Methods and systems for providing custom crawl-time metadata

摘要

A method for providing metadata to a search engine for a document that is not in a mark-up language includes sending a request for data about the document and receiving a response to the request that has a Hyper-Text Transfer Protocol (HTTP or HTTPS) header that includes metadata associated with the document in a name-value pair and the document's content. The method also includes extracting the name-value pair from the HTTP-header and creating a mark-up language tag for the name-value pair and providing the make-up language tag and the contents of the document in a mark-up language format to a search index creation component.

主权项

1. A computer-implemented method for providing metadata for a document that is not in a mark-up language format to a search index, the method comprising:
sending, from a search engine, a request for data about the document; receiving, by the search engine, a response to the request comprising a Hyper Text Transfer Protocol (HTTP) header that includes metadata about the document in a name-value pair and content associated with the document; extracting the name-value pair from the HTTP header; creating, at the search engine, a mark-up language tag for the name-value pair; and providing the mark-up language tag and contents of the document in a mark-up language format to a search index creation component of the search engine.