METHOD AND SYSTEM OF GENERATING AN AGGREGATE WEBSITE SEARCH DATABASE USING SMART INDEXES FOR SEARCHING

摘要

Signature schema documents may be pre-defined using query language to provide instructions for application by an engine to extract data from web pages of respective web sites. For a particular web page, signature schema instructions identify a web page family for the web page and extract desired data from the web page in accordance with its web page family. The instructions use signatures previously identified within web pages of the same family to distinguish the web page family from others of the web site and to distinguish the desired data from other data for the web page family. A server may make one or more requests to obtain web pages from various web sites and apply respective signature schemas maintained in a repository coupled to the engine. Indexes can be generated based upon the pre-defined data relationships to improve search capability. Extracted data and indexes can be stored to an aggregate database.