Systems and methods to evaluate search qualities

摘要

Systems and methods are provided to evaluate a search engine using search results retrieved from a plurality of different search engines. For example, a controller can randomly distribute a set of pre-formulated queries to a plurality of computers to query the search engines at random times to obtain search results and combine the search results to evaluate the completeness of a query result provided by a particular search engine. The controller may simulate the random queries from users using a set of computers and may reduce the burden on the search engines by spreading out the query activities over a period of time, such as a month.

主权项

1. A method comprising:
scheduling, by a computer controller, submission of a batch of query requests to a first search engine and a plurality of second search engines, wherein the batch of query requests is submitted to the first search engine and the plurality of second search engines by a set of computers selected by the computer controller; receiving, at the computer controller, first search results from the first search engine and second search results from the plurality of second search engines in response to submission of the batch of query requests to the first search engine and the plurality of second search engines; aggregating, by the computer controller, the first search results received from the first search engine and the second search results received from the plurality of second search engines to form a set of aggregated unique search results, wherein aggregating the first search results received from the first search engine and the second search results received from the plurality of the second search engines to form the set of aggregated unique search results comprises eliminating an invalid result from the first search result received from the first search engine; and comparing, via the computer controller, the first search results received from the first search engine to the set of aggregated unique search results to determine a performance of the first search engine, wherein the performance of the first search engine is based on a completeness of the first search results received from the first search engine in comparison with the set of aggregated unique search results.