Search engine performance evaluation using a task-based assessment metric

摘要

The subject disclosure is directed towards evaluating search engine implementation performance using a task-based assessment metric. Logged user activities associated with the search engine implementation are processed into sessions and atomic events corresponding to the user activities within the sessions. The atomic events corresponding to the user activities are classified into tasks based on similarity of the queries within the user activities. After applying the task-based assessment metric to task information associated with the search engine implementation, an evaluation mechanism determines indicia of search engine implementation performance in terms of user behavior.

主权项

1. In a computing environment, a method performed at least in part on at least one processor, comprising,
applying a task-based assessment metric to a search engine implementation, including, examining user activities associated with the search engine implementation, classifying atomic events into tasks by measuring similarity between at least some of the user activities, and evaluating each task in accordance with the task-based assessment metric to determine indicia of search engine implementation performance comprising a task success rate for the search engine implementation, including applying the task-based assessment metric to the tasks to produce a statistical distribution and comparing the statistical distribution corresponding to the search engine implementation with a distribution corresponding to another search engine implementation to generate a comparison result.