INFORMATION RETRIEVAL SYSTEM EVALUATION METHOD, DEVICE AND STORAGE MEDIUM

摘要

The present disclosure discloses an information retrieval system evaluation method, device and storage medium. A ratio between the sum of all the related object parameters of a keyword in keyword in an evaluation retrieval result set and the sum of all the related object parameters of the keyword in keyword in a retrieval result set is used to compute a recall rate of an information retrieval system. And the recall rate is introduced to evaluate the information retrieval system, thereby enhancing accuracy of quantitative evaluation of the information retrieval system, and improving the automation degree of evaluation.

主权项

1. An information retrieval system evaluation method, comprising:
obtaining behavior data sample reported by an information retrieval system within a pre-determined period; obtaining a sample retrieval keyword set and a sample retrieval result corresponding to each sample retrieval keyword according to the behavior data sample; invoking the information retrieval system to perform evaluation retrieval on a pre-determined evaluation keyword set, computing a recall rate and a correctness percentage corresponding to each keyword in the evaluation keyword set according to an evaluation retrieval result and the sample retrieval result; and computing an evaluation indicator of the information retrieval system according to the recall rate and the correctness percentage, where the recall rate is a ratio between the sum of all related object parameters in the evaluation retrieval result corresponding to a keyword and the sum of all the related object parameters in the sample retrieval result corresponding to the keyword; the correctness percentage is computed according to the number of the related objects and the number of the non-related objects in the evaluation retrieval result corresponding to the keyword; and the related object corresponding to the keyword is an object where a user performs operation in the sample retrieval result corresponding to the keyword; the non-related object corresponding to the keyword is an object where a user does not perform operation in the sample retrieval result corresponding to the keyword.