Method of processing non-responsive data items

摘要

A method of obtaining an evaluation of a data item from a human evaluator includes presenting a data item to a human evaluator, and receiving a response from the evaluator that includes an indication that a data item is non-responsive or responsive. An output determined by a computer analysis of the data item is referenced and the evaluator response is compared to the output of the computer analysis.