Performing Cross-Validation Using Non-Randomly Selected Cases

摘要

A technique to perform cross-validation using a set of randomly selected labeled cases and a set of non-randomly selected labeled cases. A training set for use during cross-validation can include cases from both sets. A test set for use during cross-validation can include cases from the randomly selected set but exclude cases from the non-randomly selected set.

主权项

1. A method for performing cross-validation, comprising:
given a first set of randomly selected labeled cases and a second set of actively selected labeled cases, training a classifier on a training set comprising a first subset of the first set and a subset of the second set; and measuring the performance of the classifier on a test set comprising a second subset of the first set that is disjoint relative to the first subset, the test set including no cases from the second set.