Predictive model evaluation and training based on utility

摘要

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a plurality of different types of predictive models using training data, wherein each of the predictive models implements a different machine learning technique. One or more weights are obtained wherein each weight is associated with an answer category in the plurality of examples. A weighted accuracy is calculated for each of the predictive models using the one or more weights.

主权项

1. A computer-implemented method, the method comprising:
obtaining training data comprising a plurality of examples wherein each example comprises one or more features and an answer; training a plurality of different types of predictive models using the training data, wherein each of the predictive models implements a different machine learning technique; obtaining weights wherein each weight is associated with a respective answer category in the plurality of examples, each of the weights indicating an amount the respective answer category will count toward a weighted accuracy for each of the predictive models; calculating the weighted accuracy for each of the predictive models using the respective weights; and selecting one of the predictive models as the most accurate model based at least partly on the calculated weighted accuracies.