| 摘要 |
A computer-implemented prediction model evaluation method includes specifying many prediction models and a benchmark model against which the prediction models will be evaluated. A primary data matrix is arranged by data indices, and the primary matrix is sampled with replacement N times to bootstrap N observation matrices. Then, all the matrices are filled with measurement criteria, with each criteria being representative of a respective data index and a respective model. A p-value estimate is returned that measures the statistical significance of the best prediction model relative to the benchmark, where the p-value represents the probability of wrongly rejecting the null hypothesis that a best prediction model has expected performance no better than that of a benchmark. The p-value accounts for the examination of all of the prediction models, i.e., the p-value depends on the examination of all of the models as a group, and not simply on a single model.
|
