GENERAL FRAMEWORK FOR CROSS-VALIDATION OF MACHINE LEARNING ALGORITHMS USING SQL ON DISTRIBUTED SYSTEMS

摘要

A general framework for cross-validation of any supervised learning algorithm on a distributed database comprises a multi-layer software architecture that implements training, prediction and metric functions in a C++ layer and iterates processing of different subsets of a data set with a plurality of different models in a Python layer. The best model is determined to be the one with the smallest average prediction error across all database segments.

主权项

1. A method of cross-validation of a supervised machine learning algorithm within a distributed database having a plurality of database segments in which data are stored, comprising:
partitioning a data set within said database into a training subset and a validation subset, wherein the partitioning data set comprises partitioning the data set according to randomly sorted data to create two data subsets that are independent and statistically equivalent; determining coefficients of a first model of said supervised machine learning algorithm using the training subset; predicting a value of a data element in said validation subset using said first model; determining a prediction error based at least in part on a difference between said predicted value and the actual value of said data element; successively repeating said partitioning k times to form k different partitions, wherein at least a subset of the k different partitions have different training and validation subsets; determining corresponding k prediction errors based at least in part on iteratively determining the coefficients, predicting the value of the data element, and determining the prediction error for each of said k partitions; and evaluating the performance of said first model using said k prediction errors.