Robust and fast model fitting by adaptive sampling

摘要

Aspects of the present disclosure relate generally to model fitting. A target model having a large number of inputs is fit using a performance model having relatively few inputs. The performance model is learned during the fitting process. Optimal optimization parameters including a sample size, a damping factor, and an iteration count are selected for an optimization round. A random subset of data is sampled based on the selected sample size. The optimization round is conducted using the iteration count and the sampled data to produce optimized parameters. The performance model is updated based on the performance of the optimization round. The parameters of the target model are then updated based on the damping factor and the parameters computed by the optimization round. The aforementioned steps are performed in a loop in order to obtain optimized parameters and fit of the data to the target model.

主权项

1. A method for training a target model on a set of training data by performing a plurality of optimization rounds, the method comprising, for each of the plurality of optimization rounds:
selecting a set of optimization parameters for the optimization round including a sample size and a damping factor; sampling the training data based on the selected sample size; performing, by one or more processors, the optimization round including optimizing an objective function for the sampled training data; updating a set of target parameters of the target model based on results of the optimization round and the damping factor; and determining, by the one or more processors and using a performance model associated with a set of performance parameters, an optimal sample size for a next optimization round in the plurality of optimization rounds, wherein the performance model is configured to forecast an expected performance of the next optimization round for a given sample size and damping factor based on results of the optimization round, the selected sample size and the damping factor.