Novel Quadratic Regularization For Neural Network With Skip-Layer Connections

摘要

According to one aspect of the invention, target data comprising observations is received. A neural network comprising input neurons, output neurons, hidden neurons, skip-layer connections, and non-skip-layer connections is used to analyze the target data based on an overall objective function that comprises a linear regression part, the neural network's unregularized objective function, and a regularization term. An overall optimized first vector value of a first vector and an overall optimized second vector value of a second vector are determined based on the target data and the overall objective function. The first vector comprises skip-layer weights for the skip-layer connections and output neuron biases, whereas the second vector comprises non-skip-layer weights for the non-skip-layer connections.

主权项

1. A method comprising:
receiving target data comprising a plurality of observations, wherein each observation in said plurality of observation comprises (a) one or more values of one or more input neurons in a neural network and (b) one or more corresponding values of one or more output neurons in the neural network, and wherein the neural network comprises one or more layers of hidden neurons, one or more skip-layer connections between the one or more input neurons and the one or more output neurons, and one or more non-skip-layer connections between neurons of the neural network; determining an overall objective function that comprises (a) a first part corresponding to linear regression, (b) a second part corresponding to the neural network's unregularized objective function, and (c) a third part corresponding to a regularization of an unregularized objective function of the neural network; and determining, based at least in part on said target data and said overall objective function, an overall optimized first vector value of a first vector and an overall optimized second vector value of a second vector, wherein said first vector comprises a set of skip-layer weights for the one or more skip-layer connections and a set of output neuron biases, and wherein said second vector comprises a set of non-skip-layer weights for the one or more non-skip-layer connections; wherein the method is performed by one or more computing devices.