SYSTEMS AND METHODS FOR ACCELERATING HESSIAN-FREE OPTIMIZATION FOR DEEP NEURAL NETWORKS BY IMPLICIT PRECONDITIONING AND SAMPLING

摘要

A method for training a deep neural network, comprises receiving and formatting speech data for the training, preconditioning a system of equations to be used for analyzing the speech data in connection with the training by using a non-fixed point quasi-Newton preconditioning scheme, and employing flexible Krylov subspace solvers in response to variations in the preconditioning scheme for different iterations of the training.

主权项

1. A method for training a deep neural network, comprising:
receiving and formatting speech data for the training; preconditioning a system of equations to be used for analyzing the speech data in connection with the training by using a non-fixed point quasi-Newton preconditioning scheme; and employing flexible Krylov subspace solvers in response to variations in the preconditioning scheme for different iterations of the training; wherein the receiving, formatting, preconditioning and employing steps are performed by a computer system comprising a memory and at least one processor coupled to the memory.