SYSTEMS AND METHODS FOR COMBINING STOCHASTIC AVERAGE GRADIENT AND HESSIAN-FREE OPTIMIZATION FOR SEQUENCE TRAINING OF DEEP NEURAL NETWORKS

摘要

A method for training a deep neural network (DNN), comprises receiving and formatting speech data for the training, performing Hessian-free sequence training (HFST) on a first subset of a plurality of subsets of the speech data, and iteratively performing the HFST on successive subsets of the plurality of subsets of the speech data, wherein iteratively performing the HFST comprises reusing information from at least one previous iteration.

主权项

1. A method for training a deep neural network, comprising:
receiving and formatting speech data for the training; performing Hessian-free sequence training on a first subset of a plurality of subsets of the speech data; and iteratively performing the Hessian-free sequence training on successive subsets of the plurality of subsets of the speech data; wherein iteratively performing the Hessian-free sequence training comprises reusing information from at least one previous iteration; and wherein the receiving, formatting, performing and iteratively performing steps are performed by a computer system comprising a memory and at least one processor coupled to the memory.