GENERATING REPRESENTATIONS OF INPUT SEQUENCES USING NEURAL NETWORKS

摘要

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating representations of input sequences. One of the methods includes receiving a grapheme sequence, the grapheme sequence comprising a plurality of graphemes arranged according to an input order; processing the sequence of graphemes using a long short-term memory (LSTM) neural network to generate an initial phoneme sequence from the grapheme sequence, the initial phoneme sequence comprising a plurality of phonemes arranged according to an output order; and generating a phoneme representation of the grapheme sequence from the initial phoneme sequence generated by the LSTM neural network, wherein generating the phoneme representation comprises removing, from the initial phoneme sequence, phonemes in one or more positions in the output order.

主权项

1. A method comprising:
receiving an input sequence comprising a plurality of inputs arranged according to an input order; processing the input sequence using a long short-term memory (LSTM) neural network to generate an initial output sequence for the input sequence, the initial output sequence for the input sequence comprising a plurality of outputs arranged according to an output order; and generating a final output sequence for the input sequence from the initial output sequence generated by the LSTM neural network, wherein generating the final output sequence comprises removing, from the initial output sequence, outputs in one or more positions in the output order.