NEURAL MACHINE TRANSLATION SYSTEMS WITH RARE WORD PROCESSING

摘要

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural translation systems with rare word processing. One of the methods is a method training a neural network translation system to track the source in source sentences of unknown words in target sentences, in a source language and a target language, respectively and includes deriving alignment data from a parallel corpus, the alignment data identifying, in each pair of source and target language sentences in the parallel corpus, aligned source and target words; annotating the sentences in the parallel corpus according to the alignment data and a rare word model to generate a training dataset of paired source and target language sentences; and training a neural network translation model on the training dataset.

主权项

1. A computer-implemented translation system for translating natural language text from a source sentence in a source language to a target sentence in a target language, the translation system comprising one or more computers and one or more storage devices storing translation instructions and translation data, wherein:
the translation data includes:
a word dictionary;a neural network translation model trained to track the origin in source sentences of unknown words in target sentences and to emit for each out-of-vocabulary (OOV) word in the target sentence a respective unknown token, the model being operable to emit (i) pointer tokens, pointer tokens being unknown tokens that identify a respective source word in the source sentence corresponding to the unknown token, and (ii) null unknown tokens, null unknown tokens being tokens that do not identify any source word in the source sentence; the translation instructions are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
for every pointer token in a target sentence emitted by the neural network translation model from a source sentence, replacing the pointer token according to the corresponding source word in the source sentence.