N-gram model smoothing with independently controllable parameters

摘要

Described is a technology by which a probability is estimated for a token in a sequence of tokens based upon a number of zero or more times (actual counts) that the sequence was observed in training data. The token may be a word in a word sequence, and the estimated probability may be used in a statistical language model. A discount parameter is set independently of interpolation parameters. If the sequence was observed at least once in the training data, a discount probability and an interpolation probability are computed and summed to provide the estimated probability. If the sequence was not observed, the probability is estimated by computing a backoff probability. Also described are various ways to obtain the discount parameter and interpolation parameters.

主权项

1. In a machine translation system or a speech recognition system, a method performed on at least one processor, comprising, processing an output candidate comprising a sequence of tokens for natural language input, including estimating an estimated probability for a token in the sequence of tokens based upon actual counts corresponding to a number of zero or more times that the sequence was observed in training data, including controlling interpolation parameters independently from controlling a discount parameter, and when the sequence was observed at least once, computing a discount probability based upon the discount parameter subtracted from a maximum likelihood probability estimate that is based upon a context corresponding to the sequence, computing an interpolation probability based upon one interpolation parameter and a smaller other context corresponding to the sequence, mathematically combining the discount probability with the interpolation probability to provide the estimated probability, and using the estimated probability to generate an alternative output for at least a portion of the natural language input.