LANGUAGE MODELING SYSTEM AND METHOD FOR FORMATION OF LANGUAGE MODEL

摘要

PURPOSE: To sharply reduce the memory requirement and converging time of a language modeling system. CONSTITUTION: Each n-gram is related to one of n classes which are not intersected with each other. The generation frequency of each n-gram in training data is counted in each n-gram and these ngrams are separated into respective classes and a complement count is determined. A factor is determined in each class by using the counted frequency and the complement count. Probability of generating a word, i.e., the probability of a language model, is determined under a condition that two preceding words are generated.