METHOD, DEVICE, AND PROGRAM FOR LANGUAGE MODEL GENERATION AND DEVICE AND PROGRAM FOR TEXT ANALYSIS

摘要

PROBLEM TO BE SOLVED: To estimate symbolic chain probability (language model) by allocating all possible classes to one symbol.

SOLUTION: As for individual symbols of a symbolic string read out of a text database 140 having text data stored on a storage medium, a plurality of corresponding classes are found by referring to a symbol-class correspondence table 150 having symbols and a single or a plurality of classes stored on the storage medium, and their class list is generated and stored on the storage medium. Then the appearance frequency of a class chain is counted for all combinations obtained by selecting classes, one by one, from N (an integer of ≥2) class lists corresponding to N symbols which are adjacent in the read symbol string, and symbolic chain probability as a language model is generated from frequency information on class appearance chains obtained as a result of the counting.

COPYRIGHT: (C)2004,JPO