CORPUS CAPACITY REDUCTION DEVICE, METHOD AND COMPUTER PROGRAM THEREFOR

摘要

PROBLEM TO BE SOLVED: To reduce capacity of the existing corpus and to prevent deterioration of perplexity of a language model.

SOLUTION: This corpus capacity reduction device 20 includes a candidate sentence selection part 40 which sequentially selects sentences included in a first corpus 22 as candidate sentences for adding to a second corpus 24, a three sentence set selection part 44 which selects all sentence sets consisting of combination of three sentences included in the second corpus for each of the selected candidate sentences, an analogical relation decision part 46 which decides whether or not analogical relation is established between each of the selected candidate sentences and any of the combination of three sentences selected by a three sentence set selection part 44 and a candidate document writing part 42 which adds a candidate sentence decided that no analogical relation is established by the analogical relation decision part 46 to the second corpus 24 for each of the candidate sentences.

COPYRIGHT: (C)2005,JPO&NCIPI