DEVICE FOR SIMULTANEOUSLY SEGMENTING BILINGUAL CORPUS, AND COMPUTER PROGRAM THEREFOR

摘要

PROBLEM TO BE SOLVED: To provide a device for simultaneously segmenting a source of a token and a target sequence without any problem of overlearning.

SOLUTION: The device includes a storage part which stores first and second sequences; a simultaneous segmentation device which simultaneously segments a block pair of the first and second sequences; a counter 74 which counts how many phrase pairs are generated; a sample extractor 88 which samples block pairs at random; a subtractor 100 which subtracts one from the number of phrase pairs in the sampled block pairs; a calculator 102 which calculates a probability of all simultaneous segmentations which are possible for the sampled block pairs; a sample extractor 106 which samples one of the possible simultaneous segmentations in accordance with the calculated probability; an update part 108 which updates the count of the phrase pairs; and a repetition control part 90 which makes the sample extractor 88 to the update part 108 repeatedly operate until an end condition is met.

COPYRIGHT: (C)2012,JPO&INPIT