APPARATUS FOR SEGMENTING CHINESE CHARACTER SEQUENCE TO CHINESE WORD SEQUENCE

摘要

PROBLEM TO BE SOLVED: To provide an apparatus for segmenting a Chinese character sequence to an appropriate word sequence.

SOLUTION: The device comprises a Chinese subword list 64 and a statistical probability model 66 of Chinese sequence of IOB tags assigned to subwords. The apparatus further comprises a subword-based IOB tagging module 88 for segmenting a Chinese sentence 80 into a first Chinese word sequence with a maximum likelihood estimation using the subword list 64 and the probability model 66. The multiple-subword words in the first Chinese word sequence are segmented into subwords each being labeled with the IOB tags according to the segmentation. The words in the Chinese subword list 64 are treated as subwords by the subword-based IOB tagging module 88 in segmenting the Chinese sentence 80.

COPYRIGHT: (C)2008,JPO&INPIT