| 摘要 |
PROBLEM TO BE SOLVED: To eliminate a wasteful processing that a dictionary is retrieved by a retrieval key containing a special character and to improve processing speed by dividing an inputted character string into partial character strings through the use of a special character table and dividing the divided partial character strings into tokens by referring to a word dictionary. SOLUTION: The Japanese character string is inputted from an input part 1. A partial character string division part 3 divides an input sentence into the partial character strings with the special character registered in a special character table 2 as a delimiter. Then, the partial character strings are divided into the tokens by using the word dictionary 4. The character registered in the special character table 2 is not contained in the index of the work dictionary 4. A word dictionary retrieval part 5 retrieves the word dictionary 4 with the Japanese character string as the key. A token division part 6 segments the token by using the result of the word dictionary retrieval part 5. Thus, the wasteful processing that the dictionary is retrieved by the retrieval key containing the special character is eliminated and the processing speed of morpheme analysis is improved. |
