Fast, efficient hardware mechanism for natural language determination

摘要

A language in which a document is written is identified by comparing the words of a document to the most frequently used words in a plurality of candidate languages. The words are stored in a plurality of sets of word tables, each set of word tables for storing a selected set of most frequently used words in a respective candidate language according to letter pairs in the words. In the preferred embodiment, each of the word tables is an NxN bit table, where each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language. A set of table access registers, is used for accessing a respective set of word tables to compare words from the document to words stored in the word tables; each table access register accesses word tables for a respective candidate language. One or more word counting registers count a number of matches for a respective candidate language. A comparator selects a candidate language which corresponds to the word counting register having the highest count as the language in which the document is written.