Method and apparatus for highlighting and categorizing documents using coded word tokens

摘要

Highlighting and categorization of documents is carried out by using word tokens which represent words appearing in a document. Elimination of certain unimportant word tokens is first completed, after which the remaining words of the document are ranked according to their word token appearance rates. These rates are then used to highlight frequently appearing words in the document which indicate the document's topic. The document can also be categorized using document profiles developed from the word tokens.