CONSTRUCTION OF A LEXICON FOR A SELECTED CONTEXT

摘要

Various technologies pertaining to constructing a lexicon for a defined context are set forth herein. Social media text is acquired, where the social media text has contextual data that corresponds thereto. The social media text is encoded to form encoded text (in Unicode), and the contextual data is assigned to the encoded text. A text corpus for a defined context is formed by filtering the encoded text based upon contextual data, such as location. Frequency of occurrence of words or phrases in the text corpus is used to identify words or phrases that are to be included in the lexicon.

主权项

1. A computing system comprising:
a processor; and memory that comprises a lexicon generator system that is executed by the processor, the lexicon generator system configured to generate a lexicon used in context text, the lexicon generator system configured to include at least one of a mixed language word or a mixed language phrase in the lexicon based upon frequency of occurrence of the mixed language word or the mixed language phrase in the context text.