PROCESSING NATURAL LANGUAGE TEXT WITH CONTEXT-SPECIFIC LINGUISTIC MODEL

摘要

Aspects of the disclosure are directed to natural language processing. An input interface of a computing device receives input (e.g., speech input) and generates a digital signal corresponding to that input. Text corresponding to the digital signal is obtained, and the text is processed using each of a context-free and a context-specific linguistic model to generate linguistic processing results for that text. The text and linguistic processing results may be processed using a NLU model to generate an NLU recognition result corresponding to the input received at the input interface. The text and the linguistic processing results may also be annotated and used to train a NLU model. The linguistic processing results may relate to, e.g., the tokenization of portions of the text, the normalization of portions of the text, sequences of normalizations for portions of the text, and rankings and prioritization of the linguistic processing results.

主权项

1. A device comprising:
one or more processors; an input interface configured to receive input and generate a digital signal corresponding to the input; and memory storing instructions that, when executed by one of the processors, cause the device to
obtain text based on the digital signal, andprocess the text using a context-free linguistic model and a context-specific linguistic model to generate one or more linguistic processing results for the text.