USING HUMAN PERCEPTION IN BUILDING LANGUAGE UNDERSTANDING MODELS

摘要

An understanding model is trained to account for human perception of the perceived relative importance of different tagged items (e.g. slot/intent/domain). Instead of treating each tagged item as equally important, human perception is used to adjust the training of the understanding model by associating a perceived weight with each of the different predicted items. The relative perceptual importance of the different items may be modeled using different methods (e.g. as a simple weight vector, a model trained using features (lexical, knowledge, slot type, . . . ), and the like). The perceptual weight vector and/or or model are incorporated into the understanding model training process where items that are perceptually more important are weighted more heavily as compared to the items that are determined by human perception as less important.

主权项

1. A method for using human perception in training a language understanding model, comprising:
accessing tagged items that are used by the language understanding model; creating queries using the tagged items including errors into a portion of the queries that create a query that affects a weight at least one of the tagged items used within the query being created; obtaining results from a knowledge source for each of the queries created; obtaining a human perception rating for each of the different results; determining a relative importance of each of the tagged items using the human perception ratings; and training the language understanding model using the relative importance of each of the tagged items as determined from the human perception rating.