AUTOMATED QUALITY ASSURANCE CHECKS FOR IMPROVING THE CONSTRUCTION OF NATURAL LANGUAGE UNDERSTANDING SYSTEMS

摘要

Aspects described herein provide quality assurance checks for improving the construction of natural language understanding grammars. An annotation module may obtain a set of annotations for a set of text samples based, at least in part, on an ontology and a grammar. A quality assurance module may automatically perform one or more quality assurance checks on the set of annotations, the ontology, the grammar, or combinations thereof. The quality assurance module may generate a list of flagged annotations during performance of a quality assurance check. The list of flagged annotations may be presented at an annotation review interface displayed at a display device. One of the flagged annotations may be selected and presented at an annotation interface displayed at the display device. Responsive to presentation of the flagged annotation, the ontology, the grammar, the flagged annotation selected, or combinations thereof may be updated based on user input received.

主权项

1. A computer-implemented method for improving text annotation comprising:
obtaining, using an annotation module, a set of annotations for a set of text samples based, at least in part, on an ontology and a grammar; performing, automatically using a quality assurance module, one or more quality assurance checks on at least one of the set of annotations, the ontology, and the grammar; generating, using the quality assurance module, a list of flagged annotations during performance of at least one of the quality assurance checks; and presenting the list of flagged annotations at an annotation review interface displayed at a display device.