Testing and Training a Question-Answering System

摘要

An approach is provided for an information handling system that includes a processor and a memory to improve the quality of question-answer sets used as inputs to a question-answering (QA) system. In the approach, a question-answer pair is analyzed using natural language processing (NLP) components. Some of the NLP components may be taken from the QA system whose input is being analyzed The question-answer pair includes a question and an answer to the question. Based on the analysis, one or more shortcomings of the question-answer pair are identified. The shortcomings relate to an ability of the target QA system to analyze the question. A human-readable feedback is provided to a user. The feedback recommends one or more possible actions to address the identified shortcomings.

主权项

1. A method, in an information handling system comprising a processor and a memory, of improving a quality of question-answer sets used as inputs to a question-answering (QA) system, the method comprising:
analyzing a question-answer pair using natural language processing (NLP) components, wherein the question-answer pair includes a question and an answer to the question, and wherein at least one of the NLP components is a configurable analyzer extension; identifying, based on the analysis, one or more shortcomings of the question-answer pair, wherein the shortcomings relate to an ability of the target QA system to analyze the question; and providing a human-readable feedback to a user, wherein the feedback recommends one or more possible actions to address the identified shortcomings.