Automatic updating of confidence scoring functionality for speech recognition systems with respect to a receiver operating characteristic curve

摘要

Automatically adjusting confidence scoring functionality is described for a speech recognition engine. Operation of the speech recognition system is revised so as to change an associated receiver operating characteristic (ROC) curve describing performance of the speech recognition system with respect to rates of false acceptance (FA) versus correct acceptance (CA). Then a confidence scoring functionality related to recognition reliability for a given input utterance is automatically adjusted such that where the ROC curve is better for a given operating point after revising the operation of the speech recognition system, the adjusting reflects a double gain constraint to maintain FA and CA rates at least as good as before revising operation of the speech recognition system.

主权项

1. A method comprising:
determining an original receiver operating characteristic (ROC) curve describing performance of a speech recognition system with respect to an original rate of false acceptance (FA) of the speech recognition system versus an original rate of correct acceptance (CA) of the speech recognition system; changing an algorithm used by the speech recognition system for sentence confidence scores, wherein changing the algorithm results in a new ROC curve with respect to a new rate of FA of the speech recognition system versus a new rate of CA of the speech recognition system; receiving a user specification of relative importance of the new rate of FA versus the new rate of CA; and based on the relative importance of the new rate of FA versus the new rate of CA, adjusting a confidence scoring functionality related to recognition reliability for a given input utterance, wherein at or above a given operating point of the speech recognition system, the new ROC curve reflects a double gain constraint relative to the original ROC curve, such that the new rate of FA is equal to or less than the original rate of FA, and the new rate of CA is equal to or greater than the original rate of CA.