METHOD AND APPARATUS FOR CLASSIFYING AND RANKING INTERPRETATIONS FOR MULTIMODAL INPUT FUSION

摘要

A method used in an electronic equipment (100) generates a set of joint multimodal interpretations (125) from a set of multimodal interpretations (115) generated by one or more modalities (105) during a turn, generates a set of integrated multimodal interpretations (135) including an integrated multimodal interpretation formed from each joint multimodal interpretation by unifying the type feature structure of each multimodal interpretation in the joint multimodal interpretation, and generates a multilevel confidence score for each integrated multimodal interpretation based on at least one of a context score, a content score, and a confidence score of the integrated multimodal interpretation. The method classifies the multimodal interpretations and generates a set of joint multimodal interpretations that comprises essentially all possible joint multimodal interpretations. The multilevel confidence scoring is based on up to eleven factors, which provides for an accurate ranking of the integrated multimodal interpretations.