| 摘要 |
A method and a system for integrating multimodal interpretations (MMIs) in user inputs to a data processing system ( 100 ) are provided. The method includes generating ( 603 ) a set of MMIs, based on the user inputs collected during a turn. The set of MMIs includes at least semantic relationship function, the semantic relationship function denoting a semantic relationship between values determined from the user inputs entered through one or more of the at least one input modality, the semantic relationship function comprising at least one semantic operator. The method further includes generating ( 604 ) sets of joint MMIs. Each set of joint MMIs includes MMIs of semantically compatible types. The method further includes resolving ( 606 ) the reference values of the references of each set of joint MMIs to generate corresponding sets of reference resolved MMIs. The method further includes generating ( 608 ) an integrated MMI for each set of reference-resolved MMIs. The generation of the integrated MMI is carried out by using semantic operator based fusion.
|
