MULTI-MODAL SYSTEM

摘要

A multi-modal system is provided to process a plurality of modalities in parallel at the same time by using SCXML(State Chart eXtensible Markup Language), and offer a user and an application maker a convenient/efficient user interface by easily processing a sequential or concurrent multi-modal I/O(Input/Output) command with the SCXML. A markup language interpreter(240) interprets an SCXML document. A markup language execution engine(250) concurrently processes the plurality of different modality information by activating a state corresponding to the number of modalities used by a client in parallel based on the interpreted SCXML document. The different modalities are voice and non-voice modality. The markup language execution engine receives voice and non-voice modality information from the client in parallel, and fuses the inputted modality information with other modality input information if the inputted modality information is the partial information.