Systems and Methods for Generating Markup-Language Based Expressions from Multi-Modal and Unimodal Inputs

摘要

When using finite-state devices to perform various functions, it is beneficial to use finite state devices representing regular grammars with terminals having markup-language-based semantics. By using markup-language-based symbols in the finite state devices, it is possible to generate valid markup-language expressions by concatenating the symbols representing the result of the performed function. The markup-language expression can be used by other applications and/or devices. Finite-state devices are used to convert strings of words and gestures into valid markup-language, for example, XML, expressions that can be used, for example, to provide an application program interface to underlying system applications.

主权项

1. A method comprising:
generating, via a processor, from an input lattice representing a speech modality of an input, a first markup-language expression; generating, via a finite-state transducer, a mapping, wherein the finite-state transducer uses the input lattice and a first finite-state machine having associated markup-language semantics to relate the first markup-language expression to a second markup-language expression representing a second modality based on a level of coincidence between the speech modality and the second modality, wherein the second modality is not speech; generating, using the input and a second finite-state machine representing the mapping, a third markup-language expression; and outputting the third markup-language expression.