| 摘要 |
A system and method are described for adaptive language understanding using multimodal language acquisition in human-computer interaction. Words, phrases, sentences, production rules (syntactic information) as well as their corresponding meanings (semantic information) are stored. New words, phrases, sentences, production rules and their corresponding meanings can be acquired through interaction with users, using different input modalities, such as, speech, typing, pointing, drawing and image capturing. This system therefore acquires language through a natural language and multimodal interaction with users. New language knowledge is acquired in two ways. First, by acquiring new linguistic units, i.e. words or phrases and their corresponding semantics, and second by acquiring new sentences or language rules and their corresponding computer actions. The system represents an adaptive spoken interface capable of interpreting the user's spoken commands and sensory inputs and of learning new linguistic concepts and production rules. Such a system and the underlying method can not only be used to build adaptive conversational or dialog systems, but also to build adaptive interactive computer interfaces and operating systems, expert systems and computer games.
|
