Co-Verbal Interactions With Speech Reference Point

摘要

Example apparatus and methods improve efficiency and accuracy of human device interactions by combining speech with other input modalities (e.g., touch, hover, gestures, gaze) to create multi-modal interactions that are more natural and more engaging. Multi-modal interactions expand a user's expressive power with devices. A speech reference point is established based on a combination of prioritized or ordered inputs. Co-verbal interactions occur in the context of the speech reference point. Example co-verbal interactions include a command, a dictation, or a conversational interaction. The speech reference point may vary in complexity from a single discrete reference point (e.g., single touch point) to multiple simultaneous reference points to sequential reference points (single touch or multi-touch), to analog reference points associated with, for example, a gesture. Establishing the speech reference point allows surfacing additional context-appropriate user interface elements that further improve human device interactions in a natural and engaging experience.

主权项

1. A method, comprising:
establishing a speech reference point for a co-verbal interaction between a user and a device, where the device is speech-enabled, where the device has a visual display, where the device has at least one non-speech input apparatus, and where a location of the speech reference point is determined, at least in part, by an input from the non-speech input apparatus; controlling the device to provide a feedback concerning the speech reference point; receiving an input associated with a co-verbal interaction between the user and the device, and controlling the device to process the co-verbal interaction as a contextual voice command, where a context associated with the voice command depends, at least in part, on the speech reference point.