Multimodal input using scratchpad graphical user interface to edit speech text input with keyboard input

摘要

A system and method for multimodal input into an application program. The method may include performing speech recognition on speech audio input to thereby produce recognized speech text input for insertion into a document of an application program, the document having keyboard focus. The method may also include identifying the document as being text service framework unaware. The method may further include displaying the recognized speech text input in a scratchpad graphical user interface for editing the recognized speech text input. The method may further include reflecting keyboard input bound for the document to the scratchpad graphical user interface, while preserving the keyboard focus of the document. The method may also include displaying the reflected keyboard input on the scratchpad graphical user interface, to thereby effect edits in the recognized speech text input.

主权项

1. A multimodal input system for use with an application program executed on a computing device, the system comprising:
a dictation module configured to receive a recognized speech text input for insertion into a document that has keyboard focus, the recognized speech text input including text recognized from speech audio input by a speech recognition module; a scratchpad graphical user interface (GUI) configured to display the recognized speech text input, to receive keyboard input to effect edits in the recognized speech text input without changing a keyboard focus from the document in the application program, and to generate an edited speech input for insertion into the document; and a document-specific keyboard handler including a reflector configured to selectively reflect keyboard input, received from a foreground queue and bound for the document, to the scratchpad GUI for editing of the recognized speech text input, wherein the reflector includes a filter that is configured to programmatically filter the keyboard input while keyboard input is being reflected, and to refrain from reflecting exempted keyboard input that is determined to match a predetermined filter parameter, and instead pass the exempted keyboard input to the document.