System and Method for Speech-Based Navigation and Interaction with a Device's Visible Screen Elements Using a Corresponding View Hierarchy

摘要

A method 900 includes: identifying 903, within an electronic device 100, a character sequence representing a first speech input that is received 902 while displaying a viewable screen having at least one selectable viewable element; accessing 904 a view hierarchy that provides a text representation of one or more of the at least one selectable viewable element; comparing 906 the character sequence to the text representation of each of the selectable elements associated with the viewable screen; and in response to detecting 908 a match of the character sequence with at least one text representation, a processor of the electronic device 100 (i) selecting 910, within the viewable screen, a corresponding selectable viewable element whose text representation matches the character sequence and (ii) triggering 918 any action linked to selecting the corresponding selectable viewable element.

主权项

1. A method for enabling screen-specific user interfacing with elements of viewable screens presented by an electronic device, the method comprising:
identifying a character sequence representing a first input that is received while displaying a viewable screen having at least one selectable viewable element; accessing an electronic file that provides a text representation of one or more of the at least one selectable viewable element; comparing the character sequence to the text representation; and in response to detecting a match of the character sequence with at least a portion of the text representation, a processor of the electronic device
(i) selecting, within the viewable screen, a selectable viewable element whose text representation matches the character sequence, and(ii) triggering any action linked to the selecting the selectable viewable element.