Method and system enabling natural user interface gestures with an electronic system

摘要

An electronic device coupleable to a display screen includes a camera system that acquires optical data of a user comfortably gesturing in a user-customizable interaction zone having a z0 plane, while controlling operation of the device. Subtle gestures include hand movements commenced in a dynamically resizable and relocatable interaction zone. Preferably (x,y,z) locations in the interaction zone are mapped to two-dimensional display screen locations. Detected user hand movements can signal the device that an interaction is occurring in gesture mode. Device response includes presenting GUI on the display screen, creating user feedback including haptic feedback. User three-dimensional interaction can manipulate displayed virtual objects, including releasing such objects. User hand gesture trajectory clues enable the device to anticipate probable user intent and to appropriately update display screen renderings.

主权项

1. A method to enable a user-object to communicate using gestures made in (x,y,z) space with an electronic device coupleable to a display screen whereon imagery is displayable, the method including the following steps:
(a) providing said electronic device with an optical acquisition system operable to capture image data of said user-object within a three-dimensional hover zone; (b) defining within said three-dimensional hover zone an interaction subzone including at least one z0 plane disposed intermediate a plane of said display screen and a plane at a maximum z-distance beyond which user-object gestures need not be recognized by said electronic device; (c) processing image data captured at step (a) representing an interaction of said user-object with at least a portion of said z0 plane, defined in step (b), to detect and to produce three-dimensional positional information of said interaction; (d) using said three-dimensional positional information produced at step (c) to determine at least one of (i) when in time, and (ii) where in (x,y,z) space said user-object interacted with said z0 plane; (e) following determination at step (d), identifying a gesture being made by said user-object; and (f) in response to identification of a gesture at step (e), altering at least one operating characteristic of said electronic device.