Systems and methods for enabling gesture control based on detection of occlusion patterns

摘要

Described is an approach to enabling gesture interactions for the viewport widget in a graphical user interface (GUI) library. The gesture interactions may include continuous operations such as panning, zooming and rotating of the viewport's content with fingers (or styluses). The approach is based on using a camera to detect occlusion patterns in a sensor grid rendered over the viewport. The sensor grid consists of sensor blobs, which are small blobs of pixels with a distinct color. A sensor blob is aware of its location in both the viewport's coordinate system and the camera's coordinate system, and triggers an occlusion event at the location when it is occluded by a finger (or stylus). Robust techniques are devised to eliminate unintentional gestures, provide visual guidance and feedback for interactions, and minimize the visual interference of the sensor grid with the viewport's content.

主权项

1. A computer-implemented method for detecting a gesture performed by a user in connection with a user interface comprising a plurality of sensor blobs, the method being performed in a computerized system comprising a central processing unit, a projector, a camera and a memory, the computer-implemented method comprising:
a. using the projector to project a content and a plurality of identical non-contiguous dot-shaped sensor blobs on a display surface; b. acquiring a plurality of images of the plurality of identical non-contiguous dot-shaped sensor blobs projected on the display surface using the camera; c. using the central processing unit to identify occlusion of at least some of the plurality of non-contiguous dot-shaped sensor blobs projected on the display surface by the user in the acquired plurality of images of the plurality of non-contiguous dot-shaped sensor blobs; and d. using the central processing unit to detect the gesture performed by the user based on the identified occlusion of at least some of the plurality of non-contiguous dot-shaped sensor blobs in the acquired plurality of images.