Systems and Methods for Interacting with a Projected User Interface

摘要

A system and method for providing a 3D gesture based interaction system for a projected 3D user interface is disclosed. A user interface display is projected onto a user surface. Image data of the user interface display and an interaction medium are captured. The image data includes visible light data and IR data. The visible light data is used to register the user interface display on the projected surface with the Field of View (FOV) of at least one camera capturing the image data. The IR data is used to determine gesture recognition information for the interaction medium. The registration information and gesture recognition information is then used to identify interactions.

主权项

1. A processing system configured to conduct Three Dimensional (3D) gesture based interactive sessions for a projected user interface display comprising:
a memory containing an image processing application; and a processor directed by the image processing application read from the memory to:
receive image data that includes visible light image data and Infrared (IR) image data,obtain visible light image data from the image data,generate registration information for a user interface display on a projected surface with a field of view of one or more image capture devices using the visible light image data,obtain IR image data from the image data, andgenerate gesture information for an interaction medium using the IR data, andidentify an interaction with an interactive object with the user interface display using the gesture information and the registration information.