Close range natural user interface system and method of operation thereof

摘要

A natural user interface system and method of operation thereof including: a display device having a display screen and a display device camera; a mobile device having a mobile device camera, an optical axis of the mobile device camera positioned at an angle to an optical axis of the display device camera; wherein: the mobile device includes a first device pairing module for pairing the mobile device with the display device; the mobile device camera and the display device camera are for: detecting a user's hand, and determining posture and movement of the user's hand; and the display device includes a motion translation module for translating the posture and movement of the user's hand into a gesture for controlling an element of a user interface on the display screen.

主权项

1. A method of operation of a natural user interface system comprising:
providing a display device having a display screen, a display device camera, a display device gyroscope, and a display device compass; providing a mobile device having a mobile device camera, a mobile device gyroscope, and a mobile device compass, an optical axis of the mobile device camera positioned at an angle to an optical axis of the display device camera; pairing the mobile device with the display device; determining an orientation of the display device with the display device gyroscope and the display device compass; determining an orientation of the mobile device with the mobile device gyroscope and the mobile device compass; synchronizing a viewing angle of the mobile device camera with the display device based on the orientation of the mobile device relative to the orientation of the display device; detecting a user's hand; determining posture and movement of the user's hand including:
determining forwards-backwards motion and side-to-side motion of the user's hand with the mobile device camera, anddetermining up-and-down and side-to-side motion of the user's hand with the display device camera; and translating the posture and movement of the user's hand into a gesture for controlling an element of a user interface on the display screen.