System and method for user object selection in geographic relation to a video display

摘要

The invention is directed to a system and method for determining position and orientation with respect to a display device. In particular, the invention is directed to acquiring an observed image from a display device using a pointing device, and calculating pointing device position and orientation using the acquired observed image and a displayed image on the display device.

主权项

1. A method of pointing to objects in a display device displayed image, said displayed image comprising both objects and background, said method comprising:
acquiring a displayed image generated by a display device, using a user aimed pointing device comprising a housing and at least one imaging sensor disposed substantially within said housing and configured to acquire an observed image of said displayed image; said displayed image comprising a plurality of displayed image pixels, wherein at least some of said displayed image pixels show said objects and other displayed image pixels show said background; wherein said at least some of said displayed image pixels that show said objects and other displayed image pixels that show said background are refreshed by said display device at the same rate; said observed image comprising a plurality of observed image pixels, wherein at least some of said observed image pixels show said objects, and other observed image pixels show said background; calculating pointing device position and orientation using the acquired observed image and a displayed image on the display device; wherein calculating pointing device position and orientation includes correlating the displayed image with the acquired observed image to generate correlation data, and computing pointing device position and orientation using the correlation data; wherein the calculating of pointing device position and orientation is performed by solving the inverse problem(xp,vp)=arg⁢⁢minxp,vp⁢T⁡(Id;xp,vp)-Iowhere xp and vp are vectors of the position and orientation of the pointing device, respectively, Id is the displayed image, Io is the acquired image, ∥•∥ is a norm, and T is a mapping describing how the displayed image is observed by the sensor in the pointing device as a function of the pointing device position and orientation.