System and apparatus for augmented reality display and controls

摘要

An augmented reality system includes a vision output device for displaying virtual images. A sensing device captures an image within a reference frame. The vision output device is captured within the image. A processing unit identifies the vision output device within the reference frame of the captured image and localizes the vision output device within the reference frame of the captured image for identifying an absolute position and orientation of the vision output device within the reference frame of the captured image. The vision output device generates virtual displays to a user at respective locations based on the absolute position and orientation of the vision output device within reference frame of the captured image. The sensing device captures a user's selection of a virtual control. The processing unit identifies the selection of the virtual control within the captured image and enables a control action of a controllable device.

主权项

1. An augmented reality system comprising:
a vision output device for displaying virtual images, the virtual images including virtually displayed information and virtual controls associated with controllable devices; a sensing device capturing an image within a reference frame, the vision output device being captured within the image; and at least one processor in communication with the sensing device, the processor obtaining the captured image from the sensing device, the processor identifying the vision output device within the reference frame of the captured image, the processor localizing the vision output device within the reference frame of the captured image and identifying an absolute position and orientation of the vision output device within the reference frame of the captured image; wherein the vision output device generates virtual displays to a user utilizing the vision output device at respective locations based on the absolute position and orientation of the vision output device within reference frame of the captured image, wherein the sensing device captures a user's selection of a virtual control within the captured image, wherein the processor identifies the user's selection of the virtual control within the captured image and enables a control action of a controllable device associated with the selected virtual control.