Gaze detection in a see-through, near-eye, mixed reality display

摘要

The technology provides various embodiments for gaze determination within a see-through, near-eye, mixed reality display device. In some embodiments, the boundaries of a gaze detection coordinate system can be determined from a spatial relationship between a user eye and gaze detection elements such as illuminators and at least one light sensor positioned on a support structure such as an eyeglasses frame. The gaze detection coordinate system allows for determination of a gaze vector from each eye based on data representing glints on the user eye, or a combination of image and glint data. A point of gaze may be determined in a three-dimensional user field of view including real and virtual objects. The spatial relationship between the gaze detection elements and the eye may be checked and may trigger a re-calibration of training data sets if the boundaries of the gaze detection coordinate system have changed.

主权项

1. A method for determining gaze in a near-eye mixed reality display device comprising:
determining boundaries of a three dimensional gaze detection coordinate system for each eye based on positions of glints detected on the respective eye, positions on the near-eye display device of illuminators for generating the glints, and a position of at least one sensor for capturing reflected eye data of the respective eye from which the positions of glints are detected; automatically checking periodically the reflected eye data for any change along a depth axis of the gaze detection coordinate system, the depth axis extending from a respective display optical system of the near-eye, mixed reality display device toward the respective eye; responsive to detecting any change along at least one depth axis, triggering redetermination of the boundaries of the three dimensional gaze detection coordinate system; determining a gaze vector for each eye based on reflected eye data including the glints and the boundaries of the three dimensional gaze detection coordinate system; determining a point of gaze based on the gaze vectors for the eyes in a three-dimensional (3D) user field of view of the near-eye mixed reality display device including real and virtual objects; and identifying any object at the point of gaze in the 3D user field of view.