Virtual interaction with image projection

摘要

Embodiments that relate to providing a low-latency interaction in a virtual environment are provided. In one embodiment an initial image of a hand and initial depth information representing an initial actual position are received. The initial image is projected into the virtual environment to an initial virtual position. A segmented version of the initial image is provided for display in the virtual environment at the initial virtual position. A subsequent image of the hand and depth information representing a subsequent actual position are received. The subsequent image is projected into the virtual environment to a subsequent virtual position. A segmented version of the subsequent image is provided for display at the subsequent virtual position. A collision is detected between a three-dimensional representation of the hand and a virtual or physical object.

主权项

1. An interface system for providing a low-latency interaction in a virtual environment, the interface system comprising:
an interface program executed by a processor of a computing device, the interface program configured to:
receive an initial actual image of a back of a user's hand from a capture device having a field of view that captures the back of the user's hand, and corresponding initial depth information that represent an initial actual position of the hand, the initial actual position being spaced from the capture device by an initial actual distance;using the initial depth information, project the initial actual image into the virtual environment to an initial virtual position, the initial virtual position corresponding to the initial actual position and spaced from a virtual object or a physical object by a virtual distance;provide a segmented version of the initial actual image to a display device for display in the virtual environment at the initial virtual position by partitioning the initial actual image into a plurality of segments, each comprising a respective set of pixels of the initial actual image, the segmented version being one of the plurality of segments and including pixels representing the back of the user's hand;receive a subsequent actual image of the back of the user's hand and corresponding subsequent depth information that represent a subsequent actual position of the hand that is different from the initial actual position, the subsequent actual position being spaced from the capture device by a subsequent actual distance different from the initial actual distance;using the subsequent depth information, project the subsequent actual image into the virtual environment to a subsequent virtual position, the subsequent virtual position being spaced from the virtual object or the physical object by the virtual distance;provide a segmented version of the subsequent actual image to the display device for display in the virtual environment at the subsequent virtual position, by partitioning the subsequent actual image into a plurality of segments, each comprising a respective set of pixels of the subsequent image, the segmented version of the subsequent actual image being one of the plurality of segments and including pixels representing the back of the user's hand; anddetect a collision between a three-dimensional representation of the hand and the virtual object in the virtual environment or the physical object.