Devices, systems, and methods for examining the interactions of objects in an enhanced scene

摘要

Systems, devices, and methods obtain a sequence of images of a physical scene that includes a physical representation of a first object; calculate a sequence of first transform values of the physical representation of the first object based on the sequence of images store the sequence of first transform values; generate an enhanced scene; maintain the first object in the enhanced scene at positions and orientations that are indicated by the sequence of first transform values; receive an indication of selected transform values in the sequence of first transform values; retrieve the selected transform values; and generate a replay image of the enhanced scene, from a second observer viewpoint, that shows the first object at the position and the orientation that are indicated by the selected transform values.

主权项

1. A method comprising:
obtaining a sequence of images of a physical scene that includes a physical representation of a first virtual object, wherein a shape of the physical representation is different than a shape of the first virtual object; calculating a sequence of first transform values of the physical representation of the first virtual object based on the sequence of images, wherein transform values in the sequence of first transform values indicate a position and an orientation of the physical representation of the first virtual object in the physical scene; storing the sequence of first transform values of the physical representation of the first virtual object on one or more computer-readable media; generating an enhanced scene, wherein the enhanced scene includes the first virtual object and includes a second virtual object at a second position and a second orientation; maintaining the first virtual object in the enhanced scene at positions and orientations that are indicated by the transform values in the sequence of first transform values; generating images of the enhanced scene from original observer viewpoints, wherein each of the images of the enhanced scene is generated from a respective original observer viewpoint of the original observer viewpoints, wherein each original observer viewpoint has a respective position and a respective orientation, and wherein the original observer viewpoints are determined based on a position and an orientation of a display device; receiving an indication of a selected transform value in the sequence of first transform values of the physical representation of the first virtual object; retrieving the selected transform value from the sequence of first transform values; generating a first replay image of the enhanced scene that shows the first virtual object at the position and the orientation that are indicated by the selected transform value from a first replay observer viewpoint, wherein the first replay observer viewpoint has a position that is different than the respective positions of each of the original observer viewpoints, and wherein the first replay observer viewpoint is determined based on the position and the orientation of the display device; receiving a new transform value for the first virtual object that is different from the selected transform value; changing the transform value of the first virtual object to the new transform value; generating a second replay image of the enhanced scene that shows the first virtual object at the position and the orientation that are indicated by the new transform value from a second replay observer viewpoint, wherein the second replay observer viewpoint has a position that is different than the respective positions of each of the original observer viewpoints and the first replay observer viewpoint, and wherein the second replay observer viewpoint is determined based on the position and the orientation of the display device; detecting an interaction between the first virtual object and the second virtual object in the enhanced scene, wherein the interaction is a predetermined proximity, which is greater than zero, of the first virtual object and the second virtual object; and in response to detecting the interaction, adding an indicator of the interaction to the second replay image of the enhanced scene.