MIXED REALITY INTERACTIONS

摘要

Embodiments that relate to interacting with a physical object in a mixed reality environment via a head-mounted display are disclosed. In one embodiment a mixed reality interaction program identifies an object based on an image from captured by the display. An interaction context for the object is determined based on an aspect of the mixed reality environment. A profile for the physical object is queried to determine interaction modes for the object. A selected interaction mode is programmatically selected based on the interaction context. A user input directed at the object is received via the display and interpreted to correspond to a virtual action based on the selected interaction mode. The virtual action is executed with respect to a virtual object associated with the physical object to modify an appearance of the virtual object. The modified virtual object is then displayed via the display.

主权项

1. A mixed reality interaction system for interacting with a physical object in a mixed reality environment, the mixed reality interaction system comprising:
a head-mounted display device operatively connected to a computing device, the head-mounted display device including a display system for presenting the mixed reality environment and a plurality of input sensors including a camera for capturing an image of the physical object; and a mixed reality interaction program executed by a processor of the computing device, the mixed reality interaction program configured to: identify the physical object based on the captured image; determine an interaction context for the identified physical object based on one or more aspects of the mixed reality environment; query a stored profile for the physical object to determine a plurality of interaction modes for the physical object; programmatically select a selected interaction mode from the plurality of interaction modes based on the interaction context; receive a user input directed at the physical object via one of the input sensors of the head-mounted display device; interpret the user input to correspond to a virtual action based on the selected interaction mode; execute the virtual action with respect to a virtual object associated with the physical object to thereby modify an appearance of the virtual object; and display the virtual object via the head-mounted display device with the modified appearance.