Interactive mixed reality system and uses thereof

摘要

An interactive mixed reality simulator is provided that includes a virtual 3D model of internal or hidden features of an object; a physical model or object being interacted with; and a tracked instrument used to interact with the physical object. The tracked instrument can be used to simulate or visualize interactions with internal features of the physical object represented by the physical model. In certain embodiments, one or more of the internal features can be present in the physical model. In another embodiment, some internal features do not have a physical presence within the physical model.

主权项

1. An interactive mixed reality system, comprising:
a physical model of an object; a tracked instrument for interacting with and penetrating the physical model, wherein the physical model is penetrable through an external or internal surface at any location and at any orientation of the tracked instrument, wherein the tracked instrument is moveable freely within the physical model; a tracking system for tracking the position and the orientation of the tracked instrument within and near the physical model; a virtual model, stored in a memory, comprising one or more 3-D virtual representations of features of the object, the features comprising internal or hidden features of the object, wherein the one or more 3-D virtual representations are abstract or concrete representations; a 3-D perspective controller device for selecting a perspective of the virtual model, wherein the perspective is selectable by a user independently of the position and orientation of the tracked instrument; and a simulation module that
accesses the virtual model stored in the memory;receives input from the tracking system regarding the position and the orientation of the tracked instrument within or near the physical model;determines, based on the input from the tracking system, a corresponding interaction between the tracked instrument within or near the physical model and the one or more 3-D virtual representations of the virtual model;determines a consequence of the corresponding interaction; andrenders, to a display, from the perspective of the virtual model selected on the 3-D perspective controller, a particular set of the one or more 3-D virtual representations, and the consequence of the corresponding interaction.