Systems and Methods for Generating 360 Degree Mixed Reality Environments

摘要

Systems and methods for generating a 360 degree mixed virtual reality environment that provides a 360 degree view of an environment in accordance with embodiments of the invention are described. In a number of embodiments, the 360 degree mixed virtual reality environment is obtained by (1) combining one or more real world videos that capture images of an environment with (2) a virtual world environment that includes various synthetic objects that may be placed within the real world clips. Furthermore, the virtual objects embedded within the 360 degree mixed reality environment interact with the real world objects depicted in the real world environment to provide a realistic mixed reality experience.

主权项

1. A system for visualizing controllable virtual 3D objects within a mixed reality application using real-world video captured from a plurality of cameras, comprising:
a processor; memory including a mixed reality application; and wherein the mixed reality application directs the processor to: obtain a plurality of real-world videos captured by one or more cameras, each real-world video capturing a different portion of a surrounding real-world environment; for each real-world video, generate a 3D mixed reality environment that includes (1) the real-world video as a background layer of the 3D mixed reality environment, (2) an occlusion layer that includes one or more transparent 3D objects that replicate real-world objects within the real-world video, and (3) one or more virtual synthetic objects, wherein the virtual synthetic objects interact with the transparent 3D objects based on 3D space locations of the objects; and combine at least one 3D mixed reality environment generated for a real-world video with a different 3D mixed reality environment generated for a different real-world video to provide a 3D mixed reality environment that replicates a larger portion of the surrounding real-world environment for use by the mixed reality application.