System and method for visualizing synthetic objects within real-world video clip

摘要

Embodiments of the present invention provide a system and a method for visualizing synthetic objects within a real-world video clip. The embodiments allow interactive, synthetic objects or characters to appear as if they are “embedded” inside or form part of a video clip. These objects move independently of the environment within the video clip and are controlled by a user. In one embodiment the video frames are set as textures in the furthermost layer in a 3D environment viewer or 3D “engine”. In another embodiment separate layers of video frames are superimposed with the synthetic objects on an output screen and synchronizing the movements and the field of view settings to give the perception that only one camera is used.

主权项

1. A method of visualizing synthetic objects within a real-world video clip, comprising:
a. building into a transparent 3D scene, via a processor, at least one of one or more objects and features from the real-world video clip, wherein the objects and features are at least one of static and dynamic based on the real-world video clip; b. visualizing, in a viewer application, the built transparent 3D scene and dynamic objects overlaid on a background layer of the real-world video clip; c. adding additional synthetic objects, static or dynamic, within a transparent 3D scene representation of the features in the real-world video clip, the transparent 3D scene representation being based on the visualization of the built transparent 3D scene; d. using a 3D transparent scene as an occlusion layer, wherein an object or part of the occlusion layer appearing behind a transparent object is occluded and occupying a place with the real-world video clip; and e. controlling the movement of the synthetic objects independent from the movement of the video clip, wherein the viewer application adjusts a speed of changing of individual frames of the video clip from zero FPS (frames per second) up to rates of 60 FPS to simulate a standard video player and to achieve a high rate of control over the speed of movement through the video clip.