REAL-TIME IMMERSIVE MEDIATED REALITY EXPERIENCES

摘要

The invention relates to creating real-time, immersive mediated reality environments using real data collected from a physical event or venue. The invention provides a virtual participant with the ability to control their viewpoint and freely explore the venue, in real time by synthesizing virtual data corresponding to a requested virtual viewpoint using real images obtained from data collectors or sources at the venue. By tracking and correlating real and virtual viewpoints of virtual participants, physical objects, and data sources, systems and methods of the invention can create photo-realistic images for perspective views for which there is not physically present data source. Systems and methods of the invention also relate to applying effect objects to enhance the immersive experience including virtual guides, docents, text or audio information, expressive auras, tracking effects, and audio.

主权项

1. A system for creating a mediated reality environment, said system comprising a server computing system comprising a processor coupled to a tangible, non-transitory memory, the system operable to:
receive, in real-time, real viewpoint information for one or more data collectors located at a venue; receive a virtual viewpoint from a computing device of a virtual participant, said computing device comprising a processor coupled to a tangible, non-transitory memory; receive one or more real-time images from the one or more data collectors where the one or more data collectors have a real viewpoint which intersects the virtual viewpoint, said one or more real-time images comprising a plurality of real pixels; create, using the server's processor, a real-time virtual image comprising a plurality of virtual pixels and corresponding to the virtual viewpoint by using pixel information from the one or more real-time images; and cause the computing device of the virtual participant to display the real-time virtual image.