AUTOMATIC GENERATION OF VIDEO AND DIRECTIONAL AUDIO FROM SPHERICAL CONTENT

摘要

A spherical content capture system captures spherical video and audio content. In one embodiment, captured metadata or video/audio processing is used to identify content relevant to a particular user based on time and location information. The platform can then generate an output video from one or more shared spherical content files relevant to the user. The output video may include a non-spherical reduced field of view such as those commonly associated with conventional camera systems. Particularly, relevant sub-frames having a reduced field of view may be extracted from each frame of spherical video to generate an output video that tracks a particular individual or object of interest. For each sub-frame, a corresponding portion of an audio track is generated that includes a directional audio signal having a directionality based on the selected sub-frame.

主权项

1. A method for generating a video with corresponding audio, comprising:
receiving a spherical video; receiving a plurality of audio channels representing audio captured concurrently with the spherical video, each of the plurality of audio channels comprising directional audio corresponding to a different direction; mapping different spatial regions of the spherical video to different audio channels; for each of a plurality of frames of the spherical video, extracting a sub-frame having a reduced field of view relative to a field of view of the spherical video; for each of the sub-frames, determining, by a processor, one or more of the spatial regions overlapping the sub-frame and generating a portion of an audio stream from one or more different audio channels corresponding to the one or more spatial regions overlapping the sub-frame; and outputting the generated video and the audio stream.