Audiovisual capture and sharing framework with coordinated user-selectable audio and video effects filters

摘要

Coordinated audio and video filter pairs are applied to enhance artistic and emotional content of audiovisual performances. Such filter pairs, when applied in audio and video processing pipelines of an audiovisual application hosted on a portable computing device (such as a mobile phone or media player, a computing pad or tablet, a game controller or a personal digital assistant or book reader) can allow user selection of effects that enhance both audio and video coordinated therewith. Coordinated audio and video are captured, filtered and rendered at the portable computing device using camera and microphone interfaces, using digital signal processing software executable on a processor and using storage, speaker and display devices of, or interoperable with, the device. By providing audiovisual capture and personalization on an intimate handheld device, social interactions and postings of a type made popular by modern social networking platforms can now be extended to audiovisual content.

主权项

1. An audiovisual processing method comprising:
using a portable computing device to capture corresponding audio and video streams from microphone and camera interfaces thereof and to store, at least temporarily, the captured audio and video streams in storage; on the portable computing device, selecting a first effect profile from a plurality of predefined effect profiles, the predefined effect profiles selective for respective and coordinated pairings of audio and visual filters to be applied to the audio and video streams, respectively; processing the audio and video streams in respective and coordinated audio and video pipelines, wherein the coordination is based at least in part on application of the coordinated audio and visual filters to the respective audio and video streams, wherein the processing the audio and video streams includes:
in the audio pipeline, segmenting the audio stream into plural segments and mapping individual ones of the segments to respective subphrase portions of a phrase template for a target song; andin the video pipeline, segmenting the video stream and mapping segments thereof in correspondence with the audio segmentation and mapping; audiovisually rendering the audio and video streams, with the coordinated audio and visual filters applied, to a display of the portable computing device; and thereafter storing, transmitting or posting the rendered audiovisual content.