Audio visual dubbing system and method.

摘要

A system and method for replacing the original sound track of a video or film sequence depicting a dubbee with an audio signal indicative of substituted utterances by a dubber associates frames of a stored or transmitted sequence with facial feature information associated with utterances in a language spoken by the dubber. The frames of the sequence are modified by conforming mouth formations of the dubbee in accordance with the facial feature information using a look up table relating detected dubber utterances to a set of speaker independent mouth formations or actual mouth formations of the dubber. In accordance with the present invention, a viewer of a currently transmitted or previously stored program may manually select between viewing the original broadcast or program or viewing a dubbed version in which a second audio track indicative of utterances in a language different than that of the dubbee is reproduced. Once such a selection is made, the second audio track is utilized to conform the mouth movements of the dubbee to those of someone making the dubber's utterance.