| 摘要 |
<p>Acoustic-assisted image processing is achieved, in accordance with the invention by a novel method and apparatus in which an audio signal is sampled at an audio-domain sampling rate; a first viseme sequence is generated at a first rate in response to the sampled audio signal, the first rate corresponding to an audio-domain sampling rate; the first viseme sequence is transformed into a second viseme sequence at second rate using a predetermined set of transformation criteria, the second rate corresponding to a video-domain frame rate; and an image is processed in response to the second viseme sequence. In an illustrative example of the invention, a video image of a face of a human speaker is animated using a three-dimensional wire-frame facial model upon which a surface texture is mapped. The three-dimensional wire-frame facial model is structurally deformed in response to a rate-transformed viseme sequence extracted from a speech signal so that the mouth region of the video image moves in correspondence with the speech. Advantageously, the animation is accomplished in real time, works with any speaker, and has no limitations on vocabulary, nor requires any special action on the part of the speaker. <IMAGE></p> |
