SYNCHRONIZING VIRTUAL ACTOR'S PERFORMANCES TO A SPEAKER'S VOICE

摘要

A system for generating and displaying holographic visual aids associated with a story to an end user of a head-mounted display device while the end user is reading the story or perceiving the story being read aloud is described. The story may be embodied within a reading object (e.g., a book) in which words of the story may be displayed to the end user. The holographic visual aids may include a predefined character animation that is synchronized to a portion of the story corresponding with the character being animated. A reading pace of a portion of the story may be used to control the playback speed of the predefined character animation in real-time such that the character is perceived to be lip-syncing the story being read aloud. In some cases, an existing book without predetermined AR tags may be augmented with holographic visual aids.

主权项

1. A method for synchronizing performance of a virtual object to performance of a person, comprising:
detecting that a first utterance has been spoken by the person; determining a speed at which the person spoke the first utterance; detecting that a portion of a second utterance has been spoken by the person; and displaying a sequence of images of the virtual object at a rate corresponding with the determined speed of the first utterance in response to detecting that the portion of the second utterance has been spoken by the person, the sequence of images including a sequence of mouth shape images displayed such that the virtual object appears to speak the second utterance at the determined speed of the first utterance.