Generating animation from actor performance

摘要

A motion library can be created by generating motion feature vectors for at least some of multiple frames of a video sequence using a 3D mesh, each motion feature vector corresponding to characteristics of the body deformation in one of the frames. The A user can select a subset of the frames. For each frame in the subset, the user can define settings for controls of an animation character, the settings selected by the user to correspond to the body deformation in the respective frame. Mappings are generated using the settings and the motion feature vectors, the mappings regulating the controls based on multiple motion feature vectors. The motion library can be used to generate an animation from an actor performance.

主权项

1. A computer-implemented method comprising:
receiving, in a computer system, video content captured of an actor performing an act that involves a body deformation, the video content including a sequence of frames and recorded without using physical markers applied to the actor that are designed for motion capture; adapting, using the video content, a three-dimensional (3D) mesh in the computer system such that the 3D mesh corresponds to at least a part of the actor, the 3D mesh configured to deform according to the body deformation; generating motion feature vectors for at least some of the frames using the 3D mesh, each motion feature vector corresponding to characteristics of the body deformation in one of the frames; specifying, for each of the motion feature vectors, settings for controls of an animation character, the settings specified using mappings that were generated in advance based on an earlier training session with the actor, wherein the settings are specified using information from two-dimensional tracking of facial contours to provide detailed local motion information and information from three-dimensional tracking of rigid motion of one or more body parts of the actor and generating an animation with the animation character using the settings, the animation including motion by the animation character corresponding to the body deformation.