Video processing

摘要

A method and apparatus for processing video is disclosed. In an embodiment, image features of an object within a frame of video footage are identified and the movement of each of these features is tracked throughout the video footage to determine its trajectory (track). The tracks are analyzed, the maximum separation of the tracks is determined and used to determine a texture map, which is in turn interpolated to provide an unwrap mosaic for the object. The process may be iterated to provide an improved mosaic. Effects or artwork can be overlaid on this mosaic and the edited mosaic can be warped via the mapping, and combined with layers of the original footage. The effect or artwork may move with the object's surface.

主权项

1. A method for processing video showing at least one deforming object comprising:
(i) capturing a sequence of video frames showing the at least one deforming object; (ii) identifying a plurality of features of the at least one deforming object within the frames, each feature having a corresponding pixel; (iii) tracking the movement of the plurality of identified features between frames, the movement of each feature of the plurality of identified features being represented by a respective track of a corresponding plurality of tracks; (iv) analyzing the plurality of tracks representing the movement of the plurality of identified features to determine, for each pair of the tracks, a difference in position along the respective tracks; (v) associating, with each track, a 2D coordinate such that a difference between a pair of the 2D coordinates associated with a corresponding pair of the tracks is similar to a maximally separated distance in video frames of the sequence of video frames for the corresponding pair of the tracks; (vi) building a map using the tracks and the associated 2D coordinates of each track; (vii) accounting for camera roll and zoom using a spatial regularizer comprising: a minimizing function comprising a determinant of a discrete mapping Jacobian and an overall 2D affine transform for each frame; and a temporal coherence term comprising a mapping term for each frame; and (viii) determining an energy for a mosaic created using the map and, if the mosaic does not match predetermined standards, performing energy minimization iteratively until said standards are reached, the predetermined standards being that a change in the energy from a previously calculated energy is less than a threshold value.