Pull frame interpolation

摘要

A method and apparatus for performing pull frame interpolation are provided. Pull frame interpolation may include identifying a plurality of input video frames, generating a plurality of motion vectors indicating motion from a first frame of the plurality of input video frames to a second frame of the plurality of input video frames, identifying an interpolation point between the first frame and the second frame, generating a plurality of candidate interpolation motion vectors indicating motion from the first frame to the interpolation point and from the second frame to the interpolation point based on the plurality of motion vectors, selecting an interpolation motion vector from the plurality of candidate interpolation motion vectors based on a metric, and generating an interpolated frame at the interpolation point based on the selected interpolation motion vector.

主权项

1. A method comprising:
identifying a plurality of input video frames; generating a plurality of motion vectors indicating motion from a first frame of the plurality of input video frames to a second frame of the plurality of input video frames; determining a degree of smoothness of the plurality of motion vectors; using the first frame or the second frame as the interpolated frame on a condition that the degree of smoothness is above a threshold; and on a condition that the degree of smoothness is within the threshold, performing pull frame interpolation by:
identifying an interpolation point between the first frame and the second frame,jointly, using a combined energy function, identifying an occlusion and generating, based on the plurality of motion vectors, a plurality of candidate interpolation motion vectors that includes:
a candidate interpolation motion vectors indicating motion from the first frame to the interpolation point;a candidate interpolation motion vectors indicating motion from the second frame to the interpolation point; anda candidate interpolation motion vector based on motion prediction for a plurality of adjacent sites, andfor each interpolation site in a plurality of interpolation sites:
selecting an interpolation motion vector from the plurality of candidate interpolation motion vectors based on smoothness constraints within the interpolated frame and on smoothness constraints between the first frame and the second frame; andselecting an interpolation motion vector and updating the interpolated frame based on the selected interpolation motion vector, andgenerating an interpolated frame at the interpolation point based on the selected interpolation motion vector, wherein the interpolated frame includes the plurality of interpolation sites, wherein generating the interpolated frame includes:
correcting an artifact in the interpolated frame based on the interpolation motion vector by blending the interpolated frame with an average of the first frame and the second frame, wherein the degree of blending is based on a gradient of a motion field associated with the interpolation motion vector, such that a portion of the interpolated frame that has a high motion gradient is replaced with a corresponding area of the average of the first frame and the second frame.