Apparatus, system, and method for temporal domain hole filling based on background modeling for view synthesis

摘要

Embodiments of apparatuses, systems, and methods for a temporal hole filling are described. Specifically, an embodiment of the present invention may include a depth-based hole filling process that includes a background modeling technique (SGM). Beneficially, in such an embodiment, holes in the synthesized view may be filled effectively and efficiently.

主权项

1. A method for temporal domain hole filling based on background modeling for view synthesis comprising:
receiving texture information associated with a plurality of pixels in a video frame; receiving depth information associated with the plurality of pixels in the video frame; determining whether an update to a background model associated with the video frame is required in response to the texture information and the depth information associated with each of the plurality of pixels in the video frame; updating the background model in response to the texture information and the depth information, wherein the updating the background model comprises:
storing texture and depth values of a current pixel in the plurality of pixels in a buffer of supporter values in response to a determination that a difference in depth between the current pixel in the plurality of pixels and a corresponding pixel in the background model exceeds a first threshold value; andreplacing the background model with a new background model in response to a determination that a difference between the texture and depth information of the current pixel in the plurality of pixels and the corresponding pixel in the background model exceeds a second threshold value, and in response to a determination that a threshold number of supporter values have been identified in the buffer of supporter values; and filling holes in a background of a synthesized virtual view of the video frame using texture information and depth information from the background model for each of the plurality of pixels in the video frame, wherein the filling holes comprises copying at least one suitable pixel from the background model to the background of the synthesized virtual view, wherein the holes are caused by a disocclusion in the video frame.