Apparatus and method for generating new viewpoint image

摘要

An apparatus for generating a new viewpoint image includes: a new viewpoint image generation unit that shifts the foreground image region in the input image, based on a depth map to generate a new viewpoint image including a missing pixel region; a representative background pixel selection unit that selects background pixels located a predetermined number of pixels away from edge pixels, as representative background pixels; a texture direction map calculation unit that calculates a texture direction of each of the selected representative background pixels, to calculate a texture direction map indicating texture directions, each corresponding to one of missing pixels, based on locations of the representative background pixels and locations of the missing pixels included in the missing pixel region; and a hole-filling processing unit that fills the missing pixel region with the obtained background pixels in accordance with the texture direction map.

主权项

1. An apparatus for generating a new viewpoint image, which generates, from an input image, a new viewpoint image that is an image of the input image viewed from a new viewpoint, the input image including a foreground image region that is an image region of an object and a background image region other than the foreground image region, the apparatus comprising:
a generation unit configured to shift the foreground image region in the input image, based on a depth map corresponding to the input image to generate a new viewpoint image including a missing pixel region, according to a depth image based rendering (DIBR) technique; a selection unit configured to select background pixels located a predetermined number of pixels away from edge pixels, from among background pixels included in the background image region, as representative background pixels, the edge pixels indicating a boundary between the foreground image region and the background image region in the input image; a texture direction map calculation unit configured to calculate a texture direction of each of the selected representative background pixels, the texture direction being an edge direction of texture in the background image region for each of the representative background pixels, and to calculate, for all of missing pixels, a texture direction map indicating texture directions each corresponding to one of the missing pixels, based on locations of the representative background pixels and locations of the missing pixels included in the missing pixel region; and a hole-filling processing unit configured to entirely fill, at one time, the missing pixel region with the background pixels without using the pixels of the foreground image region, while considering continuity at a boundary with the background image region and without considering discontinuity at a boundary with the foreground image region, in accordance with the texture direction map indicating the texture directions of the missing pixels.