Method for enhancing depth images of scenes using trellis structures

摘要

An image for a virtual view of a scene is generated based on a set of texture images and a corresponding set of depth images acquired of the scene. A set of candidate depths associated with each pixel of a selected image is determined. For each candidate depth, a cost that estimates a synthesis quality of the virtual image is determined. The candidate depth with a least cost is selected to produce an optimal depth for the pixel. Then, the virtual image is synthesized based on the optimal depth of each pixel and the texture images. The method also applies first and second depth enhancement before, and during view synthesis to correct errors or suppress noise due to the estimation or acquisition of the dense depth images and sparse depth features.

主权项

1. A method for generating a virtual image for a virtual view of a 3D scene based on a texture image and a depth image, wherein the depth image includes an associated dense depth image and associated sparse dense features, comprising steps of:
applying a first depth enhancement to the dense depth image and the sparse depth features to obtain a first enhanced depth image; determining a plurality of candidate depths for each pixel of the first enhanced depth image; determining; for each candidate depth, a cost that estimates a synthesis quality of the virtual image; selecting a candidate depth with a least cost to produce an enhanced depth for the pixel in the first enhanced depth image; and applying view synthesis to the first depth images and the corresponding texture image to generate the virtual image, wherein the least cost is based on stereo cost, a color intensity consistency between two views, and a color difference cost wherein the steps are performed in a processor.