AUTOMATIC GEOMETRY AND LIGHTING INFERENCE FOR REALISTIC IMAGE EDITING

摘要

Image editing techniques are disclosed that support a number of physically-based image editing tasks, including object insertion and relighting. The techniques can be implemented, for example in an image editing application that is executable on a computing system. In one such embodiment, the editing application is configured to compute a scene from a single image, by automatically estimating dense depth and diffuse reflectance, which respectively form the geometry and surface materials of the scene. Sources of illumination are then inferred, conditioned on the estimated scene geometry and surface materials and without any user input, to form a complete 3D physical scene model corresponding to the image. The scene model may include estimates of the geometry, illumination, and material properties represented in the scene, and various camera parameters. Using this scene model, objects can be readily inserted and composited into the input image with realistic lighting, shadowing, and perspective.

主权项

1. A method for automatically estimating illumination sources associated with a digital image depicting a scene, comprising:
detecting light sources depicted in the image that meet a given thresholding requirement; pruning the detected light sources using inferred depth; and estimating light outside of the view frustum associated with the image using a dataset of image-based lights comprising spherical high dynamic range images, wherein each spherical image is sub-sampled into rectilinear projections and matched to the image and one or more top matching candidate image based lights are used as distinct sources of light.