Systems, methods, and media for capturing scene images and depth geometry and generating a compensation image

摘要

In some embodiments, systems for capturing scene images and depth geometry are provided, comprising a projector, an optical sensor, and a digital processing device. The projector is capable of being defocused with respect to a scene and projects light having a shifting periodic illumination pattern on the scene. The optical sensor has a plurality of pixels and detects a portion of the radiance of at least one image of the scene at each of the pixels. The digital processing device is capable of being coupled to the optical sensor and obtains a temporal radiance profile from the radiance over a time period for each of the pixels, determines an amount of projection defocus at each of the of pixels using the temporal radiance profile, and, at each of the pixels, computes a depth to the scene at the pixel using the amount of projection defocus at the pixel.

主权项

1. A system for capturing scene images and depth geometry, comprising:
a projector having a projector lens that is capable of being defocused with respect to a scene and that projects a defocused image having a shifting periodic illumination pattern on the scene; an optical sensor having a plurality of pixels and that detects a portion of the radiance of at least one image of the scene at each of the pixels; and a digital processing device capable of being coupled to the optical sensor that:
obtains a temporal radiance profile from the radiance over a time period for each of the pixels, wherein the temporal radiance profile comprises a first radiance and a second radiance of a point in the scene that is out of focus and that corresponds to the pixel, wherein the first radiance is a first response of a defocus kernel of the point in the scene to the shifting periodic illumination pattern at a first time, wherein the second radiance is a second response of the defocus kernel to the shifting periodic illumination pattern at a second time, wherein the defocus kernel is based on a distance between the point in the scene to the projector lens, and wherein the first time and the second time are different;determines, for each of the pixels, a plurality of coefficients indicative of an amount of projection defocus at the point in the scene that corresponds to the pixel using the first radiance and the second radiance associated with the pixel;determines an amount of projection defocus at each of the pixels using the plurality of coefficients determined for the pixel, wherein the amount of projection defocus at each of the pixels is based on a distance between the point in the scene and the projector lens;at each of the pixels, computes a depth to the scene at the pixel using the determined amount of projection defocus at the pixel;generates a depth map that segments the scene into layers using the depth to the scene at each of the pixels; andgenerates images of the scene with varying focus using the depth map.