Array Cameras Including an Array Camera Module Augmented with a Separate Camera

摘要

Systems with an array camera augmented with a conventional camera in accordance with embodiments of the invention are disclosed. In some embodiments, the array camera is used to capture a first set of image data of a scene and a conventional camera is used to capture a second set of image data for the scene. An object of interest is identified in the first set of image data. A first depth measurement for the object of interest is determined and compared to a predetermined threshold. If the first depth measurement is above the threshold, a second set of image data captured using the conventional camera is obtained. The object of interest is identified in the second set of image data and a second depth measurement for the object of interest is determined using at least a portion of the first set of image data and at least a portion of the second set of image data.

主权项

1. An array camera, comprising:
an array camera module comprising a plurality of cameras that capture images of a scene from different viewpoints; a separate camera located a fixed baseline distance from the array camera module, where the separate camera captures an image of the scene from a different viewpoint to the viewpoints of the cameras in the array camera module; a processor; and memory in communication with the processor; wherein software directs the processor to:
obtain a set of images captured from different viewpoints using the cameras in the array camera module and the separate camera, where the images in the set of images are captured from different viewpoints;select a reference viewpoint relative to the viewpoints of the set of images captured from different viewpoints;determine depth estimates for pixel locations in an image from the reference viewpoint using the images in the set of images captured by the array camera module, wherein generating a depth estimate for a given pixel location in the image from the reference viewpoint comprises:
identifying pixels in the images captured by the array camera module that correspond to the given pixel location in the image from the reference viewpoint based upon expected disparity at a plurality of depths;comparing the similarity of the corresponding pixels identified at each of the plurality of depths; andselecting the depth from the plurality of depths at which the identified corresponding pixels have the highest degree of similarity as a depth estimate for the given pixel location in the image from the reference viewpoint; andgenerate a depth map for an image in the set of images captured by the separate camera using the depth estimates for pixel locations in an image from the reference viewpoint by:
identifying pixels in an image captured by the separate camera corresponding to pixels in the image from the reference viewpoint for which depth estimates were determined using images in the set of images captured by the cameras in the array camera module; andapplying depth estimates determined using images in the set of images captured by the array camera module to the corresponding pixels in the image captured by the separate camera.