Combined optical and electronic image stabilization

摘要

Techniques to improve a digital image capture device's ability to stabilize a video stream are presented. According to some embodiments, improved stabilization of captured video frames is provided by intelligently harnessing the complementary effects of both optical image stabilization (OIS) and electronic image stabilization (EIS). In particular, OIS may be used to remove intra-frame motion blur that is typically lower in amplitude and dominates with longer integration times, while EIS may be used to remove residual unwanted frame-to-frame motion that is typically larger in amplitude. The techniques disclosed herein may also leverage information provided from the image capture device's OIS system to perform improved motion blur-aware video stabilization strength modulation, which permits better video stabilization performance in low light conditions, where integration times tend to be longer, thus leading to a greater amount of motion blurring in the output stabilized video.

主权项

1. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:
capture a first time-sequenced set of images with an image capture device comprising a lens assembly mechanically coupled to a camera module, wherein each image in the first time-sequenced set of images is associated with one or more capture times, one or more camera motion information samples, and one or more optical image stabilization (OIS) lens motion information samples; perform a high-pass filtering operation on the one or more camera motion information samples of the first time-sequenced set of images; apply a first amount of OIS to one or more parts of the lens assembly of the image capture device based, at least in part, on only portions of the one or more camera motion information samples that passed the high-pass filtering operation; select a current image from the first time-sequenced set of images; identify a first electronic image stabilization (EIS) transform matrix for a first portion of the current image based, at least in part, on one or more camera motion information samples associated with the first portion of the current image; adjust the first EIS transform matrix based, at least in part, on the applied first amount OIS; apply the adjusted first EIS transform matrix to the first portion of the current image to stabilize the first portion of the current image; and store the stabilized first portion of the current image in a memory.