Recognizing gestures captured by video

摘要

Motions and gestures can be detected using a video capture element of a computing device even when the video capture element is not able to accurately capture the motion. Information about the background in the image information can be determined, and the way in which that background information is occluded can be used to determine the motion. In at least some embodiments, edges are detected in the video information. Images of foreground objects can then be isolated from edges of background images by comparing histograms of multiple frames of video. The remaining data is indicative of a direction and speed of motion, which can be used to infer a determined gesture even though that gesture was not visible in the captured video information.

主权项

1. A computer-implemented method of providing input to a computing device, comprising:
under control of one or more computer systems configured with executable instructions, determining a lighting condition using a light sensor of the computing device; capturing, while an illumination element is not active and using at least one image capture element of the computing device, a first image frame of a plurality of image frames; automatically activating the illumination element of the computing device based at least in part upon the lighting condition; capturing, while the illumination element is active and using the at least one image capture element of the computing device, a second image frame of the plurality of image frames, the illumination element causing a foreground portion of the first image frame to be distinguishable from a background portion of the first image frame; generating first histogram data relating to edge information for the first image frame of the plurality of image frames; generating second histogram data relating to edge information for the second image frame of the plurality of image frames, wherein the second image frame is adjacent in time to the first image frame; removing data relating to the first histogram data from the second histogram data; determining an amount of miscorrelation between the first histogram data and the second histogram data, the amount of miscorrelation being indicative of a type of motion; determining a shape representing the amount of miscorrelation, the shape including characteristics indicative of a speed and a direction of the motion; determining the shape corresponds to stored gesture information for at least one gesture; and identifying an input, for the computing device, associated with the stored gesture information.