Dynamic hand gesture recognition using depth data

摘要

The subject disclosure is directed towards a technology by which dynamic hand gestures are recognized by processing depth data, including in real-time. In an offline stage, a classifier is trained from feature values extracted from frames of depth data that are associated with intended hand gestures. In an online stage, a feature extractor extracts feature values from sensed depth data that corresponds to an unknown hand gesture. These feature values are input to the classifier as a feature vector to receive a recognition result of the unknown hand gesture. The technology may be used in real time, and may be robust to variations in lighting, hand orientation, and the user's gesturing speed and style.

主权项

1. In a computing environment, a method performed at least in part on at least one processor, the method comprising:
sensing depth data for a plurality of frames that include hand movement; for the plurality of frames, processing the depth data, wherein processing the depth data comprises:
segmenting the depth data to isolate a hand represented in the plurality of frames;determining that a principal direction of the hand is not pointing in a predefined direction in an image plane;based on the determining, rotating the hand such that the palm of the hand is substantially parallel to the image plane and such that the principal direction of the hand is pointing toward the predefined direction in the image plane;performing normalization on the rotated hand to provide normalized hand data to compensate for scale and a relative hand size of the hand; andextracting feature values corresponding to the normalized hand data; and recognizing the hand movement as a hand gesture based upon the feature values provided to a classifier.