Tracking hand/body pose

摘要

Tracking hand or body pose from image data is described, for example, to control a game system, natural user interface or for augmented reality. In various examples a prediction engine takes a single frame of image data and predicts a distribution over a pose of a hand or body depicted in the image data. In examples, a stochastic optimizer has a pool of candidate poses of the hand or body which it iteratively refines, and samples from the predicted distribution are used to replace some candidate poses in the pool. In some examples a best candidate pose from the pool is selected as the current tracked pose and the selection processes uses a 3D model of the hand or body.

主权项

1. A method of tracking pose of a hand or body comprising:
receiving a stream of images depicting the hand or body; inputting at least part of one of the images to a prediction engine to obtain a distribution over poses of the hand or body, the inputting including:
inputting the at least part of one of the images to a first layer of the prediction engine and receiving a result from the first layer; andinputting the at least part of one of the images to part of a second layer of the prediction engine, the part of the second layer being selected using the result; accessing a pool of candidate poses of the hand or body; and periodically replacing a subset of the candidate poses in the pool with samples from the distribution over poses of the hand or body.