Occluded object recognition

摘要

An object observable in various images of an environment is recognized as being the same object, even when partially occluded in some images. In one particular implementation, the object is a hand which may be used for gesture input to an augmented reality system. A flexible similarity measurement metric is computed to aid in determining whether hands captured through 2D images of the environment are the same or different, regardless of whether the hands are fully observed or partially occluded. The similarity measurement metric accommodates for translation, rotation, and scaling of the hand to enable recognition of a common hand through a sequence of images even if that hand is partially occluded in some images.

主权项

1. A system comprising:
one or more processors; one or more cameras to capture at least a first image and a second image of an environment, the first image showing at least in part a hand that is not occluded from the one or more cameras and the second image showing at least in part the hand that is at least partly occluded from the one or more cameras; one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
producing a first curvature map to represent a curvature of the hand represented in the first image;generating a first distance map using distances between points on the first curvature map, wherein the first distance map includes a first distance between a first point on the first curvature map and a second point on the first curvature map and a second distance between the second point on the first curvature map and a third point on the first curvature map;producing a second curvature map to represent a curvature of the hand represented in the second image;generating a second distance map using distances between points on the second curvature map, wherein the second distance map includes a third distance between a first point on the second curvature map and a second point on the second curvature map and a fourth distance between the second point on the second curvature map and a third point on the second curvature map;generating a first adjusted curvature map and a second adjusted curvature map to account for depth variation based at least in part on the first distance map and the second distance map so that the first adjusted curvature map and the second adjusted curvature map represent the hand represented in the first image is at approximately a same distance from the one or more cameras as the hand represented in the second image;comparing the first adjusted curvature map to the second adjusted curvature map; anddetermining that the hand represented in the first image is the hand represented in the second image based at least in part on results obtained from comparing the first adjusted curvature map to the second adjusted curvature map.