Estimating fingertip position using image analysis

摘要

A computing device and/or application executing on the device can utilize fingertip tracking using a camera. However, when the fingertip is in a dead zone (an area that is not viewable by the camera), the fingertip tracking cannot function properly. Nevertheless, the position of the fingertip, when in the dead zone, can still be estimated. An image of a user's hand can be captured by at least one camera. A portion of a pointing finger can be detected in the captured image. An orientation of the portion of the pointing finger can be determined. One or more joint lines of the pointing finger can be identified. Based on data about a slant and/or a bend of the pointing finger obtained using information relating to the identified joint line(s), and/or on data obtained via calibration and/or historic/current usage, the position of the fingertip, when in the dead zone, can be approximated.

主权项

1. A computer-implemented method comprising:
capturing, using a camera of a computing device, an image representing a hand of a user of the computing device; retrieving, from a database, historical characteristic data for the hand of the user while the hand of the user is in a field of view of the camera, the historical characteristic data including (i) historical characteristic device data relating to physical dimensions of the computing device, and (ii) historical characteristic user data relating to physical dimensions of a pointing finger of the hand of the user; determining a portion of the pointing finger of the hand in the captured image based at least in part on the historical characteristic data; determining an orientation of the portion of the pointing finger; detecting one or more linear features in the portion of the pointing finger that are less than a threshold amount of deviation from being orthogonal to the determined orientation of the portion of the pointing finger; identifying one or more joint lines in the portion of the pointing finger based at least in part upon the one or more detected linear features; calculating a slant amount for at least one of a first finger segment or a second finger segment, wherein the calculation of the slant amount utilizes at least one of a brightness contrast between at least two palmar side areas on the portion of the pointing finger or a measurement of distance between midpoints of at least two identified joint lines; calculating a bend amount for at least one of the first or the second finger segment, wherein the calculation of the bend amount is based at least in part upon evaluating one or more contrasts in brightness between the one or more identified joint lines and one or more adjacent finger segments; determining a position of a fingertip of the pointing finger based at least in part upon the calculated slant amount and the calculated bend amount for the at least one of the first or the second finger segment; and causing a cursor rendered on a display screen of the computing device to move correspondingly on the display screen based at least in part on the position of the fingertip of the pointing finger.