System and Method For Automated Sign Language Recognition

摘要

A method for sign language recognition includes receiving, with an input device, an input based on a plurality of hand movements and postures of a user that correspond to a sequence of signs, extracting a plurality of features from the input corresponding to the plurality of hand movements and postures, identifying, a start of the sequence of signs in the input based on a first set of features in the plurality of features and a first Hidden Markov Model (HMM) stored in the memory and identifying a first sign in the input based on a second set of features in the plurality of features and a second HMM stored in the memory. The method also includes generating an output corresponding to the first sign from the input.

主权项

1. A training method for automated sign language recognition comprising:
receiving, with an input device, a plurality of training inputs corresponding to a plurality of predetermined sequences of signs from hand movements and postures of a user; extracting, with the processor, a first set of features from the training inputs corresponding to hand movements and postures of the user for each sign in the plurality of predetermined sequences of signs; generating, with the processor, a first Hidden Markov Model (HMM) based on the first set of features from the training inputs corresponding to hand movements and postures for each sign in the predetermined sequences of signs; extracting, with the processor, a second set of features from the training inputs corresponding to hand movements and postures of the user for transitions between signs in the plurality of predetermined sequences of signs; generating, with the processor, a second HMM based on the second set of features from the training inputs and the predetermined sequences of signs; extracting, with the processor, a third set of features from the training inputs corresponding to hand movements and postures of the user for starting and ending each predetermined sequence of signs in the plurality of predetermined sequences of signs; generating with the processor a third HMM based on the third set of features from the training inputs and the predetermined sequences of signs; and storing, with the processor, the first HMM, second HMM, and third HMM in a memory for recognition of additional signs received from the input device.