Systems and methods for recognition of sign language for improved viewing experiences

摘要

Systems and methods are described herein for selecting a closed captioning language track. Control circuitry may capture an image or video of a user while the user is performing a gesture of a sign language. The control circuitry may access a database comprising a plurality of entries corresponding to a plurality of gestures in respective sign languages. The control circuitry may compare a first pixel map corresponding to the image or video of the user with pixel maps corresponding to each of the plurality of entries in order to identify a preferred sign language. The control circuitry may receive metadata associated with a media asset comprising a plurality of closed captioning language tracks, each closed captioning language track comprising closed captioning for the media asset in a different language and select one of the closed captioning language tracks that corresponds to the preferred sign language.

主权项

1. A method for selecting a closed captioning language track, the method comprising:
capturing an image or video of a user while the user is performing a first gesture of a sign language, the image or video of the user comprising a first pixel map; accessing a database comprising a plurality of entries, each entry comprising a pixel map that corresponds to a gesture in a respective sign language and an indication of the respective sign language; comparing the first pixel map to the pixel map of each of the plurality of entries, comprising, for each respective entry:
extracting a second pixel map from the respective entry;calculating an average pixel value for the second pixel map;normalizing the first pixel map using the calculated average pixel value;comparing pixels of the first pixel map to pixels of the second pixel map;identifying a pattern of pixels from the first pixel map that is within a threshold value of corresponding pixels from the second pixel map; identifying, based on the comparison, an entry of the plurality of entries corresponding to a second gesture that matches the first gesture; receiving metadata associated with a media asset, the metadata comprising a plurality of closed captioning language tracks, each closed captioning language track comprising closed captioning for the media asset in a different language; and selecting one of the closed captioning language tracks that corresponds to a sign language indicated in the identified entry.