Method and system for emotion and behavior recognition

摘要

A method and system for recognizing behavior is disclosed, the method includes: capturing at least one video stream of data on one or more subjects; extracting body skeleton data from the at least one video stream of data; computing feature extractions on the extracted body skeleton data to generate a plurality of 3 dimensional delta units for each frame of the extracted body skeleton data; generating a plurality of histogram sequences for each frame by projecting the plurality of 3 dimensional delta units for each frame to a spherical coordinate system having a plurality of spherical bins; generating an energy map for each of the plurality of histogram sequences by mapping the plurality of spherical bins versus time; applying a Histogram of Oriented Gradients (HOG) algorithm on the plurality of energy maps to generate a single column vector; and classifying the single column vector as a behavior and/or emotion.

主权项

1. A method for recognizing behavior, the method comprising:
capturing at least one video stream of data on one or more subjects, wherein the at least one video stream data comprises a plurality of frames; extracting body skeleton data from each frame of the at least one video stream of data; computing feature extractions on each joint of the extracted body skeleton data for each frame of the extracted body skeleton data, wherein the extracted features of the joint of the body skeleton describes a speed feature, a movement feature, and a pose feature of the joint of the body skeleton data, and wherein the speed feature describes a relative position between joint n in frame t and every joint in a preceding frame (t−k), where k is a parameter of speed estimation step size; generating a plurality of histograms for each frame by respectively projecting the extracted features for each frame to a spherical coordinate system having a plurality of spherical bins; generating an energy map for each of the plurality of histograms by mapping the plurality of spherical bins versus time; applying a Histogram of Oriented Gradients (HOG) algorithm on the plurality of energy maps to generate a single column vector; and classifying the single column vector as a behavior and/or emotion.