Multi-modal sensor based emotion recognition and emotional interface

摘要

Features, including one or more acoustic features, visual features, linguistic features, and physical features may be extracted from signals obtained by one or more sensors with a processor. The acoustic, visual, linguistic, and physical features may be analyzed with one or more machine learning algorithms and an emotional state of a user may be extracted from analysis of the features. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

主权项

1. A method for determining an emotional state of a user, comprising:
extracting features including one or more acoustic features, visual features, linguistic features, physical feature s from signals obtained by one or more sensors with a processor; analyzing the features including the acoustic features, visual features, linguistic features, and physical features with one or more machine learning algorithms implemented on a processor; wherein analyzing the acoustic features, visual features, linguistic features, and physical features includes use of separate machine learning algorithms for the acoustic, visual, linguistic, and physical features; wherein a first machine learning algorithm provides feedback to a second machine learning algorithm in a serial fashion; and extracting an emotional state of the user from analysis of the features including analysis of the acoustic features, visual features, linguistic features, and physical features with the one or more machine learning algorithms.