Method for on-the-fly learning of facial artifacts for facial emotion recognition

摘要

A method for determining a facial emotion of a user in the presence of a facial artifact includes detecting Action Units (AUs) for a first set of frames with the facial artifact; analyzing the AUs with the facial artifact after the detection; registering the analyzed AUs for a neutral facial expression with the facial artifact in the first set of frames; predicting the AUs in a second set of frames; and determining the facial emotion by comparing the registered neutral facial expression with the predicted AUs in the second set of frames.

主权项

1. A computer-implemented method for determining a facial emotion, in a video comprising a face with a facial artifact, the method comprising:
detecting Action Units (AUs) in a first set of frames of the video, wherein the first set of frames of the video comprises the face with the facial artifact, and wherein each AU comprises a fundamental action of an individual muscle or group of muscles; analyzing the AUs of the first set of frames of the video, wherein analyzing the AUs comprises:
determining frequently occurring AUs that appear in a larger subset of a sequence of frames of the video than other AUs, andregistering the frequently occurring AUs as a neutral facial expression; registering, as the neutral facial expression, the analyzed AUs in the first set of frames of the video; predicting AUs in a second set of frames of the video; and determining the facial emotion, by comparing: the AUs in the registered neutral facial expression in the first set of frames of the video, with the predicted AUs in the second set of frames of the video.