Method and system of determing user engagement and sentiment with learned models and user-facing camera images

摘要

In one exemplary embodiment, a method includes the step of obtaining a digital image of a user with a user-facing camera of a computing device. It is determined that the digital image includes a frontal image of the user. A user-sentiment score is calculated based on at least one attribute of the frontal image. A user engagement value is determined with respect to a portion of a display of the computing device. At least one of the frontal image of the user, the user-sentiment score or the gaze position of the user is communicated to an external server process or an application operating in the computing device.

主权项

1. A method comprising:
obtaining a digital image of a user with a user-facing camera of a computing device; determining that the digital image comprises a frontal image of the user; calculating a user-sentiment score based on at least one attribute of the frontal image; determining a user engagement value with respect to a portion of a display of the computing device; communicating at least one of the frontal image of the user, the user-sentiment score or the gaze position of the user to an external server process or an application operating in the computing device; wherein the step of determining that the digital image comprises a frontal image of the user further comprises:
receiving a frontal-image learned model; andcomparing the digital image with the frontal-image learned model; wherein the step of determining that the digital image comprises a frontal image of the user further comprises:
determining a sufficient similarity between the digital image and the frontal-image learned model; wherein the step of calculating a user-sentiment score based at least one attribute of the frontal image further comprises:
receiving a set of learned-smile models, wherein each learned-smile model is associated with a specified user-sentiment score; andextracting a user-smile attribute from the frontal-image; wherein the step of calculating a user-sentiment score based on at least one attribute of the frontal image further comprises:
comparing the user-smile attribute with the set of learned-smile models;determining a best-fit learned-smile model of the set of learned-smile models for the user-smile attribute; andassigning the user sentiment score the specified user-sentiment score of the best-fit learned-smile model; and wherein the step of determining a user engagement value with respect to a portion of a display of the computing device further comprises:
mapping a gaze position of the user with respect to a location of the display of the computing device.