IDENTIFYING USER ACTIVITIES USING EYE TRACKING DATA, MOUSE EVENTS, AND KEYSTROKES

摘要

A computing device classifies user activities for a person interacting with a computer user interface using one or more user interface devices. The computing device receives eye tracking data for the person, which includes a sequence of fixations ordered temporally. Each fixation corresponds to a plurality of consecutive measured gaze points. Each fixation has a duration and location based on the corresponding gaze points. For each fixation, the computing device determines a plurality of features for the fixation, including characteristics of the fixation, context features based on preceding or subsequent fixations, and user interaction features based on information from the user interface devices during the fixation. The computing device assigns a user activity label to the fixation according to the features. The label is selected from a predefined set. The computing device then analyzes the fixations and their assigned user activity labels to make recommendations.

主权项

1. A method of classifying user activities, comprising:
at a computer system having one or more processors and memory:
receiving eye tracking data for a person interacting with a computer using one or more user interface devices, the eye tracking data comprising a sequence of fixations ordered temporally, wherein each respective fixation corresponds to a plurality of respective consecutive measured gaze points, and wherein each respective fixation has a respective duration and location based on the corresponding respective measured gaze points;receiving user interaction data from the one or more user interface devices; andfor each fixation in the sequence:
determining a plurality of features for the fixation, including one or more characteristics of the fixation and one or more user interaction features based on the received user interaction data; andassociating a user activity label with the fixation in accordance with the plurality of features, wherein the user activity label is selected from a predefined set of user activity labels.