Intelligent Interactive Systems Group at Harvard

Overview

This classifier discriminates between deliberate, targeted pointing movements, and those movements that were affected by distraction, visual search, demanding cognitive task, or any other extraneous factor. See the following paper for more details:

Krzysztof Gajos, Katharina Reinecke, and Charles Herrmann. Accurate measurements of pointing performance from in situ observations. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, CHI '12, pages 3157-3166, New York, NY, USA, 2012. ACM.
[Abstract, BibTeX, Authorizer, Data and Source Code, etc.]

Code

• Source Code is structured as two separate Eclipse projects: IIS Java Utils contains many of the utilities shared among several of our projects, and ABUI-analysis contains the code for parsing, analyzing, and classifying movement traces. The primary class of interest is edu.harvard.seas.iis.abilities.classify.MovementClassifier, which contains the front end methods for interfacing with the trained classifiers. If you want to build the classifier using your own data (or ours), the entire toolchain (except for feature selection) is captured in edu.harvard.seas.iis.abilities.classify.BuildClassifier. You will want to edit edu.harvard.seas.iis.abilities.classify.Settings to tell specify the default location of your data if you end up training your own classifier.
• The executable jar file will parse a movement log file and output basic statistics about the numbers of deliberate, targeted movements found. The best way to view the output is to paste it into a spreadsheet (the output is tab-separated so it pastes cleanly into any spreadsheet for easy analysis).

Data

The raw data (115MB compressed) include movement traces from both in situ observations and the formal experiments.

The clean parsed data (20MB compressed) are represented as tables with one row per movement; the values of all the features are computed. The data collected in a natural setting have the value "implicit" in the Class column; the data collected in the experimental setting have "explicit" in the Class column.

Log Format

Mouse Events

movement:

m, x-coordinate, y-coordinate, time stamp

example: m,547,410,1303044426975

button presses:

(older logs will not have any information about the target location and more limited information about the type of the target)

[s-d|s-u|s-c],target type,mouse button,target width, target height, x-coordinate of upper left corner of the target, y-coordinate, x-coordinate of the button press event, y-coordinate, time stamp]

s-d -- mouse button down event

s-u -- mouse button up event

s-c -- mouse click event

e-d, e-u, e-c -- as above, but for mouse button events from the explicit test

mouse buttons: 1=left button, 2=right button

example: s-c,link-A->B,2,87,15,329,237,377,243,1303076189570

Exceptions:

for targets from the explicit test, the target coordinates point to its center and not upper left corner

keyboard events:

[l|r|o|backspace],[u|d],time stamp

l -- letter key that on a normal keyboard would be operated with the left hand

r -- letter key that on a normal keyboard would be operated with the right hand

o -- other character (not quite sure what goes into this category)

backspace -- well, backspace; recorded explicitly to capture corrections in typing

u -- key up event

d -- key down event

examples:

l,u,1303076206233

r,d,1303076457918

o,u,1303076220541

backspace,d,1303076207063

break:

whenever the mouse leaves the browser window or the user switches tabs, we record a break event with a time stamp, example:

Contact

The primary contact person for this project is Krzysztof Gajos.

This page was last modified on April 26, 2022.