Intelligent Interactive Systems Group at Harvard



Current and Upcoming Projects

Organic Peer Assessment

We are developing tools and techniques for organic peer assessment, an approach where assessment occurs as a side effect of students performing activities, which they find intrinsically motivating. Our preliminary results, obtained in the context of a flipped classroom, show that the quality of the summative assessment produced by the peers matched that of experts, and we encountered strong evidence that our peer assessment implementation had positive effects on achievement.

Steven Komarov and Krzysztof Z. Gajos. Organic Peer Assessment. In Proceedings of the CHI 2014 Learning Innovation at Scale workshop, 2014.
[Abstract, BibTeX, etc.]

Learnersourcing: Leveraging Crowds of Learners to Improve the Experience of Learning from Videos

Rich knowledge about the content of educational videos can be used to enable more effective and more enjoyable learning experiences. We are developing tools that leverage crowds of learners to collect rich meta data about educational videos as a byproduct of the learners' natural interactions with the videos. We are also developing tools and techniques that use these meta data to improve the learning experience for others.
Read more on the project page >>

Juho Kim, Phu Nguyen, Sarah Weir, Philip J Guo, Robert C Miller, and Krzysztof Z. Gajos. Crowdsourcing Step-by-Step Information Extraction to Enhance Existing How-to Videos. In Proceedings of CHI 2014, 2014. To appear.   Honorable Mention  
[Abstract, BibTeX, etc.]

Juho Kim, Shang-Wen (Daniel) Li, Carrie J. Cai, Krzysztof Z. Gajos, and Robert C. Miller. Leveraging Video Interaction Data and Content Analysis to Improve Video Learning. In Proceedings of the CHI 2014 Learning Innovation at Scale workshop, 2014.
[Abstract, BibTeX, etc.]

Juho Kim, Philip J. Guo, Daniel T. Seaton, Piotr Mitros, Krzysztof Z. Gajos, and Robert C. Miller. Understanding In-Video Dropouts and Interaction Peaks in Online Lecture Videos. In Proceeding of Learning at Scale 2014, 2014. To appear.
[Abstract, BibTeX, etc.]

Juho Kim, Robert C. Miller, and Krzysztof Z. Gajos. Learnersourcing subgoal labeling to support learning from how-to videos. In CHI '13 Extended Abstracts on Human Factors in Computing Systems, CHI EA '13, pages 685-690, New York, NY, USA, 2013. ACM.
[Abstract, BibTeX, etc.]

Predicting Users' First Impressions of Website Aesthetics

Users make lasting judgments about a website's appeal within a split second of seeing it for the first time. This first impression is influential enough to later affect their opinion of a site's usability and trustworthiness. In this project, we aim to automatically adapt website aesthetics to users' various preferences in order to improve this first impression. As a first step, we are working on predicting what people find appealing, and how this is influenced by their demographic backgrounds.

Katharina Reinecke and Krzysztof Z. Gajos. Quantifying Visual Preferences Around the World. In Proceedings of CHI 2014, 2014. To appear.
[Abstract, BibTeX, etc.]

Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos. Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, pages 2049-2058, New York, NY, USA, 2013. ACM.   Honorable Mention  
[Abstract, BibTeX, Data, etc.]

Curio: a platform for crowdsourcing research tasks in sciences and humanities

Curio is intended to be a platform for crowdsourcing research tasks in sciences and humanities. The platform is designed to allow researchers to create and launch a new crowdsourcing project within minutes, monitor and control aspects of the crowdsourcing process with minimal effort. With Curio, we are exploring a brand new model of citizen science that significantly lowers the barrier of entry for scientists, developing new interfaces and algorithms for supporting mixed-expertise crowdsourcing, and investigating a variety of human computation questions related to task decomposition, incentive design and quality control.

We expect to launch Curio soon. Sign up if you want to be notified when it comes on line.

Edith Law, Conner Dalton, Nick Merrill, Albert Young, and Krzysztof Z. Gajos. Curio: A Platform for Supporting Mixed-Expertise Crowdsourcing. In Proceedings of HCOMP 2013. AAAI Press, 2013. To appear.
[Abstract, BibTeX, etc.]

Lab in the Wild

Most of what we know about human-computer interaction today is based on studies conducted with Western participants, usually with American undergrads. This is despite many findings that our cultural background affects our perception and preferences. Neuroscience research has even shown that cultural exposure leads to differences in neural activity -- a finding that might affect how we interact with computers. If people around the world perceive, process, and interact with information differently, then what should their user interfaces look like in order to be most intuitive for them to use?

With Lab in the Wild we are trying to shed light on this question. Our goal is to improve the user experience and performance for computer users around the world. But Lab in the Wild doesn't just help us answer our questions. It also provides participants with personalized feedback, which lets them compare themselves and their performance to people of other countries. Try it out :)

Here are some of the most recent papers that relied on the data collected on Lab in the Wild:

Katharina Reinecke and Krzysztof Z. Gajos. Quantifying Visual Preferences Around the World. In Proceedings of CHI 2014, 2014. To appear.
[Abstract, BibTeX, etc.]

Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos. Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, pages 2049-2058, New York, NY, USA, 2013. ACM.   Honorable Mention  
[Abstract, BibTeX, Data, etc.]

Ability-Based User Interfaces

Most of today's GUIs are designed for the typical, able-bodied user. People with unusual abilities (due to a disability, a temporary injury, or who are just trying to operate a small device with cold fingers) have to adapt themselves to the user interfaces, perhaps using assistive technologies. We are working to reverse this situation: we believe that user interfaces should be adapted to the invidual abilities, devices, and preferences of the people who use them.

Several invidual projects in our group contribute to this vision. Our work on the SUPPLE system, for example, demonstrated that we can automatically generate user interfaces adapted to a person's individual motor and vision abilities. The results of our studies showed that people with motor impairments were significantly faster and strongly preferred such automatically generated ability-based interfaces to the defaults provided by the software manufacturers.

Papers, video, and other resources are available on the project web site.

PETALS Project -- A Visual Decision Support Tool For Landmine Detection

Landmines remain in conflict areas for decades after the end of hostilities. Their suspected presence renders vast tracts of land unusable for development and agriculture causing significant psychological and economical damage. Landmine removal is a slow and dangerous process. Compounding the difficulty, modern landmines use minimal amounts of metallic content making them very hard to detect and to distinguish from other metallic debris (such as bullet shells, wires, etc.) frequently present in post-combat areas. Recent research has demonstrated that the accuracy of landmine detection can be improved if deminers try to mentally represent the shape of the area where the metal detector's response gets triggered. Despite similar amounts of metallic content, mines and clutter results in areas of different shapes. Building on these findings, we have created a visual decision support tool that presents the deminer with an explicit visualization of the shapes of these response areas. The results of our study demonstrate that this tool significantly improves novice deminers' detection rates and it improves the localization accuracy.

Lahiru G. Jayatilaka, Luca F. Bertuccelli, James Staszewski, and Krzysztof Z. Gajos. Evaluating a Pattern-Based Visual Support Approach for Humanitarian Landmine Clearance. In CHI '11: Proceeding of the annual SIGCHI conference on Human factors in computing systems, New York, NY, USA, 2011. ACM.
[Abstract, BibTeX, Authorizer, etc.]

Lahiru G. Jayatilaka, Luca F. Bertuccelli, James Staszewski, and Krzysztof Z. Gajos. PETALS: a visual interface for landmine detection. In Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology, UIST '10, pages 427-428, New York, NY, USA, 2010. ACM.
[Abstract, BibTeX, Authorizer, etc.]

Recently Completed Projects

Adaptive Click and Cross: Adapting to Both Abilities and Task to Improve Performance of Users With Impaired Dexterity

Adaptive Click-and-Cross, an interaction technique for computer users with impaired dexterity. This technique combines three "adaptive" approaches that have appeared separately in previous literature: adapting the user's abilities to the interface (i.e., by modifying the way that the cursor works), adapting the user interface to the user's abilities (i.e., by modifying the user interface through enlarging items), and adapting the user interface to the user's task (i.e., by moving frequently or recently used items to a convenient location). Adaptive Click-and-Cross combines these three adaptations to minimize each approach's shortcomings, selectively enlarging items predicted to be useful to the user while employing a modified cursor to enable access to smaller items.

Louis Li and Krzysztof Z. Gajos. Adaptive Click-and-Cross: Adapting to Both Abilities and Task Improves Performance of Users With Impaired Dexterity. In Proceedings of IUI 2014, 2014. To appear.
[Abstract, BibTeX, etc.]

Louis Li. Adaptive Click-and-cross: An Interaction Technique for Users with Impaired Dexterity. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS '13, pages 79:1-79:2, New York, NY, USA, 2013. ACM.
[Abstract, BibTeX, etc.]

InProv: a Filesystem Provenance Visualization Tool

InProv is a filesystem provenance visualization tool, which displays provenance data with an interactive radial-based tree layout. The tool also utilizes a new time-based hierarchical node grouping method for filesystem provenance data we developed to match the user's mental model and make data exploration more intuitive. In an experiment comparing InProv to a visualization based on the node-link representation, participants using InProv made more accurate assessments of provenance and found InProv to require less mental effort, less physical activity, less work, and to be less stressful to use.

Michelle A Borkin, Chelsea S Yeh, Madelaine Boyd, Peter Macko, KZ Gajos, M Seltzer, and H Pfister. Evaluation of filesystem provenance visualization tools. IEEE transactions on visualization and computer graphics, 19(12):2476-2485, 2013.
[Abstract, BibTeX, etc.]

Crowdsourcing Performance Evaluations of User Interfaces

Can computer users be trusted to paricipate in user interface studies from the comfort of their home? Can user interface researchers give up control over their subjects' environment? In this project we study whether we can use Amazon Mechanical Turk to conduct user interface studies reliably. To do so, we replicated three previously known performance experiments, the "Bubble Cursor," the "Split Menus," and the "Split Interface," both in our lab and on Mechanical Turk. We compared the lab with the online population in terms of performance metrics such as speed, accuracy, and consistency. The results, which we share in our upcoming CHI paper, show that the Mechanical Turk participants perform just as well as the lab participants.

Steven Komarov, Katharina Reinecke, and Krzysztof Z. Gajos. Crowdsourcing performance evaluations of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, pages 207-216, New York, NY, USA, 2013. ACM.
[Abstract, BibTeX, Data, etc.]

SPRWeb: Preserving Subjective Responses to Website Colour Schemes through Automatic Recolouring

Colors are an important part of user experiences on the Web. Color schemes influence the aesthetics, first impressions and long-term engagement with websites. However, five percent of people perceive a subset of all colors because they have color vision deficiency (CVD), resulting in an unequal and less-rich user experience on the Web. Traditionally, people with CVD have been supported by recoloring tools that improve color differentiability, but do not consider the subjective properties of color schemes while recoloring. To address this, we developed SPRWeb, a tool that recolors websites to preserve subjective responses and improve color differentiability, thus enabling users with CVD to have similar online experiences. SPRWeb is the first tool to automatically preserve the subjective and perceptual properties of website color schemes thereby equalizing the color-based web experience for people with CVD.

David R. Flatla, Katharina Reinecke, Carl Gutwin, and Krzysztof Z. Gajos. SPRWeb: preserving subjective responses to website colour schemes through automatic recolouring. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, pages 2069-2078, New York, NY, USA, 2013. ACM.   Best Paper Award  
[Abstract, BibTeX, etc.]

Cultural Differences in Time Perception and Group Decision-Making

When discussing the effect of technology on culture, people often assume that the world is slowly homogenizing into a culture of Internet users, who share similar values and behavioral norms. Our analysis of the online scheduling behavior on Doodle argues against this hypothesis. In fact, event scheduling is not simply a matter of finding a mutually agreeable time, but a process that is shaped by social norms and values. And this can highly vary between countries. To investigate the influence of national culture on people's scheduling behavior we analyzed more than 1.5 million Doodle date/time polls from 211 countries. Our findings include that people around the world steer their availabilities towards those options that have good chances to reach consensus. But people from more group-oriented collectivist countries (e.g., India, China) seem to make a larger effort to reach mutual agreement than individualists (e.g., the US). We believe that increasing the awareness of such differences can help improve intercultural scheduling and support the acceptance of cultural differences as an interesting contribution to our lives.

Katharina Reinecke, Minh Khoa Nguyen, Abraham Bernstein, Michael Näf, and Krzysztof Z. Gajos. Doodle around the world: online scheduling behavior reflects cultural differences in time perception and group decision-making. In Proceedings of the 2013 conference on Computer supported cooperative work, CSCW '13, pages 45-54, New York, NY, USA, 2013. ACM.
[Abstract, BibTeX, Data, etc.]

Accurate Measurements of Pointing Performance from In Situ Observations

We present a method for obtaining lab-quality measurements of pointing performance from unobtrusive observations of natural in situ interactions. Specifically, we have developed a set of user-independent classifiers for discriminating between deliberate, targeted mouse pointer movements and those movements that were affected by any extraneous factors. Our results show that, on four distinct metrics, the data collected in-situ and filtered with our classifiers closely matches the results obtained from the formal experiment.

Krzysztof Gajos, Katharina Reinecke, and Charles Herrmann. Accurate measurements of pointing performance from in situ observations. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, CHI '12, pages 3157-3166, New York, NY, USA, 2012. ACM.
[Abstract, BibTeX, Authorizer, Data and Source Code, etc.]

Mobi: Human Computation Tasks with Global Constraints

An important class of tasks that are underexplored in current human computation systems are complex tasks with global constraints. One example of such a task is itinerary planning, where solutions consist of a sequence of activities that meet requirements specified by the requester. In this paper, we focus on the crowdsourcing of such plans as a case study of constraint-based human computation tasks and introduce a collaborative planning system called Mobi that illustrates a novel crowdware paradigm. Mobi presents a single interface that enables crowd participants to view the current solution context and make appropriate contributions based on current needs. We conduct experiments that explain how Mobi enables a crowd to effectively and collaboratively resolve global constraints, and discuss how the design princi- ples behind Mobi can more generally facilitate a crowd to tackle problems involving global constraints.

Haoqi Zhang, Edith Law, Rob Miller, Krzysztof Gajos, David Parkes, and Eric Horvitz. Human computation tasks with global constraints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '12, pages 217-226, New York, NY, USA, 2012. ACM.   Honorable Mention  
[Abstract, BibTeX, etc.]

PlateMate: Crowdsourcing Nutrition Analysis from Food Photographs

PlateMate allows users to take photos of their meals and receive estimates of food intake and composition. Accurate awareness of this information is considered a prerequisite to successful change of eating habits, but current methods for food logging via self-reporting, expert observation, or algorithmic analysis are time-consuming, expensive, or inaccurate. PlateMate crowdsources nutritional analysis from photographs using Amazon Mechanical Turk, automatically coordinating untrained workers to estimate a meal's calories, fat, carbohydrates, and protein. To make PlateMate possible, we developed the Management framework for crowdsourcing complex tasks, which supports PlateMate's decomposition of the nutrition analysis workflow. Two evaluations show that the PlateMate system is nearly as accurate as a trained dietitian and easier to use for most users than traditional self-reporting, while remaining robust for general use across a wide variety of meal types.

Jon Noronha, Eric Hysen, Haoqi Zhang, and Krzysztof Z. Gajos. PlateMate: Crowdsourcing Nutrition Analysis from Food Photographs. In Proceedings of the 24th annual ACM symposium on User interface software and technology, UIST '11, pages 1-12, New York, NY, USA, 2011. ACM.
[Abstract, BibTeX, Authorizer, Data, etc.]

Past Projects

Exploring The Design Space Of Adaptive User Interfaces

For decades, researchers have presented different adaptive user interfaces and discussed the pros and cons of adaptation on task performance and satisfaction. Little research, however, has been directed at isolating and understanding those aspects of adaptive interfaces which make some of them successful and others not. We have conducted several laboratory studies to systematically isolate some of the design and contextual factors that affect the impact of adaptation on users' performance and satisfaction. The results of these studies combined with the recent work of others, provide an initial characterization of the design space of adaptive graphical user interfaces.

Louis Li and Krzysztof Z. Gajos. Adaptive Click-and-Cross: Adapting to Both Abilities and Task Improves Performance of Users With Impaired Dexterity. In Proceedings of IUI 2014, 2014. To appear.
[Abstract, BibTeX, etc.]

Leah Findlater and Krzysztof Z. Gajos. Design Space and Evaluation Challenges of Adaptive Graphical User Interfaces. AI Magazine, 30(4):68-73, 2009.
[Abstract, BibTeX, etc.]

Krzysztof Z. Gajos, Katherine Everitt, Desney S. Tan, Mary Czerwinski, and Daniel S. Weld. Predictability and accuracy in adaptive user interfaces. In CHI '08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 1271-1274, New York, NY, USA, 2008. ACM.
[Abstract, BibTeX, Authorizer, etc.]

Krzysztof Z. Gajos. Automatically Generating Personalized User Interfaces. PhD thesis, University of Washington, Seattle, WA, USA, 2008.
[Abstract, BibTeX, etc.]

Krzysztof Z. Gajos, Daniel S. Weld, and Jacob O. Wobbrock. Decision-Theoretic User Interface Generation. In AAAI'08, pages 1532-1536. AAAI Press, 2008.
[Abstract, BibTeX, etc.]

Krzysztof Z. Gajos, Mary Czerwinski, Desney S. Tan, and Daniel S. Weld. Exploring the design space for adaptive graphical user interfaces. In AVI '06: Proceedings of the working conference on Advanced visual interfaces, pages 201-208, New York, NY, USA, 2006. ACM Press.
[Abstract, BibTeX, Slides, Video, Authorizer, etc.]

HemoVis: Artery Visualization for Heart Disease Diagnosis

Heart disease is the number one killer in the United States, and finding indicators of the disease at an early stage is critical for treatment and prevention. In this paper we evaluate visualization techniques that enable the diagnosis of coronary artery disease. A key physical quantity of medical interest is endothelial shear stress (ESS). Low ESS has been associated with sites of lesion formation and rapid progression of disease in the coronary arteries. Having effective visualizations of a patient's ESS data is vital for the quick and thorough non-invasive evaluation by a cardiologist. We present a task taxonomy for hemodynamics based on a formative user study with domain experts. Based on the results of this study we developed HemoVis, an interactive visualization application for heart disease diagnosis that uses a novel 2D tree diagram representation of coronary artery trees. We present the results of a formal quantitative user study with domain experts that evaluates the effect of 2D versus 3D artery representations and of color maps on identifying regions of low ESS. We show statistically significant results demonstrating that our 2D visualizations are more accurate and efficient than 3D representations, and that a perceptually appropriate color map leads to fewer diagnostic mistakes than a rainbow color map.
[Related paper]

Automatic Task Design on Amazon Mechanical Turk

A central challenge in human computation is in understanding how to design task environments that effectively attract participants and coordinate the problem solving process. We consider a common problem that requesters face on Amazon Mechanical Turk: how should a task be designed so as to induce good output from workers? In posting a task, a requester decides how to break down the task into unit tasks, how much to pay for each unit task, and how many workers to assign to a unit task. These design decisions affect the rate at which workers complete unit tasks, as well as the quality of the work that results. Using image labeling as an example task, we consider the problem of designing the task to maximize the number of quality tags received within given time and budget constraints. We consider two different measures of work quality, and construct models for predicting the rate and quality of work based on observations of output to various designs. Preliminary results show that simple models can accurately predict the quality of output per unit task, but are less accurate in predicting the rate at which unit tasks complete. At a fixed rate of pay, our models generate different designs depending on the quality metric, and optimized designs obtain significantly more quality tags than baseline comparisons.
[Related paper]

Crossing-Based User Interfaces

Jacob O. Wobbrock (UW) and Krzysztof Z. Gajos

Prior work has highlighted the challenges faced by people with motor impairments when trying to acquire on-screen targets using a mouse or trackball. Two reasons for this are the difficulty of positioning the mouse cursor within a confined area, and the challenge of accurately executing a click. We hypothesize that both of these difficulties with area pointing may be alleviated in a different target acquisition paradigm called "goal crossing." In goal crossing, users do not acquire a confined area, but instead pass over a target line. Although goal crossing has been studied for able-bodied users, its suitability for people with motor impairments is unknown. In our study, participants with motor impairments were faster with and preferred goal-crossing to area pointing. This work provides the empirical foundation from which to pursue the design of crossing-based interfaces as accessible alternatives to pointing-based interfaces.
[Related papers][Project web site]

ARNAULD: Preference Elicitation For Interface Optimization

Krzysztof Z. Gajos and Daniel S. Weld (UW)

ARNAULD Project Recent years have revealed a trend towards increasing use of optimization as a method for automatically designing aspects of an interface's interaction with the user. In most cases, this optimization may be thought of as decision-theoretic -- the objective is to minimize the expected cost of a user's interactions or (equivalently) to maximize the user's expected utility. While decision-theoretic optimization provides a powerful, flexible, and principled approach for these systems, the quality of the resulting solution is completely dependent on the accuracy of the underlying utility or cost function. Unfortunately, determining the correct utility function is a complex, time-consuming, and error-prone task. While domain-specific learning techniques have been used occasionally, most practitioners parameterize the utility function and then engage in a laborious and unreliable process of hand-tuning.
[Related papers][Project web site]

SUPPLE: Automatically Generating User Interfaces

Krzysztof Z. Gajos, Raphael Hoffmann (UW), David Christianson (UW), Anthony Wu (UW), Kiera Henning (UW), Jing Jing Long (UW), and Daniel S. Weld (UW)

SUPPLE Project SUPPLE is an application- and device-independent system that automatically generates user interfaces for a wide variety of display devices. SUPPLE uses decision-theoretic optimization to render an interface from an abstract functional specification and an interchangeable device model. SUPPLE can use information from the user model to automatically adapt user interfaces to different tasks and work styles while also prividing extensive customization mechanisms that allow for modifications to the appearance, organization and navigational structure of the user interface.

Papers, video, code, and other resources are available on the project web site.

Exploring Opportunities for Intelligent Interfaces Aiding Healthcare in Low-Income Countries

Brian DeRenzi (UW), Krzysztof Z. Gajos, Tapan S. Parikh (UC Berkeley), Neal Lesh (D-Tree International), Marc Mitchell (D-Tree International), and Baetano Borriello (UW)

Child mortality is one of the most pressing health concerns almost 10 million children die worldwide each year before reaching their fifth birthday, mostly in low-income countries. To aid overburdened and undertrained health workers the World Health Organization (WHO) and United Nations Children's Fund (UNICEF) have developed clinical guidelines, such as the Integrated Management of Childhood Illness (IMCI) to help with the classification and treatment of common childhood illness. To help with deployment, we have developed an electronic version (e-IMCI) that runs on a PDA. From July to September 2007, we ran a pilot of e-IMCI in southern Tanzania. The system guides health workers step-by-step through the treatment algorithms and automatically calculates drug doses. Our results suggest that electronic implementations of protocols such as IMCI can reduce training time and improve adherence to the protocol. They also highlight several important challenges including varying levels of education, language and expertise, which could be most adequately addressed by implementing novel intelligent user interfaces and systems.
[Related papers]

Opportunity Knocks: a System to Provide Cognitive Assistance with Transportation Services

Donald J. Patterson (UW), Lin Liao (UW), Krzysztof Gajos, Michael Collier (UW), Nik Livic (UW), Katherine Olson (UW), Shiaokai Wang (UW), Dieter Fox (UW), and Henry Kautz (UW)

Opportunity Knocks Opportunity Knocks (OK) is an automated transportation routing system, whose goal is to improve the efficiency, safety and independence of individuals with mild cognitive disabilities. OK is implemented on a combination of a Bluetooth sensor beacon that broadcasts GPS data, a GPRS-enabled cell-phone, and remote activity inference software. The system uses a novel inference engine that does not require users to explicitly provide information about the start or ending points of their journeys; instead this information is learned from users' past behavior.
[Related papers]

Alfred: End User Empowerment in Human Centered Pervasive Computing

Krzysztof Z. Gajos, Harold Fox (MIT), and Howard Shrobe (MIT)

Alfred is an electronic butler for Intelligent Environments. Alfred allows an end user to "program" the system by telling it the name of a new goal, demonstrating one or more plans for achieving that goal, and finally telling the system the conditions under which it would prefer one plan to another. Similarly, the user can name events that arise in the environment and tell the system what goals should be posted when those events arise. Each of these steps can be done by simple verbal commands or other natural forms of interaction. End users, in effect, record "macros" which, are executed adaptively and reactively.
[Related papers]

Look-to-Talk: A Gaze-Aware Interface in a Collaborative Environment

Alice Oh (MIT), Harold Fox (MIT), Max Van Kleek (MIT), Aaron Adler (MIT), Krzysztof Gajos, Louis-Philippe Morency (MIT), and Trevor Darrell (MIT)

Loot To Talk "Look-to-talk" is a gaze-aware interface for directing a spoken utterance to a software agent in a multiuser collaborative environment. Through a prototype and a Wizard-of-Oz (WOz) experiment, we showed that "look-totalk" is indeed a natural alternative to speech and other paradigms.
[Related papers]

FIRE: The Friendly Information Retrieval Engine

Krzysztof Z. Gajos, Ajay Kulkarni (MIT), and Howard Shrobe (MIT)

FIRE FIRE is a multimodal interface for information retrieval deployed in the Intelligent Room at the MIT AI Lab. FIRE extracts all the category terms related to the search query and uses entropy to generate questions that would quickly allow the user to disambiguate her query and arrive at a small set of relevant documents. FIRE presents information over several large displays in the Intelligent Room and supports both speech and gesture input for more natural interaction.
[Related papers]

Rascal: A High-Level Resource Manager For Smart Environments

Krzysztof Gajos, Luke Weisman (MIT), Howard Shrobe (MIT)

Rascal Rascal is a high-level resource management system for the Intelligent Room Project, that addresses the problem of the numerous applications competing for limited physical resources. Rascal performs the service mapping and and uses constrained search for arbitration among different requesters.
[Related papers]

This page was last modified on Thursday, 05-Jun-2014 16:24:54 EDT.