Visualization and Interaction for business and entertainment

Visualization and Interaction for Business and Entertainment (VIBE)

The Visualization and Interaction for Business and Entertainment (VIBE) group conducts research in the areas of: artificial emotion intelligence (AEI), information visualization in machine learning and big data; and human-computer interaction (HCI) in software engineering.

Affective computing

Emotions are fundamental to human interactions and influence memory, decision-making, and well-being. As AI systems and intelligent agents become more advanced, there is increasing interest in applications that can sense and respond to emotional states.

We are working on several projects to bring AEI to Microsoft products:

  1. Building a large database of labeled naturalistic data. Having a large amount of labeled data is essential for training affective computing systems. While Microsoft has an unparalleled amount of data that could be mined for this purpose, what’s missing are efficient labeling methodologies that add human judgments.
  2. Designing new methods for sensing emotion signals. We are advancing the state of the art in remote physiological measurement using webcams and other cameras to capture physiological responses, facial expression, voice tone, and language.
  3. Advancing multimodal analysis. We are combining affect measurement (for example, computer vision analysis of facial expression and scenes) with language models to generate computational models of conversation that better reflect emotions.
  4. Prototyping emotionally adaptive systems. How should systems—from search interactions to productivity tools—respond to emotions? We prototype systems that respond in real time and perform user studies to inform the design of effective affective computing applications.

Visualization in machine learning and big data

Our group looks at data visualization as a key factor in helping understand and validate machine learning models. Using trend analysis and outlier detection, we can build displays to help people attend to data at the appropriate times, and see when data has changed.

Electronic systems produce tremendous amounts of remotely collected data, or telemetry, reflecting user behavior, system actions, and much more. From all of this data, product teams need to extract the data that holds value and insight. Our team is looking at user experiences for very large datasets, to better understand how users interact with approximate query processing data samples to get interactive responses their explorations.

HCI in software engineering

Our team members study software developers to improve their productivity and well-being. We also interview researchers and product developers to document, summarize, and analyze what makes for successful technology transfers.