Improving text prediction accuracy using neurophysiology

We frequently encounter text prediction algorithms in many of the digital technologies and interfaces that we use day-to-day. However, the degree to which we choose to interact with these algorithms is often limited due to repeated errors or misinterpretations of our intended message. While these systems are initially trained on a set of linguistics data, they are typically not closed-loop systems; that is, they are not capable of registering errors and improving in real time. This project seeks to explore if neurophysiological signals could be used in a closed-loop, assistive, brain-computer interface (BCI) to improve predictive text accuracy. We conducted a study where we recorded electroencephalogram (EEG) and eye tracking (ET) data from participants while they completed a self-paced typing task in a simulated predictive text environment. Participants completed the task with different degrees of reliance on the predictive text system (completely dependent, completely independent, or their choice) and encountered both correct and incorrect text generations. Data suggest that erroneous text generations may evoke neurophysiological responses that can be measured with both EEG and pupillometry. Moreover, these responses appear to change according to users’ reliance on the predictive text system. Results show promise for use in a closed-loop, assistive BCI.

Series:
Microsoft Research Talks
Date:
Speakers:
Sophia Mehdizadeh

Series: Microsoft Research Talks