Human-Robot Collaboration

In order for robots to collaborate with humans, they must infer helpful actions in the physical world by observing the human’s language, gesture, and actions. A particular challenge for robots that operate under uncertainty is identifying and recovering from failures in perception, actuation, and language interpretation. I will describe our approaches to automatic failure recovery using language and probabilistic methods. First we describe how a robot can use a probabilistic language grounding framework to employ information-theoretic dialog strategies, asking targeted questions to reduce uncertainty about different parts of a natural language command. Second, I will show how to invert a model for interpreting language to generate targeted natural language requests for help from a human partner, enabling a robot team to actively solicit help from a person when they encounter problems. And third, I will describe steps toward incremental interpretation of language and gesture as an enabling technology for making robots that use coordination actions to establish common ground with their human partner. This approach points the way toward more general models of human-robot collaboration building world models from both linguistic and non-linguistic input, following complex grounded natural language commands, and engaging in fluid, flexible collaboration with their human partners.

Speaker Details

Stefanie Tellex is an Assistant Professor of Computer Science and Assistant Professor of Engineering at Brown University. Her group, the Humans To Robots Lab, creates robots that seamlessly collaborate with people to meet their needs using language, gesture, and probabilistic inference, aiming to empower every person with a collaborative robot. She completed her Ph.D. at the MIT Media Lab in 2010, where she developed models for the meanings of spatial prepositions and motion verbs. Her postdoctoral work at MIT CSAIL focused on creating robots that understand natural language. She has published at SIGIR, HRI, RSS, AAAI, IROS, ICAPs and ICMI, winning Best Student Paper at SIGIR and ICMI, and Best Paper at RSS. She was named one of IEEE Spectrum’s AI’s 10 to Watch and won the Richard B. Salomon Faculty Research Award at Brown University. Her research interests include probabilistic graphical models, human-robot interaction, and grounded language understanding.

Website: http://h2r.cs.brown.edu/

Series:
Microsoft Research Talks
Date:
Speakers:
Stefanie Tellex
Affiliation:
Brown University

Series: Microsoft Research Talks