Building interactive language agents that make sense of the world’s information, to communicate with and empower people.

This research theme explores how AI agents can use language as a tool to learn from and interact with the world. We envision AI companions that we can teach through conversation and demonstration, that help us in our daily lives.

Language is an abstract space for reasoning, in which modalities/domains/tasks can be translated one to another. It is also a medium for vast stores of information. How can agents harness the structure, compositionality, and associations of language to understand and predict a complex world? How can they leverage the knowledge we have built up and stored in writing over the millennia? And how can they use language to share what they’re thinking with us (and each other)? These are some of the questions that animate our research.

Our interdisciplinary work combines methods from natural language processing (NLP), including large pretrained language models, representation learning (especially self- and un-supervised), reinforcement learning, and metalearning.

Methodologically, we take two complementary approaches: Frameworks and Modelling Innovations.

• In Frameworks, we build learning environments with sequential, dynamic, multi-modal interactions. These enable us to embody agents and ground language in controlled worlds.
• In Modelling Innovations, we explore what architectures, inductive biases, memory mechanisms, and representations best empower agents to use natural language — in all its generalizing, abstract, and specific complexity. See the Projects tab for more detail.

Workshops & competitions

With industry and academic partners, our team co-organizes several research workshops and competitions. This includes the regularly recurring Wordplay workshops, which bring together researchers across a breadth of fields and showcase the interactive narrative community’s challenges, learning environments, and models to the wider NLP/RL/Gaming communities.

• The 3rd Wordplay Workshop: When Language Meets Games (opens in new tab) @ NAACL 2022
• NILLI: Novel Ideas in Learning-to-Learn through Interaction (opens in new tab) @ EMNLP 2022

See past events

Our collaborators

In addition to the organizations below, we also work closely with other Microsoft teams, including Semantic Machines.