First-person Perception and Interaction
MSR Distinguished Lecture Series: First-person Perception and Interaction
Computer vision has seen major success in learning to recognize objects from massive “disembodied” Web photo collections labeled by human annotators. Yet cognitive science tells us that perception develops in the context of acting the world—and without intensive supervision. Meanwhile, many realistic vision tasks require not only categorizing a well-composed human-taken photo, but also actively deciding where to look in the first place. In the context of these challenges, we are exploring how machine perception benefits from anticipating the sights and sounds an agent will experience as a function of its own actions. Based on this premise, we introduce methods for learning to look around intelligently in novel environments, learning from video how to interact with objects, and perceiving audio-visual streams for both semantic and spatial context. Together, these are steps towards first-person perception, where interaction with the world is itself a supervisory signal.
[Slides]
- Date:
- Speakers:
- Eric Horvitz, Kristen Grauman
- Affiliation:
- Microsoft Research, University of Texas Austin
-
-
Eric Horvitz
Chief Scientific Officer
-
-
Series: MSR AI Distinguished Lectures and Fireside Chats
-
-
Frontiers in Machine Learning: Fireside Chat
Speakers:- Christopher Bishop,
- Peter Lee,
- Sandy Blyth
-
Learning over sets, subgraphs, and streams: How to accurately incorporate graph context
Speakers:- Debadeepta Dey,
- Paul Bennett,
- Sean Andrist
-
-
-
-
Fireside Chat with Anca Dragan
Speakers:- Anca Dragan and Eric Horvitz
-
Conversations Based on Search Engine Result Pages
Speakers:- Maarten de Rijke
-
-
The Ethical Algorithm
Speakers:- Michael Kearns
-
Fireside Chat with Stefanie Jegelka
Speakers:- Alekh Agarwal
-
Fireside Chat with Peter Stone
Speakers: -
Efficient Robot Skill Learning: Grounded Simulation Learning and Imitation Learning from Observation
Speakers:- Debadeepta Dey
-
-
Building Neural Network Models That Can Reason
Speakers:- Christopher Manning
-
Fireside Chat with David Blei
Speakers: -
The Blessings of Multiple Causes
Speakers:- David Blei
-
As We May Program
Speakers:- Peter Norvig
-
Fireside Chat with Peter Norvig
Speakers:- Eric Horvitz,
- Peter Norvig
-
An Optimization-Based Theory of Mind for Human-Robot Interaction
Speakers:- Anca Dragan
-
Fireside Chat with Manuel Blum
Speakers: -
Towards a Conscious AI: A Computer Architecture inspired by Neuroscience
Speakers:- Adith Swaminathan
-
Fireside Chat with Dario Amodei
Speakers: -
-
Super-Human AI for Strategic Reasoning
Speakers:- Adith Swaminathan