Mixed Reality for Shared Autonomy

Microsoft Swiss Joint Research Center – Day 2 | Computer vision and Mixed Reality

The Applied Machine Learning Days channel features talks and performances from the Applied Machine Learning Days. AMLD is one of the largest machine learning & AI events in Europe, focused specifically on the applications of machine learning and AI, making it particularly interesting to industry and academia.

Our objective is to exploit recent developments in Mixed Reality (MR) to enhance human capabilities with robotic assistance. Robots offer mobility and power but are not capable of performing complex tasks in challenging environments such as construction, contact-based inspection, cleaning, and maintenance. On the other hand, humans have excellent higher-order reasoning, and skilled workers have the experience and training to adapt to new circumstances quickly and effectively. However, they lack in mobility and power. We envision to reduce this limitation by empowering human operators with the assistance and the capabilities provided by a robot system. This requires a human-robot interface that fully leverages the capabilities of both the human operator and the robot system. This talk will present the problem of shared autonomy for physical interaction tasks in shared physical workspaces, and the possible solutions.