{"id":633120,"date":"2020-02-02T08:18:44","date_gmt":"2020-02-02T16:18:44","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-event&p=633120"},"modified":"2020-10-27T03:31:58","modified_gmt":"2020-10-27T10:31:58","slug":"mixed-reality-and-robotics-tutorial-iros-2020","status":"publish","type":"msr-event","link":"https:\/\/www.microsoft.com\/en-us\/research\/event\/mixed-reality-and-robotics-tutorial-iros-2020\/","title":{"rendered":"Mixed Reality and Robotics – Tutorial @ IROS 2020"},"content":{"rendered":"

Conference:<\/strong> IROS 2020<\/a><\/p>\n

On-Demand Videos:\u00a0<\/strong>Direct Link<\/a><\/p>\n

Contact:<\/strong>
\njeffrey.delmerico@microsoft.com<\/a> and helen.oleynikova@microsoft.com<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

The Mixed Reality and Robotics tutorial at IROS 2020 will introduce the audience to both the high-level concepts of Mixed Reality and practically demonstrate how these concepts can be used to interact with a robot through an MR device.<\/p>\n","protected":false},"featured_media":661089,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"msr_startdate":"2020-10-25","msr_enddate":"2020-10-25","msr_location":"On-demand Virtual","msr_expirationdate":"","msr_event_recording_link":"","msr_event_link":"https:\/\/www.microsoftevents.com\/profile\/9758116","msr_event_link_redirect":false,"msr_event_time":"","msr_hide_region":true,"msr_private_event":false,"footnotes":""},"research-area":[13562],"msr-region":[239178],"msr-event-type":[197941],"msr-video-type":[],"msr-locale":[268875],"msr-program-audience":[],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-633120","msr-event","type-msr-event","status-publish","has-post-thumbnail","hentry","msr-research-area-computer-vision","msr-region-europe","msr-event-type-conferences","msr-locale-en_us"],"msr_about":"Conference:<\/strong> IROS 2020<\/a>\r\n\r\nOn-Demand Videos:\u00a0<\/strong>Direct Link<\/a>\r\n\r\nContact:<\/strong>\r\njeffrey.delmerico@microsoft.com<\/a> and helen.oleynikova@microsoft.com<\/a>","tab-content":[{"id":0,"name":"About","content":"Welcome to the Mixed Reality and Robotics Tutorial at IROS 2020<\/a>.\u00a0 This year's conference is using an on-demand, virtual format, which means that all of the content for this tutorial is available as streaming videos, with code samples to accompany the demos.\u00a0 However, the conference organizers have made registration FREE, so by signing up here<\/a>, you can gain access to all of the talks and papers, as well as the workshops and tutorials (including this one).\u00a0 \u00a0Please see the Agenda tab for more detailed information about the tutorial contents.<\/strong>\r\n\r\nOnce registered, the on-demand videos can be reached directly on the IROS On-Demand Platform<\/a>.\r\n\r\nTo follow along with the colocalization tutorial, you will also need to register for the Azure Spatial Anchors Linux SDK which can be done at this Signup Link<\/a>.\r\n

Registration<\/h3>\r\nIn order to view the tutorial videos, you will need to be registered for the IROS conference<\/strong>.\u00a0 However, to help us better understand the research interests of the audience, and to more easily contact IROS attendees who are interested in Mixed Reality, we would kindly ask that you click the link in the top left to register for this event.\u00a0 Registration for our tutorial is not binding, and is separate from the IROS conference registration.\u00a0<\/strong> In order to access the content for this tutorial through the IROS On-Demand site, you will still need to register for the IROS conference.\r\n<\/strong>\r\n

Abstract<\/h3>\r\nMixed, Augmented, and Virtual Reality offer exciting new frontiers in communication, entertainment, and productivity. A primary feature of Mixed Reality (MR) is the ability to register the digital world with the physical one, opening the door to a wide variety of robotics applications. This capability enables more natural human-robot interaction: instead of a user interfacing with a robot through a computer screen, we envision a future in which the user interacts with a robot in the same environment through MR, to see what it sees, to see its intentions, and seamlessly control it in its own representation of the world.\r\n\r\nThe purpose of this tutorial is to introduce the audience to both the high-level concepts of Mixed Reality and practically demonstrate how these concepts can be used to interact with a robot through an MR device. We will discuss how various hardware devices (mobile phones, AR\/MR\/VR headsets, and robots\u2019 on-board sensors) can integrate with cloud services to create a digital representation of the physical world, and how such a representation can be used for co-localization. Participants will have a chance to create an iOS, Android, or Microsoft HoloLens 2 app to control and interact with a virtual robot, with instructions on how to adapt the sample code to a real robot, so attendees can start using Mixed Reality in their own robotics projects.\r\n

Workshop Organizers<\/strong><\/h4>\r\nMarc Pollefeys<\/a>\r\nHelen Oleynikova<\/a>\r\nJeff Delmerico<\/a>"},{"id":1,"name":"Agenda","content":"

Tutorial Contents<\/h2>\r\nWe cover \"big picture\" ideas of Mixed Reality and how we envision that it will transform how we interact with robots, along with technical details on a few different ways to do colocalization to allow any Mixed or Augmented Reality device to share a coordinate frame with a robot.\u00a0 \u00a0Finally, there is a practical portion where we introduce a few of the tools that are necessary to create full Mixed Reality experiences with robotics. This takes the form of several demos that attendees will be able to build and run on their own, and adapt to use with their own robots.\r\n\r\nThe tutorial features five videos on the IROS 2020 streaming site<\/a>:\r\n
    \r\n \t
  1. Introduction to MR and Robotics\u00a0<\/strong>[Direct Link<\/a>]<\/li>\r\n \t
  2. Interaction\u00a0<\/strong>[Direct Link<\/a>]\r\n