{"id":428574,"date":"2017-10-19T16:12:45","date_gmt":"2017-10-19T23:12:45","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-event&#038;p=428574"},"modified":"2017-12-01T10:42:24","modified_gmt":"2017-12-01T18:42:24","slug":"microsoft-research-project-malmo-workshop","status":"publish","type":"msr-event","link":"https:\/\/www.microsoft.com\/en-us\/research\/event\/microsoft-research-project-malmo-workshop\/","title":{"rendered":"Designing Tasks for the Future of AI &#8211; Project Malmo Workshop"},"content":{"rendered":"<p><strong>Registration is now closed.<\/strong><\/p>\n<p><strong>Venue:<\/strong> <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/longbeach.regency.hyatt.com\/en\/hotel\/home.html\" target=\"_blank\" rel=\"noopener\">Hyatt Regency Long Beach<\/a><br \/>\nBeacon A Main Conference Space<br \/>\nReception in Seaview Rotunda<br \/>\n200 South Pine Avenue<br \/>\nLong Beach, CA 90802<\/p>\n<p><strong>Learn more:<\/strong><br \/>\n<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-malmo\/\" target=\"_blank\" rel=\"noopener\">Project Malmo<\/a><br \/>\n<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Microsoft\/malmo\" target=\"_blank\" rel=\"noopener\">Project Malmo on GitHub<\/a><\/p>\n<p><strong>Contact:<\/strong> <a href=\"mailto:malmoadm@microsoft.com\">malmoadm@microsoft.com<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Microsoft is sponsoring a one-day workshop to train people how to run experiments in Project Malmo. This free workshop, which will be held December 3, 2017, at the Hyatt Regency in Long Beach, CA, will include information on technical implementation and task design in this unusual and intuitive environment, as well as offer group exercises, tutorials, and networking opportunities. <\/p>\n","protected":false},"featured_media":346142,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"msr_startdate":"2017-12-03","msr_enddate":"2017-12-03","msr_location":"Long Beach, California, USA","msr_expirationdate":"","msr_event_recording_link":"","msr_event_link":"","msr_event_link_redirect":false,"msr_event_time":"9:30 AM\u20136:00 PM (PST)","msr_hide_region":true,"msr_private_event":false,"footnotes":""},"research-area":[13556],"msr-region":[197900],"msr-event-type":[197944],"msr-video-type":[],"msr-locale":[268875],"msr-program-audience":[],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-428574","msr-event","type-msr-event","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-region-north-america","msr-event-type-hosted-by-microsoft","msr-locale-en_us"],"msr_about":"<strong>Registration is now closed.<\/strong>\r\n\r\n<strong>Venue:<\/strong> <a href=\"https:\/\/longbeach.regency.hyatt.com\/en\/hotel\/home.html\" target=\"_blank\" rel=\"noopener\">Hyatt Regency Long Beach<\/a>\r\nBeacon A Main Conference Space\r\nReception in Seaview Rotunda\r\n200 South Pine Avenue\r\nLong Beach, CA 90802\r\n\r\n<strong>Learn more:<\/strong>\r\n<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-malmo\/\" target=\"_blank\" rel=\"noopener\">Project Malmo<\/a>\r\n<a href=\"https:\/\/github.com\/Microsoft\/malmo\" target=\"_blank\" rel=\"noopener\">Project Malmo on GitHub<\/a>\r\n\r\n<strong>Contact:<\/strong> <a href=\"mailto:malmoadm@microsoft.com\">malmoadm@microsoft.com<\/a>","tab-content":[{"id":0,"name":"About","content":"Project Malmo, built on the video game Minecraft, is a platform for artificial intelligence experimentation that is particularly suited for researching issues in human\/AI collaboration. To introduce AI researchers to this unique environment, Microsoft is sponsoring a one-day workshop to train people how to run experiments in Project Malmo. This free workshop, which will be held December 3, 2017, at the Hyatt Regency in Long Beach, CA, will include information on technical implementation and task design in this unusual and intuitive environment, as well as offer group exercises, tutorials, and networking opportunities. And you won\u2019t want to miss the cocktail reception at the end of the day! Hurry\u2014space is limited, so we recommend registering today.\r\n<h3>Learn more about Project Malmo<\/h3>\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-malmo\/\" target=\"_blank\" rel=\"noopener\">Project Malmo<\/a><\/li>\r\n \t<li><a href=\"https:\/\/github.com\/Microsoft\/malmo\" target=\"_blank\" rel=\"noopener\">Project Malmo on GitHub<\/a><\/li>\r\n<\/ul>"},{"id":1,"name":"Agenda","content":"<h2>Sunday, December 3<\/h2>\r\n<table class=\"msr-table-schedule\" style=\"border-collapse: collapse;border-spacing: inherit\">\r\n<thead class=\"thead\">\r\n<tr class=\"tr\">\r\n<th class=\"th\" style=\"padding: inherit;border: inherit\">Time (PST)<\/th>\r\n<th class=\"th\" style=\"padding: inherit;border: inherit\">Session<\/th>\r\n<\/tr>\r\n<\/thead>\r\n<tbody class=\"tbody\">\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">10:00 AM\u201310:05 AM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Opening | Sean Kuno<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">10:05 AM\u201310:50 AM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Keynote: Project Malmo \u2013 Overview and task development in the context of collaborative AI | Katja Hofmann, Microsoft Research<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">10:50 AM\u201311:10 AM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Break<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">11:10 AM\u201311:55 AM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Keynote: Challenges and Opportunities of General Video Game AI | Diego Perez-Liebana, University of Essex<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">11:55 AM\u201312:50 PM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Lunch &amp; networking<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">12:50 PM\u20131:35 PM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Keynote: Psychometric Testing in Project Malmo \u2013 Using an AI Experimentation Platform for Nonverbal Reasoning Assessment in Human Agents | Heinrich Peters, University of Hamburg<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">1:35 PM\u20132:00 PM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Deep-dive: task development in Malmo | Katja Hofmann<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">2:00 PM\u20134:30 PM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Group breakout session<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">4:30 PM\u20135:00 PM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Follow-up discussion | Katja Hofmann &amp; Sean Kuno<\/div><\/td>\r\n<\/tr>\r\n<tr class=\"tr\">\r\n<td class=\"td-1-4\" style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">5:00 PM\u20136:00 PM<\/div><\/td>\r\n<td style=\"padding: inherit;border: inherit\">\r\n<div class=\"msr-table-schedule-cell\">Casual reception<\/div><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n&nbsp;"},{"id":2,"name":"Speakers & Abstracts","content":"<img class=\"size-full wp-image-444525 alignleft\" style=\"margin-bottom: 10px\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/10\/microsoft-katja-hofmann.jpg\" alt=\"Portrait of Microsoft Researcher Katja Hofmann\" width=\"125\" height=\"125\" \/><span style=\"font-size: x-large\">Project Malmo \u2013 Overview and task development in the context of collaborative AI<\/span>\r\n<div style=\"height: 1px\"><\/div>\r\n<span style=\"font-size: large\"><em>Katja Hofmann, Microsoft Research<\/em><\/span>\r\n\r\n[accordion]\r\n[panel header=\"Abstract\"]\r\nProject Malmo is an open source AI experimentation platform to support fundamental research in artificial intelligence. With the platform, Microsoft aims to provide an experimentation environment in which promising approaches can be systematically and easily compared, and that fosters collaboration between researchers. Project Malmo achieves flexibility by building on top of Minecraft, which is particularly appealing due to its open-ended nature, collaboration with other players, and creativity in game-play.\r\n\r\nIn this talk I will introduce the Malmo Platform, with a focus on its capabilities for easily creating a wide range of single- and multi-agent AI tasks. In particular, I will exemplify task design and challenges in the context of the Malmo Collaborative AI Challenge \u2013 a challenge our team recently conducted to foster research in collaborative AI. Aim of the talk is to give the audience a broad overview of the possibilities of Project Malmo and how they can drive an ambitious AI research agenda.\r\n[\/panel]\r\n[panel header=\"Bio\"]\r\nI am a researcher at the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/group\/machine-intelligence-and-perception\/\">Machine Intelligence and Perception<\/a> group at <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/lab\/microsoft-research-cambridge\/\">Microsoft Research Cambridge<\/a>. I am the research lead of <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-malmo\/\">Project Malmo<\/a>, which uses the popular game Minecraft as an experimentation platform for developing intelligent technology. My long-term goal is to develop AI systems that learn to collaborate with people, to empower their users and help solve complex real-world problems.\r\n\r\nOutside of Project Malmo, I work on online evaluation and\u00a0interactive learning\u00a0for information retrieval. This means that I try to understand how we can apply machine learning an artificial intelligence to develop more intelligent search and recommendation systems.\r\n\r\nBefore joining Microsoft Research, I completed my PhD in Computer Science as part of the <a href=\"http:\/\/ilps.science.uva.nl\/\">ILPS<\/a> group at the <a title=\"\" href=\"http:\/\/www.uva.nl\/en\">University of Amsterdam<\/a>. I worked with <a href=\"http:\/\/staff.science.uva.nl\/~mdr\/\">Maarten de Rijke<\/a> and <a href=\"http:\/\/staff.science.uva.nl\/~whiteson\">Shimon Whiteson<\/a> on smart search engines that learn directly from their users. My thesis on Fast and Reliable Online Learning to Rank in Information Retrieval can be downloaded from my <a href=\"http:\/\/katja-hofmann.de\/\">personal homepage<\/a>.\r\n[\/panel]\r\n[\/accordion]\r\n<div style=\"height: 15px\"><\/div>\r\n<img class=\"size-full wp-image-441399 alignleft\" style=\"margin-bottom: 10px\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Diego-Perez-Liebana.png\" alt=\"Portrait of Diego Perez-Liebana\" width=\"125\" height=\"125\" \/><span style=\"font-size: x-large\">Challenges and Opportunities of General Video Game AI<\/span>\r\n<div style=\"height: 1px\"><\/div>\r\n<span style=\"font-size: large\"><em>Diego Perez-Liebana, University of Essex<\/em><\/span>\r\n\r\n[accordion]\r\n[panel header=\"Abstract\"]\r\nThe General Video Game AI (GVGAI) framework and competition have attracted many practitioners, researchers, and students during the last couple of years. This benchmark proposes the challenge of creating agents that are able to play any game it\u2019s given, even if it\u2019s not known in advance, in the absence of any domain knowledge. This is proposed in different settings, from single and two-player planning problems to learning without a forward model. Besides, the competition presents another two tests for generality, namely the automatic generation of levels and rules for any game. After briefly introducing the framework and its different tracks, this talk will cover the main challenges and opportunities that research on General Video Game Playing brings for academia and the games industry.\r\n[\/panel]\r\n[panel header=\"Bio\"]\r\n\r\nDiego Perez-Liebana is a Lecturer in Computer Games and Artificial Intelligence at the University of Essex (UK), where he achieved a Ph.D. in Computer Science (2015). His research focuses on Reinforcement Learning and Evolutionary Computation in Game AI. He is author of more than 50 papers at main conferences and journals, including IEEE CIG, IEEE Transactions on Evolutionary Computation and IEEE Transactions on CI and AI in Games. He has organized several international competitions on Game AI for IEEE CIG conferences, such as the Physical Travelling Salesman Problem and the GVGAI Competition. He also served as competitions chair at IEEE CEEC 2015, 2016, track chair at nucl.ai 2015 and 2016, IEEE CIG 2016 and 2018, and he\u2019s member of the CIS Student Games-Based Competitions Sub-Committee. He has several years of experience in the videogames industry, working as a games programmer (Revistronic; Spain), with titles published for both PC and consoles and as an AI for videogames developer (Game Brains; Ireland). He has lectured several modules about games development and game AI.\r\n\r\n[\/panel]\r\n\r\n[\/accordion]\r\n<div style=\"height: 15px\"><\/div>\r\n<img class=\"alignleft wp-image-441402 size-full\" style=\"margin-bottom: 10px\" src=\"https:\/\/microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/11\/Heinrich-Peters.png\" alt=\"Portrait of Heinrich Peters\" width=\"125\" height=\"125\" \/><img \/><span style=\"font-size: x-large\">Psychometric Testing in Project Malmo \u2013 Using an AI Experimentation Platform for Nonverbal Reasoning Assessment in Human Agents<\/span>\r\n<div style=\"height: 1px\"><\/div>\r\n<span style=\"font-size: large\"><em>Heinrich Peters, University of Hamburg<\/em><\/span>\r\n\r\n[accordion]\r\n[panel header=\"Abstract\"]\r\nWe used the interactive and three-dimensional environment in Minecraft\u2019s Project Malmo to create a series of psychometric tests. These overcome several drawbacks of conventional psychometric tests: First, in conventional testing options for item presentation are limited - usually, two dimensional and static content is presented on a screen or on paper. Second, response formats are limited - the psychometric information of an item is often restricted to a simple right or wrong dichotomy. And last but not least, conventional tests are known to create artificial situations associated with low motivation, test anxiety and a variety of other problems. We believe that these shortfalls can be redressed by presenting complex items in an interactive and engaging three-dimensional environment. The result is a rich set of real time behavioral data that can be analyzed in addition to the conventional test score. We are currently trialing the Malmo tests in schools with 10-year-old students. At the same time, we are planning to lay the foundations for in depth exploration of the log data in order to enhance psychometric properties and testing experience while improving our understanding of the underlying mental processes that play a role in nonverbal reasoning and problem solving. My talk will cover the basic concepts of modern psychometric testing theory, Malmo based tasks for human agents and a preliminary assessment of Malmo's potential for future psychometric testing.\r\n[\/panel]\r\n[panel header=\"Bio\"]\r\nHeinrich Peters is a psychology student at the University of Hamburg. The main focus of his study and research activity is on individual differences and psychometrics. He is interested in exploring innovative data sources and bridging psychology and machine learning. His academic experience includes assistant positions in research methods and differential psychology at the University of Hamburg as well as research placements at New York University and the University of Cambridge. Currently, he is developing and validating a game based nonverbal reasoning assessment that is implemented with Microsoft's AI experimentation platform \"Project Malmo\".\r\n[\/panel]\r\n[\/accordion]"}],"msr_startdate":"2017-12-03","msr_enddate":"2017-12-03","msr_event_time":"9:30 AM\u20136:00 PM (PST)","msr_location":"Long Beach, California, USA","msr_event_link":"","msr_event_recording_link":"","msr_startdate_formatted":"December 3, 2017","msr_register_text":"Watch now","msr_cta_link":"","msr_cta_text":"","msr_cta_bi_name":"","featured_image_thumbnail":"<img width=\"960\" height=\"298\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/01\/malmo-comp-header.jpg\" class=\"img-object-cover\" alt=\"Malmo Collaborative AI Challenge\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/01\/malmo-comp-header.jpg 11411w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/01\/malmo-comp-header-300x93.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/01\/malmo-comp-header-768x238.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/01\/malmo-comp-header-1024x318.jpg 1024w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","event_excerpt":"Microsoft is sponsoring a one-day workshop to train people how to run experiments in Project Malmo. This free workshop, which will be held December 3, 2017, at the Hyatt Regency in Long Beach, CA, will include information on technical implementation and task design in this unusual and intuitive environment, as well as offer group exercises, tutorials, and networking opportunities.","msr_research_lab":[199565],"related-researchers":[{"type":"user_nicename","display_name":"Katja Hofmann","user_id":32468,"people_section":"Section name 1","alias":"kahofman"},{"type":"user_nicename","display_name":"Noboru Sean Kuno","user_id":33122,"people_section":"Section name 1","alias":"nkuno"}],"msr_impact_theme":[],"related-academic-programs":[],"related-groups":[],"related-projects":[235753],"related-opportunities":[],"related-publications":[],"related-videos":[],"related-posts":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/428574"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-event"}],"version-history":[{"count":16,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/428574\/revisions"}],"predecessor-version":[{"id":445206,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/428574\/revisions\/445206"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/346142"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=428574"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=428574"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=428574"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=428574"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=428574"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=428574"},{"taxonomy":"msr-program-audience","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-program-audience?post=428574"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=428574"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=428574"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}