{"id":497597,"date":"2018-07-27T09:14:42","date_gmt":"2018-07-27T16:14:42","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=497597"},"modified":"2018-10-26T11:22:58","modified_gmt":"2018-10-26T18:22:58","slug":"project-malmo-reinforcement-learning-in-a-complex-world","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/project-malmo-reinforcement-learning-in-a-complex-world\/","title":{"rendered":"Project Malmo: Reinforcement learning in a complex world"},"content":{"rendered":"

\"\"France\u2019s victory over Croatia in the 2018 FIFA World Cup was as thrilling as sports competition gets. If you\u2019re as much a fan of the game as I am, you enjoyed watching 32 national teams vie for the title over a beautiful month across 11 cities in Russia.<\/p>\n

The riveting action taking place on the pitch reminded us of another kind of competition. But this one, instead of football teams, involves software agents. Two years ago, a collaborative cross-Microsoft Research team that includes participants from Microsoft Research in Redmond, Washington, New York City and Cambridge, United Kingdom launched Project Malmo \u2013 an open-ended platform to advance the state of the art in AI research, especially reinforcement learning in a complex world. The platform is designed to take what\u2019s possible today and push our research toward more ambitious and more difficult tasks.<\/p>\n

Last year we held our first competition, the Malmo Collaborative AI Challenge (opens in new tab)<\/span><\/a>. It focused on human and software agents working together to tackle certain tasks. The competition attracted many students worldwide and the winners were invited to AI Summer School 2017 hosted by Microsoft Research Cambridge. Winning teams received Azure for Research. An interesting discovery in Cambridge was the sheer diversity of approaches from participants. We were delighted to see students showing various creative approaches and well-designed implementations of their agents. Indeed, in the wake of the competition one of the winning teams from Nanyang Technological University published an AAAI paper on their approach titled, \u201cHogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge (opens in new tab)<\/span><\/a>\u201d that is absolutely worth a read.<\/p>\n

Today we\u2019re happy to share an additional milestone involving Project Malmo. Microsoft is partnering with Queen Mary University of London and CrowdAI to co-host a second competition, Learning to Play: The Multi-Agent Reinforcement Learning in Malm\u00d6 (MARL\u00d6) Competition. This competition is a brand-new challenge that proposes research on multi-agent reinforcement learning using multiple games. Participants create learning agents able to play multiple 3D games as de\ufb01ned in the Project Malmo platform. The aim of the competition is to encourage AI research on more general approaches through multi-player games. The challenge will consist of not one but several games, each involving tasks of varying di\ufb03culty and settings. This represents a very unique approach.<\/p>\n

Diego Perez-Liebana, Lecturer in Computer Games and Artificial Intelligence at Queen Mary University of London, United Kingdom talked about the potential impact of the Malmo competitions on AI research, \u201cOur research group has been running AI game-based competitions for many years and we are well aware of the multiple benefits these bring,\u201d said Perez-Liebana. \u201cThey provide a common benchmark for multiple researchers across the globe to train their AI agents in a way that is comparable, allowing us to effectively contrast different techniques in a common domain,\u201d he continued. Game AI competitions are a great resource for education, as they can be proposed as assignment or project from undergraduate to PhD level. \u201cGames are fun, and so is AI,\u201d said Perez-Liebana. Indeed, combining the two helps popularize challenges and solutions faster and more broadly than any other methods. Perez-Liebana pointed to the evolution of the Monte Carlo Tree Search methods during successive Go competitions that led to the use of this method in multiple other games and domains as a clear example of this.<\/p>\n

Sharada Prasanna Mohanty, a PhD student at EPFL, Switzerland, co-founder of CrowdAI expressed his expectations regarding the competition. \u201cWith this challenge, our principal goal is to make available a series of problems for the community of multi-agent reinforcement learning researchers to collaboratively work on,\u201d said Sharada Mohanty. \u201cWith Minecraft as the main platform enabling this research, we also hope to inspire many other researchers and engineers from various domains to get involved in reinforcement learning research. The success of this challenge can help establish these tasks as standard benchmark tasks for all multi-agent reinforcement learning researchers to compare their approaches in the future and at the same time can potentially help us better measure our own progress in multi-agent reinforcement learning research as a community over time.\u201d<\/p>\n

The competition is open to anyone worldwide. Visit the competition page (opens in new tab)<\/span><\/a> for more detail about registration and rules. Qualifying rounds last until November 12th. The top 32 teams in the qualifying rounds can move forward to the knockout rounds of the final tournament, where team agents compete each other on an exciting set of games and tasks. The tournament will be a live competition in MARLO workshop (opens in new tab)<\/span><\/a> at the 14th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (opens in new tab)<\/span><\/a> (AIIDE’18) to be held at the University of Alberta in Edmonton, AB, Canada on November 14, 2018. We\u2019re also calling for papers (opens in new tab)<\/span><\/a> at the workshop.<\/p>\n

We hope to see as many students, researchers and engineers as possible share their innovative approaches and creative ideas for multi-agent reinforcement learning at the workshop in Edmonton. Let\u2019s kick-off! You are now in possession of the ball!<\/p>\n","protected":false},"excerpt":{"rendered":"

France\u2019s victory over Croatia in the 2018 FIFA World Cup was as thrilling as sports competition gets. If you\u2019re as much a fan of the game as I am, you enjoyed watching 32 national teams vie for the title over a beautiful month across 11 cities in Russia. The riveting action taking place on the […]<\/p>\n","protected":false},"author":37074,"featured_media":497603,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[241770],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-497597","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","msr-research-area-artificial-intelligence","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[346139],"related-groups":[],"related-projects":[235753],"related-events":[454587],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"\"Project","byline":"Noboru Sean Kuno","formattedDate":"July 27, 2018","formattedExcerpt":"France\u2019s victory over Croatia in the 2018 FIFA World Cup was as thrilling as sports competition gets. If you\u2019re as much a fan of the game as I am, you enjoyed watching 32 national teams vie for the title over a beautiful month across 11…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/497597","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/37074"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=497597"}],"version-history":[{"count":6,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/497597\/revisions"}],"predecessor-version":[{"id":545802,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/497597\/revisions\/545802"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/497603"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=497597"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=497597"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=497597"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=497597"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=497597"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=497597"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=497597"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=497597"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=497597"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=497597"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=497597"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}