Designing Tasks for the Future of AI – Project Malmo Workshop

Project Malmo is an open source AI experimentation platform to support fundamental research in artificial intelligence. With the platform, Microsoft aims to provide an experimentation environment in which promising approaches can be systematically and easily compared, and that fosters collaboration between researchers. Project Malmo achieves flexibility by building on top of Minecraft, which is particularly appealing due to its open-ended nature, collaboration with other players, and creativity in game-play.

In this talk I will introduce the Malmo Platform, with a focus on its capabilities for easily creating a wide range of single- and multi-agent AI tasks. In particular, I will exemplify task design and challenges in the context of the Malmo Collaborative AI Challenge – a challenge our team recently conducted to foster research in collaborative AI. Aim of the talk is to give the audience a broad overview of the possibilities of Project Malmo and how they can drive an ambitious AI research agenda.

I am a researcher at the Machine Intelligence and Perception (opens in new tab) group at Microsoft Research Cambridge (opens in new tab). I am the research lead of Project Malmo (opens in new tab), which uses the popular game Minecraft as an experimentation platform for developing intelligent technology. My long-term goal is to develop AI systems that learn to collaborate with people, to empower their users and help solve complex real-world problems.

Outside of Project Malmo, I work on online evaluation and interactive learning for information retrieval. This means that I try to understand how we can apply machine learning an artificial intelligence to develop more intelligent search and recommendation systems.

Before joining Microsoft Research, I completed my PhD in Computer Science as part of the ILPS (opens in new tab) group at the University of Amsterdam (opens in new tab). I worked with Maarten de Rijke (opens in new tab) and Shimon Whiteson (opens in new tab) on smart search engines that learn directly from their users. My thesis on Fast and Reliable Online Learning to Rank in Information Retrieval can be downloaded from my personal homepage (opens in new tab).

The General Video Game AI (GVGAI) framework and competition have attracted many practitioners, researchers, and students during the last couple of years. This benchmark proposes the challenge of creating agents that are able to play any game it’s given, even if it’s not known in advance, in the absence of any domain knowledge. This is proposed in different settings, from single and two-player planning problems to learning without a forward model. Besides, the competition presents another two tests for generality, namely the automatic generation of levels and rules for any game. After briefly introducing the framework and its different tracks, this talk will cover the main challenges and opportunities that research on General Video Game Playing brings for academia and the games industry.

Diego Perez-Liebana is a Lecturer in Computer Games and Artificial Intelligence at the University of Essex (UK), where he achieved a Ph.D. in Computer Science (2015). His research focuses on Reinforcement Learning and Evolutionary Computation in Game AI. He is author of more than 50 papers at main conferences and journals, including IEEE CIG, IEEE Transactions on Evolutionary Computation and IEEE Transactions on CI and AI in Games. He has organized several international competitions on Game AI for IEEE CIG conferences, such as the Physical Travelling Salesman Problem and the GVGAI Competition. He also served as competitions chair at IEEE CEEC 2015, 2016, track chair at nucl.ai 2015 and 2016, IEEE CIG 2016 and 2018, and he’s member of the CIS Student Games-Based Competitions Sub-Committee. He has several years of experience in the videogames industry, working as a games programmer (Revistronic; Spain), with titles published for both PC and consoles and as an AI for videogames developer (Game Brains; Ireland). He has lectured several modules about games development and game AI.

We used the interactive and three-dimensional environment in Minecraft’s Project Malmo to create a series of psychometric tests. These overcome several drawbacks of conventional psychometric tests: First, in conventional testing options for item presentation are limited – usually, two dimensional and static content is presented on a screen or on paper. Second, response formats are limited – the psychometric information of an item is often restricted to a simple right or wrong dichotomy. And last but not least, conventional tests are known to create artificial situations associated with low motivation, test anxiety and a variety of other problems. We believe that these shortfalls can be redressed by presenting complex items in an interactive and engaging three-dimensional environment. The result is a rich set of real time behavioral data that can be analyzed in addition to the conventional test score. We are currently trialing the Malmo tests in schools with 10-year-old students. At the same time, we are planning to lay the foundations for in depth exploration of the log data in order to enhance psychometric properties and testing experience while improving our understanding of the underlying mental processes that play a role in nonverbal reasoning and problem solving. My talk will cover the basic concepts of modern psychometric testing theory, Malmo based tasks for human agents and a preliminary assessment of Malmo’s potential for future psychometric testing.

Heinrich Peters is a psychology student at the University of Hamburg. The main focus of his study and research activity is on individual differences and psychometrics. He is interested in exploring innovative data sources and bridging psychology and machine learning. His academic experience includes assistant positions in research methods and differential psychology at the University of Hamburg as well as research placements at New York University and the University of Cambridge. Currently, he is developing and validating a game based nonverbal reasoning assessment that is implemented with Microsoft’s AI experimentation platform “Project Malmo”.