Minecraft as AI Playground and Laboratory

Modern video games provide exciting challenges and opportunities for pushing the state of the art in machine learning and other research areas, and, in turn, stand to first benefit from research advances. For driving research, games provide rich data that can be used to tackle hard problems, from complex decision making to collaboration. If and when these are successfully tackled, new algorithms and insights have the potential to enable entirely new game experiences.

This talk focuses on opportunities in the setting of the game Minecraft, one of the most popular video games of all time. Minecraft is an open-world game, where players explore, create, and continuously find new ways to play and engage with each other. This open-ended nature both make the game appealing to its human fan-base, and uniquely challenging to AI algorithms. To unlock the potential of Minecraft for AI experimentation, my team has developed Project Malmo — an open source experimentation platform built on top of Minecraft to enable a wide range of research. Here, I will illustrate the capabilities of the platform with recent examples that I find particularly exciting.

I will highlight our most recent collaboration, led by a team of PhD students at Carnegie Mellon University: the MineRL competition. This ambitious competition is designed to drive advances in sample efficient reinforcement learning with human priors. Sample efficient learning is a key challenge, with current algorithms often requiring millions of samples to learn to perform individual narrow tasks, limiting the scope and applicability of these approaches. This competition is built around a complex task, large-scale demonstration data, and an evaluation setup that requires and rewards sample efficient learning and effective generalization.

Looking out into the future, I will conclude by highlight selected open questions and challenges that have high potential for impact in video games and raise key questions for current state-of-the-art AI approaches.