{"id":720673,"date":"2021-02-05T13:01:57","date_gmt":"2021-02-05T21:01:57","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=720673"},"modified":"2023-03-06T09:13:27","modified_gmt":"2023-03-06T17:13:27","slug":"research-collection-shall-we-play-a-game","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/research-collection-shall-we-play-a-game\/","title":{"rendered":"Research Collection \u2013 Shall we play a game?"},"content":{"rendered":"<aside id=accordion-c182c1d2-b051-4ea6-ab01-f77c582aa25d class=\"msr-table-of-contents-block accordion mb-5 pb-0\" data-bi-aN=\"table-of-contents\">\n\t<button class=\"btn btn-collapse bg-gray-100 mb-0 display-flex justify-content-between\" type=\"button\" data-mount=\"collapse\" data-target=\"#accordion-collapse-c182c1d2-b051-4ea6-ab01-f77c582aa25d\" aria-expanded=\"true\" aria-controls=\"accordion-collapse-c182c1d2-b051-4ea6-ab01-f77c582aa25d\">\n\t\t<span class=\"msr-table-of-contents-block__label subtitle\">In this article<\/span>\n\t\t<span class=\"msr-table-of-contents-block__current mr-4 text-gray-600 font-weight-normal\" aria-hidden=\"true\"><\/span>\n\t<\/button>\n\t<div id=\"accordion-collapse-c182c1d2-b051-4ea6-ab01-f77c582aa25d\" class=\"msr-table-of-contents-block__collapse-wrapper collapse show\" data-parent=\"#accordion-c182c1d2-b051-4ea6-ab01-f77c582aa25d\">\n\t\t<div class=\"accordion-body bg-gray-100 border-top pt-4\">\n\t\t\t<ol class=\"msr-table-of-contents-block__list\">\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"\" class=\"msr-table-of-contents-block__list-item-link\">Games as a research tool<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"\" class=\"msr-table-of-contents-block__list-item-link\">Research to improve the gaming experience<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<\/ul>\n\t\t<\/div>\n\t<\/div>\n\t<span class=\"msr-table-of-contents-block__progress-bar\"><\/span>\n<\/aside>\n\n\n\n<blockquote class=\"wp-block-quote is-style-default is-layout-flow wp-block-quote-is-layout-flow\">\n<p><em>From a research point of view, games offer an amazing environment in which to develop new machine learning algorithms and techniques. And we hope, in due course, that those new algorithms will feed back not just into gaming, but into many other domains. Beyond the very technical machine learning techniques themselves, gaming is an environment in which we can explore the relationship between AI and people, and see how they can work in partnership. It\u2019s a very rich environment in which to drive new research ideas.<\/em><\/p>\n<cite>Dr. Chris Bishop, Technical Fellow and Lab Director, Microsoft Research Cambridge<\/cite><\/blockquote>\n\n\n\n<p>The history of computer gaming goes back nearly as far as the history of computer science itself. As early as the 1950s, researchers were using games to push the boundaries of computing, driving many advances in computer graphics and human-computer interaction. Games played a role in the earliest research into artificial intelligence, as researchers trained the ancestors of modern AI systems to learn and play games like chess and tic-tac-toe. From the 1960s campus phenomenon of <em>Spacewar!<\/em> to the insatiable demand for the latest generation of consoles, games endure as a popular and engaging pastime \u2013 and, for many future researchers and developers, as a gateway to a career in computer science.<\/p>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/the-human-side-of-ai-for-chess\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"The human side of AI for chess\" data-bi-aN=\"margin-callout\" data-bi-cN=\"The human side of AI for chess\">\n\t\t\t\tThe human side of AI for chess&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>Today, researchers are working to advance the state of the art in gaming in many different ways \u2013 enabling more realistic and immersive sound and graphics, using AI to improve gameplay and make gaming more engaging for spectators, and making games more accessible, inclusive and safer to play. Games continue to play a role in advancing artificial intelligence, offering up new challenges for AI models to solve, and using game environments to train language models and autonomous systems. Chess also continues to be fertile ground for AI research; most recently, researchers from Microsoft and their collaborators explore how to better match AI to human skill levels and playing styles.<\/p>\n\n\n\n<p>Listen to <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/game-on-with-dr-chris-bishop-and-phil-spencer\/\">this podcast<\/a> with Dr. Chris Bishop and Microsoft Executive Vice President of gaming Phil Spencer about the intersection of research and gaming at Microsoft.<\/p>\n\n\n\n<p>In February 2021, Microsoft Research hosted the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/aiandgaming2020\/\">AI and Gaming Research Summit<\/a>, which brought together researchers and innovators from academia, creators from the gaming industry, Microsoft Research (MSR), and Gaming teams at Microsoft, to share ideas and discuss how AI and machine learning is transforming the landscape of gaming. All recorded talks are now available on the Videos tab at the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/aiandgaming2021\/#!videos\">event site<\/a>. <\/p>\n\n\n\n<p>This collection highlights much of the research happening across Microsoft to advance the state of the art in gaming, and put games to work to enable new breakthroughs in artificial intelligence.<\/p>\n\n\n\n\t<div class=\"border-bottom border-top border-gray-300 mt-5 mb-5 msr-promo text-center text-md-left alignwide\" data-bi-aN=\"promo\" data-bi-id=\"1085496\">\n\t\t\n\n\t\t<p class=\"msr-promo__label text-gray-800 text-center text-uppercase\">\n\t\t<span class=\"px-4 bg-white display-inline-block font-weight-semibold small\">Spotlight: Blog post<\/span>\n\t<\/p>\n\t\n\t<div class=\"row pt-3 pb-4 align-items-center\">\n\t\t\t\t\t\t<div class=\"msr-promo__media col-12 col-md-5\">\n\t\t\t\t<a class=\"bg-gray-300\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/eureka-evaluating-and-understanding-progress-in-ai\/\" aria-label=\"Eureka: Evaluating and understanding progress in AI\" data-bi-cN=\"Eureka: Evaluating and understanding progress in AI\" target=\"_blank\">\n\t\t\t\t\t<img decoding=\"async\" class=\"w-100 display-block\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/09\/NEWEureka-2024-BlogHeroFeature-1400x788-1-66e90c195464d.jpg\" alt=\"White icons representing the Eureka Experiment Pipeline -- PromptProcessing, Inference, PromptProcessing, Inference, DataProcessing, EvalReporting -- on blue to green gradient background.\" \/>\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"msr-promo__content p-3 px-5 col-12 col-md\">\n\n\t\t\t\t\t\t\t\t\t<h2 class=\"h4\">Eureka: Evaluating and understanding progress in AI<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<p class=\"large\">How can we rigorously evaluate and understand state-of-the-art progress in AI? Eureka is an open-source framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings. Learn more about the extended findings.\u00a0<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"wp-block-buttons justify-content-center justify-content-md-start\">\n\t\t\t\t\t<div class=\"wp-block-button\">\n\t\t\t\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/eureka-evaluating-and-understanding-progress-in-ai\/\" class=\"btn btn-brand glyph-append glyph-append-chevron-right\" aria-label=\"Read more\" data-bi-cN=\"Eureka: Evaluating and understanding progress in AI\" target=\"_blank\">\n\t\t\t\t\t\t\tRead more\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div><!--\/.msr-promo__content-->\n\t<\/div><!--\/.msr-promo__inner-wrap-->\n\t<\/div><!--\/.msr-promo-->\n\t\n\n\n<h2 id=\"games-as-a-research-tool\">Games as a research tool<\/h2>\n\n\n\n<h3 id=\"airsim\">AirSim<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"480\" height=\"266\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/05\/frame_54_delay-0.07s.png\" alt=\"simulated aerial drone navigating course\" class=\"wp-image-660678\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/05\/frame_54_delay-0.07s.png 480w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/05\/frame_54_delay-0.07s-300x166.png 300w\" sizes=\"(max-width: 480px) 100vw, 480px\" \/><\/figure>\n\n\n\n<p>Microsoft AirSim (<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/aerial-informatics-robotics-platform\/\">Aerial Informatics and Robotics Simulation<\/a>) is an open-source robotics simulation platform. From ground vehicles, wheeled robotics, aerial drones, and even static IoT devices, AirSim can capture data for models without costly field operations. AirSim works as a plug-in to Epic Games\u2019 Unreal Engine 4 editor, providing control over building environments and simulating difficult-to-reproduce, real-world events to capture meaningful data for AI models.<\/p>\n\n\n\n<p>Using AirSim, researchers can capture data for machine learning models in a virtual environment, without costly field operations.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Download<\/span>\n\t\t\t<a href=\"https:\/\/github.com\/microsoft\/AirSim-Drone-Racing-VAE-Imitation\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"AirSim Drone Racing VAE Imitation\" data-bi-aN=\"citation\" data-bi-cN=\"AirSim Drone Racing VAE Imitation\">\n\t\t\t\tAirSim Drone Racing VAE Imitation&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/game-of-drones-at-neurips-2019-simulation-based-drone-racing-competition-built-on-airsim\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Game of Drones at NeurIPS 2019: Simulation-based drone-racing competition built on AirSim\" data-bi-aN=\"citation\" data-bi-cN=\"Game of Drones at NeurIPS 2019: Simulation-based drone-racing competition built on AirSim\">\n\t\t\t\tGame of Drones at NeurIPS 2019: Simulation-based drone-racing competition built on AirSim&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/autonomous-systems-aerial-robotics-and-game-of-drones-with-gurdeep-pall-and-dr-ashish-kapoor\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Autonomous systems, aerial robotics and Game of Drones with Gurdeep Pall and Dr. Ashish Kapoor\" data-bi-aN=\"citation\" data-bi-cN=\"Autonomous systems, aerial robotics and Game of Drones with Gurdeep Pall and Dr. Ashish Kapoor\">\n\t\t\t\tAutonomous systems, aerial robotics and Game of Drones with Gurdeep Pall and Dr. Ashish Kapoor&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p><\/p>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"hybrid-reward-architecture\">Hybrid Reward Architecture<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"582\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2017\/06\/download-1024x582.gif\" alt=\"Hybrid Reward Architecture for Ms. Pac-Man\" class=\"wp-image-455751\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2017\/06\/download-1024x582.gif 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2017\/06\/download-300x171.gif 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2017\/06\/download-768x437.gif 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Games are popular as a testbed for new machine learning techniques because they can be very challenging and allow for easy analysis of new learning techniques in a controlled environment. For reinforcement learning (RL), where the goal is to learn good behavior in a data-driven way, the Arcade Learning Environment (ALE), which provides access to a large number of Atari 2600 games, has been a popular testing environment. The&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/hybrid-reward-architecture\/\">Hybrid Reward Architecture<\/a>&nbsp;project combines standard reinforcement learning techniques with deep neural networks, with the aim of outperforming humans in Arcade Learning Environment (ALE) games. In 2017, it achieved a perfect score on Ms. Pac-Man.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-2 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/hybrid-reward-architecture-achieving-super-human-ms-pac-man-performance\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Hybrid Reward Architecture (HRA) Achieving super-human performance on Ms. Pac-Man\" data-bi-aN=\"citation\" data-bi-cN=\"Hybrid Reward Architecture (HRA) Achieving super-human performance on Ms. Pac-Man\">\n\t\t\t\tHybrid Reward Architecture (HRA) Achieving super-human performance on Ms. Pac-Man&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/hybrid-reward-architecture-reinforcement-learning\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Hybrid Reward Architecture for Reinforcement Learning\" data-bi-aN=\"citation\" data-bi-cN=\"Hybrid Reward Architecture for Reinforcement Learning\">\n\t\t\t\tHybrid Reward Architecture for Reinforcement Learning&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/hybrid-reward-architecture-fall-ms-pac-man-dr-harm-van-seijen\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Hybrid Reward Architecture and the Fall of Ms. Pac-Man with Dr. Harm van Seijen\" data-bi-aN=\"citation\" data-bi-cN=\"Hybrid Reward Architecture and the Fall of Ms. Pac-Man with Dr. Harm van Seijen\">\n\t\t\t\tHybrid Reward Architecture and the Fall of Ms. Pac-Man with Dr. Harm van Seijen&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"project-malmo\">Project Malmo<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"247\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2016\/06\/malmo_human_ai_interaction-web.png\" alt=\"Project malmo\" class=\"wp-image-235757\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/malmo_human_ai_interaction-web.png 800w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/malmo_human_ai_interaction-web-300x93.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/malmo_human_ai_interaction-web-768x237.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/figure>\n\n\n\n<p>The Malmo platform is a sophisticated AI experimentation platform built on top of Minecraft, and designed to support fundamental research in artificial intelligence.<\/p>\n\n\n\n<p>The <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-malmo\/\">Project Malmo<\/a> platform consists of a mod for the Java version, and code that helps artificial intelligence agents sense and act within the Minecraft environment. The two components can run on Windows, Linux, or Mac OS, and researchers can program their agents in any programming language they\u2019re comfortable with.<\/p>\n\n\n\n<p>Minecraft is ideal for artificial intelligence research for the same reason it is appealing to the millions of fans who enter its virtual world every day. Unlike other computer games, Minecraft offers its users endless possibilities, ranging from simple tasks, like walking around looking for treasure, to complex ones, like building a structure with a group of teammates.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-3 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/project-malmo-reinforcement-learning-in-a-complex-world\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Malmo: Reinforcement learning in a complex world\" data-bi-aN=\"citation\" data-bi-cN=\"Project Malmo: Reinforcement learning in a complex world\">\n\t\t\t\tProject Malmo: Reinforcement learning in a complex world&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/minerl-sample-efficient-reinforcement-learning-challenge-back-for-a-second-year-benefits-organizers-as-well-as-larger-research-community\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"MineRL sample-efficient reinforcement learning challenge\u2014back for a second year\u2014benefits organizers, as well as larger research community\" data-bi-aN=\"citation\" data-bi-cN=\"MineRL sample-efficient reinforcement learning challenge\u2014back for a second year\u2014benefits organizers, as well as larger research community\">\n\t\t\t\tMineRL sample-efficient reinforcement learning challenge\u2014back for a second year\u2014benefits organizers, as well as larger research community&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/malmo-minecraft-and-machine-learning-with-dr-katja-hofmann\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Malmo, Minecraft and machine learning with Dr. Katja Hofmann\" data-bi-aN=\"citation\" data-bi-cN=\"Malmo, Minecraft and machine learning with Dr. Katja Hofmann\">\n\t\t\t\tMalmo, Minecraft and machine learning with Dr. Katja Hofmann&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/malmo-platform-artificial-intelligence-experimentation\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"The Malmo Platform for Artificial Intelligence Experimentation\" data-bi-aN=\"citation\" data-bi-cN=\"The Malmo Platform for Artificial Intelligence Experimentation\">\n\t\t\t\tThe Malmo Platform for Artificial Intelligence Experimentation&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/project-malmo-enabling-ai-technology-can-collaborate-humans\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Malmo: Enabling AI technology that can collaborate with humans\" data-bi-aN=\"citation\" data-bi-cN=\"Project Malmo: Enabling AI technology that can collaborate with humans\">\n\t\t\t\tProject Malmo: Enabling AI technology that can collaborate with humans&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"super-phoenix-suphx\">Super Phoenix (Suphx)<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1024x576.jpg\" alt=\"Suphx - Mahjong board\" class=\"wp-image-720721\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-16x9.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong.jpg 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Mahjong is a popular multi-player imperfect-information game, but it is very challenging for AI research due to its complex playing\/scoring rules and rich hidden information. In 2019, Microsoft researchers designed <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/suphx-mastering-mahjong-with-deep-reinforcement-learning\/\">Super Phoenix (Suphx)<\/a>, an AI for Mahjong based on deep reinforcement learning with some newly introduced techniques including global reward prediction, oracle guiding, and run-time policy adaptation. Suphx has demonstrated stronger performance than most top human players in terms of stable rank and is rated above 99.99% of all the officially ranked human players in the Tenhou platform. This is the first time that a computer program outperformed most top human players in Mahjong.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-4 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/news.microsoft.com\/apac\/features\/mastering-mahjong-with-ai-and-machine-learning\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"More than a game: Mastering Mahjong with AI and machine learning - Microsoft Stories Asia\" data-bi-aN=\"citation\" data-bi-cN=\"More than a game: Mastering Mahjong with AI and machine learning - Microsoft Stories Asia\">\n\t\t\t\tMore than a game: Mastering Mahjong with AI and machine learning &#8211; Microsoft Stories Asia&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/suphx-mastering-mahjong-with-deep-reinforcement-learning\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Suphx: Mastering Mahjong with Deep Reinforcement Learning\" data-bi-aN=\"citation\" data-bi-cN=\"Suphx: Mastering Mahjong with Deep Reinforcement Learning\">\n\t\t\t\tSuphx: Mastering Mahjong with Deep Reinforcement Learning&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"textworld\">TextWorld<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"528\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/08\/fig_1_textworld-1024x528.png\" alt=\"screen shot of the TextWorld interface\" class=\"wp-image-603234\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/08\/fig_1_textworld-1024x528.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/08\/fig_1_textworld-300x155.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/08\/fig_1_textworld-768x396.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/08\/fig_1_textworld.png 1376w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Text-based games may seem primitive next to the beautifully rendered graphics of today, but to succeed at even the simplest such game, humans use a special set of skills. We seamlessly and simultaneously comprehend language descriptions, plan our next moves, visualize the environment, remember important information, and generalize past experiences to new situations. AI agents don\u2019t yet possess these capabilities, but they are key to general intelligence. Microsoft&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/textworld\/\">TextWorld<\/a>&nbsp;is an open-source, extensible platform that both generates and simulates text-based games. It is a popular testing ground for AI, used by researchers to train, evaluate, analyze, and improve reinforcement learning (RL) agents that learn from and interact with the world through language.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-5 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Download<\/span>\n\t\t\t<a href=\"https:\/\/github.com\/Microsoft\/TextWorld\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"TextWorld\" data-bi-aN=\"citation\" data-bi-cN=\"TextWorld\">\n\t\t\t\tTextWorld&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/first-textworld-problems-the-competition-using-text-based-games-to-advance-capabilities-of-ai-agents\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"First TextWorld Problems, the competition: Using text-based games to advance capabilities of AI agents\" data-bi-aN=\"citation\" data-bi-cN=\"First TextWorld Problems, the competition: Using text-based games to advance capabilities of AI agents\">\n\t\t\t\tFirst TextWorld Problems, the competition: Using text-based games to advance capabilities of AI agents&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/textworld-a-learning-environment-for-training-reinforcement-learning-agents-inspired-by-text-based-games\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"TextWorld: A learning environment for training reinforcement learning agents, inspired by text-based games\" data-bi-aN=\"citation\" data-bi-cN=\"TextWorld: A learning environment for training reinforcement learning agents, inspired by text-based games\">\n\t\t\t\tTextWorld: A learning environment for training reinforcement learning agents, inspired by text-based games&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/getting-started-with-textworld\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Getting started with TextWorld\" data-bi-aN=\"citation\" data-bi-cN=\"Getting started with TextWorld\">\n\t\t\t\tGetting started with TextWorld&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h2 id=\"research-to-improve-the-gaming-experience\">Research to improve the gaming experience<\/h2>\n\n\n\n<h3 id=\"project-paidia\">Project Paidia<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"410\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-1024x410.png\" alt=\"example game screens used in Project Paidia\" class=\"wp-image-678297\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-1024x410.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-300x120.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-768x307.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560.png 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-paidia\/\">Project Paidia<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> is a research project in close collaboration between Microsoft Research Cambridge and&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/ninjatheory.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Ninja Theory<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. Its focus is to drive state of the art research in reinforcement learning to enable novel applications in modern video games, in particular: agents that learn to collaborate with human players.<\/p>\n\n\n\n<p>In contrast to traditional approaches to crafting the behavior of bots, non-player characters, or other in-game characters, reinforcement learning does not require a game developer to anticipate a wide range of possible game situations and map out and code all required behaviors. Instead, with reinforcement learning, game developers control a reward signal which the game character then learns to optimize while responding fluidly to all aspects of a game\u2019s dynamics. The result is nuanced situation and player-aware emergent behavior that would be challenging or prohibitive to achieve using traditional Game AI.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-6 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/developer.microsoft.com\/en-us\/games\/blog\/supercharge-games-with-azure-ai-and-reinforcement-learning\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Game Stack - Supercharge games with Azure AI and reinforcement learning | Microsoft Developer\" data-bi-aN=\"citation\" data-bi-cN=\"Game Stack - Supercharge games with Azure AI and reinforcement learning | Microsoft Developer\">\n\t\t\t\tGame Stack &#8211; Supercharge games with Azure AI and reinforcement learning | Microsoft Developer&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/three-new-reinforcement-learning-methods-aim-to-improve-ai-in-gaming-and-beyond\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Three new reinforcement learning methods aim to improve AI in gaming and beyond\" data-bi-aN=\"citation\" data-bi-cN=\"Three new reinforcement learning methods aim to improve AI in gaming and beyond\">\n\t\t\t\tThree new reinforcement learning methods aim to improve AI in gaming and beyond&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/inside-the-microsoft-ai-residency-program-with-dr-brian-broll\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Inside the Microsoft AI Residency Program with Dr. Brian Broll\" data-bi-aN=\"citation\" data-bi-cN=\"Inside the Microsoft AI Residency Program with Dr. Brian Broll\">\n\t\t\t\tInside the Microsoft AI Residency Program with Dr. Brian Broll&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"roomalive-toolkit\">RoomAlive Toolkit<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"933\" height=\"555\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Roomalive-Toolkit_example-scene.png\" alt=\"RoomAlive example scene\" class=\"wp-image-720949\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Roomalive-Toolkit_example-scene.png 933w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Roomalive-Toolkit_example-scene-300x178.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Roomalive-Toolkit_example-scene-768x457.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Roomalive-Toolkit_example-scene-16x10.png 16w\" sizes=\"(max-width: 933px) 100vw, 933px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/roomalive-toolkit\/\">RoomAlive Toolkit<\/a>&nbsp;is an open source SDK that transforms any room into an immersive, augmented, magical entertainment experience. RoomAlive Toolkit presents a unified, scalable approach for interactive projection mapping that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Download<\/span>\n\t\t\t<a href=\"https:\/\/github.com\/Microsoft\/RoomAliveToolkit\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"RoomAlive Toolkit\" data-bi-aN=\"citation\" data-bi-cN=\"RoomAlive Toolkit\">\n\t\t\t\tRoomAlive Toolkit&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/illumiroom-peripheral-projected-illusions-for-interactive-experiences\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"IllumiRoom: Peripheral Projected Illusions for Interactive Experiences\" data-bi-aN=\"citation\" data-bi-cN=\"IllumiRoom: Peripheral Projected Illusions for Interactive Experiences\">\n\t\t\t\tIllumiRoom: Peripheral Projected Illusions for Interactive Experiences&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"seeingvr\">SeeingVR<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"349\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2018\/12\/seeingvr-1024x349.png\" alt=\"SeeingVR's low vision tools\" class=\"wp-image-556662\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2018\/12\/seeingvr-1024x349.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2018\/12\/seeingvr-300x102.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2018\/12\/seeingvr-768x262.png 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/seeingvr\/\">SeeingVR<\/a>&nbsp;is a research toolkit for making virtual reality more accessible to people with low vision \u2013 moderate to severe visual impairments that cannot be corrected fully by glasses. It includes a number of enhancements that provide visual and audio augmentations to existing VR experiences \u2013 including magnification, edge enhancement, brightness and contrast adjustment, text augmentation and text-to-speech. The toolkit also enables VR developers to add capabilities such as object descriptions and object labeling and highlighting. SeeingVR is designed to be used with the Unity VR development platform.<\/p>\n\n\n\n<p>SeeingVR includes enhancements that help developers make VR more accessible to people with low vision.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-8 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Download<\/span>\n\t\t\t<a href=\"http:\/\/aka.ms\/seeingvropensource\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"SeeingVR Toolkit\" data-bi-aN=\"citation\" data-bi-cN=\"SeeingVR Toolkit\">\n\t\t\t\tSeeingVR Toolkit&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/inclusive-design-for-all-or-ict4d-and-4u-with-dr-ed-cutrell\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Inclusive design for all, or ICT4D and 4U! with Dr. Ed Cutrell\" data-bi-aN=\"citation\" data-bi-cN=\"Inclusive design for all, or ICT4D and 4U! with Dr. Ed Cutrell\">\n\t\t\t\tInclusive design for all, or ICT4D and 4U! with Dr. Ed Cutrell&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"http:\/\/aka.ms\/seeingvrpaper\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision\" data-bi-aN=\"citation\" data-bi-cN=\"SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision\">\n\t\t\t\tSeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"http:\/\/aka.ms\/seeingvrvideo\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision\" data-bi-aN=\"citation\" data-bi-cN=\"SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision\">\n\t\t\t\tSeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"project-triton\">Project Triton<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"683\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-1024x683.jpg\" alt=\"virtual world example in Project Acoustics\" class=\"wp-image-720952\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-1024x683.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-768x512.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-16x12.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels.jpg 1098w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-triton\/\">Project Triton<\/a>&nbsp;renders immersive acoustics that responds to the shape and materials of 3D environments in games and augmented\/virtual reality (AR\/VR). It is unique in modeling the true wave physics of sound, including diffraction, producing natural effects like smooth occlusion and scene-dependent reverberation. Detailed simulation is performed offline, yielding a lightweight runtime that scales from desktop to mobile phones. Incubated over a decade in Microsoft Research, Project Triton ships in major game titles like Gears of War, Sea of Thieves, and Borderlands 3. Plugin integrations for Unity and Unreal Engine are available as part of Project Acoustics.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Download<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=57346\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Acoustics Unity plugin and samples\" data-bi-aN=\"citation\" data-bi-cN=\"Project Acoustics Unity plugin and samples\">\n\t\t\t\tProject Acoustics Unity plugin and samples&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Documentation<\/span>\n\t\t\t<a href=\"https:\/\/docs.microsoft.com\/en-us\/gaming\/acoustics\/what-is-acoustics\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Acoustics\" data-bi-aN=\"citation\" data-bi-cN=\"Project Acoustics\">\n\t\t\t\tProject Acoustics&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/project-triton-and-the-physics-of-sound-with-dr-nikunj-raghuvanshi\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Triton and the physics of sound with Dr. Nikunj Raghuvanshi\" data-bi-aN=\"citation\" data-bi-cN=\"Project Triton and the physics of sound with Dr. Nikunj Raghuvanshi\">\n\t\t\t\tProject Triton and the physics of sound with Dr. Nikunj Raghuvanshi&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"truematch\">TrueMatch<\/h3>\n\n\n\n<p>Developed by Microsoft researchers and based on reinforcement learning, the TrueMatch matchmaking system uses AI to decide which people should play together in online multiplayer games.<\/p>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/truematch\/\">TrueMatch<\/a> dynamically adjusts players\u2019 matchmaking allowances based on their location and skill level. If there are many players nearby, then TrueMatch shrinks the allowances so that they only get fair matches with low ping. If there are few players nearby, TrueMatch expands the allowances. Previous systems expand the allowance only after a player has waited a long time in the queue. TrueMatch starts the search pre-expanded, cutting wait time in half in low population regions. TrueMatch can also give players a personalized estimate of their wait time, before they queue up to play.<\/p>\n\n\n\n<p>TrueMatch has launched in Gears of War 4, Gears of War 5, and Halo 5.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-10 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Project<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/truematch\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"TrueMatch Matchmaking System\" data-bi-aN=\"citation\" data-bi-cN=\"TrueMatch Matchmaking System\">\n\t\t\t\tTrueMatch Matchmaking System&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/levelup-a-thin-cloud-approach-to-game-livestreaming\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"LevelUp: A thin-cloud approach to game livestreaming\" data-bi-aN=\"citation\" data-bi-cN=\"LevelUp: A thin-cloud approach to game livestreaming\">\n\t\t\t\tLevelUp: A thin-cloud approach to game livestreaming&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/switchboard-a-matchmaking-system-for-multiplayer-mobile-games\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Switchboard: A Matchmaking System for Multiplayer Mobile Games\" data-bi-aN=\"citation\" data-bi-cN=\"Switchboard: A Matchmaking System for Multiplayer Mobile Games\">\n\t\t\t\tSwitchboard: A Matchmaking System for Multiplayer Mobile Games&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/youtu.be\/Q8BX0nXfPjY\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Machine Learning for Optimal Matchmaking\" data-bi-aN=\"citation\" data-bi-cN=\"Machine Learning for Optimal Matchmaking\">\n\t\t\t\tMachine Learning for Optimal Matchmaking&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"trueskill\">TrueSkill<\/h3>\n\n\n\n<p>The <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/trueskill-ranking-system\/\">TrueSkill ranking system<\/a> is a skill based ranking system for&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.xbox.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Xbox Live<\/a>&nbsp;developed at&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\">Microsoft Research<\/a>. The purpose of a ranking system is to both identify and track the skills of gamers in a game (mode) in order to be able to match them into competitive matches. TrueSkill has been used to rank and match players in many different games, from Halo 3 to Forza Motorsport 7.<\/p>\n\n\n\n<p>The classic TrueSkill ranking system only uses the final standings of all teams in a match in order to update the skill estimates (ranks) of all players in the match. The TrueSkill 2 ranking system also uses the individual scores of players in order to weight the contribution of each player to each team. As a result, TrueSkill 2 is much faster at figuring out the skill of a new player.<\/p>\n\n\n\n<p>An improved version of the TrueSkill ranking system, named&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/trueskill-2-improved-bayesian-skill-rating-system\/\" target=\"_blank\" rel=\"noreferrer noopener\">TrueSkill 2<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, launched with&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/gearsofwar.com\/en-us\/community\/news\/web-seasons\" target=\"_blank\" rel=\"noreferrer noopener\">Gears of War 4<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;and was later incorporated into&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.halowaypoint.com\/en-us\/news\/may-playlist-updates\" target=\"_blank\" rel=\"noreferrer noopener\">Halo 5<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-11 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Project<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/trueskill-ranking-system\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"TrueSkill\u2122 Ranking System\" data-bi-aN=\"citation\" data-bi-cN=\"TrueSkill\u2122 Ranking System\">\n\t\t\t\tTrueSkill\u2122 Ranking System&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/score-based-bayesian-skill-learning-2\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Score-based Bayesian Skill Learning - Microsoft Research\" data-bi-aN=\"citation\" data-bi-cN=\"Score-based Bayesian Skill Learning - Microsoft Research\">\n\t\t\t\tScore-based Bayesian Skill Learning &#8211; Microsoft Research&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/trueskilltm-a-bayesian-skill-rating-system-2\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"TrueSkill: A Bayesian Skill Rating System\" data-bi-aN=\"citation\" data-bi-cN=\"TrueSkill: A Bayesian Skill Rating System\">\n\t\t\t\tTrueSkill: A Bayesian Skill Rating System&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/trueskill-2-improved-bayesian-skill-rating-system\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"TrueSkill 2: An improved Bayesian skill rating system\" data-bi-aN=\"citation\" data-bi-cN=\"TrueSkill 2: An improved Bayesian skill rating system\">\n\t\t\t\tTrueSkill 2: An improved Bayesian skill rating system&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"watch-for\">Watch For<\/h3>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/garage\/wall-of-fame\/watch-for\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Watch For - Microsoft Garage\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Watch For - Microsoft Garage\">\n\t\t\t\tWatch For &#8211; Microsoft Garage&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>Developed by Microsoft researchers as part of a 2017 internal company hackathon, Watch For identifies and surfaces the most interesting parts of a livestream in real time. It recognizes text, symbols, objects, players and other elements that signal exciting moments in a game, and uses AI to learn over time as more streams and content are fed into the model. After winning the hackathon, the researchers worked with product groups to integrate the technology into HypeZone, which was part of the company\u2019s Mixer game streaming service. The technology was subsequently incorporated into MSN\u2019s Esports hub, and was also used by the Cloud9 professional esports organization to derive insights to support match preparation. Based on customer feedback and the team\u2019s combined AI expertise and end-to-end distributed systems knowledge, the Watch For team is continuing their exploration in this area, and working on tools for enabling non-experts to efficiently extract video data.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>From a research point of view, games offer an amazing environment in which to develop new machine learning algorithms and techniques. And we hope, in due course, that those new algorithms will feed back not just into gaming, but into many other domains. Beyond the very technical machine learning techniques themselves, gaming is an environment [&hellip;]<\/p>\n","protected":false},"author":38004,"featured_media":720721,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[244017],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[244005,243996,243993],"msr-podcast-series":[],"class_list":["post-720673","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-collection","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-promo-type-newsletter","msr-promo-type-podcast","msr-promo-type-webinar"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144923,237595,283244,395930,583324,663258,863034],"related-projects":[635574,707674,669597,607743,589879,577638,546345,442191,604002,359810,235753,171462,169873],"related-events":[632643],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-960x540.jpg\" class=\"img-object-cover\" alt=\"Suphx - Mahjong board\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-16x9.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/01\/Collection_Gaming_Suphx-Mahjong.jpg 1400w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","byline":"","formattedDate":"February 5, 2021","formattedExcerpt":"From a research point of view, games offer an amazing environment in which to develop new machine learning algorithms and techniques. And we hope, in due course, that those new algorithms will feed back not just into gaming, but into many other domains. Beyond the&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/720673"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/38004"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=720673"}],"version-history":[{"count":34,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/720673\/revisions"}],"predecessor-version":[{"id":924948,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/720673\/revisions\/924948"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/720721"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=720673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=720673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=720673"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=720673"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=720673"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=720673"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=720673"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=720673"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=720673"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=720673"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=720673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}