{"id":708031,"date":"2020-11-30T10:36:41","date_gmt":"2020-11-30T18:36:41","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=708031"},"modified":"2023-03-03T11:06:51","modified_gmt":"2023-03-03T19:06:51","slug":"the-human-side-of-ai-for-chess","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/the-human-side-of-ai-for-chess\/","title":{"rendered":"The human side of AI for chess"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_AiChess_nologo.gif\" alt=\"Animated gif showing how MAIA interprets chess moves to predict the skill of the player.\"\/><\/figure>\n\n\n\n<p><em>Editor\u2019s note: The section \u201cModeling individual players\u2019 styles with Maia\u201d has been updated as of July 12, 2021.<\/em><\/p>\n\n\n\n<p>As artificial intelligence continues its rapid progress, equaling or surpassing human performance on benchmarks in an increasing range of tasks, researchers in the field are directing more effort to the interaction between humans and AI in domains where both are active. Chess stands as a model system for studying how people can collaborate with AI, or learn from AI, just as chess has served as a leading indicator of many central questions in AI throughout the field&#8217;s history.<\/p>\n\n\n\n<p>AI-powered chess engines have consistently bested human players since 2005, and the chess world has undergone further shifts since then, such as the introduction of the heuristics-based Stockfish engine in 2008 and the deep reinforcement learning-based AlphaZero engine in 2017. The impact of this evolution has been monumental: chess is now seeing record numbers of people playing the game even as AI itself continues to get better at playing. These shifts have created a unique testbed for studying the interactions between humans and AI: formidable AI chess-playing ability combined with a large, growing human interest in the game has resulted in a wide variety of playing styles and player skill levels.<\/p>\n\n\n\n<p>There\u2019s a lot of work out there that attempts to match AI chess play to varying human skill levels, but the result is often AI that makes decisions and plays moves differently than human players at that skill level. The goal for our research is to better bridge the gap between AI and human chess-playing abilities. The question for AI and its ability to learn is: can AI make the same fine-grained decisions that humans do at a specific skill level? This is a good starting point for aligning AI with human behavior in chess.<\/p>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--left\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">PUBLICATION<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/aligning-superhuman-ai-with-human-behavior-chess-as-a-model-system\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Aligning Superhuman AI with Human Behavior: Chess as a Model System\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Aligning Superhuman AI with Human Behavior: Chess as a Model System\">\n\t\t\t\tAligning Superhuman AI with Human Behavior: Chess as a Model System&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">PUBLICATION<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/learning-personalized-models-of-human-behavior-in-chess\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Learning Personalized Models of Human Behavior in Chess\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Learning Personalized Models of Human Behavior in Chess\">\n\t\t\t\tLearning Personalized Models of Human Behavior in Chess&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>Our team of researchers at the University of Toronto, Microsoft Research, and Cornell University has begun investigating how to better match AI to different human skill levels and, beyond that, personalize an AI model to a specific player\u2019s playing style. Our work comprises two papers, \u201c<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/aligning-superhuman-ai-with-human-behavior-chess-as-a-model-system\/\">Aligning Superhuman AI with Human Behavior: Chess as a Model System<\/a>\u201d and \u201c<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/learning-personalized-models-of-human-behavior-in-chess\/\">Learning Personalized Behaviors of Human Behavior in Chess<\/a>,\u201d as well as a novel chess engine, called Maia, which is trained on games played by humans to more closely match human play. Our results show that, in fact, human decisions at different levels of skill can be predicted by AI, even at the individual level. This represents a step forward in modeling human decisions in chess, opening new possibilities for collaboration and learning between humans and AI.<\/p>\n\n\n\n<div class=\"wp-block-group alignfull msr-chess-experience is-layout-flow wp-block-group-is-layout-flow\">\n<iframe style=\"border:none;\" src=\"https:\/\/maiachess.com\/maia-viz\/#\/msr-chess-experience\" seamless=\"\" width=\"100%\">\n<\/iframe>\n<\/div>\n\n\n\n<p>AlphaZero changed how AI played the game by practicing against itself with only knowledge of the rules (\u201cself-play\u201d), unlike previous models that relied heavily on libraries of moves and past games to inform training. Our model, Maia, is a customized version of Leela Chess Zero (an open-source implementation of AlphaZero). We trained Maia on human games with the goal of playing the most human-like moves, instead of being trained on self-play games with the goal of playing the optimal moves. In order to characterize human chess-playing at different skill levels, we developed a suite of nine Maias, one for each Elo rating between 1100 and 1900. (Elo ratings are a system for evaluating players\u2019 relative skill in games like chess.) As you\u2019ll see below, Maia matches human play more closely than any chess engine ever created.<\/p>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">CODE<\/span>\n\t\t\t<a href=\"https:\/\/github.com\/CSSLab\/maia-chess\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Maia Chess\" data-bi-aN=\"citation\" data-bi-cN=\"Maia Chess\">\n\t\t\t\tMaia Chess&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t\t\t<p class=\"annotations__caption text-neutral-400 mt-2\">Explore our nine final maia models saved as Leela Chess neural networks, and the code to create more and reproduce our results.<\/p>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>If you\u2019re curious, you can play against a few versions of Maia on Lichess, the popular open-source online chess platform. Our bots on Lichess are named maia1, maia5, and maia9, which we trained on human games at Elo rating 1100, 1500, and 1900, respectively. You can also download these bots and other resources from the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/github.com\/CSSLab\/maia-chess\">GitHub repo<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/p>\n\n\n\n<h2 id=\"measuring-human-play\">Measuring human play<\/h2>\n\n\n\n<p>What does it mean for a chess engine to match human play? For our purposes, we settled on a simple metric: given a position that occurred in an actual human game, what is the probability that the engine plays the move that the human played in the game?<\/p>\n\n\n\n<p>Making an engine that matches human play according to this definition is a difficult task. The vast majority of positions seen in real games only happen once, because the sheer number of possible positions is astronomical: after just four moves by each player, the number of potential positions enters the hundreds of billions. Moreover, people have a wide variety of styles, even at the same rough skill level. And even the same exact person might make a different move if they see the same position twice!<\/p>\n\n\n\n<h2 id=\"creating-a-dataset\">Creating a dataset<\/h2>\n\n\n\n<p>To rigorously compare engines in how well they match human play, we need a good test set to evaluate them with. We made a collection of nine test sets, one for each narrow rating range. Here\u2019s how we made them:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>First, we made rating bins for each range of 100 rating points (such as 1200-1299, 1300-1399, and so on).<\/li>\n\n\n\n<li>In each bin, we put all games where both players are in the same rating range.<\/li>\n\n\n\n<li>We drew 10,000 games from each bin, ignoring games played at Bullet and HyperBullet speeds. At those speeds (one minute or less per player), players tend to play lower quality moves to not lose by running out of time.<\/li>\n\n\n\n<li>Within each game, we discarded the first 10 moves made by each player to ignore most memorized opening moves.<\/li>\n\n\n\n<li>We also discarded any move where the player had less than 30 seconds to complete the rest of the game (to avoid situations where players are making random moves).<\/li>\n<\/ul>\n\n\n\n<p>After these restrictions we had nine test sets, one for each rating range, which contained roughly 500,000 positions each.<\/p>\n\n\n\n\n\t<div class=\"border-bottom border-top border-gray-300 mt-5 mb-5 msr-promo text-center text-md-left alignwide\" data-bi-aN=\"promo\" data-bi-id=\"670821\">\n\t\t\n\n\t\t<p class=\"msr-promo__label text-gray-800 text-center text-uppercase\">\n\t\t<span class=\"px-4 bg-white display-inline-block font-weight-semibold small\">Spotlight: Microsoft research newsletter<\/span>\n\t<\/p>\n\t\n\t<div class=\"row pt-3 pb-4 align-items-center\">\n\t\t\t\t\t\t<div class=\"msr-promo__media col-12 col-md-5\">\n\t\t\t\t<a class=\"bg-gray-300\" href=\"https:\/\/info.microsoft.com\/ww-landing-microsoft-research-newsletter.html\" aria-label=\"Microsoft Research Newsletter\" data-bi-cN=\"Microsoft Research Newsletter\" target=\"_blank\">\n\t\t\t\t\t<img decoding=\"async\" class=\"w-100 display-block\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/09\/Newsletter_Banner_08_2019_v1_1920x1080.png\" alt=\"\" \/>\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"msr-promo__content p-3 px-5 col-12 col-md\">\n\n\t\t\t\t\t\t\t\t\t<h2 class=\"h4\">Microsoft Research Newsletter<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<p class=\"large\">Stay connected to the research community at Microsoft.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"wp-block-buttons justify-content-center justify-content-md-start\">\n\t\t\t\t\t<div class=\"wp-block-button is-style-fill-chevron\">\n\t\t\t\t\t\t<a href=\"https:\/\/info.microsoft.com\/ww-landing-microsoft-research-newsletter.html\" class=\"btn btn-brand glyph-append glyph-append-chevron-right\" aria-label=\"Microsoft Research Newsletter\" data-bi-cN=\"Microsoft Research Newsletter\" target=\"_blank\">\n\t\t\t\t\t\t\tSubscribe today\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div><!--\/.msr-promo__content-->\n\t<\/div><!--\/.msr-promo__inner-wrap-->\n\t<\/div><!--\/.msr-promo-->\n\t\n\n\n\n<h2 id=\"differentiating-our-work-from-prior-attempts\">Differentiating our work from prior attempts<\/h2>\n\n\n\n<p>People have been trying to create chess engines that accurately match human play for decades. For one thing, they would make great sparring partners. But getting crushed like a bug every single game isn\u2019t that fun, so the most popular attempts at engines that match human play have been some kind of <em>attenuated <\/em>version of a strong chess engine. Attenuated versions of an engine are created by limiting the engine\u2019s ability in some way, such as reducing the amount of data it&#8217;s trained on or limiting how deeply it searches to find a move. For example, the \u201cplay with the computer\u201d feature on Lichess is a series of <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/stockfishchess.org\/\">Stockfish<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> models that are limited in the number of moves they are allowed to look ahead. Chess.com, ICC, FICS, and other platforms all have similar engines. How well do these engines match human play?<\/p>\n\n\n\n<p><strong>Stockfish: <\/strong>We created several attenuated versions of Stockfish, one for each depth limit (for example, the depth 3 Stockfish can only look 3 moves ahead), and then we tested them on our test sets. In the plot below, we break out the accuracies by rating level so you can see if the engine thinks more like players of a specific skill level.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"526\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure1_AIandChess_highrest-1024x526.jpg\" alt=\"Figure 1: Accuracy of Stockfish models with depth 1, 3, 5, 7, 9, 11, 13, and 15 shown form 1100 to 1900 Elo ratings. Depth 5 matching is the lowest accuracy, starting at under 35% at 1100 and rising to just above 35% for 1900 rating. The best move matching is at Depth 15, starting at roughly 36% at 1100 and rising to over 40% at 1900.\n \" class=\"wp-image-708598\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure1_AIandChess_highrest-1024x526.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure1_AIandChess_highrest-300x154.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure1_AIandChess_highrest-768x394.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure1_AIandChess_highrest-16x8.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure1_AIandChess_highrest.jpg 1509w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 1: Move matching accuracy for Stockfish compared with the targeted player\u2019s Elo rating<\/figcaption><\/figure>\n\n\n\n<p>As you can see, it doesn\u2019t work that well. Attenuated versions of Stockfish only match human moves about 35-40% of the time. And equally importantly, each curve is strictly increasing, meaning that even depth-1 Stockfish does a better job at matching 1900-rated human moves than it does at matching 1100-rated human moves. This means that attenuating Stockfish by restricting the depth it can search doesn\u2019t capture human play at lower skill levels\u2014instead, it looks like it\u2019s playing regular Stockfish chess with a lot of noise mixed in.<\/p>\n\n\n\n<p><strong>Leela Chess Zero<\/strong>: Attenuating Stockfish doesn\u2019t characterize human play at specific levels. What about <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/lczero.org\/\">Leela Chess Zero<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, an open-source implementation of AlphaZero, which learns chess through self-play games and deep reinforcement learning? Unlike Stockfish, Leela incorporates no human knowledge in its design. Despite this, however, the chess community was very excited by how Leela seemed to play more like human players.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-2_AIandChess_high-rest-1024x572.jpg\" alt=\"Figure 2: Leela ratings from 800 to 3200 graphed for accuracy. Leela does better than Stockfish for move matching, but as Elo rating gets better, each version of Leela has better or worse accuracy. Accuracy ranges from under 20% (800-rated Leela predicting 1900-level play) to about 47% (3200-rated Leela predicting 1900-level play).\" class=\"wp-image-708592\" width=\"938\" height=\"523\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-2_AIandChess_high-rest-1024x572.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-2_AIandChess_high-rest-300x168.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-2_AIandChess_high-rest-768x429.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-2_AIandChess_high-rest-16x9.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-2_AIandChess_high-rest.jpg 1503w\" sizes=\"(max-width: 938px) 100vw, 938px\" \/><figcaption class=\"wp-element-caption\">Figure 2: Move matching accuracy for Leela compared with the targeted player\u2019s Elo rating<\/figcaption><\/figure>\n\n\n\n<p>In the analysis above, we looked at a number of different Leela generations, with the ratings being their relative skill (commentators noted that early Leela generations played particularly similar to humans). People were right in that the best versions of Leela match human moves more often than Stockfish. But Leela still doesn\u2019t capture human play at different skill levels: each version is always getting better or always getting worse as the human skill level increases. To characterize human play at a particular level, we need another approach.<\/p>\n\n\n\n<h2 id=\"maia-a-better-solution-for-matching-human-skill-levels\">Maia: A better solution for matching human skill levels<\/h2>\n\n\n\n<p>Maia is an engine designed to play like humans at a particular skill level. To achieve this, we adapted the AlphaZero\/Leela Chess framework to learn from human games. We created nine different versions, one for each rating range from 1100-1199 to 1900-1999. We made nine training datasets in the same way that we made the test datasets (described above), with each training set containing 12 million games. We then trained a separate Maia model for each rating bin to create our nine Maias, from Maia 1100 to Maia 1900.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"571\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure3_AIandChess_high-res-1024x571.jpg\" alt=\"Figure 3: Maia trained models from 1100 to 1900 ratings. These are shown predicting player moves at 1100 to 1900 ratings. Maia\u2019s worst accuracy is 46% when a 1900-rated Maia model predicts moves of a 1100-rated player. The highest is 52%, far greater than prior AI chess models.\" class=\"wp-image-708595\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure3_AIandChess_high-res-1024x571.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure3_AIandChess_high-res-300x167.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure3_AIandChess_high-res-768x428.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure3_AIandChess_high-res-16x9.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure3_AIandChess_high-res.jpg 1506w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 3: Move matching accuracy for Maia compared with the targeted player\u2019s Elo rating<\/figcaption><\/figure>\n\n\n\n<p>As you can see, the Maia results are qualitatively different from Stockfish and Leela. First off, the move matching performance is much higher: Maia\u2019s lowest accuracy, when it is trained on 1900-rated players but predicts moves made by 1100-rated players, is 46%\u2014as high as the best performance achieved by any Stockfish or Leela model on any human skill level we tested. Maia\u2019s highest accuracy is over 52%. Over half the time, Maia 1900 predicts <em>the exact move<\/em> a 1900-rated human played in an actual game.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"945\" height=\"741\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/figure-4-AI-and-Chess-highres.jpg\" alt=\"Figure 4: Figures 1, 2, and 3 combined showing that Maia\u2019s accuracy greatly surpasses prior models\u2019 performance.\" class=\"wp-image-708610\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/figure-4-AI-and-Chess-highres.jpg 945w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/figure-4-AI-and-Chess-highres-300x235.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/figure-4-AI-and-Chess-highres-768x602.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/figure-4-AI-and-Chess-highres-16x12.jpg 16w\" sizes=\"(max-width: 945px) 100vw, 945px\" \/><figcaption class=\"wp-element-caption\">Figure 4: Move matching accuracy for all the models compared with the targeted player\u2019s Elo rating<\/figcaption><\/figure>\n\n\n\n<p>Importantly, every version of Maia uniquely captures a specific human skill level since every curve achieves its maximum accuracy at a different human rating. Even Maia 1100 achieves over 50% accuracy in predicting 1100-rated moves, and it\u2019s much better at predicting 1100-rated players than 1900-rated players!<\/p>\n\n\n\n<p>This means something deep about chess: there is such a thing as \u201c1100-rated style.\u201d And furthermore, it can be captured by a machine learning model. This was surprising to us: it would have been possible that human play is a mixture of good moves and random blunders, with 1100-rated players blundering more often and 1900-rated players blundering less often. Then it would have been impossible to capture 1100-rated style, because random blunders are impossible to predict. But since we <em>can <\/em>predict human play at different levels, there is a reliable, predictable, and maybe even algorithmically teachable difference between one human skill level and the next.<\/p>\n\n\n\n<h2 id=\"maias-predictions\">Maia\u2019s predictions<\/h2>\n\n\n\n<p>You can find all of the juicy details in the paper, but one of the most exciting things about Maia is that it can predict <em>mistakes<\/em>. Even when a human makes an absolute howler\u2014\u201changing\u201d a queen, in other words letting an opponent capture it for free, for example\u2014Maia predicts the exact mistake made more than 25% of the time. This could be really valuable for average players trying to improve their game: Maia could look at your games and tell which blunders were <em>predictable<\/em> and which were random mistakes. If your mistakes are predictable, you know what to work on to hit the next level.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"669\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-5-AI-and-Chess-high-res-1024x669.jpg\" alt=\"Figure 5: Matching accuracy (predicting move quality) of Maia versus Leela. Quality prediction is much more consistent and consistently higher across the full range of Maia models, at its height above 60%, when compared with Leela, which has a much broader range of accuracy when looking at the full range of models.\" class=\"wp-image-708613\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-5-AI-and-Chess-high-res-1024x669.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-5-AI-and-Chess-high-res-300x196.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-5-AI-and-Chess-high-res-768x502.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-5-AI-and-Chess-high-res-16x10.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/Figure-5-AI-and-Chess-high-res.jpg 1099w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 5: Move matching accuracy as a function of the quality of the move played in the game<\/figcaption><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 id=\"modeling-individual-players-styles-with-maia\">Modeling individual players\u2019 styles with Maia<\/h2>\n\n\n\n<p>In current work, we are pushing the modeling of human play to the next level: can we actually predict the moves a <em>particular human player would make?<\/em><\/p>\n\n\n\n<p>It turns out that personalizing Maia gives us our biggest performance gains. Whereas base Maia predicts human moves around 50% of the time, some personalized models can predict an individual\u2019s moves with accuracies up to 65%!<\/p>\n\n\n\n<p>We achieve these results by <em>fine-tuning<\/em> Maia. Starting with a base Maia, say Maia 1900, we update the model by continuing training on an individual player\u2019s games. We achieve these results by fine-tuning Maia. Starting with a base Maia, say Maia 1900, we update the model by continuing training on an individual player\u2019s games. Below, you can see that for predicting individual players\u2019 moves, most of the personalized models show significant improvement over the non-personalized models corresponding to the players\u2019 ratings. For players that have played at least 10,000 games, the personalized models achieve over 4% higher accuracy on average than the non-personalized models. For players with fewer than 10,000 games, the non-personalized models frequently perform better, indicating that personalization via our current methods requires substantial data to be effective.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"399\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/07\/Maia_updated-Figure-1024x399.png\" alt=\"Graph showing mean accuracy data for Project Maia.\" class=\"wp-image-762259\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/07\/Maia_updated-Figure-1024x399.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/07\/Maia_updated-Figure-300x117.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/07\/Maia_updated-Figure-768x299.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/07\/Maia_updated-Figure-240x93.png 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/07\/Maia_updated-Figure.png 1081w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>The personalized models are accurate enough that given a small number of games, we can tell which player played them! In this <em>stylometry <\/em>task\u2014where the goal is to recognize an individual\u2019s playing style\u2014we train personalized models for 400 players of varying skill levels, and then have each model predict the moves from 10 games by each player. For 86% of the 10-game sets we tested, the personalized model that achieved the highest accuracy (that is, predicted the player\u2019s actual moves most often) was the one that was trained on the player who played the games. The stylometry accuracy increases to 94% if we use 30-game sets, and 98% if we use 100-game sets. Thus, using a relatively small set of games, we can pick out who played the games from a set of 400 players. The personalized models are capturing individual chess-playing style in a highly accurate way.<\/p>\n\n\n\n<h2 id=\"using-ai-to-help-improve-human-chess-play\">Using AI to help improve human chess play<\/h2>\n\n\n\n<p>We designed Maia to be a chess engine that predicts human moves at a particular skill level, and it has progressed into a personalized engine that can identify the games of individual players. This is an exciting step forward in our understanding of human chess play, and it brings us closer to our goal of creating AI chess-teaching tools that help humans improve. Among the many capabilities of a good chess teacher, two of them are understanding how students at different skill levels play and recognizing the playing styles of their students. Maia has shown that these capabilities are realizable using AI.<\/p>\n\n\n\n<p>The ability to create personalized chess engines from publicly available, individual player data opens an interesting discussion on the possible uses (and misuses) of this technology. We initiate this discussion in our papers, but there is a long road ahead in understanding the full potential and implications of this line of research. As in countless times before, Chess will be one model AI system that sets the stage for this discussion.<\/p>\n\n\n\n<h3 id=\"acknowledgments\">Acknowledgments<\/h3>\n\n\n\n<p>Many thanks to <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/lichess.org\/\">Lichess.org<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> for providing the human games that we trained on, and hosting our Maia models that you can play against. Ashton Anderson was supported in part by an NSERC grant, a Microsoft Research gift, and a CFI grant. Jon Kleinberg was supported in part by a Simons Investigator Award, a Vannevar Bush Faculty Fellowship, a MURI grant, and a MacArthur Foundation grant.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Editor\u2019s note: The section \u201cModeling individual players\u2019 styles with Maia\u201d has been updated as of July 12, 2021. As artificial intelligence continues its rapid progress, equaling or surpassing human performance on benchmarks in an increasing range of tasks, researchers in the field are directing more effort to the interaction between humans and AI in domains [&hellip;]<\/p>\n","protected":false},"author":38838,"featured_media":708646,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[1],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-708031","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-artificial-intelligence","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[759193],"related-events":[],"related-researchers":[{"type":"guest","value":"reid-mcilroy-young","user_id":"708052","display_name":"Reid McIlroy-Young","author_link":"<a href=\"https:\/\/reidmcy.com\/\" aria-label=\"Visit the profile page for Reid McIlroy-Young\">Reid McIlroy-Young<\/a>","is_active":true,"last_first":"McIlroy-Young, Reid","people_section":0,"alias":"reid-mcilroy-young"},{"type":"guest","value":"ashton-anderson","user_id":"708091","display_name":"Ashton  Anderson","author_link":"<a href=\"http:\/\/www.cs.toronto.edu\/~ashton\/\" aria-label=\"Visit the profile page for Ashton  Anderson\">Ashton  Anderson<\/a>","is_active":true,"last_first":"Anderson, Ashton ","people_section":0,"alias":"ashton-anderson"},{"type":"guest","value":"jon-kleinberg","user_id":"708094","display_name":"Jon  Kleinberg","author_link":"<a href=\"http:\/\/www.cs.cornell.edu\/home\/kleinber\/\" aria-label=\"Visit the profile page for Jon  Kleinberg\">Jon  Kleinberg<\/a>","is_active":true,"last_first":"Kleinberg, Jon ","people_section":0,"alias":"jon-kleinberg"},{"type":"user_nicename","value":"Siddhartha Sen","user_id":33656,"display_name":"Siddhartha Sen","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/sidsen\/\" aria-label=\"Visit the profile page for Siddhartha Sen\">Siddhartha Sen<\/a>","is_active":false,"last_first":"Sen, Siddhartha","people_section":0,"alias":"sidsen"}],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-960x540.jpg\" class=\"img-object-cover\" alt=\"diagram\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-1536x864.jpg 1536w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-2048x1152.jpg 2048w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-16x9.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/1400x788_Ai_Chess_with_logo_still-1920x1080.jpg 1920w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","byline":"<a href=\"https:\/\/reidmcy.com\/\" title=\"Go to researcher profile for Reid McIlroy-Young\" aria-label=\"Go to researcher profile for Reid McIlroy-Young\" data-bi-type=\"byline author\" data-bi-cN=\"Reid McIlroy-Young\">Reid McIlroy-Young<\/a>, <a href=\"http:\/\/www.cs.toronto.edu\/~ashton\/\" title=\"Go to researcher profile for Ashton  Anderson\" aria-label=\"Go to researcher profile for Ashton  Anderson\" data-bi-type=\"byline author\" data-bi-cN=\"Ashton  Anderson\">Ashton  Anderson<\/a>, <a href=\"http:\/\/www.cs.cornell.edu\/home\/kleinber\/\" title=\"Go to researcher profile for Jon  Kleinberg\" aria-label=\"Go to researcher profile for Jon  Kleinberg\" data-bi-type=\"byline author\" data-bi-cN=\"Jon  Kleinberg\">Jon  Kleinberg<\/a>, and <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/sidsen\/\" title=\"Go to researcher profile for Siddhartha Sen\" aria-label=\"Go to researcher profile for Siddhartha Sen\" data-bi-type=\"byline author\" data-bi-cN=\"Siddhartha Sen\">Siddhartha Sen<\/a>","formattedDate":"November 30, 2020","formattedExcerpt":"Editor\u2019s note: The section \u201cModeling individual players\u2019 styles with Maia\u201d has been updated as of July 12, 2021. As artificial intelligence continues its rapid progress, equaling or surpassing human performance on benchmarks in an increasing range of tasks, researchers in the field are directing more&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/708031"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/38838"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=708031"}],"version-history":[{"count":24,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/708031\/revisions"}],"predecessor-version":[{"id":924705,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/708031\/revisions\/924705"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/708646"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=708031"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=708031"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=708031"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=708031"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=708031"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=708031"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=708031"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=708031"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=708031"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=708031"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=708031"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}