{"id":954777,"date":"2023-07-20T06:00:00","date_gmt":"2023-07-20T13:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=954777"},"modified":"2023-10-25T12:02:06","modified_gmt":"2023-10-25T19:02:06","slug":"collaborators-gaming-ai-with-haiyan-zhang","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/collaborators-gaming-ai-with-haiyan-zhang\/","title":{"rendered":"Collaborators: Gaming AI with Haiyan Zhang"},"content":{"rendered":"\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1400\" height=\"788\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1.jpg\" alt=\"black and white photos of Haiyan Zhang, General Manager of Gaming AI at Microsoft Gaming, next to the Microsoft Research Podcast \"R\" logo\" class=\"wp-image-954783\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1.jpg 1400w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-1280x720.jpg 1280w\" sizes=\"(max-width: 1400px) 100vw, 1400px\" \/><\/figure>\n\n\n<div class=\"wp-block-msr-podcast-container my-4\">\n\t<iframe loading=\"lazy\" src=\"https:\/\/player.blubrry.com\/?podcast_id=108559607&modern=1\" class=\"podcast-player\" frameborder=\"0\" height=\"164px\" width=\"100%\" scrolling=\"no\" title=\"Podcast Player\"><\/iframe>\n<\/div>\n\n\n\n<p>Transforming research ideas into meaningful impact is no small feat. It often requires the knowledge and experience of individuals from across disciplines and institutions.&nbsp;<em>Collaborators<\/em>, a new Microsoft Research Podcast series, explores the relationships\u2014both expected and unexpected\u2014behind the projects, products, and services being pursued and delivered by researchers at Microsoft and the diverse range of people they\u2019re teaming up with.<em> <\/em><\/p>\n\n\n\n<p>In the world of gaming, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/hazhang\/\" target=\"_blank\" rel=\"noreferrer noopener\">Haiyan Zhang<\/a> has situated herself where research meets real-world challenges, helping to bring product teams and researchers together to elevate the player experience with the latest AI advances even before the job became official with the creation of her current role, General Manager of Gaming AI. In this episode, she talks with host Dr. Gretchen Huizinga about the variety of expertise needed to avoid the discomfort experienced by players when they encounter a humanlike character displaying <em>inhuman<\/em> behavior, the potential for generative AI to make gaming better for both players and creators, and the games she grew up playing and what she plays now.<\/p>\n\n\n\n<div class=\"wp-block-group msr-pattern-link-list is-layout-flow wp-block-group-is-layout-flow\">\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading h5\" id=\"learn-more-1\">Learn more:<\/h2>\n\n\n\n<ul class=\"list-unstyled wp-block-list\">\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/group\/game-intelligence\/\">Game Intelligence<\/a><br>Group page<\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-paidia\/\">Project Paidia<\/a><br>Project page<\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/trueskill-ranking-system\/\">TrueSkill\u2122 Ranking System<\/a><br>Project page<\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/truematch\/\">TrueMatch Matchmaking System<\/a><br>Project page<\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/grounded-conversational-characters\/\">Grounded Conversational Characters<\/a><br>Project page<\/li>\n<\/ul>\n<\/div>\n\n\n\n<section class=\"wp-block-msr-subscribe-to-podcast subscribe-to-podcast\">\n\t<div class=\"subscribe-to-podcast__inner border-top border-bottom border-width-2\">\n\t\t<h2 class=\"h5 subscribe-to-podcast__heading\">\n\t\t\tSubscribe to the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\">Microsoft Research Podcast<\/a>:\t\t<\/h2>\n\t\t<ul class=\"subscribe-to-podcast__list list-unstyled\">\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/itunes.apple.com\/us\/podcast\/microsoft-research-a-podcast\/id1318021537?mt=2\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"black\" viewBox=\"0 0 32 32\">  <path d=\"M7.12 0c-3.937-0.011-7.131 3.183-7.12 7.12v17.76c-0.011 3.937 3.183 7.131 7.12 7.12h17.76c3.937 0.011 7.131-3.183 7.12-7.12v-17.76c0.011-3.937-3.183-7.131-7.12-7.12zM15.817 3.421c3.115 0 5.932 1.204 8.079 3.453 1.631 1.693 2.547 3.489 3.016 5.855 0.161 0.787 0.161 2.932 0.009 3.817-0.5 2.817-2.041 5.339-4.317 7.063-0.812 0.615-2.797 1.683-3.115 1.683-0.12 0-0.129-0.12-0.077-0.615 0.099-0.792 0.192-0.953 0.64-1.141 0.713-0.296 1.932-1.167 2.677-1.911 1.301-1.303 2.229-2.932 2.677-4.719 0.281-1.1 0.244-3.543-0.063-4.672-0.969-3.595-3.907-6.385-7.5-7.136-1.041-0.213-2.943-0.213-4 0-3.636 0.751-6.647 3.683-7.563 7.371-0.245 1.004-0.245 3.448 0 4.448 0.609 2.443 2.188 4.681 4.255 6.015 0.407 0.271 0.896 0.547 1.1 0.631 0.447 0.192 0.547 0.355 0.629 1.14 0.052 0.485 0.041 0.62-0.072 0.62-0.073 0-0.62-0.235-1.199-0.511l-0.052-0.041c-3.297-1.62-5.407-4.364-6.177-8.016-0.187-0.943-0.224-3.187-0.036-4.052 0.479-2.323 1.396-4.135 2.921-5.739 2.199-2.319 5.027-3.543 8.172-3.543zM16 7.172c0.541 0.005 1.068 0.052 1.473 0.14 3.715 0.828 6.344 4.543 5.833 8.229-0.203 1.489-0.713 2.709-1.619 3.844-0.448 0.573-1.537 1.532-1.729 1.532-0.032 0-0.063-0.365-0.063-0.803v-0.808l0.552-0.661c2.093-2.505 1.943-6.005-0.339-8.296-0.885-0.896-1.912-1.423-3.235-1.661-0.853-0.161-1.031-0.161-1.927-0.011-1.364 0.219-2.417 0.744-3.355 1.672-2.291 2.271-2.443 5.791-0.348 8.296l0.552 0.661v0.813c0 0.448-0.037 0.807-0.084 0.807-0.036 0-0.349-0.213-0.683-0.479l-0.047-0.016c-1.109-0.885-2.088-2.453-2.495-3.995-0.244-0.932-0.244-2.697 0.011-3.625 0.672-2.505 2.521-4.448 5.079-5.359 0.547-0.193 1.509-0.297 2.416-0.281zM15.823 11.156c0.417 0 0.828 0.084 1.131 0.24 0.645 0.339 1.183 0.989 1.385 1.677 0.62 2.104-1.609 3.948-3.631 3.005h-0.015c-0.953-0.443-1.464-1.276-1.475-2.36 0-0.979 0.541-1.828 1.484-2.328 0.297-0.156 0.709-0.235 1.125-0.235zM15.812 17.464c1.319-0.005 2.271 0.463 2.625 1.291 0.265 0.62 0.167 2.573-0.292 5.735-0.307 2.208-0.479 2.765-0.905 3.141-0.589 0.52-1.417 0.667-2.209 0.385h-0.004c-0.953-0.344-1.157-0.808-1.553-3.527-0.452-3.161-0.552-5.115-0.285-5.735 0.348-0.823 1.296-1.285 2.624-1.291z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Apple Podcasts<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/subscribebyemail.com\/www.blubrry.com\/feeds\/microsoftresearch.xml\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M6.4 6a2.392 2.392 0 00-2.372 2.119L16 15.6l11.972-7.481A2.392 2.392 0 0025.6 6H6.4zM4 10.502V22.8a2.4 2.4 0 002.4 2.4h19.2a2.4 2.4 0 002.4-2.4V10.502l-11.365 7.102a1.2 1.2 0 01-1.27 0L4 10.502z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Email<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/subscribeonandroid.com\/www.blubrry.com\/feeds\/microsoftresearch.xml\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M12.414 4.02c-.062.012-.126.023-.18.06a.489.489 0 00-.12.675L13.149 6.3c-1.6.847-2.792 2.255-3.18 3.944h13.257c-.388-1.69-1.58-3.097-3.179-3.944l1.035-1.545a.489.489 0 00-.12-.675.492.492 0 00-.675.135l-1.14 1.68a7.423 7.423 0 00-2.55-.45c-.899 0-1.758.161-2.549.45l-1.14-1.68a.482.482 0 00-.494-.195zm1.545 3.824a.72.72 0 110 1.44.72.72 0 010-1.44zm5.278 0a.719.719 0 110 1.44.719.719 0 110-1.44zM8.44 11.204A1.44 1.44 0 007 12.644v6.718c0 .795.645 1.44 1.44 1.44.168 0 .33-.036.48-.09v-9.418a1.406 1.406 0 00-.48-.09zm1.44 0V21.76c0 .793.646 1.44 1.44 1.44h10.557c.793 0 1.44-.647 1.44-1.44V11.204H9.878zm14.876 0c-.169 0-.33.035-.48.09v9.418c.15.052.311.09.48.09a1.44 1.44 0 001.44-1.44v-6.719a1.44 1.44 0 00-1.44-1.44zM11.8 24.16v1.92a1.92 1.92 0 003.84 0v-1.92h-3.84zm5.759 0v1.92a1.92 1.92 0 003.84 0v-1.92h-3.84z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Android<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/open.spotify.com\/show\/4ndjUXyL0hH1FXHgwIiTWU\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M16 4C9.383 4 4 9.383 4 16s5.383 12 12 12 12-5.383 12-12S22.617 4 16 4zm5.08 17.394a.781.781 0 01-1.086.217c-1.29-.86-3.477-1.434-5.303-1.434-1.937.002-3.389.477-3.403.482a.782.782 0 11-.494-1.484c.068-.023 1.71-.56 3.897-.562 1.826 0 4.365.492 6.171 1.696.36.24.457.725.217 1.085zm1.56-3.202a.895.895 0 01-1.234.286c-2.338-1.457-4.742-1.766-6.812-1.747-2.338.02-4.207.466-4.239.476a.895.895 0 11-.488-1.723c.145-.041 2.01-.5 4.564-.521 2.329-.02 5.23.318 7.923 1.995.419.26.547.814.286 1.234zm1.556-3.745a1.043 1.043 0 01-1.428.371c-2.725-1.6-6.039-1.94-8.339-1.942h-.033c-2.781 0-4.923.489-4.944.494a1.044 1.044 0 01-.474-2.031c.096-.023 2.385-.55 5.418-.55h.036c2.558.004 6.264.393 9.393 2.23.497.292.663.931.371 1.428z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Spotify<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/www.blubrry.com\/feeds\/microsoftresearch.xml\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M6.667 4a2.676 2.676 0 00-2.612 2.13v.003c-.036.172-.055.35-.055.534v18.666c0 .183.019.362.055.534v.003a2.676 2.676 0 002.076 2.075h.002c.172.036.35.055.534.055h18.666A2.676 2.676 0 0028 25.333V6.667a2.676 2.676 0 00-2.13-2.612h-.003A2.623 2.623 0 0025.333 4H6.667zM8 8h1.333C17.42 8 24 14.58 24 22.667V24h-2.667v-1.333c0-6.618-5.382-12-12-12H8V8zm0 5.333h1.333c5.146 0 9.334 4.188 9.334 9.334V24H16v-1.333A6.674 6.674 0 009.333 16H8v-2.667zM10 20a2 2 0 11-.001 4.001A2 2 0 0110 20z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">RSS Feed<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t\t<\/ul>\n\t<\/div>\n<\/section>\n\n\n<div class=\"wp-block-msr-show-more\">\n\t<div class=\"bg-neutral-100 p-5\">\n\t\t<div class=\"show-more-show-less\" data-mount=\"show-more-show-less\">\n\t\t\t<div>\n\t\t\t\t<span>\n\t\t\t\t\t\n\n<h3 class=\"wp-block-heading\" id=\"transcript\">Transcript<\/h3>\n\n\n\n<p>[TEASER]<\/p>\n\n\n\n<p>[MUSIC PLAYS UNDER DIALOGUE]<\/p>\n\n\n\n<p><strong>HAIYAN ZHANG:<\/strong> And as animation rendering improves, we have the ability to get to a wholly life-like character. So how do we get there? You know, we&#8217;re working with animators. We want to bring in neuroscientists, game developers, user researchers. There&#8217;s a lot of great work happening across the games industry in machine learning research, looking at things like motion matching, helping to create different kinds of animations that are more human-like. And it is this bringing together of artistry and technology development that&#8217;s going to get us there.&nbsp;<\/p>\n\n\n\n<p><strong>GRETCHEN HUIZINGA:<\/strong> You&#8217;re listening to <em>Collaborators<\/em>, a Microsoft Research Podcast showcasing the range of expertise that goes into transforming mind-blowing ideas into world-changing technologies. I&#8217;m Dr. Gretchen Huizinga.<\/p>\n\n\n\n<p>[MUSIC ENDS]<\/p>\n\n\n\n\t\t\t\t<\/span>\n\t\t\t\t<span id=\"show-more-show-less-toggle-1\" class=\"show-more-show-less-toggleable-content\">\n\t\t\t\t\t\n\n\n\n<p>I&#8217;m delighted to be back in the booth today with Haiyan Zhang. When we last talked, Haiyan was Director of Innovation at Microsoft Research Cambridge in the UK and Technical Advisor to Lab Director Chris Bishop. In 2020, she moved across the pond and into the role of Chief of Staff for Microsoft Gaming. Now she&#8217;s the General Manager of Gaming AI at Xbox, and I normally say, let&#8217;s meet our collaborators right now, but today, we&#8217;ll be exploring several aspects of collaboration with a woman who has collaborating in her DNA. Welcome back, Haiyan!&nbsp;<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Hi, Gretchen. It&#8217;s so good to be here. Thank you for having me on the show.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Listen, I left out a lot of things that you did before your research work in the UK, and I want you to talk about that in a minute, but first, help us understand your new role as GM of Gaming AI. What does that mean, and what do you do?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Right. Uh, so I started this role about a year ago, last summer, actually. And those familiar with Xbox will know that Xbox has been shipping AI for over a decade and in deep collaborations with Microsoft Research, with technologies like TrueSkill and TrueMatch, uh, that came out of the Cambridge lab in the UK. So we have a rich history of transferring machine learning research into applied product launch, and I think more recently, we know that AI is experiencing this step change in its capabilities, what it can empower people to do. And as we looked across all the various teams in Xbox working on machine learning models, looking at these new foundation models, this role was really to say, hey, let&#8217;s bring everybody together. Let&#8217;s bring it together and figure out what&#8217;s our strategy moving forward? Are there new connections for collaboration that we can form across teams from the platform services team to the game development teams? Are there new avenues that we should be exploring with AI to really accelerate how we make games and how we deliver those great game experiences to our players? So my role is really, let&#8217;s bring together that strategy for Xbox AI and then to look at new innovations, new incubation, that my team is spearheading, uh, in new areas for game development and our gaming platform.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Fascinating. Are you the first person to have this role? I mean, has there been a Gaming AI person before you?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Did you \u2026 did you get that from [LAUGHS] what I just \u2026 yeah, did you get that hint from what I just said?<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right!<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> The role didn&#8217;t exist before I came into the role last summer. And sometimes, you know, when you step into a role that didn&#8217;t exist before, a part of that is just to define what the role is by looking at what the organization needs. So you are totally right. This is a completely new role, and it is very timely because we are at that intersection where AI is really transformational. I mean, we\u2019ve seen machine learning gain traction with deep learning, uh, being applied in many more areas, and now with foundation models, with these large language models, we\u2019re just going to see this completely new set of capabilities emerge through technology that we want to be ready for in Xbox.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> You know, I&#8217;m going to say a phrase I hate myself for saying, but I want to double click on that for a minute. [LAUGHS] Um, AI for a decade or more in Xbox \u2026 how much did the new advances in large learning models and the GPT phenom help and influence what you&#8217;re doing?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Well, so interestingly, Gretchen, I, I&#8217;ve actually been secretly doing this role for several years before they&#8217;ve made it official. So back in Cambridge in the UK, with Microsoft Research, uh, I was working with our various AI teams that were working within gaming. So you might have talked before to Katja Hofmann&#8217;s team working on reinforcement learning.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Absolutely.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> The team working on Bayesian inference with TrueSkill and TrueMatch. So I was helping the whole lab think about, hey, how do we take these research topics, how do we apply them into gaming? Coming across the pond from the UK to Redmond, meeting with different folks such as Phil Spencer, the CEO of Microsoft Gaming, the leadership team of Xbox and really trying to champion, how do we get more, infuse more, AI into the Xbox ecosystem? Every year, I would work with colleagues in Xbox, and we would run an internal gaming and AI conference where we bring together the best minds of Microsoft Research and the product teams to really kind of meet in the middle and talk about, hey, here are some research topics; here are some product challenges. So I think for the last five or six years, I&#8217;ve, I\u2019ve been vying for this role to be created. And finally, it happened!<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Sort of you personify the role anyway, Haiyan. Was that conference kind of a hackathon kind of thing, or was it more formal and, um, bring minds together and \u2026 academic wise?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Right. We saw a space for both. That there needed to be more of a translation layer between, hey, here are some real product challenges we face in Xbox, both in terms of how we make the games and then how we build the networking platform that players join to access the games. So here are some real-world problems we face. And then to bring that together with, hey, here are some research topics that people might not have thought about applied to gaming. In recent times, we&#8217;ve looked at topics like imitation learning. You know, imitation learning can be applied to a number of different areas, but to apply that into video games, to say, hey, how can we take real player data and be able to try to develop AI agents that can play, uh, in the style of those human players, personalized for human players? You know, this is where I think the exciting work happens between problems in the real world and, uh, researchers that are looking at bigger research topics and are looking for those real-world problems.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right. To kind of match them together. Well, speaking of all the things you&#8217;ve done, um, you&#8217;ve had sort of a <em>Where in the World is Carmen Sandiego?<\/em> kind of career path, um, everything from software engineering and user experience to hardware R&D, design thinking, blue-sky envisioning. Some of that you&#8217;ve just referred to in that spectrum of, you know, real-world problems and research topics. But now you&#8217;re doing gaming. So talk a little bit about the why and how of your technology trek, personally. How have the things you did before informed what you&#8217;re doing now? Sort of broad thinking.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Thanks, Gretchen. You know, I&#8217;ve been very lucky to work across a number of roles that I&#8217;ve had deep passion for and, you know, very lucky to work with amazing colleagues, both inside of Microsoft and outside of Microsoft, so it&#8217;s definitely not something by plan, so it&#8217;s more that just following my own passions has led me down this path, for better or for worse. [LAUGHTER] Um, so I mean, my career starts in software engineering. So I worked in, uh, Windows application development, looking at data warehousing, developing software for biomedical applications, and I think there was a point where, you know, I, I really loved software architecture and writing code, and I, I wanted to get to the <em>why<\/em> and the <em>what<\/em> of what we build that would then lead to <em>how<\/em> we build it. So I wanted to get to a discipline where I could contribute to why and what we build.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> And that led me on a journey to really focus in on user experience and design, to say, hey, why do we build things? Why do we build a piece of software? Why do we build a tool? It&#8217;s really to aid people in whatever tasks that they&#8217;re doing. And so that user experience, that <em>why<\/em>, is, is going to be so important. Um, and so I went off and did a master&#8217;s in design in Italy and then pursued this intersection of user research, user experience, and technology.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> A master&#8217;s degree in Italy \u2026 That just sounds idyllic. [LAUGHS] So as you see those things building and following your passions and now you&#8217;re here, do you find that it has informed the way you see things in your current role, and, and how might that play out? Maybe even one example of how you see something you did before just found its way into, hey, this is what&#8217;s happening now; that, that fits, that connects.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> You know, I think in this role and also being a team leader and helping to empower a team of technologists and designers to do their best work in this innovation space, it&#8217;s kind of tough. You know, it \u2026 sometimes I find it&#8217;s really difficult to wear many hats at the same time. So when I&#8217;m looking at a problem \u2013 and although I have experience in writing software, doing user research, doing user experience design \u2013 it&#8217;s really hard to bring all of those things together into a singular conversation. So I\u2019m either looking at a problem purely through the coding, purely through the technology development, or purely through the user research. So I haven&#8217;t, I haven&#8217;t actually figured out a way to integrate all of those things. So when I was in the UK, when I was more working with a smaller team and, uh, really driving the innovation on my own, it was probably easier to, uh, to bring everything together, but then I&#8217;m only making singular things. So, for example, you know, in the UK, I developed a watch that helped a young woman called Emma to overcome some of the tremor symptoms she has from her Parkinson&#8217;s disease, and I developed some software that helped a young woman \u2013 her name is Aman \u2013 who had memory loss, and the software helped her be able to listen to her classes \u2013 she was in high school \u2013 to listen to her classes, record notes, be able to reflect back on her notes. So innovation really comes in many forms: at the individual level, at the group level, at the society level. And I find it just really valuable to gain experience across a spectrum of design and technology, and in order to really make change at scale, I think the work is really, hey, how do we empower a whole team? How do we empower a whole village to work on this together? And that is a, a very unique skill set that I&#8217;m still on a journey to really grasp and, and, and learn together with all my colleagues.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah. Well, let\u2019s talk about gaming more specifically now and what&#8217;s going on in your world, and I&#8217;ll, I\u2019ll start by saying that much of our public experience with AI began with games, where we&#8217;d see headlines like \u201cAI Beats the World Chess Champion\u201d or the World Go Champion or even, um, video games like Ms. Pac-Man. So in a sense, we&#8217;ve been conditioned to see AI as something sort of adversarial, and you&#8217;ve even said that, that it&#8217;s been a conceptual envisioning of it. But do you see a change in focus on the horizon, where we might see AI as less adversary and more collaborator, and what factors might facilitate that shift?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> I think we could have a whole conversation about all the different aspects of popular culture that has made artificial intelligence exciting and personified. So I can think of movies like <em>WarGames<\/em> or <em>Short Circuit<\/em>, just like really fun explorations into what might happen if a computer program gained some expanded intelligence. So, yes, we are \u2026 I think we are primed, and I think this speaks to some core of the human psyche that we love toying with these ideas of these personalities that we don&#8217;t totally understand. I mean, I think the same applies for alien movies. You know, we have an alien and it is completely foreign! It has an intelligence that we don&#8217;t understand. And sometimes we do project that these foreign personalities might be dangerous in some way. I also think that machine learning\/AI research comes in leaps and bounds by being able to define state-of-the-art benchmarks that everyone rallies around and is either trying to replicate or beat. So the establishment of human performance benchmarks, that we take a model and we say this model can perform this much better than the human benchmark, um, is a way for us to measure the progress of these models. So I think the establishment of these benchmarks has been really important to progress AI research. Now we see these foundation models being able to be generalized across many different tasks and performing well at many different tasks, and we are entering this phase where we transition from making the technology to building the tools. So the technology was all about benchmarks: how do we just get the technology there to do the thing that we need it to do? To now, how do we now mold the technology into tools that actually assist us in our daily lives? And I think this is where we see more HCI researchers, more designers, bringing their perspectives into the tool building, and we will see this transition from beating human benchmarks to assistive tools \u2013 <em>copilots<\/em> \u2013 that are going to aid us in work and in play.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah. Well, you have a rich history with gaming, both personally and professionally. Let me just ask, when did you start being a gamer?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Oh my goodness. Yeah, I actually prefer the term \u201cplayer\u201d because I feel like \u201cgamer\u201d has a lot of baggage in popular culture \u2026<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> I like that. It\u2019s probably true. When did you start gaming?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> I have very fond memories of playing <em>Minesweeper<\/em> on my dad&#8217;s university PC. And so it really started very early on for me. It&#8217;s funny. So I was an only child, so I spent a lot of time on my own. [LAUGHS]<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> I have one!<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> And one of the first hobbies I picked up was, uh, programing on our home PC, programing in BASIC.<\/p>\n\n\n\n<p><strong>HUIZINGA: <\/strong>Right \u2026<\/p>\n\n\n\n<p><strong>ZHANG: <\/strong>And, you know, it&#8217;s very \u2026 it&#8217;s a simple language. I think I was about 11 or 12, and I think the first programs that I wrote were trying to replicate music or trying to be simple games, because that&#8217;s a way for, you know, me as a kid, to see some exciting things happening on the screen.<\/p>\n\n\n\n<p><strong>HUIZINGA: <\/strong>Right \u2026<\/p>\n\n\n\n<p><strong>ZHANG: <\/strong>Um,so I&#8217;d say that that was when I was trying to make games with BASIC on a PC and also just playing these early games like <em>Decathlon<\/em> or <em>King\u2019s Quest<\/em>. So I&#8217;ve always loved gaming. I probably have more affinity to, you know, games of my, my childhood. But yeah, so started very early on, um, and then probably progressed I&#8217;d say I probably played a little too much <em>Minecraft<\/em> in university with some like many sleepless nights. That was probably not good. Um, and then I think has transitioned into mobile games like <em>Candy Crush<\/em>, just like quick sessions on a commute or something. And that&#8217;s why I prefer the term \u201cplayer\u201d because there\u2019s, there\u2019s billions of people in the world who play games, and I don&#8217;t think they think of themselves as <em>gamers<\/em>.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> But you know, if you&#8217;re playing <em>Solitaire<\/em>, <em>Candy Crush<\/em>, you are playing video games.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Well, that was a preface to the, to the second part of the question, which is something you said that really piqued my interest. That video games are an interactive art form expressed through human creativity. But let\u2019s get back to this gaming AI thing and, and talk a little bit about how you see creativity being augmented with AI as a collaborative tool. What&#8217;s up in that world?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Right. Yeah, I, I, I really fundamentally believe that, you know, video games are an experience. They&#8217;re a way for players to enter a whole new world, to have new experiences, to meet people in all walks of life and cultures that they&#8217;ve not met before; either they are fictional characters or other players in different parts of the world. So being able to help game creators express their creativity through technology is fundamental to what we do at Xbox. With AI, there\u2019s this just incredible potential for that assistance in creativity to be expanded and accelerated. For example, you know, in 1950, Claude Shannon, who was the inventor of information theory, wrote a paper about computer chess. And this is a seminal paper where he outlined what a modern computer chess program could be \u2013 before there were really wide proliferation of computers \u2013 and in it, he talked about that there were innate advantages that humans had that a computer program could never achieve. Things like, humans have imagination; humans have flexibility and adaptability. Humans can learn on the fly. And I think in the last seven decades, we&#8217;ve now seen that AI is exhibiting potential for imagination and flexibility. And imagination is something that generative AI is starting to demonstrate to us, but purely in terms of being able to express something that we ask of it. So I think everybody has this experience of, hey, I&#8217;d really like to create this. I&#8217;d really like to express this. But I can&#8217;t draw. [LAUGHTER] I want to take that photo, but why do my photos look so bad? Your camera is so much better than mine! And giving voice to that creativity is, I think, the strength of generative AI. So developing these tools that aid that creativity will be key to bringing along the next generation of game creators.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> So do you think we&#8217;re going to find ourselves in a place where we can accept a machine as a collaborator? I mean, I know that this is a theme, even in Microsoft Research, where we look at it as augmenting, not replacing, and collaborating, you know, not canceling. But there&#8217;s a shift for me in thinking of tools as collaborators. Do you feel like there&#8217;s a bridge that needs to be crossed for people to accept collaboration, or do they \u2026 or do you feel like it just is so much cooler what we can do with this tool that we&#8217;re all going to discover this is a great thing?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> I think, in many ways, we already use tools as collaborators, uh, whether that&#8217;s a hammer, whether that&#8217;s Microsoft Word. I mean, I love the spell-check feature! Oh, geez! Um, so we already use tools to help us do our work \u00ad&#8211; to make us work faster, type faster, um, check our grammar, check our spelling. This <em>is<\/em> the next step change in that these tools, the collaboration is going to be more complex, more sophisticated. I am somebody that welcomes that because I have so much work and need to get it done. At the same time, I really advocate for us, as a society, as our community, to have an open dialogue about it.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Because we should be talking about, what is it to be human, and how do we want to frame our work and our values moving forward given these new assistive tools? You know, when we talk about art, the assistance provided by generative AI will allow us to express, through words, what we want and then to have those words appear as an image on the page, and I think this will really challenge the art world to push artists beyond perhaps what we think of art today to new heights.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah. Yeah. There&#8217;s a whole host of questions and issues and, um, concerns even that we&#8217;re going to face. And I think it may be a, like you say, a step change in even how we conceptualize what art \u2026 I mean, even now, Instagram filters, I mean, or Photoshop or any of the things that you could say, well, that&#8217;s not really the photo you took. But, um, well, listen, we\u2019ve lived for a long time in an age of what I would call specialization or expertise, and we, we\u2019ve embraced that, you know, talk to the experts, appeal to the experts. But the current zeitgeist in research is multidisciplinary, bringing many voices into the conversation. And even in video games, it\u2019s multiplayer, multimodal. So talk for a minute about the importance of the prefix \u201cmulti\u201d now and how more voices are better for collaborative innovation.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> So I want to start with the word \u201cculture,\u201d because I fundamentally believe that when we have a team building an experience for our users and players, that team should reflect the diversity of those users and players, whether that&#8217;s diversity in terms of different abilities, in terms of cultural background. Teams that build these new experiences need to be inclusive, need to have many, many voices. So I want to start the \u201cmulti\u201d discussion there, and I encourage every team building products to think about that diversity and to move to recruit towards diversity. Then, let&#8217;s talk about the technology: multi-modality. So games are a very rich medium. They combine 3D, 2D, behaviors, interactions, music, sounds, dialogue. And it&#8217;s only when these different modalities come together and converge and, and work in perfect synchronicity that you get that amazing immersive experience. And we&#8217;ve seen foundation models do great things with text, do amazing things with 2D, some 3D now we&#8217;re seeing, and this is what I&#8217;m trying to push, that we need that multi-modality in generative AI, and multi-modality in terms of bringing these different mediums together. Multi-disciplinary, you know, what I find interesting is that, um, foundation models, LLMs like GPT, are at the height of democratized technology. Writing natural language as a prompt to generate something is probably the simplest form of programing.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Oh interesting.<\/p>\n\n\n\n<p><strong>ZHANG: <\/strong>It does not get simpler than I literally write what I want, and the thing appears.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Wow.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> So you can imagine that everybody in the world is going to be empowered with AI. And going from an idea, to defining that idea through natural language, to that idea becoming into reality, whether that&#8217;s it made an app for me, it made an image for me \u2026 so when this is happening, \u201cmultidisciplinary\u201d is going to be the natural outcome of this, that a designer\u2019s going to be able to make products. A coder is going to be able to make user experience. A user researcher is going to go from interviewing people to showing people prototypes with very little gap in order to further explore their topic. So I think we will get multidisciplinary because the technology will be democratized.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right. You know, as you talk, I\u2019m, I&#8217;m thinking \u201cprompts as programing\u201d is a fascinating \u2026 I never thought of it that way, but that&#8217;s exactly it. And you think about the layers of, um, barriers to entry in technology, that this has democratized those entry points for people who say I can never learn to code. Um, but if you can talk or type, you can! [LAUGHS] So that&#8217;s really cool. So we don&#8217;t have time to cover all the amazing scientists and collaborators working on specific projects in AI and gaming, but it would be cool if you&#8217;d give us a little survey course on some of the interesting, uh, research that&#8217;s being done in this area. Can you just name one or two or maybe three things that you think are really cool and interesting that are, um, happening in this world?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Well, I definitely want to give kudos to all of my amazing research collaborators working with large language models, with other machine learning approaches like reinforcement learning, imitation learning. You know, we know that in a video game, the dialogue and the story is key. And, you know, I work with an amazing research team with, uh, Bill Dolan, um, Sudha Rao, who are leading the way in natural language processing and looking at grounded conversational players in games. So how do we bring a large language model like GPT into a game? How do we make it fun? How do we actually tell a story? You know, as I said, technology is becoming democratized, so it&#8217;s going to be easy to put GPT into games, but ultimately, how do we make that into an experience that&#8217;s really valuable for our players? On the reinforcement learning\/imitation learning front, we&#8217;ve talked before about Project Paidia. How do we develop new kinds of AI agents that play games, that can help test games, that can really make games more fun by playing those games like real human players? That is our ultimate goal.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> You know, again, every time you talk, something comes into my head, and I&#8217;m thinking GPTs for NPCs. [LAUGHS]<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> I like that.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Um, and I have to say I&#8217;m, I\u2019m not a game player. I&#8217;m not the target market. But I did watch that movie <em>Free Guy<\/em> and got exposure to the non \u2026 what is it called?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Non-player characters.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> That&#8217;s the one. Ryan Reynolds plays that, and that was a really fun experience. Well, I want to come back to the multi-modality dimension of the technical aspects of gaming and AI\u2019s role in helping make animated characters more life-like. And that&#8217;s key for a lot of gaming companies is, how real does it feel? So what different skills and expertise go into creating a world where players can maintain their sense of immersion and avoid the <em>uncanny valley<\/em>? Who are you looking to collaborate with to make that happen?<\/p>\n\n\n\n<p><strong>ZHANG: <\/strong>Right. So the uncanny valley is this space where, when you are creating a virtual character and you bring together animation, whether it&#8217;s facial animation, body movements with sound, with their voice, with eye movement, with how they interact with the player, and the human brain has an ability to recognize, subconsciously, recognize another human being. And when you play with that deep-seated instinct and you create a virtual character, but the virtual character kind of slightly doesn&#8217;t move in the right way, their eyes don&#8217;t blink in the right way, they don&#8217;t talk in the right way, it, it triggers, in the deep part of someone&#8217;s brain, a discomfort. And this is what we call the <em>uncanny valley<\/em>. And there are many games that we know they&#8217;re stylized worlds and they&#8217;re fictional, so we try to get the characters to a place where the player knows that it&#8217;s not a real person, but they&#8217;re happy to be immersed in this environment. And as animation rendering improves, we have the ability to get to a wholly life-like character. So how do we get there? You know, we&#8217;re working with animators. We want to bring in neuroscientists, game developers, user researchers. There&#8217;s a lot of great work happening across the games industry in machine learning research, looking at things like motion matching, helping to create different kinds of animations that are more human-like. And it is this bringing together of artistry and technology development that&#8217;s going to get us there.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah. Yeah, yeah. Well, speaking of the uncanny valley \u2013 and I&#8217;ve experienced that on several animated movies where they \u2026 it\u2019s just like, eww, that&#8217;s weird \u2013 um, and we, we have a quest to avoid it. We, we want them to be more real. And I guess what we&#8217;re aiming for is to get to the point in gaming where the experience is so realistic that you can&#8217;t tell the difference between the character in the game and humans. So I have to ask, and assume you&#8217;ve given some thought to it, what could possibly go wrong if indeed you get everything right?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> So one thing is that video games are definitely a stylized visual art, so we also welcome game creators who want to create a completely cartoon universe, right?<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Oh, interesting, yeah.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> But for those creators that want to make that life-like visual experience, we want to have the technology ready. In terms of, hey, as you ask, what could possibly go wrong if indeed we got everything right, I think that throughout human history, we have a rich legacy of exploring new ideas and challenging topics through fiction. For example, the novels of Asimov, looking at robots and, and the laws of robotics, I believe that we can think of video games as another realm of fiction where we might be able to explore these ideas, where you can enter a world and say, hey, what if something went wrong when you have these life-like agents? It&#8217;s a safe place for players to be able to have those thoughts and experiences and to explore the different outcomes that might happen.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> I also want to say that I think the bar for immersion might also move over time. So when you say, what could possibly go wrong if we got everything right? Well, it&#8217;s still a video game.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> And it might look real, but it&#8217;s still in a game world. And I think once we experience that, the bar might shift. So for example, when the first black-and-white movies, uh, came out and people saw movies for the first time, I remember seeing a documentary where one of the first movies was somebody filming a train, a steam train, coming towards the camera, and the audience watching that jumped! They ran away because they thought there was a steam train coming at them. And I think since then, people have understood that that is not happening. But I think this bar of immersion that people have will move over time because ultimately it is not real. It is in a digital environment.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right. Well, and that bar finds itself in many different milieus, as it were. Um, you know, radio came out with <em>War of the Worlds<\/em>, and everyone thought we were being invaded by aliens, but we weren&#8217;t. It was just fiction. Um, and we&#8217;re also dealing with both satire and misinformation in non-game places, so it&#8217;s up to humans, it sounds like, to sort of make the differentiation and adapt, which we&#8217;ve done for a long time.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> I totally agree. And I think this podcast is a great starting point for us to have this conversation in society about topics like AGI, uh, topics like, hey, how is AI going to assist us and be copilots for us? We should be having more of that discussion.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> And even specifically to gaming I think one of the things that I hope people will come away with is that we&#8217;re thinking deeply about the experiences. We want them to be good experiences, but we also want people to not become, you know, fooled by it and so on. So these are big topics, and again, we won&#8217;t solve anything here, but I&#8217;m glad you&#8217;re thinking about it. So people have called gaming &#8212; and sometimes gamers &#8212; a lot of things, but rarely do you hear words like <em>welcoming<\/em>, <em>safe<\/em>, <em>friendly<\/em>, <em>diverse<\/em>, <em>fun<\/em>, <em>inviting<\/em>. And yet Xbox has worked really hard to kind of earn a reputation for inclusivity and accessibility and make gaming for everyone. Now I sound like I&#8217;m doing a commercial for Xbox, and I&#8217;m really not. I think this has been something that&#8217;s been foregrounded in the conversation in Xbox. So talk about some of the things you&#8217;ve done to, as you put it, work <em>with<\/em> the community rather than just <em>for<\/em> the community.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Right. I mean, we estimate there are 3 billion players in the world, so gaming has really become democratized. You can access it on your phone, on your computer, on your console, on your TV directly. And that means that we have to be thinking about, how do we make gaming for everybody? For every type of player? We believe that AI has this incredible ability to make gaming more fun for more people. You know, when we think about games, hey, how do we make gaming more adaptive, more personalized to every player? If I find this game is too hard for me, maybe the game can adapt to my needs. Or if I have different abilities or if I have disabilities that prohibits my access to the game, maybe the game can change to allow me to play with my friends. So this is an area that we are actively exploring. And, you know, even without AI, Xbox has this rich history of thinking about players with disabilities and how we can bring features to allow more people to play. So for example, recently <em>Forza <\/em>racing created a feature to allow people with sight impairment to be able to drive in the game by using sound. So they introduced new 3D sounds into the game, where someone who cannot see or can only partially see the screen can actually hear the hairpin turns in the road in order to drive their car and play alongside their family and friends.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> We&#8217;ve also done things like speech-to-text in Xbox Party Chat. How do we allow somebody who has a disability to be able to communicate across countries, across cultures, across languages with other players? And we are taking someone&#8217;s spoken voice, turning it into text chat so that everybody within that party can understand them and can be able to communicate with them.<\/p>\n\n\n\n<p><strong>HUIZINGA: <\/strong>Right. That&#8217;s interesting. So across countries, you could have people that don&#8217;t speak the same language be able to play together \u2026<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Right. Exactly.<\/p>\n\n\n\n<p><strong>HUIZINGA: <\/strong>&#8230;and communicate.<\/p>\n\n\n\n<p><strong>ZHANG: <\/strong>Yeah.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Wow. Well, one of the themes on this show is the path that technologies travel from mind to market, or lab to life, as I like to say. And we talk about the spectrum of work from blue-sky ideas to blockbuster products used by millions. But Xbox is no twinkle in anyone&#8217;s eye. It&#8217;s been around for a long time. Um, even so, there&#8217;s always new ideas that are making their way into established products and literally change the game, right? So without giving away any industry secrets, is there anything you can talk about that would give us hint as to what might be coming soon to a console or a headset or a device near us?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Oh my goodness. You know I can&#8217;t give away any product secrets!<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Dang it! I thought I would get you!<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Um, I mean, I&#8217;m excited by our ability to bring more life-like visuals, more life-like behavior, allowing players to play games anywhere at a fidelity they&#8217;ve not seen before. I mean, these are all exciting futures for AI. At the same time, generative AI capabilities to really accelerate and empower game developers to make games at higher quality, at a faster rate, these are the things that the industry wants. How do we turn these models, these technologies, into real tools for game developers?<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Yeah. I&#8217;m only thinking of players, and you&#8217;ve got this whole spectrum of potential participants.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> I think the players benefit when the creators are empowered. It starts with the creators and helping them bring to life their games that the players ultimately experience.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right. And even as you talk, I&#8217;m thinking there&#8217;s not a wide gap between creator and player. I mean, many of the creators are avid gamers themselves and \u2026<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> And now when we see games like <em>Minecraft<\/em> or <em>Roblox<\/em>, where the tools of creativity are being brought to the players themselves and they can create their own experiences \u2026<\/p>\n\n\n\n<p><strong>HUIZINGA: <\/strong>Yeah.<\/p>\n\n\n\n<p><strong>ZHANG: <\/strong>\u2026I want to see more of those in the world, as well.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Exciting. Well, as we close, and I hate to close with you because you&#8217;re so much fun, um, I&#8217;d like to give you a chance to give an elevator pitch for your preferred future. I keep using that term, but I know we all think in terms of what, what we&#8217;d like to make in this world to make a mark for the next generation. You&#8217;ve already referred to that in this podcast. So we can close the circle, and I&#8217;ll ask you to do a bit of blue-sky envisioning again, back to your roots in your old life. What do video games look like in the future, and how would you like to have changed the gaming landscape with brilliant human minds and AI collaborators?<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Oh my goodness. That&#8217;s such a tall order! Uh, I think ultimately my hope for the future is that we really explore our humanity and become even more human through the assistance of AI. And in gaming, how do we help people tell more human stories? How do we enable people to create stories, share those stories with others? Because ultimately, I think, since the dawn of human existence, we&#8217;ve been about storytelling. Sitting around a fire, drawing pictures on a cave wall, and now we are still there. How do we bring to life more stories? Because through these stories, we develop empathy for other people. We experience other lives, and that&#8217;s ultimately what&#8217;s going to make us better as a society, that empathy we develop, the mutual understanding and respect that we share. And I see AI as a tool to get us there.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> From cave wall to console \u2026 Haiyan Zhang, it&#8217;s always a pleasure to talk to you. Thank you for joining us today.<\/p>\n\n\n\n<p><strong>ZHANG:<\/strong> Thank you so, much, Gretchen. Thank you.<\/p>\n\n\t\t\t\t<\/span>\n\t\t\t<\/div>\n\t\t\t<button\n\t\t\t\tclass=\"action-trigger glyph-prepend mt-2 mb-0 show-more-show-less-toggle\"\n\t\t\t\taria-expanded=\"false\"\n\t\t\t\tdata-show-less-text=\"Show less\"\n\t\t\t\ttype=\"button\"\n\t\t\t\taria-controls=\"show-more-show-less-toggle-1\"\n\t\t\t\taria-label=\"Show more content\"\n\t\t\t\tdata-alternate-aria-label=\"Show less content\">\n\t\t\t\tShow more\t\t\t<\/button>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For over a decade, Xbox has been leveraging AI to elevate gaming. Haiyan Zhang, GM of Gaming AI, explores the collaborations behind the work and the potential for generative AI to support better experiences for both players and game creators.<\/p>\n","protected":false},"author":42183,"featured_media":954783,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"https:\/\/player.blubrry.com\/id\/108559607","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[240054],"tags":[],"research-area":[13556,13551,13552,13554,13553],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[268131],"class_list":["post-954777","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-msr-podcast","msr-research-area-artificial-intelligence","msr-research-area-graphics-and-multimedia","msr-research-area-hardware-devices","msr-research-area-human-computer-interaction","msr-research-area-medical-health-genomics","msr-locale-en_us","msr-podcast-series-collaborators"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"https:\/\/player.blubrry.com\/id\/108559607","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[583324],"related-projects":[880329,669597,604002,169873],"related-events":[],"related-researchers":[{"type":"guest","value":"gretchen-huizinga-2","user_id":"444834","display_name":"Gretchen Huizinga","author_link":"<a href=\"https:\/\/www.linkedin.com\/in\/gretchen-huizinga-phd-3a4b2921?trk=people-guest_people_search-card\" aria-label=\"Visit the profile page for Gretchen Huizinga\">Gretchen Huizinga<\/a>","is_active":true,"last_first":"Huizinga, Gretchen","people_section":0,"alias":"gretchen-huizinga-2"},{"type":"user_nicename","value":"Haiyan Zhang","user_id":31986,"display_name":"Haiyan Zhang","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/hazhang\/\" aria-label=\"Visit the profile page for Haiyan Zhang\">Haiyan Zhang<\/a>","is_active":false,"last_first":"Zhang, Haiyan","people_section":0,"alias":"hazhang"}],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-960x540.jpg\" class=\"img-object-cover\" alt=\"black and white photos of Haiyan Zhang, General Manager of Gaming AI at Microsoft Gaming, next to the Microsoft Research Podcast &quot;R&quot; logo\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Haiyan-CollabPodcast-HeroFeature-1400x788-1.jpg 1400w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","byline":"<a href=\"https:\/\/www.linkedin.com\/in\/gretchen-huizinga-phd-3a4b2921?trk=people-guest_people_search-card\" title=\"Go to researcher profile for Gretchen Huizinga\" aria-label=\"Go to researcher profile for Gretchen Huizinga\" data-bi-type=\"byline author\" data-bi-cN=\"Gretchen Huizinga\">Gretchen Huizinga<\/a> and <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/hazhang\/\" title=\"Go to researcher profile for Haiyan Zhang\" aria-label=\"Go to researcher profile for Haiyan Zhang\" data-bi-type=\"byline author\" data-bi-cN=\"Haiyan Zhang\">Haiyan Zhang<\/a>","formattedDate":"July 20, 2023","formattedExcerpt":"For over a decade, Xbox has been leveraging AI to elevate gaming. Haiyan Zhang, GM of Gaming AI, explores the collaborations behind the work and the potential for generative AI to support better experiences for both players and game creators.","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/954777"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/42183"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=954777"}],"version-history":[{"count":23,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/954777\/revisions"}],"predecessor-version":[{"id":970764,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/954777\/revisions\/970764"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/954783"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=954777"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=954777"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=954777"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=954777"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=954777"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=954777"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=954777"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=954777"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=954777"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=954777"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=954777"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}