{"id":1162537,"date":"2026-03-03T10:05:09","date_gmt":"2026-03-03T18:05:09","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-video&#038;p=1162537"},"modified":"2026-03-03T10:05:10","modified_gmt":"2026-03-03T18:05:10","slug":"magentic-marketplace-testing-societies-of-agents-at-scale","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/magentic-marketplace-testing-societies-of-agents-at-scale\/","title":{"rendered":"Magentic Marketplace: Testing societies of agents at scale"},"content":{"rendered":"\n<div class=\"wp-block-group is-layout-constrained wp-block-group-is-layout-constrained\">\n<p>As AI agents move from isolated tools to active participants in multi-agent ecosystems, their success depends on more than task competence\u2014it requires strategic behavior under misaligned incentives and imperfect information. Using Magentic Marketplace, an open-source simulation of two-sided agent markets, we show that while frontier models can achieve strong welfare outcomes in ideal settings, performance degrades at scale and reveals emergent failure modes such as manipulation and speed bias, motivating a shift toward training agents for social reasoning.<\/p>\n\n\n\n<h2 class=\"wp-block-heading h5\" id=\"explore-more\">Explore more<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/magentic-marketplace-an-open-source-environment-for-studying-agentic-markets\/\">Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets<\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/microsoft\/multi-agent-marketplace\" target=\"_blank\" rel=\"noopener noreferrer\">Magentic Marketplace on GitHub<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/magentic-marketplace-an-open-source-simulation-environment-for-studying-agentic-markets\/\">Magentic Marketplace: an open-source simulation environment for studying agentic markets<\/a><\/li>\n<\/ul>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-cta\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/microsoft-research-forum\/past-episodes\/\">All Research Forum sessions<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-cta\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"http:\/\/aka.ms\/researchforum-register\" target=\"_blank\" rel=\"noreferrer noopener\">Register for the series<\/a><\/div>\n<\/div>\n\n\n<div class=\"wp-block-msr-show-more\">\n\t<div class=\"bg-neutral-100 p-5\">\n\t\t<div class=\"show-more-show-less\">\n\t\t\t<div>\n\t\t\t\t<span>\n\t\t\t\t\t\n\n<h2 class=\"wp-block-heading h3\" id=\"transcript\">Transcript<\/h2>\n\n\n\n<p><strong>Magentic Marketplace: Testing societies of agents at scale<\/strong><\/p>\n\n\n\n<p>As AI agents become more capable, they are no longer working alone. Agents start interacting with each other in shared environments. And when that happens, things can get interesting here in Redmond. Gagan Bansal from MSR, AI frontiers studies what happens when agents start transacting in a market-like settings using an open source environment called Magentic Marketplace.<\/p>\n\n\n\n<p>His team explores how agents behave when incentives don&#8217;t personally line up, and what kind of failure modes start to appear as systems scale.<\/p>\n\n\n\n<p>This is exactly the kind of early curiosity driven research that helps us understand what could go wrong before these systems light up. Take it away.<\/p>\n\n\n\n\t\t\t\t<\/span>\n\t\t\t\t<span id=\"show-more-show-less-toggle-1\" class=\"show-more-show-less-toggleable-content\">\n\t\t\t\t\t\n\n\n\n<p>Hi, I&#8217;m Gagan Bansal and I&#8217;m a researcher at Microsoft. And today I want to talk to you about our recent work on applying societies of agents in markets. And although I&#8217;m presenting, this work was a collaboration between many amazing colleagues across Microsoft.<\/p>\n\n\n\n<p>Capabilities of AI agents are improving rapidly. We&#8217;re quickly moving towards a future where each one of us will have personal agents. Now, in a world where everyone has agents, we believe societies of agents will drive new applications where our agents will have to interact with other agents. But how can we trust agents that we don&#8217;t control, or agents that might know things ours doesn&#8217;t, or even have competing goals?<\/p>\n\n\n\n<p>At Microsoft, we&#8217;ve been building on our expertise in multi-agent frameworks like AutoGen and Magentic one to create useful societies of agents, ones that add value, save time, and don&#8217;t cause harm. To enable this future, we need to understand how agents behave when they interact at scale. Recent examples from the open source community, where agents could talk freely on the forums, only underscores how timely and important this question is.<\/p>\n\n\n\n<p>Let me show you what we built and what we found. Imagine a marketplace where all the buying and selling is done by agents representing people. We call these settings as two sided agentic markets. This setting is a great testbed for society&#8217;s agents, because every agent has access to different information and competing incentives to systematically study two sided markets.<\/p>\n\n\n\n<p>We at Microsoft Research built a new simulation environment called Magentic Marketplace. Here, assistant agents represent customers, service agents represents businesses, and a marketplace sits in the middle handles search, communication and transaction between agents. Here&#8217;s a typical interaction.<\/p>\n\n\n\n<p>Suppose a customer wants to find a restaurant with something specific, like delicious empanadas and outdoor seating. Their assistant can search the marketplace. Talk to the service agents. Ask about menus, check amenities, and finally make a reservation. This framework allows us to test hundreds of agents buying and selling in parallel.<\/p>\n\n\n\n<p>We used it to systematically ask many research questions. Do these agents even add value for customers and businesses? Does the quality of search result impact their behavior? Are they vulnerable to any biases or manipulation? We built this framework as a general research tool. It can be used to ask many other questions, even for domains beyond markets.<\/p>\n\n\n\n<p>We started by asking whether agents actually add value for consumers. To find out, we implemented agents using frontier and open source models and computed the welfare that they achieve. Here, welfare is the value customers get from their purchase minus the price they paid. Higher is better. We observed that when agents have access to high quality search results, frontier models like GPT 5 and Sonnet 4 reached near optimal welfare.<\/p>\n\n\n\n<p>They talk to business agents, gathered missing information and made good choices. But we found that agent performance was tied to the quality of search results. When the search quality dropped, performance dropped. We also observed that there is still a large gap between the welfare achieved by frontier models and open source models.<\/p>\n\n\n\n<p>In addition to the impact of the quality of search results, we also want to test whether the number of search results impact welfare. So we gave agents more search results, varying them from 3 to 100, and expected the welfare to increase. But the opposite happened, resulting in a surprising paradox of choice.<\/p>\n\n\n\n<p>Welfare dropped for almost every model. This happened because agents didn&#8217;t explore enough and contacted few businesses. We also conducted experiments that tested whether the order of offers from service agents matters. It did dramatically. Almost 80 to 100% of the agents accepted the first proposal they received.<\/p>\n\n\n\n<p>They never even looked at the alternatives. Think about what this means for a real market. Speed beats quality. A business gains more from responding fast than from offering a better deal. That&#8217;s not a healthy dynamic. We also tested vulnerability to fake reviews, fake awards and prompt injection.<\/p>\n\n\n\n<p>Some frontier models resisted everything, but others were completely compromised. All payments were redirected to the attackers. These are early findings and markets are just the beginning. Societies of agents will emerge anywhere. Agents represent people with different interests such as supply chains, hiring and negotiation.<\/p>\n\n\n\n<p>Magnetic marketplace is open source and GitHub for the community to run experiments, stress test agents and help answer the harder questions. What guardrails do we need? How should markets be designed when both sides are AI? What we&#8217;ve shown is that simulation matters. Agents can add value, but they also inherit biases, fall for manipulation, and make choices that reward speed over quality.<\/p>\n\n\n\n<p>These are not edge cases. These are behaviors that only emerge when societies of agents are tested at scale. If these agents are going to take high stakes decisions on our behalf, such as making transactions to other agents, we should understand their behaviors and biases before deployment and not after.<\/p>\n\n\n\n<p>Please check out our papers and GitHub repository for more information. And thank you for attending the Microsoft Research Forum.<\/p>\n\n\t\t\t\t<\/span>\n\t\t\t<\/div>\n\t\t\t<button\n\t\t\t\tclass=\"action-trigger glyph-prepend mt-2 mb-0 show-more-show-less-toggle\"\n\t\t\t\taria-expanded=\"false\"\n\t\t\t\tdata-show-less-text=\"Show less\"\n\t\t\t\ttype=\"button\"\n\t\t\t\taria-controls=\"show-more-show-less-toggle-1\"\n\t\t\t\taria-label=\"Show more content\"\n\t\t\t\tdata-alternate-aria-label=\"Show less content\">\n\t\t\t\tShow more\t\t\t<\/button>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>As AI agents move from isolated tools to active participants in multi-agent ecosystems, their success depends on more than task competence\u2014it requires strategic behavior under misaligned incentives and imperfect information. Using Magentic Marketplace, an open-source simulation of two-sided agent markets, we show that while frontier models can achieve strong welfare outcomes in ideal settings, performance [&hellip;]<\/p>\n","protected":false},"featured_media":1163140,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_hide_image_in_river":0,"footnotes":""},"research-area":[13556],"msr-video-type":[268311],"msr-locale":[268875],"msr-post-option":[269148,269142],"msr-session-type":[256174],"msr-impact-theme":[],"msr-pillar":[],"msr-episode":[270235],"msr-research-theme":[270112],"class_list":["post-1162537","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-video-type-microsoft-research-forum","msr-locale-en_us","msr-post-option-approved-for-river","msr-post-option-include-in-river"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/Z7Sld00QLnU","msr_secondary_video_url":"","msr_video_file":"http:\/\/0","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/1162537","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":8,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/1162537\/revisions"}],"predecessor-version":[{"id":1163142,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/1162537\/revisions\/1163142"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1163140"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1162537"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1162537"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=1162537"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1162537"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1162537"},{"taxonomy":"msr-session-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-session-type?post=1162537"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1162537"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1162537"},{"taxonomy":"msr-episode","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-episode?post=1162537"},{"taxonomy":"msr-research-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-theme?post=1162537"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}