{"id":1057788,"date":"2024-07-18T06:00:00","date_gmt":"2024-07-18T13:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=1057788"},"modified":"2024-07-18T07:12:08","modified_gmt":"2024-07-18T14:12:08","slug":"abstracts-july-18-2024","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/abstracts-july-18-2024\/","title":{"rendered":"Abstracts: July 18, 2024"},"content":{"rendered":"\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1400\" height=\"788\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788.jpg\" alt=\"Microsoft Research Podcast - Abstracts\" class=\"wp-image-1057803\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788.jpg 1400w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-1280x720.jpg 1280w\" sizes=\"(max-width: 1400px) 100vw, 1400px\" \/><\/figure>\n\n\n<div class=\"wp-block-msr-podcast-container my-4\">\n\t<iframe loading=\"lazy\" src=\"https:\/\/player.blubrry.com\/?podcast_id=133091912&modern=1\" class=\"podcast-player\" frameborder=\"0\" height=\"164px\" width=\"100%\" scrolling=\"no\" title=\"Podcast Player\"><\/iframe>\n<\/div>\n\n\n\n<p>Members of the research community at Microsoft work continuously to advance their respective fields. <em>Abstracts<\/em> brings its audience to the cutting edge with them through short, compelling conversations about new and noteworthy achievements.<\/p>\n\n\n\n<p>In this episode, Senior Researcher <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/armitra\/\">Arindam Mitra<\/a> joins host Gretchen Huizinga to discuss \u201c<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/agentinstruct-toward-generative-teaching-with-agentic-flows\/\">AgentInstruct: Toward Generative Teaching with Agentic Flows<\/a>.\u201d In their paper, Mitra and his coauthors introduce\u00a0an automated multi-agent framework for creating diverse, high-quality synthetic data at scale for language model post-training. In contrast to methods that create data from a seed set of existing prompts and responses, AgentInstruct uses raw data and specifications provided by model builders. The work\u2014which post-trains a model, Orca-3, on AgentInstruct-generated data\u2014is part of project Orca. Orca aims to develop techniques for creating small language models that can perform as well as large language models. Like Orca-3, the earlier Orca, Orca-2, and Orca-Math models show the effectiveness of leveraging synthetic data in training.\u00a0<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/agentinstruct-toward-generative-teaching-with-agentic-flows\/\">Read the paper<\/a><\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-group msr-pattern-link-list is-layout-flow wp-block-group-is-layout-flow\">\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading h5\" id=\"learn-more-1\">Learn more:<\/h2>\n\n\n\n<ul class=\"list-unstyled wp-block-list\">\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4\/\">Orca: Progressive Learning from Complex Explanation Traces of GPT-4<\/a><br>Publication | June 2023<\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/orca-2-teaching-small-language-models-how-to-reason\/\">Orca-2: Teaching Small Language Models How to Reason<\/a><br>Publication | November 2023<\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/orca-math-unlocking-the-potential-of-slms-in-grade-school-math\/\">Orca-Math: Unlocking the potential of SLMs in Grade School Math<\/a><br>Publication | February 2024<\/li>\n<\/ul>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n<\/div>\n\n\n\n<section class=\"wp-block-msr-subscribe-to-podcast subscribe-to-podcast\">\n\t<div class=\"subscribe-to-podcast__inner border-top border-bottom border-width-2\">\n\t\t<h2 class=\"h5 subscribe-to-podcast__heading\">\n\t\t\tSubscribe to the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\">Microsoft Research Podcast<\/a>:\t\t<\/h2>\n\t\t<ul class=\"subscribe-to-podcast__list list-unstyled\">\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/itunes.apple.com\/us\/podcast\/microsoft-research-a-podcast\/id1318021537?mt=2\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"black\" viewBox=\"0 0 32 32\">  <path d=\"M7.12 0c-3.937-0.011-7.131 3.183-7.12 7.12v17.76c-0.011 3.937 3.183 7.131 7.12 7.12h17.76c3.937 0.011 7.131-3.183 7.12-7.12v-17.76c0.011-3.937-3.183-7.131-7.12-7.12zM15.817 3.421c3.115 0 5.932 1.204 8.079 3.453 1.631 1.693 2.547 3.489 3.016 5.855 0.161 0.787 0.161 2.932 0.009 3.817-0.5 2.817-2.041 5.339-4.317 7.063-0.812 0.615-2.797 1.683-3.115 1.683-0.12 0-0.129-0.12-0.077-0.615 0.099-0.792 0.192-0.953 0.64-1.141 0.713-0.296 1.932-1.167 2.677-1.911 1.301-1.303 2.229-2.932 2.677-4.719 0.281-1.1 0.244-3.543-0.063-4.672-0.969-3.595-3.907-6.385-7.5-7.136-1.041-0.213-2.943-0.213-4 0-3.636 0.751-6.647 3.683-7.563 7.371-0.245 1.004-0.245 3.448 0 4.448 0.609 2.443 2.188 4.681 4.255 6.015 0.407 0.271 0.896 0.547 1.1 0.631 0.447 0.192 0.547 0.355 0.629 1.14 0.052 0.485 0.041 0.62-0.072 0.62-0.073 0-0.62-0.235-1.199-0.511l-0.052-0.041c-3.297-1.62-5.407-4.364-6.177-8.016-0.187-0.943-0.224-3.187-0.036-4.052 0.479-2.323 1.396-4.135 2.921-5.739 2.199-2.319 5.027-3.543 8.172-3.543zM16 7.172c0.541 0.005 1.068 0.052 1.473 0.14 3.715 0.828 6.344 4.543 5.833 8.229-0.203 1.489-0.713 2.709-1.619 3.844-0.448 0.573-1.537 1.532-1.729 1.532-0.032 0-0.063-0.365-0.063-0.803v-0.808l0.552-0.661c2.093-2.505 1.943-6.005-0.339-8.296-0.885-0.896-1.912-1.423-3.235-1.661-0.853-0.161-1.031-0.161-1.927-0.011-1.364 0.219-2.417 0.744-3.355 1.672-2.291 2.271-2.443 5.791-0.348 8.296l0.552 0.661v0.813c0 0.448-0.037 0.807-0.084 0.807-0.036 0-0.349-0.213-0.683-0.479l-0.047-0.016c-1.109-0.885-2.088-2.453-2.495-3.995-0.244-0.932-0.244-2.697 0.011-3.625 0.672-2.505 2.521-4.448 5.079-5.359 0.547-0.193 1.509-0.297 2.416-0.281zM15.823 11.156c0.417 0 0.828 0.084 1.131 0.24 0.645 0.339 1.183 0.989 1.385 1.677 0.62 2.104-1.609 3.948-3.631 3.005h-0.015c-0.953-0.443-1.464-1.276-1.475-2.36 0-0.979 0.541-1.828 1.484-2.328 0.297-0.156 0.709-0.235 1.125-0.235zM15.812 17.464c1.319-0.005 2.271 0.463 2.625 1.291 0.265 0.62 0.167 2.573-0.292 5.735-0.307 2.208-0.479 2.765-0.905 3.141-0.589 0.52-1.417 0.667-2.209 0.385h-0.004c-0.953-0.344-1.157-0.808-1.553-3.527-0.452-3.161-0.552-5.115-0.285-5.735 0.348-0.823 1.296-1.285 2.624-1.291z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Apple Podcasts<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/subscribebyemail.com\/www.blubrry.com\/feeds\/microsoftresearch.xml\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M6.4 6a2.392 2.392 0 00-2.372 2.119L16 15.6l11.972-7.481A2.392 2.392 0 0025.6 6H6.4zM4 10.502V22.8a2.4 2.4 0 002.4 2.4h19.2a2.4 2.4 0 002.4-2.4V10.502l-11.365 7.102a1.2 1.2 0 01-1.27 0L4 10.502z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Email<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/subscribeonandroid.com\/www.blubrry.com\/feeds\/microsoftresearch.xml\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M12.414 4.02c-.062.012-.126.023-.18.06a.489.489 0 00-.12.675L13.149 6.3c-1.6.847-2.792 2.255-3.18 3.944h13.257c-.388-1.69-1.58-3.097-3.179-3.944l1.035-1.545a.489.489 0 00-.12-.675.492.492 0 00-.675.135l-1.14 1.68a7.423 7.423 0 00-2.55-.45c-.899 0-1.758.161-2.549.45l-1.14-1.68a.482.482 0 00-.494-.195zm1.545 3.824a.72.72 0 110 1.44.72.72 0 010-1.44zm5.278 0a.719.719 0 110 1.44.719.719 0 110-1.44zM8.44 11.204A1.44 1.44 0 007 12.644v6.718c0 .795.645 1.44 1.44 1.44.168 0 .33-.036.48-.09v-9.418a1.406 1.406 0 00-.48-.09zm1.44 0V21.76c0 .793.646 1.44 1.44 1.44h10.557c.793 0 1.44-.647 1.44-1.44V11.204H9.878zm14.876 0c-.169 0-.33.035-.48.09v9.418c.15.052.311.09.48.09a1.44 1.44 0 001.44-1.44v-6.719a1.44 1.44 0 00-1.44-1.44zM11.8 24.16v1.92a1.92 1.92 0 003.84 0v-1.92h-3.84zm5.759 0v1.92a1.92 1.92 0 003.84 0v-1.92h-3.84z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Android<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/open.spotify.com\/show\/4ndjUXyL0hH1FXHgwIiTWU\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M16 4C9.383 4 4 9.383 4 16s5.383 12 12 12 12-5.383 12-12S22.617 4 16 4zm5.08 17.394a.781.781 0 01-1.086.217c-1.29-.86-3.477-1.434-5.303-1.434-1.937.002-3.389.477-3.403.482a.782.782 0 11-.494-1.484c.068-.023 1.71-.56 3.897-.562 1.826 0 4.365.492 6.171 1.696.36.24.457.725.217 1.085zm1.56-3.202a.895.895 0 01-1.234.286c-2.338-1.457-4.742-1.766-6.812-1.747-2.338.02-4.207.466-4.239.476a.895.895 0 11-.488-1.723c.145-.041 2.01-.5 4.564-.521 2.329-.02 5.23.318 7.923 1.995.419.26.547.814.286 1.234zm1.556-3.745a1.043 1.043 0 01-1.428.371c-2.725-1.6-6.039-1.94-8.339-1.942h-.033c-2.781 0-4.923.489-4.944.494a1.044 1.044 0 01-.474-2.031c.096-.023 2.385-.55 5.418-.55h.036c2.558.004 6.264.393 9.393 2.23.497.292.663.931.371 1.428z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">Spotify<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\n\t\t\t\t\t\t\t<li class=\"subscribe-to-podcast__list-item\">\n\t\t\t\t\t<a class=\"subscribe-to-podcast__link\" href=\"https:\/\/www.blubrry.com\/feeds\/microsoftresearch.xml\" target=\"_blank\" rel=\"noreferrer noopener\">\n\t\t\t\t\t\t<svg class=\"subscribe-to-podcast__svg\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 32 32\"><path fill=\"currentColor\" d=\"M6.667 4a2.676 2.676 0 00-2.612 2.13v.003c-.036.172-.055.35-.055.534v18.666c0 .183.019.362.055.534v.003a2.676 2.676 0 002.076 2.075h.002c.172.036.35.055.534.055h18.666A2.676 2.676 0 0028 25.333V6.667a2.676 2.676 0 00-2.13-2.612h-.003A2.623 2.623 0 0025.333 4H6.667zM8 8h1.333C17.42 8 24 14.58 24 22.667V24h-2.667v-1.333c0-6.618-5.382-12-12-12H8V8zm0 5.333h1.333c5.146 0 9.334 4.188 9.334 9.334V24H16v-1.333A6.674 6.674 0 009.333 16H8v-2.667zM10 20a2 2 0 11-.001 4.001A2 2 0 0110 20z\"\/><\/svg>\n\t\t\t\t\t\t<span class=\"subscribe-to-podcast__link-text\">RSS Feed<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t\t<\/ul>\n\t<\/div>\n<\/section>\n\n\n<div class=\"wp-block-msr-show-more\">\n\t<div class=\"bg-neutral-100 p-5\">\n\t\t<div class=\"show-more-show-less\" data-mount=\"show-more-show-less\">\n\t\t\t<div>\n\t\t\t\t<span>\n\t\t\t\t\t\n\n<h2 class=\"wp-block-heading\" id=\"transcript\">Transcript<\/h2>\n\n\n\n<p>[MUSIC PLAYS]<\/p>\n\n\n\n<p><strong>GRETCHEN HUIZINGA:<\/strong> Welcome to <em>Abstracts<\/em>, a Microsoft Research Podcast that puts the spotlight on world-class research in brief. I\u2019m Dr. Gretchen Huizinga. In this series, members of the research community at Microsoft give us a quick snapshot\u2014or a <em>podcast abstract<\/em>\u2014of their new and noteworthy papers.<\/p>\n\n\n\n<p>[MUSIC FADES]<\/p>\n\n\n\n<p>I&#8217;m here today with Dr. Arindam Mitra, a senior researcher at Microsoft Research and the lead researcher for Microsoft&#8217;s Orca project. Dr. Mitra is coauthor of a paper called \u201cAgentInstruct: Toward Generative Teaching with Agentic Flows.\u201d Arindam, it&#8217;s a pleasure to have you on <em>Abstracts<\/em> today.<\/p>\n\n\n\n<p><strong>ARINDAM MITRA:<\/strong> Thank you, Gretchen.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> So let&#8217;s start with a brief overview of your paper. What problem does your research address, and why does it matter?<\/p>\n\n\n\n\t\t\t\t<\/span>\n\t\t\t\t<span id=\"show-more-show-less-toggle-1\" class=\"show-more-show-less-toggleable-content\">\n\t\t\t\t\t\n\n\n\n<p><strong>MITRA:<\/strong> So the post-training phase is very important for language models. You can really improve the model a lot by creating high-quality synthetic data. The problem is, however, though, high-quality synthetic data creation requires lots of human effort and expertise. The problem that we&#8217;re trying to tackle is, how do you reduce human effort? How can you create high-quality data with really low amount of human effort? When you have a language model and, let&#8217;s say, you want to apply it somewhere, you might have to train a generic model before. Which could be small or big. Doesn\u2019t matter. After that, you can specialize it on the domain that you are looking for, and when you want to do that\u2014to make it really fast, this particular process\u2014it&#8217;s best if you go for synthetic data. If you have a way to, actually, generate very high-quality synthetic data, you can fast-track this part of specialization process. Not only single model. So this year, you&#8217;re going to see a lot more multi-agent models. And when you are trying to build these multi-agent models, you&#8217;re fearing like, OK, it might increase the cost too much, the latency too much. So it&#8217;s also very much important that you have a multi-agent system and you can, sort of, replace some of those agents with specialized small models. And when you&#8217;re trying to address these goals, you want this process to be something which you know works <em>fast<\/em>. So that&#8217;s why we are trying to make sure we have a very good way to create synthetic data for your specific need.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> No research exists in a vacuum, and most of it fills some kind of a gap. So tell us what&#8217;s already been done in this field and how <em>this<\/em> work is building on it.<\/p>\n\n\n\n<p><strong>MITRA:<\/strong> So previously, actually, we have seen that in post-training, the more data you have, the better the performance goes for the model you&#8217;re training. So what we wanted to test is how much we can scale and what happens if we scale a lot and lot. But we didn&#8217;t have the tools for it. So the other approaches people previously used was you had a small set of data and how do we expand this dataset into much larger and larger amount of data. That&#8217;s where people were mostly focusing. But it&#8217;s not that easy to create that initial seed set. [LAUGHTER] You need to be very expert. The way that we&#8217;re doing is, actually, rather you define what you want to create. Like, OK, you want to create tool-use data. So you say, OK, I have a bunch of tools, and I am looking for data in the scenarios where someone can just come give me a description and then maybe that person interact with the AI to figure out how to get the job done. It&#8217;s not a one-step thing. And maybe you also have a setting where it&#8217;s more like an app developer. You have a bunch of APIs in your phone. You just want to figure out which one is best for the user request, which came through voice command. So different scenarios could be there. So what we&#8217;re saying [is], OK, we are not going through the method where you have to come up with your initial own seed data and then we expand. It is more like you define what you want to do. It&#8217;s much more abstract. And then, we are, sort of, automating the effort of data creation. So this setting actually of synthetic data creation, we are referring [to] it as <em>generative teaching<\/em>, and that&#8217;s where we are, sort of, differing. So previously, it was more like expansion, and now we are trying from specification to the data that you need.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Gotcha. Well talk a little bit more about your methodology and how you went about conducting this research.<\/p>\n\n\n\n<p><strong>MITRA:<\/strong> So first of all, what we are proposing actually is a multi-agent solution. So you start with first describing what you really need. So you describe in detail, like, I need data for this specific skill or this specific scenario. Then, what we do is like, OK, you have some unstructured data or raw data like text documents or code files that you gather from web with permissible license or use something that you own. We don&#8217;t care much about what the content is really. So it&#8217;s more like we got some random stuff, some random content. And then we&#8217;ll guide you how to convert this random something which is not meaningful for you into something which is meaningful for your data creation. For example, like, if you are creating data to teach how to use APIs, you might think about, you need lots of APIs and how do you get these APIs. So what we are saying is, like, we can take something like code and we&#8217;ll have agents which will convert these raw code files into list of APIs which is more like a library. So you create automatically this input that is very meaningful for data creation. And then once we have that, we have basically the seed instruction creation step based on your specification. Like, what do you want to create data for? So you have all these different scenarios, and we have multiple agents creating data for different scenarios. And then the last step is actually what we call refinement step. So it&#8217;s more like whatever data you created, we\u2019ll go through them and we\u2019ll make them better and better\u2014improve the quality, improve the complexity, improve the trickiness, we\u2019ll teach when not to answer, etc., etc. So make sure we cover the whole space. So by changing the stochastic seed, we are trying to cover the entire possible data space.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Right.<\/p>\n\n\n\n<p><strong>MITRA:<\/strong> So that&#8217;s the key thing. The way we, sort of, conducted this research is actually we defined 17 skills. Skills meaning reading comprehension, tool use, text modification, content creation, RAG (retrieval-augmented generation) &#8230; we have, like, list of 17 skills \u2026 conversation \u2026 and then we created one multi-agent flow for each of the skills and we generate data. So one key thing I want to highlight is, like, this work, compared to other work, it was not benchmark driven. We want to teach a skill. We don&#8217;t care which benchmarks we&#8217;re trying to evaluate it on. So we define the skill, like <em>tool use<\/em> means this to us, <em>reading comprehension<\/em> means this to us, <em>text modification<\/em> means this to us. And then we, sort of, generate the data to teach everything for that skill. And then what we did, we created actually 22 million instructions. And we had previously in Orca series, we had 3 million, around, instructions. So the 25 million is what we, sort of, have at the end. And that&#8217;s where we actually trained a Mistral model as of now. And we&#8217;re going to measure, like, how much we improve the Mistral model by this post-training.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Moving from methods to findings, I always look forward to the part of the research paper that finishes the sentence \u201cand what we found was \u2026 ,\u201d so give us a quick overview of your results. What did you find?<\/p>\n\n\n\n<p><strong>MITRA:<\/strong> Yes, so the results were actually very exciting for us. So Mistral 7B was our main, sort of, baseline because that&#8217;s where we\u2019re trying to showcase, like, how much improvement we are getting. On the other side, we have, like, frontier models\u2014ChatGPT, GPT-4. We want to also measure how far we are from those frontier models, so that&#8217;s, sort of, our evaluation setup. So on average actually, we got like 20 percent performance gain over the Mistral, and we evaluated that across 14 benchmarks that test reasoning, content creation, instruction following, format following, etc. But what was more important to us was to do a skill-specific evaluation because we are trying to teach certain skills, and we had, like, 17 skills as we mentioned earlier. So, for example, like, if you are focusing on reading comprehension as a skill, we took LSAT, SAT, and DROP, and many other benchmarks; we created a collection of reading comprehension-based benchmark. And there, we are observing, like, 20 percent improvement over Mistral, and what it means, like, we&#8217;re actually achieving GPT-4\u2013level performance. Similarly, if I&#8217;m focusing on math skill, there are many datasets which test, like, elementary math, high school math, college-level math. And we improved actually across all these different levels of math. So we see from 40 percent to 150 percent of improvement on different benchmarks of math. So it was more like what we wanted to see. We&#8217;re not optimizing for a particular benchmark. We wanted to optimize the skill, and that&#8217;s what you&#8217;re observing. So you&#8217;re observing improvement in math across all these levels, from elementary to high school to college to middle school, etc., everything. The same goes for RAG, as well. We\u2019re observing on RAG skill 92 percent, around, improvement over Mistral. The format following numbers are pretty interesting to us. So format following is very important for SLMs (small language models). You want to make these models practical. You want to make sure that they follow the format so you can parse the result. And we were able to take Mistral beyond Gemini Pro. So that was a very strong performance from the post-training that we did. For summarization, actually we were able to reduce the hallucination rate by 31 percent while achieving the GPT-4\u2013level quality. So overall, all these results were, sort of, highlighting that the methodology that we have, which we&#8217;re calling AgentInstruct, is very promising.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> I think it&#8217;s important to get practical and talk about real-world impact. So tell us who you think this research will benefit most and why.<\/p>\n\n\n\n<p><strong>MITRA:<\/strong> Yeah, so again the model builders will, sort of, find it most beneficial. So the significance of our work actually lies in the way we are trying to revolutionize the language model development through scalable, low-effort synthetic creation. And the scalable and low effort is, sort of, the key thing, right. We have shown that we can create very high-quality data. That&#8217;s what the numbers are telling us. We want to mention that this is very scalable and low effort, and that&#8217;s what we think might help the most for model builders.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> So, Arindam, let&#8217;s borrow a phrase from the machine learning lexicon and go for a little one-shot learning here: if you had to boil down why your work is important, what&#8217;s the one thing you want our listeners to take away from this research?<\/p>\n\n\n\n<p><strong>MITRA:<\/strong> The key takeaway would be, like, the AgentInstruct method enables the generation of vast, diverse, and high-quality synthetic data with very minimal human input. So that&#8217;s one thing I would, like, to remember from this paper.<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> So as we close, talk briefly about the limitations that you encountered in this project and directions for future research. What are the outstanding challenges in this field, and what&#8217;s on your research agenda to overcome them?<\/p>\n\n\n\n<p><strong>MITRA:<\/strong> Yes, so we&#8217;re exploring further automation. But apart from making this data creation more automated and less human involvement needed, we&#8217;re trying to focus on two other aspects. One is automated model debugging, and the other is automated model repairing. So now that we have the ability to generate data for a particular skill, let&#8217;s say math, for model debugging, what we need is basically an error handler. Like something we can plug in which takes the question and the answer coming from a different model and verifies if the answer is correct or not. So that&#8217;s the part we&#8217;re working on right now, figuring out this error handler. And the second aspect is repairing. So once we have the error, we figure out, OK, this is where the model is struggling. How can we give feedback or how can we give more knowledge so it can basically correct those errors? So those are some things we&#8217;re working on right now.<\/p>\n\n\n\n<p>[MUSIC PLAYS]<\/p>\n\n\n\n<p><strong>HUIZINGA:<\/strong> Well, Arindam Mitra, thanks for joining us today, and to our listeners, thanks for tuning in. If you want to read this paper, you can find a link at aka.ms\/abstracts, or you can find a preprint on arXiv. See you next time on <em>Abstracts<\/em>!<\/p>\n\n\n\n<p>[MUSIC FADES]<\/p>\n\n\t\t\t\t<\/span>\n\t\t\t<\/div>\n\t\t\t<button\n\t\t\t\tclass=\"action-trigger glyph-prepend mt-2 mb-0 show-more-show-less-toggle\"\n\t\t\t\taria-expanded=\"false\"\n\t\t\t\tdata-show-less-text=\"Show less\"\n\t\t\t\ttype=\"button\"\n\t\t\t\taria-controls=\"show-more-show-less-toggle-1\"\n\t\t\t\taria-label=\"Show more content\"\n\t\t\t\tdata-alternate-aria-label=\"Show less content\">\n\t\t\t\tShow more\t\t\t<\/button>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n\n<p><a id=\"_msocom_1\"><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Senior Researcher Arindam Mitra introduces\u00a0AgentInstruct. Using raw data sources, the automated multi-agent framework can create diverse, high-quality synthetic data at scale for the post-training of small and large language models.<\/p>\n","protected":false},"author":37583,"featured_media":1057803,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"https:\/\/player.blubrry.com\/id\/133091912","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[240054],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[243990],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[268128],"class_list":["post-1057788","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-msr-podcast","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-post-option-podcast-featured","msr-podcast-series-abstracts"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"https:\/\/player.blubrry.com\/id\/133091912","podcast_episode":"","msr_research_lab":[199565,992148],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[392600],"related-projects":[1003089,983295],"related-events":[],"related-researchers":[{"type":"guest","value":"gretchen-huizinga-2","user_id":"444834","display_name":"Gretchen Huizinga","author_link":"<a href=\"https:\/\/www.linkedin.com\/in\/gretchen-huizinga-phd-3a4b2921?trk=people-guest_people_search-card\" aria-label=\"Visit the profile page for Gretchen Huizinga\">Gretchen Huizinga<\/a>","is_active":true,"last_first":"Huizinga, Gretchen","people_section":0,"alias":"gretchen-huizinga-2"},{"type":"user_nicename","value":"Arindam Mitra","user_id":42978,"display_name":"Arindam Mitra","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/armitra\/\" aria-label=\"Visit the profile page for Arindam Mitra\">Arindam Mitra<\/a>","is_active":false,"last_first":"Mitra, Arindam","people_section":0,"alias":"armitra"}],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-960x540.jpg\" class=\"img-object-cover\" alt=\"Microsoft Research Podcast - Abstracts\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Episode-12_Abstracts_Hero_Feature_No_Text_1400x788.jpg 1400w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","byline":"<a href=\"https:\/\/www.linkedin.com\/in\/gretchen-huizinga-phd-3a4b2921?trk=people-guest_people_search-card\" title=\"Go to researcher profile for Gretchen Huizinga\" aria-label=\"Go to researcher profile for Gretchen Huizinga\" data-bi-type=\"byline author\" data-bi-cN=\"Gretchen Huizinga\">Gretchen Huizinga<\/a> and <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/armitra\/\" title=\"Go to researcher profile for Arindam Mitra\" aria-label=\"Go to researcher profile for Arindam Mitra\" data-bi-type=\"byline author\" data-bi-cN=\"Arindam Mitra\">Arindam Mitra<\/a>","formattedDate":"July 18, 2024","formattedExcerpt":"Senior Researcher Arindam Mitra introduces\u00a0AgentInstruct. Using raw data sources, the automated multi-agent framework can create diverse, high-quality synthetic data at scale for the post-training of small and large language models.","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1057788"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/37583"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=1057788"}],"version-history":[{"count":22,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1057788\/revisions"}],"predecessor-version":[{"id":1058259,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1057788\/revisions\/1058259"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1057803"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1057788"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=1057788"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=1057788"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1057788"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=1057788"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=1057788"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1057788"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1057788"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1057788"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=1057788"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=1057788"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}