{"id":983295,"date":"2023-11-20T18:02:56","date_gmt":"2023-11-21T02:02:56","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=983295"},"modified":"2024-07-31T10:51:04","modified_gmt":"2024-07-31T17:51:04","slug":"orca","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/orca\/","title":{"rendered":"Orca"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background- card-background--full-bleed\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"720\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720.jpg\" class=\"attachment-full size-full\" alt=\"ORCA project header : AI-generated whale graphic over an abstract background of data waves\" style=\"object-position: 80% 47%\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720.jpg 1920w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720-300x113.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720-1024x384.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720-768x288.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720-1536x576.jpg 1536w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720-1600x600.jpg 1600w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/11\/Orca-project_header_1920x720-240x90.jpg 240w\" sizes=\"(max-width: 1920px) 100vw, 1920px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 \">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading\" id=\"orca\">Orca<\/h1>\n\n\n\n<p><strong>Using AI, to improve AI <\/strong><\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p>Orca is a research team in Microsoft Research. Orca focuses on creating automated pipelines for creating high-quality synthetic data at scale, and training\u00a0models for specialization and model self-improvement. Orca\u2019s research areas involve self-improvement strategies, feedback-driven teaching methods between large and small models to create high-quality synthetic data and using domain specific data to specialize LMs.<\/p>\n\n\n\n<p>Orca focuses on the following directions:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Automated pipelines for generating diverse and high-quality data at scale,<\/li>\n\n\n\n<li>Training algorithms for model specialization and continual improvement,<\/li>\n\n\n\n<li>Building a general pipeline for finetuning-as-a-service (automating data generation and learning for any domain).<\/li>\n<\/ul>\n\n\n\n<div style=\"padding-bottom:0; padding-top:0\" class=\"wp-block-msr-immersive-section alignfull row wp-block-msr-immersive-section\">\n\t\n\t<div class=\"container\">\n\t\t<div class=\"wp-block-msr-immersive-section__wrapper\">\n\t\t\t<hr class=\"wp-block-separator has-text-color has-blue-color has-alpha-channel-opacity has-blue-background-color has-background\" \/>\n\n\n\n<h2 class=\"wp-block-heading has-text-align-center\" id=\"models\">Models<\/h2>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<h3 class=\"wp-block-heading\" id=\"orca-progressive-learning-from-complex-explanation-traces\">Orca: Progressive Learning from Complex Explanation Traces<\/h3>\n\n\n\n<p>Imitate reasoning processes of larger models with explanation tuning; improvements over models like Vicuna-13B by more than 100% in complex zero-shot reasoning benchmarks like Big-Bench Hard (BBH) and 42% on AGIEval.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--75cf82c3491acdfdb097be5cd6f3a25f\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4\/\">Read the paper<\/a><\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<h3 class=\"wp-block-heading\" id=\"orca-2-teaching-small-language-models-how-to-reason\">Orca-2: Teaching Small Language Models How To Reason<\/h3>\n\n\n\n<p>Enhance smaller language models with reasoning abilities traditionally seen in larger models by teaching models to choose different strategies for varied tasks; performance levels similar or better than those of models 5-10x larger on complex tasks that test advanced reasoning abilities in zero-shot settings.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-container-core-buttons-is-layout-2 wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--b8336ffb2cb7486ce7874ca411155e11\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/huggingface.co\/microsoft\/Orca-2-7b\" target=\"_blank\" rel=\"noreferrer noopener\">Orca-2-7B<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-outline is-style-outline--94fcd1a58012af46c77d4aaa3d32eae8\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/huggingface.co\/microsoft\/Orca-2-13b\" target=\"_blank\" rel=\"noreferrer noopener\">Orca-2-13B<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-outline is-style-outline--9554997fc407a330bc61a115d4244e4e\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/orca-2-teaching-small-language-models-how-to-reason\/\">Read the paper<\/a><\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<p>Orca models were designed for research settings, and its testing has only been carried out in such environments. It should not be used in downstream applications, as additional analysis is needed to assess potential harm or bias in the proposed application.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-blue-color has-alpha-channel-opacity has-blue-background-color has-background\" \/>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"agentinstruct-creating-high-quality-synthetic-data-using-agentic-flows\">AgentInstruct &#8211; Creating high-quality synthetic data using agentic flows<\/h2>\n\n\n\n<p>In Orca, we recently released <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/agentinstruct-toward-generative-teaching-with-agentic-flows\/\">AgentInstruct<\/a>, an extensible agentic framework for automatically creating large amounts of diverse and high-quality synthetic data. AgentInstruct can create both the prompts and responses, using only raw data sources like text documents and code files as seeds. We demonstrated the utility of AgentInstruct by creating a post training dataset of 25M pairs to teach language models different skills, such as text editing, creative writing, tool usage, coding, reading comprehension, etc. The dataset can be used for instruction tuning of any base model. <br><br>We post-trained Mistral-7b with the data. When comparing the resulting model (Orca-3) to Mistral-7b-Instruct (which uses the same base model), we observe significant improvements across many benchmarks. For example, 40% improvement on AGIEval, 19% improvement on MMLU, 54% improvement on GSM8K, 38% improvement on BBH and 45% improvement on AlpacaEval. Additionally, it consistently outperforms other models such as LLAMA-8B-instruct and GPT-3.5-turbo.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1661\" height=\"664\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Figure-1.png\" alt=\"chart, bar chart\" class=\"wp-image-1063512\" style=\"width:1268px;height:auto\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Figure-1.png 1661w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Figure-1-300x120.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Figure-1-1024x409.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Figure-1-768x307.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Figure-1-1536x614.png 1536w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/07\/Figure-1-240x96.png 240w\" sizes=\"(max-width: 1661px) 100vw, 1661px\" \/><figcaption class=\"wp-element-caption\">This figure from the AgentInstruct papers shows the effect of using AgentInstruct data for post-training Mistral-7B. We see significant improvements across many benchmarks for Mistral-7B post-trained on AgentInstruct data.<\/figcaption><\/figure>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n","protected":false},"excerpt":{"rendered":"<p>Using AI, to improve AI Orca is a research team in Microsoft Research. Orca focuses on creating automated pipelines for creating high-quality synthetic data at scale, and training\u00a0models for specialization and model self-improvement. Orca\u2019s research areas involve self-improvement strategies, feedback-driven teaching methods between large and small models to create high-quality synthetic data and using domain [&hellip;]<\/p>\n","protected":false},"featured_media":984930,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556,13554],"msr-locale":[268875],"msr-impact-theme":[264846],"msr-pillar":[],"class_list":["post-983295","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-human-computer-interaction","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[945906,985959,1054431],"related-downloads":[1001559,1001568,1001559,1001568],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Ahmed Awadallah","user_id":31979,"people_section":"Section name 0","alias":"hassanam"},{"type":"user_nicename","display_name":"Andres Codas","user_id":42207,"people_section":"Section name 0","alias":"andrescodas"},{"type":"user_nicename","display_name":"Hamed Khanpour","user_id":38055,"people_section":"Section name 0","alias":"hakhanpo"},{"type":"user_nicename","display_name":"Yash Lara","user_id":43341,"people_section":"Section name 0","alias":"yashlara"},{"type":"user_nicename","display_name":"Shweti Mahajan","user_id":42594,"people_section":"Section name 0","alias":"shmahaj"},{"type":"user_nicename","display_name":"Arindam Mitra","user_id":42978,"people_section":"Section name 0","alias":"armitra"},{"type":"user_nicename","display_name":"Corby Rosset","user_id":41997,"people_section":"Section name 0","alias":"corbyrosset"},{"type":"user_nicename","display_name":"Dany Rouhana","user_id":31540,"people_section":"Section name 0","alias":"danyr"},{"type":"user_nicename","display_name":"Ryen W. White","user_id":33481,"people_section":"Section name 0","alias":"ryenw"},{"type":"user_nicename","display_name":"Guoqing Zheng","user_id":37941,"people_section":"Section name 0","alias":"zheng"}],"msr_research_lab":[199565,992148],"msr_impact_theme":["Computing foundations"],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/983295"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":37,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/983295\/revisions"}],"predecessor-version":[{"id":1091127,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/983295\/revisions\/1091127"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/984930"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=983295"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=983295"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=983295"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=983295"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=983295"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}