{"id":1136575,"date":"2025-04-15T07:49:06","date_gmt":"2025-04-15T14:49:06","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=1136575"},"modified":"2026-05-18T10:04:56","modified_gmt":"2026-05-18T17:04:56","slug":"multimodal-hls-foundation-models","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/multimodal-hls-foundation-models\/","title":{"rendered":"Accelerating healthcare and life sciences discovery with AI"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background- card-background--full-bleed\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"720\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4.png\" class=\"attachment-full size-full\" alt=\"a screenshot of a video game\" style=\"\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4.png 1280w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-300x169.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-1024x576.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-768x432.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-1066x600.png 1066w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-655x368.png 655w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-240x135.png 240w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-640x360.png 640w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/04\/Personalized-Medicine-4-960x540.png 960w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 \">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading is-style-l\" id=\"health-and-life-sciences-ai-frontiers\">Health and Life Sciences        AI Frontiers<\/h1>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p>The Healthcare AI Frontiers group is dedicated to transforming healthcare through the development and deployment of advanced multimodal artificial intelligence solutions. Our work spans frontier research and real\u2011world clinical systems, with a focus on enabling collaborative, agent\u2011driven workflows that support clinicians, care teams, and health systems at scale. By translating breakthroughs in AI into integrated, multi\u2011person experiences embedded in everyday tools, we aim to reduce complexity, improve coordination, and ultimately deliver better outcomes for patients worldwide.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"innovative-multi-modal-models\">What We Do<\/h1>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"innovative-multi-modal-models\">Innovative Multimodal Models<\/h3>\n\n\n\n<p>Microsoft and our partners have made significant investments in the research and development of multi-modal models specifically designed for healthcare and life sciences. Our commitment to innovation is driven by the goal of transforming these critical fields through the implementation of advanced GenAI. See the full catalog of models here: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/aka.ms\/health-life-sciences\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/aka.ms\/health-life-sciences<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n\n\n\n<p>Microsoft and partner foundation models are built from the ground up using vast de-identified medical datasets and are ready for fine-tuning to address the unique challenges of healthcare and life sciences. These models are capable of processing and analyzing diverse types of medical data, including images, records, and genomic information, to provide comprehensive insights and support decision-making.<\/p>\n\n\n\n<p>The versatility of our models allows them to be deployed in a range of scenarios. From enhancing image quality analysis to predicting patient outcomes and identifying potential treatment pathways, our models are designed to support healthcare professionals in delivering better patient care. In life sciences, these models may facilitate groundbreaking research and accelerate the development of new therapies and medical solutions.<\/p>\n\n\n\n<p>Key activities include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Model Development<\/strong>: We develop first-party (1P) models and collaborate with third-party (3P) partners to expand our model catalog. This includes fine-tuning models for specific use cases and integrating them into clinical workflows.<\/li>\n\n\n\n<li><strong>Platform Engineering<\/strong>: Our team works on creating production code within AI Foundry, including demo experiences and documentation, to ensure seamless integration and usability for our customers.<\/li>\n\n\n\n<li><strong>Customer Engagement<\/strong>: We actively engage with healthcare providers, pharmaceutical companies, and research institutions to understand their needs and tailor our solutions accordingly. This involves structured partner engagement and training sessions to promote our solutions.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"revolutionizing-healthcare-with-cutting-edge-agentic-solutions\">Revolutionizing Healthcare with Cutting-edge Agentic Solutions<\/h3>\n\n\n\n<p>The HLS AI Frontiers team advances agentic AI frameworks purpose\u2011built for the complexity of healthcare, where meaningful outcomes depend on coordination across people, systems, and time. We design AI systems that go beyond single tasks, enabling end\u2011to\u2011end orchestration of complex clinical and operational workflows involving multiple roles, teams, and decision points. Our work emphasizes collaborative intelligence\u2014AI that works alongside clinicians and staff, supporting shared context, accountability, and human\u2011in\u2011the\u2011loop decision\u2011making. Built to integrate seamlessly with the diverse data sources that underpin healthcare\u2014from clinical systems to enterprise tools\u2014our approaches surface the right information at the right moment in the flow of work.<\/p>\n\n\n\n<p>Building on early explorations in healthcare agent orchestration, HLS AI Frontiers continues to shape how intelligent agents can responsibly and scalably support care delivery across health systems worldwide.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"microsoft-multi-modal-models-select-models-summary\">How We Partner<\/h1>\n\n\n\n<p>The Healthcare AI Frontiers team collaborates with a diverse range of partners to drive innovation and adoption of our AI solutions. Our partnerships span across various sectors, including healthcare providers, pharmaceutical companies, and technology developers.<\/p>\n\n\n\n<p>Key partnership strategies include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Custom Model Collaborations<\/strong>: We work closely with partners to fine-tune AI models for specific use cases, leveraging custom agreements led by our business development team.<\/li>\n\n\n\n<li><strong>Joint Development<\/strong>: We collaborate with partners like Paige, Nvidia, and Providence to onboard their models into our catalog and communicate the value proposition to customers.<\/li>\n\n\n\n<li><strong>Structured Engagement<\/strong>: We coordinate with partners like Accenture and Global Logic to kick off engagement and explore potential collaborations.<\/li>\n\n\n\n<li><strong>Research and Publication<\/strong>: Our team publishes papers and sample codes in collaboration with research institutions to showcase the capabilities and impact of our AI models.<\/li>\n<\/ul>\n\n\n\n<p>Through these partnerships, HLS AI Frontiers aims to create a unified vision that leverages Microsoft&#8217;s extensive resources and expertise to reshape modern healthcare. By connecting product, platform, and research, we turn organizational complexity into a strategic force capable of driving positive, human-centered transformation in the healthcare industry.<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n<\/div>\n\n\n\n\n\n<p><\/p>\n\n\n\n<p><em>The Microsoft healthcare AI models are intended for research and model development exploration. The models are not designed or intended to be deployed in clinical settings as-is nor for use in the diagnosis or treatment of any health or medical condition, and the individual models&#8217; performances for such purposes have not been established. Developers bear sole responsibility and liability for any use of the healthcare AI models, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.<\/em><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"model-catalog\">AI Foundry Model Catalog<\/h3>\n\n\n\n<p>Filtered to Health and Life Sciences:&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/aka.ms\/health-life-sciences\">https:\/\/aka.ms\/health-life-sciences<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/p>\n\n\n\n<p>We have a rich catalog of in-house developed 1P models and a carefully curated list of 3P models from partners and model providers. <\/p>\n\n\n\n<p>The HLS AI Frontiers team continues to develop foundation models that are made available for Azure AI customers. Along with deployment access, you can find details on Model Architecture, License, Training Information, Evaluation Results, Sample Inputs\/Outputs, Data and Resource Specs for Deployment, and HW Requirements.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/ai.azure.com\/explore\/models\/MedImageInsight\/version\/5\/registry\/azureml\">MedImageInsights Model Card<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> &#8211; an embeddings model<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/ai.azure.com\/explore\/models\/CxrReportGen\/version\/5\/registry\/azureml\">CXRReportGen Model Card<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> &#8211; a grounded findings generation model<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/ai.azure.com\/explore\/models\/MedImageParse\/version\/4\/registry\/azureml\">MedImageParse Model Card<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> &#8211; a 2D image segmentation model<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/ai.azure.com\/explore\/models\/MedImageParse3D\/version\/1\/registry\/azureml\">MedImageParse3D Model Card<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> &#8211; a 3D image segmentation model<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"github-samples\">GitHub Samples Repository<\/h3>\n\n\n\n<p>Designed to help you get started with Microsoft&#8217;s healthcare AI models. Whether you are a researcher, data scientist, or developer, you will find a variety of examples and solution templates that showcase how to leverage these powerful models for different healthcare scenarios. From basic deployment and usage patterns to advanced solutions addressing real-world medical problems, this repository aims to provide you with the tools and knowledge to build and implement healthcare AI solutions using Microsoft AI ecosystem effectively:&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/\">https:\/\/github.com\/microsoft\/healthcareai-examples\/<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n\n\n\n<p>Here&#8217;s a quick look at what you&#8217;ll find:<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"basic-usage-examples-and-patterns\">Basic Usage Examples and Patterns:<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/medimageparse\/medimageparse_segmentation_demo.ipynb\">MedImageParse call patterns<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; a collection of snippets showcasing how to send various image types to MedImageParse and retrieve segmentation masks. See how to read and package xrays, ophthalmology images, CT scans, pathology patches, and more.<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/medimageinsight\/zero-shot-classification.ipynb\">Zero shot classification with MedImageInsight<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; learn how to use MedImageInsight to perform zero-shot classification of medical images using its text or image encoding abilities.<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/medimageinsight\/adapter-training.ipynb\">Training adapters using MedImageInsight<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; build on top of zero shot pattern and learn how to train simple task adapters for MedImageInsight to create classification models out of this powerful image encoder. For additional thoughts on when you would use this and the zero shot patterns as well as considerations on fine tuning, read our blog on Microsoft Techcommunity Hub.<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/medimageinsight\/advanced-call-example.ipynb\">Advanced calling patterns<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; no production implementation is complete without understanding how to deal with concurrent calls, batches, efficient image preprocessing, and deep understanding of parallelism. This notebook contains snippets that will help you write more efficient code to build your cloud-based healthcare AI systems.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"advanced-examples-and-solution-templates\">Advanced Examples and Solution Templates<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/medimageinsight\/outlier-detection-demo.ipynb\">Detecting outliers in MedImageInsight<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; go beyond encoding single image instances and learn how to use MedImageInsight to encode CT\/MR series and studies and detect outliers in image collections.<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/medimageinsight\/exam-parameter-demo\/exam-parameter-detection.ipynb\">Exam Parameter Detection<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; dealing with entire MRI imaging series, this notebook explores an approach to a common problem in radiological imaging &#8211; normalizing and understanding image acquisition parameters. Surprisingly (or not), in many cases DICOM metadata cannot be relied upon to retrieve exam parameters. Look inside this notebook to understand how you can build a computationally efficient exam parameter detection system using an embedding model like MedImageInsight.<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/advanced_demos\/radpath\/rad_path_survival_demo.ipynb\">Multimodal image analysis using radiology and pathology imaging<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; can foundational models be connected to build systems that understand multiple modalities? This notebook shows a way this can be done using the problem of predicting cancer hazard score via a combination of MRI studies and digital pathology slides. Also read&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/techcommunity.microsoft.com\/blog\/healthcareandlifesciencesblog\/cancer-survival-with-radiology-pathology-analysis-and-healthcare-ai-models-in-az\/4366241\">our blog<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;that goes into more depth on this topic.<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/advanced_demos\/image_search\/2d_image_search.ipynb\">Image Search Series Pt 1: Searching for similar XRay images<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; an opener in the series on image-based search. How do you use foundation models to build an efficient system to look up similar Xrays? Read&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/techcommunity.microsoft.com\/blog\/healthcareandlifesciencesblog\/image-search-series-part-1-chest-x-ray-lookup-with-medimageinsight\/4372736\">our blog<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;for more details.<\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/microsoft\/healthcareai-examples\/blob\/main\/azureml\/advanced_demos\/image_search\/3d_image_search.ipynb\">Image Search Series Pt 2: 3D Image Search with MedImageInsight (MI2)<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;&#8211; expanding on the image-based search topics we look at 3D images. How do you use foundation models to build a system to search the archive of CT scans for those with similar lesions in the pancreas? Read&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/aka.ms\/healthcare-ai-examples-mi2-3d-image-search-blog\">our blog<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;for more details.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"research-papers\">Research Papers<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/2410.06542\">MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/microsoft.github.io\/BiomedParse\/\">BiomedParse: A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-maira\/publications\/?msockid=285b61c65a0066cb1d9275135bba6713\">MAIRA Complete List of Publications<\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/2410.13174\">Scalable Drift Monitoring in Medical Imaging AI<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/2503.10057\">Multi-Modal Mamba Modeling for Survival Prediction (M4Survive): Adapting Joint Foundation Model Representations<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"nature-publications\">Nature Publications:<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/www.nature.com\/articles\/s41591-024-03141-0\">Virchow: A foundation model for clinical-grade computational pathology and rare cancers detection<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/www.nature.com\/articles\/s41586-024-07441-w\">Gigapath: A whole-slide foundation model for digital pathology from real-world data<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/www.nature.com\/articles\/s41592-024-02499-w\">BioMedParse: A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n\n\n\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/www.nature.com\/articles\/s42256-024-00965-w\">Rad-DINO: Exploring scalable medical image encoders beyond text supervision<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"benchmarking\">Benchmarking:<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/rexrank.ai\/\">ReXrank Chest X-ray Report Generation Leaderboard (hosted on Azure; MAIRA-2\/CXRReportGen is the MSFT model)<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<\/ul>\n\n\n\n\n\n<h2 class=\"wp-block-heading\" id=\"healthcare-agent-orchestrator\">Healthcare Agent Orchestrator<\/h2>\n\n\n\n<p>Announcing our new open-source project called the&nbsp;<strong>Healthcare Agent Orchestrator<\/strong>. It\u2019s built on top of Azure AI Foundry and a deep foundation of research, with one clear goal: to make complex clinical workflows \u2014 like multidisciplinary tumor boards \u2014 dramatically easier to manage.<\/p>\n\n\n\n<p>What sets it apart is its ability to bring together multimodal AI to analyze everything from imaging and pathology to genomics and clinical notes \u2014 fast. Institutions like&nbsp;<strong>Stanford, Johns Hopkins, Providence Genomics<\/strong>, and&nbsp;<strong>UW Health<\/strong>&nbsp;are already exploring how it can help streamline care and improve outcomes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"key-capabilities\"><strong>Key capabilities:<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Hours of analysis, reduced to minutes<\/li>\n\n\n\n<li>Works with tools you already use&nbsp;(like Teams and Microsoft 365)<\/li>\n\n\n\n<li>Delivers AI insights you can trust and act on<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"learn-more\"><strong>Learn More:<\/strong><\/h3>\n\n\n\n<p>Blog: <a href=\"https:\/\/www.microsoft.com\/en-us\/industry\/blog\/healthcare\/2025\/05\/19\/developing-next-generation-cancer-care-management-with-multi-agent-orchestration\/\">Developing next-generation cancer care management with multi-agent orchestration<\/a><\/p>\n\n\n\n<p>Fireside Chat: Transforming Tumor Boards: AI Agents and the New Era of Personalized Cancer Care<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"yt-consent-placeholder\" role=\"region\" aria-label=\"Video playback requires cookie consent\" data-video-id=\"jeY8VUr3GmU\" data-poster=\"https:\/\/img.youtube.com\/vi\/jeY8VUr3GmU\/maxresdefault.jpg\"><iframe aria-hidden=\"true\" tabindex=\"-1\" title=\"Transforming Tumor Boards: AI Agents and the New Era of Personalized Cancer Care\" width=\"500\" height=\"281\" data-src=\"https:\/\/www.youtube-nocookie.com\/embed\/jeY8VUr3GmU?feature=oembed&rel=0&enablejsapi=1\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><div class=\"yt-consent-placeholder__overlay\"><button class=\"yt-consent-placeholder__play\"><svg width=\"42\" height=\"42\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><g fill=\"none\" fill-rule=\"evenodd\"><circle fill=\"#000\" opacity=\".556\" cx=\"21\" cy=\"21\" r=\"21\"\/><path stroke=\"#FFF\" d=\"M27.5 22l-12 8.5v-17z\"\/><\/g><\/svg><span class=\"yt-consent-placeholder__label\">Video playback requires cookie consent<\/span><\/button><\/div><\/div>\n<\/div><\/figure>\n\n\n\n<p>Tutorial: <\/p>\n\n\n\n<figure class=\"wp-block-video\"><video controls src=\"https:\/\/mediusdownload.event.microsoft.com\/video-7525406\/db2398f109\/OD815_v4.mp4?sv=2018-03-28&sr=c&sig=a9kacPiEK7gNvzloD0VGvCV80bJrb4W3Of1GYmSMRfI%3D&se=2030-05-17T06%3A28%3A41Z&sp=r\"><\/video><\/figure>\n\n\n\n<p>Real-world example: Stanford Medicine&#8217;s <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.youtube.com\/watch?v=DSOcjyV0oAE\" target=\"_blank\" rel=\"noopener noreferrer\">feature video<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n<p>Developer walkthrough: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/news.microsoft.com\/source\/features\/ai\/meet-4-developers-leading-the-way-with-ai-agents\/\" target=\"_blank\" rel=\"noopener noreferrer\">Developer Feature<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n<p>Open-source repository: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Azure-Samples\/healthcare-agent-orchestrator\/\" target=\"_blank\" rel=\"noopener noreferrer\">GitHub repository<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n\n\n<h2 class=\"wp-block-heading\" id=\"stanford-medicine-tumor-board-use-case\">Stanford Medicine Tumor Board Use Case<\/h2>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"yt-consent-placeholder\" role=\"region\" aria-label=\"Video playback requires cookie consent\" data-video-id=\"DSOcjyV0oAE\" data-poster=\"https:\/\/img.youtube.com\/vi\/DSOcjyV0oAE\/maxresdefault.jpg\"><iframe aria-hidden=\"true\" tabindex=\"-1\" title=\"Stanford Medicine and the healthcare agent orchestrator: Satya Nadella at Microsoft Build 2025\" width=\"500\" height=\"281\" data-src=\"https:\/\/www.youtube-nocookie.com\/embed\/DSOcjyV0oAE?feature=oembed&rel=0&enablejsapi=1\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><div class=\"yt-consent-placeholder__overlay\"><button class=\"yt-consent-placeholder__play\"><svg width=\"42\" height=\"42\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><g fill=\"none\" fill-rule=\"evenodd\"><circle fill=\"#000\" opacity=\".556\" cx=\"21\" cy=\"21\" r=\"21\"\/><path stroke=\"#FFF\" d=\"M27.5 22l-12 8.5v-17z\"\/><\/g><\/svg><span class=\"yt-consent-placeholder__label\">Video playback requires cookie consent<\/span><\/button><\/div><\/div>\n<\/div><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"oxford-agentic-ai-use-case-for-efficient-tumor-boards\">Oxford Agentic AI Use case for efficient Tumor Boards<\/h2>\n\n\n\n<p>Ignite 2025 talk about multi-agent orchestration approach and its implementation by Oxford University Hospitals<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"yt-consent-placeholder\" role=\"region\" aria-label=\"Video playback requires cookie consent\" data-video-id=\"cFJH00a12lo\" data-poster=\"https:\/\/img.youtube.com\/vi\/cFJH00a12lo\/maxresdefault.jpg\"><iframe aria-hidden=\"true\" tabindex=\"-1\" title=\"From Code to Care: Empowering Healthcare with Agentic AI | BRK373\" width=\"500\" height=\"281\" data-src=\"https:\/\/www.youtube-nocookie.com\/embed\/cFJH00a12lo?feature=oembed&rel=0&enablejsapi=1\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><div class=\"yt-consent-placeholder__overlay\"><button class=\"yt-consent-placeholder__play\"><svg width=\"42\" height=\"42\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><g fill=\"none\" fill-rule=\"evenodd\"><circle fill=\"#000\" opacity=\".556\" cx=\"21\" cy=\"21\" r=\"21\"\/><path stroke=\"#FFF\" d=\"M27.5 22l-12 8.5v-17z\"\/><\/g><\/svg><span class=\"yt-consent-placeholder__label\">Video playback requires cookie consent<\/span><\/button><\/div><\/div>\n<\/div><\/figure>\n\n\n","protected":false},"excerpt":{"rendered":"<p>The Healthcare AI Frontiers group is dedicated to transforming healthcare through the development and deployment of advanced multimodal artificial intelligence solutions. Our work spans frontier research and real\u2011world clinical systems, with a focus on enabling collaborative, agent\u2011driven workflows that support clinicians, care teams, and health systems at scale. By translating breakthroughs in AI into integrated, [&hellip;]<\/p>\n","protected":false},"featured_media":1139820,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556,13553],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1136575","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-medical-health-genomics","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[],"related-downloads":[],"related-videos":[1140590,1140600,1140628,1140759,1140793,1140799,1140811,1142319],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[1139386],"related-articles":[],"tab-content":[],"related-researchers":[{"type":"user_nicename","display_name":"Asma Ben Abacha","user_id":42558,"people_section":"Section name 0","alias":"abenabacha"},{"type":"user_nicename","display_name":"Noel Codella","user_id":41635,"people_section":"Section name 0","alias":"ncodella"},{"type":"user_nicename","display_name":"Alexander Ersoy","user_id":43862,"people_section":"Section name 0","alias":"aersoy"},{"type":"user_nicename","display_name":"Jameson Merkow","user_id":42225,"people_section":"Section name 0","alias":"jmerkow"},{"type":"user_nicename","display_name":"Mert Oez","user_id":43891,"people_section":"Section name 0","alias":"mehmetoez"},{"type":"user_nicename","display_name":"Naiteek Sangani","user_id":43887,"people_section":"Section name 0","alias":"naiteeks"},{"type":"user_nicename","display_name":"Alberto Santamaria-Pang","user_id":43863,"people_section":"Section name 0","alias":"albertosa"},{"type":"user_nicename","display_name":"Ivan Tarapov","user_id":36173,"people_section":"Section name 0","alias":"itarapov"},{"type":"guest","display_name":"Mu Wei","user_id":654207,"people_section":"Section name 0","alias":""}],"msr_research_lab":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/1136575","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":52,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/1136575\/revisions"}],"predecessor-version":[{"id":1172327,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/1136575\/revisions\/1172327"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1139820"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1136575"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1136575"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1136575"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1136575"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1136575"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}