Aurora 1.5 is a major update to the open Aurora Earth-system foundation model, adding 22 new weather variables for a broader view of atmospheric conditions, hourly forecasts, and probabilistic ensemble forecasting. Developed by Microsoft Weather as an extension of the original model from Microsoft Research AI for Science, Aurora 1.5 shows how frontier research can move into broader use: open for researchers and developers to evaluate and extend, and designed to support customers where additional data, infrastructure, and operational assurance is needed. As climate and weather-related risks continue to affect communities, infrastructure, and economies worldwide, advances in Earth-system forecasting can help improve preparedness and decision-making.

What is Aurora?

Aurora is a foundation model for the Earth system developed by Microsoft Research AI for Science, first introduced in 2024 and published in Nature (opens in new tab) in 2025. It showed that a single model could be adapted to medium-range weather, ocean waves, atmospheric chemistry, and emerging climate applications, including high-resolution weather forecasting through fine-tuning. Its growing use has reinforced the value of an open, collaborative model that is easier to adapt, evaluate, and put to use. 

This next phase of Aurora (opens in new tab) builds on that foundation by making the model openly available for the global community to adapt, extend, and build on. 

What is new in Aurora 1.5?

Aurora 1.5 advances the broader effort to make open weather foundation models practical and scalable for organizations that rely on atmospheric and Earth-system intelligence. Alongside new variables and higher temporal resolution, Aurora 1.5 adds one of the most requested capabilities from users: ensemble forecasting. Because forecasts are sensitive to initial conditions and model uncertainty, ensembles run multiple simulations to show the range and likelihood of possible outcomes. Aurora 1.5 builds on Microsoft Research’s scientific foundation with new product engineering, cloud infrastructure, managed access, and decision-support capabilities. Together, these advances make Aurora 1.5 a valuable enterprise-grade weather solution for organizations. 

The breadth update adds 22 new variables to Aurora’s original 4, including representative surface, pressure-level, wind, temperature, humidity, precipitation, and radiation fields. That broader coverage makes the model more relevant for sectors that depend on integrated Earth-system signals, from energy and agriculture to transport and resilience planning. 

The update to hourly temporal resolution enables fine-grained detail for precision operational guidance, such as the onset of precipitation, trade decisions, or a landfalling tropical cyclone. 

“Aurora 1.5 is a meaningful step toward making weather foundation models more open, useful, and practical. By releasing the model openly, we give researchers, developers, and organizations a clearer path to evaluate it, adapt it, and understand where it can help. Microsoft Weather’s role is to connect that open research foundation with the data, infrastructure, and applied workflows required by enterprises to use weather intelligence responsibly and with confidence.”

Sridhar Iyer, Corporate Vice President, Microsoft AI

video series

On Second Thought

A video series with Sinead Bovell built around the questions everyone’s asking about AI. With expert voices from across Microsoft, we break down the tension and promise of this rapidly changing technology, exploring what’s evolving and what’s possible.

Explore the series

Opens in a new tab

Ensemble Forecasting in Aurora 1.5 Unlocks More Confident Decisions in the Face of Weather Uncertainty

The ensemble version of Aurora 1.5 introduces stochastic perturbations to represent model uncertainty, allowing the generation of multiple forecast members to estimate the spread of possible futures. For a multitude of applications including power systems, transport, agriculture, extreme-weather planning, and climate risk, the model distribution matters as much as the best estimate. 

This ensemble capability was developed through multi-stage fine-tuning on top of the original Aurora model. After expanding the variable set and adding hourly temporal resolution, the team introduced controlled perturbations into the model’s latent conditioning pathway and optimized the ensemble for probabilistic forecast quality. A final round of auto-regressive fine-tuning on ECMWF High Resolution (HRES) analysis data from 2018 to 2023 improved rollout behavior and stability.

Aurora’s ensemble approach summarizes uncertainty across multiple model runs. Its probabilistic forecasts outperform those of the state-of-the-art ECWMF dynamical ensemble on 88.9% of evaluated targets (Figure 1). In evaluations on all 2024–2025 tropical cyclones, Aurora 1.5 substantially reduced track errors, including roughly one-third lower track error when comparing the ensemble median to the original Aurora. An example for the devastating Hurricane Helene shows how Aurora 1.5’s skill translates to high-impact weather applications. 

Beyond weather: Aurora as an Earth-system foundation

Beyond medium-range weather applications, Terradot – part of the Microsoft Climate Innovation Fund portfolio—is working with the AI for Good Lab (opens in new tab) and the Microsoft Research Accelerator on TerraNova, using Aurora-derived weather representations (opens in new tab) to estimate and optimize carbon dioxide removal from enhanced rock weathering under real field conditions. Sasankh Munukutla, Co-Founder of Terradot, highlights, “By building on Aurora, we’re significantly advancing our R&D timelines and accelerating our path towards gigaton-scale carbon removal.” This work shows how Earth-system foundation models can support climate mitigation and public-interest science beyond forecasting, including settings where rigorous evaluation and responsible deployment matter.

Aurora is also being explored with partners such as the UK Met Office, exploring how foundation models can work alongside established physics-based systems to tackle problems from weather to climate time scales. The aim is faster, more flexible forecasts that support decision-making without replacing the science behind trusted prediction. 

“Microsoft’s Aurora model is an exciting and promising tool, enabling Met Office scientists to bring their data and expertise to help solve climate problems and provide new kinds of climate information. Met Office and Microsoft scientists and engineers are working together every day to translate lessons from AI weather prediction into the climate information space, sharing expertise in data science and climate science. Aurora is a great platform for learning how to translate these tools for use in climate projection to make the AI climate models of the future.”

— Doug McNeall, Science lead for Data-Driven Climate Modelling, Met Office Hadley Centre 

Connecting open models to operational use

Microsoft connects open research, product engineering, responsible deployment, and partner ecosystems so that models can move from scientific advance to evaluated operational use. As an example, Aurora began in Microsoft Research AI for Science and is now being built on for operational use by Microsoft Weather, with AI for Good helping to evaluate public-interest applications. The platform path brings Aurora into Microsoft Foundry and Planetary Computer Pro, alongside Agent skills and Azure services that connect models with geospatial data, scalable infrastructure, and applied workflows. BKW provides an early proof point: the company is using Aurora 1.5 alongside existing operational Microsoft Weather models to support energy operations where weather-dependent generation, infrastructure planning, and environmental data need to come together. 

“This collaboration demonstrates how advanced AI capabilities and robust cloud infrastructure can be applied to one of the most strategic domains — energy, where weather plays a fundamental role. In a time of accelerated transformation, it supports our ambition to operate increasingly renewable-based systems, where generation is inherently weather-dependent, and to better anticipate and manage this variability with greater confidence and precision.” 

Farhat Quiñones Yamshid, Lead, AI and Technology, BKW 

From open research to broader impact

Aurora’s open-source availability is intended to help researchers, agencies, companies, and civil society evaluate, apply, and extend the model. Microsoft Weather is building on that open foundation to deliver easier access to Aurora forecasts through managed services, integrations, and responsible deployment paths for organizations that depend on weather and Earth-system intelligence.

Foundation models should complement—not replace—physics-based models and domain expertise. The opportunity is to use them responsibly, with careful evaluation and transparency, and to invite researchers, agencies, companies, and public-interest partners to test where Aurora and related Microsoft Weather capabilities can improve forecasting, planning, and climate resilience in their own settings.

About Microsoft Weather 

Microsoft Weather is the AI-based forecasting team behind weather experiences across Windows, Bing, Copilot, Edge, and MSN, reaching more than a billion devices across 180 countries. The team has been applying AI to operational weather forecasting for more than seven years and has built a proven track record of delivering high-quality forecasts at global scale. Microsoft Weather has won multiple forecasting competitions and was ranked the world’s most accurate global forecast provider by an independent third party for three consecutive years from 2022 to 2024. Building on today’s Aurora 1.5 announcement, the team plans to extend this work in the coming months with additional fit-for-purpose AI weather models designed for enterprise scenarios where forecast quality, speed, uncertainty, and operational decision support matter most.

If you are interested in exploring Aurora and Microsoft Weather solutions for commercial or organizational applications, please contact us at AIWeatherClimate@microsoft.com 

The post Aurora 1.5: Extending open foundation models for weather and Earth-system applications appeared first on Microsoft Research.

Creating a good chart requires many design decisions: how dates should be parsed, whether a scale should start at zero, how values should be formatted, how much room labels need, and which colors make the data easier to read. Modern visualization libraries such as Vega-Lite, Apache ECharts, and Chart.js expose these controls, but there is a trade-off: Short specifications that rely on system defaults often produce uninspiring charts, while polished visualizations require detailed specifications with purposely chosen parameters that are often verbose, fragile, and error-prone.

This trade-off becomes sharper as large language models (LLMs) and AI agents take on more visualization work. Agents are especially prone to errors when they must manage complex, low-level specification details, and the resulting fragile code can be difficult for people to inspect, repair, or reuse. Ideally, we need something in between: a compact specification that agents can produce reliably, people can edit directly, and a system can compile into a well-designed chart.

To address this challenge, we introduce Flint (opens in new tab), a visualization intermediate language for AI-driven chart creation. Flint helps AI agents create expressive, attractive charts from simple, human-editable chart specs. Instead of requiring verbose low-level parameters for scales, axes, spacing, and layout, the Flint compiler derives optimized chart settings from the data, semantic types, chart type, and encodings. The same Flint spec can render through multiple backends, including Vega-Lite, Apache ECharts, and Chart.js.

How Flint works

Figure 2 illustrates the how the Flint compiler turns a compact chart specification into a refined heatmap.

To produce a high-quality heatmap, traditionally, we need to explicitly tell the system with low-level chart properties about how to process the period field, how to properly label MonthYear values, size individual heatmap cells, and choose a color scale that appropriately represents positive and negative newUsers values. Without these configurations, visualization libraries must guess from field names and raw values, which can lead to charts that are technically valid but potentially misleading. While they are important, hard-coding these details can be difficult and error-prone, and they make specification fragile and hard for users to understand or adapt.

In Flint, these low-level details are systematically managed, where the compiler infers them from high-level data and chart specifications. Here, the data specification captures semantic types and optional metadata, and the chart specification defines the chart type and maps fields to visual channels such as x, y, color, size, or facet. From this information, the compiler derives the parsing rules, scales, axes, aggregations, formatting, color schemes, layout, and generates the backend-native specification, which is used to render the final polished visualization. This frees users from explicitly setting fragile and error-prone low-level details.

Furthermore, because the intermediate representation is separate from any single rendering library, Flint can target backends with very different APIs and programming models. Users can keep the same compact chart intent while compiling to Vega-Lite, ECharts, or Chart.js, and choose the backend whose capabilities best fit the visualization.

Azure AI Foundry Labs

Get a glimpse of potential future directions for AI, with these experimental technologies from Microsoft Research.

Azure AI Foundry

Opens in a new tab

Flint for AI-assisted visualization

Flint is well suited to LLM-based chart generation because semantic types are often easier for models to infer than the full set of low-level visualization parameters. Field names, value patterns, and common data knowledge can help an agent recognize whether a column represents a date, price, percentage, country, ranking, or correlation. Once those meanings are explicit, the compiler can handle many design decisions that would otherwise appear as brittle, library-specific code.

In our research study, we compared Flint with DirectVL, a baseline that asks the model to directly generate full (more complex) Vega-Lite specifications in a LLM self-evaluation pipeline. Across three tested models based on testing data from Tidy Tuesdays, Flint received higher overall LLM-judge scores: 16.27 vs. 15.91 with GPT-5.1, 16.16 vs. 15.60 with GPT-5-mini, and 15.91 vs. 15.34 with GPT-4.1. In fact, Flint has been so powerful and reliable that it is now used to power Data Formulator (opens in new tab), a Microsoft Research project for AI-assisted data analysis and visualization.

To make Flint easy for your agents to access, we also release flint-chart-mcp, a Model Context Protocol (MCP) server that allows agents to create, validate, and render charts inside a chat or coding environment. MCP calls can embed data inline or read configured local files, and the server can open an interactive chart view so users can inspect and refine the results.

Try Flint

Flint is open source and ready to use:

• Project site: https://microsoft.github.io/flint-chart/ (opens in new tab)
• GitHub: https://github.com/microsoft/flint-chart (opens in new tab)
• Flint MCP server instruction: https://microsoft.github.io/flint-chart/#/mcp (opens in new tab)

Flint points toward a shared semantic layer for visualization, where people and AI agents can work with compact chart intent while a compiler handles the careful low-level details. We invite the community to explore the project and build on it.

The post Flint: A visualization language for the AI era appeared first on Microsoft Research.

Large language models (LLMs) are increasingly deployed as agents that gather evidence, call tools, and execute multi-step tasks. For these agents, the hard problem is no longer whether they can call a tool, but whether they can complete tasks reliably and consistently. Today, agent skills typically come from three sources: experts write them by hand, a frontier model generates them one-shot, or the agent loosely revises them after execution. None of these approaches behaves like a deep-learning optimizer. They lack step-size control, held-out validation, and any memory of revisions that failed. As a result, skills tend to grow longer and drift with each rewrite, and a revision that seems perfectly reasonable can quietly degrade real task performance. This uncontrolled skill evolution has become a major obstacle on the path from agent prototype to dependable, production-grade deployment.

In our recent paper, SkillOpt: Executive Strategy for Self-Evolving Agent Skills, we reframe the question from “how do we write a better prompt?” to “how do we train the skill?” SkillOpt treats the skill file as a trainable parameter living outside a frozen target model, bringing a training-style optimization loop, consistent gains across 52 evaluation cells, and a compact skill file that stays readable, auditable, and transferable.

How SkillOpt works

SkillOpt organizes skill editing as a forward–backward–update cycle in text space. In the forward pass, the frozen target model executes a batch of training tasks with the current skill; the rollout batch size controls how much evidence each update receives. In the backward pass, a separate optimizer model reads the resulting trajectories in reflection minibatches, distilling patterns to preserve from successful trajectories and patterns to correct from failures.

In the update step, the optimizer proposes small add, delete, and replace edits; candidate edits are merged, deduplicated, ranked, and clipped by a textual learning rate—a per-step edit budget. Every candidate skill must then pass a strict validation gate: it is adopted only if it scores strictly higher than the current skill on the held-out validation split. Rejected edits are not discarded; they enter a rejected-edit buffer that serves as negative feedback for later optimizer calls in the same epoch. On a slower cadence, an epoch-wise slow/meta update consolidates longer-horizon lessons that single batches cannot reveal (Figure 2). Together, bounded edits, validation gating, and best-version selection keep skill optimization controllable and auditable, so the skill converges instead of drifting.

Consistent gains across benchmarks, models, and execution modes

We evaluated SkillOpt across six benchmarks (SearchQA, SpreadsheetBench, OfficeQA, DocVQA, LiveMathematicianBench, and ALFWorld), seven target models from frontier-scale GPT-5.5 to the small open-weight Qwen3.5-4B, and three execution modes (direct chat, Codex, and Claude Code). Counting each combination as one evaluation cell, When measured against human-written skills, one-shot LLM skills, Trace2Skill, TextGrad, GEPA, and EvoSkill, SkillOpt delivered the best or tied for -best results on all 52 cells. These performance improvements are unusually large for a method that updates no model weights. With GPT-5.5 in direct chat, SkillOpt raises the six-benchmark average from 58.8 to 82.3, a +23.5-point absolute improvement—and +5.4 points above an oracle that picks the single best competing method per cell. The largest gains appear on procedural benchmarks: SpreadsheetBench rises from 41.8 to 80.7, OfficeQA from 33.1 to 72.1, and LiveMathematicianBench from 37.6 to 66.9. The same interface carries over to agentic loops, lifting GPT-5.5 by +24.8 points inside Codex and +19.1 inside Claude Code over no skill.

Spotlight: Microsoft research newsletter

Microsoft Research Newsletter

Stay connected to the research community at Microsoft.

Subscribe today

Opens in a new tab

A small model plus a skill file

Approaching the next model tier SkillOpt also narrows the gap between small or open-weight models and frontier models—without changing any weights or adding any extra model calls at inference. After optimization, GPT-5.4-mini’s six-benchmark average (64.3) exceeds the no-skill baseline of the larger GPT-5.4 (59.7), and GPT-5.4-nano (57.4) exceeds the no-skill baseline of GPT-5.2 (51.3). Qwen3.5-4B, a 4-billion-parameter open-weight model, surpasses GPT-5.2’s no-skill baseline as well. Gains that once required a larger model can now be approximated by one optimized skill file.

Skills that transfer: train once, reuse everywhere

The optimized skill file captures reusable task-solving procedures rather than instructions overfit to a single model, benchmark, or execution environment. This is why the same skill can still improve performance when transferred across model scales, agent harnesses, and related tasks. In our transfer experiments, skills continued to deliver gains when moved across model scales, across execution harnesses, and to a nearby math benchmark. The clearest example is cross-harness transfer: a spreadsheet skill trained inside Codex, dropped into Claude Code with no further optimization, lifts the no-skill baseline from 22.1 to 81.8 (+59.7)—slightly above the 80.4 achieved by training directly inside Claude Code. Because the two harnesses expose different tool surfaces, this suggests SkillOpt learns general workflow logic, not just harness-specific recipes.

Compact, readable, and built from very few accepted edits

The deployed artifact, best_ skill.md , is neither an opaque parameter blob nor an ever-growing log. Across six case studies, the median final skill length is roughly 920 tokens, and because the validation gate rejects most proposals, only one to four edits are accepted into the final file. OfficeQA’s +39.0-point gain comes from a single accepted edit. The learned rules read like a seasoned practitioner’s advice. Component ablations confirm that the controls do the work: removing the rejected-edit buffer lowers scores on all three ablation benchmarks, and removing both the meta skill and the slow update drops SpreadsheetBench from 77.5 to 55.0. A new adaptation layer for the agent era SkillOpt points to a lighter-weight path for domain-adapting agents: instead of fine-tuning weights, hard-coding task logic, or hand-tuning prompts, teams can train a small, versionable, auditable natural-language skill layer—wherever automatic evaluation or a reliable verifier exists.

By bringing learning rates, schedules, validation splits, rejected samples, and slow updates to agent skills, SkillOpt suggests that training need not be limited to model weights. Procedural knowledge outside the model can also be optimized.

When that process is controlled, validated, and recorded, a natural-language skill becomes a stable, transferable, and reversible adapter between frontier-model capability and real-world workloads. Read the full paper, visit the project page at aka.ms/skillopt (opens in new tab), or explore the SkillOpt GitHub repository at github.com/microsoft/SkillOpt (opens in new tab). Teams building agentic workflows can use SkillOpt as a foundation for training reusable skills against their own tasks and verifiers. See also our companion project, SkillLens.

The post SkillOpt: Agent skills as trainable parameters appeared first on Microsoft Research.

Imagine a workplace AI assistant helping you run a multi-month project. Over weeks of conversations, you share constraints, agree on milestones, revise deadlines, and surface dozens of stakeholder preferences. When you later ask it to draft an update for a colleague, it should recall not just the latest decision but the journey that got you there: what was tried, what was ruled out, who weighed in. Today’s AI agents struggle with this. Modern large language models (LLMs) are powerful reasoners, but they are effectively stateless: every session starts from zero, every long conversation forces the model to re-read its entire history, and every new piece of information is either stored as raw text (fragmented and noisy) or compressed into a vague summary (precise details lost). As AI assistants and autonomous agents move into long-horizon deployments, such as copilots that track a project for many months or even research agents that build up domain expertise with long horizon usage, the absence of principled memory system has become the critical bottleneck.

A growing line of work has begun to fill this gap. Systems like Mem0 extract atomic facts from conversations; retrieval-augmented (RAG) approaches index raw text fragments for later recall; and graph-based memory systems such as Zep and GraphRAG impose structure through entity relations. Each represents real progress, yet each runs into the same wall: existing designs force an unavoidable tradeoff between specificity (preserving fine-grained detail) and abstraction (organizing memory efficiently as it grows). Memora is built to give agents both.

What is Memora

Memora is an agentic memory framework designed for long-horizon AI agents. Memora’s central insight is to decouple what is stored from how it is retrieved. Memory content can remain rich and expressive, such as a project timeline, a multi-turn discussion about constraints, while a separate, lightweight structural layer handles indexing and retrieval. The result is a memory system that scales: it consolidates related information into stable units, surfaces fine-grained details when they matter, and lets the agent navigate its own history without re-reading everything. On standard long-conversation benchmarks, Memora sets new state-of-the-art performance while using up to 98% fewer tokens than would be consumed by dumping the full history into context.

Why this is hard: the abstraction–specificity tension

Existing memory systems fall into two extremes. Content-fragmentation systems, such as RAG and Mem0, embed extracted facts or text fragments directly. This preserves detail but produces brittle, isolated entries that lose narrative coherence. Coarse-abstraction systems compress experience into compact summaries. They are efficient, but summarization strips away the constraints, edge cases, and numeric details that make memory useful in the first place. Graph-based systems add structure on top of content, yet still rely on the content itself for retrieval and typically require rigid ontologies that don’t generalize across domains. None of these resolves the underlying tension between abstraction (which keeps memory efficient) and specificity (which gives memory utility).

How Memora works

Memora resolves this tension through a harmonic organization. Each memory entry has two components: a primary abstraction, which a short phrase (6–8 words) that captures what the memory is fundamentally about, and a memory value holding the rich content itself. Crucially, only the primary abstraction is embedded for similarity search; the value is never directly retrieved through its own content. This separation means new information about an evolving topic merges into the existing memory entry under the same primary abstraction, rather than fragmenting into a chain of partial duplicates. Complementing primary abstractions, cue anchors are short, context-aware tags extracted from each memory’s value, providing alternative access paths to the same memory. They function as flexible, organically-generated metadata.

To make this concrete: suppose a user says, “Dave and Sarah agreed to push the prototype to April 1, the pilot to May 2, and the MVP to May 30.” A knowledge-graph system would need predefined entity types and relation schemas: Person → agreed_on → Milestone → has_date → Date, and any new relation type would require schema extension. In Memora, the primary abstraction Updated Project Orion timeline agreed by Dave and Sarah serves as the canonical access point, while cue anchors like Dave Project Orion update, Project Orion prototype schedule, and Project Orion pilot timeline provide alternative retrieval paths — all without committing to an ontology. A later query about Dave’s recent contributions, or the prototype schedule, or pilot timing can all route to the same underlying memory through different cues, with the full detail preserved in the memory value.

On top of this representation, Memora introduces a policy-guided retriever that treats memory access as an active reasoning process. Rather than returning the top-k semantically similar items in a single shot, the policy retriever iteratively refines its query, expands through cue anchors to surface related-but-not-similar memories, and decides when to stop. This lets the agent navigate to relevant non-local context that pure semantic search would miss, chasing multi-hop dependencies the way a human would when recalling connected events. The retrieval policy can be either hand-prompted with a strong LLM or distilled into a much smaller model via reinforcement learning.

Spotlight: AI-POWERED EXPERIENCE

Microsoft research copilot experience

Discover more about research at Microsoft through our AI-powered experience

Start now

Opens in a new tab

Results

We evaluate Memora on two long-context benchmarks: LoCoMo, where dialogues average 600 turns, and LongMemEval, with 115,000-token contexts. Memora achieves new state-of-the-art performance on both: 86.3% LLM-judge accuracy on LoCoMo and 87.4% on LongMemEval, outperforming RAG, Mem0, Nemori, Zep, LangMem, and even full-context inference. The gap is largest on multi-hop reasoning, where Memora’s ability to traverse cue anchors pays the biggest dividends. The efficiency story is just as striking: Memora stores roughly half the memory entries per conversation that Mem0 does (344 vs. 651) and reduces token consumption by up to 98% relative to full-context inference. Less to read, less to store, better answers.

Looking forward

Memora’s design has implications beyond benchmark performance. We see this work as a step toward AI agents that can sustain long-term collaboration with users and accumulate organizational knowledge over months and years, not just within a single session. Building on this foundation, we are pursuing several complementary directions. MemLoop explores how memory systems can learn from retrieval and task failures, attribute errors to specific stages of the memory pipeline, and improve themselves over time. Deferred Memory investigates when memory construction should be postponed until sufficient context, evidence, or future utility becomes available, rather than committing prematurely to what should be stored. Group Memory examines how knowledge can be shared across teams and agents while preserving provenance, access boundaries, ownership, and sensitive context. We release our code alongside the paper and invite the community to build on this representation and explore what becomes possible when AI agents are no longer stateless.

Acknowledgements

We would like to thank Shantanu Dixit (Research Fellow) Paramaguru Harimurugan (Research Fellow), Rujia Wang, Victor Rühle, and Robert Sim for contributing to this project.

The post Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity appeared first on Microsoft Research.

The explainability problem in language neuroscience

Over the past decade, LLMs have become the most accurate tools we have for predicting how the human brain responds to language. Feed an LLM the same story a person hears in an fMRI scanner, and the model’s internal representations can predict the activity of individual patches of cortex with remarkable fidelity. But this success comes with a catch: nobody can read these models. They are millions of inscrutable parameters that can’t be directly translated into interpretations. A model that predicts brain activity tells us that a region responds to language, but not what it is actually picking up on, whether it’s food, places, numbers, or something else entirely. As black-box models spread, the gap between prediction and understanding has become one of the central problems in computational neuroscience.

Turning black boxes into testable theories

In a new paper accepted in Nature Neuroscience, Microsoft Research scientists, in collaboration with scientists at the University of California, Berkeley, University of California, San Francisco, and Columbia University, introduce a framework to overcome this explainability crisis: generative causal testing (GCT). GCT distills brain-prediction models into short, readable accounts of what each patch of cortex responds to, then tests those claims. An LLM writes new stories engineered to activate a specific brain area, subjects hear them in the scanner, and if the explanation is correct, the targeted region lights up. The result is a method that translates uninterpretable predictive models back into the currency of science: concise hypotheses that can be confirmed or refuted in a follow-up experiment. An LLM writes new stories engineered to activate a specific brain area, subjects hear them in the scanner, and if the explanation is correct, the targeted region lights up. The result is a method that translates uninterpretable predictive models back into the currency of science: concise hypotheses that can be confirmed or refuted in a follow-up experiment.

How GCT works

GCT has two steps: explanation, then verification. To generate an explanation, the method starts from a predictive model for a single voxel or region and identifies the short phrases that most strongly drive its predicted response. An LLM then summarizes those words into a concise verbal explanation, often a single phrase such as “food preparation” or “location names.”

The crucial second stage closes the loop. To build trust in the explanation, GCT uses an LLM to write new stories in which each paragraph is carefully constructed to drive a brain region according to its explanation. Three subjects returned to the scanner to read these synthetic stories. If a region’s activity to its “driving” paragraphs was significantly greater than to baseline text, the explanation passed a genuine causal test, not just a correlational one.

Across all three subjects, the core approach held up: the synthetic stories reliably drove their target regions above baseline, confirming that GCT’s short explanations capture something the cortex genuinely responds to. The explanations were also most trustworthy where the underlying brain-prediction models were strongest (the more stable the model, the more reliably its explanation could be confirmed in the scanner). With the method validated on regions whose selectivity was already known, the researchers turned GCT on harder questions.

GCT also proved sharp enough to settle long-standing ambiguities. Three neighboring regions involved in processing places have often been treated as functionally similar: the retrosplenial cortex (RSC), the parahippocampal place area (PPA), and the occipital place area (OPA). At first, stories written for one region also activated the others. But by generating differential stimuli (stories designed to switch one region on while keeping its neighbors quiet), GCT teased the three apart. For example, RSC responds more strongly to proper noun location names, like Tokyo or Connecticut, rather than general location. This is the kind of nuanced, region-specific theory that a raw predictive model cannot provide on its own.

Beyond known regions, the authors discovered new prefrontal “micro-regions.” By scanning a grid of candidate locations and keeping only the most stable ones, GCT surfaced these previously unmapped regions tuned to remarkably specific concepts: one selective for dialogue between people (words like “said” or “told”), one for mentions of clock times (“one o’clock”), and one for numeric measurements (“50 feet”). These are distinctions no one had gone looking for; they emerged because the method could propose a hypothesis and immediately test it.

video series

On Second Thought

A video series with Sinead Bovell built around the questions everyone’s asking about AI. With expert voices from across Microsoft, we break down the tension and promise of this rapidly changing technology, exploring what’s evolving and what’s possible.

Explore the series

Opens in a new tab

Implications and looking forward

The significance of GCT reaches well beyond neuroscience. Researchers increasingly face the same dilemma: a model that predicts beautifully but explains nothing. GCT shows that a data-driven model need not be the end of inquiry; it can be distilled into a readable, experimentally testable theory, and that theory can be checked against reality by generating new experiments on demand.

For neuroscience specifically, GCT points toward a faster, more hypothesis-rich way of mapping the cortex—one where an AI system proposes what a brain region might encode and a closed-loop experiment confirms or rejects it within a single study. The same generate-and-verify philosophy could extend to other domains where powerful predictive models have outrun our ability to understand them. The broader lesson is hopeful: the rise of black-box models in science does not necessarily mean the retreat of human-readable theory. With the right framework, the two can advance together.

Acknowledgements

This work was a collaboration across Microsoft Research, UC Berkeley (Alex Huth, Bin Yu, Sihang Guo, and Aliyah Hsu), Columbia University (RJ Antonello, co-lead), and UCSF (Shailee Jain). We also thank the study participants and the broader language-neuroscience community whose tools and datasets made this research possible.

Read the paper (opens in new tab): “Generative causal testing to bridge data-driven models and scientific theories in language neuroscience,” accepted in Nature Neuroscience and the code on Github (opens in new tab).

The post Understanding the brain with AI-driven explanations and experiments appeared first on Microsoft Research.

Why genome reanalysis matters

Genomic testing has transformed the diagnosis of rare disease, but even with this advancement, more than half of patients remain undiagnosed after their first test. This is because our knowledge of the genome is still incomplete. Researchers are learning more every day about the function of specific genes and how they relate to disease.

However, unlike most diagnostic investigations, genomic data has a unique property: it can be stored and reexamined indefinitely. Because our understanding of the genome improves constantly, simply rerunning the analysis later can yield a diagnosis that was impossible to make the first time. This is because there are hundreds of new gene–disease associations and thousands of new variant classifications reported every year.

Reanalysis of the genomes of undiagnosed patients is the solution; a meta-analysis of nearly 9,500 undiagnosed patients found that reanalysis lifted diagnostic yield by about 10% over roughly two years. However, the problem is that reanalysis today is overwhelmingly manual. It depends on motivated clinicians, scarce laboratory staff, and inconsistent reimbursement, so the vast majority of stored genomes are never revisited and the data keep accumulating. Automation has long been proposed as the answer, but the developers of automated machinery must navigate hard trade-offs between sensitivity, specificity, how many candidate variants a human must review, and how often the analysis is rerun.

Talos (opens in new tab), developed through a collaboration spanning the Centre for Population Genomics, Australian Genomics, the Broad Institute, and Microsoft, was built to resolve those trade-offs and to demonstrate, at international scale, that systematic reanalysis is both feasible and valuable. We have recently published a journal article (opens in new tab) detailing how Talos functions and evaluating its performance on multiple rare disease cohorts.

How Talos works

Talos re-interprets a patient’s existing variant calls against the latest community knowledge each time it runs. It draws on two continuously updated public resources: PanelApp Australia (opens in new tab) for gene–disease relationships and modes of inheritance, and ClinVar (opens in new tab) for variant-level pathogenicity. It then applies a variant-prioritization algorithm designed to surface variants most likely to meet ACMG/AMP criteria for clinical reporting.

The pipeline uses newly discovered information to tag and filter variants, then refines the candidate set using family structure (for example, mode of inheritance and de novo status) and, when available, the patient’s phenotype. Talos can be used to interpret single-nucleotide variants, small insertions/deletions, copy number variants, and large structural variants from exome or genome data.

Two design choices distinguish Talos. First, it is deliberately conservative, optimized to return a small set of high confidence variants rather than a long ranked list, because in real-world genomic reanalysis the limiting factor is human review time, not algorithmic recall. Second, on repeat runs, Talos returns only variants whose supporting evidence has changed since the previous cycle, allowing clinicians to focus exclusively on findings that aregenuinely new.

Validated against expert manual analysis

We benchmarked Talos on two independent cohorts that had already undergone careful manual analysis: the Australian Acute Care Genomics (ACG) cohort of critically ill infants and children, and the U.S.-based Rare Genomes Project (RGP) cohort of families with prior uninformative testing. This included 1,089 probands in total.

On ACG trios, Talos recovered 90% of in-scope diagnoses while returning a median of just 1.3 candidate variants per family. The diagnoses it missed were largely a direct consequence of its conservative strategy, for example, recessive variants lacking ClinVar support that human analysts had classified using trans configuration or functional studies.

Crucially, Talos held the same operating point on the very different RGP cohort, agroup of families who had previously had uninformative clinical testing, with probands ranging up to 82 years of age. On RGP trios, it recovered 87% of in-scope diagnoses (47 of 54) at a median of 1.3 candidate variants per trio, showing generalizability across cohorts.

We then benchmarked head-to-head against Exomiser, a widely used prioritization tool. Talos matched its overall sensitivity for small variants, but at a very different operating point: Exomiser ranks and returns a broad list, while Talos returns a short, highly specific one. In a paired comparison, the two tools were statistically indistinguishable when all of Exomiser’s ranked variants were reviewed, but Talos came out significantly ahead once review was limited to a realistic budget—the top five (p = 0.017) or top one (p < 0.0001) ranked variants. Notably, the two tools surfaced different variants, so they are complementary and should ideally be used together in diagnostic workflows.

PODCAST SERIES

AI Testing and Evaluation: Learnings from Science and Industry

Discover how Microsoft is learning from other domains to advance evaluation and testing as a pillar of AI governance.

Listen now

Opens in a new tab

Deployed on an international scale

The experiment we were most excited about was a tested-but-undiagnosed cohort of 4,735 individuals, drawn from Australian Genomics research studies and a single diagnostic laboratory. Most patients were singletons with neurodevelopmental, cardiac, renal, and/or neurological indications.

Talos produced 241 new diagnoses in 238 individuals—a 5.1% additional yield, with every single likely-causative variant subsequently confirmed as pathogenic or likely pathogenic by accredited labs.

The sources of those diagnoses illustrate why reanalysis is such a powerful paradigm:

• 32% came from new gene–disease relationships discovered since the original test,
• 22% came from new variant-level evidence (reclassifications), and
• 45% came from improved filtering and analysis—including variant types such as CNVs and structural variants not examined originally, phenotype filters that had been set too narrowly, and other sources.

Yield was consistent across clinical areas (roughly 5–6% for neurodevelopmental, cardiac, and renal indications) but the reasons differed: new gene associations and CNVs dominated neurodevelopmental diagnoses, while variant reclassification drove most cardiac ones. Genome data outperformed exome (6.1% vs 4.8%), partly by reaching non-coding diagnoses such as RNU4-2 and a deep-intronic MRPL39 variant. A recurring theme was the lag in conventional knowledge bases: 59% of the new gene–disease diagnoses were not yet curated in OMIM at the time of reanalysis, underscoring the value of drawing on a rapidly updated resource like PanelApp Australia.

From a one-off event to a continuous program

We then ran Talos for 29 monthly iterative cycles. Most diagnoses (92%) came on a cohort’s first pass, but the iterative design proved its value on two fronts. First, it demonstrated the scalability of ongoing reanalysis: because later cycles return only newly actionable evidence, they surfaced an average of just one variant per 200 cases over the program. Second, it showed how quickly we can move from scientific discovery to diagnosis: on average just 32 days passed between new knowledge appearing in a public database and a patient receiving a diagnosis, with the fastest case turning around in a single day. Figure 2 provides timelines for three example patients showing how continual reanalysis can bring answers to families within weeks of new scientific findings. The whole pipeline is cheap enough to run continuously: annotating 1,000 genomes cost about $11, and a monthly reanalysis pass ran for a few cents per cohort.

Looking ahead

Talos reframes genomic reanalysis from a rare, labor-intensive event into a continuous, automated program that can keep pace with the science. By optimizing for specificity, it respects the real bottleneck of expert reviewer time, and by drawing on openly shared, frequently updated resources like PanelApp Australia and ClinVar, it turns the global community’s accumulating knowledge into diagnoses for individual patients, often within weeks.

We believe we’ve established a foundational capability, and we’re excited to see how the community builds on it. In particular, as more advanced AI models for understanding and predicting the consequences of genetic variation become available, we’re looking forward to leveraging them in the reanalysis of unsolved rare disease cases.

Talos is open source and straightforward to deploy in cloud environments like Azure. Our results offer a practical blueprint for health systems aiming to deliver frequent, scalable reanalysis to the many patients still searching for diagnoses.

The post Talos: Scaling rare disease diagnosis with automated, iterative genomic reanalysis appeared first on Microsoft Research.

We pointed Project Ire, Microsoft’s autonomous malware-classification agent, at a malware sample—blind—and asked for a verdict. The sample is a variant of LOTUSLITE, a Windows DLL backdoor recently documented by Acronis. Our copy’s hash isn’t in their IOC list, and as of June 4, most major EDRs (CrowdStrike Falcon, SentinelOne, Sophos, Trellix, Palo Alto, ESET) still don’t flag it as malware. Ire produced a function-by-function behavioral report—install routine, C2 packet layout, command IDs, persistence mechanism, obfuscation—that lines up with Acronis’s published analysis. One decompiler-based run, no human priors.

This is what behavioral, agentic reverse engineering can achieve when signature matching and manual inspections fall short. Variants that share TTPs but not indicators of compromise (IOC) get caught instead of slipping past signature lists. Novel malware classification is a domain with no automatic validator, requiring in-depth investigation and holistic understanding of the software’s behaviors to surface and determine intent. Ire operates without context: no origin metadata, no telemetry, no analyst prompt. It invokes decompilers and binary-analysis tools, builds an auditable chain of evidence, and reaches a malicious-or-benign verdict.

Acronis’s Threat Research Unit (TRU) published a writeup (opens in new tab) on LOTUSLITE, a DLL backdoor delivered through a politically themed ZIP, sideloaded through a renamed Tencent KuGou launcher. They attribute it to Mustang Panda at moderate confidence based on infrastructure overlap and the loader/DLL split. Hunting on VirusTotal for samples whose behavior matched the report, we surfaced one whose SHA-256 doesn’t appear in Acronis’s IOC list.

The sample: 47e51e82229e80a387c3cb100d39d3705e6360bbf9bfa1601dbc484e8d02e653 (opens in new tab). When we picked it up on May 28, VirusTotal showed 1 of 72 vendors flagging it.

A week later, that rose to 7 of 70. The cluster: Microsoft Trojan:Win32/Malgent!MSR, Kaspersky HEUR:Trojan-Dropper.Win32.Dorifel.gen, Rising Dropper.Dorifel!8.31E (CLOUD), Cynet (score 100), Elastic (moderate confidence), Kingsoft, TrendMicro-HouseCall. With Microsoft now flagging, VT’s popular threat label has shifted to dropper.dorifel / malgent. CrowdStrike Falcon, SentinelOne, Sophos, Trellix, Palo Alto, and ESET still miss it. VT lists the file type as pedll (PE DLL) and the filename as SmartPrintScreen.Print.

We analyzed the sample with Ire, using only its decompiler-based tools through a single tool call. Ire’s verdict was “malicious”; you can review the complete report on Github (opens in new tab).

On Ire’s calibration

One noteworthy observation in Ire’s report (opens in new tab) is worth highlighting first. Ire flagged the nfapi::nf_unRegisterDriver and NetFilter naming as suspicious but explicitly did not claim active packet interception. The function in question writes the Run key; it does not install a driver. This is where LLM-driven analysis can go wrong: suggestive strings can steer the verdict. A function called nf_unRegisterDriver sounds like it does kernel-level work, and a less thorough agent would write that into the report. Downstream defenders would then chase a phantom, building detection rules for behavior that may or may not be there. Ire flagged the misleading name and considered the behavior as one piece of the evidence during its final adjudication of malice.

Comparing the two reports

Comparing Ire’s output with Acronis’ report, the sample we analyzed matches the behavioral profile of the LOTUSLITE family of malware. Both show a loader/DLL split, HTTPS C2 carrying a custom binary protocol with a magic DWORD, interactive shell over pipes, directory enumeration, file primitives, chunked upload, HKCU persistence, and traffic camouflaged as Google and Microsoft services. The surface details differ—filenames, paths, magic value—but the underlying behaviors align. Ire correctly identified this sample as part of the same family of malware because of the behaviors it was able to identify through decompilation and reverse engineering, not on string match alone.

Because the sample is a DLL (pedll per VT), the sample’s install routine reads differently than it might look at first. The DLL copies two files into C:\ProgramData\SmartPrint\: the loader EXE that sideloaded it (its host process, obtained via GetModuleFileName(NULL), written as SmartPrintScreen.exe) and itself (AMPV.dll, the analyzed sample). The Run key points at the loader with –DaDaBar. On the next logon, the loader runs and sideloads AMPV.dll from the install path. This is the same Acronis-identified pattern but with different filenames.

This also explains the binary’s strange export surface. The DLL exports a long list of banking and QR-themed names (Query_Bank, BankSepah_Iran, BankToman_BMI, BankofChina, qrBankInit, JpgSymbolToBMP, and others), most of which resolve to a message box or ExitProcess. The shape suggests a hijacked banking/QR SDK shell, repurposed so the host EXE can call any one of those exports via GetProcAddress and reach the LOTUSLITE entry point. Acronis names theirs DataImporterMain. The Ire report does not surface a matching entry-point name, but it identifies that the behavioral shape is the same.

Acronis attributes the malware family to Mustang Panda at moderate confidence based on infrastructure and TTPs we don’t have access to, while our sample directly contains a literal actor-name string “BelievemeIamMustang-Panda” with no obfuscation. A string isn’t direct proof of authorship; it could be a developer artifact, a trophy, or a deliberate plant. While we are not making an attribution call, we note that the binary names the same actor that Acronis named through other means, and we leave the question open. Another consideration to make for this finding: a string like this can function as adversarial input to LLM-driven analysis, biasing the verdict.

Spotlight: AI-POWERED EXPERIENCE

Microsoft research copilot experience

Discover more about research at Microsoft through our AI-powered experience

Start now

Opens in a new tab

Why this matters

Ire statically reverse-engineers binaries and identifies the behavior from the function to the system level to describe what the software does and determine a verdict. The verdict of this sample came from a single Ire run because of the specific detail Ire was able to surface: function roles, packet layout, command IDs, persistence registry keys, and decoy strings. Ire never named LOTUSLITE in its report or chain of evidence. The family mapping is ours, after the fact, comparing Ire’s report against Acronis report. Ire described the behavior precisely enough to make the mapping straightforward of this sample to LOTUSLITE.

Stay up to date on the latest findings and other interesting sample detections from Project Ire by following along on our project page.

The post Ire identifies another LOTUSLITE specimen appeared first on Microsoft Research.

Enterprise teams increasingly rely on AI systems for analytics, but enterprise data workflows are often fragmented across storage systems and tools. Before analysis can begin, teams often need to establish governed connections, prepare metadata, manage permissions, and build workflows for combining and reshaping data across multiple systems.

Beyond data connection, analysis itself remains challenging for analysts and domain experts, many of whom lack deep coding expertise. They frequently need to compute new metrics, compare different ways of organizing data, inspect intermediate outputs, and refine visualizations as needs evolve. These workflows are difficult to reproduce inside isolated chat interactions that lack persistent access to enterprise data, workflow history, and visualization context.

Our new release, Data Formulator 0.7 (opens in new tab), is designed to address these challenges. It is an open-source AI-powered data analysis system that connects fragmented enterprise data and iterative analytical workflows. It provides a lightweight way to connect across a variety of data sources, context-aware agents that assist with data preparation, exploration, and visualization, and an interactive workspace where users can iteratively refine and share their analyses.

Spotlight: Event Series

Microsoft Research Forum

Join us for a continuous exchange of ideas about research in the era of general AI. Watch the latest episodes on demand.

Watch on-demand

Opens in a new tab

Connecting enterprise data with Data Connectors

Data Formulator helps teams bring enterprise data into an AI-ready workspace without needing to rebuild the same connections for every source of data. The Data Connectors feature supports authentication, persistent connections, previews, metadata, and a unified workspace model across databases, warehouses, BI systems, object stores, and local files. This reduces integration work for platform teams and allows users to work from centrally managed, reusable data connections rather than relying on repeated manual file uploads, as shown in Figure 1. 

Context-aware agents for data analysis

Context-aware AI agents form the core of Data Formulator. Unlike a single prompt, Data Formulator gives agents access to the full analysis workspace, including connected data sources, loaded tables, prior charts, and the user’s objective. Agents reason and act through tools rather than text alone. In a single interaction, an agent can inspect data, write and run code in an isolated environment, generate chart specifications, and explain its results while showing intermediate steps.

When a request is ambiguous, the agent asks clarifying questions before proceeding. This allows agents to carry out more complex analytical workflows: aligning analyses with the user’s goal, preparing and transforming data, suggesting follow-up questions, generating tables and charts in batch, and creating verifiable, reproducible code for every result.

A workspace for iterative data analysis

Data Formulator pairs these agents with a multimodal interface designed for open-ended analysis workflows. Users work with agents through the Data Thread, a structured chat that records every question, intermediate finding, and chart throughout the analysis process. Long sessions stay navigable: users can revisit earlier steps, branch into alternative analyses, and compare them side by side without losing context.

As illustrated in Figure 2, the interactive canvas complements Data Thread by allowing users to directly edit visualizations. When users shift from exploration to communication, they can refine charts directly on the canvas or describe changes in natural language and let the agent adjust labels, annotations, layout, color, and emphasis. Analysts can also generate reports and share their findings with others.

View the Data Formulator demo here (opens in new tab), or explore the Data Formulator GitHub repository (opens in new tab). Teams developing analytics workflows for enterprise data can use the project as a foundation for adapting these capabilities to their own systems and requirements.

The post Data Formulator 0.7: AI-powered data analytics for enterprise data appeared first on Microsoft Research.

AI systems today can write essays, generate code, summarize complex ideas, and carry on conversations with remarkable fluency. Yet those same systems still struggle with tasks humans find intuitive: reliably tracking objects through change, reasoning compositionally in unfamiliar situations, or distinguishing truth from plausible fiction. These contradictions have fueled polarized debates about AI. Some see current systems as early forms of human-like intelligence; others dismiss them as sophisticated autocomplete. 

In recent interdisciplinary work – including Adam Frank, Marcelo Gleiser, and Evan Thompson’s The Blind Spot (opens in new tab) and DeepMind researcher Alexander Lerchner’s The Abstraction Fallacy (opens in new tab) – a different picture is emerging. Rather than asking whether AI systems are becoming intelligent in the human sense, these approaches ask a more basic question: What if AI systems work because they rely on structures that are rooted in human cognition? This shift in perspective, which draws on the phenomenology of Edmund Husserl, helps make sense of both the capabilities and the limits of modern AI. 

In our recent paper, The Origins of Artificial Intelligence in Natural Intelligence, we argue that modern AI systems are best understood neither as human minds nor as trivial statistical tricks. Instead, they extend structures that originate in human cognition itself. Further drawing on the phenomenology of Husserl, the paper proposes that language already contains sedimented structures of human understanding —structures that AI systems learn to model and extend. This perspective helps explain both the capabilities and the boundaries of contemporary AI.  

Human perception is not simply passive reception of sensory data. We experience the world as stable things unfolding through change: a cup remains the same cup as we move around it; a melody remains recognizable even as individual notes pass away. Language emerges by expressing these stable structures in conceptual form. Words like “red,” “round,” or “larger than” articulate relationships that originate in lived experience. 

Large language models learn statistical relationships within this linguistic world. They capture how concepts tend to relate across enormous bodies of human writing. This explains why AI systems can produce coherent responses across many domains. But it also explains why they hallucinate. Humans remain answerable to the world: experience continually corrects our expectations and beliefs. AI systems, by contrast, extend patterns within text itself. They can continue a line of reasoning with remarkable fluency, but they lack the lived engagement with the world that anchors meaning and truth.

This framework helps explain several recurring challenges in AI research. One is the “compositionality gap”—the tendency for language models to perform well on familiar reasoning patterns while failing when asked to combine concepts in genuinely novel ways. Research increasingly shows that larger models improve fluency and factual recall much faster than they improve true compositional reasoning. From our perspective, this is not simply an engineering limitation but a structural boundary: AI systems can extend patterns already sedimented in language, but they do not possess the world-directed understanding that allows humans to generate genuinely new conceptual relations. 

A similar pattern appears in multimodal systems that combine language and vision. These systems can often label images correctly while still failing at robust reasoning about objects and their parts. They learn correlations between visual patterns and language rather than perceiving stable objects unfolding through time in the way humans do. The result is systems that can appear impressively fluent while remaining surprisingly brittle outside familiar patterns. 

This perspective also reframes debates about AI safety. Public discussion often swings between fears of “rogue superintelligence” and claims that AI poses little meaningful risk. Our research suggests that both extremes misunderstand the nature of current systems. The most immediate risks arise not because AI possesses human-like intentions, but because it can extend patterns of reasoning without reflective responsibility to the world. Systems can generate persuasive but ungrounded outputs, automate flawed decisions at scale, or execute harmful actions if embedded in poorly governed environments.

This helps explain why AI safety is increasingly shifting from model safety to system safety. In practice, organizations already rely on layered safeguards—what the industry increasingly calls “harnesses”—to constrain, validate, and monitor AI behavior. Rather than temporary patches, our paper argues that these mechanisms reflect something fundamental about AI architecture itself: trustworthy behavior emerges from the work of builders of AI systems responsible for their behavior, a responsibility that cannot be delegated to or shared with models.

This interpretation aligns closely with how enterprises increasingly approach trustworthy AI deployment. Organizations need systems that can extend human intelligence while remaining governable, auditable, and aligned with human oversight. Understanding AI as a derived form of intelligence clarifies why layered governance, evaluation, and operational controls matter so deeply.

Spotlight: AI-POWERED EXPERIENCE

Microsoft research copilot experience

Discover more about research at Microsoft through our AI-powered experience

Start now

Opens in a new tab

Looking ahead, we believe phenomenology offers more than a critique of AI—it offers a framework for understanding its promise. AI systems reveal something profound about human cognition itself: that meaning can be formalized, extended, and scaled in powerful new ways.  The central societal risk of AI thus turns out to be kicking away the ladder of its origins in human experience and cognition – misinterpreting AI as a rival intelligence that diminishes our humanity and thus, in turn, diminishes the true promise of AI itself. 

The question, then, is not whether AI will replace human intelligence. It is how we can responsibly build systems that extend human understanding while remaining grounded in the world from which that understanding arises. If we mistake AI systems for autonomous minds, we risk over-trusting them. If we dismiss them as trivial tricks, we risk overlooking one of the most important technological developments of our time. A more grounded interpretation recognizes both truths at once: AI is a genuine extension of human intelligence—and precisely because of that, humans remain responsible for how it is understood, governed, and used.

The post Extending Human Intelligence Through AI appeared first on Microsoft Research.

Today, Microsoft Research AI Frontiers releases MagenticLite (opens in new tab), an experimental agentic application designed for small models. As the next generation of Magentic-UI, it works across the browser and local file system in a single workflow.

MagenticLite is powered by two purpose-built models: MagenticBrain, for reasoning, delegation, and terminal use, and Fara1.5, a computer-use model family for browser-based tasks. The three components were designed to work together as a single system. The result is an agent that runs efficiently, keeps data on the user’s machine, and supports a broad range of agentic tasks. It also points toward a broader goal: capable agents that can run directly on users’ hardware.

The project is built around a key research bet: that agentic capability depends on tool orchestration and action rather than knowledge alone. That insight makes it possible to use smaller models while still enabling a broad range of agentic tasks at a fraction of the cost.

MagenticLite also reflects how we approach agentic AI end-to-end—from training data and model design to orchestration, interaction design, and human oversight throughout the experience.

Included in this release

MagenticLite (opens in new tab)

The next generation of Magentic-UI, our experimental agentic experience, is powered by an agent harness rebuilt for small models, with an updated user interface informed by community feedback. It works across users’ browsers and local file systems in a single workflow.

MagenticBrain (opens in new tab)

MagenticBrain is MagenticLite’s planner, coder, and delegator in one. It turns vague requests into concrete plans, selects the right tool or subagent for each step, writes code when needed, and recovers should something break mid-task. 

Fara1.5

The next generation of our computer-use model family, Fara1.5 comes  in three sizes, with a flagship 9-billion-parameter model for most use cases. Fara1.5 sets new state-of-the-art (SOTA) results among small computer-use models and nearly doubles Fara-7B’s performance on web navigation, with sharper handling of forms, credentialed sites, and long-running tasks.

Each component is useful on its own, but they work best together. Codesigning the app, models, and the harness enables capable and reliable agentic performance at this scale.

Our research approach: Doing more with less

We started with a simple question: what does it take to make a small model genuinely good at agentic tasks? The answer spanned the full lifecycle—data generation, training objectives, model design, and orchestration had to be redesigned together rather than in isolation.

We identified requirements from real-world use cases like filling out forms, conducting browser research, and managing files locally, and built an evaluation dataset around them. Standard benchmarks capture part of the picture, but they are not always a direct measure of real-world usefulness. Scenario-based evaluations complemented those benchmarks and became a key signal for iterative improvement across both the models and the harness, as shown in Figure 2.

For the user experience, we retained key elements from Magentic-UI, including visibility into the agent’s reasoning and actions, the ability for users to take direct control, and explicit approval at critical points. Based on recent user studies, we also made MagenticLite easier to learn and collaborate with through updated browser and chat views, designed to make it easier for users to understand the agent’s actions and intervene when needed. This is illustrated in Figure 3.

PODCAST SERIES

The AI Revolution in Medicine, Revisited

Join Microsoft’s Peter Lee on a journey to discover how AI is impacting healthcare and what it means for the future of medicine.

Listen now

Opens in a new tab

System components

Fara1.5: A computer-use model that outperforms its weight class

Fara1.5 is the next generation of our computer-use model family, which is available in three sizes, with a flagship 9B model recommended for most use cases. Fara1.5 achieves new SOTA performance among small computer-use models and nearly doubles Fara-7B’s performance on web navigation, with better handling of forms, credentialed sites, and long-running tasks.

Last November, we released Fara-7B, a small agentic model built for completing tasks in a web browser. It was trained using a novel synthetic data generation engine that enabled best-in-class performance. Fara1.5 is the next step in that bet: a family of three models (4B, 9B, 27B) based on Qwen 3.5, designed to close the gaps we saw in the prior release.

What’s new

State-of-the-art results. On the popular Online-Mind2Web benchmark, which contains 300 tasks across widely used web domains, Fara1.5 sets new SOTA results for models in its size class. Fara1.5 outperforms all similarly sized models and nearly doubles the performance of Fara-7B. The larger Fara1.5-27B variant achieves more than 90% performance on the same benchmark.

Improved user experience. In addition to improvements on benchmarks, we improved the user experience of Fara1.5. Users should observe stronger performance on everyday tasks like filling out forms, handling logins for credentialed sites, and booking appointments. These improvements are driven by the next evolution of our FaraGen data generation pipeline. Alongside training on live websites, we also trained the model on highly realistic synthetic environments designed to simulate scenarios like logins and irreversible actions.

A native action space tuned for long-running tasks. Beyond clicks and keyboard actions, Fara1.5 has built-in tools to store key information in its context across hundreds of steps and ask the user for permission or preferences when needed, helping it stay coherent on tasks that span many minutes of real work.

Recalibrated critical points. Fara-7B was trained to detect critical points for activities like transactions, login flows, or irreversible submissions and flag them. In Fara1.5, we refined our design around critical points based on our learnings from real use, so safety triggers still occur when they should but do not block useful tasks, such as form-filling.

MagenticBrain: The orchestrator model

MagenticBrain is a 14B-parameter orchestration model—planner, coder, and delegator in one. Fine-tuned from Qwen 3 14B, MagenticBrain was trained end-to-end within the MagenticLite harness with the same tool schemas and execution environment it will encounter at inference time. As a result, there is no gap between how it learned to orchestrate and how it runs.

In many agentic systems, orchestration (planning and coordination) is the most reasoning-intensive component, so teams have historically relied on their most capable models for this role. Our bet is that small models can handle this role without sacrificing capability. Two design choices make that possible.

The first involves combining multistep tool-calling trajectories—where the model learns to pick the right tool and call it correctly—with coding and terminal trajectories—where the right answer is sometimes five lines of Python, not a tool call. This is paired with tight coupling between the tool format used during training and inference.

The second is computer-use agent (CUA) delegation. A key part of the orchestrator’s job is knowing when not to act itself and instead handing off a task to Fara1.5. Our data pipeline includes explicit delegation trajectories: sequences where the orchestrator recognizes a browser or user interface (UI) task, issues a structured handoff to the CUA model, waits for the result, and resumes the task. The result is an orchestrator model that reasons, codes, calls tools, and delegates fluidly within a single 14B footprint. We are releasing MagenticBrain which is designed for use with MagenticLite. 

The Harness: Built for small models

The harness combines the orchestrator and browser-use models into a single workflow. Three design choices matter most:

• Step-by-step planning. The harness plans incrementally, keeping the system flexible and enabling smoother course correction and recovery throughout long-running tasks.
• Active context management. Small models have smaller effective context windows and degrade faster as context grows. The harness actively curates what each model receives at each step, keeping prompts focused, surfacing only the necessary information, condensing earlier interactions into concise summaries, and offloading the rest, so the orchestrator and Fara1.5 remain effective across long tasks.
• Delegation through subagents. Rather than relying on a single small model for every task, the orchestrator acts as the main agent and delegates specialized work to subagents. This means handing off browser tasks to Fara1.5. This pattern plays to the strengths of small language models by allowing each model to handle a narrower, more specialized part of the problem. It also lays the foundation for future expansion: later versions could introduce additional subagents and run them in parallel for richer, more efficient workflows.

The harness preserves the human-in-the-loop guarantees from Magentic-UI 1.0. Critical points across both browser and code actions still pause for explicit user approval, and the entire system runs inside Quicksand (opens in new tab), an open-source wrapper created for a QEMU-based sandbox, which isolates browser sessions and code execution from the host system.

See it in action

MagenticLite can perform a wide range of tasks across the browser and local file system, such as filling out forms, making appointments, organizing local files, and searching for and analyzing information.

Try it, and build with us

MagenticLite, MagenticBrain, and Fara1.5 are research releases intended to support continued exploration and development. We are releasing them to encourage experimentation, evaluation, and feedback from the broader community.

• MagenticLite is an updated release of Magentic-UI, it’s available on GitHub (opens in new tab). 
• MagenticBrain is available on Microsoft Foundry (opens in new tab). 
• Fara1.5 models are available on Microsoft Foundry (opens in new tab). 

Contributors

• Agentic experience: Cheng Tan, Maya Murad, Weili Shi
• Agentic harness: Adam Fourney, Tyler Payne
• Fara1.5: Alexey Taymanov, Andrew Zhao, Aravind Rajeswaran, Corby Rosset, Hussein Mozannar, Luiz Do Valle, Spencer Whitehead, Vibhav Vineet, Zach Nussbaum, Sahil Gupta, Yadong Lu
• MagenticBrain: Ahmed Elgohary Ghoneim, Akshay Nambi, Amir Saeidi, Caio César Teodoro Mendes, Harkirat Behl, Karan Gupta, Pashmina Cameron, Pranav Vajreshwari, Shital Shah, Yash Lara, Yash Pandya
• Collaborators: Abhishek Gowami, Amanda Swearngin, Michael Harrison, Sara Abdali, Sarthak Harne, Vidhisha Balachandran
• Project leads: Ahmed Awadallah, Rafah Hosn
• Sponsors: Ahmed Awadallah, Ece Kamar, Rafah Hosn, Saleema Amershi, Shital Shah

The post MagenticLite, MagenticBrain, Fara1.5: An agentic experience optimized for small models appeared first on Microsoft Research.