Start light, scale intentionally: Choosing the right Microsoft agent architecture

There’s a moment in almost every organization’s AI journey when experimentation gives way to something more difficult: Deciding which tool to rely on to build mission-critical agentic systems. These mainline systems operate across teams, connect to governed data, and accomplish real tasks for your workforce.

At Microsoft, we’re developing a growing agent ecosystem that offers diverse options for your workplace demands. This collection of tools acknowledges that there is no single “right” way to build an agent.

“The best agent architecture isn’t the most powerful one. It’s the simplest one that delivers value today while giving you the flexibility to grow tomorrow.”

Patrice Pelland, partner engineering manager, HR Employee Experience

Microsoft 365 Copilot, declarative agents, Copilot Studio, Microsoft Fabric data agents, and Microsoft Foundry agent service all represent valid patterns that are optimized for different kinds of work. Newer ideas like scout-style personal agents and Frontier Tuning (how we fine-tune our agent compliance) offer even more flexibility.

When choosing the best agentic technology for you, the rule of thumb is simple: Select the lightest architecture that fits your user experience, data needs, action model, and governance requirements.

“The best agent architecture isn’t the most powerful one,” says Patrice Pelland, a partner engineering manager in HR Employee Experience. “It’s the simplest one that delivers value today, while giving you the flexibility to grow tomorrow.”

Start with the surface, not the stack

The fastest way to get agent architecture wrong is to begin with technology instead of the workload. A better starting point is the user surface—where the work happens and how people expect to interact with it.

Our agent ecosystem is broad, but each of the core options map cleanly to a different kind of challenge:

• Microsoft 365 Copilot is the default entry point for AI in daily work, embedded across apps like Teams, Outlook, and Word. It excels at summarization, drafting, and reasoning over Microsoft Graph data without requiring users to leave their most familiar tools.
• Microsoft Copilot Cowork takes Microsoft 365 Copilot beyond just answering questions to actually executing on work assignments. Designed for complex, multi-step tasks, it can plan, coordinate, and carry out work across Microsoft 365 applications and connected systems while keeping users in control through approval checkpoints and oversight.
• Declarative agents extend Microsoft 365 Copilot with scoped instructions, knowledge, and lightweight actions. They’re the simplest way to create focused, role-based assistants while staying entirely within the Copilot experience.
• Copilot Studio agents provide a low-code environment for building workflow-driven agents with connectors, triggers, approvals, and multi-channel delivery. They’re best suited for business processes that need structure, governance, and integration across systems.
• Microsoft Foundry Agent Service is a code-first platform for building and operating custom agents with full control over models, orchestration, and runtime. It fits engineering-heavy scenarios that require scalability and tailored architecture.
• Microsoft Fabric data agents specialize in governed analytics over business data, using semantic models, ontologies, and curated datasets. They are designed to answer questions about KPIs, entities, and operations with trusted, auditable context rather than general document retrieval.
• Microsoft Scout and Autopilots introduce always-on personal agents that can coordinate work in the background across Microsoft 365. They focus on continuity—tracking commitments, preparing context, and proactively surfacing what matters next.
• Frontier Tuning enables organizations to teach agents how work actually gets done using company-specific data, workflows, and evaluation signals. It’s a later-stage capability that refines behavior inside compliance boundaries, rather than serving as a starting point.

Taken together, these are layers in a system. The practical decision is to pick the one closest to the user.

Picking the right agentic tool

Use Microsoft 365 Copilot as the default front door for broad productivity and Microsoft 365-grounded experiences. Extend it with declarative agents, connectors, plugins, or APIs when a scenario needs more domain-specific knowledge or actions. Do not treat plain Copilot as the primary architecture when the real need is a departmental specialist, governed workflow, analytics specialist, custom channel, or custom runtime. Those needs fit Copilot Studio, Fabric, or Foundry patterns better.

Use Microsoft Cowork when you want to complete a meaningful piece of work across multiple steps, files, applications, and systems. Cowork is best suited for long-running tasks that require planning, orchestration, and execution over time, while remaining grounded in Microsoft 365 context. Do not use Cowork as a replacement for workflow platforms, analytics specialists, or custom agent architectures. When a task requires governed business processes, specialized analytics, or deep runtime customization, Cowork should coordinate with other agentic tools rather than replace them.

Use Copilot Studio when governed workflow, connectors, approvals, and channel reach matter as much as the chat experience. Do not use it as the default for a small Microsoft 365 Copilot extension or for engineering-first scenarios that require custom model choice, runtime hosting, private networking, or deep orchestration; those fit declarative agents or Foundry better.

Use Microsoft Foundry when low-code or Microsoft 365-native layers hit real limits around orchestration, runtime control, model selection, deployment architecture, or engineering operations. Do not start here for a simple business workflow, no/low-code service agent, or focused Microsoft 365 assistant unless custom runtime control is truly the differentiator.

Use Fabric data agents when you’re looking for analytics expertise in a broader architecture. Fabric IQ, semantic models, and ontology can help translate raw data into business meaning. Do not use them as a general enterprise assistant or workflow engine: they are read-only, analytics-focused, and not intended for broad action-taking, unstructured document help, or custom model/runtime control.

Use Microsoft Scout when you need a personal front-door and coordination layer. It should not serve as a replacement for every enterprise agent architecture. When work crosses into governed process, business analytics, custom orchestration, or domain systems, Scout should delegate to Copilot Studio, Fabric data agents, MCP tools, or Foundry specialists.

Use Frontier Tuning only after the basics are in place: clear workflows, governed data, meaningful evaluation signals, and operational maturity. It is a later-stage optimization for institutional know-how, not the first step for a lightweight assistant or a substitute for sound architecture, data governance, or human-in-the-loop controls.

AI-ready data: The foundation of effective agents

The success of an AI agent depends less on the model it uses and more on the quality of the data that grounds it. Agents can only reason effectively when they have access to information that is trusted, discoverable, secure, and connected to the broader decision-making context. Organizations that invest in AI-ready data create the foundation for more reliable and useful AI experiences.

“The competitive advantage in the age of AI won’t come from access to models alone. Organizations also need to prioritize AI-ready data, which allows agents to reason accurately, securely, and at enterprise scale.”

Ravi Goripathy, principal software engineering manager, HR Employee Experience

AI-ready data goes beyond standard measures of data quality, such as accuracy, completeness, and consistency. It also includes the semantics, governance, and contextual information that help AI systems understand what the data means, how it should be used, and the way it relates to other information across the business. High-quality data is essential, but AI systems need data that is also meaningful, connected, and trustworthy.

As organizations deploy more copilots and agents, data readiness increasingly becomes the limiting factor for success, rather than model availability. A strong AI-ready data foundation helps agents produce better results, apply business context consistently, and operate within appropriate security and governance boundaries. The most effective AI strategies treat data readiness as a core architectural requirement.

“The competitive advantage in the age of AI won’t come from access to models alone,” says Ravi Goripathy, a principal software engineering manager in HR Employee Experience. “Organizations also need to prioritize AI-ready data, which allows agents to reason accurately, securely, and at enterprise scale.”

Building reliable agent systems

As soon as an agent moves from answering questions to taking actions, architecture matters more.

A useful pattern is to separate responsibilities into layers:

MCP tools play a key role here by exposing actions as controlled, reusable capabilities. Instead of giving an agent broad access to systems, you expose bounded functions with defined inputs, permissions, and telemetry. This improves reliability, governance, and auditability.

Similarly, not every interaction belongs in free-form chat. Tasks that require review, approval, or comparison—like approving a budget or evaluating candidates—benefit from structured UI components. MCP apps let you surface those experiences inside conversational flows without rebuilding entire applications. Rather than sending users to a separate system to complete a task, MCP apps can render forms, grids, dashboards, and other transactional experiences directly within the conversation, enabling users to review information, approve requests, update records, and take action in context.

As solutions mature, new patterns emerge. Always-on personal agents—like Scout-style Autopilots—can monitor commitments, prepare context, and proactively surface issues. In parallel, Frontier Tuning introduces a way to teach agents how work actually gets done in a specific organization, using real workflows, evaluation signals, and internal knowledge.

These are powerful capabilities, but they’re not starting points. They’re extensions of an architecture that already works.

A layered approach beats a ‘super agent’

It’s tempting to imagine a single agent that handles context, analytics, workflows, and actions. In practice, a more durable approach is a set of loosely connected specialists that each focus on a specific responsibility.

A Microsoft 365–native assistant can serve as the front door for user interaction, delegating analytics to Fabric data agents, actions to Copilot Studio workflows or MCP tools, and more complex orchestration to Foundry-based agents when needed. This separation makes systems easier to govern, test, and evolve over time.

Before moving to production, architecture should include clear ownership, permission design, governance controls, human approval for sensitive actions, evaluation processes, and monitoring. These are all essential aspects of building an agent system that can be trusted.

Deciding where to start

When faced with multiple valid options, a simple decision path helps:

• If a packaged Microsoft experience already meets most of the need, use it.
• If the experience should live in Microsoft 365, start with Copilot and a declarative agent.
• If the core problem is workflow—approvals, routing, integration—lean toward Copilot Studio.
• If the value depends on governed business data, bring in Fabric data agents early.
• If you need deep control over orchestration, models, or runtime behavior, step up to Foundry.

And across all of these: add structure only where the workload demands it. Use MCP tools for actions, structured UI where chat breaks down, and human review where risk increases.

Most importantly, resist the urge to start at the most complex layer. Many successful solutions begin as simple Microsoft 365 extensions and evolve incrementally into multi-layered systems.

Looking ahead: From tools to systems of work

The trajectory of agent architecture is clear. We’re moving from isolated assistants toward interconnected systems that combine personal productivity, business data, workflows, and continuous coordination.

“Agentic AI is shifting from isolated capabilities to connected systems of work. The organizations that lead will be those that design for evolution—starting simple, learning continuously, and scaling with intention.”

Ravi Goripathy, principal software engineering manager, HR Employee Experience

In the near term, that means more layering: personal agents coordinating work, workflow agents handling processes, analytics agents grounding decisions, and platform agents orchestrating across systems. Over time, capabilities like Frontier Tuning will make these systems more adaptive, learning from how organizations actually operate while staying within governance boundaries.

But the core principle is unlikely to change. The most successful teams won’t be the ones who adopt the most advanced architecture first. They’ll be the ones who start with the smallest viable solution, validate it in real workflows, and scale intentionally—adding complexity only when it delivers clear value.

“Agentic AI is shifting from isolated capabilities to connected systems of work,” Goripathy says. “The organizations that lead will be those that design for evolution—starting simple, learning continuously, and scaling with intention.”