There was excitement, curiosity, skepticism, and no shortage of questions about what this technology meant for the future of IT.

At Microsoft Digital—the company’s IT organization—we didn’t start with a grand transformation plan. Instead, we started with a realization: AI wasn’t just another tool to roll out. It was a fundamental shift in how engineering work could happen.

For years, our IT teams have been focused on scale, reliability, and operational excellence. Those priorities didn’t change. What changed were the possibilities.

Suddenly, engineers could draft code in seconds, summarize complex systems instantly, or automate work that had once consumed hours or days. It was an opportunity to take the skills and capabilities of our people and amplify them with AI.

That realization forced us to step back and ask harder questions.

How do you help thousands of engineers understand what AI can actually do to impact their day-to-day work? How do you move from experimentation to trust? And how do you adopt AI in a way that strengthens engineering fundamentals instead of eroding them?

The answer came in the form of a phased journey grounded in people, culture, and continuous learning.

Phase 1: Awareness and access

It might sound surprising when speaking about engineering processes, but our first challenge wasn’t technology; it was understanding.

When generative AI entered the conversation, most engineers saw the headlines and dabbled in various tools, but few understood fully what it meant for their work. Some were excited, others were wary. Many simply didn’t know where to start. That gap between awareness and practical value was the first barrier we had to address.

We realized early that top-down mandates wouldn’t work. Telling engineers to “use AI” without context or relevance would only deepen skepticism. Instead, we focused on something both simpler and more difficult: Exposure.

We started by making AI visible and accessible in the tools engineers already used. GitHub Copilot. Microsoft 365 Copilot. Early copilots embedded directly into engineering workflows. The goal wasn’t immediate productivity gains. It was familiarity. Letting engineers see, firsthand, what AI could and couldn’t do.

“We encouraged tool usage and adoption so people would at least play around with AI. And once they did, they started seeing the value. That’s when the mindset shifted from ‘AI might replace me’ to ‘AI can be my companion.’”

Mukul Singhal, partner group engineering manager, Microsoft Digital

Just as important, we talked openly about limitations.

AI wasn’t perfect. It hallucinated. It made confident mistakes. And that honesty mattered. By framing AI as an assistant, we reinforced the role of engineering judgment. Engineers didn’t need to fear losing control. They needed to understand how to stay in control.

We also made experimentation safe.

No quotas. No forced adoption metrics. Engineers were encouraged to try AI on low‑risk tasks: summarizing documentation, generating test cases, or exploring unfamiliar codebases. Small wins built confidence, confidence built curiosity, and curiosity drove organic adoption.

As that experimentation took hold, the mindset began to shift.

“We encouraged tool usage and adoption so people would at least play around with AI,” says Mukul Singhal, a partner group engineering manager in Microsoft Digital. “And once they did, they started seeing the value. That’s when the mindset shifted from ‘AI might replace me’ to ‘AI can be my companion.’”

Over time, conversations changed from ‘Should we use AI?’ to ‘Where does AI help most?’

Engineers began sharing prompts, tips, and lessons learned with one another. What started as individual exploration turned into community learning. Awareness gave way to momentum.

Phase one was about providing access to explore, to question, and to learn. And that foundation made everything that followed possible.

Phase 2: Culture shift

Access created awareness and awareness created curiosity.

As more engineers began experimenting with AI, we noticed a pattern. Some teams were moving faster, learning faster, and reducing friction in their day‑to‑day work. Others stalled after initial trials. The difference wasn’t technical skill or capability, it was mindset.

“People started shifting from the mindset of ‘Will AI work?’ to ‘AI is working for me.’ I think that was a very transformational shift, to where I believe a lot of engineers in the organization started believing in AI.”

Veera Mamilla, principal group engineering manager, Microsoft Digital

To move forward, we had to shift how AI was perceived from something optional or experimental to something that was simply part of how modern engineering gets done.

That meant normalizing AI as a trusted partner in the engineering process.

Leaders played a critical role in that shift. Rather than positioning AI as a productivity shortcut, they framed it as a way to strengthen engineering fundamentals: clearer design discussions, better documentation, faster feedback loops, and more time for deep problem‑solving. The message was intentional and consistent. Using AI wasn’t about cutting corners, it was about reimagining how work gets done.

We also had to address a fear that surfaced early: that AI adoption was a signal of replacement rather than empowerment.

“People started shifting from the mindset of ‘Will AI work?’ to ‘AI is working for me,’” says Veera Mamilla, a principal group engineering manager in Microsoft Digital. “I think that was a very transformational shift, to where I believe a lot of engineers in the organization started believing in AI.”

That framing mattered.

As engineers incorporated AI into their workflows, success stopped being measured by output alone. The focus shifted to outcomes. Did AI help you understand a system faster? Did it surface risks earlier? Did it free up time to focus on higher‑value work?

Over time, AI stopped feeling like a novelty. It became part of the engineering fabric. We reinforced it through leadership modeling, peer learning, and shared success stories. Teams no longer asked whether AI belonged in their workflows. They asked how to use it responsibly and effectively.

Phase 3: Upskilling and role evolution

Once AI moved from curiosity to expectation, the challenge of skill building became unavoidable.

From the start, we made a deliberate choice: This would be an upskilling and reskilling journey, not a wholesale replacement of roles. The goal wasn’t a new workforce. It was an investment in the one we had.

That decision shaped everything that followed.

Early upskilling efforts focused on practical entry points. Prompt engineering. Tool literacy. Understanding how copilots and early agents behaved in real engineering workflows. We treated these as something every engineer needed to experiment with, regardless of discipline.

But it quickly became clear that skills alone weren’t the full story. Roles themselves were starting to evolve.

“Your title might still be software engineer or principal engineer. But if you’re acting like an AI engineer, what does that actually mean? That question helped us start defining how these roles were evolving.”

Ragini Singh, partner group engineering manager, Microsoft Digital

Across software development, service engineering, and cloud network engineering, the work was shifting from manual execution toward orchestration and oversight. Engineers were no longer expected to do every task end‑to‑end by hand. Instead, they were learning how to guide AI, review its output, and decide where automation made sense and where it didn’t.

As part of this shift, we began researching how the industry itself was redefining engineering roles. Leaders examined emerging job descriptions from across the market and compared them with Microsoft’s own role frameworks. At the time, there was no formal “AI engineer” role in the internal job library. Rather than creating a new title, the focus stayed on evolving expectations within existing roles.

The idea of an “AI‑native engineer” emerged not as a job description, but as a mindset.

An AI‑native engineer still understands systems, architecture, and risk. What’s different is how that expertise gets applied. Routine tasks are delegated to AI. Judgment, design, and accountability stay with the human. Engineers move from doing all the work themselves to supervising work done in partnership with AI.

“Your title might still be software engineer or principal engineer,” says Ragini Singh, a partner group engineering manager in Microsoft Digital. “But if you’re acting like an AI engineer, what does that actually mean? That question helped us start defining how these roles were evolving.”

This evolution looked different across disciplines. Software engineers focused on AI‑assisted coding, test generation, and spec‑driven development. Service engineers leaned into AI for incident response, knowledge capture, and operational decision support. Cloud network engineers began moving from manual intervention toward intelligent orchestration and agent‑assisted troubleshooting. The common thread wasn’t identical tooling, it was a shared shift toward higher‑order work and reduced toil.

Phase 4: Embedding AI across the engineering lifecycle

By this phase, we knew individual productivity gains were simply the starting point for larger and broader benefits.

Early on, most AI usage showed up in familiar places: Code suggestions, documentation summaries, quick answers. Useful, but fragmented. The bigger opportunity emerged when we stepped back and asked a harder question: What would it look like if AI were embedded across the entire engineering lifecycle, not just used at isolated moments?

We stopped thinking in terms of tools and started thinking in terms of flow. Design. Build. Test. Deploy. Operate. Improve. AI needed to show up across all of it, in ways that reinforced how engineers already worked.

“If AI is only showing up at one step, you don’t get the full value. The real impact comes when it’s integrated across the lifecycle, where engineers can design, build, operate, and learn faster as a system.”

Sudhakar Sadasivuni, principal group engineering manager, Microsoft Digital

In software engineering, that meant pulling AI earlier into the process. We began using it to help draft requirements, reason through design options, and review code with broader system context to accelerate how quickly we could get to informed decisions. Coding assistance mattered, but it was no longer the center of gravity.

Testing and quality followed a similar pattern. AI supported test generation, defect analysis, and code review, reducing repetitive effort and helping issues surface sooner. That gave engineers more time to focus on quality and architecture instead of cleanup.

In service engineering, we embedded AI into incident management and operational workflows. Engineers used it to summarize incidents, surface relevant knowledge, and analyze signals across systems. In cloud network engineering, AI helped shift work away from manual intervention toward orchestration and intelligent troubleshooting. Across disciplines, the principle stayed the same: AI should reduce friction, not introduce it.

As we scaled this approach, one thing became clear. Embedding AI wasn’t just a technical exercise. It was a systems change.

“If AI is only showing up at one step, you don’t get the full value,” says Sudhakar Sadasivuni, a principal group engineering manager in Microsoft Digital. “The real impact comes when it’s integrated across the lifecycle, where engineers can design, build, operate, and learn faster as a system.”

As AI became part of core workflows, engineers remained accountable for outcomes. AI output was reviewed, tested, and validated like any other engineering input. Embedding AI didn’t lower the bar for rigor. It raised expectations around judgment, oversight, and data quality. We became more deliberate about responsibility and governance.

Over time, these integrations created compound benefits.

Faster design cycles reduced downstream rework. Better testing lowered operational noise. Improved operational insight shortened recovery times. AI stopped being something we used occasionally and became something the engineering system itself was built around.

Phase 5: Eliminating toil and accelerating outcomes

At some point, every AI story hits the same test. Does it actually make engineers’ days better? For us, that proof showed up fastest in elimination of toil.

Across Microsoft Digital, engineers have always spent time on work that was necessary but draining. It included tasks such as manual troubleshooting, repetitive diagnostics, log analysis, and routine operational tasks that kept systems running but didn’t move the organization forward.

AI gave us a chance to change that.

“Toil reduction is the biggest thing. That’s where engineers’ eyes light up. If we can eliminate toil, people engineers will flock to use AI. I really believe it.”

Beth Garrison, principal cloud network engineer, Microsoft Digital

In cloud network engineering, for example, troubleshooting used to require manually reconstructing what happened, such as logging into devices, chasing configurations, and piecing together context after the fact. As we began introducing agents and machine learning into these workflows, that work shifted. Instead of spending time assembling the picture, engineers could generate the views they needed faster and focus on resolving issues.

The same shift showed up in how we used operational data.

Rather than reacting to incidents after impact, we started using machine learning to analyze logs, identify patterns, and surface anomalies earlier. That moved teams from reactive response toward proactive monitoring and prevention.

One thing became clear very quickly: Toil reduction wasn’t just a benefit; it was the catalyst for adoption.

“Toil reduction is the biggest thing. That’s where engineers’ eyes light up,” says Beth Garrison, a principal cloud network engineer at Microsoft Digital. “If we can eliminate toil, people engineers will flock to use AI. I really believe it.”

Service engineering followed a similar arc.

Across governance, operations, productivity, and cost management, we began applying agents and automation to simplify complex work and reduce manual review cycles. Governance and compliance workflows became faster and more consistent. Operational processes benefited from guided remediation and earlier insight. Knowledge capture improved as documentation and remediation guidance could be generated and updated automatically.

When we removed repetitive work such as manual triage, rote diagnostics, endless documentation cleanup, we transformed how engineers spent their time. More focus on design. More proactive problem‑solving. More energy directed toward improving systems instead of just maintaining them.

Toil reduction made the value of AI tangible. It’s the moment AI stopped being interesting and became indispensable, and our engineering teams started asking where else we can apply it next.

Measuring what matters

By the time AI was embedded across our engineering lifecycle, a new question came into focus: “How do we know it’s working?”

In the early days, we paid close attention to usage. Which tools engineers were trying, where adoption was growing, or where it stalled. Those signals mattered and adoption was the leading indicator that people were getting comfortable and starting to integrate AI into real work.

“Adoption was always the starting point. But we were clear from the beginning that usage isn’t the destination. The real goal is impact; more time for engineers to focus on the work that truly matters.”

Ullas Kumble, principal group software engineering manager, Microsoft Digital

But using AI doesn’t automatically mean better outcomes. So, we shifted the conversation and started asking, “What’s different now that our engineers are using AI?”

That change reframed how we thought about measurement. We began looking beyond tool activity to understand impact across the engineering system. Faster design cycles. Earlier defect detection. Reduced time spent on repetitive operational work. Shorter incident resolution. Clearer documentation. Fewer handoffs. Less rework.

These weren’t abstract metrics. They showed up in the flow of work.

We were intentional about not forcing a single definition of value across every role. Software engineers, service engineers, and cloud network engineers experience impact differently. What mattered was that each team could point to tangible improvements in how work moved through the system.

That perspective shaped how leadership talked about success.

“Adoption was always the starting point,” says Ullas Kumble, a principal group software engineering manager at Microsoft Digital. “But we were clear from the beginning that usage isn’t the destination. The real goal is impact; more time for engineers to focus on the work that truly matters.”

Over time, this approach changed the quality of our conversations. Instead of debating whether AI was worth the investment, teams talked about where it was removing friction and where it still wasn’t delivering enough value. Measurement became a tool for learning and prioritization.

Moving forward

Looking ahead, one lesson stands out: this journey isn’t complete.

AI tools will continue to evolve. Agents will become more capable. Roles will keep shifting. What it means to be an engineer will continue to change. And that means our approach must stay grounded in the same principles that guided us from the start: invest in people, reinforce fundamentals, embed AI into real workflows, and stay honest about what’s working and what isn’t.

We didn’t set out to build an AI‑driven engineering organization overnight, we built it phase by phase.

By meeting engineers where they were
By reshaping culture before redefining roles.
By embedding AI across the lifecycle, not bolting it on.
By reducing toil and measuring impact where it mattered most.

The result is better engineering: powered by AI, guided by human judgment, and built to keep evolving.

Key takeaways

Here’s a set of approaches you can take to establish AI-led engineering for your organization:

• Start with access and understanding. Give engineers safe, easy access to AI in the tools they already use so curiosity and confidence can develop organically before you push for outcomes.
• Frame AI as a partner, not a replacement. Position AI as an assistant that strengthens engineering judgment and fundamentals rather than a shortcut or a threat to roles.
• Normalize experimentation without pressure. Encourage low‑risk experimentation and peer sharing instead of mandates, allowing adoption to grow through visible, practical wins.
• Invest in upskilling. Focus on evolving skills and expectations within existing roles so engineers learn how to guide, review, and stay accountable for AI‑assisted work.
• Embed AI across the full engineering lifecycle. Look beyond isolated productivity gains and integrate AI into design, build, test, operate, and improve workflows to unlock system‑level impact.
• Measure impact where engineers feel it. Move past usage metrics and track outcomes like reduced toil, faster feedback, and improved flow so teams can see where AI is truly making work better.

Try it out

Try GitHub Copilot.

Related links

• Learn how an AI-led SDLC builds an end-to-end agentic software development lifecycle with Azure and GitHub.
• Discover why developers will lead AI transformation across the enterprise.

We’d like to hear from you!

• Want more information? Email us and include a link to this story and we’ll get back to you.

The post Powering the new age of AI-led engineering in IT at Microsoft appeared first on Inside Track Blog.

They dubbed it the “Scream Test.”

“We didn’t have a great server inventory and tracking system, and we didn’t always know who owned a server,” says Brent Burtness, a principal software engineer in Commerce Financial Platforms, who was one of the leaders for the effort in his group. “So, we essentially just turned them off. If someone screamed—‘Hey, why’d you turn off my server?’—then we’d know it was still being used.”

Today, the basic idea behind the Scream Test is being used across the company, but in a more holistic way. Importantly, it’s been incorporated into the overall lifecycle management of our computing infrastructure. And, through the automation tools provided by Microsoft Azure, we have a much more efficient process for making sure that we’re saving time and money by reducing the number of underused machines we operate, monitor, and maintain.

“We thought we were going to get rid of a small number of machines that weren’t being used. But we found the actual share was about 15% of all machines, which saved us a lot of effort of moving those unused machines to the cloud. In other words, we downsized on the way to the cloud, rather than after the fact.”

Pete Apple, cloud network engineering architect, Microsoft Digital

Uncovering more than expected

The Scream Test was part of the huge effort to evaluate our on-premises compute resources before we began moving to the Azure cloud. After all, why spend resources moving something that isn’t needed?

Pete Apple, who helped develop the concept of the Scream Test, is a cloud network engineering architect in Microsoft Digital, the company’s IT organization. Looking back, he remembers the surprising results that emerged when they began shutting down specific servers to see who noticed.

“We thought we were going to get rid of a small number of machines that weren’t being used,” Apple says. “But we found the actual share was about 15% of all machines, which saved us a lot of effort of moving those unused machines to the cloud. In other words, we downsized on the way to the cloud, rather than after the fact.”

As part of this process, Apple explains, our engineers looked at two related factors to reduce inefficiencies in our usage of computing resources.

The first was to identify systems that were used infrequently, at a very low level of CPU (sometimes called “cold” servers). From that, we could determine which systems in our on-premises environments were oversized—meaning someone had purchased physical machines according to what they thought the load would be, but either that estimate was incorrect or the load diminished over time. We took this data and created a set of recommended Microsoft Azure Virtual Machine (VM) sizes for every on-premises system to be migrated.

“We learned that there’s a lot of orphaned, or underutilized, resources out there,” Burtness says. “These were cases where the workload was so small on a server—like under 5% CPU—that it didn’t make sense to host it on its own machine. We could then move the task or application and get it down to just one or two CPUs on a virtual machine.”

At the time, we did much of this work manually, because we were early adopters. The company now has a number of products available to assist with this review of your on-premises environment, led by Azure Migrate.

Another part of the process was determining which systems were being used for only a few days a month or at certain busy times of the year. These development machines, test/QA machines, and user acceptance testing machines (reserved for final verification before moving code to production) were running continuously in the datacenter but were really only needed during limited windows. For these situations, we applied the tools available in Azure Resource Manager Templates and Azure Automation to ensure the machines would only run when needed.

Automating with Azure

Today, we don’t have to rely on anything as crude as the Scream Test to find unused and underused computing resources. With 98% of our IT resources operating in the Azure cloud, we have much greater insight into how efficient our network is, so much of the process can be automated.

“We’ve found this effort much easier to manage in the cloud, because all our computing resources are integrated with the Azure portal,” Apple says. “They have an API system and offer various tools within Azure Update Manager and Azure Advisor to help with cost efficiency. It’s kind of like a modern version of Clippy—’Hey, it looks like your VM isn’t being used much. Do you want to downsize that or turn it off?'”

(For the uninitiated, Clippy was the Microsoft Office animated paperclip assistant introduced in the late 1990s. It offered tips and help with tasks, like writing and formatting documents. Clippy became iconic for its quirky suggestions, including recommending that you remove things from your desktop that you weren’t using.)

“With everything being in the Azure portal or in Azure Resource Graph, it’s much more streamlined, and makes it easier to get that data out to the teams. They can then go into the portal and clean up the resource.”

Brent Burtness, principal software engineer, Commerce Financial Platforms

And simply taking the step of turning off stuff that we weren’t using turned out to be very effective. Thanks, Clippy!

Today, we approach this challenge in a more efficient and sophisticated way, taking advantage of Azure tools like Update Manager and Advisor.

“With everything being in the Azure portal or in Azure Resource Graph, it’s much more streamlined, and makes it easier to get that data out to the teams,” Burtness says. “We can run automated queries with Azure Resource Graph. Then we bring that information into our internal Service 360 tool, which we use to give action items to our developers. Each item gives them a link to Azure portal, and they can then go into the portal and clean up the resource.”

Managing for the lifecycle

One of the most important things we learned by using the Scream Test to identify inefficiencies and moving our systems from on-premises servers to the cloud was that it’s an ongoing process, not a fixed-end project.

“We had this idea that it was going to be a one-time event, that we’ll move to the cloud and then we’ll be done,” Apple says. “A better understanding is that it’s a lifecycle. We have integrated this concept of continual evaluation into our processes around everything that’s still on-premises, because we still have labs, we still have physical infrastructure.”

We continue to do this evaluation on a regular basis with both physical and virtual computing resources, because needs and usage are constantly changing.

Cutting our cloud costs

“Now we have a basic process around a six-month cycle,” Apple says. “So, every six months we ask, does this still need to be on-premises or should we start moving it to the cloud? And we do the same thing with our cloud resources. Who’s still using these VMs? And we still go through the same review process to see if it’s needed, or if we can shut it down or move it.”

This has resulted in significant cost savings for the company. “We’re up to about 15% to 20% less compute cost, depending on the organization, because of this much better understanding of our business needs,” Apple says.

Better governance, increased security

Another major benefit of this process was establishing much stronger governance of compute resources across the entire organization.

“When we first did the Scream Test, we weren’t always really sure who owned what, in some cases,” Apple says. “We’ve fixed that as part of this process. This governance aspect is a key part of being more efficient with our resources.”

Burtness explains why this is so important.

“It’s critical to know exactly who to contact when there’s something wrong with the server,” Burtness says. “Now, with clearer ownership, clearer accountability, and better inventory, it’s a much better experience.”

Better governance also means tighter security, according to both Apple and Burtness.

“This is really important when it comes to threat-actor response,” Apple says. “Unused servers can often be an entry point for hackers. Or, say we discover that a machine or server is getting hacked; you need to talk to who owns it. If you don’t know, it takes you longer to track them down and combat the hack. That’s not great. Improving our governance has definitely made securing our environment easier.”

Key takeaways

Here are some things to keep in mind when managing your own enterprise compute resources for greater efficiency:

• It’s not a one-time exercise. For the best results, you should be evaluating your computing resources on a regular schedule to identify ”cold” servers and unused infrastructure.
• Adjust for variable usage patterns. It’s not just about unused servers. Some machines may only be needed for a business function during certain busy times of the year. Consider turning the machines on just to handle the load during those periods and turning them off the rest of the year.
• Use Azure tools for greater insight. If you’re operating your infrastructure in the Azure cloud, you can much more easily monitor and address orphaned resources using automated tools such as Azure Advisor, Azure Resource Graph, and the Azure portal.
• Apply your savings to other priorities. “The more efficient you are, the more savings can be applied to other projects or given back to your manager—who is going to be very happy with you,” Apple says.
• Saving money is not the only benefit. You’ll not only save operating costs, you’ll have a reduced maintenance and monitoring load, better governance, and fewer security vulnerabilities.

Try it out

• Get started on your own cloud journey with the Cloud Adoption Framework for Microsoft Azure.
• Learn about Azure process automation and sign up for a free trial.

Related links

• Read about our IT journey to becoming a cloud-first organization with Microsoft Azure.
• Discover the five principles that guided our move from on-premises infrastructure to the cloud at Microsoft.
• Explore the benefits of Azure Advisor.
• Get more details about the Azure portal.

We’d like to hear from you!

• Want more information? Email us and include a link to this story and we’ll get back to you.

The post Moving from a ‘Scream Test’ to holistic lifecycle management: How we manage our Azure services at Microsoft appeared first on Inside Track Blog.

These agents help employees reach beyond Microsoft Graph and Microsoft 365 applications to do their work more thoroughly and efficiently. By empowering users to experiment with AI-driven assistance and capabilities internally at Microsoft, we’re unlocking efficiency, process automation, and data-driven insights tailored to specific individuals’ or teams’ needs.

One tool is making Copilot extensions accessible to more employees than ever before: Microsoft Copilot Studio.

This low-code solution makes it possible for both technical and non-technical users to create their own agents and tailor Copilot’s capabilities to their work. At Microsoft Digital, the company’s IT organization, we’re the first to implement Copilot Studio and develop a methodology for empowering our employees to create while establishing guardrails to keep our organization’s data safe.

As a result, we’ve built best practices that help us protect employees while enabling helpful agents—from individualized tools to organization-wide utilities. We’ve also learned lessons that can help customers navigate their own Copilot Studio journey.

Extending enterprise AI with Microsoft Copilot Studio

Microsoft Copilot Studio, a part of Microsoft Power Platform, empowers employees to build their own agents or use them to extend Microsoft 365 Copilot’s value. It uses the same low-code connector model as Power Platform to power actions through first-party and third-party services.

“Copilot Studio is a way for a non-technical person to spin up an agent quickly.”

Amy Rosenkranz, principal product manager, Copilot Extensibility, Microsoft Digital

As a result, users can create their own agents tailored to specific professional needs and business functions. These agents can narrow the focus of knowledge within the Microsoft 365 Graph, reach outside of it, and even take actions.

There are several ways to create agents. They range from simple natural language queries in Copilot Studio agent builder through Copilot Chat in Microsoft Teams and SharePoint to the full-featured Copilot Studio graphical authoring environment to a combination of Copilot Studio and Azure AI.

“Copilot Studio is a way for a non-technical person to spin up an agent quickly,” says Amy Rosenkranz, principal product manager responsible for Copilot extensibility at Microsoft Digital. “You can pull from a SharePoint site, from a graph connector, or from the web, and so employees are using it to tailor their experience to their business process.”

Building agents with Copilot Studio

“There’s an important role for Copilot Studio in helping customize the solutions our employees create, whether they want to use existing functionality, extend their knowledge, or expand their skill compatibility.”

Eileen Zhou, senior program manager, Microsoft Digital

Ultimately, the goal is to help employees work more efficiently by putting them in the driver’s seat through the power of self-directed agent creation. It also helps alleviate strain on business functions by getting people to the answers they need faster, without the need for human intervention.

“There’s an important role for Copilot Studio in helping customize the solutions our employees create, whether they want to use existing functionality, extend their knowledge, or expand their skill compatibility,” says Eileen Zhou, senior program manager in Microsoft Digital. “And it provides opportunities for both non-technical creators who want to create individualized solutions and people with advanced knowledge who are building more enterprise-focused agents.”

To empower our employees for this kind of creativity, we needed to put guardrails in place that ensure they can build agents confidently without putting themselves or the company at risk.

Managing the scale and sophistication of Copilot Studio creations

Building guardrails around agent production meant developing a system for classifying them according to their purpose, reach, and potential risk.

On one end of the spectrum, simple retrieval agents might only access content that individuals author and own. Non-technical employees typically create this kind of agent through natural language prompts in Copilot Studio agent builder.

“There’s a time and place for personal agents that integrate with business workflows, but if something is a business-critical service, we need to think security-first.”

Jake Visser, principal architect manager, Copilot and AI apps

On the other end, more elaborate tools—task or autonomous agents that combine knowledge, action, and orchestration—need to cross data boundaries to accomplish their work. More technically advanced IT employees and professional developers build these agents for larger-scale business functions using the full-featured Copilot Studio authoring environment.

Agent capabilities

This simple taxonomy doesn’t capture the whole picture though. As a result of the varying reaches and risk profiles for different agents, we tend to group them into three categories:

• Personal self-service agents created by employees to meet highly individual business needs.
• Line-of-business agents created by individual organizations within Microsoft to fulfil discipline-specific work functions.
• Agents intended for publishing across the entire organization.

“If an employee is building a service, we need to manage it like a service,” says Jake Visser, principal architect manager for Copilot and AI apps. “There’s a time and place for personal agents that integrate with business workflows, but if something is a business-critical service, we need to think security-first.”

Microsoft Digital is responsible for developing and enforcing guidelines for managing those services.

“Thanks to our early experiences with Copilot Studio, we’ve been able to develop gates and controls based on the type of agents that creators want to build.”

Aisha Hasan, Power Platform and Copilot Studio product manager, Microsoft Digital

Governance, processes, and policy for enabling Microsoft Copilot Studio

Establishing guardrails around the different agents our employees can create in Microsoft Copilot Studio involved asking a lot of questions. What Power Platform features apply to Copilot Studio workflows? What additional areas of risk do agents introduce? How can we build policies and processes around low-code AI creations? How can we help employees understand the implications of the agents they create?

“Thanks to our early experiences with Copilot Studio, we’ve been able to develop gates and controls based on the type of agents that creators want to build,” says Aisha Hasan, Power Platform and Copilot Studio product manager for Microsoft Digital. “Through predetermined groups and rules, we can allow freedom and experimentation at different scales without putting our internal tenant at risk.”

Since Copilot Studio exists within Power Platform, that tool’s capabilities provided a solid foundation for managing agents. We have extensive experience empowering citizen developers while maintaining good governance through Microsoft Power Platform. So it was easy for us to apply existing administration and governance best practices to this new framework.

At the outset of our journey, we already had robust systems in place for securing custom connectors, and Microsoft 365’s built-in governance capabilities ensure Microsoft 365 agents respect our labeling taxonomy and the policies it articulates. Finally, we have the power to introduce sharing limits that restrict how widely creators can distribute agents depending on their purpose and scope.

Together, these features and capabilities helped us extend existing administration and governance structures to the new world of agents. But thoughtful process and policy are equally important.

For the simpler self-service agents that individual employees create and use, we’re able to define our policies at the Copilot Studio environment level. Tenant administrators and partners on the Microsoft Security team apply data loss prevention policies to configure what individual employees can and can’t do. At this level, everyone in the company has the same configuration and tools available, and automation largely handles agent reviews and assessments based on pre-configured settings.

For more wide-reaching apps that operate at the line-of-business level or that we might publish enterprise-wide, we need to apply greater rigor. Thanks to our experience administrating and governing Power Platform, Microsoft Digital already had a robust process in place to review internally created enterprise apps. Discipline-specific professionals in security, privacy, and other spaces conduct these reviews to ensure internal teams meet our high standards.

By building onto that structure, we’ve updated our custom environment review process for agents created in Copilot Studio. We step through a review process that includes phases for security assessments, threat modeling, privacy assessments, and Responsible AI reviews.

Our goal is to properly scope our governance controls into what people are building. If we can easily enable things we consider low-risk like retrieval agents, we let employees build those in their personal development environment, but more powerful or far-reaching custom agents require more thorough oversight.

Configuration, review, and assessment are only parts of the puzzle. We also flighted user awareness efforts to help employees understand not just how to use Copilot Studio, but also its implications for security, privacy, and Responsible AI.

These campaigns included field readiness through Viva Learning, Copilot Champs sessions, newsletters, marketing campaigns through Viva Amplify, office hours, internal roadshows, and elite programs. We even launched an agent-building contest that invited employees to design whatever they liked.

Providing employees with opportunities for learning and experimentation has helped jumpstart interest in creating agents. Together with product features, process, and policy, it ensures we unlock the full value of Copilot Studio safely and effectively.

Unlocking Copilot Studio value

With the freedom to create using Microsoft Copilot Studio and the protection of robust guardrails, individuals and teams are flexing their imaginations to create highly useful agents. We’re in the early days of our own Copilot extensibility journey, but agents are already driving faster and more accurate access to information and greater productivity.

Two examples stand out:

• The IDEAS Copilot, a retrieval agent, empowers informed decision-making by democratizing access to our IDEAS knowledge base and its insights on app usage. Through natural language queries, IDEAS lets users take action on crucial usage data without the need for technical expertise.
• The Employee Self-Service Agent in Microsoft 365 Copilot, a more advanced and organization-spanning agent, provides access to HR, IT, and facilities-related information and tools through two interfaces: Copilot or our company sites.

As the capabilities of Copilot Studio continue to grow, Microsoft Digital is actively collaborating with the product team to ensure administration and governance features keep pace with its technical elements. Our experience as the first and largest adopters of this new framework mean that every lesson we learn internally helps the product accommodate businesses’ needs more effectively.

“Everyone wants to move fast, and people are enthusiastic to explore this new framework for enterprise AI. Our guiding principle is making the product secure by default so businesses can make it happen safely.”

David Johnson, principal program manager, Microsoft Digital

Thanks to our experience at Microsoft, the product has incorporated several new features:

• A set of controls for Copilot Studio connectors that allow guardrails for self-service.
• The ability to specify data sources including SharePoint sites, public URLs, internal documents, or others.
• Connector endpoint filtering that lets administrators govern the SharePoint sites and other connectable endpoints when creators build apps, flows, or agents.
• Different channels for publishing agents, like Microsoft Teams, websites, or integrations into tools like Dynamics 365.
• Suggested configuration defaults, for example requiring authentication so people can’t create anonymous Copilots.

Between built-in features and emerging best practices, Copilot Studio is unlocking the freedom to create like never before while maintaining organizational safety. For our customers and Copilot users, that means multiplying AI’s impact by setting employees free to create tools that will help them do their work faster, better, more creatively, and more insightfully.

“Everyone wants to move fast, and people are enthusiastic to explore this new framework for enterprise AI,” says David Johnson, principal program manager architect for governance at Microsoft Digital. “Our guiding principle is making the product secure by default so businesses can make it happen safely.”

Key takeaways

Here are some tips for getting started with Copilot Studio at your company:

• Have an all-up tenant strategy. Create separate Power Platform environments based on what people want to build, what data they want to use, and what controls you need as a result.
• Take this opportunity to make sure that your governance is up to date and aligns properly between Power Platform and Microsoft 365.
• Educating your users is key. Recognize that most difficulties arise from inefficiency and error, not nefarious intention.
• Evaluate your risk tolerance for different kinds of Copilot Studio creation and structure your security and governance efforts around that.
• Take advantage of dev environments to learn and practice.

Try it out

• Curious what Copilot Studio can accomplish for your business? Try a demo here.

Related links

• Learn how we’re using good governance to ride the wave of agents washing over Microsoft.
• Find out how we’re embracing this new ‘agentic’ moment at Microsoft.
• Check out our guide for deploying and driving adoption of Microsoft 365 Copilot at your company.
• Discover how we’re driving deeper AI value with Copilot extensibility.

We’d like to hear from you!

• Want more information? Email us and include a link to this story and we’ll get back to you.

The post Unlocking enterprise AI extensibility at Microsoft with Microsoft Copilot Studio appeared first on Inside Track Blog.

The introduction of Microsoft 365 Copilot has brought about a significant transformation in the way our engineers work here at Microsoft. Tools like GitHub Copilot and Copilot Studio have become essential for streamlining coding tasks, but the real revolution has been the arrival of Copilot.

It has opened new possibilities for enhancing crucial soft skills like communication, program management, collaboration, and emotional intelligence, which are sometimes overlooked. Copilot has been a powerful ally with these softer skills, with capabilities that extend beyond coding and development.

Challenges in promoting Microsoft 365 Copilot

When Microsoft 365 Copilot was first introduced to our engineering team in Central & Eastern Europe (CEE), we faced several challenges. Our engineers were skeptical about the ability of Copilot to meet their needs and make them more productive. Many perceived it as more suited for administrative or managerial tasks rather than core engineering functions. Additionally, there was a lack of awareness about the full spectrum of capabilities offered by Copilot, leading to underutilization. We had our work cut out for us as we attempted to change perception and demonstrate real value for our engineers.

Identifying needs and driving usage

To address these challenges, we embarked on a series of activities aimed at demonstrating the value of Copilot to engineers. In Microsoft Digital, the company’s IT organization, we’ve learned that the best way to confront a skeptical audience is to listen carefully to their concerns, provide in-person hands-on training, and “make it real” by applying AI to their day-to-day challenges.

Some ways we did this include:

• Focus group sessions on understanding engineering persona: We conducted focus group sessions to discuss engineers’ day-to-day scenarios and identify needs that could be addressed by using Copilot. We looked for benefits beyond engineers’ core work as programmers, as many of them were satisfied with the coding assistance available from GitHub Copilot.
• Hands-on training sessions: We gave our engineers opportunities to see first-hand how they could leverage Copilot to assist with tasks such as ideation, project planning, and technical documentation. As a result of these sessions, they learned how to use Copilot to generate diagrams, create user stories, and draft technical documents efficiently. This hands-on approach helped engineers understand the practical benefits of using Copilot in their daily work.
• Showcasing practical use cases: We highlighted real-life examples of how our engineers’ peers have integrated Copilot into their daily routines. Highlighted examples included summarizing emails, drafting documents, and generating technical design documentation. By demonstrating these practical applications, we showcased the immediate benefits of using Copilot in everyday tasks.

Maximizing the value of Copilot for engineers

Based on the insights gained from our adoption activities, here are 10 practical suggestions for how engineers can make the most of Copilot’s features:

1. Streamline meeting preparation: Copilot assists in setting up meetings by generating agendas and drafting meeting invites using simple prompts. This reduces the time spent on administrative tasks and ensures that meetings are well organized. Additionally, Copilot can help summarize notes and action items from previous meetings to enhance collaboration and efficiency.

2. Master effective communication: Utilize Copilot to draft and refine emails, documents and presentations. This ensures that messages are clear, concise, and impactful, improving overall communication with team members and stakeholders. Copilot can also suggest tone adjustments and provide templates for various communication scenarios to enhance the professionalism and effectiveness of engineers’ interactions.

3. Optimize time management: Copilot can help prioritize tasks and set reminders for important deadlines. This helps engineers manage their time more efficiently and stay organized. By integrating with engineers’ calendars and task-management tools, Copilot can provide a holistic view of their schedule and suggest optimal times for focused work and collaboration.

4. Generate comprehensive technical documentation: Copilot in Word can be used to generate technical design documents. Engineers can quickly create diagrams and information needed to produce comprehensive and accurate documentation. Copilot can also assist in maintaining version control and ensuring that documentation is consistently updated and aligned with project developments.

5. Simplify gathering feedback: Copilot in Forms simplifies the process of creating surveys and forms to gather information from within and outside the organization. Copilot in Word or Excel can also analyze survey results and generate insights, helping engineers make data-driven decisions.

6. Prepare for standups efficiently: For daily standups, engineers use Copilot Chat to create user stories and update tasks in Azure DevOps. This helps them stay on track with their sprint goals and collaborate effectively with product managers. Copilot can also provide summaries of previous standups and highlight key discussion points, ensuring continuity and focus.

7. Foster collaboration and teamwork: Use Copilot in Whiteboard to facilitate collaborative brainstorming sessions and project planning. This fosters a collaborative environment where team members can contribute ideas and work together seamlessly. Copilot can also capture and organize brainstorming outputs, making it easier to translate ideas into actionable plans.

8. Enhance leadership and management: Copilot can assist in creating structured feedback, performance reviews, and development plans for team members. This helps engineering managers provide constructive feedback and support their team’s growth effectively. Copilot can also suggest personalized development resources and track progress to ensure continuous improvement.

9. Resolve conflicts thoughtfully: Use Copilot to draft thoughtful and empathetic responses to resolve conflicts or address concerns within the team and promote a positive and respectful work environment. Copilot can also provide conflict resolution strategies and mediation techniques to help employees navigate challenging situations with confidence.

10. Encourage continuous learning: Leverage Copilot to identify and recommend learning resources tailored to engineering needs and interests. Whether it’s technical skills, soft skills, or industry trends, Copilot can curate a personalized learning path and provide access to relevant courses, articles, and workshops. This ensures that employees stay updated and continuously develop their expertise.

By integrating these innovative practices into their daily workflow, engineers can leverage Microsoft 365 Copilot to not only boost their technical productivity but also significantly enhance their soft skills. This holistic approach will help them become more effective and well-rounded professionals, ready to tackle the diverse challenges of the modern workplace.

Unleashing the power of Copilot: a transformative journey

The journey to promote Copilot within our engineering community is underway and initial results are promising. By addressing initial skepticism and demonstrating the practical benefits, our engineers are now seeing the value of Copilot to enhance their technical and soft skills. With a solid foundation to build on, our engineers will continue to explore and embrace these capabilities, finding new ways to innovate, collaborate, and excel in their roles.

Key takeaways

Here are some ways engineering teams can use Copilot to enhance your engineers’ productivity and collaboration:

• Use Copilot for meeting preparation: Using Copilot to generate agendas and draft meeting invitations ensures well-organized and efficient meetings.
• Enhance communication skills: Copilot can draft and refine emails and presentations, improving the clarity and impact of communication.
• Optimize time management: Manage time efficiently and stay organized by using Copilot to prioritize tasks and set reminders.
• Generate technical documentation: Ensure accuracy and consistency with Copilot in Word and create comprehensive technical design documents. and
• Foster collaboration: Facilitate brainstorming sessions and project planning with Copilot in Whiteboard to promote teamwork and idea sharing.

Try it out

Get started with Microsoft 365 Copilot.

Related links

• Learn how we’re measuring the impact of Microsoft 365 Copilot and AI at Microsoft.
• Find out how we’re promoting Microsoft 365 Copilot adoption with gamification.
• Discover how we’re becoming an AI-first frontier firm at Microsoft.
• See how we’re implementing a Zero Trust security model at Microsoft.
• Learn how we’re using the new Microsoft 365 Copilot-powered Facilitator feature at Microsoft.

We’d like to hear from you!

• Want more information? Email us and include a link to this story and we’ll get back to you.

The post Ten tips to unlock Microsoft 365 Copilot for your engineers appeared first on Inside Track Blog.

With the launch of Microsoft 365 Copilot and the ability to extend its impact through agents, we saw an opportunity to roll the value of AI-powered assistants into the information-rich ecosystem of SharePoint. By infusing SharePoint with Copilot features, we’re making the search for authoritative content more accurate and more streamlined for users while giving site administrators, site editors, and content owners greater control and more opportunities to enable their colleagues.

As we’ve implemented this new kind of AI assistant internally at Microsoft, we in Microsoft Digital, the company’s IT organization, have gained first-hand knowledge of how to deploy, manage, and optimize the new capabilities—and learned key lessons that can help you use SharePoint agents to their full potential.

SharePoint in the age of AI extensibility

As a content management and collaboration platform, SharePoint is so deeply integrated into the fabric of business that it’s easy to take it for granted. Every day, people add almost two billion documents to Microsoft 365 Copilot apps (Outlook, Teams, Word, and so on). Searching through those vast quantities of content and information can be a challenge for users.

“SharePoint agents are a way for users to ask very specific, scoped questions in order to receive authoritative answers from that specific content source. We want to add value to the user’s workflow to ultimately improve their productivity.”

Sunitha Bodhanampati, senior product manager, Microsoft Digital

One of extensibility’s core principles is using agents to bring AI capabilities into any canvas or endpoint. With the emergence of this new framework, connecting retrieval agents to the SharePoint experience was an instinctive move.

SharePoint is a natural place for agents to live—SharePoint makes enterprise content accessible to employees and agents simplify and enhance the workflows needed to that and because they make it easier to find that content.

Transforming enterprise content accessibility

At their core, SharePoint agents are about surfacing insights, scaling expertise, and powering more-informed decisions.

Every SharePoint site includes an agent scoped for the site’s content. They allow users to search a site using natural language queries like “Summarize last week’s files on benefits” or “Create an executive summary of last quarter’s sales reports.” That means people can find answers without combing through the site or wrestling with cumbersome search terms.

Here’s a rundown of the goals and benefits of SharePoint agents for different users:

“SharePoint agents are a way for users to ask very specific, scoped questions in order to receive authoritative answers from that specific content source,” says Sunitha Bodhanampati, senior product manager working on SharePoint agents with Microsoft Digital. “We want to add value to the user’s workflow to ultimately improve their productivity.”

Ready-made agents are a helpful starting point, but SharePoint agent builder introduces even more targeted capabilities. It gives site administrators, editors, and content owners the opportunity to create, customize, and control agents to provide greater assistance to their users.

“For human beings, the more content you give them, the less they engage, so agents are a way to narrow that field of inquiry to make your site more helpful.”

Siobhan Flanigan, senior marketing communications manager, Microsoft Customer and Partner Solutions

In just a few clicks, anyone with SharePoint site editing permissions can create agents based on content that’s relevant to specific projects or tasks. They can customize their agent’s branding and purpose, specify the sites, pages, and files that it should access, and define customized prompts tailored to its objectives and scope.​​​​​​​ This flexibility ensures that the right people get the best possible access to content while ensuring security and adherence to governance guardrails.

Most importantly, it’s easy to create SharePoint agents. This technology isn’t just accessible to software developers. SharePoint agent builder’s inherent simplicity means that people in communications, HR, marketing, or any other role can create digital assistants in just a few minutes and a few clicks.

“We’re making knowledge accessible at a level it’s never been before,” says Siobhan Flanigan, senior marketing communications manager for Worldwide Learning in Microsoft Customer and Partner Solutions. “For human beings, the more content you give them, the less they engage, so agents are a way to narrow that field of inquiry to make your site more helpful.”

“There’s a lot of intelligent creation and summarization with Copilot experiences, so naturally there are fears around organizational risk from overexposure, hallucinations, or misdirections that lead to user frustration.”

Swapna Malekar, principal product manager, Microsoft 365 Copilot

Beyond SharePoint sites, employees can easily share agents via email or within Microsoft Teams chats, granting colleagues access to the same accurate and relevant information through natural language queries. Not only are coworkers able to use each other’s agents, but @mentioning the agent in a group chat setting gives the team a digital subject matter expert, ready to assist and facilitate collaboration. 

Building these capabilities and implementing them securely required extensive collaboration between the SharePoint product group and Microsoft Digital, the company’s IT organization. As the first business to implement this technology at scale, we had to be confident that it met our standards for trustworthy administration, governance, security, and responsible AI.

“With any AI-specific experience, there needs to be guardrails and governance to manage its behaviors,” says Swapna Malekar, principal product manager for Information Discovery and Experiences in Microsoft 365 Copilot. “There’s a lot of intelligent creation and summarization with Copilot experiences, so naturally there are fears around organizational risk from overexposure, hallucinations, or misdirections that lead to user frustration.”

In the simplest terms, SharePoint agents are scoped versions of Copilot Chat. As a facet of agents in Microsoft 365 apps, SharePoint agents benefit from all of the same governance controls that protect our tenants in any other Copilot-enabled context.

That alignment with pre-existing tooling and policy means that SharePoint agents respect permission-trimming when they provide responses. Because the content itself honors permissions according to Microsoft 365 Copilot governance policies, users who don’t have access to that content won’t receive it as part of the agent’s outputs.

These capabilities arose from our iterative development process and experience as an enterprise, but it’s just the beginning. In our early experiments with SharePoint agents, we’ve also developed some helpful scenarios and best practices our customers can use.

Creating agent-friendly content ecosystems in SharePoint

Early adopters here at Microsoft have already created some highly useful SharePoint agents. In the Microsoft Customer and Partner Solutions (MCAPS) business group, the Worldwide Learning team has used the following agents to support employees in specific contexts:

Ask MCAPS Academy

This agent makes it easy for learners to query the Microsoft learning catalog to find specific answers contained in our course content. For example, before a salesperson demonstrates Microsoft Fabric, they could ask for best practices without having to take an hour-long course.

Ask MCAPS Tech Connect

MCAPS Tech Connect is a strategic training event for technical field roles, designed to help them uplevel their expertise and build confidence through collaborative learning and hands-on skilling. The Ask MCAPS Tech Connect agent gives employees easy access to content from more than 70 sessions. Users ask questions about the material, and the agent retrieves Microsoft PowerPoint decks and summarizes sessions so they can determine if they want to watch full videos.

During the process of creating these agents and others, our internal site editors and administrators have developed best practices to make sure employees get the most value out of their new digital assistants. The following techniques can help you create your own agents:

• Understand agent instructions. It’s helpful to think about creating agents with two sets of parameters: sources and behaviors. Sources are how you define the sites, folders, and content your agent will encompass. A more expansive scope will be more likely to return an answer, but that answer might be too broad. A more limited scope will provide better accuracy, but it might not have access to answers from a wider content base and therefore not return results at all. Meanwhile, behaviors are the explicit instructions and guidance you provide your agent, for example fine-tuning the structure of the summaries it delivers or specifying the technical level of responses the agent should provide.
• Optimize your libraries for AI. Just like it’s important to structure web content for search engine optimization (SEO), it’s helpful to structure your SharePoint sites for AI optimization—what some super-users are calling “AIO.” We recommend using all available metadata to ensure content is highly available to SharePoint agents; for example, headers, meta-tags, and alt-text. File names are particularly impactful. We recommend naming a file according to the way a user is most likely to search for it, like “Q3 AI impact executive summary.” It’s also helpful to name files associated with each other in similar ways. For example, the PowerPoint presentation and recording transcript for the same conference session should have similar titles.
• Recognize human behaviors. If site administrators and editors want to enable their users, they need to think about how to accommodate the ways they work. Plenty of employees will know to access SharePoint agents through the built-in chat, but why not provide even easier onramps? Our insiders have learned that it’s extra helpful to share agents through Microsoft Teams chats, in communications, and anywhere else people might need content support. It’s also helpful to use the UX design capabilities in SharePoint to create explicit call-to-action buttons that direct users to particular agents.

“SharePoint agents unlock and scale knowledge,” Flanigan says. “If there’s an answer locked somewhere in a content library, agents essentially turn that library into a responsive assistant, and people can ask it questions to get the information that empowers their work.”

The agentic future of enterprise knowledge

As our teams continue to experiment with SharePoint agents, they continue to find value in more accessible and authoritative knowledge. Site editors and administrators across Microsoft are eagerly seeking out advice and opportunities for more and more agents to support their organizations.

“SharePoint revolutionized enterprise content management and collaboration once before. Now, we have an incredible opportunity to use the power of AI to help people get the information and insights they need.”

Jeff Teper, president, Microsoft 365 Collaborative Apps and Platforms

Our product teams are also extending SharePoint agents’ capabilities to amplify their impact even further. In addition to linking to agents in Microsoft Teams chats, they’ll soon be available in channels to provide AI assistants as digital liaisons for specific projects or teams.

Other, more complex features are on the way as well. These improvements will lead to even greater value, all stemming from the combination of enterprise content and AI assistance.

“SharePoint revolutionized enterprise content management and collaboration once before,” says Jeff Teper, president of Microsoft 365 Collaborative Apps and Platforms. “Now, we have an incredible opportunity to use the power of AI to help people get the information and insights they need, driving more informed decision-making, better collaboration, and more streamlined business processes.”

Key takeaways

Here are some things to think about as you consider getting started with SharePoint agents at your company:

• Experiment with different scopes and behaviors by iterating your SharePoint agents over time.
• Pay special attention to the metadata in your SharePoint sites and files to ensure they’re optimized for AI discoverability. This resource shares best practices for managing metadata.
• Tailor your SharePoint agents and how you disseminate them to human needs and behaviors to encourage uptake.

Try it out

Want to start streamlining access to content for your employees? Get started with SharePoint agents here.

Related links

• See how our Microsoft legal team is helping clients find answers faster with SharePoint agents.
• Find out how we’re reimagining content creation with our Azure AI-powered Inside Track story bot.
• Discover ways we’re unlocking enterprise AI extensibility at Microsoft with Microsoft Copilot Studio.
• Learn how we’re embracing this new ‘agentic’ moment at Microsoft.

We’d like to hear from you!

• Want more information? Email us and include a link to this story and we’ll get back to you.

The post Unlocking knowledge through intelligence: Lessons learned using SharePoint agents at Microsoft appeared first on Inside Track Blog.

To address these challenges, knowledge must be captured, presented, and made accessible so that individuals can quickly find answers on their own. Our legal team supporting marketing at Microsoft developed a SharePoint agent to help achieve just that.

Over the years, our Microsoft legal team, Corporate, External, and Legal Affairs, has developed rich, comprehensive, and curated content accessible through SharePoint. This includes guidelines, policies, summaries of laws, self-service tools, and more; all presented in a way that’s understandable for a non-legal audience. The marketing section of this SharePoint site alone drives approximately 8,000 page views per month, resulting in significant cost savings.

When Microsoft released SharePoint agents, it created an opportunity to do even more. Now, the marketing legal team’s newly developed SharePoint agent sits on top of its robust SharePoint site, adding the power of AI to answer legal questions and further unlocking the value of the existing resources in an elegant and streamlined way.

SharePoint agents are natural language AI assistants tailored to specific tasks and subject matter, providing trusted and precise answers and insights to support informed decision-making. Each SharePoint site includes an agent based on the site’s content. Or, with a single click users can create and share a custom agent that accesses only the information they select. 

“At Microsoft, AI is transforming how our legal teams operate, creating new opportunities to enhance workflow efficiency,” says Hossein Nowbar, chief legal officer and corporate vice president for Microsoft. “We’ve used SharePoint agents to improve the discoverability and delivery of legal resources, scale our legal advice, and gain critical insights into content usage. This saves considerable time for teams that need advice and those that provide it, all the while driving greater legal compliance and consistency.”

To determine whether using the SharePoint agent shown in the demo was better than using search and navigation alone, the legal team ran a test consisting of six legal questions for which five participants were asked to find answers. For each question, the participants were timed using search and navigation alone, and then using the new SharePoint agent.

In timing each participant, we stopped the clock either when they were satisfied that they had found the correct answer, or at five minutes if they did not find the correct answer. In the first test, using search and navigation, participants only found the answer 83.3% of the time, leaving 16.7% of the questions unanswered. Using the SharePoint agent, participants found the correct answer 100% of the time.

Not only were participants more successful at finding correct answers, but they also found the answers much more quickly using the SharePoint agent. Participants found and confirmed the answer in under 1 minute 46.7% of the time and found and in under two minutes 100% of the time. On average, participants found the correct answers 2.97 times faster using the SharePoint agent when compared with using site search and navigation.

We know from experience and feedback that when people can find answers to their legal questions quickly and easily using self-service resources, the legal department can focus on more complex issues. A SharePoint agent is an essential tool for any organization seeking to harness the power of AI to make answers readily available, reduce the need for live support, and bring their existing content to life.

“The Microsoft Legal team was an ideal early adopter of SharePoint agents due to their well-curated content,” says CJ Tan, principal group product manager for SharePoint agents. “They recognized the value of an agent in scaling support and handling easily addressable questions, allowing the team to focus on more complex, unique business scenarios. Instead of learning how to build an agent, they could concentrate on helping marketers surface and use the right content for their business needs. As subject matter experts, they were also well-positioned to validate and test their agent before publishing it on their SharePoint site.”

As we build out our array of Microsoft 365 agents, we continue to look to our internal experiences to guide the product’s evolution for our customers. We are exploring new ways for SharePoint agents to be shared and extensible across a variety of content sources. Lastly, we know that governance controls and analytics are critical as organizations introduce new features within their workflow and are excited about the roadmap for additional insights available and coming soon from Copilot Analytics, SharePoint Advanced Management, and SharePoint Purview.

“Organizations rely on SharePoint, creating more than two million sites and uploading more than two billion files daily,” says Jared Spataro, chief marketing officer of AI at Work @ Microsoft. “By giving every SharePoint site an agent, employees can quickly tap into this massive knowledge base with a single click.”

As with any new product and technology innovation, we’re focused on education and customer learnings. At the Microsoft 365 Community Conference we will host a variety of sessions on SharePoint agents, going deeper into business use cases and best practices for creation and usage.

Connect with author Brent Sanders on LinkedIn.

Here are some of our top tips for getting started with SharePoint agents at your company:

• Prepare your content: Ensure your SharePoint content is highly curated, accurate, complete, and unique. This helps agents provide more accurate and relevant responses.​ Organize content into smaller, manageable sets to improve response accuracy (e.g., using smaller document libraries with fewer files and minimal graphics).
• Maintain your content: Updates made to content sources are reflected in the SharePoint agent responses, so make sure that content sources are maintained. Also, be sure to regularly check that file permissions are accurate, based on the agent audience.
• Use ready-made agents: Each SharePoint site comes with a ready-made agent scoped to the content of the site. SharePoint admins can approve this agent to help jump-start usage. Use our communication kit to help announce SharePoint agent availability and increase awareness.
• Identify where custom SharePoint agents can add value: SharePoint agents can be grounded in specific sites, folders, or files. Collaborate with business stakeholders to identify business objectives and priorities to create specialized expert and informational agents.
• Target no more than 20 content sources: If you are selecting a site or folder, you can have any number of files underneath. However, when selecting items individually, we recommend capping it at 20 sites, folders, or files for best results.
• Encourage users to provide feedback: Your employees can use “thumbs up or thumbs down” to give feedback on the SharePoint agent’s response. This feedback can be used to continuously improve content and enhance response accuracy over time.
• Measure the impact: We have a variety of analytics resources to help measure adoption and usage of SharePoint agents, including; the SharePoint document library, SharePoint Advanced Management, Microsoft Purview, and additional reports coming to Copilot Analytics.

For organizations with at least 50 Microsoft 365 Copilot licenses, any employee in the organization will be able to create, share, and interact with SharePoint agents. Learn more about SharePoint agents.

• Learn more about implementing and using SharePoint agents.
• Check out the Microsoft Learn content about how you can get started with SharePoint agents.
• Discover how we’re managing the wave of agents washing over Microsoft with good governance.
• Read about how we’re extending our infrastructure at Microsoft to manage the proliferation of agents.

• Want more information? Email us and include a link to this story and we’ll get back to you.

The post Boosting efficiency with SharePoint agents: How our Microsoft legal team is helping clients find answers faster appeared first on Inside Track Blog.

Microsoft is a big company with thousands of teams working in different ways based on the work they do. Despite that complexity, when our employees go looking for something, they expect an internal search portal that will find exactly what they’re looking for instantly—just like when they search on the internet. Yet when talking to these employees, each of them defines the scope of what they’re looking for quite differently.

• A developer may want HR info, stack overflow, other technical info specific to their organization, or technical info from places like Microsoft Azure and Microsoft.com.
• A salesperson may want HR info, customer information from our account management software and support services, or the latest public information about their customers.

This blog explores the challenge of delivering the full scope of content each employee expects to find in search from their subjective view. This is what we call search completeness.

To start on the journey of getting to search completeness, you first must understand your user community:

• How do they search? Do they use smart phones? Do they use Bing’s Work search tool? Do they use a corporate SharePoint portal? For Microsoft employees, it’s a mix of all of these.
• Why are they searching? Are they trying to find another person? Are they researching content? Are they trying to find reference material?
• What are they searching for? What content is most important for your employees to find?

[Read the first blog in our series, making content more accessible and searches more efficient at Microsoft.]

Understanding your user community

Reviewing search term frequency was one of our early steps in understanding our users. Looking at the number of times each search term was used, then looking at a sampling of those search terms made it very clear that the most common searches are for common employee actions, and that less common searches are typically persona specific. The chart below shows this well: high volume search terms that are common across most employees, and low-volume ones that tend to be org- or persona-specific.

Sometimes we could easily identify desired content from these popular search terms such as search terms related to documents. Microsoft.com and Stack overflow were also fairly popular.

Next, we realized there was a lot of content that was impossible to identify from search terms. We needed some other way of identifying desired content and found a way via Microsoft Entra ID (formerly known as Microsoft Azure Active Directory, or AAD).

By using its authentication volume, we are able to see the most popular registered apps within the company. Many of these are included in Microsoft Search by default. SharePoint and OneDrive are good examples. Others have their own search capability that meets user expectations and doesn’t need its content included in enterprise search. Outlook is such an example. This left us with a significant volume of highly used apps whose content would be beneficial to add to enterprise search. The chart below gives you a taste of these results.

Gathering the list of popular apps left us with a challenge of identifying popular content that isn’t defined as an app in Entra ID. We explored various ways of capturing this information but, so far, have not found any better method than user feedback and surveys.

The result of this work has yielded a “Top 100” list of content we want to add to enterprise search. So how do we go about getting this content added into our search results?

Methods of achieving search completeness

Microsoft Search provides a number of different methods with which to bring in all the content. Each method has its own strengths and weaknesses, which we’ve summarized in the table below.

What we are doing

So now the stage is set, we know the content we want to include, we know the methods available for doing it, we just need to implement the right method in each case.

We also use Microsoft Viva Topics and other product capabilities, which will be discussed in a future blog post.

At this point, search indexing encompasses 70 percent of the Microsoft Entra ID Top Apps list, as weighted by usage volume. We expect to reach 80 percent within the next year.

• The content added through connectors and federated search is receiving 75,000 clicks per month––about 8 percent of our total click volume.
• These connections have added 10 percent to the admin effort. For more detail, see the previous blog post in this series: Generating great results: Administering search at Microsoft.

We’ve also realized there are occasions where content should not be included in enterprise search but should be included in targeted custom search portals. The same methods described above can typically be used to support such custom portals. Our learning thus far will also be described in a future post.

We see some continuing challenges for which we do not yet have answers:

1. At some point the administrative and resource overhead associated with adding additional sources of content will outweigh the benefit because we will be getting down to very seldom used content. We don’t know where that boundary is yet.
2. We need to figure out how to stay in touch with continuing changes across the company, deprecating content when appropriate while adding new content sources when they come up.
3. We haven’t figured out how to tune search relevance in a manner that works well for each persona.

Please return to the Inside Track blog for future stories in our ongoing series on transforming search completeness here at Microsoft.

• Read the first blog post in our series about making content more accessible and searches more efficient at Microsoft.
• Explore generating great results: Administering search at Microsoft.

Want more information? Email us and include a link to this story and we’ll get back to you.

The post Getting to ‘search completeness’ internally at Microsoft appeared first on Inside Track Blog.