{"id":23935,"date":"2026-06-04T09:00:00","date_gmt":"2026-06-04T16:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/insidetrack\/blog\/?p=23935"},"modified":"2026-06-09T13:04:31","modified_gmt":"2026-06-09T20:04:31","slug":"measuring-the-impact-of-our-ai-investments-in-it-at-microsoft","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/insidetrack\/blog\/measuring-the-impact-of-our-ai-investments-in-it-at-microsoft\/","title":{"rendered":"Measuring the impact of our AI investments in IT at Microsoft"},"content":{"rendered":"\n
As an IT organization, we need to understand which of our AI investments are creating business value for Microsoft. We need to know how that value shows up, whether we can measure it, if we can trend it, and how we can use what we learn to make better decisions for the company. <\/p>\n\n\n\n
That\u2019s why, as part of our broader approach to AI at Microsoft, we\u2014Microsoft Digital, the company\u2019s IT organization\u2014are building a framework to measure the impact of the AI investments we\u2019re making on behalf of the company.<\/p>\n\n\n\n \u201cIf we want to measure the business impact of AI, the conversation quickly moves toward identifying the agents or AI efforts that are driving the most value and satisfying business outcomes. We know that those conversations can be complex, so we use a value measurement framework to capture and assess the signals we have available.\u201d<\/p>\nDon Campbell, principal group technical program manager, Microsoft Digital<\/cite><\/blockquote>\n\n\n\n Our framework is helping us move from AI enthusiasm to AI accountability. It creates a common way for us to talk about value across our different initiatives, teams, and business processes. It also helps us ask a harder, more specific question every time we assess the impact of AI at Microsoft Digital: If AI saves time, reduces costs, improves quality, or lowers risk, what changes are we making to take advantage of that? <\/p>\n\n\n\n We don\u2019t have the full answer yet\u2014we\u2019re still improving the way we measure. Some of our signals are instrumented, some rely on strong hypotheses, and some need better telemetry. But we\u2019re not waiting for perfect results to start learning.<\/p>\n\n\n\n \u201cIf we want to measure the business impact of AI, the conversation quickly moves toward identifying the agents or AI efforts that are driving the most value and satisfying business outcomes,\u201d says Don Campbell, a principal group technical program manager in Microsoft Digital. \u201cWe know that those conversations can be complex, so we use a value measurement framework to capture and assess the signals we have available.\u201d<\/p>\n\n\n\n AI councils at Microsoft<\/strong><\/p>\n\n\n\n Check out our series on how employee councils are guiding how we use AI here at Microsoft.<\/p>\n\n\n\n AI value doesn\u2019t show up the same way everywhere. One investment we make might help employees complete a task faster, while another might improve quality, reduce risk, increase coverage, or lower operational costs. Some of that value can be measured directly, while in other cases it starts as a hypothesis that needs to be tested. That range is why we needed a common framework instead of a single metric.<\/p>\n\n\n\n Our value measurement framework helps our Microsoft Digital teams answer three basic questions before and after they build:<\/p>\n\n\n\n Revenue impact: <\/strong>How an AI investment contributes to our business growth, sales activity, customer targeting, or deal velocity.<\/p>\n<\/div>\n\n\n\n Productivity and efficiency: <\/strong>How AI helps our people complete tasks faster, increase throughput, optimize processes, or automate work.<\/p>\n<\/div>\n\n\n\n Security and risk management: <\/strong>How AI helps us identify, prevent, or manage security vulnerabilities, risk exposure, or Responsible AI compliance.<\/p>\n<\/div>\n\n\n\n Employee and customer experience: <\/strong>How AI improves satisfaction, engagement, or the quality of a product or service experience.<\/p>\n<\/div>\n\n\n\n Quality improvement: <\/strong>How AI improves deliverables, accuracy, confidence in outputs, or process quality.<\/p>\n<\/div>\n\n\n\n Cost savings: <\/strong>How AI reduces our operational cost, improves our resource allocation, or helps us avoid future cost.<\/p>\n<\/div>\n\n\n\n Our framework doesn\u2019t require every AI investment that we make to create value in all six areas. In fact, that rarely occurs. A support automation scenario might focus primarily on productivity, employee experience, and cost avoidance. A security scenario might involve risk reduction, vulnerability coverage, or the ability to address more issues than a team could handle manually. A process quality scenario might relate to measures that are specific to a particular workflow and be harder to roll up into a single number.<\/p>\n\n\n\n \u201cMeasurement sources vary based upon the type of AI initiative. The six value areas give us a framework for measurement, but we need to observe and collect information where measurement starts to become practical. Teams need to understand the processes affected, including how long work takes, how many resources are involved, and what the workflow looks like before and after AI.\u201d<\/p>\nDavid Laves, director business programs, Microsoft Digital<\/cite><\/blockquote>\n\n\n\n The framework creates consistency in how we talk about value, while giving teams room to measure what actually matters for their scenario. Some measures roll up easily, including cost savings, time savings, and certain risk measures. Others are more specific to the process being improved. Those measures still matter because they help business owners understand whether AI is changing the work in a meaningful way.<\/p>\n\n\n\n \u201cMeasurement sources vary based upon the type of AI initiative,\u201d says David Laves, a director of business programs in Microsoft Digital. \u201cThe six value areas give us a framework for measurement, but we need observe and collect information where measurement starts to become practical. Teams need to understand the processes affected, including how long work takes, how many resources are involved, and what the workflow looks like before and after AI.\u201d<\/p>\n\n\n\n Our framework also helps us make better investment decisions. Before we commit to an AI scenario, we can map the opportunity to the value areas that matter most, estimate the value we think it can create, and decide what needs to be measured. After implementation, we can compare results against the baseline, review the data with the right business and AI owners, and adjust the work based on what we\u2019re seeing.<\/p>\n\n\n\n That last step is critical. Our framework creates a way for us to create an operating rhythm around AI value. It helps us take a promising scenario and prove its worth (or lack thereof) by establishing expected value, evaluating what is being measured, and deciding what to change because of the results.<\/p>\n\n\n\n It\u2019s a continual evolution of business value measurement that prioritizes progress over perfection, and it\u2019s helping ensure that our AI approach stays grounded in the outcomes that are most meaningful to our business.<\/p>\n\n\n\n A framework only matters if teams use it to make decisions. For us, that means moving measurement out of one-off reporting and into a regular management cadence. We track our highest-business-value AI initiatives by priority and business function, then review the KPIs that show whether those investments are creating the value expected.<\/p>\n\n\n\n Some KPIs roll up cleanly. Our cost savings, time savings, and certain risk measures can be summarized across initiatives and discussed at a leadership level. Other KPIs stay closer to the individual scenario because they\u2019re tied to a specific workflow, process, or business outcome. We need both. Our rollup metrics help our leaders see broad progress, while the scenario-level metrics help our teams understand what\u2019s changing inside their work.<\/p>\n\n\n\n \u201cWe actually create monthly targets and end-of-year targets for every top AI-enabled initiative,\u201d Campbell says. \u201cThen we basically reiterate every single month with our leadership team to look at the value we\u2019re driving and have conversations about it.\u201d<\/p>\n\n\n\n That monthly rhythm helps us proactively manage AI value. If a measure is trending positively, we look at what\u2019s working and where else the pattern might apply. If a measure is off track, teams can dig into the supporting data, review the assumptions, and decide whether they need to adjust the solution, the measurement, or the operating process around it.<\/p>\n\n\n\n This process supercharges prioritization. Our team here in Microsoft Digital has a large set of AI opportunities, and not every idea can move at the same pace. By mapping initiatives to value areas, estimating expected impact, and tracking results over time, we can have a more grounded conversation about where to invest, where to scale, and where to keep learning.<\/p>\n\n\n\n That discipline becomes more important as AI moves deeper into business processes. We don\u2019t want teams to measure only adoption or usage if the real goal is a better business outcome. Usage matters, but it doesn\u2019t tell the whole story. A tool can be used often and still fail to improve the process it was meant to change.<\/p>\n\n\n\n Consider the following example of applying the framework from our Global Support team. This team is currently examining how AI can help automate specific pieces of the ticket management process.<\/p>\n\n\n\n \u201cIn Global Support, the processes that often matter most from a value perspective are the ones with high repetition. If a process runs thousands of times a month and can operate autonomously, without human input, that\u2019s where AI can deliver meaningful, measurable impact.\u201d<\/p>\nDavid Finney, principal program manager, Microsoft Digital<\/cite><\/blockquote>\n\n\n\n As part of this effort, we examined a support process that depended on manual follow-up. In this process, after a Global Support team member marks an issue as resolved, the team waits for the user to confirm that the ticket can be closed. If the user doesn\u2019t respond, the agent must follow up once a day for up to three days. After the third attempt, the agent simply closes the ticket.<\/p>\n\n\n\n This user flow gave us a practical way to test the value framework. It has repetition, because it runs often; it has autonomy potential, because the steps are deterministic and rule-driven; and it has a clear time-savings opportunity, because a human agent spends time checking the ticket, writing the follow-up, and sending the message. It also has a measurable implementation effort, because the data exists in the ticket process but the solution still needs to integrate with ServiceNow.<\/p>\n\n\n\n \u201cIn Global Support, the processes that often matter most from a value perspective are the ones with high repetition,\u201d says David Finney, a principal program manager in Microsoft Digital. \u201cIf a process runs thousands of times a month and can operate autonomously, without human input, that\u2019s where AI can deliver meaningful, measurable impact.\u201d<\/p>\n\n\n\n Finney estimated that about 5,000 tickets a month execute this process. Because each ticket can require up to three follow-ups, that can create up to 15,000 manual email follow-ups a month. At about three minutes per follow-up, that\u2019s roughly 750 hours of productivity spent on one small piece of the process each month.<\/p>\n\n\n\n The framework helps us look at that work through both value and effort. On the value side, we can evaluate repetition, autonomy potential, and time savings. On the effort side, we can assess whether the data exists, how complex the solution is, whether engineering work or system integration is required, and how long implementation may take.<\/p>\n\n\n\n \u201cWe started to evolve our conversation to, \u2018So what?\u2019\u201d Campbell says. \u201cYou saved money, you saved hours. What did you do with it? Where did the actual business outcome sit?\u201d<\/p>\nDon Campbell, principal group technical program manager, Microsoft Digital<\/cite><\/blockquote>\n\n\n\n That structure matters because a high-value opportunity still needs the right implementation path. In this case, the process is part of the ticket workflow, so the needed data exists. The complexity comes from integrating the automated agent with ServiceNow so it can interact with the ticket, check whether the user responded, send follow-ups, and resolve the ticket according to the defined process.<\/p>\n\n\n\n Instead of trying to automate all of support at once, the team identifies specific subprocesses where AI has a clear role and the value can be measured. \u201cIt\u2019s taking a sort of bite-sized approach to AI rather than trying to solve for everything in one big go,\u201d Finney says.<\/p>\n\n\n\n That\u2019s the kind of practical example the framework is designed to surface. It helps us find work that\u2019s frequent enough to matter, structured enough for automation, and measurable enough to prove whether the AI investment changed the process.<\/p>\n\n\n\n Measuring value starts the next conversation. If an AI investment saves time, reduces cost, increases coverage, or improves quality, we need to know what happens next. The number is relevant, but the business outcome is more important.<\/p>\n\n\n\n \u201cWe started to evolve our conversation to, \u2018So what?\u2019\u201d Campbell says. \u201cYou saved money, you saved hours. What did you do with it? Where did the actual business outcome sit?\u201d<\/p>\n\n\n\n That\u2019s the harder part of AI value measurement. A team might use AI to reduce time spent on repetitive work, but the real value depends on how that recovered capacity gets used. In some cases, the reinvestment path is clear. A team can point to more programs delivered, more backlog reduced, more issues reviewed, or faster service delivery.<\/p>\n\n\n\n In other cases, the value is harder to trace. Some AI improvements return small amounts of time to many employees. Those minutes matter, but it\u2019s difficult to prove exactly where each person reinvested them.<\/p>\n\n\n\n We\u2019re careful when it comes to measuring ROI. We know our leaders will ask for it, and it belongs in the broader value conversation. But we don\u2019t want ROI at the center of the story before we have the right cost model, telemetry, and approved data to support it.<\/p>\n\n\n\n For now, we\u2019re focused on the operating discipline: Define expected value, baseline the current state, instrument the AI-enabled process, track results, review the data, and act on what we learn. That discipline is teaching us a number of practical lessons:<\/p>\n\n\n\n Consistency matters as much as the metric itself. When our teams review value on a regular rhythm, they can see trends, test assumptions, and adjust the solution or the process around it. Some measures will be mature, and others will be directional. Some will need better instrumentation. The point is to keep improving the quality of the measurement while keeping the conversation focused on business value.<\/p>\n\n\n\n We\u2019re continuing to build our value measurement muscle across Microsoft Digital. We\u2019re not looking for one perfect formula for every AI investment. Instead, we\u2019re creating a repeatable way to define value, measure it, review it, and use it to guide our next action.<\/p>\n\n\n\n As our AI investments and overall strategy mature, that framework helps us stay honest about what we know, clear about what we still need to learn, and focused on the outcomes that AI is designed to improve.<\/p>\n\n\n\n Key takeaways<\/p>\n<\/div>\n\n\n\n Here are five actions you can take to help measure the impact of AI investments at your organization, based on what we\u2019ve learned in our own efforts:<\/p>\n\n\n\n Try it out<\/p>\n<\/div>\n\n\n\n Get started with Microsoft Agent 365.<\/a><\/p>\n<\/div>\n\n\n\n Related links<\/p>\n<\/div>\n\n\n\n As an IT organization, we need to understand which of our AI investments are creating business value for Microsoft. We need to know how that value shows up, whether we can measure it, if we can trend it, and how we can use what we learn to make better decisions for the company. That\u2019s why, […]<\/p>\n","protected":false},"author":92,"featured_media":23937,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_hide_featured_on_single":false,"_show_featured_caption_on_single":true,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[71],"tags":[199,868,137,852,827],"coauthors":[550],"class_list":["post-23935","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-featured","tag-ai","tag-ai-deployment-and-adoption","tag-change-management","tag-it-and-business-operations","tag-microsoft-365-copilot","m-blog-post"],"yoast_head":"\n
<\/figure>\n\n\n\n\n
\n
Building a framework for AI business value<\/h2>\n\n\n\n
\n
We organize the answers to those questions around six value areas:<\/h3>\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n\n
Turning measurement into an operating rhythm<\/h2>\n\n\n\n
Applying the framework: Global Support<\/h2>\n\n\n\n
<\/figure>\n\n\n\n\n
\n
Moving from savings to reinvestment<\/h2>\n\n\n\n
\n
<\/figure>\n\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n\n