Microsoft automates revenue processing with Power Automate

Microsoft has automated revenue processing for legacy systems that lack an API, making them poor contenders for automation. Learn how optical character recognition (OCR) technology in Microsoft Power Automate was used to reach 100 percent process adherence across 1,100 newly automated processes.

Revenue processing at Microsoft spans 14 business units, 257 regions and territories, 18 million emails, 2.5 million transactions, and more than $100 billion per year in revenue. Roughly 4,000 business process outsourcing (BPO) agents positioned at various regional operation centers (ROCs) throughout the world manage the contracts and transactions that keep those operations moving. Historically, BPOs have performed the entirety of that mission-critical work manually, until Microsoft Digital built an end-to-end automated case management platform to lighten the load of manually scanning papers, filling forms, validating data, and verifying accuracy.

That platform—Customer Obsessed Solution Management and Incident Care, or COSMIC—is built on Microsoft Dynamics 365, Microsoft Power Platform, and Microsoft Azure. In addition, the use of Microsoft Cognitive Services and AI technologies increase the platform’s efficiency, lower the risk of human error, and ensure 100 percent process adherence. However, COSMIC only addressed a portion of the overall workload. The technical landscape that BPOs rely on consists of a mixture of modern and legacy apps, and COSMIC is compatible only with modern platforms. To accomplish the same automation for legacy systems, Microsoft Digital used robotic process automation (RPA). A new feature of Power Automate, RPA fills in the gap, enabling automated revenue processing operations and bringing the efficiency, compliance, and cost savings of automation to legacy platforms.

This paper will explore the benefits of RPA, how it works, and why it matters.

Revenue processing at Microsoft

BPO agents each perform one of several specialized tasks in the case management workflow:

• Gatekeepers add incoming work to the appropriate queue, depending on the case type.
• Data capture agents extract data from the queued paperwork.
• Validators verify the validity of the casework, such as whether requested parts can be ordered or whether it requires additional approvals.
• Data entry persons prepare the final data set to be entered into the target system.
• Auditors audit the finalized data for accuracy and completeness.
• Invoice generators create the invoice and send it to the appropriate partner or customer.

COSMIC automated much of the business process workflow using three distinct phases:

1. Optimal character recognition (OCR) automates manual data entry and extraction.
2. A business rules engine automates the manual validation of data extracted in the previous phase.
3. RPA automates the subsequent data entry process for legacy systems.

Phase 1: Data capture via OCR

A single enterprise agreement might contain hundreds of forms. To process each agreement, BPO agents open each of those forms individually, then validate the information provided in the form according to the instructions provided to them for that particular case.

AI Builder, a capability of Microsoft Power Platform, automates this process. Microsoft Power Platform is a low-code platform that enables organizations to analyze data, act on it through applications, and automate business processes. AI Builder provides a low-code way to create and customize Power Apps and flows. Generally, AI Builder provides automation and prediction capabilities infused with Microsoft AI. Within RPA, AI Builder provides the OCR and Forms Processing functionality needed to automate BPO agent workflows.

Within RPA, AI Builder scans BPO case documents to identify and categorize the required fields (as few as 10 fields may be necessary to process a 40-page agreement, for example). All required forms are processed, the required data is captured via OCR, and AI Builder presents the result to the BPO agent for manual verification. This process is handled at scale in Power Automate. It’s not uncommon for AI Builder to process 20,000 scans per week.

Meanwhile, the process feeds training data to a machine learning model, continually optimizing the process and increasing accuracy.

Phase 2: Data validation via a business rules engine

After data from the required fields has been extracted, that data is processed by a business rules engine. The work instructions given to BPOs contain specific business rules pertaining to each case. Prior to COSMIC, the agent would need to verify adherence to these rules manually, usually by checking an external system. To do that, the agent would copy the number pertaining to the business rule from a document form and paste the number into the external system to validate the data.

Business rules may be quite simple. For example, an agent may simply need to verify that the customer or partner placing an order is doing so within the specified data ranges of the official agreement between that customer or partner and Microsoft. However, business rules may also be quite complex, which is often the case with specialized pricing. In that case, the agent needs to check each SKU individually against an external pricing system. In an order containing hundreds of SKUs, this task alone can be extremely time-consuming, especially if the agent is required to check many different external systems for a single order. Some business rules are also applied purely to ensure data accuracy.

Currently, COSMIC uses a legacy business rules engine to automate this process, though Microsoft Digital is migrating the business rules engine used in COSMIC to Power Automate. To continue with the pricing example, OCR captures the SKU, automatically connects to the relevant external system, validates the data, and extracts the correct price. This process results in one of two outputs: pass or fail. Passes happen in the background, because they require no intervention from the agent, whereas failures are presented to the agent for manual validation.

Phase 3: Data entry via RPA

When possible, APIs are utilized to perform data entry after OCR has extracted the necessary data and that data has been validated via the business rules engine. When an API is not available, RPA is used.

In total, COSMIC automated case management for more than 2.5 million transactions across 14 business units at Microsoft. The efficiencies created in that transformation yielded approximately $14.2 million per year in savings.

How RPA works

RPA takes place within the standard flow captured in Figure 2:

1. Microsoft suppliers or partners upload agreements.
2. AI Builder processes the forms in those agreements and identifies the required fields.
3. The resulting data is stored in Common Data Service (CDS).
4. Using the CDS data, robotic data entry performs the necessary data entry.
5. The output is sent to Microsoft Power BI and displayed as a dashboard or incorporated into a business process.

Although this process is continuous, RPA—the process of recording, parameterizing, and storing the data—is occasionally run in parallel:

1. A recording is made of the manual process (the specific steps BPO agents take in their case management duties, such as opening webpages and filling in various forms), as illustrated by step “A” in Figure 2.
2. The output is created as a JSON doc. The data in that doc is then parameterized, as illustrated by step “B” in Figure 3.
3. The parameters created in the first step are married to existing data by connecting to the appropriate data source, such as CDS or Microsoft Dynamics 365, depending on the task.

With the process complete, the recording can then be played with the ingested data, triggered either manually by a BPO or automatically, depending on the maturity level of the automation process in that particular Power Automate UI flow.

This process takes two forms: one for web-based UIs and one for desktop flows. Together, these UI flows and desktop flows—which are, at their core, simply low-code automation of UI-based tasks—provide the RPA capabilities for Power Automate. Microsoft AI is incorporated into AI Builder’s handling of OCR and forms understanding (Form Processing).

As illustrated in Figure 3, roughly 40 percent of case management work is currently automated through RPA. Analytics can be applied to UI flows to recommend tasks for automation. As we automate more and more of the entire BPO process, we increase efficiency at each step.

Crucially, these flows are also reusable, retaining the same structure to create a common output, which allows for telemetry logging.

RPA vs. APIs

Because RPA depends on recording, documenting, and automating a particular user click-step pattern and mouse motion, the process needs to be updated if the UI being recorded changes (for example, if a button or text field is moved). For this reason, APIs are the preferred method of automation. Before you adopt a UI flow solution for data entry, we suggest checking whether an API or an existing Power Automate connector is available. If either is available, you should use it. If neither is available, you should use RPA.

By virtue of being designed specifically for environments in which an API doesn’t exist, or where legacy record-keeping systems are used (assuming that the legacy system isn’t updated often, which means the UI remains relatively constant), RPA is well-suited to such environments. When an API doesn’t currently exist but is on the horizon, RPA can act as a worthwhile stopgap to provide ROI while the API is being built. When an API isn’t planned because the record-keeping software is a legacy platform and there is no middle layer upon which to write an API, RPA is also a viable alternative.

The workbench model

RPA is embedded in a more comprehensive approach to streamlining BPO operations, including the implementation of the workbench model of task delegation. This model doesn’t require BPO agents to actively seek work, but instead matches case tasks to qualified agents based on their availability, rather than requiring agents to check a queue or waiting for someone to assign the work.

Under the workbench model, which is designed to automate end-to-end transactional processing, workflows are broken into stages. Stages are then assigned to workbenches, which consist of a number of tasks. Tasks may be automated, semi-automated, or performed manually, depending on the context. A semi-automated task may still require human supervision; for example, for compliance reasons or when text is illegible and OCR is likely to fail. Semi-automated tasks are executed automatically, but require guidance and so are overseen by a BPO agent. For a particular semi-automated task, the agent may expect a value to fall within a certain range, for example. The agent can verify that the task returns a value within that range, then re-validate if and when all goes as planned.

When the task is complete, it undergoes a quality check. If the task passes the quality check, it’s sent to another agent further down the “workbench” line to enter the next phase. For example, a task may be transferred from a data capture agent to a validator, or from a data entry person to an auditor. This separation of duties also ensures compliance by following certain rules, such as when an audit should be conducted by someone other than the agent who entered the data.

The workbench model removes the inefficiency of siloed, case-specific agent knowledge and brings process control into the workflow. It also eliminates the need for system operating procedure (SOP) documentation by breaking workflows into finite tasks. In COSMIC, Microsoft Power Apps display workbench tasks alongside the Dynamics 365 business process stages, recommending the next best action to agents at each step of the process.

Process optimization

The workbench model maximizes efficiency by introducing a smart, controlled assembly line model for BPO tasks. Analytics provide further insights and opportunities for optimization. Some tasks naturally take more time than others, and the analytics bear out that variance. They may also uncover redundant tasks (in which case, the redundant task can be eliminated altogether) or tasks that are unnecessary because the data to be collected by the task already exists elsewhere. Variations or errors in the data (such as misspellings) also arise from time to time. When they do, data fields can be standardized for greater efficiency.

In this way, the platform benefits greatly from having access to Microsoft data sources. If the goal of a task is to collect or create data that already exists in a connected data source, it can be eliminated altogether.

Best practices

For those looking to implement RPA to automate certain processes, keeping these best practices in mind can smooth the road ahead.

Validate data before it’s ingested. Before RPA ingests prepared data, it is checked against a set of validation rules so we can ensure the data—such as the order date or the number of SKUs—is valid. This not only reduces failure rate but saves users time by virtually eliminating the possibility that they enter a flow only to have it fail at a late stage.

Validation can take place through a systematized approval process, but even if a systematic approach isn’t possible, manual validation is better than no validation at all.

Create a preferred hierarchy for Selenium IDE selectors. We utilized Selenium IDE for the recording process. During recording, selectors define various UI elements, like buttons, as they do in cascading style sheets (CSS). Selectors may include a unique ID, name, or class, or an XPath selector. In this use case, using XPath selectors may slow down the RPA process simply because the platform needs to first find the parent element before selecting the target variable, whereas using a unique ID provides a direct path. When that small inefficiency is tallied across an entire process, it adds up, significantly slowing down the playback and data entry processes.

We recommend creating a preferred hierarchy for selectors, with the most direct path as the default option. In other words, if a unique ID is available, the platform uses the ID; if not, it uses the element name. If the element name isn’t available, the platform uses a CSS selector, and so on.

XPath is not recommended unless no other selector in the preferred hierarchy is available. If XPath is used and the UI being recorded changes, the XPath for particular elements also change, causing the RPA process to fail. Using unique IDs when available bypasses that problem. It’s also worth noting that Internet Explorer doesn’t natively support XPath, which further hinders performance. If Internet Explorer support is unavoidable—as it is in our case—we recommend using other selector methods.

Incorporate step time when needed. Power Automate includes a configuration option called step delay, which defines how much time transpires between steps. If an end user needs to oversee the data entry process, you should implement a delay. Otherwise, following the recording can be difficult.

For example, if the UI flow includes opening a web browser and executing a web search, those two actions happen almost instantaneously without a delay, making it difficult to perceive the separation between steps. Adding a one-second delay between each step makes it much easier to follow. You can configure step delays in Power Automate via the Web UI flow card.

You can also add wait commands, either as explicit or implicit delays. Explicit delays are absolute. Implicit delays rely on defined variables. For example, in an environment with low internet bandwidth that results in slow page loads, an implicit delay might be configured to wait until a button is loaded on a page to perform an action on that button, rather than repeatedly trying to execute the command while the page is still loading. For similar reasons, implicit delays can also be helpful for legacy browser support.

Other available implicit commands include:

• Waiting for an element to become editable
• Waiting for an element to become not editable
• Waiting for an element to be present
• Waiting for an element to not be present
• Waiting for an element to become visible
• Waiting for an element to become not visible

Identify potential failures and create exceptions for them. Doing so deliberately causes the process to fail when the data doesn’t meet the defined parameters. For example, if a documented agreement between Microsoft and a supplier is only valid until a certain date, any purchase orders that include a purchase date beyond that deadline will fail.

Make use of “store” commands. Store commands are useful when values on a web page are used at some point, but not immediately. These following commands store the text from a label or text field for later use:

• The “store JSON” command stores an entire JSON file to be used in a script. This can be useful to get data from external sources as a parameter to a UI flow.
• The “store text” command stores the label of a web page element.
• The “store value” command stores the value of a text box or drop-down menu on a webpage.
• The “store xPath count” command can be used to check presence of an element on a webpage, or to get the number of rows in a grid on a page.

Make use of the “verify” and “assert” commands. The “verify” command is useful when confirming that a particular condition is met. For example, if the goal is to execute a command when a new page is loaded, the verify command can verify that the page header has changed (which indicates that the new page has loaded).

The “assert” command performs the same function, but also causes the process to fail, whereas the “verify” command does not.

Extend functionality with other modules. You can combine RPA with other Power Automate modules to create a complete solution capable of adapting to almost any scenario:

• The trigger module. You can use this to trigger the RPA process in various scenarios—Power Apps, Power Automate approvals or Power Virtual Agents, for example.
• Data collection from multiple sources. The RPA process may need access to multiple sources of data, such as a database, CRM, or COSMOS DB. This module ensures that the necessary data from all necessary sources is available at the time of data entry.
• Handling of duplicate runs. When a manual trigger is present, duplicate runs can be triggered, leading to duplicate data. This module prevents duplicate runs to prevent duplicate data.

• Logging and retrieval module. Using proper error logging and fetching results from CDS, this module stores error logs and screenshots in CDS, which can then be displayed to the agent.

Other best practices include:

• Creating a feature flag to quickly enable and disable RPA, which is useful when a bug is identified and needs to be addressed.
• Utilizing the Solutions for UI flow, which facilitates the migration of all UI flows from one environment to another.
• Creating a module that monitors the RPA process and provides an alert in case of process failure.
• Using comments in each step of the UI flow and its parent flow for proper documentation and sharing.
• Recording a happy path first, then iterating over multiple steps, or by stitching the steps together based on conditions.
• Using implicit wait commands instead of explicit delays where possible.

Looking forward

RPA performs data entry for all enterprise agreements entered into by Microsoft that are still processed on legacy platforms. Implementing RPA has resulted in a 71 percent time savings and an estimated $15 million in cost savings over the next two to three years. While the platform is in operation, next steps will focus on optimizing the platform to spot inefficiencies across all ROCs and BPO agents. For example, it will be possible in the near future to spot BPO agents at different ROCs that are each performing the same task.

In addition, optimizations are planned to incorporate the idiosyncrasies and regulations specific to a particular region or department. In some cases, because of the need for strict compliance in matters of revenue documentation, BPO agents perform tasks in the precise order laid out by existing documentation, regardless of whether that order is necessary or efficient. Insights derived from the platform may be able to indicate when a task is no longer required and when it was done in a previous or subsequent step and is therefore redundant.

In the near future, we will add support for the unattended execution of RPA tasks to a wider number of machines at Microsoft, as well as enhancing the recording experience using Microsoft AI. We’re also expanding the platform to handle orders submitted manually via Microsoft Excel documents, which will yield significant efficiency gains (some Microsoft Excel orders contain hundreds of rows of data).

Conclusion

Every day, between 50 and 55 million system transactions are processed through Microsoft Power Platform, in a massive operation that runs 24 hours a day, 7 days a week. Because so much of the case management surrounding those transactions involves manual work, the sheer volume means a degree of inefficiency and human error is unavoidable. Modern systems have the advantage of APIs to eliminate that inefficiency and the possibility of human error. For legacy systems that don’t have the luxury of an API, RPA accomplishes the same thing.

For the thousands of BPO agents stationed throughout the world, using RPA as a virtual worker to replace agents for mundane, repetitive tasks frees them up for more impactful work. To date we’ve incorporated 1,100 business rules into RPA, which translates to upwards of 10 million instances of those rules being automated—bringing the efficiency and security of digital transformation to legacy revenue processing platforms.