Creating efficiencies in finance with Dynamics 365 and machine learning

At Microsoft, we’ve combined Dynamics 365 for Customer Service and Microsoft AI to perform topic prediction and automatic routing for incoming issues and requests from Microsoft Finance customers. This new solution has reduced the time it takes to get a request into the hands of an agent and enabled a more intelligent, streamlined management process – resulting in shorter response and resolution times for our customers.

In today’s world of digital transformation every organization is searching for ways to work more efficiently. The Microsoft Finance organization recently created a solution to automate the process and tools our agents use to access and manage incoming issues and requests from Microsoft Finance customers. Using Microsoft Dynamics 365 and machine learning, the new solution has reduced the time it takes to get a request into the hands of an agent and enabled a more intelligent and streamlined management process.

Understanding Microsoft Finance

Microsoft Finance provides financial and regulatory services for Microsoft. Microsoft Finance Operations is the first point of contact for employee and supplier support and guidance in the Microsoft Procure-to-Pay process. Finance Operations provides support personnel that handle queries for more than 150,000 internal employees and more than 130,000 suppliers. Finance Operations covers 120 countries, and the breadth of queries come from our large scale of transactions per year, including the following;

• 1.2 million expense reports
• 580,000 purchase orders
• 1.5 million invoices
• 5.6 million corporate credit card transactions
• $65 billion in payee payments
• $1 billion in employee expense payouts

Managing finance operations with OneFinance

Our OneFinance solution provides the primary interface between Microsoft Finance Operations and the customers that consume Microsoft Finance services—from audit, to treasury, to tax and trade. Almost 1,100 OneFinance agents in five locations manage over 1 million support tickets per year, which we refer to as cases. They also receive and process over 18 million emails.

Previously, a team of agents called gatekeepers were responsible for reviewing each incoming customer request email, and then routing the request to the proper OneFinance agent queue, where agents would receive the request and work towards resolving it. With the volume of requests that came in to OneFinance, gatekeeping was a large, time-consuming, and error-prone task. The manual gatekeeping process caused issues in several different parts of the organization:

• Customer. Our customers waited too long for a reply to the issue they submitted. The manual gatekeeping process was a bottleneck in the workflow. Also, cases were sometimes assigned to the wrong queues so agents that picked the cases up didn’t have the expertise to help the customer, which left the customer wondering why the agent didn’t understand the issue.
• Agent. Our agents had to spend the time picking cases out of the different subject matter—related queues. This created an extra step for them and consumed time they could be using to resolve cases. Agents also had to deal with cases assigned to the wrong queue, further hindering their productivity.
• Agent manager. Our agent managers were left wondering why their agents took so much time to resolve cases. The managers also realized they needed to cross-train agents to empower them to resolve cases from a wider set of queues.
• Business. Our high-level managers observed the case creation and resolution process and wondered why so many people were needed to resolve a case. They did understand the need for a more efficient process.

Automating case management with Dynamics 365 and machine learning

Microsoft Finance, in partnership with Microsoft Digital and the Dynamics 365 team has worked to create a new solution to transform the customer request process and create a better, more efficient gatekeeping function. We wanted to remove as much manual process from the case creation workflow as possible. To create a more efficient and effective case creation process, we’ve automated case creation, categorization, and assignment using Dynamics 365 for Customer Service, and machine learning. With the automated solution, agents no longer need to sort through incoming queues to prioritize and filter the requests. The solution monitors, filters, and automates the case management so our agents are working with valid, actionable customer requests and are assigned those requests only when they are prepared to handle them.

High-level OneFinance request process flow

The automated case creation solution is designed to create a more efficient, automated process to manage incoming requests to OneFinance. The high-level process begins when a customer has a request to submit through to OneFinance:

1. A customer submits a request to OneFinance through email. Requests come from internal employees and external vendors, and include questions about purchase orders, expense reports, payment dispersal, refund requests, and order status, to name a few.
2. The automated gatekeeper performs AI prediction and routing for incoming cases.
3. The email is converted to a case and placed in the Dynamics 365 for Customer Service queue.
4. The automated gatekeeper reviews the case in the top of the first-in-first-out (the oldest email first) queue and routes it to the proper queue, such as Expense, Refunds, or Orders.

After the agent receives the customer request through OneFinance, they can take appropriate actions based on the status of the request or its attached case. The automated solution has reduced the time it takes to get a request into the hands of an agent from 15 minutes to less than 1 minute, on average.

The automated process uses Dynamics 365 for Customer Service for intake, routing, assignment, and Gatekeeping web service. The web service uses a machine learning model developed in Microsoft Azure Machine Learning Studio. The machine learning model analyzes the body of the incoming cases and predicts the primary and secondary topics within the case, returning the results to Dynamics 365 for Customer Service.

Building on Dynamics 365 for Customer Service

Dynamics 365 for Customer Service provides a framework upon which we build OneFinance functionality. It provides several native capabilities that enable us to build and release OneFinance functional components over time, including automated case creation. These native capabilities include the ability to:

• Guide agents to the right actions with AI-driven insights surfaced at the right time on a single interface.
• Treat every customer like a VIP with a complete perspective on each customer’s experience so that agents can personalize interactions.
• Avoid service and support issues with predictive care by analyzing data from connected devices and taking appropriate actions before warning signs become a problem.
• Make it easier to find answers through self-service communities by intelligently routing cases from any channel to the right agent for quick resolution.

Using Dynamics 365 for intelligent routing and assignment

Dynamics 365 is at the core of automated case creation functionality. Incoming emails are routed to Dynamics 365 for Customer service, which handles the bulk of automation and routing tasks. The end-to-end case creation process requires no manual case management tasks; everything is hosted and automated within Dynamics 365 for Customer Service and Azure.

Understanding our agents with agent data

Dynamics 365 for Customer Service gives us the opportunity to personalize the information for each agent so the new solution understands which cases to route to an agent, and when that agent is ready for a case to be assigned. Dynamics 365 for Customer Service stores two sets of agent data:

• Persona. The persona dataset contains information about who the agent is, including each agent’s product expertise, experience, and languages spoken.
• Calendar. The calendar dataset contains information about when the agent is available to handle cases, including working hours and the region in which they reside.

Routing smarter with Intelligent Case Routing

We’re using Intelligent Case Routing in Dynamics 365 for Customer Service to route and assign cases to our agents accurately and efficiently. Intelligent Case Routing is infused with AI, which we’ve used to create three primary match algorithms:

• Match topics to agent skills. The topics derived from the machine learning analysis of the case information are matched to the skills stored in the agent’s persona data.
• Match region or language to agent region and language. The case routing process matches the region and language derived from the case information to an agent with the same region and language skillset.
• Match case assignment to availability of agent. Cases are only assigned to agents if they are within active working hours.

Based on these three matches, the case is assigned to the agent that best fits the case and has the smallest number of cases in their personal queue.

Enabling intelligent automation with machine learning

Our automated case solution is informed by machine learning models we’ve built using Azure Machine Learning Studio. The machine learning capabilities in the automated solution enable our automated case creation process to make intelligent, context-based and data-driven decisions based on the input received from the customer request email. The overall process flow is illustrated in the following figure.

Data for the process is extracted from email messages that come in from Dynamics 365 for Customer Service. The data is processed by a machine learning web service, which returns data back to Dynamics 365 for Customer service. This detailed process involves several core processes that form the machine learning flow:

• Load and pre-process the data. The data from Dynamics 365 is loaded and transformed within Azure SQL, where the email subject and body are extracted from the email.
• Extract data. The data is processed through a machine learning web service. We’ve built a REST API that interacts with Dynamics 365 for Customer Service to push the required data into the machine learning model, which performs several processing tasks, including:
  • Remove records that have missing, incomplete, or unactionable data.
  • Remove special characters, numbers, and punctuation.
  • Convert all text to lower case.
  • Remove stop words. These include content that is not relevant to retrieving the primary and secondary topics of the request.
  • Find and group stem and root words that have similar meaning (for example: run, running, ran)
• The machine learning model hosted in the web service is based on the Multiclass Neural Network module and it provides two key outputs: a primary topic and a secondary topic.
• Transform machine learning model output. The primary and secondary topic are stored in a database that provides numeric index values for each topic. The numeric value associated with the primary and secondary topic from a case are stored and passed on to Dynamics 365 for Customer Service for routing and assignment.
• Train the model. We split the data, using 70 percent for model training, 15 percent for parameter tuning, and 15 percent for testing. Our initial training contained five years of historical data that provided a starting point for the model.
• Retrain the model. When the machine learning model detects a topic incorrectly, the incorrect records go back into the model as retraining records. This process improves the accuracy of the model and adapts to new information or changes in OneFinance functionality, such as new queues or business groups being added. Retraining is critical to the proper functionality of the machine learning model and we retrain using continuous integration and continuous development to ensure our model is as effective as possible. With retraining, we improved accuracy from 76 percent in the first 3 months to 88 percent in the most recent 3 months.

Benefits

Our entire business benefits from the automated case creation process. Each of our key personas receives an improved experience that directly impacts their ability to be more productive and efficient. These persona benefits include:

• Customer. Our customers receive quicker acknowledgement of their case assignment and the best answer to their problem from an agent who is a trained expert on the subject.
• Agent. Our agents don’t have to go out and find the next case that fits their skill sets. Cases are automatically assigned and sent to the agents, and the cases that each agent receives are specific to his or her area of expertise, region, language, and availability.
• Agent manager. Our agent managers know that their agents are working on cases that they can resolve with confidence. Because routing and assignment is tracked and reported against, the managers know which topics are used most, so they can cross-train their agents in those areas.
• Business. The business has reduced cost for the case management process. Gatekeeping agents can now convert to agents who resolve cases, creating greater efficiency and output. Cases are resolved in a timely and efficient manner, which results in improved customer satisfaction for the business.

We also realized several specific benefits from the automated case creation solution, including:

• Better categorization of incoming requests. Our agents receive more accurate and usable information for cases with the new automated solution. The machine learning model is assessing primary case category with 88 percent accuracy. As a result, our agents are given usable, actionable information in the cases they handle.
• Faster case resolution. Automating case creation and categorization has revolutionized how our agents interact with and resolve cases. We’ve saved over 18,000 hours of agent time in the past calendar year with the automated solution. This gives our agents time to focus on the cases that need their attention and be more productive within their case workload.
• Greater case-handling capability. The automated case creation process is faster than any human agent could ever be. This results in more customer requests being processed and put in the hands of agents who can resolve the issue more quickly.

Moving forward

We’re working on improvements to our automated case creation solution that create a more accurate and efficient case creation process. Our service is currently used within OneFinance, but we’re making changes to enable other applications to consume the automated case creation solution as a service that can be integrated with other applications. We’re also refining our machine learning model to detect incoming requests that don’t require action from an agent. These scenarios include SPAM mail, out-of-office replies, or any other non-actionable email message.

Conclusion

Dynamics 365 for Customer Service and Microsoft AI have dramatically increased the efficiency and effectiveness of case management for Microsoft Finance. The automated case creation solution has reduced the time it takes to get a customer request into the hands of an agent and enabled a more intelligent and streamlined case creation process. Using Dynamics 365 and machine learning, Microsoft Finance has improved the request handling process with more accurate categorization, faster case resolution, and better case-handling capability, resulting in shorter response and resolution times for our customers.