{"id":4687,"date":"2019-06-25T16:06:37","date_gmt":"2019-06-25T23:06:37","guid":{"rendered":"https:\/\/www.microsoft.com\/insidetrack\/blog\/?p=4687"},"modified":"2023-06-27T15:43:36","modified_gmt":"2023-06-27T22:43:36","slug":"key-sox-control-at-microsoft-is-transformed-through-ai","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/insidetrack\/blog\/key-sox-control-at-microsoft-is-transformed-through-ai\/","title":{"rendered":"Key SOX control at Microsoft is transformed through AI"},"content":{"rendered":"

\"MicrosoftRoyalty statements quality review can be a royal pain.<\/p>\n

Like many companies, Microsoft pays royalties for games, videos, content, software, and other creative work it uses in its products and services.<\/p>\n

The company pays a lot in royalties\u2014nearly $5 billion a year.<\/p>\n

\u201cThe quality review and approval of payments has been a time-consuming manual process for our operations teams,\u201d says Rohit Amberker, a finance director on Microsoft Finance\u2019s royalty team. \u201cWe review nearly 30,000 statements each year.\u201d<\/p>\n

Now that frustration is going away.<\/p>\n

How?<\/p>\n

Microsoft Digital has built an experience that digitizes the company\u2019s royalty payment process from end to end, says Jagannathan Venkatesan, principal group engineering manager for Microsoft Digital, which provides IT support for Microsoft Finance.<\/p>\n

[Go here to learn more about Driving Microsoft\u2019s transformation with AI<\/a>.\u00a0Click here to find out how Microsoft increases sales by using AI for lead qualification<\/a>.\u00a0Learn how Microsoft uses machine learning to develop smart energy solutions<\/a>.]<\/p>\n

\u201cThe Royalties team has been making significant strides in our service-maturity journey, enabling our business partners to close our month-end activities in a stable manner,\u201d Venkatesan says. \u201cHaving accomplished a successful migration from an on-premises system to a cloud-based distributed system performing at scale, we focused on helping our business partners with the effectiveness of operational processes.\u201d<\/p>\n

Venkatesan adds: \u201cWe targeted the statement-approval process, a monthly recurring activity, involving many thousands of contracts, currently performed manually. The ML\/AI-based solution for statement approval dovetails nicely into a larger effort across Microsoft Digital to digitize how it supports internal business partners like Microsoft Finance.\u201d<\/p>\n

Complexity untangled<\/strong><\/p>\n

Part of the challenge has been the complex nature of royalty payments, says Katie Lencioni, a principal program manager at Microsoft Digital.<\/p>\n

\u201cWe collect all transactions that are subject to a royalty payment based on views, downloads, or plays depending on the type of content,\u201d she says. \u201cFrom there, we calculate the payment amount according to the contract of the content owner, create a statement, and ensure the payments are processed.\u201d<\/p>\n

With 30,000 statements per year, there is ample opportunity for errors.<\/p>\n

\u201cWe perform three-way recon, comparing source transactions from Microsoft Marketplace with auto-calculated statements and against a third independent source,\u201d says Rajmohan Venkatesan, a software engineer at Microsoft Digital.<\/p>\n

\u201cThis was a completely manual process, using reports imported to Excel sheets and macros to identify mismatches,\u201d Venkatesan says. \u201cIt was labor-intensive, with multiple people performing the analysis in different regions. Now, we pull the various streams into an HD Insight cluster and use business rules to filter the royalty-bearing transactions to ensure parity.\u201d<\/p>\n

From there, the team had to apply machine learning to determine anomalies in payment variances. To do this, they had to understand the thought process and business rules used by analysts to determine the validity of a payment.<\/p>\n

\u201cThe Royalties team expects payments for a contract to fall between a minimum and maximum range. Outside the range is an outlier,\u201d says Geeth Priya Namasivayam, also a Microsoft Digital software engineer. \u201cTo truly automate is not just to identify the anomaly, but to determine which dimension caused the anomaly. We use a quantile regression algorithm, which allows us to clearly define the variance based on cause.\u201d<\/p>\n

Machine learning requires a lot of data to learn. To get a big enough data sample, the team loaded 24 months of payment and transaction data into the pipeline. This provided transactions, payment history, seasonality, and launch events, which allowed the team to create a model that included all typical variances.<\/p>\n

\u201cThe machine-learning job is initiated as each statement is released, using current data for the contract and loading historical data for comparison,\u201d Namasivayam says. \u201cIf an anomaly is detected, the data set is reduced and additional dimensions such as seasonality, launch periods, and geography are used to isolate the source of the anomaly.\u201d<\/p>\n

The goal was to make the application as easy to use as possible.<\/p>\n

\u201cThe user interface is very simple. A statement is classified into three categories: green meaning no anomalies, yellow meaning an anomaly is detected but it\u2019s due to seasonality or launch, and red meaning an unexplained anomaly is detected,\u201d Namasivayam says. \u201cWe incorporated a thumbs-up or thumbs-down button, which allows the user to validate the classification and provide learning reinforcement for the algorithm.\u201d<\/p>\n

Trying it out<\/strong><\/p>\n

With the data pipeline and algorithm in place, the team was ready for real-life testing.<\/p>\n

\u201cWe had to prove to our stakeholders that our algorithm was accurate. This took several months of automated validation and manual review of the data by the finance team. We had to build trust,\u201d Venkatesan says.<\/p>\n

After the accuracy of the system was validated, the proof-of-concept was moved into production.<\/p>\n

\u201cWith confidence in reconciliation accuracy and no anomalies, the finance team was comfortable to automatically approve transactions for amounts below certain thresholds,\u201d Venkatesan says.<\/p>\n

Embracing a new technology that the team would trust to take on an important process regulated by federal Sarbanes-Oxley Act (SOX) rules was not an easy leap for the operations team to make. Microsoft\u2019s internal Controls and Compliance teams, along with the Corporate Finance Controls group, partnered closely to ensure the risks were properly addressed.<\/p>\n

\u201cIt was uncomfortable and a little scary yielding an important process to automation and machine learning,\u201d says Mohammad Shafaqat, global process owner on the Royalties team in Microsoft Finance. \u201cAs things went along, we learned that we could keep the machine in check by providing ongoing feedback.\u201d<\/p>\n

This work has not only delivered significant time savings but has also improved morale in many ways. Professionals no longer perform painful mechanical work but instead are focused on building partner relationships and delivering insights.<\/p>\n

It\u2019s all making a major difference, Amberker says.<\/p>\n

\u201cWith Azure ML services, approximately 90 percent of Xbox statements and payments are now automated, saving 20,000 hours per year in the steady state,\u201d he says.<\/p>\n

Amberker says this is just the start; the anomaly detection-based engineering approach is now being applied in many other functions at Microsoft. \u201cStay tuned for more updates,\u201d he says.<\/p>\n

\"Related<\/p>\n