Royalty statements quality review can be a royal pain.
Like many companies, Microsoft pays royalties for games, videos, content, software, and other creative work it uses in its products and services.
The company pays a lot in royalties—nearly $5 billion a year.
“The quality review and approval of payments has been a time-consuming manual process for our operations teams,” says Rohit Amberker, a finance director on Microsoft Finance’s royalty team. “We review nearly 30,000 statements each year.”
Now that frustration is going away.
How?
Microsoft Digital has built an experience that digitizes the company’s royalty payment process from end to end, says Jagannathan Venkatesan, principal group engineering manager for Microsoft Digital, which provides IT support for Microsoft Finance.
[Go here to learn more about Driving Microsoft’s transformation with AI. Click here to find out how Microsoft increases sales by using AI for lead qualification. Learn how Microsoft uses machine learning to develop smart energy solutions.]
“The Royalties team has been making significant strides in our service-maturity journey, enabling our business partners to close our month-end activities in a stable manner,” Venkatesan says. “Having accomplished a successful migration from an on-premises system to a cloud-based distributed system performing at scale, we focused on helping our business partners with the effectiveness of operational processes.”
Venkatesan adds: “We targeted the statement-approval process, a monthly recurring activity, involving many thousands of contracts, currently performed manually. The ML/AI-based solution for statement approval dovetails nicely into a larger effort across Microsoft Digital to digitize how it supports internal business partners like Microsoft Finance.”
Complexity untangled
Part of the challenge has been the complex nature of royalty payments, says Katie Lencioni, a principal program manager at Microsoft Digital.
“We collect all transactions that are subject to a royalty payment based on views, downloads, or plays depending on the type of content,” she says. “From there, we calculate the payment amount according to the contract of the content owner, create a statement, and ensure the payments are processed.”
With 30,000 statements per year, there is ample opportunity for errors.
“We perform three-way recon, comparing source transactions from Microsoft Marketplace with auto-calculated statements and against a third independent source,” says Rajmohan Venkatesan, a software engineer at Microsoft Digital.
“This was a completely manual process, using reports imported to Excel sheets and macros to identify mismatches,” Venkatesan says. “It was labor-intensive, with multiple people performing the analysis in different regions. Now, we pull the various streams into an HD Insight cluster and use business rules to filter the royalty-bearing transactions to ensure parity.”
From there, the team had to apply machine learning to determine anomalies in payment variances. To do this, they had to understand the thought process and business rules used by analysts to determine the validity of a payment.
“The Royalties team expects payments for a contract to fall between a minimum and maximum range. Outside the range is an outlier,” says Geeth Priya Namasivayam, also a Microsoft Digital software engineer. “To truly automate is not just to identify the anomaly, but to determine which dimension caused the anomaly. We use a quantile regression algorithm, which allows us to clearly define the variance based on cause.”
Machine learning requires a lot of data to learn. To get a big enough data sample, the team loaded 24 months of payment and transaction data into the pipeline. This provided transactions, payment history, seasonality, and launch events, which allowed the team to create a model that included all typical variances.
“The machine-learning job is initiated as each statement is released, using current data for the contract and loading historical data for comparison,” Namasivayam says. “If an anomaly is detected, the data set is reduced and additional dimensions such as seasonality, launch periods, and geography are used to isolate the source of the anomaly.”
The goal was to make the application as easy to use as possible.
“The user interface is very simple. A statement is classified into three categories: green meaning no anomalies, yellow meaning an anomaly is detected but it’s due to seasonality or launch, and red meaning an unexplained anomaly is detected,” Namasivayam says. “We incorporated a thumbs-up or thumbs-down button, which allows the user to validate the classification and provide learning reinforcement for the algorithm.”
Trying it out
With the data pipeline and algorithm in place, the team was ready for real-life testing.
“We had to prove to our stakeholders that our algorithm was accurate. This took several months of automated validation and manual review of the data by the finance team. We had to build trust,” Venkatesan says.
After the accuracy of the system was validated, the proof-of-concept was moved into production.
“With confidence in reconciliation accuracy and no anomalies, the finance team was comfortable to automatically approve transactions for amounts below certain thresholds,” Venkatesan says.
Embracing a new technology that the team would trust to take on an important process regulated by federal Sarbanes-Oxley Act (SOX) rules was not an easy leap for the operations team to make. Microsoft’s internal Controls and Compliance teams, along with the Corporate Finance Controls group, partnered closely to ensure the risks were properly addressed.
“It was uncomfortable and a little scary yielding an important process to automation and machine learning,” says Mohammad Shafaqat, global process owner on the Royalties team in Microsoft Finance. “As things went along, we learned that we could keep the machine in check by providing ongoing feedback.”
This work has not only delivered significant time savings but has also improved morale in many ways. Professionals no longer perform painful mechanical work but instead are focused on building partner relationships and delivering insights.
It’s all making a major difference, Amberker says.
“With Azure ML services, approximately 90 percent of Xbox statements and payments are now automated, saving 20,000 hours per year in the steady state,” he says.
Amberker says this is just the start; the anomaly detection-based engineering approach is now being applied in many other functions at Microsoft. “Stay tuned for more updates,” he says.
