{"id":10969,"date":"2020-02-19T15:02:45","date_gmt":"2020-02-19T23:02:45","guid":{"rendered":"https:\/\/www.microsoft.com\/insidetrack\/blog\/?p=10969"},"modified":"2023-06-11T16:01:50","modified_gmt":"2023-06-11T23:01:50","slug":"microsoft-reinvents-sales-processing-and-financial-reporting-with-azure","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/insidetrack\/blog\/microsoft-reinvents-sales-processing-and-financial-reporting-with-azure\/","title":{"rendered":"Microsoft reinvents sales processing and financial reporting with Azure"},"content":{"rendered":"
\n
\n
<\/div>\n
This content has been archived, and while it was correct at time of publication, it may no longer be accurate or reflect the current situation at Microsoft.<\/p>\n<\/div>\n<\/div>\n
Moving the company\u2019s revenue reporting platform to Microsoft Azure is giving Microsoft Digital the opportunity to redesign the platform\u2019s infrastructure and functionality. With the major components in Azure, we\u2019ve already seen how Spark Streaming and Azure Data Factory have made dramatic improvements in the platform\u2019s performance and scalability. As our journey to host this solution in Azure continues, we\u2019re finding new ways to improve it with Azure capabilities.<\/p>\n
Microsoft Digital has moved the company\u2019s revenue reporting platform, MS Sales, from on-premises datacenters to the cloud and Microsoft Azure. This is more than just a move to the cloud\u2014it\u2019s an opportunity to reimagine and redesign the way MS Sales infrastructure functions. To prepare for the migration, we examined several options for hosting MS Sales in Azure and came away with a clear direction for transition design, planning, and deployment.<\/p>\n
MS Sales for revenue and sales data<\/h2>\n
MS Sales manages Microsoft product and service revenue data. Transaction data is conformed, aggregated, and enriched by MS Sales to provide accurate revenue reporting and analysis. MS Sales gives a consistent view of Microsoft businesses and production revenue, and it enables better, faster strategic decisions. People can query purchase, sales, budget, and forecast data and drill down to see more transaction details.<\/p>\n
The MS Sales environment includes the following:<\/p>\n
\n
Many thousands of users at Microsoft, which includes people in finance, sales, marketing, and executives.<\/li>\n
Large collection of internal and external data sources.<\/li>\n
Many years of sales data, depending on the reporting pivot.<\/li>\n
Millions of daily transactions.<\/li>\n<\/ul>\n
MS Sales publishes data that\u2019s aligned with the Microsoft financial calendar. The publishing processes include daily, weekly, and\u2014the most critical\u2014fiscal month-end (FME) data for restatement, forecasting, and budgeting. The restatement includes the attribution of past revenue into current business structures and product lines. The system needed more processing capacity to keep pace with the expanding number of revenue records and details.<\/p>\n
We had been experiencing several challenges with the on-premises MS Sales environment, including limited scalability and agility, a complex ecosystem, cumbersome data-processing models, and increasing costs. The transition of Microsoft\u2019s business model to services has created an exponential curve in the number of transactions. With the legacy design, we would reach the limit of processing to meet our service-level agreements (SLAs) to users. The goal in migrating MS Sales to Azure was to address these challenges and position MS Sales for success well into the future.<\/p>\n
MS Sales previously<\/h3>\n
The MS Sales system was built 20 years ago to report Microsoft revenue. It\u2019s the company\u2019s standard revenue reporting system, and it is pivotal to strategic business, management, and financial decisions. Timely, accurate sales data is crucial to assessing Microsoft performance and maintaining a competitive position.<\/p>\n
The original solution for MS Sales was hosted in on-premises datacenters and included:<\/p>\n
\n
Fifty-three servers<\/li>\n
Thirty Microsoft SQL Server databases<\/li>\n
Approximately 35 TB of data storage<\/li>\n<\/ul>\n
MS Sales components and functionality<\/h4>\n
The original MS Sales architecture ingested data, processed and transformed significant data, created star schema data marts, and distributed these marts for querying and consumption by other systems. Querying was primarily via Microsoft Reporting Analytics (MSRA), an Excel add-in that generates and executes SQL queries based on the user\u2019s definitions. The architecture supported five major functions:<\/p>\n
\n
Ingestion.<\/strong>\u2002MS Sales ingests data from more than one thousand sources. Most of the sources are external partners, such as manufacturers, distributors, and retailers. With the original MS Sales, data was ingested through a process that was supported by SQL jobs, and Windows services data was batch loaded into the MS Sales Warehouse (a SQL database). One of our focus points in the next phase was to stream partner and customer data for ingestion.<\/li>\n
Warehouse.<\/strong>\u2002Ingested data ended up in the MS Sales Warehouse SQL Server database. Data from other sources\u2014such as forecasting, budgeting, and planning data\u2014and SAP were directly loaded here. The warehouse held approximately 5 TB of data.<\/li>\n
Factory.<\/strong>\u2002Data from the warehouse was log-shipped to the factory server, where all major business data was processed, and the final MS Sales database produced for reporting. Total factory data was about 4.5 TB of uncompressed data. Jobs were scheduled via NT Batch, and 15 complex stored procedures were executed. The factory server required a high level of processing capability. For example, in a continuing push to adopt hardware that could support MS Sales functionality, we moved the factory to a two-server architecture, using two 72-core servers with 2 TB of RAM to run factory processes in parallel.<\/li>\n
Distribution.<\/strong>\u2002As the MS Sales database was created, it was cloned to 18 reporting servers. Each reporting server hosted slightly more than 500 GB of compressed data. MS Sales data was copied to more than 220 other environments, where it was merged with other data for various reporting needs.<\/li>\n
User experience.<\/strong>\u2002The MSRA tool offers ad hoc query definition in a pivot table. Users could immediately schedule or execute queries. MSRA generates Transact-SQL (T-SQL) based on source-specific metadata. The T\u2011SQL is executed by a middle-tier server, and results are stored or retrieved immediately. Results are presented in an Excel PivotTable, where users can take full advantage of Excel capabilities. We have many thousands of MSRA users who can run ad hoc queries against MS Sales. Approximately three million queries were run in the past fiscal year.<\/li>\n<\/ul>\n
Figure 1 illustrates the MS Sales on-premises infrastructure.<\/p>\n