At Microsoft, our Microsoft Digital Employee Experience (MDEE) team is using Microsoft Azure telemetry tools to get key insights on our business processes that flow through our SAP instance, one of the largest in the world. Our new platform provides our leadership with a comprehensive view of our business-process health and allows our engineering teams to create a more robust and efficient SAP environment.
Like many enterprises, we use SAP—the global enterprise resource planning (ERP) software solution—to run our various business operations. Our SAP environment is critical to our business performance, and we integrate it into most of our business processes. SAP offers functionality for enterprise services at Microsoft, such as human resources, finance, supply-chain management, and commerce. We use a wide variety of SAP applications, including:
- SAP S/4HANA
- ERP Central Component (ECC)
- Global Trade Screening (GTS)
- Business Integrity Screening (BIS) on S4
- Master Data Governance (MDG) on S4
- Governance, Risk, Compliance (GRC)
- Revenue Management, Contract Accounting (RMCA)
- OEM Services (OER)
- SAP SaaS (Ariba, IBP, Concur, SuccessFactors)
Since 2018, Microsoft’s instance of SAP is 100 percent migrated to Microsoft Azure. This project entailed moving all SAP assets to more than 800 Azure virtual machines and numerous cloud services.
We approached the migration by using both vertical and horizontal strategies.
From a horizontal standpoint, we migrated systems in our SAP environment that were low risk—training systems, sandbox environments, and other systems that weren’t critical to our business function. We also looked at vertical stacks, taking entire parts of our SAP landscape and migrating them as a unified solution.
We gained experience with both migration scenarios, and we learned valuable lessons in the early migration stages that helped us smoothly transition critical systems later in the migration process.
[Unpack how we’re optimizing SAP for Microsoft Azure. | Discover how we’re protecting Microsoft’s SAP workload with Microsoft Sentinel. | Explore how we’re unlocking Microsoft’s SAP telemetry with Microsoft Azure.]
Operating as Microsoft Azure-native
At Microsoft, we develop and host all new SAP infrastructure and systems on Microsoft Azure. We’re using Azure–based cloud infrastructure and SAP–native software as a service (SaaS) solutions to increase our architecture’s efficiency and to grow our environment with our business. The following graphic represents our SAP landscape on Azure.
The benefits of SAP on Microsoft Azure
SAP on Microsoft Azure provides several benefits to our business, many of which have resulted in significant transformation for our company. Some of the most important benefits include:
- Business agility. With Microsoft Azure’s on-demand SAP–certified infrastructure, we’ve achieved faster development and test processes, shorter SAP release cycles, and the ability to scale instantaneously on demand to meet peak business usage.
- Efficient insights. SAP on Microsoft Azure gives us deeper visibility across our SAP landscape. On Azure, our infrastructure is centralized and consolidated. We no longer have our SAP infrastructure spread across multiple on-premises datacenters.
- Efficient real-time operations and integration. We can leverage integration with other Microsoft Azure technologies such as Internet of Things (IoT) and predictive analytics to enable real-time capture and analysis of our business environment, including areas such as inventory, transaction processing, sales trends, and manufacturing.
- Mission-critical infrastructure. We run our entire SAP landscape—including our most critical infrastructure—on Microsoft Azure. SAP on Azure supports all aspects of our business environment.
Identifying potential for improved monitoring
As we examined our SAP environment on Microsoft Azure, we found several key areas where we could improve our monitoring and reporting experience:
- Monitoring SAP from external business-process components. External business process components had no visibility into SAP. Our monitoring within individual SAP environments provided valuable insight into SAP processes, but we needed a more comprehensive view. SAP is just one component among many in our business processes, and the owners of those business processes didn’t have any way to track their processes after they entered SAP.
- Managing and viewing end-to-end processes. It was difficult to manage and view end-to-end processes. We couldn’t capture the end-to-end process status to effectively monitor individual transactions and their progress within the end-to-end process chain. SAP was disconnected from end-to-end monitoring and created a gap in our knowledge of the entire process pipeline.
- Assessing overall system health. We couldn’t easily assess overall system health. Our preexisting monitoring solution didn’t provide a holistic view of the SAP environment and the processes with which it interacted. The overall health of processes and systems was incomplete because of missing information for SAP, and issues that occurred within the end-to-end pipeline were difficult to identify and problematic to troubleshoot.
Our SAP on Microsoft Azure environment was like a black box to many of our business-process owners, and we knew that we could leverage Azure and SAP capabilities to improve the situation. We decided to create a more holistic monitoring solution for our SAP environment in Azure and the business processes that defined Microsoft operations.
Creating a telemetry solution for SAP on Microsoft Azure
The distributed nature of our business process environment led us to examine a broader solution—one that would provide comprehensive telemetry and monitoring for our SAP landscape and any other business processes that constituted the end-to-end business landscape at Microsoft. The following goals drove our implementation:
Integrate comprehensive telemetry into our monitoring.
- Move toward holistic health monitoring of both applications and infrastructure.
- Create a complete view of end-to-end business processes.
- Create a modern, standards-based structure for our monitoring systems.
Guiding design with business-driven monitoring and personas
We adopted a business-driven approach to building our monitoring solution. This approach examines systems from the end-user perspective, and in this instance, the personas represented three primary business groups: business users, executives, and engineering teams. Using the synthetic method, we planned to build our monitoring results around what these personas wanted and needed to observe within SAP and the end-to-end business process, including:
- Business user needs visibility into the status of their business transactions as they flow through the Microsoft and SAP ecosystem.
- Executives need to ensure that our business processes are flowing smoothly. If there are critical failures, they need to know before customers or partners discover them.
- Engineers need to know about business-process issues before those issues affect business operations and lead to customer-satisfaction issues. They need end-to-end visibility of business transactions through SAP telemetry data in a common consumption format.
Creating end-to-end telemetry with our Unified Telemetry Platform
The MDEE team developed a telemetry platform in Microsoft Azure that we call the Unified Telemetry Platform (UTP). UTP is a modern, scalable, dependable, and cost-effective telemetry platform that’s used in several different business-process monitoring scenarios in Microsoft, including our SAP–related business processes.
UTP is built to enable service maturity and business-process monitoring across MDEE. It provides a common telemetry taxonomy and integration with core Microsoft data-monitoring services. UTP enables compliance with and maintenance of business standards for data integrity and privacy. While UTP is the implementation we chose, there are numerous ways to enable telemetry on Microsoft Azure. For additional considerations, access Best practices for monitoring cloud applications on the Azure documentation site.
Capturing telemetry with Microsoft Azure Monitor
To enable business-driven monitoring and a user-centric approach, UTP captures as many of the critical events within the end-to-end process landscape as possible. Embracing comprehensive telemetry in our systems meant capturing data from all available endpoints to build an understanding of how each process flowed and which SAP components were involved. Azure Monitor and its related Azure services serve as the core for our solution.
Microsoft Azure Application Insights
Application Insights provides a Microsoft Azure–based solution with which we can dig deep into our Azure–hosted SAP landscape and extract all necessary telemetry data. By using Application insights, we can automatically generate alerts and support tickets when our telemetry indicates a potential error situation.
Microsoft Azure Log Analytics
Infrastructure telemetry such as CPU usage, disk throughput, and other performance-related data is collected from Azure infrastructure components in the SAP environment by using Log Analytics.
Microsoft Azure Data Explorer
UTP uses Microsoft Azure Data Explorer as the central repository for all telemetry data sent through Application Insights and Microsoft Azure Monitor Logs from our application and infrastructure environment. Azure Data Explorer provides enterprise big-data interactive analytics; we use the Kusto query language to connect the end-to-end transaction flow for our business processes, for both SAP process and non–SAP processes.
Microsoft Azure Data Lake
UTP uses Microsoft Azure Data Lake for long-term cold-data storage. This data is taken out of the hot and warm streams and kept for reporting and archival purposes in Azure Data Lake to reduce the cost associated with storing large amounts of data in Microsoft Azure Monitor.
Constructing with definition using common keys and a unified platform
UTP uses Application Insights, Microsoft Azure Data Explorer, and Microsoft Azure Data Lake as the foundation for our telemetry data. This structure unifies our data by using a common schema and key structure that ties telemetry data from various sources together to create a complete view of business-process flow. This telemetry hub provides a central point where telemetry is collected from all points in the business-process flow—including SAP and external processes—and then ingested into UTP. The telemetry is then manipulated to create comprehensive business-process workflow views and reporting structures for our personas.
Common schema
UTP created a clearly defined common schema for business-process events and metrics based on a Microsoft-wide standard. That schema contains the metadata necessary for mapping telemetry to services and into processes, and it allows for joins and correlation across all telemetry.
Common key
As part of the common schema for business process events, the design includes a cross-correlation vector (XCV) value, common to all stored telemetry and transactions. By persisting a single value for the XCV and populating this attribute for all transactions and telemetry events related to a business process, we can connect the entire process chain related to an individual business transaction as it flows through our extended ecosystem.
Multilayer telemetry concept for SAP
For SAP on Microsoft Azure, our MDEE team focused on four specific areas for telemetry and monitoring:
- SAP Business Process layer
- SAP Application Foundation layer
- Infrastructure layer
- Surrounding API layer
The result was holistic telemetry and monitoring across these layers, a structure that leverages Microsoft Power BI as the engine behind our reporting and dashboarding functionality.
Our MDEE team created reporting around business-driven monitoring and constructed standard views and dashboards that offer visibility into important areas for each of the key business personas. Dashboards are constructed from Kusto queries, which are automatically translated in the Microsoft Power BI M formula language. For each persona, we’ve enabled a different viewpoint and altitude of our business process that allows the persona to view the SAP monitoring information that’s most critical to them.
Microsoft Azure Monitor for SAP Solutions
Microsoft previously announced the launch of Microsoft Azure Monitor for SAP Solutions (AMS) in public preview—an Azure-native monitoring solution for customers who run SAP workloads on Azure. With AMS, customers can view telemetry of their SAP landscapes within the Azure portal and efficiently correlate telemetry between various layers of SAP. AMS is available through Microsoft Azure Marketplace in the following regions: East US, East US 2, West US 2, West Europe, and North Europe. AMS doesn’t require a license fee.
Our MDEE team worked in close collaboration with Microsoft Azure product teams to build and release SAP NetWeaver provider in Microsoft Azure Monitor for SAP solutions.
- The SAP NetWeaver provider in Microsoft Azure Monitor for SAP Solutions enables SAP on Microsoft Azure customers to monitor SAP NetWeaver components and processes on Azure in the Azure portal. The SAP NetWeaver provider includes default visualizations and alerts that can be used out of the box or customized to meet customer requirements.
- SAP NetWeaver telemetry is collected by configuring the SAP NetWeaver provider within AMS. As part of configuring the provider, customers are required to provide the host name (Central, Primary, and/or Secondary Application server) of SAP system and its corresponding Instance number, Subdomain, and System ID (SID).
For more information, go to AMS quick start video and SAP NetWeaver monitoring-Azure Monitoring for SAP Solutions.
Our telemetry platform provides benefits across our SAP and business-process landscape. We have created a solution that facilitates end-to-end SAP business-process monitoring, which in turn enables our key personas to do their jobs better.
Persona benefits
Benefits for each persona include the following:
- Business users no longer need to create service tickets to get the status of SAP transaction flows. They can examine our business processes from end to end, including SAP transactions and external processes.
- Executives can trust that their business processes execute seamlessly and that any errors are proactively addressed with no impact to customers or partners.
- Engineers no longer need to check multiple SAP transactions to investigate business-process issues and identify in which step the business process failed. They can improve their time-to-detect and time-to-resolve numbers with the correct telemetry data and avoid business disruption for our customers.
Organization-wide benefits
The benefits of our platform extend across Microsoft by providing:
- End-to-end visibility into business processes. Our Unified Telemetry Platform (UTP) provides visibility into business processes across the organization, which then facilitates better communication and a clearer understanding of all parts of our business. We have a more holistic view of how we’re operating, which helps us work together to achieve our business goals.
- Decreased time to resolve issues. Our visibility into business processes informs users at all levels when an issue occurs. Business users can examine the interruption in their workflow, executives are notified of business-process delays, and engineers can identify and resolve issues. This activity all occurs before our customers are affected.
- More efficient business processes. Greater visibility leads to greater efficiency. We can demonstrate issues to stakeholders quickly, everyone involved can recognize areas for potential improvement, and we can monitor modified processes to ensure that improvement is happening.
We learned several important lessons with our UTP implementation for SAP on Microsoft Azure. These lessons helped inform our progress of UTP development, and they’ve given us best practices to leverage in future projects, including:
- Perform a proper inventory of internal processes. You must be aware of events within a process before you can capture them. Performing a complete and informed inventory of your business processes is critical to capturing the data required for end-to-end business-process monitoring.
- Build for true end-to-end telemetry. Capture all events from all processes and gather telemetry appropriately. Data points from all parts of the business process—including external components—are critical to achieving true end-to-end telemetry.
- Build for Microsoft Azure-native SAP. SAP is simpler to manage on Azure and instrumenting SAP processes becomes more efficient and effective when SAP components are built for Azure.
- Encourage data-usage models and standards across the organization. Data standards are critical for an accurate end-to-end view. If data is stored in different formats or instrumentation in various parts of the business process, the end reporting results won’t accurately represent the business-process’ state.
We’re continuing to evaluate and improve as we discover new and more efficient ways to track our business processes in SAP. Some of our current focus areas include:
- Machine learning for predictive analytics. We’re using machine learning and predictive analytics to create deeper insights and more completely understand our current SAP environment. Machine learning also helps us anticipate growth and change in the future. We’re leveraging anomaly detection in Microsoft Azure Cognitive Services to track SAP business service-health outliers.
- Actionable alerting. We’re using Microsoft Azure Monitor alerts to create service tickets, generate service-level agreement (SLA) alerts, and provide a robust notification and alerts system. We’re working toward linking detailed telemetry context into our alerting system to create intelligent alerting that enables us to more accurately and quickly identify potential issues within the SAP environment.
- Telemetry-based automation. We’re using telemetry to enable automation and remediation within our environment. We’re creating self-healing scenarios to automatically correct common or easy-to-correct issues to create a more intelligent and efficient platform.
We’re continually refining and improving business-process monitoring of SAP on Microsoft Azure. This initiative has enabled us to keep key business users informed of business-process flow, provided a complete view of business-process health to our leadership, and helped our engineering teams create a more robust and efficient SAP environment. Telemetry and business-driven monitoring have transformed the visibility that we have into our SAP on Azure environment, and our continuing journey toward deeper business insight and intelligence is making our entire business better.
