![\"Microsoft](\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2021\/10\/ms-digital-technical-stories-300x112.png\")
As part of our continuing digital transformation journey, our Microsoft Digital Employee Experience (MDEE) team is constantly looking for ways to improve our business processes and detect issues and anomalies before they become serious problems.<\/p>\n
Sometimes these failures happen sporadically within the application framework and often go undetected. Even if a user detects an anomaly, they need to decide how to react, which is time consuming.<\/p>\n
To do this, we\u2019re using Microsoft Azure Anomaly Detector to examine transactions across our SAP environment, which helps us identify issues before they become problems. In turn this enables us to proactively improve the performance, consistency, and reliability of our entire SAP landscape.<\/p>\n
[Unpack how we\u2019re optimizing SAP for Microsoft Azure<\/a>. | Discover how we\u2019re protecting Microsoft\u2019s SAP workload with Microsoft Sentinel<\/a>. | Explore how we\u2019re upgrading Microsoft\u2019s core Human Resources system with SAP SuccessFactors<\/a>.]<\/em><\/p>\n At Microsoft, our SAP environment comprises many complex processes across multiple lines of business. To avoid having disparate environments and isolated monitoring and reporting data, we wanted to build a single codebase solution for monitoring and anomaly detection that each line-of-business can use with minimal code implementation.<\/p>\n We wanted to build intelligence to detect anomalies and inconsistencies in business process flow to improve platform health. Improved platform health improves engineering service-level agreements (SLAs) and reduces revenue loss by being proactive rather than reactive.<\/p>\n There were hundreds of areas that could benefit from anomaly detection in our SAP portfolio, but we wanted to identify a single area for our pilot project. In the Master Data Management (MDM) space, we create thousands of objects representing business entities such as customers and business partners.<\/p>\n Most of these objects are created by using an application programming interface (API), and no human interaction is needed. However, it\u2019s extremely difficult to identify if issues related to MDM are occurring in upstream systems, so we needed a way to capture issues in advance, proactively and quickly.<\/p>\n In the MDM space, we have SAP Master Data Governance (MDG) background processes, such Customer Master data creation, which run without any user interaction. Across various batch and scheduled jobs, process runtime varies based on data volume, time of day, time of year, and resource availability.<\/p>\n Understanding the potential for issues in each process and the larger process environment involves several challenging questions, including:<\/p>\n Detecting these issues by using human triage was difficult and time and resource intensive. Many issues went undetected, resulting in poor customer experience and the loss of potential revenue, in addition to lost capacity that could have been used for more productive purposes.<\/p>\n To solve this problem, we required a solution that was reliable, scalable, and easy to integrate with our SAP systems. The solution that we wanted would be process agnostic, implemented as a single codebase, and require no human intervention to detect issues.<\/p>\n The Microsoft Azure Anomaly Detector service, available within Microsoft Azure Cognitive Services, fits all our requirements.<\/p>\n The Anomaly Detector API enabled us to monitor and detect abnormalities in our data without having to know machine learning. The Anomaly Detector API’s algorithms adapt by automatically identifying and applying the best-fitting models to data, regardless of industry, scenario, or data volume, which greatly reduced our development efforts. Our primary steps were quite simple:<\/p>\n For Anomaly Detector to identify anomalies, it requires time-series data, which is a series of data points indexed in time-based order.<\/p>\n For example, your car might have embedded sensors that send information regarding engine health, speed, tire pressure, and gasoline capacity. This information about your car is constantly updated over time and, as such, it can be used as time\u2011series data.<\/p>\n Most data received throughout time can be manipulated to be time\u2011series data if it\u2019s a consistent data sequence with a time stamp. Time-series data with a single variable is considered a univariate series, while time-series data with more than one variable is considered a multivariate time series. Anomaly detector supports both univariate and multivariate series.<\/p>\n A data anomaly is outlying data that doesn’t fit within expected boundaries. The graphic below depicts the visual pattern of the time-series data with highlighted anomaly points in the time-series data. The graphic contains each of the time\u2011series data on the plot.<\/p>\n Data should be within minimum and maximum boundaries. In the figure, the boundary is filled with a light color. Most of the data points are within the expected boundaries. However, some data points that exceed the expected boundaries are highlighted in red in the figure, are data anomalies.<\/p>\n For example, a stock price that drops below the expected limit is a data anomaly. If the temperature reading of a power plant core exceeds the acceptable limit, the reading is a data anomaly, and the technicians at the power plant should be immediately notified so that they can act based on the anomaly.<\/p>\n Not all data anomalies are negative.<\/p>\n For example, if you have an article on your website that\u2019s trending and experiencing larger traffic volume than normal, you likely want to be notified about the anomaly.<\/p>\n Or, if you have an e\u2011commerce website and receive a sudden spike in product demand, you, as the product supplier, should be notified so that you can act immediately. The graphic below contains examples of inputs and results for the Anomaly Detector service.<\/p>\n To enable integration with our SAP portfolio, we\u2019ve implemented several decoupled software components. Each component has a specific use case, and we decouple business logic and the presentation layers to the extent possible. All application code is committed to a Microsoft Azure DevOps repository and is built as a Microsoft Azure-native solution.<\/p>\n As depicted in the graphic below, various SAP applications post their business-process logs into the Application Insights instance. The Web App hosts the core application, including the presentation layer and user interaction. The two Function Apps perform extract and process data from the Application Insights service and control interaction with the Anomaly Detector service. The Function Apps send the final results from the Anomaly Detector service for display and consumption in the Web App.<\/p>\n One of our key business processes that we onboarded to the Anomaly Detector\u2013based solution was the Master Data Management (MDM) business-partner creation that uses SAP Master Data Governance (MDG).<\/p>\n We constantly create and update business-partner data in our SAP system via API calls from various upstream tenants and front-end systems. Based on incoming telemetry sources, the Anomaly Detector solution detects if there is a sudden drop in creation or update processes because of API failure or network issues.<\/p>\n The detection algorithm can detect these issues automatically, in real time, which helps our system users to take corrective action. This simple addition to the issue-detection process helps us supply a better customer experience and eliminates major negative effects on revenue.<\/p>\n We\u2019re planning to implement the same solution design across many other business processes, such as batch-job monitoring.<\/p>\n Currently, we have several hundred batch jobs that range from a runtime of a few seconds to several hours. It\u2019s extremely difficult to monitor them manually and individually.<\/p>\n Sometimes, due to system issues or transaction locking, these jobs take more time, further affecting downline processes. Anomaly detection will play a critical role in detecting those issues, creating automatic alerts, and reducing manual monitoring.<\/p>\n This application has many potential use cases across multiple business scenarios. We\u2019re planning to explore several of these use cases, including:<\/p>\n Using Microsoft Azure Anomaly Detector has enabled us to quickly and efficiently build a solution to detect abnormalities in our SAP processes without having to know machine learning. The Anomaly Detector API’s algorithms help us to identify issues before they become problems, thereby proactively improving the performance, consistency, and reliability of our entire SAP landscape.<\/p>\n As part of our continuing digital transformation journey, our Microsoft Digital Employee Experience (MDEE) team is constantly looking for ways to improve our business processes and detect issues and anomalies before they become serious problems. Sometimes these failures happen sporadically within the application framework and often go undetected. Even if a user detects an anomaly, […]<\/p>\n","protected":false},"author":133,"featured_media":9013,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"_hide_featured_on_single":false,"_show_featured_caption_on_single":true,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[1],"tags":[434,317,335,115,188],"coauthors":[646],"class_list":["post-9010","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-automation","tag-device-health","tag-machine-learning","tag-microsoft-azure","tag-sap","program-microsoft-digital-technical-stories","m-blog-post"],"jetpack_publicize_connections":[],"yoast_head":"\nUnderstanding the need for anomaly detection in SAP<\/h2>\n
\n
Assessing Microsoft Azure Cognitive Services and Microsoft Azure Anomaly Detector<\/h3>\n
\n
Using time-series data and data anomalies<\/h2>\n
![\"Time-series](\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2022\/11\/10254-figure1.png\")
![\"Examples](\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2022\/11\/10254-figure2.png\")
Using Microsoft Azure services to create a business solution<\/h3>\n
\n
The Function App transforms the data into JavaScript Object Notation (JSON) format with time-series subformatting. The second Function App captures the precompiled time-series data from the first Function App, makes a call to the Anomaly Detector service, and then retrieves the result. The Web App presentation layer displays the results in a graph format. Function App endpoints are protected with access tokens.<\/div>\n<\/li>\nImplementation architecture<\/h3>\n
![\"Diagram](\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2022\/11\/10254-figure3.png\")
Business implementation and benefits<\/h2>\n
![\"Key](\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2021\/10\/key-takeaways-300x74.png\")
<\/p>\n\n
![\"Related](\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2021\/10\/related_links-300x81.png\")
<\/p>\n\n