At Microsoft we have integrated our ServiceNow environment with Microsoft Azure Monitor to create a comprehensive, end-to-end monitoring solution that helps us to ensure performance, reliability, and service excellence across our ServiceNow environment. Combining ServiceNow with Azure Monitor helps us monitor the availability, performance, and usage of the ServiceNow platform and provide actionable insights to our engineering team. Our new monitoring environment results in greater platform reliability and health and a better experience for all our ServiceNow users.
We on the Microsoft Digital Employee Experience (MDEE) team support more than 170 thousand employees and partners in more than 150 countries and regions. We use ServiceNow to meet several service and support needs for our organization, including incident management, problem management, service-level agreement (SLA) measurement, request fulfillment, self-service support, and knowledge management. ServiceNow provides built-in, real-time reporting and dashboarding for the aforementioned areas. Global Helpdesk, our support and service management organization, receives more than 3,000 incoming service requests each day.
We have been using ServiceNow as our primary service and support platform since 2015. In July 2019, we established a strategic partnership with ServiceNow to help accelerate digital transformation for our customers. As part of this collaborative effort, we have been using the ServiceNow platform to automate internal support processes while helping ServiceNow to develop and improve their platform to create a more reliable, efficient, and trusted framework for employee support.
[Learn how we’re using Microsoft Teams and ServiceNow to enhance end-user support.]
Examining ServiceNow monitoring at Microsoft
We’ve used ServiceNow’s native reporting features to better understand our support environment since we started using the product in 2015. ServiceNow offers a robust reporting module. Users can generate ad hoc and scheduled reports in ServiceNow to observe current data or visualize and analyze other data, including most major ServiceNow module data. ServiceNow provides an integrated report designer interface and integrated dashboarding capabilities.
However, the massive scope of our ServiceNow environment makes end-to-end visibility of ServiceNow functionality difficult to achieve with the built-in ServiceNow reporting features. While we can report on many aspects of ServiceNow’s business and process-oriented functionality, we wanted to capture a more comprehensive instrumentation of ServiceNow functionality from the perspective of all roles that use ServiceNow. These roles include support agents, engineers, product owners, program managers, external partners, and users. We identified several areas of ServiceNow monitoring and reporting that we wanted to improve, including:
- Overall platform health monitoring. While ServiceNow’s reporting provided great data for the processes happening within each module, we couldn’t use it to observe the platform itself, including the functional components and architecture that kept ServiceNow running.
- End-to-end monitoring of business and technical functionality. We wanted to monitor ServiceNow from both business-oriented and technology-oriented perspectives, end to end. Our support and service metrics were important, but so was knowing that the systems that our support and service teams use were operating properly and effectively.
- Proactive reporting. We wanted a less reactive monitoring environment in which we could actively search for issues and determine the health status of any or all parts of our ServiceNow architecture and functionality.
- Near real-time monitoring. We wanted near real-time monitoring for all aspects of ServiceNow, not just the business-process results from each module, such as incident management or problem management. We needed to know immediately if an underlying technical component had an issue.
- Comprehensive objective reporting. We wanted a better way to monitor ServiceNow from the outside, with an objective monitoring tool that wasn’t part of the ServiceNow platform.
Extending ServiceNow monitoring with Azure Monitor
To address the challenges and make the improvements we identified, we in MDEE have integrated Microsoft Azure Monitor with ServiceNow to create more robust reporting capabilities. Our solution is based in Azure Monitor, using Azure Monitor Application Insights to store data from ServiceNow that we’ve extracted and transformed using both ServiceNow’s native reporting data and an internally developed app that translates ServiceNow data and stores it in Application Insights.
Capturing complete data from ServiceNow
ServiceNow provides several native data-output streams. While the end-to-end process of capturing ServiceNow data in Application Insights involved a complex data flow, retrieving raw data from ServiceNow was simple. We used two of ServiceNow’s default data-output streams to capture data for Application Insights:
- Node and semaphore information from xmlstats.do. Xmlstats.do contains comprehensive data for ServiceNow node and semaphore data. ServiceNow nodes host core functionality, while semaphores define how the load for those nodes is managed and distributed.
- Usage and business process data from transaction logs. ServiceNow transaction logs contain records for all browser activity and REST API usage for an instance, including performance metrics such as response time and network latency.
Bridging the gap from ServiceNow to Application Insights with Azure Function Apps
We’re using two approaches to extract this data from ServiceNow, with one approach for each of the data types. Our push model assembles data in ServiceNow and pushes it to a REST API endpoint for consumption, while the pull model collects data from a ServiceNow API endpoint. Both models are based in a Microsoft Azure Function App that we call the Translator App. We use the Translator App to translate ServiceNow’s telemetry output, transform the data into usable JSON formatted output, and place the data in Application Insights.
Push model for platform data in nodes and semaphores
Our push model is rooted in an automation routine in ServiceNow that assembles and exports node and semaphore data to the Translator app. This data is collected from the xmlstats.do page in ServiceNow in five-minute increments and pushed to the Translator app in 15-minute intervals. The automation for the push model is built entirely in ServiceNow Studio, ServiceNow’s built-in editor for custom applications. The push model collects data for system health, notification events, and other key datapoints within the ServiceNow node or semaphore architecture. The push model works in near real-time and is best suited to smaller collections of datapoints or when we must perform calculations on the data points prior to exporting the data.
Pull model business data in transaction logs
The pull model is designed to manage a much larger data load. We use the pull model to extract all transaction log data from ServiceNow and place it into Application Insights. The pull model’s functionality is based in the Translator app. Using Azure Function App capability, the Translator app accesses a ServiceNow REST API endpoint and pulls data from a read replica database. Because we use a read replica database in ServiceNow, requests from the pull model don’t affect production functionality or affect the performance of production databases. We pull approximately 2 million to 6 million transactions each day from ServiceNow into Application Insights by using the Translator app.
Creating actionable reporting with Application Insights
After our Translator app transforms and loads data from ServiceNow into Application Insights, we can take advantage of a robust and highly configurable reporting and monitoring environment. We can use Application Insights to monitor technical and business processes across the entire ServiceNow platform. Our Application Insights instance is the primary source for all our querying, reporting, and dashboarding activities for ServiceNow.
Reporting and dashboarding
With the data in Application Insights, we can use any reporting tools included with or compatible with Microsoft Azure Monitor to query and visualize our data. We use several important reporting tools, including:
- Microsoft Power BI. We use Power BI for most of our dashboarding needs for end-to-end ServiceNow monitoring. Power BI uses a simple, yet powerful interface that users, like our program managers, can use to create their own insights into ServiceNow performance and functionality.
- Log Analytics. Our engineers use Log Analytics extensively to quickly locate and identify specific transaction log behavior or trends. Log Analytics query language makes it easy for engineers to quickly find what they’re searching for, especially if they’re familiar with the underlying dataset and schema of the ServiceNow data sources.
Microsoft Azure Monitor provides a flexible and consistent platform on which we’ve built a broad and comprehensive monitoring solution for ServiceNow. Our new solution, built in Azure, helps us ensure that we deliver a reliable service as our ServiceNow environment grows and changes with our business. Combining ServiceNow with Azure Monitor helps us monitor the availability, performance, and usage of the platform. We can provide actionable insights to our engineering team, which results in higher platform reliability and health and a better experience for all our ServiceNow users.
- ServiceNow has the extensibility to write custom apps to be able pull data out of your instances and Microsoft Azure gives you the flexibility to be able to ingest the data.
- Monitor system health and create alerts and notifications in near real-time when certain metrics fall below thresholds. With the new solution, we can capture this data across the entire ServiceNow scope.
- Track service health over time, correlate, and trend data. Historical and trend-based reporting helps us to anticipate upcoming events and adjust to meet upcoming demand. We can also more accurately predict failures and outages across the environment.
- Report on feature usage within ServiceNow. This data empowers our program managers and engineers to understand how our customers are using or not using the features and modules that we release to determine return on our investment.
