{"id":9028,"date":"2023-11-16T07:05:50","date_gmt":"2023-11-16T15:05:50","guid":{"rendered":"https:\/\/www.microsoft.com\/insidetrack\/blog\/?p=9028"},"modified":"2023-11-16T08:04:46","modified_gmt":"2023-11-16T16:04:46","slug":"moving-to-next-generation-siem-at-microsoft-with-microsoft-azure-sentinel","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/insidetrack\/blog\/moving-to-next-generation-siem-at-microsoft-with-microsoft-azure-sentinel\/","title":{"rendered":"Moving to next-generation SIEM at Microsoft with Microsoft Sentinel"},"content":{"rendered":"
Our internal security team works diligently 24 hours a day, 7 days a week to help protect Microsoft IP, its employees, and its overall business health from security threats.<\/p>\n
We recently implemented Microsoft Sentinel to replace a preexisting, on-premises solution for security information and event management (SIEM). With Microsoft Sentinel, we can ingest and appropriately respond to more than 20 billion cybersecurity events per day.<\/p>\n
Microsoft Sentinel supplies cloud-scale SIEM functionality that allows integration with crucial systems, provides accurate and timely response to security threats, and supports the SIEM requirements of our team.<\/p>\n
Our team is responsible for maintaining security and compliance standards across Microsoft. Managing the massive volume of incoming security-related data is critical to Microsoft\u2019s business health. Historically, we have performed SIEM using a third-party tool hosted on-premises in Microsoft datacenters.<\/p>\n
However, we recognized several areas in which they could improve their service by implementing a next-generation SIEM tool. Some of the challenges when using the old tool included:<\/p>\n
As part of our ongoing digital transformation, we\u2019re moving to cloud-based solutions with proven track records and active, customer-facing development and involvement. We need our technology stack to evolve at the speed of our business.<\/p>\n
[Read more about how we\u2019re securing our enterprise and responding to cybersecurity attacks with Microsoft Sentinel.<\/a> | Discover how we\u2019re improving our security by protecting elevated-privilege accounts at Microsoft.<\/a>]<\/em><\/p>\n In response to the challenges presented, we began assessing options for a new SIEM environment that would address the challenges positioning our team to manage continued growth of the cybersecurity landscape.<\/p>\n In partnership with the Microsoft Sentinel product team, our internal security division assessed whether Sentinel would be a suitable replacement for our previous solution. Sentinel is a Microsoft-developed, cloud-native enterprise SIEM solution that uses the cloud\u2019s agility and scalability to ensure rapid threat detection and response through:<\/p>\n To move to Microsoft Sentinel, we needed to verify that equivalent features and capabilities were available in the new environment. We aligned security teams across Microsoft to ensure that we met all requirements. Some of these teams had mature monitoring and detection definitions in place, and we needed to understand those scenarios to accommodate feature-performance requirements. The issues that our previous solution presented narrowed our focus with respect to whether Sentinel would work, including throughput, agility, and usability.<\/p>\n Throughout the assessment period and into migration, we worked closely with the Microsoft Sentinel product team to ensure that Microsoft Sentinel could provide the feature set we required. Our engagement with the Microsoft Sentinel team addressed two sets of needs simultaneously. We received significant incident-response benefits from Microsoft Sentinel while the product team worked with us as if we were a customer. This close collaboration meant that the product team could identify what enterprise-scale customers needed more quickly.<\/p>\n Not only were our requirements met, but we were able to provide feedback and testing for the Microsoft Sentinel product team. This helped them better serve their large customers that have similar challenges, requirements, and needs.<\/p>\n As we developed standards that met our new requirements, we also evaluated our previous SIEM solution\u2019s functionality to determine how it would transition to Microsoft Sentinel. We examined three key aspects of incoming security data ingestion and event detection:<\/p>\n Throughout this process, we worked with the Microsoft Security Operations team to evaluate detections end-to-end. They got involved in the detection and data-source refinement process and were exposed to how these detections and data sources would work in Microsoft Sentinel.<\/p>\n After feature parity and throughput capabilities were confirmed, we began the migration process from our previous solution to Microsoft Sentinel. Based on our initial testing, we added several implementation steps to ensure that our Sentinel environment would readily meet our security environment\u2019s needs.<\/p>\n Properly onboarding data sources was a critical component in our implementation and<\/em> one of the biggest benefits of the Microsoft Sentinel environment. With the massive amount of default connectors available in Sentinel, we were able to connect to most of our data sources without further customization. This included cloud data sources such as Microsoft Azure Active Directory, Microsoft Defender for Cloud, and Microsoft Defender. However, it also included on-premises data sources, such as Windows Events and firewall systems.<\/p>\n We also connected to several enrichment sources that supplied more information for threat-hunting queries and detections. These enrichments sources included data from human-resources systems and other nontypical data sources. We used playbooks to create many of these connections.<\/p>\n We keep Microsoft Sentinel data in hot storage for 90 days, using Kusto Query Language (KQL) queries for detections, hunting, and investigation. We also use Microsoft Azure Data Explorer for warm storage and Microsoft Azure Data Lake for cold storage and retrieval for up to two years.<\/p>\n Based on testing, we refined our detection definitions further in Sentinel to support better alert suppression and aggregation. We didn\u2019t want to overwhelm our Security Operations team with incidents. Therefore, we refined our detection definitions to include suppression logic when notification wasn\u2019t required and aggregation logic to ensure that similar and related events were grouped together and not surfaced as multiple, individual alerts.<\/p>\n We used dedicated clusters for Microsoft Azure Monitor Log Analytics to support the data-ingestion scalability we required. At a large enterprise scale, our previous solution was exceeding its capacity at 10 billion events per day. With dedicated clusters, we were able to accommodate that initial volume and add additional data sources to improve alert detection, thereby increasing our event ingestion to > 20 billion events per day.<\/p>\n Our environment required several customizations to Sentinel functionality, which we implemented by using standard Microsoft Sentinel features and extension capabilities to meet our needs while still staying within the boundaries of standard functionality. Using common features for customization made our changes to Sentinel easy to document and helped our security operations team better and more quickly understand and use the new features. We made several important customizations including:<\/p>\nModernizing SIEM with Microsoft Sentinel<\/h2>\n
Feature assessment and planning<\/h3>\n
\n
Defining and refining SIEM detections<\/h3>\n
\n
Implementation<\/h2>\n
Onboarding data sources<\/h3>\n
Refining detections<\/h3>\n
Increasing scale with the cloud<\/h3>\n
Customizing functionality<\/h3>\n
\n