{"id":8745,"date":"2024-03-13T07:36:16","date_gmt":"2024-03-13T14:36:16","guid":{"rendered":"https:\/\/www.microsoft.com\/insidetrack\/blog\/?p=8745"},"modified":"2024-03-13T07:43:41","modified_gmt":"2024-03-13T14:43:41","slug":"managing-microsoft-azure-expressroute-in-a-global-enterprise","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/insidetrack\/blog\/managing-microsoft-azure-expressroute-in-a-global-enterprise\/","title":{"rendered":"Transforming how Microsoft uses Microsoft Azure ExpressRoute"},"content":{"rendered":"
Our Microsoft Digital Employee Experience (MDEE) team manages one of the largest Microsoft Azure enterprise environments.<\/p>\n
Migrating our virtual machine resources into Microsoft Azure necessitated significant connectivity between our on-premises networks and Azure. We have many Microsoft Azure ExpressRoute circuits connecting our on-premises resources to our Azure environment, and to ensure proper management of a network environment this large, we\u2019ve moved our Microsoft Azure networking environment to a hub and spoke shared circuit model. This gives uniform access for our internal business groups to ExpressRoute and other Azure networking components, and it also allows us to centrally manage and monitor how we provision our Azure networking.<\/p>\n
Understanding Microsoft Azure ExpressRoute usage at Microsoft<\/h2>\n
Microsoft Azure ExpressRoute is the default connectivity method we use for connecting Microsoft Azure to our internal networks. It\u2019s the best solution in Azure for our large-scale network environment, compared to alternatives such as point-to-site or site-to-site virtual networks. We use ExpressRoute to connect Azure to datacenters, corporate headquarters, and regional offices across many different business groups. We use ExpressRoute for two primary purposes:<\/p>\n
\n
To connect internal or partner on-premises resources to a Microsoft Azure network containing virtual machines\u2014typically to form a contiguous, isolated network space between Azure and an on-premises network.<\/li>\n
To provide dedicated and measurable bandwidth between on-premises resources and Microsoft Azure components.<\/li>\n<\/ul>\n
Rethinking Microsoft Azure ExpressRoute business group subscriptions<\/h3>\n
When we first began our Microsoft Azure services, Microsoft Azure ExpressRoute connections were provisioned at the request of our business groups. We perform the ExpressRoute provisioning and assign IP address ranges to accompany the ExpressRoute circuit within the Azure subscription that belongs to the business group or partner organization. Although this approach has the benefit of leaving the ExpressRoute ownership in the hands of the business group that owns the subscription, it does have some management challenges:<\/p>\n
\n
It doesn\u2019t scale well. As more and more of our business teams began to embrace Microsoft Azure, it became quickly clear that a one-to-one relationship between Microsoft Azure ExpressRoute circuits and subscriptions wasn\u2019t going to scale.<\/li>\n
A business team might initially request an IP address range with a Microsoft Azure ExpressRoute circuit based on their current use of resources, and then require more at a later date. Reconfiguring the ExpressRoute circuit to add a larger IP range requires additional work to add or change hardware.<\/li>\n
It requires a central account in each subscription to manage the circuits. Multiple accounts are complex and time-consuming to manage.<\/li>\n
There\u2019s not a single point of reference for where our Microsoft Azure ExpressRoute circuits have been provisioned.<\/li>\n
Microsoft Azure ExpressRoute circuits must be provisioned individually. Provisioning tasks don\u2019t only include creating the ExpressRoute circuit\u2014it also includes allocating IP addresses, configuring routers, and other network-related tasks. Having to configure routers for every request quickly started to impact the amount of time required for creating connections.<\/li>\n<\/ul>\n
Using the provider subscription model<\/h2>\n
The provider subscription model is intended to simplify the provisioning and management of Microsoft Azure ExpressRoute circuits and virtual networks that can leverage them. In this model, we use a single shared provider subscription that owns every physical ExpressRoute circuit. These ExpressRoute circuits are then used by other consumer subscriptions that belong to our business groups by connecting virtual network to virtual network across subscriptions.<\/p>\n
The Microsoft Azure ExpressRoute hub and spoke shared circuit model.<\/figcaption><\/figure>\n
Implementing a hub shared provider subscription<\/h2>\n
To better manage the virtual networks for our Microsoft Azure ExpressRoute environment, we created shared provider subscriptions to specifically manage the ExpressRoute circuits. These provider subscriptions are owned by our central MDEE Infrastructure team. This team provides services out to the wider business units within MDEE and the larger corporate teams within all of Microsoft.<\/p>\n
The process for linking a Microsoft Azure ExpressRoute circuit to an external Microsoft Azure consumer subscription is a straightforward process and, after it\u2019s complete, our business groups can use the ExpressRoute connectivity in their own Azure environments. When we organize the ExpressRoute circuits this way, we can share connectivity between multiple business groups who may be using different subscriptions. This is beneficial when a single business group might not require the full amount of available bandwidth associated with an ExpressRoute circuit. It also adds greater flexibility to increase the number of IPs required later.<\/p>\n
In the graphic below, MSIT-CORP-ERPROVIDER-1 represents the Microsoft Azure subscription that is designated as the provider subscription for owning Microsoft Azure ExpressRoute circuits. Within the subscription are Azure resource groups, named for Azure regions, into which are placed the ExpressRoute circuits that are then used by the consumer subscriptions that belong to our business units.<\/p>\n
The MDEE implementation of the provider subscription model for Microsoft Azure ExpressRoute networks.<\/figcaption><\/figure>\n
The provider subscription has a simple setup. For each region where we\u2019re going to connect a Microsoft Azure ExpressRoute for connectivity, we create a resource group. This resource group owns the resources associated with that virtual network connectivity. We work with our central IP management team to allocate a large IP range for each circuit, from which smaller IP address ranges are assigned. As a particular region grows, we might need an additional circuit added. The table below provides an example of our configuration and naming conventions.<\/p>\n
Naming conventions for Microsoft Azure subscriptions, resource groups, and Microsoft Azure ExpressRoute circuits.<\/em><\/p>\n
As you can see from the table, we have Microsoft Azure ExpressRoute circuits configured across various parts of the United States, as well as Europe and Southeast Asia. We used a defined naming convention for subscriptions, resource groups, and ExpressRoute circuits\u2014this gives us a standardized environment and simplifies ExpressRoute maintenance and usage. All of these resources can be managed within the same centralized shared provider subscription. As you can see, some regions have more circuits configured for them than others, reflecting our ability to use this model to grow based on need. As might be expected, the WestUS2 region has the most circuits (five) because it\u2019s located near the main Microsoft campus in Redmond, Washington.<\/p>\n
Getting our internal customers connected<\/h2>\n
As part of pivoting our Microsoft Azure ExpressRoute services for our internal customers, we had to provide a way to request an ExpressRoute connection for their subscription. This was initially implemented by creating a website that could gather pertinent information from them to implement the request and then pass it on for use in the implementation process. The graphic below shows the new network form on our internal website that collects the following information about the request:<\/p>\n
\n
Request name:<\/strong> Plain text to identify the request, like \u201cNew Finance IT ExpressRoute Request.\u201d<\/li>\n
Microsoft Azure subscription:<\/strong> The Subscription ID (GUID) for the subscription to connect.<\/li>\n
Network type: <\/strong>Drop down to pick between our corporate network and our perimeter network.<\/li>\n
Region:<\/strong> Choose the region they need for the virtual network connection.<\/li>\n
Network size:<\/strong> \/26 or a \/27 virtual network. Customers that require something larger can contact us to request an exception.<\/li>\n
Service name:<\/strong> The service associated as defined within our Microsoft Azure service hierarchy.<\/li>\n
Associated users:<\/strong> We required at least two named contacts for notifications, follow-up, and to contact if there are any concerns regarding the configuration. Our security team will also use these contacts as required for detected compliance issues.<\/li>\n<\/ul>\n