This story reflects updated guidance from Microsoft Digital—it was first published in July 2024. 

Providing exemplary support is critical to how we empower our customers to achieve more with Microsoft technologies and services.

We in Microsoft Digital, the company’s IT organization, recently migrated our global Microsoft customer service network to Microsoft Azure, creating a cloud network-based solution to connect our customers to the support services they need at Microsoft. With the new solution, our customers and customer service team members are connected faster, more reliably, and with improved network performance while maintaining secure and compliant connections.

Building a better customer support network

Our Support Experience Group (SxG) within our Cloud + AI division is driving transformation for Microsoft Support solutions, building on Microsoft solutions and infusing cutting-edge innovation to improve customer and agent experiences across all our businesses. Our SxG team provides platforms and services to almost 80,000 Microsoft support advocates, including technical teams and customer support advocates from our network of global contact centers.

Our customer support advocates and partners are integral to maintaining high-quality customer service and support for Microsoft products and solutions. Microsoft customer services handle almost 200,000 support calls daily in 37 different languages worldwide. It’s a diverse and fast-paced environment where connecting support staff to the customer and the Microsoft services they support can be complex.

Our previous global network backbone served us for years through the deployment of key regional central hub sites. Hub sites were connected by physical point-to-point Multiprotocol Label Switching (MPLS) circuits deployed strategically to various sites globally. The MPLS network design is complex, costly, and inflexible.

By redesigning our network with Microsoft Azure Cloud Network solutions at the center, we’re addressing several challenges associated with traditional MPLS networks, such as:

• Cost and complexity: MPLS networks are often expensive and complex to deploy. 
• Inflexibility: MPLS is designed for stable, point-to-point connections and can be too rigid for the dynamic and distributed nature of modern cloud computing. It struggles to efficiently handle the traffic patterns created by enterprises running workloads across multiple clouds.
• Deployment speed: Setting up or modifying MPLS connections can take weeks or even months, which is not conducive to the agility required by businesses today. Cloud networks can be deployed and scaled much more rapidly.
• Security and encryption: Traditional MPLS doesn’t offer encryption, which is increasingly important as operations move toward the cloud. A cloud network can provide consistent protection regardless of how users connect.

At the core of our transformation is a newly designed global, cloud-based network built on Azure Virtual WAN services called the SxG Cloud Network, built specifically for Microsoft customer services. The SxG Cloud Network directly connects advocates at Microsoft contact centers, remote advocates and internal support teams to the required services.

The SxG Cloud Network provides a highly reliable and high-performing network path into Azure, where support team members can access the tools and environments required to support our customers fully. Within the network, our customer service teams are connected to Azure Virtual Desktops that supply the tools and connectivity they need for troubleshooting, enabling them to connect with Microsoft customers worldwide through virtual private network (VPN) and Azure Virtual Network (VNet) peering.

The SxG Cloud Network resides on the Microsoft Azure tenant and consists of several virtual WAN hubs in key Azure regions across the globe. These hubs use Microsoft Azure Firewall to secure traffic flows within the cloud network using URL filtering, TLS inspection, and intrusion detection and prevention.

The Azure-based hubs provide a single access point that simplifies connectivity and creates a unified and consistent environment for all support advocates. We provide several connectivity methods for our Microsoft customer support advocates irrespective of location, including:

• Point-to-site (P2S) VPN: This provides connectivity for the remote user working from home.
• Site-to-site (S2S) VPN: We use S2S VPN to connect Microsoft contact centers using an S2S encrypted tunnel between the partner VPN concentrator and the SxG Cloud Network gateway.
• VNet peering: We also support peering between a partner Azure tenant and the SxG Cloud Network Azure tenant. VNets on both tenants are directly peered and secured by Azure Firewall.

Point-to-site VPN

Remote Microsoft customer support advocates use Azure P2S VPN to connect directly to Microsoft services in Azure. We maintain several VPN hubs across global Azure regions to ensure that advocates experience the most direct network path to Azure. We use Azure networking components within Azure to connect to the required internal Azure resources.

To ensure that only necessary traffic goes through the VPN, VPN profiles are configured with split-tunnel routing that routes Microsoft specific traffic to Azure and the rest to the partner network or the public internet. This ensures that users can access local websites in the correct locale and languages they need, while also enabling low-latency access to the Microsoft corporate edge network.

The Azure VPN client facilitates connectivity between the local device and the Azure Virtual WAN gateway hosted in the SxG network. We use a single VPN profile configured with split tunneling for all VPN users. This is made possible by a key feature of Azure Virtual WAN that automatically connects P2S users directly to the closest region. Authentication is required to access the VPN and users authenticate using their Microsoft credentials through Entra ID and multi-factor authentication.

Site-to-site VPN

S2S VPN connections provide a secure encrypted VPN connection over the public internet to connect our contact centers to Microsoft customer support services in Azure. The contact center partner manages their network and the configuration of the device on their network, which establishes a VPN tunnel to the Azure Virtual WAN gateway hosted in the SxG Cloud Network.

VNet peering

When partners already have an Azure presence, Microsoft can connect the partner Azure network to the virtual WAN using Azure VNet peering. Traffic between the peered VNets doesn’t leave the global Azure backbone network. We use SxG VNet peering to connect VNets in the Microsoft tenant with VNets in the partner’s Azure tenant. VNet peering establishes a high-performance, trusted connection using Azure Firewall in the SxG Cloud Network to provide flow control and traffic protection.

SxG Cloud Network infrastructure

Managing connectivity for voice services

Our advocates often support our customers with voice calls, and supporting an effective and efficient voice service is integral to the SxG Cloud Network.

We use Azure ExpressRoute connections to create a direct private network path from all our Azure Virtual WAN gateways to our voice services platform environment using an MPLS backbone. These global connections to our voice services hosted in Azure enable advocates connected to the SxG Cloud Network via P2S, S2S, or VNet peering to use our voice services. The Interhub feature in Azure Virtual WAN also provides seamless connectivity between hubs, ensuring that user network traffic takes the best path with minimal latency while traversing the Microsoft backbone network.

Microsoft customer service advocates voice services are now migrated to Azure Communication Services, which is connected to the SxG Cloud Network with ExpressRoute and keeps traffic on the reliable Azure backbone network.

The SxG Cloud Network has modernized how we connect to voice and data services hosted in Azure and can provide advocates access without needing to deploy physical circuits to contact center locations, saving time and money. It also creates a unified network environment, simplifying access points and functionality for our advocates.

With the flexibility and scalability of the SxG Cloud Network, we can manage our bandwidth needs better and have fewer physical circuits that are oversized for the traffic volume. This alone is reducing network costs by more than 60% in specific cases. While exact figures for cost savings and performance improvements can vary depending on the specific circumstances of a deployment, businesses often report significant reductions in total cost of ownership (TCO) and enhancements in network performance when migrating from MPLS to Azure cloud-based solutions.

Looking forward

As we look to the immediate future of the SxG Cloud Network, we’re excited about increasing Azure Communication Services traffic on our network for voice support, further unifying our services and leading to more significant cost savings and efficiency. We’ll continue searching for ways to improve the SxG Cloud Network, including moving the network edge closer to our users with new global virtual WAN hubs. This helps us deliver more effective and easy-to-use support services for Microsoft customers and the advocates who support them.

We’re benefiting from the SxG Cloud Network in several areas, including:

• Experience enhanced support: Connect faster and more reliably to support services thanks to our migration to the Azure-based SxG Cloud Network, ensuring high-quality assistance whenever Microsoft customers need it.
• Global reach, local service: The SxG Cloud Network spans countries and languages, providing a seamless support experience through a diverse team of professionals ready to assist customers.
• Secure and simplified connectivity: Azure Virtual WAN offers various connection options, including VPN and VNet, to ensure a secure, direct connection to support resources.
• Future-ready voice services: Azure Communication Services is creating a more integrated and cost-effective voice support system, enhancing the support experience while maintaining the highest network reliability standards.

Create a P2SUser VPN connection using Azure Virtual WAN at your company.

• Discover how we’re moving Microsoft’s global network to the cloud with Microsoft Azure.
• Unpack transforming Microsoft’s enterprise network with next-generation connectivity.
• Explore our Total Cost of Ownership (TCO) calculator.

• Want more information? Email us and include a link to this story and we’ll get back to you.

The post Running our customer service and support contact centers on Microsoft Azure appeared first on Inside Track Blog.

We’re building a more agile, resilient, and stable virtual wide-area network (VWAN) to create a better experience for our employees to connect and collaborate globally. By implementing a continuous integration/continuous deployment (CI/CD) approach to building our VWAN-based network infrastructure, we can automate the deployment and configuration processes to ensure rapid and reliable delivery of network changes. Here’s how we’re making that happen internally at Microsoft.

Infrastructure as code (IaC)

Infrastructure as code (IaC) is the fundamental principle underlying our entire VWAN infrastructure. Using IaC, we can develop and implement a descriptive model that defines and deploys VWAN components and determines how the components work together. IaC allows us to create and manage a massive network infrastructure with reusable, flexible, and rapid code deployments.

We created deployment templates and resource modules using the Bicep language in our implementation. These templates and modules describe the desired state of our VWAN infrastructure in a declarative manner. Bicep is a domain-specific language (DSL) that uses declarative syntax to deploy Microsoft Azure resources.

We maintain a primary Bicep template that calls separate modules—also maintained in Bicep templates—to create the desired resources for the deployment in alignment with Microsoft best practices. We use this modular approach to apply different deployment patterns to accommodate changes or new requirements.

With IaC, changes and redeployments are as quick as modifying templates and calling the associated modules. Additionally, parameters for each unique deployment are maintained in separate files from the templates so that different iterations of the same deployment pattern can be deployed without changing the source Bicep code.

Version control

We use Microsoft Azure DevOps, a source control system using Git, to track and manage our IaC templates, modules, and associated parameter files. With Azure DevOps, we can maintain a history of changes, collaborate within teams, and easily roll back to previous versions if necessary.

We’re also using pull requests to help track change ownership. Azure DevOps tracks changes and associates them with the engineer who made the change. Azure DevOps is a considerable help with several other version control tasks, such as requiring peer reviews and approvals before code is committed to the main branch. Our code artifacts are published to (and consumed from) a Microsoft Azure Container Registry that allows role-based access control of modules. This enables version control throughout the module lifecycle, and it’s easy to share Azure Container Registry artifacts across multiple teams for collaboration.

Automated testing

Responsible deployment is essential with IaC when deploying a set of templates could radically alter critical network infrastructure. We’ve implemented safeguards and tests to validate the correctness and functionality of our code before deployment. These tests include executing the Bicep linter as part of the Azure DevOps deployment pipeline to ensure that all Bicep best practices are being followed and to find potential issues that could cause a deployment to fail.

We’re also running a test deployment to preview the proposed resource changes before the final deployment. As the process matures, we plan to integrate more testing, including network connectivity tests, security checks, performance benchmarks, and enterprise IP address management (IPAM) integration.

Configuration management

Azure DevOps and Bicep allow us to automate the configuration and provisioning of network objects and services within our VWAN infrastructure. These tools make it easy to define and enforce desired configurations and deployment patterns to ensure consistency across different network environments. Using separate parameter files, we can rapidly deploy new environments in minutes rather than hours without changing the deployment templates or signing in to the Microsoft Azure Portal.

Continuous deployment

The continuous integration (CI) pipeline automates the deployment process for our VWAN infrastructure when the infrastructure code passes all validation and tests. The CI pipeline triggers the deployment process automatically, which might involve deploying virtual machines, building and configuring cloud network objects, setting up VPN connections, or establishing network policies.

Monitoring and observability

We’ve implemented robust monitoring and observability practices for how we deploy and manage our VWAN deployment. Monitoring and observability are helping us to ensure that our CI builds are successful, detect issues promptly, and maintain the health of our development process. Here’s how we’re building monitoring and observability in our Azure DevOps CI pipeline:

• We’re creating built-in dashboards and reports that visualize pipeline status and metrics such as build success rates, durations, and failure details.
• We’re generating and storing logs and artifacts during builds.
• We’ve enabled real-time notifications to help us monitor build status for failures and critical events.
• We’re building-in pipeline monitoring review processes to identify areas for improvement including optimizing build times, reducing failures, and enhancing the stability of our pipeline.

We’re continuing to iterate and optimize our monitoring practices. We’ve created a feedback loop to review the results of our monitoring. This feedback provides the information we need to adjust build scripts, optimize dependencies, automate certain tasks, and further enhance our pipeline.

By implementing comprehensive monitoring and observability practices in our Azure DevOps CI pipeline, we can maintain a healthy development process, catch issues early, and continuously improve the quality of our code and builds.

Rollback and rollforward

We’ve built the ability to rollback or rollforward changes in case of any issues or unexpected outcomes. This is achieved through infrastructure snapshots, version-controlled configuration files, or using features provided by our IaC tool.

Improving through iteration

We’re continuously improving our VWAN infrastructure using information from monitoring data and user experience feedback. We’re also continually assessing new requirements, newly added Azure features, and operational insights. We iterate on our infrastructure code and configuration to enhance security, performance, and reliability.

By following these steps and using CI/CD practices, we can build, test, and deploy our VWAN network infrastructure in a controlled and automated manner, creating a better employee experience by ensuring faster delivery, increased stability, and more effortless scalability.

Here are some tips on how you can start tackling some of the same challenges at your company:

• You can use Infrastructure as code (IaC) to create and manage a massive network infrastructure with reusable, flexible, and rapid code deployments.
• Using IaC, you can make changes and redeployments quickly by modifying templates and calling the associated modules.
• Don’t overlook version control. Tracking and managing IaC templates, modules, and associated parameter files is essential.
• Perform automated testing. It’s necessary to validate the correctness and functionality of the code before deployment.
• Use configuration management tools to simplify defining and enforcing desired configurations and deployment patterns. This ensures consistency across different network environments.
• Implement continuous deployment to automate the deployment process for network infrastructure after the code passes all validation and tests.
• Use monitoring and observability best practices to help identify issues, track performance, troubleshoot problems, and ensure the health and availability of the network infrastructure.
• Building rollback and roll-forward capabilities enables you to quickly respond to issues or unexpected outcomes.

Try using a Bicep template to manage your Microsoft Azure resources.

• Discover boosting our connectivity with our own next-generation optical network.
• Unpack running on VPN: How Microsoft is keeping its remote workforce connected.
• Explore implementing Microsoft Azure cost optimization internally at Microsoft.
• Check out building a secure and efficient self-service application using Azure ACI, Azure Compute Gallery, and the Microsoft Azure SDK.

Please share your feedback with us—take our survey and let us know what kind of content is most useful to you.

The post How we’re deploying our VWAN infrastructure using infrastructure as code and CI/CD appeared first on Inside Track Blog.

We’ve recently helped SAP, one of our key business partners, replace an outdated VPN connectivity solution with a highly secure, cost-efficient connection into Azure for integration with ADP payroll using Azure VWAN and Azure VPN.

As cloud networking has matured, so have the requirements for security and traffic control. SAP’s pre-existing solution for connecting their VPN clients lacked support for critical IKEv2 security and traffic control mechanisms such as AES256 and SHA256 encryption and border gateway protocol (BGP). To support these new requirements, the existing hardware used in SAP’s VPN solution needed to be replaced to be compliant with IKEv2 standards.

Our cloud networking engineers at Microsoft Digital (MSD), the company’s IT organization, proposed a different solution: a cloud-first VPN solution using Azure VWAN and Azure VPN.

The cloud-first solution uses Azure VPN policies and tunneling to bypass the requirement for routing hardware in SAP’s environment. It also provides full support for IKEv2 and removes dependency on outdated VPN hardware and controls.

This is the first Azure-based solution using IKEv2 policy-based VPN connectivity for SAP and ADP support. This cutting-edge solution introduces a highly secure and cost-efficient way for business partners to connect with Azure while ensuring strict adherence to regulatory compliance.

The introduction of IKEv2 VPN connectivity allows partners’ systems to securely communicate with Microsoft’s SAP environment in Azure, fostering a seamless integration of services. It sets the stage for unprecedented connectivity, creating a robust ecosystem for business partners to collaborate and exchange data securely.

Azure VWAN provides native IKEv2 VPN connectivity support. By taking advantage of Azure’s powerful networking capabilities, the solution is instantly scalable and highly available. The Azure-native approach streamlines the entire connectivity process by simplifying set up, management, and monitoring.

With IKEv2 VPN and Azure VWAN, all communications between partners’ or providers’ systems and Microsoft’s SAP environment are encrypted and authenticated, safeguarding sensitive information from unauthorized access. As a result, we can provide our partners with peace of mind and uphold the highest standards of data protection.

By eliminating the need for complex hardware and streamlining the set-up process, we enable our internal businesses and partners to achieve significant cost savings. This cost-effectiveness empowers them to invest resources strategically and fuel their growth.

Regulatory compliance is non-negotiable in today’s business landscape. Our VPN connectivity and native network solutions are designed with strict adherence to regulatory requirements in mind. By meeting industry standards and regulatory mandates, we ensure that our business partners can confidently operate within the bounds of compliance, mitigating potential risks and challenges.

The first IKEv2 policy-based VPN connectivity solution for SAP and ADP support through Azure native network solutions represents an entirely new way to approach SAP and ADP connectivity for Microsoft partners. With improved security, better cost efficiency, and support for regulatory compliance adherence, we’re reshaping the standards for business partner connectivity in the cloud. We’re continuing to innovate and explore new approaches to secure and efficient VPN connectivity to support SAP systems.

To navigate the integration of your network with Microsoft Azure networking solutions, consider the following:

• Explore Azure VWAN and VPN. Evaluate your network for potential improvements by considering Azure Virtual WAN and VPN for enhanced security and efficiency.
• Assess security and compliance. Review your network’s security protocols and compliance standards against Azure’s IKEv2 encryption and regulatory adherence.
• Initiate a pilot project. Test the impact of Azure VWAN and VPN by launching a small-scale pilot within your network, focusing on performance and cost benefits.

Here’s how you can create a peer-to-site VPN connection using Azure Virtual WAN.

• Explore moving Microsoft’s global network to the cloud with Azure.
• Read our ongoing series on moving our network to the cloud.

Want more information? Email us and include a link to this story and we’ll get back to you.

Please share your feedback with us—take our survey and let us know what kind of content is most useful to you.

The post Revolutionizing SAP and ADP connectivity with Microsoft Azure VWAN and VPN appeared first on Inside Track Blog.