{"id":828337,"date":"2022-03-22T10:00:00","date_gmt":"2022-03-22T17:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=828337"},"modified":"2022-08-17T09:46:04","modified_gmt":"2022-08-17T16:46:04","slug":"powering-the-next-generation-of-trustworthy-ai-in-a-confidential-cloud-using-nvidia-gpus","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/powering-the-next-generation-of-trustworthy-ai-in-a-confidential-cloud-using-nvidia-gpus\/","title":{"rendered":"Powering the next generation of trustworthy AI in a confidential cloud using NVIDIA GPUs"},"content":{"rendered":"\n<figure class=\"wp-block-image alignwide size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1400\" height=\"788\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_animation_final.gif\" alt=\"Animation showing the process of how encrypted data is transferred between the GPU drive and the GPU through a secure channel. The GPU driver on the host CPU and the SEC2 microcontroller on the NVIDIA A100 Tensor Core GPU work together to achieve end-to-end encryption of data transfers\" class=\"wp-image-829054\"\/><\/figure>\n\n\n\n<p>Cloud computing is powering a new age of data and AI by democratizing access to scalable compute, storage, and networking infrastructure and services. Thanks to the cloud, organizations can now collect data at an unprecedented scale and use it to train complex models and generate insights.&nbsp;&nbsp;<\/p>\n\n\n\n<p>While this increasing demand for data has unlocked new possibilities, it also raises concerns about privacy and security, especially in regulated industries such as government, finance, and healthcare. One area where data privacy is crucial is patient records, which are used to train models to aid clinicians in diagnosis. Another example is in banking, where models that evaluate borrower creditworthiness are built from increasingly rich datasets, such as bank statements, tax returns, and even social media profiles. This data contains very personal information, and to ensure that it&#8217;s kept private, governments and regulatory bodies are implementing strong privacy laws and regulations to govern the use and sharing of data for AI, such as the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/gdpr-info.eu\/\" target=\"_blank\" rel=\"noreferrer noopener\">General Data Protection Regulation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (GDPR) and the proposed <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/artificialintelligenceact.eu\/\" target=\"_blank\" rel=\"noreferrer noopener\">EU AI Act<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. You can learn more about some of the industries where it\u2019s imperative to protect sensitive data in this <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/aka.ms\/accgpu\" target=\"_blank\" rel=\"noreferrer noopener\">Microsoft Azure Blog post<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/p>\n\n\n\n<h2 id=\"commitment-to-a-confidential-cloud\">Commitment to a confidential cloud<\/h2>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/oblivious-multi-party-machine-learning-trusted-processors\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Oblivious Multi-Party Machine Learning on Trusted Processors\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Oblivious Multi-Party Machine Learning on Trusted Processors\">\n\t\t\t\tOblivious Multi-Party Machine Learning on Trusted Processors&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>Microsoft recognizes that trustworthy AI requires a trustworthy cloud\u2014one in which security, privacy, and transparency are built into its core. A key component of this vision is <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/confidentialcomputing.io\/faq\/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>confidential computing<\/em><span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u2014a set of hardware and software capabilities that give data owners technical and verifiable control over how their data is shared and used. Confidential computing relies on a new hardware abstraction called <em>trusted execution environments<\/em> (TEEs). In TEEs, data remains encrypted not just at rest or during transit, but also during use. TEEs also support remote attestation, which enables data owners to remotely verify the configuration of the hardware and firmware supporting a TEE and grant specific algorithms access to their data.&nbsp;&nbsp;<\/p>\n\n\n\n<p>At Microsoft, we are committed to providing a <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2021\/03\/Toward-Confidential-Cloud-Computing.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">confidential cloud<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, where confidential computing is the default for all cloud services. Today, Azure offers a rich confidential computing platform comprising different kinds of confidential computing hardware (Intel SGX, AMD SEV-SNP), core confidential computing services like <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/attestation\/overview\" target=\"_blank\" rel=\"noreferrer noopener\">Azure Attestation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/key-vault\/managed-hsm\/overview\" target=\"_blank\" rel=\"noreferrer noopener\">Azure Key Vault managed HSM,<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> and application-level services such as <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/relational-databases\/security\/encryption\/always-encrypted-database-engine?view=sql-server-ver15\" target=\"_blank\" rel=\"noreferrer noopener\">Azure SQL Always Encrypted<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/azure-confidential-ledger\/\" target=\"_blank\" rel=\"noreferrer noopener\">Azure confidential ledger<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/confidential-computing\/confidential-containers\" target=\"_blank\" rel=\"noreferrer noopener\">confidential containers on Azure<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. However, these offerings are limited to using CPUs. This poses a challenge for AI workloads, which rely heavily on AI accelerators like GPUs to provide the performance needed to process large amounts of data and train complex models.&nbsp;&nbsp;<\/p>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--left\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/graviton-trusted-execution-environments-on-gpus\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Graviton: Trusted Execution Environments on GPUs\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Graviton: Trusted Execution Environments on GPUs\">\n\t\t\t\tGraviton: Trusted Execution Environments on GPUs&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>The <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/theme\/confidential-computing\" target=\"_blank\" rel=\"noreferrer noopener\">Confidential Computing<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> group at Microsoft Research identified this problem and defined a vision for <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/confidential-ai\/\" target=\"_blank\" rel=\"noreferrer noopener\">confidential AI<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> powered by confidential GPUs, proposed in two papers, \u201c<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/oblivious-multi-party-machine-learning-trusted-processors\/\" target=\"_blank\" rel=\"noreferrer noopener\">Oblivious Multi-Party Machine Learning on Trusted Processors<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u201d and \u201c<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/graviton-trusted-execution-environments-on-gpus\/\" target=\"_blank\" rel=\"noreferrer noopener\">Graviton: Trusted Execution Environments on GPUs<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.\u201d In this post, we share this vision. We also take a deep dive into the NVIDIA GPU technology that&#8217;s helping us realize this vision, and we discuss the collaboration among NVIDIA, Microsoft Research, and Azure that enabled NVIDIA GPUs to become a part of the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/azure.microsoft.com\/en-us\/solutions\/confidential-compute\/\" target=\"_blank\" rel=\"noreferrer noopener\">Azure confidential computing<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> ecosystem.<\/p>\n\n\n\n\t<div class=\"border-bottom border-top border-gray-300 mt-5 mb-5 msr-promo text-center text-md-left alignwide\" data-bi-aN=\"promo\" data-bi-id=\"1045002\">\n\t\t\n\n\t\t<p class=\"msr-promo__label text-gray-800 text-center text-uppercase\">\n\t\t<span class=\"px-4 bg-white display-inline-block font-weight-semibold small\">Spotlight: Event<\/span>\n\t<\/p>\n\t\n\t<div class=\"row pt-3 pb-4 align-items-center\">\n\t\t\t\t\t\t<div class=\"msr-promo__media col-12 col-md-5\">\n\t\t\t\t<a class=\"bg-gray-300\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/cvpr-2024\/\" aria-label=\"Microsoft at CVPR 2024\" data-bi-cN=\"Microsoft at CVPR 2024\" target=\"_blank\">\n\t\t\t\t\t<img decoding=\"async\" class=\"w-100 display-block\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/05\/CVPR_WebBanner_1920x720.png\" alt=\"teal background triangular pattern\" \/>\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"msr-promo__content p-3 px-5 col-12 col-md\">\n\n\t\t\t\t\t\t\t\t\t<h2 class=\"h4\">Microsoft at CVPR 2024<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<p class=\"large\">Microsoft is a proud sponsor and active participant of\u00a0<a href=\"https:\/\/cvpr2023.thecvf.com\/Conferences\/2024\" target=\"_blank\" rel=\"noreferrer noopener\">CVPR 2024<\/a>, which focuses on advancements in computer vision and pattern recognition. <\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"wp-block-buttons justify-content-center justify-content-md-start\">\n\t\t\t\t\t<div class=\"wp-block-button\">\n\t\t\t\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/cvpr-2024\/\" class=\"btn btn-brand glyph-append glyph-append-chevron-right\" aria-label=\"Learn more\" data-bi-cN=\"Microsoft at CVPR 2024\" target=\"_blank\">\n\t\t\t\t\t\t\tLearn more\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div><!--\/.msr-promo__content-->\n\t<\/div><!--\/.msr-promo__inner-wrap-->\n\t<\/div><!--\/.msr-promo-->\n\t\n\n\n<h2 id=\"vision-for-confidential-gpus\">Vision for confidential GPUs<\/h2>\n\n\n\n<p>Today, CPUs from companies like Intel and AMD allow the creation of TEEs, which can isolate a process or an entire guest virtual machine (VM), effectively eliminating the host operating system and the hypervisor from the trust boundary. <strong>Our vision is to extend this trust boundary to GPUs<\/strong>, <strong>allowing code running in the CPU TEE to securely offload computation and data to GPUs<\/strong>.&nbsp;&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"775\" height=\"532\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_Vision-graphic.png\" alt=\"Diagram showing the trust boundary extended from the host trusted execution environment of the CPU to the trusted execution environment of the GPU through a secure channel.\" class=\"wp-image-828403\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_Vision-graphic.png 775w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_Vision-graphic-300x206.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_Vision-graphic-768x527.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_Vision-graphic-240x165.png 240w\" sizes=\"(max-width: 775px) 100vw, 775px\" \/><figcaption>Figure 1: Vision for confidential computing with NVIDIA GPUs.<\/figcaption><\/figure>\n\n\n\n<p>Unfortunately, extending the trust boundary is not straightforward. On the one hand, we must protect against a variety of attacks, such as man-in-the-middle attacks where the attacker can observe or tamper with traffic on the PCIe bus or on a <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.nvidia.com\/en-us\/design-visualization\/nvlink-bridges\/\" target=\"_blank\" rel=\"noreferrer noopener\">NVIDIA NVLink<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> connecting multiple GPUs, as well as impersonation attacks, where the host assigns an incorrectly configured GPU, a GPU running older versions or malicious firmware, or one without confidential computing support for the guest VM. At the same time, we must ensure that the Azure host operating system has enough control over the GPU to perform administrative tasks. Furthermore, the added protection must not introduce large performance overheads, increase thermal design power, or require significant changes to the GPU microarchitecture.&nbsp;&nbsp;<\/p>\n\n\n\n<p>Our research shows that this vision can be realized by extending the GPU with the following capabilities:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>A new mode where all sensitive state on the GPU, including GPU memory, is isolated from the host<\/li><li>A hardware root-of-trust on the GPU chip that can generate verifiable attestations capturing all security sensitive state of the GPU, including all firmware and microcode&nbsp;<\/li><li>Extensions to the GPU driver to verify GPU attestations, set up a secure communication channel with the GPU, and transparently encrypt all communications between the CPU and GPU&nbsp;<\/li><li>Hardware support to transparently encrypt all GPU-GPU communications over NVLink&nbsp;&nbsp;<\/li><li>Support in the guest operating system and hypervisor to securely attach GPUs to a CPU TEE, even if the contents of the CPU TEE are encrypted<\/li><\/ul>\n\n\n\n<h2 id=\"confidential-computing-with-nvidia-a100-tensor-core-gpus\">Confidential computing with NVIDIA A100 Tensor Core GPUs<\/h2>\n\n\n\n<p>NVIDIA and Azure have taken a significant step toward realizing this vision with a new feature called Ampere Protected Memory (APM) in the NVIDIA A100 Tensor Core GPUs. In this section, we describe how APM supports confidential computing within the A100 GPU to achieve end-to-end data confidentiality.&nbsp;&nbsp;<\/p>\n\n\n\n<p>APM introduces a new confidential mode of execution in the A100 GPU. When the GPU is initialized in this mode, the GPU designates a region in high-bandwidth memory (HBM) as protected and helps prevent leaks through memory-mapped I\/O (MMIO) access into this region from the host and peer GPUs. Only authenticated and encrypted traffic is permitted to and from the region.&nbsp;&nbsp;<\/p>\n\n\n\n<p>In confidential mode, the GPU can be paired with any external entity, such as a TEE on the host CPU. To enable this pairing, the GPU includes a hardware root-of-trust (HRoT). NVIDIA provisions the HRoT with a unique identity and a corresponding certificate created during manufacturing. The HRoT also implements authenticated and measured boot by measuring the firmware of the GPU as well as that of other microcontrollers on the GPU, including a security microcontroller called SEC2. SEC2, in turn, can generate attestation reports that include these measurements and that are signed by a fresh attestation key, which is endorsed by the unique device key. These reports can be used by any external entity to verify that the GPU is in confidential mode and running last known good firmware.&nbsp;&nbsp;<\/p>\n\n\n\n<p>When the NVIDIA GPU driver in the CPU TEE loads, it checks whether the GPU is in confidential mode. If so, the driver requests an attestation report and checks that the GPU is a genuine NVIDIA GPU running known good firmware. Once confirmed, the driver establishes a secure channel with the SEC2 microcontroller on the GPU using the Security Protocol and Data Model (SPDM)-backed Diffie-Hellman-based key exchange protocol to establish a fresh session key. When that exchange completes, both the GPU driver and SEC2 hold the same symmetric session key.\u202f&nbsp;<\/p>\n\n\n\n<p>The GPU driver uses the shared session key to encrypt all subsequent data transfers to and from the GPU. Because pages allocated to the CPU TEE are encrypted in memory and not readable by the GPU DMA engines, the GPU driver allocates pages outside the CPU TEE and writes encrypted data to those pages. On the GPU side, the SEC2 microcontroller is responsible for decrypting the encrypted data transferred from the CPU and copying it to the protected region. Once the data is in high bandwidth memory (HBM) in cleartext, the GPU kernels can freely use it for computation.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"856\" height=\"501\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_AmpereProtectedMemory-graphic.png\" alt=\"Diagram showing how the GPU driver on the host CPU and the SEC2 microcontroller on the NVIDIA Ampere GPU work together to achieve end-to-end encryption of data transfers.\" class=\"wp-image-828406\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_AmpereProtectedMemory-graphic.png 856w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_AmpereProtectedMemory-graphic-300x176.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_AmpereProtectedMemory-graphic-768x449.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_AmpereProtectedMemory-graphic-480x280.png 480w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/Confidential-GPUs_AmpereProtectedMemory-graphic-240x140.png 240w\" sizes=\"(max-width: 856px) 100vw, 856px\" \/><figcaption>Figure 2: The GPU driver on the host CPU and the SEC2 microcontroller on the NVIDIA A100 Tensor Core GPU work together to achieve end-to-end encryption of data transfers.<\/figcaption><\/figure>\n\n\n\n<h2 id=\"accelerating-innovation-with-confidential-ai\">Accelerating innovation with confidential AI<\/h2>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/aka.ms\/accgpu\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Azure confidential computing with NVIDIA GPUs for trustworthy AI\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Azure confidential computing with NVIDIA GPUs for trustworthy AI\">\n\t\t\t\tAzure confidential computing with NVIDIA GPUs for trustworthy AI&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>The implementation of APM is an important milestone toward achieving broader adoption of confidential AI in the cloud and beyond. APM is the foundational building block of <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/aka.ms\/accgpu\" target=\"_blank\" rel=\"noreferrer noopener\">Azure Confidential GPU VMs<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, now in <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/forms.office.com\/Pages\/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR37R7JFLKbBAml_g6YTMEqtUN0lDSFVRTVNJUFNRNlBaNk9RMVBZMjhOSS4u\" target=\"_blank\" rel=\"noreferrer noopener\">private preview<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. These VMs, designed in collaboration with NVIDIA, Azure, and Microsoft Research, feature up to four A100 GPUs with 80 GB of HBM and APM technology and enable users to host AI workloads on Azure with a new level of security.&nbsp;&nbsp;<\/p>\n\n\n\n<p>But this is just the beginning. We look forward to taking our collaboration with NVIDIA to the next level with NVIDIA\u2019s Hopper architecture, which will enable customers to protect both the confidentiality and integrity of data and AI models in use. We believe that confidential GPUs can enable a confidential AI platform where multiple organizations can collaborate to train and deploy AI models by pooling together sensitive datasets while remaining in full control of their data and models. Such a platform can unlock the value of large amounts of data while preserving data privacy, giving organizations the opportunity to drive innovation.&nbsp;&nbsp;<\/p>\n\n\n\n<p>A real-world example involves <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.bosch.com\/research\/\" target=\"_blank\" rel=\"noreferrer noopener\">Bosch Research<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, the research and advanced engineering division of <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.bosch.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Bosch<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, which is developing an AI pipeline to train models for autonomous driving. Much of the data it uses includes personal identifiable information (PII), such as license plate numbers and people\u2019s faces. At the same time, it must comply with GDPR, which requires a legal basis for processing PII, namely, consent from data subjects or legitimate interest. The former is challenging because it is practically impossible to get consent from pedestrians and drivers recorded by test cars. Relying on legitimate interest is challenging too because, among other things, it requires showing that there is a no less privacy-intrusive way of achieving the same result. This is where confidential AI shines: Using confidential computing can help reduce risks for data subjects and data controllers by limiting exposure of data (for example, to specific algorithms), while enabling organizations to train more accurate models.&nbsp;&nbsp;&nbsp;<\/p>\n\n\n\n<p>At Microsoft Research, we are committed to working with the confidential computing ecosystem, including collaborators like NVIDIA and Bosch Research, to further strengthen security, enable seamless training and deployment of confidential AI models, and help power the next generation of technology.<\/p>\n\n\n\n<h3 id=\"about-confidential-computing-at-microsoft-research\">About confidential computing at Microsoft Research&nbsp;&nbsp;<\/h3>\n\n\n\n<p>The <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/theme\/confidential-computing\/\">Confidential Computing team<\/a> at Microsoft Research Cambridge conducts pioneering research in system design that aims to guarantee strong security and privacy properties to cloud users. We tackle problems around secure hardware design, cryptographic and security protocols, side channel resilience, and memory safety. We are also interested in new technologies and applications that security and privacy can uncover, such as blockchains and multiparty machine learning. Please visit our <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/theme\/confidential-computing\/opportunities\/\" target=\"_blank\" rel=\"noreferrer noopener\">careers page<\/a> to learn about opportunities for both researchers and engineers. We\u2019re hiring.<\/p>\n\n\n\n<h3 id=\"related-gtc-conference-sessions\">Related GTC Conference sessions<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.nvidia.com\/gtc\/session-catalog\/?tab.scheduledorondemand=1583520458947001NJiE&search=azure%20confidential#\/session\/1642801739576001k8ly\" target=\"_blank\" rel=\"noreferrer noopener\">Unlocking New Possibilities for Privacy-Preserving Data Analytics with Azure Confidential Computing&nbsp;<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/li><\/ul>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.nvidia.com\/gtc\/session-catalog\/?tab.scheduledorondemand=1583520458947001NJiE&search=azure%20confidential#\/session\/1642804134716001xaA0\" target=\"_blank\" rel=\"noreferrer noopener\">Advancing AI Trustworthiness with Azure Confidential GPUs<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Cloud computing is powering a new age of data and AI by democratizing access to scalable compute, storage, and networking infrastructure and services. Thanks to the cloud, organizations can now collect data at an unprecedented scale and use it to train complex models and generate insights.&nbsp;&nbsp; While this increasing demand for data has unlocked new [&hellip;]<\/p>\n","protected":false},"author":40306,"featured_media":828352,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[1],"tags":[],"research-area":[13558],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[243984],"msr-impact-theme":[261676],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-828337","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-security-privacy-cryptography","msr-locale-en_us","msr-post-option-blog-homepage-featured"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":["Trust"],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[559983],"related-projects":[648207],"related-events":[],"related-researchers":[{"type":"user_nicename","value":"Kapil Vaswani","user_id":32487,"display_name":"Kapil Vaswani","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/kapilv\/\" aria-label=\"Visit the profile page for Kapil Vaswani\">Kapil Vaswani<\/a>","is_active":false,"last_first":"Vaswani, Kapil","people_section":0,"alias":"kapilv"},{"type":"user_nicename","value":"Antoine Delignat-Lavaud","user_id":31056,"display_name":"Antoine Delignat-Lavaud","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/antdl\/\" aria-label=\"Visit the profile page for Antoine Delignat-Lavaud\">Antoine Delignat-Lavaud<\/a>","is_active":false,"last_first":"Delignat-Lavaud, Antoine","people_section":0,"alias":"antdl"},{"type":"user_nicename","value":"Stavros Volos","user_id":35437,"display_name":"Stavros Volos","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/svolos\/\" aria-label=\"Visit the profile page for Stavros Volos\">Stavros Volos<\/a>","is_active":false,"last_first":"Volos, Stavros","people_section":0,"alias":"svolos"},{"type":"user_nicename","value":"C\u00e9dric Fournet","user_id":31819,"display_name":"C\u00e9dric Fournet","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/fournet\/\" aria-label=\"Visit the profile page for C\u00e9dric Fournet\">C\u00e9dric Fournet<\/a>","is_active":false,"last_first":"Fournet, C\u00e9dric","people_section":0,"alias":"fournet"},{"type":"user_nicename","value":"Manuel Costa","user_id":32794,"display_name":"Manuel Costa","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/manuelc\/\" aria-label=\"Visit the profile page for Manuel Costa\">Manuel Costa<\/a>","is_active":false,"last_first":"Costa, Manuel","people_section":0,"alias":"manuelc"}],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-960x540.jpg\" class=\"img-object-cover\" alt=\"Graphic showing the process of how encrypted data is transferred between the GPU drive and the GPU through a secure channel. The GPU driver on the host CPU and the SEC2 microcontroller on the NVIDIA A100 Tensor Core GPU work together to achieve end-to-end encryption of data transfers\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-1536x865.jpg 1536w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-2048x1153.jpg 2048w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/03\/1400x788_Confidential_GPU_still-1920x1080.jpg 1920w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","byline":"","formattedDate":"March 22, 2022","formattedExcerpt":"Cloud computing is powering a new age of data and AI by democratizing access to scalable compute, storage, and networking infrastructure and services. Thanks to the cloud, organizations can now collect data at an unprecedented scale and use it to train complex models and generate&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/828337"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/40306"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=828337"}],"version-history":[{"count":26,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/828337\/revisions"}],"predecessor-version":[{"id":870606,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/828337\/revisions\/870606"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/828352"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=828337"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=828337"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=828337"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=828337"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=828337"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=828337"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=828337"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=828337"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=828337"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=828337"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=828337"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}