{"id":702310,"date":"2020-11-09T12:12:21","date_gmt":"2020-11-09T20:12:21","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=702310"},"modified":"2023-03-06T09:14:21","modified_gmt":"2023-03-06T17:14:21","slug":"research-collection-re-inventing-storage-for-the-cloud-era","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/research-collection-re-inventing-storage-for-the-cloud-era\/","title":{"rendered":"Research Collection \u2013 Re-Inventing Storage for the Cloud Era"},"content":{"rendered":"<aside id=accordion-66813573-f461-4e2f-b17b-52382d231a08 class=\"msr-table-of-contents-block accordion mb-5 pb-0\" data-bi-aN=\"table-of-contents\">\n\t<button class=\"btn btn-collapse bg-gray-100 mb-0 display-flex justify-content-between\" type=\"button\" data-mount=\"collapse\" data-target=\"#accordion-collapse-66813573-f461-4e2f-b17b-52382d231a08\" aria-expanded=\"true\" aria-controls=\"accordion-collapse-66813573-f461-4e2f-b17b-52382d231a08\">\n\t\t<span class=\"msr-table-of-contents-block__label subtitle\">In this article<\/span>\n\t\t<span class=\"msr-table-of-contents-block__current mr-4 text-gray-600 font-weight-normal\" aria-hidden=\"true\"><\/span>\n\t<\/button>\n\t<div id=\"accordion-collapse-66813573-f461-4e2f-b17b-52382d231a08\" class=\"msr-table-of-contents-block__collapse-wrapper collapse show\" data-parent=\"#accordion-66813573-f461-4e2f-b17b-52382d231a08\">\n\t\t<div class=\"accordion-body bg-gray-100 border-top pt-4\">\n\t\t\t<ol class=\"msr-table-of-contents-block__list\">\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#re-imagining-storage\" class=\"msr-table-of-contents-block__list-item-link\">Re-Imagining Storage<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#optics-for-the-cloud\" class=\"msr-table-of-contents-block__list-item-link\">Optics for the Cloud<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#holographic-storage\" class=\"msr-table-of-contents-block__list-item-link\">Holographic Storage<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#project-silica\" class=\"msr-table-of-contents-block__list-item-link\">Project Silica<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#dna-storage\" class=\"msr-table-of-contents-block__list-item-link\">DNA Storage<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<\/ul>\n\t\t<\/div>\n\t<\/div>\n\t<span class=\"msr-table-of-contents-block__progress-bar\"><\/span>\n<\/aside>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p><em>\u201cOne of the challenges for us as a company, and us as an industry, is that many of the technologies we rely on are beginning to get to the point where either they are at the end, or they\u2019re starting to get to the point where you can see the end. Moore\u2019s Law is a well-publicized one and we hit it some time ago. And that\u2019s a great opportunity, because whenever you get that rollover, you get an opportunity to be able do things differently, to have new ways of doing things.\u201d<\/em><\/p>\n<cite>\u2013 Ant Rowstron, Distinguished Engineer and Deputy Lab Director, Microsoft Research Cambridge<\/cite><\/blockquote>\n\n\n\n<p>It is projected that around 125 <em>zettabytes<\/em> of data will be generated annually by 2024. Storing this data efficiently and cost-effectively will be a huge challenge. Growth in storage capabilities using SSDs, HDDs or magnetic tape has not kept up with the exponential growth in compute capacity, the surge in data being generated, or the novel economics and storage needs of cloud services.<\/p>\n\n\n\n<p>Future demands from intelligent edge and Internet of Things (IoT) deployments, streaming audio, video, virtual and mixed reality, \u201cdigital twins\u201d and use cases we haven\u2019t yet predicted will generate lots of bits \u2013 but where will we keep them?<\/p>\n\n\n\n<p>This requires more than incremental improvements \u2013 it demands disruptive innovation. For many years, Microsoft researchers and their collaborators have been exploring ways to make existing storage approaches more efficient and cost-effective, while also forging entirely new paths \u2013 including storing data in media such as glass, holograms and even DNA.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"582\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Cloud-Storage-landscape-1024x582.png\" alt=\"Diagram showing the cloud storage landscape\" class=\"wp-image-689577\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Cloud-Storage-landscape-1024x582.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Cloud-Storage-landscape-300x171.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Cloud-Storage-landscape-768x437.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Cloud-Storage-landscape.png 1437w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Research efforts in storage cover the spectrum from \u201chot\u201d data, which is read and written frequently and requires very fast performance, to \u201ccool\u201d data, which is generally archival in nature and is retrieved infrequently (see diagram).<\/figcaption><\/figure>\n\n\n\n<p>In this clip from the Microsoft Research Podcast, Ant Rowstron discusses how various storage media are reaching their useful limits in the datacenter. <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/optics-for-the-cloud-storage-in-the-zettabyte-era-with-dr-ant-rowstron-and-mark-russinovich\/\">Listen to the full episode<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/storage_ant_excerpt.mp3\"><\/audio><\/figure>\n\n\n\n<p class=\"has-small-font-size\"><em>If you think of most of the technologies we use to store data today \u2013 things like flash, things like hard disk drives, things like tape \u2013 it\u2019s true to say that they were all designed before the cloud existed. And in fact, they were all designed to work in multiple scenarios. You know, the flash chips have to work in your phone as well as in the datacenters. Or the hard disk drives have to work with desktop machines as well as in the datacenters. So, there\u2019s compromises that have been made. And, in fact, many of those technologies, if you think of a sort of S-curve where you got time, and you\u2019ve got the metric of interest, you get a new technology comes along like a hard disk drive, it starts off very slowly and then hits its sort of main speed and it\u2019s sort of going nicely. It\u2019s, you know, doubling every two years or three years if you\u2019re a hard disk drive. And then you get to the point where it actually stops just, basically, exploiting the property they were exploiting, and it rolls over and becomes flat. And I think one of the challenges for us as a company, and us as an industry, is that many of the technologies we rely on are beginning to get to the point where either they are at the end, or they\u2019re starting to get to the point where you can see the end. And of course, Moore\u2019s Law is a well-publicized one and we hit it some time ago. And that\u2019s a great opportunity, because whenever you get that rollover, you get an opportunity to be able do things differently, to have new ways of doing things.<\/em><\/p>\n\n\n\n<h3 id=\"re-imagining-storage\">Re-Imagining Storage<\/h3>\n\n\n\n<p>Researchers have taken a holistic approach to making storage more efficient and cost-effective, using the emergence of the cloud as an opportunity to completely re-think storage in an end-to-end fashion. They are co-designing new approaches across layers that are traditionally thought of as independent \u2013 blurring the lines between storage, memory and network. At the same time, they\u2019re re-thinking storage from the media up \u2013 including storing data in media such as glass, holograms and even DNA.<\/p>\n\n\n\n<p>This work extends back more than two decades: in 1999, Microsoft researchers began work on <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/farsite-federated-available-and-reliable-storage-for-an-incompletely-trusted-environment\/\">Farsite<\/a>, a secure and scalable file system that logically functions as a centralized file server, but is physically distributed among a set of untrusted computers. This approach would utilize the unused storage and network resources of desktop computers to provide a service that is reliable, available and secure despite running on machines that are unreliable, often unavailable and of limited security. In 2007, researchers published a <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/the-farsite-project-a-retrospective\/\">paper<\/a> that explored the conditions under which such a system could be scalable, the software engineering environment used to build the system, and the lessons learned in its development.<\/p>\n\n\n\n<p>At Microsoft Research Cambridge, researchers began exploring optimizing enterprise storage using off-the-shelf hardware in the late 2000s. They explored off-loading data from overloaded volumes to virtual stores <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/write-off-loading-practical-power-management-for-enterprise-storage-2\/\">to reduce power consumption<\/a> and <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/everest-scaling-down-peak-loads-through-io-off-loading\/\">to better accommodate peak I\/O request rates<\/a>. This was an early example of storage virtualization and software-defined storage, ideas that are now widely used in the cloud. As solid-state drives (SSDs) became more commonplace in PCs, researchers&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/migrating-enterprise-storage-to-ssds-analysis-of-tradeoffs\/\">considered<\/a>&nbsp;their application in the datacenter \u2013 and concluded that they were not yet cost-effective for most workloads at current prices. In 2012, they&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/whole-system-persistence-with-non-volatile-memories\/\">analyzed<\/a>&nbsp;potential applications of non-volatile main memory (NVRAM) and proposed whole-system persistence (WSP), an approach to database and key-value store recovery in which memory rather than disk is used to recover an application\u2019s state when it fails \u2013 blurring the lines between memory and storage.<\/p>\n\n\n\n<p>In 2011, researchers established the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/software-defined-storage-architectures\/\">Software-Defined Storage Architectures<\/a> project, which brought the idea of separating control flow from data from networking to storage, to provide predictable performance and reduced cost. <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/ioflow-a-software-defined-storage-architecture\/\">IOFlow<\/a> is a software-defined storage architecture that uses a logically centralized control plane to enable end-to-end policies and QoS guarantees, which required rethinking across the data center storage and network stack. This principle was extended to other cloud resources to create a <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/end-to-end-performance-isolation-through-virtual-datacenters\/\">virtual data center<\/a> per tenant. In this <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/treating-storage-stack-like-network\/\">2017 article<\/a> the researchers describe the advantages of intentionally blurring the lines between virtual storage and virtual networking.<\/p>\n\n\n\n<p>Established in 2012, the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/farm\/\">FaRM<\/a> project explored new approaches to using main memory for storing data with a distributed computing platform that exploits remote direct memory access (RDMA) communication to improve both latency and throughput by an <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/farm-fast-remote-memory\/\">order of magnitude compared to main memory systems that use TCP\/IP<\/a>. By providing <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/no-compromises-distributed-transactions-with-consistency-availability-and-performance\/\">both strong consistency and high performance<\/a> \u2013 challenging the conventional wisdom that high performance required weak consistency \u2013 FaRM allowed developers to focus on program logic rather than handling consistency violations. Initial developer experience highlighted the need for strong consistency for aborted transactions as well as committed ones \u2013 and this was then <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/fast-general-distributed-transactions-with-opacity\/\">achieved using loosely synchronized clocks<\/a>.<\/p>\n\n\n\n<p>At the same time, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-pelican\/\">Project Pelican<\/a> addressed the storage needs of \u201ccold\u201d or infrequently-accessed data. &nbsp;Pelican is a rack-scale disk-based storage unit that trades latency for cost; using a unique data layout and IO scheduling scheme to constrain resources usage so that only 8% of its drives can spin concurrently. Pelican was an example of <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/rack-scale-computing\/\">rack-scale co-design<\/a>: rethinking the storage, compute, and networking hardware as well as the software stack at a rack scale to deliver value at cloud scale.<\/p>\n\n\n\n<p>To further challenge traditional ways of thinking about the storage media and controller stack, researchers began to consider whether a general-purpose CPU was even necessary for many operations. To this end, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/honeycomb\/\">Project Honeycomb<\/a> tackles the challenges of building complex abstractions using FPGAs in CPU-free custom hardware, leaving CPU-based units to focus on control-plane operations.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide has-text-align-wide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/02\/245-kash.pdf\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"DC-DRF: Adaptive Multi-Resource Sharing at Public Cloud Scale\" data-bi-aN=\"citation\" data-bi-cN=\"DC-DRF: Adaptive Multi-Resource Sharing at Public Cloud Scale\">\n\t\t\t\tDC-DRF: Adaptive Multi-Resource Sharing at Public Cloud Scale&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/treating-storage-stack-like-network\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Treating the Storage Stack Like a Network\" data-bi-aN=\"citation\" data-bi-cN=\"Treating the Storage Stack Like a Network\">\n\t\t\t\tTreating the Storage Stack Like a Network&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/pelican-a-building-block-for-exascale-cold-data-storage\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Pelican: A Building Block for Exascale Cold Data Storage\" data-bi-aN=\"citation\" data-bi-cN=\"Pelican: A Building Block for Exascale Cold Data Storage\">\n\t\t\t\tPelican: A Building Block for Exascale Cold Data Storage&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/farm-fast-remote-memory\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"FaRM: Fast Remote Memory\" data-bi-aN=\"citation\" data-bi-cN=\"FaRM: Fast Remote Memory\">\n\t\t\t\tFaRM: Fast Remote Memory&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/ioflow-a-software-defined-storage-architecture\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"IOFlow: A Software-Defined Storage Architecture\" data-bi-aN=\"citation\" data-bi-cN=\"IOFlow: A Software-Defined Storage Architecture\">\n\t\t\t\tIOFlow: A Software-Defined Storage Architecture&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/fast-general-distributed-transactions-with-opacity\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Fast General Distributed Transactions with Opacity\" data-bi-aN=\"citation\" data-bi-cN=\"Fast General Distributed Transactions with Opacity\">\n\t\t\t\tFast General Distributed Transactions with Opacity&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/end-to-end-performance-isolation-through-virtual-datacenters\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"End-to-end Performance Isolation through Virtual Datacenters\" data-bi-aN=\"citation\" data-bi-cN=\"End-to-end Performance Isolation through Virtual Datacenters\">\n\t\t\t\tEnd-to-end Performance Isolation through Virtual Datacenters&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/migrating-enterprise-storage-to-ssds-analysis-of-tradeoffs\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Migrating Enterprise Storage to SSDs: Analysis of Tradeoffs\" data-bi-aN=\"citation\" data-bi-cN=\"Migrating Enterprise Storage to SSDs: Analysis of Tradeoffs\">\n\t\t\t\tMigrating Enterprise Storage to SSDs: Analysis of Tradeoffs&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/everest-scaling-down-peak-loads-through-io-off-loading\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Everest: Scaling down peak loads through I\/O off-loading\" data-bi-aN=\"citation\" data-bi-cN=\"Everest: Scaling down peak loads through I\/O off-loading\">\n\t\t\t\tEverest: Scaling down peak loads through I\/O off-loading&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/the-farsite-project-a-retrospective\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"The Farsite Project: a Retrospective\" data-bi-aN=\"citation\" data-bi-cN=\"The Farsite Project: a Retrospective\">\n\t\t\t\tThe Farsite Project: a Retrospective&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/farsite-federated-available-and-reliable-storage-for-an-incompletely-trusted-environment\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"FARSITE: Federated, available and reliable storage for an incompletely trusted environment\" data-bi-aN=\"citation\" data-bi-cN=\"FARSITE: Federated, available and reliable storage for an incompletely trusted environment\">\n\t\t\t\tFARSITE: Federated, available and reliable storage for an incompletely trusted environment&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/no-compromises-distributed-transactions-with-consistency-availability-and-performance\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"No compromises: distributed transactions with consistency, availability, and performance\" data-bi-aN=\"citation\" data-bi-cN=\"No compromises: distributed transactions with consistency, availability, and performance\">\n\t\t\t\tNo compromises: distributed transactions with consistency, availability, and performance&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/whole-system-persistence-with-non-volatile-memories\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Whole-system Persistence with Non-volatile Memories\" data-bi-aN=\"citation\" data-bi-cN=\"Whole-system Persistence with Non-volatile Memories\">\n\t\t\t\tWhole-system Persistence with Non-volatile Memories&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/write-off-loading-practical-power-management-for-enterprise-storage-2\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Write off-loading: Practical power management for enterprise storage\" data-bi-aN=\"citation\" data-bi-cN=\"Write off-loading: Practical power management for enterprise storage\">\n\t\t\t\tWrite off-loading: Practical power management for enterprise storage&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"optics-for-the-cloud\">Optics for the Cloud<\/h3>\n\n\n\n<p>Microsoft Research Cambridge\u2019s Optics for the Cloud program conducts research to advance and enable the adoption of optical technologies in the rapidly growing field of cloud computing. In addition to novel applications of optics to reinvent local and wide-area networking, this program also is exploring two new approaches to data storage \u2013 glass and holography.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide has-text-align-wide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-2 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/optics-for-the-cloud-research-alliance-establishes-collaborative-research-approach-to-improving-cloud-technology\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Optics for the Cloud Research Alliance establishes collaborative research approach to improving cloud technology\" data-bi-aN=\"citation\" data-bi-cN=\"Optics for the Cloud Research Alliance establishes collaborative research approach to improving cloud technology\">\n\t\t\t\tOptics for the Cloud Research Alliance establishes collaborative research approach to improving cloud technology&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/optics-for-the-cloud-storage-in-the-zettabyte-era-with-dr-ant-rowstron-and-mark-russinovich\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Optics for the cloud: storage in the zettabyte era with Dr. Ant Rowstron and Mark Russinovich\" data-bi-aN=\"citation\" data-bi-cN=\"Optics for the cloud: storage in the zettabyte era with Dr. Ant Rowstron and Mark Russinovich\">\n\t\t\t\tOptics for the cloud: storage in the zettabyte era with Dr. Ant Rowstron and Mark Russinovich&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/could-cloud-storage-be-disrupted-in-the-next-decade\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Could Cloud Storage Be Disrupted in the Next Decade? \" data-bi-aN=\"citation\" data-bi-cN=\"Could Cloud Storage Be Disrupted in the Next Decade? \">\n\t\t\t\tCould Cloud Storage Be Disrupted in the Next Decade? &nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"holographic-storage\">Holographic Storage<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"617\" height=\"712\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Lab-photo.png\" alt=\"Holographic storage research testbed at Microsoft Research Cambridge\" class=\"wp-image-693336\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Lab-photo.png 617w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/09\/Lab-photo-260x300.png 260w\" sizes=\"(max-width: 617px) 100vw, 617px\" \/><figcaption class=\"wp-element-caption\">Holographic storage research testbed at Microsoft Research Cambridge <\/figcaption><\/figure>\n\n\n\n<p>Holographic storage was first proposed back in the 1960s, shortly after the invention of the laser. Holographic optical storage systems store data by recording the interference between the wave-fronts of a modulated optical field, containing the data, and a reference optical field, as a refractive index variation inside the storage media. It is this information containing refractive index variation that is the \u201chologram\u201d. The stored data can then be retrieved by diffracting only the reference field off the hologram to reconstruct the original optical field containing the data.<\/p>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/how-does-holographic-storage-work\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"How does Holographic Storage Work?\" data-bi-aN=\"margin-callout\" data-bi-cN=\"How does Holographic Storage Work?\">\n\t\t\t\tHow does Holographic Storage Work?&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>Capitalizing on recent exponential improvements and commoditization in optical technologies, Project HSD is a collaboration between Microsoft Research Cambridge and Microsoft Azure to re-vitalize and re-imagine holographic storage with a cloud-first design. Researchers in physics, optics, machine learning and storage systems are working together to design mechanical movement free, high-endurance cloud storage that is both performant and cost-effective. This has led to new research challenges and breakthroughs in areas ranging from materials to machine learning.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide has-text-align-wide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-3 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/in-search-for-future-of-cloud-storage-researchers-look-to-holographic-storage-solutions\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"In search for future of cloud storage, researchers look to holographic storage solutions\" data-bi-aN=\"citation\" data-bi-cN=\"In search for future of cloud storage, researchers look to holographic storage solutions\">\n\t\t\t\tIn search for future of cloud storage, researchers look to holographic storage solutions&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Project<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/hsd\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project HSD\" data-bi-aN=\"citation\" data-bi-cN=\"Project HSD\">\n\t\t\t\tProject HSD&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"project-silica\">Project Silica<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/3er1viui9wo30pkxh1v2nh4w-wpengine.netdna-ssl.com\/wp-content\/uploads\/prod\/sites\/549\/2019\/11\/ignite-project-silica-superman-james-clegg_1920x1280.jpg\" alt=\"Project Silica\"\/><\/figure>\n\n\n\n<p>Another project at Microsoft Research is exploring a different medium that can meet our future storage needs for long-lived \u201ccold\u201d data \u2013 glass. Glass is an incredibly durable material \u2013 it can withstand being boiled in water, microwaved, flooded, scoured and demagnetized.<\/p>\n\n\n\n<p>Project Silica is developing the first-ever storage technology designed and built from the media up, using glass, for the cloud. Researchers are leveraging recent discoveries in ultrafast laser optics to store data in quartz glass by using femtosecond lasers, and building a completely new storage system designed from scratch around this technology. This opens up exciting opportunities to challenge and completely re-think traditional storage system design, and to co-design the future hardware and software infrastructure from the cloud.<\/p>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Story<\/span>\n\t\t\t<a href=\"https:\/\/news.microsoft.com\/innovation-stories\/ignite-project-silica-superman\/?utm_source=msr-yt&utm_campaign=1639\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Silica proof of concept stores Warner Bros. \u2018Superman\u2019 movie on quartz glass\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Project Silica proof of concept stores Warner Bros. \u2018Superman\u2019 movie on quartz glass\">\n\t\t\t\tProject Silica proof of concept stores Warner Bros. \u2018Superman\u2019 movie on quartz glass&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>As a proof of concept, Microsoft and Warner Bros. collaborated to successfully store and retrieve the entire 1978 \u201cSuperman\u201d movie on a piece of glass roughly the size of a drink coaster. This project leverages advances first developed at the University of Southampton Optoelectonics Research Centre, and was featured in a Microsoft Ignite 2017 <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noreferrer noopener\" href=\"https:\/\/youtu.be\/sOw8On1iCa4\" target=\"_blank\">keynote<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> on future storage technologies.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide has-text-align-wide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-4 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Presentation<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/project-silica-storage-solutions-for-the-zettabyte-era\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Silica: Storage Solutions for the Zettabyte Era\" data-bi-aN=\"citation\" data-bi-cN=\"Project Silica: Storage Solutions for the Zettabyte Era\">\n\t\t\t\tProject Silica: Storage Solutions for the Zettabyte Era&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Project<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-silica\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Silica\" data-bi-aN=\"citation\" data-bi-cN=\"Project Silica\">\n\t\t\t\tProject Silica&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/glass-a-new-media-for-a-new-era\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Glass: A New Media for a New Era?\" data-bi-aN=\"citation\" data-bi-cN=\"Glass: A New Media for a New Era?\">\n\t\t\t\tGlass: A New Media for a New Era?&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/project-silica-storing-data-in-glass\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Project Silica \u2013 Storing Data in Glass\" data-bi-aN=\"citation\" data-bi-cN=\"Project Silica \u2013 Storing Data in Glass\">\n\t\t\t\tProject Silica \u2013 Storing Data in Glass&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"dna-storage\">DNA Storage<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/3er1viui9wo30pkxh1v2nh4w-wpengine.netdna-ssl.com\/wp-content\/uploads\/prod\/sites\/549\/2019\/03\/hello-data-dna-storage-atcg-automation-sytem-1600x900.jpg\" alt=\"DNA Storage\"\/><\/figure>\n\n\n\n<p>In addition to optical methods, DNA is also an attractive possibility for long-term data storage. DNA is extremely dense \u2013 in practice, it could store up to 1 exabyte per cubic inch. It is very durable, and can last for hundreds or even thousands of years. And it is future-proof: humans will be interested in reading their DNA for as long as humans are around, so technologies to read and write DNA will surely endure. Starting in 2015, Microsoft and University of Washington researchers have been collaborating to use DNA as a high-density, durable and easy-to-manipulate storage medium. The DNA Storage project enables molecular-level data storage into DNA molecules by leveraging biotechnology advances in synthesizing, manipulating and sequencing DNA to develop archival storage.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"480\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2016\/07\/dna-hero-1024x480.jpg\" alt=\"DNA storage: image of a pencil tip and a test tube tip\" class=\"wp-image-266451\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/07\/dna-hero-1024x480.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/07\/dna-hero-300x141.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/07\/dna-hero-768x360.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/07\/dna-hero.jpg 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>In this clip from the Microsoft Research Podcast, Karin Strauss discusses several properties of DNA that make it a promising medium for data storage. <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/storing-digital-data-in-synthetic-dna-with-dr-karin-strauss\/\">Listen to the full episode<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2020\/11\/storage_strauss_excerpt.mp3\"><\/audio><\/figure>\n\n\n\n<p class=\"has-small-font-size\"><em>We\u2019re very excited about DNA for at least three of its properties. The first one is density. So instead of really storing the bits into devices that we have to manufacture, we are really looking at a molecule, storing data in a molecule itself. And so, a molecule can be a lot smaller than the devices we\u2019re making. Just to give you an example, you could store the information, today stored in a datacenter, one exabyte of data, into a cubic inch of DNA. So that\u2019s quite tiny. Durability is the next interesting property of DNA. And so, DNA, if preserved under the right conditions, can keep for a very long time, which is not necessarily possible with media that\u2019s commercial today. I think the longest commercial media is rated for thirty years. That\u2019s tape. Still tape! DNA, if encapsulated in the right conditions, has been shown to survive thousands of years. And so, it\u2019s very interesting from a data preservation perspective as well. And then, one other property is that, now that we know how to read DNA and we\u2019ll always have the technology to read it. So now we\u2019ll have those readers\u2026 (if we don\u2019t have those readers, we have a real problem) \u2026but we\u2019ll have those readers forever, as long as there is civilization. So, it\u2019s not like floppy disks that are in the back of a drawer just gathering dust. We\u2019ll really have technology to read it.<\/em><\/p>\n\n\n\n<p>While DNA storage at scale is not yet practical, due to the current state of DNA synthesis and sequencing, these technologies are improving quite rapidly with advances in the biotech industry. At the same time, the impending limits of silicon technology as Moore\u2019s Law comes to an end create an incentive to further explore the use of biotechnology for computation as well as storage. For instance, DNA-based databases were proposed decades ago; more recent evidence of their practicality has renewed interest in related theory and applications. In this area, Microsoft researchers have designed a DNA-based digital data store equipped with a mechanism for content-based similarity search (see paper below).<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide has-text-align-wide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-5 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Project<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/dna-storage\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"DNA Storage\" data-bi-aN=\"citation\" data-bi-cN=\"DNA Storage\">\n\t\t\t\tDNA Storage&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/molecular-digital-data-storage-using-dna\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Molecular Digital Data Storage using DNA\" data-bi-aN=\"citation\" data-bi-cN=\"Molecular Digital Data Storage using DNA\">\n\t\t\t\tMolecular Digital Data Storage using DNA&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Story<\/span>\n\t\t\t<a href=\"https:\/\/news.microsoft.com\/innovation-stories\/hello-data-dna-storage\/?utm_source=elevate&utm_campaign=1385\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"With a \u201chello,\u201d Microsoft and UW demonstrate first fully automated DNA data storage\" data-bi-aN=\"citation\" data-bi-cN=\"With a \u201chello,\u201d Microsoft and UW demonstrate first fully automated DNA data storage\">\n\t\t\t\tWith a \u201chello,\u201d Microsoft and UW demonstrate first fully automated DNA data storage&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/microsoft-and-uw-demonstrate-first-fully-automated-dna-data-storage\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Microsoft and UW Demonstrate First Fully-Automated DNA Data Storage\" data-bi-aN=\"citation\" data-bi-cN=\"Microsoft and UW Demonstrate First Fully-Automated DNA Data Storage\">\n\t\t\t\tMicrosoft and UW Demonstrate First Fully-Automated DNA Data Storage&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Webinar<\/span>\n\t\t\t<a href=\"https:\/\/note.microsoft.com\/MSR-Webinar-Ultra-Dense-Data-Storage-Registration-On-Demand.html\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Ultra-dense storage and extreme parallelism with electronic-molecular systems\" data-bi-aN=\"citation\" data-bi-cN=\"Ultra-dense storage and extreme parallelism with electronic-molecular systems\">\n\t\t\t\tUltra-dense storage and extreme parallelism with electronic-molecular systems&nbsp;<span class=\"glyph-append glyph-append-share glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>\u201cOne of the challenges for us as a company, and us as an industry, is that many of the technologies we rely on are beginning to get to the point where either they are at the end, or they\u2019re starting to get to the point where you can see the end. Moore\u2019s Law is a [&hellip;]<\/p>\n","protected":false},"author":38004,"featured_media":619044,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[1,244017],"tags":[246652],"research-area":[13552,13547],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-702310","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","category-research-collection","tag-storage-technologies","msr-research-area-hardware-devices","msr-research-area-systems-and-networking","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199561],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-960x540.jpg\" class=\"img-object-cover\" alt=\"Video: Project Silica - Storing Data in Glass\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/10\/Read-Write-Final.mp4.00_02_57_03.Still003.jpg 1920w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","byline":"","formattedDate":"November 9, 2020","formattedExcerpt":"\u201cOne of the challenges for us as a company, and us as an industry, is that many of the technologies we rely on are beginning to get to the point where either they are at the end, or they\u2019re starting to get to the point&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/702310"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/38004"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=702310"}],"version-history":[{"count":50,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/702310\/revisions"}],"predecessor-version":[{"id":924951,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/702310\/revisions\/924951"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/619044"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=702310"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=702310"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=702310"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=702310"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=702310"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=702310"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=702310"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=702310"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=702310"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=702310"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=702310"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}