{"id":167585,"date":"2014-10-06T00:00:00","date_gmt":"2014-10-06T00:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/msr-research-item\/pelican-a-building-block-for-exascale-cold-data-storage\/"},"modified":"2018-10-16T22:00:48","modified_gmt":"2018-10-17T05:00:48","slug":"pelican-a-building-block-for-exascale-cold-data-storage","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/pelican-a-building-block-for-exascale-cold-data-storage\/","title":{"rendered":"Pelican: A building Block for Exascale Cold Data Storage"},"content":{"rendered":"<div class=\"asset-content\">\n<p>A significant fraction of data stored in cloud storage is rarely accessed. This data is referred to as <em>cold data<\/em>; cost-effective storage for cold data has become a challenge for cloud providers. Pelican is a rack-scale hard disk based storage unit designed as the basic building block for exabyte scale storage for cold data. In Pelican, server, power, cooling and interconnect bandwidth resources are provisioned by design to support cold data workloads; this <em>right-provisioning<\/em> significantly reduces Pelican\u2019s total cost of ownership compared to traditional disk-based storage.<\/p>\n<p>Resource right-provisioning in Pelican means only 8% of the drives can be concurrently spinning. This introduces complex resource management to be handled by the Pelican storage stack. Resource restrictions are expressed as constraints over the hard drives. The data layout and IO scheduling ensures that these constraints are not violated. We evaluate the performance of a prototype Pelican, and compare against a traditional resource overprovisioned storage rack using a cross-validated simulator. We show that compared to this over-provisioned storage rack Pelican performs well for cold workloads, providing high throughput with acceptable latency.<\/p>\n<\/div>\n<p><!-- .asset-content --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A significant fraction of data stored in cloud storage is rarely accessed. This data is referred to as cold data; cost-effective storage for cold data has become a challenge for cloud providers. Pelican is a rack-scale hard disk based storage unit designed as the basic building block for exabyte scale storage for cold data. In [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13547],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-167585","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-systems-and-networking","msr-locale-en_us"],"msr_publishername":"","msr_edition":"Inproceedings","msr_affiliation":"","msr_published_date":"2014-10-06","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"217351","msr_publicationurl":"https:\/\/www.usenix.org\/conference\/osdi14\/technical-sessions\/presentation\/balakrishnan","msr_doi":"","msr_publication_uploader":[{"type":"file","title":"osdi2014-Pelican.pdf","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/10\/osdi2014-Pelican.pdf","id":217351,"label_id":0},{"type":"url","title":"https:\/\/www.usenix.org\/conference\/osdi14\/technical-sessions\/presentation\/balakrishnan","viewUrl":false,"id":false,"label_id":0}],"msr_related_uploader":"","msr_attachments":[{"id":0,"url":"https:\/\/www.usenix.org\/conference\/osdi14\/technical-sessions\/presentation\/balakrishnan"},{"id":217351,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/10\/osdi2014-Pelican.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"shobanab","user_id":33627,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=shobanab"},{"type":"user_nicename","value":"rjblack","user_id":33417,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=rjblack"},{"type":"user_nicename","value":"austind","user_id":31148,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=austind"},{"type":"user_nicename","value":"pengland","user_id":33226,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=pengland"},{"type":"text","value":"Adam Glass","user_id":0,"rest_url":false},{"type":"user_nicename","value":"dharpe","user_id":31621,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=dharpe"},{"type":"user_nicename","value":"serleg","user_id":33580,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=serleg"},{"type":"text","value":"Aaron Ogus","user_id":0,"rest_url":false},{"type":"user_nicename","value":"ericpete","user_id":31753,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=ericpete"},{"type":"user_nicename","value":"antr","user_id":31061,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=antr"}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[],"msr_project":[171302,382118],"publication":[],"video":[],"download":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":171302,"post_title":"Rack-scale Computing","post_name":"rack-scale-computing","post_type":"msr-project","post_date":"2014-02-19 03:07:37","post_modified":"2017-06-14 09:04:32","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/rack-scale-computing\/","post_excerpt":"&nbsp; New hardware technology such as systems- and networks-on-chip (SOCs and NOCs), switchless network fabrics, silicon photonics, and RDMA, are redefining the landscape of data center computing, enabling interconnecting thousands of cores at high speed at the scale of today's racks. We refer to this new class of hardware as rack-scale computers (RSCs) because the rack is increasingly replacing the individual server as the basic building block of modern data centers. Early examples of RSCs&hellip;","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171302"}]}},{"ID":382118,"post_title":"Project Pelican","post_name":"project-pelican","post_type":"msr-project","post_date":"2017-05-18 08:38:07","post_modified":"2018-11-05 09:39:32","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-pelican\/","post_excerpt":"Pelican: A building block for exascale cold data storage Pelican aims to store infrequently accessed (cold) data as inexpensively as possible. The amount of data stored is growing at a huge rate, but not all of it is \u201chot,\u201d i.e. frequently accessed.\u00a0 There is little reason to store cold data in the same high-performance, high-cost systems as hot data. Our goal was to design a storage system\u2014called Pelican\u2014specifically to take advantage of the needs of&hellip;","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/382118"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/167585"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/167585\/revisions"}],"predecessor-version":[{"id":541195,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/167585\/revisions\/541195"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=167585"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=167585"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=167585"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=167585"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=167585"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=167585"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=167585"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=167585"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=167585"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=167585"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=167585"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=167585"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=167585"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=167585"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=167585"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=167585"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}