{"id":165291,"date":"2013-11-01T00:00:00","date_gmt":"2013-11-01T00:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/msr-research-item\/ioflow-a-software-defined-storage-architecture\/"},"modified":"2018-10-16T21:57:25","modified_gmt":"2018-10-17T04:57:25","slug":"ioflow-a-software-defined-storage-architecture","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/ioflow-a-software-defined-storage-architecture\/","title":{"rendered":"IOFlow: A Software-Defined Storage Architecture"},"content":{"rendered":"<div class=\"asset-content\">\n<p>In data centers, the IO path to storage is long and complex. It comprises many layers or \u201cstages\u201d with opaque interfaces between them. This makes it hard to enforce end-to-end policies that dictate a storage IO flow\u2019s performance (e.g., guarantee a tenant\u2019s IO bandwidth) and routing (e.g., route an untrusted VM\u2019s traffic through a sanitization middlebox). These policies require IO differentiation along the flow path and global visibility at the control plane. We design IOFlow, an architecture that uses a logically centralized control plane to enable high-level flow policies. IOFlow adds a queuing abstraction at data-plane stages and exposes this to the controller. The controller can then translate policies into queuing rules at individual stages. It can also choose among multiple stages for policy enforcement.<\/p>\n<p>We have built the queue and control functionality at two key OS stages\u2013 the storage drivers in the hypervisor and the storage server. IOFlow does not require application or VM changes, a key strength for deployability. We have deployed a prototype across a small testbed with a 40 Gbps network and storage devices. We have built control applications that enable a broad class of multipoint flow policies that are hard to achieve today.<\/p>\n<\/div>\n<p><!-- .asset-content --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In data centers, the IO path to storage is long and complex. It comprises many layers or \u201cstages\u201d with opaque interfaces between them. This makes it hard to enforce end-to-end policies that dictate a storage IO flow\u2019s performance (e.g., guarantee a tenant\u2019s IO bandwidth) and routing (e.g., route an untrusted VM\u2019s traffic through a sanitization [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13547],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-165291","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-systems-and-networking","msr-locale-en_us"],"msr_publishername":"ACM","msr_edition":"SOSP'13: The 24th ACM Symposium on Operating Systems Principles","msr_affiliation":"","msr_published_date":"2013-11-01","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"SOSP\u201913, Nov. 3\u20136, 2013, Farmington, PA, USA. ACM 978-1-4503-2388-8\/13\/11. http:\/\/dx.doi.org\/10.1145\/2517349.2522723","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"218119","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","title":"ioflow-sosp13.pdf","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2013\/11\/ioflow-sosp13.pdf","id":218119,"label_id":0}],"msr_related_uploader":"","msr_attachments":[{"id":218119,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2013\/11\/ioflow-sosp13.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"etheres","user_id":31766,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=etheres"},{"type":"user_nicename","value":"hiballan","user_id":32008,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=hiballan"},{"type":"user_nicename","value":"gregos","user_id":31908,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=gregos"},{"type":"user_nicename","value":"thomkar","user_id":34031,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=thomkar"},{"type":"user_nicename","value":"antr","user_id":31061,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=antr"},{"type":"text","value":"Tom Talepy","user_id":0,"rest_url":false},{"type":"user_nicename","value":"rjblack","user_id":33417,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=rjblack"},{"type":"text","value":"Timothy Zhu","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[],"msr_project":[171193,170657],"publication":[],"video":[],"download":[169402],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":171193,"post_title":"Software-Defined Storage (SDS) Architectures","post_name":"software-defined-storage-architectures","post_type":"msr-project","post_date":"2013-08-14 01:44:29","post_modified":"2018-08-20 10:33:12","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/software-defined-storage-architectures\/","post_excerpt":"In data centers, the IO path to storage is long and complex. It comprises many layers or \u201cstages\u201d with opaque interfaces between them. This makes it hard to enforce end-to-end policies that dictate a storage IO flow\u2019s performance (e.g., guarantee a tenant\u2019s IO bandwidth) and routing (e.g., route an untrusted VM\u2019s traffic through a sanitization middlebox). We are researching architectures that decouple control from data flow to enable such policies.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171193"}]}},{"ID":170657,"post_title":"Predictable Data Centers (PDC)","post_name":"predictable-data-centers-pdc","post_type":"msr-project","post_date":"2011-02-11 10:10:03","post_modified":"2018-08-20 10:30:03","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/predictable-data-centers-pdc\/","post_excerpt":"Performance predictability is a key requirement for high-performant applications in today's multi-tenant datacenters. Online services running in infrastructure datacenters need such predictability to satisfy applications SLAs. Cloud datacenters require guaranteed performance to bound customer costs and spur adoption. However, the network and storage stack used in today\u2019s datacenters is unaware of such application requirements. This projects examines how to enable preditable datacenters. Performance predictability is a key requirement for high-performant applications in today's multi-tenant data&hellip;","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/170657"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/165291","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/165291\/revisions"}],"predecessor-version":[{"id":540634,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/165291\/revisions\/540634"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=165291"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=165291"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=165291"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=165291"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=165291"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=165291"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=165291"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=165291"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=165291"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=165291"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=165291"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=165291"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=165291"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=165291"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=165291"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=165291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}