{"id":465501,"date":"2018-02-08T16:20:46","date_gmt":"2018-02-09T00:20:46","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=465501"},"modified":"2020-03-13T17:36:40","modified_gmt":"2020-03-14T00:36:40","slug":"multitenancy","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/multitenancy\/","title":{"rendered":"Multitenancy in Autopilot"},"content":{"rendered":"<p>We leverage spare capacity in Bing to run batch workloads (i.e., data analytics). From this project we started multiple research contributions:<\/p>\n<ul>\n<li>This project originally aimed to harvest idle resources in large scale datacenters. We leverage the historical patterns of primary tenants to harvest both compute and storage from latency sensitive services. The research details are described in our <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/harvesting-spare-cycles-and-storage-in-large-scale-datacenters\/\">paper<\/a> at <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.usenix.org\/conference\/osdi16\" target=\"_blank\" rel=\"noopener noreferrer\">OSDI 2016<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/li>\n<li>We needed to scale HDFS to tens of thousands of servers. The research details are described in our <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/scaling-distributed-file-systems-resource-harvesting-datacenters\/\">paper<\/a> at <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.usenix.org\/conference\/atc17\" target=\"_blank\" rel=\"noopener noreferrer\">USENIX ATC 2017<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. We build\u00a0<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/hadoop.apache.org\/docs\/current\/hadoop-project-dist\/hadoop-hdfs-rbf\/HDFSRouterFederation.html\" target=\"_blank\" rel=\"noopener noreferrer\">HDFS Router-based Federation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u00a0which is contributed back to <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/hadoop.apache.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">Apache Hadoop<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/li>\n<li>The large heterogeneous environment was very prone to long tail latencies. Our proposal to manage tail latency when accessing data in HDFS is described in our <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/managing-tail-latency-in-datacenter-scale-file-systems-under-production-constraints\/\">paper<\/a> at <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.eurosys2019.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">EuroSys 2019<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>The Multitenancy in Autopilot project leverages spare capacity in Bing to run batch workloads (i.e., data analytics).<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13547],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-465501","msr-project","type-msr-project","status-publish","hentry","msr-research-area-systems-and-networking","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"2014-06-02","related-publications":[392756,168898,568647],"related-downloads":[],"related-videos":[],"related-groups":[144927,282170],"related-events":[391415,309251],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Ricardo Bianchini","user_id":33393,"people_section":"Section name 1","alias":"ricardob"},{"type":"user_nicename","display_name":"Sameh Elnikety","user_id":33503,"people_section":"Section name 1","alias":"samehe"},{"type":"user_nicename","display_name":"&Iacute;&ntilde;igo Goiri","user_id":32102,"people_section":"Section name 1","alias":"inigog"}],"msr_research_lab":[199565],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/465501","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":14,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/465501\/revisions"}],"predecessor-version":[{"id":643242,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/465501\/revisions\/643242"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=465501"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=465501"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=465501"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=465501"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=465501"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}