{"id":958941,"date":"2023-08-07T15:05:54","date_gmt":"2023-08-07T22:05:54","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&#038;p=958941"},"modified":"2024-01-23T14:51:32","modified_gmt":"2024-01-23T22:51:32","slug":"towards-building-autonomous-data-services-on-azure","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/towards-building-autonomous-data-services-on-azure\/","title":{"rendered":"Towards Building Autonomous Data Services on Azure"},"content":{"rendered":"<p>Modern cloud has turned data services into easily accessible commodities. With just a few clicks, users are now able to access a catalog of data processing systems for a wide range of tasks. How- ever, the cloud brings in both complexity and opportunity. While cloud users can quickly start an application by using various data services, it can be difficult to configure and optimize these services to gain the most value from them. For cloud providers, managing every aspect of an ever-increasing set of data services, while meeting customer SLAs and minimizing operational cost is becoming more challenging. Cloud technology enables the collection of significant amounts of workload traces and system telemetry. With the progress in data science (DS) and machine learning (ML), it is feasible and desirable to utilize a data-driven, ML-based approach to automate various aspects of data services, resulting in the creation of autonomous data services. This paper presents our perspectives and insights on creating autonomous data services on Azure. It also covers the future endeavors we plan to undertake and unresolved issues that still need attention.<\/p>\n<span id=\"label-external-link\" class=\"sr-only\" aria-hidden=\"true\">Opens in a new tab<\/span>","protected":false},"excerpt":{"rendered":"<p>Modern cloud has turned data services into easily accessible commodities. With just a few clicks, users are now able to access a catalog of data processing systems for a wide range of tasks. How- ever, the cloud brings in both complexity and opportunity. While cloud users can quickly start an application by using various data [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13556,13563],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-field-of-study":[246694,248884],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2023-6-1","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/08\/3555041.3589674.pdf","id":"1001319","title":"3555041-3589674","label_id":"243109","label":0},{"type":"doi","viewUrl":"false","id":"false","title":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3555041.3589674","label_id":"243109","label":0}],"msr_related_uploader":"","msr_attachments":[{"id":1001319,"url":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/3555041.3589674.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"Yiwen Zhu","user_id":39438,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Yiwen Zhu"},{"type":"user_nicename","value":"Yuanyuan Tian","user_id":40708,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Yuanyuan Tian"},{"type":"user_nicename","value":"Joyce Cahoon","user_id":40012,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Joyce Cahoon"},{"type":"user_nicename","value":"Subru Krishnan","user_id":33746,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Subru Krishnan"},{"type":"text","value":"Ankita Agarwal","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Rana Alotaibi","user_id":42168,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Rana Alotaibi"},{"type":"user_nicename","value":"Jes\u00fas Camacho Rodr\u00edguez","user_id":40693,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Jes\u00fas Camacho Rodr\u00edguez"},{"type":"text","value":"Bibin A Chundatt","user_id":0,"rest_url":false},{"type":"text","value":"Andrew Chung","user_id":0,"rest_url":false},{"type":"text","value":"Niharika Dutta","user_id":0,"rest_url":false},{"type":"text","value":"Andrew Fogarty","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Anja Gruenheid","user_id":40696,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Anja Gruenheid"},{"type":"user_nicename","value":"Brandon Haynes","user_id":40000,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Brandon Haynes"},{"type":"user_nicename","value":"Matteo Interlandi","user_id":39784,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Matteo Interlandi"},{"type":"text","value":"Minu Iyer","user_id":0,"rest_url":false},{"type":"text","value":"Nick Jurgens","user_id":0,"rest_url":false},{"type":"text","value":"Sumeet Khushalani","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Brian Kroth","user_id":40024,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Brian Kroth"},{"type":"text","value":"Manoj Kumar","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Jyoti Leeka","user_id":40066,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Jyoti Leeka"},{"type":"text","value":"Sergiy Matusevych","user_id":0,"rest_url":false},{"type":"text","value":"Minni Mittal","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Andreas Mueller","user_id":40030,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Andreas Mueller"},{"type":"text","value":"Kartheek Muthyala","user_id":0,"rest_url":false},{"type":"text","value":"Harsha Nagulapalli","user_id":0,"rest_url":false},{"type":"text","value":"Yoonjae Park","user_id":0,"rest_url":false},{"type":"text","value":"Hiren Patel","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Anna Pavlenko","user_id":40009,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Anna Pavlenko"},{"type":"user_nicename","value":"Olga Poppe","user_id":40054,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Olga Poppe"},{"type":"text","value":"Santhosh Ravindran","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Karla Saur","user_id":39991,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Karla Saur"},{"type":"user_nicename","value":"Rathijit Sen","user_id":39450,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Rathijit Sen"},{"type":"text","value":"Steve Suh","user_id":0,"rest_url":false},{"type":"text","value":"Arijit Tarafdar","user_id":0,"rest_url":false},{"type":"text","value":"Kunal Waghray","user_id":0,"rest_url":false},{"type":"text","value":"Demin Wang","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Carlo Curino","user_id":31352,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Carlo Curino"},{"type":"user_nicename","value":"Raghu Ramakrishnan","user_id":40051,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Raghu Ramakrishnan"}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[684024],"msr_project":[],"publication":[],"video":[],"download":[],"msr_publication_type":"inproceedings","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/958941"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/958941\/revisions"}],"predecessor-version":[{"id":1001328,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/958941\/revisions\/1001328"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=958941"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=958941"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=958941"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=958941"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=958941"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=958941"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=958941"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=958941"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=958941"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=958941"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=958941"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=958941"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=958941"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=958941"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}