Demand-driven resource allocation for cloud databases has become a popular research direction. Recent approaches have evolved from reactive policies to proactive decision making. These approaches leverage not only the current resource demand but also the predicted demand to make more informed resource allocation decisions for each database and thus improve the quality of service and reduce the operational costs. We present an infrastructure that enables proactive resource allocation capabilities for millions of serverless Azure SQL databases. Our solution finds near-optimal middle ground between high availability of resources, low operational costs, and low computational overhead of the proactive policy. We describe the design principles we followed and the architectural decisions we made during this cross-team, multi-year journey. Given the size and scope of our solution, we believe that the relational cloud databases in other companies could benefit from the proactive resource allocation capabilities.<\/p>\n","protected":false},"excerpt":{"rendered":"
Demand-driven resource allocation for cloud databases has become a popular research direction. Recent approaches have evolved from reactive policies to proactive decision making. These approaches leverage not only the current resource demand but also the predicted demand to make more informed resource allocation decisions for each database and thus improve the quality of service and […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13563],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[249832,251908,246691,251647],"msr-conference":[260221],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1017390","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-data-platform-analytics","msr-locale-en_us","msr-field-of-study-autonomous-system-internet","msr-field-of-study-cloud-database","msr-field-of-study-computer-science","msr-field-of-study-distributed-database"],"msr_publishername":"ACM","msr_edition":"","msr_affiliation":"","msr_published_date":"2024-3-1","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"ACM","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/03\/prorp.pdf","label_id":"243109","label":0},{"type":"doi","viewUrl":"false","id":"false","title":"10.1145\/3626246.3653371","label_id":"243106","label":0}],"msr_related_uploader":"","msr_attachments":[{"id":1023093,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/04\/prorp.pdf"},{"id":1023087,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/04\/Proactive_Resume_and_Pause_of_Resources-5.pdf"},{"id":1017399,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/03\/Proactive_Resume_and_Pause_of_Resources-5.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"Olga Poppe","user_id":40054,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Olga Poppe"},{"type":"text","value":"Pankaj Arora","user_id":0,"rest_url":false},{"type":"text","value":"Sakshi Sharma","user_id":0,"rest_url":false},{"type":"text","value":"Jie Chen","user_id":0,"rest_url":false},{"type":"text","value":"Sachin Pandit","user_id":0,"rest_url":false},{"type":"text","value":"Rahul Sawhney","user_id":0,"rest_url":false},{"type":"text","value":"Vaishali Jhalani","user_id":0,"rest_url":false},{"type":"text","value":"Willis Lang","user_id":0,"rest_url":false},{"type":"text","value":"Qun Guo","user_id":0,"rest_url":false},{"type":"text","value":"Anupriya Inumella","user_id":0,"rest_url":false},{"type":"text","value":"Sanjana Dulipeta Sridhar","user_id":0,"rest_url":false},{"type":"text","value":"Dheren Gala","user_id":0,"rest_url":false},{"type":"text","value":"Nilesh Rathi","user_id":0,"rest_url":false},{"type":"text","value":"Morgan Oslake","user_id":0,"rest_url":false},{"type":"text","value":"Alexandru Chirica","user_id":0,"rest_url":false},{"type":"text","value":"Sarika Iyer","user_id":0,"rest_url":false},{"type":"text","value":"Prateek Goel","user_id":0,"rest_url":false},{"type":"text","value":"Ajay Kalhan","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[],"msr_project":[723823],"publication":[],"video":[],"download":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":723823,"post_title":"Proactive Resource Allocation","post_name":"moneyball","post_type":"msr-project","post_date":"2021-04-24 17:47:08","post_modified":"2024-07-13 10:59:09","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/moneyball\/","post_excerpt":"Modern cloud services aim to allocate resources if and only if the customers use these resources to optimize quality of service and operational cost efficiency. Recent approaches have evolved from merely reactive policies to proactive decision making. These approaches leverage not only the current resource demand but also the predicted future demand to make more informed resource allocation decisions for each database. We define a proactive resource allocation policy and design a distributed infrastructure that…","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/723823"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1017390","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":4,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1017390\/revisions"}],"predecessor-version":[{"id":1023090,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1017390\/revisions\/1023090"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1017390"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=1017390"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=1017390"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1017390"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=1017390"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=1017390"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=1017390"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=1017390"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=1017390"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1017390"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1017390"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=1017390"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=1017390"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=1017390"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1017390"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1017390"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}