{"id":306476,"date":"2009-08-10T09:00:03","date_gmt":"2009-08-10T16:00:03","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=306476"},"modified":"2016-10-17T11:30:48","modified_gmt":"2016-10-17T18:30:48","slug":"energy-efficiency-work-reaps-rewards","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/energy-efficiency-work-reaps-rewards\/","title":{"rendered":"Energy-Efficiency Work Reaps Rewards"},"content":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research<\/em><\/p>\n

These days, more than ever, it\u2019s important for computing to be energy-efficient. Particularly in data centers, energy requirements represent a significant portion of operational costs, and power and cooling needs help dictate where data centers can be located, how close to capacity they can operate, and how robust they are to failure.<\/p>\n

In part, however, this is true because computers are precision machines. They\u2019re hard-wired that way. Ask a computer what the average daily temperature is in Seattle on May 16, and you might get an answer such as 57.942368 degrees. While most of us would be hard-pressed to discern the atmospheric difference between 57.942368 degrees and 58, computers are able to provide that precision\u2014even when such precision is all but useless.<\/p>\n

Surely, there must be a way to save the energy utilized to offer such superfluous exactitude. And if you could do so, given the thousands upon thousands of machines housed in a typical data center, surely significant savings would result.<\/p>\n

Trishul Chilimbi<\/a> certainly believes so. Chilimbi, senior researcher and manager of the Runtime Analysis and Design team within Microsoft Research Redmond<\/a>\u2019s Research in Software Engineering<\/a> organization, is leading an effort to underscore the importance of energy-efficient computing, as evidenced in the paper Green: A System for Supporting Energy-Conscious Programming Using Principled Approximation<\/em><\/a>.<\/p>\n

\"Trishul

Trishul Chilimbi<\/p><\/div>\n

\u201cThe Green project,\u201d Chilimbi explains, \u201cis looking at how we can save energy efficiency purely looking at software. The high-level goal is to give programmers high-level abstractions that they can use to express domain knowledge to the underlying system so it can take advantage.\u201d<\/p>\n

In other words, round that temperature off to the nearest degree, please.<\/p>\n

\u201cEspecially in the cloud-computing space and data centers,\u201d Chilimbi says, \u201cthere are service-level agreements that you\u2019re not always going to get a very precise answer. And even where they\u2019re not in place, programmers know that there is an asymmetric importance of results in certain domains.<\/p>\n

\u201cLet\u2019s take search for an example. My query results on the first few pages are very important. The ones on the 100th page might not be as important. We\u2019d like to devote more resources to focusing on the first several pages and not as much to those much lower down the rank system.\u201d<\/p>\n

So, with a bit of programmer-defined approximation, computers can take much less time to provide results that, while not as precise as possible, deliver valuable information for real-world problems in a fraction of the time and the resource requirements necessitated by precision measurements.<\/p>\n

\u201cComputer programming languages have been built on a mathematical foundation to provide very precise answers,\u201d Chilimbi says, \u201cand in domains where you don\u2019t need that precision, you end up overcomputing and throwing away results. If programmers specify what their requirements are, we\u2019ll avoid this and do exactly what you need to meet those requirements, but no more.\u201d<\/p>\n

The energy savings, of course, depend upon the level of approximation the programmer specifies.<\/p>\n

\u201cSay 99.999 percent of all results should be similar as if you didn\u2019t have this Green system in place,\u201d Chilimbi says. \u201cThat\u2019s a very high threshold. And with that, we see savings of about 20 percent, which translates directly into energy, because if you do less work, you\u2019re consuming fewer resources and less energy.<\/p>\n

\u201cWe found that if you\u2019re willing to tolerate even lower levels of accuracy\u2014say I have a speech-recognition program, and I can tolerate a loss of accuracy of 5 percent\u2014maybe I can get it back with smart algorithms and understanding use context and get a factor of 2X improvement.\u201d<\/p>\n

Such results might seem astounding, but not when you stop to think about it.<\/p>\n

\u201cIt\u2019s not surprising there\u2019s this diminishing margin of returns,\u201d Chilimbi says. \u201cAs you go for the last improvement, you\u2019re using more and more resources, and as you\u2019re willing to scale back, you get fairly large savings. It can be very significant, and it\u2019s up to the programmer to intelligently specify what\u2019s realistic for the domain.<\/p>\n

\u201cIn many domains, such as graphics in a video game, anything beyond 60 frames per second the user can\u2019t distinguish. You have potential for a graceful degradation, and it\u2019s fairly significant.\u201d<\/p>\n

Unnecessary Precision<\/h2>\n

But, he stresses, the savings depend on the domain.<\/p>\n

\u201cYou don\u2019t want to do this for banking,\u201d Chilimbi smiles. \u201cThere are certain domains in which you need absolute precision. But I think there are more domains for which you don\u2019t need this precision. A lot of them are concerned with who the consumers of the information are. If they are human, they\u2019re limited by audio and visual sensory perception. There is leeway to play around with that.\u201d<\/p>\n

You\u2019ve heard the question: If a tree falls in a forest and nobody\u2019s around to hear it, does it make a sound? Now, try this one: If a human isn\u2019t able to perceive the difference between a 10-millisecond response time and one that takes 5 milliseconds, why not save the energy the extra speed demands?<\/p>\n

Graphics is one such domain. Speech recognition is another.<\/p>\n

\u201cThere is a certain amount of fidelity we can deal with,\u201d Chilimbi says. \u201cIf my battery is going to die, and I really want to see the end of this movie, I might be willing to compromise image quality for a little bit. Search is another classic example.\u201d<\/p>\n

Lest you get the idea that the Green project is applied research just to squeeze more capability out of data centers, think again. As Chilimbi notes, there are legitimate, intriguing, basic research questions being refined as part of this effort. One is to rethink programming languages from the perspective of a requirement of approximate versus absolute precision. How do you provide guarantees to support your approximation techniques? And how can programmers communicate their high-level knowledge about programs and application domains to underlying hardware?<\/p>\n

The answer to the latter question might require designing hardware and software in tandem.<\/p>\n

\u201cOne idea might be to use lower-energy components,\u201d Chilimbi says, \u201cbut to get the performance back, you could have software communicate more of its intent, so hardware doesn\u2019t have to infer it. That\u2019s another high-level goal: How can we co-design software and hardware so that the whole system is energy-efficient, rather than trying to deal with these pieces in isolation?\u201d<\/p>\n

Quality of Service<\/h2>\n

Providing quality-of-service guarantees is equally challenging.<\/p>\n

\u201cSay I have a function in a program,\u201d Chilimbi explains, \u201cand there is a quality-of-service contract this function enforces. We have modular programs that have a notion of abstraction\u2014I don\u2019t need to know how this contract is implemented, just that this is the contract this module enforces. You could have a similar kind of quality-of-service contract in modules. Then the hard part becomes: How do I compose these quality-of-service agreements and construct an overall quality of service for the program?<\/p>\n

\u201cThat\u2019s a challenge, because you can have non-linear effects of combining these things. What kind of guarantees can you give? Can you give static guarantees, or are they only runtime guarantees?\u201d<\/p>\n

So, if you have approximated certain procedures sufficiently that they provide good-enough results while using fewer resources, and you analyze the level of precision necessary for a specific task, you\u2019re home-free, right?<\/p>\n

Not so fast. What, now, can you do with the model you have just constructed?<\/p>\n

\u201cYou could say, \u2018Well, I\u2019m done if I have the model,\u2019 \u201cChilimbi says. \u201cI\u2019m going to use the problem, and I\u2019m going to assume that everything follows this model, and if that\u2019s true, everything will be good because the model has been designed so that it is only approximate when it guarantees that it can meet whatever quality of service is required.\u201d<\/p>\n

The problem is, the real world isn\u2019t always as accepting of lab-derived models as researchers would like.<\/p>\n

\u201cUnfortunately,\u201d Chilimbi states, \u201cin the real world, you might get unanticipated scenarios or inputs. You have to prepare for all possibilities. That\u2019s why the static model is a good starting point, but it\u2019s not sufficient, especially if you have a scenario very different from the ones you\u2019re seen. To handle that, you need a third part that says, \u2018Every once in a while, I\u2019m going to check the scenarios I\u2019ve seen, the usage I\u2019ve seen, to ensure that I\u2019m still meeting the quality-of-service requirement.<\/p>\n

\"Green

This diagram outlines how the Green project is designed to work.<\/p><\/div>\n

\u201cThe way we check that is, on the same scenario, we execute both the precise version and the approximate version our model suggests and measure the quality-of-service error and see if it\u2019s still above the threshold. If everything is fine, we\u2019re OK. But if the scenario is very different from what we anticipated, then we might recalibrate the model to cope with this. You need this third part to guarantee that no matter what the situation is, you will meet quality of service. That is crucial, because you want to have this guarantee for your users, and just modeling limited inputs doesn\u2019t allow you to give that guarantee.\u201d<\/p>\n

Chilimbi says he always has been interested in computer performance and optimization, but in recent years, he has seen his interests shift a little bit.<\/p>\n

\u201cAs we\u2019ve been moving to cloud computing with data centers and software as a service,\u201d he says, \u201cenergy becomes, from a business perspective, very important, both on the data-center side and on the client side. On the data-center side, it\u2019s about monthly operating costs and power bills. On the client side, it\u2019s about battery life.\u201d<\/p>\n

And then there\u2019s a new focus on the environment.<\/p>\n

\u201cI\u2019m still interested in performance, but it\u2019s not just performance at any cost\u2014it\u2019s performance per joule,\u201d he says. \u201cEnergy efficiency is a natural extension. Data centers have interesting new workloads, I want to do research there, and it\u2019s not just pure performance. It\u2019s performance per joule and energy efficiency.\u201d<\/p>\n

In fact, the Green project is part of a bigger effort\u2014one that includes a broad swath of personnel across Microsoft\u2014to examine what would happen if the entire software and hardware stack were rebuilt with energy efficiency in mind.<\/p>\n

\u201cWhat if we could start from scratch?\u201d Chilimbi ponders. \u201cHow could we rethink all parts of it\u2014the programming language, the runtime system, the operating system, the microarchitectural design\u2014so that everything is up for grabs?<\/p>\n

\u201cAre we going to design a microprocessor from scratch? Maybe not. But what we can do is see what current microprocessors are good at and what they\u2019re bad at and add things like hardware accelerators to make them better. Rather than starting with something that\u2019s very high-performance, expensive, and inefficient, you could start with something that\u2019s very efficient but not very high-performance and see what parts of that inefficient piece are crucial for data-center workloads that would deliver high performance but still be energy-efficient?\u201d<\/p>\n

Of course, it helps when considering such scenarios to have Microsoft-scale resources behind you.<\/p>\n

\u201cBeing at Microsoft,\u201d Chilimbi says, \u201cI feel I have an advantage, because I have access to data centers and data-center workloads that a lot of people don\u2019t. By looking at that, you get access to interesting real problems.\u201d<\/p>\n

One of the real problems Chilimbi and colleagues have been examining has been search.<\/p>\n

\u201cThe search domain was what we first focused on,\u201d Chilimbi says. \u201cIt\u2019s our largest property, we care about energy efficiency, let\u2019s see what we can do.\u201d<\/p>\n

Search Expense<\/h2>\n

They identified the expensive functions in search and what they did, specifically the portions that identify all documents that match a query and then rank these documents.<\/p>\n

\u201cWhat we found,\u201d Chilimbi reports, \u201cis that, many times, you return a certain set of results, but to return those, you look at many more documents than needed. If you had an oracle, you would look at just a certain set of documents and return those. But you don\u2019t have that. You need to look at more and rank them.<\/p>\n

\u201cSo we said, \u2018We can design an algorithm that can look at fewer documents but give the same results by deciding to stop early.\u2019 When we started with that, we said, \u2018Hey, wait a second! This is generalizable. We can this for other programs, as well.\u2019 That\u2019s where the whole framework and abstraction came about. But it was really targeted at search initially. The search people told us, \u2018We have so many machines that if you improve it by this percent, it goes into millions of dollars really fast. That seemed motivating, so we decided to focus on search and how we could search fewer documents but return pretty much the same results. And then we can use those savings to improve the quality of ranking the documents that really matter.\u201d<\/p>\n

For Microsoft\u2019s search efforts, it wasn\u2019t so much a way to save money as it was a way to be efficient and use the savings to improve other parts of the search experience. And the technology is being used in Bing<\/a> search today.<\/p>\n

Chilimbi and his colleagues\u2014including, at various points, Woongki Baek, a Stanford University Ph.D. candidate; Kushagra Vaid, from the Microsoft Global Foundation Services<\/a> group; Utkarsh Jain, from Bing; and Benjamin Lee from Microsoft Research Redmond\u2019s Computer Architecture Group<\/a>\u2014aren\u2019t done yet. They hope to find a way to automate more of the approximation process, to learn how to compose quality-of-service metrics, and to identify the information software can communicate to hardware to make the two work together more efficiently.<\/p>\n

\u201cIt\u2019s a different way of looking at computing,\u201d Chilimbi says. \u201cWe\u2019ve been programmed to think of computers as these mathematical units that compute functions, whereas real-world problems are often much more fuzzy and approximate. We\u2019ve forced these real-world problems into mathematical-function form.<\/p>\n

\u201cWe want to see how far we can go with the results we have to date with more approximate models of computing. In the long run, this might make perfect sense. Green has been the first few steps, but the results are pretty good, so they justify continuing along this very interesting, novel path. Approximate computing is very interesting, especially if we can bound and guarantee a specified level of approximation, and I think we should continue investigating it. There\u2019s a lot of opportunity there.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research These days, more than ever, it\u2019s important for computing to be energy-efficient. Particularly in data centers, energy requirements represent a significant portion of operational costs, and power and cooling needs help dictate where data centers can be located, how close to capacity they can operate, and how robust […]<\/p>\n","protected":false},"author":39507,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[194488,194460],"tags":[186889,186617,214910,214907,214919,214913,214916],"research-area":[13560,13555],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-306476","post","type-post","status-publish","format-standard","hentry","category-program-languages-and-software-engineering","category-search-and-information-retrieval","tag-cloud-computing","tag-data-centers","tag-energy-utilization","tag-energy-efficient-computing","tag-programmer-defined-approximation","tag-runtime-analysis-and-design-team","tag-the-green-project","msr-research-area-programming-languages-software-engineering","msr-research-area-search-information-retrieval","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144666,144812],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"August 10, 2009","formattedExcerpt":"By Rob Knies, Managing Editor, Microsoft Research These days, more than ever, it\u2019s important for computing to be energy-efficient. Particularly in data centers, energy requirements represent a significant portion of operational costs, and power and cooling needs help dictate where data centers can be located,…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/306476"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=306476"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/306476\/revisions"}],"predecessor-version":[{"id":306779,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/306476\/revisions\/306779"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=306476"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=306476"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=306476"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=306476"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=306476"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=306476"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=306476"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=306476"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=306476"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=306476"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=306476"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}