DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, Esha Choukse
HPCA | March 2025
Íñigo Goiri
Principal Research SDE
Publications
Publications by Year
-
-
Mosaic: Harnessing Micro-architectural Resources of Servers in Serverless Environments
Jovan Stojkovic, Esha Choukse, Enrique Saurez, Íñigo Goiri, Josep Torrellas
MICRO | November 2024
Mnemosyne: Parallelization Strategies for Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximations
Amey Agrawal, Junda Chen, Íñigo Goiri, Ramachandran Ramjee, Chaojie Zhang, Alexey Tumanov, Esha Choukse
September 2024
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
SmartOClock: Workload- and Risk-Aware Overclocking in the Cloud
Jovan Stojkovic, Pulkit Misra, Íñigo Goiri, Sam Whitlock, Esha Choukse, Mayukh Das, Chetan Bansal, Jason Lee, Zoey Sun, Haoran Qiu, Reed Zimmermann, Savyasachi Samal, Brijesh Warrier, Ashish Raniwala, Ricardo Bianchini
ISCA | June 2024
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, Ricardo Bianchini
ISCA | June 2024
Towards Cloud Efficiency with Large-scale Workload Characterization
A. Parayil, Jue Zhang, Xiaoting Qin, Íñigo Goiri, Chetan Bansal
Published by Arxiv | May 2024, Vol abs/2405.07250
Characterizing Power Management Opportunities for LLMs in the Cloud
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ASPLOS | April 2024
Making Kernel Bypass Practical for the Cloud with Junction
Joshua Fried, Gohar Irfan Chaudhry, Enrique Saurez, Esha Choukse, Íñigo Goiri, Sameh Elnikety, Rodrigo Fonseca, Adam Belay
NSDI | April 2024
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Josep Torrellas
EMC2 at ASPLOS | April 2024
-
POLCA: Power Oversubscription in LLM Cloud Providers
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ArXiv | August 2023, Vol abs/2308.12908
How Different are the Cloud Workloads? Characterizing Large-Scale Private and Public Cloud Workloads
Xiaoting Qin, Minghua Ma, Yueng Zhao, Jue Zhang, Chao Du, Yudong Liu, A. Parayil, Chetan Bansal, Saravan Rajmohan, Íñigo Goiri, Eli Cortez, Si Qin, Qingwei Lin 林庆维, Dongmei Zhang
DSN | June 2023
Palette Load Balancing: Locality Hints for Serverless Functions
Mania Abdi, Sam Ginzburg, Charles Lin, Jose M Faleiro, Íñigo Goiri, Gohar Irfan Chaudhry, Ricardo Bianchini, Daniel S. Berger, Rodrigo Fonseca
EuroSys | May 2023
Unlocking unallocated cloud capacity for long, uninterruptible workloads
Anup Agarwal, Shadi Noghabi, Íñigo Goiri, Srinivasan Seshan, Anirudh Badam
NSDI | April 2023
Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs
Fangkai Yang, Lu Wang, Zhenyu Xu, Jue Zhang, Liqun Li, Bo Qiao, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Terry Yang, Victor Ruehle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2023
Research Avenues Towards Net-Zero Cloud Platforms
Daniel S. Berger, Fiodar Kazhamiaka, Esha Choukse, Íñigo Goiri, Celine Irvene, Pulkit Misra, Alok Kumbhare, Rodrigo Fonseca, Ricardo Bianchini
NetZero 2023: 1st Workshop on NetZero Carbon Computing | February 2023
-
Overclocking in Immersion-Cooled Datacenters
Pulkit Misra, Ioannis Manousakis, Esha Choukse, Majid Jalili, Íñigo Goiri, Ashish Raniwala, Brijesh Warrier, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
IEEE Micro Top Picks from the 2021 Computer Architecture Conferences | July 2022
Spot Virtual Machine Eviction Prediction in Microsoft Cloud
Fangkai Yang, Bowen Pang, Jue Zhang, Bo Qiao, Lu Wang, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Senthil Baladhandayutham, Victor Rühle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
TheWebConf 2022 | April 2022
Memory-Harvesting VMs in Cloud Platforms
Alex Fuerst, Stanko Novakovic, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, Ricardo Bianchini
Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2022
-
Faa$T: A Transparent Auto-Scaling Cache for Serverless Applications
Francisco Romero, Gohar Irfan Chaudhry, Íñigo Goiri, Pragna Gopa, Paul Batum, Neeraja J. Yadwadkar, Rodrigo Fonseca, Christos Kozyrakis, Ricardo Bianchini
Proceedings of the Symposium on Cloud Computing (SoCC) | November 2021
Faster and Cheaper Serverless Computing on Harvested Resources
Yanqi Zhang, Íñigo Goiri, Gohar Irfan Chaudhry, Rodrigo Fonseca, Sameh Elnikety, Christina Delimitrou, Ricardo Bianchini
Proceedings of the International Symposium on Operating Systems Principles (SOSP) | October 2021
Cost-Efficient Overclocking in Immersion-Cooled Datacenters
Majid Jalili, Ioannis Manousakis, Íñigo Goiri, Pulkit Misra, Ashish Raniwala, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
Proceedings of the International Symposium on Computer Architecture (ISCA) | June 2021
-
Providing SLOs for Resource-Harvesting VMs in Cloud Platforms
Pradeep Ambati, Íñigo Goiri, Felipe Vieira Frujeri, Alper Gun, Ke Wang, Brian Dolan, Brian Corell, Sekhar Pasupuleti, Thomas Moscibroda, Sameh Elnikety, Marcus Fontoura, Ricardo Bianchini
Proceedings of the Symposium on Operating Systems Design and Implementation (OSDI) | November 2020
Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider
Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri, Gohar Irfan Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, Ricardo Bianchini
Proceedings of the USENIX Annual Technical Conference (ATC) | July 2020
Community Award
-
Managing Tail Latency in Datacenter-Scale File Systems Under Production Constraints
Pulkit Misra, María F. Borge, Íñigo Goiri, Alvin R. Lebeck, Willy Zwaenepoel, Ricardo Bianchini
Proceedings of the 14th European Conference on Computer Systems (EuroSys) | March 2019
-
Uncertainty Propagation in Data Processing Systems
Ioannis Manousakis, Íñigo Goiri, Ricardo Bianchini, Sandro Rigo, Thu D. Nguyen
Proceedings of the Symposium on Cloud Computing (SoCC) | November 2018
-
Scaling Distributed File Systems in Resource-Harvesting Datacenters
Pulkit Misra, Íñigo Goiri, Jason Kace, Ricardo Bianchini
Proceedings of USENIX Annual Technical Conference (ATC) | July 2017
-
History-Based Harvesting Spare Cycles and Storage in Large-Scale Datacenters
Yunqi Zhang, George Prekas, Giovanni Matteo Fumarola, Marcus Fontoura, Íñigo Goiri, Ricardo Bianchini
Proceedings of the International Symposium on Operating Systems Design and Implementation (OSDI) | November 2016
Morpheus: Towards Automated SLOs for Enterprise Clusters
Sangeetha Abdu Jyothi, Carlo Curino, Ishai Menache, Shravan Matthur Narayanamurthy, Alexey Tumanov, Jonathan Yaniv, Ruslan Mavlyutov, Íñigo Goiri, Subru Krishnan, Janardhan (Jana) Kulkarni, Sriram Rao
2016 International Symposium on Operating Systems Design and Implementation (OSDI) | November 2016
-
CoolProvision: Underprovisioning Datacenter Cooling
Ioannis Manousakis, Íñigo Goiri, Sriram Sankar, Thu D. Nguyen, Ricardo Bianchini
Proceedings of the International Symposium on Cloud Computing (SoCC) | August 2015
ApproxHadoop: Bringing Approximations to MapReduce Frameworks
Íñigo Goiri, Ricardo Bianchini, Santosh Nagarakatte, Thu D. Nguyen
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2015
CoolAir: Temperature- and Variation-Aware Management for Free-Cooled Datacenters
Íñigo Goiri, Thu D. Nguyen, Ricardo Bianchini
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2015
-
Parasol and GreenSwitch: Managing Datacenters Powered by Renewable Energy
Íñigo Goiri, William Katsak, Kien Le, Thud D. Nguyen, Ricardo Bianchini
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2013
Publications by Research Area
-
Mnemosyne: Parallelization Strategies for Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximations
Amey Agrawal, Junda Chen, Íñigo Goiri, Ramachandran Ramjee, Chaojie Zhang, Alexey Tumanov, Esha Choukse
September 2024
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
Towards Cloud Efficiency with Large-scale Workload Characterization
A. Parayil, Jue Zhang, Xiaoting Qin, Íñigo Goiri, Chetan Bansal
Published by Arxiv | May 2024, Vol abs/2405.07250
Spot Virtual Machine Eviction Prediction in Microsoft Cloud
Fangkai Yang, Bowen Pang, Jue Zhang, Bo Qiao, Lu Wang, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Senthil Baladhandayutham, Victor Rühle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
TheWebConf 2022 | April 2022
-
DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, Esha Choukse
HPCA | March 2025
Mosaic: Harnessing Micro-architectural Resources of Servers in Serverless Environments
Jovan Stojkovic, Esha Choukse, Enrique Saurez, Íñigo Goiri, Josep Torrellas
MICRO | November 2024
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
SmartOClock: Workload- and Risk-Aware Overclocking in the Cloud
Jovan Stojkovic, Pulkit Misra, Íñigo Goiri, Sam Whitlock, Esha Choukse, Mayukh Das, Chetan Bansal, Jason Lee, Zoey Sun, Haoran Qiu, Reed Zimmermann, Savyasachi Samal, Brijesh Warrier, Ashish Raniwala, Ricardo Bianchini
ISCA | June 2024
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, Ricardo Bianchini
ISCA | June 2024
Towards Cloud Efficiency with Large-scale Workload Characterization
A. Parayil, Jue Zhang, Xiaoting Qin, Íñigo Goiri, Chetan Bansal
Published by Arxiv | May 2024, Vol abs/2405.07250
Characterizing Power Management Opportunities for LLMs in the Cloud
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ASPLOS | April 2024
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Josep Torrellas
EMC2 at ASPLOS | April 2024
Making Kernel Bypass Practical for the Cloud with Junction
Joshua Fried, Gohar Irfan Chaudhry, Enrique Saurez, Esha Choukse, Íñigo Goiri, Sameh Elnikety, Rodrigo Fonseca, Adam Belay
NSDI | April 2024
POLCA: Power Oversubscription in LLM Cloud Providers
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ArXiv | August 2023, Vol abs/2308.12908
How Different are the Cloud Workloads? Characterizing Large-Scale Private and Public Cloud Workloads
Xiaoting Qin, Minghua Ma, Yueng Zhao, Jue Zhang, Chao Du, Yudong Liu, A. Parayil, Chetan Bansal, Saravan Rajmohan, Íñigo Goiri, Eli Cortez, Si Qin, Qingwei Lin 林庆维, Dongmei Zhang
DSN | June 2023
Palette Load Balancing: Locality Hints for Serverless Functions
Mania Abdi, Sam Ginzburg, Charles Lin, Jose M Faleiro, Íñigo Goiri, Gohar Irfan Chaudhry, Ricardo Bianchini, Daniel S. Berger, Rodrigo Fonseca
EuroSys | May 2023
Unlocking unallocated cloud capacity for long, uninterruptible workloads
Anup Agarwal, Shadi Noghabi, Íñigo Goiri, Srinivasan Seshan, Anirudh Badam
NSDI | April 2023
Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs
Fangkai Yang, Lu Wang, Zhenyu Xu, Jue Zhang, Liqun Li, Bo Qiao, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Terry Yang, Victor Ruehle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2023
Research Avenues Towards Net-Zero Cloud Platforms
Daniel S. Berger, Fiodar Kazhamiaka, Esha Choukse, Íñigo Goiri, Celine Irvene, Pulkit Misra, Alok Kumbhare, Rodrigo Fonseca, Ricardo Bianchini
NetZero 2023: 1st Workshop on NetZero Carbon Computing | February 2023
Overclocking in Immersion-Cooled Datacenters
Pulkit Misra, Ioannis Manousakis, Esha Choukse, Majid Jalili, Íñigo Goiri, Ashish Raniwala, Brijesh Warrier, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
IEEE Micro Top Picks from the 2021 Computer Architecture Conferences | July 2022
Spot Virtual Machine Eviction Prediction in Microsoft Cloud
Fangkai Yang, Bowen Pang, Jue Zhang, Bo Qiao, Lu Wang, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Senthil Baladhandayutham, Victor Rühle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
TheWebConf 2022 | April 2022
Memory-Harvesting VMs in Cloud Platforms
Alex Fuerst, Stanko Novakovic, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, Ricardo Bianchini
Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2022
Faa$T: A Transparent Auto-Scaling Cache for Serverless Applications
Francisco Romero, Gohar Irfan Chaudhry, Íñigo Goiri, Pragna Gopa, Paul Batum, Neeraja J. Yadwadkar, Rodrigo Fonseca, Christos Kozyrakis, Ricardo Bianchini
Proceedings of the Symposium on Cloud Computing (SoCC) | November 2021
Faster and Cheaper Serverless Computing on Harvested Resources
Yanqi Zhang, Íñigo Goiri, Gohar Irfan Chaudhry, Rodrigo Fonseca, Sameh Elnikety, Christina Delimitrou, Ricardo Bianchini
Proceedings of the International Symposium on Operating Systems Principles (SOSP) | October 2021
Cost-Efficient Overclocking in Immersion-Cooled Datacenters
Majid Jalili, Ioannis Manousakis, Íñigo Goiri, Pulkit Misra, Ashish Raniwala, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
Proceedings of the International Symposium on Computer Architecture (ISCA) | June 2021
Providing SLOs for Resource-Harvesting VMs in Cloud Platforms
Pradeep Ambati, Íñigo Goiri, Felipe Vieira Frujeri, Alper Gun, Ke Wang, Brian Dolan, Brian Corell, Sekhar Pasupuleti, Thomas Moscibroda, Sameh Elnikety, Marcus Fontoura, Ricardo Bianchini
Proceedings of the Symposium on Operating Systems Design and Implementation (OSDI) | November 2020
Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider
Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri, Gohar Irfan Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, Ricardo Bianchini
Proceedings of the USENIX Annual Technical Conference (ATC) | July 2020
Community Award
Managing Tail Latency in Datacenter-Scale File Systems Under Production Constraints
Pulkit Misra, María F. Borge, Íñigo Goiri, Alvin R. Lebeck, Willy Zwaenepoel, Ricardo Bianchini
Proceedings of the 14th European Conference on Computer Systems (EuroSys) | March 2019
Uncertainty Propagation in Data Processing Systems
Ioannis Manousakis, Íñigo Goiri, Ricardo Bianchini, Sandro Rigo, Thu D. Nguyen
Proceedings of the Symposium on Cloud Computing (SoCC) | November 2018
Scaling Distributed File Systems in Resource-Harvesting Datacenters
Pulkit Misra, Íñigo Goiri, Jason Kace, Ricardo Bianchini
Proceedings of USENIX Annual Technical Conference (ATC) | July 2017
Morpheus: Towards Automated SLOs for Enterprise Clusters
Sangeetha Abdu Jyothi, Carlo Curino, Ishai Menache, Shravan Matthur Narayanamurthy, Alexey Tumanov, Jonathan Yaniv, Ruslan Mavlyutov, Íñigo Goiri, Subru Krishnan, Janardhan (Jana) Kulkarni, Sriram Rao
2016 International Symposium on Operating Systems Design and Implementation (OSDI) | November 2016
History-Based Harvesting Spare Cycles and Storage in Large-Scale Datacenters
Yunqi Zhang, George Prekas, Giovanni Matteo Fumarola, Marcus Fontoura, Íñigo Goiri, Ricardo Bianchini
Proceedings of the International Symposium on Operating Systems Design and Implementation (OSDI) | November 2016
CoolProvision: Underprovisioning Datacenter Cooling
Ioannis Manousakis, Íñigo Goiri, Sriram Sankar, Thu D. Nguyen, Ricardo Bianchini
Proceedings of the International Symposium on Cloud Computing (SoCC) | August 2015
CoolAir: Temperature- and Variation-Aware Management for Free-Cooled Datacenters
Íñigo Goiri, Thu D. Nguyen, Ricardo Bianchini
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2015
ApproxHadoop: Bringing Approximations to MapReduce Frameworks
Íñigo Goiri, Ricardo Bianchini, Santosh Nagarakatte, Thu D. Nguyen
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2015
Parasol and GreenSwitch: Managing Datacenters Powered by Renewable Energy
Íñigo Goiri, William Katsak, Kien Le, Thud D. Nguyen, Ricardo Bianchini
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2013
-
Spot Virtual Machine Eviction Prediction in Microsoft Cloud
Fangkai Yang, Bowen Pang, Jue Zhang, Bo Qiao, Lu Wang, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Senthil Baladhandayutham, Victor Rühle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
TheWebConf 2022 | April 2022
-
Spot Virtual Machine Eviction Prediction in Microsoft Cloud
Fangkai Yang, Bowen Pang, Jue Zhang, Bo Qiao, Lu Wang, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Senthil Baladhandayutham, Victor Rühle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
TheWebConf 2022 | April 2022
-
Spot Virtual Machine Eviction Prediction in Microsoft Cloud
Fangkai Yang, Bowen Pang, Jue Zhang, Bo Qiao, Lu Wang, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Senthil Baladhandayutham, Victor Rühle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
TheWebConf 2022 | April 2022
Publications by Type
-
Mnemosyne: Parallelization Strategies for Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximations
Amey Agrawal, Junda Chen, Íñigo Goiri, Ramachandran Ramjee, Chaojie Zhang, Alexey Tumanov, Esha Choukse
September 2024
-
DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, Esha Choukse
HPCA | March 2025
Mosaic: Harnessing Micro-architectural Resources of Servers in Serverless Environments
Jovan Stojkovic, Esha Choukse, Enrique Saurez, Íñigo Goiri, Josep Torrellas
MICRO | November 2024
SmartOClock: Workload- and Risk-Aware Overclocking in the Cloud
Jovan Stojkovic, Pulkit Misra, Íñigo Goiri, Sam Whitlock, Esha Choukse, Mayukh Das, Chetan Bansal, Jason Lee, Zoey Sun, Haoran Qiu, Reed Zimmermann, Savyasachi Samal, Brijesh Warrier, Ashish Raniwala, Ricardo Bianchini
ISCA | June 2024
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, Ricardo Bianchini
ISCA | June 2024
Characterizing Power Management Opportunities for LLMs in the Cloud
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ASPLOS | April 2024
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Josep Torrellas
EMC2 at ASPLOS | April 2024
Making Kernel Bypass Practical for the Cloud with Junction
Joshua Fried, Gohar Irfan Chaudhry, Enrique Saurez, Esha Choukse, Íñigo Goiri, Sameh Elnikety, Rodrigo Fonseca, Adam Belay
NSDI | April 2024
How Different are the Cloud Workloads? Characterizing Large-Scale Private and Public Cloud Workloads
Xiaoting Qin, Minghua Ma, Yueng Zhao, Jue Zhang, Chao Du, Yudong Liu, A. Parayil, Chetan Bansal, Saravan Rajmohan, Íñigo Goiri, Eli Cortez, Si Qin, Qingwei Lin 林庆维, Dongmei Zhang
DSN | June 2023
Palette Load Balancing: Locality Hints for Serverless Functions
Mania Abdi, Sam Ginzburg, Charles Lin, Jose M Faleiro, Íñigo Goiri, Gohar Irfan Chaudhry, Ricardo Bianchini, Daniel S. Berger, Rodrigo Fonseca
EuroSys | May 2023
Unlocking unallocated cloud capacity for long, uninterruptible workloads
Anup Agarwal, Shadi Noghabi, Íñigo Goiri, Srinivasan Seshan, Anirudh Badam
NSDI | April 2023
Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs
Fangkai Yang, Lu Wang, Zhenyu Xu, Jue Zhang, Liqun Li, Bo Qiao, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Terry Yang, Victor Ruehle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2023
Research Avenues Towards Net-Zero Cloud Platforms
Daniel S. Berger, Fiodar Kazhamiaka, Esha Choukse, Íñigo Goiri, Celine Irvene, Pulkit Misra, Alok Kumbhare, Rodrigo Fonseca, Ricardo Bianchini
NetZero 2023: 1st Workshop on NetZero Carbon Computing | February 2023
Spot Virtual Machine Eviction Prediction in Microsoft Cloud
Fangkai Yang, Bowen Pang, Jue Zhang, Bo Qiao, Lu Wang, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Senthil Baladhandayutham, Victor Rühle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang
TheWebConf 2022 | April 2022
Memory-Harvesting VMs in Cloud Platforms
Alex Fuerst, Stanko Novakovic, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, Ricardo Bianchini
Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2022
Faa$T: A Transparent Auto-Scaling Cache for Serverless Applications
Francisco Romero, Gohar Irfan Chaudhry, Íñigo Goiri, Pragna Gopa, Paul Batum, Neeraja J. Yadwadkar, Rodrigo Fonseca, Christos Kozyrakis, Ricardo Bianchini
Proceedings of the Symposium on Cloud Computing (SoCC) | November 2021
Faster and Cheaper Serverless Computing on Harvested Resources
Yanqi Zhang, Íñigo Goiri, Gohar Irfan Chaudhry, Rodrigo Fonseca, Sameh Elnikety, Christina Delimitrou, Ricardo Bianchini
Proceedings of the International Symposium on Operating Systems Principles (SOSP) | October 2021
Cost-Efficient Overclocking in Immersion-Cooled Datacenters
Majid Jalili, Ioannis Manousakis, Íñigo Goiri, Pulkit Misra, Ashish Raniwala, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
Proceedings of the International Symposium on Computer Architecture (ISCA) | June 2021
Providing SLOs for Resource-Harvesting VMs in Cloud Platforms
Pradeep Ambati, Íñigo Goiri, Felipe Vieira Frujeri, Alper Gun, Ke Wang, Brian Dolan, Brian Corell, Sekhar Pasupuleti, Thomas Moscibroda, Sameh Elnikety, Marcus Fontoura, Ricardo Bianchini
Proceedings of the Symposium on Operating Systems Design and Implementation (OSDI) | November 2020
Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider
Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri, Gohar Irfan Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, Ricardo Bianchini
Proceedings of the USENIX Annual Technical Conference (ATC) | July 2020
Community Award
Managing Tail Latency in Datacenter-Scale File Systems Under Production Constraints
Pulkit Misra, María F. Borge, Íñigo Goiri, Alvin R. Lebeck, Willy Zwaenepoel, Ricardo Bianchini
Proceedings of the 14th European Conference on Computer Systems (EuroSys) | March 2019
Uncertainty Propagation in Data Processing Systems
Ioannis Manousakis, Íñigo Goiri, Ricardo Bianchini, Sandro Rigo, Thu D. Nguyen
Proceedings of the Symposium on Cloud Computing (SoCC) | November 2018
Scaling Distributed File Systems in Resource-Harvesting Datacenters
Pulkit Misra, Íñigo Goiri, Jason Kace, Ricardo Bianchini
Proceedings of USENIX Annual Technical Conference (ATC) | July 2017
Morpheus: Towards Automated SLOs for Enterprise Clusters
Sangeetha Abdu Jyothi, Carlo Curino, Ishai Menache, Shravan Matthur Narayanamurthy, Alexey Tumanov, Jonathan Yaniv, Ruslan Mavlyutov, Íñigo Goiri, Subru Krishnan, Janardhan (Jana) Kulkarni, Sriram Rao
2016 International Symposium on Operating Systems Design and Implementation (OSDI) | November 2016
History-Based Harvesting Spare Cycles and Storage in Large-Scale Datacenters
Yunqi Zhang, George Prekas, Giovanni Matteo Fumarola, Marcus Fontoura, Íñigo Goiri, Ricardo Bianchini
Proceedings of the International Symposium on Operating Systems Design and Implementation (OSDI) | November 2016
CoolProvision: Underprovisioning Datacenter Cooling
Ioannis Manousakis, Íñigo Goiri, Sriram Sankar, Thu D. Nguyen, Ricardo Bianchini
Proceedings of the International Symposium on Cloud Computing (SoCC) | August 2015
CoolAir: Temperature- and Variation-Aware Management for Free-Cooled Datacenters
Íñigo Goiri, Thu D. Nguyen, Ricardo Bianchini
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2015
ApproxHadoop: Bringing Approximations to MapReduce Frameworks
Íñigo Goiri, Ricardo Bianchini, Santosh Nagarakatte, Thu D. Nguyen
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2015
Parasol and GreenSwitch: Managing Datacenters Powered by Renewable Energy
Íñigo Goiri, William Katsak, Kien Le, Thud D. Nguyen, Ricardo Bianchini
Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2013
-
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
Towards Cloud Efficiency with Large-scale Workload Characterization
A. Parayil, Jue Zhang, Xiaoting Qin, Íñigo Goiri, Chetan Bansal
Published by Arxiv | May 2024, Vol abs/2405.07250
-
POLCA: Power Oversubscription in LLM Cloud Providers
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ArXiv | August 2023, Vol abs/2308.12908
Overclocking in Immersion-Cooled Datacenters
Pulkit Misra, Ioannis Manousakis, Esha Choukse, Majid Jalili, Íñigo Goiri, Ashish Raniwala, Brijesh Warrier, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
IEEE Micro Top Picks from the 2021 Computer Architecture Conferences | July 2022