DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, Esha Choukse
HPCA | March 2025
Publications
Publications by Year
-
-
Input-Dependent Power Usage in GPUs
Theo Gregersen, Pratyush Patel, Esha Choukse
Sustainable Supercomputing at SC24 | November 2024
Memory Allocation under Hardware Compression
Muhammad Laghari, Yuqing Liu, Gagandeep Panwar, David Bears , Chandler Jearls , Raghavendra Srinivas , Esha Choukse, Kirk Cameron, Ali R. Butt, Xun Jian
MICRO | November 2024
Mosaic: Harnessing Micro-architectural Resources of Servers in Serverless Environments
Jovan Stojkovic, Esha Choukse, Enrique Saurez, Íñigo Goiri, Josep Torrellas
MICRO | November 2024
Optimizing GPU Data Center Power
Tawfik Rahal-Arabi, Paul Van der Arend, Ashish Jain, Mehdi Saidi, Rashad Oreifej, Sriram Sundaram, Srilatha Manne, Indrani Paul, Rajit Seahra, Frank Helms, Esha Choukse, Nithish Mahalingam, Brijesh Warrier, Ricardo Bianchini
APCCAS | November 2024
Mnemosyne: Parallelization Strategies for Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximations
Amey Agrawal, Junda Chen, Íñigo Goiri, Ramachandran Ramjee, Chaojie Zhang, Alexey Tumanov, Esha Choukse
September 2024
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
Designing Cloud Servers for Lower Carbon
Jaylen Wang, Daniel S. Berger, Fiodar Kazhamiaka, Celine Irvene, Chaojie Zhang, Esha Choukse, Kali Frost, Rodrigo Fonseca, Brijesh Warrier, Chetan Bansal, Jonathan Stern, Ricardo Bianchini, Akshitha Sriraman
ISCA | June 2024
DyLeCT: Achieving Huge-page-like Translation Performance For Hardware-compressed Memory
Gagan Panwar, Muhammad Laghari, Esha Choukse, Xun Jian
ISCA | June 2024
SmartOClock: Workload- and Risk-Aware Overclocking in the Cloud
Jovan Stojkovic, Pulkit Misra, Íñigo Goiri, Sam Whitlock, Esha Choukse, Mayukh Das, Chetan Bansal, Jason Lee, Zoey Sun, Haoran Qiu, Reed Zimmermann, Savyasachi Samal, Brijesh Warrier, Ashish Raniwala, Ricardo Bianchini
ISCA | June 2024
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, Ricardo Bianchini
ISCA | June 2024
Characterizing Power Management Opportunities for LLMs in the Cloud
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ASPLOS | April 2024
Making Kernel Bypass Practical for the Cloud with Junction
Joshua Fried, Gohar Irfan Chaudhry, Enrique Saurez, Esha Choukse, Íñigo Goiri, Sameh Elnikety, Rodrigo Fonseca, Adam Belay
NSDI | April 2024
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Josep Torrellas
EMC2 at ASPLOS | April 2024
-
POLCA: Power Oversubscription in LLM Cloud Providers
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ArXiv | August 2023, Vol abs/2308.12908
Myths and Misconceptions Around Reducing Carbon Embedded in Cloud Platforms
Jialun Lyu, Jaylen Wang, Kali Frost, Chaojie Zhang, Celine Irvene, Esha Choukse, Rodrigo Fonseca, Ricardo Bianchini, Fiodar Kazhamiaka, Daniel S. Berger
2nd Workshop on Sustainable Computer Systems (HotCarbon’23) | July 2023
Towards Improved Power Management in Cloud GPUs
Pratyush Patel, Zibo Gong, S. Rizvi, Esha Choukse, Pulkit A. Misra, T. Anderson, Akshitha Sriraman, Esha Choukse, Pulkit Misra
IEEE Computer Architecture Letters | June 2023, Vol 22: pp. 141-144
Research Avenues Towards Net-Zero Cloud Platforms
Daniel S. Berger, Fiodar Kazhamiaka, Esha Choukse, Íñigo Goiri, Celine Irvene, Pulkit Misra, Alok Kumbhare, Rodrigo Fonseca, Ricardo Bianchini
NetZero 2023: 1st Workshop on NetZero Carbon Computing | February 2023
-
Translation-optimized Memory Compression for Capacity
Gagandeep Panwar, Muhammad Laghari, D. Bears, Yuqing Liu, Chandler Jearls, Esha Choukse, K. Cameron, A. Butt, Xun Jian, Esha Choukse
2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) | September 2022, pp. 992-1011
Overclocking in Immersion-Cooled Datacenters
Pulkit Misra, Ioannis Manousakis, Esha Choukse, Majid Jalili, Íñigo Goiri, Ashish Raniwala, Brijesh Warrier, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
IEEE Micro Top Picks from the 2021 Computer Architecture Conferences | July 2022
-
Buddy compression: enabling larger memory for deep learning and HPC workloads on GPUs
Esha Choukse, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Stephen W. Keckler
2020 International Symposium on Computer Architecture | May 2020
-
PruneTrain: fast neural network training by dynamic sparse model reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez
2019 IEEE International Conference on High Performance Computing, Data, and Analytics | November 2019
Publications by Research Area
-
Mnemosyne: Parallelization Strategies for Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximations
Amey Agrawal, Junda Chen, Íñigo Goiri, Ramachandran Ramjee, Chaojie Zhang, Alexey Tumanov, Esha Choukse
September 2024
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
PruneTrain: fast neural network training by dynamic sparse model reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez
2019 IEEE International Conference on High Performance Computing, Data, and Analytics | November 2019
-
DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, Esha Choukse
HPCA | March 2025
Input-Dependent Power Usage in GPUs
Theo Gregersen, Pratyush Patel, Esha Choukse
Sustainable Supercomputing at SC24 | November 2024
Optimizing GPU Data Center Power
Tawfik Rahal-Arabi, Paul Van der Arend, Ashish Jain, Mehdi Saidi, Rashad Oreifej, Sriram Sundaram, Srilatha Manne, Indrani Paul, Rajit Seahra, Frank Helms, Esha Choukse, Nithish Mahalingam, Brijesh Warrier, Ricardo Bianchini
APCCAS | November 2024
Memory Allocation under Hardware Compression
Muhammad Laghari, Yuqing Liu, Gagandeep Panwar, David Bears , Chandler Jearls , Raghavendra Srinivas , Esha Choukse, Kirk Cameron, Ali R. Butt, Xun Jian
MICRO | November 2024
Mosaic: Harnessing Micro-architectural Resources of Servers in Serverless Environments
Jovan Stojkovic, Esha Choukse, Enrique Saurez, Íñigo Goiri, Josep Torrellas
MICRO | November 2024
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
Designing Cloud Servers for Lower Carbon
Jaylen Wang, Daniel S. Berger, Fiodar Kazhamiaka, Celine Irvene, Chaojie Zhang, Esha Choukse, Kali Frost, Rodrigo Fonseca, Brijesh Warrier, Chetan Bansal, Jonathan Stern, Ricardo Bianchini, Akshitha Sriraman
ISCA | June 2024
SmartOClock: Workload- and Risk-Aware Overclocking in the Cloud
Jovan Stojkovic, Pulkit Misra, Íñigo Goiri, Sam Whitlock, Esha Choukse, Mayukh Das, Chetan Bansal, Jason Lee, Zoey Sun, Haoran Qiu, Reed Zimmermann, Savyasachi Samal, Brijesh Warrier, Ashish Raniwala, Ricardo Bianchini
ISCA | June 2024
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, Ricardo Bianchini
ISCA | June 2024
Characterizing Power Management Opportunities for LLMs in the Cloud
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ASPLOS | April 2024
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Josep Torrellas
EMC2 at ASPLOS | April 2024
Making Kernel Bypass Practical for the Cloud with Junction
Joshua Fried, Gohar Irfan Chaudhry, Enrique Saurez, Esha Choukse, Íñigo Goiri, Sameh Elnikety, Rodrigo Fonseca, Adam Belay
NSDI | April 2024
POLCA: Power Oversubscription in LLM Cloud Providers
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ArXiv | August 2023, Vol abs/2308.12908
Myths and Misconceptions Around Reducing Carbon Embedded in Cloud Platforms
Jialun Lyu, Jaylen Wang, Kali Frost, Chaojie Zhang, Celine Irvene, Esha Choukse, Rodrigo Fonseca, Ricardo Bianchini, Fiodar Kazhamiaka, Daniel S. Berger
2nd Workshop on Sustainable Computer Systems (HotCarbon’23) | July 2023
Research Avenues Towards Net-Zero Cloud Platforms
Daniel S. Berger, Fiodar Kazhamiaka, Esha Choukse, Íñigo Goiri, Celine Irvene, Pulkit Misra, Alok Kumbhare, Rodrigo Fonseca, Ricardo Bianchini
NetZero 2023: 1st Workshop on NetZero Carbon Computing | February 2023
Overclocking in Immersion-Cooled Datacenters
Pulkit Misra, Ioannis Manousakis, Esha Choukse, Majid Jalili, Íñigo Goiri, Ashish Raniwala, Brijesh Warrier, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
IEEE Micro Top Picks from the 2021 Computer Architecture Conferences | July 2022
Buddy compression: enabling larger memory for deep learning and HPC workloads on GPUs
Esha Choukse, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Stephen W. Keckler
2020 International Symposium on Computer Architecture | May 2020
PruneTrain: fast neural network training by dynamic sparse model reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez
2019 IEEE International Conference on High Performance Computing, Data, and Analytics | November 2019
-
DyLeCT: Achieving Huge-page-like Translation Performance For Hardware-compressed Memory
Gagan Panwar, Muhammad Laghari, Esha Choukse, Xun Jian
ISCA | June 2024
Towards Improved Power Management in Cloud GPUs
Pratyush Patel, Zibo Gong, S. Rizvi, Esha Choukse, Pulkit A. Misra, T. Anderson, Akshitha Sriraman, Esha Choukse, Pulkit Misra
IEEE Computer Architecture Letters | June 2023, Vol 22: pp. 141-144
Translation-optimized Memory Compression for Capacity
Gagandeep Panwar, Muhammad Laghari, D. Bears, Yuqing Liu, Chandler Jearls, Esha Choukse, K. Cameron, A. Butt, Xun Jian, Esha Choukse
2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) | September 2022, pp. 992-1011
Buddy compression: enabling larger memory for deep learning and HPC workloads on GPUs
Esha Choukse, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Stephen W. Keckler
2020 International Symposium on Computer Architecture | May 2020
PruneTrain: fast neural network training by dynamic sparse model reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez
2019 IEEE International Conference on High Performance Computing, Data, and Analytics | November 2019
-
PruneTrain: fast neural network training by dynamic sparse model reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez
2019 IEEE International Conference on High Performance Computing, Data, and Analytics | November 2019
Publications by Type
-
Mnemosyne: Parallelization Strategies for Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximations
Amey Agrawal, Junda Chen, Íñigo Goiri, Ramachandran Ramjee, Chaojie Zhang, Alexey Tumanov, Esha Choukse
September 2024
-
DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, Esha Choukse
HPCA | March 2025
Input-Dependent Power Usage in GPUs
Theo Gregersen, Pratyush Patel, Esha Choukse
Sustainable Supercomputing at SC24 | November 2024
Optimizing GPU Data Center Power
Tawfik Rahal-Arabi, Paul Van der Arend, Ashish Jain, Mehdi Saidi, Rashad Oreifej, Sriram Sundaram, Srilatha Manne, Indrani Paul, Rajit Seahra, Frank Helms, Esha Choukse, Nithish Mahalingam, Brijesh Warrier, Ricardo Bianchini
APCCAS | November 2024
Memory Allocation under Hardware Compression
Muhammad Laghari, Yuqing Liu, Gagandeep Panwar, David Bears , Chandler Jearls , Raghavendra Srinivas , Esha Choukse, Kirk Cameron, Ali R. Butt, Xun Jian
MICRO | November 2024
Mosaic: Harnessing Micro-architectural Resources of Servers in Serverless Environments
Jovan Stojkovic, Esha Choukse, Enrique Saurez, Íñigo Goiri, Josep Torrellas
MICRO | November 2024
DyLeCT: Achieving Huge-page-like Translation Performance For Hardware-compressed Memory
Gagan Panwar, Muhammad Laghari, Esha Choukse, Xun Jian
ISCA | June 2024
Designing Cloud Servers for Lower Carbon
Jaylen Wang, Daniel S. Berger, Fiodar Kazhamiaka, Celine Irvene, Chaojie Zhang, Esha Choukse, Kali Frost, Rodrigo Fonseca, Brijesh Warrier, Chetan Bansal, Jonathan Stern, Ricardo Bianchini, Akshitha Sriraman
ISCA | June 2024
SmartOClock: Workload- and Risk-Aware Overclocking in the Cloud
Jovan Stojkovic, Pulkit Misra, Íñigo Goiri, Sam Whitlock, Esha Choukse, Mayukh Das, Chetan Bansal, Jason Lee, Zoey Sun, Haoran Qiu, Reed Zimmermann, Savyasachi Samal, Brijesh Warrier, Ashish Raniwala, Ricardo Bianchini
ISCA | June 2024
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, Ricardo Bianchini
ISCA | June 2024
Characterizing Power Management Opportunities for LLMs in the Cloud
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ASPLOS | April 2024
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Josep Torrellas
EMC2 at ASPLOS | April 2024
Making Kernel Bypass Practical for the Cloud with Junction
Joshua Fried, Gohar Irfan Chaudhry, Enrique Saurez, Esha Choukse, Íñigo Goiri, Sameh Elnikety, Rodrigo Fonseca, Adam Belay
NSDI | April 2024
Myths and Misconceptions Around Reducing Carbon Embedded in Cloud Platforms
Jialun Lyu, Jaylen Wang, Kali Frost, Chaojie Zhang, Celine Irvene, Esha Choukse, Rodrigo Fonseca, Ricardo Bianchini, Fiodar Kazhamiaka, Daniel S. Berger
2nd Workshop on Sustainable Computer Systems (HotCarbon’23) | July 2023
Research Avenues Towards Net-Zero Cloud Platforms
Daniel S. Berger, Fiodar Kazhamiaka, Esha Choukse, Íñigo Goiri, Celine Irvene, Pulkit Misra, Alok Kumbhare, Rodrigo Fonseca, Ricardo Bianchini
NetZero 2023: 1st Workshop on NetZero Carbon Computing | February 2023
Buddy compression: enabling larger memory for deep learning and HPC workloads on GPUs
Esha Choukse, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Stephen W. Keckler
2020 International Symposium on Computer Architecture | May 2020
PruneTrain: fast neural network training by dynamic sparse model reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez
2019 IEEE International Conference on High Performance Computing, Data, and Analytics | November 2019
-
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling
A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky
August 2024
-
POLCA: Power Oversubscription in LLM Cloud Providers
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
ArXiv | August 2023, Vol abs/2308.12908
Towards Improved Power Management in Cloud GPUs
Pratyush Patel, Zibo Gong, S. Rizvi, Esha Choukse, Pulkit A. Misra, T. Anderson, Akshitha Sriraman, Esha Choukse, Pulkit Misra
IEEE Computer Architecture Letters | June 2023, Vol 22: pp. 141-144
Translation-optimized Memory Compression for Capacity
Gagandeep Panwar, Muhammad Laghari, D. Bears, Yuqing Liu, Chandler Jearls, Esha Choukse, K. Cameron, A. Butt, Xun Jian, Esha Choukse
2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) | September 2022, pp. 992-1011
Overclocking in Immersion-Cooled Datacenters
Pulkit Misra, Ioannis Manousakis, Esha Choukse, Majid Jalili, Íñigo Goiri, Ashish Raniwala, Brijesh Warrier, Husam Alissa, Bharath Ramakrishnan, Phillip Tuma, Christian Belady, Marcus Fontoura, Ricardo Bianchini
IEEE Micro Top Picks from the 2021 Computer Architecture Conferences | July 2022