AI Infrastructure

Towards efficient AI/ML deployment

Publications

View by:

- SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference
  
  Anish Biswas, Kanishk Goel, Srivarshinee S, Jayashree Mohan, Alind Khare, A. Parayil, Ramachandran Ramjee, Chetan Bansal
  
  2027 Architectural Support for Programming Languages and Operating Systems | April 2027
  
  PDF
- LLM-42: Enabling Determinism in LLM Inference with Verified Speculation
  
  Raja Gond, Aditya K Kamath, Ramachandran Ramjee, Ashish Panwar
  
  SOSP 2026 | September 2026
  
  Preprint
- GraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads
  
  Abhishek Ghosh, Ajay Nayak, Ashish Panwar, Arkaprava Basu
  
  Operating Systems Design and Implementation | July 2026
  
  Preprint
- QoServe : Breaking the Silos of LLM Inference Serving
  
  Kanishk Goel, Jayashree Mohan, Nipun Kwatra, Ravi Shreyas Anupindi, Ramachandran Ramjee
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2026 | March 2026
  
  Publication
- ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  ACM Symposium on Cloud Computing (SoCC) 2025 | November 2025
  
  Preprint Project
- POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
  
  Aditya K Kamath, Ramya Prabhu, Jayashree Mohan, Simon Peter, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  Publication
- vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
  
  Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  PDF Preprint Github
- ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  February 2025
  
  Arxiv | February 2025
  
  Publication Project
- Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
  
  Amey Agrawal, Anmol Agarwal, Nitin Kedia, Jayashree Mohan, Souvik Kundu, Nipun Kwatra, Ramachandran Ramjee, Alexey Tumanov
  
  July 2024
  
  Preprint
- VIDUR: A Large-Scale Simulation Framework for LLM Inference
  
  Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
  
  MLSys | May 2024
  
  PDF Github
- Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
  
  Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
  
  European Conference on Computer Systems | April 2024
  
  DOI Publication
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
  
  Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, Ramachandran Ramjee
  
  OSDI ’24 | March 2024
  
  DOI Publication Publication Github
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
  
  Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
  
  September 2023
  
  Preprint
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication
- Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
  
  Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
  
  EuroSys 2022 | April 2022
  
  Best Paper Award
  
  Publication PDF
- Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
  
  Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
  
  February 2022
  
  arXiv preprint | February 2022
  
  Publication
- Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
  
  Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
  
  Fifteenth European Conference on Computer Systems (EuroSys’20) | April 2020
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication Project
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication Project Project
- Gandiva: Introspective Cluster Scheduling for Deep Learning
  
  Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
  
  13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
  
  Publication Project Project

- SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference
  
  Anish Biswas, Kanishk Goel, Srivarshinee S, Jayashree Mohan, Alind Khare, A. Parayil, Ramachandran Ramjee, Chetan Bansal
  
  2027 Architectural Support for Programming Languages and Operating Systems | April 2027
  
  PDF
- LLM-42: Enabling Determinism in LLM Inference with Verified Speculation
  
  Raja Gond, Aditya K Kamath, Ramachandran Ramjee, Ashish Panwar
  
  SOSP 2026 | September 2026
  
  Preprint
- GraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads
  
  Abhishek Ghosh, Ajay Nayak, Ashish Panwar, Arkaprava Basu
  
  Operating Systems Design and Implementation | July 2026
  
  Preprint
- QoServe : Breaking the Silos of LLM Inference Serving
  
  Kanishk Goel, Jayashree Mohan, Nipun Kwatra, Ravi Shreyas Anupindi, Ramachandran Ramjee
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2026 | March 2026
  
  Publication
- ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  ACM Symposium on Cloud Computing (SoCC) 2025 | November 2025
  
  Preprint Project
- POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
  
  Aditya K Kamath, Ramya Prabhu, Jayashree Mohan, Simon Peter, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  Publication
- vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
  
  Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  PDF Preprint Github
- Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
  
  Amey Agrawal, Anmol Agarwal, Nitin Kedia, Jayashree Mohan, Souvik Kundu, Nipun Kwatra, Ramachandran Ramjee, Alexey Tumanov
  
  July 2024
  
  Preprint
- VIDUR: A Large-Scale Simulation Framework for LLM Inference
  
  Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
  
  MLSys | May 2024
  
  PDF Github
- Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
  
  Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
  
  European Conference on Computer Systems | April 2024
  
  DOI Publication
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
  
  Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, Ramachandran Ramjee
  
  OSDI ’24 | March 2024
  
  DOI Publication Publication Github
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
  
  Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
  
  September 2023
  
  Preprint
- Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
  
  Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
  
  EuroSys 2022 | April 2022
  
  Best Paper Award
  
  Publication PDF
- Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
  
  Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
  
  February 2022
  
  arXiv preprint | February 2022
  
  Publication
- Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
  
  Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
  
  Fifteenth European Conference on Computer Systems (EuroSys’20) | April 2020
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication Project
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication Project Project
- Gandiva: Introspective Cluster Scheduling for Deep Learning
  
  Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
  
  13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
  
  Publication Project Project
- SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference
  
  Anish Biswas, Kanishk Goel, Srivarshinee S, Jayashree Mohan, Alind Khare, A. Parayil, Ramachandran Ramjee, Chetan Bansal
  
  2027 Architectural Support for Programming Languages and Operating Systems | April 2027
  
  PDF
- ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  ACM Symposium on Cloud Computing (SoCC) 2025 | November 2025
  
  Preprint Project
- ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  February 2025
  
  Arxiv | February 2025
  
  Publication Project
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication Project
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication Project Project
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication

- SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference
  
  Anish Biswas, Kanishk Goel, Srivarshinee S, Jayashree Mohan, Alind Khare, A. Parayil, Ramachandran Ramjee, Chetan Bansal
  
  2027 Architectural Support for Programming Languages and Operating Systems | April 2027
  
  PDF
- LLM-42: Enabling Determinism in LLM Inference with Verified Speculation
  
  Raja Gond, Aditya K Kamath, Ramachandran Ramjee, Ashish Panwar
  
  SOSP 2026 | September 2026
  
  Preprint
- GraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads
  
  Abhishek Ghosh, Ajay Nayak, Ashish Panwar, Arkaprava Basu
  
  Operating Systems Design and Implementation | July 2026
  
  Preprint
- QoServe : Breaking the Silos of LLM Inference Serving
  
  Kanishk Goel, Jayashree Mohan, Nipun Kwatra, Ravi Shreyas Anupindi, Ramachandran Ramjee
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2026 | March 2026
  
  Publication
- ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  ACM Symposium on Cloud Computing (SoCC) 2025 | November 2025
  
  Preprint Project
- POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
  
  Aditya K Kamath, Ramya Prabhu, Jayashree Mohan, Simon Peter, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  Publication
- vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
  
  Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  PDF Preprint Github
- Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
  
  Amey Agrawal, Anmol Agarwal, Nitin Kedia, Jayashree Mohan, Souvik Kundu, Nipun Kwatra, Ramachandran Ramjee, Alexey Tumanov
  
  July 2024
  
  Preprint
- VIDUR: A Large-Scale Simulation Framework for LLM Inference
  
  Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
  
  MLSys | May 2024
  
  PDF Github
- Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
  
  Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
  
  European Conference on Computer Systems | April 2024
  
  DOI Publication
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
  
  Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, Ramachandran Ramjee
  
  OSDI ’24 | March 2024
  
  DOI Publication Publication Github
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication
- Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
  
  Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
  
  EuroSys 2022 | April 2022
  
  Best Paper Award
  
  Publication PDF
- Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
  
  Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
  
  Fifteenth European Conference on Computer Systems (EuroSys’20) | April 2020
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication Project
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication Project Project
- Gandiva: Introspective Cluster Scheduling for Deep Learning
  
  Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
  
  13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
  
  Publication Project Project
- ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  February 2025
  
  Arxiv | February 2025
  
  Publication Project
- Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
  
  Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
  
  February 2022
  
  arXiv preprint | February 2022
  
  Publication
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
  
  Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
  
  September 2023
  
  Preprint

AI Infrastructure

Publications

SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference

LLM-42: Enabling Determinism in LLM Inference with Verified Speculation

GraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads

QoServe : Breaking the Silos of LLM Inference Serving

ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving

POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference

vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention

ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving

Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems

VIDUR: A Large-Scale Simulation Framework for LLM Inference

Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills

Renee: End-to-end training of extreme classification models

Varuna: Scalable, Low-cost Training of Massive Deep Learning Models

Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads

Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning

Quiver: An Informed Storage Cache for Deep Learning

Astra: Exploiting Predictability to Optimize Deep Learning

Gandiva: Introspective Cluster Scheduling for Deep Learning

SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference

LLM-42: Enabling Determinism in LLM Inference with Verified Speculation

GraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads

QoServe : Breaking the Silos of LLM Inference Serving

ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving

POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference

vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention

Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems

VIDUR: A Large-Scale Simulation Framework for LLM Inference

Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills

Varuna: Scalable, Low-cost Training of Massive Deep Learning Models

Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads

Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning

Quiver: An Informed Storage Cache for Deep Learning

Astra: Exploiting Predictability to Optimize Deep Learning

Gandiva: Introspective Cluster Scheduling for Deep Learning

SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference

ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving

ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving

Renee: End-to-end training of extreme classification models

Quiver: An Informed Storage Cache for Deep Learning

Astra: Exploiting Predictability to Optimize Deep Learning

Renee: End-to-end training of extreme classification models

SUTRADHARA : An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference

LLM-42: Enabling Determinism in LLM Inference with Verified Speculation

GraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads

QoServe : Breaking the Silos of LLM Inference Serving

ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving

POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference

vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention

Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems

VIDUR: A Large-Scale Simulation Framework for LLM Inference

Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

Renee: End-to-end training of extreme classification models

Varuna: Scalable, Low-cost Training of Massive Deep Learning Models

Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning

Quiver: An Informed Storage Cache for Deep Learning

Astra: Exploiting Predictability to Optimize Deep Learning

Gandiva: Introspective Cluster Scheduling for Deep Learning

ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving

Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads

SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills