Ankur Mallick

Performance Aware LLM Load Balancer for Mixed Workloads

Kunal Jain, A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan

EuroMLSys 2025 | April 2025

Publication Project

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

ArXiv | August 2024, Vol abs/2408.13510

Preprint

Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruehle, Laks V. S. Lakshmanan, Ahmed Awadallah

ICLR 2024 | February 2024

Publication

Matchmaker: Data Drift Mitigation in Machine Learning for Large-Scale Systems

Ankur Mallick, Kevin Hsieh, Behnaz Arzani, Gauri Joshi

MLSys | January 2022

Publication

Performance Aware LLM Load Balancer for Mixed Workloads

Kunal Jain, A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan

EuroMLSys 2025 | April 2025

Publication Project

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

ArXiv | August 2024, Vol abs/2408.13510

Preprint

Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruehle, Laks V. S. Lakshmanan, Ahmed Awadallah

ICLR 2024 | February 2024

Publication

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

ArXiv | August 2024, Vol abs/2408.13510

Preprint

Matchmaker: Data Drift Mitigation in Machine Learning for Large-Scale Systems

Ankur Mallick, Kevin Hsieh, Behnaz Arzani, Gauri Joshi

MLSys | January 2022

Publication

Performance Aware LLM Load Balancer for Mixed Workloads

Kunal Jain, A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan

EuroMLSys 2025 | April 2025

Publication Project

Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruehle, Laks V. S. Lakshmanan, Ahmed Awadallah

ICLR 2024 | February 2024

Publication

Matchmaker: Data Drift Mitigation in Machine Learning for Large-Scale Systems

Ankur Mallick, Kevin Hsieh, Behnaz Arzani, Gauri Joshi

MLSys | January 2022

Publication

Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Kunal Jain, A. Parayil, Ankur Mallick, Rujia Wang, Renee St. Amant, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Shashwat Jaiswal, Yogesh Simmhan, Anoop Kulkarni, Steve Kofsky

ArXiv | February 2025

Publication

Ensuring Fair LLM Serving Amid Diverse Applications

Kunal Jain, Ankur Mallick, A. Parayil, Renee St. Amant, Rujia Wang, Victor Ruehle, Chetan Bansal, Saravan Rajmohan, Redwan Ibne Seraj Khan, Haiying Shen, Anoop Kulkarni, Steve Kofsky, Pankhuri Choudhary, Yue Cheng

ArXiv | November 2024

Publication

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

ArXiv | August 2024, Vol abs/2408.13510

Preprint