Victor Ruehle

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

August 2024

Preprint

Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers

Rya Sanovar, Srikant Bharadwaj, Renee St. Amant, Victor Ruehle, Saravan Rajmohan

May 2024

Publication

LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression

Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang

The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024

Preprint Project

Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruehle, Laks V. S. Lakshmanan, Ahmed Awadallah

ICLR 2024 | February 2024

Publication

Hybrid Retrieval-Augmented Generation for Real-time Composition Assistance

Menglin Xia, Xuchao Zhang, Camille Couturier, Guoqing Zheng, Saravan Rajmohan, Victor Ruehle

February 2024

Preprint

Bayesian Estimation of Differential Privacy

Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Ahmed Salem, Victor Ruehle, Andrew Paverd, Mohammad Naseri, Boris Köpf, Daniel Jones

2023 International Conference on Machine Learning | July 2023

Editor(s): Barbara Engelhardt, Emma Brunskill, Kyunghyun Cho

Publication Project

Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs

Fangkai Yang, Lu Wang, Zhenyu Xu, Jue Zhang, Liqun Li, Bo Qiao, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Terry Yang, Victor Ruehle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang

Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2023

Publication

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

August 2024

Preprint

Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers

Rya Sanovar, Srikant Bharadwaj, Renee St. Amant, Victor Ruehle, Saravan Rajmohan

May 2024

Publication

LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression

Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang

The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024

Preprint Project

Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruehle, Laks V. S. Lakshmanan, Ahmed Awadallah

ICLR 2024 | February 2024

Publication

Hybrid Retrieval-Augmented Generation for Real-time Composition Assistance

Menglin Xia, Xuchao Zhang, Camille Couturier, Guoqing Zheng, Saravan Rajmohan, Victor Ruehle

February 2024

Preprint

Bayesian Estimation of Differential Privacy

Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Ahmed Salem, Victor Ruehle, Andrew Paverd, Mohammad Naseri, Boris Köpf, Daniel Jones

2023 International Conference on Machine Learning | July 2023

Editor(s): Barbara Engelhardt, Emma Brunskill, Kyunghyun Cho

Publication Project

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

August 2024

Preprint

Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers

Rya Sanovar, Srikant Bharadwaj, Renee St. Amant, Victor Ruehle, Saravan Rajmohan

May 2024

Publication

Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs

Fangkai Yang, Lu Wang, Zhenyu Xu, Jue Zhang, Liqun Li, Bo Qiao, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Terry Yang, Victor Ruehle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang

Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2023

Publication

Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers

Rya Sanovar, Srikant Bharadwaj, Renee St. Amant, Victor Ruehle, Saravan Rajmohan

May 2024

Publication

LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression

Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang

The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024

Preprint Project

Bayesian Estimation of Differential Privacy

Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Ahmed Salem, Victor Ruehle, Andrew Paverd, Mohammad Naseri, Boris Köpf, Daniel Jones

2023 International Conference on Machine Learning | July 2023

Editor(s): Barbara Engelhardt, Emma Brunskill, Kyunghyun Cho

Publication Project

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

A. Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Ruehle, Saravan Rajmohan, Kunal Jain, Anoop Kulkarni, Steve Kofsky

August 2024

Preprint

Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers

Rya Sanovar, Srikant Bharadwaj, Renee St. Amant, Victor Ruehle, Saravan Rajmohan

May 2024

Publication

Hybrid Retrieval-Augmented Generation for Real-time Composition Assistance

Menglin Xia, Xuchao Zhang, Camille Couturier, Guoqing Zheng, Saravan Rajmohan, Victor Ruehle

February 2024

Preprint

LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression

Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang

The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024

Preprint Project

Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruehle, Laks V. S. Lakshmanan, Ahmed Awadallah

ICLR 2024 | February 2024

Publication

Bayesian Estimation of Differential Privacy

Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Ahmed Salem, Victor Ruehle, Andrew Paverd, Mohammad Naseri, Boris Köpf, Daniel Jones

2023 International Conference on Machine Learning | July 2023

Editor(s): Barbara Engelhardt, Emma Brunskill, Kyunghyun Cho

Publication Project

Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs

Fangkai Yang, Lu Wang, Zhenyu Xu, Jue Zhang, Liqun Li, Bo Qiao, Camille Couturier, Chetan Bansal, Soumya Ram, Si Qin, Zhen Ma, Íñigo Goiri, Eli Cortez, Terry Yang, Victor Ruehle, Saravan Rajmohan, Qingwei Lin 林庆维, Dongmei Zhang

Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) | March 2023

Publication