CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys 2025 | April 2025
Shan Lu
Senior Principal Research Manager
Publications
Publications by Year
-
-
If At First You Don’t Succeed, Try, Try, Again…? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems
Bogdan Alexandru Stoica, Utsav Sethi , Yiming Su , Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath
ACM SOSP 2024 | November 2024
DroidSpeak: KV Cache Sharing for Efficient Multi-LLM Serving
Yuhan Liu, Yuyang Huang, Jiayi Yao, Zhuohan Gu, Kuntai Du, Hanchen Li, Yihua Cheng, Junchen Jiang, Shan Lu, Madan Musuvathi, Esha Choukse
November 2024
Automated Proof Generation for Rust Code via Self-Evolution
Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
ICLR 2025 | October 2024
AutoVerus: Automated Proof Generation for Rust Code
Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
September 2024
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications
Lei Chen, Shiafun Liu, Chenxi Wang, Haoran Ma, Yifan Qiao, Zhe Wang, Chenggang Wu, Youyou Lu, Xiaobing Feng, Huimin Cui, Shan Lu, Harry Xu
OSDI 2024 | June 2024
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
ExChain: Exception Dependency Analysis for Root Cause Diagnosis
Ao Li, Shan Lu, Suman Nath, Rohan Padhye, Vyas Sekar
USENIX NSDI | April 2024
-
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
HotGPT: How to Make Software Documentation More Useful with a Large Language Model?
Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Hot Topics in Operating Systems (HotOS) | June 2023
-
Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics
Shan Lu, Soyeon Park, Eunsoo Seo, Yuanyuan Zhou
2008 Architectural Support for Programming Languages and Operating Systems | February 2008
ACM SIGARCH/SIGPLAN/SIGOPS ASPLOS Influential Paper Award
Publications by Research Area
-
Automated Proof Generation for Rust Code via Self-Evolution
Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
ICLR 2025 | October 2024
AutoVerus: Automated Proof Generation for Rust Code
Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
September 2024
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics
Shan Lu, Soyeon Park, Eunsoo Seo, Yuanyuan Zhou
2008 Architectural Support for Programming Languages and Operating Systems | February 2008
ACM SIGARCH/SIGPLAN/SIGOPS ASPLOS Influential Paper Award
-
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys 2025 | April 2025
If At First You Don’t Succeed, Try, Try, Again…? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems
Bogdan Alexandru Stoica, Utsav Sethi , Yiming Su , Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath
ACM SOSP 2024 | November 2024
DroidSpeak: KV Cache Sharing for Efficient Multi-LLM Serving
Yuhan Liu, Yuyang Huang, Jiayi Yao, Zhuohan Gu, Kuntai Du, Hanchen Li, Yihua Cheng, Junchen Jiang, Shan Lu, Madan Musuvathi, Esha Choukse
November 2024
Automated Proof Generation for Rust Code via Self-Evolution
Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
ICLR 2025 | October 2024
AutoVerus: Automated Proof Generation for Rust Code
Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
September 2024
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications
Lei Chen, Shiafun Liu, Chenxi Wang, Haoran Ma, Yifan Qiao, Zhe Wang, Chenggang Wu, Youyou Lu, Xiaobing Feng, Huimin Cui, Shan Lu, Harry Xu
OSDI 2024 | June 2024
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
ExChain: Exception Dependency Analysis for Root Cause Diagnosis
Ao Li, Shan Lu, Suman Nath, Rohan Padhye, Vyas Sekar
USENIX NSDI | April 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
HotGPT: How to Make Software Documentation More Useful with a Large Language Model?
Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Hot Topics in Operating Systems (HotOS) | June 2023
Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics
Shan Lu, Soyeon Park, Eunsoo Seo, Yuanyuan Zhou
2008 Architectural Support for Programming Languages and Operating Systems | February 2008
ACM SIGARCH/SIGPLAN/SIGOPS ASPLOS Influential Paper Award
-
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys 2025 | April 2025
DroidSpeak: KV Cache Sharing for Efficient Multi-LLM Serving
Yuhan Liu, Yuyang Huang, Jiayi Yao, Zhuohan Gu, Kuntai Du, Hanchen Li, Yihua Cheng, Junchen Jiang, Shan Lu, Madan Musuvathi, Esha Choukse
November 2024
Automated Proof Generation for Rust Code via Self-Evolution
Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
ICLR 2025 | October 2024
AutoVerus: Automated Proof Generation for Rust Code
Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
September 2024
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
Publications by Type
-
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys 2025 | April 2025
If At First You Don’t Succeed, Try, Try, Again…? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems
Bogdan Alexandru Stoica, Utsav Sethi , Yiming Su , Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath
ACM SOSP 2024 | November 2024
Automated Proof Generation for Rust Code via Self-Evolution
Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
ICLR 2025 | October 2024
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications
Lei Chen, Shiafun Liu, Chenxi Wang, Haoran Ma, Yifan Qiao, Zhe Wang, Chenggang Wu, Youyou Lu, Xiaobing Feng, Huimin Cui, Shan Lu, Harry Xu
OSDI 2024 | June 2024
ExChain: Exception Dependency Analysis for Root Cause Diagnosis
Ao Li, Shan Lu, Suman Nath, Rohan Padhye, Vyas Sekar
USENIX NSDI | April 2024
HotGPT: How to Make Software Documentation More Useful with a Large Language Model?
Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Hot Topics in Operating Systems (HotOS) | June 2023
Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics
Shan Lu, Soyeon Park, Eunsoo Seo, Yuanyuan Zhou
2008 Architectural Support for Programming Languages and Operating Systems | February 2008
ACM SIGARCH/SIGPLAN/SIGOPS ASPLOS Influential Paper Award
-
DroidSpeak: KV Cache Sharing for Efficient Multi-LLM Serving
Yuhan Liu, Yuyang Huang, Jiayi Yao, Zhuohan Gu, Kuntai Du, Hanchen Li, Yihua Cheng, Junchen Jiang, Shan Lu, Madan Musuvathi, Esha Choukse
November 2024
AutoVerus: Automated Proof Generation for Rust Code
Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
September 2024
-
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291