You Only Prefill Once: Combining Cached Knowledge for Large Language Model Serving with CacheBlend
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys | April 2025
Shan Lu
SR PRINCIPAL RESEARCH MANAGER
Publications
Publications by Year
-
-
If At First You Don’t Succeed, Try, Try, Again…? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems
Bogdan Alexandru Stoica, Utsav Sethi , Yiming Su , Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath
2024 Symposium on Operating Systems Principles | November 2024
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications
Lei Chen, Shiafun Liu, Chenxi Wang, Haoran Ma, Yifan Qiao, Zhe Wang, Chenggang Wu, Youyou Lu, Xiaobing Feng, Huimin Cui, Shan Lu, Harry Xu
OSDI 2024 | June 2024
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
ExChain: Exception Dependency Analysis for Root Cause Diagnosis
Ao Li, Shan Lu, Suman Nath, Rohan Padhye, Vyas Sekar
NSDI | April 2024
-
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
HotGPT: How to Make Software Documentation More Useful with a Large Language Model?
Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Hot Topics in Operating Systems (HotOS) | June 2023
Publications by Research Area
-
You Only Prefill Once: Combining Cached Knowledge for Large Language Model Serving with CacheBlend
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys | April 2025
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
-
You Only Prefill Once: Combining Cached Knowledge for Large Language Model Serving with CacheBlend
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys | April 2025
If At First You Don’t Succeed, Try, Try, Again…? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems
Bogdan Alexandru Stoica, Utsav Sethi , Yiming Su , Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath
2024 Symposium on Operating Systems Principles | November 2024
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications
Lei Chen, Shiafun Liu, Chenxi Wang, Haoran Ma, Yifan Qiao, Zhe Wang, Chenggang Wu, Youyou Lu, Xiaobing Feng, Huimin Cui, Shan Lu, Harry Xu
OSDI 2024 | June 2024
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
ExChain: Exception Dependency Analysis for Root Cause Diagnosis
Ao Li, Shan Lu, Suman Nath, Rohan Padhye, Vyas Sekar
NSDI | April 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
HotGPT: How to Make Software Documentation More Useful with a Large Language Model?
Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Hot Topics in Operating Systems (HotOS) | June 2023
-
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291
Publications by Type
-
You Only Prefill Once: Combining Cached Knowledge for Large Language Model Serving with CacheBlend
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang
EuroSys | April 2025
If At First You Don’t Succeed, Try, Try, Again…? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems
Bogdan Alexandru Stoica, Utsav Sethi , Yiming Su , Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath
2024 Symposium on Operating Systems Principles | November 2024
CacheGen: Fast Context Loading for Language Model Applications via KV Cache Streaming
Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, Junchen Jiang
SIGCOMM | August 2024
ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications
Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire, Shan Lu
OSDI 2024 | July 2024
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications
Lei Chen, Shiafun Liu, Chenxi Wang, Haoran Ma, Yifan Qiao, Zhe Wang, Chenggang Wu, Youyou Lu, Xiaobing Feng, Huimin Cui, Shan Lu, Harry Xu
OSDI 2024 | June 2024
ExChain: Exception Dependency Analysis for Root Cause Diagnosis
Ao Li, Shan Lu, Suman Nath, Rohan Padhye, Vyas Sekar
NSDI | April 2024
HotGPT: How to Make Software Documentation More Useful with a Large Language Model?
Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Hot Topics in Operating Systems (HotOS) | June 2023
-
Keeper: Automated Testing and Fixing of Machine Learning Software
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yuhan Liu, Henry Hoffmann, Michael Maire, Shan Lu
ACM Transactions on Software Engineering and Methodology | May 2024
Run-Time Prevention of Software Integration Failures of Machine Learning APIs
Chengcheng Wan, Yuhan Liu, Kuntai Du, Henry Hoffmann, Junchen Jiang, Michael Maire, Shan Lu
OOPSLA | October 2023, Vol 7: pp. 264-291