TurboAttention: Efficient Attention Approximation For High Throughputs LLMs
Hao Kang, Srikant Bharadwaj, James Hensman, Tushar Krishna, Victor Ruehle, Saravan Rajmohan
ArXiv | December 2024, Vol abs/2412.08585
James Hensman
Sr Principal Researcher
Publications
Publications by Year
-
KBLaM: Knowledge Base augmented Language Model
Xi Wang, Liana Mikaelyan, Taketomo Isazawa, James Hensman
October 2024
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, James Hensman, Pashmina Cameron
2024 Neural Information Processing Systems | March 2024
Preprint
Publications by Research Area
-
TurboAttention: Efficient Attention Approximation For High Throughputs LLMs
Hao Kang, Srikant Bharadwaj, James Hensman, Tushar Krishna, Victor Ruehle, Saravan Rajmohan
ArXiv | December 2024, Vol abs/2412.08585
KBLaM: Knowledge Base augmented Language Model
Xi Wang, Liana Mikaelyan, Taketomo Isazawa, James Hensman
October 2024
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, James Hensman, Pashmina Cameron
2024 Neural Information Processing Systems | March 2024
Preprint
Publications by Type
-
TurboAttention: Efficient Attention Approximation For High Throughputs LLMs
Hao Kang, Srikant Bharadwaj, James Hensman, Tushar Krishna, Victor Ruehle, Saravan Rajmohan
ArXiv | December 2024, Vol abs/2412.08585
-
KBLaM: Knowledge Base augmented Language Model
Xi Wang, Liana Mikaelyan, Taketomo Isazawa, James Hensman
October 2024
-
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, James Hensman, Pashmina Cameron
2024 Neural Information Processing Systems | March 2024
Preprint