What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression
Mengnan Du, Subhabrata (Subho) Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Awadallah
EACL | April 2023
Publications
Publications par année
-
-
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
-
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
Subhabrata (Subho) Mukherjee, Ahmed Awadallah, Jianfeng Gao
ArXiv | June 2021
arXiv:2106.04563
-
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
-
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019
Filtre par zone de recherche
-
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019
-
What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression
Mengnan Du, Subhabrata (Subho) Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Awadallah
EACL | April 2023
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
Subhabrata (Subho) Mukherjee, Ahmed Awadallah, Jianfeng Gao
ArXiv | June 2021
arXiv:2106.04563
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019
Publication Type
-
What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression
Mengnan Du, Subhabrata (Subho) Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Awadallah
EACL | April 2023
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
-
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
Subhabrata (Subho) Mukherjee, Ahmed Awadallah, Jianfeng Gao
ArXiv | June 2021
arXiv:2106.04563
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019