Not All Tokens Are What You Need for Pretraining
Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Jian Jiao, Nan Duan, Weizhu Chen
2024 Neural Information Processing Systems | October 2024
Best Paper Runner Up
Xiao Liu
Senior Researcher
Publications
Publications by Year
-
Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, Weizhu Chen
March 2024
Using Left and Right Brains Together: Towards Vision and Language Planning
Jun Cen, Chenfei Wu, Xiao Liu, Sheng-Siang Yin, Yixuan Pei, Jinglong Yang, Qifeng Chen, Nan Duan, Jianguo Zhang
ICML 2024 | February 2024
-
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
NeurIPS 2023 | December 2023
-
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-Rong Wen, Nan Duan, Weizhu Chen
EMNLP 2022 | October 2022
Publications by Research Area
-
Not All Tokens Are What You Need for Pretraining
Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Jian Jiao, Nan Duan, Weizhu Chen
2024 Neural Information Processing Systems | October 2024
Best Paper Runner Up
Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, Weizhu Chen
March 2024
Using Left and Right Brains Together: Towards Vision and Language Planning
Jun Cen, Chenfei Wu, Xiao Liu, Sheng-Siang Yin, Yixuan Pei, Jinglong Yang, Qifeng Chen, Nan Duan, Jianguo Zhang
ICML 2024 | February 2024
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
NeurIPS 2023 | December 2023
-
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-Rong Wen, Nan Duan, Weizhu Chen
EMNLP 2022 | October 2022
-
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-Rong Wen, Nan Duan, Weizhu Chen
EMNLP 2022 | October 2022
Publications by Type
-
Not All Tokens Are What You Need for Pretraining
Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Jian Jiao, Nan Duan, Weizhu Chen
2024 Neural Information Processing Systems | October 2024
Best Paper Runner Up
Using Left and Right Brains Together: Towards Vision and Language Planning
Jun Cen, Chenfei Wu, Xiao Liu, Sheng-Siang Yin, Yixuan Pei, Jinglong Yang, Qifeng Chen, Nan Duan, Jianguo Zhang
ICML 2024 | February 2024
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
NeurIPS 2023 | December 2023
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-Rong Wen, Nan Duan, Weizhu Chen
EMNLP 2022 | October 2022
-
Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, Weizhu Chen
March 2024