Microsoft at NAACL 2021

When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages

Stojan Trajanovski, Chad Atalla, Kunho Kim, Vipul Agarwal, Milad Shokouhi, Chris Quirk

APo-VAE: Text Generation in Hyperbolic Space

Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, JJ (Jingjing) Liu

MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning

Mengzhou Xia, Guoqing Zheng, Subhabrata (Subho) Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Awadallah

Noisy Self-Knowledge Distillation for Text Summarization

Yang Liu, Sheng Shen, Mirella Lapata

Enhancing Factual Consistency of Abstractive Summarization

Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, Meng Jiang

Self-training with Weak Supervision

Giannis Karamanolakis, Subhabrata (Subho) Mukherjee, Guoqing Zheng, Ahmed Awadallah

LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu

Modular Networks for Compositional Instruction Following

Rodolfo Corona, Daniel Fried, Coline Devin, Dan Klein, Trevor Darrell

Posterior Differential Regularization with f-divergence for Improving Model Robustness

Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao

The Curious Case of Hallucinations in Neural Machine Translation

Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt

Understanding by Understanding Not: Modeling Negation in Language Models

Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville

Structure-Grounded Pretraining for Text-to-SQL

Xiang Deng, Ahmed Awadallah, Chris Meek, Alex Polozov, Huan Sun, Matthew Richardson

Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle

Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville

Mask Attention Networks: Rethinking and Strengthen Transformer

Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang

ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding

Dongling Xiao, Yu-Kun Li, Han Zhang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

Modeling Event Plausibility with Consistent Conceptual Abstraction

Ian Porada, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung

SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding

Yu-An Chung, Chenguang Zhu, Michael Zeng

Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks

Siddharth Dalmia, Brian Yan, Vikas Raunak, Florian Metze, Shinji Watanabe

DA-Transformer: Distance-aware Transformer

Chuhan Wu, Fangzhao Wu, Yongfeng Huang

Are NLP Models Really Able to Solve Simple Math Word Problems?

Arkil Patel, Satwik Bhattamishra, Navin Goyal

Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge

Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian J. McAuley, Furu Wei

Language Scaling for Universal Suggested Replies Model

Qianlan Ying, Payal Bajaj, Budhaditya Deb, Yu Yang, Wei Wang, Bojia Lin, Milad Shokouhi, Xia Song, Yang Yang, Daxin Jiang (姜大昕)

Detoxifying Language Models Risks Marginalizing Minority Voices

Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, Dan Klein

From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding

Rob van der Goot, Ibrahim Sharaf, Aizhan Imankulova, Ahmet Üstün, Marija Stepanović, Alan Ramponi, Siti Oryza Khairunnisa, Mamoru Komachi, Barbara Plank

Learning to Decompose and Organize Complex Tasks

Yi Zhang, Sujay Kumar Jauhar, Julia Kiseleva, Ryen W. White, Dan Roth

FLIN: A Flexible Natural Language Interface for Web Navigation

Sahisnu Mazumder, Oriana Riva

Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention

Pengcheng Yin, Hao Fang, Graham Neubig, Adam Pauls, Emmanouil Antonios Platanios, Yu Su, Sam Thomson, Jacob Andreas

Reading and Acting While Blindfolded: The Need for Semantics in Text Game Agents

Shunyu Yao, Karthik Narasimhan, Matthew Hausknecht

FUDGE: Controlled Text Generation with Future Discriminators

Kevin Yang, Dan Klein

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

Privacy Regularization: Joint Privacy-Utility Optimization in Language Models

Fatemehsadat Mireshghallah, Huseyin Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, Robert Sim

UniDrop: A Simple Yet Effective Technique to Improve Transformer without Extra Cost

Zhen Wu, Lijun Wu, Qi Meng, Yingce Xia, Shufang Xie, Tao Qin, Xinyu Dai, Tie-Yan Liu

Smart-Start Decoding for Neural Machine Translation

Jian Yang, Shuming Ma, Dongdong Zhang, Juncheng Wan, Zhoujun Li, Ming Zhou

Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge

Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley, Julian J. McAuley

Extracting a Knowledge Base of Mechanisms from COVID-19 Papers

Tom Hope, Aida Amini, David Wadden, Madeleine van Zuylen, Sravanthi Parasa, Eric Horvitz, Daniel Weld, Roy Schwartz, Hannaneh Hajishirzi

Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval

Chen Zhao, Chenyan Xiong, Jordan L. Boyd-Graber, Hal Daumé III

Constructing Taxonomies from Pretrained Language Models

Catherine Chen, Kevin Lin, Dan Klein

Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization

Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, Jianfeng Gao

An Empirical Study on Neural Keyphrase Generation

Rui Meng, Xingdi Yuan, Tong Wang, Sanqiang Zhao, Adam Trischler, Daqing He

Text Editing by Command

Felix Faltings, Michel Galley, Gerold Hintz, Chris Brockett, Chris Quirk, Jianfeng Gao, Bill Dolan

BBAEG: Towards BERT-based Biomedical Adversarial Example Generation for Text Classification

Ishani Mondal

Targeted Adversarial Training for Natural Language Understanding

Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi

TITA: A Two-stage Interaction and Topic-Aware Text Matching Model

Xingwu Sun, Yanling Cui, Hongyin Tang, Qiuyu Zhu, Fuzheng Zhang, Beihong Jin

Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning

Jason Wei, Chengyu Huang, Soroush Vosoughi, Yu Cheng, Shiqi Xu

GPT Perdetry Test: Generating new meanings for new words

Nikolay Malkin, Sameera Lanka, Pranav Goel, Sudha Rao, Nebojsa Jojic

NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction

Ahmed Elgohary, Chris Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, Ahmed Awadallah

Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph

Yucheng Zhou, Xiubo Geng, Tao Shen, Wenqiang Zhang, Daxin Jiang (姜大昕)

QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, Dragomir R. Radev

MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization

Chenguang Zhu, Yang Liu, Jie Mei, Michael Zeng