Do GPTs Produce Less Literal Translations?
Vikas Raunak, Arul Menezes, Matt Post, Hany Hassan Awadalla
ACL 2023 | June 2023
Vikas Raunak
Senior Research Scientist
Publications
Publications by Year
-
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, Hany Hassan Awadalla
February 2023
-
Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models
Vikas Raunak, Matt Post, Arul Menezes
NeurIPS 2022 Workshop on Human Evaluation of Generative Models (HGEM) | November 2022
Rank-One Editing of Encoder-Decoder Models
Vikas Raunak, Arul Menezes
NeurIPS 2022 Workshop on Interactive Learning for Natural Language Processing | November 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks
Vikas Raunak, Arul Menezes
EMNLP 2022 | October 2022
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Vikas Raunak, et. al.
EMNLP 2022 | October 2022
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
Vikas Raunak, et. al.
EMNLP 2022 | June 2022
SALTED: A Framework for SAlient Long-tail Translation Error Detection
Vikas Raunak, Matt Post, Arul Menezes
EMNLP 2022 | May 2022
-
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
Siddharth Dalmia, Brian Yan, Vikas Raunak, Florian Metze, Shinji Watanabe
May 2021
The Curious Case of Hallucinations in Neural Machine Translation
Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt
NAACL 2021 | April 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Chinenye Emezue, Varun Gangal, Cristina Garbacea, Tatsunori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Mihir Kale, Dhruv Kumar, Faisal Ladhak, Aman Madaan, Mounica Maddela, Khyati Mahajan, Saad Mahamood, Bodhisattwa Prasad Majumder, Pedro Henrique Martins, Angelina McMillan-Major, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Andre Niyongabo Rubungo, Salomey Osei, Ankur Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanam, João Sedoc, Thibault Sellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, Jiawei Zhou
2021 Natural Language Generation | February 2021
Publications by Research Area
-
Do GPTs Produce Less Literal Translations?
Vikas Raunak, Arul Menezes, Matt Post, Hany Hassan Awadalla
ACL 2023 | June 2023
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, Hany Hassan Awadalla
February 2023
Rank-One Editing of Encoder-Decoder Models
Vikas Raunak, Arul Menezes
NeurIPS 2022 Workshop on Interactive Learning for Natural Language Processing | November 2022
Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models
Vikas Raunak, Matt Post, Arul Menezes
NeurIPS 2022 Workshop on Human Evaluation of Generative Models (HGEM) | November 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks
Vikas Raunak, Arul Menezes
EMNLP 2022 | October 2022
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Vikas Raunak, et. al.
EMNLP 2022 | October 2022
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
Vikas Raunak, et. al.
EMNLP 2022 | June 2022
SALTED: A Framework for SAlient Long-tail Translation Error Detection
Vikas Raunak, Matt Post, Arul Menezes
EMNLP 2022 | May 2022
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
Siddharth Dalmia, Brian Yan, Vikas Raunak, Florian Metze, Shinji Watanabe
May 2021
The Curious Case of Hallucinations in Neural Machine Translation
Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt
NAACL 2021 | April 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Chinenye Emezue, Varun Gangal, Cristina Garbacea, Tatsunori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Mihir Kale, Dhruv Kumar, Faisal Ladhak, Aman Madaan, Mounica Maddela, Khyati Mahajan, Saad Mahamood, Bodhisattwa Prasad Majumder, Pedro Henrique Martins, Angelina McMillan-Major, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Andre Niyongabo Rubungo, Salomey Osei, Ankur Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanam, João Sedoc, Thibault Sellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, Jiawei Zhou
2021 Natural Language Generation | February 2021
-
Rank-One Editing of Encoder-Decoder Models
Vikas Raunak, Arul Menezes
NeurIPS 2022 Workshop on Interactive Learning for Natural Language Processing | November 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks
Vikas Raunak, Arul Menezes
EMNLP 2022 | October 2022
SALTED: A Framework for SAlient Long-tail Translation Error Detection
Vikas Raunak, Matt Post, Arul Menezes
EMNLP 2022 | May 2022
Publications by Type
-
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, Hany Hassan Awadalla
February 2023
-
Do GPTs Produce Less Literal Translations?
Vikas Raunak, Arul Menezes, Matt Post, Hany Hassan Awadalla
ACL 2023 | June 2023
Rank-One Editing of Encoder-Decoder Models
Vikas Raunak, Arul Menezes
NeurIPS 2022 Workshop on Interactive Learning for Natural Language Processing | November 2022
Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models
Vikas Raunak, Matt Post, Arul Menezes
NeurIPS 2022 Workshop on Human Evaluation of Generative Models (HGEM) | November 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks
Vikas Raunak, Arul Menezes
EMNLP 2022 | October 2022
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Vikas Raunak, et. al.
EMNLP 2022 | October 2022
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
Vikas Raunak, et. al.
EMNLP 2022 | June 2022
SALTED: A Framework for SAlient Long-tail Translation Error Detection
Vikas Raunak, Matt Post, Arul Menezes
EMNLP 2022 | May 2022
The Curious Case of Hallucinations in Neural Machine Translation
Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt
NAACL 2021 | April 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Chinenye Emezue, Varun Gangal, Cristina Garbacea, Tatsunori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Mihir Kale, Dhruv Kumar, Faisal Ladhak, Aman Madaan, Mounica Maddela, Khyati Mahajan, Saad Mahamood, Bodhisattwa Prasad Majumder, Pedro Henrique Martins, Angelina McMillan-Major, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Andre Niyongabo Rubungo, Salomey Osei, Ankur Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanam, João Sedoc, Thibault Sellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, Jiawei Zhou
2021 Natural Language Generation | February 2021
-
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
Siddharth Dalmia, Brian Yan, Vikas Raunak, Florian Metze, Shinji Watanabe
May 2021