Towards Modular LLMs by Building and Reusing a Library of LoRAs
O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
ICML 2024 | May 2024
Alessandro Sordoni
Principal Researcher
Publications
Publications by Year
-
-
Guiding Language Model Reasoning with Planning Tokens
Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
October 2023
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
NeurIPS 2023 | October 2023
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2023 | October 2023
-
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
ACL 2022 | May 2022
Unsupervised Dependency Graph Network
Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
ACL 2022 | May 2022
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
ICLR 2022 | April 2022
Learning to Dequantise with Truncated Flows
Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
ICLR 2022 | April 2022
-
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
EMNLP 2021 | November 2021
The emergence of the shape bias results from communicative efficiency
Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
Conference on Computational Natural Language Learning (CoNLL) | November 2021
Decomposed Mutual Information Estimation for Contrastive Representation Learning
Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
2021 International Conference on Machine Learning | July 2021
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
May 2021
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
Linguistic dependencies and statistical dependence
Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
2021 Empirical Methods in Natural Language Processing | April 2021
-
Exploring and Predicting Transferability Across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
May 2020
Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
arXiv preprint arXiv:2003.01680 | March 2020
-
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
Ordered Memory
Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
NeurIPS 2019 | November 2019
ArXiv
Robust Natural Language Inference Models with Example Forgetting
Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
November 2019
Metalearned Neural Memory
Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
NeurIPS 2019 | July 2019
Preprint under review
Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
ICLR 2019 | May 2019
ICLR 2019 Best Paper Award
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
ICLR 2019 | April 2019
-
Towards Text Generation with Adversarially Learned Neural Outlines
Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
NeurIPS 2018 | December 2018
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
European Workshop on Reinforcement Learning (EWRL) | October 2018
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
Proceedings of ACL | July 2018
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
ICML 2018 | June 2018
Augmented cyclegan: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
ICML 2018 | June 2018
ArXiv
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
ICML 2018 | June 2018
Twin Networks: Matching the Future for Sequence Generation
Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
ICLR 2018 | January 2018
-
Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
Learning Algorithms for Active Learning
Philip Bachman, Alessandro Sordoni, Adam Trischler
34th International Conference on Machine Learning (ICML 2017) | August 2017
Publications by Research Area
-
Towards Modular LLMs by Building and Reusing a Library of LoRAs
O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
ICML 2024 | May 2024
Guiding Language Model Reasoning with Planning Tokens
Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
October 2023
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2023 | October 2023
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
NeurIPS 2023 | October 2023
Learning to Dequantise with Truncated Flows
Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
ICLR 2022 | April 2022
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
ICLR 2022 | April 2022
The emergence of the shape bias results from communicative efficiency
Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
Conference on Computational Natural Language Learning (CoNLL) | November 2021
Decomposed Mutual Information Estimation for Contrastive Representation Learning
Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
2021 International Conference on Machine Learning | July 2021
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
May 2021
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
arXiv preprint arXiv:2003.01680 | March 2020
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
Robust Natural Language Inference Models with Example Forgetting
Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
November 2019
Ordered Memory
Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
NeurIPS 2019 | November 2019
ArXiv
Metalearned Neural Memory
Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
NeurIPS 2019 | July 2019
Preprint under review
Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
ICLR 2019 | May 2019
ICLR 2019 Best Paper Award
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
ICLR 2019 | April 2019
Towards Text Generation with Adversarially Learned Neural Outlines
Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
NeurIPS 2018 | December 2018
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
European Workshop on Reinforcement Learning (EWRL) | October 2018
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
Proceedings of ACL | July 2018
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
ICML 2018 | June 2018
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
ICML 2018 | June 2018
Augmented cyclegan: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
ICML 2018 | June 2018
ArXiv
Twin Networks: Matching the Future for Sequence Generation
Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
ICLR 2018 | January 2018
Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
Learning Algorithms for Active Learning
Philip Bachman, Alessandro Sordoni, Adam Trischler
34th International Conference on Machine Learning (ICML 2017) | August 2017
-
Unsupervised Dependency Graph Network
Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
ACL 2022 | May 2022
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
ACL 2022 | May 2022
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
EMNLP 2021 | November 2021
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
Linguistic dependencies and statistical dependence
Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
2021 Empirical Methods in Natural Language Processing | April 2021
Exploring and Predicting Transferability Across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
May 2020
-
Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
-
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
Publications by Type
-
Towards Modular LLMs by Building and Reusing a Library of LoRAs
O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
ICML 2024 | May 2024
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
NeurIPS 2023 | October 2023
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
NeurIPS 2023 | October 2023
Unsupervised Dependency Graph Network
Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
ACL 2022 | May 2022
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
ACL 2022 | May 2022
Learning to Dequantise with Truncated Flows
Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
ICLR 2022 | April 2022
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
ICLR 2022 | April 2022
The emergence of the shape bias results from communicative efficiency
Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
Conference on Computational Natural Language Learning (CoNLL) | November 2021
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
EMNLP 2021 | November 2021
Decomposed Mutual Information Estimation for Contrastive Representation Learning
Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
2021 International Conference on Machine Learning | July 2021
Linguistic dependencies and statistical dependence
Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
2021 Empirical Methods in Natural Language Processing | April 2021
Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
Ordered Memory
Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
NeurIPS 2019 | November 2019
ArXiv
Metalearned Neural Memory
Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
NeurIPS 2019 | July 2019
Preprint under review
Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
ICLR 2019 | May 2019
ICLR 2019 Best Paper Award
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
ICLR 2019 | April 2019
Towards Text Generation with Adversarially Learned Neural Outlines
Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
NeurIPS 2018 | December 2018
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
European Workshop on Reinforcement Learning (EWRL) | October 2018
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
ICML 2018 | June 2018
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
ICML 2018 | June 2018
Augmented cyclegan: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
ICML 2018 | June 2018
ArXiv
Twin Networks: Matching the Future for Sequence Generation
Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
ICLR 2018 | January 2018
Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
Learning Algorithms for Active Learning
Philip Bachman, Alessandro Sordoni, Adam Trischler
34th International Conference on Machine Learning (ICML 2017) | August 2017
-
Guiding Language Model Reasoning with Planning Tokens
Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
October 2023
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
May 2021
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
May 2021
Exploring and Predicting Transferability Across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
May 2020
Robust Natural Language Inference Models with Example Forgetting
Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
November 2019
-
Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
arXiv preprint arXiv:2003.01680 | March 2020
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
Proceedings of ACL | July 2018