Getting Modular with Language Models: Building, Reusing a Library of Experts for Task Generalization

Microsoft Research Forum | Episode 2 | March 5, 2024

Alessandro Sordoni shared recent efforts on building and re-using large collections of expert language models to improve zero-shot and few-shot generalization to unseen tasks.

See more at https://aka.ms/ResearchForum-Mar2024 (opens in new tab)

Series:: Microsoft Research Forum
Date:: March 5, 2024

- Alessandro Sordoni
  
  Principal Researcher
Research Area
- Artificial intelligence
Research Lab
- Microsoft Research Lab – Montréal
Event
- Microsoft Research Forum | Episode 2
Publication
- LoRA: Low-Rank Adaptation of Large Language Models
Download
- LoRA
- Phi-2

Series: Microsoft Research Forum

Fostering appropriate reliance on AI
September 3, 2024
Speakers:

Mihaela Vorvoreanu
A generative model of biology for in-silico experimentation and discovery
September 3, 2024
Speakers:

Kevin Kaichuang Yang
Project Aurora: The first large-scale foundation model of the atmosphere
September 3, 2024
Speakers:

Megan Stanley
Direct Nash Optimization: Teaching language models to self-improve with general preferences
September 3, 2024
Speakers:

Corby Rosset
Analog optical computing for sustainable AI and beyond
September 3, 2024
Speakers:

Jiaqi Chu,

Francesca Parmigiani
Research Forum 4 | Panel: Beyond Language: The future of multimodal models in healthcare, gaming, and AI
September 3, 2024
Speakers:

John Langford,

Hoifung Poon,

Katja Hofmann

, et. al.
Research Forum 4 | Keynote: Phi-3-Vision: A highly capable and "small" language vision model
September 3, 2024
Speakers:

Jianfeng Gao
AutoGen Update: Complex Tasks and Agents
June 4, 2024
Speakers:

Adam Fourney
MatterGen: A Generative Model for Materials Design
June 4, 2024
Speakers:

Tian Xie
Driving Industry Evolution: Exploring the Impact of Generative AI on Sector Transformation
June 4, 2024
Speakers:

Jiang Bian
Insights into the Challenges and Opportunities of Large Multi-Modal Models for Blind and Low Vision Users: CLIP
June 4, 2024
Speakers:

Daniela Massiceti
Research Forum 3 | Panel: Generative AI for Global Impact: Challenges and Opportunities
June 4, 2024
Speakers:

Jacki O'Neill,

Tanuja Ganu,

Sunayana Sitaram

, et. al.
Research Forum 3 | Keynote: Building Globally Equitable AI
June 4, 2024
Speakers:

Jacki O'Neill
Generative AI and Plural Governance: Mitigating Challenges and Surfacing Opportunities
March 5, 2024
Speakers:

Madeleine Daepp
GigaPath: Foundation Model for Digital Pathology
March 5, 2024
Speakers:

Naoto Usuyama
Getting Modular with Language Models: Building, Reusing a Library of Experts for Task Generalization
March 5, 2024
Speakers:

Alessandro Sordoni
The Metacognitive Demands and Opportunities of Generative AI
March 5, 2024
Speakers:

Lev Tankelevitch
What's new in AutoGen?
March 5, 2024
Speakers:

Chi Wang
Research Forum 2 | Panel Discussion: Transforming the Natural Sciences with AI
March 5, 2024
Speakers:

Bonnie Kruft,

Alex Lu,

Tristan Naumann

, et. al.
Research Forum 2 | Keynote: The Revolution in Scientific Discovery
March 5, 2024
Speakers:

Christopher Bishop
Generative AI meets Structural Biology: Equilibrium Distribution Prediction
January 30, 2024
Speakers:

Shuxin Zheng
Kahani: Visual Storytelling through Culturally Nuanced Images
January 30, 2024
Speakers:

Sameer Segal
Research Forum: Panel Discussion: AI Frontiers
January 30, 2024
Speakers:

Ashley Llorens,

Sébastien Bubeck,

Ahmed Awadallah

, et. al.
Research Forum: Closing Remarks and Announcements
January 30, 2024
Speakers:

Ashley Llorens
Evaluation and Understanding of Foundation Models
January 30, 2024
Speakers:

Besmira Nushi