Reducing AI’s Carbon Footprint
Pursuing more efficient AI
Télécharger
XtremeDistil
avril 2022
XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale.
LoRA
avril 2022
This repo contains the source code of the Python package loralib and several examples of how to integrate it with PyTorch models, such as those in HuggingFace. We only support PyTorch for now. See our paper for a detailed description…
Maximal Update Parametrization (μP)
mars 2022
Maximal Update Parametrization (μP) and Hyperparameter Transfer (μTransfer), in association with the paper: Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Archai – Reproducible Rapid Research for Network Architecture Search
octobre 2020
Archai is a platform for Neural Network Search (NAS) that allow you to generate efficient deep networks for your applications. Archai aspires to accelerate NAS research by enabling easy mix and match between different techniques while ensuring reproducibility, self-documented hyper-parameters…
UniLM – Unified Language Model Pre-training
octobre 2019
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities.