Downloads
Stochastic Mixture-of-Experts
October 2021
This PyTorch package implements Taming Sparsely Activated Transformer with Stochastic Experts.
Efficient Self-Supervised Vision Transformers (EsViT)
July 2021
This is a research project in exploring self-supervised learning (SSL) for computer vision. It aims to learn general-purpose image features from raw pixels without relying on manual supervisions, and the learned networks serve as the backbone of various downstream tasks.…
Vision Longformer for Object Detection
May 2021
This project provides the source code for the object detection part of vision longformer paper.
Sent2Vec
July 2015
Sent2vec maps a pair of short text strings (e.g., sentences or query-answer pairs) to a pair of feature vectors in a continuous, low-dimensional space where the semantic similarity between the text strings is computed as the cosine similarity between their…
ReinMax
September 2023
Bridging Discrete and Backpropagation: Straight-Through and Beyond—Guided by our findings, we propose a novel method called ReinMax, which integrates Heun’s Method, a second-order numerical method for solving ODEs, to approximate the gradient. Our method, ReinMax, achieves second-order accuracy without requiring…
LiST (Lite Self-Training)
October 2021
We present a new method LiST for efficient fine-tuning of large pre-trained language models (PLMs) in few-shot learning settings. LiST significantly improves over recent methods that adopt prompt fine-tuning using two key techniques. The first one is the use of…