CV Group Header Image

Computer Vision Group

Empowering technologies for real-world vision-based systems

Nouvelles et reportages

ACAV100M text on top of a series of small images.
Blog de recherche Microsoft

| Yale Song

The natural association between visual observations and their corresponding sounds has exhibited powerful self-supervision signals for learning video representations (opens in new tab), which makes the ever-growing amount of online video an attractive data source for self-supervised learning. However, online…

A graphic depicting audio and video content items passing through an audio transformer layer and a video transformer layer, respectively, before being combined while passing through a multimodal transformer layer
Blog de recherche Microsoft

| Yale Song

Understanding video is one of the most challenging problems in AI, and an important underlying requirement is learning multimodal representations that capture information about objects, actions, sounds, and their long-range statistical dependencies from audio-visual signals. Recently, transformers have been successful in…