DIABLo: a Deep Individual-Agnostic Binaural Localizer

In this project, we have developed and studied a deep neural network-based individual-agnostic general-purpose binaural localizer (BL) for sound sources located at arbitrary directions on the $4\pi$ sphere. Unlike binaural localization models trained with an HRIR catalog associated with a specific head and ear shape, an individual-agnostic model aims for the generalization over the individuality of HRIRs, and does not assume a-priori knowledge about the HRIRs which the sound wave is filtered through at recording time. The proposed model was evaluated via localization tests using public binaural room impulse responses (BRIRs) and binaural recording datasets and was found to deliver more robust and accurate localization in noisy and reverberant conditions and unknown recording-time HRIRs compared to BLs trained on a single subject’s HRIR catalog. The proposed model is also designed to support multiple or moving sources, and demonstrations for these scenarios are provided.

Speaker Details

Shoken Kaneko received the B. Eng. degree in Electronic Engineering from the University of Tokyo in 2008 and the M. Sci. degree in Physics from Radboud University in 2010. He has been working as a Research Engineer at Yamaha Corporation from 2011 to 2019 in the field of spatial audio for virtual/augmented reality and numerical acoustic simulations. His major contribution at Yamaha includes the development of a binaural spatial audio technology based on statistical ear shape modeling branded as ViReal headphonesTM, which was licensed to a major video game company and was deployed in some million-seller video games which have sold more than 22 million copies in total by May 2020. Currently, he is a Ph.D. student in the Computer Science program at the University of Maryland, College Park. His research interests include scientific computing methods for numerical acoustic simulation and immersive audio technologies for virtual/augmented reality and next-generation telecommunication/telepresence.

Series:
Microsoft Research Talks
Date:
Speakers:
Shoken Kaneko
Affiliation:
University of Maryland, College Park

Series: Microsoft Research Talks