{"id":885639,"date":"2022-10-12T11:48:45","date_gmt":"2022-10-12T18:48:45","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/"},"modified":"2022-10-12T12:02:23","modified_gmt":"2022-10-12T19:02:23","slug":"towards-all-purpose-full-sphere-binaural-localization","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/towards-all-purpose-full-sphere-binaural-localization\/","title":{"rendered":"Towards all-purpose full-sphere binaural localization"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/LOCATA-task2_results.png\" alt=\"Localization results for LOCATA task 2\" width=\"1297\" height=\"1257\" class=\"aligncenter size-full wp-image-885657\" \/>Sound source localization from binaural signals has important applications ranging from machine listening to psychoacoustics, yet challenges including generalization and robustness to noise and reverberation remain. Here we propose a binaural localizer (BL) framework that produces a full-sphere spatial activity map for every audio input frame. The framework enables individual-agnostic training of a convolutional neural network using head-related impulse response (HRIR) sets with arbitrary measurement grids and is shown to perform well on unseen HRIRs and binaural recordings. Unlike BLs trained with the HRIRs of a specific known subject or dummy head, the proposed individual-agnostic BL is intended to perform robustly without any a priori knowledge about the process creating the binaural signals. Localization tests with binaural speech renderings and recordings show that the proposed BL performs well in the presence of noise and reverberation and compares favorably to individual-specific BLs. Furthermore, preliminary results indicate that the proposed BL is applicable to the localization of multiple simultaneous and moving sources.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/LOCATA-task2_results.png\" alt=\"Localization results for LOCATA task 2\" width=\"600\" class=\"aligncenter size-full wp-image-885657\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Sound source localization from binaural signals has important applications ranging from machine listening to psychoacoustics, yet challenges including generalization and robustness to noise and reverberation remain. Here we propose a binaural localizer (BL) framework that produces a full-sphere spatial activity map for every audio input frame. The framework enables individual-agnostic training of a convolutional neural [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[243062],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-885639","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-audio-acoustics","msr-locale-en_us"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2022-8-1","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/Deep_binaural_localization-2.pdf","id":"885645","title":"deep_binaural_localization-2","label_id":"243132","label":0}],"msr_related_uploader":"","msr_attachments":[{"id":885645,"url":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/Deep_binaural_localization-2.pdf"}],"msr-author-ordering":[{"type":"text","value":"Shoken Kaneko","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Hannes Gamper","user_id":31943,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Hannes Gamper"}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[144923],"msr_project":[212079],"publication":[],"video":[],"download":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":212079,"post_title":"Spatial Audio","post_name":"spatial-audio","post_type":"msr-project","post_date":"2015-12-01 18:14:03","post_modified":"2022-01-21 12:44:12","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/spatial-audio\/","post_excerpt":"Spatial audio, also known as 3D stereo sound, is about creating a 3D audio experience by using headphones. Applications of this technology include augmented and virtual reality, listening to music, and watching a movie on a tablet or PC.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/212079"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/885639"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":6,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/885639\/revisions"}],"predecessor-version":[{"id":885672,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/885639\/revisions\/885672"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=885639"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=885639"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=885639"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=885639"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=885639"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=885639"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=885639"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=885639"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=885639"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=885639"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=885639"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=885639"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=885639"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=885639"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=885639"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=885639"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}