Automatically detecting sound events with Artificial Intelligence (AI) has become increasingly popular in the field of bioacoustics, particularly for wildlife monitoring and conservation. Conventional methods predominantly employ supervised learning techniques that depend on substantial amounts of manually annotated bioacoustics data. However, manual annotation in bioacoustics is tremendously resource-intensive, both in terms of human labor and financial resources, and requires considerable domain expertise. This consequently undermines the validity of crowdsourcing annotation methods, such as Amazon Mechanical Turk. Additionally, the supervised learning framework restricts application scope to predefined categories within a closed setting. To address these challenges, we present a novel approach leveraging a multi-modal contrastive learning technique called Contrastive Language-Audio Pretraining (CLAP). CLAP allows for flexible class definition during inference through the use of descriptive text prompts and is capable of performing Zero-Shot Transfer on previously unencountered datasets. In this study, we demonstrate that without specific fine-tuning or additional training, an out-of-the-box CLAP model can effectively generalize across 9 bioacoustics benchmarks, covering a wide variety of sounds that are unfamiliar to the model. We show that CLAP achieves comparable, if not superior, recognition performance compared to supervised learning baselines that are fine-tuned on the training data of these benchmarks. Our experiments also indicate that CLAP holds the potential to perform tasks previously unachievable in supervised bioacoustics approaches, such as foreground \/ background sound separation and the discovery of unknown animals. Consequently, CLAP offers a promising foundational alternative to traditional supervised learning methods for bioacoustics tasks, facilitating more versatile applications within the field.<\/p>\n","protected":false},"excerpt":{"rendered":"
Automatically detecting sound events with Artificial Intelligence (AI) has become increasingly popular in the field of bioacoustics, particularly for wildlife monitoring and conservation. Conventional methods predominantly employ supervised learning techniques that depend on substantial amounts of manually annotated bioacoustics data. However, manual annotation in bioacoustics is tremendously resource-intensive, both in terms of human labor and […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13556,243062,198583],"msr-publication-type":[193726],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-967266","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-research-area-audio-acoustics","msr-research-area-ecology-environment","msr-locale-en_us"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2023-8-1","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/www.researchsquare.com\/article\/rs-3180218\/v1","label_id":"252679","label":0},{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/doi.org\/10.21203\/rs.3.rs-3180218\/v1","label_id":"243106","label":0}],"msr_related_uploader":"","msr_attachments":[],"msr-author-ordering":[{"type":"user_nicename","value":"Zhongqi Miao","user_id":42462,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Zhongqi Miao"},{"type":"text","value":"Benjamin Elizalde","user_id":0,"rest_url":false},{"type":"text","value":"Soham Deshmukh","user_id":0,"rest_url":false},{"type":"text","value":"Justin Kitzes","user_id":0,"rest_url":false},{"type":"text","value":"Huaming Wang","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Rahul Dodhia","user_id":41401,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Rahul Dodhia"},{"type":"user_nicename","value":"Juan M. Lavista Ferres","user_id":39552,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Juan M. Lavista Ferres"}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[696544],"msr_project":[1016418,784627],"publication":[],"video":[],"download":[],"msr_publication_type":"unpublished","related_content":{"projects":[{"ID":1016418,"post_title":"Advance Sustainability - AI for Good","post_name":"advance-sustainability-ai-for-good","post_type":"msr-project","post_date":"2024-04-02 08:57:43","post_modified":"2024-10-14 15:56:57","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/advance-sustainability-ai-for-good\/","post_excerpt":"Climate change requires swift, collective action and technological innovation. We are committed to meeting our own goals while enabling others to do the same.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/1016418"}]}},{"ID":784627,"post_title":"Bioacoustics","post_name":"bioacoustics","post_type":"msr-project","post_date":"2021-12-17 10:04:48","post_modified":"2024-06-06 18:56:29","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/bioacoustics\/","post_excerpt":"Bioacoustics is a cross-disciplinary science that combines biology and acoustics. Usually, it refers to the investigation of sound production, dispersion and reception in animals (including humans). In our research lab, we collaborate with conservation organizations and research labs to leverage machine learning and deep learning models to automatically process and analyze large volumes of audio recordings.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/784627"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/967266"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/967266\/revisions"}],"predecessor-version":[{"id":967272,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/967266\/revisions\/967272"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=967266"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=967266"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=967266"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=967266"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=967266"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=967266"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=967266"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=967266"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=967266"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=967266"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=967266"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=967266"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=967266"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=967266"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=967266"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=967266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}