{"id":1002690,"date":"2024-01-29T10:10:01","date_gmt":"2024-01-29T18:10:01","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&#038;p=1002690"},"modified":"2024-01-29T10:10:01","modified_gmt":"2024-01-29T18:10:01","slug":"watching-the-air-rise-learning-based-single-frame-schlieren-detection","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/watching-the-air-rise-learning-based-single-frame-schlieren-detection\/","title":{"rendered":"Watching the Air Rise: Learning-Based Single-Frame Schlieren Detection"},"content":{"rendered":"<p><span dir=\"ltr\" role=\"presentation\">Detecting air flows caused by phenomena such <\/span><span dir=\"ltr\" role=\"presentation\">as heat convection is valuable in multiple scenarios, including <\/span><span dir=\"ltr\" role=\"presentation\">leak identification and locating thermal updrafts for extending <\/span><span dir=\"ltr\" role=\"presentation\">UAVs\u2019 flight duration. Unfortunately, these flows\u2019 heat signature <\/span><span dir=\"ltr\" role=\"presentation\">is often too subtle to be seen by a thermal camera. While <\/span><span dir=\"ltr\" role=\"presentation\">convection also leads to fluctuations in air density and hence <\/span><span dir=\"ltr\" role=\"presentation\">causes so-called<\/span> <span dir=\"ltr\" role=\"presentation\">schlieren<\/span> <span dir=\"ltr\" role=\"presentation\">\u2013 intensity and color variations in <\/span><span dir=\"ltr\" role=\"presentation\">images<\/span> <span dir=\"ltr\" role=\"presentation\">\u2013<\/span> <span dir=\"ltr\" role=\"presentation\">existing<\/span> <span dir=\"ltr\" role=\"presentation\">techniques<\/span> <span dir=\"ltr\" role=\"presentation\">such<\/span> <span dir=\"ltr\" role=\"presentation\">as<\/span> <span dir=\"ltr\" role=\"presentation\">Background-oriented <\/span><span dir=\"ltr\" role=\"presentation\">schlieren (BOS) allow detecting them only against a known <\/span><span dir=\"ltr\" role=\"presentation\">background and from a static camera, making these approaches <\/span><span dir=\"ltr\" role=\"presentation\">unsuitable for moving vehicles. In this work we demonstrate <\/span><span dir=\"ltr\" role=\"presentation\">the feasibility of visualizing air movement by predicting the <\/span><span dir=\"ltr\" role=\"presentation\">corresponding<\/span> <span dir=\"ltr\" role=\"presentation\">schlieren-induced<\/span> <span dir=\"ltr\" role=\"presentation\">optical<\/span> <span dir=\"ltr\" role=\"presentation\">flow<\/span> <span dir=\"ltr\" role=\"presentation\">from<\/span> <span dir=\"ltr\" role=\"presentation\">a<\/span> <span dir=\"ltr\" role=\"presentation\">single <\/span><span dir=\"ltr\" role=\"presentation\">greyscale<\/span> <span dir=\"ltr\" role=\"presentation\">image<\/span> <span dir=\"ltr\" role=\"presentation\">captured<\/span> <span dir=\"ltr\" role=\"presentation\">by<\/span> <span dir=\"ltr\" role=\"presentation\">a<\/span> <span dir=\"ltr\" role=\"presentation\">moving<\/span> <span dir=\"ltr\" role=\"presentation\">camera<\/span> <span dir=\"ltr\" role=\"presentation\">against<\/span> <span dir=\"ltr\" role=\"presentation\">an <\/span><span dir=\"ltr\" role=\"presentation\">unfamiliar background. We first record and label a set of optical <\/span><span dir=\"ltr\" role=\"presentation\">flows in an indoor setup using standard BOS techniques. We <\/span><span dir=\"ltr\" role=\"presentation\">then train a convolutional neural network (CNN) by applying <\/span><span dir=\"ltr\" role=\"presentation\">the previously collected optical flow distortions to a dataset <\/span><span dir=\"ltr\" role=\"presentation\">containing a mixture of real and synthetically generated images <\/span><span dir=\"ltr\" role=\"presentation\">to predict the two-dimensional optical flow from a single image. <\/span><span dir=\"ltr\" role=\"presentation\">Finally, we evaluate our approach on the task of extracting the <\/span><span dir=\"ltr\" role=\"presentation\">optical flow caused by schlieren from both a static and moving <\/span><span dir=\"ltr\" role=\"presentation\">camera on previously unseen flow patterns and background <\/span><span dir=\"ltr\" role=\"presentation\">images.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Detecting air flows caused by phenomena such as heat convection is valuable in multiple scenarios, including leak identification and locating thermal updrafts for extending UAVs\u2019 flight duration. Unfortunately, these flows\u2019 heat signature is often too subtle to be seen by a thermal camera. While convection also leads to fluctuations in air density and hence causes [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13556],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[259330,246685,249835],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1002690","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-field-of-study-drone","msr-field-of-study-machine-learning","msr-field-of-study-robotics"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2024-5-17","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/Watching-the-Air-Rise.pdf","id":"1002702","title":"watching-the-air-rise","label_id":"243109","label":0}],"msr_related_uploader":"","msr_attachments":[{"id":1002702,"url":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/Watching-the-Air-Rise.pdf"}],"msr-author-ordering":[{"type":"text","value":"Florian Achermann","user_id":0,"rest_url":false},{"type":"text","value":"Julian Andreas Haug","user_id":0,"rest_url":false},{"type":"text","value":"Tobias Zumsteg","user_id":0,"rest_url":false},{"type":"text","value":"Nicholas Lawrance","user_id":0,"rest_url":false},{"type":"text","value":"Jen Jen Chung","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Andrey Kolobov","user_id":30910,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Andrey Kolobov"},{"type":"text","value":"Roland Siegwart","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[862206],"msr_project":[502862],"publication":[],"video":[],"download":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":502862,"post_title":"Project Frigatebird: AI for Autonomous Soaring","post_name":"project-frigatebird-ai-for-autonomous-soaring","post_type":"msr-project","post_date":"2018-08-27 23:27:11","post_modified":"2024-04-04 10:26:39","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-frigatebird-ai-for-autonomous-soaring\/","post_excerpt":"Autonomous Soaring: an Open-World Challenge for AI Techniques for automatic decision making under uncertainty have been making great strides in their ability to learn complex policies from streams of observations. However, this progress is happening mostly in --- and has a bias towards --- settings with abundant data or readily available high-fidelity simulators, such as games. Learning algorithms in these environments enjoy luxuries unavailable to AI agents in the open world, including resettable training episodes,&hellip;","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/502862"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1002690"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1002690\/revisions"}],"predecessor-version":[{"id":1002705,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1002690\/revisions\/1002705"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1002690"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=1002690"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=1002690"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1002690"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=1002690"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=1002690"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=1002690"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=1002690"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=1002690"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1002690"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1002690"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=1002690"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=1002690"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=1002690"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1002690"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1002690"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}