{"id":883593,"date":"2022-10-06T08:02:00","date_gmt":"2022-10-06T15:02:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-blog-post&#038;p=883593"},"modified":"2022-10-06T08:25:20","modified_gmt":"2022-10-06T15:25:20","slug":"farmvibes-ai","status":"publish","type":"msr-blog-post","link":"https:\/\/www.microsoft.com\/en-us\/research\/articles\/farmvibes-ai\/","title":{"rendered":"FarmVibes.AI"},"content":{"rendered":"\n<h3 id=\"extracting-intelligence-from-farm-data-and-remote-sensing-sources\">Extracting intelligence from farm data and remote sensing sources<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-1024x576.jpg\" alt=\"Andrew Nelson studies a FarmVibes.AI image on a tablet\" class=\"wp-image-883251\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/10\/FarmVibes-tablet-09_1400x788.jpg 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>No single data source gives us the complete information about a farm. Sensors capture temporal data for a location, satellites or drones give spatial information at an instant in time, and neither have enough information extract details about properties below the soil\u2019s surface.<\/p>\n\n\n\n<p>Our key hypothesis is that merging a variety of data sources can help us create the ultimate truth about a farm. Through FarmVibes.AI, we have done this for a variety of data sources, including:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Heat maps<\/strong> &#8212; by combining Sensor + Aerial Imagery, e.g., soil moisture, soil temperature, carbon, weed, nutrients, and soil pH heat maps (accepted to<br><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/democratizing-data-driven-agriculture-using-affordable-hardware\/\" target=\"_blank\" rel=\"noreferrer noopener\">Read the paper presented at IEEE Micro 2022 ><\/a><\/li><li><strong>Seeing through clouds<\/strong> &#8212; by combining Optical + RADAR imagery to see through Clouds, with a technique called SpaceEye<br><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/seeing-through-clouds-in-satellite-images\/\" target=\"_blank\" rel=\"noreferrer noopener\">Read the paper ><\/a><\/li><li><strong>Super-res satellite imagery<\/strong> &#8212; by combining low res, cloud-free satellite imagery (from SpaceEye) with higher res cloudy satellite imagery<\/li><li><strong>Microclimate prediction<\/strong> &#8212; by combining on-farm sensor data with weather predictions from weather services <br><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/micro-climate-prediction-multi-scale-encoder-decoder-based-deep-learning-framework\/\" target=\"_blank\" rel=\"noreferrer noopener\">Read the paper presented at KDD 2021 ><\/a><\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-1024x576.png\" alt=\"FarmVibes - diagram showing fused datasets\" class=\"wp-image-882063\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-1024x576.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-300x169.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-768x432.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-1066x600.png 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-655x368.png 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-343x193.png 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-240x135.png 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-640x360.png 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-960x540.png 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788-1280x720.png 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/09\/farmvibes-AI_1400x788.png 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption>Fusing datasets this way helps generate more robust insights. This project contains several fusion models as well (published and shown to be key for agriculture related problems) as a fusion framework to help build robust remote sensing, earth observation, and geospatial models with ease.<\/figcaption><\/figure>\n\n\n\n<p>In addition to research, we are making these tools available to the broader community. Scientists, researchers, and partners can build new workflows leveraging these AI models, to estimate farming practices, the amount of emissions, and the carbon sequestered in soil.&nbsp;&nbsp;<\/p>\n\n\n\n<p>As an agri\/food or sustech data scientist or researcher interacting, fusing, deriving insights from various such spatial-temporal datasets is hard. With FarmVibes.AI, one can develop rich insights for agriculture with ease. They can build models that fuse multiple geospatial and spatiotemporal data from the field to obtain insights regarding its carbon footprint (as determined by tilling, fertilization, and cover crops), the nutrition of food that it is growing (e.g., yield and protein content), and the long-term sustainability of the soil and the water (e.g., detect if topsoil erosion is being arrested and water ways are designed to retain rain and flood water). These insights are not otherwise possible (with regard to robustness, precision and\/or accuracy) without the fusion techniques in FarmVibes.AI. You can fuse together satellite imagery (RGB, SAR, multispectral, different resolutions), drone imagery (RGB, multispec, hyperspec), weather data (historic, current, and forecasts), sensor timeseries data (soil as well as atmospheric), and more (e.g., information from industrial vehicles such as tractors).<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link\" href=\"https:\/\/github.com\/microsoft\/farmvibes-ai\" target=\"_blank\" rel=\"noreferrer noopener\">Download FarmVibes.AI<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>FarmVibes.AI merges data sources, like heat maps, super-res satellite imagery, and other technologies to create a big picture view about a farm.<\/p>\n","protected":false},"author":40306,"featured_media":883251,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-content-parent":881235,"footnotes":""},"research-area":[],"msr-locale":[268875],"msr-post-option":[],"class_list":["post-883593","msr-blog-post","type-msr-blog-post","status-publish","has-post-thumbnail","hentry","msr-locale-en_us"],"msr_assoc_parent":{"id":881235,"type":"project"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/883593"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-blog-post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/40306"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/883593\/revisions"}],"predecessor-version":[{"id":883596,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/883593\/revisions\/883596"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/883251"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=883593"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=883593"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=883593"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=883593"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}