{"id":26004,"date":"2020-03-12T09:00:26","date_gmt":"2020-03-12T08:00:26","guid":{"rendered":"https:\/\/www.microsoft.com\/en-gb\/industry\/blog\/?p=26004"},"modified":"2020-03-12T09:59:04","modified_gmt":"2020-03-12T08:59:04","slug":"interconnected-data-for-an-interconnected-planet","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-gb\/industry\/blog\/cross-industry\/2020\/03\/12\/interconnected-data-for-an-interconnected-planet\/","title":{"rendered":"Interconnected data for an interconnected planet"},"content":{"rendered":"
There\u2019s a limited amount of data and metrics surrounding the way we produce, supply, and consume food. Unfortunately, much of this information is fragmented. Until recently it\u2019s been impossible to bring that data together in a meaningful way.<\/p>\n
We set up Agrimetrics<\/a> to help address challenges in the food system using new technology and data to remedy the situation. We highlight how a range of technologies can tackle complexity in the food system and make it more resilient.<\/p>\n While it\u2019s clear that technology is powerful \u2013 the challenge is creating effective business models that support solving these problems.<\/p>\n I get excited about the idea that sharing and connecting data can yield insights that would not be possible without the latest technologies. However, it requires a connection to data that is hard to build in a sustainable way.\u00a0And quite often, people don\u2019t share data without a value exchange. Also, organisations have to weigh the risk vs. value equation in sharing that data \u2013 because we all know that data can be misused.<\/p>\n In my previous role as Director of the Centre for Food Security, one of the emerging themes was the complexity of the food system. The lack of sharing data has made the food system inherently unpredictable and vulnerable.<\/p>\n The food system is dependent on factors beyond human control like the weather. Add on to that the fact that food is at the heart of human existence, and issues in the system can have far reaching consequences. The food system is global. It\u2019s not inconceivable that a drought in one part of the world could impact food shortages elsewhere.<\/p>\n <\/p>\n Serendipitously, I came upon the opportunity to bid for funding to create a Centre for Agri-Informatics and Metrics of Sustainable Intensification. I jumped at the chance. A colleague and I started to discuss how we could use data to reduce complexity. In particular, we wanted to tackle the challenge of reconnecting the farmed, natural, and human ecosystems. These have tended to be managed independently of one another.<\/p>\n These are of course one ecosystem. By building close connections between the ecosystem and its digital representation through data, we can fix this disconnect.<\/p>\n Agrimetrics is a company that is truly at the intersection of science and technology companies. This relationship has always existed, but now it is going beyond the core technical aspects into creating something bigger. We will discuss what this means in practice, and some of the technologies that have changed the way we can build on data.<\/p>\n We are essentially the food and farming sector’s Data Marketplace: a place to find, manage and monetise agri-food data. Our mission is to accelerate the sector’s ability to maximise the value of its data. We want to see a sector where the sharing of data powers the next generation of innovation.<\/p>\n Making the most of the data in agriculture is harder than in many industries because of its fragmentation. At a much more practical level, we are using technology to provide an infrastructure that supports an agri-food data marketplace.<\/p>\n The key requirements for a data marketplace include:<\/p>\n All of these requirements bring technical challenges, including the need for detailed permissioning below the level of the data set. The most interesting to me however is interoperability. Many are tempted to think that standardisation is the route to interoperability. However, this imposes rigidity on the data model when there is a range of data in agriculture.<\/p>\n Agricultural data includes numerical data like prices and yields. It also includes things like plant names, which might be in Latin or a local language. There is also the human challenge. Some data-standards that are different in one area compared to another. Persuading one community of users to abandon their cherished standard in favour of another is likely to be problematic.<\/p>\n We are adopting an alternative. By using semantic data models we are providing comprehensive, machine readable descriptions of the entities which the data is intended to represent as well as the relationships between them. The value of this approach is that it begins with a fundamental description of the world which can be made real in many different ways.<\/p>\n The description begins with the general case, for example: a cow produces milk, humans drink milk, and ends at the specific: ermintrude is a cow; ermintrude produced 5000 litres of milk last year; Susan is a human; Susan drank a pint of milk yesterday.<\/p>\n The value of this approach is that it begins with a fundamental description of the world which can be made real in many different ways. For example the cow entity can be specified in any language or it can be made to represent a specific cow by using data. This data can be supplied in any form as the example illustrates with differing units used to measure volumes of milk.<\/p>\n To build our data marketplace, we realised we needed a sophisticated and connected dataset. We created a knowledge graph with rich semantics to make data interoperable. A knowledge graph uses machine learning to provide structure and create smart relations throughout the dataset.<\/p>\n Graph databases are quite challenging to make performant and useable. They\u2019re built with SPARQL, which isn\u2019t a friendly language and a small mistake can easily bring down a database. This is why you need to combine a range of technologies to tackle any problems. In our case, we used elastic to allow for rapid querying, SQL to store numerical data, and an API called graphDB to take care of the semantic data.<\/p>\n Azure allows us to scale as our data grows. Most importantly, Azure is built with security-by-design to help us keep the data safe. It even uses machine learning and AI to stay ahead of modern security threats and keep its cybersecurity intelligence up to date<\/p>\n The volumes of data that are being created today are transforming the way we do science. With plentiful data, we can have much larger samples to produce more actionable insights for the food system.<\/p>\n As we have highlighted above however, making the most of the data in agriculture is harder than in many industries because of its fragmentation. Therefore, we have taken a more practical approach leveraging the power of Azure and machine learning to provide a robust infrastructure that supports the agri-food data marketplace.<\/p>\n By simplifying complex food systems through the use of the data, analytics, and AI we\u2019re improving resilience and helping solve the global challenge of economically, ethically, and environmentally sustainable food production.<\/p>\nUntangling a complex system<\/h2>\n
Innovating the industry by combining science and technology<\/h2>\n
Connecting fragmented data<\/h2>\n
\n
Leveraging machine learning and the power of Azure<\/h2>\n
Helping create sustainable food production<\/h2>\n
Find out more<\/h2>\n