Data inter/intra operability in the agriculture and food industry
The agriculture and food industry is a highly complex and interconnected system that relies on various stakeholders to ensure the production and delivery of quality food products. In recent years, the industry has experienced a rapid growth in the amount of data generated from various sources, including IoT sensors, GPS trackers, and weather stations, among others. However, this abundance of data has created new challenges, especially regarding data interoperability and interconnectivity.
In a 2021 report on the topic, the Farm Foundation discusses the importance of data interoperability in agriculture. It is pointed out that a Trust in Food survey with US farmers found that nearly a third of respondents primarily store and manage their data on paper records, and more than half did not rely on farm-level data software; of the 50% of farmers who do use Farm Management Information Systems (FMIS), less than half were completely satisfied with the software outputs, this is closely related to an absence of industry-wide structural and semantic interoperability,
What is data inter and intra operability in the Agriculture and Food Industry and why is it important?
Data interoperability refers to the ability of different systems and technologies to exchange and use data effectively. In the context of the agriculture and food industry, and at the farm level, data interoperability enables seamless integration of different data sources, such as weather data, soil data, crop data, financial data, and many others, to provide a comprehensive view of the farming environment. The same can be applied to any other segments of the value chain, like input manufacturers, distribution channels, food manufacturers, and so on.
In contrast, data intra operability refers to the ability of different components within a system to communicate and share data effectively. For example, intra operability enables different IoT sensors to communicate with each other to provide accurate and timely data about the farming environment. From an ag and food business perspective, it can mean companies using the data from different internal data silos in a connected/shared manner, amplifying their analytical capacity.
There are 3 types of interoperability that need to be considered:
• TECHNICAL INTEROPERABILITY: Infrastructure and protocols: the physical infrastructure is in place to transport bits of data.
• SYNTACTIC INTEROPERABILITY: Common data structure: the ability to communicate and exchange data between two or more systems through a standardized structure and format; shared syntax.
• SEMANTIC INTEROPERABILITY: Common data definition: the ability to exchange data between systems and for the data to be understood by each system; shared meaning
Why are startups needed in this space?
The agriculture and food industry faces several challenges related to data inter and intra operability, including data silos, lack of standardization, and limited interoperability among different technologies. Startups can play a critical role in addressing these challenges by developing innovative solutions that enable seamless data integration and communication across different systems and technologies.
Moreover, startups can also leverage emerging technologies, such as artificial intelligence and machine learning, to develop predictive analytics tools that enable real-time decision-making and data-driven insights. By doing so, startups can help the agriculture and food industry become more efficient, sustainable, and resilient in the face of future challenges.
Looking at the current agricultural data landscape, there are four primary approaches to achieving interoperability, and startups are very well-positioned to leverage them:
1) Point-to-point integrations: One-off point-to-point integration between applications
2) Linking systems of systems: The development and adoption of open source, commonly used tools and frameworks to connect different systems.
3) Formal standardization: The development and adoption of shared, open-source frameworks through formal multi-party agreement
4) Walled garden integrations: Proprietary data integrations driven by a large player (or players) not necessarily intended for use beyond their own ecosystem.
Challenges regarding Traceability and Transparency in the Agriculture and Food industry: A Jobs to be Done perspective:
Below we explore some opportunities regarding data inter/intra operability in agriculture and food from a number of different angles. We use the lens of the Jobs-to-be-done framework conceived by Clayton Christensen and popularized by Alexander Osterwalder in his Value Proposition Canvas framework. The Jobs-to-be-Done (JTBD) framework can help identify the emerging challenges and guide data inter/intra operability-focused startups to develop innovative solutions to address the needs of the future.
One way to organize the overarching jobs in this space is to consider a hierarchy of needs from data functionality to actionable insight. The entire stack of technical needs in this theme serve to unblock users’ other functional, social, and emotional jobs to be done. Using that lens, we can propose several “meta-jobs” under which we can find many specific jobs across various customers:
1. Job: Standardize data in use
The agri-food sector is notable for aging systems for data management. Many of the largest firms have homegrown systems, and data collection and management varies widely within value chain segments and even more widely along the value chain. At the most basic level, data for any use needs to be functional for it to create any value. This is a basic requirement to move up the hierarchy of needs.
Ask: What is the ultimate job to be done that better data would support? What is the value of solving that goal with data? What data is needed to support it? Based on the needs and value, what about its structure is holding back progress?
2. Job: Facilitate integrations and movement of data
Functional data’s next bottleneck is movement. Many operators, including producers, have silos within their own walls. Moving data up and down a value chain is a technical challenge, but the greater barrier to interoperability is often embedded in incentives.
Ask: To achieve the business objective, what data needs to move between systems? If it needs to move across value chain stages, what is preventing it from doing so? If a conflict of incentives may exist, how can it be resolved to solve jobs for both parties?
3. Job: Ensure security of data and systems
Systems bear risk. The more reliant a firm is on technology to operate, the greater the cost of a vulnerability. Protecting customer information and proprietary data can be an existential need for a business. Agri-food is also seeing a rise in ransomware and other malware attacks that can grind production to a halt as they have in healthcare. Whether it concerns loose data or a malicious attack, security is a growing concern.
Ask: What are the vulnerabilities to operational goals created by data risks? What is the probability-adjusted cost of those risks?
4. Job: Access and combine relevant data
Some tasks require more data than any one system or player has by itself. As new data and analytics capabilities come online in firms of all sizes, the demand for data will continue to grow. When asking a question about customer behavior, a firm will create blind spots if it can’t merge multiple points of view. However, more data alone is often not sufficient to create new value.
Ask: What is the ultimate objective of an analytical process? What is the incremental value that could be captured? Can more data help to capture it? What else is needed?
5. Job: Enable interpretation of data
Data that has been made clean and brought to the right place needs to be analyzed to inform any actions. At a step between data management and action is the problem of deriving meaning. Today, the industry faces a shortage of data scientists and advanced tools for making data ready for higher-order applications or use by others. Data only has value because of the knowledge and action it can be used to create.
Ask: Beyond technical constraints, who needs to be able to draw what type of conclusion from data? What is holding back analysis and application?
6. Job: Access useful models
We are entering a new era of modeling potential. When a co-op decides to build a localized digital agronomist or an input provider wants to pressure-test its forecasts, models need to be built with their data. But significant parts of the power of those models will be generalized. To accomplish their business objectives, these customers need to start from the right base before customizing.
Ask: What is the ultimate user job to be done that a firm is trying to solve by building a model? To what degree are those circumstances unique? What are similar problems that have been modeled in other circumstances?
7. Job: Translate data to insight and action
Some users just need the answer. In a world where the technical pre-requisites are sufficiently solved, operators can focus on making progress on their core jobs to be done. Specific answers, direct decision support, and even the automation of decision making can create new ways to solve those jobs. But in the end, progress on even this highest-order data interoperability need is in service of another more fundamental job.
Ask: What is the user’s job to be done outside of the data questions raised? What is the best version of solving that job using this stack of capabilities?
DIAL Ventures, the innovation arm of the Purdue Applied Research Institute, tackles big problems facing the U.S. and the world such as food safety, supply chain efficiency, sustainability, and environmental impact. DIAL Ventures creates new companies that drive innovation in the agri-food industry which, in turn, makes a positive impact on our lives and lifestyles for years to come.
If you are interested in becoming a DIAL Ventures Fellow or Corporate Partner, contact us at info@dialventures.com.
In a 2021 report on the topic, the Farm Foundation discusses the importance of data interoperability in agriculture. It is pointed out that a Trust in Food survey with US farmers found that nearly a third of respondents primarily store and manage their data on paper records, and more than half did not rely on farm-level data software; of the 50% of farmers who do use Farm Management Information Systems (FMIS), less than half were completely satisfied with the software outputs, this is closely related to an absence of industry-wide structural and semantic interoperability,
What is data inter and intra operability in the Agriculture and Food Industry and why is it important?
Data interoperability refers to the ability of different systems and technologies to exchange and use data effectively. In the context of the agriculture and food industry, and at the farm level, data interoperability enables seamless integration of different data sources, such as weather data, soil data, crop data, financial data, and many others, to provide a comprehensive view of the farming environment. The same can be applied to any other segments of the value chain, like input manufacturers, distribution channels, food manufacturers, and so on.
In contrast, data intra operability refers to the ability of different components within a system to communicate and share data effectively. For example, intra operability enables different IoT sensors to communicate with each other to provide accurate and timely data about the farming environment. From an ag and food business perspective, it can mean companies using the data from different internal data silos in a connected/shared manner, amplifying their analytical capacity.
There are 3 types of interoperability that need to be considered:
• TECHNICAL INTEROPERABILITY: Infrastructure and protocols: the physical infrastructure is in place to transport bits of data.
• SYNTACTIC INTEROPERABILITY: Common data structure: the ability to communicate and exchange data between two or more systems through a standardized structure and format; shared syntax.
• SEMANTIC INTEROPERABILITY: Common data definition: the ability to exchange data between systems and for the data to be understood by each system; shared meaning
Why are startups needed in this space?
The agriculture and food industry faces several challenges related to data inter and intra operability, including data silos, lack of standardization, and limited interoperability among different technologies. Startups can play a critical role in addressing these challenges by developing innovative solutions that enable seamless data integration and communication across different systems and technologies.
Moreover, startups can also leverage emerging technologies, such as artificial intelligence and machine learning, to develop predictive analytics tools that enable real-time decision-making and data-driven insights. By doing so, startups can help the agriculture and food industry become more efficient, sustainable, and resilient in the face of future challenges.
Looking at the current agricultural data landscape, there are four primary approaches to achieving interoperability, and startups are very well-positioned to leverage them:
1) Point-to-point integrations: One-off point-to-point integration between applications
2) Linking systems of systems: The development and adoption of open source, commonly used tools and frameworks to connect different systems.
3) Formal standardization: The development and adoption of shared, open-source frameworks through formal multi-party agreement
4) Walled garden integrations: Proprietary data integrations driven by a large player (or players) not necessarily intended for use beyond their own ecosystem.
Challenges regarding Traceability and Transparency in the Agriculture and Food industry: A Jobs to be Done perspective:
Below we explore some opportunities regarding data inter/intra operability in agriculture and food from a number of different angles. We use the lens of the Jobs-to-be-done framework conceived by Clayton Christensen and popularized by Alexander Osterwalder in his Value Proposition Canvas framework. The Jobs-to-be-Done (JTBD) framework can help identify the emerging challenges and guide data inter/intra operability-focused startups to develop innovative solutions to address the needs of the future.
One way to organize the overarching jobs in this space is to consider a hierarchy of needs from data functionality to actionable insight. The entire stack of technical needs in this theme serve to unblock users’ other functional, social, and emotional jobs to be done. Using that lens, we can propose several “meta-jobs” under which we can find many specific jobs across various customers:
1. Job: Standardize data in use
The agri-food sector is notable for aging systems for data management. Many of the largest firms have homegrown systems, and data collection and management varies widely within value chain segments and even more widely along the value chain. At the most basic level, data for any use needs to be functional for it to create any value. This is a basic requirement to move up the hierarchy of needs.
Ask: What is the ultimate job to be done that better data would support? What is the value of solving that goal with data? What data is needed to support it? Based on the needs and value, what about its structure is holding back progress?
2. Job: Facilitate integrations and movement of data
Functional data’s next bottleneck is movement. Many operators, including producers, have silos within their own walls. Moving data up and down a value chain is a technical challenge, but the greater barrier to interoperability is often embedded in incentives.
Ask: To achieve the business objective, what data needs to move between systems? If it needs to move across value chain stages, what is preventing it from doing so? If a conflict of incentives may exist, how can it be resolved to solve jobs for both parties?
3. Job: Ensure security of data and systems
Systems bear risk. The more reliant a firm is on technology to operate, the greater the cost of a vulnerability. Protecting customer information and proprietary data can be an existential need for a business. Agri-food is also seeing a rise in ransomware and other malware attacks that can grind production to a halt as they have in healthcare. Whether it concerns loose data or a malicious attack, security is a growing concern.
Ask: What are the vulnerabilities to operational goals created by data risks? What is the probability-adjusted cost of those risks?
4. Job: Access and combine relevant data
Some tasks require more data than any one system or player has by itself. As new data and analytics capabilities come online in firms of all sizes, the demand for data will continue to grow. When asking a question about customer behavior, a firm will create blind spots if it can’t merge multiple points of view. However, more data alone is often not sufficient to create new value.
Ask: What is the ultimate objective of an analytical process? What is the incremental value that could be captured? Can more data help to capture it? What else is needed?
5. Job: Enable interpretation of data
Data that has been made clean and brought to the right place needs to be analyzed to inform any actions. At a step between data management and action is the problem of deriving meaning. Today, the industry faces a shortage of data scientists and advanced tools for making data ready for higher-order applications or use by others. Data only has value because of the knowledge and action it can be used to create.
Ask: Beyond technical constraints, who needs to be able to draw what type of conclusion from data? What is holding back analysis and application?
6. Job: Access useful models
We are entering a new era of modeling potential. When a co-op decides to build a localized digital agronomist or an input provider wants to pressure-test its forecasts, models need to be built with their data. But significant parts of the power of those models will be generalized. To accomplish their business objectives, these customers need to start from the right base before customizing.
Ask: What is the ultimate user job to be done that a firm is trying to solve by building a model? To what degree are those circumstances unique? What are similar problems that have been modeled in other circumstances?
7. Job: Translate data to insight and action
Some users just need the answer. In a world where the technical pre-requisites are sufficiently solved, operators can focus on making progress on their core jobs to be done. Specific answers, direct decision support, and even the automation of decision making can create new ways to solve those jobs. But in the end, progress on even this highest-order data interoperability need is in service of another more fundamental job.
Ask: What is the user’s job to be done outside of the data questions raised? What is the best version of solving that job using this stack of capabilities?
DIAL Ventures, the innovation arm of the Purdue Applied Research Institute, tackles big problems facing the U.S. and the world such as food safety, supply chain efficiency, sustainability, and environmental impact. DIAL Ventures creates new companies that drive innovation in the agri-food industry which, in turn, makes a positive impact on our lives and lifestyles for years to come.
If you are interested in becoming a DIAL Ventures Fellow or Corporate Partner, contact us at info@dialventures.com.