The ‘Big 5’ of Challenges Facing Farm Data

We believe in a bright future for agriculture; one where data is invaluable in helping Britain’s expert farmers and advisors make the most informed and therefore the best decisions for their needs.

There is no shortage of farms and agribusinesses wanting to do more with their data, and looking for where to start. 

The elephant in the room is that farming data is big, ugly, and complicated. It’s not simple.

At YAGRO we’ve methodically built our capabilities to meet the unique challenges facing farming data. It’s been the hardest single challenge of our careers. Here are the top challenges to farm data, how we’ve overcome them, and why YAGRO’s capability is head and shoulders above anyone else in UK farming: and the single most trusted place for your data.

Data Governance

You probably don’t know this is international Data Privacy Week. So where better to start! 
We’ve always had a simple rule at YAGRO: Do The Right Thing. That means of course GDPR compliance, it also means so much more: for us to be trusted with critical business data, integrity is paramount – whether that’s personal data, business data, or aggregated data. Any use of that data should be crystal-clear and require explicit permission. With professional legal support, we have invested huge amounts of time and energy producing industry-leading, crystal-clear data rights that put the owner of that data (the farmer) first.  

The result? We’ve just been announced as Data Privacy Week Champions. Businesses across the industry trust us with their data. We’re not going to be ‘one of those’ companies.

Data Quality

Are you confident in the quality of your data? Is everything accurately recorded? Every application, cost, sale, yield, machinery movement?
So when you compare figures that are hand-entered by farmers in a simple benchmarking system, do you have total faith in what’s shown?
The vast majority of farms out there are not 100% confident that every line of their data is accurate. 
We built our Data Ingestion Engine to easily handle such errors. It is a marvel of data engineering. Having ingested millions of data points from our customers, our system has learned where and how to spot such gaps or errors in data. It will rigorously trawl through data, highlighting inconsistencies with methods to correct them. Our expert data analysts also carry out regular quality checks to ensure the system is doing its job.

The result? Clean data you can trust, absolutely.
If you’re unsure about your data quality, it’s actually a great way to run a quick audit and see where your gaps are. We’re happy to help.

Data Comparability

Winter Wheat. Wheat. W Wheat. WW. WW2021. My Wheat. Top Wheat. Wheet. W W. ww.
Yes, these are all ways of telling a computer you’re growing wheat in a field. But a traditional computer will see these as all different. We see different field names, varieties, product names, product units, acres or hectares, and on and on.
So what good having great quality data, when none of it can be compared?
Enter our Data Ingestion Engine once again. Having cleaned up all your lively data, we apply a translation step. Using the millions of lines of data, we’ve learned how people write. For example, we know “Winter Wheat” and “Wheet” actually mean the same thing. So we automatically translate and standardise data to our framework of data.

The result? You keep using your language and systems – we’ll work it out, and make it directly comparable to others, which allows us to provide insight from masses of data.

Data Complexity

The data in the first YAGRO system was relatively simple. We looked at input transactions, where the data included Product, Volume, Price, and Date. 4 bits of data pairs. Some maths: when we’re analysing this data, it means there are 6 different combinations of data to understand correlations. 
Fast forward, we now deal with over 130 data types from farms. That makes over 8’000 different combinations of data pairs – and many, many more aggregate data layers to analyse – over many years of data…!
You can imagine Excel spinning away, with the inevitable systems crash…
At YAGRO we make use of cutting edge technology that allow us to handle all this smoothly and efficiently. From cloud databases, to smart ways of storing calculated results (‘caching’), to machine learning models.

The result? Huge data set calculations made available fast and efficiently to your screen.

Data Security

Given the value and sensitivity in data, few of us check on the security of our data holders? Security is too often assumed. 
The 15 biggest hacks so far this century exposed data of some 8.3 billion accounts.
At YAGRO we understand security is paramount for the long term trust of our clients. Our approach to data management follows the latest security principles from industry leaders like Amazon Web Services. We willingly bear substantial costs of rigorous cloud security software, insurance, and substantial engineering effort to bolster our ‘vault’ of data.

The result? Security you can trust.

Should you be looking at your own journey in farm data, overcoming these five challenges will put you on the path to success.

Or of course, you could speak to our expert team at YAGRO, to see how we can help.