Linux Foundation starts AgStack, an open-source agtech initiative
investableuniverse.com
investableuniverse.com
I've been doing some work in ag-tech for the past few years (having previously worked in ISP/telco, general SME web/mobile development, then consumer travel tech).
The tools available to farmers/growers for what should be quite basic things - i.e., web-connected weather stations and monitoring systems for soil moisture, frost alarms and irrigation control are terrible.
The industry is full of small players who take a look and think "I know how to build stuff with [Arduino/Raspberry Pi/Zigbee etc], I can build an ag-tech monitoring/control product easily".
So everyone builds a hardware product from scratch, then as a quick afterthought, a web app with an SQL database and Highcharts to show data graphs. You have an MVP within a few months, then a few paying customers in your local community, then suddenly your customers start asking for features you'd never thought of, like sensor inputs you didn't know existed, or combined displays of different kinds of data you'd never considered anyone would want, or needing it to be faster (SQL turns out to be really slow for this kind of data) or more reliable (bugfixing is hard when your devices are hundreds of miles away and connected via weak 3G/4G data links), and you realise your hardware and software architecture doesn't support any of this, and you'd have to completely start over to actually deliver something that customers really need, but you're out of money and energy.
The result is a lot of farmers/growers dissatisfied/frustrated at never being able to get the monitoring systems they need (some of them having tried 3+ different vendors), and a lot of new companies trying to build new solutions and going bust.
I've been thinking there needs to be some widely accepted open standards in the industry for hardware and software platforms, so solutions providers can avoid trying to re-invent everything and instead focus on integration of tried-and-true building blocks.
If this initiative brings that about, it would be a big win for everyone in the industry.
Nobody makes money in open-source selling software, either. Everything is either consulting, or SAAS.
I do not agree that SQL is "slow" for these types of problems. I've built a number of systems that support this issue effectively. You _could_ use a tool that has schemaless/unstructured data as a first-class feature, but if your goal is to reduce complexity a Postgres instance is just fine. As with all data projects, indexing is important and needs to be thoughtful (from the beginning). For sensor data, it's also a good idea to think about data retention and removal policies immediately (keep your metrics/aggregates, move raw data to cold storage after a while).
In my experience, the problems are a result of the manufactured constraints on the physical hardware/sensor devices. Mountings and power and connectivity. Receiving data from an array of devices into a shared storage system for analysis is a well solved problem. Receiving data from multiple sensors on a fixed interval, when the sensors may be made by different companies/work totally differently, is where the complexity lives. Combined with each company trying to build their own awful closed source proprietary data system on top of their sensors, you've got a really terrible time.
> SQL turns out to be really slow for this kind of data
I think this is just poor modeling. SQL is just fine for the work you're talking about.
The more I think about right to repair, the more I become convinced it's a symptom of hazy interface specifications.
Radical idea: Ban sensor / actuator companies from building software on top of them in-house.
They're welcome to offer a turn-key solution to market, but it must (1) have hardware and software built by two separate, independent companies & (2) publish its interface specs, between those two companies, to all customers or end users.
Things would cost more. But I'm not convinced this would be a worse world, in aggregate.
I'm as big a booster of good old SQL as anyone, but there's a lot to be said for more targeted time series solutions when it comes to sensors.
I'm working on a platform for monitoring water water tank levels. It slices Grafana and influxdb horizontally to share the resources between multiple users and multiple tanks.
The productivity of such a stack is high, when it comes to getting beautifully rendered, interactive graphs of e.g stacked water levels. And with influxdb flux language, you can write joins that join data from the time series database and the rdbms (for more reference data, like the names and calibration data of individual tanks).
Yes you can do anything with SQL but there's a reason for the presence of dedicated time series databases.
Sensors aren't really different from any other timeseries data.
> The productivity of such a stack is high, when it comes to getting beautifully rendered, interactive graphs of e.g stacked water levels. And with influxdb flux language, you can write joins that join data from the time series database and the rdbms (for more reference data, like the names and calibration data of individual tanks).
Your productivity being high with a given tech stack does not disqualify an alternative tech stack from having equally high (or much higher) productivity for equally trained users.
> Yes you can do anything with SQL but there's a reason for the presence of dedicated time series databases.
The reason you're describing is "marketing"
Try implementing classic timescale features like down sampling in your straight RDBMS.
Certainly you can do it, just as you can build a house with a hammer and nails rather than a nail gun.
But you'll spend lots of time building undifferentiated infrastructure that you could have got out of the box.
It was fun because we were hired to do things that in house employees had failed to do for whatever reason, so when I built the little dashboard I was hired for, everyone was so amazed. I just completed the job I was given, but to them I was a superstar. Consulting can be fun like that.
Check out Phenome Force, they do weekly webinars featuring the open source/DIY tools people are building: https://phenome-force.github.io/PhenomeForce/
I know a few people who work outside of academia, e.g. agtech companies under the Flagship Pioneering umbrella (Inari etc, same parent as Moderna and various other ag/biotech) and the software teams are very remote friendly I believe
If you're interested in a staff position at a university there's a chance my PI might be hiring for full time dev positions later this year (dependent on luck with grant applications), lmk if interested
* technical writers
* security pentesting/code reviews
* hosting and infrastructure support services for open source projects
Pretty sure that there are a lot of people who would be happy to donate their time to opensource projects that benefit farmers, so there is a bigger opportunity for matchmaking here.
There is incentive to deploy those network now. A agreement to deploy is all what is needed ( private property style ). Electricity is a plus but that work well on solar too.
But was disappointing to see the way the LF is organizing this - no representation from the ag industry's heavy hitters and no specific and attainable goals.
Should this be done by an Operating Systems organization? Other than centralized control and logistical systems, ag engineering is not the realm of microprocessors and operating systems, and code monkeys. Ag engineering is a world of hardware, micro-controllers, FSMs, and scheduler stacks.
But then, if not the LF, whom has the organization to do this?
there are blockchain, mainframe, embedded os projects, countless more, under LF.
worth noting that Linux Foundation began as a merger of between Open Source Development Labs (who worked to push Linux adoption in enterprise) and the Free Standards Group (who worked to push open source standards) to standardize Linux. Only one of these groups started with an exclusive Linux focus, and that same group also focused on the enterprise & driving adoptability.
well, sort of. there were a couple of half-assed pushes with things like linux for data centers, but mostly it was a data center filled with hardware that got very little use other than one asterisk system sitting in a corner, a stack of machines/disk that could be "checked out", but mostly got used for automated testing, and an NEC numa itanium machine that barely worked. add to that, projects that intel no longer wanted to support being shoved off, and you have a pretty good view of day to day life at the OSDL.
source: worked there, doing day to day life at the OSDL.
My basic understanding is that on one side you have companies who want to sponsor open source tech they depend on, and on the other side you have open source developement teams who would like to be supported; the Foundation facilitates interaction between these 2 sides.
"The AgStack Foundation will not engage in building software applications but will instead focus on the software infrastructure (tools, frameworks, and models) that will be needed to build, manage and run applications by the members and users"
The biggest issue I see right now is where to get valid data. Model is nothing without a huge amount of validating datasets, which only have ag giants. They will not share the data so easy since all of them build their digital platforms and understand the value of data.
It is a kind of like a CMS/ERP for agtech. I am using it for basic farm management purposes like inventorying equipment, chemicals, etc. It is built on Drupal which I personally do not like. Overall, the open source nature of the project allows anyone to contribute new modules.
I was building my own app before I found it and am glad it's one less project I have to 20%
If you are on firefox, try this: View -> Page Style -> No Style and scroll down. Hit <cntl>+ to enlarge if needed.
I do this on many sites nowadays - wish there was a way to make this a default setting for all sites. (Other browsers probably have a similar feature but I don't have one in front of me to give the step-by-step instructions.)
Something that concerns me personally is the globalistic nature of the food supply chain and just from a risk management perspective it would be great to push it into a more localized state. But I don't think a world where everybody moves to homesteading is realistic, quite the opposite.
Just superficial pondering. If there's some nice initiatives / startups / other working on this would be keen to learn more about them.
e: and we're working on this :)
So, scan the QR code and pick the produce with the shortest supply chain. If consumers demand it, local food production WILL be a big thing.
There's a lot of fraud in food.
We're trying make small farms more automated and enable them as economically feasable businesses. Our main project is a precision farming robot, and within the next year were hoping to ship dev kits
I have some arable land a couple hours away from home so will be following your progress.
Would you mind elaborating a bit more?
There is a tension between closed and open source here that is going to occur in every industry, in every country.
And we shall all sit on the sidelines bemoaning the lack of standards ... unless ... err
I have to admit I don't know how to build dynamic detail based standards that don't die in committee - how do we get YAML / JSON and not SOAP?
I suspect it is best to start with a 100 Million dollars, and create open mailing lists for each industry, and invite the best of each industry to conferences each month.
I think what some of the projects involved are doing is laudable but it's not really revolutionary.
Then again I don't do farming though some of my family and friends of my family do.
You have supercharging current methods which most of the projects in the OpenTeam initiative are doing which seems to be turning into the AgStack Foundation if I understand it correctly. Think adding smart sensors might be some of what they are planning with the framework and importantly all the planning and managing of farms which the strangely named FarmOs is doing ( I say strangely since it's not an OS ).
The other is doing things with hardware and software that aren't mimicking or supercharging current methods but just implementing a new and more efficient way of doing things. Think the difference between making a perfect robot hand and making thing that grips. Both do the same thing but one is not trying to do the same things as the other. So here you have things like Farmbot that probably need so be split into separate stages of operation if it is going to scale at a farm field level.
I feel like we are past the making a better tractor and other farm machinery phase and are moving into automating everything we can phase. Because the problem with our current processes is that they are stressing soil and the nature too much. By having diverse crops in the same field you both minimize the impact of soil, groundwater problems, salting and stuff like that, but also minimizing the risks in farming where you rely on futures commodity markets instead of having a wide balance of produce.
But I'm no expert and haven't put much thought into this. Just want us to head in the right direction and make sure we invest in the right things.
When I was working as a surveyor for my families construction company, we gradually transitioned to having surveying capabilities added to our machines.
And I can tell you they are overpriced pieces of garbage that still work but boy they are backwards. There is some really cool tech but it's obscured by really old buggy software.
The problem with bolting things to existing equipment is you are going to face so many hurdles that is why only a certain type of companies even try to do it. Those include being allowed to integrate with the hardware. Usually not companies that developers interested in elegant solutions. Not that I don't think people that work for those companies can't do interesting things they are usually hindered in doing so.
That is why most companies trying to innovate go the route of building things themselves.
That is also where you would find people that believe in open source that want to do interesting things.
Why bolt things to a tractors whey you can just mod something like a 4x4 vehicle.
I'm a person that would not build a farmbot or probably work much on it in my free time. Have plenty of side projects. But I think their tech is cool and their methodology I feel is right. So I would probably join them as a developer if things lined up. I can't say the same for any of the other projects.
What you need to look out for is when developers are excited about something. It doesn't mean it's the right solution but it's really good starting point.
Yeah having stationary structures doesn't make sense in large scale farming but AUV with the tech that is in Farmbots makes a whole lot of sense.
But I might be missing something. Also the farming I have experience is just in Iceland where things don't get that big. I've been around a lot of different tangential things to do with farming since my father side went from farming to construction. But I feel like a lot of the things are similar.
But what I know most about is software and I am mostly judging things based on that and I'm really impressed with Farmbot and the choices they made.
...said AgStack executive director Sumer Johal according to venturebeat (1) in another meaningless statement that provided no concrete details.
Their high level architecture (2) comfortably encompasses everything from ML to shells to security; “like an operating system” the press release claimed, careful to avoid saying they were actually building an operating system, because that would be daft.
...but who is Sumer Johal? Well, no one very interesting (3) it turns out, but he’s been involved in agtech for some time...
So... I guess I’m left puzzled?
What does this have to do with the Linux foundation?
Well, turns out the place to go to find out is... surprise, the Linux foundation:
https://www.linuxfoundation.org/en/press-release/linux-found...
“Through the AgStack Project, the Linux Foundation will provide valuable cohesion and development capacity to support shared, community-maintained infrastructure.”
Or something. You read it and decide for yourself; 20 different companies coming together to collaborate with completely different ideas of how and on what.
Sounds like open office.
Can’t wait to see how it goes...
[1] - https://venturebeat.com/2021/05/05/linux-foundation-launches... [2] - https://agstack.org/projects/ [3] - https://www.crunchbase.com/person/sumer-johal
I'm not a big fan of improving pesticide application techniques (one of the examples given in the article) as at the end of the day it is still pumping poison no matter how efficient but if it means less chemicals leeching into our food and environment, I suppose it's better than doing nothing.
LifeTrac is an interesting Open Source project that might benifit from this foundational type of support.
When it comes to agriculture, there is already a very large grass roots, open community of farmers and growers. Eg. Market gardeners, Seed sharers, Homesteaders even Hay farmers. I believe this group is primed to turn into a powerful societal movement but does require financial support to compete against the corporate giants that at present literally dominate the AG field(s).
I don't see any real agriculture heavy hitters (e.g. Cargill, ADM).
But yeah it would be awesome if Cargil got onboard. They are into everything, I used to run multi purpose cables for Flir trafficon cameras that was made by a Cargil company. (FLIR could be a huge supporter too for that matter)
The big problem agtech has isn't that we don't have enough "open-source agtech initiatives". The big problem is the divide between 'ag' and 'tech'. A few individuals bridge it well, but for the most part most ag people don't know about tech and most tech people don't know about ag. And often they don't want to know.
The standards mentioned are largely ignored and that will continue to be the case. Public data would be useful, but I don't see it as a game changer.
Happy to be proven wrong, but problems here aren't in tech, it's in the lack of two way communication that lets us apply it effectively to solve industry problems.
We need more software/hardware engineers in open source ag. Be warned, you may need to live in a somewhat rural area, accept a modest paycheck, and occasionally get your hands dirty, but it's a lot more fun than writing code to shuffle forms or finances around imo
And @tomhoward on here brings up loads of good points -- in N farms we've seen N unique implementations ("bespoke") of monitoring, recording, measuring and pumps and processes and power-systems and all that.
What this does is extract even cheaper labor from the masses. 9 in 10 people used to be farmers. Now it's 1 in 300. I find it hard to believe we have a problem with productivity in this industry.
The end game here is to further centralize the power structure into the corporations who already control most of the supply chain. Look at how they treat seed farmers around the world. They pay them pennies for seeds and then charge thousands of dollars for them on the global market.
Again... I don't see it. I see the trees, but where's the forest?
Agricultural overconsumption with a more efficient or open technology supply chain is still overconsumption.
Also "livestock accounts for 77% of global farming land. While livestock takes up most of the world’s agricultural land it only produces 18% of the world’s calories and 37% of total protein"
And the trend is growth..
Raising efficiency and means of exploitation may also induce demand ("oh we're so eco-friendly").
The point is: Life needs (contiguous) undisturbed wilderness.
And ecological management? That this would even be necessary is testament to our fuckup, but yeah that might actually be topic's biggest benefit I suppose.
Pre-industrialisation life managed itself just fine for millenia.
What do you mean by that, specifically? Clearly we benefit enormously from what has happened since, including food, shelter, peace, freedom, knowledge, etc. etc. I just got something called a vaccine, which protects me from a deadly disease. Someone stuck a needle in my arm, but they had figured out how to do that perfectly safely. The vaccine was driven to the provider, kept cold in refrigeration, and I also drove the provider after making prior arrangements via telecommunications. You're literate and reading this on a website using a computer, a vast collection of manufactured items ... You get the idea.
https://en.wikipedia.org/wiki/Life
Christ, how far have things come
Anyway, what is your solution? Starvation doesn't seem like an option.
Regulate the supply side with land use restrictions and restoration of natural habitat; introduce subsidies for underdeveloped and emerging economies to prevent further ecosystem degradation out of economic necessity.
With a set limit on land use, market prices would adjust accordingly (i.e. meat price rises), consumption will reduce (accompanied by some whining) and food innovators get more time and market share for sustainable alternatives.
See https://www.theguardian.com/environment/2018/may/31/avoiding...
https://en.wikipedia.org/wiki/Climate_change_and_agriculture
However this problem is not of a technological nature but a political/institutional one.
If there had been a strong forward-looking consensus to keep agricultural ecosystem use in check, it would never have come to this point.
As so often the free market moves first, society gets to clean up afterwards.
Can’t they just be stewards of Linux kernel development and focus on that? If there are other initiatives just create other foundations or redirect funding to those areas.
The way a lot of these other initiatives are organized is they are their own foundations/projects under the LF umbrella which lets them share infrastructure, piggtback on events, etc.
Meanwhile Linux seems to be doing quite well from where I sit. I'm not sure what activities are lacking around Linux that a more focused (and inevitably much leaner) organization would drive.