23 karma · joined September 15, 2019
[ my public key: https://keybase.io/rvs; my proof: https://keybase.io/rvs/sigs/zesua9Y9WeYYqsiD7SpAtepGFNXztRIKx3P1HqRnGgA ]
So on #1, if you look very casually at Buildroot and Yocto -- they will clearly come on top over this next generation of systems. It appears they have WAY more upstream components already available for you to chose from. Compared to them the list here looks almost laughable https://github.com/bottlerocket-os/bottlerocket/tree/develop... and https://github.com/linuxkit/linuxkit/tree/master/pkg The problem though is combinatoric explosion of how you can compose all these upstream components. The canonical example here is the choice of your init system. You pick one -- and your choice in everything else gets severely restricted. So to some extent that apparent embarrassment of riches that Buildroot and Yocto offer is misleading.
These next generation systems, on the other hand, don't pretend that you can build a host OS in any shape or form you want (hence very few base packages) but rather that you build "just enough of Linux to run containerd" -- the rest of what you would typically put into your baseOS goes into various containers. This is a very different approach to constructing the bootable system, but subtly so -- which I don't think a lot of people on either side of this debate appreciate.
I honestly think that what makes Yocto and Buildroot difficult is that they want to be all things to all people and they want it at the level of baseOS -- complexity-wise, this is a wrong approach these days.
That scores one point for these next generation systems in my book.
The question #2 is not even a comparison. In Buildroot most of the integration/package logic is implemented in Makefiles and usability-wise (if you're trying to actively change the system or add a new package) it falls apart pretty quickly (it is still great if you're just using what's already there btw). In Bitbake -- the codebases is REALLY complex Python which suffers from the same issue. Contrast it with Linuxkit/Project EVE where all that logic is golang and bottle rocket which uses Rust and ask yourself whether you would rather debug a complex issue with a dozen of Makefiles all full of non-trivial recipes or look over go/rust codebase (yes, I know all these things are turing complete and thus equivalent -- but life is to short to debug Makefile).
If you don't quite believe me, there's been a number of studies about using Buildroot and Yocto for building containers. Pretty much all of them came back with the same conclusion -- the usability aspect of extending them makes it a non-starter. Here's the one from the last Kubecon that VMWare guys did: https://blogs.vmware.com/opensource/2020/02/27/distribution-...
One more thing on the good side: the TUF implementation in Rust seems really interesting. I'll be digging some more and may actually steal it for linuxkit (and by extension Project EVE)
Fun fact: a lot of the patches you will find in more system level packages like grub seems to trace their lineage to CoreOS (and potentially Project EVE) but I haven't seen acknowledgments anywhere. This is of course all fine from licensing perspective -- but I still would be curious to know whether it is indeed where it was taken from.
It is also interesting to note that every step on that journey seems to have picked the coolest runtime to implement it in (C/early Go, established Go, and now Rust)
Honestly, I think #2 is the most exciting part of EVE and there’s an evolving doc here: https://github.com/lf-edge/eve/blob/master/docs/ECOS.md
EVE does have a pretty decent support for various gateways: https://wiki.lfedge.org/display/EVE/Hardware+Platforms+Suppo... but as you can see there’s a heavy tilt towards Intel-based systems (simply because they are the ones with way more existing workloads available for them).
Now, let's talk about business side of things: Balena and ZEDEDA are very different when it comes to the value add on top of the open source platform. Balena aims at being more of a PaaS solution (sort of like Heroku of that space) where ZEDEDA is clearly an IaaS/CaaS (or better yet ECasS) solution. That said, both companies have their Cloud Based portions. Balena has https://www.balena.io/cloud/ and ZEDEDA has https://zedcontrol.zededa.net/
In EVE’s terminology that zedcontrol service in the cloud is called a Controller https://github.com/lf-edge/eve/tree/master/docs#eve-controll...
At ZEDEDA we believe that we’re providing the most scalable controller to our customers, however, there’s an open source controller implementation as well: https://github.com/lf-edge/adam
Adam is somewhat new and we’re still working on making it a bit more polished so all of your pull requests will be very much welcome!
Finally, there are other architectural differences that are mentioned in the doc I linked about. Things like Balena is build on top of Yocto, while EVE leverages amazing linuxkit project (the same one that powers Docker Desktop product). I'm only mentioning this because ironically enough, while Balena was a pioneer of Docker containers in IoT space, the way the system itself is put together is still very much embedded-like.