Buildroot – Making Embedded Linux Easy | Hacker News Reader

Buildroot – Making Embedded Linux Easy

buildroot.org

I worked on an OpenWrt-derived product for some years and I found their build system quite good. If I remember correctly it's based on buildroot but they went separate ways somewhere in the past?

In any case, I found it quite easy to add new packages, update existing ones, rebuild as needed, add support for new devices, image generation, ... Also, saving the build config and used package versions to rebuild the same image later was easy. Documentation could be better (obviously...) but there are plenty examples and one could always ask advice on IRC or mailing list.

I don't have experience with Buildroot, Yocto, OpenEmbedded, ... but at least the OpenWrt build system was much better than the homegrown stuff I worked with before...

So if you're looking to build software for a Linux-based device that has some networking functionality then you might want to look at OpenWrt. If not for OpenWrt as such then perhaps just the build system.

If you don't need Linux, there's also Unikraft[1]. It's great for building tiny operating systems. It was fairly easy to port QuickJS, and having an OS that can run untrusted code in QEMU+KVM within 400ms is pretty cool (total time it takes for the OS to boot, run JavaScript and shut down in Linux running in VMWare)

[1] http://www.unikraft.org/

Linux kernel can also start similarly quickly, depending on HW.

This sounds pretty cool. A lot of the hardware ideas I have only really need a simple NodeJS runtime running. Lots of clear security benefits to this approach.

I had a look on their website and did some Googling but didn’t find anything about getting QuickJS working on Unikraft.

Do you have any relevant resources you could share?

Buildroot is awesome! I have been using it w/ Elixir/Nerves to build a small rpi powered GPS tracker. I had to get under the hood to recompile the kernel to support a usb wifi adapter and honestly it was a breeze. Really impressed with the tooling here.

>"Buildroot is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation."

Can handle everything

Cross-compilation toolchain, root filesystem generation, kernel image compilation and bootloader compilation.

Is very easy

Thanks to its kernel-like menuconfig, gconfig and xconfig configuration interfaces, building a basic system with Buildroot is easy and typically takes 15-30 minutes."

Whereas using Yocto is a long process of learning how it works to make your first build, and then learning again when you want to update some dependencies.

Afaik yocto is the go to when u need to manually specify every version and have everything in ur hands

Took my boss like 2 months to get lamp stack fully working

Or need deeper customizations / very special operating systems

idk it's pretty easy to patch buildroot to customize the packages.

At this point my go to for image creation is guix though.

Does guix build for any embedded ish devices?

Like u-boot+ custom dtb's kernel etc?

Mmh I should have a look at how guix n nixos support rpi and similar devices, maybe it's adaptable

Nix is very much customizable, at every level. It's also in my experience much much faster than bitbake and friends.

Yes, see the recent changes to the guix system image command (the info/web pages haven't been updated for it quite yet.)

This is true. Plus add debug bitbake, and debugging (or ignoring) broken caching, missing make semantic (delete a result artifact and see bitbake claim there's nothing to do) and generally causing a massive head-ache.

But ... it has all the hooks you need, and even if it's a particularly ugly and broken behemoth, you can slowly steer it roundabout in the direction you want it to go. yocto also has more apes hitting the typewriter, so in some distant future it might even be sensible.

My colleagues have evaluated buildroot vs. yocto several times, only to end up in yocto each time. I stopped challenging them: Once you have to hook into the build process in various places and implement your "around" hooks, yocto, while still sucking, and still standing in your way, and still pointing every gun there is at your feet ... doesn't make it impossible to achieve your goal without rewriting the whole thing.

Also you can make more money with using a complicated, shiny tool, of course.

NB: I'm using yocto professionally for three years now after a decade of experience in cross-building embedded targets using NetBSD. It is no joy (in contrast to NetBSD). If you don't have a problem that involves rewriting half of buildroot, choose buildroot :D

FWIW we recently started using the Yocto-based BalenaOS[0] in our project and have been rather happy with it because we get to focus on developing and running our software and don't have to bother with the low-level hardware stuff too much.

Unfortunately, as we're getting closer to shipping our product, we've realized that Balena seems to be rather cavalier when it comes to respecting software licenses. They offer pre-built .img files and Docker base images but don't provide a comprehensive list of licenses (let alone the source code) of third-party software components anywhere, as this issue[1] attests to. I'm currently talking to one of their customer service representatives and, as of this morning (when they sent me another email), they don't even acknowledge that this is an issue.

Looks like we'll now have to compile this list on our own. :\

[0]: https://www.balena.io/os

[1]: https://github.com/balena-os/meta-balena/issues/1955

I don't know BalenaOS, but if it's yocto based, using this bbclass might help you out:

https://gist.github.com/weber-martin/fa9ca5bdceffc9504175f6e...

add "extra_license" to your IMAGE_CLASSES_append in your (machine, distro) config.

add EXTRA_COPY_LIC_ROOTFS = "1" to your liking.

You will end up with a directory of licenses, including license texts, of the packages you have in your image.

Yeah you will have to build your image yourself, but isn't that $trivial with yocto (FSVO $trivial ?)

Thanks so much, I will look into this! :)

Coming from spending a lot of time working in OpenEmbedded build systems to now trying to work in buildroot has been quite painful and annoying for me. Buildroot lacks a LOT of the features and capabilities which OE and bitbake offer.

Possibly I am just not familiar enough yet with buildroot, or I just am too stuck in thinking how OE/bitbake work, but I much prefer working with OE/bitbake over using buildroot in a professional context.

Yes, OE/bitbake have a MUCH steeper learning curve once you get past the "hello world" level of examples. But once you're trying to do somewhat complicated things, I have found OE/bitbake to be amazingly flexible and easy to customize.

No you have it exactly right, Yocto is a lot less pain unless you just configure your vendor's default BSP.

I've never found vendor provided OE BSP layers to be easy to use or useful beyond initial proof-of-concept type work. In general, hardware vendor provided software has sucked greatly in my experience.

Taking the vendor provided thing and then removing a lot of the junk, or taking the upstream software project and then selectively cherry-picking some key changes which the vendor has made works pretty well. But this isn't a fast process and requires quite a bit of education about how/why the vendor has done things a certain way and how the upstream open source code in question works. But long term it's been a much better solution for the projects I've worked on when I've been able to do this.

I'm not saying vendor BSPs are easier or harder to use in OE, but that OE is a lot easier if you do extensive modifications. And especially if you have a family of related products.

I used buildroot since 2011 and yocto since 2019. Both have their strengths, and their weaknesses, but I actually prefer yocto now since it is more predictable in what it builds.

Buildroot is faster to launch and run, but I'm infinitely more confident that everything will build again after cleaning with Yocto than I am with buildroot because Yocto creates a sysroot for each recipe which contains only the specified dependencies, unlike buildroot which contains whatever has been installed into "target" by that point in the build process. This is even more troublesome when using multiple cores to build since the state of "target" is changing rapidly based on how quickly different packages are able to be built and installed.

Yocto is slow and bitbake can be a pain until you get it, but it's really an amazing thing once you understand it and can leverage what it has to offer. Layers are wonderful if you need to support multiple devices which share a common base. By using a shared state cache, I can have other developers building complete images in minutes rather than hours.

I am happy that I took the time to learn how to use Yocto, even after having a good experience with buildroot.

I concur with your technical argument, except for the trust of what's being built. I have repeatedly found the shared state cache to be the achilles heel of my builds. You never want to have a false-positive hit in your cache, yet yocto has repeatedly bit me with that. I have had instances where bitbake claimed it was done, just to find wrong or missing artifacts in my image. Cleaning and rebuilding from scratch is a continuous exercise that's IME sadly necessary. ...let alone letting others use my build directories!

I accept yocto's right to exist, but I'm far from being happy for using it.

I don't want to be the person who says "you're doing it wrong", but you probably are doing something wrong. I've been using this for quite a while and have never had to clean the world to get things straightened out, or had a false positive in the cache.

One thing that I had to do, however, was run bitbake-hashserv when running the shared state cache. Even so, this was because I was getting cache misses.

If I may ask, what are you doing when it claims to be done but things aren't in a good state? If you're deleting files / directories manually instead of using bitbake commands you're going to have a bad time.

I've had similar issues. It's rare but a nightmare to debug when it happens. One thing which consistently caused problems was OOM errors on compilers (fairly easy to encounter during a parallel yocto build). This was suprisingly effective at poisoning the state of the system (I'm not sure why this compared to any other failure was so problematic).

> Yocto creates a sysroot for each recipe which contains only the specified dependencies

It didn't used to. This was a major improvement in how yocto acts when it was added.

I've worked at a Yocto shop, and I currently work at a Buildroot shop. Between the two of them, I like Yocto much, much better.

Yocto (or OpenEmbedded if ya nasty) is a much more complicated beast, for sure. But that's a feature, not a bug. I've hit Buildroot's limits a whole bunch of times. It's messy, especially if you're trying to support a family of designs from a single repo.

Right off the bat, here are a few of the things Yocto does really well:

1. File dependencies. With Buildroot, I have to intentionally clean targets all the time. I can't just modify a source file somewhere and expect Buildroot to know what needs to be rebuilt when I run `make all`. Yocto has really good file-level dependency tracking, and will almost always do the right thing if your dependencies are correctly expressed. It's hard to include a source in Yocto without also tracking it in the dependency graph.

2. Hardware variants. Buildroot's kconfig system is nice for managing a single rootfs for a single piece of hardware. But as soon as you want to share configuration across multiple boards (some with different packages), it kind of blows up. All of a sudden, you're using a preprocessor to generate your defconfig from fragments, and you can't reliably use `make savedefconfig` to guarantee that you aren't missing package/config dependencies. And you also can't use `make menuconfig` any more, which is kind of the whole point of using kconfig in the first place.

3. License tracking. Yocto is really strict about licensing. It forces you to specify the license of every package, and also forces you to specify a file that it can check to make sure the license doesn't change unexpectedly. When building an image, you can easily generate license reports for every piece of software in the rootfs. You can also do things like forcing it to exclude GPLv3 software.

4. Toolchains in the rootfs. For some dumb reason, Buildroot doesn't give you recipes for building a compiler toolchain and including it in your rootfs. This is trivial in Yocto. If you support developers that like to work on-target over SSH, it's a big limitation for Buildroot.

5. Scripting. If you want to do something fancy in a package recipe (or in an image recipe), it's way easier in Yocto. You can run arbitrary Python code at parse-time, and modify your build's internal metadata if you want. Each recipe can also have arbitrary build steps that you can trigger manually if you want (or hook into the dependency graph).

6. Layers. Yocto lets you put all of your customizations in their own folder, which can be in a separate folder from the upstream sources. You can extend, override, or modify the behavior of any upstream recipe, and source-control your tweaks without having to vendor your own copy of the upstream. This is really nice for upgrading Yocto releases, because it's usually as simple as "swap out the upstream submodules, then tweak your custom bbappends (if any)". You can have as many layers as you want, and you have full control over their evaluation order. Lots of hardware vendors distribute their BSPs as a Yocto layer.

7. Packages. Yocto builds every package as a .deb (or a .rpm, or a .ipk depending on how you set up the build). This makes it very easy to build a package on your dev machine, then install it on-target if you're experimenting with some new package or you want to install $tool one time for a test.

Add in the shared state cache as #8. It makes it very easy to bootstrap new developers since there's no need for them to rebuild glibc, etc, when they are mostly interested in adding new recipes or customizing images.

yocto and bitbake are basically slightly earlier and less formal versions of the same ideas as in nix and nixos, and they do suffer a bit from that. Given that, I would actually be very interested in a version of nixos focused on targeting embedded systems.

If there is one main reason Yocto very much won over Buildroot in production use, it's the ease of managing dependencies.

(started with crosstools in mid-00s myself, used buildroot for better half of a decade)

Buildroot is much easier to get started than Yocto, I agree. You can hit the limits pretty quick with Buildroot though, and then things get ugly pretty fast.

Yocto has a much steeper learning curve. But the tradeoff is that you get incredibly precise control over all aspects of your build. Yocto builds are also much more composable, making it easy to have a "lots of variants that share some common configuration" kind of setup.

What's particularly neat about Buildroot is the support for reproducible builds.

I have an old project for building container images using buildroot ( https://github.com/br-containers/br-containers ) that I need to pick back up soon. Highly reproducible redis server images weighing in at under 10mb, with your choice of libc under it.

3.7M output/images/rootfs.tar

sorry, redis on uclibc in under 5mb, and no commits to my repo in 3 years while continuing to build against buildroot/master.

Reply on news.ycombinator.com