Hacker News Reader

Comment by cbrake | Hacker News Reader

I’ve been experimenting with what a next-generation embedded Linux build system might look like: native builds on the target architecture, modern language package managers as first-class citizens, and AI as a primary interface to the system.

Instead of cross-compiling with a large meta-layer stack, the tool builds kernel, rootfs, and applications together using one engine, with a CLI, TUI, and AI assistant talking to the same core. All you need is the tool, Docker, and Git — no global SDKs or hidden state.

It’s pre-1.0 and rough around the edges; I’m sharing it early to get feedback from people who live in Yocto/OpenEmbedded, Buildroot, Nix, etc. I’d love to hear where this breaks on your boards, what workflows feel wrong, and whether the “native builds + AI-aware build graph” direction seems promising.

View on HN

>native builds

This is the complete opposite way, actually.

We need cross-compiling that is just as effortless as native compilation.

You should be able to build complex software on a powerful computer and perform costly optimization, then run it on a low-powered device.

Yocto and Buildroot will compile, from scratch, an entire gcc crosstool chain with standard library suite and headers to build fast and then deploy to your target. This exists.

I wouldn’t call either of those effortless, though. I’m thinking of something similar to Zig’s DX for cross-compilation.

zig makes cross-compiling easy for a single project - but not everything is zig (or supported by zig cc)

the various embedded build systems make cross-compiling easy at scale, which means you often don't need to think about it when adding a new package

It was true in 2005, but still? As I described in another post here, a modern strategy is to get a beefy server than can run the same ABI, then start a docker container and assemble a system from Alpine's package repo, then compile a kernel and a few in-house things, then extract a subset of that into an image. No cross-compile, and most of the useful software is in pre-built binary packages with the compile-time options you would have selected anyway.

Even if you don't have a beefy server of the same architecture, you can probably run it in qemu instead of docker to the same effect. And even if qemu is slow, you can run a build of the kernel and your in-house stuff in parallel on 64 cores and not really be affected by the qemu slowdown.

I'm interested to hear counterexamples, though.

I don’t have counter examples, but hear me out.

What if this hegemony in ISAs is partially because of the high costs of porting software and toolchains?

Maybe the next generation of embedded devices will have ISAs that are, say, extremely optimized for real time industrial controllers, but terrible at compiling C++.

It seems a little silly to give up on cross-compilation just because _today_ everything runs on ARM.

Especially because cross-compilation is not a particularly hard problem, the problem is with decades of tooling assuming the build and target environments are the same.

Well, I'd say cross-compiling is at least a little problematic... if nothing else, it is a hurdle to running automated tests. And cross-compiling still needs to exist for the people building the package repositories. I'm just sort of advocating that if you have limited amounts of time but want to put together a system image to run in an embedded or embedded-adjacent manner, the least pain / most agile solution is by building on top of a popular binary distro and then using native compilation on appropriate hardware inside a chroot or container that resembles the target.

There are no beefy RISC-V servers. There are no beefy 32-bit ARM servers.

Beowulf time?

I had been maintaining a fork of the U of KY BDR with SBCs and Arm devices in mind, might need to revive it for RISC-V.

> There are no beefy 32-bit ARM servers.

"Ampere processors natively support both 32 & 64 bit Android applications and require no binary translation for maximum instance density" ... "Run up to 120+ 3D cloud game instances per socket" ... "DRAM: 384GB - 512GB"

https://amperecomputing.com/solutions/arm-native

I've not personally tested this, but it very much appears that just like x86, you can use a 64-bit ARM kernel that has 32-bit support enabled to run a 32-bit userspace, likely compatible with docker, or even just in a chroot. Then with 120 cores, "make -j 120"

> It was true in 2005, but still? As I described in another post here, a modern strategy is to get a beefy server than can run the same ABI

I’m already sitting at an expensive computer, if everything else were equal I’d rather cross compile than buy more hardware.

I’m open to reasons that’s presently too hard of a workflow to enable so I’m (literally) paying for a penalty, but I don’t see how it’s a better idea.

Armbian and Qemu worked well for me when I needed to compile packages for an orange pi without enough RAM to actually run cargo build. Built the image on an emulator with more RAM then the target system, dropped the customized image on the SD card and booted right into the entry script.

I wish you luck! The pain points you identified are definitely real and solving them would be valuable.

The workflow for user space can definitely improve some of this pain but I feel like a large portion of any embedded Linux development effort still ends up in the weeds for boot related items (secure boot, proper updates, nuanced kernel patches, bootloaders, device trees, and supporting machine variants, etc). Solving those to make them easy is a hard problem for sure.

(For the unaware, OP above is the article author.)

As a long time user of Bitbake and Yocto, and watcher of Yoe Linux... I kind of have to respectfully disagree with the approach.

One look at the Python or Node packaging ecosystem, and you can see how difficult trying to integrate those in a 'sane' way in embedded, with repeatability and security in mind, nevertheless wrangling something like C/C++ dependencies and native packages in a qemu-cross (or binfmt emulation) environment.

I feel like Bitbake and the wider Yocto ecosystem has essentially 'solved' cross compiling. Sure there are the incredibly complex codebases like Chromium that require lots and lots of study and patience, or esoteric compilers, etc. but for most applications I feel that especially AI tooling can write up a good basic recipe for integration with Yocto.

The unfortunate thing about AI tooling is Kernighan's law (where debugging is twice as hard as writing it the first time.) Especially in Embedded Linux, where 99.99% of the product code is written by others, trying to figure out where the AI didn't quite get it right or missed something can break things unexpectedly and impossibly for the unaware developer to fix, even breaking at runtime. So the build environment has to be simple and predictable. I think Yocto/Bitbake already strikes a good balance here, with other nice things like offline builds, repeatable builds, sstate caching across machines, PRbuild servers... things useful for big teams, so consider those as well.

Although, if someone does a 'uv'-like rewrite of Bitbake into Rust with the same feature set but faster, I am all for it...

Maybe, having Yoe or some other centralized distro make specific opinionated choices and then provide an open sstate feed, working inside Bitbake, could get a lot of the speed and customizability gains back without a big rewrite effort.

Native builds on target can be very slow?

Reply on news.ycombinator.com