Hacker News Reader

Comment by xyzzy_plugh | Hacker News Reader

Are you proposing compiling on the target? For a vast number of embedded systems that is not only impossible (insufficient disk and memory) but also incredibly slow.

At the end of the day you need some cross compilation just for board bring up.

If you're playing with some platform for which this has already been done, then sure, but that's not really the "normal" way of doing embedded.

View on HN

Embedded just means ARM 99% of the time and it's cheaper and easier to use native ARM servers (AWS has them cheap) than to make 100% of software cross compile. Some parts of the firmware might need to be cross compiled but those projects are designed to cross compile.

Your target build environment often differs significantly from your server environment, so you end up needing a different toolchain and all of the problems that come with cross-compilation anyway. The toolchain and ABI settings that produce small, battery or instruction cache efficient are usually not you want on servers.

I'm not familiar with the problems that would cause "target build environment often differs significantly from your server environment". If you have an ARM server, then you should be able to download and run Alpine docker images, and then fetch most system utilities from the Alpine package repo and then compile a kernel and a few other binaries on top of that, then export an image.

The main reason to not run Debian is that Debian usually makes lots of compile-time decisions for you, and chooses to maximize the enabled options of the software. Alpine makes the opposite choices for you, creating very minimal feature sets.

It's been working well for me, but my use cases is a lot more like a custom in-house DDWRT firmware kind of a thing. If you've experienced additional constraints that make this pattern unworkable, I'd be interested to hear about them.

My toolchain for both is managed by the Nix derivation wrappers. It’s trivial to set custom C/C++/Rust/Go/NodeJS flags. As long as I have one flexible build tool, I can customize it for both server and embedded.

Run on server, chroot into target environment, configure compiler flags for eventual target, done. It's really just an afternoon of setup and then the problem is solved.

I'm talking about cross building an image, and flashing an entire image onto the target. This is the purpose of environments like yocto and buildroot.

My preference is to cross build packages for each application on my workstation, and maintain a local package repository from which individual packages are installed on the target.

I find archlinux extremely useful in this configuration, because even if I'm not building on the target, the same package maintenance and other system commands are identical to my development workstation.

Additionally, archlinux provides a well documented, and fairly simple process for hosting your own local repositories.

There are several ways to compile for the target:

1) Cross compile. Yoe build still does this where it makes sense. (Go apps, likely kernel builds in the future) 2) QEMU user mode - 5-20x slow, but fine for some things. 3) Run yoe-build on any ARM machine (AWS, rPI5, Jetson) 4) Farm unit builds out to runners on cloud ARM machines. (future)

The yoe-build architecture allows for this. Choose the container, host architecture, and location that makes the most sense on a unit-by-unit basis.

Caching is also part of the vision, so we never rebuild something twice, unless it changes.

Doing this on target seems terrible. What an awful development experience.

Agreed, building on the typical target is a terrible idea. None of my suggestions are to build on target, but rather specific units on some capable hardware of the same architecture. The main build will still run on your own workstation of any architecture. Only specific units get farmed out to another machine, but worst case can be run in QEMU user mode emulation if there is no native machine runner available.

This remote build runner is not implemented yet, just ideas.

Reply on news.ycombinator.com