Build Initramfs Rootless | Hacker News Reader

Build Initramfs Rootless

blog.izissise.net

108 pointstodsacerdoti46 comments

If you want devtmpfs mounted automatically by the kernel within an initramfs, then you may need to patch your kernel slightly, like: https://lore.kernel.org/lkml/25e7e777-19f9-6280-b456-6c9c782...

By default the kernel will not mount devtmpfs automatically within an initramfs, even if you've configured your kernel to automatically mount devtmpfs, because it waits until the real rootfs has been mounted to do it. This is fine unless you never plan to actually mount a real rootfs as you've made your initramfs have everything a normal rootfs would have and you want devtmpfs to Just Work.

Thanks--you just saved me an hour or two of debugging. I thought, CONFIG_DEVTMPFS_MOUNT=y, what more do I need to do? I should've just RTFM though: even an unpatched kernel mentions this in the help for that config option.

It's also easy enough to just add a mount command to your initramfs init, or embed required device nodes in the CPIO.

It is, unless you're just trying to use an off the shelf distro as your initramfs and that distro's default init choice makes the assumption that the kernel will mount devtmpfs for it.

My understanding is that pretty much every modern init system has a service for mounting the devtmpfs, even if it's not in the fstab, and depending on the switch_root binary being used, mounts made in the initramfs may or may not be re-mounted in the new root.

You don't even need a devtmpfs in the initramfs environment, and it may or may not even persist into the fully booted system. I believe Systemd has a tmpfs unit file, and OpenRC has a service for it which is enabled by default.

You don't even need a devtmpfs in the initramfs environment, and it may or may not even persist into the fully booted system.

In this case, my point is for a system where the "fully booted" state is still within an initramfs. There is no "new root" past the initramfs, it IS the rootfs which will run forever until the system is rebooted or shutdown.

You can just mount then exec the real init, no?

I wish that there were some more tools to build archives based on script-like description. E.g. "i want this file under this path with this metadata, go build me a zip".

Without it, my favorite way of building images rootless is still fakeroot. fakeroot + mknod + cpio can fully replace gen_init_cpio

The python `zipfile` module (`tarfile` likewise) is fairly straight-forward to use for this use case (just with a literal script instead of a script-like).

Using the zipfile/tarfile/etc. modules' functionality, it's easy to write a script that would take an input file of any format you want (such as the one in the example) and create an output file the same way it works in the article.

It also allows for you to create archives compressed with gz, bz2, lzma, xz (via lzma), or potentially any other option that Python can provide.

I did consider writing such a script as an example, but it occurred to me that just using ad-hoc imperative python scripts might be simpler/clearer (i.e. do we actually benefit from using a "DSL" here, even if it's a trivial one?)

bsdtar can do this! It supports reading an mtree file (a text description of a filesystem or of the contents of an archive) as an input and producing ZIP as an output!

Given an mtree file foo.mtree:

    #mtree
    baz/bar type=file content=quux

You can produce a ZIP containing baz/bar with:

    bsdtar -c --format=zip -f out.zip @foo.mtree

There’s a much better explanation in the bsdtar manual at https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&f...

If you’re on Windows, “tar.exe” is bsdtar. With the caveat that the mtree file should be saved with LF line endings, this should still work.

You can use podman and skip that dance. Just install gen_init_cpio from a rootless container.

It seems like podman needs root to enable containers to run mknod.

You don't actually have to make a device node, you can just create it directly within the CPIO archive.

Here's my old Python CPIO writer. It's not very maintained right now and it's probably missing some useful file types, but it works fine!

https://github.com/amluto/virtme/blob/master/virtme/cpiowrit...

Yes please, and for archive types like ISOs and disk images (where there may be a partition table in play as well).

Debian has jigdo, but the recipe/template for it is derived after that fact by scanning the ISO and then blanking out the parts of it that match external files— there's no mechanism for authoring them pre asset creation.

Here is a tiny containerized and deterministic Linux build process with a minimal init system in rust for secure enclaves:

https://git.distrust.co/public/enclaveos/src/branch/master/C...

And another one for a simple busybox based ISO for airgap operations:

https://git.distrust.co/public/airgap/src/branch/stagex-rewr...

Making Linux images for server/appliance use cases is always more secure and almost always easier than trying to adapt workstation distros like debian. Lots of ways to do it depending on your tooling preferences.

Why does the first example need 10GB of RAM?

It should not need anywhere near that much. Maybe some defaults in your docker setup, or a docker-machine VM is in play?

My bad. I misread the README. It's development requirements not normal user requirements.

A good tool for creating images, also initrd images: http://mkosi.systemd.io/

Just use bsdtar, which will convert a tar archive to cpio: https://unix.stackexchange.com/a/581014

I have used it to convert docker images to Linux bootable initramfs archives for Rpi4.

Do you have link to that conversion? I wanna use containers to provision the pis, such a pain.

I've had good luck making environments in a qemu user chroot, then you can more or less just copy the whole filesystem to a SD card and boot it, considering the boot partition of course.

You can also roll the same CPIO archive into your linux kernel by going into the build system and specifying that you want the initramfs to contain the files in the gen_init_cpio list referenced here. I forget exactly which flags do this but I use it all the time right now.

Just point CONFIG_INITRAMFS_SOURCE at a folder or cpio archive

This kind of thing always makes me want to build a programming language with monadic file IO where the FS actually gets logically copied and modified at each write (but maybe this can get optimized away by a compiler).

I've been working on this for the last year or so: https://github.com/desultory/ugrd

It's mostly made to work with Gentoo (uses pax-utils for finding dependencies), but should work with any distro. It does most of the initramfs creation process rootless. The main consideration is that you need privileges to make device nodes, but I just create them synthetically within the cpio image.

On most systems, this tool still requires root to copy the mount binary.

This is right up my alley, I’d love for a more programmatic way to generate initramfs. Being able to template different use cases would be great for specific appliances. One of my projects aims to create a hackable minimal Linux distro, and thinking about extension has always been on my mind. https://github.com/snacsnoc/snacklinux

Have you found any specific niches that scratch an itch with your tool?

    printf '%s\n' '#include <stdio.h>' '#include <unistd.h>' 'int main() { printf("Hello world!\n"); sleep(999999999); }' | gcc -fpic -static -xc - -o init

This is completely irrelevant, but that line reminded me of the old “real programmers write device drivers with cat” joke. I guess even realer programmers would write device drivers directly into gcc stdin.

Are there FreeBSD/NetBSD equivalents to do the same without creating a UFS volume:

- programatically declare the /dev/ files and their permissions

- link the binaries you want to use

I've learned about crunched binaries (like busybox, but in reverse), so I wonder if there's a BSD way to do that.

https://duckduckgo.com/?t=ftsa&q=mfsbsd ?

There have been others in the past, but this seems to be the most polished and ready to use ATM(for FBSD).

Another would be NanoBSD (also FBSD).

For NetBSD you're on your own, starting from there https://wiki.netbsd.org/tutorials/how_to_create_bootable_net... , and/or asking on https://daemonforums.org/ , https://www.unitedbsd.com/ (taking inspiration from some 'live-distro' discussed there. like 'OS-108'), reddit(?), 'crap-overflow', and even https://www.linuxquestions.org/.

Of course you're free to use the official NetBSD mailing lists, and some obscure IRC-channels in even more obscure IRC-networks also :-)

(You won't be spoon-fed, and are expected to have read the manuals and other documentations...)

I'm very new to BSD in general, but I find it very fun and interesting!

However, I need pointers to get started.

> You won't be spoon-fed, and are expected to have read the manuals and other documentations...

I read a lot of FreeBSD and NetBSD documentation to get to the point of compiling my own kernels, but I don't think I ever read about the equivalent concept of Linux cpio/initramfs for BSD. My minimal images use a UFS filesystem.

Here, after checking https://mfsbsd.vx.sk/ and https://github.com/mmatuska/mfsbsd/blob/master/scripts/mdini... I think mfsbsd is just a using tmpfs so it may not exactly the same thing as initramfs, that allows booting linux from a bzImage + initrd

I'll keep searching, it's not super high priority at the moment, but it's something I'd like to do with (Free|Net)BSD.

If you're coming from Linux some terminology may be uncommon, different, or not even there, because equivalent mechanisms don't (or didn't) exist. So there's a different oral history, which makes it harder to get instant search results, besides 'BSD is dying', which leads to less results in general, because it's simply less used.

Back to topic, does it matter if it's using different mechanisms, if achieving the same result at the end of the day? Like running in RAM, and not necessarily having to be 'installed' on some medium, being very minimal, and optionally 'rebooting/remounting/pivoting' into something larger, which may also reside in RAM?

That OS-108 may have been misleading, because it seams rather dead, but I thought it did something like that, eons ago, when I looked at it in emulation.

https://github.com/OS108/OS108/tree/master/LiveImgScripts

Anyways, it's based on this https://github.com/tsutsui/netbsd-teokureliveimage , which isn't, most relevant would be mkimagebuilder.sh and mkliveimage.sh from there, I guess. If that doesn't get you further than your own minimal images, I've understood you wrong, and probably wasted your time.

Editorialized title.

Kind of necessary, a post simply titled Initramfs wouldn't be useful.

But isn't the article about building initramfs bottom-up, presenting the most basic tools?

I did not see anything special that would make it rootless. That seemed to be a point made by the submitter.

The article itself says:

>Note this is rootless, we don't need to re-create the filesystem tree locally or use fakeroot to create special file in the archive.

Sure it does. But that's not because the article shows us any new trick, it's just because they use the original and most basic tool to generate an initramfs.

gen_init_cpio is not the original cpio archiver utility. It's only obvious to know to use it if you already know about it. The Linux kernel docs cover it, sure, but the whole point of the OP is a more approachable introduction to the same information.

The article says it is but that is potentially misleading. Building an initramfs is very simple. Copy all the files you want in it and make a cpio archive. Arch's tool for doing it is just a 80 line or so bash script. In principle, it doesn't matter what user does this but only if all the files you need are readable by a non-root user.

You can't make device nodes as non-root.

You can create any data as non-root, including the description of a device node in a cpio.

But the point is that the tooling to do this is non-obvious. The same issue exists for all these kinds of archives— container image layers, ISOs, VMDKs, debootstrap outputs, etc. All of them are things that should be possible to just emit directly into a user-owned archive, but often require mounts, overlays, chown, and who knows what else.

Another related one is about installing grub into an archived filesystem or disk image. Should be trivial, right? But most instructions for this require mounting it, which immediately means you need SYS_ADMIN and therefore a privileged container in your CI setup.

You can't make them on the filesystem itself, gen_init_cpio lets you specify device nodes to be created within the archive, but they are not actually created on the live system and do not require privileges.

I made my own CPIO library: https://github.com/desultory/pycpio and one of the main goals was this behavior, so I could create CPIO archives with device nodes without privileges. It's really as simple as writing some bytes to a file saying "when you extract this, create this device node". It's not like any archive actually has a device node, just instructions for where to put them, and what properties they have.

Certainly, you can't extract these archives as non-root, but you have to go out of your way to make an initramfs have non-root privileges.

Reply on news.ycombinator.com