Debian putting everything on the /usr | Hacker News Reader

Debian putting everything on the /usr

theregister.co.uk

In my opinion, this is getting rid of a minor historical relic, instead of the major relic of Linux booting. That ugly hack known as initramfs should have been obsoleted at least ten years ago. It was useful back when you had to worry about keeping all your boot files below the 2gb mark on a disk, but these days, all it does is create a second, ephemeral, userland in a more difficult to debug container. Put the various boot binaries into /bin and /sbin, statically linked, and completely visible to even the running system. Cleaning relics should start with getting rid of entire filesystems that just make it harder to administer a system.

initramfs isn't (and wasn't) at all about keeping your boot files below the 2gb mark on a disk. It's about actually mounting your root filesystem, either on a fully-modularized kernel where all of your (hundreds of) block drivers are in modules, or where root is coming over a network block device, or a USB disk, or a SAN, or a complex RAID, or an encrypted disk where you need to enter a password or use a hardware token or whatever.

Basically, rather than encoding tons and tons of crazy special-purpose logic into the kernel to mount every insane iteration of root filesystem, you just get the bootloader to stick a CPIO image in RAM for the kernel to get at and then use the normal userland tools to do what you want.

IMHO, initramfs is the opposite of an ugly hack.

Though I take your point and mostly agree, is there any reason this couldn't be done with a static boot binary entirely in RAM? Since the kernel is modular anyway, much of the special purpose cruft loaded by the initramfs could just be loaded and unloaded with a boot script of some kind right? I mean: it could all sit on the same filesystem without having a CPIO image on disk? Grub supports pretty much every filesystem already so loading this initial kernel image from the network or wherever could just start there.

Mounting root file system might nowadays conceivably involve some non-trivial userspace code that never even was implemented in kernel space. Few examples:

- Searching for root filesystem based on it's contents (in simple case UUID, but mechanisms like "contains this file" are used sometimes) instead of "physical" location - Encrypted root filesystem (requires some kind of UI for entering the passphrase) - LVM2 (which is set of userspace utilities that configure more general kernel modules that know nothing about the actual on-disk format) - while there is slightly hackish kernel support for configuring network interfaces before having userspace and mounting rootfs from NFS, such thing cannot possibly deal with all variants of network configuration (eg. VLANs, 802.1x, ...) and possible network filesystems/block devices. - root on ramdisk, copy-on-write overlay filesystem or something like that.

To summarize: it's entirely possible to have the system boot of PXE, authenticate via WPA-EAP on wifi and fetch it's own root filesystem into ramdisk via HTTPS while storing writes onto NFS somewhere. Implementing all which is required for that in kernel space is neither possible nor useful.

By the way, the point of initramfs is that it is loaded into memory by bootloader using whatever means the bootloader has and the image does not have to be on rootfs accessible to kernel (for PXE it typically isn't).

Ok. That's a great summary and good point about non-trivial userspace bits. I hadn't considered how complex "booting" as a process has become...

Most initramfs designs are based on shell scripts and use tools like `mount -t ext4`, cryptsetup, etc - normal userspace tools, tied together by shell scripts. What you're suggesting is actually more special purpose than what we already have. Modern initramfs design is really great and not crufty at all.

Wait are you suggesting that something tied together with shell scripts is great?

Oh yes. Bash is definitely not a good general purpose programming langauge, hell I'd hesitate to even call it a programming language. However, it's very much so the right tool for the job in an initramfs. How familiar are you with the early boot process on a modern Linux distro? Ever had to do it manually from an initramfs rescue shell? Well, if not, it basically just involes invoking a bunch of commands and minimal parsing. Most early boot configs like /etc/fstab are easily readable with awk and are well suited to handling with shell scripts. All you basically have is a stripped down userspace with coreutils plus the commands necessary to prepare the system for the full boot. Shell scripts are the natural choice for tying it all together. I actually really like bash in general, in the niches it's suited for it's very nice.

> However, it's very much so the right tool for the job in an initramfs.

Do you remember Erik Troan's nash ?

Not flexible enough. Bash is definitely the tool for the job in a _modern_ initramfs.

There are various RAM based distros in which the only file system is initramfs.

Makes no sense to propose getting rid of this.

I don't think the argument is as much about getting rid of it in the kernel, as trying to avoid yet another case of "i have a hammer, thus all my problems are nails".

But then the linux "community" is rampant with this line of thinking. just look at the slow moving recreation of "containerize all the thing!!!" meme.

Definitely.

If ever I have a hosed Linux machine, the wrangling with systemd, initramfs, grub2, mdraid and all the associated shenanigans fills me with dread. Sometimes it's easier just to reimage and restore.

Free/OpenBSD get this 100% on the mark.

initramfs is also about keeping userland tools, like LVM out of the kernel. But to run the LVM tools you need access to a filesystem, which might be on a LVM. And there are quite a few other scenarios too. But if you can do without, be my quest, less things that can fail.

What's the point? Why not do it the other way around and put everything in /bin? It almost feels like /usr is the new root.

1. Less disruption. Most software is already installed to /usr.

2. Installing all software to /usr means that /usr can be made read-only when not installing/upgrading software. This isn't possible if installing to the root, as directories like /etc and /var have to be left writable.

Never heard of /usr/etc and /usr/var ...

/etc doesn't generally have to be writable, and in a read-only root environment /var will be on its own slice.

Do you mean the partition containing /usr will be mounted ad read-only?

He said can be made, that's an option.

Ah, thanks for the quick explanation.

So, I'm of two minds on this:

On one hand, I'd go farther. I like apt, but I wish that distros would use the same packaging system and same package format. There is little value to them being different between distros, it just causes more work for repackaging/deployment/config. Even distros that build everything could all do that the same way, so any distro and its apps could be fully built from source, if the source is available.

On the other hand, this isn't great. Having variety in paths, although it's basically security via obscurity like using non-standard ports, is still an additional level of security. And think of all of the existing config/repackaging scripts/etc. that will fail.

But, I'm not an admin nor in devops, so I'll defer to those that actually are impacted...

On one hand, I'd go farther. I like apt, but I wish that distros would use the same packaging system and same package format

I agree however that's rather off topic. This is not about harmonising between Linux distributions. The fact that other distros did it is merely used as an argument to show it's possible and without any major issues for anyone involved.

Having variety in paths, although it's basically security via obscurity like using non-standard ports, is still an additional level of security.

No, it isn't. The way you find binaries (when paths aren't hard coded) is by looking through the $PATH environment variable. Both /usr/bin and /bin are part of that on any Linux distro so there is no obscurity or any kind of security gained from this. Though the advantage of having everything in /usr/bin is that /usr can be mounted read-only meaning an attacker will have a much harder time modifying the binaries initially (unless they can do some local privilege escalation and remount the partition).

And think of all of the existing config/repackaging scripts/etc. that will fail.

They won't. Even those with hardcoded paths to things in /bin won't be affected since usrmerge will provide the symlinks for those binaries. And anything properly written will be searching for binaries through $PATH and be equally unaffected. There's a few crucial tools that will likely need to be updated to be aware of this change but the Debian project will take care of those.

You seem convinced that $PATH handles everything. That's not accurate. Deployment scripts can make automated changes to config files and need paths. One package assumes a hardcoded path for another package- typically a symlink, but if those symlinks are moving around, that could break.

And the package management system isn't off-topic if one of the reasons for moving paths is to become more standard. Apt, Yum, Pacman, etc. all have different ways of doing things, and if the goal is to become more standard, then you have to wonder why the Linux community has not banded together on a single packaging system standard.

Deployment scripts can make automated changes to config files and need paths.

And proper deployment systems can handle this. Or it's templated and can be changed with a single variable. We're also not talking about every possible binary ever, we're only talking about /bin which is a limited and well-known set of things, for which symlinks will exist. It will not affect any piece of software ever and associated configuration files, the blast radius is very small and well understood.

And the package management system isn't off-topic if one of the reasons for moving paths is to become more standard.

That's not the reason. The reason the move is happening is because there is no real point in having /bin and /usr/bin separate. It's a historical relic which offers very little benefit to users and is largely confusing to newcomers. Debian isn't making the move to standardise with what other distro's are doing, they're doing it b/c it makes sense for their users. It just so happens that others have come to the same conclusion and since they've made that change before we can learn from how and why they did it.

then you have to wonder why the Linux community has not banded together on a single packaging system standard.

You should read up on how dpkg and rpm were created, and why every other package format and package manager ever has been too. Besides the history, every single one of them has their flaws and issues. If we had one that was perfect, that solved every single issue related to package management ever, in a platform independent way that we can all agree on I'm pretty sure everyone would be using it. I'm all ears for proposals though, but so far any time this has been attempted it hasn't gotten anywhere. And this is a completely different domain than moving /bin around.

I think you're going to need to explain how /bin and /usr/bin provides more security than just /usr/bin. I don't see how it is at all like changing ports for services.

Packages and scripts will not (often) fail because they're adding symlinks from the old locations to the new. Other distros have already made this move, without anyone really noticing. It'll be fine.

When you look into a common packaging format, it turns out that what you really want is common packaging policies. The packaging software is not relevant when you try to install something for one Linux distribution (be it Debian, Fedora or possibly Android) on another where it doesn't hook into configuration scripts, enable services, set up firewall rules or desktop icons as the user expected. Whereas something like Debian "just works" for the end user.

Even with apt's format, distros have different packaging philosophy. Arch doesn't split doc, headers and binaries IIUC.

I wonder if there would be a way to harmonize things cleanly from source.

Reply on news.ycombinator.com