Distri: 20x faster initramfs (initrd) from scratch

initramfs is essentially a tiny Linux operating root image that Linux boots to RAM, before it goes on to boot from hard disk.

You can actually configure Linux to never go onto the next step of booting from hard disk, and run the operating system purely from RAM. Indeed there are distros such as Tinycore Linux that are designed exactly this way which can make things very fast compact and secure and diskless.

After more then 10 years of PXE booting most systems, I personally can definitely say it is rock solid, reduce maintenance times, and makes scaling simple.

>"How will our userland program know which kernel modules to load? Linux kernel modules declare patterns for their supported hardware as an alias, e.g.:"

  initrd# grep virtio_pci lib/modules/5.4.6/modules.alias
  alias pci:v00001AF4d*sv*sd*bc*sc*i* virtio_pci

  initrd# cat /sys/devices/pci0000:00/*/modalias
  pci:v00001AF4d00001005sv00001AF4sd00000004bc00scFFi00
  pci:v00001AF4d00001004sv00001AF4sd00000008bc01sc00i00

Could someone elaborate on this a bit? I'm familiar with udevd. Is /sysfs enumerated first and then a match is searched for in lib/modules/5.4.6/modules.alias? Or the other way around? Are these files parsed before normal kernel PCI bus enumeration then? Any other specifics or detail would be greatly appreciated.

It's not 20x faster to boot?

It gets even worse when you enable debug. It's a series of really nested bash scripts, and enabling debug does "set -x" for everything. That spews out so much information over serial console / screen that actually tracking down the bug is almost impossible. The sheer complexity of the embedding and sequence of events means that even innocuous changes can have a way bigger impact than expected. RedHat changed something between RHEL7.6 and RHEL7.7 and suddenly we were having all sorts of issues with CentOS7 instances booting off iSCSI.

The one big positive about it (from my perspective at least) is that it _does_ make it pretty easy to inject additional functionality in to the initramfs. Just drop in a bash script with a number prefix relevant to where you want it to occur in the boot order.

I can’t remember the last time I sat in front of a Linux box during reboot. Is that a pittance or a noticeable improvement?

this means that an optimized initramfs creator ought to take as long as a tar file creator, because it basically is a glorified tar file creator. on my system, with a full cache, mkinitcpio takes about 1.5 seconds to run. my custom initramfs generator (not released yet) takes 0.03 seconds to run using "cat" as compressor. it is written entirely in POSIX shell and uses only the external commands ldd, gen_init_cpio, and the compressor. however, if gzip -9 is used for the compressor, then compressing the 12 MB takes 1.8 seconds. so, we can see that compression can significantly inflate (pun intended) the time consumed. as a matter of fact, looking at https://github.com/distr1/distri/blob/master/cmd/distri/init..., it appears to pass no options to gzip, so I assume the default is used, probably -6. http://man7.org/linux/man-pages/man8/dracut.8.html indicates that dracut uses gzip -9 by default, which would increase the time required. if dracut is configured in a non-default manner (e.g. lz4 -12) that would further increase the time.

the author credits Go, concurrency, no external dependencies, and threads (?!) for the improved performance. this is manifestly unnecessary: no threads are needed to improve the performance of "tar", firstly because it is I/O bound, and secondly because the compressor is usually the slowest part anyways. it might be possible to slightly improve the performance by using native code, but there is no substantial difference between running mkinitcpio in 1.0 vs 1.5 seconds, and there's definitely no difference between 10.0 and 11.5 seconds (lz4 -12 takes about 10 seconds to run, and I assume the remaining 1.5 seconds can be fully optimized to require 0 seconds).

regarding the rest of the post, it's... a little weird. I don't understand why anyone would manually trudge through modules.dep files when that is the main objective of the "modprobe" command, as opposed to "insmod", which does no dependency resolution. (modprobe also supports config files, aliases, and some other things, but the main and original objective is dependency resolution.) it's also a bad idea to reimplement libblkid: it supports a ton of filesystems, many of which one might actually want to use as a root filesystem, but are not supported by this basic implementation, including xfs, btrfs, or zfs. it also doesn't appear to support LUKS2. libblkid isn't even necessary: busybox findfs works perfectly well for most common filesystems, and runs very quickly (0.02 seconds on my machine). but, again, the main cost when booting with a necessarily cold cache is likely to be I/O, not spawning a process. I don't understand why someone would faff about with manually parsing ELF files when ldd works just fine with simple text parsing and handles interpreter, rpath, and transitive dependencies built-in.

this all just seems overly complicated to me. my initramfs generator is 77 lines of code, including automatic ldd dependency generation, and init is 37 lines, including LUKS decryption, dropbear for remote password entry, and e2fsck with automatic reboot if requested by e2fsck, for a total of about 110 lines. I didn't count the lines of code for minitrd, but adding together the two files linked is already about 1000 lines of code. mkinitcpio has far more flexibility and is only about 1500 lines for the core code.

Also, I think that people's efforts should in general be spent on rallying behind dracut and making it work faster, thereby benefiting all Linux distributions that use dracut (increasingly more)."

As described on the website unixsheikh.com recently posted to HN^1 and making the front page^2, I think this illustrates one difference between "Linux" and BSD. Much of what unixsheikh describes has been said before however I think it still remains true today. I am currently using Linux and open to being persuaded otherwise. I want to learn "the Linux way" if there is such a thing.

That said, I always found booting to ramdisk, "root on mfs", "mfsroot", or whatever people might call it, very straightforward on BSD. They maintained and provided all the necessary tools, some of them have no Linux equivalent. Things stayed more or less the same so once the process was learnt, there seemed little chance of the goalposts moving. One did not need to be a contributor to understand these processes. Sometimes I get the feeling I am just not smart enough for Linux, and it seems like these processes are constantly subject to change. (I hope to find the truth is otherwise.)

As a BSD end user, I found it relatively easy to compile a custom kernel and a custom, statically-linked, multi-call binary as a userland, like busybox but using utilities "as is" not pared down with features missing, to be inserted into the kernel. I could boot from USB stick^3 to ramdisk root, pull out the USB stick, e.g., if I needed the USB port, then chroot into any userland. This was extremely flexible and allowed for easy experimentation. Sometimes I might have that userland prepared on internal media, the USB stick, or some other external media, or I might create it dynamically by downloading and extracting binary sets. This "targetroot"^4 userland might be mounted, usually r/o, on HDD or it might just be mfs/tmpfs.

Apparently, the author was inspired to make his own distribution because he found the package installation process too slow for large packages, e.g. qemu.^5 He tried using SquashFS images to speed up the process. I did the same thing in 2012 using cloop2 compressed filesystem images for large packages, e.g. ghostscript, on NetBSD. I would keep them on the USB stick and just mount them over an executable directory as necessary. I was not using these large programs too frequently and I was running entirely from tmpfs, no HDD, so I did not want them occupying space and depleting RAM.

1. https://news.ycombinator.com/from?site=unixsheikh.com

2. https://news.ycombinator.com/front?day=2020-01-20

3. Thanks to the excellent BSD bootloaders I could manually choose the kernel during boot. I would have several kernels on the USB stick. Of course the kernel did not have to reside on the USB stick, I could load it from any connected media. At one point I was able to boot NetBSD kernels using the FreeBSD bootloader.

4. TBH, I still do not know the true story behind targetroot. Things in BSD always have a history behind them. Surely there is a story behind the "targetroot" directory.

5 https://michael.stapelberg.ch/posts/2019-08-17-linux-package...

"How can minitrd be 20 times faster than dracut?

dracut is mainly written in shell, with a C helper program. It drives the generation process by spawning lots of external dependencies (e.g. ldd or the dracut-install helper program). I assume that the combination of using an interpreted language (shell) that spawns lots of processes and precludes a concurrent architecture is to blame for the poor performance."

Interesting that dracut uses bash, not dash for its scripts. Would dash be any faster. Are bash features a necessity for the simple tasks dracut performs.

Also, could dracut use readelf instead of the bash shell script ldd. According to Wikipedia, the Linux man page for ldd asserts it is a security risk.

Does default Ubuntu include a manpage for ldd. Mine is missing.

https://github.com/slashbeast/better-initramfs