The Distribution Vector That Changed Linux Forever | Hacker News Reader

The Distribution Vector That Changed Linux Forever

tedium.co

41 pointstodsacerdoti28 comments

I was the lead developer of the Linuxcare business card and the later LNX-BBC, and I also want to mention that Knoppix is a huge name in Linux live CD history.

Knoppix (developed by Klaus Knopper) ended up using the cloop driver, which Rusty Russell originally wrote for LNX-BBC. It added transparent gzip compression to the loopback filesystem mount concept and I think it allowed us to ship about twice as much software on a 50 megabyte CD-ROM! (And similarly, Knoppix could then ship about twice as much software on a full-size CD-ROM.)

In general, doing better than raw ISO9660 for read-only file storage was an important challenge for would-be tiny live and embedded Linux systems. I think there are several things that have followed cloop, directly or indirectly inspired by it.

I think you could still do quite a lot with a live USB system today. At Linuxcare we heard from a lot of IT desktop support people who were regularly using the bootable business card to fix/back up Linux and Windows systems, or sometimes for forensics, lost password resets, or other creative applications.

Edit: Knoppix existed slightly earlier than the Linuxcare project, even though it adopted that one particular cloop technology from us. I think it was probably an inspiration for the Linuxcare group. I don't know my Knoppix history very well, but I think I saw some form of Knoppix as early as 1997!

Edit 2: According to Wikipedia my recollection may be wrong because the first known official Knoppix release is 2000 (though that's version 1.3, so maybe there was a 1.0 sometime before that?) See https://en.wikipedia.org/wiki/Knoppix and https://en.wikipedia.org/wiki/Klaus_Knopper (amazing to see that Knoppix is still being actively developed to this day!).

Edit 3: Thanks to someone elsewhere in this thread for mentioning tomsrtbt by Tom Oehser (http://www.toms.net/rb/), which was also definitely an inspiration for the Linuxcare distribution. It had the motto "the most Linux on 1 floppy disk", later modified to "the most GNU/Linux on 1 floppy disk". While it didn't have a GUI, like the QNX floppy, you could do quite a bit with it!

Do you happen to know what the early bootloading situation was?

I'm familiar with ISOLINUX[1] (a very small bootloader for iso9660 cd roms) and SYSLINUX (for bootloading floppies (from which isolinux is derived, i believe?)). Was this a key enabling technology? Were other bootloading solutions ever used?

[1] https://wiki.syslinux.org/wiki/index.php?title=ISOLINUX

The Linuxcare bootable business card always used ISOLINUX and it was very pleasant (as it could read ISO9660 filesystems to find various objects on them, such as kernel and root filesystem images). I'm not positive what other systems used.

I think LILO was less widely used on live CDs because BIOSes had trouble letting it access CD-ROMs. My recollection of the details of the exact problems with this has grown a little hazy, though.

I think more modern systems are able to use GRUB in a relatively media-independent way, but I haven't followed this area enough to understand how GRUB is able to be so media-independent.

CD-ROMs and some other boot media use something called the El Torito standard (https://wiki.osdev.org/El-Torito), which is so named because a bunch of computer industry people -- I think in 1993 or 1994 or something? -- got together at a restaurant called El Torito to hash out the details. This describes how the BIOS can find the OS boot loader within the ISO9660 filesystem on the optical media. The tools for making an ISO9660 image had an option where you could indicate which file should be the El Torito special next-stage image, so I think we were supposed to indicate the ISOLINUX output this way, and it would get the right location (and maybe other metadata) so that the BIOS would be happy to boot it.

El Torito support in BIOSes was surprisingly universal considering how relatively unofficial a standard it was. In particular, ISO9660 was published by the ISO as a work product of a formal committee, while the El Torito extension was, I think, just some people who met at a restaurant this one time!

Edit: 1994 in Irvine, CA, apparently, according to https://en.wikipedia.org/wiki/ISO_9660#El_Torito; if the coordinates in the article are to be believed, the restaurant is still there, at Main and Mercantile, literally right across the street from SNA airport. Which might, in turn, be the reason they picked that restaurant?

I still have my Linuxcare BBC. And tomsrtbt and QNX media. Fun times.

On the topic of cloop, any idea why it never made it to mainline? Its hugely useful in itself, and I use it for some custom boot images built with buildroot, but it breaks on nearly every kernel upgrade. Also finding the latest cloop source always feels like a treasurehunt.

Everytime I ask about cloop in mainline, I get told to switch to squashfs - which is fine, just that cloop has uses well outside squashfs' domain.

I guess one answer is just that we never tried, and the kernel was stable enough during the time that we were releasing our distribution that I think we only needed a total of one patch from Rusty and/or from Klaus Knopper. Presumably it would (just) have needed someone to be willing to maintain it in the kernel at the time.

One element of this is that cloop was, quite literally, a quick hack (I think Rusty wrote the original version in a single day), which then worked incredibly well for our use case.

Sorry about that.

I came here just to say how in Germany knoppix was huge (and for most people the first exposure to Linux). In addition to most Linux users I know many advanced Windows users who always had a a knoppix CD laying around to fix broken PCs.

Trinux was another early floppy-based Linux bootable distribution. It's claim to fame over the better-known (and more self-contained) tomsrtbt was that it could read from successive floppies (typically 2-3 discs) to make a wider range of tools (mostly security / network forensics) available.

The old dev site is still up on Sourceforge:

http://trinux.sourceforge.net/legacy/

No mention of knoppix, I think? For me that was the first live distro (hell, first distro at all) that cracked reliable hardware detection.

I'm disappointed at the lack of a Knoppix mention, myself.

First time I tried Knoppix, booting to a desktop from CD without having to install anything with almost all of my hardware being functional...

...was probably the last time anything I did on a PC actually felt like magic.

And we still can't do that with Windows today. It took Microsoft nearly 20 years to even give us a USB installer option.

Yeah. It's still magic.

Yeah, the idea of running an operating system without installing it was amazing in the PC world twenty years ago. (This is how DOS and CP/M worked on PCs with no hard drive, in the sense that you would boot them from removable media, but that model didn't seem to outlast the 1990s.)

One of the magical experiences I remember with LNX-BBC was when we got Two-Kernel Monte working, which was a tool that let you boot a Linux kernel using a Linux userspace as the bootloader (not emulated or virtualized, but directly on the hardware, replacing the prior kernel and userspace). We had all kinds of fun with that, like netbooting over HTTP.

In fact I remember a whole discussion like "hey, PXE is too restrictive, we can ship 50 MB of code that does an arbitrary netboot using non-BIOS-included drivers, protocols, and authentication logic". I think someone did make a custom image to do this in some environment.

One thing I didn't know until I checked just now is that the Two-Kernel Monte replacement kexec is still maintained in the kernel! https://en.wikipedia.org/wiki/Kexec

Somehow I thought it had fallen by the wayside after a couple of years, but apparently not. So now I should go try some netbooting over HTTPS... :-)

take a look at https://netboot.xyz/

I really liked the very first run-from-CD distro (Yggdrasil). It provided a full-featured Linux environment back when many systems lacked sufficient hard disk capacity to run one from there. I believe my initial Linux system had a 70MB hard disk, later upgraded to 200MB. The Yggdrasil disc had ~600MB of content and worked well (if you didn't mind the slowness).

https://en.wikipedia.org/wiki/Yggdrasil_Linux/GNU/X

https://archiveos.org/yggdrasil/

> At the time, CD-ROMs spun at slower speeds with data transfer rates of below 1 megabyte, which made the CD-ROM less useful than it might have been in later iterations.

It's all relative, I suppose. Around the same time I had Debian (server) installation that took up less than 40 MB of storage. Nowadays, the kernel itself is larger than that.

Those "slow" CD-ROMs were still an order of magnitude or more faster than the most common alternative, floppy disks, by the way. Not to mention that floppy disks were notoriously unreliable. It was always a challenge to find enough working floppies to be able to install the "A" and "N" disk sets required to get a (minimally working) Slackware box up and running!

I think the first CD-ROM drive that I personally owned was a "4x" drive which, if memory serves, would have been ~600 kB/s. A "standard" 3.5" (1.44 MB) floppy drive might have been able to read data at, what, maybe 25 kB/s at the most?

Nowadays, when I boot a live CD it's almost always over the network (via PXE) -- at 10 or 40 Gbps -- and the entire thing gets copied into RAM before being launched...

It's interesting to think about but it's impossible to imagine how many of today's "hard (upper) limits" -- and which ones -- will seem absurd and completely miniscule in another 25 years or so.

For me, it was Unix that introduced me to Linux. At work, we had Sun Solaris and the console ran CDE[0] WM which for some reason I really liked. At the same time, I started to learn/write shell scripts and loved the experience. Later when I bought my computer, I wanted to install Solaris on it so that I could have CDE on it and practice shell scripts at home but me and a colleague couldn't make it run on x86 (I think we had the right build but some hardware wasn't compatible). At the same time, PCQuest[1] was the first magazine that I bought that had a live-cd with PCQLinux on it - can't remember how it was. Later I got another issue that had Knoppix with it - that basically blew my mind because practically everything just worked - KDE was a nice WM and I could write shell scripts on Linux at my home! I also found many programs/utilities that were included in Knoppix to be fascinating. Eventually managed to get the dial-up working too and there was no looking back after that.

[0]: https://en.wikipedia.org/wiki/Common_Desktop_Environment

[1]: https://en.wikipedia.org/wiki/PCQuest_(magazine)#PCQ_Linux_I...

In case you weren't aware, CDE was eventually open sourced and now happily runs on modern Linux systems (as well as the assorted BSDs and Illumos distros). If you want to have that again, you totally can.

I wasn't aware - thanks for sharing; I've come to like the more modern looking WMs though and will probably continue to use GNOME3 with Dash to Panel now :)

The full distro live CD was really great. As a young teen, it also appealled to my wannabe "hacker" attitude ;) I loved this idea of carrying my own almost self-sufficient OS in a pocket that I could just boot on anyone's computer. I spent a ridiculous amount of time configuring and building custom Live CD distros that I rarely used in any other situation :)

I remember that Ygdrassil Linux Bible. That and Slackware floppies was how I first installed Linux. The book was mostly a compilation of free howtos and books but still useful in printed form.

Fun to see adamj's work appreciated again. :-)

Is that before or after the floppy-based QNX?

The QNX floppy was apparently from late 1999 (http://toastytech.com/guis/qnxdemo.html), so I'd say "almost contemporaneous with" because, in my recollection, the Linuxcare project would have dated to sometime in 2000.

The QNX floppy was extremely impressive for our group and we never really thought about how to get closer to its size. (Today one part of the answer might be "use busybox for your whole userspace".)

I suspect Yggdrasil came before the QNX floppy, at least the one I played with at the time (http://toastytech.com/guis/qnxdemo.html).

Even if I'm wrong there have probably been earlier Linux boot floppy disks (e.g.tomsrtbt, mulinux - which was great!).

Somewhat related, at some point even SCO made a demo boot floppy for SCO Unix (or perhaps Openserver), it's still available on ftp.sco.com somewhere. No X and very few useful utilities in there.

> data transfer rates of below 1 megabyte

That doesn't bode well for my confidence

Why? It's fairly customary[1][2] to measure CD transfer rates in bytes/sec and not bits/sec.

[1] http://www.sysu.com/cdspeed.htm

[2] https://en.wikipedia.org/wiki/CD-ROM#Original_speed

The "/sec" bit is critical, and missing.

> Jeff Bezos, Eric Schmidt, and Bill Gates LOVE This Secret Asset Class

blog spam with little content

There was lots of interesting content in the article. But being annoyed at the ad, what's the difference between that stupid ad, and any other ad on the web. I'd say not much.

Reply on news.ycombinator.com