Linux from Scratch
linuxfromscratch.org
linuxfromscratch.org
https://github.com/dslm4515/Musl-LFS
I played with it for a while as the base OS for my KVM hypervisor servers.
A couple of years ago, I tried to find all the source archives for the old LFS releases. I ended up using the Debian and kernel.org archives to find most of the old package versions. Packages from before 2004 started disappearing like as86 and the PowerPC patches. I was never able to find the PowerPC patches because they were never mirrored on anything but FTP sites. The Internet Archive did not pick up any of those mirrors. https://archive.org/details/linuxfromscratch-releases
I eventually replaced the Musl-LFS build and replaced it with a buildroot set of configs and scripts when I had issues replicating my work with later package versions. Buildroot allowed me to more rapidly test and deploy my hypervisor config. I wanted a simple /etc/rc init like what was in SunOS 4, no GRUB, and QEMU with VDE networking.
I ended up with SYSLINUX for BIOS MBR and gummiboot(before it was converted into systemd-boot) for EFI because my EFI machine (Dell R810) did not allow for direct booting of EFI binaries.
- How to compile things
- How to follow instructions
- How to overcome failure and troubleshoot
- How to design a package manager (if you get that far)
- Basic configuration
Also takes a good few hours (it took me 2 days but I had to restart once and had subpar hardware). Weigh the advantages/disads
The first time, it is whatever is suggested. Then whatever works for You. After doing LFS or similar, You have most of the knowledge to choose. Me, I use rc files, but then I'm a Slackware Linux/OpenBSD snob.
(Oh, init.d drives me up the wall. Learn which kill signals do what You want) Edit: s/ if / of /
I have my own LFS and no systemd... neither init.d. I have only an idiotic parallel startup SH script (busybox), already overkill. The main trade-off is I do compile as built-in the linux SH script support module.
Well, what I remember from it (about 20 years ago now) is mainly this: make/compiler error -> search -> modify obscure file -> repeat. For days in row. At that time I didn't exactly know a lot about programming so most of it was just dumb copy-pasting without knowing what was going on. I do think it was useful (hard to tell - I seem to be able to fix pretty much everything in the SW depertment these days but that's probably a sign of attitude in general, much less thanks to LfS), but also lead to a bit of a distaste for linux. I.e. after having spent weeks like that with lfs, and then again when deciding to go for gentoo, and then again with debian, there wasn't much fun in it anymore. Only a 'been there, done that, now give me something which just works' feeling.
On a personal, nostalgic, note: running LFS on your laptop in my university was the ultimate Linux nerd cred. A step below, immediately following, there was Slackware.
The reason I feel confident enough working with Linux today is because I started back in the early '00 using Slackware, which recently had a new release[1].
Don't get me wrong, compiling your own kernel, solving shared libraries compilation hell, messing up your init.d and starting over, leaving your Pentium 4 compile for several hours was a massive waste of my 20s. But, boy, did I enjoy it!
I had the same experience. I'm surprised how far I went with tasks I wouldn't particularly like to attempt now, purely motivated by the reward of tinkering with things without a sense of fear.
Apropos to this, I remember back in early 2010s arch linux IRC had a bot which used to make fun of Gentoo for its compile time. One of them was something like "Watching compile text fly by is fun for the whole family". I still think about this some time. Which is funny because in day to day I don't use any compiled languages :P
Edit: forgot to quote relevant text.
PS: If someone does remember the exact quote, please tell me, it would make my day.
It does help figure out informational crap from errors in other situations than being overwhelmed by wall of texts.
So much you can learn and like you say "dispel" from doing this. Even if you're 'just' running a bunch of scripts with the right amount of curiosity paired with it it can teach so much! Like trying out a bunch of optimizations on the C compiler and actually getting programs to fail because of it and such. Fond memories!
Personally I "moved on" to Gentoo at some point. And I remember "arguing" w/ my dad about compiling my own operating system from scratch. Of course I was that "teenager with lots of time on my hands" and he was working on production systems w/ uptime guarantees. Thanks for the fun memories in crazy times!
Like me telling him how there are people on usenet and mailing lists that don't know how to quote properly! And he'd just chuckle and tell me how quoting a whole thread could actually be good. I was furious (not him). How could that be? FWIW, I still think I was right for the usenet and mailing list use case, where anyone could just see the whole thread. He was right as well though for corporate conversations like I learned a few years later at my first job where you were glad that everyone in the thread did a full quote and 5 levels down in the first forward you got you actually saw what was actually being asked and could shortcut a whole escalated discussion with a simple "Err guys, the original question was just X, here's the answer. Can we all just go back to work now?".
Pros and Cons. Everything has at least two sides. Nothing is like it seems. Ever.
I'm a hardware engineer, and enjoyed playing this in my spare time. You create small components from nand gates with an extremely simple made-up HDL and gradually integrate them together into larger and larger functional blocks until you have a basic but working CPU.
Definitely on the "Basic" side of hardware programming, but it helped me personally fill in some gaps in my understanding.
Not a game but a pretty good read/work through.
Had fun doing distributed compiling with distcc.
Then again, I suppose there are so many things we never got to play with (and instead played w/ the aforementioned). Like when I told my dad I was playing around with vmware and xen and such and he'd just be like "Oh yeah right, virtualization, like what we've been running on our IBM mainframe since like forever". But sure, have fun, explore, learn!
If anything, Linux and all this free software has done a great job of taking this out of the hands of huge corporations and having to work for them just be allowed to play and learn. Anyone could do that now.
I did it out if necessity and there was no excellent documentation like LFS, but it was still doable and extremely educational. I highly recommend it.
I still recommend this to people interested in learning more about linux and operating systems.
I do write programs in Python and SQL but never touch compiling things, except in hobby projects which involves VSCode and some gcc.
Spin up a VM and dive in.
If you want a gentler slope, maybe something like (in this order, and keep doing stuff on a "earlier level" on the list):
1) Install packages via your packagemanager from the cli and other sysadm tasks
2) Change settings for stuff by files and not by gui
3) Install things not packaged by your package manager
4) Try arch (no compiling but a lot of setup)
5) Try gentoo (lots of setup and lots of compiling)
6) Try LFS in a VM (NEVER on a system, that you actually need)
Part 1: https://www.youtube.com/watch?v=tCh0XjyIAKU
Part 2: https://www.youtube.com/watch?v=egzyIFsQ3Nw
He also managed to dig up a copy of the proto LiveCD of Yggdrasil Linux and get it working.
Maybe using Arch or even Gentoo would be a sane next step.
https://missing.csail.mit.edu/
Try it.
Compiling every package is a pointless exercise. Even benchmarks showed Ubuntu was faster than Gentoo.
Package based build your own distros like Arch or Slackware give you most of the same experience in 1/100th the time.
Source/details? I would easily believe that Ubuntu was nearly as fast, or perhaps even exactly as fast, but I can't think of any way for a custom-compiled program to be slower than a distro-provided binary.
Could also be talking about boot time maybe - isn't SystemD supposed to boot faster than OpenRC much of the time?
Boot time crossed my mind, too, but Gentoo is happy to run with systemd so that's not really "Gentoo vs Ubuntu" so much as "this exact variant of Gentoo vs default Ubuntu" which is kind of fair. (Assuming, of course, that OpenRC is slower to boot than systemd, which would also be an interesting benchmark)
I would as well. My understanding is that SystemD does more parallelization/magic to decrease boot time, but is a bigger system to try to comprehend and has more surface in general.
I believe parallelization is a configurable option on OpenRC though, I'll add that test to the list to try some time.
The first time I switched to systemd, years ago (when Debian moved to systemd), I noticed the boot becoming much faster. So, yup, it was a thing. But now, years later, I got rid of systemd and honestly I don't see my system booting much slower. I mean: if Debian takes, say, 0.3s to show me the login prompt while Devuan (the Debian fork with all the systemd stuff stripped away) takes 0.4s, who cares...
(fwiw I always boot into text console, log in there, then manually launch the graphical display)
I don't think that the "faster boot time" is an argument in favor of systemd anymore, at least not on a modern system. And in case you need to spin a container in a heartbeat, you may be using some ultra minimal Linux distro like Alpine anyway.
In my searching, did find this which shows trading blows with others... yet to see a comparison showing that compiling everything leads to performance gains...
https://www.phoronix.com/scan.php?page=article&item=ec2-holi...
Of course there may exist interpretations that bring your statement to the level of debatable, but if you keep them inexplicit, should the reader divinate?
The last time I tried LFS seemed abandoned or so out of date nothing worked, links to packages abandoned.
> The LFS system will be built by using an already installed Linux distribution
It would be totally cool if there wasn't a chicken-and-egg problem with LFS, imho. I've tried to cross-build it from OpenBSD but failed miserably.https://trac.clfs.org/wiki/download if anyone else wants to attempt this
It is easy to compile NetBSD kernel on Linux but not vice versa. The significance of that if any is left as question for the reader.
Linux from Scratch with Training Wheels - https://news.ycombinator.com/item?id=28820602 - Oct 2021 (41 comments)
Linux from Scratch 10.0 - https://news.ycombinator.com/item?id=24350738 - Sept 2020 (49 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=24238015 - Aug 2020 (86 comments)
Major Proposed Changes to Linux From Scratch - https://news.ycombinator.com/item?id=23787526 - July 2020 (93 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=20168343 - June 2019 (15 comments)
Ask HN: Is the Linux From Scratch project still relevant? - https://news.ycombinator.com/item?id=20149111 - June 2019 (7 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=16823110 - April 2018 (1 comment)
Linux from Scratch Version 8.2 released - https://news.ycombinator.com/item?id=16510333 - March 2018 (2 comments)
Linux from Scratch – build your own Linux distro - https://news.ycombinator.com/item?id=11829373 - June 2016 (57 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=8392057 - Sept 2014 (1 comment)
Welcome to Linux From Scratch - https://news.ycombinator.com/item?id=4488162 - Sept 2012 (71 comments)
Linux From Scratch 7.1 Published - 3.2.6 Kernel + GCC 4.6.2 - https://news.ycombinator.com/item?id=3677350 - March 2012 (13 comments)
Linux From Scratch 7 Released - https://news.ycombinator.com/item?id=3171448 - Oct 2011 (27 comments)
Ask HN: Linux from Scratch.. Should I try it? - https://news.ycombinator.com/item?id=1779665 - Oct 2010 (58 comments)
Ask HN: The best site to learn linux from scratch? - https://news.ycombinator.com/item?id=1513838 - July 2010 (1 comment)
How to build custom Linux from source code - https://news.ycombinator.com/item?id=743843 - Aug 2009 (1 comment)
With Gentoo, waiting for your DE or web browser to compile was a great excuse to step away from the computer and go for a walk. The little things in life.
[1] tbh I don't rememeber the exact emerge syntax, it was almost 20 years ago!
It actually teaches you better to learn the inner workings of Linux if you follow Gentoo (or the likes) that makes you actually type in each of the steps to get the OS running like setting up swap and configuring kernel options but LFS tells you what the OS is made of like what packages are required to get a minimal system going.
Good to learn both if you're interested.
Indeed. I will say though, I'm seriously considering using this for a custom built appliance, probably for audio work - I always get annoyed at purpose focused distros because I can't quite fit it all in brainspace, and then it updates and changes under me. Audio specifically because I can never remember if my laptop is set up with Pulse/ALSA/JACK/OSS/Pipewire, and always break things when I try to set things up (I'm kinda dumb).
But perhaps a custom LFS which never needs to be changed or connected to the internet after installation...
Minimal yes, unmaintainable maybe, but insecure? Really depends on your threat model; courtesy of "minimal", there's not much attack surface to go around.
> Only think it will teach you is why distros and package managers exist.
And more importantly, how a distro works and what a package manager (and package build process) has to do. LFS isn't a terribly practical output for many people, but building it is a good learning experience regardless of if/how you use the built system.
I don't think LFS does that, it's simply page after page of .configure..make..make install. It would be more valuable to read a sysadmin book on unix/linux and play around with a minimal distro.
Just because you don’t see the value, doesn’t mean it has none.
Maybe it’s just me but the instructions seemed to focus too much on what to do and not nearly enough on why. I came out of it thinking that I learned much less than I expected.
In the "good old days" it was recommended to install the linux-howtos package and there was one howto https://tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.htm... which explained the boot process. There are also other howtos who talk about X.
Nowadays with systemd and the (stupid) tendency of using a lot of shared libraries the things have become more complicated.
Edit: looks like i can, cool!