Bored? How about trying a Linux speedrun? (2020) | Hacker News Reader

Bored? How about trying a Linux speedrun? (2020)

rachelbythebay.com

212 pointsmdwalters71 comments

It's more like an escape room than a speed run.

First person to watch a specific youtube video, which contains the password, wins.

Or maybe you're literally locked in a room, and you have to bootstrap the PC to the point where you can connect to the local network, then connect to the insecure Citrix server which is running the security software that controls the door locks. From there you can unlock the door to your cell and escape.

Only to find you're standing alone on an abandoned oil rig in the north Atlantic.

Just thought I might jump in to point out that the phrase ‘insecure Citrix’ may contain a redundant term

...and ~10hrs after you post your comment, someone posts a story[1] about an ongoing attack backdooring hundreds of Citrix servers!

[1] https://news.ycombinator.com/item?id=36977282

> First person to watch a specific youtube video, which contains the password, wins.

Why would that be difficult?

It is. Skills like this you can learn via HITB and likewise. Btw, Nix is a nice way to install userland for user (in case you don't have or want to be root).

I recommend anyone wanting to learn Linux to deploy a Gentoo machine. It gives you a crash course in all of the various components, allows you to hand roll/customize your kernel, and provides incredible granularity in tuning precisely what is on your system. No bloat. No extra dependencies. Apps compiled with support for "superfluous" features disabled, etc.

I owe my love of chroot to Gentoo.

The guide[1] is incredibly straightforward and easy to follow.

1: https://wiki.gentoo.org/wiki/Handbook:AMD64

Linux From Scratch gets even more into the weeds.

My arc was gentoo -> lfs -> arch Linux when I realized I didn’t want to fiddle as much.

Slackware comes pretty close, too. Yeah, you can use the TUI installer... but Slackware packages are pretty much just tarballs that you could extract directly onto / (plus an extra folder for metadata and post-install scripts). Starting with a bare-minimum Linux install and extracting enough packages one-by-one to get slackpkg working is a fun exercise (one which I've indeed done on everything from various LXC-based budget VPSs to old thin clients with disk space in the hundreds of MB at best).

my arc was slackware -> gentoo -> arch. very similar vibes across them all I thought.

That's indeed why I lump them together as the "SAG Trifecta".

Similar arc, but have fiddled with them all. These days most of my machines are arch(btw) or Alpine.

My experience with LFS is checking a few makefiles, the doing ./configure && male && make install because I got tired os checking everything by hand.

Personally, gentoo is nice because, once you know your way, you can do a big emerge and go get a coffe while it compiles everything.

I started to go through LFS, but with the intent of scripting it. What ultimately though slowed me down to a halt was a persistent VirtualBox issue I couldn't solve. But I think I just easily jump back into it now :D

Ah I’m actually mid gentoo install on an old MacBook of mine for that exact reason. Minimum cd from bootable usb stick altho I’ve currently approached a roadblock with getting the damn wifi set up lol. Ik I can ssh in, but since the goal is a learning experience it seems like kind of a cop out

It seems that pairing this with the original post, the best outcome is a single shell file, that takes a few arguments, such as device location (USB at /dev/sde) and username/password, etc, and compiles the bootloader, the kernel, etc from scratch using the most trusted basic sources, and then places that on the target device.

Obviously it should also install any additional apps like vim, adjust terminal colors, create and copy your ssh keys to machines needed, and configure any windows managers, etc as well from the single shell command.

Making that single shell command as small, reliable, readable, and maintainable as possible is the real challenge.

`archinstall` on a fresh arch iso gives you something similar in TUI fashion and can export a json output you can save to clone the install to additional machines.

Why is 'superfluous' in quotes?

Whether something is superfluous or just complete/rounded out is in the eye of the beholder.

Yes but I read the sentence to mean that parent was talking about features they actually found superfluous.

Can't plus-one this enough. The Gentoo [installation] handbook is a true treasure for the linux newbie.

Especially if you are the kind of learner who needs to be challenged to truly ingest the material at hand, just read every word and apply what you have learned. You'll be astonished by the distance that you covered in a month!

Whenever Gentoo is brought up, somebody always brings up LFS.

LFS becomes repetitive fast. I also don't think trying to use esoteric build systems that come with cryptic readme files is newbie-friendly. Just learn to read ebuild scripts if you are after that kind of skill :)

> Then someone would decide it's too easy if you have this thing or that thing, and they'd cut down the starting conditions.

Related: https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-... and https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed... - AIUI, the idea is that you can bootstrap from a tiny hex program (small enough to hand-enter) and then iterate compilers up to current gcc on normal GNU/Linux.

I'm astonished that this thread hasn't yet mentioned https://www.ecb.torontomu.ca/~elf/hack/recovery.html

Long story short, someone accidentally did rf -rf * in the wrong place as root, and they had to rebuild the system enough to restore from the backups, including writing an executable in machine code as hex and then converting it to binary to call the appropriate system call to effectively "mkdir /etc".

It wasn't Linux, but close enough.

Thanks for sharing it, it was really nice read!

There’s folklore, I don’t know where I heard it, or if I just made it up, so take it with a grain of salt.

But the story is the Chuck Moore, when he wrote ColorFORTH, essentially started with DOS, DEBUG, and a floppy drive.

And from this sprang forth (ahem) ColorFORTH. Essentially he hand keyed in the most minimal core necessary to start working with his own tools until he had a boot floppy. Then he tossed DOS and DEBUG and off he went.

Appreciate that the original FigForth is about 80 screens of code. And a 3.5” PC floppy can store over 1400 screens.

Chuck seems to be the kind of person that could type in a rough Forth in raw assembly and get it editable a feed on itself to get more powerful.

It’s an interesting thought experiment to be sure.

Yeh, I think one of the first steps would be to write a tiny lisp or forth. Maybe something like tcl or perl/awk subset.

Probably the most bang-for-buck.

bootstrap, aka becoming sovereign aka self-made man.

> the original FigForth is about 80 screens of code

Was "screens" a common unit of code length back in the day? Interesting.

Only in Forth, as far as I know

Indeed.

Take a look at the Fig Forth Installation Manual, which has the source code to Fig Forth, in Fig Forth. The source is listed as Forth screens.

http://archive.6502.org/books/forth_interest_group/fig_forth...

It is forth-specific. It refers to blocks of 1024 bytes of code, arranged in 64 columns by 16 lines and manipulated through a more-or-less standard vocabulary[1], supported by most Forth systems.

You can see it in action in e.g. the CollapseOS emulator[2], as it comes preloaded with ~300 blocks: try writing "0 LIST" (uppercase, as it is case-sensitive - see also [3]).

[1]: https://forth-standard.org/standard/block/

[2]: https://schierlm.github.io/CollapseOS-Web-Emulator/

[3]: https://incoherency.co.uk/collapseos/blk.txt.html

Today I'd still call it screens, as there's a default that xterm uses of 80x24, damn near the 80x25 in DOS years. I suspect xterm used 80x24 as a terminal program would normally use the last line for info, like baud rate.

https://retrocomputing.stackexchange.com/a/5634

Well bash has a built in TCP client which kind of ruins the fun. libc has a DNS library to query a resolver so no fun banging DNS over UDP required to download from gnu.org (:

If people don't know about this feature, you can search for "tcp" in your bash man page!

Then, let's exclude bash and libc (if we can)

At least the man pages.

The fun thing about speedruns is the variety of categories. Just add rules for new kinds of runs.

Made me think of "C compiler used as bootloader, that eats source code for Linux kernel while booting, and boots that".

Ah, still on the net:

https://bellard.org/tcc/tccboot.html

Uses TinyCC in place of a bootloader. Of course that would require other software like a shell / BusyBox or similar besides the kernel. But after booting on-the-fly-compiled-Linux that seems like a minor issue.

Project is from 2004. Surely doing this with source code for current(ish) Linux kernels would be more involved? Especially those including Rust code. :-)

Would be fun to have a more up to date something of the above. Anyone?

I speed-run every install; how else would I improve anything about it? :p

https://wiki.realmofespionage.xyz/distros:fedora_workstation...

Fresh Fedora installs involve me opening Terminal, Firefox, going to that page, and copy/pasting commands. I can reproduce my set-up in less than an hour.

For physical machines I maintain a debian preseed that runs an ansible playbook post install. Install + setup is fully unattended and generally takes about 15 minutes.

https://wiki.debian.org/DebianInstaller/Preseed

Quite different than bootstrapping a system as described in the OP, but relevant in the context of your comment.

I don't have an unattended install, but I do have a post install script I wrote in pyinfra.

Mostly sets up repos, installs packages, does some basic configuration. But also calls a bash script that sets up KDE/Gnome, down to the launcher icons, widgets, themes, a wallpaper, and application settings.

Basically "cattle not pets" approach to workstations and laptops.

The only thing I struggle with are Firefox extensions. The last time I checked automating them was such a drag. Surprised there's nothing like apt or dnf for them.

https://pyinfra.com/ https://stackoverflow.com/questions/37728865/install-webexte...

An end user variant of this might be counting total key strokes/mouse clicks to install Linux on any machine, including all the disgusting BIOS/UEFI fiddling to enable USB boot, disabling SecureBoot, and anything else necessary. Whoever wins has found the best end user hardware/distro combo.

Do the machines that ship with GNU/Linux preinstalled win by default? (Zero keys/clicks on system76, say) I suppose to be a fun game we'd have to exclude them.

We'd have to exclude them, otherwise it just raises the question of how the machine was installed originally. And if that was done using some complicated automated setup, the cost and complexity of the setup draws the result down. Assume the storage has some unknown content.

Gentoo users win then. They don't need to boot from USB, because their installer is tar. The ones that use a BINHOST don't even have to fiddle with compiling anything but the kernel unless they demand non-standard USE flags

It'd be interesting to compare that with installing Windows, which in 30 years of using Linux I have still to successfully do.

I worked on Fuchsia for about 5 years. Somewhere maybe around half way through, I think it was during a Christmas vacation, someone wrote in that they were still away from their desk, and they'd not bricked their test device, but nearly - they could no longer install new software via the regular path as they'd done something to one of the filesystem runtimes or the like and that broke some of the components. Along the way they'd also bricked the alternate boot path, so they couldn't use the regular "paving" strategy (like a flash, but for regular disks and a bit faster/higher level).

I'd been working on the system bootstrap and verified execution chain, and often had to cobble stuff together in our shell. The shell on Fuchsia is a bit weird though (also my doing, largely), we'd avoid installing most software, and instead most of the programs in the shell are little stubs containing e.g. `#!resolve fuchsia-pkg://fuchsia.com/vim#bin/vim`. If you have bricked your update system you can't run those programs, as you can't resolve an up to date and safe version of vim. What was there though, was always a very small number of utilities from sbase. If you were to pick between sbase, toybox, busybox and so on, I'm not necessarily sure you'd pick sbase, but for Fuchsia it was definitely one of the easier to port for a variety of reasons.

So you have basically a hexdump tool, cat, and a hacked up version of dash as a shell (a POSIX shell, essentially, on a system without fork, ttys, signals or sessions, so it worked a little differently). A similar set of constraints to this article.

I sent them an email which looked something like: run lsblk on target and capture the output. mount the efi device to /mnt, assuming it's good that's fine. copy files using the pattern "cat file | ssh target 'cat - > target-path'". Copy the following files in build-name:target-path form: zircon.bin:/mnt/zircon.bin bootx64.efi:/mnt/EFI/BOOT/bootx64.efi ramdisk.bin:/mnt/ramdisk.bin. umount /mnt. On target do 'for i in {1..4096}; do echo -n a > /dev/{path-to-minfs}; done`, the system will crash and reboot shortly after, and should boot into your new ramdisk.

At the time there was a bug that'd trigger the final reboot when a mounted but broken partition had its magic header overwritten. A couple of months later I made this way harder to recover from, but we also added an A/B/R partition scheme so that there was essentially always a recovery strategy even if you send really broken software.

If you're on a Linux system though, I think some of this gets in some ways easier. There's some ABI junk to deal with, but you can get really far with disgusting behavior in a posix shell that has a tty and signals, as you can write a bunch of not very feature complete, but very functional programs quickly in shell. A quick editor - probably 20 lines or so, and each time you make it better it gets easier to be better. Already got some read/write loop going on in that editor, great, plain text http isn't far off.

This is all a bit of an interesting set of trivialities though, not to discount the idea. DNS is a good example as in the article, but if you have no DNS tools you are missing ip(1) and friends, and you need to go implement netlink first - otherwise having a DNS client doesn't matter, you don't have configured network interfaces! If you have ip(1) then you likely also have ss(1) and you can trick the latter into acting as a dns client. If you have to hand roll netlink without headers, a compiler and a linker though, that's not really a from-memory job there, and the netlink part is far worse than the DNS part really. If you're on a systemd system, and all you have is systemd a shell and ip tools, you're basically completely set from the get-go, as you can jump straight to `machinectl pull-tar` and you're off to the races.

Of course if you're on an EFI system that hasn't trimmed down much from the reference tianocore build, then you can probably reboot into an efi shell and get network configuration and pull an image from there, which is likely easier than escalating from a zero compiler zero network environment against the linux abi.

Fun idea, I'm just not totally sure how to put a good modern bound around "minimal linux system".

It feels like a CTF, but kind of in reverse. Following that theme perhaps it should be "escape the pot", modelling the stripped down system as a honeypot.

Consider a minimal install of linux/unix with no C compiler and no binutils.

Editing text. Construct an ed-like editor in shell - the first character/token of each input line is a command. You could use awk or shell for this.

Creating binary files. If you have gnu coreutils, it has a powerful echo option to output bytes. If you have posix/busybox/toybox, use od(1), head and tail to extract a library of 1-byte files from existing binaries. Cat these together to create multibyte binaries.

Assembler and linker. You don't need to use conventional notation. Create an assembly syntax around the same first-token-is-command approach used for the text editor. Implement an assembler that reads that notation and outputs binary. You would need to create elf headers. You would need to map system calls.

Can someone please do this and stream it so I can learn?

Tetris OS[1] has some good content to learn from.

1: https://www.youtube.com/watch?v=FaILnmUYS_U

(2020)

Previous discussion: https://news.ycombinator.com/item?id=22846250

I did something like this with a 486 box recently. It only had DOS, ISA only, and no network card. By the end of the night I had Win 95 running, copying files over FTP, and SSH'd into my router from it. It was fun.

>and no network card.

> copying files over FTP, and SSH'd into my router from it.

Did you add a network card or connect through another machine with serial/ppp/uucp?

Yes, I took the easy route and added an 3Com 3C509 ISA network card. However, it does have a serial port and that would be a far more interesting way to get it online!

I tried LFS on a 486 box when I still had it. Compiles went on for so long, time lost all meaning and I lived about 70 lifetimes as the clouds flew by to the motion picture soundtrack written by Philip Glass.

Isn't the fastest way to do this to download a linux bootable image and then reboot over that?

The problem being how to download when you have no tools to download (or even do DNS requests for you). So you'd need to code those first...

It looks like they didn't create a user, unless pacstrap does it for them?

One can learn all sorts of interesting skills when one starts to imagine all the possible ways simple tools can be used.

I'd like to see something like this, but with hardware included - something like Junkyard Wars, but with random computer hardware - which then people need to get going to be able to do specific functional things.

I created a Linux image that allows for this about a year ago:

> Linux "distro" that only includes a Lua interpreter and a Lua library to issue syscalls

https://github.com/FranchuFranchu/lua-lfs-hard

>I mean speedrunning the Linux situation itself. Start with a minimal system and get yourself to the point where you can do something meaningful (like reading cat pictures on Reddit).

So basically an Arch or Gentoo install speedrun. I'm pretty sure I've already seen those on Youtube.

Too easy.

Try you only have a single bootloader and a little bit of source code and building all the way up to a complete linux distro

https://github.com/fosslinux/live-bootstrap

(checkout the builder-hex0/fiwix work)

The “desert island machine” concept brings to my mind this incredible talk from David Beazley: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8

Reminds me of something similar I went through on my first job fresh out of school. Maybe not as exciting as uncovering secrets in a 1.5TB source code vault, but we had these locked down workstations with nothing but a browser, Office and some vendor specific tools you could install from a software catalog. Nothing useful like an IDE or SDK and every other binary downloaded that was not approved would not run. Imagine my reaction when I found that one of the vendor tools included a Python runtime (albeit an old one like 1.6 or so). Which allowed me to script against the Exchange's server IMAP and automate my morning tasks of sifting through hundreds of monitoring emails from hours to minutes. Saving precious time I could invest in further developing the tool to include a full web interface and all sorts of nonsense.

Here’s a version of the idea: bootstrap speedrun. Start with the source of everything but the binaries of nothing that isn’t the basics (cat/chmod) &c. Write a script that compiles gcc. Fastest benchmark time wins.

Genuinely curious: how is this different from nand-to-tetris sort of challenges? (I haven't tried those, just curious to know if people have heard of it.)

Always wanted to get into speedruns, maybe this would be the perfect thing to do. Now all we need are set rules and a category on speedrun.com.

Offtopic, but

> I mean speedrunning the Linux situation itself.

I think “the Linux situation” is an amazing choice of words. It truly covers it all, doesn’t it?

That's it, no more hacker news for me, I think this is the post that made me log off for good lol

Bored? How about learning about mobile-friendly CSS?

Reply on news.ycombinator.com