Interview with Andreas Kling of Serenity OS (2022)

You probably won't supplant Linux with your toy OS for your own research, but for a software engineer, writing a OS is the closest thing we have to the exhilarating feeling of seeing your code affect real hardware. You haven't lived until a bug of your own causes a triple-fault exception that causes a literal system shutdown.

I started this career writing my own tiny OS in my teens (https://github.com/1player/klesh) and for 20 years I've been wanting to go back, with all the experience I have accumulated.

Just don't make it another UNIX, please. We have enough of those already ;)

I'd love to try it out when (if ever) I have the time.

If you have all of your programs reside in the same virtual memory space (which is the implication with running them in ring 0), then you get a unique performance advantage in that you don't have to invalidate the TLB. I have no idea how much the amortized performance impact of virtual memory is, unfortunately, so the gains might be trivial, but who knows.

Fuchsia's "Signals" are designed in that direction. You get a "handle" to a resource, and the resource has a 32-bit integer with signal bits that you could subscribe to. The meaning of each bit is different for each type of resource.

(In general operating system terminology, "Signal" means just a message without any payload that would have required any memory allocation. There are many ways to do it, not just the Unix way. Message ports and other abstractions are sometimes built on top of types of signals)

I strongly encourage you to do this!

I think writing your own OS is one of the things every dev should do at least once, just for the educational value. The others are writing a game and writing a compiler and linker.

Go one step further and have the kernel expose a much more structured view of storage to user space by default. 99.9% of user space programs I've written don't want to deal with file IO, they want a database, or some other way for arbitrary in memory objects to be persisted and shared between processes. There's so much room to improve on the 1970s era IO primitives we use by default.

There was at least one Unix startup (IIRC it was Tolerant Systems) in mid 80s that applied transactional semantics to filesystem operations. I did some contract work for them and found the performance was absolutely abysmal and the system had some deadlocks. Now may be their implementation was not great but I suspect the converse is also true: database semantics are terrible for filesystems :-)

We’ve moved on from “back in my day, the 80s…” computer UI nostalgia to 90s it seems.

10 more years we’ll have Mac OSX Snow Leopard clone project get abandoned (have had a lot of young Millennials and older GenZ hackers talk it up at me as peak computing)

Just because they’re not grumpy doesn’t mean they aren’t nostalgia driven iconoclasts making serenity out of disdain for status quo

Nobody younger than an older millennial who grew up learning computers via Windows 95 has any interest in “retvrning” to this ethos of computing. Hell, I just described myself and even I could care less about this project other than a passing “huh that’s kinda neat” which is a rather tepid response proportional to the work that has been invested here.

There are far more interesting projects rethinking the OS right now that are actually trying their hand at unique architectures. This is just an homage to nostalgia (a technically impressive one, but still nothing inspiring here). The saddest part to me is I’m not sure the graybeards on HN will ever really get that.

Sometimes the grumpy greybeard fuddy-duddies act as they do due to experience. One can have mailing lists which are very distributed and easy to put on a searchable web page somewhere. One could also tie everything to Github or Gitlab or Atlassian, and then have it pulled, or have rates jacked up, or whatnot. One can go on IRC, and easily migrate from Freenode to Liberanet if need be. One can also go into the walled garden of Slack or Microsoft Teams or some other proprietary format, so that some company can hike rates or pull the rug out at their convenience.

The most innovative concepts historically would be Plan9, BeOS, and seL4.

Plan9's approach of "everything is a file" is something that was unique.

BeOS's extensive use of journalling and metadata in the filesystem gave it big advantages.

seL4 being the most rigorous application of a micro-kernel architecture, and making use of formal verification make this one stand out.

Capability-based operating systems also offer immense benefits.

Fuchsia and Redox are clearly drawing some inspiration from these. Along with GNU Hurd, which you didn't mention.

As for shells, nushell is clearly revolutionary. Something that points to what a better world might have been.

I would also recommend looking at Lennart Poettering's vision that extends on innovations like cgroups, journalling filesystems, ostree, etc.

https://0pointer.net/blog/projects/stateless.html

And of course, Nix and Guix both align with this vision of reproducibility, statelessness, and transactionality... along with Fedora Silverblue.

I kinda think of this as making the filesystem work more like git. A distro is a branch. A rollback is a branch from a previous commit.

And then the build system would be completely idempotent... every compilation from any system would be byte-for-byte identical, and thus verifiable in a security sense. And every upgrade on any system could be undone.

Ever since I heard this, I've been protective of what I talk about and it seems to have helped me ship more things than before.

The go-to guide for anyone that wants to build their own operating system has always been the OSDev community wiki and forum: https://wiki.osdev.org/Main_Page

I'm confused by the use of quotes. Do you think ADHD is a made-up thing?

While not black-magic, dealing with hardware and its quirks is a different skillset than writing React logic. A bug might need a reboot, or in some rare cases with terrible hardware, turning it into a paperweight.

In fact, I believe that learning to write low-level code for simple hardware (say, an Arduino or a RP2040), is a smoother path for a complete newbie than learning the massively complex frontend world, and provides much better foundational knowledge.

You're right, there's no magic here. It's just lots and lots of work. And there are important decisions to be made about how you design and abstract everything. None of them are hard, but making so many decisions is the very thing that's hard about most programming.

The answer of course is to have lots of programming experience. Which is why building an actual OS to the level of SerenityOS is not something a reasonably skilled graduate would be able to do.

Its linux but stripped down to the very bare essentials

https://elixir.bootlin.com/linux/0.01/source

It teaches how to do a hello world, with proper setup.

It's a bit more than a single C file, with some assembly/linker script/custom GCC toolchain required.

From there, that wiki is probably the most helpful resource you can find, along with the CPU manuals.

Then you compile this with GCC flags that tell GCC to not include a standard library (e.g. compile for bare metal).

Then you link with a very simple linker script that tells ld to put the multiboot header before your actual program.

Then, if you want to run on your own hardware, you change the GRUB config file to load your binary and reboot. Or if you want to run in QEMU, you just load the binary with QEMU. Using QEMU is probably simpler, since you're less likely to run into hardware weirdness at the beginning.

Then, you'd start with just writing to a text buffer, which has a hardcoded address on old architectures (it makes sense to start with an architecture where that's the case). That way, you can print stuff to the screen. That's enough to do a lot of very cool stuff already. After that, you configure interrupts, which you do by hard-coding some function pointers in a table using assembly. Then you use some special instructions to load these interrupt handler tables on startup. Now you can do all kinds of interactive things.

    CREATE TABLE filesystem(inode AS BIGINT UNIQUE NOT NULL PRIMARY KEY, is_dir AS BIT(1) NOT NULL, name AS VARCHAR(4096) UNIQUE NOT NULL, in_this_dir_inode AS INT NOT NULL, bloblist_ptr AS INT NOT NULL, metadata_ptr AS BIGINT),

    CREATE TABLE bloblist(blob_id AS INT UNIQUE NOT NULL, onwer_inode_ptr AS INT, thisblob AS LONGBLOB, prev_blob_ptr AS INT NOT NULL, next_blob_ptr AS INT NOT NULL)

   CREATE TABLE file_metadata(metadata_record_id AS BIGINT UNIQUE NOT NULL, created_at AS DATETIME NOT NULL, last_accessed at AS DATETIME, last_modified_at AS DATETIME, owner_uid AS INT NOT NULL, owner_gid AS INT NOT NULL, mode AS BIT(4) NOT NULL)