SerenityOS – a graphical Unix-like OS for x86, with 90s aesthetics
github.com
github.com
If you would like to see the system in action, I did a demo video in March[1] and one in April[2].
I’m happy to answer any questions you might have about the system. I’m not an expert on operating systems, but I can be pretty productive :)
You singlehandedly created something that took many programmers many decades to build.
The original developers had to come up with the concept of, say, a taskbar from scratch.
They understood it has to run separately from other parts of the OS in case of crashes, it should support icons that are references to the same icons that windows have, etc.
But they were also limited by the resources of their day, and had to get clever about resource allocation. That's extremely time-consuming.
But that said, 90% of this project is simply a very good creator, being incredibly productive.
The mythical 100x programmer isn't always a myth.
I'm in awe.
Serenity is an unapologetic remix/mash-up of all the things I like in software. Not just visually, but also code-wise. If you look into the sources, you'll find C++ heavily inspired by Qt and WebKit.
I remember being excited that I was able to splurge on a 486 with 4 Mb RAM.
I don't think I have much of a point except that you could do development on that machine in relatively modern ways in a way I could never imagine on a ZX-81 or ZX Spectrum.
[1]: https://en.wikipedia.org/wiki/ZX81 [2]: https://en.wikipedia.org/wiki/Intel_DX4 [3]: https://en.wikipedia.org/wiki/Cello_(web_browser)
Will it be possible at some time to compile Qt based applications in SerenityOS?
From the state a month ago, it’s very reasonable. From scratch however...
How do you even start building an OS?
I don't know if any two operating systems started out the same.. Serenity came into this world because I found myself with a lot of free time and in need of a programming project.
The first three components I wrote were an ELF parser, an Ext2 filesystem dumper, and a GUI library (on top of SDL.) They all ran independently on my Linux box. Once they worked well enough, I glued it all together with a simple x86 kernel and just kept going.
Probably not the answer you wanted, but I assume from a poc/toy bootloader. I wasn't disciplined enough to go further, but I did play around getting a toy OS working, and stopped a little bit past that part.
This effort, from a one man show, is amazing.
So far I've learned C [0], x86 asm [1], read OSTEP [2] and begun hacking on xv6 ...and I still feel like I have years to go.
[0] http://www.knking.com/books/c2/index.html
Now that you have networking and a compiler, you could build a "simple" package manager. It could be just a list of supported Git repositories that are downloaded (downloader) and built (gcc), that would at least allow people to write stand-alone applications (no libraries) for your system that others could use.
A package manager is definitely coming sooner or later. There's actually an SDL port in the works (done by a stellar contributor, not myself.) I suspect that we're gonna need some sort of ports system in the Serenity tree to be able to maintain it. :)
Any plans for 64 bit?
Any plans for SMP?
The old readme mentions a gcc patch. Is it still needed? What does it do? Just the final executable's format?
Is there an FAQ somewhere that I missed? :)
The answer is basically the same for 64-bit and SMP support. Some day I'll wake up with the burning desire to implement those things, and then I'll start on it. Or someone will show up with patches. We'll see! Nothing really depends on those features, although they would certainly be huge improvements to the architecture.
Yeah we still need patches for binutils and GCC. Any custom operating system does, you can see the patches here: https://github.com/SerenityOS/serenity/tree/master/Toolchain...
It's all about setting up the executable formats, providing the system names, and telling the compiler how we like things linked, etc.
There's no FAQ but maybe I should compile one..
I hope your OS finds a stable niche to live in, and you get lots of great contributors. Thank you for writing this, its existence makes me deeply happy :)
The bitmap sharing between WindowServer and clients is using a custom Serenity API ("create_shared_buffer") which is specifically tailored for this system. It can probably be replaced with some equivalent X11/Wayland mechanism in a few hours, if someone felt inclined to do that. :)
understatement of the century
The inspiration for this system comes from using Visual Basic as a kid and just having fun trying to mimic the Microsoft Office look&feel of those days. It also comes from using Slackware Linux and realizing that everything can be done with text commands, and configured with text files.
Does that system only design forms, and you have to manually "glue" them to code - or is there a visual programming system behind it?
I guess what I'm getting at is that you mentioned here being inspired by VB from your youth - so I was wondering how far you have taken that particular part?
I worked professionally with VB for over a decade (from VB3 to VB6) years ago; BASIC in general, and VB as well - both hold places in my heart, since that is where I started my coding journey to a career in it (I started when I was 10-11 years old, back in 1984, using a TRS-80 Color Computer 2, which ran a version of BASIC from Microsoft).
This is a great project, something I've seen posted before here and elsewhere, but it's always worth another look when it pops up.
It would be fun to build a visual programming system in the style of VB6~, but it's not something I've started on. :)
In other words, a Windows GUI with a Unix-like CLI? That's something I can certainly agree with --- Windows has traditionally been very lacking with its CLI, while the Unix-ish systems (including Linux) seem to be quite fragmented and awkward at GUIs.
I also thought the terminal font looked familiar, then realised it's either the same as or a very close version of the "misc-fixed" fonts[1] that I use in my terminals and for editing plaintext (including source code) --- it even has the slashed zero that I added to mine, and which the original fonts lacked. The general UI font looks like a proportional version of it, which I've honestly never seen before but think it's quite pleasing too.
All the fonts are hand-bitmapped by me, using the included FontEditor application. I was trying to construct the imaginary love-child of Tahoma and Fixed, so you are definitely on the right track there.
Although AmigaDOS isn't UNIX compatible, it had lots of cool features.
And in what concerns having a GUI with REPL-like CLI, workstation OSes from Xerox, ETHZ and the last iteraction of Plan9, Inferno.
Maybe I'm looking for the equivalent of a flying car, but I wish someone would come up with an OS, or even just an OS feature that makes people say "wow!"
Is there a college lab or a research department or some other place where people are trying really different things?
Then, after all that, come back and tell us that a modern desktop is the utilitarian endgame of OS innovation :).
Do you have other writings?
Are there any books about this?
Do you think about this often?
Some of the folks in this camp are firmly in the traditional Unix "camp" and mock cat -v, but there's a pretty refreshing diversity of views there. Some swear by the Unix Haters' Handbook, others come from (what's left of) the Amiga community.
I don't know if there's a central gathering place, but if you look around the Fediverse, IRC, various tildeverse servers, you'll eventually run into someone :).
A 1995 book that somehow ended up in my old home town's public library called Tog On Software Design, describing a project at Sun to build a desktop of the future, was one influence as a kid. Definitely check out the celebrity endorsements on Amazon.
https://norrisnode.com/tabs-pads-and-boards/
http://lowendmac.com/ed/rosen/10ar/ubiquitous-computing.html
https://www.ics.uci.edu/~redmiles/ics203b-SQ05/papers/Weiser...
RIP Terry Davis.
C/C++ programmers have been building their own hobby OS's non-stop since Linux came out. There's lots of example code out there for initializing the bare metal, writing bootloaders, task switching, controlling the MMU there for those languages. I even got a certain distance myself nearly 20 years ago, and I'm a complete hack. I only gave up because I never figured out how to initialize VESA graphics from protected mode, and I sure as hell didn't want do it with 16-bit code from the bootloader.
We all know what a disaster area C/C++ are, the number of CVE's we see in the Linux kernel and userland because of this, etc etc, we hear about it every day on HN. So I personally like to see some of those people put their money where their mouth is. I think that would be an innovation.
Although it's not clear that it has a GUI toolkit up to the level of complexity that SerenityOS is showing though. If it does it's not being shown off, and I'd love to see the performance.
I'd love to see more examples if anyone can find them.
Well that makes sense on Intel anyway, the first stage bootloader has to fit into a single 512-byte sector (the MBR) doesn't it? Not sure if this still true in these modern days of EFI.
Great to see examples that are getting close though, although I think my point stands that they are nowhere near as numerous or complete as their C/C++ counterparts.
As far as the UI is concerned, are you talking about increased productivity or just looking for the wow factor? I'm not convinced a change in UI paradigms is necessary for what people use computers for. Touch UIs are OK for consumption, but I feel severely handicapped without a keyboard/mouse when doing any moderately involved task. Email - slow, chat - slow, browsing - slow, file/photo/document management - slooow, content creation - slow, gaming - okay, programming - heh.
To me to Windows95 UI (not the OS) is peak productivity usability. All UI innovations that came later were just distractions except maybe tiling window managers and the "Exposé" function in Mac Os X which later was completely botched in Snow Leopard. At some point no further innovation is required.
I'd rather see innovation on another level: Do away with C and introduce a new programming language which gets rid of the need for a MMU or preemptive Multitasking at compile time.
The elephant in the room is Spectre. It's no longer clear you can actually keep any secrets inside an address space on speculating CPUs. So if all your code runs in a single address space, even if it works in theory, it may not work in practice. But this opens up the interesting research problem of Spectre mitigations :)
One idea I've wondered about is whether you can use the very new Intel features to rapidly change access permissions on page table entries, such that you could have a totally unified but segmented heap, but entering or leaving lower privileged code modules triggers a fast address space transition.
[1]: https://en.wikipedia.org/wiki/Inferno_(operating_system)
At its heart it’s a lie: network resources are not files. They are not yet in RAM.
Unifying two abstractions into one is great, IF the data behaves in a perfectly analogous way. Otherwise the “unification” actually makes the domain of interaction more complex.
There’s a reason HTTP has succeeded.
Don't forget search. Especially if it can correctly guess what the user is probably trying to get to. Search in Vista let me ditch SlickRun for quickly getting to stuff.
"Do away with C and introduce a new programming language which gets rid of the need for a MMU or preemptive Multitasking at compile time."
You'll still need it for hardware or compiler failures. Cosmic rays being a big source of them. On compilers, they'll sometimes optimize away security checks. Then there's side channels which require low-level view of hardware and software interactions. Probably better to keep the MMU's or some hardware protections for defense in depth.
They have all been rejected by the community at large, sadly.
I have a different conception of what a desktop OS should be than you seem to, though. I want one that embraces simplicity. It should elicit a resounding "meh" because it doesn't do anything particularly impressive and is just a really good execution of already existing ideas.
I happen to (wishfullly) think the next major OS will have capability-based permissions at the core level, a model a bit like Mobile permissions but composable. Good thing that Google's Fuschia is based on capabilities. It's in C++ though.
From a UI perspective, I've found this to be very innovative and interesting - but I'm not sure how practical it is:
...and in a more open fashion and inspired by it:
Both of these appear to be something known as "tangible user interfaces" - or are a subset of such; if you google around, you'll find a ton of research on such systems. A similar kind of interface is the "augmented reality sandbox":
https://arsandbox.ucdavis.edu/
And indeed AR is a part of all of this; but rather than wearing a headset (or using a phone) to overlay virtual objects or such on the real world, instead projectors are used, which in effect makes it easier to use and more natural, while also allowing for easier collaboration.
I have a theory that LISP or a LISP-inspired language (perhaps LOGO) might also be something that could be incorporated into a new system (I honestly believe that both of these languages haven't been as fully explored as they deserve - LOGO moreso than LISP).
Lastly - the idea of a DSL (Domain Specific Language) might also have relevance to a new paradigm; I've mentioned this particular offering in the past:
https://www.jetbrains.com/mps/
https://github.com/JetBrains/MPS
I'm only as familiar with it as what I've read about it, both on JetBrains site and elsewhere, but it looks both interesting and promising.
In many ways, I think the state of modern personal computing is evidence we are, in fact, in the darkest possible timeline for it.
They're still useful for making music and get used for that occasionally, but I honestly love just booting them up and using the Finder. I wish I could still use them for communication somehow.
Bitlbee + IRC client.
Also, and surely not at all correlated with my youth, it was the peak of good music.
It was based on NeXTSTEP, first released in 1988 and developed up to 2000.
rox filer for the file manager and shortcuts and it’s pretty nice
Unless you use a CRT, i'd recommend to get a 1366x768 monitor with a VA panel. The low DPI makes pixels look crisp, if you game you'll get great performance even with low end hardware while you'll be able to play a lot of older games in a natively supported resolution (1024x768, centered) without UI issues and (the reason for VA) you get deep blacks (not as deep as OLED but certainly better than the washed out garbage you get with IPS panels) which are great on both movies and games. The main negative of VA is low refresh, but unless you are playing competitive games you should be fine.
If you use a CRT, then carry on. Although note that those guns wont live forever and they do wear out. I use mine only on the occasion i want to play some old 2D game that simply doesn't work with anything other than its "native" resolution.
But yeah, if there was a brand new one i'd buy it without much thought.
I'm never going to ask anyone for money, but I'm extremely grateful that some people have chosen to support this project financially.
The only "goal" I have set up on Patreon is hitting 1000 supporters. If things ever get that crazy, I'll stop whatever I'm doing and focus only on Serenity. For now that's just a fun daydream though :)
I originally thought I would learn something new instead of programming, but then I started working on this and got caught up in all the fun. It wasn't a "business decision" of any kind :)
I sympathize since I am also doing open source projects full time so I know that the struggle is real!
Unfortunately although I know C++ I don't really know much about OSdev. If there's any way I can help, I have a few months of free time (summer break) that I'm looking to fill!
Wanna write a Tetris game for an obscure new OS? Here’s your chance! How about a bitmap editor (like MS Paint)? Maybe some cool graphical effect demos? Perhaps you’d like to write a telnet client? An FTP client? Or a disk usage visualization tool! Just look at all the stuff that comes with an OS like Win2K or your average Linux distro. :)
My short term goal is to make a settings manager.
And staying within a Unix-y paradigm means that if some part(s) become too good, they can be semi-easily ripped out and turned into portable libraries/programs instead.
Imagine if working a car was as complicated as your average desktop OS. There would be a lot less people driving that's for sure.
In the immediate future though, the visual builder tool is limited to generating C++ that you then glue together with the rest of your program. But one day :)
https://github.com/SerenityOS/serenity/tree/9153666e72effdf5...
Also using something like boost ties you to a specific version of boost due to the wonders of c++ abi - your OS libraries can’t expose anything that explicitly or implicitly includes things that aren’t ABI stable.
(I’m sure kling is already requiring 100% stable abi :) )
Its use of the C++ language strives to be modern. There is no attempt to use modern 3rd-party C++ frameworks.
I'd rather use something like Qt (which is far from great, but better than STL/StdC++). Well, personally i'd rather not use C++ in the first place, but if i'm going to use C++ i'd rather use something else (which frankly, would be my own container and string classes that provide a much better API).
There are some parts of the code where I don't want to depend on anything other than the C++ language. Some folks are working on porting parts of this to some very constrained platforms, and bringing in boost or even STL would really gum things up.
But let's also be honest: It's fun making string and vector classes, and designing them to work just the way I like. :)
Just curious: Why did you choose to take on the entire OS instead of building on top of an existing distro as a base?
Speaking more emotionally, I feel like *nix used to be a lot of fun and full of hacker's spirit, and it's just not like that anymore. Things aren't simple anymore, and they can't easily be taken apart and understood in an evening, like they used to. So I figured I can just make my own then, just the way I like it :)
(A lot of performance work followed after that recording. There's still a lot to do, but the 30 second compile time is now down to ~2.5 seconds. 80% of the speedup was from implementing DMA disk transfers, and the other 20% from a faster malloc().)
Windows 95: https://en.wikipedia.org/wiki/Windows_95#/media/File:Windows...
vs Nextstep: https://en.wikipedia.org/wiki/NeXTSTEP#/media/File:NeXTSTEP_...
Yeah they're both light grey, but other than that, they don't really look similar to me...
>90s
>aesthetics
wat?
Modern GUIs are ridiculous and stupid and overcomplicated. Much like most of modern computing.
You're over-romanticizing the past. There were plenty of apps even in the Windows 95 era that drew their own widgets and only half-respected the colors you set in the control panel, and that consequently looked like pureed ass if you changed the colors from the default.
One of the big complaints about Java used to be that it didn't use native widgets and consequently looked out of place, but nowadays that is basically every application.
It's ridiculous.
"Flat" UI is both lazy and an overreaction to the ridiculous skeuomorphism that plagued us years ago.
I like to think that by trial and error we will eventually get back there. Microsoft is already moving away from the flat aesthetic to a more "materialish" design.
I'll continue iterating on the look though. Here's a more recent demo that shows a bunch of pixel-related improvements: https://www.youtube.com/watch?v=pS56tlXFmEk
It does a lot to make the UI feel less dark
I should eventually add color theming so this can be done from a single place, and then I'll be sure to add a warmer option, too. And probably make it the default after I see how nice it looks :)
Also for reference for those unfamiliar with the difference in greys: http://actsofvolition.com/images/screenshots/Display-95-thro...