Ideal OS: Rebooting the Desktop Operating System (2017)
joshondesign.com
joshondesign.com
The text reminded me of the way FTP clients pop up the default text editor. Such a boring technology, no one thought anything of it. Why isn't my mail client doing this? It doesn't seem hard to make web browsers do it either. Trying to do code highlighting in a <textarea> is possibly the worse technology we have. We should really have a dedicated client for forms as well which seems to logically fold out of a database file system?
The text also reminded me of Picassa the photo organizer, as soon as I had everything scanned and organized in it it felt like I couldn't use the normal file system anymore. It was such a clumsy clunker by comparison.
Also hilarious was having to install Thunar because Nautilus couldn't open my big folders.
I was also reminded of textpad. While Atom couldn't open files larger than 2mb the old textpad on a pentium 4 opened the 1+ TB wikipedia xml dump in 0 seconds. Editing was not sluggish, replace macros were not sluggish and saving the document worked just fine. The thought stuck with me: How the f did it just do that??? I think the first release in 1992 probably explains it. Squeezing wine from a rock was still a thing back then.
Users don't want to start from scratch to learn every program they need. Also, they often bounce off of games with controls outside their comfort zone.
As a fan of desktop GUI software that conforms to well-developed platform UI guidelines such as the old Apple Human Interface Guidelines, I believe the rise of mobile computing and the Web hurt the desktop. Software companies ship cross-platform applications using least-common-denominator frameworks instead of writing apps with UIs that are tailor-made to each platform. While this saves development time, and while it’s popular to say “having an Electron app is better than having no app,” it makes the desktop experience less consistent and more difficult to use. In addition, the desktop UI has become more mobile-like even in ways that are inappropriate for desktop computing, such as the proliferation of hamburger windows, the tendency to hide settings (thus requiring additional clicks to perform operations that used to be done in one or two clicks), oversized bars, title bars being crammed with buttons (making it harder to move windows via dragging the title bar), excessive padding, etc. There is a difference between making things easier for users versus dumbing things down, but I feel that a lot of software designers these days don’t know the difference.
I would like to see a return to UI consistency across applications, and I’d also like to see a desktop environment that caters to power users.
A dream of mine is some type of “component-based” desktop operating system, where everything is an object that can be manipulated through either a GUI, the command line, or programmatically. The backend would be inspired by technologies like Smalltalk, GNU Emacs, Lisp machines, and Apple OpenDoc, but with modern considerations such as security and parallelism.
And not having open or shared standards was viewed as a feature to drive down competition.
ATT started out UNIX with their hands tied. Hence BSD was birthed and then later ATT tried to put the genie back in the bottle. But it the economics were different. Hardware costs were the main driver. The OS was a given.
Starting over sounds great. But if history repeats itself, which it usually does, then we are far more likely to see something take (copy) ideas from current OSes and evolve them rather than a radical sea change where we star over.
Also now that Microsoft is shifting (shifted?) to a business model where selling OSws isn't really profitable will their behavior change?
Got some bad news for ya...
Microsoft keeps failing at making .NET more relevant across the whole OS stack, as WinDev keeps pushing for C++ and COM, no matter what, and did a big mess of how UWP could have improved the whole experience, after the Longhorn sabotage from WinDev.
There is a key difference between today and the period of the 80s/90s when we had true OS efflorescence. Back then, the real concern was compatibility of software and in particular file/data formats. These days we have so many data format standards and protocols that a lot of this interoperability is essentially system-agnostic. I think that's a huge advantage.
Just want to point out that starting an OS from scratch, even if reusing an existing kernel, is a monstrous epic.
I find it difficult that one would be able to just use existing kernels like Linux without some considerable changes to support a more secure userspace (which seems like a requirement in order to do wild new things).
But, I have commented about most of the article, and about my own ideas of operating system design (which is capability-based); there are some similarities, but many things are done differently. However, some of the functionality they describe would be possible with how my ideas are doing, too.
I have written some on Hacker News as well, such as: https://news.ycombinator.com/item?id=40651272 https://news.ycombinator.com/item?id=40676995
(I had also written some on Usenet, and also on IRC.)
Perhaps I could set up a NNTP for discussion, that work can be made to write the specifications (which can be changed as the actual implementation is made, in case that reveals problems with the specifications), and to discuss implementations.
I do not understand your motivations here: inventing new Internet protocols, apparently intentionally misusing terminology, etc.
("I set up a NNTP" makes no more sense than "here, try to connect over brown": NNTP is a protocol and the normal usage is to set up a news server over which one communicates via a news client over NNTP.)
Maybe this is meant to be funny, or sarcastic/ironic, or performance art, or something else; I can't tell. What it is not is helpful, informative, or clear.
I do not know what reply was expected.
> Maybe this is meant to be funny, or sarcastic/ironic, or performance art, or something else; I can't tell. What it is not is helpful, informative, or clear.
It is meant to be helpful, informative, and clear. Unfortunately, that is not always the case.
It is none of these. Please try harder.
I was interested in following these links and reading about this, but your strange attitude, opaque and incomprehensibly cryptic responses, and general lack of cooperation mean that I have now lost interest.
If you want to play silly games, invent your own protocols and your own unique way of misusing existing language, post "links" that only work using your own client software, and other strange lateral responses, that is of course up to you. Perhaps it is some sort of act.
But do not expect everyone to like it or to be amused. I am not. I am just annoyed. I suspect most people won't even give it that much attention.
(Simple explanation: Use port 1517. Send R and then a space and then the full URL and then carriage return and line feed.)
The file format is a binary format, although there is also a plain text file describing the protocol and file format at: scorpion://zzo38computer.org/specification.txt
What can connect to it? How do I use it? If I need to implement my own protocol, then the answer is "no" followed by "what are you on?"
https://gavinhoward.com/2024/05/a-new-architecture-for-a-gui...
- This API should be programming language agnostic (like, with a well defined ABI). Ideally an IPC based one, with a well defined wire format and protocols.
- Not a singular API, but an “API system” with extensibility taken into account from the beginning (with things like versioning). It should be able to support arbitrary new API definitions so that other apps could consume, and also produce.
- Such an API system should be used by the OS itself too, of course. So the OS would have the resulting benefits and the whole system would be more uniform.
I guess you and I have different use cases and definition of "modern application usage". I assume you spend most of your day in a GUI, while I spend most of mine in a terminal.
Pipes can handle binary data and sure you could pipe Skypes output through sed or awk... Though something like ffmpeg is more likely, and it will happily take stdin as input.
I just want eyeball tracking for the mouse and voice to LLM to bash.
So true. And so is every company. They have very little incentives to make your data accessible. Or is it "their" data?
You can also go "turtles all the way down"... and discover: the object oriented programming model. It also hides, eh "encapsulates" your data, you only get access through messages.
This is what operating systems are build on. And it works very well for Microsoft / Apple / Google.
I think Unix had the right idea. Make the data part of the outside and programs only process this data. Programs should be stateless.
Unfortunately Unix kind of got stuck in the text based world. Text is really great, because it's so visible. But there is no reason binary data can't be the same. You'd need to develop general purpose binary file format. That can be opened and edited outside of any application. Which would let you extract for example a single image of a video to edit in a paint app.
Then make all of this live update, so that the mouse for example is just a struct with position and other data you can use everywhere.
Then let us have history on all the data.
[1] https://github.com/organix/uFork
[2] http://www.dalnefre.com/wp/2022/08/memory-safety-simplifies-...
I agree with the four items listed in "Problems with Today's Desktop Environments and Applications". However, these are not the only issues. (I even thing the older Windows 3.1 and even older UI were well enough; you don't need such fancy graphics.)
Your "Monolithic Applications versus Composable Tools" also is reasonable. What you list about Unix philsophy are the good ideas of Unix philsophy; it is too bad that many modern programs don't support it. I would allow command-line and GUI and easily interact between them.
My idea involves capability-based security with proxy capabilities, and a program can send/receive/wait with capabilities it has access to; a message send/received consists of bytes and/or capabilities. (It is also similar than actor model in some ways.) This is what the kernel does (and programs can create their own capabilities called "proxy capabilities"); all I/O is required to use capabilities. (When a program starts, it must receive an initial message containing some capabilities, so that it can do I/O.)
My idea also is to make the format which will be called "Common Data Format"; it can store documents, databases, and other stuff, and can be used in command-line and GUI. (It also involves some of the ideas of OpenDoc, and of TRON Application Databus, but is different from both of them.)
There are also many other considerations: hypertext file system, window indicators, security, TRON character code, network transparency, emulation, etc.
About "Not a New Operating System", my idea would be that the specification can be done and implementations of the individual parts can be done independently and interchangeable, and that an implementation can be made to run stand-alone or inside of another operating system (something that Inferno also does); this can avoid the problems of drivers at first, while when it is made stand-alone, drivers can be made for it in that way, too. However, I do not intend to include a web browser by default.
(For making such a thing on Linux, I don't know exactly what capsicum blocks; capabilities are supposed to required for all I/O including accessing the current date/time. My system idea is similar like if you have only socketpair to create new channels of communication, and being able to send/receive messages that may contain SCM_RIGHTS, and the ability to wait for objects.)
Just look at what Android had to do to implement a secure platform with somewhat of a well defined non-1980's OS API - it basically removed the Unix part of the OS and built a Microkernel-like thing on top of it.
Driver support would be a very important part though.
...Huh? Pretty much all Android adds to the kernel is Binder, which is really a performance optimization for it's version of IPC, which today has equivalents that mean it probably wouldn't need it's own version now if it didn't need compatibility.
The security model allows native apps, which have access to all the normal libc, or even direct syscalls. Some hardware access is marshalled through userspace daemons, which may implement security for things like camera/microphone access, but that's been a pretty common thing on other unix-style OSs for ages.
Though the separation of each app into it's own UID might not have been the original intent, but it seems to work implemented with those same unix capabilities.
Yeah, most apps probably don't use POSIX APIs directly, and instead the java Android APIs, but they are very much build on top of the "crufty 1980s OS APIs", rather than bypassing them.
Binder is not just a simple IPC system. It requires userspace parts which aren't even the parties who are talking through IPC. Like you said it has lots of performance optimizations, and that requires it hooking up to a number of kernel subsystems. At the beginning it was met with a lot of resistance from kernel maintarners when tried to be merged.
Why did Google came up with it? When talking about the 'Android sandbox' people usually talk about how Android gives each app an different uid/gid. But that's just the beginning of the story. It wasn't enough for sandboxing needs since the beginning. Thats why they took the binder idea from BeOS and used on Android.
Since then Google has blocked everything. Native code is needed on any practical system, but today's native code has access mostly to nothing, just the common needs to run (like libc which is used due backward compatibility). The proper way to get something is through Binder (and even that is fine-grained controlled on a per process basis).
Linux and Unixes in general don't implement true capabilities. Android does through Binder, but since Binder lives “inside Linux” everything else has to be blocked to make these capabilities useful.
> Some hardware access is marshalled through userspace daemons
Most of stuff are, at least initially, orchestrated through userspace software - daemons like you said, but mostly HALs (which are also daemons).
The main takeaway is that binder is _the core_ of Android userspace. It's tightly integrated with SELinux and everything passes through it at some point. Permissions are enforced through it. In the end, the whole “binder subsystem” is architectured like a Microkernel system.
At the point that binder was created, there wasn't a good kernel-level IPC system integrated into linux, true, but I'm not sure I agree with the idea that it's in any way tied into the capabilities/permissions system outside from being a communication channel to the userspace daemons things that do.
And native code does have access to every service on the device - just as any app. It's just more painful than the Java layer, and some things aren't guaranteed to be stable over API versions. Less useful for apps, but still not part of the security model. At an extreme level, the java code is run in the same memory address space as any native code, there's no protection from that. Hell, some apps even used to patch dalvik/ART live. It was insane and fragile, but entirely possible.
And the "disallowed" syscalls are those not exposed by libc though some more are restricted for user apps (mostly around UID management, which is the big "major" deviation from standard unix, and system stuff like loading kernel modules or rebooting...) [1] So maybe it doesn't allow some linux extensions, or could remove the "old" versions of replaced syscalls (when arguments were found not to be large enough, flags arguments added etc.). But there's not really anything missing from there as used in 99% of unix userspace programs anyway.
So I maintain it's still a system build on top of a unix kernel and libc with a few additions on the side, and all that is still accessible to apps. Perhaps you think I'm including all the stuff people associate with modern "Desktop Linux"? Like X11 and/or wayland? Or the entire FHS? That's the main thing Android has completely divorced from, but that also isn't "Unix", or even really from the 80s even if it has some ancestors there.
[0] https://android.googlesource.com/kernel/common/+/refs/heads/...
[1] See SECCOMP*.txt and SYSCALLS.txt from https://android.googlesource.com/platform/bionic/+/refs/head...
I'm going to describe some flows so you can have a perspective on what's going on. I'm describing them as I remember when I worked with this stuff.
First, you have to know about SEAndroid which is a thing built on top of SELinux. It doesn't add anything new from a kernel perspective but there's a whole 'firmware image build-time framework' to _legislate_/declare permissions about all sensitive stuff on the system. Like, you want to add a new daemon to the system? Maybe this daemon will serve some AIDL (such daemon is called a service)? Oh, then you should declare its SELinux stuff, like "it's a daemon", "it should have access to sockets/whatever" and "it will serve such and such AIDL interfaces" and so on. You should state the things it should have access to. The important part is that the SEAndroid stuff has 'Binder services' as a first class citizen, much like a fs path, socket or anything else SELinux is capable of legislating about (even though the kernel and kernel's binder knows nothing about binder services from an SELinux perspective). A good part of these policies don't talk about vanilla Linux stuff (like processes, files, sockets, etc.) but are about _services_ from "binder realm" - the kernel doesn't know about and ignores them. During the system image build process these policies are "compiled" and put somewhere on a filesystem.
When Android boots, after all the device-mapper mess (which, as most things, is a subject on its own) eventually the "base" filesystems are mounted (this mess comes in part from Project Treble). At this point there's no SELinux stuff loaded (which means, IIRC the SELinux status from the kernel is "off"/insecure/whatever). Then comes the time when the SEAndroid/SELinux policies are to be loaded. A system process looks up and loads the SELinux blobs generated during the system's image build process, and then enables SELinux.
Ok, now a flow from runtime:
You have process A, which in this example will be on the client role, and process B which will be the server. Process A comes from a program A which is a plain ELF binary on the filesystem. Same for process B. The SEAndroid policies loaded during boot define the SELinux contexts associated with process A and B. Program's B context includes the list of AIDL interfaces/services it implements (or serves). And program's A context includes an statement that it should have access to such and such AIDL services (it states nothing about program B or others who implement the AIDL services).
Every binder driver device is exposed to userspace through the /dev fs and they each need to have an associated _service manager_ process. When I worked with these there were 3 binder devices, and so 3 associated service managers. Originially there was just 1 binder. So process A needs to have access to, say /dev/binder, and this is enforced with vanilla Linux acccess controls. Same for process B on this part. Process A tries to establish communication with whatever process is implementing the AIDL service it's trying to talk to (process B) (services have a string name, it's a binder thing). This attempt actually ends up on the binder's service manager, which asks the kernel's SELinux policy 'Can this process A here access that AIDL interface?'. The kernel doesn't know about AIDL interfaces, but the SELinux policy blobs downloaded into it earlier encode this info in such a way that such questions can be asked and answered. If so, it awakes/starts process B and communication goes on.
As an addendum, binder supports transporting file descriptors and that's what I meant by true capabilities. That has lots of implications and caveats given it's going on a Linux-based system but that's another conversation...
> And native code does have access to every service on the device
No, it dependes on which process, or more specifically, which SELinux context the process running that code has been attributed to.
> At an extreme level, the java code is run in the same memory address space as any native code, there's no protection from that
Yes, on the level this conversation is concerned about, it doesn't matter what you program is doing, if it's a Java thing, python, web browser, if it's a code dynamically linked into your process from a shared library object - what matters most is that it ends up calling syscalls and that each process has an identity from the SELinux perspective inside the kernel (so the SELinux subsystem inside the kernel knows which process is calling a given syscall).
> Perhaps you think I'm including all the stuff people associate with modern "Desktop Linux"? Like X11 and/or wayland? Or the entire FHS?
My comment about this part would be that libc is... not related at all. It's importance comes from 'Oh, this is C/C++ code so it doesn't actually issue syscalls, it does all that though a libc because, well, this is C and Unix'. On this abstraction level I would put it as just an implementation detail as apps being programmed in Java.
- New IPC system: presumably it's abstracted, so can use any existing IPC for implementation. Or, purely as optimization, use Android's Binder or bring back bus1 (it was such a great idea)
- Document Database: pure userspace; in the most extreme case use raw block device
- Compositor / WM / Windows: this was always userspace already
- Rebuild the apps: userspace as well
Then once you new fancy APIs are ready, you use seccomp and/or per-app users and/or namespaces to isolate apps from everything except the few things people want.
I agree.
The hard part is, as usual, on the details. How do you take away stuff so that you still end up with an usable general purpose OS? That's definitely not easy, specially if you want to make it in a proper way, without hacking things up.
For example, suppose you make the decision to use the Linux kernel, which is a good one - you get driver support, networking, filesystems, and all the good stuff and general primitives for a common OS of today.. Given that what one wants is to remove stuff, in order to make that secure (which is requirement) you'd need to isolate and block everything because on Linux everything is accessible to a process by default, and the common primitives for control are very basic - you'd have to do a _lot_ of work in userspace to block it. It would contaminate your userspace architecture with details and stuff. And then you'd probably end up needing something to make the userspace parts able to communicate with each other, like Binder on Androids case (see my other comment in this thread).
Looks to me like he has some of the same ideas as I had in this talk:
https://fosdem.org/2024/schedule/event/fosdem-2024-3095-one-...
I turned it into an article series:
> Why can't I have a file in two places at once on my filesystem?
Hard links do that, but as usual the UI around this isn't great
> two places at once on my filesystem
(a) the entire permanent storage system
(b) a particular technology for organizing bits on a storage system
(c) a subset of (a) that has been mount(1)-ed somewhere within (a)
I tend to use (c) most often. No idea what the author or you might do.
But hardware vendors have for years, and still do, just say "our use case, or the highway, you don't really own the thing you're buying." And there's not much out there for one to "vote with one's wallet" on it.
"FS can't store a file in two spots" … I want to accuse OP of being a Windows user, but they're obviously not [only]. So that's … an odd critique?
I still think hierarchical+symlinks beats trying to tag stuff, and I've lived through enough decades of AI trying to tag stuff and deciding things like that Mozart is "Rap [explicit]". I don't have the mental capacity for it. Just give me the dumb hierarchy with (sym)links. (And bind mounts.) (But I wouldn't oppose an OS trying it.)
"Let's ditch all IPC" … buuuut adds IPC right back in. Okay :). I suppose it's "only message passing". (And I do think message passing is a great default, and what most things need.)
"Native apps are bloated". The most bloated apps I have are all the Electron ones. I don't think most people are concluding this...
I'd love a multi-item clipboard, but the devil is in the details: sensitive items and large transfers.
> it's the pure text nature that bothers me. Instead of chaining CLI apps together with text streams we need something richer, like serialized object streams (think JSON but more efficient).
You have to consider the bulk cases. But very much yes; I wish at least that Unix had settled on typed bytestreams, like MIME+stream, and some form of content negotiation. Unix scripts are rife with examples of `a | b` where a & b speak different formats and I feel like it took forever to get good tooling around strongly specified formats (instead of just bullshitting with awk/grep/cut/tr/etc.) with tools like jq, yq, xsv, etc. I just wish applications could just have a means to understand "oh, I'm getting this kind of data" or "I have no idea what this is, and I shouldn't just GIGO".
> In our new OS every window is tab dockable. Or side dockable. Or something else
Have you tried a tiling WM?
If you design an OS ... make the FS Unicode only, and ban "\n" from filenames. Life will be simpler.
How far we've come.