Hacker News Reader

> made me realize how extremely unixoid almost every OS I know is

It's a bit disappointing that every time somebody decides to write their own kernel, the first thing they do is implement some subset of the POSIX spec.

Indeed. But to be fair, in today's world it is not only somewhat hard to imagine an OS that does not rely on things such as a filesystem with opaque binary files, directories, and hierarchical paths connecting these, but in order to properly communicate with the "outside world" you more or less have to have those things. Even AS/400 / IBM i added IFS, which is basically a bolted on hierarchical filesystem, to their system a long time ago by now.

Still sad that it has to be that way. I've long come off of thinking "everything is a file", and a file then just being a binary blob, is a good thing. (That's not even talking about other concepts from Unix we take for fully granted yet.)

Indeed. This reminds me of Rob Pike’s famous 2000 polemic “Systems Software Research is Irrelevant,” where he laments the decline in innovative operating system designs of the era.

Additionally, we need to consider the career incentive structures of researchers, whether they are in industry, academia, or some other institution. Writing an operating system is difficult and laborious, and coming up with innovative interfaces and subsystems is even more difficult. When a researcher’s boss in industry is demanding research results that are productizable, or when a researcher’s tenure committee at a university is demanding “impact” measured by publications, grants, and awards, it’s riskier betting a career on experimental systems design that could take a few person-years to implement and may not pan out (as research is inherently risky) versus pursuing less-ambitious lines of research.

It’s hard for a company to turn its back on compatibility with standards, and it’s hard for academic researchers to pursue “out-there” ideas in operating systems in a “publish-or-perish” world, especially when implementing those ideas is labor-intensive.

The widespread availability of Unix, from a source-available proprietary system with generous licensing costs to universities back in the 1970s, to the birth of FOSS clones and derivatives such as Linux and the BSDs, has made it much easier for CS researchers to not need to reinvent the OS wheel, instead focusing on narrower subsystems, but at the cost of discouraging research that could very well lead to the discovery of whole new ways of carrying things, metaphorically speaking. Sometimes reinventing the wheel is a good thing.

I still dream, though, of writing my own operating system in my spare time. Thankfully as a community college professor I have no publication pressures, and I get three months off per year to do whatever I want, so….

For sure. Also, we simply live in a world where computers, and their operating systems, are giant, impossibly complex structures. No single person can even fully know in all detail even what you would relatively consider a small part of neither any commercial computer, nor their operating system. In my job, I work with 15000 (!) page specifications, and that's only concerning one part of very many.

That pretty much guarantees that change can only be incremental.

> It's a bit disappointing that every time somebody decides to write their own kernel, the first thing they do is implement some subset of the POSIX spec.

Well, not quite _every_ time. For example, I’m deliberately not doing POSIX with my latest one[0], so I can be a bit more experimental.

[0] https://github.com/roscopeco/anos

Kudos for doing so! This is seriously a great endeavor. Regarding its relation to UNIX concepts, I do spot a classical hierarchical file system there, though. ;) Is it only an "add-on" (similar to IFS on IBM i), or is it fundamental?

Thank you!

At this early stage, the filesystem exists only to prove the disk drivers and the IPC interfaces connecting them. I chose FAT32 for this since there has to be a FAT partition anyway for UEFI.

The concept of the VFS may stick around as a useful thing, but it’s strictly for storage, there is no “everything is a file” ethos. It’s entirely a user space concept - the kernel knows nothing of virtual filesystems, files, or the underlying hardware like disks.

That makes sense.

Kudos for trying something else.

Ah… but don’t you need some POSIX to get tools like grep and vim working? Who really wants to live in a universe without grep and vim?

"Who wants to live without horses for transportation?"

Isn't the point that we don't even consider alternatives?

POSIX is dead on arrival. Horrible API's. You still cannot use grep to search for strings, only byte buffers (not unicode aware). Horrible signals and concurrency concepts. Blocking IO. Unsafe system languages all over. Monolithic kernels.

Very true!

I think it’d be amazing if things like asynchronous I/O were the standard instead of an afterthought in both kernel and user space.

Also feels like Microsoft got something right with their handles… (Try doing select/poll to wait for an socket and a semaphore — without eventfd or something else bolted on)

> every time somebody decides to write their own kernel, the first thing they do is implement some subset of the POSIX spec.

yet most of them target virtual machines. Nobody is programming the bare hardware anymore.

Qemu is nice, but you have to read 5 internet pages to start it properly.

That feels like an orthogonal problem to whether or not POSIX is implemented, though.

I've read that in AS/400, everything is an object, rather than a file. Could you expand on that at all?

Honestly, it's a bit much for a single HN comment. But I've look around, and I found this, which upon first glance gives a good (and not too long) overview: http://www.varietysoftworks.com/jbaldwin/Education/single-le...

Crucially, it also describes the "single level store" that everything lives in. In short, the boundary between "main memory" or "hard disk" or other things is abstracted away, and you just have pointers to "things" instead. While in UNIX accessing some piece of main memory is fundamentally different from, say, opening a file.

Even though UNIX has a little bit of the opposite concept of trying -- but in my mind failing -- to represent as much as it can as a "file".

It's quite tricky to explain, but yes, everything in AS/400 is an object. There is also the notion of a context, called a library, that these files-as-objects exist in. AS/400 files very frequently act in a way that today we would describe as databases (like sqlite files, say) and the library context guides how information ('records') in them is returned, eg FIFO, LIFO, etc.

I think the best explanation is contained in this very old porting guide from IBM that explains how to move UNIX programs to AS/400. It's written in a remarkably un-IBM fashion, not at all straitlaced.

https://www.redbooks.ibm.com/redbooks/pdfs/sg244438.pdf

For any experts out there, please correct me, it's been 30 years...

I’m not ready for 1995 to be 30 years ago …

I just finished reading "Fortress Rochester" by Frank Soltis, the designer behind the IBM System38/OS400/iSeries sw.

It's a fascinating book, very approachable for the density of the technical detail contained, and shows how very different choices were made w.r.t. how OS/ 400 system software was designed, and how hardware was developed to support it.

As I understand from reading it, there's like three layers to the software-

-->Things you'd think of as applications on Unix, running in user space...this includes the DB/400 database engine, queue systems etc.

-->Machine Interface (MI) components, which include the management of the single-level store, I/O devices (OS/400 still supports dedicated mainframe-style I/O channels to an extent), and compilers/interpreters. All of this layer and above are considered "Technology Independent" software, where programs written in C/C++, RPG IV, COBOL, Java etc. are compiled into 128-bit object code that gets translated into PPC instructions on the fly.

-->the SLIC (System? Licensed Internal Code) which is referring to both some of the iSeries firmware, stuff that might be considered part of the kernel in Unix, as well as the PowerVM hypervisor.

The craziest thing (to me) about the single-level store is that there's no user-visible filesystem to administer; objects created in memory are automatically persisted onto permanent storage, unless you explicitly tell the system otherwise.

The OS/400 objects also have capabilities built-in; i.e. things like executables can only be run, and not edited at runtime; these are flagged during memory access by setting bits in ECC memory using a dedicated CPU instruction on Power CPUs that was added explicitly for OS/400 use.

For someone who is used to Unix, iSeries a really fascinating thing to learn about. Pity that it's not more accessible to hobbyists, but as the Soltis boook makes clear, the purpose of the design was to satisfy a specific set of customers, who had well-defined line-of-business applications to run.

> I just finished reading "Fortress Rochester" by Frank Soltis, the designer behind the IBM System38/OS400/iSeries sw.

There's a copy of the previous version, Inside the AS/400, also by Frank Soltis on archive.org

https://archive.org/details/insideas4000000solt/

Also it is one of the successful bytecode OSes, were userspace is mostly bytecode based and AOT compilation takes place at installation time, or after updates.

If you could wave a magic wand and suddenly all Unix systems (and variants such as Linux) could be instantly changed to suit your wishes, what would you wish for?

For me, it would be:

    - removal of 'root' user and replaced with various predefined sysadmin groups and standardized audit logs for each group
    - get a batch processing system (similar in spirit to z/OS but without JCL). It should allow you to see a list of the jobs you've submitted, cancel individual jobs, re-order existing jobs that haven't started, have a standard place to see stdout/stderr logs.
    - get something like AIX's smit/smitty for sysadmin tasks
    - have either ZFS built-in or something with equivalent functionality
    - sensible and easy-to-use containerized/jailing capabilities
    - built-in support for append-only file permissions
    - ability to quickly/easily adjust priorities for all processes belonging to a user
    - ability to quickly/easily cap a user's processes cpu/ram resources (without having to use a container/jail)

- Accessibility built in, eg the OS takes care of adapting to what the user can do ( vision, subtitles, adaptive keyboard input etc).

- trivially replaceable kernel, to encourage research and experimentation, real time use, etc.

- ruthless, consistent separation of user data, user configuration, and an unbreakable standard for where programs get installed. Just look at dotfiles, hier/LFS, the windows Registry etc to see what a mess this is.

- a native grid compute protocol that allowed any two or more computers to merge into a single system image and automatically distribute workloads across them. We have clustering and the insane complexity of k8s today but it imagine something as easy as "Bonjour and Plan 9"!

Interesting. What you are wishing for is, in essence, (Open)VMS. It exists!

Solaris from 10 onwards to OpenIndiana/illumos/SmartOS/OmniOS has some of this.

I don't know if you could remove the root user, but you can have a lot of control using pfexec. This might require you to design and assemble it however.

ZFS of course for ZFS filesystem.

Solaris Containers are great: sensible and easy-to-use containerized/jailing capabilities

Solaris has "projects" and the FSS (Fair Share Scheduler) which should allow you to cap in both absolute and relative terms (like a share of CPU time or RAM) on a per-user or per-project/group basis, even if not in a container.

As well, you can create virtualized network interfaces called VNICs and have bandwidth management by VNIC or by port (e.g. port 443, port 25). So you could always reserve say 10% of bandwidth for SSH traffic so you never get starved, etc.

Sounds like systemd :-).

At home I had an 8-bit so played around with CP/M and BBS'ing trying to gather shareware (software was too expensive for a kid of course). My first serious experience of "big systems" were VAX 6000 (1GB RAM in 1991!). I'm reminded that many of the useful concepts of that system, some of which you list.. never made it into Unix. One of the reasons I stuck with Solaris for so long later is because it had a few of them.

My first experience with a multiuser system was also on VAX/VMS. It felt more complete and cohesive (perhaps not the exactly right terms...) than early Unixes. I still experiment with it in emulators.

For me, it was Plan 9. In some ways, it was mind-bending to not have root. I haven't played with it recently; maybe it has gotten easier to play with alternative OSes since I tried 10 years ago.

Already the use of PL/I, while we keep trying to add safety back into C.