Comment by linguae | Hacker News Reader

Comment by linguae | Hacker News Reader

Parent Full thread

Through various market forces, Unix (and its descendants/clones) and Windows killed most of the rest of the OS ecosystem over 20 years ago. There are generations of software engineers and computer scientists who’ve never studied operating systems that weren’t Unix- or Windows-based. Most leading OS textbooks (Tanenbaum’s books, the dinosaur book, Three Easy Pieces) have a Unix slant. Even the systems software research community is heavily Unix-centric; I say that as someone who used to be immersed in the research storage systems community. The only non-Unix or Windows operating systems many practitioners and even researchers may have used in their lives are MS-DOS and the classic Mac OS, and there’s a growing number of people who weren’t even born yet by the time these systems fell out of common use.

However, the history of computing contains examples of other operating systems that didn’t survive the marketplace but have interesting lessons that can apply to improving today’s operating systems. The Unix Hater’s Handbook is a nice example of alternative worlds of computing that were still alive in the 1980s and early 1990s. VMS, IBM mainframe systems, Smalltalk, Symbolics Genera, Xerox Mesa and Cedar, Xerox Interlisp-D, and the Apple Newton were all real-world systems that demonstrate alternatives to the Unix and Windows ways of thinking. Project Oberon is an entire system developed by Wirth (of Pascal fame) whose design goal is to build a complete OS and development environment that is small enough to be understood for pedagogical purposes, similar to MINIX but without any Unix compatibility. Reading the history of failed Apple projects such as the planned Lisp OS for the Newton and the ill-fated Pink/Taligent project are also instructive. Microsoft Research did a lot of interesting research in the 2000s on operating systems implemented in memory-safe languages, notably Singularity and Midori.

From learning about these past projects, we can then imagine future directions for OS design.

Another key issue: the concept of open source barely existed. Sure, you got a copy of the source code used to SYSGEN your RT-11 or RSTS/E system, but it had comments removed and was not for redistribution. In the end, the closest in feel to the old DEC world today would be the traditional Windows command line (not PowerShell).

The rise of GNU and the general open source movement was a reaction to the rug-pull when access to Unix source was restricted, and that gave us Linux (MINIX wasn't initially open source, but could be put into a state that made Linux possible).

The Unix paradigm stuck with us, IMO, because Unix (and later Linux and BSDs) have been ported to so many vastly different architectures. So many other operating systems have never moved beyond (and often died with) their initial platforms.

Thats not really fair to DCL - cmd is much more.. lacking in facilities than DCL is. Nor is it fair to DCL as an interactive environment either.

True enough. CP/M and MS-DOS (which give us the old-school Windows command shell) predate DCL, though; it was a fairly late addition to the PDP-11 operating systems, after it was rolled out in VMS. RSTS/E up to version 8 logged users into BASIC by default, while version 9 made DCL the default run-time system.

DCL to my eyes at least clearly inherited some ideas from TOPS-20.

cmd for all of its braindead nature is surprisingly usable for complex tasks though. I have to use it often, tbh.

So far, historically, no OS has been successful unless (in addition to whatever whiz bang funtionality it had to offer) it provided a process abstraction in which the application thinks it has the real hardware machine to itself. This is the big thing in Unix and others: not how the file system is, or that there are pipes and whatnot: but that you get an address space in which you run machine code for the Intel, SPARC, PPC or whatever. The machine isn't hidden. The OS is in the background; you tap into it via some privileged instruction or trap.

It was not possible to develop just an OS which does this; it needs hardware support. There has to be virtual memory if there is a requirement to juggle multiple such applications each of which thinks it has a whole machine all to itself. There has to be protection: user/supervisor separation, and that protection cannot depend on a trusted compiler, because the application can execute arbitrary machine code produced by a compiler of the developer's choosing (or even by hand). It has to be hardware.

Okay, so successful OSs so far have been ones that provide decent multiprogramming, by following the hardware's model for virtual memory and privilege separation. What about the rest of the shape of the OS; the environment? Like files being just arrays of bytes?

Files being arrays of bytes is a form of democracy. Any sort of structured storage introduced on one OS will look indistinguishable from a proprietary file format which silos your data.

Files are exchanged between operating systems in bytes. No matter what you do, that's not going to go away.

Even some modern enhancements and complications in file representation cause interoperability headaches for the users, when they need to take their files to a different system.

The rich object features are provided by applications. For instance, an object-oriented database can just store its stuff in a block of bytes. Same with a graphical office suite, or a Lisp system saving its image.

Nobody wants an opinionated operating system in which files are fancy trees of objects done in One Way that everyone has to follow who codes for that operating system.

> Nobody wants an opinionated operating system in which files are fancy trees of objects done in One Way that everyone has to follow who codes for that operating system.

And yet there is ChromeOS. There is iOS. There is Android. These are highly opinionated platforms. So I think that yes, people do want these things. Developers in particular want lots of high level APIs because it makes software production easier, which is why so much software targets the web (effectively an "OS" these days). The web follows implements zero of the standard UNIX API primitives yet is very successful.

There's a lot of stuff that can be done with operating systems research, I'm really pretty sure of it. That doesn't mean you can't layer things properly, or that you can't implement your "OS" also as an app in the same way that Chrome is both an app and an OS. We're constrained by lack of imagination and the difficulties in getting market adoption for new approaches. Developers prefer to be locked in to a multi-vendor API with open source implementations, than a single vendor API with a proprietary implementation, even though they'll do it when a market is there (mobile apps). But that preference kills the commercial incentives to innovate, and open source devs just won't do it, which is why every new OS project you see is a clone of commercial operating systems from the past.

When I realized that the linux operating system had to have hardware support (traps for syscalls and interrupts), it was eye opening.

I guess your post highlights how operating systems and hardware developed together, and to innovate outside of what we currently have there is a need to somehow jump outside of this loop.

What are the hardware features are modern operating systems reliant on and the hardware features they aren't reliant on?

How would one go about exploring “other” real-world systems? I’ve read some old OS textbooks, poked at Symbian and OS/2 books some, and have texts on Oberon and Symbolics in my queue, but the docs for RSX-11 and VMS seem to bury me in operational minutiae without really explaining the design choices, and the Multics docs look like a huge pile of research notes, which is going a bit too far in the other direction. The current bytecode on IBM i is apparentily outright NDA’d, and the RPG docs are eager to presume I know how to operate the original punch-card tabulators. Any pointers?

Depending on the system this could range from an easy project that one can do in a few hours to a very long quest. For some research systems (particularly those implemented in the pre-FOSS era or those from companies) it may be impossible to try out the systems without attempting to implement them yourself. However, there are other systems that are available to try out. The ideal situation is a FOSS release, like the MIT CADR Lisp machine code and Interlisp-D. Barring that, the next best thing is some type of freeware or non-commercial license; I think this is the case for OpenVMS, though I could be mistaken. Some systems cannot be obtained easily without traveling the high seas, if you catch my drift, matey (cough Open Genera cough).

I think a lot of value can be gained from not only using software, but by reading papers and manuals, especially when the software is unavailable or unattainable. There’s a site called Bitsavers that is a treasure trove of old documents.

Come to think of it, a significant reason for Unix’s dominance in research and education is its availability, going all the way back to the 1970s when Bell Labs sold licenses to universities at very low prices. Even when licensing became more restrictive in the 1980s, this spurred the development of MINIX, GNU, later BSDs, and finally the Linux kernel, in order to recreate the environment students enjoyed with 1970s Unix. This openness is a far cry from the work of Xerox and Lisp machine vendors, where one needed to have been privileged enough to work for one of these vendors or their customers to use these environments, which were far more expensive than Unix workstations and especially personal computers. Thankfully there’s a wealth of documentation about these non-Unix systems, as well as testimony from their users. In addition, some systems were open sourced. But we must remember the times that these systems emerged, and we must remember why Unix became so dominant in the first place; its openness and availability set it apart from its competitors.

As an amateur OS nerd by 'exploring' do you mean use or read about or both? :)

You can get free access to play around on an IBM i (aka AS400/iSeries/i5) here [0].

Archive.org has (for the moment) Inside the AS/400 [1] by Frank Soltis. He was the father of the System/38 /AS400, and also worked on the Cell processor. I haven't read this edition, but I did read the following edition called 'Fortress Rochester', and it is a good and detailed overview of the system - history, design choices, and hardware/software. As for RPG, it was made to be familiar to punch-card operators/programmers so they could switch to a 'real computer'.

bitsavers has a lot of info [2]. A tech report on the Tandem Nonstop Guardian OS / system [3] was interesting.

[0] https://pub400.com/

[1] https://archive.org/details/insideas4000000solt

[2] https://bitsavers.org/pdf

[3] http://bitsavers.org/pdf/tandem/technical_reports/TR90-08_Gu...

Your best bet is probably the old MVS that was open sourced back in the 70s or 80s and lives on via an emulator called Hercules. Its sufficiently different to be worth a look, it's got a living direct descendent (although that has become increasingly turned into a Unix over the years), and there is enough of a community around it due to it having a very expensive direct descendent that the docs and other tools necessary for working with it are relatively high quality and up to date in comparison to most long dead operating systems.

The issue with it is that there is no way around having to do some pretty serious assembly language work for an instruction set that no longer exists and whose modern descendents probably microcode to hell in order to run some of the old instructions.

"Introduction to Operating System Abstractions using Plan 9 from Bell Labs" is the best operating system book I've seen - and it works through Plan9: https://archive.org/details/plan9designintro It isn't quite real world as Plan9 was never used for serious production, but it is mind opening and doesn't wallow in minutiae.

Get 9front and you would be surprised. Extra software for 9front:

http://shithub.us/

Also:

https://pspodcasting.net/dan/blog/2019/plan9_desktop.html

"Shithub is a community service, written by people who avoid browsers."

Project Oberon is amazing. Shame it isn't more popular.

Beautifully put. I couldn't have said it better myself.

Reply on news.ycombinator.com