Unix's technical history is mostly old now
utcc.utoronto.ca
utcc.utoronto.ca
* ITS (of PDP-10 hacker fame) - processes could debug and introspect their child processes. The debugger was always available, basically. The operating system provided support for breakpoints, single-stepping, examining process memory, etc. Whatever part of the system manages processes, is the natural place for the debugger, really. Some of this has been brought to Unix systems recently, but you still can't trivially freeze a process, tweak a few values in its memory, and resume it as an integrated part of the operating system. Why not? It seems very basic to an operating system, now that I think about it.
* KeyKOS (developed by Tymshare for their commercial computing services in the 1970s) - A capability operating system. If everything in UNIX was a file, then everything in KeyKOS was a memory page, and capabilities (keys) to access those pages. The kernel has no state that isn't calculated from values in the virtual memory storage. The system snapshots the virtual memory state regularly. There are subtle consequences from this. Executing processes are effectively memory-mapped files that constantly rewrite themselves, with only the snapshots being written out. Snapshotting the virtual memory state of the system snapshots everything -- including the state of running processes. There's no need for a file system, just a means to map names to sets of pages, which is done by an ordinary process. After a crash, processes and their state are internally consistent, and continue running from their last snapshot. For those who are intrigued, there's a good introduction, written in 1979, by the system's designers available here: http://cap-lore.com/CapTheory/upenn/Gnosis/Gnosis.html (It was GNOSIS before being renamed KeyKOS.) And a later document written in the 90s aimed at UNIX users making the case: http://cap-lore.com/CapTheory/upenn/NanoKernel/NanoKernel.ht... Some work on capability systems continues, but it seems the lessons learned have largely been forgotten.
My own reconstruction of what I think happened (this was all before I was born, I'm just making some inferences based on what I know about the topic) – Microsoft was heavily influenced by DEC operating systems (Bill Gates and Paul Allen wrote the first version of Microsoft Basic on a PDP-10). Microsoft didn't like CP/M's native option syntax, so for their development tools for CP/M, they adopted a DEC-style option syntax instead. Since many ISVs used Microsoft's development tools, a lot of third party software on CP/M ended up using the same option syntax too. Tim Paterson at Seattle Computer Products was influenced a lot by Microsoft (even before he worked for them), so QDOS/86-DOS/PC-DOS/MS-DOS ended up with the DEC/Microsoft-style forward slash as an option character. When DOS 2.0 came around, Microsoft wanted to switch it to dash/minus, for better Unix/Xenix compatibility. In support of this, they added a CONFIG.SYS setting SWITCHCHAR= to let you choose between forward slash and dash/minus, and an API (INT 21,37) to get/set that config setting at runtime. However, it was decided that the backward compatibility cost of that change was too great, so they pulled the feature from the documentation (although it still existed in the code; in later DOS versions, they disabled the ability to change it to anything other than forward slash, but the undocumented API to get the setting was never removed.)
People talk about Windows NT being influenced by OpenVMS, via Dave Cutler – the influence is very obvious in some areas (e.g. asynchronous IO), but path name syntax is one area in which Windows is much more influenced by Unix than by OpenVMS (even considering warts such as drive letters, support for backslash instead of forward slash, and reserved file names such as NUL and CON.)
the sockets api was modeled after berkeley and the libc itself is posix compliant to this day, i believe.
Then IBM and MS fell out and Microsoft put all their resources into Win32.
However, Windows NT did actually have those three APIs – the Win32 subsystem (which also incorporated Win16/DOS support), the POSIX subsystem, and the OS/2 subsystem. But the POSIX and OS/2 subsystems were always rather half-baked, and never saw that much use. The OS/2 subsystem was removed in Windows 2000; it only ever supported OS/2 1.x apps, and only character mode unless you purchased a separate graphics mode add-on from Microsoft. The POSIX subsystem was not removed until Windows 8.1, although by then it had changed its name to "Subsystem for Unix-based Applications", and had grown a lot compared to the original NT 3.x POSIX subsystem. WSL is effectively the replacement for the POSIX subsystem, however its implementation is very different (which is true of both WSL1 and WSL2, despite the fact that they are also very different in implementation from each other.)
the important part was the posix api functions that made porting unix daemons that had been written in c/c++ to nt pretty easy. there were some quirks around shared memory/mmap, getpwnam, the sockets api and threading support, but beyond that doing a port was pretty easy.
the work there was plugging into the service control and event logging apis. and doing an installshield, that was no fun.
if a program did use fork as part of its concurrency model (like say, preforking apache) then making use of actually functional nt threading support and gratuitous ifdefs was usually the answer.
Both Cygwin and POSIX/Interix/SFU/SUA use the same workaround - maintain a separate Unix PID for each process. exec() changes the NT PID but leaves the Unix PID unchanged. So exec() creates a new process, but to Unix apps it looks like the same process.
You are right that for many apps, rather than rely on emulated fork/exec, it was better to switch to native Windows facilities. Still, there was some value in being able to recompile Unix software with minimal changes (as Cygwin demonstrates). Even WSL now confirms that, albeit in a rather different way. Also, it was necessary to pass the POSIX test suite, which Microsoft briefly was concerned about (until they successfully lobbied the US government to drop POSIX certification as a procurement requirement).
Interix 2.2[0] shipped with its own PSXSS.EXE. I don't have a contemporaneous version of the PSXSS.EXE that shipped with Windows at that time to compare handy.
Finding a copy of OpenNT prior to the Microsoft acquisition is proving to be difficult.
So much history is slipping away.
However (putting aside the legalities of downloading "abandonware"), they won't let you download anything unless you first upload something they don't have. Heck, I probably do have something they don't have in my box of old floppy disks (which I keep on telling myself I will image one of these days), but my curiosity about this topic isn't strong enough to motivate me to do that.
- Duplicate all threads. No one does this because it would be total chaos. If another thread was writing to a file or a socket, parts of those writes might happen twice, etc.
- Duplicate only the calling thread. This isn't total chaos, so everyone does this, but this also sucks. Any locks that were held by any other threads will never be released in the new process, so you need to make sure not to touch any locks at all post-fork. But that's a huge restriction, because e.g. malloc() touches locks sometimes. Code that runs post-fork ends up being as restricted as code that runs in signal handlers.
I guess the advantage of fork relative to other hypothetical methods of creating child processes is essentially just getting a copy of all the parent's state before the fork. So maybe the question is how often getting a copy of all the parent's state is actually useful?
However, fork() is still not necessarily a good design, and the reason for that is that the vast majority of the time the reason you want to run fork() is to immediately run exec() afterwards because you're trying to launch a separate process. So, the fork() does all the hard work of duplicating the entire address space of the process, and then exec() throws it all away again.
One thought is, if the OS/libc was designed so that exec() always created a child process, then you could get the same functionality as fork() by creating a child process using exec() and passing along just the info that the child process needed, possibly using IPC. This seems like better encapsulation, for the same reason you wouldn't write a function which required a ton of arguments it didn't use. It also makes the issue of forking a multithreaded process irrelevant.
But I could imagine this approach being slow if there's a lot of state you want to pass, if copying entire pages of memory (the way fork does?) is significantly faster than than sending the same data via IPC.
I wonder what the most common non-exec() use of fork() is.
There are quite a lot of things that survive exec(). Not least of which is the set of open files. This is how a shell will pass on the definitions of stdin/stdout/stderr to a command that you want it to run, for example. Also, environment variables survive exec().
Does this change depending on whether the environment variable definition was prepended with 'export'?
I’m not sure if it is the same, but back in my Amiga-days we had Arexx which I thought was an amazing tool to do tool-automation.
To this day I haven’t seen anything quite like it. The closest is P to ably Windows and COM, but nobody likes that ;)
"Eric Bier Demonstrates Cedar"
https://www.youtube.com/watch?v=z_dt7NG38V4
One for Oberon,
https://www.youtube.com/watch?v=OJGnpmnXR5w
IBM Redbooks are a source of information for IBM i and z
And Burroughs is still sold as ClearPath MCP.
https://public.support.unisys.com/search2/DocumentationSearc...
Can't you do this with gdb? Running `gdb -p <some pid>` pauses that process, you can inspect it, set breakpoints, change values in memory, etc, and then resume execution.
Because ITS had the debugger (DDT) as its shell. ITS wasn't the only OS for which this was true – the same was effectively true of CTSS, since in CTSS debugging was just a bunch of shell commands, not launching a separate "debugger". The same is true of IBM VM/CMS, for assembler-level debugging. (Source-level debugging is launching a separate program though.)
One interesting feature of ITS – it had an API (called "valret" or "valretting") which an application could call to run a command in the OS shell/debugger. So basically, an application could actually control the debugger debugging it. I've never seen any other system with that feature, although it is possible to implement with GDB – allocate a buffer, put a "run this debugger command" request in it, pass it to a dummy function which just returns. Have a "processed" flag in the buffer – without a debugger, calling the dummy function will not set that flag, so the app knows no debugger is listening for commands. Now write a GDB Python script which sets a breakpoint on that function, when the breakpoint is hit, it reads the command out of the buffer, runs it, optionally writes any reply back in to it, sets the processed flag, then continues execution. The app can see the processed flag is set, so it knows the command was run, and can read any reply from the buffer too.
MS-DOS COMMAND.COM did have a somewhat related feature – INT 0x2E. You could use that to tell COMMAND.COM to run a command. The difference from e.g. the C system() function, is it didn't spawn a child process COMMAND.COM to run the command, it told the parent COMMAND.COM to run it for you.
(What many people today don't understand about MS-DOS, is it was actually a pseudo-multitasking operating system – it supported having multiple processes in memory at once, unlike many earlier operating systems such as CP/M which only supported having a single process in memory at a time – but unlike an actual multi-tasking OS, only the single foreground process could actually be running, all other processes would suspended – so you could have a child process, but you'd be suspended while it ran. An ancestor process could export an API to its descendant processes by installing an interrupt handler, which is exactly what COMMAND.COM did by installing a handler for INT 0x2E.)
https://erlang.org/download/armstrong_thesis_2003.pdf
Chapter 5 is especially relevant:
excerpt:
What is a fault-tolerant system and how can we program it? This question is central to this thesis and to our understanding of how to build fault-tolerant systems. In this chapter we define what we mean by “fault-tolerance” and present a specific method for programing fault-tolerant systems. We start with a couple of quotations:
We say a system is fault-tolerant if its programs can be properly executed despite the occurrence of logic faults. — [16]
...
To design and build a fault-tolerant system, you must under- stand how the system should work, how it might fail, and what kinds of errors can occur. Error detection is an essential com- ponent of fault tolerance. That is, if you know an error has occurred, you might be able to tolerate it by replacing the ocending component, using an alternative means of compu- tation, or raising an exception. However, you want to avoid adding unnecessary complexity to enable fault tolerance be- cause that complexity could result in a less reliable system. — Dugan quoted in Voas [67].
The presentation here follows Dugan’s advice, I explain exactly what happens when an abnormal condition is detected and how we can make a sodware structure which detects and corrects errors.
The remainder of this chapter describes:
• A strategy for programming fault-tolerance — the strategy is to fail immediately if you cannot correct an error and then try to do some- thing that is simpler to achieve.
• Supervision hierarchies — these are hierarchical organisations of tasks.
• Well-behaved functions — are functions which are supposed to work correctly. The generation of an exception in a well-behaved function is interpreted as a failure.
* A History of Erlang
* The Development of Erlang
I think the subject is right: systems software "research" is irrelevant. Because academia could no longer keep up.
IBM and Bell Labs, maybe.
It's interesting to read a manpage on a slick, modern-seeming system like macOS and notice a date at the bottom which is literally last century.
I guess the key thing is the birth of self-sustaining "software ecosystems": even if the ecosystem is flawed, the benefit of being part of the ecosystem is really high, and choosing to be part of the ecosystem means adding to it, which makes the benefit of being part of it even greater. Unix is self-sustaining despite any flaws in the same way qwerty and the English language are self-sustaining despite any flaws. Unix is like a protocol for software projects to coordinate with each other and with human beings.
Ironically, if it wasn't for rapid development of hardware and broad penetration of computers throughout society, we might still be in a phase of rapid systems evolution, since the "ecosystems" wouldn't be as well-developed. Popularity increase -> lock-in.
A bunch of systems admins who were puzzling over how to create a standard way to administer Unix systems. The premise was "in the old IBM mainframe days, every shop ran the exact same way, so it was easy to train a new hire. Now, with Unix, there are a million different customs, so how do we standardize it?"
You probably never even thought this was a problem. My big problem was staying awake. I guess now that it's all Linux, it's easier. Back then, there was HP-UX, IBM AIX, Sequent's whatever-it-was-called, Pyramid, SCO & all the other PC variants...
The group was called Moses, which makes it almost unsearchable since there are so many other uses of that word.
I agree. In fact, since there's fewer actively used unix variants, the same job would seem simpler in many ways. I think the fragmentation of tech above the OS would be much more intimidating. So many more languages, build+deploy systems, frameworks, things like api gateways, caching platforms, load balancers, and so on. In the 1990's an admin could maintain a decent amount of personal knowledge about what a developer might be doing on the platform.
I'm more of an
$ apt remove --purge nano > /dev/null
Then they ran out of space and needed to spill over onto another drive, so home directories got moved to `/home`.
IIRC Ubuntu atleast comes with vi, but vi != vim.
Apple's Unix variant seems fine without it though ... Of course macOS uses launchd [0] which I guess is somewhat similar. And if I am not mistaken it's released under a permissive license.
Perhaps Linux could benefit from using launchd instead of systemd?
---
1) the presence of launchd doesn't make systemd necessary
2) the previous absence of anything like systemd on Linux distros means I don't know what you mean by "necessary"
It isn't. Systemd has dependencies and many directives to express them. Launchd doesn't, and instead demands that every service either waits for its dependencies by itself or crashes deliberately if they are not met - so the autorestart by means of its brute force eventually brings the system fully up.
It's not entirely clear whether that proposition's actually true.
I think in particular when you look at things like the ability of that operating system abstraction to handle stuff like GPU accelerated computation (CUDA and the like) it's possible that the insistence that the only abstractions we need are processes, pipes, sockets and inodes starts feeling a little limiting.
UNIX/POSIX never went beyond the basic of CLI applications and server daemons.
Indeed, I would say that it's true. I have a reference book about Unix & GNU/Linux wrote on mid/late 90's, and roughly 80% of it keeps applying to modern Linux. ps keeps being ps. Same thing with bg, chmod, etc. What really got outdated on these book, was X11 configuration (thanks!), referring to an editor called "joe", and a few bits of how install a Red Hat Linux 5.1
A complete superset of NTFS's ACLs, thus providing good cross-platform compatibility, and already implemented in illumos, Mac OS X, and FreeBSD, Linux is the only holdout on them.
Mind also, "old POSIX ACLs" came from a POSIX draft: they never made it into POSIX. While being an extremely simple expansion of the Unix modes, they are only ever additive and do not support fine-grained permissions that NFSv4 allows for. They're sometimes better than the standard mode bits, but they very often come up short of being useful in the real world.
Everyone who uses systemd. Try it yourself: do a getfacl on the files inside /var/log/journal on a system with persistent logging enabled (if it's disabled, these files will be at /run/log/journal instead).
As someone who has had to deal with NFSv4-styles ACLs (on an Isilon server handling Linux HPC clients): they can get really messy, really quickly (especially the inheritance aspect of them).
I'm not saying your use case is invalid; I'm actually expressing sincere curiosity: what do NFSv4 ACLs bring to the table and what problems does it solve?
NFSv4 ACLs are far more powerful, modeled after NTFS ACLs (a proper superset, with an added two permission types). In network environments (where you'd use them anyway), they can assign access control entries (ACE) with Security IDs (SIDs), generally more stable and consistent across a network than ad-hoc Unix user IDs/group IDs. Managing with UIDs/GIDs is still possible, too. They add both allow and deny permissions to ACLs; if you want to deny "fred" from reading a file, you can make a deny entry for "fred" specifically.
The fine-grained permissions you can get from NFSv4 ACLs: READ_DATA, LIST_DIRECTORY, WRITE_DATA, ADD_FILE, APPEND_DATA, ADD_SUBDIRECTORY, READ_NAMED_ATTRS, WRITE_NAMED_ATTRS, EXECUTE, DELETE_CHILD, READ_ATTRIBUTES, WRITE_ATTRIBUTES, DELETE, READ_ACL, WRITE_ACL, WRITE_OWNER, SYNCHRONIZE.
READ_DATA, WRITE_DATA, EXECUTE are equivalent to the read/write/execute Unix mode bits. The two expanded features I find I use most frequently are blocking the deletion of files and making them append-only.
This is largely true for any server you’re likely to ssh into. But the most common Linux distributions are Android which make extensive use of SELinux features.
While such systems still exist, they are a much smaller percentage of all installed systems than they used to be. Most contemporary Unix(-like) systems fall into one of two categories:
(a) single-user machines (such as a laptop)
(b) application servers, database servers, etc, which serve many users, but those users are defined at the application layer, the OS and filesystem don't know anything about them
Maybe part of the reason why filesystem ACL adoption is weaker than people expected, is that (a) and (b) don't have the same need for filesystem ACLs as the classic multi-user timesharing environments do.
> But the most common Linux distributions are Android which make extensive use of SELinux features
SELinux context and filesystem ACLs are two separate things. I see many machines with heaps of the former and little of the later (such as most Red Hat boxes).
Even with single-user machines, there is still some value in inter-process security – application sandboxing, corporate-managed devices, etc – but there are many technologies available now to meet those requirements, and filesystem ACLs are often not the most useful among them.
... added to https://github.com/globalcitizen/taoup
But if we could include these things, I'd say also:
* Epoll and kqueue. The recognition that select(2) and later poll(2) do not scale.
* Interactivity. One thing I remember about 90s Unix is how terrible the usability was on those old terminal emulators or termcaps. You'd type a backspace and see ^H. Arrow keys that didn't do the right thing. You'd log into a Linux box and by contrast you'd get good tab completion, colors, etc.
* Secure defaults. First noticed this in OpenBSD. But the internet forced everybody to reconsider the set of daemons you get on a default install.
* Deprecation of unsafe libc functions
One other improvement worth calling out is that /dev/random (or substitute your RNG method of choice) is a thousandfold better now than it was in the 90s, though Linux has a big wart with entropy pool exhaustion. BSD leading the way there from what I understand, having an RNG that Just Works. This isn't just an invisible implementation detail, it means application developers can request and use random data without a bunch of "just-in-case" bit shuffling like discarding lower bits.
Sometimes something is so painful everyone unconsciously decides choice is bad and we're all going to use $X.
That is probably true as many of those old tools kept compatibility, but at the same time we got a whole bunch of new tools, more powerful than `ps` for gaining more insights. For an example look at all the changes in service management, eBPF and such. Not to mention that the shell prompt itself (see zsh etc.) is more powerful than back then, which could give a very different experience.
So yeah, old tools keep mostly working, but improvement goes on.
Well they might freak out about what you did with telnet and how you logged in without a password.
levenez.com has been posted multiple times before in this context. There is a /unix and a /lang page under it.