Unix – The Hole Hawg of Operating Systems (1999)
team.net
team.net
Decades have passed and it has gone beyond the cold rooms to our warm pockets. It still works tirelessly. You can still depend on it. It has changed its shape and wears colours if you want it to. But deep down, I know it to have the same kernel of truth. If you want to talk to it in those ancient grunts, it will still answer back. And that to me still is the most beautiful language.
That's a poor representation of the big smile I had on my face while reading this.
I've spent 25 years now "messing" with Linux. Installing over-the-air interactive TV Linux-based mpeg stream devices in central London-based TV channels. Managing web servers and compute clusters for US universities. Wrangling SANs and VMware together to create email services for education. Load balancers. Macs. Private compute environments on VMware for developers. Kubernetes clusters for what felt like a million microservices.
All the while, Unix has been there for me to hack on, mess with, debug, wrangle, chase down, curse, and praise.
It's an old, dependable friend now, but has lost none of its vigour. I imagine it will be there long after I'm gone, probably long after any of us are around.
As a young outsider, I could gain access to this world only by breaking in or fast-talking someone. Once inside I'd have my own room and could learn magic at my own pace. I could explore. The wizard might banish me if he finds out.
This was literally half of the books I'd read growing up come to life.
My very first programming experience was the integrated shell of Python 2.6, maybe 2.7. I followed wikis and tutorials in order to print various fun little things. Everything past that was further layers of abstraction, buttons, magic, and GUIs.
I was whisked away into the faintly surreal and inexplicable tornado of Android Studio and XCode until my first year of college, at which point a college professor, during the third lecture of class, was asked, "how many students fail out during this course?"
The professor paused for a moment, and said, "I'm not sure -- let me count the number of failing grades from last semester's final," at which point he piped his grade book into a terminal, straight into a series of bash commands that gave him the precise number of final exams that had scored below a 55.
Thinking back, it's not the fact that he could do that -- I sort of knew that was possible, even with a fairly basic understanding of computers. It was the ease, the fluency with which he did it.
Right then, I understood that programming was the closest I was ever going to get to my childhood understanding of alchemy/magic/psionic powers. And maybe even a little bit further than that.
That is what Unix will always mean to me. A genuine, actual, factual spellbook, inexplicably and suddenly laid at my feet.
It really does feel like a spellbook sometimes, or a huge Lego model where brick A can fit onto plate B in a dozen different ways to create anything from a dinosaur to a SHIELD Helicarrier :)
cat | grep -i | grep -v | awk '{print $4}' | something | something --else
And from the chaos, beauty emerges. awk '/[Pp]attern1/ && !/pattern2/ {print $4}' file | \
something --specific
What I wouldn't give to just spend every day awk golfing.As an excellent qa person, he would write everything down, and do that thing, that way, in that situation. Repeatability.
I gradually convinced him, no, it's a composable language. Look, you can put this thing that you know together with this other thing that you know, and get these new and useful results.
And the lights went on.
Like, just look at something like the -print0 option to find.
This is the most irritating thing about Stephenson's little tract and the "UNIX is the best" mindset that its fans have: the things the he praises is just the user environment, a surface level feature which can be significantly replicated on any modern OS. Now, don't get me wrong; the UNIX user environment deserves its praise but it says nothing about the merits of UNIX/Linux itself at an architectural level. Most people can't even describe the differences between Linux, Windows, OS X, Fuchsia, Haiku, Midori etc. in an architectural sense.
That's because Unix won. When he wrote this article Linux was just moving out of the hobbyist realm, we were on OS 9 (or 8?), and Windows was busy "finding itself". The landscape was different.
Microsoft were trying to sell Windows NT to chip designers on the premise that they didn't need to use scripts.
MacOS at the time didn't even have a commandline or interactive text-driven shell.
MVS TSO/ISFP had JCL, but the principle mechanism for interprocess communication was DASD and structured datafiles, not piplines. (VM/CMS more closely approached the Unix model, but was far less widely used.) VMS had COMfiles and DCL, but lacked the full nuance of Unix scripts.
And none of those systems were available on commodity x86 hardware. Linux (and numerous proprietary Unix clones) could run on 386s, and in some cases 286 systems. Need a system at low cost and with sufficient flexibility that the disadvantages of low-power were largely obviated? Unix had you covered.
MacOS and Andriod adopted Unix or Linux because the were clearly superior options. Fuschia ... I think ... is following similar lines (I've not followed it closely). Microsoft introduced Powershell after it became clear that no, GUI toolbuilding simply lacks the flexibility of the commandline. (And they still came up with something incompatible. So WSL got bolted on as part of the stock build.)
> "MacOS and Andriod adopted Unix or Linux because the were clearly superior options"
If by "superior" you mean "free as in beer", yes, they were the superior option.
https://en.wikipedia.org/wiki/In_the_Beginning..._Was_the_Co...
Such a philosophy makes it impossible to create a truly easy-to-use system because the "OS" cannot make assumptions about its state. e.g. you have a configuration file and the root user can edit it with a text editor and introduce inconsistencies.
So either you don't allow the user to introduce "not anticipated" states or you let them and on their head be it. That seems to me to be the main UNIX philosophy - "ultimately the choice is yours".
That doesn't mean you can't screw it up and make it something else. I think the corporate interests who are paying for the development of Linux have made choices that have moved it a few steps away. Thanks to open source, however, one can escape their clutches for a price.
Anti-Foreword By Dennis Ritchie for the Linux Hater Handbook https://web.mit.edu/~simsong/www/ugh.pdf
To the contributers to this book: I have succumbed to the temptation you offered in your preface: I do write you off as envious malcontents and romantic keepers of memories. The systems you remember so fondly (TOPS-20, ITS, Multics, Lisp Machine, Cedar/Mesa, the Dorado) are not just out to pasture, they are fertilizing it from below.
Your judgments are not keen, they are intoxicated by metaphor. In the Preface you suffer first from heat, lice, and malnourishment, then become prisoners in a Gulag. In Chapter 1 you are in turn infected by a virus, racked by drug addiction, and addled by puffiness of the genome. Yet your prison without coherent design continues to imprison you.
How can this be, if it has no strong places? The rational prisoner exploits the weak places, creates order from chaos: instead, collectives like the FSF vindicate their jailers by building cells almost compatible with the existing ones, albeit with more features. The journalist with three undergraduate degrees from MIT, the researcher at Microsoft, and the senior scientist at Apple might volunteer a few words about the regulations of the prisons to which they have been transferred.
Your sense of the possible is in no sense pure: sometimes you want the same thing you have, but wish you had done it yourselves; other times you want something different, but can't seem to get people to use it; sometimes one wonders why you just don't shut up and tell people to buy a PC with Windows or a Mac. No Gulag or lice, just a future whose intellectual tone and interaction style is set by Sonic the Hedgehog. You claim to seek progress, but you succeed mainly in whining.
Here is my metaphor: your book is a pudding stuffed with apposite observations, many well-conceived. Like excrement, it contains enough undigested nuggets of nutrition to sustain life for some. But it is not a tasty pie: it reeks too much of contempt and of envy.
Bon appetit!
You may be suprised if you delve into the command line environment in Haiku and see just how intimate you can get with the kernel and resource management.
Unix - The Hole Hawg - https://news.ycombinator.com/item?id=2174519 - Feb 2011 (5 comments)
Like a hole hawg, that power can rapidly cause things to go awry. Unlike a hole hawg, the extent of the damage can extend across the world.
And then there is shell. Win10 has made big progress here (by using Linux subsystem), but it is still only half useable. For example, I really miss "select to copy & middle click to paste" support.
Indeed, because it is right click to paste.
*nix operating systems were and continue to be successful because they are a complete and powerful toolbox for computing at all different levels.
And yes, if I had to identify a "Hole Hawg" of Unix it would probably be the `dd' command. But I certainly don't use it on a daily basis :)
(Milwaukee sells “demolition screwdrivers” designed to be used as prybars and chisels because that’s what people do with them: https://www.homedepot.com/p/Milwaukee-5-16-in-Slotted-6-in-D... - akin to modifying a program for the use case vs the original designed purpose).
In all seriousness, most enormous flathead screwdrivers are intentionally designed to work as a prying tool. Now, do that with an insulated screwdriver and you might as well prepare yourself for the cable whipping from your senior.
(I've definitely had my share of being "spun around the hole" with accidental dd misfires.)
There is nothing about the UNIX command line that isn't done better by REPL based environments.
http://people.cs.georgetown.edu/~clay/classes/spring2010/os/...
killshot. this is what unix is like.
In Stephenson's analogy, the OS is the Milwaukee drill in question.
it does describe both unix & the unix enthusiast. we get to know our tools, there is apparent & unadorned solidity to them and a truthfulness in interactions & capabilities beyond what the gussied up consumerized product could create.
but yeah. upvoting ya for sure.
Since my impression is that HN people are [a] xNix fans [b] often quite young therefore [c] have little exposure to other OSes, let me try to unpack what Stephenson was getting at, *in context*.
The Hole Hawg is a dangerous and overpowered tool for most non-professionals. It is big and heavy. It can take on big tough jobs with ease, but its size and its brute power mean that it is not suitable for precision work. It has relatively few safety features, so that if used inexpertly, it will hurt its operator.
DIY stores are full of smaller, much less powerful tools. This is for good reasons:
• because for non-professional users, those smaller, less-powerful tools are much safer. A company which sells a tool to untrained users which tends to maim or kill them will go out of business.
• because smaller, less-powerful tools are better for smaller jobs, that a non-professional might undertake, such as hanging a picture, or putting up some shelves.
• professionals know to use the right tool for the job. Surgeons do not operate with chainsaws. Carpenters do not use axes.
The Hole Hawg, as described, is a clumsy tool that needs things attached to it in order to be used, and even then, you need to know the right way or it will hurt you.
Compare with a domestic drill with a pistol grip that is ready to use out of its case. Modern ones are cordless, increasing their convenience.
One is a tool for someone building a house; the other is a _better_ tool for someone _living in that house._
That's the drill part.
Now, let's discuss the OSes talked about in the rest of the 1999 piece from which that's a clipping: https://en.wikipedia.org/wiki/In_the_Beginning..._Was_the_Co...
There are:
• Linux, before KDE, with no free complete desktop environments yet.
• Windows, meaning Windows 98SE or NT 4.
• Classic MacOS – version 9.
• BeOS.
Stephenson points out that Linux is as powerful as any of them, cheaper, but slower, ugly and unfriendly.
He points out that MacOS 9 is as pretty, friendly, and comprehensible as OSes get, but it doesn't multitask well, it is not very stable, and when a program crashes, your entire computer probably goes with it.
He points out that Windows is overpriced, performs poorly, and is not the best option for anyone – but that everyone runs it and most people just conform with what the mainstream does.
He praises BeOS very highly, which was 100% justified at the time: it was faster than anything else, by a large margin. It has superb multimedia support and integration, better than anything else at the time. It was standards-compliant but not held back by it. For its time, it has a supermodern OS, eliminating tonnes of legacy cruft.
But it didn't have many apps so it was mainly for people in narrow niches, such as music production or maybe video editing.
It was manifestly the future, though. But we're living in the future and it wasn't. This was 23 years ago, nearly a quarter of a century, before KDE and GNOME, before Windows XP, before Mac OS X. You need to know that.
What Unix people interpret as praise here is in fact criticism.
That Unix is very unfriendly and can easily hurt its user. (Think `rm -rf /` here.)
That Unix has a great deal of raw power but maybe more than most people need.
That Unix is, frankly, kinda ugly, and only someone who doesn't care about appearances would choose it.
That something of this brute power is not suitable for fine precision work. (Which it still mostly isn't -- Mac OS X is Unix, tuned and polished, and that's what the creative pros use now.)
Here's a response from 17 years ago: http://garote.bdmonkeys.net/commandline/index.html
- ACID compliant data storage is not something that operating systems excel at, databases are much better. (I've seen some monstrosities where somebody tried to use the filesystem as a database and eventually they always got horrible race conditions and data corruption). This is a table saw; extremely good at one specific thing (rip cuts), can do other types of things in a pinch (cross cuts for table saws, db-as-a-job-queue for databases) and is absolutely rubbish for other things like drilling holes.
- Going up in complexity: Most operating systems I've seen are relatively self-contained on a single box. The OS controls everything on the box and no other OS-es exist on the same server. If you have a large multi-server system the OS typically does not provide too many capabilities except a low level networking stack, something like Erlangs BEAM VM or Kubernetes fills up the gaps to make all the operating systems work together properly. This would be like a CNC machine; large, complex and expensive but very worth it from a certain scale onward.
- Going down in complexity: Embedded systems are often too small for a proper operating system and implement everything they need themselves. Realtime operating systems exist but are often not a good solution for the constraints. This might be something like an impact wrench, super specialized for a specific type of small job.
No contractor would use a Hole Hawg to drill 1/4” holes in something - maybe quarter feet. The amount of adapters you’d need to even get it able to reliably hold a 1/4” bit precludes it.
The point of the analogy remains - both tools can do things others can’t and both can easily “get away” from you.
But the real lesson is learning that the normal drill (and operating system) can also get away from you, and you may be less braced for it when it happens.
EDIT: found it https://randsinrepose.com/archives/the-foamy-rules-for-rabid...
The last line of the article is a killer because I’m writing this on an Apple unix system on a train at the moment after balancing my books.
As a pdp11 v7, vax 4.1bsd, 4.2bsd, Cray unicos and Unix 32V user I am slightly bemused by this because to all intents and purposes Darwin is unix
If he means "we feel a bit tawdry using a GUI" well sometimes, yes, except SunView and X10R4 and the blit and gnot and perq are kind of out there. I liked the perq. And Ultrix.
I think Stephenson has kind of walked off the reservation a bit. Even plan9 fanatics use a gui.
"In the beginning was the command line" is better.
The oldest version of this article in the wayback machine is from 2003, but I see a reference from 2001 here:
https://arstechnica.com/civis/viewtopic.php?f=20&t=912937
So I suspect it's from before the introduction of OS X.
which was published in 1999
"You guessed right: I embraced OS X as soon as it was available and have never looked back. So a lot of 'In the beginning was the command line' is now obsolete. I keep meaning to update it, but if I'm honest with myself, I have to say this is unlikely."
Given that Stephenson didn't stick with CLI himself, the repostings of "In the Beginning was the Command Line" really should have ended long ago.
(As an aside, the interview is worth a read, particularly his response to "In a fight between you and William Gibson, who would win?")
Imagine plugging in a USB bluetooth dongle to play some music on your headphones. In response the OS needs to load some drivers, start the bluetooth service, tell the audio service to output audio to your headphones etc. Windows can do this out of the box. Linux can be made to do this, either with a massive proprietary Android task or systemd/pulseaudio/bluez/whatever if you're in the open-source world. At this point Linux can be hardly called a Hole-Hawg.
The perceived superiority of Unix also leads to the false conclusion about Android/macOS/Desktop Linux being superior to Windows (or any other hypotethical fully fledged OS). In order to handle the massive complexity imposed by the complex scenarios, the systems themselves have to be complex as well, with varying success. I'm willing to bet, most of the code that allows me to write this comment on macOS on a HW accelerated GUI using a bluetooth keyboard is part of the Apple-monolith and has nothing to do with Unix/POSIX. The same applies to the other "Unixes" I mentioned earlier.
Arguably much of the difference can be attributed to being able to get Windows running easily - forgoing the painful process of having to learn what you’re doing (and at the same time learning how to tune it and where the slowdowns might be).
We’re now in a much different world. (Arguably the main difference remaining is that a Linux comes with “everything” whereas the others need out-of-band installations: in the 90s those things cost money whereas they were free on Linux - compilers used to be a pay product!)
My point is, full-fat desktop OS-es are much more complex beasts that a bare-bones Linux server install, that probably is a bunch of init-scripts and a server process on top of the kernel. Once you try to turn a UNIX into a desktop OS, it'll have the same problems as Windows, for the same reasons.
The Unix command line has a lot to recommend it. Notably, the simple ergonomics of the pipeline are indeed great (other OSs have similar pipeline concepts; they just don't get used the same way because they are too clunky).
The handle has to be removed after every use if you want to put it back in it's case.
The handle is just pipe with a rubber handle at the end, whereas most tools use molded plastic that slides over the chuck.
And yeah, the body is a cube of metal, as opposed to large unibody plastic castings.
https://i.ebayimg.com/images/g/FHcAAOSwLfVg0K4w/s-l640.jpg
https://i.ebayimg.com/images/g/LosAAOSwZS9g2quf/s-l1600.jpg