Unix: A History and a Memoir, by Brian Kernighan
cs.princeton.edu
cs.princeton.edu
I've been lucky enough to work and hang out with some of the co-founders of very impactful projects, such as OpenStack and Cloud Foundry, and there are so many stories I've heard that I'm sure would be insightful and valuable lessons for whomever is embarking on new ideas. And yet, we all move so fast, that there is no time to stop and write them down.
I'm glad BK did. UNIX is foundational to essentially all software-driven technology today, in one way or another. His book (along with Dennis Ritchie) on the C Programming Language made a huge impact for me as a CompSci student in the 80s, as did UNIX itself (Ultrix and DG/OS were my fist UNIX variants).
I look forward to read his book.
I try and keep the following in mind (even when my ego fights it).
(A) I am very unlikely to be the smartest person who's ever thought about a problem. (B) Even if their technology differed, some parts of their approach are useful. (C) Clues and references pop up in the oddest places, so ears open and notepad ready. (D) Accept references graciously, in the spirit of a gift. (E) Name dropping historical references without providing actual context is always in poor taste.
Why do you feel this way? ACM has had a History of Programming Languages conference since the late 70s where the well known History of Lisp paper was presented. All sorts of popular accounts are, well, popular. Soul of A New Machine was published in '81, Hackers in '84, Accidental Empires in '92, just as a few examples off the top of my head.
If history exists, but nobody reads it, then are we not still lost?
I don't work in SV, but I like to think I work in a fairly high-skill coding environment. And I regularly have to elaborate on historical references or suggestions.
In the meantime, how is the (Bayesian) evidence from one's experience to be properly measured next to the existence of books, papers, journals, conferences, videos, some with easily accessible and fairly accurate sales or views numbers to prove a level of popularity? In my office I doubt more than 20 people, perhaps no more than 10, out of the thousand or so spread across the floors (which includes many non-devs) could name over 200 Pokemon. This doesn't negate the existence of millions of people who can do that and more elsewhere, and the millions of dollars in that market. But I think it is at least suggestive about the type of people my company, and companies like mine, tend to hire. Is it really a stretch to extrapolate and make a bet that "perhaps only 2% of salaried devs can name over 200 Pokemon"? How about something just as trivial, the names of all 50 united states?
Pokemon is Pokemon, it doesn't matter much for our field, but where the widespread self-hating of one's profession comes in is that depending on one's experience the bets might not seem to be that different for things more programmers (or other roles like managers) probably ought to know about, their field's own short history just one of them. I recently quipped to someone that I doubt more than 5% of developers are even familiar with the SOLID principles thus it's not very fruitful to ask about them in an interview unless knowing about them is a hard requirement. Even fewer have a good grasp of what each principle means, and even fewer know the trivia that the L is named after a woman (with even fewer still knowing the less trivial nature of what else she contributed to our industry). Maybe I'm over-extrapolating, or a bit pessimistic, maybe I'm letting my enterprise day-job and my conversations with others at places (both enterprise and not) jade me in my expectations for what I think the "average everydev" is like.
My underlying theory is that power laws are everywhere, which means if the distributions on facets like "knows Pokemon", "knows about SOLID", "frequents Hacker News", "knows important bits of computing's history", "writes/buys/reads technical books", et al. both trivial and non-trivial are power laws then the average level would be terribly low compared to the narrow peak on one side of the distribution. The mode, which is probably what influences a lot of personal-experience feelings about the average, is lower still. If your experience is mostly around the peak it may seem otherwise.
Who's to say how much it matters anyway, it's often a pointless discussion / complaint to me. Normal laws are also everywhere, and we know that at least IQ is normal. "Smart but ignorant" is acceptable for many things, and besides, ignorance need not be a permanent condition. One doesn't need to know that Lisp machines in 1990 could do full 3D rigging, editing, and effects (https://www.youtube.com/watch?v=6VmJVNYfxDc) in order to be effective for their Java shop employer.
You're saying computer science history objectively exists.
I'm lamenting a lack of knowledge of it.
In the same way that the medical field has similar historical output, but the number of everyday doctors versed in medical history is quite low.
I'll dig around today and see if I can find representative polls related to computer history. Surely SO has asked a question at some point...
I see it from the perspective of someone working at a fast-moving, Silicon Valley-based software vendor. In that context, I don't see a lot of appreciation for history, in the day-to-day basis.
But it's not because people intrinsically don't care about it. It's more about our daily lives. We not only have to do what we all have to do, but we also have to keep up with the latest if we want to survive here. That line of thinking constantly pushes us forward, and doesn't reward looking at the past.
If anything, the great speed of the industry's development has made it particularly concerned with its own history - things become history quickly, the principals are typically still around, etc. The ACM HOPL conference I mentioned started when high level languages had a total history of barely over 20 years. When McCarthy (who was surely also a very busy person) wrote his History of Lisp paper, Lisp was younger than Java is today. And as I said in a sibling comment, the interest extends far beyond industry participants - there are many popular accounts aimed at general audiences and many new ones are written at a seemingly increasing rate. There's a Computer History Museum which prides itself on its working exhibits right here in Silicon Valley. Industry history-related articles are hugely popular on HN - there's a couple of them on the front page right now.
Glad it has finally arrived!
I agree. At the same time, unless you immerse yourself in oral tradition and try to assemble a picture of the diversity of what was going on from it, it's hard to get a sense of what has happened on a scale larger than a particular community, and sometimes not even then.
I occasionally think about trying to write a one volume history of computing.
> UNIX is foundational to essentially all software-driven technology today, in one way or another.
Except Windows, which comes from VMS, which predates Unix. And SQL, which comes from IBM mainframe land. I could probably come up with a few more, but it's late and I'm tired.
Unix was in development in 1969, had a manual released inside AT&T in 1971 and was announced publicly in 1973. The first source license was sold in 1975.
VMS was released in 1977 as VAX/VMS on the VAX series of computers. Before VMS the DEC hardware ran various other operating systems such as RSX-11, TOPS-10, and optionally AT&T Unix (which was developed on the PDP series, first the PDP-7).
> VMS was released in 1977 as VAX/VMS on the VAX series of computers. Before VMS the DEC hardware ran various other operating systems such as RTX-11, TOPS-10, and optionally AT&T Unix
Before VMS there was RSX-11 (note RSX not RTX):
https://en.wikipedia.org/wiki/RSX-11
"From 1971[5] to 1976 the RSX-11M project was spearheaded by noted operating system designer Dave Cutler, then at his first project.[5] Principles first tried in RSX-11M appear also in later designs led by Cutler, DEC's VMS and Microsoft's Windows NT.[6][7][8]"
In Dave's words:
https://tech-insider.org/windows/research/1992/11.html
"RSX-11M was introduced in 1973, 18 months after we started building"
Note Dave Cutler and note the years. I'd say the history is comparable.
Note "heavily influenced" doesn't make RSX-11 any more VMS than it makes Multics into Unix, or CP/M into MS-DOS, or the NeXT into a Mac, or an Alto into a Mac or Windows.
Also note that 1973 doesn't significantly predate 1973. Don't confuse the first source license same with the announcement of availability, which I also noted. And Unix was in use internally at AT&T the same year DEC started on RSX-11.
I don't claim that his entire life work predates the work of somebody else, just that the products he delivered are definitely comparable in its commercial availability with Unix.
I would also be not surprised if his products had more users at these first years than Unix did. To do the history right one should not project the results visible today (or which happened much later) to the history.
Pascal was not bad at that role, and it would have been possible to develop all the code which was eventually written in C in Pascal too. At least in the dialects that weren’t made just for education.
I think this illustrates how the philosophy of Pascal differed from that of C.
(No tagged unions, though. Systems guys are allergic to 'em, apparently).
Apple also added object-oriented features, but I don't count those as addressing anything borken or awkward in the base language.
Ahh, I was right (PDF file here): http://mirror.informatimago.com/next/developer.apple.com/tec...
These are not closures, they're still strictly nested and on the native stack, so the usual crashes happen if you execute a callback to something that has gone out of scope.
Remind me to tell you about "Fork Queues", a solution we had on the Newton for dealing with multiple entities that all wanted to have the main event loop (of which you could really only ever have one, but we cheated).
https://herbsutter.com/2012/05/03/reader-qa-what-about-vc-an...
https://www.reddit.com/r/cpp/comments/4oruo1/windows_10_code...
If you'd care to learn some real history, I suggest you read The UNIX-HATERS Handbook: http://web.mit.edu/%7Esimsong/www/ugh.pdf
With this book, you'll learn that UNIX and C are nothing admirable and have actually been responsible for successfully destroying much better systems and languages in the popular eye; languages including Lisp, APL, Simula, ALGOL, Smalltalk, and Forth all existed before C; systems such as ITS and Multics addressed concerns UNIX users still suffer under today.
Make no mistake, for all of RMS' admirable qualities, he's basically responsible for UNIX proliferating by copying it for GNU. You also shouldn't look at your modern BSD or GNU system and think this is what UNIX users used decades ago, because for all of their faults, GNU and BSD actually try to produce programs which work correctly and GNU goes much farther than several of the BSDs in this respect. The UNIX attitude is one of getting half the job done and leaving it at that.
In closing, UNIX has no philosophy. The UNIX philosophy is simply brand-named simplicity. The ideas of modularity and simplicity predate automatic computing and recorded history, and yet people will claim you're following UNIX if you write a program which adheres to these basic ideals. Further, those other qualities of this philosophy result in programs that aren't modular, simple, nor beautiful.
Blaming (or crediting, however you want to think about it) GNU for proliferation of Unix is anachronistic. RMS has repeatedly said he doesn't care for the design of Unix, but he chose it for the ease of implementation. GNU wasn't even bootable as a stand-alone OS before 1990s and certainly not production ready until Linux was. Using GNU utilities in proprietary Unices was popular, at least since the 90s, but I never heard anyone consider them a "killer app" for Unix.
Had Microsoft and IBM kept serious about their POSIX compatibility subsystems instead of a bullet point to win government contracts, and history would have played out much differently.
RMS worked on EmacsLisp.
Computing is absolutely full of widely deployed working technologies that enable people to get work done all day but that have rough edges. For every one of these, there is somebody saying that you should use <pet technology> instead. Usually with missionary levels of zeal. And yet at no point do they seriously address why people might have good reasons for adopting the allegedly inferior solution.
With an emphasis on "allegedly" sometimes. For example, C is hugely superior to Python in terms of machine efficiency. Does C support faster development cycles? That's right, you can write code that's practically as fast as tightly-optimized machine code! Does C prevent potentially catastrophic errors? That's right, you can write code that's practically as fast as tightly-optimized machine code! A lot of the True Zealots aren't quite as monomaniacal on a single narrow point, but the lack of ability to see a total solution is diagnostic.
So was ITS better than Unix? Not if you prioritize usability, support for application software, or ability to run on more than a single family of very expensive mainframe computers the world had begun to abandon by the time Unix hit its big growth phase. You can say similar things about LispMs, although they were more usable.
https://multicians.org/myths.html
> 1. Myth: Multics failed in 1969. Bell Labs quit, Multics survived.
Now that we can use Multics about as easily as we can use Ancient Unix versions under emulation, you can spin up a perfectly functional 1980s-era Multics and see that, no, really, Multics evolved into something you can do stuff on.
That's the problem with narratives: They're both inevitable and inevitably limiting, narrowing the focus to what makes a comprehensible story as opposed to a day-by-day list of what happened. Humans create narratives as naturally, and as unavoidably, as breathing, but we have to be aware of what they do to our comprehension of history.
Since history belongs to winners, if it wasn't for the accessibility of old conference papers and computer manuals, that would be indeed the only one we had to believe on.
The systems you remember so fondly (TOPS-20, ITS, Multics, Lisp Machine, Cedar/Mesa, the Dorado) are not just out to pasture, they are fertilizing it from below.
Your judgments are not keen, they are intoxicated by metaphor. In the Preface you suffer first from heat, lice, and malnourishment, then become prisoners in a Gulag. In Chapter 1 you are in turn infected by a virus, racked by drug addiction, and addled by puffiness of the genome.
Yet your prison without coherent design continues to imprison you. How can this be, if it has no strong places? The rational prisoner exploits the weak places, creates order from chaos: instead, collectives like the FSF vindicate their jailers by building cells almost compatible with the existing ones, albeit with more features. The journalist with three undergraduate degrees from MIT, the researcher at Microsoft, and the senior scientist at Apple might volunteer a few words about the regulations of the prisons to which they have been transferred.
[0] https://en.wikipedia.org/wiki/Wollongong [1] https://en.wikipedia.org/wiki/Royal_National_Park
If we take ENIAC [0] as the first computer (1945), then our industry is 74 years old. Let’s compare this with another tech industry – the aviation industry. The Wright brothers flew in 1903 [1]. 1903 + 74 = 1977.
I was a kid in the 1970s, fascinated by aerospace and aviation. There was a massive amount of aviation history around (I still have some of the aviation history books I bought then). So I think there has been plenty of time for the history of computing to appear. Even if we added another 15 years to shift the start of the history of computing (software engineering) at the development of COBOL [2] (approx the same as FORTRAN or LISP), there has still been a lot of time.
So I don’t think it’s industry youth / age alone.
[0] https://en.wikipedia.org/wiki/ENIAC
They have an excellent YouTube channel with thousands of hours of interviews with important people in the industry. They posted an interview with Brian Kernighan last week.
We appreciate history far too much. Unix is holding computing back, and it needs to go. It served its purpose, but it's done. It represent very few of the things that are important to computers in 2019, and it actively hinders many of them.
Unix needs be respectfully let to go into retirement, and new things given a chance to replace it finally.
It looks to me like our current crop of unix derivatives and unix-likes is catering for a lot of needs very well. We have windows which is non-Unix.
I think the onus is on you to show that something else could be better, and better enough that the pain of relearning would be worth it.
FWIW Chris Harrison (https://www.researchgate.net/profile/Chris_Harrison2) taught a compulsory history of computing class at UMIST in the UK when I was there in 1999-2003. This seems pretty unusual, especially now that I live in the US and understand more about how universities work here.
That is British-grade understatement right there. :)
Recently I discovered the AT&T history channel, with this gem: https://www.youtube.com/watch?v=tc4ROCJYbm0
There is a massive difference in appearance and clarity between him and the other people appearing in that video, even the "presenter"...
I go back to this video every once in a while ever since I discovered it a few years ago. I just think it is super relaxing. When I first watched it, I was beginning to use Linux and, when I opened my terminal emulator, I was like: "It's a Unix system, I know this!" The pipelines explanation was incredibly clear.
I also love the Computerphile episodes with professor Kernighan.
-----Original Message----- From: Brian Kernighan <bwk@cs.princeton.edu> To: Mike Russo <mike@papersolve.com> Subject: Re: please publish ebook of Unix memoir!! Date: Mon, 28 Oct 2019 11:12:48 -0400
Mike --
I have just uploaded a Kindle version, but it has to go through Amazon's approval, which could take a day. I also can't see a preview on a physical device, so I have no idea whether it will actually look right. If it doesn't, I'll have to pull it and figure out an alternative.
Brian K
On Mon, 28 Oct 2019, Mike Russo wrote:
if you get an ebook of it out there it will sell even more!! and thanks for writing it!
--
Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 Your random quote for today: ..you could spend all day customizing the title bar. Believe me. I speak from experience. -- Matt Welsh
Australia: A$11.99 [my local site] https://www.amazon.com.au/dp/B07ZQHX3R1/
or US$8.20 https://www.amazon.com/dp/B07ZQHX3R1/
vs printed book US$18 https://www.amazon.com/dp/1695978552
I own and have read his following books and they were all superb!
The Go Programming Language
The Practice of Programming
The C Programming Language
The AWK Programming LanguageFor a long while I maintained a program that took the compiler output (intermediate form) from one toolchain, and converted it to another (intermediate form).
Why awk? It was faster, and more readable, than any of the alternatives, whilst being much much shorter.
In a way, awk, like sed, cut(1), paste(1), comm(1) is closer to the original Unix design philosophy of pipeline composition of simple, single function tools. cut, paste, and comm are simple, single function. sed is programmable, and awk even more so, but they're still optimized for being used in pipelines. Perl still has lots of options designed for pipeline usage (-p, -n, -a, -0, -F), but was always capable of doing general purpose programming, and python is primarily general purpose programming.
Personally, I've never learned sed, cut, paste, or comm; the only traditional text processing tools I use are sort & uniq. I know awk, but the one time I wanted to use it in the last 5 years in an official script, I was shouted down by confused youngsters. I use perl for one liners, and python for any nontrivial scripting.
I once wrote code with curl, awk, sed, jq, xargs and in less than 30 lines implemented something that would have taken about 200 lines of go code. I did it too in less than an hour when I was told it would take a day. It was urgent and I needed the result fast.
There are different kind of hammers, different kind of nails and screws. Know your tools, use them when it's called for. You can use a wrench as a hammer, doesn't make it a great one.
Youtube-dl and awk to grep out of a YouTube channel.
If I were doing it again I would use youtube-dl -o to just print the base64, instead of "manually" renaming it to that with ls>awk:mv>sh.
Brian Kernighan mentions in the book that awk provides "the most bang for the programming buck of any language--one can learn much of it in 5 or 10 minutes, and typical programs are only a few lines long" [p. 116, UNIX: A History and Memoir]. Also keep in mind Larry Wall's (inventor of Perl) famous quote/signature line: "I still say awk '{print $1}' a lot."
More background on awk from Brian Kernighan in a 2015 talk on language design: https://youtu.be/Sg4U4r_AgJU?t=19m45s
Database Internals, Snowden book, Algorithms book, BPF Performance tools, Your Linux Toolbox, The Go Programming Language, The Pragmatic Programmer, Quantum Computing, A Programmer's Introduction to Mathematics, An Elegant Puzzle
I had to go up to page 14 of the items list to find all of the items you listed and to find the wire-type soldering iron tip cleaner.
Edit: Took out unneeded snark. It seems I fell prey to Amazon algorithms.
I only see these items plus another cooler and a Macbook stand.
It's basically a fancy version of those metal pot scourers for cleaning the burnt bits off, but without damaging the metal plating on the tip.
You put a bit of solder on the tip, but that's just to help apply heat to the joint.
Amusingly, Alexa notified me when it arrived, and the notification was "Your purchase UNIX has arrived."
I have stopped buying paper books over past several years as they simply accumulate moisture and mould in the climate I live in and make rooms smell musty. It is incomprehensible to me why primarily textual books would be published without and ebook version these days.
This is already #243 out of all Amazon, so I'd say he was correct in this ...
edit: he replied to me to say he's uploaded a version today that has to be approved.
Source: A friend of mine who has been trying to get onto the NYT best seller list.
I can’t remember a single mainstream title that hasn’t had an ebook version for preorder and first day sales. This includes “Bad Blood”, “Super Pumped”, and “Catch and Kill”.
I highly recommend hitting up a Vintage Computer Festival if you've got the opportunity (http://vcfed.org/wp/festivals/). Not only did Kernighan and Thompson speak, but also Joe Decuir of Atari & Amiga fame.
On that note, this post reminds me of another great book (eye-opener of sorts) that I read a while ago that takes the reader through the history of some important milestones in hardware and software https://www.amazon.com/Computer-Book-Artificial-Intelligence...
After I read this, I was surprised to learn that the concept/ideas which seem very recent because of the interest/research around it have actually been around for a long time. For instance, I learnt that "Secure Multi Party Communication" has been around since 1982, Verilog since 1984, AI Medical diagnosis since 1975.
While the story/writing is interesting, like most self-published titles this book needs some professional layout/editing help. The second and sixth pages of chapter 1 are full page Google Maps of central NJ and Google Satellite view of Bell Labs. Eeek.
This may also be why there is no Kindle version yet. Many pages have full-color images and would need significant changes for a decent Kindle reading experience.
like, that's all books on Amazon US
The book is a great read otherwise!