Hacker News Reader

Comment by dbcurtis | Hacker News Reader

Brian Kernighan gives credit to Doug McIlroy for coming up with the idea in 1964. Quoting from _UNIX A History and a Memoir_, page 68:

"To put my strongest concerns in a nutshell: 1. we should have some way of coupling programs like garden hose..."

from a Bell Labs internal memo.

Kernighan's book is a great read, BTW. Highly recommend to all UNIX nerds.

View on HN

A scan of the original 1964 memo:

https://www.oreilly.com/library/view/making-sense-of/9781492...

Thank you very much for digging this up, and to all the other commenters here for the useful references.

FYI: it's literally in the linked article from this HN post, maybe start reading that, not just the headline and then commenting...

FYI: I'm the author of the linked article, and I added a link to the image after it was posted here.

This has got to be one of the funniest exchanges I have ever seen on Hacker News.

Thanks for adding context, that wasn't clear at all to me, it seemed like the classical "Reads headline and then some comments, and post 'smart' stuff which the original article read anyway" response to me.

Sorry for the "accusation" then.

Now I'm confused and curious as to what the ‘library filing scheme’ means. Does it mean shared libraries? IIRC Unix stuff was statically-linked at first. And then again, what are ‘indexing’ and ‘data path switching’?

+1 for Kernighan's book. I'm not one for impulse purchases, but as when I saw a link on HN, I immediately bought it and read through it. I lent it to a professor of mine and he's using as a text for his history course next semester.

Here's a nice McIlroy interview with broken formatting: https://www.princeton.edu/~hos/mike/transcripts/mcilroy.htm .

Does anyone know how the pipe idea compares to Erlang messaging?

As far as I can tell the implementation of each might look like the other in some ways (a queue of some kind with MutEx guaranteed somehow).

Does anyone have the knowledge to write a little on this? Or maybe point to a similar article for that style of messaging?

Pipes are bytes Streams. Communication between actors as well as CSP channels are object streams.

Pipes exercise backpressure on the writer - if the pipe is full the writer is blocked. Actor systems mostly use seemingly unbounded queues. The sender will not get blocked.

> Pipes exercise backpressure on the writer - if the pipe is full the writer is blocked. Actor systems mostly use seemingly unbounded queues. The sender will not get blocked.

Erlang does suspend (block) processes that send to ports or nodes when the buffers for that get full; but not when sending to local processes. There used to be an optional reduction count punishment for senders when sending to messages with larger mailboxes, but it seems that may have been removed. I don't think it would be too hard to add a feature where sending to a local mailbox over a specified size caused the sender to be suspended, but tracking it might be a little difficult.

Here's a short 2014 paper from McIlroy himself discussing his early thinking back then: https://www.cs.dartmouth.edu/~doug/sieve/sieve.pdf

TL;DR: McIlroy was applying the concept of coroutines, which was described by Melvin Conway in 1963. Two processes communicating over a pipe are basically coroutines, except instead of passing structured data they're just passing bytes.

A Unix pipeline is a set of multiple coroutines. In fact, Tony Hoare's 1978 Communicating Sequential Processes paper cites the UNIX shell[1] for the concept of coroutines, and discussion of coroutines figures prominently in that paper. See https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf CSP basically models the behavior of a large set of coroutines.

AFAIU, Erlang was partly inspired by CSP. You can draw a straight line from coroutines, through Unix pipelines and CSP, to Erlang's processes.

[1] Specifically, Ken Thompson's 1976 paper, "The UNIX command language." See https://archive.org/details/theunixcommandlanguage

"...a practical exploration of using Python coroutines..."

  http://www.dabeaz.com/coroutines/index.html

Reply on news.ycombinator.com