Hacker News Reader

Comment by dale_glass | Hacker News Reader

One weird thing I noticed over time is the enshrinement of old and crappy technologies, in the face of all reason. Some examples:

Some people for some reason I can't understand have warm and fuzzy feelings about FTP, despite it being a horrible protocol in multiple ways. Yet, for a long time it was often thought of being the "proper" way to download stuff, even though even HTTP is technically better in many respects.

You still find plenty people harping on about the Unix Philosophy, even though that in modern times it's nigh irrelevant, and significant parts of it are technically obsolete. Yeah, parsing text streams was sorta okay in the 80s, but is very troublesome and brittle in the modern age.

And there's an odd amount of people who are still mentally stuck on the Linux of the 90s and having trouble to understand that things have gotten more complicated since then, and software has had to adapt to things like laptops and wifi.

View on HN

I feel the need to be "that guy" cause this is hacker news, but the Unix philosophy isn't really about parsing text streams, it's about decomposability of a problem. It's way easier to test and debug 4 simple programs feeding into one another other like a flow chart than building a mega program with an insane internal state (the your problem can be decomposed into 4 subproblems).

The basic core revelation that the Unix folks had was "gee wiz, I seem to need to sort data/open a socket/make a directory super often, and every time I personally try to code one of these operations up it's a huge headache that adds hours to dev time"

Everything has good and bad cases where it can be applied of course.

But I see this as a good example of a religious issue because you can often find people think that it's a principle worth rigidly adhering to, regardless of whether it actually does produce better results in a given case.

For instance my experience is that such a model tends to get bad when things get sufficiently complex. Things like GPG and cdrecord are painful to interact with, and would arguably work much better if they were a library because a text stream is actually a pretty bad communication channel.

And sure, I agree that back when Unix was created the model made sense, but it's increasingly less and less relevant.

As the previous poster said, it’s not about text streaming, it’s about composability. One “modern” paradigm that follows the Unix philosophy and has gained a lot of traction is micro services which are mostly viewed positively. You need to extract the idea from its original implementation to apply it today.

> Things like GPG

I, for one, would very much prefer GPG to be decomposed to follow the unix philosophy - I think I would have a much easier time understanding and composing commands like `gpg-sign --keyfile ~/mykeyfile somefile | gpg-encrypt --to ~/someperson.pub | gpg-asciiarmor > somefile.gpg` than whatever the monolithic incantation is. This would make it much less painful to interact with, especially because GPG is doing a series of transformations on either streams of text or raw binary.

And that'd be awful for many use cases, because maximizing that would make it incredibly unwieldy and brittle.

For instance if you want to deal with keyrings, and be fully philosophy compliant, then that might be a gpg-keyring tool, which means now we have a problem. A signature operates on both a key and a file, and a pipe only works on one stream. Then there's the issue with error handling, since any of those bits can go wrong, and then somebody has to figure out a good way of dealing with all of that.

Or, we could have libgpg instead, in which case our mail client could avoid screwing around with text streams, signals and process management, and just have a nice API with features like callbacks and well defined datatypes that would be a lot more comfortable to use.

EDIT: Even with just the 3 primitives you've outlined, we already can create a huge mess. Is it sign then encrypt, or encrypt then sign? We can combine signing, armoring and encrypting in any order and any amount, requiring the recipient of this insanity to somehow figure out what we did, and then somebody else will do it differently.

You're still on the text streams!!

On the 4 simple programs vs a complex one...I feel this a microcosm of the microservices vs monoliths discussion, which can also devolve into quasi-religious battles.

> You still find plenty people harping on about the Unix Philosophy, even though that in modern times it's nigh irrelevant, and significant parts of it are technically obsolete. Yeah, parsing text streams was sorta okay in the 80s, but is very troublesome and brittle in the modern age.

Have you confused the Unix Philosophy with Unix itself? The Unix Philosophy doesn't have any technical components which might become technically obsolete.

It's a fuzzy matter because it doesn't have a single agreed on definition. But yes, some parts are definitely showing their age:

> Write programs that do one thing and do it well.

Putting things together from a kit of parts isn't necessarily the optimal way to do many things. In many cases tight integration actually produces superior results, and debugging a system made from many cooperating bits isn't necessarily any easier.

> Write programs to handle text streams, because that is a universal interface.

Or don't, because it's a bad interface, and a cause of many bugs and security issues.

>> Write programs that do one thing and do it well.

> Putting things together from a kit of parts isn't necessarily the optimal way to do many things. In many cases tight integration actually produces superior results, and debugging a system made from many cooperating bits isn't necessarily any easier.

Define "optimal" here. Without a definition this is a meaningless statement. But even if your definition of "optimal" is "easiest to debug", I'd have to disagree. Tightly coupled systems are nigh impossible to debug because you have to have the whole system composed together to debug anything. Loosely coupled systems allow you to debug each part separately, and small components mean they have little you have to debug. The next part of debugging is the interaction points, but that's easily constrained with a moment's thought when designing and developing a system.

If "optimal" is about performance, that's a debatable thing. The tightly coupled shit some people put together at my office is terrible with respect to performance. It's bordering on negligence the way they designed it and trying to fix the performance issues is almost impossible again because of the tight coupling of different components.

Regarding text streams: Ok, one weak point in some definitions/descriptions of it. The idea of utilizing a more universal data format though still stands. 1 million or so different binary formats isn't much better.

That is kind of the point, you have to define the problem and the tradeoffs involved. Then sometimes one approach better than another. That determination should be made on a case-by-case basis, rather than as a religious matter.

So for instance stringing too many things together also leads to confusion due to the many interactions between disparate bits.

At some point you can end up too much parsing/serialization in the mix, too much accumulation of startup costs, or dealing a system that has components written in shell script, Perl, Python, C, C++ and Rust making the whole system difficult for any random person to comprehend fully.

>Putting things together from a kit of parts isn't necessarily the optimal way to do many things. In many cases tight integration actually produces superior results, and debugging a system made from many cooperating bits isn't necessarily any easier.

The reason why this is part of the Unix philosophy is not merely because of the productivity it provides, but largely because of the power it affords end users. Most of the programs on my computer are GUIs that don't interact with each other. How much more powerful and flexible would my PC be if they were designed to do one thing well and be slotted together.

I think back to my days teaching at a college whose CS labs ran on NetWare 3. For “reasons” the user accounts were things like PASCAL; all the students in a course that used Pascal used this account (and kept their files on floppies). I wanted to move to an environment where students had individual accounts. The IT folks said that was way too much work to implement, and showed me the GUI (or TUI?) app they used to manage accounts; I agreed that this made individual accounts very difficult to manage. I sketched out a solution that used awk to identify adds and drops from the class lists the Registrar's office produced, along with scripts for creating and deactivating accounts as appropriate. This elicited much mystification from the IT folks.

A year or so later, we got individual student accounts. I resolutely refused to discover how they implemented it.

The Unix Philosophy is about building small components that compose well together. The fact that some applications benefit from tight integration in no way derogates from its value.

So absolutely true. I work in an “operational facing” engineering group. We are engineering but process data (daily/monthly data processing for factory and customer requests). Having a composable process using the Unix philosophy makes it trivial to stop processing at any point in the work flow for that one customer request that comes up once a year. There is no ROI to make the work flow support these edge cases, it’s cheaper even after a decade or more just to one-off these special cases. But this would be completely impossible if we didn’t architect the system in smaller components. As you say, it’s not text processing, it is units of processing that can be composed, stopped, reordered or have a custom step in the middle.

Stream is a bad interface in general and a cause of (…). Our worst ones were mostly binary, afair.

The name is the reason.

You see, it is the File Transfer Protocol.

Do you need to transfer a file? Use the protocol designed to transfer files! Why would you use something silly like a Hypertext protocol for binary!?

I see similar conflation of the name-vs-reality all the time. My favourite is the “secure” network that is differentiated not by firewall access policy, but by name only. Literally just a label.

Do you need to transfer a file? Use the protocol designed to transfer files! Why would you use something silly like a Hypertext protocol for binary!?

Because it’s faster with http, doesn’t require firewall/nat trickery at either side, is secure out of box via https, can be easily gui-ed via index plugin, cached via a proxy/cdn, supports “moved elsewhere” and other issue reporting.

And if you ignore that “Hypertext” word that is there historically and look at http as just a metadata+data transfer protocol, it is clearly superior to ftp for any sort of data transfer.

It's a pity there's no font-style for sarcasm, like italic or bold...

> ftp

Don't know if that fits the bill, but for a lot of people (me included) ftp is the reminder of simpler times, where you'd just put your username /password and address in an explorer window and drag and drop files.

It wasn't secure, nor reliable, but it felt really simple and easy to grasp, especially for inexperienced users. We have way better choices now, but I've spawn many more quick and dirty ftp servers at home than I'd be willing to publicly admit. Hell, my printer pushes scanned files to an ftp.

I much prefer Unix philosophy to contenerized microservices for Todo list app philosophy.

I'm not convinced they are different. Take a large program and break it down into smaller programs with a singular purpose.

Unix is text processing.

Contenerized microservices is also text processing, but you add 2 new layers of abstraction for every operation you do.

Containerized microservices for a todo list app is an example. No one seriously claims that an example like that needs to be implemented that way.

People were using the “Unix philosophy” in the 2000s to try to satisfy the demands of the consumer internet. It failed miserably at that, and was replaced.

Microservices is UNIX philosophy of multiprocessing and IPC taken to the network level.

Sun Microsystems business quote on their manuals, "The network is the computer".

Then again, besides being a famous book quote, most commercial UNIXes never cared about such philosophy.

Engineering is all about trade-offs, and taking good trade-offs to the next level usually makes them bad. This is a prime example.

Reply on news.ycombinator.com