Hacker News Reader

Comment by dale_glass | Hacker News Reader

It's terrible, but the Unix philosophy leaves much to be desired. Like the example:

    cat file | grep "foo" | awk '{print $2}'

That's terrible in this day and age. Text streams are terrible to work with in modern times. We've long moved past /etc/passwd-like formats, and even with those such tooling has been extremely subpar. What if you put a colon in a file with colon separated fields?

There's no end to the nonsense you have to deal with that just shouldn't even be a problem in 2025.

View on HN

Perhaps because it is a dumb example.

  awk '/foo {print $2}' file

does the same thing, without a pipeline.

If one wanted robust rich pipelines, one could use PowerShell (which I used as my UNIX and Linux shell for a while when I was programming in it everyday as part of dayJob).

This is literally the best thing about Unix tools. You can write a compact little Awk script and pipe that with a few coreutils to parse mountains of text into sensible data. I use this kind of thing every day.

Text streams have their advantages and disadvantages. Advantages include being easy to create, modify, view without special tooling. Given the nature of the Unix shell, you can apply that equally to both data and code. Of course, the disadvantage includes the potential of poorly defined streams, or of steams being difficult to interpret due to them being human readable. But that was as true when Unix was created as it is today.

> What if you put a colon in a file with colon separated fields?

This problem has been solved for thirty years.

https://en.wikipedia.org/wiki/Escape_character

awk -F: will not recognize that

Okay, what's the actual problem? The only material complaint I see in your comment is that you have to not use control characters as data. And I guess that's true, but I really don't think it's that big of a deal.

That it's kind of terrible and decades out of date.

Yeah, most problems are solvable somehow, but you still have to solve them instead of getting things done, and the results are often fragile. Like awk -F: /etc/passwd works nicely, until there's an \: in there that awk completely fails to understand.

So the alternative is opaque types? Powershell tried adding structured types for the output, and it has some benefits, but now your pipeline composition options are severely limited. Only the tools that understand the output structure of the prior step are options. What if you have a new structure output type? You now need all new tools to do anything with it. I don't say text is an ideal format by any means, but it's a _universal_ format. That unviversality is necessary to be useful long term. Unix just picked text since it was easy for a human to look at and reason about.

Or are you suggesting monolithic tools instead of composable ones? Tomorrow I have a need you haven't thought of. Now your tool is useless to me and I have no way to do what I need. Modular composition isn't just a Unix philosophy, it's a basic development principle because it puts the power in the hands of the user to create almost anything they can think of, rather than restricting it to only what the creator thought of and implemented.

I'm not really sure what you're arguing is "terrible" about the solution. If you don't like the hazards and difficulty of dealing with something like your colon separator being in the data too, let me introduce you to data handling 101. It doesn't matter what format you put data in, the same problem exists. Even raw binary needs a delineator between elements, and eventually that delineator is going to need to be in the data itself somewhere. Unless you suggest we shouldn't be able to represent all data? It's the reason virtually every language supports escaping special characters (E.g. JSON's backslash escaping of quotes).

Yes, I think Powershell is a step in the right direction. Not ideal of course, but where we should be going in general. At a minimum everything should be some sort of JSON-ish structure, if possible with support for common datatypes (strings, integers, floats, arrays, hashes, dates)

It's terrible because while it was a good idea back in the day, today everything tries to get away from it as quickly as possible. Most any tool gets wrapped in a library that actually exposes a much more pleasant to use and safer interface.

And the usage of text streams means everyone is reinventing the same wheel decade after decade, everyone in a slightly different manner and with different footguns.

Reply on news.ycombinator.com