Hacker News Reader

Comment by grumbel | Hacker News Reader

> Output should be free from headers or other decoration.

One way to solve this nicely, that seems to be getting more common, is to use `isatty()` to check if the output is a terminal and if so print with decorations, otherwise leave them away.

`ls` for example will output unprintable characters, even just space, in quoted form on a terminal:

    $ touch 'foo bar'
    $ ls
    'foo bar'

But when redirected, it will output the raw value:

    $ ls | cat -
    foo bar

View on HN

> One way to solve this nicely, that seems to be getting more common, is to use `isatty()` to check if the output is a terminal and if so print with decorations, otherwise leave them away.

This is not a good idea because it can lead to surprising the user (ls's behavior is actually bad from that perspective). For example you run a program

    $ foo
    ID Thing   What
    4  Cat     Mews
    2  Dog     Woofs
    5  Canary  Tweets

...then you run the result through sort and trying to avoid the header...

    $ foo | tail -n +2 | sort

...except instead of the expected result you get...

    2  Dog     Woofs
    5  Canary  Tweets

...because the program tried to be smart instead of consistent. This is also against the GNU guidelines as mentioned elsewhere.

The little surprise is worth the general improvement in usability (e.g. colors, progress bars, filenames you can copy&paste, terminal not getting corrupted by escape sequences, etc). It also makes it clear that the terminal output is for user interaction, so programs no longer have to be both UI and API at the same time, they can focus on one or the other, making both much better and cleaner as a result.

> ...then you run the result through sort and trying to avoid the header...

The much more common scenario would be doing `foo | sort` and then ending up with random header text in the sorted data. Few people will add a `tail` the first time they type that command or remember do it every time they use it interactively. With `isatty()` it behaves as the user expects it right from the start.

> `ls` for example will output unprintable characters, even just space, in quoted form on a terminal:

Pedantic quibble GNU does this for special characters, as in your example the space is very much printable. Specifically this became default with coreutils 8.25 https://www.gnu.org/software/coreutils/quotes.html

I noticed that last week and was quite surprised by it.

I was writing a script to get the permissions of all files. I wrote something like ls -l | grep -oE '^[^ ]+' and ran it in my home directory for testing. And then I was surprised that the output was wrong. Turned out I had files with \n in their name there and ls was printing them on two lines which confused the grep. (I still used that script, since I did not have any \n files on the real system)

I was actually building an exam for a course involving shell scripting. A common question was, do something with all the files in the current directory, like grep them or delete them. The lecture notes said to use * for all those files, but then I realized rm * would not work in all possible cases. I spend like an hour to find a hopefully correct solution. However, the professor said, the students would never figure it out in an exam, and I should just put * as model solution. The shells is extremely brittle

When dealing with filenames one has to get used to always using '\0' separated output instead of newline separated output, as filenames in Linux can contain everything except '\0' and '/'. Luckily most tools are prepared for this and have options to either output '\0' separators or accept them as input.

Dealing with all the files in the current directory would look something like this (for demonstration, can be made shorter by using '-name' or '-regex' option from 'find'):

   find . -maxdepth 1  -type f -print0 | grep -z needle | xargs -n 1 -0 echo

Looping over '\0' separated output is possible as well with this:

   find . -print0 | while read -r -d $'\0' filename; do echo "$filename"; done

It stops being brittle when used correctly, but can take a bit getting used to and can get a little verbose.

The one I found to delete all files was rm -- .[!.]* ..?* * 2>/dev/null

Well, if you're parsing ls then you're in a world of hurt no matter what you do anyways: https://mywiki.wooledge.org/ParsingLs

We should stop using ls as a part of a example pipe, it's usually a very poor example.

Reply on news.ycombinator.com