Hacker News Reader

Comment by eqvinox | Hacker News Reader

Not sure I see the point of this. Remembering a bunch of options on one tool is no better than a bunch of tools/constructs? Especially if the latter are useful elsewhere. And it can't be used for scripts unless you start shipping it alongside, which… nah.

Just go with

  foo | while read X; do bar "$X"; done

View on HN

Bash while loops are pretty readable and the above would be a nice to iterate over lines if it wasn't for that gnarly pipe, which is a common source of errors in this construct. Remember that a pipe starts a new shell. So:

  grep stuff file.txt | while read key value ; do [ "$key" = "target" ] && found="$value" ; done

where you might expect $found to end up with the value for the line that has "target" in the first column. Then you notice that darn pipe symbol. The variable found is set in a subshell that terminates and the value is lost. This is a problem every time you need to keep some sort of state when looping. If you can tolate a bash-ism then you could do:

  while read key value ; do [ "$key" = "target" ] && found="$value" ; done < <(grep stuff file.txt)

but that doesn't read as nice and isn't compatible. It does avoid a common source of problems though, and might be worth getting into muscle memory for the times it is needed.

> and isn't compatible

is that a GNU v POSIX type of compatibility issue?

bash vs. POSIX sh (or some other shells)

zsh should be bash compatible on this AFAIR

You can put the "< <( foo )" before the "while" btw, for pipe-like ordering of things. All redirections can be anywhere on the command line.

I use "| while read" as well, because it works well in a pipeline and handles embedded spaces. It doesn't handle embedded newlines, but in practice, real files have embedded spaces, while embedded newlines only happen in test cases and exploits. (You can do actual NUL-delimited reads with `-d ''`, but for a quick command-line operation that's generally not necessary, and if you're going to be that careful you probably also need `-r`.)

Yes, I think the author is just showing their ignorance, not actually doing anything about that ignorance, and re-inventing the wheel:

    $ find /path -name "*.pdf" -print0 | xargs -0 -I {} echo "Processing: {}" # handles paths properly, obviates the need to write anything new whatsoever

Seriously kids, learn your tools and check yourself before you wreck yourselves writing tools that really, really don't need to be written.

find print0 with xargs 0 is such an awesome combination. One day I'll remember to use it instead of piping into while read ln; do ...

Depending on the actual command, this can be far slower and less efficient than xargs. You're creating a separate process for each invocation of bar when a lot of commands will take many targets for a single invocation.

Try this with find and grep vs xargs. There's a big difference.

3 reasons against that:

* in a lot of cases the performance just doesn't matter

* xargs gets you the spaces in filenames landmine

* some commands don't even support multiple target filename arguments

For scripts in long term use, yeah, sure, figure out xargs maybe. Any other situation with a "| while read" solution, especially on an interactive session, is an oddball and simplicity wins.

xargs supports null termination.

While it's true not every command supports multiple targets, presumably you know if you're using one of those commands.

xargs sure does support null termination.

Does whatever I'm piping into it though? It's "find" in maybe 30% of cases for me, no more, and null termination is not exactly common in other tools.

I'm not sure what you're trying to say. If you need null termination you're going to be a lot better off adding it at the source (say, with sed) than trying to make a while loop work in a shell script. If you don't need null termination, then... don't use it.

I am saying I have a whole bunch of other (not find) sources of items (not even filenames necessarily) that I need to loop over, none of which do null termination on output. It's an item per line, with spaces unescaped. I would either need "| tr '\n' '\0' | xargs -0" or "| xargs -d '\n'".

Considering the body of a while loop is more flexible too, I spend the brain capacity on remembering the details on that rather than xargs.

I'm well aware xargs is sometimes called for, but it's rare and when it happens I go look up the man page.

>* xargs gets you the spaces in filenames landmine

Ignorance of xargs gets you the landmine.

Understanding of xargs, helps you complete the mission without blowing off limbs.

    $ find . -name "Some Files With Spaces*.txt" -print0 | xargs -0 -I {} echo "Processing: {}"  # landmine avoided

cf. sibling, find is not the only thing to pipe from.

While FTW! I just wondered whether it could be simplified because I get a bit tired of typing out my own belt and braces version of it:

  { # code that generates one item/filename per line of output }  | { while read ITEM; do # do something with "$ITEM"; done; }

So I just tried this and it seems to work for a trivial case although you need 1 escape:

  enum() {
    local source=$1; shift; local action=$1; shift  
    { eval "$source" ; } | { while read ITEM; do eval "$action"; done; }
  }

  $ enum "find /tmp" "echo found: \$ITEM"

  found: /tmp/.XIM-unix
  found: /tmp/.ICE-unix

I just use while’s default variable name, $REPLY most of the time. But `while` is my preferred tool for the xargs problem in most cases.

  printf 'silent data loss' | while read l; do printf '%s\n' "$l"; done

though at the point I need to write

  while IFS= read -r line || [ -n "$line" ]; do
    printf '%s\n' "$line"
  done

I'll use a different language.

Reply on news.ycombinator.com