Hacker News Reader

Comment by aaaaaaaaaaab | Hacker News Reader

Some additional tips:

1. You don't need the parentheses.

2. If you use process substitution [1] instead of a pipe, you will stay in the same process and can modify variables of the enclosing scope:

    i=0
    while read -r v; do
        ...
        i=$(( i + 1))
    done < <(do_something)

The drawback is that this way `do_something` has to come after `done`, but that's bash for you ¯\_(ツ)_/¯

[1] https://www.gnu.org/software/bash/manual/html_node/Process-S...

View on HN

I use this exact pattern a lot. One thing to consider is that in the process substitution version, do_something can't modify the enclosing variables. The vast majority of the time I want to modify variables in the loop body and not the generating process, but it's worth keeping in mind.

One common pattern I use this for is running a bunch of checks/tests, e.g.

    EXIT_CODE=0
    while read -r F
    do
        do_check "$F" || EXIT_CODE=1
    done < <(find ./tests -type f)
    exit "$EXIT_CODE"

This is a more complicated alternative to the following:

    find ./tests -type f | while read -r F
    do
      do_check "$F" || exit 1
    done

The simpler version will abort on the first error, whilst the first version will always run all of the checks (exiting with an error afterwards, if any of them failed)

I usually write zsh scripts and I think there’s a shell option in zsh that allows the loop at the end of the pipe to modify variables in the enclosing body: I remember at least one occasion where I was surprised about this discrepancy between shells.

Interesting! Indeed, Greg's BashFAQ notes it too: https://mywiki.wooledge.org/BashFAQ/024

>Different shells exhibit different behaviors in this situation:

>- BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator ('<', '>').

>- BASH, Yash and PDKsh-derived shells create a new process only if the loop is part of a pipeline.

>- KornShell and Zsh creates it only if the loop is part of a pipeline, but not if the loop is the last part of it. The read example above actually works in ksh88, ksh93, zsh! (but not MKsh or other PDKsh-derived shells)

>- POSIX specifies the bash behaviour, but as an extension allows any or all of the parts of the pipeline to run without a subshell (thus permitting the KornShell behaviour, as well).

Check out bash lastpipe option.

Yeah, although I use the parentheses mostly because I like how it reads. And that process substitution trick is important too.

I think the redirection can come first, though (not at a computer to test):

    < <( do_something ) while read . . .

Yeah, for commands, the input/output redirections can precede them, but for some reason it doesn't work for builtin constructs like `while`:

    $ < <( echo foo ) while read -r f; do echo "$f"; done
    -bash: syntax error near unexpected token `do'
    $ < <( echo foo ) xargs echo
    foo

    $ bash --version
    GNU bash, version 5.1.4(1)-release (x86_64-apple-darwin20.2.0)

Maybe wrap the loop either with parentheses or braces?

Tried that, but nope :D I'll let you figure this one out once you get near a computer!

Redirection like this doesn't seem to work if it comes first on GNU bash 5.0.17(1)-release.

For documentation purposes, this is the exact thing I tried to run:

    $ < <(echo hi) while read a; do echo "got $a"; done
    -bash: syntax error near unexpected token `do'

    $ while read a; do echo "got $a"; done < <(echo hi)
    got hi

Maybe there is another way...

One way which isn't great, but an option nonetheless… The zsh parser is happy with that form:

    $ zsh -c '< <(echo hi) while read a; do echo "got $a"; done'
    got hi

My position isn't that it is a good reason to switch shells, but if you're using it anyway then it is an option.

I’ve always preferred zsh and, as I’ve slowly adopted nix, I’ve slowly stopped writing bash in favor of zsh

This is not POSIX compliant though.

These days bash and/or zsh are available nearly every place I care about, so I find POSIX compliance to be much less relevant.

No, process substitution must be provided by the kernel/syslibs, it is not feature of bash. For example there is bash on AIX, but process substitution is not possible because the OS do not support it.

ksh93 depends exclusively on the kernel implementation of /dev/fd devices. I just checked `cat <(ls)` a moment ago on both Linux and AIX 7.2--the latter fails in ksh93t+.

Bash uses /dev/fd when available, but also appears to have an internal implementation which silently creates named pipes and cleans them up. In Bash 5.0.18 on AIX, fake process substitution works just fine, in my testing.

Yes, you are right. Bash 5 on AIX 7.2 works with process substitution. Thanks for the advise!

Reply on news.ycombinator.com