Comment by chubot | Hacker News Reader

Comment by chubot | Hacker News Reader

I'm working on fixing errexit / set -e in Oil:

https://github.com/oilshell/oil/issues/709

Summary of problems:

1. The "if myfunc" problem -- error checking is skipped. This is specified by POSIX, but it's undesirable.

2. The f() { test -d /tmp && echo "exists" } problem. The exit code of the function is probably not what you expect!

3. the local x=$(false) problem. This happens in all shells because of the definition of $?. (local is a shell builtin with its own exit code, not part of the language) This one is already fixed with shopt -s more_errexit in Oil.

4. the x=$(false) problem. This is a bash problem with command substitution. For example, dash and zsh don't have this problem. Test case:

    bash -c 'set -e; x=$(false); echo should not get here'

Newer versions of bash fix it with inherit_errexit. Oil also implements inherit_errexit and turns it on by default! (in Oil, not OSH)

-----

So 1 and 2 relate to the confusing combination of shell functions and errexit.

And 3 and 4 relate to command subs. Oil has fixed these with opt-in OSH shell options (on by default in Oil), but not 1 and 2.

If you know of any other problems, please chime in on the bug!

> 2. The f() { test -d /tmp && echo "exists" } problem. The exit code of the function is probably not what you expect!

The exit code is 0 (assuming /tmp/, stdout, /bin/test and /bin/echo are all working correctly; with /tmp1 it's 1), as expected; is this referencing a bug in sh and/or bash that I've fixed locally and then forgotten about?

(Also, I'm pretty sure it should be:

  f() { test -d /tmp && echo "exists"; }

unless the parse error for missing ';' was your point (I haven't bothered to fix that one, but maybe Oil has).)

Yeah I didn't explain that very well. It's actually a variation on the problem here:

https://news.ycombinator.com/item?id=24738274

except with a shell function rather than a subshell.

Demo:

https://github.com/oilshell/blog-code/blob/master/errexit /demo.sh#L21

Run ./demo.sh subshell-compare and func-compare.

I have seen multiple people point it out as confusing. The problem is that in the first case, there are two exit codes:

1. false (suppressed on LHS of &&)

2. true

And in the second case, there are 3:

1. false (suppressed on LHS of &&)

2. true

3. the exit code of the function/subshell, which is the exit code of the last statement, which is nonzero. This then CAUSES THE PROGRAM TO FAIL, whereas it didn't before.

(edit: removed incorrect code)

-----

So it's kind of like a "ghost" exit code created by the subshell/function! Which has an unexpected interaction with errexit.

Does that make sense?

I am not sure exactly how to fix it, but it will probably involve limiting && to stuff like this:

    if test -d /tmp && test -d /tmp/foo; then echo yes; fi

And disallow it when standing alone, because it doesn't make much sense there, if errexit is on.

Feel free to chime in bug with any possible solutions or more problems.

----

(And yes Oil fixes the brace problem with shopt -s parse_brace, which is on by default in Oil! Try it out and let me know if you like it :) )

> Yeah I didn't explain that very well.

Not at all well, I'm afraid, but I think I understand:

  f(){ false && true; } ; set -e ; f ; echo hi
  # this works correctly (f returns false and errexit triggers)
  
  set -e; false && true ; echo hi
  # despite `false && true` failing, this doesn't trigger errexit

I was confused by the implication that the function version was what was wrong (rather than "false && true is broken, and you'd expect the function to work the same way"). Probably partly because I was confusing it with `command || true`, which is a idiomatic way to suppress errexit-like mechanisms by making the exit status always 0.

Well, which one is broken depends on your viewpoint, and the exact use case.

Another way to look at it is the difference between exercise 4 and 5 here:

https://mywiki.wooledge.org/BashFAQ/105

So taking that concrete example

    test -d nosuchdir && echo "warning: no directory";

instead of

    false && true

You didn't want the function to fail. You just wanted to print a warning if the directory doesn't exist. But the whole function fails, while it doesn't for the if statement.

The whole thing is inherently confusing ... The language confuses success/failure and true/false. Both of those are stuffed into an exit code.

test -d is really for true/false, but then functions also have success/failure (did the last command succeed).

So there's no way for shell to really know what you want.

I think Oil might end up with something like "if catch foo" for success/fail, and "if boolean grep" for true/false.

> You just wanted to print a warning if the directory doesn't exist.

That code prints a warning if the directory does exist, actually ("-d FILE FILE exists and is a directory"). Did you mean something like:

  test -e outfile && echo "trying to overwrite preexisting outfile"

> You didn't want the function to fail.

Uh, I kind of do, actually? If I wanted to discard the exit status I'd write:

  test -e outfile && echo "trying to overwrite preexisting outfile" || true;

This is awkward, and suggests a proper if-then operator (perl-6-style `??` collides with globbing, but maybe `&?`?), but it's better then having something like:

  mkfoo -o foo && cp -T foo otherfoo
  do-something foo bar

mostly-silently mix old and new data together because mkfoo failed and the error code was swallowed.

> So there's no way for shell to really know what you want.

Yep, and if we have to pick one problem, things failing completely in a obvious and also easily-fixable way is usually a much less awful problem to have than silently corrupting data and state, especially when you've opted-in to a mechanism (like errexit) specifically designed to do the former.

Reply on news.ycombinator.com