Elegant Bash Conditionals
timvisee.com
timvisee.com
There will be people - seasoned professional developers, even - reading these languages that don't know the first thing about them!
`[` is one of those things that are second nature for seasoned bash people but that are utterly ungoogleable by a bash beginner. Beginners don't know `[` is an executable, and it would never occur to them to `man [`. It can become quite the rabbit hole to figure out that there's also `[[` (which is NOT an executable), that `[ a > b ]` is very very different from `[[ a > b ]]`, etc.
So do these people a favor and put the `if` there so they at least have some hope of stumbling upon a stack overflow post talking about some of this stuff.
It is usually also a symlink to the 'test' executable but not in all Unix-like systems.
https://www.gnu.org/software/bash/manual/html_node/Bourne-Sh...
They also exist as executables for ancient backwards compatibility.
"[[" is a so-called "conditional construct":
https://www.gnu.org/software/bash/manual/html_node/Condition...
I have a somewhat quirky style and almost always use "if test ..." since I find it clearer than "[" and usually don't need the additional functionality that "[[" provides.
(For similar reasons, whenever I'm grepping for a nontrivial regex, I always use egrep rather than grep, because the escaping semantics are simpler—"if it's escaped with a \, then it's not special"—and again I do not want to have the information of "which characters are special only when escaped in base grep" in my brain.)
Tangential but relevant to the Edit thread today were vis was mentioned, “No more {very-,}{no,}magic modes with different semantics and escaping rules. Instead we use the familiar and predictable POSIX Extended Regular Expression variant.”¹ is a real advantage over vim.
¹https://github.com/martanne/vis/wiki/Differences-from-Vi(m)#...
$ type -a [
[ is a shell builtin
[ is /usr/bin/[
$ type -a [[
[[ is a shell keywordBut never in scripts used in production that must be maintained by a group.
That rule served me well for many years. YMMV
https://www.gnu.org/software/bash/manual/bash.html#Bash-Cond...
>>if [ -r ~/.profile ]; then
>> source ~/.profile
>>fi
>>We can simplify this using control operators:
>>[ -r ~/.profile ] && . ~/.profile
I wonder which version I would prefer for readability three months down the road.
Like so:
# Download the binary
wget --quiet --output-document /usr/local/bin/mybinary "$download_url" || {
error 'Failed downloading the CLI'
exit 1
}
debug "Making CLI executable"
# Make it executable
chmod +x /usr/local/bin/mybinary || {
error 'Failed making CLI executable'
exit 1
}
This has been completely transformative for me and has made writing maintainable + debuggable scripts so much easier. if ! wget --quiet --output-document /usr/local/bin/mybinary "$download_url"; then
error 'Failed downloading the CLI'
exit 1
fi
?I know someone who wrote most of his Bash that way, using `cmd && { ...; }` instead of `if cmd; then ...; fi` and `cmd || { ...; }` instead of `if ! cmd; then ...; fi`. I never thought it was particularly clear or maintainable--that it was more on the "clever" side than on the "clear" side.
Sure, it's a little clunky to have an `if` for everything that might fail, but uh... these days I write Go for a living.
if err := thingThatMightFail(); err != nil {
...
}
so I was saying that of course I think lots of `if` statements for error handling are reasonable. (Hmm, thinking back, I learned Go in 2012, and my Bash probably peaked in 2013, I wonder if having learned Go made me more amenable to this in Bash.)I think a single-line `cmd || die "msg"` like in Perl or Ruby is handy and fine. Or maybe even a two-line
some command that might fail ||
die "Some message that is long enough that it wants to be its own line"
But once it starts getting to have multiple statements that you're grouping with `{ }`, just use an `if` statement. If I were reviewing someone's Ruby that had multiple lines after the "or" in "or die", I'd tell them to just use unless thingThatMightFail
...
endThe bash "or" syntax reads very clearly. Do X or do Y. Do X or fail like so. It's very English-like and intuitive. And understanding why it works is just a matter of understanding logical operator short circuiting, which you have to understand to work in C-like languages anyway.
if condition ; then
command1
command2
else
command3
command4
fi
versus: condition && (command1; command2) || (command3; command4)
GNU Bash has a fix for that: non-forking braces. #!/usr/bin/env bash
set -xeuo pipefael
-x to print the commands
-e to exit on error
-u to error out if there are unbound variables
-o pipefail to exit if a command that's not the last in a pipeline fails
No need to || {}.Only word of warning here is that -x can print out secrets if you're not careful.
thingThatCanFail() {
echo "step one succeeded"
echo "step two failed"
false
echo "step three was run too"
}
if ! thingThatCanFail; then
echo "thingThatCanFail failed!"
fi
With or without `set -e`, step three is run, and the function returns success, even though you might expect the failure of step two to prevent step three from running.If "thingThatCanFail" is called _outside_ of an if statement, then `set -e` causes different behavior (i.e. step three _is_ skipped).
I instead use lots of chaining with && (as in the article), or explicit checks after each command. I have two utility functions I define in nearly every script:
warn() { >&2 printf "%s\\n" "$*"; }
abort() { warn "$@"; exit 1; }
Then I do lots of: stepOne || abort "step one failed"
stepTwo || abort "step two failed"
...
It can get a little verbose, but much better than trying to reason about `set -e` in my opinion. commandThatMayFail || true
in order to continue the script if some optional step fails.Bonus note: The notation "do something else die 'with message'" stems from Perl, AFAIK. See for instance https://perldoc.perl.org/Carp
die() { echo "$@" >&2; exit 1; }
wget --quiet --output-document /usr/local/bin/mybinary "$download_url" ||
die 'Failed downloading the CLI'
chmod +x /usr/local/bin/mybinary ||
die 'Failed making CLI executable'
If you need more complex cleanup, that doesn't work as well... but (a) hopefully you can avoid that cleanup and just use a plain "|| die", and (b) if you need cleanup, hopefully you can put that into one function (possibly "die" itself) and have its cleanup work no matter where it's called from, and still run everything as "|| cleanup_and_die" or whatever you'd call it.in the beginning of your scripts put:
set -e # stop script if error set -u # stop with undefined var set -o pipefail # stop with pipe fail
Now your scripts will have a sane default behavior
# panic STRING-FORMAT [...]
#
# write printf-formatted message to stderr, prepending
# script name, and exit failure
panic() {
printf "%s: %.0s${1}\n" "${0##*/}" "$@" >&2
exit 1
}
Prepending the script name is immensely helpful when diagnostic messages are mixed on stderr. It's also standard style for Unix utilities in general. The BSD extensions err(3) and warn(3), provided by all modern Unix libc implementations and commonly use by shell utilities, prepend the program name.Another common style guideline is to include context for the diagnostic, often prepended. For example,
wget --quiet --output-document /usr/local/bin/mybinary "$download_url" \
|| panic '%s: unable to download' "$download_url" echo Option -n invalid # OK
echo -n is not valid # Uh oh
printf '%s ' -n is not valid # Always okay
Generally you should use printf because of its consistency. For a simpler example of OP's: panic() { printf '%s\n' "${0}: ${*}"; exit 1; }
Is fine. I don't see the need to pass a format string to your function. Just call: panic "${download_url}: unable to download"The real pro tip is: don't use bash if it's not trivial to read
Plus, the POSIX specification for the shell is significantly more concise and clear than the manuals for Bash, Zsh, etc. See Shell & Utilities -> Shell Command Language at https://pubs.opengroup.org/onlinepubs/9699919799/ I know exactly where to go if I can't remember the difference between ${foo:-X} vs ${foo+Y}.
Also, don't run a command that directly overwrites a file unless you know it's going to succeed. A temp file lets you atomically replace the file only if the download succeeded. And a cleanup trap helps clean this up in the event of errors. Finally, you can die on unset variables or errors (cleanup still works) and you can enable tracing if environment variable DEBUG=1 was set.
#!/usr/bin/env sh
set -eu
[ "${DEBUG:-0}" = "1" ] && set -x
cleanup() { rm -f /usr/local/bin/mybinary.tmp ; } ; trap cleanup EXIT
panic() { printf "$0: Error: %s\n" "$1" >&2; exit 1; }
mkdir -p /usr/local/bin
[ -e /usr/local/bin/mybinary ] || \
wget -q -O /usr/local/bin/mybinary.tmp "$url" \
|| panic "wget was unable to download '$url'"
mv -f /usr/local/bin/mybinary.tmp /usr/local/bin/mybinaryhttps://mywiki.wooledge.org/BashPitfalls#cmd1_.26.26_cmd2_.7...
I love shell scripting and this highlights one of the major problems with bash: small changes like this can appear completely interchangeable with other mechanisms for doing the same thing but introduce edge cases that can unexpectedly bite you.
For example... consider what happens if "[ $foo -eq 0 ] && /bin/do-something" is the last statement within a function or the end of your script.
I have wasted many hours of my life tracking down buts where the idiom in the OP was used in a function. It leads to really hard to find bugs.
Beware!!!
So how does `if [ <condition> ]` work then? Is it some kind of special case? No! The way it works is that there's an executable named `[` in UNIX that that takes the condition expression as arguments, and returns "success" or "failure" depending on if the condition evaluates to true. It's right there in the filesystem at /bin/[
Now, some sticklers might argue that maybe not the greatest idea to spawn a process every time you need to evaluate an if statement, but I really do admire the UNIX purity of it: of COURSE that's how bash conditionals work! This is UNIX, after all! Why add an expression parser/evaluator to bash when you can just have it's own process for that? Do one thing and do it well!
You can verify this by writing a simple test script and doing (at least on Linux) "strace ./test.sh 2>&1 | grep exec" and observe no explicit call to that binary.
And there are tons of gotchas like that that made me terrified of bash. Even the simplest construct can attack you from any angle and the trial and error and distrust is everywhere. I'm getting better at it but for me and I guess many others, I just don't use bash enough to remember these quirks and the resulting experience is rather poor. Which is a shame!
The savior for me was shellcheck. Instead of needing character-perfect memory of each construct and common snippet I can run shellcheck on it and it will tell me what common pitfall I might have stumbled upon and most importantly, why it is an issue. For a beginner I think it is an absolute must.
Honestly the idea that a certain benign-looking syntax could (but maybe not!) spawn some external subprocess is terrifying to me, and I don't find it delightful at all.
Even if you `set -e`, the command in the test can fail and your script won't exit.
Parameter expansion is a great way to simplify your code, but it can be obtuse to the casual reader. Setting a default variable using DEFAULT="${DEFAULT:-myvalue}" is a bit easier to understand than just ${DEFAULT:=myvalue}.
Read the dash manual and try to stick to just those features as it's almost entirely POSIX. Most scripts do not need to rely on bash functionality. https://linux.die.net/man/1/dash
If you find it will really simplify your life to have arrays, hashes/maps/dicts, a while loop reading from a subshell's output, etc, then use Bash and use those features. Otherwise, stick to POSIX semantics. You can do almost everything you need with parameter expansion, expr and other unix tools.
With any programming language, you should try to use the least amount of syntax and functionality possible to accomplish your goal, as long as it is readable, maintainable, and does not hide tricky behavior. Being verbose is always preferable to being hard to maintain; verbose and uncomplicated things can easily be simplified later.
Every developer has read enough if statements to understand if then else at a glance. Few people write enough bash scripts to have that same instant parsing of [[ condition ]] && cmd or [[ condition ]] || cmd, especially when intermingled.
I like the idea of cmd || { raise error; } that others have mentioned, but I hate the idea of using these in place of an if statement to save 2 lines.
condition && action
condition || action
(or the equivalent for languages where the logical operators are written differently) are not bash-exclusive idioms; I've seen them in a number of expression-oriented languages (less frequently on statement-oriented languages where the particular action is a function rather than a statement, but the fact that it isn't usable consistently in those.
languages tends to make it less idiomatic there.)>The echo command always exists [sic] with 0, so this propagates to exit if the first expression is truthful. (emphasis his)
In both bash and dash (and probably most), echo will fail if there is no stdout. E.g.:
j.sh:
echo hello && exit 2
$ bash j.sh >&-
j.sh: line 1: echo: write error: Bad file descriptor
$ echo $?
1
Note the exit value ($?) is 1, not 2.EDIT: Also note that not exiting the script could cause a flurry of cascading errors (likely permission errors in the article's example).
I really don't understand what is the problem with that simple if statement to receive this kind of reaction. :shrug:
so you can do things like
[[ 1 < 2 ]] && echo true # where < is now not redirect, like to (( 1 < 2 ))
[[ "x" == 'y' || x == x ]] # note how || is not behaving the same as [ '' || 'y' ]
`man 1 test` explains the more conventional `[ ]` where the bash man page would better explain the [[ case.Here it is in pdksh:
# [ "Z" == "z" ] && echo Bob
# [ "Z" == "Z" ] && echo Bob
Bob
I chose pdksh on purpose - because _it_ also supports [[ (builtin like bash)[2], so while [[ is a "bash-ism" it's actually present in some/many other shells as well. However, the operational aspect is not identical between [ and [[, both in the bash and pdksh implementations and let's add zsh to show the same line of shell break: # [[ "Z" == "Z" && "A" == "A" ]] && echo Bob
Bob
# [ "Z" == "Z" && "A" == "A" ] && echo Bob
pdksh: [: missing ]
# [[ "Z" == "Z" && "A" == "A" ]] && echo Bob
Bob
# [ "Z" == "Z" && "A" == "A" ] && echo Bob
-bash: [: missing `]'
# [[ "Z" == "Z" && "A" == "A" ]] && echo Bob
Bob
# [ "Z" == "Z" && "A" == "A" ] && echo Bob
zsh: = not found
The [ operator requires that you use the bash/pdksh method to combine them with && "outside the brackets" in a more POSIX like use, like so: # if ([ "Z" == "Z" ] && [ "A" == "A" ]); then echo Bob; fi;
Bob
...except on zsh, because (surprise!) zsh has decided to internalize the [ command rather than use the external one (which works OK); but it also has a problem with the "=" sign being used like this[3] with the test operator. zsh requires we then handle the == sign by quoting it (or unsetting a value internally): # if [ "Z" == "Z" ] && [ "A" == "A" ]; then echo Bob; fi;
zsh: = not found
# if [ "Z" '==' "Z" ] && [ "A" '==' "A" ]; then echo Bob; fi;
Bob
This last example is a subtle point - the use of [[ is actually _more_ compatible between bash/pdksh/zsh than the use of [ due to the way zsh handles the input which is different than bash/pdksh and even old school POSIX Bourne shell (/bin/sh).This got kinda long sorry, hope it helps.
[1] there's actually a binary "[" and a binary "test" in the `coreutils` package on most systems, however I'm not sure why they have different binary file sizes to be honest
[2] https://linux.die.net/man/1/pdksh search "[["
if command_or_condition
then
branch
fi
I suppose that to many, this looks wasteful and consequently ugly, so one looks for alternatives.I always inline the `then`, which requires a semicolon:
if command_or_condition; then
branch
fi
If I was forced to use the traditional format, I'd be definitely annoyed :) if command_or_condition
then branch
more_branch
fi
Given the relationship between ; and newline, and the positions of required ;s in if/then/fi, I assumed that was intended. [ -r ~/.profile ] && . ~/.profile
> Only if the readability check expression is truthful/succeeds, we want to source the file, so we use the && operator.Almost. The real way to do this is to check for the non-existence of the file as the “success” case and do the action via a || on failure.
Otherwise if you run in a strict mode with error on any unhandled non-zero command (set -e), you’ll exit the script with failure when the profile doesn’t exist:
[[ ! -r ~/.profile ]] || . ~/.profile
Note that the if expression does not have this issue as they don’t trigger the error on exit handling. Only the && approach does.cat ~/.profile && echo This is your profile || echo Failed to read profile
I think these are commonly called short-circuits. Using these with brackets too, in order to gain greater control or more complex comparisions is super useful if not aware of it.
A simple example would be something like:
# (foo && bar) && (bish || bash) && echo 0 || (echo "not bosh" && exit 1)
I'm no bash expert, just a sysadmin with a little dangerous knowledge, but constructing things that way just feels more natural (to me, at least, it's probably 'teaching your grandmother to suck eggs' to a proper bash hacker).
Be careful with using parenthesis -- this means you're now in a subshell! That "exit 1" will NOT terminate the entire script, but instead set "$?" to 1 for the next command. Additionally, any variable or environmental changes will be lost (which may be desirable) once you're outside of that block.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
[1] https://www.cs.ait.ac.th/~on/O/oreilly/unix/ksh/appa_02.htm
-[ $EUID -ne 0 ] && { echo You must be root; exit 1 }
+[ $EUID -ne 0 ] && { echo You must be root; exit 1; }Explicit if [].. is more readable.
Chaining commands with && is better just accomplished by running `set -e` which you should be doing at the start of your script anyway.
And of course both of these are getting pretty close to the "if you have to Google what this bash syntax does, just write your script in Python" rule.
[[ -f $input_file ]] || { echo "${input_file} does not exist"; exit 1; }
So you can read the left-hand side bit as an assertion that something is true in the following part of the script, and skim over the right-hand side.I get that the authors point was more focused on the conditional usage, but this seems like a pretty bad way to check if the user is root and isn’t really any more complicated than checking EUID = 0 or some of the other methods.
I have tended to use it more recently for vertical compactness, and in python because pylint doesn't like single-line if statements but happily tolerates "x or y".
( foo && bar && baz ) || ( fallback1 && fallback2 ) || echo 'Argh, even fallback failed!'
Using parentheses is also preferred for readability.no closing `fi`, use multiline or single line, add an arbitrary number of elif statements. this only requires the short option is set in your z shell.
if (condition) { statements; } else { do other thing; }