Advanced Shell Scripting Techniques: Automating Complex Tasks with Bash

    if test 1 -gt 2
    then echo 123
    fi

    if test 1 -gt 2
    then
        echo 123
    fi

I've seldom had to worry about the performance of my scripts. It almost always is dominated by the performance of whatever the script is running. I did have one very notable exception, though. It's kind of embarrassing.

On one project---for "reasons"---I had written several bash scripts to collect system performance metrics to push into Nagios or Graphite. A couple of these had to calculate some floating point, and I was invoking `dc` to do that, as bash only supports integer arithmetic. The script would loop once every 5 seconds or so, iirc, invoking dc each time. One small process every 5 seconds doesn't sound like a lot, but the systems also ran ClamAV---again, for "reasons"---and were already under high load, and that extra process invocation had a noticeable and significant impact on the system.

My workaround was to run a single instance of dc using bash's `coproc` keyword, then redirect all the calculations through the returned descriptors. This had a drastic improvement.

Except... it turns out that dc leaked 80 bytes of memory whenever it read and parsed a float. So, the long-running dc process would eventually get oom-killed. So, I added a check to restart the dc process every 5000 iterations, or something like that.

Which all sounds like a big headache, but for "reasons" this was far easier than getting some other solution into that environment.

I did submit a bug report (2020 Feb 19), and Ken Pizzini (dc's original author, I think) was kind enough to reply even though he wasn't the maintainer. Unfortunately, my patch never got upstreamed.

Then again, who else is crazy enough to use dc as long-running math server?

test 1 -> 2.70s true -> 2.55s : -> 2.25s [[ 1 ]] -> 2.10s

Like, I don't even have a way to measure the overhead of just the for loop as [[ is faster than that.

But we can do something and see the cost of that: let's do a -z $a when a=. We don't need to quote the $a with [[, but I'll do it both ways:

test -z "$a" -> 3.25s [[ -z "$a" ]] -> 2.65s [[ -z $a ]] -> 2.10s

So, sure: it isn't a performance difference that is likely to kill a use case, but it does exist. I also agree that it isn't necessarily worth bothering with... but I just can't get behind something -- using test -- that is both slower and more complex... like, to me, it isn't [[ that has "functionality" that I might not need, it is test: with test I can programmatically decide operators by putting them in variables, while with [[ everything must be syntax. This is an incredible amount of extra power that you get with test that I frankly don't just not need... I actively would prefer not having except in a scant handful of scenarios. It is much easier to glance at a use of [[ and know 100% for sure what it is doing than with test, where you must carefully consider all of the quoting rules, and even then the behavior might be dynamic.

However I continue to see shell scripts from what I believe are probably competent scripters, i.e., scripts in tarballs of software from authors I trust, and they almost always seem to prefer using "[". Wondering if there is a reason or it is purely style.

1. Bash is too big and slow for scripting. The bash author admits it is "too big and slow" in the manpage. The world's most popular Linux distribution does not use bash as its scripting shell; it uses something smaller and faster derived from NetBSD Almquist shell. For scripting, there are more efficient shells for scripting than bash.

NB. Nothing against bash. As an experiment I have been using busybox bash as an interactive shell. Normally I use NetBSD ash. I like the way busybox bash does not keep duplicates in history and its command search is quick. But like its full-sized counterpart it is too slow for scripting. Using busybox to do scripting is so slow that it is bascially useless for anything but the smallest scripts.

This reads like someone's personal cheat sheet more than an informative post.

There are some good suggestions but I'd really like to know why the author has these opinions.

Try a small benchmark to see the results of /usr/bin/printf vs printf. It really makes a difference (not invoking an external program is a huge performance win).

The author is indeed confused though. `test` for example, is also a builtin. You can have an empty PATH= and still invoke printf, test, echo and some few others.

The list is weird.

Sounds very bash-ish.

https://omid.dev/posts/

They are good at generating code with some stilted prose around it

Also yeah the cadence and number and topics of the posts makes me suspicious -- I don't a theme or purpose, just a bunch of "content"

Rather than typing `useradd foo`, you can type `user.sh add foo`! So much easier?

    # df -h | grep "$partition" | awk '{print $5}' | sed 's/%//'
    24
    # df -h | grep -Po "\d+(?=%\s+$partition)"
    #

I think the first one is much easier to read.

Purpose built binaries are extremely useful and should replace a ton of script usage, however, scripts will always have their place.