Use the unofficial Bash strict mode | Hacker News Reader

Use the unofficial Bash strict mode

redsymbol.net

96 pointsredsymbol36 comments

Better than set -e is trapping ERR. You can set up a trap that prints out the command that failed, and the line it's on, rather than just dying quietly like set -e does. It's much easier to debug that than trying to work ahead from the last command with output. For bonus points, when you have a bunch of scripts together, put the trap code in its own file and source it in all the other scripts (rather than duplicating code).

It's also amazing that you can trap SIGTERM to perform clean-up tasks when the user Ctrl-Cs a script.

That would be SIGINT - SIGTERM is sent when you kill the process without specifying a specific signal to send, or when the system is shutting down.

Example usage with backtrace:

    set -o errtrace
    trap 'err_handler $?' ERR

    err_handler() {
      trap - ERR
      let i=0 exit_status=$1
      echo "Aborting on error $exit_status:"
      echo "--------------------"
      while caller $i; do ((i++)); done
      exit $?
    }

    The set -e option instructs bash to immediately exit if
    any command has a non-zero exit status. You wouldn't
    want to set this for your command-line shell,

...but, it's a great addition to your buddy's `.bashrc`. For maximum effectiveness, be physically present, perhaps with a camera. :)

Oooh, this is my new favorite. Previously it was "echo exit >> ~/.bashrc" ;)

Place this entire line @ the bottom of your target's .bashrc

    echo 'sleep .1' >> ~/.bashrc

By week 2 my poor friend had conditioned himself to work within 1 session since spawning new sessions became unbearably slow :)

The disappointment with set -e is that it does not work everywhere:

- with an unset 'hello' variable, try:

  set -eu
  echo "`echo $hello` world"
  echo "huh still going?!"
  foo="`echo $hello` world"
  echo "not reaching this as expected"

Fun, now you have to manually write your whole program in what's basically SSA (static single assignment) form.

- It's deactivated in if/then context, which at first makes sense, but then when you try to force it on explicitely like:

  set -eu
  if (set -eu; false; true); then
      echo "huh why still true??"
  else
      echo false
  fi

and your declaration is just ignored.. you begin to wonder how many places set -e misses really.

I don't think that setting IFS this way is a good idea. Of your variables do happen to contain tabs and newlines, you still get unwanted word expansion. Much better is to just use double quoted expansion primitives that always expand one element to each word: "${foo[@]}"

Agreed. The IFS trick is shortly followed by an explanation, which includes a bunch of anti-patterns, such as bare ${foo[@]} and $@ usage.

Rule of thumb: quote all your variables.

http://mywiki.wooledge.org/Quotes

That's the common answer, and while I practice that myself, I have to disagree with it as guidance. Maybe you and I are fastidious enough to always remember to quote all our variables, but many are not - I know I was writing shell scripts for a couple of years before I realized its importance, and it's very common that even experienced engineers don't know or care to do it. If someone writing or editing the code forgets to quote the variable that allows subtle bugs to sneak in.

It's unfortunate the semantics of bash don't have variable references behave like they are quoted by default. I really wish it did.

While setting IFS=$"\t\n" may make problems less likely when strings contain spaces, it's still not correct. File names (and other strings) can contain tabs and newlines, too. That's relatively rarer, but the quoting approach always works.

Hm, is that true? When I run this script:

  #!/bin/bash
  items=(
      'a'
      'b c'
      "d\te"
      "f\ng"
  )
  
  echo "Unquoted:"
  for item in ${items[@]}; do
      echo -e ".  $item"
  done
  
  echo "Quoted:"
  for item in "${items[@]}"; do
      echo -e ".  $item"
  done
  
  set -euo pipefail
  IFS=$'\n\t'
  echo "Unquoted strict mode:"
  for item in ${items[@]}; do
      echo -e ".  $item"
  done

... I get this output:

  Unquoted:
  .  a
  .  b
  .  c
  .  d    e
  .  f
  g
  Quoted:
  .  a
  .  b c
  .  d    e
  .  f
  g
  Unquoted strict mode:
  .  a
  .  b c
  .  d    e
  .  f
  g

Note the output for "Quoted" and "Unquoted strict mode" are identical.

(GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu))

If you can remember to use $@ rather than $*, you should be able to remember to use "$@". The only reason $@ exists is that it behaves differently when enclosed in double quotes.

Unquoted variables are an attack vector. Plain and simple. The IFS tip, as described in the original article, does nothing to remedy it.

https://www.owasp.org/index.php/Command_Injection

Can you give an example of an exploit in bash? The link you give is excellent information, but doesn't have any shell script examples.

My first thought: It seems like only $@, $* and variables read from the environment could be possible attack vectors. But since shell scripts can't be setuid, it's hard for me to immediately imagine how unquoting could enable a new exploit.

EDIT: the most dangerous example I can think of right now is a script like this, where a webapp invokes the script and passes a GET param as an argument:

  #!/bin/bash
  exec $1

But if I execute this like so:

  ./badscript.sh 'some-benign-command ; cat /etc/passwd'

... then passwd is not exposed, because the tokens ";", "cat" and "/etc/passwd" are passed into the argv of some-benign-command.

(And if "cat /etc/password" itself can be passed as $1, then quoting won't help...)

Aside from the comments above, I would add "shopt -s shift_verbose" which enables "overshifting" detection.

E.g. when a function receives less arguments than necessary and you retrieve them like this:

    function f()
    {
        declare -r x="$1";  shift
        declare -r y="$1";  shift
        declare -r z="$1";  shift
    }

with less than three arguments passed and shift_verbose on you will get an error message the moment you shift and with "set -e" in addition, execution will be aborted.

See https://www.gnu.org/software/bash/manual/html_node/The-Shopt...

Also, "set -o noclobber" might be useful. If you try to redirect to an existing file with ">", it will fail. If you explicitly want to overwrite the file without triggering the error, use ">|".

See https://www.gnu.org/software/bash/manual/html_node/Redirecti...

When using `set -eu`, how do you check the number of arguments?:

    if [ -z "$1" ]; then
      usage();
      exit 1;
    fi;

If I'm using `set -eu`, that dies on the `$1` with a nasty error message, rather than printing my usage message. I've resorted to moving `set -eu` to after these kind of checks, but that makes me uneasy.

Use the `$#` variable which contains the number of arguments.

You can use default values to avoid the nasty error. ${1:-} will expand to an empty string if your script is called without arguments.

In case it matters, michaelmior and Splognosticus's solutions are both POSIX shell compliant, so they should work anywhere. (eg, busybox, dash, tcsh, zsh)

Good info for writing bash scripts that are large enough to need debugging, but why would one do that in the first place? Python or Perl would be better choices at that point.

Portability without dependencies.

Until you get bit by versioning quirks in bash.

I'm still trying to recall which I got hit with (it's been a few years), but IIRC there were a few at both the 3.x and 4.x transition points.

It's happened more than once that I've started something in bash, thinking it's a small enough job, only to see it growing in time - a lot. Sure, in theory you could rewrite it in Python, but you don't always have that luxury in practice.

Bottom line is - keep bash scripts small, but be prepared to deal with the cases when they grow like Jack's magic beanstalk.

Bash is not just Bash but also sed, awk and all the other goodies. You can grow Bash scripts from the command line. You don’t have to worry about things, everything is a string is a number is a stream. One could go so far as to argue that shell pipes behave quite distinctly functional-programming-ly(?).

Plus it’s some sort of middle ground between “BDFL tells you to put a space there“ and “one day, archaeologists will find a script written in Perl, awk and csh and it will become the new millenium’s Rosetta stone“.

You sometimes have to use the tools you have available.

When you are fundamentally running shell commands, using python/perl really don't make a lot of sense. Use the best tool for the job. Note that for my dayjob I write python almost fulltime, but if I'm almost exclusively running shell commands, I'll write a shell script. Just because you can do something one way doesn't necessarily mean I should.

See I'm of the opinion that if you need arrays and associative arrays, bash is the wrong tool for the job. If you have a recent bash it has both of those, just seems wrong in such a clunky language with awful scoping.

I write a lot of bash scripts, but lately I've been doing most of my shell interop in Ruby instead. Backticks are pretty good and I can munge things in easier ways than bash or Python. And with `ruby -n`, the script is invoked line-by-line--perfect for processing piped content.

Use the best tool for the job. If you don't know bash/bourne shell well, use ruby/perl/python/etc. I started out as a sysadmin years ago and know bourne shell/bash very very well. It is all about what works best for the problem.

Sure. And don't get me wrong, I write a lot of bash scripts. =) I think any logic beyond string replacement is probably edging out of where it's a good idea, if only because other people then have to read my stuff later, but it's totally fine for that. I'm saying more that I think Ruby (or Perl) make more sense than Python given the tools it provides.

The problem with the `-e` flag is if you want to output custom errors, say json. I prefer to have bash inspect status codes.

    if [ $? != 0 ]; then
        echo "{\"error\": \"Failed to connect to the database.\"}" >&2;
        exit 1;
    fi

Just do it all in the condition instead of separately:

    if ! command_that_may_error; then
        echo "{\"error\": \"Failed to connect to the database.\"}" >&2;
        exit 1;
    fi

Regarding setting $IFS:

  for arg in $@; do

A better way to do it is to quote it:

  for arg in "$@"; do

because then you can capture newlines and tabs. Bash will automatically convert it to separate parameters, even though there is only one quoted variable.

See footnote 2. http://redsymbol.net/articles/unofficial-bash-strict-mode/#f...

I wonder why not to set IFS='' then. I haven't used this myself in production, but quick testing seems to do what I expected: make $foo behave like "$foo".

Reply on news.ycombinator.com