Unofficial bash strict mode

unofficial-bash-strict-mode.md

Unofficial Bash Strict Mode

Sometimes a programming language has a "strict mode" to restrict unsafe constructs. E.g., Perl has use strict, Javascript has "use strict", and Visual Basic has Option Strict. But what about bash? Well, bash doesn't have a strict mode as such, but it does have an unofficial strict mode:

set -euo pipefail

set -e

Setting the -e, a.k.a., errexit, option, causes the script to exit immediately when an unhandled error occurs, instead of just continuing on.

set -u

Setting the -u option causes the script to treat references to unset variables as errors. E.g., if you misspell $MASTER_LINKS as $MASTERLINKS, which is unset, this will be treated as an error.

set -o pipefail

Setting the pipefail option causes the shell to treat an error in any command of a multi-stage pipeline to be an error in the pipeline as a whole. This is in contrast to the surprising default behavior of only considering the exit status of the last command in the pipeline.

Further explanation

set -e

Ways to set and unset this option:

set -e                # Set the errexit option
set -o errexit        # Equivalent
shopt -s -o errexit   # Equivalent (TMTOWTDI FTW!)

set +e                # Unset the errexit option
set +o errexit        # Equivalent
shopt -u -o errexit   # Equivalent

When set, the errexit option causes the script to exit immediately when an unhandled error occurs. See the bash reference manual for details and exceptions:

https://www.gnu.org/software/bash/manual/bash.html#index-set

Example:

# remove the temporary files
cd "$JOB_HOME/job001/tmp"
rm *

If the cd command failed, you wouldn't want to proceed with the execution of the rm command. The errexit option helps protect against such blunders.

Without errexit, you'd need to be vigilant in checking the exit status or each command:

die() {
  echo "$@" >&2
  exit 1
}
...
# remove the temporary files
cd "$JOB_HOME/job001/tmp" || die "Couldn't cd into $JOB_HOME/job001/tmp"
rm *

Criticism of errexit

• http://mywiki.wooledge.org/BashFAQ/105
• https://lists.gnu.org/archive/html/bug-bash/2012-12/msg00093.html

Rebuttal of criticism of errexit

It is true that the use of set -e involves some adjustments to your shell scripting style, but the safety it affords is well worth it.

set -u

Setting the -u option causes the script to treat references to unset variables as errors.

Consider this example:

# Create or update the hardlink
ln -f proc.mk "$MASTERLINKS"/

In this example, we accidentally mispelled the $MASTER_LINKS environment variable as $MASTERLINKS, which is unset.

Without the -u option set, the script will attempt to create the hard link at the root of the filesystem, /.

With the -u option, the script's reference to the unset variable $MASTERLINKS will be treated as an error, and no attempt to create the hard link at the wrong location will be made.

set -o pipefail

By default, bash has the POSIX-mandated behavior of only considering the last command in a pipeline when determining the exit status of the pipeline as a whole. Setting the pipefail option will cause the script to have the less surprising behavior of considering an error in any stage of the pipeline to be an error of the pipeline as a whole.

Consider this example:

$ cat myscript.sh
#!/bin/bash
false | echo 'hi'
echo "$?"
set -o pipefail
false | echo 'hi'
echo "$?"

$ ./myscript.sh
hi
0
hi
1

You see that when the pipefail option is set, the non-zero exit status of the first stage of the pipeline causes the pipeline as a whole to have that same non-zero exit status. Without the pipefail option set, the behavior is surprising: the pipeline as a whole has a zero exit status; the error is silently ignored!

Inspired by

• Use the Unofficial Bash Strict Mode (Unless You Looove Debugging)
  http://redsymbol.net/articles/unofficial-bash-strict-mode/

but without the stuff about changing IFS.

Discussion of that blog post: https://news.ycombinator.com/item?id=8054440

See also

• ShellCheck
  https://www.shellcheck.net

ShellCheck helps find unsafe constructs in your shell script.

Be careful with pipefail, as some constructs are built with the idea the failure (non-zero) value of one element in the pipeline might be expected, and indeed even handled in some way. Bash provides a volatile indexed variable PIPESTATUS to check for the exit code of each member of the pipeline if you need that methodology.

A simple example of this is when using the find command as non-root and ignoring "Permission denied" messages, like so:

$ find /etc -name hosts 2>/dev/null | xargs grep -c 127
/etc/avahi/hosts:0
/etc/hosts:2

$ echo $?
0

vs.

$ set -o pipefail
$ find /etc -name hosts 2>/dev/null | xargs grep -c 127
/etc/avahi/hosts:0
/etc/hosts:2

$ echo $?
1

This is a silly example we could code a better way, but it illustrates where the expectation is to ignore the first member of the pipe STDERR and not really care if it fails, what we're after is the grep output of some sort (which we then might extend this pipeline to feed into something else, again it's a silly example).

An example of using PIPESTATUS in this regard would be a neat logging function that runs a command, logs it, then returns the expected output of that pipeline:

# run action, log output, return exit code
# - passing in 'sed' should be avoided
# - functions can only return 0..254
# -- set a global to check as needed
ACTLOG=/var/log/foobar.log
_ACTRET=0
function logact() {
  local ACTION
  ACTION="$*"
  ${ACTION} 2>&1 | tee -a "${ACTLOG}"
  _ACTRET=${PIPESTATUS[0]}
  return ${_ACTRET}
}

In the above, we don't care if tee fails (let's pretend the $ACTLOG is non-writable - bad, but not fatal to script operational needs) we only care about the first member of the pipeline, not the entire thing as we're returning that to the caller in the above example. Setting a global pipefail can have unexpected results (which can be coded around if needed, not saying there isn't a solution), just depends what your script code is doing and intending as a result.

The silent behavior of errexit means user might not realize at all the script stopped in the middle.

https://disconnected.systems/blog/another-bash-strict-mode/ suggests handling the virtual ERR trap instead of setting errexit — this approach allows printing an error message and providing more details e.g. exact line where the script failed.
I love that because

@robin-a-meade Great gist! Maybe add shopt -s inherit_errexit to?

shopt -s inherit_errexit
    # shopt              => This builtin allows you to change additional shell optional behavior.
    # -s inherit_errexit => If set, command substitution inherits the value of the errexit option, instead of unsetting it in the subshell environment.