Skip to content

Instantly share code, notes, and snippets.

@dimo414

dimo414/_README.md

Last active Apr 3, 2021
Embed
What would you like to do?
Bash array expansion patterns for use with -u

Expanding Bash arrays safely with set -u

Prior to Bash 4.4 set -u treated empty arrays as "unset", and terminates the process. There are a number of possible workarounds using array parameter expansion, however almost all of them fail in certain Bash versions.

This gist is a supplement to this StackOverflow post.

tl;dr

The only safe option for expanding an array across Bash versions under set -u is:

${array[@]+"${array[@]}"}

or:

${@+"$@"}

Notice the quotes are inside the expansion, not surrounding the whole expression, and that it uses +, not :+.

Alternatives

See results.txt for a detailed breakdown across several versions of Bash. Some of these alternatives are "obviously" wrong due to incorrect quoting, but they're included for completeness.

shorthand $@ example ${arr[@]} example broken in notes
":+" "${@:+"$@"}" "${arr[@]:+"${arr[@]}"}" 4.2+
":+ "${@:+$@}" "${arr[@]:+${arr[@]}}" 4.2+
"+" "${@+"$@"}" "${arr[@]+"${arr[@]}"}" 4.2
"+ "${@+$@}" "${arr[@]+${arr[@]}}" 4.2
:+" ${@:+"$@"} ${arr[@]:+"${arr[@]}"} *
:+ ${@:+$@} ${arr[@]:+${arr[@]}} *
+" ${@+"$@"} ${arr[@]+"${arr[@]}"} N/A works in all tested versions (> 3.0)
+ ${@+$@} ${arr[@]+${arr[@]}} *
":0/1 "${@:1}" "${arr[@]:0}" 4.2 crashes, presumably a regression
:0/1 ${@:1} ${arr[@]:0} * also crashes in 4.2
":- "${@:-}" "${arr[@]:-}" *
:- ${@:-} ${arr[@]:-} *
"- "${@-}" "${arr[@]-}" *
- ${@-} ${arr[@]-} *

If we exclude v4.2 several other expansions do work, including "${@+"$@"}" and "${@:1}", but so long you intend to support that version these expansions are not safe.

Reproduction

To reproduce the contents of results.txt run:

for v in 3.1 3.2 4.0 4.1 4.2 4.3 4.4 5.0; do
  docker run -v "$PWD:/mnt" "bash:$v" bash /mnt/expansions.sh
done

Bash 3.0 is intentionally excluded from the reported results, but you can run the script against that version too to see what breaks (hint: it's a lot).

#!/bin/bash
echo "Bash Version: $BASH_VERSION"
expansion() {
local incantation=$1 expected=$2 actual
shift 2
actual=$(
set -u
eval "$(printf 'copy=( %s ); printf "${#copy[@]}"' "$incantation")" 2>/dev/null
) || { printf '\t\e[1;33m%s\e[0m' "!"; return 0; }
if (( actual == expected )); then
printf '\t\e[32m%s\e[0m' ""
else
printf '\t\e[31m%s\e[0m' ""
fi
}
test_expansion() {
local arr=("$@")
printf '$#:%s' "$#"
for incantation in \
'"${@:+"$@"}"' '"${@:+$@}"' '"${@+"$@"}"' '"${@+$@}"' '${@:+"$@"}' '${@:+$@}' \
'${@+"$@"}' '${@+$@}' '"${@:1}"' '${@:1}'
do
expansion "$incantation" "$#" "$@"
done
printf '\n'
printf '#arr:%s' "${#arr[@]}"
for incantation in \
'"${arr[@]:+"${arr[@]}"}"' '"${arr[@]:+${arr[@]}}"' '"${arr[@]+"${arr[@]}"}"' \
'"${arr[@]+${arr[@]}}"' '${arr[@]:+"${arr[@]}"}' '${arr[@]:+${arr[@]}}' \
'${arr[@]+"${arr[@]}"}' '${arr[@]+${arr[@]}}' '"${arr[@]:0}"' '${arr[@]:0}'
do
expansion "$incantation" "$#"
done
printf '\n'
}
printf '\t%s' '":+"' '":+' '"+"' '"+' ':+"' ':+' '+"' '+' '":0/1' ':0/1'
printf '\n'
test_expansion
test_expansion ''
test_expansion '' ''
test_expansion a 'b c'
Bash Version: 3.1.23(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 3.2.57(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.0.44(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.1.17(2)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.2.53(2)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ! ! ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.3.48(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ! ! ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.4.23(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 5.0.17(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
@dimo414

This comment has been minimized.

Copy link
Owner Author

@dimo414 dimo414 commented May 1, 2020

If the eval-magic feels a bit too handwavy for you, an earlier version of the script doesn't use it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment