Skip to content

Instantly share code, notes, and snippets.

@dimo414
Last active Jun 30, 2022
Embed
What would you like to do?
Bash array expansion patterns for use with -u

Expanding Bash arrays safely with set -u

Prior to Bash 4.4 set -u treated empty arrays as "unset", and terminates the process. There are a number of possible workarounds using array parameter expansion, however almost all of them fail in certain Bash versions.

This gist is a supplement to this StackOverflow post.

tl;dr

The only safe option for expanding an array across Bash versions under set -u is:

${array[@]+"${array[@]}"}

Notice the quotes are inside the expansion, not surrounding the whole expression, and that it uses +, not :+.

Alternatives

See results.txt for a detailed breakdown across several versions of Bash. Some of these alternatives are "obviously" wrong due to incorrect quoting, but they're included for completeness.

shorthand $@ example ${arr[@]} example broken in notes
":+" "${@:+"$@"}" "${arr[@]:+"${arr[@]}"}" 4.2+
":+ "${@:+$@}" "${arr[@]:+${arr[@]}}" 4.2+
"+" "${@+"$@"}" "${arr[@]+"${arr[@]}"}" 4.2
"+ "${@+$@}" "${arr[@]+${arr[@]}}" 4.2
:+" ${@:+"$@"} ${arr[@]:+"${arr[@]}"} *
:+ ${@:+$@} ${arr[@]:+${arr[@]}} *
+" ${@+"$@"} ${arr[@]+"${arr[@]}"} N/A works in all tested versions (> 3.0)
+ ${@+$@} ${arr[@]+${arr[@]}} *
":0/1 "${@:1}" "${arr[@]:0}" 4.2 crashes, presumably a regression
:0/1 ${@:1} ${arr[@]:0} * also crashes in 4.2
":- "${@:-}" "${arr[@]:-}" *
:- ${@:-} ${arr[@]:-} *
"- "${@-}" "${arr[@]-}" *
- ${@-} ${arr[@]-} *

If we exclude v4.2 several other expansions do work, including "${@+"$@"}" and "${@:1}", but so long you intend to support that version these expansions are not safe.

Reproduction

To reproduce the contents of results.txt run:

for v in 3.1 3.2 4.0 4.1 4.2 4.3 4.4 5.0; do
  docker run -v "$PWD:/mnt" "bash:$v" bash /mnt/expansions.sh
done

Bash 3.0 is intentionally excluded from the reported results, but you can run the script against that version too to see what breaks (hint: it's a lot).

#!/bin/bash
echo "Bash Version: $BASH_VERSION"
expansion() {
local incantation=$1 expected=$2 actual
shift 2
actual=$(
set -u
eval "$(printf 'copy=( %s ); printf "${#copy[@]}"' "$incantation")" 2>/dev/null
) || { printf '\t\e[1;33m%s\e[0m' "!"; return 0; }
if (( actual == expected )); then
printf '\t\e[32m%s\e[0m' ""
else
printf '\t\e[31m%s\e[0m' ""
fi
}
test_expansion() {
local arr=("$@")
printf '$#:%s' "$#"
for incantation in \
'"${@:+"$@"}"' '"${@:+$@}"' '"${@+"$@"}"' '"${@+$@}"' '${@:+"$@"}' '${@:+$@}' \
'${@+"$@"}' '${@+$@}' '"${@:1}"' '${@:1}'
do
expansion "$incantation" "$#" "$@"
done
printf '\n'
printf '#arr:%s' "${#arr[@]}"
for incantation in \
'"${arr[@]:+"${arr[@]}"}"' '"${arr[@]:+${arr[@]}}"' '"${arr[@]+"${arr[@]}"}"' \
'"${arr[@]+${arr[@]}}"' '${arr[@]:+"${arr[@]}"}' '${arr[@]:+${arr[@]}}' \
'${arr[@]+"${arr[@]}"}' '${arr[@]+${arr[@]}}' '"${arr[@]:0}"' '${arr[@]:0}'
do
expansion "$incantation" "$#"
done
printf '\n'
}
printf '\t%s' '":+"' '":+' '"+"' '"+' ':+"' ':+' '+"' '+' '":0/1' ':0/1'
printf '\n'
test_expansion
test_expansion ''
test_expansion '' ''
test_expansion a 'b c'
Bash Version: 3.1.23(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 3.2.57(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.0.44(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.1.17(2)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.2.53(2)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ! ! ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.3.48(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ! ! ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 4.4.23(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
Bash Version: 5.0.17(1)-release
":+" ":+ "+" "+ :+" :+ +" + ":0/1 :0/1 ":- :- "- -
$#:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
#arr:0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓
$#:1 ✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:1 ✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
$#:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
#arr:2 ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✓ ✗
@dimo414
Copy link
Author

dimo414 commented Jun 30, 2022

Ok yes I follow you now. You're right that $@ is not affected by the same set -u issue as other arrays, I believe I just included it in the table for completeness - as you've found $@ expansion sometimes behaves differently than ${arr[@]} expansion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment