Skip to content

Instantly share code, notes, and snippets.

@sarnobat
Last active June 20, 2024 18:13
Show Gist options
  • Save sarnobat/49b85f98d0979084bff1acaf44fbf085 to your computer and use it in GitHub Desktop.
Save sarnobat/49b85f98d0979084bff1acaf44fbf085 to your computer and use it in GitHub Desktop.
Obscure Shell Syntax

(draft 1) Obscure Shell Syntax

  • Like irregular verbs in a foreign language, the most common primitives are optimized for brevity rather than consistency, so often are illogical and idiomatic (you have to "just memorize" them to get the benefit).
  • --I'm organizing this by symbol rather than concept since there is a universal closed set of characters, but concepts have subjective implicit meaning (like key bindings)

$

  • $'' - verbatim string, no interpretting (useful if you have filenames with non ASCII characters)

    • $"" - verbatim string, no interpretting (except locale)
cat "1    Winner" | awk -F \$'\t' '{print $1,$2}'
  • You could think of $ as a dereferencing operation, like C's * for variables.

  • $_ - last argument of last command

mkdir `date -I` && cd $_
  • ${@}

  • ${1}

  • There are others that are less useful, but are good for perspective in understanding

    • $?
    • $$ - PID of newly created subshell (useful as a random number)
    • ${1:?"missing arg"}
    • ${TERM+"term is set"} - return value when variable is set
    • ${TERM-"term is not set"} - return value when variable is NOT set
    • echo "${PWD/$HOME/~}" (if you want to use variables in the substitution, the expression must be in double quotes - somewhat obviously)
      • to match from the beginning of the string, add a hash: "${PWD/#HOME/~}"
    • ${PWD:3:5} - characters from positions 3 to 5
    • ${PWD#*/} - remove everything BEFORE the FIRST slash
    • ${PWD##*/} - remove everything BEFORE the LAST slash
    • (doesn't work)${PWD%*/} - remove everything AFTER the FIRST slash
    • (doesn't work)${PWD%%*/} - remove everything AFTER THE LAST SLASH
    • "${!TERM*}" - only in bash, not zsh (indirect expansion)
    • Read about "parameter substitution" for more

!

  • You could think of ! as escaping the shell into a meta-shell that accesses the command history

    • For more, read about "word designators" and "event designators"
  • !:0

  • !:1

  • !:2

  • !:2-4

  • !:2-

  • !$ - last argument of last command (do not confuse with $!). Shorthand for !!:$

  • !^ - first arg of previous command, excluding the command (same as !:1)

  • !ls - last command beginning with ls . Zsh supports tab expansion. Can be useful when a server shell doesn't have history search configured with a key binding.

  • !! - same as !:-1

  • !? - last command that resulted in an error

  • !* all args in previous line, but not the 0th
  • !:-5:2 - 2nd arg of 5 commands ago

  • !!:gs/foo/bar/ - greedy substitution

  • !fi:2 2nd arg of the last command starting with fi

%

  • %1

``

  • `some_command` is syntactic sugar for eval "some_command"
  • Inside double quotes, it will get executed and the result inserted into the string. Inside single quotes, the command itself will remain in the string. (e.g. `cat ~/.sshpass`)

()

  • (some_command) is syntacitc sugar foir sh -c "some_command".
    • You can use - inside the brackets to read stdin.
    • (cd /tmp/; zip web.xml my.jar) | (cd /tmp/; unzip -l my.jar)
      • I'm not sure if we can do something similar using xargs, or whether we have to resort to a pipe to a while loop
    • It's a bit like javascript promises actually
    • useful also for suppressing output of a command you want to run in the background using &
  • Creates a subshell (variables created will not be valid outside it)
  • find | (while read LINE; basename "$LINE"; dirname "$LINE")
    • you can change the delimeter using read -d
  • $(some command) can be thought of as an anonymous variable that you assign the output of a command to. It is not a dash standard
  • <(some command) can be thought of as an anonymous file that a command writes to, and outside you can read from (not dash compatible). e.g. diff
  • >(some command) can be thought of as an anonymous file that a command reads from, and outside you can write to (not dash compatible). Useful writing to multiple files e.g. 2>(), 1>(), but is otherwise the same as | (some command)

{}

For commands:

  • Groups commands for associativity only (no subshell). Useful for printing an error message and exiting if a precondition fails. Binding precedence of && and || is identical in shell, not like in OOP.
  • semicolon is required for the last (or only) command
  • spaces are required after opening and before closing
test -f /tmp/in.txt || { echo "not found" && exit 1; }

test -f

For variables (tldp parameter substitution explains it well):

  • $1
  • read stdin if no args given
while read line; do; echo $line; done < "${1:-/dev/stdin}"
  • ${FILE:=}- return default value and assign
  • ${FILE:-/tmp/in.txt} - return d``efault value, but do not assign
    • it will work without the : . See paramter substitution for exact difference.
echo ${username-`whoami`}
echo $PATH:gs/\/home\/sarnobat/~/
echo ${PATH/#$HOME/~}

[]

:

^

  • ^foo^bar^ quick substitution (!!:s^foo^bar^) - only substitutes the first occurrence

>

  • &>

  • 2> - send to stderr

<<

  • Multiple heredocs
cat <<AAA <<BBB
a
AAA
b
BBB

<<<

  • Zsh only

&

  • I wonder if we can use this with xargs
  • &1 - shorthand for /dev/stdout
  • |& - shorthand for 2>&1 |

\

  • at the end of a line, you can think of it as escaping the newline that follows it (that your editor renders graphically but in memory is just another character)
  • use builtin command, not alias with the same name

test

  • test -z is empty string

  • test $1 = 'build'

  • test -e exists

  • test -f is file

  • test -d is dir

  • Good for failing fast

  • You can use if statements instead of test ... && , but that encourages memory sharing

    • [[...]] && is (also a shorthand) but I find that harder to understand

set

  • set

  • set -e fail fast (same effect as using && between every statement)

exit

  • failure: exit 1 (do not confuse with -1 in OOP)

decl

read

trap

select

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment