Understanding SSH and exit codes in Bash scripts.

ssh-exit-codes.sh

#!/bin/bash
#
# SSH Exit Codes
#
# Using SSH in scripting is pretty standard, but sometimes you want to stop execution of a script
# if a command inside an SSH session fails to exit cleanly (return 0).  The key to remember is that
# the ssh command's exit code will be that of the *last executed* command inside the ssh session, just
# like a bash script ends with the exit code of the last command executed unless you specifically
# call exit.
#
# See the examples below for various ways of handling ssh connections running multiple commands
# and how that interacts with it's exit value and the usage of "set -e" in the shell.

REMOTE_SERVER="insert-test-server-here.example.org"

echo "Testing exit codes from HEREDOC ssh connections with a set -e"

ssh $REMOTE_SERVER << EOF
  set -e
  someNonExistantCommand
  hostname
EOF

echo "Return of SSH Connection: $?"
# This version returns non-zero because of failed command, and never runs hostname
# The set -e allows this to happen.


###############################################################################
echo
echo "Testing exit codes from HEREDOC ssh connections without a set -e"

ssh $REMOTE_SERVER << EOF
  someNonExistantCommand
  hostname
EOF

echo "Return of SSH Connection: $?"
# This returns zero and runs hostname
# Without the set -e designated in the ssh shell's environment, all commands
# will happily run.  Since hostname shouldn't fail, it will return a 0



###############################################################################
echo
echo "Testing exit codes from ssh connections with a set -e using quotations and semicolons"

ssh $REMOTE_SERVER "
  set -e ;
  someNonExistantCommand ;
  hostname
"

echo "Return of SSH Connection: $?"
# This version returns non-zero because of failed command, and never runs hostname
# The set -e stops execution at the bad command.



###############################################################################
echo
echo "Testing exit codes from ssh connections without a set -e using quotations and semicolons"

ssh $REMOTE_SERVER "
  someNonExistantCommand ;
  hostname
"

echo "Return of SSH Connection: $?"
# This returns zero and runs hostname
# Similar to the HEREDOC version, the shell continues to run commands after a failed one.
# The semicolon doesn't stop additional execution from happening.



###############################################################################
echo
echo "Testing exit codes from ssh connections with a set -e using quotations and ampersands"

ssh $REMOTE_SERVER "
  set -e &&
  someNonExistantCommand &&
  hostname
"

echo "Return of SSH Connection: $?"
# This version returns non-zero because of failed command, and never runs hostname
# Again, the set -e here stops our execution before ever running hostname.



###############################################################################
echo
echo "Testing exit codes from ssh connections without a set -e using quotations and ampersands"

ssh $REMOTE_SERVER "
  someNonExistantCommand &&
  hostname
"

echo "Return of SSH Connection: $?"
# This version returns non-zero because of failed command, and never runs hostname
# Here, it's actually the && that stops our execution.  This boolean operator will
# only continue if the previous command executes cleanly, so when the bad command
# exits with a non-zero return code, it stops execution and hostname never runs.

The key here is to remember that the ssh exit code will be the same as the last executed command in the session. Using set -e, and using && and || operators effect what that last executed command will be.