Skip to content

Instantly share code, notes, and snippets.

@grahamhelton

grahamhelton/cool bash tricks.md

Created June 23, 2024 18:55
Show Gist options
  • Save grahamhelton/35399893fba899031fba2aa291ee7c9c to your computer and use it in GitHub Desktop.
Save grahamhelton/35399893fba899031fba2aa291ee7c9c to your computer and use it in GitHub Desktop.
Download ZIP
Quick notes from looking at bash scripts that are way better than mine
Raw
cool bash tricks.md

Learning from well written bash

These are just some quick notes I jotted down when looking at https://github.com/WoozyMasta/kube-dump/blob/master/kube-dump, theres lots of cool little tricks within the initilization of the script.

Notes

#!/usr/bin/env bash
  • Running /usr/bin/env bash looks for the default version of the program in your environment. This way if bash is not in the default /bin/bash location (such as on [[nixos]]), it will still find bash.
    • Interestingly, there is a small security concern. If someone can get their exectuable for bash in your path it might be executed
set -e
  • set -e: Stops the execution of the script if a command or pipline exits with a non 0 exit code.
# Messages
log () {
  printf '%s [%s] %s\n' "$(date '+%Y/%m/%d %H:%M:%S')" "$1" "${@:2}"
}
msg-start () {
  [ "$silent" == 'true' ] && return; if [ -t 1 ]; then
    printf '\e[1;33m%-15s\e[m%-30s%s\n' 'Processing' "$1" "${@:2}"
  else log INFO "Processing dump $*"; fi
}
msg-end () {
  [ "$silent" == 'true' ] && return; if [ -t 1 ]; then
    printf '\e[1A\e[1;32m%-15s\e[m%-30s%s\n' 'Success' "$1" "${@:2}"
  else log INFO "Successfully dumped $*"; fi
}
msg-fail () {
  [ "$silent" == 'true' ] && return; if [ -t 1 ]; then
    printf '\e[1A\e[1;31m%-15s\e[m%-30s%s\n' 'Fail' "$1" "${@:2}"
  else log WARNING "Failed dump $*"; fi
}
success () {
  [ "$silent" == 'true' ] && return; if [ -t 1 ]; then
    printf '%s \e[1;36m%s\e[m %s\n' "$1" "$2" "${@:3}"
  else log INFO "$*"; fi
  score=$((score+1))
}
heading () {
  [ "$silent" == 'true' ] && return; if [ -t 1 ]; then
  printf '%s \e[1;34m%s\e[m %s\n%-15s%-30s%s\n' \
         "$1" "$2" 'started' 'STATE' 'RESOURCE' 'NAME'
  else log INFO "$*"; fi
}
warn () {
  if [ -t 1 ]; then
    >&2 printf '\e[1;31m%-10s\e[m%s\n' 'Warning:' "$*"
  else log WARNING "$*"; fi
}
fail () {
  if [ -t 1 ]; then
    >&2 printf '\n\e[1;31m%-10s\e[m%s\n' 'Error:' "$*"; exit 1
  else log ERROR "$*"; exit 1; fi
}
  • This is an interesting way to print text to the screen. Why not use varaibles for color codes?
# Check command is exist
require () {
  for command in "$@"; do
    if ! [ -x "$(command -v "$command")" ]; then
      fail "'$command' util not found, please install it first"
    fi
  done
}
  • This is a great way of checking for dependencies. Check each dependency by passing a list to the require function
# Usage message
usage () {
  cat <<-EOF
Dump kubernetes cluster resources

Usage:
  ${0##*/} [command] [[flags]]

Available Commands:
  all, dump                     Dump all kubernetes resources
  ns,  dump-namespaces          Dump namespaced kubernetes resources
  cls, dump-cluster             Dump cluster wide kubernetes resources

The command can also be passed through the environment variable MODE.
All flags presented below have a similar variable in uppercase, with underscores
For example:
  --destination-dir == DESTINATION_DIR

Flags:
  -h, --help                    This help
  -s, --silent                  Execute silently, suppress all stdout messages
  -d, --destination-dir         Path to dir for store dumps, default ./data
  -f, --force-remove            Delete resources in data directory before launch
      --detailed                Do not remove detailed state specific fields
      --output-by-type          Organize output into directories by resource type
      --flat                    Organize all resources of the same type in the same file

Kubernetes flags:
  -n, --namespaces              List of kubernetes namespaces
  -r, --namespaced-resources    List of namespaced resources
  -k, --cluster-resources       List of cluster resources
      --kube-config             Path to kubeconfig file
      --kube-context            The name of the kubeconfig context to use
      --kube-cluster            The name of the kubeconfig cluster to use
      --kube-insecure-tls       Skip check server's certificate for validity

Git commit flags:
  -c, --git-commit              Commit changes
  -p, --git-push                Commit changes and push to origin
  -b, --git-branch              Branch name
      --git-commit-user         Commit author username
      --git-commit-email        Commit author email
      --git-remote-name         Remote repo name, defualt is origin
      --git-remote-url          Remote repo URL

Archivate flags:
  -a, --archivate               Create archive of data dir
      --archive-rotate-days     Rotate archives older than N days
      --archive-type            Archive type xz, gz or bz2, default is tar

Example of use:
  ${0##*/} dump-namespaces -n default,dev -d /mnt/dump -spa --archive-type gz

Report bugs at:
  https://github.com/WoozyMasta/kube-dump/issues
  <woozymasta@gmail.com>

EOF
  exit 0
}
  • Using a heredoc to print the usage
# Set common vars
working_dir="$(pwd)"
timestamp="$(date '+%Y.%m.%d_%H-%M')"

# Read vars from env
# shellcheck disable=SC1090,SC1091
[ -f "$working_dir/.env" ] && . "$working_dir/.env"
  • -f "$working_dir/.env" checks that the .env file is a regular file, essentially checking if the file exists
  • If it does exist, it reads the contents of the .env file by sourcing it and any variables inside of .env will be used in the script
# Parse args commands
if [[ "${1:-$MODE}" =~ ^(dump|all|dump-namespaces|ns|dump-cluster|cls)$ ]]; then
  mode="${1:-$MODE}"; else usage; fi
  • This snippet checks if the provided argument is one of the valid otpions listed.
  • "{1:-$MODE}" expands the first argument passed to the command line ($1). If no argument is provided, it uses the value of the variable $MODE
  • =~ is the matching operator used for regular expressions
  • ^(dump|all|dump-namespaces|ns|dump-cluster|cls)$ is regex that checks if the argument matches any of the following options: dump, all,dump-namespaces,ns,dump-cluster,cls
  • mode="${1:-$MODE}"; else usage; fi: Assigns the value of the first argument to MODE else, prints the usage
# Parse args flags
args=$(
  getopt \
    -l "namespaces:,namespaced-resources:,cluster-resources:" \
    -l "kube-config:,kube-context:,kube-cluster:,kube-insecure-tls" \
    -l "help,silent,destination:,force-remove,detailed,output-by-type,flat" \
    -l "git-commit,git-push,git-branch:,git-commit-user:,git-commit-email:" \
    -l "git-remote-name:,git-remote-url:" \
    -l "archivate,archive-rotate-days:,archive-type:" \
    -o "n:,r:,k:,h,s,d:,f,c,p,b:,a" -- "${@:2}"
)
  • Stores the command line arugments passed to the $args variable
  • Long options are defined using the -l flag followed by a comma separated list of options
    • Each option cna have an optiona argument if it's followed by a :
    • Options with no arugments don't have a :
    • Long options can be grouped together with commas
  • Short options are defined using the -o flag
    • The -- "${@:2} is interesting.
    • -- Seperates the list of otpions from the remainting arguments passed to the script
    • ${@:2} expands all arguments passed to the script starting from the third artument. The first two arguments passed to the script are likely the script name and first option (the mode) which is processed previously
eval set -- "$args"
  • eval is needed otherwise set would interpret this in odd ways depending on the contents of $args
while [ $# -ge 1 ]; do
  case "$1" in
# Resources
    -n|--namespaces)            namespaces+="$2,";                shift; shift;;
    -r|--namespaced-resources)  namespaced_resources+="$2,";      shift; shift;;
    -k|--cluster-resources)     cluster_resources+="$2,";         shift; shift;;
# Kubectl opts
       --kube-config)           kube_config="$2";                 shift; shift;;
       --kube-context)          kube_context="$2";                shift; shift;;
       --kube-cluster)          kube_cluster="$2";                shift; shift;;
       --kube-insecure-tls)     kube_insecure_tls='true';                shift;;
# Common opts
    -h|--help)                  usage;;
    -s|--silent)                silent='true';                           shift;;
    -d|--destination-dir)       destination_dir="$2";             shift; shift;;
       --detailed)              detailed='true';                         shift;;
       --output-by-type)        output_by_type='true';                   shift;;
       --flat)                  output_flat='true';                      shift;;
# Dump opts
    -f|--force-remove)          force_remove='true';                     shift;;
# Commit opts
    -c|--git-commit)            git_commit='true';                       shift;;
    -p|--git-push)              git_push='true';                         shift;;
    -b|--git-branch)            git_branch="$2";                  shift; shift;;
       --git-commit-user)       git_commit_user="$2";             shift; shift;;
       --git-commit-email)      git_commit_email="$2";            shift; shift;;
       --git-remote-name)       git_remote_name="$2";             shift; shift;;
       --git-remote-url)        git_remote_url="$2";              shift; shift;;
# Archivate opts
    -a|--archivate)             archivate='true';                        shift;;
       --archive-rotate-days)   archive_rotate="$2";              shift; shift;;
       --archive-type)          archive_type="$2";                shift; shift;;
# Final
       --)                                                        shift; break;;
       -*)                      fail "invalid option $1";;
  esac
done
  • The case statement starts with a pattern matching the option (long or short)
    • If the option is mateched, append the argument value (with a comma) to the corresponding varaible. IE: namespaces+="$2,"
    • Other options set variables to true or false such as --kube-insecure-tls
    • -- Exits the loop when encountered
    • -*) Catches any unknown options that start with a - and then calls the fail function
    • Shift is used to "shift" the arguments off the array of arguments
if [[ -n "$*" && "$OSTYPE" != "darwin"* ]]; then
  fail "extra arguments $*"
fi

# Set vars
: "${silent:=$SILENT}"
: "${detailed:=$DETAILED}"
: "${output_by_type:=$OUTPUT_BY_TYPE}"
: "${output_flat:=$FLAT}"
: "${kube_config:=$KUBE_CONFIG}"
: "${kube_context:=$KUBE_CONTEXT}"
: "${kube_cluster:=$KUBE_CLUSTER}"
: "${kube_insecure_tls:=$KUBE_INSECURE_TLS}"
: "${git_commit:=$GIT_COMMIT}"
: "${git_branch:=$GIT_BRANCH}"
: "${git_commit_user:=$GIT_COMMIT_USER}"
: "${git_commit_email:=$GIT_COMMIT_EMAIL}"
: "${git_remote_name:=$GIT_REMOTE_NAME}"
: "${git_remote_url:=$GIT_REMOTE_URL}"
: "${git_push:=$GIT_PUSH}"
: "${archivate:=$ARCHIVATE}"
: "${archive_rotate:=$ARCHIVE_ROTATE}"
: "${archive_type:=$ARCHIVE_TYPE}"
  • This allows the script to leverage environment variables from the previously sourced .env file for configuration while providing defaults within the script itself.
  • The : forces the parameter expansion to happen even if the varaible is empty
  • The expansion checks if silent has a value. If not, it assigns the default value of "true"
    • If theres an environment varaible named $SILENT with a value, that value becomes the default for $silent. If $SILENT doesn't exist or is empty, an empty string is assigned to $silent
# Check dependency
require kubectl jq yq
[ "$git_commit" == 'true' ] && \
require git
[ "$archivate" == 'true' ] && [ "$archive_type" == 'xz' ] && \
require tar xz
[ "$archivate" == 'true' ] && [ "$archive_type" == 'gzip' ] && \
require tar gzip
[ "$archivate" == 'true' ] && [ "$archive_type" == 'bzip2' ] && \
require tar bzip2
  • calls the require function for the three necessary tools kubectl, jq, and yq. Then if other arguments are supplied with other dependencies, it checks for those dependencies by calling the require function
# Kubectl args
[ -n "$kube_config" ] && k_args+=("--kubeconfig=$kube_config")
[ -n "$kube_context" ] && k_args+=("--context=$kube_context")
[ -n "$kube_cluster" ] && k_args+=("--cluster=$kube_cluster")
[ "$kube_insecure_tls" == 'true' ] && \
  k_args+=("--insecure-skip-tls-verify=true")
  • If the $kube_config, $kube_context, $kube_cluster, and $kube_insecure_tls arguments are set, append to the k_args varaible
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment