Backup a docker-compose project, including all images, named and unnamed volumes, container filesystems, config, logs, and databases.

docker-compose-backup.sh

#!/usr/bin/env bash

### Bash Environment Setup
# http://redsymbol.net/articles/unofficial-bash-strict-mode/
# https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html
# set -o xtrace
set -o errexit
set -o errtrace
set -o nounset
set -o pipefail
IFS=$'\n'

# Fully backup a docker-compose project, including all images, named and unnamed volumes, container filesystems, config, logs, and databases. 
project_dir="${1:-$PWD}"
if [ -f "$project_dir/docker-compose.yml" ]; then
    echo "[i] Found docker-compose config at $project_dir/docker-compose.yml"
else
    echo "[X] Could not find a docker-compose.yml file in $project_dir"
    exit 1
fi

project_name=$(basename "$project_dir")
backup_time=$(date +"%Y-%m-%d_%H-%M")
backup_dir="$project_dir/data/backups/$backup_time"

# Source any needed environment variables
[ -f "$project_dir/docker-compose.env" ] && source "$project_dir/docker-compose.env"
[ -f "$project_dir/.env" ] && source "$project_dir/.env"


echo "[+] Backing up $project_name project to $backup_dir"
mkdir -p "$backup_dir"

echo "    - Saving docker-compose.yml config"
cp "$project_dir/docker-compose.yml" "$backup_dir/docker-compose.yml"


# Optional: run a command inside the contianer to dump your application's state/database to a stable file
echo "    - Saving application state to ./dumps"
mkdir -p "$backup_dir/dumps"
# your database/stateful service export commands to run inside docker go here, e.g.
#   docker-compose exec postgres env PGPASSWORD="$POSTGRES_PASSWORD" pg_dump -U "$POSTGRES_USER" "$POSTGRES_DB" | gzip -9 > "$backup_dir/dumps/$POSTGRES_DB.sql.gz"
#   docker-compose exec redis redis-cli SAVE
#   docker-compose exec redis cat /data/dump.rdb | gzip -9 > "$backup_dir/dumps/redis.rdb.gz"

# Optional: pause the containers before backing up to ensure consistency
# docker-compose pause

for service_name in $(docker-compose config --services); do
    image_id=$(docker-compose images -q "$service_name")
    image_name=$(docker image inspect --format '{{json .RepoTags}}' "$image_id" | jq -r '.[0]')
    container_id=$(docker-compose ps -q "$service_name")

    service_dir="$backup_dir/$service_name"
    echo "[*] Backing up ${project_name}__${service_name} to ./$service_name..."
    mkdir -p "$service_dir"
    
    # save image
    echo "    - Saving $image_name image to ./$service_name/image.tar"
    docker save --output "$service_dir/image.tar" "$image_id"
    
    if [[ -z "$container_id" ]]; then
        echo "    - Warning: $service_name has no container yet."
        echo "         (has it been started at least once?)"
        continue
    fi

    # save config
    echo "    - Saving container config to ./$service_name/config.json"
    docker inspect "$container_id" > "$service_dir/config.json"

    # save logs
    echo "    - Saving stdout/stderr logs to ./$service_name/docker.{out,err}"
    docker logs "$container_id" > "$service_dir/docker.out" 2> "$service_dir/docker.err"

    # save data volumes
    mkdir -p "$service_dir/volumes"
    for source in $(docker inspect -f '{{range .Mounts}}{{println .Source}}{{end}}' "$container_id"); do
        volume_dir="$service_dir/volumes$source"
        echo "    - Saving $source volume to ./$service_name/volumes$source"
        mkdir -p $(dirname "$volume_dir")
        cp -a -r "$source" "$volume_dir"
    done

    # save container filesystem
    echo "    - Saving container filesystem to ./$service_name/container.tar"
    docker export --output "$service_dir/container.tar" "$container_id"

    # save entire container root dir
    echo "    - Saving container root to $service_dir/root"
    cp -a -r "/var/lib/docker/containers/$container_id" "$service_dir/root"
done

echo "[*] Compressing backup folder to $backup_dir.tar.gz"
tar -zcf "$backup_dir.tar.gz" --totals "$backup_dir" && rm -Rf "$backup_dir"

echo "[√] Finished Backing up $project_name to $backup_dir.tar.gz."

# Resume the containers if paused above
# docker-compose unpause

This is great! Does an equally convenient way to restore containers with their associated volumes exist?

Not equally convenient, I've only manually browsed the output so far to restore individual files or volumes a few times.

A safety net at line 33.5
[[ -z "$container_id" ]] && continue
Or, even echo a warning about a service that isn't currently running.

Fixed @JesseChisholm, thanks

How would one go about restoring one of the backups created by this convenient script?

How much do you need restored? @jslettengren if you only need the volume data it's trivial, just copy the stuff out of the volume data folder into a new dir and point your new container at it. If you need the image too then you'll have to load the image from the tar file docker load < some_backed_up_image.tar. If you need the whole container filesystem or root dir then you'll have to manually do some copy pasting in the running container to get it working. There's no one command solution because it depends on exactly what you need. Restoring too much could be potentially harmful or more complicated than needed, so just restore what you need.

Thanks a lot @pirate

Right now I have no real need, other than I just want to know what to to do when the need arises :-)

To use the script, I run sudo bash docker-compose-backup.sh. I had to change in one of the cp commands from:
cp "$project_dir/docker-compose.yml" > "$backup_dir/docker-compose.yml"
to
cp "$project_dir/docker-compose.yml" "$backup_dir/docker-compose.yml"

Why was the > in there in the first place?

Awsome script, could you edit it so you can add environment variables for containers in the docker compose file so you can specify if you want to make backups of the attached volumes or not? For Example when backing-up Plex i dont want to backup the volume that contains my music and movies.

Thanks a lot @pirate
Would you give us a migration or restore example ??

@ppkliu https://gist.github.com/pirate/265e19a8a768a48cf12834ec87fb0eed#gistcomment-3525603