Use docker-compose on TrueNAS SCALE 22.12 (Bluefin) without Kubernetes

readme.md

⚠️ In October 2023, TrueNAS SCALE Cobia will be released. At that time, ix-systems is making the switch to containerd and Docker will be removed. While it may remain possible to enable apt and install docker, I have no plans to do so here.

However, there is an alternative way of running docker, based on systemd-nspawn now available for testing!

enable-docker.sh

#!/usr/bin/env bash

#
# Enable docker and docker-compose on TrueNAS SCALE (no Kubernetes)
#
# This script is a hack! Use it at your own risk!!
# Using this script to enable Docker is NOT SUPPORTED by ix-systems!
# You CANNOT use SCALE Apps while using this script!
#
# 1  Create a dedicated Docker dataset in one of your zpools
# 2  Save this script somewhere else on your zpool, not in the Docker dataset
# 3  Edit line 20 of the script, set a path to the Docker dataset you created
# 4  You can now start Docker by running the script from the SCALE console
#
# For these changes to persist after SCALE reboots and upgrades, run the script at start-up
#   Schedule this script to run via System Settings -> Advanced -> Init/Shutdown Scripts
#   Click Add -> Type: Script and choose this script -> When: choose to run as Post Init

## Set a path to your docker dataset
docker_dataset='/mnt/tank/docker'


## HEREDOC: docker/daemon.json
read -r -d '' JSON << END_JSON
{
  "data-root": "${docker_dataset}",
  "storage-driver": "overlay2",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
}
END_JSON

## path to docker daemon file
docker_daemon='/etc/docker/daemon.json'

if [ ${EUID} -ne 0 ]; then
  echo "Please run this script as root or using sudo"
elif [ "$(systemctl is-enabled k3s)" == "enabled" ]; then
  echo "You can not use this script while k3s is enabled"
elif [ "$(systemctl is-active k3s)" == "active" ]; then
  echo "You can not use this script while k3s is active"
elif ! which docker &> /dev/null; then
  echo "Docker executable not found"
elif ! chmod +x /usr/bin/docker-compose &> /dev/null; then
  echo "Failed to make docker-compose executable"
elif ! install -d -m 755 -- /etc/docker &> /dev/null; then
  echo "Failed to install directory: /etc/docker"
elif ! zfs list "${docker_dataset}" &> /dev/null; then
  echo "Dataset not found: ${docker_dataset}"
else
  echo "Checking file: ${docker_daemon}"
  if test "${JSON}" != "$(cat ${docker_daemon} 2> /dev/null)"; then
    echo "Updating file: ${docker_daemon}"
    jq -n "${JSON}" > ${docker_daemon}
    if [ "$(systemctl is-active docker)" == "active" ]; then
      echo "Restarting Docker"
      systemctl restart docker
    elif [ "$(systemctl is-enabled docker)" != "enabled" ]; then
      echo "Enable and starting Docker"
      systemctl enable --now docker
    fi
  fi
fi

Considering they will remove Docker in the next update of Scale, it won't be their solution, but a regular Docker install on a Debian server.

Considering they will remove Docker in the next update of Scale, it won't be their solution, but a regular Docker install on a Debian server.

They are removing Docker now!?

Considering they will remove Docker in the next update of Scale, it won't be their solution, but a regular Docker install on a Debian server.

They are removing Docker now!?

Yeah, it was discussed here, in 23.10 they will remove Docker from TrueNAS, but my script should bring it back on the next reboot right after the update. (Still not tested, of course…) They'd give us more of a headache if they removed apt, but that would still be solvable. This will always be a Linux server with a network connection, so almost everything is possible.

IMO the best way to run docker, if you don't want to run it inside a 'jail' with jailmaker, is not to enable the package manager and install docker. Instead, download and run the binaries. You can set it up in a portable way so your docker install will be preserved on upgrades. And you minimize the interaction with the TrueNAS host (not enabling and installing from the package manager with possibly unintended side effects). I think this way it should even be possible to get it to run alongside TrueNAS Apps. If someone would create a script for this I recommend to take a look at what I'm doing with ubernerd. It uses nerdctl (which is a mostly docker compatible tool to run containers) which has the additional benefit of being able to run 'jails'. This docker install script could be inspiring too. It can install the latest docker binaries, but it's missing the part to make the installation contained to a single directory (to keep it preserved after upgrade and limit the amount of interaction with the host rootfs).