Using Docker on TrueNAS SCALE (no Kubernetes)

boot.sh

#!/usr/bin/env bash

#
# Enable docker and docker-compose on TrueNAS SCALE (no Kubernetes)
#
# This script is a hack! Use it at your own risk!!
# Using this script to enable Docker is NOT SUPPORTED by ix-systems!
# You CANNOT use SCALE Apps while using this script!
#
# 1  Create a dedicated Docker zvol on one of your zpools: zfs create -V 100G data/_docker
# 2  Create an ext4 filesystem on your new zvol: mkfs.ext4 /dev/zvol/data/_docker
# 3  Create a directory where you can mount the ext4 filesystem: mkdir /mnt/data/_docker
# 4  Save this script somewhere else on your zpool, not in the Docker dataset
# 5  Edit line 24 and 25 of the script, set paths to the Docker directory and zvol you created
# 7  If this is NOT the first time you run this script (updating to new location/zvol):
#     - Check the contents of /etc/fstab, and remove the old # docker-zvol-ext4 rule if it's there
# 6  You can now start Docker by running the script from the SCALE console
#
# For these changes to persist after SCALE reboots and upgrades, run the script at start-up
#   Schedule this script to run via System Settings -> Advanced -> Init/Shutdown Scripts
#   Click Add -> Type: Script and choose this script -> When: choose to run as Pre Init

## set the path to your docker directory and specify the zvol
docker_directory='/mnt/data/_docker'
docker_zvol='/dev/zvol/data/_docker'

## this part will add a new fstab rule, if the fstab file doesn't contain our special docker-zvol-ext4 comment
## if this is not the first time you run this script, our special comment is already there
## so if you ever change the docker_directory and docker_zvol paths in the future, also remove the old
## fstab rule and comment from /etc/fstab
grep -q 'docker-zvol-ext4' /etc/fstab || {
    printf "# docker-zvol-ext4\n${docker_zvol} ${docker_directory} ext4 rw,relatime,stripe=4 0 0\n" >> /etc/fstab
    mount "${docker_directory}" && echo 'Mounted docker directory'
}

## HEREDOC: docker/daemon.json
read -r -d '' JSON << END_JSON
{
  "storage-driver": "overlay2",
  "data-root": "${docker_directory}",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
}
END_JSON

## path to docker daemon file
docker_daemon='/etc/docker/daemon.json'

if [ ${EUID} -ne 0 ]; then
  echo "Please run this script as root or using sudo"
elif [ "$(systemctl is-enabled k3s)" == "enabled" ]; then
  echo "You can not use this script while k3s is enabled"
elif [ "$(systemctl is-active k3s)" == "active" ]; then
  echo "You can not use this script while k3s is active"
elif ! which docker &> /dev/null; then
  echo "Docker executable not found"
elif ! chmod +x /usr/bin/docker-compose &> /dev/null; then
  echo "Failed to make docker-compose executable"
elif ! install -d -m 755 -- /etc/docker &> /dev/null; then
  echo "Failed to install directory: /etc/docker"
elif [ ! -d "${docker_directory}" ] ; then
 echo "Directory not found: ${docker_directory}"
else
  echo "Checking file: ${docker_daemon}"
  if test "${JSON}" != "$(cat ${docker_daemon} 2> /dev/null)"; then
    echo "Updating file: ${docker_daemon}"
    jq -n "${JSON}" > ${docker_daemon}
    if [ "$(systemctl is-active docker)" == "active" ]; then
      echo "Restarting Docker"
      systemctl restart docker
    elif [ "$(systemctl is-enabled docker)" != "enabled" ]; then
      echo "Enable and starting Docker"
      systemctl enable --now docker
    fi
  fi
fi

Yeah, I saw the comment I was just confused about if "Scale Apps" referred to their apps (not including the UI) or if the UI was decoupled from their app repo and pointed to native docker. I am looking to use portainer and was underwhelmed by the performance I got running through a VM) so I will proceed using this.

May I ask what your reason for changing Line 21 from Post-Init to Pre-Init?

Yeah, I saw the comment I was just confused about if "Scale Apps" referred to their apps (not including the UI) or if the UI was decoupled from their app repo and pointed to native docker. I am looking to use portainer and was underwhelmed by the performance I got running through a VM) so I will proceed using this.

May I ask what your reason for changing Line 21 from Post-Init to Pre-Init?

SCALE Apps refers to the complete apps system for TrueNAS SCALE. GUI and backend.
It's fully kubernetes backed and does not 'point' to native docker.

May I ask what your reason for changing Line 21 from Post-Init to Pre-Init?

Well spotted! Since the script mounts the ext4 filesystem if it's not already in fstab it needs to run as early as possible.

New zfs version has overlayfs support natively baked in btw :)

Yeah I just noticed! Super awesome! 😃Looking forward to using it on a stable SCALE release.

Sources indicate it also fixes about 75% of the CPU load issues on SCALE Apps as well :)

Wonderful :) would be worth revisiting the Docker Compose TrueCharts app by that time. It really is comforting to know that one exists for when this Docker hack stops working.

@Jip-Hop - I'm on RC 1 now - I only needed to add the "storage-driver": "overlay2", to the HEREDOC section of the script and everything seems to be working. I did not have to create, format, or even mount an extra zvol.

EDIT: Maybe worth noting, I started with a fresh docker-dataset after I changed the storage-driver.

Great! Thanks for testing and reporting. Didn't expect overlay2 support to be included so quickly. Then I know what I must do when SCALE 22.12.0 is released next month. Glad I can drop the ext4 workaround.😄

From a docker level this should be the only required change.
Though this also means one of the primary reasons not to go with SCALE Apps (performance issues) are also solved for a large portion as well.

I'm back to using the script by tprelog now that I've updated to Bluefin. No need for the ext4 zvol :)

⚠️ In October 2023, TrueNAS SCALE Cobia will be released. At that time, ix-systems is making the switch to containerd and Docker will be removed.

The alternative way of running docker, based on systemd-nspawn is available for testing! 😄

Please vote for this issue: Support for systemd-nspawn Linux Containers (jails for SCALE).

So we have to make our own scripts to "hack" what ix-systems has designed.

If you (or anyone) has useful scripts to enable Docker/Portainer on TrueNAS SCALE then please post them here. 😃

https://forum.level1techs.com/t/truenas-scale-native-docker-vm-access-to-host-guide/190882