README.md

Cronjob and docker?

Some applications consist of multiple services. In those cases it can be useful to use a tool like docker-compose. With docker-compose you can define all your services within a single docker-compose yaml file, and add the file to your repository. This way you keep all services definition at a single place, with version control.

But what if you want to run a cronjob?

Cron on the host

One solution is to add a call to docker-compose to the crontab in the host system. In this example we would like to run a script called cleanup-old-files.py.

This can look like:

Dockerfile

ADD cleanup-old-files.py /opt/app/cleanup-old-files.py

docker-compose.yaml

services:
  cron-service:
    image: some-fancy-image
    command: python3 /opt/app/cleanup-old-files.py

crontab -l

@daily cd /some/directory && docker-compose up cron-service

On of the disadvantages of this approach is the dependency of the crontab of the host system. When I like to install the entire application I have to remember to add a cronjob. I also have to be careful to not accidentally start the cron-service when I (re)start other services using docker-compose.

Cron in a docker container (crontab in container)

What if the cron can run inside a docker container, defined in docker-compose.yaml? This allows me to define everything the application needs (regular services and cron jobs) in the same file, and have it in version control.

Dockerfile

# Make sure the docker image has cron installed
RUN apt update && apt install -y cron

# Add the start_cron.sh script
ADD start_cron.sh /opt/app/start_cron.sh
ADD cleanup-old-files.py /opt/app/cleanup-old-files.py

docker-compose.yaml

services:
  cron-service:
    restart: always
    image: some-fancy-image
    command: /opt/app/start_cron.sh "@daily python3 /opt/app/cleanup-old-files.py"

start_cron.sh see below The helper script manages the crontab and makes sure all output (stdout and stderr) of your task gets send to the logging of docker.

Another option: Cron in a docker container (crontab mounted from host)

Robert Klep proposed to bind-mount a directory as /etc/cron.d/ into the container. This saves the effort of passing the crontab via command line and writing it within the container. How will this look like?

docker-compose.yaml

services:
  cron-service:
    image: some-fancy-image
    volumes:
      cron.d:/etc/cron.d
    command: cron && tail -F /var/log/syslog | grep CRON

cron.d/name-of-your-crontab.conf

@daily python /opt/app/cleanup-old-files.py

This solution doesn't log the output of the python script to the output of cron-service. The syslog file only contains information like Nov 2 21:15:01 srv01 CRON[2846271]: (koen) CMD (python /opt/app/cleanup-old-files.py).

To solve this a solution is to have every command in the crontab log to syslog using logger.

docker-compose.yaml

services:
  cron-service:
    image: some-fancy-image
    volumes:
      cron.d:/etc/cron.d
    command: cron && tail -F /var/log/cronjob.log

cron.d/name-of-your-crontab.conf

@daily python /opt/app/cleanup-old-files.py 2>&1 | logger -f /var/log/cronjob.log

Some minor disadvantages are:

1. Every crontab configuration requires 2>&1 | logger -f /var/log/cronjob.log to be appended to every command
2. Log rotation needs to be added in your container to prevent the logs growing indefinitely
3. The configuration for the jobs (which image, limits, mounts, command, etc) are split over the docker-compose file and the crontab file

start_cron.sh

#!/bin/bash

# Stop when something fails
set -e

if [ $# -eq 0 ]
  then
    echo "usage: $0 <crontab entry>"
    exit 1
fi

# Create a log fifo so tail can wait for it
if [ -f /var/log/cronjob.log ]; then
  echo "Removing old log file"
  rm /var/log/cronjob.log
fi
echo "Creating log fifo"
mkfifo /var/log/cronjob.log

# Add the crontab entry
# First add all the environment variables. Otherwise it might not be able to find certain python packages.
# Alternative method is to use /etc/environment
cat <<EOT | crontab
`env`

$1 2>&1 | tee -a /var/log/cronjob.log
EOT

echo "Current crontab entries"
crontab -l


# Stop cron if running
if [ -f /var/run/crond.pid ]; then
  kill `cat /var/run/crond.pid`
  rm /var/run/crond.pid
fi

# Start cron daemon in background
echo "Starting cron deamon"
cron

# Watch the log file
echo "Tailing the log file"
# Use the "-f" flag to make it wait for new data.
tail -f /var/log/cronjob.log