Dokku setup

dokku_setup.sh

#!/usr/bin/env bash

# ================================================================================================================
# author:   Victor Miti <https://github.com/engineervix>

# NOTES:
# 1. this is not meant to be executed as a script!
#    rather, it's a documented sequence of steps to follow
#    in setting up Dokku & deploying your web applications.
# 2. this is mostly based on my experience deploying a
#    wagtail/django project, and draws from various resources on the web, including:
#    - https://www.accordbox.com/blog/how-deploy-django-project-dokku/#introduction
#    - [Setting up Dokku with DigitalOcean and Namecheap (GitHub gist)](https://gist.github.com/djmbritt/10938092)
#    - https://vitobotta.com/2022/02/16/deploying-an-app-with-dokku/
#    - https://blog.semicolonsoftware.de/securing-dokku-with-lets-encrypt-tls-certificates/
#    - https://spiffy.tech/dokku-with-lets-encrypt-behind-cloudflare
#    - [Dokku Docs](https://dokku.com/docs/)
# =================================================================================================================

# 0. First things first
#    + spin up a new server with your preferred service provider (Digital Ocean, Vultr, Hetzner, Linode, etc).
#      Make sure the server's hostname is the exact same as your domain, as this will ensure 
#      `/etc/hostname` and the `hostname` command respond correctly (something dokku relies on).
#    + Configure DNS.
#      - You'll need an A record for the naked domain (the "@" one) pointing to your IP with the lowest TTL possible
#      - You'll need a wildcard A record (use '`*`') pointing to your IP with the lowest TTL possible
#    + [bootstrap your server using this script](https://github.com/engineervix/pre-dokku-server-setup) or
#      [Jason Heec's script](https://github.com/jasonheecs/ubuntu-server-setup)

# 1. install Dokku
# update the version accordingly
wget https://raw.githubusercontent.com/dokku/dokku/v0.27.6/bootstrap.sh && \
sudo DOKKU_TAG=v0.27.6 bash bootstrap.sh

# 2. (on local machine) assuming:
#   - you named your VPS as dokku in ~/.ssh/config
#   - your SSH key is also called dokku, and is in ~/.ssh/
# cat ~/.ssh/dokku.pub | ssh dokku sudo dokku ssh-keys:add admin

# 3. run docker without sudo (Log out and log back in so that your group membership is re-evaluated)
sudo usermod -aG docker "$(whoami)"
# you can also run the following command to activate the changes to groups:
# newgrp docker

# 4. create your app
sudo dokku apps:create app-name

# 5. add a domain to your app
sudo dokku domains:add app-name app.example.com
# check domains report
sudo dokku domains:report app-name
sudo dokku domains:report --global
sudo dokku domains:report app-name

# 6. setup postgres | https://github.com/dokku/dokku-postgres
sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres
# feel free to choose the image & image version of your choice here (except for timescaledb! see below):
sudo dokku postgres:create postgres-app-name --image "postgis/postgis" --image-version "14-3.2"
sudo dokku postgres:link postgres-app-name app-name
# Note: DATABASE_URL is automagically set for you, so no need to worry about it

# If using timescaledb, you might have issues, here's a workaround/fix
# ref: https://github.com/timescale/timescaledb-docker/issues/99 and
# https://github.com/dokku/dokku-postgres/issues/153
sudo dokku postgres:create temp-db --image "postgres" --image-version "14.2"
sudo dokku postgres:create postgres-app-name --image "timescale/timescaledb" --image-version "latest-pg14"

sudo cp -v /var/lib/dokku/services/postgres/temp-db/data/server.crt /var/lib/dokku/services/postgres/postgres-app-name/data/server.crt
sudo cp -v /var/lib/dokku/services/postgres/temp-db/data/server.key /var/lib/dokku/services/postgres/postgres-app-name/data/server.key

sudo dokku postgres:info postgres-app-name
sudo dokku postgres:link postgres-app-name app-name
sudo dokku postgres:destroy temp-db

# if you're migrating your app from somewhere, import database dump:
# if you wanna import from heroku ...
# https://devcenter.heroku.com/articles/heroku-postgres-import-export#download-backup
# heroku pg:backups:capture
# heroku pg:backups:download
sudo dokku postgres:import postgres-app-name < database.dump

# set up authentication for backups on the postgres service
# Datastore backups are supported via AWS S3 and S3 compatible services
# like https://github.com/minio/minio and https://www.backblaze.com/b2/cloud-storage.html
# sudo dokku postgres:backup-auth <service> <aws-access-key-id> <aws-secret-access-key> <aws-default-region> <aws-signature-version> <endpoint-url>
sudo dokku postgres:backup-auth postgres-app-name access-key-id secret-access-key us-west-004 v4 https://s3.us-west-004.backblazeb2.com
# postgres:backup postgres-app-name <bucket-name>
sudo dokku postgres:backup postgres-app-name my-bucket
# everyday at 2:30AM, 10:30AM, 6:30 PM
sudo dokku postgres:backup-schedule postgres-app-name "30 2,10,18 * * *" my-bucket
# check: cat the contents of the configured backup cronfile for the service
sudo dokku postgres:backup-schedule-cat postgres-app-name

# 7. setup redis | https://github.com/dokku/dokku-redis
# https://tute.io/install-setup-redis-dokku
sudo dokku plugin:install https://github.com/dokku/dokku-redis.git redis
sudo dokku redis:create redis-app-name
sudo dokku redis:link redis-app-name app-name
# take note of the REDIS_URL that is automagically set and
# will be shown to you here ... you'll set this as CELERY_BROKER_URL in next step if using celery

# 8. set env variables for your app
# Note 1: if using django-compressor, set DISABLE_COLLECTSTATIC=1 and 
#  run collectstatic as a post-compile step
#  ref: https://github.com/django-compressor/django-compressor/issues/486#issuecomment-341791105
# Note 2: feel free to adjust WEB_CONCURRENCY based on the memory requirements of your processes
#  ref: https://docs.gunicorn.org/en/stable/settings.html
#  The suggested number of workers is (2*CPU)+1
sudo dokku config:set --no-restart app-name WEB_CONCURRENCY=3 && \
sudo dokku config:set --no-restart app-name DISABLE_COLLECTSTATIC=1 && \
sudo dokku config:set --no-restart app-name PYTHONHASHSEED=random && \
sudo dokku config:set --no-restart app-name DJANGO_SECRET_KEY=YOURSECRETKEY && \
sudo dokku config:set --no-restart app-name DJANGO_SETTINGS_MODULE=config.settings.production && \
sudo dokku config:set --no-restart app-name CONN_MAX_AGE=60 && \
sudo dokku config:set --no-restart app-name DEBUG=False && \
sudo dokku config:set --no-restart app-name EMAIL_RECIPIENTS='John Doe <john@example.co.zm>,somebody@someplace.com,user@example.co.zm,anotheruser@example.co.zm' && \
sudo dokku config:set --no-restart app-name DEFAULT_FROM_EMAIL='Jane Doe <jane@example.co.zm>' && \
sudo dokku config:set --no-restart app-name DJANGO_SERVER_EMAIL= && \
sudo dokku config:set --no-restart app-name DJANGO_EMAIL_SUBJECT_PREFIX= && \
sudo dokku config:set --no-restart app-name ALLOWED_HOSTS=app.example.com && \
sudo dokku config:set --no-restart app-name BASE_URL=https://app.example.com && \
sudo dokku config:set --no-restart app-name MAPBOX_ACCESS_TOKEN=YOURMAPBOXACCESSTOKEN && \
sudo dokku config:set --no-restart app-name RECAPTCHA_PUBLIC_KEY=YOURRECAPTCHAPUBLICKEY && \
sudo dokku config:set --no-restart app-name RECAPTCHA_PRIVATE_KEY=YOURRECAPTCHAPRIVATEKEY && \
sudo dokku config:set --no-restart app-name SENDGRID_API_KEY=YOUR_SENDGRIDAPIKEY && \
sudo dokku config:set --no-restart app-name SENDGRID_GENERATE_MESSAGE_ID=True && \
sudo dokku config:set --no-restart app-name SENDGRID_MERGE_FIELD_FORMAT=None && \
sudo dokku config:set --no-restart app-name DJANGO_AWS_ACCESS_KEY_ID=WHATEVERITIS && \
sudo dokku config:set --no-restart app-name DJANGO_AWS_SECRET_ACCESS_KEY=WHATEVERITIS && \
sudo dokku config:set --no-restart app-name DJANGO_AWS_STORAGE_BUCKET_NAME=app-name && \
sudo dokku config:set --no-restart app-name DJANGO_AWS_S3_REGION_NAME=eu-central-1 && \
sudo dokku config:set --no-restart app-name XYNLE_API_KEY=WHATEVERITIS && \
sudo dokku config:set --no-restart app-name XYNLE_API_USERNAME=some-username && \
sudo dokku config:set --no-restart app-name XYNLE_API_SECRET=WHATEVERITIS && \
sudo dokku config:set --no-restart app-name XYNLE_API_SENDER_ID=WHATEVERITIS && \
sudo dokku config:set --no-restart app-name VONAGE_API_KEY=something && \
sudo dokku config:set --no-restart app-name VONAGE_API_SECRET=YOURAPISECRET && \
sudo dokku config:set --no-restart app-name VONAGE_DEFAULT_FROM=WHATEVERITIS && \
sudo dokku config:set --no-restart app-name DJANGO_SENTRY_LOG_LEVEL=20 && \
sudo dokku config:set --no-restart app-name SENTRY_DSN=https://something@thing.ingest.sentry.io/anotherthing && \
sudo dokku config:set --no-restart app-name SENTRY_ENVIRONMENT=production && \
sudo dokku config:set --no-restart app-name SENTRY_TRACES_SAMPLE_RATE=0.5 && \
sudo dokku config:set --no-restart app-name REDIS_KEY_PREFIX=whatever && \
sudo dokku config:set --no-restart app-name CELERY_BROKER_URL=WHATEVER

# 9. configure buildpacks if your app has multiple buildpacks
# this is an example for a Django project which
#  - uses Node.js to build frontend assets
#  - uses PostGIS (has geospatial features)
sudo dokku buildpacks:add --index 1 app-name https://github.com/heroku/heroku-buildpack-nodejs.git
sudo dokku buildpacks:add --index 2 app-name https://github.com/heroku/heroku-geo-buildpack.git
sudo dokku buildpacks:add --index 3 app-name https://github.com/heroku/heroku-buildpack-python.git
# list buildpacks
sudo dokku buildpacks:list app-name

# 10. NGIИX
sudo ufw allow 'Nginx Full'
sudo rm -fv /etc/nginx/sites-enabled/default
sudo systemctl restart nginx
sudo dokku nginx:validate-config
sudo dokku nginx:show-config app-name
# Customize Nginx | set `client_max_body_size`, to make upload feature work better in Django project, for example
sudo dokku nginx:set app-name client-max-body-size 50m
# regenerate config
sudo dokku proxy:build-config app-name

# 11. (on local machine) add a remote branch `dokku` to your git repo, and deploy 🚀
# git remote add dokku dokku@dokku:app-name
# change if your branch is main or something else
# git push dokku master:master

# 12. SSL
# # if your domain is on cloudflare, you need to disable Cloudflare's proxying behavior on your domain
# while you get Let's Encrypt set up on Dokku (https://spiffy.tech/dokku-with-lets-encrypt-behind-cloudflare)
sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
sudo dokku config:set --no-restart --global DOKKU_LETSENCRYPT_EMAIL=your@emailaddress.com
sudo dokku letsencrypt:enable app-name
# this would setup cron job to update letsencrypt certificate
sudo dokku letsencrypt:cron-job --add

# 13. Run Dokku command without sudo
sudo visudo /etc/sudoers
# add the following (a comment plus a single line)

## Allow members of group dokku to run Dokku commands
# %dokku ALL=(ALL:ALL) NOPASSWD:SETENV: /usr/bin/dokku

# then add current user to the dokku group
sudo usermod -a -G dokku "$(whoami)"

# 14. Other issues

# 14.1 storage (Dokku uses up lots of disk space with each build)
# ref: https://github.com/dokku/dokku-mariadb/issues/38
# setup a cron job as current user (`crontab -e`) to cleanup:
# this runs every 12 hours (feel free to adjust according to your needs)
# use https://crontab.guru/ for quick and simple editing of cron schedule expressions
# 0 */12 * * * docker rm -v $(docker ps -a -q -f status=exited)

# 14.2 cron tasks for your app
# recommended way is to have an `app.json` in yur repo with the details of cron tasks
# see https://dokku.com/docs/processes/scheduled-cron-tasks/#dokku-managed-cron
# an example for Django:
# 
# {
#   "cron": [
#     {
#       "command": "python manage.py clearsessions",
#       "schedule": "@daily"
#     }
#   ]
# }

# 14.3 Add/remove domains to/from your app
# say, for example, you were testing things on a temp domain, before puchasing the actual domain
# remember: if your domain is on cloudflare, you need to disable Cloudflare's proxying behavior on your domain
# while you get Let's Encrypt set up on Dokku (https://spiffy.tech/dokku-with-lets-encrypt-behind-cloudflare)
dokku domains:add app-name myapp.com
dokku domains:remove app-name app.example.com
# for Django apps, you'd have to adjust your env variables accordingly:
# dokku config:set app-name ALLOWED_HOSTS=myapp.com && \
# dokku config:set app-name BASE_URL=https://myapp.com

# 14.4 If your app isn't using the global domain ...
# if someone accesses the global domain, there'll probably be nothing,
# since we deleted the default nginx page, which would have been otherwise shown!
# to remedy this, create a new static app, let's call it placeholder
dokku apps:create placeholder
dokku domains:add placeholder example.com
# (on machine): the bare minimum you need is a repo with these files:
#  - `index.html`  --> with your (placeholder) content
#  - `.static`     --> an empty file, see https://github.com/dokku/heroku-buildpack-nginx
# then, all you need to do is
# git remote add dokku dokku@dokku:placeholder
# git push dokku main:main (or `git push dokku master:master` or whatever branch you're using!)

# 14.5 Continuous Integration
# The Dokku project has an official GitHub Action, see 
# https://dokku.com/docs/deployment/continuous-integration/github-actions/
# For GitLab CI, see https://dokku.com/docs/deployment/continuous-integration/gitlab-ci/
# Other: https://dokku.com/docs/deployment/continuous-integration/generic/#generic-cicd-integration

# 14.6 Security Headers
# use https://securityheaders.com/ to analyze your security headers and update your app config accordingly
# for Django, here are some recommended resources:
#  - https://adamj.eu/tech/2019/04/10/how-to-score-a+-for-security-headers-on-your-django-website/
#  - https://github.com/adamchainz/django-permissions-policy
#  - https://github.com/mozilla/django-csp | https://django-csp.readthedocs.io

# 14.7 Zero Downtime Deploys
# Reference: https://dokku.com/docs/deployment/zero-downtime-deploys/
# 
# By default, Dokku will wait 10 seconds after starting each container before assuming
# it is up and proceeding with the deploy. Once this has occurred for all containers
# started by an application, traffic will be switched to point to your new containers.
# Dokku will also wait a further 60 seconds after the deploy is complete before terminating
# old containers in order to give time for long running connections to terminate.
# In either case, you may have more than one container running for a given application.
# 
# You may both create user-defined checks for web processes using a CHECKS file, as well as
# customize any and all parts of this experience using the checks plugin.

# 15. Next Steps

# 15.1 figure out how to deploy with Docker, which is particularly useful
#      in a situation where your app has certain dependencies, for example, pandoc
#      wkhtmltopdf, ffmpeg, etc.
# https://www.accordbox.com/blog/how-deploy-django-project-dokku-docker/ is a good
# starting point,
# also see https://www.accordbox.com/blog/deploy-django-project-heroku-using-docker/
# if you need to build frontend assets in Dockerfile

# 15.2 use Dokku client
# Given the constraints, running Dokku commands remotely via SSH is fine.
# For certain configurations, the extra complication of manually invoking ssh can be a burden.
# The easiest way to interact with Dokku remotely is by using the official client. 
# see https://dokku.com/docs/deployment/remote-commands/#official-client

# 16. Technical Reference

# 16.1 run a one-off command under an app, e.g. Django shell:
# dokku run app-name python manage.py shell_plus
#   This will start a new container and run the desired command within that container.
#   The container image will be the same container image as was used to start the currently deployed app.
#   As of v0.25.0, this container will be removed after the process exits.