Caddy + docker-compose mastodon + ntfy for the IPv6 world

Caddyfile

# You'll notice that webfinger is redundant with the main reverse_proxy.
# That's here in case you want to have your home page on something other than mastodon
phh.me www.phh.me {
        reverse_proxy mastodon_web:3000
        handle_path /ntfy/* {
                rewrite * {path}
                reverse_proxy ntfy:80
        }
        handle /.well-known/webfinger {
                reverse_proxy mastodon_web:3000
        }
        handle /api/v1/streaming/* {
                reverse_proxy mastodon_streaming:4000
        }
}

docker-compose.yml

version: "3.8"
services:
  mastodon_db:
    image: postgres:16-alpine
    restart: always
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "mastodon"]
    environment:
      POSTGRES_USER: mastodon
      POSTGRES_PASSWORD: SETME-and-in-mastodon-env
      POSTGRES_DB: mastodon
    volumes:
      - ./mastodon_db:/var/lib/postgresql/data
    networks:
      mastodon_net:

  mastodon_redis:
    restart: always
    image: redis:7-alpine
    networks:
      - mastodon_net
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    volumes:
      - ./mastodon/redis:/data

  mastodon_es:
    restart: always
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.1
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "cluster.name=es-mastodon"
      - "discovery.type=single-node"
      - "bootstrap.memory_lock=true"
      - "xpack.security.enabled=false"
      - "xpack.security.http.ssl.enabled=false"
    networks:
      - mastodon_net
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl --silent --fail localhost:9200/_cluster/health || exit 1",
        ]
    volumes:
      - ./mastodon_elasticsearch:/usr/share/elasticsearch/data
    ulimits:
      memlock:
        soft: -1
        hard: -1

  mastodon_web:
    build: .
    image: ghcr.io/mastodon/mastodon:v4.2
    restart: always
    env_file: mastodon-env
    command: bundle exec puma -C config/puma.rb
    networks:
      mastodon_net:
      nat:
    healthcheck:
      # prettier-ignore
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
    depends_on:
      - mastodon_db
      - mastodon_redis
    volumes:
      - ./mastodon/public-system:/mastodon/public/system

  mastodon_streaming:
    build: .
    image: ghcr.io/mastodon/mastodon:v4.2
    env_file: mastodon-env
    restart: always
    command: node ./streaming
    networks:
      - mastodon_net
    healthcheck:
      # prettier-ignore
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']
    depends_on:
      - mastodon_db
      - mastodon_redis

  mastodon_sidekiq:
    build: .
    image: ghcr.io/mastodon/mastodon:v4.2
    env_file: mastodon-env
    restart: always
    command: bundle exec sidekiq
    networks:
      - mastodon_net
      - nat
    healthcheck:
      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
    depends_on:
      - mastodon_db
      - mastodon_redis
    volumes:
      - ./mastodon/public-system:/mastodon/public/system

  ntfy:
    image: binwiederhier/ntfy
    restart: unless-stopped
    healthcheck:
        test: ["CMD-SHELL", "wget -q --tries=1 http://localhost:80/v1/health -O - | grep -Eo '\"healthy\"\\s*:\\s*true' || exit 1"]
        interval: 60s
        timeout: 10s
        retries: 3
        start_period: 40s
    environment:
      NTFY_BASE_URL: https://phh.me/ntfy
      NTFY_CACHE_FILE: /var/lib/ntfy/cache.db
      NTFY_AUTH_FILE: /var/lib/ntfy/auth.db
      NTFY_AUTH_DEFAULT_ACCESS: deny-all
      NTFY_BEHIND_PROXY: true
      NTFY_ATTACHMENT_CACHE_DIR: /var/lib/ntfy/attachments
      NTFY_ENABLE_LOGIN: true
      NTFY_WEB_PUSH_PUBLIC_KEY: SETME
      NTFY_WEB_PUSH_PRIVATE_KEY: SETME
      NTFY_WEB_PUSH_FILE: /var/lib/ntfy/webpush.db
      NTFY_WEB_PUSH_EMAIL_ADDRESS: phh@phh.me
    volumes:
      - ./ntfy:/var/lib/ntfy/
    networks:
      internal_http:
    command: serve

  caddy:
    image: caddy:latest
    restart: unless-stopped
    volumes:
      - "./Caddyfile:/etc/caddy/Caddyfile"
      - /srv/www:/files
      - ./caddy_data:/data
    networks:
      internal_http:
      mastodon_net:
      ipv6pub:
        # Set the public IP you want for your HTTP(S) server.
        ipv6_address: 2a01:e0a:dead:beef:ffff:0000:0000:0010

networks:
  ipv6pub:
    enable_ipv6: true
    driver: ipvlan
    driver_opts:
      # Change me to your LAN ethernet interface
      parent: enp0s3
      ipvlan_mode: l2
    ipam:
      driver: default
      config:
        # This is your real public IPv6 subnet. Change it.
        - subnet: "2a01:e0a:dead:beef::/64"
        # We don't declare a gateway because it'll get picked up by RA

  nat:
    enable_ipv6: true
    ipam:
      driver: default
      # This is a site-local subnet to be used for the NAT-ed IPv6 world.
      # Feel free to keep this, or something else. You just can't use the same subnet as the one declared before
      config:
        - subnet: "fec0:dead:beaf::/64"
  internal_http:
    internal: true
  mastodon_net:
    internal: true

gistfile1.txt

This is an "IPv6-only" mastodon setup inside docker, without using ugly port forwarder, but actually using IPv6.
Caddy is used as the front HTTP reverse proxy to distribute the requests, with automatic SSL config.

This has been tested on ARM64. (a VM inside a Freebox Server Delta with 15GB RAM)

I've always had a hard time to make a proper IPv6 setup docker, I'm now almost happy with the current state.

Feel free to ping me on @phh@phh.me to discuss this doc.

Ntfy is completely unrelated to mastodon, I just wanted to show an additional service in addition to just mastodon

The setup is pretty annoying, but I'll give pointers.

We'll extensively use docker "networks" (which are each a bridge).

We'll have four networks:
- Two which are fully internal and doesn't go over the internet ("internal_http" and "mastodon_net"). ntfy, postgresql, redis and friends will be there
- One that is fully NAT-ed [1] ("nat"). So it can access the internet but not the other way. Mastodon web server and sidekiq will be there
- One that is public ("ipv6pub"). Caddy our front HTTP proxy

We'll use a site-local prefix for the NAT network (the docker example uses a public subnet which ugh).

Please note that we'll refer to the container names at various places.
When using networks (other than the default one) in a container, docker will export the IP to that container.
For instance, caddyfile refers to "mastodon_web", docker will resolve it for us to the IP of the "mastodon_web" container declared in the docker yaml.

The public network will use ipvlan to allow having a container with its own public IPv6 rather than using the IPv6 of the host.

[1] Yes with proper IPv6 we wouldn't NAT, we would firewall, but docker's documentation is... uh... yeah ok.

This docker-compose config will store everything stateful in the local folder.

To do the setup, you need:
1. Edit all IPs, domain names, mail addresses, base url, and POSTGRES_PASSWORD
2. (fake) start the docker: `docker compose up`. Give it a minute for thing to ty to start and fail, then do ctrl-c to stop it again
3. Fix permissions: `chown 1000:1000 -R mastodon_elasticsearch` ; `chown 991 mastodon/public-system`
4. Generate ntfy webkeys with `docker run --rm binwiederhier/ntfy webpush keys` and add the output to docker-compose.yml environment
5. Generate mastodon secrets with `docker run --rm -ti --entrypoint=/bin/sh ghcr.io/mastodon/mastodon:v4.2` then inside it `rake secret` twice to set a SECRET_KEY_BASE and OTP_SECRET
6. Inside that same docker `rake mastodon:webpush:generate_vapid_key` to generate VAPID_PUBLIC_KEY and VAPID_PRIVATE_KEY
7. Relaunch `docker compose up`
8. You need to connect to the mastodon_web container to initialize DB. You can attach to it with `docker exec -ti XXX-mastodon_web-1` where XXX is your folder name, and then do `RAILS_ENV=production bundle exec rake mastodon:setup` to start mastodon setup. At the end of this, you'll be greeted with your admin password

mastodon-env

# Most of those comments come from original mastodon config

LOCAL_DOMAIN=phh.me

# Redis
# -----
REDIS_HOST=mastodon_redis
REDIS_PORT=6379

# PostgreSQL
# ----------
DB_HOST=mastodon_db
DB_USER=mastodon
DB_NAME=mastodon
DB_PASS=SETME
DB_PORT=5432

# Elasticsearch (optional)
# ------------------------
ES_ENABLED=true
ES_HOST=mastodon_es
ES_PORT=9200
# Authentication for ES (optional)
ES_USER=elastic
ES_PASS=

# Secrets
# -------
# Make sure to use `rake secret` to generate secrets
# -------
SECRET_KEY_BASE=SETME
OTP_SECRET=SETME

# Web Push
# --------
# Generate with `rake mastodon:webpush:generate_vapid_key`
# --------
VAPID_PRIVATE_KEY=SETME
VAPID_PUBLIC_KEY=SETME

# Sending mail
# ------------
SMTP_SERVER=
SMTP_PORT=587
SMTP_LOGIN=
SMTP_PASSWORD=
SMTP_FROM_ADDRESS=notifications@example.com

# File storage (optional)
# -----------------------
S3_ENABLED=false
S3_BUCKET=files.example.com
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
S3_ALIAS_HOST=files.example.com

# IP and session retention
# -----------------------
# Make sure to modify the scheduling of ip_cleanup_scheduler in config/sidekiq.yml
# to be less than daily if you lower IP_RETENTION_PERIOD below two days (172800).
# -----------------------
IP_RETENTION_PERIOD=31556952
SESSION_RETENTION_PERIOD=31556952