fast mastodon instance /w docker-compose

...README.md

• you need the latest version of docker and docker-compose installed
• these instructions presume that you're running apache and letsencrypt; if you have a different web server/proxy and cert mgmt strategy you are hopefully smart enough to figure it out yourself
• run sysctl -w vm.max_map_count=262144 or elasticsearch will fail; update sysctl.conf to make it permanent
• you'll need the following apache modules loaded:
  • proxy*
  • ssl
  • slotmem_shm
  • headers
• create your server directory; in the examples here it's /data/mastodon
• mkdir -p /data/mastodon/{public,elasticsearch,postgres14,redis}
• put docker-compose.yml and .env.production into /data/mastodon
• put your actual domain name into the LOCAL_DOMAIN and (if hosting on a subdomain but you want usernames to be in the root domain) WEB_DOMAIN sections of .env.production
  • WARNING: if LOCAL_DOMAIN and WEB_DOMAIN are different, you will want to set up aliases or redirects from https://LOCAL_DOMAIN/.well-known/webfinger and https://LOCAL_DOMAIN/.well-known/host-meta to https://WEB_DOMAIN/.well-known/webfinger and https://WEB_DOMAIN/.well-known/host-meta -- in apache-speak that means roughly

    RewriteEngine on
    RewriteCond %{REQUEST_URI} =/.well-known/webfinger
    RewriteRule ^ https://<WEB_DOMAIN>%{REQUEST_URI} [END,NE,R=permanent]
    RewriteCond %{REQUEST_URI} =/.well-known/host-meta
    RewriteRule ^ https://<WEB_DOMAIN>%{REQUEST_URI} [END,NE,R=permanent]
    

• generate a postgres password (echo $(tr -dc A-Za-z0-9_ < /dev/urandom | head -c 16) is a reasonable approach) put this value into both docker-compose.yml and .env.production in the noted places
• use rake to generate a SECRET_KEY_BASE and OTP_SECRET: docker-compose run --rm web bundle exec rake secret (run this twice to generate two keys) put these values into .env.production
• use rake to generate VAPID_PUBLIC_KEY and VAPID_PRIVATE_KEY: docker-compose run --rm web bundle exec rake mastodon:webpush:generate_vapid_key put these values into .env.production
• (OPTIONAL) if you want email notifications to work, fill out the "Sending mail" section of .env.production with the hostname and (if necessary) login/password for an email server that can relay mail for you. (Beware that most IP address ranges of most major hosting providers are considered spam sources by gmail/outlook/etc; if you want to do this for real you will probably want to set up Amazon SES or similar service to handle the DKIM/DMARC/etc heavy lifting for you and doing so is waaaaaay beyond the scope of this little doc.)
• set up the database: docker-compose run --rm web rails db:migrate IF THIS STEP FAILS: inspect the output carefully and delete the contents of the postgres14 directory before trying again
• note on using S3 for assets: mastodon seemed to choke on any bucket name with dots in it; I created a bucket entirely for it to use and created an IAM user with only R/W access to that bucket. From all accounts you really really REALLY want to use S3 or some similar object store for file storage from the get-go: strongly advise biting the bullet here and doing the work up-front.
• at this point, docker-compose up should bring up everything
• create and validate a user via the signup interface (this will also tell you if outbound email is working)
• exec into the streaming container: docker exec -it mastodon_streaming_1 /bin/bash ...and make yourself an admin: RAILS_ENV=production bin/tootctl accounts modify MYUSERNAME --role Admin
• reload the web interface, go to preferences -> administration -> site settings; turn off signups!
• once you've kicked the tires a bit, you probably want to make it persistent: put mastodon.service into /etc/systemd/system and then run sudo systemctl start mastodon followed by sudo systemctl status mastodon and optionally journalctl -u mastodon -f to tail the logs as it runs
• the one thing this setup currently does NOT provide is a postgres connection pooler. for a self-hosted, single-digit-userbase server, you can probably get away without it, but it is 100% guaranteed that this will be the first limit you hit if you try to scale up even slightly.

.env.production

# Federation
# ----------
# This identifies your server and cannot be changed safely later
# ----------
LOCAL_DOMAIN=mydomain.foo
# SET THIS IF YOU ARE NOT HOSTING MAST ON YOUR DOMAIN ROOT
# WEB_DOMAIN=mastodon.mydomain.foo

# Redis
# -----
REDIS_HOST=redis
REDIS_PORT=6379

# PostgreSQL
# ----------
DB_HOST=db
DB_USER=postgres
DB_NAME=mastodon_production
DB_PASS=<SET YOUR PG PASS HERE; MUST MATCH COMPOSE FILE>
DB_PORT=5432

# Elasticsearch (optional)
# ------------------------
ES_ENABLED=true
ES_HOST=es
ES_PORT=9200
# Authentication for ES (optional)
#ES_USER=elastic
#ES_PASS=password

# Secrets
# -------
# Make sure to use `rake secret` to generate secrets
# -------
SECRET_KEY_BASE=<GENERATE WITH RAKE; SEE NOTES>
OTP_SECRET==<GENERATE WITH RAKE; SEE NOTES>

# Web Push
# --------
# Generate with `rake mastodon:webpush:generate_vapid_key`
# --------
VAPID_PRIVATE_KEY=<GENERATE WITH RAKE; SEE NOTES>
VAPID_PUBLIC_KEY=<GENERATE WITH RAKE; SEE NOTES>

# Sending mail
# ------------
SMTP_SERVER=<YOUR SMTP SERVER>
SMTP_PORT=587
SMTP_LOGIN=<SMTP SASL LOGIN>
SMTP_PASSWORD=<SMTP SASL PASS>
SMTP_FROM_ADDRESS=notifications@mydomain.foo

# File storage (theorectically this is optional but empirically it's mandatory: you WILL regret
# your decisions if you try to use local file storage.  If you're allergic to S3, find one of
# the gazillion online guides to setting up other object stores for mastodon.)
# -----------------------
S3_ENABLED=true
S3_BUCKET=<BUCKET NAME -- SEE NOTES>
S3_REGION=<BUCKET REGION>
AWS_ACCESS_KEY_ID=<AWS ACCESS KEY ID>
AWS_SECRET_ACCESS_KEY=<AWS SECRET ACCESS KEY>
S3_PROTOCOL=https
S3_HOSTNAME=s3-<BUCKET REGION>.amazonaws.com

# IP and session retention
# -----------------------
# Make sure to modify the scheduling of ip_cleanup_scheduler in config/sidekiq.yml
# to be less than daily if you lower IP_RETENTION_PERIOD below two days (172800).
# -----------------------
IP_RETENTION_PERIOD=31556952
SESSION_RETENTION_PERIOD=31556952

## Rails general stuff
# setting log level to WARN because otherwise the logs are INSANELY verbose and almost
# certainly redundant with your apache/nginx/whatever logs
RAILS_LOG_LEVEL=warn

apache_site.conf

<IfModule mod_ssl.c>
  <VirtualHost _default_:443>
    ServerName      mastodon.mydomain.foo
    DocumentRoot    /data/mastodon/public
    ErrorLog ${APACHE_LOG_DIR}/mastodon-ssl-error.log
    CustomLog ${APACHE_LOG_DIR}/mastodon-ssl-access.log combined

    ProxyPreserveHost On
    RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
    RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}
    ProxyAddHeaders On
    ProxyPass /api/v1/streaming http://localhost:4000/
    ProxyPass / http://localhost:3000/
    ProxyPassReverse / http://localhost:3000/

    SSLEngine on

    <Directory "/data/mastodon/public">
      Options Indexes FollowSymLinks MultiViews Includes
      AllowOverride All
      DirectoryIndex index.html
      Require all granted
    </Directory>

    # this is all letsencrypt stuff and should automatically be added for you by "certbot --apache"
    Include               /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile    /etc/letsencrypt/live/mastodon.mydomain.foo/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/mastodon.mydomain.foo/privkey.pem
  </VirtualHost>
</IfModule>

<VirtualHost *:80>
    ServerName      mastodon.mydomain.foo
    DocumentRoot    /data/mastodon/public
    ErrorLog ${APACHE_LOG_DIR}/mastodon-error.log
    CustomLog ${APACHE_LOG_DIR}/mastodon-access.log combined
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =mastodon.mydomain.foo
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

docker-compose.yml

version: '3.7'

x-base: &base
  restart: always

x-serviceinternal: &intsvc
  <<: *base
  networks:
    - internal_network

x-serviceexternal: &extsvc
  <<: *base
  networks:
    - internal_network
    - external_network

x-servicemast: &mastsvc
  <<: *extsvc
  image: tootsuite/mastodon:v4.1.0
  env_file: .env.production

services:
  db:
    <<: *intsvc
    image: postgres:14-alpine
    shm_size: 256mb
    healthcheck:
      test: ['CMD', 'pg_isready', '-U', 'postgres']
    volumes:
      - ./postgres14:/var/lib/postgresql/data
    environment:
      - 'POSTGRES_HOST_AUTH_METHOD=trust'
      - 'POSTGRES_PASSWORD=<SET YOUR POSTGRES PASSWORD HERE>'
      - 'POSTGRES_DB=mastodon_production'

  redis:
    <<: *intsvc
    image: redis:7-alpine
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    volumes:
      - ./redis:/data

  es:
    <<: *intsvc
    image: docker.elastic.co/elasticsearch/elasticsearch:7.17.4
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m -Des.enforce.bootstrap.checks=true"
      - "xpack.license.self_generated.type=basic"
      - "xpack.security.enabled=false"
      - "xpack.watcher.enabled=false"
      - "xpack.graph.enabled=false"
      - "xpack.ml.enabled=false"
      - "bootstrap.memory_lock=true"
      - "cluster.name=es-mastodon"
      - "discovery.type=single-node"
      - "thread_pool.write.queue_size=1000"
    healthcheck:
       test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"]
    volumes:
       - ./elasticsearch:/usr/share/elasticsearch/data
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - '127.0.0.1:9200:9200'

  web:
    <<: *mastsvc
    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
    healthcheck:
      # prettier-ignore
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
    ports:
      - '127.0.0.1:3000:3000'
    depends_on:
      - db
      - redis
      - es
    volumes:
      - ./public/system:/mastodon/public/system

  streaming:
    <<: *mastsvc
    command: node ./streaming
    healthcheck:
      # prettier-ignore
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']
    ports:
      - '127.0.0.1:4000:4000'
    depends_on:
      - db
      - redis

  sidekiq:
    <<: *mastsvc
    command: bundle exec sidekiq
    depends_on:
      - db
      - redis
    volumes:
      - ./public/system:/mastodon/public/system
    healthcheck:
      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]

  ## Uncomment to enable federation with tor instances along with adding the following ENV variables
  ## http_proxy=http://privoxy:8118
  ## ALLOW_ACCESS_TO_HIDDEN_SERVICE=true
  # tor:
  #   <<: *extsvc
  #   image: sirboops/tor
  #
  # privoxy:
  #   <<: *extsvc
  #   image: sirboops/privoxy
  #   volumes:
  #     - ./priv-config:/opt/config

networks:
  external_network:
  internal_network:
    internal: true

mastodon.service

[Unit]
Description=%i service with docker compose
PartOf=docker.service
After=docker.service

[Service]
Type=simple
RemainAfterExit=false
WorkingDirectory=/data/memory/mastodon
ExecStart=/usr/bin/docker-compose up --remove-orphans
ExecStop=/usr/bin/docker-compose down

[Install]
WantedBy=multi-user.target

sysctl.conf

# elasticsearch :(
vm.max_map_count=262144

Note:

exec into the streaming container: docker exec -it mastodon_streaming_1 /bin/bash ...and make yourself an admin: RAILS_ENV=production bin/tootctl accounts modify MYUSERNAME --role admin

s/b

exec into the streaming container: docker exec -it mastodon_streaming_1 /bin/bash ...and make yourself an admin: RAILS_ENV=production bin/tootctl accounts modify MYUSERNAME --role Admin

Capitalization turns out to matter.

oh geeze. updated.