Skip to content

Instantly share code, notes, and snippets.

@omltcat
Last active December 8, 2024 15:46
Show Gist options
  • Save omltcat/241ef622070ca0580f2876a7cfa7de67 to your computer and use it in GitHub Desktop.
Save omltcat/241ef622070ca0580f2876a7cfa7de67 to your computer and use it in GitHub Desktop.
Caddy with Docker Labels: Easy config for wildcard certs and Authelia

What is this?

One great feature of caddy-docker-proxy is that you can quickly define config rules with Docker Compose labels in each containers on the fly like Traefik, instead of at a centralized place. With this feature, we can define everything in compose files, and don't ever need to mess with a config file (Caddyfile or JSON).

Taking advantage of snippets, I created this docker-compose.yaml example so that you can quickly define routing rules and add authetication like Authelia with just 3 lines of labels below each docker container you use.

With this example, should not ever need to manually edit Caddyfile config.

When you add a new container, you just need to do this:

networks:
  caddy_net:
    external: true

services:
  snapdrop:
    image: linuxserver/snapdrop:latest
    container_name: snapdrop
    networks:
      - caddy_net
    restart: always
    #πŸ‘‡ Magic happening below
    labels:
      caddy: drop.example.com                   #πŸ‘ˆ Subdomain using wildcard cert
      caddy.reverse_proxy: "{{upstreams 80}}"   #πŸ‘ˆ Container port
      caddy.import: auth                        #πŸ‘ˆ One-line enable Authelia

Now with auto_https prefer_wildcard option merged, we get a even better config structure if wanting to use wildcard SSL certs for HTTPS.

Caddy Dockerfile

To use wildcard certs, need to build a custom Docker image of Caddy with Cloudflare module (if that is you DNS nameserver) to handle DNS challenge.
Put this Dockerfile in the same directory as the Docker Compose file below.

ARG CADDY_VERSION=2
FROM caddy:${CADDY_VERSION}-builder AS builder

# no need the "v2.9.0-beta.2" part after new version release
RUN xcaddy build v2.9.0-beta.2 \ 
    --with github.com/lucaslorentz/caddy-docker-proxy/v2 \
    --with github.com/caddy-dns/cloudflare

FROM caddy:${CADDY_VERSION}-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

CMD ["caddy", "docker-proxy"]

Caddy Docker Compose

Besure to modify to your info where indicated. Some examples are provided at the end.

.env file

EMAIL=your_email@example.com           #πŸ‘ˆ Your email for SSL cert
CF_API_TOKEN=your_cloudflare_api_token #πŸ‘ˆ Get your token from Cloudflare
AUTH_HOST_INTERNAL=authelia:9091       #πŸ‘ˆ Authelia container name and port
AUTH_HOST_EXTERNAL=auth.example.com    #πŸ‘ˆ Public facing domain of Authelia 

Caddy Main Container

# docker-compose.yaml
networks:
  caddy_net:  #ℹ️ Caddy ingress network
    name: caddy_net
    ipam:
      driver: default  

services:
  caddy:
    container_name: caddy
    build: .
    restart: always
    environment:
      CADDY_INGRESS_NETWORKS: caddy_net
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./caddy/data:/data/caddy        #πŸ‘ˆ where to save SSL certs
      - ./caddy/config:/config/caddy    #πŸ‘ˆ where to save configs
    networks:
      - caddy_net
    ports:
      - 80:80
      - 443:443
    extra_hosts:
      - host.docker.internal:host-gateway

# to be continued below...

Caddy Labels

Before using, we need to configure Caddy itself first. This can be done purely with compose labels. We group labels with _numbers as yaml keys must be unique. You can read more about Labels to Caddyfile conversion here.

While we CAN put these labels directly under the main container above, it is better to use a separate, lightweight container. As commented by @coandco, if we change any labels under the main container, Caddy has to be restarted and thus interrupt existing connections.

Personally, I use the traefik/whoami image, which can double as a troubleshooting tool.

# docker-compose.yaml continued
  caddy-config:
    container_name: caddy-config
    image: traefik/whoami:latest
    networks:
      - caddy_net
    restart: always
    labels:
    #############################################
    # Settings and snippets to get things working
    # You shouldn't need to modify this normally
    # Custom settings and definitions are below
    #############################################

      #### Global Settings ####
      caddy_0.email: "{env.EMAIL}"
      caddy_0.auto_https: prefer_wildcard

      #### Snippets ####
      # Get wildcard certificate
      caddy_1: (wildcard)
      caddy_1.tls.dns: "cloudflare {env.CF_API_TOKEN}"
      caddy_1.tls.resolvers: 1.1.1.1 1.0.0.1
      caddy_1.handle.abort: ""

      # Secure a site with Authelia
      caddy_2: (auth)
      caddy_2.forward_auth: "{$$AUTH_HOST_INTERNAL}"
      caddy_2.forward_auth.uri: /api/verify?rd=https://{$$AUTH_HOST_EXTERNAL}
      caddy_2.forward_auth.copy_headers : Remote-User Remote-Groups Remote-Name Remote-Email

      # Skip TLS verify for backend with self-signed HTTPS
      caddy_3: (https)
      caddy_3.transport: http
      caddy_3.transport.tls: ""
      caddy_3.transport.tls_insecure_skip_verify: ""

    ###########################################
    # Custom settings. Modify things below πŸ‘‡:
    # Make sure they have unique label numbers
    ###########################################

      # Custom global settings, add/edit as needed
      # caddy_0.log: default
      # caddy_0.log.format: console

      # Uncomment this during testing to avoid hitting rate limit.
      # It will try to obtain SSL from Let's Encrypt's staging endpoint.
      # acme_ca: "https://acme-staging-v02.api.letsencrypt.org/directory" # πŸ‘ˆ Staging

      ## Setup wildcard sites
      caddy_10: "*.example.com"   #πŸ‘ˆ Change to your domain
      caddy_10.import: wildcard

      # Add our first site, which this container itself
      caddy_99: whoami.example.com                       #πŸ‘ˆ Subdomain using wildcard cert
      caddy_99.reverse_proxy: "{{upstreams 80}}"         #πŸ‘ˆ Container port
      caddy_99.import: auth                              #πŸ‘ˆ Enable protection by Authelia

# to be continued below...

If we have some non-docker sites that need to be reverse proxied, we can also add their configs here:

# docker-compose.yaml continued
      # e.g.: Pi-Hole on another machine in the same LAN
      caddy_100: pihole.example.com                      #πŸ‘ˆ Subdomain using wildcard cert
      caddy_100.reverse_proxy: 192.168.1.4:88            #πŸ‘ˆ LAN IP and port
      caddy_100.import: auth                             #πŸ‘ˆ Enable protection by Authelia

      # e.g. OpenMediaVault on the host machine, with self-signed https at port 4430
      caddy_101: omv.example.com                         #πŸ‘ˆ Subdomain using wildcard cert
      caddy_101.reverse_proxy: host.docker.internal:4430 #πŸ‘ˆ Port on host machine
      caddy_101.reverse_proxy.import: https              #πŸ‘ˆ Allow self-signed cert between OMV and Caddy
      caddy_101.import: auth                             #πŸ‘ˆ Enable protection by Authelia

Authelia Docker Compose

Make sure Authelia is in the same network with Caddy. Add two labels under your existing Authelia compose:

networks:
  caddy_net:
    external: true
  # ... other networks used by authelia ...

services:
  authelia:
    container_name: authelia
    hostname: authelia
    image: authelia/authelia:latest
    networks:
      - caddy_net
      - #... other networks ...
    expose:
      - 9091
    # ... the rest of your regular ...
    # ... authelia compose here ...
    # ...
    # Add this:
    labels:
      caddy: auth.example.com                    #πŸ‘ˆ Public facing subdomain of Authelia
      caddy.reverse_proxy: "{{upstreams 9091}}"  #πŸ‘ˆ Authelia container port

Add new sites from other containers

Three labels! Just the subdomain, port and if you want Authelia.

networks:
  caddy_net:
    external: true

  snapdrop:
    image: linuxserver/snapdrop:latest
    container_name: snapdrop
    networks:
      - caddy_net
    restart: always
    labels:
      caddy: drop.example.com                    #πŸ‘ˆ Subdomain
      caddy.reverse_proxy: "{{upstreams 80}}"    #πŸ‘ˆ Container port
      caddy.import: auth                         #πŸ‘ˆ Authelia

Some extra tricks

Caddy Admin Endpoint and Metrics (Homepage)

The docker version of Caddy needs its admin endpoint internally for reload, and does not allow you to modify or expose it outside the container.

If you need it, such as for the Caddy widget of Homepage dashboard, you can expose this portion with the following labels for Caddy:

# docker-compose.yaml
# under caddy-config container
      caddy_20: :2020
      caddy_20.handle: /reverse_proxy/upstreams
      caddy_20.handle.reverse_proxy: localhost:2019
      caddy_20.handle.reverse_proxy.header_up: Host localhost:2019

And for your Homepage widget: (make sure they are on the same network)

# services.yaml
        ...
        widget:
            type: caddy
            url: http://caddy:2020

More to come...

Please leave a ⭐ if this helps you!

@coandco
Copy link

coandco commented Oct 14, 2024

What specific features are you using 2.9.0-beta2 for? Why have that over mainline 2.8.4?

@omltcat
Copy link
Author

omltcat commented Oct 15, 2024

What specific features are you using 2.9.0-beta2 for? Why have that over mainline 2.8.4?

The https wildcard PR is merged very recently and only in beta now
caddyserver/caddy#6146

@coandco
Copy link

coandco commented Oct 15, 2024

Ah! Yeah, that's pretty important. Thanks!

@coandco
Copy link

coandco commented Oct 16, 2024

Have you been able to get it to work with multiple wildcard domains? I'm trying something like this:

      caddy_10: "*.one.example.com"
      caddy_10.import: wildcard
      
      caddy_11: "*.two.example.com"
      caddy_11.import: wildcard

But when I do that, it's not getting any of the wildcard domains.

@coandco
Copy link

coandco commented Oct 17, 2024

Ah ha, I was able to provide them a minimal test case and now they're fixing it in beta3: caddyserver/caddy#6636

@omltcat
Copy link
Author

omltcat commented Oct 18, 2024

Ah ha, I was able to provide them a minimal test case and now they're fixing it in beta3: caddyserver/caddy#6636

Ha I've used the wildcard PR for several months and never realized there was a bug here. Good find!

@coandco
Copy link

coandco commented Oct 20, 2024

One change that I'd suggest: running a separate container with your base config on it. That way, you don't have to restart caddy (and thus interrupt any existing connections) if you want to update any of the caddy_* labels on the base container. Something like this:

networks:
  caddy_net:  #ℹ️ Caddy ingress network
    name: caddy_net
    ipam:
      driver: default  

services:
  caddy:
    container_name: caddy
    build: .
    restart: always
    environment:
      EMAIL: your_email@example.com           #πŸ‘ˆ Your email for SSL cert
      CF_API_TOKEN: your_cloudflare_api_token #πŸ‘ˆ Get your token from Cloudflare
      AUTH_HOST_INTERNAL: authelia:9091       #πŸ‘ˆ Authelia container name and port
      AUTH_HOST_EXTERNAL: auth.example.com    #πŸ‘ˆ Public facing domain of Authelia 
      CADDY_INGRESS_NETWORKS: caddy_net
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro #### needs socket to read events
      - ./caddy/data:/data/caddy        #πŸ‘ˆ where to save certificates
      - ./caddy/config:/config/caddy    #πŸ‘ˆ where to save configs
    networks:
      - caddy_net
    ports:
      - 80:80
      - 443:443
    extra_hosts:
      - host.docker.internal:host-gateway
  
  caddy-config:
    restart: unless-stopped
    image: alpine
    command: ["/bin/sleep", "infinity"]
    networks:
      - caddy_net
    labels:
    #############################################
    # Settings and snippets to get things working
    # You shouldn't need to modify this normally
    # Custom settings and definitions are below
    #############################################

      #### Global Settings ####
      caddy_0.email: "{env.EMAIL}"
      caddy_0.auto_https: prefer_wildcard

      #### Snippets ####
      # Get wildcard certificate
      caddy_1: (wildcard)
      caddy_1.tls.dns: "cloudflare {env.CF_API_TOKEN}"
      caddy_1.tls.resolvers: 1.1.1.1 1.0.0.1
      caddy_1.handle.abort: ""

      # Staging cert server for testing
      caddy_2: (staging)
      caddy_2.tls.ca: "https://acme-staging-v02.api.letsencrypt.org/directory"

      # Secure a site with Authelia
      caddy_3: (auth)
      caddy_3.forward_auth: "{env.AUTH_HOST_INTERNAL}"
      caddy_3.forward_auth.uri: /api/verify?rd=https://{env.AUTH_HOST_EXTERNAL}
      caddy_3.forward_auth.copy_headers : Remote-User Remote-Groups Remote-Name Remote-Email

      # Skip TLS verify for backend with self-signed HTTPS
      caddy_4: (https)
      caddy_4.transport: http
      caddy_4.transport.tls: ""
      caddy_4.transport.tls_insecure_skip_verify: ""

    ###########################################
    # Custom settings. Modify things below πŸ‘‡:
    # Make sure they have unique label numbers
    ###########################################

      # Custom global settings, edit as needed
      #caddy_0.log: default
      #caddy_0.log.format: console

      ## Setup wildcard sites
      caddy_10: "*.example.com"   #πŸ‘ˆ Change to your domain
      caddy_10.import: wildcard

      #ℹ️ Examples: Setup non-docker sites to use Caddy as reverse proxy
      # e.g. OpenMediaVault on host, with self-signed https at port 4430
      caddy_100: omv.example.com                         #πŸ‘ˆ Subdomain using wildcard cert
      caddy_100.reverse_proxy: host.docker.internal:4430 #πŸ‘ˆ Port on host machine
      caddy_100.reverse_proxy.import: https              #πŸ‘ˆ Allow self-signed cert between OMV and Caddy
      caddy_100.import: auth                             #πŸ‘ˆ Enable protection by Authelia

      # e.g.: Pi-Hole on another machine in LAN
      caddy_101: pihole.example.com                      #πŸ‘ˆ Subdomain using wildcard cert
      caddy_101.reverse_proxy: 192.168.1.4:88            #πŸ‘ˆ LAN IP and port
      caddy_101.import: auth                             #πŸ‘ˆ Enable protection by Authelia

That way if you (for example) want to add a new static forward, you can just update the labels on the caddy-config container and docker-compose up -d it, loading the new config without interrupting existing connections.

@omltcat
Copy link
Author

omltcat commented Oct 20, 2024

@coandco I am not sure if existing connection will not be interrupted this way. Have you tested this way? AFAIK, whenever the caddy labels are updated by any container, Caddy reloads internally.

Anyway, if this is true, maybe use something else other than command: ["/bin/sleep", "infinity"]? The container can be properly shutdown (kill signal) if it is sleeping and has to wait for a timeout. I usually use something light like traefik/whoami

@coandco
Copy link

coandco commented Oct 20, 2024

Yes, I've confirmed that existing connections aren't touched when the labels are updated on containers other than the base Caddy container -- I started a download of a several-gig file and tested updating labels on the caddy-config container, and the download finished successfully, whereas if I update labels on the base caddy container it restarts it and all existing connections are interrupted.

sleep responds to signals just fine, and the caddy-config container doesn't have to wait for any timeouts when it's being shut down. I was using sleep infinity as a way to consume the least system resources I could while having a container up with the labels needed.

@coandco
Copy link

coandco commented Oct 23, 2024

I've written up a worked example of my setup here: https://github.com/coandco/caddy-homeserver

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment