Caddy on fly.io as reverse proxy to services on Tailscale network

README.md

This is my attempt to run a webapp python -m http.server 8000 --bind 0.0.0.0 on my home machine (Mac mini) and serve it on the public Internet using Caddy as a reverse proxy to my local machine over encrypted mesh VPN (Tailscale)

• Follow this article and set an ephemeral Tailscale key as a secret in Fly, using flyctl secrets set TAILSCALE_AUTHKEY="tskey-<key>"
• Put all these files in a folder and run flyctl deploy
• Make sure to modify the domain name and IP/port in Caddyfile. Since Fly.io handles HTTPS certificates, we don't need Caddy to do this.

This is not currently working. When this app boots up, it never gets into a healthy state and the flyctl logs does not provide a helpful message.

Caddyfile

log stdout
errors stdout
auto_https off

http://myapp.fly.dev {
  reverse_proxy 100.120.108.62:8000
}

Dockerfile

FROM caddy:2-alpine as builder
WORKDIR /app
COPY . ./
COPY ./Caddyfile /etc/caddy/Caddyfile


FROM alpine:latest as tailscale
WORKDIR /app
COPY . ./
ENV TSFILE=tailscale_1.16.2_amd64.tgz
RUN wget https://pkgs.tailscale.com/stable/${TSFILE} && tar xzf ${TSFILE} --strip-components=1
COPY . ./


# https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
FROM caddy:2-alpine
RUN apk update && apk add ca-certificates iptables ip6tables && rm -rf /var/cache/apk/*

# Copy binary to production image
COPY --from=builder /app/start.sh /app/start.sh
COPY --from=builder /etc/caddy/Caddyfile /etc/caddy/Caddyfile
COPY --from=tailscale /app/tailscaled /app/tailscaled
COPY --from=tailscale /app/tailscale /app/tailscale
RUN mkdir -p /var/run/tailscale /var/cache/tailscale /var/lib/tailscale

#caddy is running on 80
EXPOSE 80

# Run on container startup.
CMD ["/app/start.sh"]

fly.toml

# fly.toml file generated for tailproxy on 2021-11-14T03:39:04Z

app = "tailproxy"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 80
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "4s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

start.sh

#!/bin/sh

/app/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/var/run/tailscale/tailscaled.sock &
until /app/tailscale up --authkey=${TAILSCALE_AUTHKEY} --hostname=flyio
do
    sleep 0.1
done
echo "tailscale has started ........... now reloading caddy"
# caddy reload

echo "caddy is up, now we wait for requests to come"
tail -f /dev/null