Skip to content

Instantly share code, notes, and snippets.

@andreivasiliu

andreivasiliu/acme.md

Created October 20, 2023 06:48
Show Gist options
  • Save andreivasiliu/796e813f82d62d40dc821da7e9b5631c to your computer and use it in GitHub Desktop.
Save andreivasiliu/796e813f82d62d40dc821da7e9b5631c to your computer and use it in GitHub Desktop.
Download ZIP
Demi ACME
Raw
acme.md

Demi Cluster:

  • A cluster of two halves
  • There is no master or slave; both demi hosts are masters
  • Security through isolation:
    • All web services (HTTP, DNS) run inside Demi VMs
    • Demi VMs may not access anything on the LAN, or on their host
    • Demi VMs may access each other
  • Security through minimalism:
    • The Demi VMs run on minimal Demi bare metal hosts
    • The bare metal hosts run no services, and cannot be reached from the Internet
    • The bare metal hosts only install software from trusted sources (e.g. Debian repo)
    • The bare metal hosts install as little software as possible (e.g. Ansible and libvirt)
  • Security through regeneration:
    • Demi VMs are assumed to be untrusted and unsafe; they can always become vulnerable
    • Demi VMs will be completely wiped and regenerated from scratch
    • Demi VMs may not persist anything between runs
    • Demi VMs may mount things from the bare metal hosts, but only as read-only

ACME:

  • ACME (the protocol behind Let's Encrypt) works in two steps:
    • Request dns-01 or http-01 challenge
      • dns-01: ACME server gives a TXT record
        • It must become available on _acme-challenge.demi.ro
        • Having two TXT records works, but it seems ACME only reads one
      • http-01: ACME server gives a file name and content
        • It must become available on demi.ro/.well-known/acme-challenge/
        • The names will be random and different for each challenge
    • Request certificate
      • ACME will perform the challenge, and if successful, it will return a certificate
      • This has a limit of at most 5 times a day (has a much higher limit on the staging/testing endpoint)

ACME approaches:

  • Separate Nginx service on port 80
    • Start port 80, add to load balancer, start acme, start port 443, add to loadbalancer
    • Load balance each one separately
    • Distribute challenge to both hosts, with a timeout?
    • Or make one host send the challenge request to the other if it's not found?
  • Custom memcached/etcd-like kv store
    • Would degrade and/or fail quorum?
  • Use iptables to funnel HTTP or DNS UDP packets to the right demi host
    • Wouldn't work for HTTP and DNS TCP: syn packet has no information
    • Might work for DNS UDP? But will fail for DNSSEC and DNSTLS, which I might want
  • Ask the router which hosts are active and push challenge text to them
    • Wouldn't work to bootstrap (unless 80 is separate)
      • Actually, wouldn't work at all: no hosts will be up initially
    • Race condition on host health
  • Per-demi-host DNS or HTTP port
    • Ask ACME to send the challenge to a specific port
    • Does not seem to be supported by ACME
    • Is there any other identifier possible?
  • DNS server on router
    • Serves TXT, routes everything else
    • Not sure how to implement it on OpenWrt
  • Make Nginx route request to the other demi host
    • Without the 80/443 split it might not work to bootstrap

Separate Nginx service on port 80:

  • Seems to be the only viable solution
  • SystemD services:
    • Port 80 and 81 Nginx service
    • ACME one-shot, reloads Nginx for port 443
    • Timer that re-runs ACME one-shots
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment