Skip to content

Instantly share code, notes, and snippets.

@aaronmboyd
Last active December 1, 2020 15:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aaronmboyd/ad17e1cec27da3321c78ff5ac5a1f9c1 to your computer and use it in GitHub Desktop.
Save aaronmboyd/ad17e1cec27da3321c78ff5ac5a1f9c1 to your computer and use it in GitHub Desktop.

Celo Attestation Service & TLS

Motivation

To create a companion piece to the Celo forum guide on adding TLS to your attestation web server (https://forum.celo.org/t/tls-for-attestation-service/649) that uses Docker instead of requiring nginx / certbot installation on your host VM. This guide is an simplified version of some of the documentation for the source docker containers (referenced at the end of this page).

Prerequisites

  1. You have a hosted domain you have control over and can create a DNS A record
  2. Be ready to update your metadata file for your validator after changing to a fully qualified domain name. You can follow the guide here: https://docs.celo.org/validator-guide/attestation-service#registering-metadata
  3. Access to manage the docker containers serving the attestation service. For simplicity this guide will simply use docker run from the host VM but this approach should work the same with compose. If you are using a more mature container management system like Kubernetes, it's probably better to manage the regular way with a dedicated nginx ingress pod, as you might not have access to the docker socket that is used here.

Steps

  1. Create a new DNS A record pointing to the current IP address of your attestation service.

  2. In addition to the existing port 80 for the attestation service, make sure port 443 ingress to your attestation is available. For cloud servers this might involve updating security groups etc, and/or iptables management if you are managing your hosts directly.

  3. Run the nginx-proxy. The volume mounts are important here as they are reused by the letsencrypt helper.

docker run --detach \
    --name nginx-proxy \
    --publish 80:80 \
    --publish 443:443 \
    --volume /etc/nginx/certs \
    --volume /etc/nginx/vhost.d \
    --volume /usr/share/nginx/html \
    --volume /var/run/docker.sock:/tmp/docker.sock:ro \
    jwilder/nginx-proxy
  1. Run the letsencrypt-nginx-proxy-companion. It's recommended to enter your email for the certificate registration. You copy the volumes from the nginx-proxy and again mount the docker socket.
docker run --detach \
    --name nginx-proxy-letsencrypt \
    --volumes-from nginx-proxy \
    --volume /var/run/docker.sock:/var/run/docker.sock:ro \
    --volume /etc/acme.sh \
    --env "DEFAULT_EMAIL=<your email>" \
    jrcs/letsencrypt-nginx-proxy-companion
  1. Restart your attestation web service, but add in the following new environment variables VIRTUAL_HOST, VIRTUAL_PORT, and LETSENCRYPT_HOST. Set the VIRTUAL_HOST and LETSENCRYPT_HOST to your fully qualified domain name pointed to in your DNS record. In this example, I also changed the port of the express instance to 3000, just as added security / sanity check since 3000 doesn't match any of the ingress ports (80/443).
    -p 3000:3000 \
    -e PORT=3000 \
    -e VIRTUAL_HOST="baklava.attestation.a.pretoriaresearchlab.io" \
    -e VIRTUAL_PORT="3000" \
    -e LETSENCRYPT_HOST="baklava.attestation.a.pretoriaresearchlab.io" \
  1. That's it! No nginx.conf configuration, no manual certificate management.

How Does It Work?

The docker socket gives access to the docker system API, where a container can inspect the state of new containers. The nginx-proxy listens for new containers with the named environment variables (VIRTUAL_HOST etc), checks which ports are published and configures itself to proxy to those new services on that port.

The letsencrypt helper works in the same way and checks whether it needs to request and issue a new certificate for the new LETSENCRYPT_HOST that appeared on a new container. The shared volumes between the proxy and helper allows nginx to have the cert store in sync. Every hour it will re-check to make sure the certificate is issued and valid, and if not will re-issue.

You can view the logs of each container to get an idea of what they are doing and I encourage you to take a look.

Troubleshooting

  • If you are using a custom bridge network for the containers that make up your attestation service, you will need to start the nginx-proxy and letsencrypt helper on the same network.
  • You might need to bounce your containers if the order you run them in is mixed up at some point.

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment