In order to be issued a certificate, Let's Encrypt needs to verify you have control over the domain. This process is automated through challenges, such as setting up a local webserver, or adding DNS TXT records. I don't like the DNS-TXT challenge, as it requires giving every host a DNS service account.
I came up with my own solution. The .well-known/acme-challenge can be hosted using S3, allowing Let's Encrypt to perform an HTTP-01 challenge.
This method requires the least amount of privileges on the host (no ports need to be open)
The primary reason I want internal hosts to have valid certs is quite simple - invalid/untrusted certificates break things! A secondary reason is that constant browser warning are annoying and false alarms are detrimental to security.
In public DNS, my intranet domains point to S3 via a CNAME. Each intranet domain has its own bucket.(I can use this to log external queries & other things). At the end, I will demonstrate integration with Cloudflare, so origin hosts can obtain Let's Encrypt certs, without having to create additional DNS records.
Place the following in /etc/letsencrypt/s3-auth.sh
##########################
# Upload the token to S3 #
##########################
if [ -z "$CERTBOT_S3_ROOT" ]; then
CERTBOT_S3_ROOT="$CERTBOT_DOMAIN/.well-known/acme-challenge"
fi
echo $CERTBOT_VALIDATION | s3cmd put -P - s3://$CERTBOT_S3_ROOT/$CERTBOT_TOKEN
Place the following in /etc/letsencrypt/s3-cleanup.sh
##########################
# Delete the token on S3 #
##########################
# /etc/letsencrypt/s3-cleanup.sh
if [ -z "$CERTBOT_S3_ROOT" ]; then
CERTBOT_S3_ROOT="$CERTBOT_DOMAIN/.well-known/acme-challenge/"
fi
s3cmd rm s3://$CERTBOT_S3_ROOT/$CERTBOT_TOKEN
sudo certbot certonly \
--preferred-challenges=http \
--manual-auth-hook=/etc/letsencrypt/s3-auth.sh \
--manual-cleanup-hook=/etc/letsencrypt/s3-cleanup.sh \
--manual-public-ip-logging-ok \
--manual --agree-tos --no-eff-email \
--staging -m user@etherarp.net -d cert-example.etherarp.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for cert-example.etherarp.net
Output from s3-auth.sh:
upload: '<stdin>' -> 's3://etherarp.net/.well-known/acme-challenge/SQqXyzU6TDrQM_WDsv9Bn7Zjt1OEH39bW1oILkhNLf4' [part 1, 88B]
88 of 88 100% in 0s 196.20 kB/s
88 of 88 100% in 0s 175.78 B/s done
Public URL of the object is: http://etherarp.net.s3.amazonaws.com/.well-known/acme-challenge/SQqXyzU6TDrQM_WDsv9Bn7Zjt1OEH39bW1oILkhNLf4
Waiting for verification...
Cleaning up challenges
Output from s3-cleanup.sh:
delete: 's3://etherarp.net/.well-known/acme-challenge/SQqXyzU6TDrQM_WDsv9Bn7Zjt1OEH39bW1oILkhNLf4'
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/cert-example.etherarp.net-0001/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/cert-example.etherarp.net-0001/privkey.pem
Your cert will expire on 2019-08-25. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
"certbot renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
To run without root privileges, add the following flags
--work-dir=$PWD --logs-dir=$PWD --config-dir=$PWD
Next, let's create an IAM policy, allowing read-write access to well-known/acme-challenge/ across all buckets.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowCertbot",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectACL",
"s3:DeleteObject"
],
"Condition": {
"IpAddress": {
"aws:SourceIp": "0.0.0.0/0"
}
},
"Resource": [
"arn:aws:s3:::*/.well-known/acme-challenge/*"
]
}
]
}
IAM policies should never be attached directly to a user. The policy is attached to a group and the group is attached to a user.
The Let's Encrypt http-01 challenge can follow HTTP redirects (limit 10). This means the primary web server can route /.well-known/acme-challenge to another endpoint.
In the previous example, the DNS record pointed directly to S3. In this example, the DNS record points to Cloudflare.
In order to route the Let's Encrypt verification to the approrpriate S3 endpoint, a page rule needs to be configured in cloudflare.
Now let's try it out
$ certbot certonly --preferred-challenges=http-01 --manual --manual-auth-hook=/etc/letsencrypt/s3-auth.sh --manual-public-ip-logging-ok --staging -d etherarp.net --work-dir=$PWD --logs-dir=$PWD --config-dir=$PWD -m rohan@etherarp.net --agree-tos --no-eff-email
Saving debug log to /home/rohan/le2/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for etherarp.net
Output from s3-auth.sh:
upload: '<stdin>' -> 's3://etherarp.net/.well-known/acme-challenge/gRlXWpRHQ0MzSVQ6g7VXbbGDvnd9PyEk44TrGQjkvAk' [part 1, 88B]
88 of 88 100% in 0s 96.13 kB/s
88 of 88 100% in 0s 316.17 B/s done
Public URL of the object is: http://s3.amazonaws.com/etherarp.net/.well-known/acme-challenge/gRlXWpRHQ0MzSVQ6g7VXbbGDvnd9PyEk44TrGQjkvAk
Waiting for verification...
Cleaning up challenges
Non-standard path(s), might not work with crontab installed by your operating system package manager
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/home/rohan/le2/live/etherarp.net/fullchain.pem
Your key file has been saved at:
/home/rohan/le2/live/etherarp.net/privkey.pem
Your cert will expire on 2019-08-25. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
"certbot renew"
- Your account credentials have been saved in your Certbot
configuration directory at /home/rohan/le2. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
Let's take a look at the logs
{
"identifier": {
"type": "dns",
"value": "etherarp.net"
},
"status": "valid",
"expires": "2019-06-26T11:55:04Z",
"challenges": [
{
"type": "tls-alpn-01",
"status": "pending",
"url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/dsOziM-Xs9qhIDsMUzbts5OCK0dMynq1FtzWP2JVXng/315781756",
"token": "lqRFfaJZa8q-LdlA5YLOLQ5T26-ZBfk7atd-ZwgL6gs"
},
{
"type": "http-01",
"status": "valid",
"url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/dsOziM-Xs9qhIDsMUzbts5OCK0dMynq1FtzWP2JVXng/315781757",
"token": "gRlXWpRHQ0MzSVQ6g7VXbbGDvnd9PyEk44TrGQjkvAk",
"validationRecord": [
{
"url": "http://etherarp.net/.well-known/acme-challenge/gRlXWpRHQ0MzSVQ6g7VXbbGDvnd9PyEk44TrGQjkvAk",
"hostname": "etherarp.net",
"port": "80",
"addressesResolved": [
"104.24.101.85",
"104.24.100.85",
"2606:4700:30::6818:6455",
"2606:4700:30::6818:6555"
],
"addressUsed": "2606:4700:30::6818:6455"
},
{
"url": "http://etherarp.net.s3.amazonaws.com/.well-known/acme-challenge/gRlXWpRHQ0MzSVQ6g7VXbbGDvnd9PyEk44TrGQjkvAk",
"hostname": "etherarp.net.s3.amazonaws.com",
"port": "80",
"addressesResolved": [
"52.216.179.163"
],
"addressUsed": "52.216.179.163"
}
]
}
]
}
We can see the two hops.