someburner/le2018.md

## le2018.md

      
    Raw
  

              le2018.md
            
          
    Let's Encrypt for Nginx on Ubuntu 17.10, 18.04 with wildcards

Guide info:

we're setting up mydomain.com, *.mydomain.com
HTML is served from /var/www/mydomain
Challenges are served from /var/www/letsencrypt.
As of this writing, SSL Labs gives it an A+
Took info from here and here on wilcard certs.

Auto-renewal

This guide will not result in a "never have to look it again" configuration, at least, not without some extra work. This announcement states:

Additionally, wildcard domains must be validated using the DNS-01 challenge type. This means that you’ll need to modify DNS TXT records in order to demonstrate control over a domain for the purpose of obtaining a wildcard certificate.

The DNS-01 challenge is what is shown in this guide, but this means renewal also requires this challenge (in addition to issuance). Fortunately, many registrars now have APIs available to made record edits. It seems that lexicon is almost a one-stop shop for doing this and is written specifically for LE.
Prerequisites


The following are required:
A record: mydomain.com
And that's it. For better or for worse LE supports wildcards now so best be usin' it.
Download the file and run the following, then open in a markdown viewer

## dont forget to escape the dots
sed -i -e "s|mydomain\.com|YOURDOMAINDOTCOM|g" le2018.md
sed -i -e "s|mydomain|YOURDOMAIN|g" le2018.md
sed -i -e "s|myemail|YOUREMAIL|g" le2018.md
Also, install certbot if you haven't yet. NOTE: wildcard certs depend on having
at least v0.22.x. For Ubuntu 17.10, I last tested with 0.22.2. For Ubuntu 18.04, I last tested with 0.23.0.
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install certbot


Nginx snippets

First we create two snippets (to avoid duplicating code in every virtual host configuration).
Create a file /etc/nginx/snippets/letsencrypt.conf containing:
location ^~ /.well-known/acme-challenge/ {
	default_type "text/plain";
	root /var/www/letsencrypt;
}

Create a file /etc/nginx/snippets/ssl.conf containing:
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;

ssl_protocols TLSv1.2;
ssl_ciphers EECDH+AESGCM:EECDH+AES;
ssl_ecdh_curve secp384r1;
ssl_prefer_server_ciphers on;

ssl_stapling on;
ssl_stapling_verify on;

add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;

Create the folder for the challenges:
mkdir -p /var/www/letsencrypt/.well-known/acme-challenge


Nginx virtual hosts (HTTP-only)

We don't have a certificate yet at this point, so the domain will be served only as HTTP.
Create a file /etc/nginx/sites-available/mydomain.conf containing:
server {
	listen 80 default_server;
	listen [::]:80 default_server ipv6only=on;
	server_name mydomain.com www.mydomain.com;

	include /etc/nginx/snippets/letsencrypt.conf;

	root /var/www/mydomain;
	index index.html;
	location / {
		try_files $uri $uri/ =404;
	}
}

Enable the site:
rm -f /etc/nginx/sites-enabled/default
ln -s /etc/nginx/sites-available/mydomain.conf /etc/nginx/sites-enabled/mydomain.conf
Make sure nginx likes it:
nginx -t
And reload Nginx:
systemctl reload nginx


Get the certificate

For wildcard cert, you must specify LE's staging server as below.
This requirement may go away in the future. Also, not sure if this can be
done automatically or if adding the DNS is required for wildcard. Either way,
1-and-done for all future subdomains is worth the extra time.
Note: The flag --no-eff-email opts out of signing up for the EFF mailing list,
remove the flag if you'd like to signup.
## acme v2 (almost final)
certbot certonly --agree-tos --no-eff-email --email myemail --server https://acme-v02.api.letsencrypt.org/directory --manual -d *.mydomain.com -d mydomain.com
Should give you:

Please deploy a DNS TXT record under the name _acme-challenge.mydomain.com with the following value:


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


Before continuing, verify the record is deployed.


Step 1 - Create the DNS record

Using gandi.net as an example:

go to records section for mydomain.com
select add record
Type: TXT
TTL: 300 seconds
Name: _acme-challenge
Text value: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Step 2 - Verify it

Press enter once you've verified the following;
nslookup -type=TXT _acme-challenge.mydomain.com
Should give you:

nslookup -type=TXT _acme-challenge.mydomain.com
Server:		127.0.0.53
Address:	127.0.0.53#53


Non-authoritative answer:
_acme-challenge.mydomain.com	text = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"


Step 3

Certbot should then ask you to add the challenge file:

Create a file containing just this data:


aaaaaaaaaaaaaaaaaaaaaaaaaaa-bbbbbbbbbbbbbbb.ccccccccccccccccccccccccccccccccccccccccccc


And make it available on your web server at this URL:


http://mydomain.com/.well-known/acme-challenge/aaaaaaaaaaaaaaaaaaaaaaaaaaa-bbbbbbbbbbbbbbb

Do what it says: (replace the a's b's and c's from above)
_cfile="aaaaaaaaaaaaaaaaaaaaaaaaaaa-bbbbbbbbbbbbbbb"
_cdata="aaaaaaaaaaaaaaaaaaaaaaaaaaa-bbbbbbbbbbbbbbb.ccccccccccccccccccccccccccccccccccccccccccc"
cd /var/www/letsencrypt/.well-known/acme-challenge
touch $_cfile
echo "$_cdata" > $_cfile

# Now verify
wget http://mydomain.com/.well-known/acme-challenge/aaaaaaaaaaaaaaaaaaaaaaaaaaa-bbbbbbbbbbbbbbb


Step 4 - Results

Press enter, and you should be presented with this:

Waiting for verification...
Cleaning up challenges


IMPORTANT NOTES:


Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/mydomain.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/mydomain.com/privkey.pem
Your cert will expire on 2018-06-20. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew all of your certificates, run
"certbot renew"
Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.

It will save the file in /etc/letsencrypt/live/mydomain.com/.
To see the wildcard cert in action, go to ssllabs
and enter in some arbitrary subdomain, eg: testabc.mydomain.com and you should
see the cert result show up for *.mydomain.com. Cool beans!


Nginx virtual hosts (HTTPS-only)

Now that you have a certificate for the domains, switch to HTTPS by editing the
file /etc/nginx/sites-available/mydomain.conf and replacing contents with:
## http://mydomain.com redirects to https://mydomain.com
server {
	listen 80;
	listen [::]:80;
	server_name mydomain.com;

	include /etc/nginx/snippets/letsencrypt.conf;

	location / {
		return 301 https://mydomain.com$request_uri;
	}
}

## http://www.mydomain.com redirects to https://www.mydomain.com
server {
	listen 80 default_server;
	listen [::]:80 default_server ipv6only=on;
	server_name www.mydomain.com;

	include /etc/nginx/snippets/letsencrypt.conf;

	location / {
		return 301 https://www.mydomain.com$request_uri;
	}
}

## https://mydomain.com redirects to https://www.mydomain.com
server {
	listen 443 ssl http2;
	listen [::]:443 ssl http2;
	server_name mydomain.com;

	ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem;
	ssl_trusted_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
	include /etc/nginx/snippets/ssl.conf;

	location / {
		return 301 https://www.mydomain.com$request_uri;
	}
}

## Serves https://www.mydomain.com
server {
	server_name www.mydomain.com;
	listen 443 ssl http2 default_server;
	listen [::]:443 ssl http2 default_server ipv6only=on;

	ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem;
	ssl_trusted_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
	include /etc/nginx/snippets/ssl.conf;

	root /var/www/mydomain;
	index index.html;
	location / {
		try_files $uri $uri/ =404;
	}
}

Test:
nginx -t
Then reload Nginx:
systemctl reload nginx
Note that http://mydomain.com redirects to https://mydomain.com (which
redirects to https://www.mydomain.com) because redirecting to
https://www.mydomain.com directly would be incompatible with HSTS.


Nginx proxy-pass (HTTPS for some non-HTTPS server on port 1234)

The wildcard cert can now be used with nginx to cover https for all your
non-HTTPS services. For this example, lets say you have a service running on
port 1234.
This assumes you have set up an A record pointing to the same IP address, so
set that up if you haven't already.
Create a file /etc/nginx/sites-available/myservice.conf with the following:
server {
	listen 80 http2;
	listen [::]:80 http2;
	server_name myservice.mydomain.com;

	include /etc/nginx/snippets/letsencrypt.conf;

	rewrite ^ https://$server_name$request_uri? permanent;
}

server {
	listen 443 ssl http2;
	listen [::]:443 ssl http2;

	include /etc/nginx/snippets/ssl.conf;

	server_name myservice.mydomain.com;

	location / {
		proxy_pass http://127.0.0.1:1234;
		proxy_set_header Host $host;
		proxy_ignore_client_abort on;
	}

	location ~ /.well-known {  allow all; }

	ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem;
}

Enable the site:
ln -s /etc/nginx/sites-available/myservice.conf /etc/nginx/sites-enabled/myservice.conf
Test:
nginx -t
Then reload Nginx:
systemctl reload nginx
Now browse to https://myservice.mydomain.com and viola - https.


Automatic renewal using Cron

Certbot can not renew certs without some help. See top section again for why. This will fail asking for manual hooks:
certbot renew --dry-run


WIP-ish: lexicon setup

See lexicon to see if your provider will work. If not you're on on your own, sorry.
Otherwise, install it --> dns-lexicon <--:
# NOT lexicon :)
pip3 install dns-lexicon
Go to your registrars website and get the necessary API key. Before doing anything further, you should write a test script to make sure lexicon works with your registrar.
In my case Gandi had a new API scheme and I had to manually add a new file (in this case I called it gandi2.py) that was an unmerged PR. If you find yourself in a similar situation, here are some tips:

the location to add/edit provider python scripts is /usr/local/lib/python3.x/dist-packages/lexicon/providers/
Replace python3.x with whatever pip3 is linked to
or just do find /usr -name 'lexicon' to see where it's at

Now on to writing a little test script. For example, test-dns-auth.sh:
#!/bin/bash

# This are just used for this test script to make it clear that these should be changed.
_PROVIDER="gandi2"
_DOMAIN="mydomain.com"

# note the naming convention:
# In this case "gandi2" will result in lexicon using gandi2.py, and accepting
# _GANDI2_ for env. variables. See lexicon docs for more examples/why.
export LEXICON_GANDI2_USERNAME="myusername"
export LEXICON_GANDI2_TOKEN="xxxxxxxxxxxxxxxxxxxxxxxx"

lexicon $_PROVIDER list $1 TXT
res=$?;
echo "lexicon got res=$res";
Now do:
chmod +x test-dns-auth.sh
./test-dns-auth.sh
If it works, you should get a printout of a python array of dicts, and at the end of the output you should see the dns challenge from earlier. Mine looked something like this (formatted here for readability).
list_records: 
[{
    'type': 'TXT',
    'name': '_acme-challenge.mydomain.com.',
    'ttl': 300,
    'content': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'id': '_acme-challenge'
}]

If you have deleted the DNS records (as you should after renewal), you should see an empty array:
list_records: []

Please consult lexicon or your provider if you're having trouble here. Do not proceed until you have the above working.


WIP-ish: Lexicon renewal

Now copy the boilerplate certbot hook script.
cd /etc/letsencrypt
wget 'https://github.com/AnalogJ/lexicon/blob/master/examples/certbot.default.sh'
chmod +x certbot.default.sh

# edit PROVIDER and PROVIDER_CREDENTIALS
nano certbot.default.sh
Again, this is only the boilerplate. But it's pretty well self-documented. So use creds from the test and enter them in the boilerplate script.
NOTE: I was about up for renewal and the above script combined with the script below appears to have worked for me- that is, I was able to renew my certs without manually adding/editing the DNS records myself. For now, I am okay with manually logging in and running the recert script every 2 months and just making sure it goes smoothly. Feel free to share your experiences in comments.
Create a new file le-manual-renew.sh:
#!/bin/bash

certbot certonly \
--manual \
--preferred-challenges "dns" \
--server "https://acme-v02.api.letsencrypt.org/directory" \
--manual-auth-hook "/etc/letsencrypt/certbot.default.sh auth" \
--manual-cleanup-hook "/etc/letsencrypt/certbot.default.sh cleanup" \
-d *.mydomain.com -d mydomain.com
Then:
chmod +x le-manual-renew.sh
# and when you are up for renewal, try it out:
./le-manual-renew.sh

# Note that this is "manual mode"
# You will be presented with 1-2 manual things you need to enter:
# See below
Example for what I entered using manual mode:
... skipping to important stuff (you may or may not be presented with this 1st notice) ...

Cert not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/mydomain.com.conf)

What would you like to do?
-------------------------------------------------------------------------------
1: Keep the existing certificate for now
2: Renew & replace the cert (limit ~5 per 7 days)
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then [enter] 
(press 'c' to cancel):                                          2 <-- 2 [enter]


... then a bit later ...


-------------------------------------------------------------------------------
NOTE: The IP of this machine will be publicly logged as having requested this
certificate. If you're running certbot in manual mode on a machine that is not
your server, please ensure you're okay with that.

Are you OK with your IP being logged?
-------------------------------------------------------------------------------
(Y)es/(N)o: Y                                                     <-- Y [enter]


... you should see some stuff and then ...


create_record: True
Delaying for 30 seconds..


... and more stuff and then ...


delete_record: True


IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:

... and there you go

For me, this has worked on 2 servers w/ 2 different domain names on Ubuntu 17.10 and 18.04.


Won't fix: Cron

You only need to renew your certs once every couple months (or less perhaps?). So either:

A.) You have just a couple small sites, and renewing them should be no big deal and will probably save you time in the long run should something change in either lexicon, certbot, or the LE renewall process.
B.) You run a ton of websites, and reallly need auto-renew, you should be competent enough to write a script to do that. It's not that hard, and trying to do that generically seems counter to the direction LE wants to go with this.


Conclusion

Congratulations, you should now be able to see your website at https://*.mydomain.com.
You can now also test that your domain has A SLL rating:

https://www.ssllabs.com/ssltest/analyze.html?d=mydomain.com

I would also recommend setting up content-specific features like
Content Security Policy and Subresource Integrity:

Mozilla Observatory: submit a domain to get content-specific advices
Mozilla Security Guidelines

If Let's Encrypt is useful to you, and you're not broke, consider donating to Let's Encrypt or donating to the EFF.
end