Skip to content

Instantly share code, notes, and snippets.

@kennwhite
Last active March 18, 2016 16:25
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kennwhite/aa74c164bcdf092a7a10 to your computer and use it in GitHub Desktop.
Save kennwhite/aa74c164bcdf092a7a10 to your computer and use it in GitHub Desktop.
Notes from using Letsencrypt with Debian & CentOS/RedHat on GCE and AWS

UPDATE 10-31-2015: I finally got Cent6.x working, skip to bottom.

These are my notes from getting the LetsEncrypt beta client running on Debian and CentOS using Google Compute Engine (GCE) and AWS.

Preface

Please take my impressions and feedback in the spirit intended. Lets Encrypt is an amazing project, and they have made ridiculously fast progress over the past year. The whole point of a beta before General Availability is to work out the kinks from the myriad environments in the real world. After only a couple of hours, I was able to reliably request and receive certificates on Debian and CentOS on GCE and AWS, and achieve an A+ rating with SSLLabs. Woot!

![Image of SSLLabs analysis] (http://i.imgur.com/iiQpIBi.png)

Quick Summary

In my opinion, unless you're really adept at running multiple versions of Python, don't even bother with Debian Wheezy or CentOS/RedHat 6.7 (at least for now). Instead target Jessie (Debian 8.1) or CentOS 7.x., or alternatively, consider one of the alternative LetsEncrypt (hereafter, "LE") clients.

Beware that there is a "staging" API server (the default unless otherwise specified) which returns certs with a bogus "Happy Hacker" root CA. This stung me partly because I didn't appreciate the nauance between the default staging server versus the (closed to beta testers only) production API server, either of which the client tools will happily talk to.

Also, perhaps obvious in retrospect, but if you generate a cert on the staging server, you can't revoke it & reissue on the production server.

As detailed below, one of the side effects of installing the official LE client is that it silently invokes sudo and installs lots of additional development tools and packages (gcc, etc.). For many use cases, this might prove too risky for production deployments, and the "manual" certificate fetcher might be preferred, possibly as a central server, which pushes keys and certs to web and app servers.

Finally, it took me many, many false starts debugging cryptic python stack traces to realize that although I was an approved beta tester with "whitelisted" domains example•com, www•example•com, and example•org, I was not whitelisted for blog•example•com, and any attempts to issue certs (on the production server) returned an unpleasant trace including this line:

An unexpected error occurred. Error: unauthorized :: The client lacks sufficient authorization :: Error creating new authz :: Name is not whitelisted

The Rabbit Hole to the Promised Land

The core OS Python SSL libraries require at least version 2.7.9/3.0 or higher to properly verify encrypted peer connections, and although there are lots of suggested fixes, for me anyway, they never resolved fatal crashes with traces like this:

InsecurePlatformWarning Updating letsencrypt and virtual environment dependencies..... /home/me/.local/share/letsencrypt/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning

Curious readers with any passing interest in security will likely find that page amusing, because ultimately, it suggests fixes which include either changing core https url libraries in your code (not really an attractive option for many of us), or simply disabling the warnings.

On CentOS 6.7 and Debian Wheezy, no amount of Stack Overflow or community advice, eg:

http://stackoverflow.com/a/29202163
http://stackoverflow.com/a/29099439
https://community.letsencrypt.org/t/error-while-executing-instructions-mentioned-in-beta-invite-letsencrypt-auto/2189
certbot/certbot#1046

could solve the problem. So for these systems, you have a couple of options. Either run the client from a different (newer OS) machine (and copy/paste or push keys & certs), dig into the native LE code, or possibly consider one of the unofficial clients. These two alternative client (really, component library) projects (a Ruby gem and a Go client) had pretty good feedback and might be worth a look:

https://github.com/unixcharles/acme-client
https://github.com/xenolf/lego

Update: Just tried the brilliant [@diafygi] (https://twitter.com/diafygi) nosudo client, and I highly recomment it:
https://github.com/diafygi/letsencrypt-nosudo

What I really like about it is it's a single script, and is dead-simple to use. One note though, it's not clear what version of Python is needed, but 2.6 seems too early and barfed with fatal errors about unknown standard library calls. CentOS 7 worked fine (with the default Python 2.7.5).

Full Monty

So, here are my notes in all their glory (repeated several times with clean VMs to verify behavior) for Debian and CentOS. Hope they're useful.


Must-read (latest on the beta program):

https://community.letsencrypt.org/t/beta-program-announcements/

Recommended Nginx config:

https://community.letsencrypt.org/t/nginx-configuration-sample/2173

Note: I was able to get an A+ score from SSL Labs, with broad browser/client support, but I never could eliminate the "Extra Download" needed on the certificate trust chain. It looks like I'm in good company though: https://www.ssllabs.com/ssltest/analyze.html?d=helloworld.letsencrypt.org

Random land mines

The beta invitations stated that there is a per-IP and a per-domain rate limit for client calls. I never hit that, so I almost think might be only for actually-issued certs (versus half-executed handshakes). There were some reports of possible caching related issues, and occassional needs to execute the same request multiple times to get success. Honestly, I'm not sure the production & staging servers are really to blame, or if it's a side effect of weird native python client behavior.

Just to be sure, current system status is here: http://letsencrypt.status.io/
And a status twitter account is here: https://twitter.com/letsencrypt_ops

Howto: Debian Jessie on Google Compute Engine

Launch a micro VM on GCE with Debian Jessie 8.1, and allow http and https traffic:

gcloud compute instances create debian-jessie-test-vm \
 --machine-type "f1-micro" \
 --tags "http-server","https-server" \
 --image "https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/debian-8-jessie-v20150929" \
 --boot-disk-size "10" \
 --boot-disk-type "pd-ssd"

gcloud compute firewall-rules create "default-allow-http" \
  --allow tcp:80 \
  --network "default" \
  --source-ranges "0.0.0.0/0" \
  --target-tags "http-server"

gcloud compute firewall-rules create "default-allow-https" \
  --allow tcp:443 \
  --network "default" \
  --source-ranges "0.0.0.0/0" \
  --target-tags "https-server"

gcloud compute ssh debian-jessie-test-vm

# *NOT* as root user
$ sudo apt-get -qq update
$ sudo apt-get install git
$ git clone https://github.com/letsencrypt/letsencrypt
$ cd letsencrypt

./letsencrypt-auto \
 --agree-dev-preview \
 --server \
 https://acme-v01.api.letsencrypt.org/directory \
 -d example.com \
 auth

I recommend the Manual option, as I run nginx and the client is very much a work in progress for now. Also, it might be easier to take the second option and run the temporary python script as root on the destination box to verify the domain versus dealing with a json header and a text file.

When completed, your certificates and keys will be located:

ls -lh /etc/letsencrypt/live/[your-domain]

Howto: CentOS/RedHat 7 on Google Compute Engine

gcloud compute instances create "cent-7-1x-test-vm" \
--machine-type "f1-micro" \
--tags "http-server","https-server" \
--image "https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-7-v20150929" \
--boot-disk-size "10" \
--boot-disk-type "pd-ssd" \
--boot-disk-device-name "cent-7-1x-test-vm"

gcloud compute firewall-rules create "default-allow-http" \
  --allow tcp:80 \
  --network "default" \
  --source-ranges "0.0.0.0/0" \
  --target-tags "http-server"

gcloud compute firewall-rules create "default-allow-https" \
  --allow tcp:443 \
  --network "default" \
  --source-ranges "0.0.0.0/0" \
  --target-tags "https-server"

gcloud compute ssh cent-7-1x-test-vm

$ sudo yum update -y -q
$ sudo yum install git

# *Not* as root!
$ git clone https://github.com/letsencrypt/letsencrypt
$ cd letsencrypt

# *Not* as root!
./letsencrypt-auto \
 --agree-dev-preview \
 --server \
 https://acme-v01.api.letsencrypt.org/directory \
 -d example.com \
 auth

Footnotes

Debian packages added from stock Jessie by LE client:

# grep install /var/log/dpkg.log | tail

augeas-lenses
binutils
cpp
cpp-4.9
dh-python
dialog
gcc
gcc-4.9
git-core
libasan1
libatomic1
libaugeas0
libc-dev-bin
libc6-dev
libcilkrts5
libcloog-isl4
libexpat1-dev
libffi-dev
libgcc-4.9-dev
libgomp1
libisl10
libitm1
liblsan0
libmpc3
libmpdec2
libmpfr4
libpython-dev
libpython2.7
libpython2.7-dev
libpython3-stdlib
libpython3.4-minimal
libpython3.4-stdlib
libquadmath0
libssl-dev
libtsan0
libubsan0
linux-libc-dev
python-chardet-whl
python-colorama-whl
python-dev
python-distlib-whl
python-html5lib-whl
python-pip-whl
python-requests-whl
python-setuptools-whl
python-six-whl
python-urllib3-whl
python-virtualenv
python2.7-dev
python3
python3-minimal
python3-pkg-resources
python3-virtualenv
python3.4
python3.4-minimal
virtualenv
zlib1g-dev

CentOS/RedHat 7 packages added to stock minimal distro by LE client:

# rpm -qa --qf '%{INSTALLTIME} %-40{NAME} %{INSTALLTIME:date}\n' | sort -n | cut -d' ' -f2- 

augeas-libs
cpp
dialog
gcc
glibc-devel
glibc-headers
kernel-headers
keyutils-libs-devel
krb5-devel
libcom_err-devel
libffi-devel
libmpc
libselinux-devel
libsepol-devel
libverto-devel
mpfr
openssl-devel
pcre-devel
python-devel
python-virtualenv
zlib-devel

Update: I managed to get the client working with CentOS 6.7 reliably (might want to spin up a new VM)

Thanks to Eva2000 at centminmod for pointing me in the right direction

# CentOS 6.x

sudo yum -y -q update

# install non-system parallel version

sudo yum -y -q install centos-release-SCL
sudo yum -y -q install epel-release
sudo yum -y -q install python-pip
sudo pip install --upgrade pip
sudo pip install virtualenv
sudo yum -y -q install git python27-devel python27-pip python27-setuptools python27-tools python27-virtualenv


# Note the leading space (run shell in current env)
. /opt/rh/python27/enable
sudo su -c "echo $LD_LIBRARY_PATH > /etc/ld.so.conf.d/letsencrypt_tmp.conf"
sudo ldconfig


git clone https://github.com/letsencrypt/letsencrypt

cd letsencrypt

# modify generic Python v2 refs to 2.7-specific

sed -i 's/--python python2/--python python2.7/g' letsencrypt-auto

./letsencrypt-auto \
--agree-dev-preview \
--server https://acme-v01.api.letsencrypt.org/directory \
-d www.example.com \
--authenticator manual \
certonly

# You can ignore the InsecurePlatformWarning
# (which will be followed by a minute or so of pause before another warning
# and then the console GUI comes up)
#

# You'll get a big echo dump of commands, choose the second option and run that on
# the receiving www server getting the cert (as root; make sure to disable/stop apache or nginx temporarily to avoid port conflicts)
#
# Once the stand-alone python is running on the www server getting the cert,
# switch back to the first terminal and hit Enter
#
# And fin.

sudo su -
ls -lh /etc/letsencrypt/live/[www.example.com]

# Enjoy your certs
@darKoram
Copy link

One way to avoid the os issues if you are already running docker is to containerize letsencrypt. eg. https://github.com/smashwilson/lets-nginx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment