Many sites have a requirement to use an enterprise-wide certificate authority. They either have a "real" signing cert that chains to a public root CA or an internal root (usually air-gapped) which only signs issuing CA certificates, one per PKI application.
Puppet does not have a currently supported configuration which fits into this model. The existing documentation describes using an "external CA" instead of Puppet's internally generated CA (which is a combined self-signed Root and issuing CA in one), but requires that the user turn off Puppet's issuance code and leaves the whole certificate generation and distribution workflow as an "exercise to the reader".
The procedure in this document describes a supportable configuration which bridges the gap between these two positions: it is possible to use Puppet's internal signing code to issue certificates from an intermediate CA cert which was externally generated and signed. There are a couple of caveats to the configuration which I've tried to call out at the start, but this should present a substantial improvement over the previous situation.
This assumes you have a x509 certificate and RSA keypair which are:
-
Issued by a Root certificate authority (we have not tested 3-level "EV" or cross-signed certs)
-
Valid for signing, meaning they have the following x509v3 extensions:
X509v3 extensions: X509v3 Key Usage: critical Digital Signature, Certificate Sign, CRL Sign
You will also need the certificate of your Root CA (not its private key, though!)
These instructions assume a modern Puppet / Puppet Enterprise infrastructure. Tested configuration is PE2016.1.2, using Puppet 4.4 and Puppet Server 2.3.
There are two general security-related caveats to this configuration, both of which may be addressed by future Puppet releases.
-
Due to a limitation in the agent (tracked at PUP-3788), when using a chained certificate the agent errors trying to validate the CRL (certificate revocation list); it does not load a CRL chain properly but requires a CRL from each CA in the chain, and therefore the connection does not complete. You will a series of errors containing the message:
SSL_connect returned=1 errno=0 state=error: certificate verify failed: [unable to get certificate CRL for CA DN]
The workaround for this is to use the --no-certificate_revocation command line option, or set certificate_revocation = false in your agent's puppet.conf, inside the [main] section of config. Be aware of the implications: this means that your master certificates can not be revoked in a way that causes agents to refuse talking to them. Agent certificates can still be revoked as normal, and masters will correctly prevent revoked agents from connecting.
- You will need to distribute the CA bundle out-of-band, ideally by pre-seeding it onto your agents operating system images or early in the post-installation phase, before puppet runs. See the CA Bundle section below for more information. This is because the file on the CA's filesystem which is returned when the agent requests
/v1/certificate/ca(in order to bootstrap trust) is also the certificate used for signing, and the CA code requires a single cert in that file, not a bundle. This bug is tracked in SERVER-1315.
Generating the Signing CA cert is outside the scope of the doc, since it will depend on the exact type of certificate authority you're using. For this installation, we used Windows 2012 Server Certificate Services with the "SubCA" profile. We then exported a PFX (pkcs12) bundle and transferred it to the puppetmaster, so the root CA, signing private key, and signing certificate would be included. We extracted the key and certs, then stripped the passphrase off the signing key so it would match the expectations of the Puppet CA:
cd /etc/puppetlabs/puppet/ssl/ca
openssl pkcs12 -in signing_bundle.pfx -nocerts -out protected_ca_key.pem -nodes
openssl pkcs12 -in signing_bundle.pfx -nokeys -out ca_crt.pem
openssl rsa -in protected_ca_key.pem -out ca_key.pem
openssl pkcs12 -cacerts -in signing_bundle.pfx -out root_crt.pem
chown pe-puppet ca*; chmod 600 ca*;
We then placed a CA bundle in the appropriate place for the agent and service certificate subsystems on the host:
# from /etc/puppetlabs/puppet/ssl/ca
cat ca_crt.pem root_crt.pem > ../certs/ca.pem
The first certificate to generate is the one for the host itself; this is used both for the puppetserver service as well as the puppet agent running on the master. It therefore needs to be crafted with correct dns_alt_names in order to answer incoming requests; the exact list of dns_alt_names you require will vary -- essentially, whatever the server value is set to on the agents needs to be in the server's certificate or you will get a "Server hostname 'x' did not match server certificate" error.
puppet cert generate puppetserver.my.domain.net --dns_alt_names=puppetserver,puppet
At this point you should be able to start up the puppetserver and run puppet agent -tv --no-certificate_revocation --server puppetserver. The next steps will depend on how much additional infrastructure you have, but the goal is to re-key all of the server components which use SSL to use the new Signing CA. The instructions for Puppet Enterprise is at "Regenerating Certificates for Puppet Enterprise" doc; this should be a superset of the steps needed for an open-source deployment.
As we mentioned in the Caveats section, you'll need to put the CA certs on the agents through some out-of-band mechanism due to a conflict between the agent and server's expectation for the contents of that file. By default, the Puppet agent will try to download the CA certificate from a URL composed of https://$ca_server:$ca_port/v1/certificate/ca and save the response to $ssldir/certs/ca.pem. If there is a pre-existing file in that location, it will not attempt the download, so curling it from a non-Puppet-managed location prior to running the agent would suffice. Alternately, if you have SSL termination on a load balancer in front of the puppetserver you could re-direct that URL to the bundle instead of the actual CA file. Assuming it's in place, an agent run with --no-certificate_revocation should succeed.