Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

OpenShift Named Certificates

This is a short guide explaining how to deploy and manage custom certificates via openshift-ansible. These custom certificates will be served for public facing UI/API.

1. Set up

NOTE: I'm running ansible from the first master so I'm creating the certificates in /root/ on my first master. The certificates will be copied to the masters (within /etc/origin/master/named_certificates/) by ansible. I could create them on my local system and run ansible from my local system instead.

1.1 Create CA Certificate

openssl genrsa -out rootCA.key 2048
openssl req -x509 -new -nodes -key rootCA.key -days 1024 -subj '/C=US/ST=North Carolina/L=Raleigh/O=Red Hat, Inc./OU=OpenShift/CN=public-example.com/emailAddress=none@public-example.com/' -out rootCA.pem

1.2 Create master.public-example.com.crt

openssl genrsa -out master.public-example.com.key 2048
openssl req -new -key master.public-example.com.key -subj '/C=US/ST=North Carolina/L=Raleigh/O=Red Hat, Inc./OU=OpenShift/CN=master.public-example.com/emailAddress=none@public-example.com/' -out master.public-example.com.csr
openssl x509 -req -in master.public-example.com.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out master.public-example.com.crt -days 500

1.3 Create wildcard-flibberty-jibbet.com.crt

openssl genrsa -out wildcard-flibberty-jibbet.com.key 2048
openssl req -new -key wildcard-flibberty-jibbet.com.key -subj '/C=US/ST=North Carolina/L=Raleigh/O=Red Hat, Inc./OU=OpenShift/CN=*.flibberty-jibbet.com/emailAddress=none@flibberty-jibbet.comm/' -out wildcard-flibberty-jibbet.com.csr
openssl x509 -req -in wildcard-flibberty-jibbet.com.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out wildcard-flibberty-jibbet.com.crt -days 500

1.4 Modify /etc/hosts

192.168.122.224 master.public-example.com master.flibberty-jibbet.com

2. Install using master.public-example.com certificate

Create an HA OpenShift cluster and specify the openshift_master_named_certificates inventory variable. Modify paths to match the certificate paths on the system where you will be running ansible and where the certificates were created.

openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]

2.1 Verify Configuration for master.public-example.com

2.1.1 Ensure certificates exist in /etc/origin/master/named_certificates/

# ls /etc/origin/master/named_certificates/
master.public-example.com.crt  master.public-example.com.key

2.1.2 Verify master configuration

servingInfo:
  bindAddress: 0.0.0.0:8443
  bindNetwork: tcp4
  certFile: master.server.crt
  clientCA: ca.crt
  keyFile: master.server.key
  maxRequestsInFlight: 500
  requestTimeoutSeconds: 3600
  namedCertificates:
  - certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
    keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
    names:
    - "master.public-example.com"

2.1.3 Ensure API serves the default certificate

# curl -vk https://192.168.122.224:8443
* About to connect() to 192.168.122.224 port 8443 (#0)
*   Trying 192.168.122.224...
* Connected to 192.168.122.224 (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
*       subject: CN=172.30.0.1
*       start date: Nov 17 00:53:36 2015 GMT
*       expire date: Nov 16 00:53:37 2017 GMT
*       common name: 172.30.0.1
*       issuer: CN=openshift-signer@1447721613
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.122.224:8443
> Accept: */*

2.1.4 Ensure API serves certificate for master.public-example.com

# curl -vk https://master.public-example.com:8443
* About to connect() to master.public-example.com port 8443 (#0)
*   Trying 192.168.122.224...
* Connected to master.public-example.com (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
*       subject: E=none@public-example.com,CN=master.public-example.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
*       start date: Nov 17 00:22:26 2015 GMT
*       expire date: Mar 31 00:22:26 2017 GMT
*       common name: master.public-example.com
*       issuer: E=none@public-example.com,CN=public-example.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: master.public-example.com:8443
> Accept: */*

3. Re-run ansible with wildcard-flibberty-jibbet.com certificate

When we re-run ansible with different openshift_master_named_certificates the original certificates should remain in place and they should also remain configured.

Modify host inventory, commenting the original openshift_master_named_certificates variable and add a new variable for wildcard-flibberty-jibbet.com.{crt,key}.

#openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]

3.1 Verify Configuration for wildcard-flibberty-jibbet.com

3.1.1 Ensure certificates exist in /etc/origin/master/named_certificates/

Both sets of certificates will exist on the filesystem.

# ls /etc/origin/master/named_certificates/
master.public-example.com.crt  master.public-example.com.key  wildcard-flibberty-jibbet.com.crt  wildcard-flibberty-jibbet.com.key

3.1.2 Verify master configuration

Both certificates will be configured.

servingInfo:
  bindAddress: 0.0.0.0:8443
  bindNetwork: tcp4
  certFile: master.server.crt
  clientCA: ca.crt
  keyFile: master.server.key
  maxRequestsInFlight: 500
  requestTimeoutSeconds: 3600
  namedCertificates:
  - certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
    keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
    names:
    - "master.public-example.com"
  - certFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.crt
    keyFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.key
    names:
    - "*.flibberty-jibbet.com"

3.1.3 Ensure API serves the default certificate

# curl -vk https://192.168.122.224:8443
* About to connect() to 192.168.122.224 port 8443 (#0)
*   Trying 192.168.122.224...
* Connected to 192.168.122.224 (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
*       subject: CN=172.30.0.1
*       start date: Nov 17 00:53:36 2015 GMT
*       expire date: Nov 16 00:53:37 2017 GMT
*       common name: 172.30.0.1
*       issuer: CN=openshift-signer@1447721613
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.122.224:8443
> Accept: */*

3.1.4 Ensure API serves certificate for master.flibberty-jibbet.com

[root@master4 ~]# curl -vk https://master.flibberty-jibbet.com:8443
* About to connect() to master.flibberty-jibbet.com port 8443 (#0)
*   Trying 192.168.122.224...
* Connected to master.flibberty-jibbet.com (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
*       subject: E=none@flibberty-jibbet.comm,CN=*.flibberty-jibbet.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
*       start date: Nov 17 00:22:37 2015 GMT
*       expire date: Mar 31 00:22:37 2017 GMT
*       common name: *.flibberty-jibbet.com
*       issuer: E=none@public-example.com,CN=public-example.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: master.flibberty-jibbet.com:8443
> Accept: */*

4. Re-run ansible with no certificates

If we have previously ran ansible with openshift_master_named_certificates and then re-run with no variable set, the original configuration and certificates should remain in place.

Comment all previous openshift_master_named_certificates inventory variables.

#openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
#openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]

4.1 Verify Configuration with no certificates

4.1.1 Ensure certificates exist in /etc/origin/master/named_certificates/

Both sets of certificates will exist on the filesystem.

# ls /etc/origin/master/named_certificates/
master.public-example.com.crt  master.public-example.com.key  wildcard-flibberty-jibbet.com.crt  wildcard-flibberty-jibbet.com.key

4.1.2 Verify master configuration

Both certificates will be configured.

servingInfo:
  bindAddress: 0.0.0.0:8443
  bindNetwork: tcp4
  certFile: master.server.crt
  clientCA: ca.crt
  keyFile: master.server.key
  maxRequestsInFlight: 500
  requestTimeoutSeconds: 3600
  namedCertificates:
  - certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
    keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
    names:
    - "master.public-example.com"
  - certFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.crt
    keyFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.key
    names:
    - "*.flibberty-jibbet.com"

5. Re-run ansible and overwrite configured certificates

Modify host inventory, uncommenting the openshift_master_named_certificate inventory variable for master.public-example.com, commenting the openshift_master_named_certificates inventory variable for wildcard-flibberty-jibbet.com and add openshift_master_overwrite_named_certificates=true.

openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
#openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]
openshift_master_overwrite_named_certificates=true

5.1 Verify Configuration with overwritten certificates

5.1.1 Ensure correct certificates exist in /etc/origin/master/named_certificates/

wildcard-flibbery-jibbet.{crt,key} should be absent.

# ls /etc/origin/master/named_certificates/
master.public-example.com.crt  master.public-example.com.key

5.1.2 Verify master configuration

Similarly, wildcard-flibbery-jibbet.{crt,key} should be absent from master configuration.

servingInfo:
  bindAddress: 0.0.0.0:8443
  bindNetwork: tcp4
  certFile: master.server.crt
  clientCA: ca.crt
  keyFile: master.server.key
  maxRequestsInFlight: 500
  requestTimeoutSeconds: 3600
  namedCertificates:
  - certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
    keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
    names:
    - "master.public-example.com"

6. Re-run ansible and clear certificates

Modify host inventory, setting openshift_master_overwrite_named_certificates=true. If this variable is set and no openshift_master_named_certificates are configured, all certificates and configuration will be cleared.

#openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]                                                                       
#openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]                                                               
openshift_master_overwrite_named_certificates=true

6.1 Verify Configuration with cleared certificates

.1.1 Ensure no certificates exist in /etc/origin/master/named_certificates/

The named_certificates directory should be absent.

# ls /etc/origin/master/named_certificates/
ls: cannot access /etc/origin/master/named_certificates/: No such file or directory

6.1.2 Verify master configuration

No certificates will be configured.

servingInfo:
  bindAddress: 0.0.0.0:8443
  bindNetwork: tcp4
  certFile: master.server.crt
  clientCA: ca.crt
  keyFile: master.server.key
  maxRequestsInFlight: 500
  requestTimeoutSeconds: 3600
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.