Skip to content

Instantly share code, notes, and snippets.

@amcginlay
Last active February 16, 2023 11:42
Show Gist options
  • Save amcginlay/dc7b3422480a11c17e468d9e3455aa49 to your computer and use it in GitHub Desktop.
Save amcginlay/dc7b3422480a11c17e468d9e3455aa49 to your computer and use it in GitHub Desktop.
Adding a new cluster to Enterprise Jetstack Secure

Adding clusters to TLS Protect For Kubernetes (TLSPK)

TLSPK Auth

jsctl auth login
jsctl config set organization <ORG_NAME>    # e.g. gallant-wright
jsctl registry auth output 2>&1 > /dev/null # force an image pull secret to be created as necessary

Create a new cluster (KinD)

Create new KinD cluster.

k8s_cluster_name=kind-$(date +"%y%m%d%H%M")
cat <<EOF | kind create cluster --config -
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: ${k8s_cluster_name}
nodes:
- role: control-plane
EOF

Note: the use of date/time in name to get some clean air for JSS, avoiding a known problem with name re-use (issue here).

Add cluster to TLSPK

The following step installs the agent workload into the jetstack-secure namespace

k8s_cluster_name_tlspk=$(tr "-" "_" <<< ${k8s_cluster_name})
jsctl clusters connect ${k8s_cluster_name_tlspk}

Note: KinD doesn't like '_' whilst TLSPK doesn't like '-' so replace these (issue here).

The cluster will appear shortly in your TLSPK org

Install TLSPK Enterprise components

The current list of TLSPK Components can be researched here

Install TLSPK Operator (js-operator)

The following step installs the js-operator component into the jetstack-secure namespace. This component is responsible for bootstrapping and managing the other TLSPK components.

jsctl operator deploy --auto-registry-credentials

Install other TLSPK components

These components are as follows:

  • cert-manager-approver-policy-enterprise
  • cert-discovery-venafi
  • venafi-oauth-helper

These TLSPK components include features that require TPP integration, so we need to pass in that info.

Set some variables first (you will need vcert installed)

tpp_url='<URL>'                 # e.g. uvo1sou257gbsuv8wdi.env.cloudshare.com (ensure visibility from k8s!)
tpp_zone='<ZONE>'               # e.g. jetstack-secure                        (zone==policy)
tpp_tlspk_username='<USERNAME>' # e.g. jetstack-secure (see **Note** below)
tpp_tlspk_password='<PASSWORD>'
unset tpp_access_token
tpp_access_token=$(
  vcert getcred -u https://${tpp_url} \
  --username ${tpp_tlspk_username} \
  --password ${tpp_tlspk_password} \
  --client-id cert-manager.io \
  --scope certificate:manage,revoke \
  --format json | jq -r .access_token
)

Note: whatever value is used for tpp_tlspk_username should also be set as the "User/Team" against the Jetstack Cert-Manager (alias cert-manager.io) API-Integration in TPP. This API-Integration ships with TPP by default, albeit in an unconfigured state. Without this setting in place both the venafi-oauth-helper deployment and venafi ClusterIssuer will fail.

Check you're happy with the variables.

set | grep ^tpp_

If tpp_access_token is not set it could indicate your TPP instance is offline.

The following step installs the other TLSPK components into the jetstack-secure namespace

jsctl operator installations apply --auto-registry-credentials \
    --tier=enterprise-plus \
    --cert-manager-replicas 1 \
    --venafi-oauth-helper \
    --cert-discovery-venafi \
    --experimental-cert-discovery-venafi-connection cert-discovery-venafi \
    --experimental-venafi-issuers tpp:venafi:venafi \
    --experimental-venafi-connections-config <(cat << EOF
venafi:
  url: "${tpp_url}"
  zone: "${tpp_zone}"
  username: "${tpp_tlspk_username}"
  password: "${tpp_tlspk_password}"
cert-discovery-venafi:
  url: "${tpp_url}"
  zone: "${tpp_zone}"
  access-token: "${tpp_access_token}"
EOF
)

js-operator Certificate Issuers

If you've used cert-manager in the past you will have gotten accustomed to defining Issuer (or ClusterIssuer) manifests. js-operator expects Certificate Issuer definitions to be embedded within its own, single, Installation resource. Furthermore, because js-operator enforces the use of an enterprise-ready approver-policy, each Issuer resource must be paired with a CertificateRequestPolicy resource defining the minimium requirements certificates must meet before they can be forwarded for processing. When Issuers are defined within the Installation manifest, js-opertor creates and pairs these resources on your behalf. For convenience, unless you explicitly configure the issuers.policy field in the Installation manifest, js-operator will default this to allowAll: true.

Pre-Installed Certificate Issuers

js-operator promotes the use of the Venafi Issuer type. You can observe that a ClusterIssuer named venafi is created by default.

kubectl get ClusterIssuers

Alternate Certificate Issuers

You can add new Issuer resources to your cluster by editing the Installation manifest. Open the manifest, in whatever EDITOR you have configured, as follows.

kubectl edit Installation installation

You will see the venafi ClusterIssuer already present.

You can add, for example, a SelfSigned ClusterIssuer by inserting the following snippet into the issuers: section of the Installation manifest.

  - clusterScope: true
    name: self-signed
    selfSigned: {}

To configure an issuer for an in-cluster dev-mode install of Hashicorp Vault you could first create a secret for the token ...

kubectl -n jetstack-secure create secret generic cert-manager-vault-token --from-literal=token=<TOKEN_VALUE_HERE>

... then update the Installation manifest as follows.

  - clusterScope: true
    name: vault
    vault:
      path: pki/sign/example-dot-com
      server: http://vault.vault.svc.cluster.local:8200
      auth:
        tokenSecretRef:
          name: cert-manager-vault-token
          key: token

Venafi Enhanced Issuer

At the time of writing the Venafi Enhanced Issuer is not included in the set of components js-operator can install for you. Installing the Venafi Enhanced Issuer can be done as follows.

helm upgrade demo oci://eu.gcr.io/jetstack-secure-enterprise/charts/venafi-enhanced-issuer \
    --namespace jetstack-secure \
    --install \
    --wait \
    --set 'global.imagePullSecrets[0].name=jse-gcr-creds' \
    --registry-config <(jsctl registry auth output --format=dockerconfig) \
    --version v0.1.7

TODO create a VenafiClusterIssuer resource and demo it in use (maybe requires Hashicorp Vault). See this

Testing Certificate Issuance

Create a namespace to play in.

kubectl create namespace demos

Testing SelfSigned certificate issuance

Now, just to kick the tires, let's create a SelfSigned certificate, as follows.

cat << EOF | kubectl -n demos apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: www.selfsigned.com
spec:
  secretName: www-selfsigned-com-tls
  dnsNames:
    - www.selfsigned.com
  issuerRef:
    name: self-signed
    kind: ClusterIssuer
    group: cert-manager.io
EOF

Then check the secret for tls.key, ca.crt and tls.crt.

kubectl -n demos describe secret www-selfsigned-com-tls

Testing Vault certificate issuance

cat << EOF | kubectl -n demos apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: test.example.com
spec:
  secretName: test-example-com-tls
  commonName: test.example.com
  issuerRef:
    name: vault
    kind: ClusterIssuer
    group: cert-manager.io
EOF

Then check the secret for tls.key, ca.crt and tls.crt.

kubectl -n demos describe secret test-example-com-tls

Testing Venafi certificate issuance

Complete the tests by creating a Venafi certificate, as follows.

cat << EOF | kubectl -n demos apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: www.venafi.com
spec:
  secretName: www-venafi-com-tls
  commonName: www.venafi.com          # Common Name is a Venafi baseline requirement
  dnsNames:
    - www.venafi.com
  issuerRef:
    name: venafi
    kind: ClusterIssuer
    group: cert-manager.io
EOF

Then check the secret for tls.key, ca.crt and tls.crt.

kubectl -n demos describe secret www-venafi-com-tls

Appendix

Deleting the installation manifest

The jsctl operator installations apply operation creates a global Installation resource named "installation". jsctl does not yet provide an operation to reverse this out but you can, if necessary, approximate the same as follows.

kubectl delete Installation installation # *** DANGER, destructive operation! ***

Unregistering and terminating your cluster

The quickest way to undo ALL of the above is to destroy the KinD cluster and delete its registration entry in JSS

kind delete cluster --name ${k8s_cluster_name}
jsctl clusters delete ${k8s_cluster_name_tlspk} --force

Remember to mop up dangling service account for the cluster in TLSPK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment