Terminology:
- TLSPK: TLS Protect for Kubernetes (previously Jetstack Secure or JSS)
- TLSPC: TLS Protect Cloud (previously Venafi as a Service or VaaS)
- TLSP: TLS Protect Data Centre (previously Venafi Trust Protection Platform or TPP)
- VEI: Venafi Enhanced Issuer (not to be confused with the native cert-manager issuer for Venafi)
cert-manager's native Venafi issuer requires Kubernetes secrets to hold Venafi credentials (e.g. API keys). Ideally you wish to eliminate the use of all secrets as these create a potential attack vector.
With the help of TLSPK js-operator, your goal is to use Venafi credentials stored off-cluster to request certificates from either TLSP or TLSPC.
In this is example your certificate issuance platform will be TLSPC and your credentials will be stored in Hashicorp Vault.
NOTE: the version of VenafiEnhancedIssuer currently enabled with js-operator (jsctl v0.1.17) pre-dates VenafiConnection, which is why that CRD is missing and, for now, you need the separate helm install.
Until further notice, do not attempt to kubectl edit installation.
It is generally good practice to monitor your Kubernetes objects, ensuring your dependencies are in place (i.e. Running) at the point of need
watch "kubectl get pods,crds -A | grep -v system"
k8s_name=kind-$(date +"%y%m%d%H%M")
cat <<EOF | kind create cluster --config -
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: ${k8s_name}
nodes:
- role: control-plane
EOF
# rm -rf ~/.jsctl/ # (optional)
jsctl auth login
jsctl config set organization gallant-wright # (change org as appropriate)
jsctl registry auth init
kubectl create namespace jetstack-secure
jsctl operator deploy --auto-registry-credentials
sleep 2 && kubectl -n jetstack-secure wait --for=condition=Available=True --all deployments --timeout=-1s
sleep 5 && jsctl operator installations apply --auto-registry-credentials --cert-manager-replicas 1
sleep 2 && kubectl -n jetstack-secure wait --for=condition=Available=True --all deployments --timeout=-1s
helm -n jetstack-secure upgrade -i vei \
oci://eu.gcr.io/jetstack-secure-enterprise/charts/venafi-enhanced-issuer \
--version v0.3.2 \
--registry-config <(jsctl registry auth output --format=dockerconfig) \
--set 'global.imagePullSecrets[0].name=jse-gcr-creds' \
--wait
At this point you may choose to update your watch command to monitor new CRD type objects, for example
watch "kubectl get pods,venaficonnections,venaficlusterissuers,certificaterequestpolicies -A | grep -v system"
The VenafiConnection object you are about to create will use this service account to establish communication with Vault.
Observe how this references serviceaccounts/token.
cat <<EOF | kubectl -n jetstack-secure apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
name: tlspc-sa
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: create-tokens-for-tlspc-issuer
rules:
- apiGroups: [""]
resources: ["serviceaccounts/token"]
verbs: ["create"]
resourceNames: ["tlspc-sa"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tlspc-issuer-sa-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: create-tokens-for-tlspc-issuer
subjects:
- kind: ServiceAccount
name: venafi-connection
namespace: jetstack-secure
EOF
helm repo add hashicorp https://helm.releases.hashicorp.com && helm repo update
helm -n vault upgrade -i vault \
hashicorp/vault \
--create-namespace \
--set "server.dev.enabled=true" \
--wait
sleep 5 && kubectl -n vault wait -l statefulset.kubernetes.io/pod-name=vault-0 --for=condition=ready pod --timeout=-1s
Note this references tlspc-sa created previously.
kubectl exec -n vault pods/vault-0 -- \
vault auth enable -path=tlspc kubernetes
kubectl exec -n vault pods/vault-0 -- \
vault write auth/tlspc/config kubernetes_host=https://kubernetes.default.svc
kubectl exec -n vault pods/vault-0 -- \
vault write auth/tlspc/role/tlspc-role \
role_type=jwt \
bound_audiences=vault.vault.svc.cluster.local \
user_claim=sub \
bound_service_account_names=tlspc-sa \
bound_service_account_namespaces=jetstack-secure \
policies=tlspc-policy-readonly \
ttl=5m
cat <<EOF | kubectl exec -i -n vault pods/vault-0 -- vault policy write tlspc-policy-readonly -
path "secret/data/tlspc-connection/creds" {
capabilities = ["read"]
}
EOF
api_key=<TLSPC_API_KEY> # <- get this from https://ui.venafi.cloud/platform-settings/user-preferences
kubectl exec -n vault pods/vault-0 -- vault kv put -mount=secret tlspc-connection/creds api-key=${api_key}
Note how this uses tlspc-sa to pull the TLSPC API key out of Vault.
cat <<EOF | kubectl -n jetstack-secure apply -f -
apiVersion: jetstack.io/v1alpha1
kind: VenafiConnection
metadata:
name: tlspc-connection
spec:
vaas:
apiKey:
- serviceAccountToken:
name: tlspc-sa
audiences: [vault.vault.svc.cluster.local]
- hashicorpVaultOAuth:
authInputType: OIDC
role: tlspc-role
authPath: /v1/auth/tlspc/login
url: http://vault.vault.svc.cluster.local:8200
- hashicorpVaultSecret:
secretPath: /v1/secret/data/tlspc-connection/creds
fields: ["api-key"]
url: http://vault.vault.svc.cluster.local:8200
EOF
Create the issuer object which references the VenafiConnection object you just created. Observe how this depends upon the Default\Default zone. Please adjust to suit.
When creating the object directly like this, no matching CertificateRequestPolicy object is created. You will address this shortly.
cat <<EOF | kubectl apply -f -
apiVersion: jetstack.io/v1alpha1
kind: VenafiClusterIssuer
metadata:
name: tlspc-issuer
spec:
venafiConnectionName: tlspc-connection
zone: Default\Default
EOF
When cert-manager is installed via TLSPK it comes with the policy-approver component. When you create an issuer via the operator (or its installation manifest) it automatically OPENs the policy to allow ALL certs to pass through. This case does not apply here.
As you created your venafi-enhanced-issuer object by hand you do not get automatic policy provision and the policy-approver will block ALL cert requests until you configure an appropriate policy, also by hand.
The following policy (and RBAC) allows certs to be defined in terms of ANY commonName and ANY dnsNames which is enough to cover the certs you require today.
cat << EOF | kubectl apply -f -
apiVersion: policy.cert-manager.io/v1alpha1
kind: CertificateRequestPolicy
metadata:
name: certificate-request-policy-tlspc
spec:
allowed:
commonName:
value: '*'
dnsNames:
values:
- '*'
selector:
issuerRef:
group: jetstack.io
kind: VenafiClusterIssuer
name: tlspc-issuer
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: certificate-request-policy-tlspc:rbac
rules:
- apiGroups: ["policy.cert-manager.io"]
resources: ["certificaterequestpolicies"]
verbs: ["use"]
resourceNames: ["certificate-request-policy-tlspc"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: certificate-request-policy-tlspc:rbac
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: certificate-request-policy-tlspc:rbac
subjects:
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
EOF
We can test the above works by requesting a certificate.
kubectl create namespace tests
cat << EOF | kubectl -n tests apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: www.vei-demo.com
spec:
secretName: www-vei-demo-com-tls
commonName: www.vei-demo.com
dnsNames:
- www.vei-demo.com
issuerRef:
name: tlspc-issuer
kind: VenafiClusterIssuer
group: jetstack.io
EOF
Check the CertificateRequest is "Ready".
kubectl -n tests get cr
If it is, inspect the secret.
kubectl -n tests get secret www-vei-demo-com-tls -o 'go-template={{index .data "tls.crt" | base64decode}}' | openssl x509 -noout -text | head -11
kind delete cluster --name ${k8s_name}
If you wish to connect your cluster to TLSPK (optional), you may do so as follows
k8s_name_tlspk=$(tr "-" "_" <<< ${k8s_name})
jsctl clusters connect ${k8s_name_tlspk}
# jsctl clusters delete ${k8s_name_tlspk} --force