Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
tkg-ytt-overlay-additional-ca-certs
#@ load("@ytt:overlay", "overlay")
#@ load("@ytt:data", "data")
#! This ytt overlay adds additional custom CA certificates on TKG cluster nodes, so containerd and other tools trust these CA certificates.
#! It works when using Photon or Ubuntu as the TKG node template on all TKG infrastructure providers.
#! Trust your custom CA certificates on all Control Plane nodes.
#@overlay/match by=overlay.subset({"kind":"KubeadmControlPlane"})
---
spec:
kubeadmConfigSpec:
#@overlay/match missing_ok=True
files:
#@overlay/append
- content: #@ data.read("tkg-custom-ca.pem")
owner: root:root
permissions: "0644"
path: /etc/ssl/certs/tkg-custom-ca.pem
#@overlay/match missing_ok=True
preKubeadmCommands:
#! For Photon OS
#@overlay/append
- '! which rehash_ca_certificates.sh 2>/dev/null || rehash_ca_certificates.sh'
#! For Ubuntu
#@overlay/append
- '! which update-ca-certificates 2>/dev/null || (mv /etc/ssl/certs/tkg-custom-ca.pem /usr/local/share/ca-certificates/tkg-custom-ca.crt && update-ca-certificates)'
#! Trust your custom CA certificates on all worker nodes.
#@overlay/match by=overlay.subset({"kind":"KubeadmConfigTemplate"})
---
spec:
template:
spec:
#@overlay/match missing_ok=True
files:
#@overlay/append
- content: #@ data.read("tkg-custom-ca.pem")
owner: root:root
permissions: "0644"
path: /etc/ssl/certs/tkg-custom-ca.pem
#@overlay/match missing_ok=True
preKubeadmCommands:
#! For Photon OS
#@overlay/append
- '! which rehash_ca_certificates.sh 2>/dev/null || rehash_ca_certificates.sh'
#! For Ubuntu
#@overlay/append
- '! which update-ca-certificates 2>/dev/null || (mv /etc/ssl/certs/tkg-custom-ca.pem /usr/local/share/ca-certificates/tkg-custom-ca.crt && update-ca-certificates)'
-----BEGIN CERTIFICATE-----
CA certificate content
-----END CERTIFICATE-----
@jessehu
Copy link
Author

jessehu commented Mar 7, 2021

This TKG ytt overlay adds additional custom CA certificates on TKG cluster nodes, so containerd and other tools trust these CA certificates, and containerd is able to pull images from the container registry that uses the CA certificate. It works when using Photon or Ubuntu as the TKG node template on all TKG infrastructure providers.

  1. Download the above files and put them under TKG ytt directory ~/.tkg/providers/ytt/03_customizations/ (for TKG 1.2.x) or ~/.tanzu/tkg/providers/ytt/03_customizations/ (for TKG 1.3.x).
  2. Add your CA certificates into tkg-custom-ca.pem.
  3. Create TKG management clusters or workload clusters.

Loading

@jessehu
Copy link
Author

jessehu commented Mar 11, 2021

This ytt overlay works for TKG AMI image for AWS as the infrastructure provider: https://github.com/scottd018/tkg-configs/blob/main/additional_ca_cert_overlay.yaml

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment