Running_v1.24.12_in_CAPVCD.md

Running Kubernetes 1.24 with CAPVCD.

❗ This is tested with Vanilla Kubernetes nodes, it may not work with Tanzu.

Image-builder

• Clone the image-builder repo.
• Edit images/capi/ansible/node.yml
• Add the following tasks at the end (e.g. before the sysprep section)

    - name: Removing 99-installer.cfg
      file:
        path: "/etc/cloud/cloud.cfg.d/99-installer.cfg"
        state: absent
    - name: creating_98-cse-vmware-datasource
      copy:
        dest: "/etc/cloud/cloud.cfg.d/98-cse-vmware-datasource.cfg"
        content: |
          datasource_list: [ "VMware" ]

• Configure the image as you need [custom-config.json]. e.g. my values 👇

{
  "kubernetes_deb_version": "1.24.12-00",
  "kubernetes_rpm_version": "1.24.12-0",
  "kubernetes_semver": "v1.24.12",
  "kubernetes_series": "v1.24",
  "cluster": "xxx",
  "convert_to_template": "false",
  "create_snapshot": "true",
  "datacenter": "xxx",
  "datastore": "xxx",
  "folder": "xxx",
  "insecure_connection": "true",
  "network": "xxx",
  "password": "xxx",
  "username": "xxx",
  "vcenter_server": "xxx"
}

• Create the image make PACKER_VAR_FILES=custom-config.json build-node-ova-vsphere-ubuntu-2004
• Then export the VM as OVF and upload it to VCD.

CPI selector and tolerations

giantswarm/cloud-provider-cloud-director-app#43

The CPI deployment must include the following tolerations (on top of the default ones):

  - effect: NoSchedule
    key: node-role.kubernetes.io/control-plane
  - effect: NoSchedule
    key: node-role.kubernetes.io/master

The node-selector should also be changed from node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane.

Cloud-init changes

A number of changes must be made to the cloud-init script that configures the node along with a couple changes in the code.

See this commit for details.

You will need to fork the CAPVCD repo, make the changes and build the image.

Cluster manifest

Change the kubernetes version from v1.2x.xx+vmware.1 to v1.2x.xx.

Change the repositories to upstream ones and fix versions.

👍