MicroK8s OpenStack cloud controller manager

README.md

Deploy OpenStack Cloud Controller Manager

Service configuration

a. Using Launch Configurations

Make sure to set --cloud-provider=external in Kubelet arguments. If not installed yet, use a launch configuration like this:

echo '
version: 0.2.0
addons:
  - name: dns
extraKubeletArgs:
  --cluster-domain: cluster.local
  --cluster-dns: 10.152.183.10
  --cloud-provider: external
' | sudo tee /etc/microk8s.yaml

sudo snap install microk8s --classic --channel 1.28

b. After installation

If MicroK8s is already installed:

echo '--cloud-provider=external' | sudo tee -a /var/snap/microk8s/current/args/kubelet
sudo snap restart microk8s.daemon-kubelite

Then, make sure to set the following taint on all existing cluster nodes, so that the cloud controller manager configures them:

for node in $(microk8s kubectl get node -o jsonpath='{.items[*].metadata.name}'); do
    microk8s kubectl taint node "${node}" node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule
done

Credentials

Prepare a secret with credentials and configuration for the OpenStack cloud, based on secret.yaml

microk8s kubectl apply -f secret.yaml

Helm configuration

Adjust values.yaml and set a unique cluster name. Then, deploy cloud controller manager with:

microk8s helm repo add openstack https://kubernetes.github.io/cloud-provider-openstack
microk8s helm repo update
microk8s helm install --namespace kube-system openstack-cloud-controller-manager openstack/openstack-cloud-controller-manager -f values.yaml

Test LoadBalancer service

Create a simple deployment, then expose as a LoadBalancer service:

microk8s kubectl create deploy nginx --replicas 3 --image nginx
microk8s kubectl expose deploy nginx --type LoadBalancer --port 80

After a while, the LoadBalancer service should have an external IP and be accessible. Check the IP address using microk8s kubectl get svc nginx

NAME         TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
nginx        LoadBalancer   10.152.183.59   10.100.0.121   80:31179/TCP   3h7m

And then verify access with curl http://10.100.0.121

Troubleshooting

1. Revisit the configuration of the [LoadBalancer] section on the cloud config secret. After changing the secret, restart the cloud controller manager with microk8s kubectl rollout restart -n kube-system ds/openstack-cloud-controller-manager
2. Check the logs of the openstack cloud controller manager with microk8s kubectl logs -n kube-system ds/openstack-cloud-controller-manager

secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: openstack-ccm-cloud-config
  namespace: kube-system
stringData:
  cloud.conf: |
    [Global]
    auth-url = https://$keystone_ip:5000/v3
    region = $region
    username = $username
    password = $password
    tenant-name = $project
    domain-name = $domain
    tenant-domain-name = $domain

    # # if cloud uses self-signed CA
    # ca-file = /etc/config/ca.crt

    [LoadBalancer]
    # if using octavia-ovn-provider
    lb-provider = ovn
    lb-method = SOURCE_IP_PORT

    # if network requires security groups
    manage-security-groups = true

    # # if using multiple external networks
    # floating-network-id = da8be501-da7b-492c-9591-20e6503ecd5a

  # Only needed if using self-signed CA, else leave empty
  ca.crt: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

values.yaml

cluster:
  # pick a unique name for different clusters
  name: microk8s-cluster01

secret:
  enabled: true
  name: openstack-ccm-cloud-config
  create: false

tolerations:
  - key: node.cloudprovider.kubernetes.io/uninitialized
    value: "true"
    effect: NoSchedule

extraVolumes: []
extraVolumeMounts: []