Set up a Kind cluster with working ingress

kind-with-ingress.md

To get a Kind cluster to work with an ingress controller such as NGINX Ingress Controller, you need some custom configuration of the cluster. Such a configuration file could look like:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    kubeadmConfigPatches:
      - |
        kind: InitConfiguration
        nodeRegistration:
          kubeletExtraArgs:
            node-labels: "ingress-ready=true"
      - |
        apiVersion: kubeadm.k8s.io/v1beta2
        kind: ClusterConfiguration
        controllerManager:
          extraArgs:
            bind-address: 0.0.0.0
        scheduler:
          extraArgs:
            bind-address: 0.0.0.0
        etcd:
          local:
            extraArgs:
              listen-metrics-urls: http://0.0.0.0:2381
    extraPortMappings:
      - containerPort: 80
        hostPort: 80
        protocol: TCP
      - containerPort: 443
        hostPort: 443
        protocol: TCP
  - role: worker
  - role: worker

Using kind create cluster --config /path/to/config.yaml creates a cluster with one control-plane node, and two worker nodes. But more importantly, you open ports from the local host to the cluster, which can be utilized by the ingress controller.

Install the NGINX Ingress Controller with helm install ingress-nginx --repo https://kubernetes.github.io/ingress-nginx ingress-nginx -f nginx-values.yaml where nginx-values.yaml contain the following:

controller:
  watchIngressWithoutClass: true

  service:
    # externalTrafficPolicy: Local
    type: NodePort # Defaults to LoadBalancer
    external:
      enabled: false
  extraArgs:
    publish-status-address: localhost

  hostPort:
    enabled: true # For Kind
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate # Recreate might be necessary when using hostPort
  nodeSelector:
    ingress-ready: "true"
  tolerations:
    - key: node-role.kubernetes.io/control-plane
      operator: Exists

With this, ingress will work using hosts such as http://foo.localhost. Test the connection with:

---
kind: Pod
apiVersion: v1
metadata:
  name: foo-app
  labels:
    app: foo
spec:
  containers:
    - command:
        - /agnhost
        - netexec
        - --http-port
        - "8080"
      image: registry.k8s.io/e2e-test-images/agnhost:2.39
      name: foo-app
---
kind: Service
apiVersion: v1
metadata:
  name: foo-service
spec:
  selector:
    app: foo
  ports:
    # Default port used by the image
    - port: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
spec:
  rules:
    - host: foo.localhost
      http:
        paths:
          - pathType: Prefix
            path: /
            backend:
              service:
                name: foo-service
                port:
                  number: 8080

curl foo.localhost/ returns HTTP/1.1 200 OK.