Kind cluster with MetalLB and Docker Mac Net Connect

README.md

Prerequisites:

• Running on MacOS
• Install Docker Desktop
• Install Kind
• Install Docker Mac Net Connect

Instructions:

• Create a cluster:
  kind create cluster --config kind-cluster.yaml
• Instal Ingress NGINX Controller:
  kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
• Install 2 pods, their corresponding services and an Ingress:
  kubectl apply -f ingress-service.yaml
• Test:
  curl localhost/foo/hostname
  returns 'foo-app'
  curl localhost/bar/hostname
  returns 'bar-app'
• Install MetalLB:
  kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml
• Find out CIDR used by the Docker network used by the kind cluster:
  docker network inspect -f '{{.IPAM.Config}}' kind
• Verify that 'spec.addresses' in 'metallb-config.yaml' are comprised in the kind network CIDR. Modify the file if required.
• Configure MetalIB:
  kubectl apply -f metallb-config.yaml
• Create a Service of type LoadBalancer that load balances across the previously created pods:
  kubectl apply -f lb-service.yaml
• Test the load balanced service through the Ingress:
  curl localhost/foo-bar/hostname
  return 'foo-app' or 'bar-app' approximately evenly distributed
• Test the load balancer directly:
  LB_IP=`kubectl get svc foo-bar-service -o json | jq ".status.loadBalancer.ingress.[].ip" | tr -d '"'`
  curl $LB_IP/hostname
  return 'foo-app' or 'bar-app' approximately evenly distributed

References

• Docker Mac Net Connect
• Kind | User Guide | Ingress
• Kind | User Guide | Load Balancer
• LoadBalancer Services using Kubernetes in Docker (kind)
• Building the best Kubernetes test cluster on MacOS

ingress-service.yaml

kind: Pod
apiVersion: v1
metadata:
  name: foo-app
  labels:
    app: foo
    service: foo-bar
spec:
  containers:
  - command:
    - /agnhost
    - netexec
    - --http-port
    - "8080"
    image: registry.k8s.io/e2e-test-images/agnhost:2.39
    name: foo-app
---
kind: Service
apiVersion: v1
metadata:
  name: foo-service
spec:
  selector:
    app: foo
  ports:
  # Default port used by the image
  - port: 8080
---
kind: Pod
apiVersion: v1
metadata:
  name: bar-app
  labels:
    app: bar
    service: foo-bar
spec:
  containers:
  - command:
    - /agnhost
    - netexec
    - --http-port
    - "8080"
    image: registry.k8s.io/e2e-test-images/agnhost:2.39
    name: bar-app
---
kind: Service
apiVersion: v1
metadata:
  name: bar-service
spec:
  selector:
    app: bar
  ports:
  # Default port used by the image
  - port: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /foo(/|$)(.*)
        backend:
          service:
            name: foo-service
            port:
              number: 8080
      - pathType: Prefix
        path: /bar(/|$)(.*)
        backend:
          service:
            name: bar-service
            port:
              number: 8080
      - pathType: Prefix
        path: /foo-bar(/|$)(.*)
        backend:
          service:
            name: foo-bar-service
            port:
              number: 80

kind-cluster.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP

lb-service.yaml

kind: Service
apiVersion: v1
metadata:
  name: foo-bar-service
spec:
  type: LoadBalancer
  selector:
    service: foo-bar
  ports:
  # Default port used by the image
  - port: 80
    targetPort: 8080

metalb-config.yaml

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: example
  namespace: metallb-system
spec:
  addresses:
  - 172.18.255.200-172.18.255.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: empty
  namespace: metallb-system