Expose KubeAPI with Nginx

README.md

Goals

Can expose KubeAPI with NGINX Reverse Proxy on Kubespray cluster edition.

Make sure our FQDN has recorded on SAN KubeAPI Certificate

• to make sure we can dump SAN information on openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text
• If SAN not yet on recorded cert, on Kubespray we can declare it in nano inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml in supplementary_addresses_in_ssl_keys section.

Create Nginx Block

• Assume KubeAPI domain is k8s.adaptivenetworklab.org will be resolve in 172.20.1.79

server {
    listen 6443;
    ssl on;
    server_name k8s.adaptivenetworklab.org;

    ssl_certificate /etc/letsencrypt/live/k8s.adaptivenetworklab.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/k8s.adaptivenetworklab.org/privkey.pem;

    location / {
        proxy_pass https://172.20.1.79:6443;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header host k8s.adaptivenetworklab.org;
        proxy_ssl_certificate /etc/letsencrypt/live/k8s.adaptivenetworklab.org/client-crt.pem;
        proxy_ssl_certificate_key /etc/letsencrypt/live/k8s.adaptivenetworklab.org/client-key.pem;
    }
}

Info

• proxy_ssl_certificate we can get in /etc/kubernetes/pki/apiserver-kubelet-client.crt
• proxy_ssl_certificate_key we can get in /etc/kubernetes/pki/apiserver-kubelet-client.key
• ssl_certificate we can get in /etc/kubernetes/pki/apiserver.crt
• ssl_certificate_key we can get in /etc/kubernetes/pki/apiserver.key

Cons

• If we use L7 like this, prox_ssl_certificate use client certificate, so RBAC with certificate will be not working with this technique.
• Recommendation expose with L4 on Nginx (upstream Technique)

    server {
      listen       6443;
      proxy_pass  172.20.1.79:6443;
    }
  }
  

Reference

• https://www.henryxieblogs.com/2021/12/how-to-expose-kube-api-server-via-nginx.html