How to fix Nginx ingress controller "certificate signed by unknown authority" error?

nginx-controller-certificate-signed-by-unknown-authority-error.md

Problem

How to fix Nginx ingress controller "certificate signed by unknown authority" error?

Error:

"Internal error occurred: failed calling webhook \"validate.nginx.ingress.kubernetes.io\": failed to call webhook: Post \"https://nginx-ingress-ingress-nginx-controller-admission.default.svc:443/networking/v1/ingresses?timeout=10s\": x509: certificate signed by unknown authority"

Solution

1. Compare the CA stored in ValidatingWebhookConfiguration vs in the secret where the *nginx-controller* pods are running

   In this case, the *nginx-controller* pods are in default namespace - yours may be in different namespace

   $ k get ValidatingWebhookConfiguration nginx-ingress-ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5
   d41d8cd98f00b204e9800998ecf8427e
   
   $ k -n default get secret nginx-ingress-ingress-nginx-admission -o jsonpath='{.data.ca}' | md5
   bbf6ef16566994f9f65facc7e8f07b16

2. It's clear that they are not same because the MD5 hashes are different. Let's fix this...

3. Copy the CA from the secret where the *nginx-controller* pods are running

   In this case, the *nginx-controller* pods are in default namespace - yours may be in different namespace

   $ CA=$(kubectl -n default get secret nginx-ingress-ingress-nginx-admission -o jsonpath='{.data.ca}')

4. Patch the ValidatingWebhookConfiguration

   $ kubectl patch validatingwebhookconfigurations nginx-ingress-ingress-nginx-admission --type='json' -p='[{"op": "add", "path": "/webhooks/0/clientConfig/caBundle", "value":"'$CA'"}]'

5. Repeat step 1 and make sure both MD5 hashes are same

   $ k get ValidatingWebhookConfiguration nginx-ingress-ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5
   bbf6ef16566994f9f65facc7e8f07b16
   
   $ k -n default get secret nginx-ingress-ingress-nginx-admission -o jsonpath='{.data.ca}' | md5
   bbf6ef16566994f9f65facc7e8f07b16

Thank you for saving me half an hour of reading through nginx-ingress code! Please accept this humble gift of half a year of free ChatGPT Plus (if I guessed your email right).

P.S. You can factor out the namespace and ingress name into variables for ease of copy-paste.

@rbtz-openai - I'm glad you found this piece useful and able to unblock you. Thank you for the free ChatGPT Plus!

For RKE2-clusters:
CHECK:
k get ValidatingWebhookConfiguration rke2-ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5sum
k -n kube-system get secret rke2-ingress-nginx-admission -o jsonpath='{.data.ca}' | md5sum

PATCH:
CA=$(kubectl -n default get secret rke2-ingress-nginx-admission -o jsonpath='{.data.ca}')
kubectl patch ValidatingWebhookConfiguration rke2-ingress-nginx-admission --type='json' -p='[{"op": "add", "path": "/webhooks/0/clientConfig/caBundle", "value":"'$CA'"}]'

Thank you for solution!

For k3s clusters

Checking

# Validating webhook CA
k get validatingwebhookconfigurations ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5sum

# CA in secret
k get secrets -n ingress-nginx ingress-nginx-admission -o jsonpath='{.data.ca}'

Fixing

CA=$(k get secrets -n ingress-nginx ingress-nginx-admission -o jsonpath='{.data.ca}')
k patch validatingwebhookconfigurations ingress-nginx-admission --type='json' -p='[{"op": "add", "path": "/webhooks/0/clientConfig/caBundle", "value":"'$CA'"}]'