Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save zulhfreelancer/36d451b172637c3f4df32c63fb2d2244 to your computer and use it in GitHub Desktop.
Save zulhfreelancer/36d451b172637c3f4df32c63fb2d2244 to your computer and use it in GitHub Desktop.
How to fix Nginx ingress controller "certificate signed by unknown authority" error?

Problem

How to fix Nginx ingress controller "certificate signed by unknown authority" error?

Error:

"Internal error occurred: failed calling webhook \"validate.nginx.ingress.kubernetes.io\": failed to call webhook: Post \"https://nginx-ingress-ingress-nginx-controller-admission.default.svc:443/networking/v1/ingresses?timeout=10s\": x509: certificate signed by unknown authority"

Solution

  1. Compare the CA stored in ValidatingWebhookConfiguration vs in the secret where the *nginx-controller* pods are running

    In this case, the *nginx-controller* pods are in default namespace - yours may be in different namespace

    $ k get ValidatingWebhookConfiguration nginx-ingress-ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5
    d41d8cd98f00b204e9800998ecf8427e
    
    $ k -n default get secret nginx-ingress-ingress-nginx-admission -o jsonpath='{.data.ca}' | md5
    bbf6ef16566994f9f65facc7e8f07b16
  2. It's clear that they are not same because the MD5 hashes are different. Let's fix this...

  3. Copy the CA from the secret where the *nginx-controller* pods are running

    In this case, the *nginx-controller* pods are in default namespace - yours may be in different namespace

    $ CA=$(kubectl -n default get secret nginx-ingress-ingress-nginx-admission -o jsonpath='{.data.ca}')
  4. Patch the ValidatingWebhookConfiguration

    $ kubectl patch validatingwebhookconfigurations nginx-ingress-ingress-nginx-admission --type='json' -p='[{"op": "add", "path": "/webhooks/0/clientConfig/caBundle", "value":"'$CA'"}]'
  5. Repeat step 1 and make sure both MD5 hashes are same

    $ k get ValidatingWebhookConfiguration nginx-ingress-ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5
    bbf6ef16566994f9f65facc7e8f07b16
    
    $ k -n default get secret nginx-ingress-ingress-nginx-admission -o jsonpath='{.data.ca}' | md5
    bbf6ef16566994f9f65facc7e8f07b16
@zulhfreelancer
Copy link
Author

zulhfreelancer commented Feb 8, 2024

@rbtz-openai - I'm glad you found this piece useful and able to unblock you. Thank you for the free ChatGPT Plus!

@daemonadmin
Copy link

daemonadmin commented Apr 9, 2024

For RKE2-clusters:
CHECK:
k get ValidatingWebhookConfiguration rke2-ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5sum
k -n kube-system get secret rke2-ingress-nginx-admission -o jsonpath='{.data.ca}' | md5sum

PATCH:
CA=$(kubectl -n default get secret rke2-ingress-nginx-admission -o jsonpath='{.data.ca}')
kubectl patch ValidatingWebhookConfiguration rke2-ingress-nginx-admission --type='json' -p='[{"op": "add", "path": "/webhooks/0/clientConfig/caBundle", "value":"'$CA'"}]'

Thank you for solution!

@DovydasNavickas
Copy link

For k3s clusters

Checking

# Validating webhook CA
k get validatingwebhookconfigurations ingress-nginx-admission -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | md5sum

# CA in secret
k get secrets -n ingress-nginx ingress-nginx-admission -o jsonpath='{.data.ca}'

Fixing

CA=$(k get secrets -n ingress-nginx ingress-nginx-admission -o jsonpath='{.data.ca}')
k patch validatingwebhookconfigurations ingress-nginx-admission --type='json' -p='[{"op": "add", "path": "/webhooks/0/clientConfig/caBundle", "value":"'$CA'"}]'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment