Skip to content

Instantly share code, notes, and snippets.

@joshwget

joshwget/stalesecret.md

Last active August 21, 2017 09:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save joshwget/ac657afebc6260514adbfb4de76622c1 to your computer and use it in GitHub Desktop.
Save joshwget/ac657afebc6260514adbfb4de76622c1 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
stalesecret.md

Containers get stuck in a state of ImagePullBackOff or CrashLoopBackOff and show a detailed error like:

Cannot initialize Kubernetes connection: the server has asked for the client to provide credentialspanic: runtime error: invalid memory address or nil pointer dereference

This is caused by stale credentials within Rancher (rancher/rancher#8388).

The following instructions can be used to fix the issue in the meantime.

Delete the token first.

kubectl -n kube-system describe serviceaccount io-rancher-system
kubectl –n kube-system delete secret <token name>

Then remove the add-ons.

kubectl -n kube-system get pods

Delete Tiller, KubeDNS, Kubernetes Dashboard, and Heapster pods. They'll be recreated and should be able to access the Kubernetes API afterwards.

kubectl -n kube-system delete ...
@mvisonneau
Copy link

Thanks ! worked for me :)

@DanielMSchmidt
Copy link

Thanks for sharing 👍 Could it be that you need to set the namespace at the end of the command?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment