Skip to content

Instantly share code, notes, and snippets.

@juniorz
Last active May 12, 2022
Embed
What would you like to do?
Annoying Kubernetes Issues

kubectl logs has quirks due to log-rotation

kubectl logs does not take into account container log rotation (kubelet config containerLogMaxSize and containerLogMaxFiles). You can still find the latest rotated logs in /var/log/pods/<namespace>_<pod-name>_<pod-uid>/ though.

Don’t assume --since=0s will have all logs since the container started because it won't look into previous log files. Don't assume a long-running --follow will always give you latest logs, because it will not switch over to the new log after current is rotated.

See: kubernetes/kubernetes#59902

KUBECONFIG is not portable

Problem: KUBECONFIG was created for a specific version of a "credential plugin". The user configured for the context has user.exec.apiVersion = client.authentication.k8s.io/v1beta1 but its user.exec.command returns a client.authentication.k8s.io/v1alpha1 document. The client-go has no machanism to know how to build a v1beta1 from a v1alpha1.

This probably happens when KUBECONFIG is generated in one system but used in another, like when two completely different containers generate and use it.

Example:

Error: [VERB URL]: getting credentials: exec plugin is configured to use API version client.authentication.k8s.io/v1beta1, plugin returned version client.authentication.k8s.io/v1alpha1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment