Skip to content

Instantly share code, notes, and snippets.

@mehstg

mehstg/kube-dns-debugging.txt Secret

Created February 3, 2017 14:35
Show Gist options
  • Save mehstg/0b8016f5398a8781c3ade8cf49c02680 to your computer and use it in GitHub Desktop.
Save mehstg/0b8016f5398a8781c3ade8cf49c02680 to your computer and use it in GitHub Desktop.
Download ZIP
Output of kubectl describe pods kube-dns for debugging purposes
Raw
kube-dns-debugging.txt
core@coreos1 ~ $ kubectl describe pods kube-dns-v20-h4w7m --namespace=kube-system
Name: kube-dns-v20-h4w7m
Namespace: kube-system
Node: 192.168.1.203/192.168.1.203
Start Time: Fri, 03 Feb 2017 14:01:16 +0000
Labels: k8s-app=kube-dns
version=v20
Status: Running
IP: 10.10.18.3
Controllers: ReplicationController/kube-dns-v20
Containers:
kubedns:
Container ID: docker://8783d9fbc4c62151bf021ae920dd113029c64c9e9ebd3336f6e8c52c0390d92c
Image: gcr.io/google_containers/kubedns-amd64:1.8
Image ID: docker-pullable://gcr.io/google_containers/kubedns-amd64@sha256:39264fd3c998798acdf4fe91c556a6b44f281b6c5797f464f92c3b561c8c808c
Ports: 10053/UDP, 10053/TCP
Args:
--domain=cluster.local.
--dns-port=10053
Limits:
memory: 170Mi
Requests:
cpu: 100m
memory: 70Mi
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Fri, 03 Feb 2017 14:15:10 +0000
Finished: Fri, 03 Feb 2017 14:17:19 +0000
Ready: False
Restart Count: 6
Liveness: http-get http://:8080/healthz-kubedns delay=60s timeout=5s period=10s #success=1 #failure=5
Readiness: http-get http://:8081/readiness delay=3s timeout=5s period=10s #success=1 #failure=3
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-641jn (ro)
Environment Variables: <none>
dnsmasq:
Container ID: docker://2ac895d257c80d6430998e13e85d113249fc341e801f2671266678e6760ddd3f
Image: gcr.io/google_containers/kube-dnsmasq-amd64:1.4
Image ID: docker-pullable://gcr.io/google_containers/kube-dnsmasq-amd64@sha256:a722df15c0cf87779aad8ba2468cf072dd208cb5d7cfcaedd90e66b3da9ea9d2
Ports: 53/UDP, 53/TCP
Args:
--cache-size=1000
--no-resolv
--server=127.0.0.1#10053
--log-facility=-
State: Running
Started: Fri, 03 Feb 2017 14:17:20 +0000
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Fri, 03 Feb 2017 14:15:10 +0000
Finished: Fri, 03 Feb 2017 14:16:49 +0000
Ready: True
Restart Count: 7
Liveness: http-get http://:8080/healthz-dnsmasq delay=60s timeout=5s period=10s #success=1 #failure=5
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-641jn (ro)
Environment Variables: <none>
healthz:
Container ID: docker://104c4102d8be1827b4f9c2679e0d3f4656e2adab80be1f0234d34ef08d5d5e02
Image: gcr.io/google_containers/exechealthz-amd64:1.2
Image ID: docker-pullable://gcr.io/google_containers/exechealthz-amd64@sha256:503e158c3f65ed7399f54010571c7c977ade7fe59010695f48d9650d83488c0a
Port: 8080/TCP
Args:
--cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null
--url=/healthz-dnsmasq
--cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null
--url=/healthz-kubedns
--port=8080
--quiet
Limits:
memory: 50Mi
Requests:
cpu: 10m
memory: 50Mi
State: Running
Started: Fri, 03 Feb 2017 14:01:41 +0000
Ready: True
Restart Count: 0
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-641jn (ro)
Environment Variables: <none>
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
default-token-641jn:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-641jn
QoS Class: Burstable
Tolerations: CriticalAddonsOnly=:Exists
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
17m 17m 1 {default-scheduler } Normal Scheduled Successfully assigned kube-dns-v20-h4w7m to 192.168.1.203
17m 17m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Pulling pulling image "gcr.io/google_containers/kubedns-amd64:1.8"
16m 16m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Pulled Successfully pulled image "gcr.io/google_containers/kubedns-amd64:1.8"
16m 16m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Created Created container with docker id 189afaa1eb0d; Security:[seccomp=unconfined]
16m 16m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Pulling pulling image "gcr.io/google_containers/kube-dnsmasq-amd64:1.4"
16m 16m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Started Started container with docker id 189afaa1eb0d
16m 16m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Pulled Successfully pulled image "gcr.io/google_containers/kube-dnsmasq-amd64:1.4"
16m 16m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Created Created container with docker id a9625236c9a7; Security:[seccomp=unconfined]
16m 16m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Started Started container with docker id a9625236c9a7
16m 16m 1 {kubelet 192.168.1.203} spec.containers{healthz} Normal Pulling pulling image "gcr.io/google_containers/exechealthz-amd64:1.2"
16m 16m 1 {kubelet 192.168.1.203} spec.containers{healthz} Normal Pulled Successfully pulled image "gcr.io/google_containers/exechealthz-amd64:1.2"
16m 16m 1 {kubelet 192.168.1.203} spec.containers{healthz} Normal Created Created container with docker id 104c4102d8be; Security:[seccomp=unconfined]
16m 16m 1 {kubelet 192.168.1.203} spec.containers{healthz} Normal Started Started container with docker id 104c4102d8be
14m 14m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Killing Killing container with docker id a9625236c9a7: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "dnsmasq" is unhealthy, it will be killed and re-created.
14m 14m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Killing Killing container with docker id 189afaa1eb0d: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "kubedns" is unhealthy, it will be killed and re-created.
14m 14m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Started Started container with docker id e9df4ad7de2c
14m 14m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Created Created container with docker id e9df4ad7de2c; Security:[seccomp=unconfined]
14m 14m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Created Created container with docker id 5e5110c2c825; Security:[seccomp=unconfined]
14m 14m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Started Started container with docker id 5e5110c2c825
12m 12m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Killing Killing container with docker id 5e5110c2c825: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "dnsmasq" is unhealthy, it will be killed and re-created.
11m 11m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Killing Killing container with docker id e9df4ad7de2c: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "kubedns" is unhealthy, it will be killed and re-created.
11m 11m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Created Created container with docker id 6bb789dd5cad; Security:[seccomp=unconfined]
11m 11m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Started Started container with docker id 6bb789dd5cad
11m 11m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Started Started container with docker id ae623a958bff
11m 11m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Created Created container with docker id ae623a958bff; Security:[seccomp=unconfined]
10m 10m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Killing Killing container with docker id ae623a958bff: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "dnsmasq" is unhealthy, it will be killed and re-created.
9m 9m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Killing Killing container with docker id 6bb789dd5cad: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "kubedns" is unhealthy, it will be killed and re-created.
9m 9m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Created Created container with docker id 9cca030a8acd; Security:[seccomp=unconfined]
9m 9m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Started Started container with docker id 0cfa05e8f812
9m 9m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Created Created container with docker id 0cfa05e8f812; Security:[seccomp=unconfined]
9m 9m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Started Started container with docker id 9cca030a8acd
8m 8m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Killing Killing container with docker id 9cca030a8acd: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "dnsmasq" is unhealthy, it will be killed and re-created.
7m 7m 1 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Killing Killing container with docker id 0cfa05e8f812: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "kubedns" is unhealthy, it will be killed and re-created.
5m 5m 1 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Killing Killing container with docker id 3c0eeac9a891: pod "kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)" container "dnsmasq" is unhealthy, it will be killed and re-created.
14m 3m 6 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Pulled Container image "gcr.io/google_containers/kubedns-amd64:1.8" already present on machine
15m 2m 11 {kubelet 192.168.1.203} spec.containers{kubedns} Warning Unhealthy Liveness probe failed: Get http://10.10.18.3:8080/healthz-kubedns: dial tcp 10.10.18.3:8080: getsockopt: no route to host
15m 2m 11 {kubelet 192.168.1.203} spec.containers{dnsmasq} Warning Unhealthy Liveness probe failed: Get http://10.10.18.3:8080/healthz-dnsmasq: dial tcp 10.10.18.3:8080: getsockopt: no route to host
16m 1m 94 {kubelet 192.168.1.203} spec.containers{kubedns} Warning Unhealthy Readiness probe failed: Get http://10.10.18.3:8081/readiness: dial tcp 10.10.18.3:8081: getsockopt: no route to host
5m 1m 5 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Killing (events with common reason combined)
14m 1m 7 {kubelet 192.168.1.203} spec.containers{dnsmasq} Normal Pulled Container image "gcr.io/google_containers/kube-dnsmasq-amd64:1.4" already present on machine
7m 1m 7 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Started (events with common reason combined)
7m 1m 7 {kubelet 192.168.1.203} spec.containers{kubedns} Normal Created (events with common reason combined)
1m 5s 7 {kubelet 192.168.1.203} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "kubedns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=kubedns pod=kube-dns-v20-h4w7m_kube-system(3a545c95-ea19-11e6-aa7c-52540021bfab)"
1m 5s 7 {kubelet 192.168.1.203} spec.containers{kubedns} Warning BackOff Back-off restarting failed docker container
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment