Skip to content

Instantly share code, notes, and snippets.

@mumoshu
Created April 20, 2017 02:10
Show Gist options
  • Save mumoshu/2e0b9e6887a85ad542c83c2b9745b9d3 to your computer and use it in GitHub Desktop.
Save mumoshu/2e0b9e6887a85ad542c83c2b9745b9d3 to your computer and use it in GitHub Desktop.
kube-aws v0.9.6-rc.2+dirty NodeLost + CrashLoopBackOff kube-dns/dashboard
$ ETCD_COUNT=3 CONTROLLER_COUNT=2 KUBE_AWS_CLUSTER_NAME=kubea7 LIMIT_SSH_ACCESS=1 KUBE_AWS_USE_CALICO=1 ETCD_SNAPSHOT_AUTOMATED=1 ./run kubectl get po --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
kube-system calico-node-1bqcl 1/1 NodeLost 0 6d 10.0.0.52 ip-10-0-0-52.ap-northeast-1.compute.internal
kube-system calico-node-btqkg 1/1 Running 0 1h 10.0.0.164 ip-10-0-0-164.ap-northeast-1.compute.internal
kube-system calico-node-ggjx3 1/1 Running 0 18h 10.0.0.197 ip-10-0-0-197.ap-northeast-1.compute.internal
kube-system calico-node-lbmgg 1/1 Running 0 18h 10.0.0.104 ip-10-0-0-104.ap-northeast-1.compute.internal
kube-system calico-node-qh16k 1/1 Running 0 18h 10.0.0.83 ip-10-0-0-83.ap-northeast-1.compute.internal
kube-system calico-node-qzckn 1/1 Running 0 1h 10.0.0.128 ip-10-0-0-128.ap-northeast-1.compute.internal
kube-system calico-node-rl00f 1/1 Running 0 18h 10.0.0.139 ip-10-0-0-139.ap-northeast-1.compute.internal
kube-system calico-policy-controller-2947401832-0bxgv 1/1 Unknown 0 6d 10.0.0.52 ip-10-0-0-52.ap-northeast-1.compute.internal
kube-system calico-policy-controller-2947401832-swvt6 1/1 Running 0 18h 10.0.0.83 ip-10-0-0-83.ap-northeast-1.compute.internal
kube-system heapster-v1.3.0-268032834-cxdtw 2/2 Running 0 1h 10.2.84.2 ip-10-0-0-197.ap-northeast-1.compute.internal
kube-system kube-apiserver-ip-10-0-0-139.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.139 ip-10-0-0-139.ap-northeast-1.compute.internal
kube-system kube-apiserver-ip-10-0-0-52.ap-northeast-1.compute.internal 1/1 Unknown 0 6d 10.0.0.52 ip-10-0-0-52.ap-northeast-1.compute.internal
kube-system kube-apiserver-ip-10-0-0-83.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.83 ip-10-0-0-83.ap-northeast-1.compute.internal
kube-system kube-controller-manager-ip-10-0-0-139.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.139 ip-10-0-0-139.ap-northeast-1.compute.internal
kube-system kube-controller-manager-ip-10-0-0-52.ap-northeast-1.compute.internal 1/1 Unknown 0 6d 10.0.0.52 ip-10-0-0-52.ap-northeast-1.compute.internal
kube-system kube-controller-manager-ip-10-0-0-83.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.83 ip-10-0-0-83.ap-northeast-1.compute.internal
kube-system kube-dns-3816048056-m7c3v 2/4 CrashLoopBackOff 54 1h 10.2.106.2 ip-10-0-0-104.ap-northeast-1.compute.internal
kube-system kube-dns-3816048056-tgvrp 3/4 CrashLoopBackOff 51 1h 10.2.84.4 ip-10-0-0-197.ap-northeast-1.compute.internal
kube-system kube-dns-autoscaler-1464605019-62442 1/1 Running 0 1h 10.2.84.3 ip-10-0-0-197.ap-northeast-1.compute.internal
kube-system kube-proxy-ip-10-0-0-104.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.104 ip-10-0-0-104.ap-northeast-1.compute.internal
kube-system kube-proxy-ip-10-0-0-128.ap-northeast-1.compute.internal 1/1 Running 0 1h 10.0.0.128 ip-10-0-0-128.ap-northeast-1.compute.internal
kube-system kube-proxy-ip-10-0-0-139.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.139 ip-10-0-0-139.ap-northeast-1.compute.internal
kube-system kube-proxy-ip-10-0-0-164.ap-northeast-1.compute.internal 1/1 Running 0 1h 10.0.0.164 ip-10-0-0-164.ap-northeast-1.compute.internal
kube-system kube-proxy-ip-10-0-0-197.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.197 ip-10-0-0-197.ap-northeast-1.compute.internal
kube-system kube-proxy-ip-10-0-0-52.ap-northeast-1.compute.internal 1/1 Unknown 0 6d 10.0.0.52 ip-10-0-0-52.ap-northeast-1.compute.internal
kube-system kube-proxy-ip-10-0-0-83.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.83 ip-10-0-0-83.ap-northeast-1.compute.internal
kube-system kube-scheduler-ip-10-0-0-139.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.139 ip-10-0-0-139.ap-northeast-1.compute.internal
kube-system kube-scheduler-ip-10-0-0-52.ap-northeast-1.compute.internal 1/1 Unknown 1 6d 10.0.0.52 ip-10-0-0-52.ap-northeast-1.compute.internal
kube-system kube-scheduler-ip-10-0-0-83.ap-northeast-1.compute.internal 1/1 Running 0 18h 10.0.0.83 ip-10-0-0-83.ap-northeast-1.compute.internal
kube-system kubernetes-dashboard-v1.5.1-0fx3g 0/1 CrashLoopBackOff 23 1h 10.2.106.3 ip-10-0-0-104.ap-northeast-1.compute.internal
$ ETCD_COUNT=3 CONTROLLER_COUNT=2 KUBE_AWS_CLUSTER_NAME=kubea7 LIMIT_SSH_ACCESS=1 KUBE_AWS_USE_CALICO=1 ETCD_SNAPSHOT_AUTOMATED=1 ./run kubectl get no
Using the kube-aws command at /Users/cw_kuoka/Gopath/src/github.com/kubernetes-incubator/kube-aws/bin/kube-aws(kube-aws version 30dccf548e78fc8e96569fbf680c7098dff3f771+dirty). Set KUBE_AWS_CMD=path/to/kube-aws to override.
NAME STATUS AGE VERSION
ip-10-0-0-104.ap-northeast-1.compute.internal Ready 18h v1.6.1+coreos.0
ip-10-0-0-128.ap-northeast-1.compute.internal Ready 1h v1.6.1+coreos.0
ip-10-0-0-139.ap-northeast-1.compute.internal Ready 18h v1.6.1+coreos.0
ip-10-0-0-164.ap-northeast-1.compute.internal Ready 1h v1.6.1+coreos.0
ip-10-0-0-197.ap-northeast-1.compute.internal Ready 18h v1.6.1+coreos.0
ip-10-0-0-52.ap-northeast-1.compute.internal NotReady 6d v1.6.1+coreos.0
ip-10-0-0-83.ap-northeast-1.compute.internal Ready 18h v1.6.1+coreos.0
@cknowles
Copy link

What do the DNS logs say? Anything like the server has asked for the client to provide credentials?

@cknowles
Copy link

@mumoshu, were you able to look at the logs? I'm trying to diagnose this issue further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment