Skip to content

Instantly share code, notes, and snippets.

@gamename
Created October 22, 2018 17:57
Show Gist options
  • Save gamename/c19e87e015a9a686b205135d1f40b68b to your computer and use it in GitHub Desktop.
Save gamename/c19e87e015a9a686b205135d1f40b68b to your computer and use it in GitHub Desktop.
$ sudo kubeadm init --pod-network-cidr 10.244.0.0/16 --apiserver-advertise-address 10.0.0.1
[init] using Kubernetes version: v1.12.1
[preflight] running pre-flight checks
[preflight/images] Pulling images required for setting up a Kubernetes cluster
[preflight/images] This might take a minute or two, depending on the speed of your internet connection
[preflight/images] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[preflight] Activating the kubelet service
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Generated etcd/ca certificate and key.
[certificates] Generated etcd/healthcheck-client certificate and key.
[certificates] Generated apiserver-etcd-client certificate and key.
[certificates] Generated etcd/server certificate and key.
[certificates] etcd/server serving cert is signed for DNS names [k8s localhost] and IPs [127.0.0.1 ::1]
[certificates] Generated etcd/peer certificate and key.
[certificates] etcd/peer serving cert is signed for DNS names [k8s localhost] and IPs [10.0.0.1 127.0.0.1 ::1]
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [k8s kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.0.0.1]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] valid certificates and keys now exist in "/etc/kubernetes/pki"
[certificates] Generated sa key and public key.
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[controlplane] wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
[init] this might take a minute or longer if the control plane images have to be pulled
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
couldn't initialize a Kubernetes cluster
@gamename
Copy link
Author

$ sudo journalctl -xeu kubelet
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.452572   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Node: Get https://10.0.0.1:6443/api/v1/nodes?fiel
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.453054   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:442: Failed to list *v1.Service: Get https://10.0.0.1:6443/api/v1/service
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.492709   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.593292   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.693849   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.794580   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.895111   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:56 k8s kubelet[19944]: E1022 18:01:56.995830   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.096518   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.197086   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.297546   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.398249   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.454041   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://10.0.0.1:6443/api/v1/pod
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.455583   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Node: Get https://10.0.0.1:6443/api/v1/nodes?fiel
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.458554   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:442: Failed to list *v1.Service: Get https://10.0.0.1:6443/api/v1/service
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.498764   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.599396   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.699982   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: I1022 18:01:57.796543   19944 kubelet_node_status.go:276] Setting node annotation to enable volume controller attach/detach
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.801261   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.817816   19944 azure_dd.go:147] failed to get azure cloud in GetVolumeLimits, plugin.host: k8s
Oct 22 18:01:57 k8s kubelet[19944]: I1022 18:01:57.819255   19944 kubelet_node_status.go:70] Attempting to register node k8s
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.822913   19944 kubelet_node_status.go:92] Unable to register node "k8s" with API server: Post https://10.0.0.1:6443/api/v1/nodes: dial tcp 10.0.0.1
Oct 22 18:01:57 k8s kubelet[19944]: E1022 18:01:57.901864   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.002555   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.103080   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.203720   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.304368   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.404736   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.457633   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://10.0.0.1:6443/api/v1/pod
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.459280   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Node: Get https://10.0.0.1:6443/api/v1/nodes?fiel
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.462082   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:442: Failed to list *v1.Service: Get https://10.0.0.1:6443/api/v1/service
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.505249   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.605826   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.706311   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.806777   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:58 k8s kubelet[19944]: E1022 18:01:58.907268   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.007892   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.108388   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.208996   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.236435   19944 event.go:212] Unable to write event: 'Patch https://10.0.0.1:6443/api/v1/namespaces/default/events/k8s.155ff3d8e88c5803: dial tcp 10
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.252662   19944 certificate_manager.go:348] Failed while requesting a signed certificate from the master: cannot create certificate signing request:
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.309563   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: W1022 18:01:59.324649   19944 cni.go:188] Unable to update cni config: No networks found in /etc/cni/net.d
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.326143   19944 kubelet.go:2167] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.412923   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.461188   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://10.0.0.1:6443/api/v1/pod
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.463575   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Node: Get https://10.0.0.1:6443/api/v1/nodes?fiel
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.464990   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:442: Failed to list *v1.Service: Get https://10.0.0.1:6443/api/v1/service
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.513531   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.614019   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.714542   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.817928   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:01:59 k8s kubelet[19944]: E1022 18:01:59.918416   19944 kubelet.go:2236] node "k8s" not found
lines 948-1001/1001 (END)```

@gamename
Copy link
Author

$ sudo systemctl status kubelet -l
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Mon 2018-10-22 14:15:08 UTC; 3h 48min ago
     Docs: https://kubernetes.io/docs/home/
 Main PID: 19944 (kubelet)
    Tasks: 22 (limit: 4915)
   Memory: 39.9M
      CPU: 31min 48.377s
   CGroup: /system.slice/kubelet.service
           └─19944 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgro

Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.012993   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Node: Get https://10.0.0.1:6443/api/v1/nodes?fiel
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.020511   19944 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://10.0.0.1:6443/api/v1/pod
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.097124   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.197706   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.299211   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.399904   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.502778   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.603245   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.703817   19944 kubelet.go:2236] node "k8s" not found
Oct 22 18:04:08 k8s kubelet[19944]: E1022 18:04:08.805213   19944 kubelet.go:2236] node "k8s" not found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment