Skip to content

Instantly share code, notes, and snippets.

@tiswanso

tiswanso/magnum minion kubelet

Created February 6, 2018 16:32
Show Gist options
  • Save tiswanso/459f78f5156639c69f70182fe9872362 to your computer and use it in GitHub Desktop.
Save tiswanso/459f78f5156639c69f70182fe9872362 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
magnum minion kubelet
Mod'd /etc/kubernetes/kubelet to add --require-kubeconfig:
<snip>
# Add your own!
KUBELET_ARGS="--pod-manifest-path=/etc/kubernetes/manifests --cadvisor-port=4194 --require-kubeconfig --kubeconfig=/etc/kubernetes/kubeconfig.yaml --hostn
ame-override=k8s-cluster-5ibc6pt6avhz-minion-0 --cluster_dns=10.254.0.10 --cluster_domain=cluster.local --cloud-provider=openstack --cloud-config=/etc/kub
ernetes/kube_openstack_config --cgroup-driver=systemd"
</snip>
[root@k8s-cluster-5ibc6pt6avhz-minion-0 kubernetes]# systemctl restart kubelet
[root@k8s-cluster-5ibc6pt6avhz-minion-0 kubernetes]# systemctl status kubelet
● kubelet.service - kubernetes-kubelet
Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2018-02-06 16:23:57 UTC; 3s ago
Process: 20587 ExecStart=/bin/runc --systemd-cgroup run kubelet (code=exited, status=1/FAILURE)
Main PID: 20587 (code=exited, status=1/FAILURE)
CPU: 41ms
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Unit entered failed state.
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Failed with result 'exit-code'.
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Stopped kubernetes-kubelet.
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Start request repeated too quickly.
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Failed to start kubernetes-kubelet.
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Unit entered failed state.
Feb 06 16:23:57 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Failed with result 'exit-code'.
[root@k8s-cluster-5ibc6pt6avhz-minion-0 kubernetes]# journalctl -t kubelet
-- No entries --
[root@k8s-cluster-5ibc6pt6avhz-minion-0 kubernetes]# runc list
ID PID STATUS BUNDLE CREATED OWNER
kube-proxy 2084 running /sysroot/ostree/deploy/fedora-atomic/var/lib/containers/atomic/kube-proxy.0 2018-02-06T03:06:08.186508617Z root
[root@k8s-cluster-5ibc6pt6avhz-minion-0 kubernetes]# systemctl start kubelet
[root@k8s-cluster-5ibc6pt6avhz-minion-0 kubernetes]# systemctl status kubelet
● kubelet.service - kubernetes-kubelet
Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2018-02-06 16:31:32 UTC; 6s ago
Process: 21013 ExecStart=/bin/runc --systemd-cgroup run kubelet (code=exited, status=1/FAILURE)
Main PID: 21013 (code=exited, status=1/FAILURE)
CPU: 40ms
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Unit entered failed state.
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Failed with result 'exit-code'.
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Stopped kubernetes-kubelet.
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Start request repeated too quickly.
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Failed to start kubernetes-kubelet.
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Unit entered failed state.
Feb 06 16:31:32 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Failed with result 'exit-code'.
< snip >
Feb 06 16:26:08 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Feb 06 16:26:08 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Stopped kubernetes-kubelet.
Feb 06 16:26:08 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Started kubernetes-kubelet.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: libcontainer-20724-systemd-test-default-dependencies.scope: Scope has no PIDs. Refusing.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: libcontainer-20724-systemd-test-default-dependencies.scope: Scope has no PIDs. Refusing.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Created slice libcontainer_20724_systemd_test_default.slice.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Removed slice libcontainer_20724_systemd_test_default.slice.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Started libcontainer container kubelet.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Unit entered failed state.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Failed with result 'exit-code'.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Stopped kubernetes-kubelet.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Started kubernetes-kubelet.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: libcontainer-20743-systemd-test-default-dependencies.scope: Scope has no PIDs. Refusing.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: libcontainer-20743-systemd-test-default-dependencies.scope: Scope has no PIDs. Refusing.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Created slice libcontainer_20743_systemd_test_default.slice.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Removed slice libcontainer_20743_systemd_test_default.slice.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Started libcontainer container kubelet.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Unit entered failed state.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Failed with result 'exit-code'.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Stopped kubernetes-kubelet.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Start request repeated too quickly.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: Failed to start kubernetes-kubelet.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Unit entered failed state.
Feb 06 16:26:09 k8s-cluster-5ibc6pt6avhz-minion-0.novalocal systemd[1]: kubelet.service: Failed with result 'exit-code'.
</snip>
@tiswanso
Copy link
Author

tiswanso commented Feb 6, 2018 

[root@k8s-cluster-5ibc6pt6avhz-minion-0 sysconfig]# systemctl cat kubelet
# /etc/systemd/system/kubelet.service
[Unit]
Description=kubernetes-kubelet
After=docker.service

[Service]
ExecStart=/bin/runc --systemd-cgroup run 'kubelet'
ExecStop=/bin/runc --systemd-cgroup kill 'kubelet'
Restart=on-failure
WorkingDirectory=/sysroot/ostree/deploy/fedora-atomic/var/lib/containers/atomic/kubelet.0

[Install]
WantedBy=multi-user.target

[root@k8s-cluster-5ibc6pt6avhz-minion-0 sysconfig]# /bin/runc --systemd-cgroup run 'kubelet'
JSON specification file config.json not found
[root@k8s-cluster-5ibc6pt6avhz-minion-0 sysconfig]# cd /sysroot/ostree/deploy/fedora-atomic/var/lib/containers/atomic/kubelet.0
[root@k8s-cluster-5ibc6pt6avhz-minion-0 kubelet.0]# /bin/runc --systemd-cgroup run 'kubelet'
2018-02-06 16:48:36.813185 I | proto: duplicate proto type registered: google.protobuf.Any
2018-02-06 16:48:36.813467 I | proto: duplicate proto type registered: google.protobuf.Duration
2018-02-06 16:48:36.813551 I | proto: duplicate proto type registered: google.protobuf.Timestamp
Flag --api-servers has been deprecated, Use --kubeconfig instead. Will be removed in a future version.
I0206 16:48:36.863566    1716 feature_gate.go:144] feature gates: map[]
Error: failed to run Kubelet: could not init cloud provider "openstack": Resource not found
Error: failed to run Kubelet: could not init cloud provider "openstack": Resource not found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment