Created
November 23, 2020 15:55
-
-
Save JohnBat26/c90ed5609867cdc2c673145250c3791f to your computer and use it in GitHub Desktop.
"device or resource busy" when update k3s
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| kubectl describe node rnd-cloud1-master2 | |
| Name: rnd-cloud1-master2 | |
| Roles: etcd,master | |
| Labels: beta.kubernetes.io/arch=amd64 | |
| beta.kubernetes.io/instance-type=k3s | |
| beta.kubernetes.io/os=linux | |
| k3s-upgrade=true | |
| k3s.io/hostname=rnd-cloud1-master2 | |
| k3s.io/internal-ip=172.20.1.1 | |
| kubernetes.io/arch=amd64 | |
| kubernetes.io/hostname=rnd-cloud1-master2 | |
| kubernetes.io/os=linux | |
| node-role.kubernetes.io/etcd=true | |
| node-role.kubernetes.io/master=true | |
| node.kubernetes.io/instance-type=k3s | |
| Annotations: etcd.k3s.cattle.io/node-address: 172.20.1.1 | |
| etcd.k3s.cattle.io/node-name: rnd-cloud1-master2-40e80463 | |
| flannel.alpha.coreos.com/backend-data: {"VtepMAC":"62:8d:cd:11:97:e6"} | |
| flannel.alpha.coreos.com/backend-type: vxlan | |
| flannel.alpha.coreos.com/kube-subnet-manager: true | |
| flannel.alpha.coreos.com/public-ip: 172.20.1.1 | |
| k3s.io/node-args: ["server","--server","https://rnd-cloud1-master1:6443","--disable","traefik"] | |
| k3s.io/node-config-hash: GF5JJRHTULXEHDPBCPKERU5SVO4FM4WQB56VBPXDN7S5LZNZAC7Q==== | |
| k3s.io/node-env: | |
| {"K3S_DATA_DIR":"/var/lib/rancher/k3s/data/223e6420f8db0d8828a8f5ed3c44489bb8eb47aa71485404f8af8c462a29bea3","K3S_TOKEN":"********"} | |
| node.alpha.kubernetes.io/ttl: 0 | |
| volumes.kubernetes.io/controller-managed-attach-detach: true | |
| CreationTimestamp: Mon, 16 Nov 2020 21:22:41 +0300 | |
| Taints: node.kubernetes.io/unschedulable:NoSchedule | |
| Unschedulable: true | |
| Lease: | |
| HolderIdentity: rnd-cloud1-master2 | |
| AcquireTime: <unset> | |
| RenewTime: Mon, 23 Nov 2020 18:54:50 +0300 | |
| Conditions: | |
| Type Status LastHeartbeatTime LastTransitionTime Reason Message | |
| ---- ------ ----------------- ------------------ ------ ------- | |
| NetworkUnavailable False Thu, 19 Nov 2020 00:00:30 +0300 Thu, 19 Nov 2020 00:00:30 +0300 FlannelIsUp Flannel is running on this node | |
| MemoryPressure False Mon, 23 Nov 2020 18:54:30 +0300 Mon, 16 Nov 2020 21:22:40 +0300 KubeletHasSufficientMemory kubelet has sufficient memory available | |
| DiskPressure False Mon, 23 Nov 2020 18:54:30 +0300 Mon, 16 Nov 2020 21:22:40 +0300 KubeletHasNoDiskPressure kubelet has no disk pressure | |
| PIDPressure False Mon, 23 Nov 2020 18:54:30 +0300 Mon, 16 Nov 2020 21:22:40 +0300 KubeletHasSufficientPID kubelet has sufficient PID available | |
| Ready True Mon, 23 Nov 2020 18:54:30 +0300 Mon, 16 Nov 2020 21:22:51 +0300 KubeletReady kubelet is posting ready status | |
| Addresses: | |
| InternalIP: 172.20.1.1 | |
| Hostname: rnd-cloud1-master2 | |
| Capacity: | |
| cpu: 4 | |
| ephemeral-storage: 74203752Ki | |
| hugepages-2Mi: 0 | |
| memory: 12139344Ki | |
| pods: 110 | |
| Allocatable: | |
| cpu: 4 | |
| ephemeral-storage: 72185409889 | |
| hugepages-2Mi: 0 | |
| memory: 12139344Ki | |
| pods: 110 | |
| System Info: | |
| Machine ID: ebf4b5a6219f4b02bea8dca5c0832f0d | |
| System UUID: 42188C99-82E4-8782-2158-6EFE60C24362 | |
| Boot ID: 3b7bccb1-63cb-4ddb-930c-e5d5854feaf4 | |
| Kernel Version: 3.10.0-693.el7.x86_64 | |
| OS Image: CentOS Linux 7 (Core) | |
| Operating System: linux | |
| Architecture: amd64 | |
| Container Runtime Version: containerd://1.4.1-k3s1 | |
| Kubelet Version: v1.19.3+k3s3 | |
| Kube-Proxy Version: v1.19.3+k3s3 | |
| PodCIDR: 10.42.1.0/24 | |
| PodCIDRs: 10.42.1.0/24 | |
| ProviderID: k3s://rnd-cloud1-master2 | |
| Non-terminated Pods: (6 in total) | |
| Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE | |
| --------- ---- ------------ ---------- --------------- ------------- --- | |
| drupal mysite-mariadb-0 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m21s | |
| ingress svclb-traefik-dcj4p 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6d21h | |
| lens-metrics node-exporter-4b6gk 10m (0%) 200m (5%) 24Mi (0%) 100Mi (0%) 6d20h | |
| lens-metrics prometheus-0 100m (2%) 0 (0%) 512Mi (4%) 0 (0%) 6d20h | |
| system-upgrade apply-k3s-latest-on-rnd-cloud1-master2-with-e50d232791db2-sjn2s 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m36s | |
| wp svclb-wordpress-6ql29 0 (0%) 0 (0%) 0 (0%) 0 (0%) 28h | |
| Allocated resources: | |
| (Total limits may be over 100 percent, i.e., overcommitted.) | |
| Resource Requests Limits | |
| -------- -------- ------ | |
| cpu 110m (2%) 200m (5%) | |
| memory 536Mi (4%) 100Mi (0%) | |
| ephemeral-storage 0 (0%) 0 (0%) | |
| hugepages-2Mi 0 (0%) 0 (0%) | |
| Events: | |
| Type Reason Age From Message | |
| ---- ------ ---- ---- ------- | |
| Normal NodeSchedulable 5m15s kubelet Node rnd-cloud1-master2 status is now: NodeSchedulable | |
| Normal NodeNotSchedulable 5m5s (x2 over 10m) kubelet Node rnd-cloud1-master2 status is now: NodeNotSchedulable |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ip: check network namespace closed: remove netns: unlinkat /var/run/netns/cni-30e725ee-1a01-5fb2-9d3a-0e674002c5a7: device or resource busy
Nov 23 18:51:07 rnd-cloud1-master2 k3s[2458]: E1123 18:51:07.941740 2458 pod_workers.go:191] Error syncing pod d5344bc5-d65a-4b99-a344-90187792767f ("svclb-wordpress-6ql29_wp(d5344bc5-d65a-4b99-a344-90187792767f)"), skipping: rpc error: code = Unknown desc = failed to get sandbox ip: check network namespace closed: remove netns: unlinkat /var/run/netns/cni-1efffad6-8a0c-9363-0d4b-b082af4cc0ff: device or resource busy