alban/kubectl-trace-demo-fosdem.md

## kubectl-trace-demo-fosdem.md

      
    Raw
  

              kubectl-trace-demo-fosdem.md
            
          
    To enable cgroup-v2 in a Kubernetes cluster.
To enable the hybrid mode on Flatcar Linux:

sudo vi /usr/share/oem/grub.cfg
set linux_append="...... systemd.unified_cgroup_hierarchy=false systemd.legacy_systemd_cgroup_controller=false"
sudo reboot

To enable this in Docker:
$ echo 'DOCKER_OPTS="--exec-opt native.cgroupdriver=systemd"' |  sudo tee -a /run/metadata/torcx

Or alternatively:
$ sudo mkdir  /etc/systemd/system/docker.service.d
$ printf '[Service]\nEnvironment=DOCKER_OPTS="--exec-opt native.cgroupdriver=systemd"\n' | sudo tee /etc/systemd/system/docker.service.d/10-docker.conf

And then:
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
$ sudo docker run -ti --rm busybox cat /proc/self/cgroup |grep ^[01]:
1:name=systemd:/system.slice/docker-5631ac3634cfd17a2ae5d9068c4121fbe2f392020b9db3d780e571c6b8954db3.scope
0::/system.slice/docker-5631ac3634cfd17a2ae5d9068c4121fbe2f392020b9db3d780e571c6b8954db3.scope

If you use the kubelet on the node, ensure it is started with --cgroup-driver=systemd:
vim /etc/systemd/system/kubelet.service
...

If you use containerd on the node, you need:
sudo mount -o remount,rw /run/torcx/unpack/
printf '\n\n[plugins.cri]\nsystemd_cgroup = true\n' | sudo tee -a /run/torcx/unpack/docker/usr/share/containerd/config.toml
sudo sed -i 's/^disabled_plugins.*$/disabled_plugins = []/g' /run/torcx/unpack/docker/usr/share/containerd/config.toml
sudo systemctl restart containerd

(it will work for processes started in the container, but not for processes entering via kubectl exec since containerd will not setup the cgroup-v2 there)