Skip to content

Instantly share code, notes, and snippets.

View ganeshmaharaj's full-sized avatar

Ganesh Maharaj Mahalingam ganeshmaharaj

23 followers · 23 following
  • Somewhere over the rainbow
View GitHub Profile
@ganeshmaharaj
ganeshmaharaj / cpu-cgroups-udev-rules.md
Last active April 4, 2022 16:56
automate update all cgroup cpus in a system using udev rules

cgroups, containers and udev rules

This gist is mostly to document my journey of understanding how cpusets are setup for containers when cores go offline and how I used udev rules to automate the cpuset files.

PS: This topic is only valid for cgroups v1. I am yet to play this out with cgroupsv2 and see how it behaves

Problem

From within a container, setting thread affinity to an offlined and onlined core will fail even though the core is back online and you can schedule onto it from the host.

cgroups and cpusets

I will not cover cgroups in-depth here. There are a ton of articles in the internet that talk and explain cgroups much better than I can. I will stick with cpusets and my observations of that with containers.

@ganeshmaharaj
ganeshmaharaj / kind-net.md
Created October 26, 2021 16:58
Kind confgure subnet

Kind network

By default, kind uses docker to create the network subnet that will be used by it's containers. Docker's default network range has 172.{30,31}.XX.XX in the list and if your network overlaps with that, you will run into interesting issues of not being able to reach your intranet systems.

The alternate would be to create a docker network for kind and ask it to use that instead of creating a default subnet from it's pool.

docker network create --subnet 192.168.80.0/16 --gateway 192.168.80.1 -d=bridge -o com.docker.network.bridge.enable_ip_masquerade=true kind
@ganeshmaharaj
ganeshmaharaj / kata-1.11.1-virtiofs-crash.log
Created July 2, 2020 22:16
Jul 02 22:06:43 clr-01 systemd-coredump[424552]: Process 378019 (virtiofsd) of user 0 dumped core.
Stack trace of thread 378563:
#0 0x00000000004e2627 __clock_gettime (/opt/kata/bin/virtiofsd + 0xe2627)
#1 0x0000000000427361 g_get_monotonic_time (/opt/kata/bin/virtiofsd + 0x27361)
#2 0x00000000004568af g_async_queue_timeout_pop (/opt/kata/bin/virtiofsd + 0x568af)
#3 0x000000000043cbfe g_thread_pool_thread_proxy (/opt/kata/bin/virtiofsd + 0x3cbfe)
#4 0x000000000043c155 g_thread_proxy (/opt/kata/bin/virtiofsd + 0x3c155)
@ganeshmaharaj
ganeshmaharaj / .config.rb
Created July 12, 2019 17:18
vagrant-stuff-voerride
$os = "fedora30"
$cpus = 6
$mem = 8192
@ganeshmaharaj
ganeshmaharaj / kata-release-packaging-fix.txt
Created June 19, 2019 15:50
ganeshma@ganeshma-desk <> packaging % git diff
diff --git a/obs-packaging/gen_versions_txt.sh b/obs-packaging/gen_versions_txt.sh
index 09d96ca..3fd888d 100755
--- a/obs-packaging/gen_versions_txt.sh
+++ b/obs-packaging/gen_versions_txt.sh
@@ -9,6 +9,7 @@ set -e
set -o errexit
set -o nounset
set -o pipefail
+set -o xtrace
@ganeshmaharaj
ganeshmaharaj / kata-virtiofs-mem-hotplug.log
Created June 14, 2019 21:25
Jun 14 21:15:24 virtiofs dockerd[15027]: time="2019-06-14T21:15:24.287258626Z" level=warning msg="Your kernel does not support swap limit capabilities,or the cgroup is not mounted. Memory limited without swap."
Jun 14 21:15:24 virtiofs kernel: docker0: port 1(veth96a104a) entered blocking state
Jun 14 21:15:24 virtiofs kernel: docker0: port 1(veth96a104a) entered disabled state
Jun 14 21:15:24 virtiofs kernel: device veth96a104a entered promiscuous mode
Jun 14 21:15:24 virtiofs kernel: IPv6: ADDRCONF(NETDEV_UP): veth96a104a: link is not ready
Jun 14 21:15:24 virtiofs systemd-udevd[16305]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 14 21:15:24 virtiofs systemd-udevd[16305]: Could not generate persistent MAC address for vethb0d974b: No such file or directory
Jun 14 21:15:24 virtiofs networkd-dispatcher[903]: WARNING:Unknown index 13 seen, reloading interface list
Jun 14 21:15:24 virtiofs systemd-udevd[16306]: link_config: autonegotiation is unset or enabled, the
@ganeshmaharaj
ganeshmaharaj / clearlinux-crio.log
Created June 13, 2019 20:20
Jun 13 19:07:51 clr-01 systemd[1]: crio.service: Current command vanished from the unit file, execution of the command list won't be resumed.
Jun 13 19:07:58 clr-01 systemd[1]: Stopping Open Container Initiative Daemon...
Jun 13 19:07:58 clr-01 systemd[1]: crio.service: Succeeded.
Jun 13 19:07:58 clr-01 systemd[1]: Stopped Open Container Initiative Daemon.
Jun 13 19:07:58 clr-01 systemd[1]: Starting Open Container Initiative Daemon...
Jun 13 19:07:58 clr-01 crio[19147]: time="2019-06-13 19:07:58.244706642+01:00" level=debug msg="found valid runtime 'runc' for runtime_path '/usr/bin/runc'
Jun 13 19:07:58 clr-01 crio[19147]: "
Jun 13 19:07:58 clr-01 crio[19147]: time="2019-06-13 19:07:58.244782537+01:00" level=debug msg="found valid runtime 'kata' for runtime_path '/usr/bin/kata-runtime'
Jun 13 19:07:58 clr-01 crio[19147]: "
Jun 13 19:07:58 clr-01 crio[19147]: time="2019-06-13 19:07:58.244796150+01:00" level=debug msg="found valid runtime 'kata-qemu' for runtime_path '/opt/kata/bin/kata-qemu'
@ganeshmaharaj
ganeshmaharaj / clearlinux-crio.conf
Created June 13, 2019 19:21
# The CRI-O configuration file specifies all of the available configuration
# options and command-line flags for the crio(8) OCI Kubernetes Container Runtime
# daemon, but in a TOML format that can be more easily modified and versioned.
#
# Please refer to crio.conf(5) for details of all configuration options.
# CRI-O reads its storage defaults from the containers-storage.conf(5) file
# located at /etc/containers/storage.conf. Modify this storage configuration if
# you want to change the system's defaults. If you want to modify storage just
@ganeshmaharaj
ganeshmaharaj / crio.log
Created June 13, 2019 18:33
This file has been truncated, but you can view the full file.
-- Logs begin at Wed 2019-06-12 10:34:54 PDT, end at Thu 2019-06-13 11:24:37 PDT. --
Jun 13 11:24:37 ubuntu-01 crio[30579]: time="2019-06-13 11:24:37.098741403-07:00" level=debug msg="ContainerStatusResponse: &ContainerStatusResponse{Status:&ContainerStatus{Id:67c19030f31bb2e475159a7e563f0527fb7a9efd3d8a0a5c939387e1bfdc38c0,Metadata:&ContainerMetadata{Name:coredns,Attempt:5,},State:CONTAINER_EXITED,CreatedAt:1560449385409753008,StartedAt:1560449385473380514,FinishedAt:1560450264336000000,ExitCode:0,Image:&ImageSpec{Image:k8s.gcr.io/coredns:1.3.1,},ImageRef:k8s.gcr.io/coredns@sha256:638adb0319813f2479ba3642bbe37136db8cf363b48fb3eb7dc8db634d8d5a5b,Reason:Completed,Message:,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-fb8b8dccf-vq4vp,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: ee0d1a37-8d55-11e9-a26e-5254002b6b5c,},Annotations:map[string]string{io.kubernetes.container.hash: a5a650f6,io.kubernetes.container.ports: [{\"name\":\"dns\",\"container
@ganeshmaharaj
ganeshmaharaj / crio.conf
Created June 13, 2019 17:59
# The CRI-O configuration file specifies all of the available configuration
# options and command-line flags for the crio(8) OCI Kubernetes Container Runtime
# daemon, but in a TOML format that can be more easily modified and versioned.
#
# Please refer to crio.conf(5) for details of all configuration options.
# CRI-O supports partial configuration reload during runtime, which can be
# done by sending SIGHUP to the running process. Currently supported options
# are explicitly mentioned with: 'This option supports live configuration