Debugging cc-agent

debugging-cc-agent.md

Create a functional Clear Container image using osbuilder

sudo -E IMG_SIZE=520M EXTRA_PKGS="dbus-bin dbus-autostart util-linux-bin p11-kit-bin bash shadow ca-certs dist-pam-configs xz-bin tar-bin grep-bin sed-bin pigz-bin iproute2-bin procps-ng-bin psstop-bin htop-bin curl nano make-bin" make rootfs
sudo -E IMG_SIZE=520M make image

Start a new VM

/usr/bin/qemu-system-x86_64 -machine pc,accel=kvm,kernel_irqchip,nvdimm -cpu host -device nvdimm,id=nv0,memdev=mem0 \
-object memory-backend-file,id=mem0,mem-path=/home/fedora/container.img,size=419430400 -m 2048M,slots=2,maxmem=16998M \
-smp 4,cores=4,threads=1,sockets=1 -vga none -nographic -no-user-config -nodefaults -kernel \
/home/fedora/vmlinuz-4.9.35-76.container -append "root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw \
rootfstype=ext4 tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 \
i8042.noaux=1 noreplace-smp reboot=k panic=1 console=hvc0 console=hvc1 initcall_debug iommu=off cryptomgr.notests net.ifnames=0 \
console=ttyS0" -device virtio-serial-pci,id=virtio-serial0 -chardev stdio,id=charconsole0,signal=off -device \
virtconsole,chardev=charconsole0,id=console0 -net nic,model=virtio -net user,id=net0,net=192.168.76.0/24 -device \
virtserialport,chardev=charch0,id=channel0,name=sh.hyper.channel.0 -chardev socket,id=charch0,path=hyper.sock,server,nowait \
-device virtserialport,chardev=charch1,id=channel1,name=sh.hyper.channel.1 -chardev \
socket,id=charch1,path=tty.sock,server,nowait

To debug systemd and unit files append next options to the kernel command line

systemd.journald.forward_to_console=1 systemd.log_level=debug systemd.log_target=console

To debug cc-agent through console.sock add next line to cc-agent.target (CC 2.x) or clear-containers.target (CC 3.0)

Wants=cc-debug-console.service

and create /usr/lib/systemd/system/cc-debug-console.service with next content

[Unit]
Description=Container debug console

[Service]
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
StandardInput=tty
StandardOutput=tty
PrivateDevices=yes
Type=simple
ExecStart=/usr/bin/bash

connect to console.sock using socat

socat stdin,raw,echo=0,escape=0x11 unix-connect:/path/to/console.sock

To fix failed to load system roots and no roots provided

Error response from daemon: Get https://registry-1.docker.io/v2/: x509: failed to load system roots and no roots provided

systemctl start p11-kit-trigger.service