mcastelino

QEMU command line

qemu-system-x86_64 \
    -enable-kvm \
    -bios OVMF.fd \
    -smp sockets=1,cpus=4,cores=2 -cpu host \
    -m 1024 \
    -vga none -nographic \
    -netdev user,id=mynet0,hostfwd=tcp::${VMN}0022-:22,hostfwd=tcp::${VMN}2375-:2375 \

mcastelino

Overview

This documents how to get Kubernetes running with Clear Containers using CRIO.

Base OS

Ubuntu 16.04 4.10.0-27-generic

Known working commits

mcastelino

apiVersion: v1
kind: Pod
metadata:
  annotations:
    io.kubernetes.cri-o.trusted: "false"
  name: nginx-untrusted
spec:
  containers:
  - name: nginx
    image: sameo/nginx-kubecon:1.0

mcastelino

Experimental Docker Libnetwork DHCP Driver

The DHCP driver is intended for users to be able to integrate Docker IP address management with their existing IPAM strategies that use DHCP for dynamic address assignment. DHCP enables users to allocate addresses in an organized fashion that will prevent overlapping IP address assignment by associating a unique MAC address from the container eth0 Ethernet interface to an IP address as determined by the DHCP pools defined in the DHCP configuration.

This driver only provides the DHCP client functionality. It does not include a DHCP server. The default driver offers single-host IPAM or for distributed multi-host orchestrated IPAM see the libnetwork overlay driver.

Getting Started

• Download the driver compiled into Docker Engine - docker binary with libnetwork test dhcp client spam driver

mcastelino

# Get qemu
git clone https://github.com/qemu/qemu
# Apply configure patch
curl https://raw.githubusercontent.com/clearcontainers/packaging/master/qemu-lite/configure.patch | patch -p1
# Configure qemu line
./configure --disable-static --disable-bluez --disable-brlapi --disable-bzip2 --disable-curl --disable-curses --disable-debug-tcg --disable-fdt --disable-glusterfs --disable-gtk --disable-libiscsi --disable-libnfs --disable-libssh2 --disable-libusb --disable-linux-aio --disable-lzo --disable-opengl --disable-qom-cast-debug --disable-rbd --disable-rdma --disable-sdl --disable-seccomp --disable-slirp --disable-snappy --disable-spice --disable-strip --disable-tcg-interpreter --disable-tcmalloc --disable-tools --disable-tpm --disable-usb-redir --disable-uuid --disable-vnc --disable-vnc-jpeg --disable-vnc-png --disable-vnc-sasl --disable-vte --disable-xen --enable-attr --enable-cap-ng --enable-kvm --enable-virtfs --target-list=x86_64-softmmu --extra-cflags="-fno-semantic-interposition -O3 -falign-functions=32" -

mcastelino

#First Cleanup
export KUBECONFIG=/etc/kubernetes/admin.conf
sudo -E kubeadm reset
sudo systemctl stop kubelet
sudo systemctl stop docker
for c in `sudo crioctl ctr list | grep ^ID | cut -c5-`; do sudo crioctl ctr stop --id $c; sudo crioctl ctr remove --id $c ; done
for c in `sudo crioctl pod list | grep ^ID | cut -c5-`; do sudo crioctl pod stop --id $c; sudo crioctl pod remove --id $c ; done

sudo systemctl stop crio

mcastelino

Using a Docker CNM plugin to play with SRIO-V

This gist describes the setup necessary for testing SRIO-V based connectivity between two physical boxes which are each setup as described here, and directly connected via their respective SRIO-V enabled NICs.

Setup host system's packages

For this scenario, I'm setting up two Ubuntu 16.04 systems which have a SRIO-V enabled interface as well as a second port for accessing the SUT. To setup:

mcastelino

When you setup github with two factor auth, the easiest way to access it is via ssh. However this can be challenging behind a proxy.

One way is to use connect-proxy

.gitconfig

[url "git@github.com:"]
 insteadOf = https://github.com/

mcastelino

Observation

The issue is only visible when both port forwarding is done on the server side and SNAT is active on the client side. If only one of the two is active (i.e. the client or server is not being NATed) the issue does not seem to show up. At least not easily observed

Test setup:

Server: (IP 10.0.0.6)

mcastelino

How to debug QEMU or Kernel issues for Clear Containers

Manual command line for Clear Container Q35 Launch. This lets you debug w/o needing the entire kubernetes or docker framework.

If you have clear containers installed you can get the kernel and rootfs image from /usr/share/clearcontainers

For q35 machine type

./x86_64-softmmu/qemu-system-x86_64 \