mcastelino

MACVLAN

ip netns del net1
ip netns add net1
ip link add macvlan1 link enp130s0f0 type macvlan mode bridge
ip link set macvlan1 netns net1 up
ip netns exec net1 ip a add 192.168.1.200/24 dev macvlan1
ip netns exec net1 ping 192.168.1.101 -c1

mcastelino

Prepare the host

Get the image clear-XXXXX-kvm.img

wget https://download.clearlinux.org/releases/30080/clear/clear-30080-kvm.img.xz
unxz clear-30080-kvm.img.xz

Add intel_iommu=on to the kernel boot parameters

mkdir mount_dir

mcastelino

Vagrant-libvirt QEMU customization

Vagrant libvirt allows the user to customize the virtual machines to add features and use QEMU options that are not exposed via the plugin itself. For this it uses the libvirt's custom QEMU arguments options which appends the args at the very end of the command line. This allows you to not only add new arguments, it allows you to also override the plugin constructed defaults.

In the snippet below the

• machine and memory default plugin arguments are over-ridden
• an additional NVDIMM device is created and added to the machine

mcastelino

Most of the difference comes from the virtio-blk boot. Systemd to kata-agent boot times are similar

[    1.002949] Freeing unused kernel image memory: 2016K                                                                                                                           |[    0.844629] Freeing unused kernel image memory: 2016K
[    1.018758] Freeing unused kernel image memory: 1204K                                                                                                                           |[    0.844878] Freeing unused kernel image memory: 1204K
[    1.018838] Run /usr/lib/systemd/systemd as init process                                                                                                                        |[    0.845072] Run /usr/lib/systemd/systemd as init process
[    1.248975] systemd[1]: systemd 241 running in system mode. (+PAM +AUDIT -SELINUX +IMA -APPARMOR -SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +B|[    1.004513] systemd[1]

mcastelino

Quick guide for getting Kata+containerd (using v2 shim) up and running super quick on bionic

Installation of Kube stuff on bionic:

Use the following sh:

sudo bash -c "cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial-unstable main
EOF"

mcastelino

group Updates in Kata

Containerd Handling Today

The hierarchy and cgroup handling seems pragmatic in the case of containerd. The container cgroups are currently placed under the podcgroup.

Output from containerd guaranteed pod, with two containers:

mcastelino

In the not-dockershim and not-CRIO normal socket path case, we are handled by the cri stats provider: https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/stats/cri_stats_provider.go

The 'magic' happens in the listPodStats function

Looping over each managed container, kubelet calculates the container statics at https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/stats/cri_stats_provider.go#L198, then calculate a running total of the pod usage at https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/stats/cri_stats_provider.go#L200, and then eventually return the results.

Potential issue

Initial potential issue I noticed was that we will run into is at the top of addPodCPUMemoryStats function. The

mcastelino

Workload

When running a simple workload such as

apiVersion: v1
kind: Pod
metadata:
  name: guar-2kc
spec:

mcastelino

Footloose as a Kubernetes POD. So a VM in Kubernetes launched and managed as a container using the Kata runtimeClass.

apiVersion: v1
data:
    authorized_keys: |
      ssh-rsa PUT_YOUR_KEY_HERE
kind: ConfigMap
metadata:
  name: ssh-pub-key

mcastelino

Machine containers with Kata

kata-footloose$ docker info  | grep runtime
WARNING: No swap limit support
Runtimes: kata-runtime runc
Default Runtime: kata-runtime
kata-footloose$ footloose show
  NAME                HOSTNAME    PORTS      IP          IMAGE                                  CMD         STATE    
  cluster-fedora0     fedora0     22->32768  172.17.0.2  quay.io/footloose/fedora29:latest      /sbin/init  Running