mcastelino

Overview

The goal of this document to cover all aspects of Kubernetes management, including how resources are expressed, constrained and accounted for. This started a way to ensure that alternate container runtime implementation like Kata containers will behave from a resource accounting and consumption point of view in the same manner as runc.

Location of the latest version of this document: https://gist.github.com/mcastelino/b8ce9a70b00ee56036dadd70ded53e9f

If you do not understand cgroups please refer to a quick primer at the bottom of this document. This will help you understand how the resource enforcement actually works.

Kubernetes Resource Management

mcastelino

Create a process with its own network and pid namespace

$ unshare --user --map-root-user --net --mount
[root@incensed-gawain ~]# echo $$
2646

Run the slirp process on the host

mcastelino

Setup ccloudvm

For more info https://github.com/intel/ccloudvm

go get github.com/intel/ccloudvm/...
$ ccloudvm setup

Create a ccloud fedora VM

mcastelino

What is running

ps auxw | grep k3s
root      1626  0.0  0.0  63972  4328 pts/1    S    23:26   0:00 sudo ./k3s server
root      1627 11.9  1.4 333516 240412 pts/1   Sl   23:26   0:44 ./k3s server
root      1679  1.6  0.5 191392 83812 pts/1    Sl   23:26   0:06 containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd
root      2014  0.0  0.0 108760  5672 pts/1    Sl   23:27   0:00 containerd-shim -namespace k8s.io -workdir /var/lib/rancher/k3s/agent/containerd/io.containerd.runtime.v1.linux/k8s.io/951dd6bb47ef012424dc526ef663bdeef1fe20942aa416a1b0c0ac5fcccf4b40 -address /run/k3s/containerd/containerd.sock -containerd-binary /var/lib/rancher/k3s/data/4df430e1473d0225734948e562863c82f20d658ed9c420c77e168aec42eccdb5/bin/containerd
root      2092  0.0  0.0 108760  5452 pts/1    Sl   23:27   0:00 containerd-shim -namespace k8s.io -workdir /var/lib/rancher/k3s/agent/containerd/io.containe

mcastelino

Trying Kata Containers with Firecracker (and QEMU)

Clearlinux bundles Kata Containers as well a firecracker.

To quickly experience how Kata Containers can be used to setup a cluster that can run Kubernetes with different types of isolation mechanisms we have created a simple developer enviornment. With this you can run workloads with runc (using cgroups, namespaces provided by the host kernel for isolation), Kata with QEMU/KVM (uses VT-x for isolation and QEMU as the hypervisor) and Kata with Firecracker (uses VT-x for isolation and the minimal Firecracker VMM).

mcastelino

How to do a Kata Containers Release

Hi if you are reading this document you may also want to create a kata Containers Release.

The Kata Containers Release Process is defined in the follwoing documents.

https://github.com/kata-containers/documentation/blob/master/Releases.md

To simply the process of read each release we have created a checklist for it.

mcastelino

{
  "kind": "AdmissionReview",
  "apiVersion": "admission.k8s.io/v1beta1",
  "request": {
    "uid": "1a41e160-f369-11e8-b020-000d3afdac62",
    "kind": {
      "group": "",
      "version": "v1",
      "kind": "Pod"
    },

mcastelino

Creating Clearlinux VM's using Libvirt

Download the KVM image and OVMF.fd from

https://download.clearlinux.org/image/

Create a directory to hold your images

Uncompress the clearlinux image and call it clear.img

mcastelino

1. mkdir somedir
2. cd somedir
3. create a file function.go with

/*
Copyright 2017 The Nuclio Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

mcastelino

https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf

• Legacy pin interrupts

— For devices that use legacy methods for interrupt routing (such as either through direct wiring to the I/OxAPIC input pins, or through INTx messages), the I/OxAPIC hardware generates the interrupt-request transaction. To identify the source of interrupt requests generated by I/OxAPICs, the interrupt-remapping hardware requires each I/OxAPIC in the platform (enumerated through the ACPI Multiple APIC Descriptor Tables (MADT)) to include a unique 16-bit source-id in its requests. BIOS reports the source-id for these I/OxAPICs via ACPI