Skip to content

Instantly share code, notes, and snippets.

@mcastelino
Forked from egernst/notes.md
Created April 12, 2019 19:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mcastelino/f24baf26712283e95e96f76b30c2e954 to your computer and use it in GitHub Desktop.
Save mcastelino/f24baf26712283e95e96f76b30c2e954 to your computer and use it in GitHub Desktop.
containerd + kubernetes, and making clr-examples do what i want on Bionic

Quick guide for getting Kata+containerd (using v2 shim) up and running super quick on bionic

Installation of Kube stuff on bionic:

Use the following sh:

sudo bash -c "cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial-unstable main
EOF"
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo -E apt update
sudo -E apt install -y kubelet kubeadm kubectl

VERSION="1.2.5"
echo "Install Containerd ${VERSION}"
wget -q https://storage.googleapis.com/cri-containerd-release/cri-containerd-${VERSION}.linux-amd64.tar.gz
sudo tar -C / -xzf cri-containerd-${VERSION}.linux-amd64.tar.gz
sudo systemctl start containerd
sudo swapoff -a
sudo modprobe br_netfilter

Helper script for starting/resetting:

Grab cloud-native-setup:

git clone https://github.com/clearlinux/cloud-native-setup.git

Apply patch, which I still need to upstream:

Update to use containerd instead of CRIO, and make minimal more minimal

cd cloud-native-setup
wget  https://gist.githubusercontent.com/egernst/138def9599a42376f409fbce2e95de5d/raw/9e1b952b75f92fc125f65a068563f0286e159f60/z0001-Use-containerd-and-let-me-configure-Kata.patch
git am z0001-Use-containerd-and-let-me-configure-Kata.patch  

Launch kube:

pushd cloud-native-setup/cl-k8s-examples
./create-stack.sh minimal

Setup Kata latest inside the running cluster:

Use kata deploy, and register the runtimeclass, kata:

kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-deploy.yaml
kubectl apply -f https://raw.githubusercontent.com/clearlinux/cloud-native-setup/master/clr-k8s-examples/8-kata/kata-qemu-runtimeClass.yaml

Run Kata example:

Here's a simple example pod. Of not is the runtimeClass definition

kubectl apply -f https://gist.githubusercontent.com/egernst/9cbbb3d6ba566afca98be2a4e29328d6/raw/0f6e332ba1b7306ed895e5ff91b2939ca3f0bb42/two-pod.yaml
From aa5abd9ab621e5ca2dd7a1e0adfcd5d4469f0bd2 Mon Sep 17 00:00:00 2001
From: Eric Ernst <eric.ernst@intel.com>
Date: Fri, 12 Apr 2019 03:51:57 +0000
Subject: [PATCH 1/1] Use containerd, and let me configure Kata
Signed-off-by: Eric Ernst <eric.ernst@intel.com>
---
clr-k8s-examples/create_stack.sh | 2 +-
clr-k8s-examples/kubeadm.yaml | 2 +-
clr-k8s-examples/reset_stack.sh | 10 +++++-----
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/clr-k8s-examples/create_stack.sh b/clr-k8s-examples/create_stack.sh
index ffd811f..55d5677 100755
--- a/clr-k8s-examples/create_stack.sh
+++ b/clr-k8s-examples/create_stack.sh
@@ -98,7 +98,7 @@ function miscellaneous() {
function minimal() {
cluster_init
cni
- kata
+ #kata
metrics
}
diff --git a/clr-k8s-examples/kubeadm.yaml b/clr-k8s-examples/kubeadm.yaml
index de436b2..92defd7 100644
--- a/clr-k8s-examples/kubeadm.yaml
+++ b/clr-k8s-examples/kubeadm.yaml
@@ -1,7 +1,7 @@
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
nodeRegistration:
- criSocket: /var/run/crio/crio.sock
+ criSocket: /var/run/containerd/containerd.sock
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
diff --git a/clr-k8s-examples/reset_stack.sh b/clr-k8s-examples/reset_stack.sh
index 77a5118..08d6e32 100755
--- a/clr-k8s-examples/reset_stack.sh
+++ b/clr-k8s-examples/reset_stack.sh
@@ -3,7 +3,7 @@
set -o nounset
#Cleanup
-sudo -E kubeadm reset -f --cri-socket="/var/run/crio/crio.sock"
+sudo -E kubeadm reset -f --cri-socket="/var/run/containerd/containerd.sock"
for ctr in $(sudo crictl ps --quiet); do
sudo crictl stop "$ctr"
@@ -17,7 +17,7 @@ done
#Forcefull cleanup all artifacts
#This is needed is things really go wrong
sudo systemctl stop kubelet
-sudo systemctl stop crio
+sudo systemctl stop containerd
sudo pkill -9 qemu
sudo pkill -9 kata
sudo pkill -9 kube
@@ -39,8 +39,8 @@ sudo -E bash -c "rm -r /var/run/kata-containers/*"
sudo rm -rf /var/lib/rook
sudo systemctl daemon-reload
-sudo systemctl enable kubelet crio
-sudo systemctl restart crio
+sudo systemctl enable kubelet containerd
+sudo systemctl restart containerd
sudo systemctl restart kubelet
-sudo -E kubeadm reset -f --cri-socket="/var/run/crio/crio.sock"
+sudo -E kubeadm reset -f --cri-socket="/var/run/containerd/containerd.sock"
--
2.17.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment