surajssd

The cluster was in HEALTH_WARN state with backfill errors. So I followed the advise from https://centosquestions.com/how-to-resolve-ceph-pool-getting-activeremappedbackfill_toofull/.

See the health:

# ceph health detail
HEALTH_WARN 1 backfillfull osd(s); 1 pool(s) backfillfull
OSD_BACKFILLFULL 1 backfillfull osd(s)
    osd.8 is backfill full
POOL_BACKFILLFULL 1 pool(s) backfillfull

surajssd

$ ./hyperkube kubelet --node-ip=10.88.81.5  --anonymous-auth=false --authentication-token-webhook --authorization-mode=Webhook --client-ca-file=/etc/kubernetes/ca.crt --cluster_dns=10.3.0.10 --cluster_domain=cluster.local --cni-conf-dir=/etc/kubernetes/cni/net.d --config=/etc/kubernetes/kubelet.config --kubeconfig=/etc/kubernetes/kubeconfig --lock-file=/var/run/lock/kubelet.lock --network-plugin=cni --pod-manifest-path=/etc/kubernetes/manifests --read-only-port=0 --volume-plugin-dir=/var/lib/kubelet/volumeplugins --node-labels=node.kubernetes.io/node,
metallb.universe.tf/my-asn=65000,metallb.universe.tf/peer-asn=65530 --register-with-taints= --address=10.88.81.5

Flag --anonymous-auth has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --authentication-token-webhook has been deprecated, This parameter should be set via the config file specified by 

surajssd

apt-get update && \
apt-get -y upgrade && \
apt-get install -y make git byobu linux-generic 

systemctl reboot

apt-get install -y linux-headers-`uname -r` && \
apt install -y virtualbox && \
apt-get install -y vagrant && \
dpkg-reconfigure virtualbox-dkms

surajssd

echo '
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: bash
  name: bash
spec:
  replicas: 1
  selector:

surajssd

Deploy Local storage provisioner

To test the issue with /mnt in kubelet in Lokomotive kinvolk-archives/lokomotive-kubernetes#160.

Add following flag to apiserver, kube-controller-manager, kube-scheduler

- --feature-gates=BlockVolume=true

surajssd

#!/bin/bash

set -e

echo 'In `cluster.tf` file:'
echo
echo '
```tf

```

surajssd

apt-get update -y
apt-get install -y virt-manager libvirt-clients libvirt-daemon-system qemu-kvm byobu git

surajssd

Using Kind

Multinode PSP enabled

kind create cluster --name multi-node-psp --config psp-multi-cluster.yaml
export KUBECONFIG="$(kind get kubeconfig-path --name="multi-node-psp")"
kubectl apply -f https://raw.githubusercontent.com/kinvolk/terraform-render-bootkube/kinvolk-master/resources/manifests/psp-privileged.yaml
kubectl apply -f https://raw.githubusercontent.com/kinvolk/terraform-render-bootkube/kinvolk-master/resources/manifests/psp-restricted.yaml

surajssd

Configs

• These configs are for the blog https://suraj.io/post/docker-seccomp-manual/
• The default seccomp profile is taken at https://github.com/moby/moby/blob/238f8eaa31aa74be843c81703fabf774863ec30c/profiles/seccomp/default.json

surajssd

apiVersion: flagger.app/v1alpha3
kind: Canary
metadata:
  name: podinfo
  namespace: test
spec:
  # deployment reference
  targetRef:
    apiVersion: apps/v1
    kind: Deployment