Skip to content

Instantly share code, notes, and snippets.

@mcastelino

mcastelino/footloose_kata_kubernetes_k8s.md

Last active August 17, 2020 17:07
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mcastelino/9b9634226cb732474a4535391eb32cfe to your computer and use it in GitHub Desktop.
Save mcastelino/9b9634226cb732474a4535391eb32cfe to your computer and use it in GitHub Desktop.
Download ZIP
Footloose as a Kubernetes POD. So a VM in Kubernetes launched and managed as a container using the Kata runtimeClass.
Raw
footloose_kata_kubernetes_k8s.md

Footloose as a Kubernetes POD. So a VM in Kubernetes launched and managed as a container using the Kata runtimeClass.

apiVersion: v1
data:
    authorized_keys: |
      ssh-rsa PUT_YOUR_KEY_HERE
kind: ConfigMap
metadata:
  name: ssh-pub-key
---
apiVersion: v1
kind: Pod
metadata:
  name: footfed
spec:
  runtimeClassName: kata
  volumes:
  - name: runv
    emptyDir:
      medium: "Memory"
  - name: runlockv
    emptyDir:
      medium: "Memory"
  - name: tmpv
    emptyDir:
      medium: "Memory"
  - name: fakecgroup
    hostPath:
      path: /sys/fs/cgroup
  - name: ssh-dir
    emptyDir:
      medium: "Memory"
  - name: ssh-pub-key
    configMap:
      name: ssh-pub-key
      defaultMode: 384
  containers:
  - name: footfed
    image: quay.io/footloose/fedora29:latest
    command: ["/sbin/init"]
    volumeMounts:
    - name: runv
      mountPath: /run
    - name: runlockv
      mountPath: /run/lock
    - name: tmpv
      mountPath: /tmp
    - name: fakecgroup
      readOnly: true
      mountPath: /sys/fs/cgroup
    - name: ssh-dir
      mountPath: /root/.ssh
    - name: ssh-pub-key
      mountPath: /root/.ssh/authorized_keys
      subPath: authorized_keys
  # These containers are run during pod initialization
  initContainers:
  - name: install
    image: busybox
    command: ["sh", "-c", "chmod 700 /root/.ssh"]
    volumeMounts:
    - name: ssh-dir
      mountPath: /root/.ssh

It talks like a duck, and walks like a duck. So it is a VM.

mrcastel@bored-pelinor:~$ kubectl get po -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP            NODE            NOMINATED NODE   READINESS GATES
footfed   1/1     Running   0          10m   10.244.0.78   bored-pelinor   <none>           <none>
mrcastel@bored-pelinor:~$ ssh root@10.244.0.78
[root@footfed ~]# uname -r
4.19.24
mrcastel@bored-pelinor:~$ kubectl exec footfed -- uname -r
4.19.24
mrcastel@bored-pelinor:~$ uname -r
4.15.0-20-generic

mrcastel@bored-pelinor:~$ kubectl exec footfed -- systemctl status
● footfed
    State: running
     Jobs: 0 queued
   Failed: 0 units
    Since: Mon 2019-04-08 21:48:18 UTC; 1min 39s ago
   CGroup: /kubepods/besteffort/pod02d6637e-5a48-11e9-88f0-525400123456/db4e409762c1efd4adda6bb5b959c09eea3ca694302a465934466a65acb7517a
           ├─60 systemctl status
           ├─init.scope
           │ └─1 /sbin/init
           └─system.slice
             ├─systemd-journald.service
             │ └─17 /usr/lib/systemd/systemd-journald
             ├─sshd.service
             │ └─43 /usr/sbin/sshd -D -oCiphers=aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc -oMACs=hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512 -oGSSAPIKexAlgorithms=gss-gex-sha1-,gss-group14-sha1- -oKexAlgorithms=curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1 -oHostKeyAlgorithms=rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01@openssh.com -oPubkeyAcceptedKeyTypes=rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01@openssh.com
             └─dbus-daemon.service
               └─26 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only


mrcastel@bored-pelinor:~$ kubectl exec footfed -- systemctl status sshd
● sshd.service - OpenSSH server daemon
   Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2019-04-08 21:03:44 UTC; 6min ago
     Docs: man:sshd(8)
           man:sshd_config(5)
 Main PID: 45 (sshd)
    Tasks: 1 (limit: 2386)
   Memory: 3.3M
   CGroup: /kubepods/besteffort/podc6e7cbd8-5a41-11e9-88f0-525400123456/9aeb989cdda6009926dbe365fbec16468b9097cd2048059cdff3748fd7e15480/system.slice/sshd.service
           └─45 /usr/sbin/sshd -D -oCiphers=aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc -oMACs=hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512 -oGSSAPIKexAlgorithms=gss-gex-sha1-,gss-group14-sha1- -oKexAlgorithms=curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1 -oHostKeyAlgorithms=rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01@openssh.com -oPubkeyAcceptedKeyTypes=rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01@openssh.com

Apr 08 21:03:44 footfed systemd[1]: Starting OpenSSH server daemon...
Apr 08 21:03:44 footfed sshd[45]: Server listening on 0.0.0.0 port 22.
Apr 08 21:03:44 footfed systemd[1]: Started OpenSSH server daemon.
Apr 08 21:03:44 footfed sshd[45]: Server listening on :: port 22.
mrcastel@bored-pelinor:~$ systemctl status sshd
● ssh.service - OpenBSD Secure Shell server
   Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-04-03 06:29:49 UTC; 5 days ago
 Main PID: 3913 (sshd)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/ssh.service
           └─3913 /usr/sbin/sshd -D
mrcastel@bored-pelinor:~$ ps auxw | grep qemu
mrcastel 22218  0.0  0.0  14856  1100 pts/0    S+   21:18   0:00 grep --color=auto qemu
root     28284  2.3  3.0 4431216 248008 ?      Sl   21:03   0:21 /opt/kata/bin/qemu-system-x86_64 -name sandbox-c5c86bed932fb34033c430831f478116c49a3cc8169723c15f90024d2325f6aa -uuid 74c5fe31-bcaf-4498-a5bf-fc8aae67b5b3 -machine pc,accel=kvm,kernel_irqchip,nvdimm -cpu host,pmu=off -qmp unix:/run/vc/vm/c5c86bed932fb34033c430831f478116c49a3cc8169723c15f90024d2325f6aa/qmp.sock,server,nowait -m 2048M,slots=10,maxmem=8999M -device pci-bridge,bus=pci.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile= -device virtio-serial-pci,disable-modern=true,id=serial0,romfile= -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/c5c86bed932fb34033c430831f478116c49a3cc8169723c15f90024d2325f6aa/console.sock,server,nowait -device nvdimm,id=nv0,memdev=mem0 -object memory-backend-file,id=mem0,mem-path=/opt/kata/share/kata-containers/kata-containers-image_clearlinux_1.6.0_agent_9fdc29c7762.img,size=134217728 -device virtio-scsi-pci,id=scsi0,disable-modern=true,romfile= -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng,rng=rng0,romfile= -device virtserialport,chardev=charch0,id=channel0,name=agent.channel.0 -chardev socket,id=charch0,path=/run/vc/vm/c5c86bed932fb34033c430831f478116c49a3cc8169723c15f90024d2325f6aa/kata.sock,server,nowait -device virtio-9p-pci,disable-modern=true,fsdev=extra-9p-kataShared,mount_tag=kataShared,romfile= -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/c5c86bed932fb34033c430831f478116c49a3cc8169723c15f90024d2325f6aa,security_model=none -netdev tap,id=network-0,vhost=on,vhostfds=3,fds=4 -device driver=virtio-net-pci,netdev=network-0,mac=82:77:ce:4a:e3:41,disable-modern=true,mq=on,vectors=4,romfile= -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic -daemonize -kernel /opt/kata/share/kata-containers/vmlinuz-4.19.24-31 -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 quiet systemd.show_status=false panic=1 nr_cpus=8 init=/usr/lib/systemd/systemd systemd.unit=kata-containers.target systemd.mask=systemd-networkd.service systemd.mask=systemd-networkd.socket systemd.mask=systemd-journald.service systemd.mask=systemd-journald.socket systemd.mask=systemd-journal-flush.service systemd.mask=systemd-udevd.service systemd.mask=systemd-udevd.socket systemd.mask=systemd-udev-trigger.service systemd.mask=systemd-timesyncd.service systemd.mask=systemd-update-utmp.service systemd.mask=systemd-tmpfiles-setup.service systemd.mask=systemd-tmpfiles-cleanup.service systemd.mask=systemd-tmpfiles-cleanup.timer systemd.mask=tmp.mount -pidfile /run/vc/vm/c5c86bed932fb34033c430831f478116c49a3cc8169723c15f90024d2325f6aa/pid -smp 1,cores=1,threads=1,sockets=1,maxcpus=8
@mcastelino
Copy link
Author

Using a sidecar to push the keys into the VM

apiVersion: v1
data:
    authorized_keys: |
      ssh-rsa USE_SIDECAR_TO_ADD_KEYS_kubectl_cm_edit_ssh-pub-key
kind: ConfigMap
metadata:
  name: ssh-pub-key
---
apiVersion: v1
kind: Pod
metadata:
  name: footfed
spec:
        #runtimeClassName: kata
  volumes:
  - name: runv
    emptyDir:
      medium: "Memory"
  - name: runlockv
    emptyDir:
      medium: "Memory"
  - name: tmpv
    emptyDir:
      medium: "Memory"
  - name: fakecgroup
    hostPath:
      path: /sys/fs/cgroup
  - name: ssh-dir
    emptyDir:
      medium: "Memory"
  - name: ssh-pub-key
    configMap:
      name: ssh-pub-key
      defaultMode: 384
  containers:
  # This sidecar is used to constantly update the keys allowing the VM
  # keys to be revoked or updated
  - name: key-sidecar
    image: pstauffer/inotify
    command: ["sh", "-xc", "chmod 700 /root/.ssh; cp /tmp/keys/authorized_keys /root/.ssh/authorized_keys; inotifywait -m -r -e create /tmp/ | while read IGNOREME; do echo /tmp/keys/authorized_keys; cp /tmp/keys/authorized_keys /root/.ssh/authorized_keys; chmod 600 /root/.ssh/authorized_keys; done"]
    volumeMounts:
    - name: ssh-dir
      mountPath: /root/.ssh
    - name: ssh-pub-key
      mountPath: /tmp/keys
  - name: footfed
    image: quay.io/footloose/fedora29:latest
    command: ["/sbin/init"]
    volumeMounts:
    - name: runv
      mountPath: /run
    - name: runlockv
      mountPath: /run/lock
    - name: tmpv
      mountPath: /tmp
    - name: fakecgroup
      readOnly: true
      mountPath: /sys/fs/cgroup
    - name: ssh-dir
      mountPath: /root/.ssh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment