K3OS on Raspberry Pi

install.md

Imaging

Assuming macOS and an SD card presented as /dev/rdisk3:

For ARMv7 (Pi 2):

diskutil unmountDisk disk3
xzcat ubuntu-18.04.2-preinstalled-server-armhf+raspi2.img.xz | sudo dd of=/dev/rdisk3 bs=32m

For ARM64 (Pi >= 3+):

diskutil unmountDisk disk3
xzcat ubuntu-18.04.2-preinstalled-server-arm64+raspi3.img.xz | sudo dd of=/dev/rdisk3 bs=32m

Boot the Pis and SSH into each one, setting the root password when prompted.

K3OS setup

I've baked this into Ansible but it's easy enough to perform the steps independently.

Update cmdline.txt. You could also do this step before ejecting the SD card after imaging. Otherwise, the booted Ubuntu system will have the file located at /boot/firmware/cmdline.txt.

net.ifnames=0 dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait cgroup_memory=1 cgroup_enable=memory rw

The changes we've made have been to add cgroup_memory=1 cgroup_enable=memory rw

Now, download the K3OS rootfs and expand it. Here we use v0.2.1. For ARMv7:

curl -sL https://github.com/rancher/k3os/releases/download/v0.2.1/k3os-rootfs-arm.tar.gz | tar --strip-components=1 -zxvf - -C /

For ARM64:

curl -sL https://github.com/rancher/k3os/releases/download/v0.2.1/k3os-rootfs-arm64.tar.gz | tar --strip-components=1 -zxvf - -C /

Now, add a config.yaml file to /k3os/system/config.yaml. The configuration will change based on whether you're provisioning a "server" node or an "agent".

Server configs do not specify a server_url:

ssh_authorized_keys:
- YOUR_PUBLIC_KEY_HERE
hostname: YOUR_SERVER_HOSTNAME

k3os:
  data_sources:
  - cdrom
  dns_nameservers:
  - 1.1.1.1
  ntp_servers:
  - 0.us.pool.ntp.org
  - 1.us.pool.ntp.org

  token: A_RANDOM_TOKEN_YOU_LIKE

Agent configs must specify k3os.server_url, as well as the same token used by the server:

ssh_authorized_keys:
- YOUR_PUBLIC_KEY_HERE
hostname: YOUR_AGENT_HOSTNAME

k3os:
  data_sources:
  - cdrom
  dns_nameservers:
  - 1.1.1.1
  ntp_servers:
  - 0.us.pool.ntp.org
  - 1.us.pool.ntp.org

  server_url: https://YOUR_SERVER_IP:6443
  token: A_RANDOM_TOKEN_YOU_LIKE

Reboot the systems and you should have a functioning K3S cluster.

As mentioned in the docs, you are going to want to pull /etc/rancher/k3s/k3s.yaml from the master and modify the hostname, if you want to run kubectl from a standalone machine outside the cluster.