Skip to content

Instantly share code, notes, and snippets.

@syncom
Last active June 1, 2021 04:25
Show Gist options
  • Save syncom/7c6e90708bc28cc9ede2c3245c203e32 to your computer and use it in GitHub Desktop.
Save syncom/7c6e90708bc28cc9ede2c3245c203e32 to your computer and use it in GitHub Desktop.
Set up k3s for kubernetes on Ubuntu on Raspberry Pi 4

Steps for setting up k3s on Ubuntu 20.04.2 on Raspberry Pi 4 Cluster

We use three Raspberry Pi 4's (RPI4's) for the setup. One of them is used as the control plane, and the other two are used as workers.

  1. On all three RPI4's, snstall Ubuntu server 20.04.

    • Follow these instructions
    • Assign static IPs to these RPI4s (via router's DHCP settings), call them rpi4-k8s-ctrl, rpi4-k8s-wkr00, and rpi4-k8s-wkr01, respectively, and add them to /etc/hosts for all three RPI4s as well as the PC used for the setup
  2. On all three RPI4's, enable cgroup (reference)

    • Add cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory to the end of /boot/firmware/cmdline.txt, and reboot
  3. On all three RPI4's, install docker (reference)

    • Install docker
      $ curl -fsSL https://get.docker.com -o get-docker.sh
      $ sudo sh get-docker.sh   
      
    • Rootless setup
      $ sudo sh -eux <<EOF
      # Install newuidmap & newgidmap binaries
      apt-get install -y uidmap
      EOF
      $ dockerd-rootless-setuptool.sh install
      
    • Add the following lines to the end of ~/.bashrc
      # docker
      export PATH=/usr/bin:$PATH
      export DOCKER_HOST=unix:///run/user/1000/docker.sock
      
  4. Install k3s on control plane node rpi4-k8s-ctrl (reference)

    • One-click installation
      $ curl -sfL https://get.k3s.io | sh -s - --docker
      
    • Check installation
      $ sudo systemctl status k3s
      $ sudo kubectl get nodes -o wide
      
    • Configure firewall (reference)
      $ sudo ufw allow 6443/tcp
      $ sudo ufw allow 443/tcp
      
    • Extract node token from the control plane node for worker setup later
      $ sudo cat sudo cat /var/lib/rancher/k3s/server/node-token
      
  5. Install k3s on both worker nodes, rpi4-k8s-wkr00 and rpi4-k8s-wkr01(reference)

    • One-click installation, with information for control plane node

       # Adapt K3S_URL and K3S_TOKEN to your control plane node's hostname/IP and node token, respectively
       $ curl -sfL https://get.k3s.io | K3S_URL=https://rpi4-k8s-ctrl:6443 K3S_TOKEN=<NODE-TOKEN> sh -s - --docker
      
    • Check installation

      On worker node

      $ sudo systemctl status k3s-agent
      

      On control plane node

      $ sudo kubectl get nodes
      # Should see something like below
      NAME                 STATUS   ROLES                  AGE   VERSION
      rpi4-k8s-wkr00   Ready    <none>                 14m   v1.20.7+k3s1
      rpi4-k8s-ctrl    Ready    control-plane,master   8d    v1.20.7+k3s1
      rpi4-k8s-wkr01   Ready    <none>                 17s   v1.20.7+k3s1
      
  6. On PC (localhost), install kubectl, by following the official documentation. Copy content of rpi4-k8s-ctrl:/etc/rancher/k3s/k3s.yaml to localhost:~/.kube/config, and change server: https://127.0.0.1:6443 to server: https://rpi4-k8s-ctrl:6443. Check installation with kubectl get nodes on local PC.

  7. On local PC, deploy a microservice, openfaas-figlet (reference).

    • Create a service
      cat <<EOF > openfaas-figlet-svc.yaml
      apiVersion: v1
      kind: Service
      metadata:
        name: openfaas-figlet
        labels:
          app: openfaas-figlet
      spec:
        type: NodePort
        ports:
          - port: 8080
            protocol: TCP
            targetPort: 8080
            nodePort: 31111
        selector:
          app: openfaas-figlet
      EOF
      
    • Create a deployment
      cat <<EOF > openfaas-figlet-dep.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: openfaas-figlet
        labels:
         app: openfaas-figlet
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: openfaas-figlet
        template:
          metadata:
            labels:
              app: openfaas-figlet
          spec:
            containers:
            - name: openfaas-figlet
              image: functions/figlet:latest-armhf
              imagePullPolicy: Always
              ports:
              - containerPort: 8080
                protocol: TCP
      EOF
      
    • Apply the configuration
      $ kubectl apply -f openfaas-figlet-dep.yaml,openfaas-figlet-svc.yaml
      deployment.apps/openfaas-figlet created
      service/openfaas-figlet created
      # Check status
      $ kubectl rollout status deploy/openfaas-figlet
      kubectl rollout status deploy/openfaas-figlet
      
    • Invoke the function
      $ echo -n "Hello SN" | curl --data-binary @- http://rpi4-k8s-ctrl:31111
       _   _      _ _         ____  _   _ 
      | | | | ___| | | ___   / ___|| \ | |
      | |_| |/ _ \ | |/ _ \  \___ \|  \| |
      |  _  |  __/ | | (_) |  ___) | |\  |
      |_| |_|\___|_|_|\___/  |____/|_| \_|
      
      
    • Scale up (or down)
      $ kubectl scale deploy/openfaas-figlet --replicas=3
      
    • Check pods info
      $ kubectl get pods -o wide
      NAME                               READY   STATUS    RESTARTS   AGE   IP           NODE                  NOMINATED NODE   READINESS GATES
      openfaas-figlet-77cb68d644-kchcr   1/1     Running   0          60m   10.42.1.3    rpi4-k8s-wkr00   <none>           <none>
      openfaas-figlet-77cb68d644-kbssv   1/1     Running   0          60m   10.42.0.20   rpi4-k8s-ctrl    <none>           <none>
      openfaas-figlet-77cb68d644-2c55b   1/1     Running   0          60m   10.42.1.4    rpi4-k8s-wkr00   <none>           <none>
      
    • Get logs for all pods
      $ kubectl logs -f -l app=openfaas-figlet --all-containers
      
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment