Create a gist now

Instantly share code, notes, and snippets.

@alexellis /k8s-pi.md
Last active Oct 20, 2017

What would you like to do?
K8s on Raspbian

K8s on (vanilla) Raspbian Lite

Yes - you can create a Kubernetes cluster with Raspberry Pis with the default operating system Raspbian. Carry on using all the tools and packages you're used to with the officially-supported OS.

Pre-reqs:

  • You must use an RPi2 or 3 for Kubernetes

Master node setup

  • Flash Raspbian to a fresh SD card.

Use Raspbian Jessie

https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2017-07-05/

  • Install Docker

This installs 17.05 - the latest release for Jessie.

$ curl -sSL get.docker.com | sh
$ sudo usermod pi -aG docker
  • Edit cmdline.txt

Add this text, but don't create any new lines:

cgroup_enable=cpuset cgroup_enable=memory 
  • Add repo lists & install kubeadm
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - && \
  echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list && \
  sudo apt-get update -q && \
  sudo apt-get install -qy kubeadm

I realise this says 'xenial' in the apt listing, don't worry. It still works.

  • Check commands exist:
$ kubeadm
$ kubectl
  • Initialize your master node:
$ sudo kubeadm init

Optionally also pass --apiserver-advertise-address=192.168.0.27 with the IP of the Pi.

This will take some time.

Run the snippet given to you on the command-line:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

  • Check everything worked:
$ kubectl get pods --namespace=kube-system
NAME                            READY     STATUS    RESTARTS   AGE
etcd-k8s-1                      1/1       Running   0          4m
kube-apiserver-k8s-1            1/1       Running   0          4m
kube-controller-manager-k8s-1   1/1       Running   0          4m
kube-dns-2459497834-63fw6       0/3       Pending   0          3m
kube-proxy-jzt2r                1/1       Running   0          3m
kube-scheduler-k8s-1            1/1       Running   0          4m
  • Setup networking

Install Weave network driver

$ kubectl apply -f https://git.io/weave-kube-1.6

Join other nodes

On the other RPis, repeat everything apart from kubeadm init.

  • Join the cluster

Replace the token / IP for the output you got from the master node:

$ sudo kubeadm join --token 1fd0d8.67e7083ed7ec08f3 192.168.0.27:6443

You can now run this on the master:

$ kubectl get nodes
NAME      STATUS     AGE       VERSION
k8s-1     Ready      5m        v1.7.4
k8s-2     Ready      10m       v1.7.4

Deploy a container

function.yml

apiVersion: v1
kind: Service
metadata:
  name: markdownrender
  labels:
    app: markdownrender
spec:
  type: NodePort
  ports:
    - port: 8080
      protocol: TCP
      targetPort: 8080
      nodePort: 31118
  selector:
    app: markdownrender
---
apiVersion: apps/v1beta1 # for versions before 1.6.0 use extensions/v1beta1
kind: Deployment
metadata:
  name: markdownrender
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: markdownrender
    spec:
      containers:
      - name: markdownrender
        image: functions/markdownrender:latest-armhf
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP

Deploy and test:

$ kubectl create -f function.yml 
$ curl localhost:31118 -d "# test"

Start up the dashboard

$ curl -sSL https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml | \
  sed "s/amd64/arm/" | kubectl create -f -

You can then find the IP and port via kubectl get svc -n kube-system

Lewiscowles1986 commented Oct 12, 2017 edited

This is great, but it'd be very cool to have this operate unattended. (or as unattended as possible)

I started playing today and got a reasonable amount done using the stretch raspbian (I know the blog post said Jessie, I'm too fussy I suppose, but I wanted stretch)

Work to date https://gist.github.com/Lewiscowles1986/4f451bfaabb238dc6a3147194a759e81

Output

kubectl get pods --namespace=kube-system
NAME                                  READY     STATUS    RESTARTS   AGE
etcd-raspberrypi                      1/1       Running   0          19m
kube-apiserver-raspberrypi            1/1       Running   0          19m
kube-controller-manager-raspberrypi   1/1       Running   0          19m
kube-dns-66ffd5c588-59r2p             0/3       Pending   0          20m
kube-proxy-b9856                      1/1       Running   0          20m
kube-scheduler-raspberrypi            1/1       Running   0          19m

Essentially I setup an access point image connected to my main network, then I setup an image for nodes with the access point name and password (all done by adding /boot/ssh and a working /boot/wpa_supplicant.conf (one wpa_supplicant.conf for controller, one for nodes)

I'm assuming kube-dns-66ffd5c588-59r2p 0/3 Pending 0 20m reflects the 3 pi's connected but not yet setup as nodes.

What I need to be able to work out is how to save the node connection string to use it; then loop over the nodes one at a time to run the setup and init command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment