Welcome to this week's Insiders Update! Insiders gain exclusive access to early previews, tutorials, updates, news, and events on my OSS work.
✅ Subscribe today and pay whatever you want 👉 through GitHub Sponsors
Insiders Update: 1st May 2020 - Netbooting the Raspberry Pi for Kubernetes and OpenFaaS enhancements
Pictured: netbooted Raspberry Pi cluster with Kubernetes (k3s)
It turns out that SD cards are not reliable storage for operating systems, especially not when they are running software like Kubernetes which constantly writes to the disk. This is one of the reasons that network filesystems can be useful. I spent my evenings this week hacking away with netbooting the Raspberry Pi 4 and now have a consistent way to run as many as my network can handle without SD cards.
There are two viable approaches I've had success with:
-
- Netbooting with TFTP, PXE using NFS for the root filesystem - simplest to set up, however cannot run Kubernetes. Adding a USB pen drive to each node fixes that, and if the pen-drive dies, it's not a big deal. You just insert a new one and format it. NFS provides file storage over the network which is relatively fast.
-
- Netbooting with TFTP, PXE and iSCSI - this mechanism runs everything over the network, and so no local storage is required at all. It's also the most complicated and slower since it involves emulating block storage over the network. In most cases, you may not have a NAS with iSCSI enabled, so have to install a software package on a Linux server.
Here is approach 1 in action, so far compared to netbooting the original RPi3 this has been faultless and worked on every boot. The speed is also very good with the storage backed by my Intel NUC
So should you burn your SD cards? No, SD cards are super fast and convenient. I even have a cluster with 28 of them in it. Keep an eye out on my Twitter feed for more on this topic.
As a subscriber to The Treasure Trove, you can download my netbooting workshop - note that you must not reproduce this or distribute it on the Internet, and it's not licensed for commercial use. Download from Google Drive here: My netbooting workshop
In the meantime, you may enjoy these two recent tutorials:
- Five years of Raspberry Pi Clusters
- Walk-through — install Kubernetes to your Raspberry Pi in 15 minutes
While we're here, you can get a public IP for your Raspberry Pi cluster with inlets PRO, why not get a free 14-trial and try this tutorial out over the weekend? Expose Your IngressController and get TLS from LetsEncrypt
The queue-worker now has two big changes:
-
Multiple parallel queues - have a regular queue and a slower queue for tasks like video processing. This was released last week. Read the docs: Async: multiple-queues
-
Matthias Hanel from the NATS team sent a PR to the queue-worker to enable much more efficient concurrency. This means that you can now set a
max_inflightoption for the queue worker via Arkade and helm, so that a single queue worker can process many openfaas function invocations at once. Matthias also optimized a few flags with best practices with NATS. Thank you!
Try it out with KinD, k3s or minikube - install fresh or upgrade:
arkade install openfaas --max-inflight 10
# Then invoke a function via its asynchronous endpoint many times over
faas-cli store deploy figlet
curl -d Test http://127.0.0.1:8080/async-function/figletUp to 10 requests will be processed in parallel, you can still get HA for the queue-worker by setting more replicas, to get 20 parallel executions of functions.
arkade install openfaas --max-inflight 10 \
--queue-workers 2In the presentation with the Weaveworks User Group I covered the use-cases for Serverless and how that differs from SaaS-based functions from cloud vendors. I then compared and contrasted using Git for operations with a push vs. pull mechanism.
Serverless 2.0 landscape updated for 2020
The Weaveworks team have a strict definition that GitOps only means systems that pull changes from a git repository via polling, however many companies including Google Cloud and GitLab have expanded the definition to mean a system where git is the source of truth. Wherever you land with GitOps, I hope you enjoy the presentation and my demos with Flux and OpenFaaS Cloud.
- Windows users must download the latest OpenFaaS CLI which fixes recent regression on constructing paths.
- A vulnerability in faasd (OpenFaaS with containerd) was reported and fixed, this affected the authenticated API so the risk is low. You should however download the latest faasd version at your earliest convenience.
Thank you to Utsav for designing this new Twitter banner! Check it out and follow @inletsdev
-
New blog post: This blog runs on a Raspberry Pi! by inlets contributor Utsav Anand
-
I started building out a new helm chart to help you use inlets PRO as a VPN, to connect site-to-site or service-to-service. The use-case would be to bridge a private network to a public network without exposing your service. Imagine a private database on-premises that you need to consume in the cloud, say on an EKS cluster. If that's interesting to you, let me know and take a look at the suggested Helm chart values.
-
May weekend offer:
The price for an inlets PRO license has been dropped. Until the end of the weekend, you'll also get an inlets t-shirt or mug for free when you purchase the 6-mo license.
If you're considering buying for personal use or for your homelab, but have questions or can't afford the cost, feel free to email me alex@openfaas.com to see what can be done to help you out.
Thanks for being an Insider, if you enjoyed this update please share my Tweet for reach.
Have a great weekend and stay safe.
Want to join the discussion about netbooting Raspberry Pi? Comment on the thread
If you need help, want to share what you're building, or want to hang out, feel free to join the OpenFaaS Slack