This is a rought draft document describing the build of VyOS on microk8s for use as a home WAN firewall/router. I also use an Ubiquiti UAP Pro Access Point to broadcast my home wifi networks.
Here is a visual of the silent fanless intel i7 kubernetes node, access point, and power-over-ethernet adapter.
This is a slightly over estimated annual power cost estimate for powering the whole setup.
Load average:
Pod List:
These are the hardware choices I made allowing for uses other than just dedicated firewall/router duties in my home. While I opted for more expensive selections it is perfectly reasonable to choose other access points and cheaper qotom, protectli, dell optiplex, or other hardware to serve the same purpose. The key feature you will need in your kubernetes node is to ensure enough RAM and that the number network interfaces (nic) on the hardware you select or build is enough to support your physical network layout. The cheapest option without sacrificing capability would likely be to select a second hand dell optiplex or other similar discarded enterprise equipment. An i5 or better is recommended. For access points, I have only been using Ubiquiti devices for years but have also heard good things about aruba devices for home use.
DISCLAIMER: I have no incentive financial or otherwise for sharing my hardware selection.
- Ubiquiti UAP Pro Access Point
- Qotom i7
- Added:
- 16GB RAM
- 4TB SSD
- Added:
For the OS, I installed Fedora Server 36 Server. I also enabled the "Headless Management" packages during install.
For Networking, I configured one network interface on a bridge named wan0 with no IP address. The rest of my devices
interfaces are attached to a br0 bridge for all internal networking.
This is a draft manual proceedure demonstrating the microk8s + kubevirt path I took.
Here are early testing artifacts for my deployment of VyOS on Kubevirt. I have since improved my methods and will continue to enhance them to accomodate a few other improvements before publishing a formal writeup.
I configured this microk8s node with the microk8s registry plugin and set it to pull the vyos container image from it's internal registry. This prevents chicken and egg issues where the unit may not have internet access until after the vyos virtual machine boots, but internet access is provided through vyos instance.
I am using ephemeral virtual machines running directly from the container disk without any persistent data. To update my vyos router configuration, I update the user-data secret with my new configuration commands, then delete the vyos router. Within about 25-35 seconds this microk8s box spawns a new vyos vm and it is configured and operational per my user-data commands without any further interaction required.
I have run github actions in a container on this microk8s box to automate this user-data secret update and vyos vm rotation. This has not been a very aesthetic method however given the github actions runner looses connection for an unreasonable amount of time. The vm replacement network outage could be averted by creating a pair of highly available VyOS vm's for redundancy and setting the pod rotation budget to 50%.
Alternatively local gitea and tekton have also proven effective in accomodating the short duration network outage during pod rotation.
This is a rudimentary edge firewall/gateway/wifi solution. I will be adding more services such as:
- pull through image registry mirrors
- squid3 ssl bump proxy for DPI & cache acceleration
- Local OpenDNS server & DNS based Add Blocking service (similar to piHole)
- Kong Ingress Controller
- Wireguard VPN Service
- Cert Manager
- and more
Hi there, are there any networking issues/config when deploying Mini PC router nodes in K8s cluster, especially for multiple nic cards and wifi interfaces. My scenario is wifi mesh using these nodes.