usrbinkat/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Kubernetes Single Node Edge Firewall, Gateway, and Ubiquiti AP Controller

This is a rought draft document describing the build of VyOS on microk8s for use as a home WAN
firewall/router. I also use an Ubiquiti UAP Pro Access Point to broadcast my home wifi networks.
Here is a visual of the silent fanless intel i7 kubernetes node, access point, and power-over-ethernet adapter.

This is a slightly over estimated annual power cost estimate for powering the whole setup.

Load average:


Pod List:


Hardware

These are the hardware choices I made allowing for uses other than just dedicated firewall/router duties in my home.
While I opted for more expensive selections it is perfectly reasonable to choose other access points and cheaper qotom,
protectli, dell optiplex, or other hardware to serve the same purpose. The key feature you will need in your kubernetes
node is to ensure enough RAM and that the number network interfaces (nic) on the hardware you select or build is enough
to support your physical network layout. The cheapest option without sacrificing capability would likely be to select a
second hand dell optiplex or other similar discarded enterprise equipment. An i5 or better is recommended. For access
points, I have only been using Ubiquiti devices for years but have also heard good things about aruba devices for home use.
DISCLAIMER: I have no incentive financial or otherwise for sharing my hardware selection.

Ubiquiti UAP Pro Access Point
Qotom i7 

Added:

16GB RAM
4TB SSD


Operating System & Host Configuration:

For the OS, I installed Fedora Server 36 Server. I also enabled the "Headless Management" packages during install.
For Networking, I configured one network interface on a bridge named wan0 with no IP address. The rest of my devices
interfaces are attached to a br0 bridge for all internal networking.
Kubernetes + Kubevirt

This is a draft manual proceedure demonstrating the microk8s + kubevirt path I took.
Vyos on Kubevirt

Here are early testing artifacts for my deployment of VyOS on Kubevirt.
I have since improved my methods and will continue to enhance them to accomodate a few other improvements before publishing a formal writeup.
Considerations

I configured this microk8s node with the microk8s registry plugin and set it to pull the vyos container image from it's internal registry. This prevents
chicken and egg issues where the unit may not have internet access until after the vyos virtual machine boots, but internet access is provided through
vyos instance.
I am using ephemeral virtual machines running directly from the container disk without any persistent data. To update my vyos router configuration,
I update the user-data secret with my new configuration commands, then delete the vyos router. Within about 25-35 seconds this microk8s box spawns
a new vyos vm and it is configured and operational per my user-data commands without any further interaction required.
I have run github actions in a container on this microk8s box to automate this user-data secret update and vyos vm rotation. This has not been a very
aesthetic method however given the github actions runner looses connection for an unreasonable amount of time. The vm replacement network outage
could be averted by creating a pair of highly available VyOS vm's for redundancy and setting the pod rotation budget to 50%.
Alternatively local gitea and tekton have also proven effective in accomodating the short duration network outage during pod rotation.
Opportunities

This is a rudimentary edge firewall/gateway/wifi solution. I will be adding more services such as:

pull through image registry mirrors
squid3 ssl bump proxy for DPI & cache acceleration
Local OpenDNS server & DNS based Add Blocking service (similar to piHole)
Kong Ingress Controller
Wireguard VPN Service
Cert Manager
and more