[Minikube on WSL2 with Podman] Tips for running a local development Kubernetes cluster on WSL2 with Podman #wsl2 #minikube #podman #kubernetes #docker

minikube-on-wsl2-with-podman.md

Work in progress, updating a guide on:

minikube on WSL2 with Podman

Tips for running a single-node Kubernetes cluster on WSL2 with Podman

Instructions

1. Install Podman

2. Edit /etc/containers/containers.conf:

   • Set cgroup_manager to cgroupfs
   • Set events_logger to file

3. Install minikube

   • Use Linux install.

4. Configure your login with /etc/sudoers.

   • Instructions for what to configure will be given by minikube in next step.

5. Run minikube start --driver=podman NOTE: start with crio runtime instead of docker?

Local Container Registry

1. Enable minikube local container registry addon:

    minikube addons enable registry
   

Using Podman

2. Run: minikube docker-env

To check:

            minikube podman-env

That will provide you with something similar to: export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://127.0.0.1:44577" export DOCKER_CERT_PATH="/home/username/.minikube/certs" export MINIKUBE_ACTIVE_DOCKERD="minikube"

Copy these rows

3. With git-bash, paste the 4 export commands but change the DOCKER_CERT_PATH to have .minikube directory in you home directory.

4. Using Explorer, open \wsl$, copy the directory .minikube from the location, into a directory in Windows 10 that you have access to, same as in previous step.

5. Executing docker images should now list the images that are present in your minikube container registry.

Using Docker

Note the response that: "Registry addon with podman driver uses port 39919 please use that instead of default port 5000".

2. Install docker-ce-cli package for your distro
3. Configure the environment to use minikube's Docker daemon (ie. eval $(minikube -p minikube docker-env))
4. Push/pull images to/from localhost:5000/imagename

Note: Instead of pushing/pulling images into/from the registry, you can build and tag an image and set containers' imagePullPolicy to IfNotPresent or Never

Reaching minikube from Windows (Minikube <= v1.11.x)

1. Make sure minikube is started (ie. run minikube start)

2. Open an elevated PowerShell prompt and execute the following commands:

   Run wsl --list and make sure the default WSL distro is set correctly

    $minikube_ip = "$(wsl minikube ip)"
    $wsl_ip = "$(wsl sh -c "ip addr show `$(ip route list default | grep -Po ' dev \K\w+') | grep -Po ' inet \K[\d.]+'")"
    route delete $minikube_ip 2>&1 | Out-Null
    route add $minikube_ip mask 255.255.255.255 $wsl_ip
   

Note: You can also route the whole podman network subnet instead of only the minikube ip

Note: Instructions above should be followed after every restart

Troubleshooting

Podman does not play nicely with iptables-nft

Some distros like Debian, use the nftables framework by default. So in case of getting error messages like:

Error adding network: failed to list chains: running [...] iptables [...] (nf_tables)

Try switching to the legacy version of iptables.

Podman might not be able to pull the kicbase image in 240 seconds

The timeout value for timedCreateHost is currently hardcoded in the Minikube source-code!

In case minikube terminates with an error message like:

StartHost failed, but will try again: creating host: create host timed out in 240.000000 seconds

Try pulling the image manually before starting minikube:

BASE_IMAGE="$(minikube start --help | sed -En "s:^.*base-image='([^@]+).*:\1:p")"
sudo podman pull $BASE_IMAGE