Skip to content

Instantly share code, notes, and snippets.

@fardjad
Last active June 3, 2024 13:48
Show Gist options
  • Save fardjad/6c95cda623d061bb830538c6c631d2e6 to your computer and use it in GitHub Desktop.
Save fardjad/6c95cda623d061bb830538c6c631d2e6 to your computer and use it in GitHub Desktop.
[Minikube on WSL2 with Podman] Tips for running a local development Kubernetes cluster on WSL2 with Podman #wsl2 #minikube #podman #kubernetes #docker

minikube on WSL2 with Podman

Tips for running a single-node Kubernetes cluster on WSL2 with Podman

Instructions

  1. Install Podman

  2. Edit /etc/containers/containers.conf:

    • Set cgroup_manager to cgroupfs
    • Set events_logger to file
  3. Install minikube

  4. Run minikube start --driver=podman

Local Registry

  1. Enable minikube local registry addon:

     minikube addons enable registry
    
  2. Install docker-ce-cli package for your distro

  3. Configure the environment to use minikube's Docker daemon (ie. eval $(minikube -p minikube docker-env))

  4. Push/pull images to/from localhost:5000/imagename

Note: Instead of pushing/pulling images into/from the registry, you can build and tag an image and set containers' imagePullPolicy to IfNotPresent or Never

Reaching minikube from Windows (Minikube <= v1.11.x)

  1. Make sure minikube is started (ie. run minikube start)

  2. Open an elevated PowerShell prompt and execute the following commands:

     # Run "wsl --list" and make sure the default WSL distro is set correctly
     
     $minikube_ip = "$(wsl minikube ip)"
     $wsl_ip = "$(wsl sh -c "ip addr show `$(ip route list default | grep -Po ' dev \K\w+') | grep -Po ' inet \K[\d.]+'")"
     route delete $minikube_ip 2>&1 | Out-Null
     route add $minikube_ip mask 255.255.255.255 $wsl_ip
    

Note: You can also route the whole podman network subnet instead of only the minikube ip

Note: Instructions above should be followed after every restart

Troubleshooting

Podman does not play nicely with iptables-nft

Some distros like Debian, use the nftables framework by default. So in case of getting error messages like:

Error adding network: failed to list chains: running [...] iptables [...] (nf_tables)

Try switching to the legacy version of iptables.

Podman might not be able to pull the kicbase image in 240 seconds

The timeout value for timedCreateHost is currently hardcoded in the Minikube source-code!

In case minikube terminates with an error message like:

StartHost failed, but will try again: creating host: create host timed out in 240.000000 seconds

Try pulling the image manually before starting minikube:

BASE_IMAGE="$(minikube start --help | sed -En "s:^.*base-image='([^@]+).*:\1:p")"
sudo podman pull $BASE_IMAGE
@dancohenenterprise
Copy link

Can you please expand on what all these settings do? Podman on Windows using WSL works great without minikube;
what does step # 2 do? [Edit /etc/containers/containers.conf]
why is docker-ce-cli needed if minikube supports podman? https://minikube.sigs.k8s.io/docs/drivers/podman/

@fardjad
Copy link
Author

fardjad commented Mar 16, 2023

Hi @dancohenenterprise

Thank you for your comment. This is an old gist and as you mentioned Docker works perfectly fine nowadays in WSL thanks to the built-in systemd support. IIRC the reason I used podman back then was to work around the issues I had with running the Docker Engine daemon.

@dancohenenterprise
Copy link

@fardjad interesting; I would like to update the gist; I would like to stay away from docker due to the enterprise licensing requirements. I'd like to use WSL, Podman for Windows (uses WSL machine) and Any good K8s for Desktop distribution (minikube, etc). If you have guidance, I would much appreciate it; I have tried rootless podman + minikube on Windows, but this does not seem to work; I will try rootful config next

@fardjad
Copy link
Author

fardjad commented Mar 16, 2023

I’ll be happy to update this gist if you find out how to make it work or please feel free to fork it and add your own instructions.

As far as I know, Docker engine (not Docker Desktop) is still free to use for enterprises and you can enable systemd (see the link in my previous comment) and simply install docker engine within your WSL distro.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment