Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
k3s on WSL2

Instructions to hack up WSL2 on Windows 10 Build 18917 to run k3s (Kubernetes) and rio

Install WSL2

https://docs.microsoft.com/en-us/windows/wsl/wsl2-install

I already had Ubuntu-18.04 installed in wsl 1. So I just did wsl --set-version Ubuntu-18.04 2

Compile Kernel

Using Ubuntu 18.04 (I'm sure any distro will work), inside WSL2 download https://thirdpartysource.microsoft.com/download/Windows%20Subsystem%20for%20Linux%20v2/May%202019/WSLv2-Linux-Kernel-master.zip and extract to a folder. The latest version of the kernel source is available at (https://github.com/microsoft/WSL2-Linux-Kernel)

In the extracted kernel source run

# Install basically "compile kernel" crap
sudo apt install build-essential flex bison libssl-dev libelf-dev

# Setup kernel config
cp Microsoft/config-wsl .config
cat >> .config << EOF
CONFIG_BRIDGE_NETFILTER=y
CONFIG_NETFILTER_XT_MATCH_COMMENT=y
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=y
CONFIG_NETFILTER_XT_MATCH_OWNER=y
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=y
CONFIG_VXLAN=y
CONFIG_GENEVE=y
EOF
make oldconfig

# Compile and drink beverage of choice
make -j $(nproc)

Update WSL kernel

  1. Copy arch/x86/boot/bzImage to somewhere outside WSL2 (/mnt/c/Users/${USER}/Desktop)
  2. Shutdown WSL2 wsl --shutdown
  3. Copy bzImage to c:\Windows\System32\lxss\tools\kernel. Make sure you backup the existing kernel file (although the original is still in another location). I had to copy through explorer.exe because of permissions issues.

Install K3s

You can't install k3s using the curl script because there is no supervisor (systemd or openrc) in WSL2.

  1. Download k3s binary from https://github.com/rancher/k3s/releases/latest
  2. chmod +x k3s
  3. Run k3s sudo ./k3s server

Setup access

  1. Copy /etc/rancher/k3s/k3s.yaml from WSL to your home in Windows to %HOME%.kube\config. Edit the copied file and change the server URL from https://localhost:6443 to the IP of the your WSL2 instance (ip addr show dev eth0). So something like https://192.168.170.170:6443.
  2. Run kubectl from windows (got to download kubectl.exe from somewhere)

Bonus Install Rio

  1. Download rio-windows-amd64 from https://github.com/rancher/rio/releases/tag/v0.2.0-rc1 to rio.exe
  2. Run ./rio install
  3. Start doing stuff with rio :)
@justicel

This comment has been minimized.

Copy link

@justicel justicel commented Jul 12, 2019

As of the latest release of the kernel code the .config modifications are no longer necessary! microsoft/WSL2-Linux-Kernel@a9f6b6e

@JuryA

This comment has been minimized.

Copy link

@JuryA JuryA commented Sep 21, 2019

Awesome! ☸️😲👍

@mkostersitz

This comment has been minimized.

Copy link

@mkostersitz mkostersitz commented Oct 17, 2019

Awesome. Just got it to work :)

@juliostanley

This comment has been minimized.

Copy link

@juliostanley juliostanley commented Dec 8, 2019

Made a little script to make the install somewhat easier. It creates a brand new wsl2 named liifi-k3s to avoid collisions. It uses the official docker image to create the WSL2 distro rancher/k3s:v1.0.0

If you run the following on the latest wsl2 enabled windows 10 (make sure you have it set as the default), you should get k3s with rio on it, inside WSL2. It will also add k3s and rio cli using scoop.

iex (new-object net.webclient).downloadstring('https://gist.github.com/juliostanley/622486d82cb8ed334d270ead5f684e89/raw/cc39b22add65d71c32dc4e9f9a227c1ce1e3d7e3/install-k3s-rio-wsl2.ps1')

Here is a README with more details.

Also, you can point VSCode to //wsl$/liifi-k3s/etc/rancher/k3s/k3s.yaml but there is a caveat (you must first modify the file to use localhost instead of 127.0.0.1, the above script will take care of it)

@DereMann

This comment has been minimized.

Copy link

@DereMann DereMann commented Mar 9, 2020

k3s server runs nicely on WSL 2 - kernel configs not needed anymore.

Combining it with microsoft/WSL#4150 forwarding port 6443 from the host to WSL makes your server accessible for agents in the network.

@avoidik

This comment has been minimized.

Copy link

@avoidik avoidik commented Oct 1, 2020

if you're looking for the cleanup scripts, this is the way how to get them

# download installer
curl -fsSL https://raw.githubusercontent.com/rancher/k3s/release-1.19/package/rpm/install.sh -o install.sh
# make it executable
chmod +x install.sh
# comment out non-relevant steps at the end of the file, preserve setup_env, create_killall, create_uninstall steps
sed -e '/verify_system$/ s/^#*/#/' -i install.sh
sed -e '/download_and_verify$/ s/^#*/#/' -i install.sh
sed -e '/create_symlinks$/ s/^#*/#/' -i install.sh
sed -e '/systemd_disable$/ s/^#*/#/' -i install.sh
sed -e '/create_env_file$/ s/^#*/#/' -i install.sh
sed -e '/create_service_file$/ s/^#*/#/' -i install.sh
sed -e '/service_enable_and_start$/ s/^#*/#/' -i install.sh
# generate cleanup scripts
INSTALL_K3S_BIN_DIR="." ./install.sh
# uninstall
./k3s-uninstall.sh
@kable-wilmoth

This comment has been minimized.

Copy link

@kable-wilmoth kable-wilmoth commented Oct 1, 2020

If you take a few pieces out of this post you can get systemd running and then k3s's regular install just works. No k3s workflow modifications needed.
https://forum.snapcraft.io/t/running-snaps-on-wsl2-insiders-only-for-now/13033

  • apt-get install daemonize
  • /usr/sbin/start-systemd-namespace
  • /usr/sbin/enter-systemd-namespace
  • sudoers
  • /etc/bash.bashrc
  • Windows env

Now to get this started somehow automagically when windows restarts (probably have to auto log in user and lock the computer). (windows is a required platform for a few of our customers)

@avoidik

This comment has been minimized.

Copy link

@avoidik avoidik commented Oct 1, 2020

unfortunately neither do not work with curl -sfL https://get.k3s.io | sh -s - --docker (docker runtime)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment