kostorv/debian-wsl2-kvm-win10.md

## debian-wsl2-kvm-win10.md

      
    Raw
  

              debian-wsl2-kvm-win10.md
            
          
    Build An Accelerated KVM Guest Custom Kernel for WSL 2 - Debian edition

In this gist I try to build an accelerated KVM Guest Custom Kernel for WSL2 for Debian distro. In this link Hayden Barnes implements it on OpenSUSE Tumbleweed distro. Though the procedure seems pretty straightforward I am stumbling upon some issues which I will describe below.
Fresh installation of WSL2 Debian

$ uname -r 
5.10.60.1-microsoft-standard-WSL2 (This changes with different Windows Updates for WSL)
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
...

This feature, for AMD processors, works with the latest Windows 11 update (or build 20175 for Insiders Program only). In general Microsoft has made nested virtualization available from Windows 10 Build 19636.
Step 1: Upgrade Distro


Upgrade WSL2 Debian to the latest distro version: Debian 11 bullseye (Upgrade debian). Afterwards, when we will try to build the custom Kernel we need our distro to have the latest packages available or we will need to do some extra tinkering to solve occuring issues. To do that, first we need to update and upgrade to latest current version and packages:

$ sudo apt update && sudo apt upgrade


Then simply edit the /etc/apt/sources.list file and replace the word stretch to bullseye. Notice that in the latest entry we need to change the whole source:

...
# change this
deb http://security.debian.org/debian-security/ stretch/updates main
# to this
deb http://security.debian.org/debian-security/ bullseye-security main


and again we update and upgrade to latest packages followed by a full-upgrade

$ sudo apt update && sudo apt upgrade
$ sudo apt full-upgrade


Reboot WSL2

on CMD: wsl --shutdown
rerun WSL Debian.


Now we upgraded to Debian 11 bullseye!
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
...

Step 2: Install packages


Now to be able to build the kernel we need to install some packages:

$ sudo apt -y install ssh ssh-askpass curl aria2 git jq tmux vim \
  bison flex python3 libssl-dev libelf-dev build-essential \
  libncurses-dev zlib1g-dev dwarves bc \
  virt-manager qemu-kvm libvirt-clients libvirt-daemon-system virtinst


Explanation:

1st row       : helper packages
2nd & 3rd rows: kernel build*
4th row       : KVM-virtualization packages


* These packages are recommended from microsoft/WSL2-Linux-Kernel official repo, and I added some when errors occured.
Step 3: Build Kernel


Download Custom Linux Kernel, untar, cd in dir:

$ curl -s https://api.github.com/repos/microsoft/WSL2-Linux-Kernel/releases/latest | jq -r '.name' | sed 's/$/.tar.gz/' | sed 's#^#https://github.com/microsoft/WSL2-Linux-Kernel/archive/refs/tags/#' | aria2c -i -
$ tar -xf *.tar.gz
$ cd WSL<tab>


Copy Microsoft's default config file:

$ cp Microsoft/config_wsl .config


Change entries:
Here I used $ make menuconfig and selected manually some options for AMD support:


Processor type and features >

Linux guest support >

KVM Guest support


Processor family >

Generic-x86_64


Supported processor vendors >

Support AMD processors (select yours accordingly)


Virtualization >

Kernel-based Virtual Machine (KVM) support
KVM for AMD processors support


!Save on exit!
and then followed the tutorial's sed commands:
$ sed -i 's/# CONFIG_KVM_GUEST is not set/CONFIG_KVM_GUEST=y/g' .config
$ sed -i 's/# CONFIG_ARCH_CPUIDLE_HALTPOLL is not set/CONFIG_ARCH_CPUIDLE_HALTPOLL=y/g' .config
$ sed -i 's/# CONFIG_HYPERV_IOMMU is not set/CONFIG_HYPERV_IOMMU=y/g' .config
$ sed -i '/^# CONFIG_PARAVIRT_TIME_ACCOUNTING is not set/a CONFIG_PARAVIRT_CLOCK=y' .config
$ sed -i '/^# CONFIG_CPU_IDLE_GOV_TEO is not set/a CONFIG_CPU_IDLE_GOV_HALTPOLL=y' .config
$ sed -i '/^CONFIG_CPU_IDLE_GOV_HALTPOLL=y/a CONFIG_HALTPOLL_CPUIDLE=y' .config
$ sed -i 's/CONFIG_HAVE_ARCH_KCSAN=y/CONFIG_HAVE_ARCH_KCSAN=n/g' .config
$ sed -i '/^CONFIG_HAVE_ARCH_KCSAN=n/a CONFIG_KCSAN=n' .config


Build the Kernel

$ make -j 8


Last step

Copy the created kernel to your Windows user home folder:
$ sudo cp ./arch/x86_64/boot/bzImage /mnt/c/Users/<your-user>/bzImage
Create a global wsl config file to instruct wsl to use your newly created Kernel:
$ sudo vim /mnt/c/Users/<your-user>/.wslconfig

[wsl2]
kernel=C:\\Users\\kostorv\\bzImage
nestedVirtualization=true


Reboot your WSL
Check if you are using your new Kernel:
$ uname -r
Check if you are supporting nested Virtualization:
$ dmesg | grep kvm
Should prompt something like: kvm: Nested Virtualization enabled


KVM, libvirtd, virt-manager, virsh Tweaks

In order to use virtualization inside WSL2 Debian you have 2 options:

full terminal
GUI

but either way you will need an X window opened in Windows through WSL.
Configuration for Xterm support through Windows


Configure

$ vim ~/.bashrc

# add the following under aliases section
# set DISPLAY to use X terminal
export DISPLAY=HOST_HOME_IP:0.0     ### TODO: Find a dynamic way of finding HOST_IP

# Check if necessary services are running; else start them
# this is necessary because when WSL shuts down on reboot
# no service is running
function check_service {
    sudo service $1 status > /dev/null
    exit_code=$?
    if [ $exit_code -eq 3 ]; then
        sudo service $1 start
    elif [ $exit_code -eq 0 ]; then
        echo "$1 service is running!"
    else
        echo "check services and config!"
    fi
}

check_service ssh
check_service libvirtd
check_service virtlogd


On Windows download MobaXterm and run it.
Connect through ssh to WSL using MobaXterm, WSL listens at 127.0.0.1


After logged in with ssh will be prompted:
/usr/bin/xauth:  file /home/kostorv/.Xauthority does not exist
..... at this step run once "xauth" to create .Xauthority file
then close everything and reconnect with MobaXterm through ssh


Troubleshooting

!IMPORTANT! (openwrt/openwrt#9019)
First upgrade the system to latest Debian 11 then start implementing all steps. If for any reason on
make -j 8 step where we build the kernel pops a message complaining about pahole's version or
something like that:
user@wsl-debian:~/WSL2-Linux-Kernel-linux-msft-wsl-5.10.93.2$ make -j 8
  DESCEND  objtool
  DESCEND  bpf/resolve_btfids
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/byteswap-16.h', needed by '/home/user/WSL2-Linux-Kernel-linux-msft-wsl-5.10.93.2/tools/bpf/resolve_btfids/fixdep.o'.  Stop.
make[3]: *** [Makefile:41: /home/user/WSL2-Linux-Kernel-linux-msft-wsl-5.10.93.2/tools/bpf/resolve_btfids/fixdep-in.o] Error 2
make[2]: *** [/home/user/WSL2-Linux-Kernel-linux-msft-wsl-5.10.93.2/tools/build/Makefile.include:5: fixdep] Error 2
make[1]: *** [Makefile:71: bpf/resolve_btfids] Error 2
make: *** [Makefile:1948: tools/bpf/resolve_btfids] Error 2
make: *** Waiting for unfinished jobs....
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/byteswap-16.h', needed by '/home/user/WSL2-Linux-Kernel-linux-msft-wsl-5.10.93.2/tools/objtool/fixdep.o'.  Stop.
make[3]: *** [Makefile:41: /home/user/WSL2-Linux-Kernel-linux-msft-wsl-5.10.93.2/tools/objtool/fixdep-in.o] Error 2
make[2]: *** [/home/user/WSL2-Linux-Kernel-linux-msft-wsl-5.10.93.2/tools/build/Makefile.include:5: fixdep] Error 2
make[1]: *** [Makefile:68: objtool] Error 2
make: *** [Makefile:1948: tools/objtool] Error 2
  CALL    scripts/atomic/check-atomics.sh
  CALL    scripts/checksyscalls.sh

try to type:
$ make distclean