🖥 Lightweight diskless VM with QEMU (Arch Linux variant)

README.md

🖥 Lightweight diskless VM with QEMU (Arch Linux variant)

This document describes a process to quickly setup a Linux VM for testing or kernel development.

The VM requires no disk image, instead it boots off a directory on the host (using virtiofs). This is:

• more performant (no FS-in-FS overhead)
• more convenient for experiments (both host and guest can modify files concurrently)
• safer (no corruption* in case of kernel panic)
• easier to set up

The lightweight microvm platform is used, with minimal VIRTIO hardware (serial console, filesystem, network, ballooning).

Steps

• Prerequisites:

  sudo pacman -S qemu-system-x86 virtiofsd arch-install-scripts

• Prepare the rootfs directory for the VM:

  sudo mkdir qemu-rootfs   # <- sudo, so that directory is owned by root
  sudo mount -m --bind qemu-rootfs qemu-rootfs-bind
  sudo pacstrap -K qemu-rootfs-bind base linux
  sudo arch-chroot qemu-rootfs-bind passwd   # <- set a root password
  sudo arch-chroot qemu-rootfs-bind sh -c 'echo -e "MODULES=(virtio_mmio virtiofs)\nHOOKS=(base)" > etc/mkinitcpio.conf && mkinitcpio -P'

• Place the provided qemu script in the directory, and run it (with sudo) to start the VM:

  curl -fLO https://gist.github.com/mildsunrise/fa1a4269c56e50f7436b8b8d64b55d14/raw/qemu
  chmod +x qemu
  sudo ./qemu

Using the network

The qemu script adds a network interface with the user backend, which requires no setup or special host privileges. You can use it to access the Internet from the VM, just make sure the interface is configured with DHCP, i.e.:

cat > /etc/systemd/network/20-wired.network <<EOF
[Match]
Name=ens4

[Network]
DHCP=yes
EOF

systemctl enable systemd-networkd
systemctl enable systemd-resolved
reboot

The script also sets up a port redirection that makes the VM's SSH port accessible at port 22001 on the host. To use it, start an SSH server:

pacman -S openssh
# enable root-login in /etc/ssh/sshd-config
systemctl enable sshd
reboot

And use ssh -p 22001 root@localhost. For convenience you can set up an entry in your ~/.ssh/config:

Host linux-vm
Hostname localhost
Port 22001
User root

Then use ssh linux-vm.

qemu

#!/bin/sh
cd "$(dirname "$0")"

sudo /usr/lib/virtiofsd \
  --socket-path=qemu-virtiofs.sock --socket-group="$(id -grn)" \
  --shared-dir="$PWD"/qemu-rootfs --cache=always &

sleep 0.1s

# FIXME: enable DAX when upstreamed

# '-cpu host' temporarily removed because of a bug in qemu:
# <https://gitlab.com/qemu-project/qemu/-/issues/2142>

exec qemu-system-x86_64 \
  -enable-kvm \
  -smp 1 -m 512 \
  \
  -nodefaults -no-user-config -nographic \
  -machine microvm,memory-backend=mem,x-option-roms=off,pit=off,pic=off,isa-serial=on,rtc=off \
  -bios /usr/share/qemu/qboot.rom \
  -global virtio-mmio.force-legacy=false \
  -serial stdio \
  \
  -device virtio-balloon-device \
  \
  -chardev socket,id=char0,path=qemu-virtiofs.sock \
  -device vhost-user-fs-device,queue-size=1024,chardev=char0,tag=rootfs \
  -object memory-backend-memfd,id=mem,size=512M,share=on \
  \
  -netdev user,id=net0,hostfwd=tcp:127.0.0.1:22001-:22 \
  -device virtio-net-device,netdev=net0 \
  \
  -kernel qemu-rootfs/boot/vmlinuz-linux \
  -initrd qemu-rootfs/boot/initramfs-linux.img \
  -append "earlyprintk=ttyS0 console=ttyS0 rootfstype=virtiofs root=rootfs rw"