plembo/rpioskvm.md

## rpioskvm.md

      
    Raw
  

              rpioskvm.md
            
          
    Raspberry Pi OS on KVM

NOTICE

This steps and links provided in this gist are significantly outdated (having been created in 2020) and should not be followed. I will look into updating them when time permits.
Introduction

Raspberry Pi OS is a Debian Linux derivative for the Raspberry Pi. "KVM" is a native open source virtual machine management service for Linux workstations and servers that leverages QEMU and libvirt. There are a variety of recipes for getting Raspberry Pi OS guest virtual machine up and running on KVM. Having tried many of these, this is the one I now use. Tested on Ubuntu 18.04 LTS Desktop, with all that implies (YMMV).
This is a "just for fun" project that I don't expect to make serious use of. With several actual Pi boards around the workshop, being able to reliably run on an emulator isn't really a major need.
Procedure [OUTDATED: DO NOT USE]


Verify that the KVM service is running on the target host system. Since there is no uniformity among Linux distributions in the installation and configuration of KVM, check the documentation available for the target host system distro for details [1].
Download and unzip the latest official Raspberry Pi OS image (I use the "lite" version), https://www.raspberrypi.com/software/operating-systems/. For example, 2021-05-07-raspios-buster-armhf-lite.img [2].
Download the following QEMU-RPI kernel image and compiled device tree binary (dtb) files from https://github.com/dhruvvyas90/qemu-rpi-kernel : kernel-qemu-4.19.50-buster and versatile-pb-buster.dtb [3].
Create a directory to hold the kernel image and dtb file. In this example, "/var/lib/libvirt/custom/qemu-rpi" [4]:

$ sudo mkdir -p /var/lib/libvirt/custom/qemu-rpi
$ sudo cp kernel-qemu-4.19.50-buster /var/lib/libvirt/custom/qemu-rpi
$ sudo cp versatile-pb-buster.dtb /var/lib/libvirt/custom/qemu-rpi


Make a copy of the OS image and convert to qcow2 format [5]:

$ cp 2021-05-07-raspios-buster-armhf-lite.img rpitest.img
$ qemu-img convert -f raw -O qcow2 rpitest.img rpitest.qcow2


Resize the OS image to something reasonable for your purposes:

$ qemu-img resize rpitest.qcow2 +7G


Copy the resized disk image to where guest disk images go on your system:

sudo cp rpitest.qcow2 /var/lib/libvirt/images


Create an XML file for defining the guest (in this example, named "rpitest.xml"):

<domain type='qemu'>
  <name>rpitest</name>
  <uuid>bafbe3e4-a42c-4acc-8f50-41923a56e847</uuid>
  <title>rpitest</title>
  <description>Raspberry Pi OS</description>
  <memory unit='KiB'>262144</memory>
  <currentMemory unit='KiB'>262144</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <os>
    <type arch='armv7l' machine='versatilepb'>hvm</type>
    <kernel>/var/lib/libvirt/custom/qemu-rpi/kernel-qemu-4.19.50-buster</kernel>
    <cmdline>root=/dev/sda2</cmdline>
    <dtb>/var/lib/libvirt/custom/qemu-rpi/versatile-pb-buster.dtb</dtb>
    <boot dev='hd'/>
  </os>
  <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>arm1176</model>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-arm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rpitest.qcow2'/>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='scsi' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:cc:cc:cc'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
      <gl enable='no' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/>
    </graphics>
    <video>
      <model type='virtio' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </video>
  </devices>
</domain>

The above config is just an example. At a minimum the following items should be customized: name, uuid, kernel, dtb, disk/source, mac address. A new uuid can be generated using uuidgen. Here's a one-liner to generate a random MAC Address, openssl rand -hex 6 | sed 's/\(..\)/\1:/g; s/.$//'. After doing that you might want to replace the first three octets with those your other virtual machines have in common.
Once these changes are made, "define" the guest domain (i.e. virtual machine):

$ virsh define rpitest.xml


Start the guest with virsh start rpitest and connect to it (after a decent interval for booting, usually around a minute), virsh console rpitest.
Shut down the guest with virsh destroy rpitest.
Make a copy of the disk image, then expand the image filesystem [6]:

$ export IMAGEDIR=/var/lib/libvirt/images
$ sudo cp $IMAGEDIR/rpitest.qcow2 $IMAGEDIR/rpitest-copy.qcow2
$ sudo virt-resize --expand /dev/sdb2 $IMAGEDIR/rpitest-copy.qcow2 $IMAGEDIR/rpitest.qcow2


Reboot the guest, and verify the partition has been expanded using lsblk.

Notes

[1] Sad, but true, that the major distributions and their derivatives have been unable (or unwilling) to standardize even on the names of the required packages, let alone how they're configured.
[2] The choices presented on the main software download page are... limited. The links on the operating-systems page are for the latest release.
[3] The 4.19.50-buster kernel and versatile-pb-buster.dtb files work with the latest (2021-05-07) Raspberry OS image, even though the OS image actually ships with Linux kernel version 5.10. The 5.x kernels and dtb files in the repository do not work, and are the subject of an unresolved bug report from April, 2021. One commenter suggests this is a QEMU related problem, which makes sense (especially given that I'm using the ancient version of QEMU 2.11 that shipped with Ubuntu 18.04 LTS).
[4] The path given is based on the default for KVM in all major Linux distributions. My personal preference is to put disk images and custom files on a separate disk to conserve space on the root volume, but setting that up is beyond the scope of this gist.
[5] Pay careful attention to the syntax here. It requires a copy of the image as the utility can't make changes "in place".
[6] Notice how I cleverly avoided additional typing by creating a variable for the disk image path! After first boot a  Raspberry Pi disk image will have two partitions. In this case, /boot on /dev/sdb1, and / (root) on /dev/sdb2. The above virt-resize command expands the / (root) parition using the additional disk space created by qemu-img resize.