vfio.org

Pci Passthrough

Table of Contents

• VFIO setup
• VM Settings
  • NVIDIA GPUs
  • Setup Multicore CPUs for Windows
  • Add PCI Passthrough devices
  • Add Memory Shim for LookingGlass
• Setup Windows
  • Install IVSHMEM Driver
  • Install LookingGlass
  • Set User to Login Automaticly
• Additional Resources

VFIO setup

Enable AMD-V (SVT) or VT-d and IOMMU in your BIOS.

Add amd or intel iommu on switch and pcie_acs_override to kernel boot parameters

GRUB_CMDLINE_LINUX="... amd_iommu=on intel_iommu=on pcie_acs_override=downstream,multifunction ..."

Run lspci -nn to list your devices and add the IDs of your GPU and it’s soundcard, from the last [] brackets to /etc/modprobe.d/vfio.conf:

                     <gpu id>  <audio id>
options vfio-pci ids=1002:687f,1002:aaf8

Add vfio_pci to /etc/modules-load.d/vfio.conf to load the vfio driver at boot.

Now create a VM in virt-manager like normal and install an OS in it.

VM Settings

Some changes are needed to the VM to make passthrough work.

NVIDIA GPUs

NVIDIA likes to sell more expensive cards and have thus implemented some trickery into the driver so it fails with the omnious error Code 43, only to be found in GPU details in the device manager.

To trick the VM to think it’s a real host multiple things have to be done.

the features section of the VM’s config needs to be modified as follows:

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
+     <vendor_id state='on' value='ab1234567890'/>
    </hyperv>
+   <kvm>
+     <hidden state='on'/>
+   </kvm>
    <vmport state='off'/>
+   <ioapic driver='kvm'/>
  </features>

• vendor_id set’s a vendor ID so a random string that isn’t easily identified as belonging to qemu or kvm.
• kvm - hidden enables kvm’s features to hide it’s virtualisation.
• ioapic ensures kvm’s io driver with the hiding functionality is used.

Setup Multicore CPUs for Windows

The CPUs need to be configured as follows:

• Untick “Copy host CPU configuration” and set it to a processor similar to your architecture or try using host-passthrough.
• Tick “Manually set CPU topology”. Set the Sockets and threads to 1, and set as many cores as you want. Also adjust the “Current allocation” up top. This is needed as consumer versions of windows can only use 2 Physical CPUs maximium and qemu simulates multiple CPUs instead of multiple cores by default.

Add PCI Passthrough devices

Open the VMs settings in virt-manager and press Add Hardware. Select PCI Host Device and choose your GPU’s entry and add it. Also do this for it’s audio device.

Add Memory Shim for LookingGlass

Add the following to the configuration in the <devices> section:

 <devices>
+ <shmem name='looking-glass'>
+  <model type='ivshmem-plain'/>
+  <size unit='M'>32</size>
+ </shmem>
...
 </devices>

This will create a memory shim in /dev/shm/looking-glass through which LookingGlass comunicates and sends it’s picture. The size has to be adjusted depending on your resolution and can be calucated with this formula:

((width * height * 4 * 2) / 1024 / 1024) + 2

This value then needs to be rounded up to the nearest power of two. The default value of 32MB is good for 1080p and 1440p.

The user in linux you want to use LookingGlass with needs to have access to this device, but only the qemu user has access by default. ou can chown it to that user with an qemu hook in /etc/libvirt/hooks/qemu/own-looking-glass-shm.sh:

#!/bin/sh
chown <user>:kvm /dev/shm/looking-glass

and make that script executable.

Setup Windows

Now boot the VM again and install your GPUs drivers, check that the GPU worked by either checking if plugged in monitors show up in the Windows settings or that it is correctly listed in the device manager.

Install IVSHMEM Driver

This driver is needed for the LookingGlass host to use the memory shim. download version 0.1.161 or later here and unpack it. Then open the Device Manager, go to System Devices and find PCI standard RAM Controller. Install the driver by going to it’s properties, and clicking Update Driver. then choose to install the driver from your local computer and locate the unpacked driver.

Install LookingGlass

As a last step you need to download the actual LookingGlass host application. You can get it from the official site. Download a Version matching to your client.

You can add the host application to auto start with User login by executing the following in an admin command prompt and changing the path to point to your looking-glass-host.exe:

SCHTASKS /Create /TN "Looking Glass" /SC  ONLOGON /RL HIGHEST /TR C:\Users\<YourUserName>\<YourPath>\looking-glass-host.exe

Set User to Login Automaticly

Press Win+R to get a Run dialog and run netplwiz. Untick Users must enter a user name and password to use this computer and click OK. Confirm by entering your Password. You will automaticly login after a reboot.

After setting up the VM, shut it down and remove the SPICE Server and Virtual GPU. These can prevent LookingGlass to start.

Additional Resources

• LookingGlass Installation Guide
• Setup evdev for switchable usb passthrough
• Enable Remote Desktop in Windows