libvirt-vga-passtrough

libvirt-vga-passtrough.rst

New guide for vga-passtrough VMs with STOCK kernel on Arch with libvirt.

Install libvirt, virt-manager, latest stock kernel, ovmf-bin. ovmf-bin may need to come from the AUR. Check if your graphics card is UEFI compilant. If not, search for a rom/gpu bios file that does support UEFI and store it somewhere.

** You may need to run your host on UEFI as well if using the i915 as primary graphics. **

If you get permission issues, try running the qemu instances as root. Edit /etc/libvirt/qemu.conf for this.

You can find the ovmf file here: '/usr/share/ovmf/ovmf_x64.bin' if you need it later.

In '/etc/mkinitcpio.conf':

MODULES="pci-stub vfio-pci"

If libvirt later comes up with a 'IOMMU groups not found' kind like error, add 'vfio_iommu_type_1' to this list. Run a 'mkinitcpio -p linux' after editing mkinitcpio.conf.

In '/etc/default/grub' set your GPU's pci ids in the CMDLINE like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pci-stub.ids=1002:6819,1002:aab0"

Update your grub config afterwards. (grub-mkconfig -o /boot/grub/grub.cfg)

Open up virt-manager. Create a new VM and create it for your Windows installation. Do not customize the settings, just create the VM and force poweroff it when it opens. Do not install Windows. Do NOT choose the q35 chipset. USE i440fx!

Open the VM configuration in virt-manager, and remove all Spice stuff (mouse, keyboard, display, tablet). Set your disks to VirtIO. Not VirtIO SCSI. You may need to remove and re-add your disk manually if virt-manager does not show you this option.

Mount your Windows installation ISO as IDE CDROM. Ensure the Windows ISO is EFI compliant. Download the virtio driver ISO from here: http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/bin/. Add the ISO to your VM as a second IDE CDROM.

Set your CPU topology manually. By default libvirt exposes as seperate sockets, but Windows won't let you use more than two which causes performance issues.

Add your keyboard and mouse as usb devices to your VM using the virt-manager interface and usb device passtrough. Alternatively passtrough the PCI controller they're on. Also pci passtrough your graphics card, using the virt-manager interface. It's advised to not passtrough the 'HDMI audio' device your GPU comes with. (you'll need to create a second passtrough device for this if you wanted to, but just don't).

Close virt-manager. Fire up a shell. Launch 'EDITOR=vim virsh -c qemu:///system edit <name_of_vm>'. If authentication fails, search on the Arch wiki for libvirt polkit rules.

Under the <os> tag, add a <loader> element with the path to the bios. Like this:

<os>
 <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type>
 <loader type='pflash'>/usr/share/ovmf/ovmf_x64.bin</loader>
</os>

If you need to inject a UEFI bios for your vm, search for you hostdev pci device and make it look like this:

<hostdev mode='subsystem' type='pci' managed='yes'>
  <source>
    <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
  </source>
  <rom file='/home/maikel/bios7850random.rom'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</hostdev>

The <rom> tag contains the path to the rom you downloaded. Save the file and exit vim. If you haven't rebooted after making the mkinitcpio changes, do it now.

Launch virt-manager. Start your VM and check if something happens. You may get dropped to the UEFI shell. It will mount a device (mostly called FS0:) if a efi partition is found. It works like a Windows shell, 'fs0:' changes to the cdrom device, and cd, dir and ls work like expected. Search for the EFI file (mostly in /efi/boot/), and type it's name to execute it.

The Windows installer will complain it didn't find any disks. Search for drivers on the second cdrom, they're in WIN8/amd64/. The disk driver is called something like 'RedHat VIRTIO SCSI disk device'. Choose that. Now your disk appears and you can install Windows like usual.

first of all: excellent gist, thanks
second, a bit of assistance might be great
tried it with:

amd fx 8320
m5a97 pro
8 gb ram
asus r9 270 (for arch host)
sapphire hd 6850 (for windows 10 guest (tested it on another pc with the exact same windows as a host - everything's fine))
750 w psu

everything works and boots, except when i try to install catalyst: it gives BSOD: VIDEO_TDR_FAILURE (atikmpag.sys). Pci-stub claims appropriate devices (saw it from dmesg)

I only have an Intel system, maybe things differ for AMD.

Have a look at https://bbs.archlinux.org/viewtopic.php?id=162768
and http://vfio.blogspot.com

Too late, already solved it, did passthrough the wrong way

Very useful links, also used them..
Windows 10's working and updating to new builds, but sometimes it can stuck on boot with windows logo - no spinning circle underneath for long time, vm hard reset helps

@walk1ndude Hey man, I know it's really late, but how did you do passthrough wrong?

@PaulBGD
Hmm.. trying to remember. It seems I had another video adapter with qxl or something similar..
Anyway, here is my current config (windows 10 insider builds with crimson driver work):

<domain type='kvm'>
  <name>windows</name>
  <uuid>80831171-02e1-4c05-be41-011401d13184</uuid>
  <memory unit='KiB'>12582912</memory>
  <currentMemory unit='KiB'>12582912</currentMemory>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>Opteron_G5</model>
    <feature policy='require' name='nx'/>
    <feature policy='require' name='invtsc'/>
    <feature policy='require' name='bmi1'/>
    <feature policy='require' name='perfctr_nb'/>
    <feature policy='require' name='perfctr_core'/>
    <feature policy='require' name='topoext'/>
    <feature policy='require' name='nodeid_msr'/>
    <feature policy='require' name='tce'/>
    <feature policy='require' name='lwp'/>
    <feature policy='require' name='wdt'/>
    <feature policy='require' name='skinit'/>
    <feature policy='require' name='ibs'/>
    <feature policy='require' name='osvw'/>
    <feature policy='require' name='cr8legacy'/>
    <feature policy='require' name='extapic'/>
    <feature policy='require' name='cmp_legacy'/>
    <feature policy='require' name='fxsr_opt'/>
    <feature policy='require' name='mmxext'/>
    <feature policy='require' name='osxsave'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='vme'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-wrapper</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/usr/share/virtio/virtio-win.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback' io='threads'/>
      <source file='/var/lib/libvirt/images/windows.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback' io='threads'/>
      <source file='/home/walkindude/windows/driveD.qcow2'/>
      <target dev='vdb' bus='virtio'/>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='usb' index='3' model='nec-xhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0f' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </controller>
    <interface type='direct'>
      <mac address='52:54:00:a6:c0:c9'/>
      <source dev='eth0' mode='bridge'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:fc:01:a4'/>
      <source network='isolated'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </hostdev>
    <watchdog model='i6300esb' action='reset'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/>
    </watchdog>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
    <panic model='isa'>
      <address type='isa' iobase='0x505'/>
    </panic>
  </devices>
</domain>

qemu-wrapper is a little script to use vga driver. I'm using seabios, tried omvf but after some number of reboots I always getting black screen - it was a year ago, maybe now it can work.. don't want to reinstall windows)
qemu-wrapper:

#!/bin/sh
exec /usr/sbin/qemu-system-x86_64 `echo "\$@" | sed 's|06:00.0|06:00.0,x-vga=on|g'`

About stucking.. It's gone: maybe because I upgraded to Sabertooth 990FX R2.0 (at least no more IRVS table fixing by kernel boot parameters), maybe because new windows and linux builds are getting better...

Note: If you want to upgrade to the new Windows build - make sure the following lines are present in your xml:

<features>
  <acpi/>
  <kvm>
    <hidden state='on'/>
  </kvm>
</features>
<cpu mode='custom' match='exact'>
  <model fallback='allow'>core2duo</model>
  <feature policy='require' name='nx'/>
</cpu>

http://ubuntuforums.org/showthread.php?t=2289210
Keep them during the upgrade, when its finished you can revert back to your cpu

What I haven't solved (or maybe too lazy to look - using pf all the time anyway):
On stock arch kernel when I boot windows linux freezes but windows works as expected
If I do the same on linux-pf which I build from AUR, both systems work and I successfully share mouse and keyboard between them by Synergy.

Booting kernel with:
root=/dev/mapper/lvmVolume-lvmRoot rw irqpoll iommu=pt iommu=1 vfio-pci.ids=1002:6811,1002:aab0