Instantly share code, notes, and snippets.

Embed
What would you like to do?
[GUIDE] Optimus laptop dGPU passthrough

Reddit post (Archived)

Table of Content

This is a guide for passing through you dGPU on your laptop for your VM. This guide only apply to laptops that does not load dGPU firmware through acpi call, which include all MUXed laptop and some MUXless laptop. For laptops that use acpi call to load dGPU firmware, please refer to to u/jscinoz 's optimus-vfio-docs.

Sorry but currently I don't know how to check if your dGPU load its firmware through acpi call.

Update: Use hexadecimal id directly instead of convert it to decimal, add some note for romfile option

Update: Forget that -vga none would cause Guest has not initialized the display (yet) problem if you don't have a system installed

Update: Use qemu 2.11.2 with pulse audio patch and vcpupin, add some caveats for 18.04

What to expect?

Depends on your hardware, you can have a laptop that:

  • Physically running a Linux distribution as the host machine,
  • Can power on/off and utilize your Nvidia dGPU on demand with bumblebee,
  • Can pass your Nvidia dGPU to your VM when you don't need it in your host machine,
  • Can have your dGPU back when the VM shutdown,
  • Can use your dGPU with bumblebee again without any problem,
  • No need to reboot during this dGPU binding/unbinding process,
  • No need for external display (depend on your hardware and the version of Windows your VM running),
  • Can connect external display directly to your VM (only some machine with specific setup).

Frame rate test

Unigine Heaven 4.0 Basic test

Steam in-home streaming between Windows VM and host:

  • Both game use high preset with V-Sync enabled.
  • Max fps of Witcher 3 has set to 60.
  • No extra monitor what so ever.

DOOM

Witcher 3

*This is my laptop running in Optimus mode with a 1080p@120Hz panel (I swapped the original 1080p@60Hz myself) and a MXM form factor Quadro P5000(QS). This laptop is MUXed.

Some TLDR about the idea behind

As you might read after, this tutorial is pretty much the same as most passthrough guide. The keypoint, however, is to assign Subsystem ID for the dGPU using some vfio-pci options. My dGPU appears to have a Subsystem ID 00000000 inside the VM by default.

About one display setup, although frames are rendered in GPU memory, display ports is not the only way to get those frames. Nvidia itself provides API to capture things in GPU memory, this is why we can have technology like Steam in-home streaming and Geforce experience. For me, I have RemoteFX working, and that is the only reason why I put that in this tutorial. Despite I use a Quadro, this mobile version GPU does not support NvFBC capture API (the same as other consumer card), which means it's capability is no more than a GeForce, so you should be able to get RemoteFX working with Geforce.

Some might be heard of gnif's phenomenal work, which made a huge step forward for one-display setup. Unfortunately, a dummy device is still required for that setup, which is a no go for laptop. Even with a MUXed laptop, having a dummy device plug-in still means that your GPU needs to expose some form of display output signal physically, but most Laptop don't support this. As far as I know, Dell precision 7000 line-up can enable DisplayPortDirectOutput mode in BIOS, which would route GPU signal directly to video output port, while keeping iGPU rendering the built-in display.

Prerequisites

Please noted that this tutorial does not support every Optimus laptop. Generally, a good laptop with some specific hardware capability is required. If you have a laptop that have a swappable MXM form factor graphics card, its highly possible that you'll success.

Also, due to the nature that laptop varies so much from manufacture to manufacture, there is no way you can tell if it is MUXed, or MUXless, or how a MUXless laptop load its firmware before you get your hands on it. However, the firmware loading mechanism of your GPU is crucial for resolving the infamous Code 43 problem. So please do enough homework (find some success report in particular) before you plan to purchase a laptop for this purpose.

Hardware

  • A CPU that support hardware virtualization (Intel VT-x) and IOMMU (Intel VT-d).

    • Check here for a full list of qualified CPU
  • A motherboard that support IOMMU with decent IOMMU layout e.g. your dGPU is in its own IOMMU group aside from other devices.

    • For the reason that there is no ACS support for laptop (maybe some bare-bone does), so far, a decent IOMMU layout is crucial since the ACS override patch is not applicable.
  • Verification:

    • Boot with intel_iommu=on kernel parameter and use dmesg | grep -i iommu to verify you IOMMU support, this will also print your IOMMU layout.

    • Example:

      • # From "lspci":
        # 00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x16) (rev 05)
        # 01:00.0 VGA compatible controller: NVIDIA Corporation Device 1bb6 (rev a1)
        
        # From "dmesg | grep iommu"
        [    0.000000] DMAR: IOMMU enabled
        [    0.086383] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
        [    1.271222] iommu: Adding device 0000:00:00.0 to group 0
        [    1.271236] iommu: Adding device 0000:00:01.0 to group 1
        [    1.271244] iommu: Adding device 0000:00:04.0 to group 2
        [    1.271257] iommu: Adding device 0000:00:14.0 to group 3
        [    1.271264] iommu: Adding device 0000:00:14.2 to group 3
        [    1.271277] iommu: Adding device 0000:00:15.0 to group 4
        [    1.271284] iommu: Adding device 0000:00:15.1 to group 4
        [    1.271293] iommu: Adding device 0000:00:16.0 to group 5
        [    1.271301] iommu: Adding device 0000:00:17.0 to group 6
        [    1.271313] iommu: Adding device 0000:00:1c.0 to group 7
        [    1.271325] iommu: Adding device 0000:00:1c.2 to group 8
        [    1.271339] iommu: Adding device 0000:00:1c.4 to group 9
        [    1.271360] iommu: Adding device 0000:00:1f.0 to group 10
        [    1.271367] iommu: Adding device 0000:00:1f.2 to group 10
        [    1.271375] iommu: Adding device 0000:00:1f.3 to group 10
        [    1.271382] iommu: Adding device 0000:00:1f.4 to group 10
        [    1.271390] iommu: Adding device 0000:00:1f.6 to group 10
        [    1.271395] iommu: Adding device 0000:01:00.0 to group 1
        [    1.271407] iommu: Adding device 0000:02:00.0 to group 11
        [    1.271418] iommu: Adding device 0000:03:00.0 to group 12
        
      • Here the GPU and its root port are in the same group, and there is no other device in this group, thus make it a decent IOMMU layout.

System & Software

  • Host:
    • I'm currently running Ubuntu 16.04 (with 4.15 kernel), but it should also work on other distribution.
    • System should be installed in UEFI mode, and boot via UEFI.
  • Guest:
    • Windows that support RemoteFX. Windows 10 Pro for example.
  • QEMU:
    • Currently running QEMU 2.11.2 with pulse audio and vcpupin patch
    • If you you use QEMU 2.10 or higher and encounter a boot hang (dots spinning forever), check your OVMF version, it might need an upgrade. Refer here for further detail.
  • RDP Client:
    • Freerdp 2.0 or above for RDP 8 with RemoteFX connection.

Note: Keep your dual-boot Windows if you still want to run software like XTU.

Update: Attention for MUXless laptop

Not sure anyone succeseded with a MUXless laptop yet (Or failed with a MUXed laptop). If you do success, please consider leave a comment with your setup (laptop model, year of production/purchase, etc.), so that other people can have some reference.

Now for switchable graphics, there are three different solutions: MUXed(Old), MUXless and MUXed(New)

Circuits diagram

Most modern Optimus laptop use MUXless scheme, while some others, HP/Thinkpad/Dell mobile workstation, Clevo P650, some Alienware, etc. use MUXed scheme. At the dark age before Optimus solution came out, there is an old MUXed scheme which require reboot to switch graphics card and can only use one at a time, while the modern MUXed allow switch between Optimus and dGPU only, and can even have display output port hooked directly to the dGPU when using Optimus (only applicable for some laptop).

For people who encounter Code 43 with a MUXless scheme, that is to say, you can see your dGPU in guest, can even have nvidia driver installed without any problem, but still have this error code. This is because ACPI call failed for firmware loading, in short:

  • Nvidia driver try to read your dGPU ROM from system BIOS instead of using the ROM you provided through vfio-pci (this is actually how a real MUXless dGPU get its ROM).

  • Please refer to u/jscinoz 's optimus-vfio-docs if you encounter such problem

Some success report

Perdouille got a MUXless laptop working, with MSI GS60-040XFR with an i7-4720HQ and a 970m.

qgnox got a MUXless laptop working, with MSI GS60 2PC with GTX860M.

ASUS G751JM (i7-4710HQ, NVIDIA gtx 860m), thanks to @d0ku

Bumblebee setup guide

Note: For people who don't want to setup bumblebee, follow this to get your GPU's ACPI address, and power it on/off by refering script here. (Credit to Verequies from reddit)

Note: You might need to disable secure boot before following continue on this part.

We will first go through my bumblebee setup process. I did install bumblebee first and setup passthrough the second. But it should work the other way around.

  1. (Optional) Solving the known interference between TLP and Bumblebee

    • If you don't want to use tlp, please skip this part.

    • TLP is a must have for a Linux laptop since it provides extra policies to save your battery. Install TLP by sudo apt install tlp

    • Add the output of lspci | grep "NVIDIA" | cut -b -8 to RUNTIME_PM_BLACKLIST in /etc/default/tlp, uncomment it if necessary. This will solve the interference.

  2. Install Nvidia proprietary driver through Ubuntu system settings (Or other install method you prefer).

  3. (Trouble shooting) Solving the library linking problem in Nvidia driver.

    • If error messages show up after executing sudo prime-select intel or sudo prime-select nvidia, follow instructions below.

    • # Replace 'xxx' to the velrsion of nvidia driver you installed
      # You might need to perform this operation everytime your upgrade your nvidia driver.
      sudo mv /usr/lib/nvidia-xxx/libEGL.so.1 /usr/lib/nvidia-xxx/libEGL.so.1.org
      sudo mv /usr/lib32/nvidia-xxx/libEGL.so.1 /usr/lib32/nvidia-xxx/libEGL.so.1.org
      sudo ln -s /usr/lib/nvidia-xxx/libEGL.so.375.66 /usr/lib/nvidia-xxx/libEGL.so.1
      sudo ln -s /usr/lib32/nvidia-xxx/libEGL.so.375.66 /usr/lib32/nvidia-xxx/libEGL.so.1
    • If everything work correctly, sudo prime-select nvidia and then logout will give you a login loop. While sudo prime-select intel (do this in other tty with Ctrl+Alt+F2) will solve the login loop problem.

    • It is recommended to switch back and forth for once, if you run into some problem after a nvidia driver update.

  4. Blocking nouveau

    • Adding content below to /etc/modprobe.d/blacklist-nouveau.conf:

      • blacklist nouveau
        options nouveau modeset=0
        
    • sudo update-initramfs -u when finish.

    • If you have a DM running under wayland (such as Ubuntu 18.04, it runs GDM in wayland mode, despite GNOME is running under X11), some extra work might be needed to prevent nouveau from loading. Refer here for details.

  5. (Optional) Install CUDA, since the CUDA installation process is well guided by Nvidia, I will skip this part.

    • For CUDA, I personally recommend runfile installation. It is far more easy to maintain compare to other installation method. Just make sure neither the display driver (self-contain in the runfile) nor the OpenGL libraries is checked during the runfile installation process. ONLY install the CUDA Toolkit and don't run nvidia-xconfig.
  6. Solve some ACPI problem before bumblebee install:

  7. Install bumblebee

    • # For Ubuntu 18.04, the official ppa should work
      sudo add-apt-repository ppa:bumblebee/testing
      sudo apt update
      
      sudo apt install bumblebee bumblebee-nvidia
    • Edit /etc/bumblebee/bumblebee.conf:

      • Change Driver= to Driver=nvidia
      • Change all occurrences of nvidia-current to nvidia-xxx (xxx is your nvidia driver version)
      • KernelDriver=nvidia-xxx
      • It appears that nvidia driver change its location in Ubuntu 18.04, refer here for details and solutions.
    • Save the file and sudo service bumblebeed restart

  8. Kernel module loading modification:

    • Make sure corresponding section in /etc/modprobe.d/bumblebee.conf look like below

      • # Again, xxx is your nvidia driver version.
        blacklist nvidia-xxx
        blacklist nvidia-xxx-drm
        blacklist nvidia-xxx-updates
        blacklist nvidia-experimental-xxx
    • Add content below to /etc/modules-load.d/modules.conf

      • i915
        bbswitch
        
    • sudo update-initramfs -u when finish.

  9. (Optional) Create a group for bumblebee so that you don't need to sudo every time:

    • If cat /etc/group | grep $(whoami) already gives your user name under bumblebee group, skip this part.

    • groupadd bumblebee && gpasswd -a $(whoami) bumblebee

  10. (Trouble shooting) Try optirun nvidia-smi, if encounter [ERROR][XORG] (EE) Failed to load module "mouse" (module does not exist, 0), add lines below to /etc/bumblebee/xorg.conf.nvidia

  • Section "Screen"
      Identifier "Default Screen"
      Device "DiscreteNvidia"
    EndSection
    
  • Check here for more information about this problem.

  1. Verification:

    • cat /proc/acpi/bbswitch should gives you Ouput:0000:01:00.0 OFF

    • optirun cat /proc/acpi/bbswitch should gives you Ouput:0000:01:00.0 ON

    • nvidia-smi should give you something like:

      • NVIDIA-SMI couldn't find libnvidia-ml.so library in your system. Please make sure that the NVIDIA Display Driver is properly installed and present in your system.
        Please also try adding directory that contains libnvidia-ml.so to your system PATH.
        
    • optirun nvidia-smi should gives you something like:

      • Wed Nov 15 00:36:53 2017       
        +-----------------------------------------------------------------------------+
        | NVIDIA-SMI 384.90                 Driver Version: 384.90                    |
        |-------------------------------+----------------------+----------------------+
        | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
        | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
        |===============================+======================+======================|
        |   0  Quadro P5000        Off  | 00000000:01:00.0 Off |                  N/A |
        | N/A   44C    P0    30W /  N/A |      9MiB / 16273MiB |      3%      Default |
        +-------------------------------+----------------------+----------------------+
                                                                                       
        +-----------------------------------------------------------------------------+
        | Processes:                                                       GPU Memory |
        |  GPU       PID   Type   Process name                             Usage      |
        |=============================================================================|
        |    0      7934      G   /usr/lib/xorg/Xorg                             9MiB |
        +-----------------------------------------------------------------------------+
        
  2. Congratulations, stay and enjoy this moment a little bit before running into the next part.

dGPU passthrough guide

System & Environment setup

  1. Set up QEMU:

    • QEMU from Ubuntu official PPA should work, just sudo apt install qemu-kvm qemu-utils qemu-efi ovmf.

      • Please note that QEMU 2.10 or above require a higher version of OVMF (say if you use UEFI for your VM), otherwise will cause boot hang. Refer here for details about which version. Simplest solution is to use ovmf package from 18.04 ppa directly.
    • Here I use QEMU 2.11.2 with pulse audio patch from spheenik to provide better audio quality and resolve the crackling issue, and vcpupin patch from saveriomiroddi for better performance.

    • Follow instructions below to build the QEMU I use (only if you prefer):

      • # Clone saveriomiroddi's vcpupin version of QEMU
        git clone https://github.com/saveriomiroddi/qemu-pinning.git qemu
        cd qemu
        git checkout v2.11.2-pinning
        
        # Apply pulseaudio from spheenik's git, we're applying the v1 version.
        wget -O - https://gist.githubusercontent.com/spheenik/8140a4405f819c5cd2465a65c8bb6d09/raw/9735bcfaaaef45cf47e1b5d92c5006adf6ecd737/v1.patch | patch -p0
        
        # (Optional)
        # You might need to set your git email or name before commiting changes
        git commit -am "Apply pulse audio patch"
        
        # Install dependencies
        sudo apt install libjpeg-turbo8-dev libepoxy-dev libdrm-dev libgbm-dev libegl1-mesa-dev libboost-thread1.58-dev libboost-random1.58-dev libiscsi-dev libnfs-dev libfdt-dev libpixman-1-dev libssl-dev socat libsdl1.2-dev libspice-server-dev autoconf libtool xtightvncviewer tightvncserver x11vnc libsdl1.2-dev uuid-runtime uuid uml-utilities bridge-utils python-dev liblzma-dev libc6-dev libusb-1.0-0-dev checkinstall virt-viewer cpu-checker nettle-dev libaio-dev
        
        # Prepare to build
        mkdir build
        cd build
        
        # QEMU does not support python3
        ../configure --prefix=/usr \
            --audio-drv-list=alsa,pa,oss \
            --enable-kvm \
            --disable-xen \
            --enable-sdl \
            --enable-vnc \
            --enable-vnc-jpeg \
            --enable-opengl \
            --enable-libusb \
            --enable-vhost-net \
            --enable-spice \
            --target-list=x86_64-softmmu \
            --python=/usr/bin/python2
        
        make -j8
        
        # QEMU does not provide 'make uninstall'
        # Use checkinstall here so that you can remove it by 'dpkg -r'
        # Assign a version number start with numeric number is mandatory when using checkinstall
        sudo checkinstall
  2. Setup kernel module and parameters:

    • Add intel_iommu=on,igfx_off kvm.ignore_msrs=1 to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub, then sudo update-grub.

      • From here: Since some windows guest 3rd patry application / tools (like GPU-Z / Passmark9.0) will trigger MSR read / write directly, if it access the unhandled msr register, guest will trigger BSOD soon. So we added the kvm.ignore_msrs=1 into grub for workaround.
    • Add content below to /etc/initramfs-tools/modules (order matters!)

      • vfio
        vfio_iommu_type1
        vfio_pci
        vfio_virqfd
        vhost-net
        
      • sudo update-initramfs -u when finish.

    • Reboot.

    • lsmod for verification.

  3. (Optional) Setup hugepages

    • Reasons to use hugepages

    • Check cat /proc/cpuinfo see if it has the pse flag (for 2MB pages) or the pdpe1gb flag (for 1GB pages)

    • For pdpe1gb:

      • Add default_hugepagesz=1G hugepagesz=1G hugepages=8 transparent_hugepage=never to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub, this will assign a 8GB huge page.
    • For pse:

      • Add default_hugepagesz=2M hugepagesz=2M hugepages=4096 transparent_hugepage=never to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub, this does the same thing above.
    • sudo update-grub when finish.

    • Reboot.

    • ls /dev | grep hugepages for verification.

Prepare your script

  1. Get your Subsystem ID (SSID) and Subsystem Vendor ID (SVID):

    • Run optirun lspci -nnk -s 01:00.0, which will gives you an output like this:

      • 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1bb6] (rev a1)
        	Subsystem: Dell Device [1028:07b1]
        	Kernel driver in use: nvidia
        	Kernel modules: nvidiafb, nouveau, nvidia_384_drm, nvidia_384
        
    • Here, 1028 is the SVID and 07b1 is the SSID. We will use them later.

  2. Setup audio:

  3. Setup VM:

    • Note: Command here only serve as a reference, checkout QEMU documentation for more detail.

    • Note: I personally don't prefer libvirt as editing xml is annoying for me. Use libvirt if you like. virsh domxml-from-native qemu-argv xxx.sh can help you converting a QEMU startup script to libvirt XML. Refer here for more information.

    • Note: If you would like to put you GPU at some other address, refer here for details about ICH9 and GMCH (Graphics & Memory Controller Hub) defines. Layout of PCIe devices of your guest machines should follow these guidelines, as to prevent potential problem.

    • Note: The romfile option in the script below is not required if there is a stand alone GPU ROM chip bundled with your GPU (the case for MXM, not sure for soldered). However, if you decide to use the romfile option, please extract it yourself instead of download a copy from the Internet.

    • Create a disk image for your VM:

      • qemu-img create -f raw WindowsVM.img 75G
    • Install iptables and tunctl if you don't have it.

    • Create two script for tap networking:

      • tap_ifup (check files below in this gist)
      • tap_ifdown (check files below in this gist)
    • Use dpkg -L ovmf to locate your OVMF_VARS.fd file, copy that to the directory where you store your VM image, then rename it to WIN_VARS.fd(or other names you like).

    • Create a script for starting your VM:

      • Recall that our GPU have a SVID 1028, and a SSID 07b1, use these two value to set the corresponding vfio-pci options (see script below).

        • This will solve the SSID/SVID all zero problem inside the VM.
      • Don't forget to get a copy of VirtIO Driver

      • #!/bin/bash
        
        # Set audio output options
        export QEMU_AUDIO_DRV=pa
        export QEMU_PA_SERVER="<your-pulse-socket>"
        export QEMU_AUDIO_TIMER_PERIOD=500
        
        # Use command below to generate a MAC address
        # printf '52:54:BE:EF:%02X:%02X\n' $((RANDOM%256)) $((RANDOM%256))
        
        # Refer https://github.com/saveriomiroddi/qemu-pinning for how to set your cpu affinity properly
        qemu-system-x86_64 \
          -name "Windows10-QEMU" \
          -machine type=q35,accel=kvm \
          -global ICH9-LPC.disable_s3=1 \
          -global ICH9-LPC.disable_s4=1 \
          -enable-kvm \
          -cpu host,kvm=off,hv_vapic,hv_relaxed,hv_spinlocks=0x1fff,hv_time,hv_vendor_id=12alphanum \
          -smp 6,sockets=1,cores=3,threads=2 \
          -vcpu vcpunum=0,affinity=1 -vcpu vcpunum=1,affinity=5 \
          -vcpu vcpunum=2,affinity=2 -vcpu vcpunum=3,affinity=6 \
          -vcpu vcpunum=4,affinity=3 -vcpu vcpunum=5,affinity=7 \
          -m 8G \
          -mem-path /dev/hugepages \
          -mem-prealloc \
          -balloon none \
          -rtc clock=host,base=localtime \
          -device ich9-intel-hda -device hda-output \
          -device qxl,bus=pcie.0,addr=1c.4,id=video.2 \
          -vga none \
          -nographic \
          -serial none \
          -parallel none \
          -k en-us \
          -spice port=5901,addr=127.0.0.1,disable-ticketing \
          -usb \
          -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
          -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-pci-sub-device-id=0x07b1,x-pci-sub-vendor-id=0x1028,multifunction=on,romfile=MyGPU.rom \
          -drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE.fd \
          -drive if=pflash,format=raw,file=WIN_VARS.fd \
          -boot menu=on \
          -boot order=c \
          -drive id=disk0,if=virtio,cache=none,format=raw,file=WindowsVM.img \
          -drive file=windows10.iso,index=1,media=cdrom \
          -drive file=virtio-win-0.1.141.iso,index=2,media=cdrom \
          -netdev type=tap,id=net0,ifname=tap0,script=tap_ifup,downscript=tap_ifdown,vhost=on \
          -device virtio-net-pci,netdev=net0,addr=19.0,mac=<address your generate>
          -device pci-bridge,addr=12.0,chassis_nr=2,id=head.2 \
          -device usb-tablet
          
        # The -device usb-tablet will not be accurate regarding the pointer in some cases, another option is to use 
        # -device virtio-keyboard-pci,bus=head.2,addr=03.0,display=video.2 \
        # -device virtio-mouse-pci,bus=head.2,addr=04.0,display=video.2 \
      • For libvirt, refer here for an example of how to masquerade your Subsystem ID. (Credit to jscinoz)

Run your VM and configure guest side

  1. Binding your dGPU to vfio-pci driver:
    • echo "10de 1bb6" > "/sys/bus/pci/drivers/vfio-pci/new_id"
  2. Run the script to launch your VM
    • Install your Windows system through host side VNC (remote-viewer spice://127.0.0.1:5930).
      • -device qxl,bus=pcie.0,addr=1c.4,id=video.2 need to be comment out, change -vga none to -vga qxl so that QXL would become the first GPU and can see POST screen from spice client.
      • Change back once you have everything working.
    • IMPORTANT: Driver could be a cause for Code 43, please try both the driver your manufacture provided, and the driver from Nvidia website.
    • Add 192.168.99.0/24 to your Windows VM firewall exception:
      • In Control Panel\System and Security\Windows Defender Firewall, click Advance settings in the right panel, and Inbound Rules -> New rules.
      • Make sure you can ping to your VM from host.
      • Some details about setting up VirtIO driver not included here.
    • Enable remote desktop in Windows VM:
      • Right click This PC, click Remote settings in the right panel.
    • Verify that your GPU (in guest) have the correct hardware ID. Device manager -> double click your dGPU -> Detailtab -> Hardware Ids
      • For me, its PCI\VEN_10DE&DEV_1BB6&SUBSYS_07B11028. I'll get PCI\VEN_10DE&DEV_1BB6&SUBSYS_00000000 if I did't have it masqueraded.
      • In some cases, you will find your dGPU as a Video controller(VGA compatible) under Unknown Device before your install nvidia driver.
    • Install the official nvidia driver.
      • If everything goes smoothly, you will now be able to see your GPU within Performance tab in Task Manager.
  3. Post VM shut down operation:
    • Unbind your dGPU from vfio-pci driver, echo "0000:01:00.0" > "/sys/bus/pci/drivers/vfio-pci/0000:01:00.0/driver/unbind"
    • Power off your dGPU, echo "OFF" >> /proc/acpi/bbswitch
    • Run optirun nvidia-smi for verification.

RemoteFX configure and fine tuning

Configure RemoteFX

  1. Run gpedit.msc through Win+R.
  2. Locate yourself to Computer Configuration -> Administrative Templates -> Windows Components -> Remote Desktop Service -> Remote Desktop Session Host -> Remote Session Environment
    • Enable Use advanced RemoteFX graphics for RemoteApp
    • (Optional) Enable Configure image quality for RemoteFX adaptive Graphics, set it to High
    • Enable Enable RemoteFX encoding for RemoteFX clients designed for Windows Servier 2008 R2 SP1
    • Enable Configure compression for RemoteFX data, set it to Do not use an RDP compression algorithm
      • Connection compression will result extra latency for encode and decode, we don't want this.
  3. Locate yourself to Computer Configuration -> Administrative Templates -> Windows Components -> Remote Desktop Service -> Remote Desktop Session Host -> Remote Session Environment -> RemoteFX for Windows Server 2008 R2
    • Enable Configure RemoteFX
    • (Optional) Enable Optimize visual experience when using RemoteFX, set both option to Highest.

FreeRDP client configuration:

  • Make sure your have FreeRDP 2.0 (Do NOT use Remmina from Ubuntu Official PPA)
    • Compile one yourself or get a nightly build from here
  • Get your Windows VM IP address (or assign a static one), here we use 192.168.99.2 as an example.
  • xfreerdp /v:192.168.99.2:3389 /w:1600 /h:900 /bpp:32 +clipboard +fonts /gdi:hw /rfx /rfx-mode:video /sound:sys:pulse +menu-anims +window-drag
    • Refer here for more detail.

Lifting 30-ish fps restriction:

  1. Start Registry Editor.
  2. Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations
  3. On the Edit menu, click New, and then click DWORD(32-bit) Value.
  4. Type DWMFRAMEINTERVAL, and then press Enter.
  5. Right-click DWMFRAMEINTERVAL, click Modify.
  6. Click Decimal, type 15 in the Value data box, and then click OK. This sets the maximum frame rate to 60 frames per second (FPS).

Verify codec usage and fine tuning your frame rate:

  • Bring up your task manager, if a simple start menu pop-up animation (Windows 10) could consume you 40+ Mbps, then you are NOT using RemoteFX codec but just vanilla RDP. With a 1600x900 resolution, the start menu pop-up animation should consume a bandwidth less than 25 Mbps, while a 1600x900 Heaven benchmark consume less than 170 Mbps at peak.
  • Fire up a benchmark like Unigine Heaven in the VM, check if your dGPU can maintain a higher than 90~95% utility stably. If not, tune down your resolution and try again. You will find a sweet spot that suits your hardware.
  • For those don't concern much about image quality, try adding /gfx-h264:AVC444 option to your FreeRDP script. This will use RDP 8.1 with H.264 444 codec, which consume only 20~30-ish bandwidth even when runing full window Heaven benchmark. But artifacts this codec bring is more than noticeable.

For gaming:

  • 1600x900 or lower resolution RFX connection is recommended for most Core i7 laptop.
  • 1080p connection with game running at 1600x900 windowed mode have the same performance as above.

For other task:

  • Tasks that are more GPU compute intensive (which does its operation asynchronously from display update) will not be bottlenecked by CPU, thus you can choose a higher resolution like 1080p.

Steam in-home Streaming

For the limitations of RemoteFX, service like Steam in-home streaming or Geforce Experience is more recommended for gaming scenario.

Extra precautions should be taken for Steam in-home Streaming:

  • A Remote desktop connection that use dGPU inside the VM to render its display is still required, or the game will literally not running on the dGPU you just passed.
    • Not 100 percent about this. Maybe manually tell the game to use which GPU is possible?
    • One more thing, Nvidia control panel is not accessible within a RDP session. Nothing will pop-up no matter how hard you click it.
  • Make sure your dGPU is the ONLY display adapter enabled inside the VM.
  • Use this method to unlock the remote screen, note that current RDP session will be terminated once unlock success.
    • Pro or higher version of Windows is required.
    • Do not launch the script until the game appears in taskbar, otherwise it won't use your dGPU.

External display setup

External display require a BIOS setting that can rarely be seen on Optimus laptop.

  • For some Dell laptop (such as mine), There is a Display port direct output mode option in Video -> Switchable Graphics, enable it and it will assign all display port (mDP, HDMI, Thunder Bolt etc.) directly to the dGPU. Check if your BIOS offer some similar options.
  • However, you will lose your capability to extend your host machine display. As there is no display output port connect to the iGPU, e.g. your host.
  • While RemoteFX will compress the image in exchange for performance (which is not good if you required extreme image quality for professional use), such problem don't exist for external display setup, as it hook the dGPU directly.

Looking glass

  • If your machine can expose video output port to dGPU, then using Looking Glass is possible.
  • Moreover, if you have a Quadro card, you can load EDID directly from file in Nvidia Control Panel, and don't need to plug anything. Can even run without physical video output port expost to dGPU.
    • Though you still need to plug something for the first time setup otherwise Nvidia Control Panel won't show.

FAQ

How did you extract you vBIOS?

Well, except for laptop that use MXM graphics card, vBIOS of onboard graphics card is actually part of the system BIOS.

  • For the record, I did success without romfile option, but there is no guarantee for this approach.
  • For MXM graphics card, try using nvflash instead of GPU-Z. (In Windows) Disable your dGPU in device manager and run command nvflash -6 xxx.rom with privilege will extract your vBIOS as xxx.rom (This is the way I did). Try different version of nvflash if you fail.
  • For on board GPU:
    • Put the AFUDOS.EXE (or other BIOS backup tool depending on your BIOS) in a DOS-bootable USB device, then use it to extract your entire BIOS.
    • Then boot to windows and use PhoenixTool (or other similar tools) to extract modules contain in that BIOS.
      • Noted that those extracted modules will have weird name thus you can't be sure which one is for your onboard graphics card.
    • Finally use some vBIOS Tweaker (MaxwellBiosTweaker or Mobile Pascal Tweaker or other equivalence) to find out which module is your vBIOS.
      • Simply drag those module rom to the tweaker. Module roms that are not a vBIOS will be displayed as Unsupport Device, while vBIOS (typically around 50~300KB in size) will be successfully readed and show is information like device ID and vendor ID.
      • Manufactures tend to include several vBIOS for generic purpose. Be sure you find the correct vBIOS that have the same device ID as the one shown in device manager.
      • Disclaimer: I just know that you can use this method to extract the vBIOS of onboard graphics in the old days. However laptop BIOS may vary and I am not sure either the extraction process can go smoothly or the extracted and identified vBIOS rom can be used in QEMU without any problem.

Regarding AMD CPU/GPU?

Never own a laptop with AMD CPU/GPU myself, worth trying though.

What about GVT-g? Can I replicate a Optimus system inside a VM?

Recently GVT project successful expose guest screen with dmabuf, might be some hope?

Last time I try this, passing dGPU to a GVT-g VM is possible, but the dGPU will report Code 12 with "no enough resources" inside the VM. No idea why.

What about those bare-bone laptop?

Bare-bone laptop with desktop CPU already have their iGPU disabled in a way you cannot revert (as far as I know), and can only use their dGPU to render the display. Thus there will be no display if you pass it to your VM.

For those bare-bone laptops who have two dGPUs, passing one to your VM sounds possible? Not sure. Just take extra care if you have two identical dGPU. Check here for more detail.

Options other than RemoteFX?

Try nvidia gamestream with moonlight client, or Parsec. Or just pick whatever handful for you.

Known issue

For RemoteFX connection with xfreerdp:

  • Only windowed game can work, full screen will triger d3d11 0x087A0001 cannot set resolution blablabla problem. Media player does not affect by this.
    • As a solution, use borderless gaming or other equivalence.
    • Windowed client doesn't seems to have this problem.
  • Mouse will go wild due to relative mouse is unsupported in RDSH/RDVH connection.

Reference

XPS-15 9560 Getting Nvidia To Work on KDE Neon

Hexadecimal to Decimal Converter

FreeRDP-User-Manual

PCI passthrough via OVMF - Arch Wiki

CUDA installation guide

Frame rate is limited to 30 FPS in Windows 8 and Windows Server 2012 remote sessions

#!/bin/bash
# tap device name
TAP=tap0
# Network information
NETWORK=192.168.99.0
NETMASK=255.255.255.0
GATEWAY=192.168.99.1
DNSMASQPID=$(cat "/var/run/qemu-dnsmasq-$TAP.pid")
if [ ! "$DNSMASQPID" = "" ]; then
kill -s SIGTERM $DNSMASQPID && echo "DNSMASQ terminated"
fi
ip link set $TAP down
ip addr flush dev $TAP
iptables -t nat -D POSTROUTING -s $NETWORK/$NETMASK -j MASQUERADE
iptables -D INPUT -i $TAP -s $NETWORK/$NETMASK -d $NETWORK/$NETMASK -j ACCEPT
iptables -D INPUT -i $TAP -p tcp -m tcp --dport 67 -j ACCEPT
iptables -D INPUT -i $TAP -p udp -m udp --dport 67 -j ACCEPT
iptables -D INPUT -i $TAP -p tcp -m tcp --dport 53 -j ACCEPT
iptables -D INPUT -i $TAP -p udp -m udp --dport 53 -j ACCEPT
iptables -D FORWARD -i $TAP -o $TAP -j ACCEPT
iptables -D FORWARD -s $NETWORK/$NETMASK -i $TAP -j ACCEPT
iptables -D FORWARD -s $GATEWAY -i $TAP -j ACCEPT
iptables -D FORWARD -d $NETWORK/$NETMASK -o $TAP -m state --state RELATED,ESTABLISHED -j ACCEPT
echo 0 | dd of=/proc/sys/net/ipv4/ip_forward > /dev/null && echo "ip_forward disabled"
#!/bin/bash
# Set to the name of your tap device
TAP=tap0
# Network information
NETWORK=192.168.99.0
NETMASK=255.255.255.0
GATEWAY=192.168.99.1
DHCPRANGE=192.168.99.2,192.168.99.10
check_tap() {
if ip link show | grep $TAP > /dev/null; then
return
else
exit 1
fi
}
enable_ip_forward() {
echo 1 | dd of=/proc/sys/net/ipv4/ip_forward > /dev/null
}
start_dnsmasq(){
dnsmasq \
--strict-order \
--interface=$TAP \
--listen-address=$GATEWAY \
--bind-interfaces \
--dhcp-range=$DHCPRANGE \
--dhcp-no-override \
--pid-file=/var/run/qemu-dnsmasq-$TAP.pid
}
add_iptable_rules() {
iptables-restore -n <<EOF
*nat
-A POSTROUTING -s $NETWORK/$NETMASK -j MASQUERADE
COMMIT
*filter
-A INPUT -i $TAP -s $NETWORK/$NETMASK -d $NETWORK/$NETMASK -j ACCEPT
# Allow port 67 for DHCP, port 53 for dnsmasq
-A INPUT -i $TAP -p tcp -m tcp --dport 67 -j ACCEPT
-A INPUT -i $TAP -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i $TAP -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i $TAP -p udp -m udp --dport 53 -j ACCEPT
# Connect the external network
-A FORWARD -i $TAP -o $TAP -j ACCEPT
-A FORWARD -s $NETWORK/$NETMASK -i $TAP -j ACCEPT
-A FORWARD -s $GATEWAY -i $TAP -j ACCEPT
-A FORWARD -d $NETWORK/$NETMASK -o $TAP -m state --state RELATED,ESTABLISHED -j ACCEPT
COMMIT
EOF
}
start_tap() {
enable_ip_forward
check_tap
# Flush old config and set new config
ip addr flush dev $TAP
ip addr add $GATEWAY/$NETMASK dev $TAP
ip link set $TAP up
start_dnsmasq
add_iptable_rules
}
start_tap
@durranitech

This comment has been minimized.

Show comment
Hide comment
@durranitech

durranitech Apr 29, 2018

I am going to try this on bionic beaver and Metabox/Clevo P650-RS. Wish me luck

durranitech commented Apr 29, 2018

I am going to try this on bionic beaver and Metabox/Clevo P650-RS. Wish me luck

@marfillaster

This comment has been minimized.

Show comment
Hide comment
@marfillaster

marfillaster May 4, 2018

To pass the actual SUBSYSTEMID, add this snippet in virsh edit vmname

...
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-pci-sub-vendor-id=4136'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-pci-sub-device-id=2050'/>
  </qemu:commandline>
</domain>

The figures are the decimal values of the hardwareID (hex) in the device manager.

marfillaster commented May 4, 2018

To pass the actual SUBSYSTEMID, add this snippet in virsh edit vmname

...
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-pci-sub-vendor-id=4136'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-pci-sub-device-id=2050'/>
  </qemu:commandline>
</domain>

The figures are the decimal values of the hardwareID (hex) in the device manager.

@3kynox

This comment has been minimized.

Show comment
Hide comment
@3kynox

3kynox May 6, 2018

Just Fantastic to bring finally a working GPU Passthrough setup!

You just Rock!

I'm about to apply all of this on a Dell 9560 Laptop and will bring the results here to contribute to this gist.

Next step I will look later is to bring eGPU through thunderbolt on VM. (cf. https://www.reddit.com/r/VFIO/comments/5cy5uh/gpu_passthrough_over_thunderbolt_3_results_in/ )

Thanks again !

3kynox commented May 6, 2018

Just Fantastic to bring finally a working GPU Passthrough setup!

You just Rock!

I'm about to apply all of this on a Dell 9560 Laptop and will bring the results here to contribute to this gist.

Next step I will look later is to bring eGPU through thunderbolt on VM. (cf. https://www.reddit.com/r/VFIO/comments/5cy5uh/gpu_passthrough_over_thunderbolt_3_results_in/ )

Thanks again !

@gbrand-salesforce

This comment has been minimized.

Show comment
Hide comment
@gbrand-salesforce

gbrand-salesforce May 15, 2018

@durranitech Did you get it working on Bionic Beaver?

gbrand-salesforce commented May 15, 2018

@durranitech Did you get it working on Bionic Beaver?

@3kynox

This comment has been minimized.

Show comment
Hide comment
@3kynox

3kynox May 15, 2018

I'm maybe close to the end, when first connecting througth vnc, I currently have the message:

Guest has not initialized the display (yet)

What's that mean ?

Thanks

3kynox commented May 15, 2018

I'm maybe close to the end, when first connecting througth vnc, I currently have the message:

Guest has not initialized the display (yet)

What's that mean ?

Thanks

@brunoais

This comment has been minimized.

Show comment
Hide comment
@brunoais

brunoais May 19, 2018

@Misairu-G How do I install the firmware for step 1 of bumblebee?
The .bin files won't just execute.
The page doesn't tell anything that helps the process.

brunoais commented May 19, 2018

@Misairu-G How do I install the firmware for step 1 of bumblebee?
The .bin files won't just execute.
The page doesn't tell anything that helps the process.

@Babbie

This comment has been minimized.

Show comment
Hide comment
@Babbie

Babbie May 21, 2018

I'm having the exact same issue as @3kynox. So close, yet so far...
It might be relevant that I was unable to extract my vROM, so I had to grab one from the internet.

Babbie commented May 21, 2018

I'm having the exact same issue as @3kynox. So close, yet so far...
It might be relevant that I was unable to extract my vROM, so I had to grab one from the internet.

@samcoinc

This comment has been minimized.

Show comment
Hide comment
@samcoinc

samcoinc May 27, 2018

I have a lenovo w530 and am having problems with this. I feel like I am close but again far.
I don't know if this is the best forum - but I thought I would start here. I have a W530 lenovo with the nvidia optimus. (intel video + K2000m) I am trying to pass the k2000m to the windows 10 virtual machine. I feel like I am so close but so far.

This is stretch with the bios set to nvidia optimus and os aware.

First I followed the directions here for the most part
https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Isolating_the_GPU
and
https://www.youtube.com/watch?v=dsDUtzMkxFk&t=608s
and
https://gist.github.com/Misairu-G/616f7b2756c488148b7309addc940b28

among other sources.

This is my hardware

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 3rd Gen Core processor DRAM Controller [8086:0154] (rev 09)
IOMMU Group 10 00:1c.2 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 3 [8086:1e14] (rev c4)
IOMMU Group 11 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
IOMMU Group 12 00:1f.0 ISA bridge [0601]: Intel Corporation QM77 Express Chipset LPC Controller [8086:1e55] (rev 04)
IOMMU Group 12 00:1f.2 SATA controller [0106]: Intel Corporation 7 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e03] (rev 04)
IOMMU Group 12 00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller [8086:1e22] (rev 04)
IOMMU Group 13 02:00.0 System peripheral [0880]: Ricoh Co Ltd MMC/SD Host Controller [1180:e822] (rev 08)
IOMMU Group 13 02:00.3 FireWire (IEEE 1394) [0c00]: Ricoh Co Ltd R5C832 PCIe IEEE 1394 Controller [1180:e832] (rev 04)
IOMMU Group 14 03:00.0 Network controller [0280]: Intel Corporation Centrino Advanced-N 6205 [Taylor Peak] [8086:0085] (rev 34)
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107GLM [Quadro K2000M] [10de:0ffb] (rev a1)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09)
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
IOMMU Group 4 00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
IOMMU Group 5 00:19.0 Ethernet controller [0200]: Intel Corporation 82579LM Gigabit Network Connection [8086:1502] (rev 04)
IOMMU Group 6 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
IOMMU Group 7 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
IOMMU Group 8 00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
IOMMU Group 9 00:1c.1 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 [8086:1e12] (rev c4)

I have the video card using vfio-pci

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107GLM [Quadro K2000M] [10de:0ffb] (rev a1)
	Subsystem: Lenovo GK107GLM [Quadro K2000M] [17aa:21f5]
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau

the video card is passed through the wm

This is my xml

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win10-2</name>
  <uuid>c90236d7-a6e1-4b1d-b6ef-e665f2fa440a</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-2.8'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10-2_VARS.fd</nvram>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <vmport state='off'/>
  </features>
  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/win10-2.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/skunkworks/Downloads/virtio-win-0.1.149.iso'/>
      <target dev='sda' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/skunkworks/Downloads/Win10_1803_English_x64.iso'/>
      <target dev='sdb' bus='sata'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
      <model name='i82801b11-bridge'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
      <model name='pci-bridge'/>
      <target chassisNr='2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='3' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='4' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='5' port='0xa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='6' port='0xb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='7' port='0xc'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='8' port='0xd'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:ba:c5:04'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/>
    </sound>
    <video>
      <model type='virtio' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x1a2c'/>
        <product id='0x0e24'/>
      </source>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-drive'/>
    <qemu:arg value='if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='ivshmem-plain,memdev=ivshmem'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M'/>
  </qemu:commandline>
</domain>

This is dmesg when I start the vm

skunkworks@skunkworks:~$ dmesg
[ 6526.439955] virbr0: port 2(vnet0) entered blocking state
[ 6526.439958] virbr0: port 2(vnet0) entered disabled state
[ 6526.440076] device vnet0 entered promiscuous mode
[ 6526.476104] virbr0: port 2(vnet0) entered blocking state
[ 6526.476108] virbr0: port 2(vnet0) entered listening state
[ 6527.914102] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[ 6528.479783] virbr0: port 2(vnet0) entered learning state
[ 6528.960091] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[ 6530.495584] virbr0: port 2(vnet0) entered forwarding state
[ 6530.495586] virbr0: topology change detected, propagating
[ 6531.001778] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0x0000
[ 6531.196960] kvm: zapping shadow pages for mmio generation wraparound
[ 6531.197636] kvm: zapping shadow pages for mmio generation wraparound
[ 6545.159633] kvm [15134]: vcpu1, guest rIP: 0xfffff803fac962bc unhandled rdmsr: 0x611
[ 6545.810951] kvm [15134]: vcpu0, guest rIP: 0xfffff803fc7223f8 unhandled rdmsr: 0x606

I am currently stuck. I don't know what else to try. I feel like I am really close.

I will add - the video card shows up in the vm but it isn't working. (the driver install but it is stopped - code 43)

samcoinc commented May 27, 2018

I have a lenovo w530 and am having problems with this. I feel like I am close but again far.
I don't know if this is the best forum - but I thought I would start here. I have a W530 lenovo with the nvidia optimus. (intel video + K2000m) I am trying to pass the k2000m to the windows 10 virtual machine. I feel like I am so close but so far.

This is stretch with the bios set to nvidia optimus and os aware.

First I followed the directions here for the most part
https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Isolating_the_GPU
and
https://www.youtube.com/watch?v=dsDUtzMkxFk&t=608s
and
https://gist.github.com/Misairu-G/616f7b2756c488148b7309addc940b28

among other sources.

This is my hardware

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 3rd Gen Core processor DRAM Controller [8086:0154] (rev 09)
IOMMU Group 10 00:1c.2 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 3 [8086:1e14] (rev c4)
IOMMU Group 11 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
IOMMU Group 12 00:1f.0 ISA bridge [0601]: Intel Corporation QM77 Express Chipset LPC Controller [8086:1e55] (rev 04)
IOMMU Group 12 00:1f.2 SATA controller [0106]: Intel Corporation 7 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e03] (rev 04)
IOMMU Group 12 00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller [8086:1e22] (rev 04)
IOMMU Group 13 02:00.0 System peripheral [0880]: Ricoh Co Ltd MMC/SD Host Controller [1180:e822] (rev 08)
IOMMU Group 13 02:00.3 FireWire (IEEE 1394) [0c00]: Ricoh Co Ltd R5C832 PCIe IEEE 1394 Controller [1180:e832] (rev 04)
IOMMU Group 14 03:00.0 Network controller [0280]: Intel Corporation Centrino Advanced-N 6205 [Taylor Peak] [8086:0085] (rev 34)
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107GLM [Quadro K2000M] [10de:0ffb] (rev a1)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09)
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
IOMMU Group 4 00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
IOMMU Group 5 00:19.0 Ethernet controller [0200]: Intel Corporation 82579LM Gigabit Network Connection [8086:1502] (rev 04)
IOMMU Group 6 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
IOMMU Group 7 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
IOMMU Group 8 00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
IOMMU Group 9 00:1c.1 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 [8086:1e12] (rev c4)

I have the video card using vfio-pci

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107GLM [Quadro K2000M] [10de:0ffb] (rev a1)
	Subsystem: Lenovo GK107GLM [Quadro K2000M] [17aa:21f5]
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau

the video card is passed through the wm

This is my xml

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win10-2</name>
  <uuid>c90236d7-a6e1-4b1d-b6ef-e665f2fa440a</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-2.8'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10-2_VARS.fd</nvram>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <vmport state='off'/>
  </features>
  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/win10-2.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/skunkworks/Downloads/virtio-win-0.1.149.iso'/>
      <target dev='sda' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/skunkworks/Downloads/Win10_1803_English_x64.iso'/>
      <target dev='sdb' bus='sata'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
      <model name='i82801b11-bridge'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
      <model name='pci-bridge'/>
      <target chassisNr='2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='3' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='4' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='5' port='0xa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='6' port='0xb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='7' port='0xc'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='8' port='0xd'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:ba:c5:04'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/>
    </sound>
    <video>
      <model type='virtio' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x1a2c'/>
        <product id='0x0e24'/>
      </source>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-drive'/>
    <qemu:arg value='if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='ivshmem-plain,memdev=ivshmem'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M'/>
  </qemu:commandline>
</domain>

This is dmesg when I start the vm

skunkworks@skunkworks:~$ dmesg
[ 6526.439955] virbr0: port 2(vnet0) entered blocking state
[ 6526.439958] virbr0: port 2(vnet0) entered disabled state
[ 6526.440076] device vnet0 entered promiscuous mode
[ 6526.476104] virbr0: port 2(vnet0) entered blocking state
[ 6526.476108] virbr0: port 2(vnet0) entered listening state
[ 6527.914102] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[ 6528.479783] virbr0: port 2(vnet0) entered learning state
[ 6528.960091] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[ 6530.495584] virbr0: port 2(vnet0) entered forwarding state
[ 6530.495586] virbr0: topology change detected, propagating
[ 6531.001778] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0x0000
[ 6531.196960] kvm: zapping shadow pages for mmio generation wraparound
[ 6531.197636] kvm: zapping shadow pages for mmio generation wraparound
[ 6545.159633] kvm [15134]: vcpu1, guest rIP: 0xfffff803fac962bc unhandled rdmsr: 0x611
[ 6545.810951] kvm [15134]: vcpu0, guest rIP: 0xfffff803fc7223f8 unhandled rdmsr: 0x606

I am currently stuck. I don't know what else to try. I feel like I am really close.

I will add - the video card shows up in the vm but it isn't working. (the driver install but it is stopped - code 43)

@caiot5

This comment has been minimized.

Show comment
Hide comment
@caiot5

caiot5 Jun 2, 2018

Hello there,

Is extraction of vBIOS still required for dGPU Passthrough setup?
It's a "no go" for me.

caiot5 commented Jun 2, 2018

Hello there,

Is extraction of vBIOS still required for dGPU Passthrough setup?
It's a "no go" for me.

@ixevix

This comment has been minimized.

Show comment
Hide comment
@ixevix

ixevix Jun 3, 2018

Hey, how in the world do you extract the romfile? I cannot find any reference to this.

Edit: Seems impossible since I have the Geforce 940MX which has it's vBIOS in the computer BIOS...

ixevix commented Jun 3, 2018

Hey, how in the world do you extract the romfile? I cannot find any reference to this.

Edit: Seems impossible since I have the Geforce 940MX which has it's vBIOS in the computer BIOS...

@sh0rez

This comment has been minimized.

Show comment
Hide comment
@sh0rez

sh0rez Jun 16, 2018

What Laptop are you using? And are there any recommendations for current hardware? Or criteria to choose? The hardware requirements above are kinda general .. not helping in choosing a specific system.

sh0rez commented Jun 16, 2018

What Laptop are you using? And are there any recommendations for current hardware? Or criteria to choose? The hardware requirements above are kinda general .. not helping in choosing a specific system.

@bash64

This comment has been minimized.

Show comment
Hide comment
@bash64

bash64 Jun 17, 2018

I have been using gpu passthrough for years on an msi gt70-2pe.
It is a muxless laptop as far as i know.
not sure.
basically, if you use uefi (ovmf) it will not work as the windows side sees optimus and demands that you have the intel video shared with qemu also, thus the code 43 error.
if i use seabios then windows cannot see that i have optimus and the nvdiai driver will work.
i have an gtx 880m 8gb.
i boot windows 7 with seabios.
i upgraded a copy of windows 7 to windows 10, thus maintaining seabios and thus code 43 does not happen.

bash64 commented Jun 17, 2018

I have been using gpu passthrough for years on an msi gt70-2pe.
It is a muxless laptop as far as i know.
not sure.
basically, if you use uefi (ovmf) it will not work as the windows side sees optimus and demands that you have the intel video shared with qemu also, thus the code 43 error.
if i use seabios then windows cannot see that i have optimus and the nvdiai driver will work.
i have an gtx 880m 8gb.
i boot windows 7 with seabios.
i upgraded a copy of windows 7 to windows 10, thus maintaining seabios and thus code 43 does not happen.

@crisboarna

This comment has been minimized.

Show comment
Hide comment
@crisboarna

crisboarna Jun 18, 2018

@3kynox , @Babbie
I have the same issue as you do with the Guest has not initialized the display (yet) message.

Have you managed to get it working / solve this issue ?

crisboarna commented Jun 18, 2018

@3kynox , @Babbie
I have the same issue as you do with the Guest has not initialized the display (yet) message.

Have you managed to get it working / solve this issue ?

@metabubble

This comment has been minimized.

Show comment
Hide comment
@metabubble

metabubble Jun 20, 2018

Dell Precision 7510 with Intel Xeon E3-1535M and Optimus with Quadro M2000M.

Basically I have my card seen in the Windows Guest, driver installs but device stays with error 43. Tried messing with subsystem IDs and installing driver with modified inf to no success.
I guess this Workstation is not considered "High-End" then...

metabubble commented Jun 20, 2018

Dell Precision 7510 with Intel Xeon E3-1535M and Optimus with Quadro M2000M.

Basically I have my card seen in the Windows Guest, driver installs but device stays with error 43. Tried messing with subsystem IDs and installing driver with modified inf to no success.
I guess this Workstation is not considered "High-End" then...

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jun 20, 2018

@3kynox , @Babbie, @crisboarna, sorry, my mistake. -vga none will not show the POST screen.

Owner

Misairu-G commented Jun 20, 2018

@3kynox , @Babbie, @crisboarna, sorry, my mistake. -vga none will not show the POST screen.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jun 20, 2018

@metabubble

Precision 7510 should work fine. It is a MUXed machine. Though I never see 42 before.

Owner

Misairu-G commented Jun 20, 2018

@metabubble

Precision 7510 should work fine. It is a MUXed machine. Though I never see 42 before.

@metabubble

This comment has been minimized.

Show comment
Hide comment
@metabubble

metabubble Jun 20, 2018

@Misairu-G Sorry, Error 43. It was already late 6 hours ago.

metabubble commented Jun 20, 2018

@Misairu-G Sorry, Error 43. It was already late 6 hours ago.

@metabubble

This comment has been minimized.

Show comment
Hide comment
@metabubble

metabubble Jun 21, 2018

These are my configs and outputs, does anyone spot if I got something wrong? Getting Error 43 even on mid 2017 drivers. https://pastebin.com/spdP8Htk

Dell Precision 7510

Edit: Added vfio-pci.ids=10de:13b0 to my kernel options to bypass loading of nvidia module and disabled bumblebee. Still error 43.

metabubble commented Jun 21, 2018

These are my configs and outputs, does anyone spot if I got something wrong? Getting Error 43 even on mid 2017 drivers. https://pastebin.com/spdP8Htk

Dell Precision 7510

Edit: Added vfio-pci.ids=10de:13b0 to my kernel options to bypass loading of nvidia module and disabled bumblebee. Still error 43.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jun 21, 2018

@metabubble Configs seems OK to me. Does the GPU shows its subsystem id correctly inside the VM? If subsystem id are all correct, driver without inf modification should work. I think it might be other problem that lead to Code 43.

Also, would you like to change hv_vendor_id=whatvendor to hv_vendor_id=whatvendor12 and see if this works?

Edit: Maybe try use it with out specifying any romfile? Or use the Subsystem ID when you turn "Switchable graphics" off in BIOS?

Edit 2: I mean, record the Subsystem ID when you turn "Switchable graphics" off, then turn it back on and test the VM.

Edit 3: Sorry, my hands are tight. But since your laptop is 7510, you can try enable the "DisplayPortDirectOutput" in your BIOS, comment any QXL option, and plug a monitor to debug the VM directly. And, be sure to use drivers from Dell support site, do not use drivers from Nvidia (or Geforce experience).

Owner

Misairu-G commented Jun 21, 2018

@metabubble Configs seems OK to me. Does the GPU shows its subsystem id correctly inside the VM? If subsystem id are all correct, driver without inf modification should work. I think it might be other problem that lead to Code 43.

Also, would you like to change hv_vendor_id=whatvendor to hv_vendor_id=whatvendor12 and see if this works?

Edit: Maybe try use it with out specifying any romfile? Or use the Subsystem ID when you turn "Switchable graphics" off in BIOS?

Edit 2: I mean, record the Subsystem ID when you turn "Switchable graphics" off, then turn it back on and test the VM.

Edit 3: Sorry, my hands are tight. But since your laptop is 7510, you can try enable the "DisplayPortDirectOutput" in your BIOS, comment any QXL option, and plug a monitor to debug the VM directly. And, be sure to use drivers from Dell support site, do not use drivers from Nvidia (or Geforce experience).

@metabubble

This comment has been minimized.

Show comment
Hide comment
@metabubble

metabubble Jun 21, 2018

@Misairu-G
Hardware ID shows up in Windows as PCI\VEN_10DE&DEV_13B0&SUBSYS_06D91028 - driver installed fine.
and setting it to whatvendor12 does not change anything... I have seen different strings all over the web and tried all of them (probably a waste of time :D)

Edit: When I turn off switchable graphics, my primary grapics card will be the NVidia. I doubt I can use the Intel HD Graphics for my host system in that case, but I will try tomorrow when I have time to reboot. I already tried without romfile aswell...

metabubble commented Jun 21, 2018

@Misairu-G
Hardware ID shows up in Windows as PCI\VEN_10DE&DEV_13B0&SUBSYS_06D91028 - driver installed fine.
and setting it to whatvendor12 does not change anything... I have seen different strings all over the web and tried all of them (probably a waste of time :D)

Edit: When I turn off switchable graphics, my primary grapics card will be the NVidia. I doubt I can use the Intel HD Graphics for my host system in that case, but I will try tomorrow when I have time to reboot. I already tried without romfile aswell...

@metabubble

This comment has been minimized.

Show comment
Hide comment
@metabubble

metabubble Jun 22, 2018

Your idea about turning off switchable graphics did not help. But seeing that there are multiple ones in the .inf file, I tried them

%NVIDIA_DEV.13B0.06D9.1028% = Section120, PCI\VEN_10DE&DEV_13B0&SUBSYS_06D91028
%NVIDIA_DEV.13B0.16D9.1028% = Section122, PCI\VEN_10DE&DEV_13B0&SUBSYS_16D91028

Still the same error, so I tried all other ones for 10DE&DEV_13B0. 2 Hours later and still no success. Worth a try though...

Giving up, changing hardware.

metabubble commented Jun 22, 2018

Your idea about turning off switchable graphics did not help. But seeing that there are multiple ones in the .inf file, I tried them

%NVIDIA_DEV.13B0.06D9.1028% = Section120, PCI\VEN_10DE&DEV_13B0&SUBSYS_06D91028
%NVIDIA_DEV.13B0.16D9.1028% = Section122, PCI\VEN_10DE&DEV_13B0&SUBSYS_16D91028

Still the same error, so I tried all other ones for 10DE&DEV_13B0. 2 Hours later and still no success. Worth a try though...

Giving up, changing hardware.

@OEvgeny

This comment has been minimized.

Show comment
Hide comment
@OEvgeny

OEvgeny Jun 23, 2018

Maybe it'll save someone some time.

I tested HP dv6-7252-er which isn't a young guy. This model seem to have no IOMMU support on motherboard. There is no such option to toggle in bios (there is only virtualization options which is on in my setup).
Here is the result after dmesg with proper kernel boot options for IOMMU:

> dmesg | grep -i iommu
[    0.000000] DMAR: IOMMU enabled

Maybe someone get luck by modifying bios for this model to enable IOMMU. This is the only way I see. But I'm not going to try.

OEvgeny commented Jun 23, 2018

Maybe it'll save someone some time.

I tested HP dv6-7252-er which isn't a young guy. This model seem to have no IOMMU support on motherboard. There is no such option to toggle in bios (there is only virtualization options which is on in my setup).
Here is the result after dmesg with proper kernel boot options for IOMMU:

> dmesg | grep -i iommu
[    0.000000] DMAR: IOMMU enabled

Maybe someone get luck by modifying bios for this model to enable IOMMU. This is the only way I see. But I'm not going to try.

@Babbie

This comment has been minimized.

Show comment
Hide comment
@Babbie

Babbie Jun 23, 2018

I'm working on this again with the new version of the guide. When trying to run QEMU, I'm getting the following:

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-pci-sub-device-id=338,x-pci-sub-vendor-id=5464,multifunction=on,romfile=MyGPU.rom: vfio error: 0000:01:00.0: failed to open /dev/vfio/1: No such file or directory

Can you help me out?

Babbie commented Jun 23, 2018

I'm working on this again with the new version of the guide. When trying to run QEMU, I'm getting the following:

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-pci-sub-device-id=338,x-pci-sub-vendor-id=5464,multifunction=on,romfile=MyGPU.rom: vfio error: 0000:01:00.0: failed to open /dev/vfio/1: No such file or directory

Can you help me out?

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jun 23, 2018

@Babbie seems like vfio cannot bind the device, use lspci -nnk make sure no module is loaded for you GPU before bind it to vfio, also make sure vfio kernel module is loaded during boot, check it by lsmod | grep vfio

Update 01: @Babbie Sorry but the above two are the only situations I encountered, might be better search other place.
Update 02: @Babbie Wait a minute, why it is qemu that report the problem, have you bind your GPU to vfio before QEMU launch? QEMU won't bind the device automatically.
Update 03: Have you try OEM drivers?
Update 04: For my experience, ROM is unlikely to be a problem, as long as the system does not load the rom through ACPI call. Anyway, good luck.

Owner

Misairu-G commented Jun 23, 2018

@Babbie seems like vfio cannot bind the device, use lspci -nnk make sure no module is loaded for you GPU before bind it to vfio, also make sure vfio kernel module is loaded during boot, check it by lsmod | grep vfio

Update 01: @Babbie Sorry but the above two are the only situations I encountered, might be better search other place.
Update 02: @Babbie Wait a minute, why it is qemu that report the problem, have you bind your GPU to vfio before QEMU launch? QEMU won't bind the device automatically.
Update 03: Have you try OEM drivers?
Update 04: For my experience, ROM is unlikely to be a problem, as long as the system does not load the rom through ACPI call. Anyway, good luck.

@Babbie

This comment has been minimized.

Show comment
Hide comment
@Babbie

Babbie Jun 24, 2018

@Misairu-G lspci reports

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM206M [GeForce GTX 965M] [10de:1427] (rev ff)
	Kernel modules: nvidiafb, nouveau, nvidia_396, nvidia_396_drm

so it seems to me that no module is loaded. lsmod says

vfio_pci               45056  0
vfio_virqfd            16384  1 vfio_pci
irqbypass              16384  2 kvm,vfio_pci
vfio_iommu_type1       24576  0
vfio                   28672  2 vfio_iommu_type1,vfio_pci

It also does not list anything for nvidia or nouveau.

Update: I put in the wrong numbers for the binding! I entered the numbers for the second entry, not the first. Currently installing Windows, it seems to have picked up the GPU just fine! I'll update this comment if I have more issues or when I get it fully running.

Update 2: I'm getting a 43...

Update 3: Yes, I've manually installed Nvidia's drivers. I'm thinking it might be an issue with the vROM, so when I have time I'll try some other sources since manually dumping doesn't work.

Babbie commented Jun 24, 2018

@Misairu-G lspci reports

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM206M [GeForce GTX 965M] [10de:1427] (rev ff)
	Kernel modules: nvidiafb, nouveau, nvidia_396, nvidia_396_drm

so it seems to me that no module is loaded. lsmod says

vfio_pci               45056  0
vfio_virqfd            16384  1 vfio_pci
irqbypass              16384  2 kvm,vfio_pci
vfio_iommu_type1       24576  0
vfio                   28672  2 vfio_iommu_type1,vfio_pci

It also does not list anything for nvidia or nouveau.

Update: I put in the wrong numbers for the binding! I entered the numbers for the second entry, not the first. Currently installing Windows, it seems to have picked up the GPU just fine! I'll update this comment if I have more issues or when I get it fully running.

Update 2: I'm getting a 43...

Update 3: Yes, I've manually installed Nvidia's drivers. I'm thinking it might be an issue with the vROM, so when I have time I'll try some other sources since manually dumping doesn't work.

@MarcSN311

This comment has been minimized.

Show comment
Hide comment
@MarcSN311

MarcSN311 Jun 25, 2018

@samcoinc did you ever manage to get your W530 working? I tried but ended up with error 43...

MarcSN311 commented Jun 25, 2018

@samcoinc did you ever manage to get your W530 working? I tried but ended up with error 43...

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jun 25, 2018

@metabubble @Babbie I've reply your comment, but I prefer not to have too much redundant comment down this gist, so I'll reply by updating my previous comment. Normally I'll reply within 24 hours, but please inform me when you have your problem resolved or would like to look some other place.

Owner

Misairu-G commented Jun 25, 2018

@metabubble @Babbie I've reply your comment, but I prefer not to have too much redundant comment down this gist, so I'll reply by updating my previous comment. Normally I'll reply within 24 hours, but please inform me when you have your problem resolved or would like to look some other place.

@kenkrelous

This comment has been minimized.

Show comment
Hide comment
@kenkrelous

kenkrelous Jul 8, 2018

@Misairu-G
Hi
First, I want thank you for this Guide. I followed your instructions and I applied some modifications due to as a different OS (I am running Manjaro ).
I writing to you in goal to explain my situation. I stuck when I install the nvidia driver on the VM. The virtual Windows doesn’t recognize my card. In the device manager I could saw that my subsystem id is null PCI\VEN_10DE&DEV_1C8D&SUBSYS_00000000&REV_A1
I want to understand what is the problem, I hope that you can help me ! If you need more information (like file or screenshot) ask to me.

Thank for reading me, please note that I don’t understand everything about Gpu-passtrough.

PS: Sorry for my english I am french.

kenkrelous commented Jul 8, 2018

@Misairu-G
Hi
First, I want thank you for this Guide. I followed your instructions and I applied some modifications due to as a different OS (I am running Manjaro ).
I writing to you in goal to explain my situation. I stuck when I install the nvidia driver on the VM. The virtual Windows doesn’t recognize my card. In the device manager I could saw that my subsystem id is null PCI\VEN_10DE&DEV_1C8D&SUBSYS_00000000&REV_A1
I want to understand what is the problem, I hope that you can help me ! If you need more information (like file or screenshot) ask to me.

Thank for reading me, please note that I don’t understand everything about Gpu-passtrough.

PS: Sorry for my english I am french.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 8, 2018

@kenkrelous Can you paste your VM script somewhere? So that I can check it for you. And the output of lspci -nnk of your graphics card.

Update: Looks like domain type problem, in that case qemu commandline options won't work, change <domain type='kvm'> to <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> and subsystem ID should be correct. However, as 01:00.0 appears to be a 3D controller, I'm afraid this guide will not works.

Owner

Misairu-G commented Jul 8, 2018

@kenkrelous Can you paste your VM script somewhere? So that I can check it for you. And the output of lspci -nnk of your graphics card.

Update: Looks like domain type problem, in that case qemu commandline options won't work, change <domain type='kvm'> to <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> and subsystem ID should be correct. However, as 01:00.0 appears to be a 3D controller, I'm afraid this guide will not works.

@kenkrelous

This comment has been minimized.

Show comment
Hide comment
@kenkrelous

kenkrelous Jul 8, 2018

Hi, I am using libvirt so I have put the xml configuration into a hastebin file https://hastebin.com/sagupopefi.xml.
And this the output of lspci -nnk https://hastebin.com/gasocinire.sql

Thanks

Update: Thanks but the it's doesn't work. Thanks

kenkrelous commented Jul 8, 2018

Hi, I am using libvirt so I have put the xml configuration into a hastebin file https://hastebin.com/sagupopefi.xml.
And this the output of lspci -nnk https://hastebin.com/gasocinire.sql

Thanks

Update: Thanks but the it's doesn't work. Thanks

@s2156945

This comment has been minimized.

Show comment
Hide comment
@s2156945

s2156945 Jul 13, 2018

@Misairu-G - thanks for this writeup and your continuing help.

I was stuck with Windows "Code 31" (not "Code 43") this seemed to be because I had not set the SSID / SVID in the vfio-pci parameters. Googling VFIO code 31 doesn't find anything, could you stick it in your doc to help those yet to get stuck with their googling?

Question 1:
how do I figure out if I have a MUXed or Muxless laptop?
It's a Metabox (Clevo) P950ER.
I suspect it is a fourth type not in your diagram: muxless with the intel card wired to the LCD and the nvidia wired to the ports. Reasons:

  • in windows native the nvidia control panel "fills out" (from 2 entries in the side panel to ~10) when an external monitor is connected; The "PhysX" tab shows the external monitors connected to the nvidia card
  • In linux native I get two "VGA compatible controller"s:
    $ lspci | grep -i vga
    00:02.0 VGA compatible controller: Intel Corporation Device 3e9b
    01:00.0 VGA compatible controller: NVIDIA Corporation GP104M [GeForce GTX 1070 Mobile] (rev a1)

Question 2:
Now I'm stuck with "Error 43" which I suspect is caused by the windows driver trying to get the ROM from ACPI (I guess I can try and confirm by turning on some ACPI debugging?)
Is the ROM being loaded by the nouveau driver in native linux, i.e. is the ROM ACPI voodoo known already to the open source community?
How important is the ROM to the VM? i.e. the ROM seems to be mostly card-specific, in the VM we only really have the nvidia chip - would a way forward be to set up the VM to look like we're passing through a vanilla Desktop card with the same chip?
Then the windows nvidia drivers will look for the ROM in the vanilla place, and we give it the vanilla ROM, and it's happy?

cheers,
Woody

s2156945 commented Jul 13, 2018

@Misairu-G - thanks for this writeup and your continuing help.

I was stuck with Windows "Code 31" (not "Code 43") this seemed to be because I had not set the SSID / SVID in the vfio-pci parameters. Googling VFIO code 31 doesn't find anything, could you stick it in your doc to help those yet to get stuck with their googling?

Question 1:
how do I figure out if I have a MUXed or Muxless laptop?
It's a Metabox (Clevo) P950ER.
I suspect it is a fourth type not in your diagram: muxless with the intel card wired to the LCD and the nvidia wired to the ports. Reasons:

  • in windows native the nvidia control panel "fills out" (from 2 entries in the side panel to ~10) when an external monitor is connected; The "PhysX" tab shows the external monitors connected to the nvidia card
  • In linux native I get two "VGA compatible controller"s:
    $ lspci | grep -i vga
    00:02.0 VGA compatible controller: Intel Corporation Device 3e9b
    01:00.0 VGA compatible controller: NVIDIA Corporation GP104M [GeForce GTX 1070 Mobile] (rev a1)

Question 2:
Now I'm stuck with "Error 43" which I suspect is caused by the windows driver trying to get the ROM from ACPI (I guess I can try and confirm by turning on some ACPI debugging?)
Is the ROM being loaded by the nouveau driver in native linux, i.e. is the ROM ACPI voodoo known already to the open source community?
How important is the ROM to the VM? i.e. the ROM seems to be mostly card-specific, in the VM we only really have the nvidia chip - would a way forward be to set up the VM to look like we're passing through a vanilla Desktop card with the same chip?
Then the windows nvidia drivers will look for the ROM in the vanilla place, and we give it the vanilla ROM, and it's happy?

cheers,
Woody

@Pilo11

This comment has been minimized.

Show comment
Hide comment
@Pilo11

Pilo11 Jul 14, 2018

@s2156945

I've got the exact same problem as you (Clevo N957TP6). I used Arch Linux as native OS and created vm with virtmanager and got Code 43. I am very sure it has something to do with the NVidia driver for Windows.
Why am I so sure?

Because I created a qemu VM with the same usual settings as hidden kvm, fake hyperv id et cetera...
But I installed an Arch Linux guest. And this guest was able to use the Nvidia card (with ROM. Without it did not work). I ran Unigine Heaven to test it on Linux (I did not use nouveau, I use original nvidia driver for Linux). Furthermore I could plug in an external monitor (DP) and saw the Linux guest there.

I am too silly to understand these ROM UEFI ACPI things. But now I am thinking it is most likely a Windows nvidia driver problem... Some people thing that the Windows guest needs the complete Optimus infrastructure (iGPU + dGPU) to work properly. But no one knows what this nvidia driver is really checking... I'm close to give up...

This is also interesting: https://github.com/jscinoz/optimus-vfio-docs/blob/master/README.md

He also describes a working Linux guest Passthrough. But that Windows passthrough needs extra stuff to be done.

"What doesn't work (yet)
Windows guest
Will need custom ACPI table to get VBIOS, as detailed above"

"Long term we need to build a custom ACPI table (provided to qemu with the -acpitable option) that has _ROM implemented at the correct path. The _ROM implementationm would need to seek over a hard-coded buffer stored elsewhere and return the VBIOS in 4kb chunks as expected by nvidia driver"

Another good up-to-date thread for our problem and the muxless guys: https://www.reddit.com/r/VFIO/comments/8gv60l/current_state_of_optimus_muxless_laptop_gpu/

Pilo11 commented Jul 14, 2018

@s2156945

I've got the exact same problem as you (Clevo N957TP6). I used Arch Linux as native OS and created vm with virtmanager and got Code 43. I am very sure it has something to do with the NVidia driver for Windows.
Why am I so sure?

Because I created a qemu VM with the same usual settings as hidden kvm, fake hyperv id et cetera...
But I installed an Arch Linux guest. And this guest was able to use the Nvidia card (with ROM. Without it did not work). I ran Unigine Heaven to test it on Linux (I did not use nouveau, I use original nvidia driver for Linux). Furthermore I could plug in an external monitor (DP) and saw the Linux guest there.

I am too silly to understand these ROM UEFI ACPI things. But now I am thinking it is most likely a Windows nvidia driver problem... Some people thing that the Windows guest needs the complete Optimus infrastructure (iGPU + dGPU) to work properly. But no one knows what this nvidia driver is really checking... I'm close to give up...

This is also interesting: https://github.com/jscinoz/optimus-vfio-docs/blob/master/README.md

He also describes a working Linux guest Passthrough. But that Windows passthrough needs extra stuff to be done.

"What doesn't work (yet)
Windows guest
Will need custom ACPI table to get VBIOS, as detailed above"

"Long term we need to build a custom ACPI table (provided to qemu with the -acpitable option) that has _ROM implemented at the correct path. The _ROM implementationm would need to seek over a hard-coded buffer stored elsewhere and return the VBIOS in 4kb chunks as expected by nvidia driver"

Another good up-to-date thread for our problem and the muxless guys: https://www.reddit.com/r/VFIO/comments/8gv60l/current_state_of_optimus_muxless_laptop_gpu/

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 14, 2018

@s2156945 Hi Woody,

  1. According to your description, Code 43 seems can be resolved by plugging something to the HDMI port? To my view, whether Nvidia wired to ports is not important, some machine can control that from the BIOS, and I don't think this will cause the Code 43 error.

  2. Sorry but I'm just some enthusiast, don't know how to do the ACPI voodoo (at least not yet). And the idea to fool VM that we're passing a vanilla Desktop card sounds interesting, and yes you can do that by adding x-pci-vendor-id and x-pci-device-id options for vfio-pci

Cheers

Owner

Misairu-G commented Jul 14, 2018

@s2156945 Hi Woody,

  1. According to your description, Code 43 seems can be resolved by plugging something to the HDMI port? To my view, whether Nvidia wired to ports is not important, some machine can control that from the BIOS, and I don't think this will cause the Code 43 error.

  2. Sorry but I'm just some enthusiast, don't know how to do the ACPI voodoo (at least not yet). And the idea to fool VM that we're passing a vanilla Desktop card sounds interesting, and yes you can do that by adding x-pci-vendor-id and x-pci-device-id options for vfio-pci

Cheers

@s2156945

This comment has been minimized.

Show comment
Hide comment
@s2156945

s2156945 Jul 15, 2018

Thanks guys for the answers!

@Pilo11 - I have skipped some of the things I have tried. I have been booting the Linux Mint 18.3 ISO with the same settings as windows which I try everytime I think I am making progress. It reports that nouveau can’t work the card either. (unknown something 0xffffffff)

I’ve been struggling with other things:
-Ubuntu 18.04 desktop ISO installer doesn’t work with a keyboard with my laptop for some reason. Instead I have used the server install.

  • nouveau + bbswitch modules hang the laptop fairly regularly; I’m using the acpi-call stuff instead but I’m not sure how to confirm that the card is powered
  • it’s been a while since I mucked around at such a low level, I had slackware and redhat and debian back in the day but have been bludging on Ubuntu since 5.04. I worked out dkms but shamefully failed when I tried to compile my own kernel for the first time in probably 15 years.

Thanks for the links, I’d seen jcinoz ( In Australia like me?) but not the other one, skimmed it - lots to try.

@Misairu-G no sorry I haven’t had any success getting the card running in a VM, by “windows native” and “linux native” I mean on bare metal.

Next things for me to try:

  • get the linux nvidia driver running on baremetal first and try to boot that under kvm
  • examine the system bios and find where the video bios is in it and the acpi calls to get it and build up an acpi table
  • spoof a more vanilla card to the VM and see if loading my cards bios a more traditional way works
  • get GPU passthrough working on another computer (I have an older desktop with Vt-d to try plus a Metabox/Clevo P650 something with a 970m)

Also fully document what I have tried somewhere.

s2156945 commented Jul 15, 2018

Thanks guys for the answers!

@Pilo11 - I have skipped some of the things I have tried. I have been booting the Linux Mint 18.3 ISO with the same settings as windows which I try everytime I think I am making progress. It reports that nouveau can’t work the card either. (unknown something 0xffffffff)

I’ve been struggling with other things:
-Ubuntu 18.04 desktop ISO installer doesn’t work with a keyboard with my laptop for some reason. Instead I have used the server install.

  • nouveau + bbswitch modules hang the laptop fairly regularly; I’m using the acpi-call stuff instead but I’m not sure how to confirm that the card is powered
  • it’s been a while since I mucked around at such a low level, I had slackware and redhat and debian back in the day but have been bludging on Ubuntu since 5.04. I worked out dkms but shamefully failed when I tried to compile my own kernel for the first time in probably 15 years.

Thanks for the links, I’d seen jcinoz ( In Australia like me?) but not the other one, skimmed it - lots to try.

@Misairu-G no sorry I haven’t had any success getting the card running in a VM, by “windows native” and “linux native” I mean on bare metal.

Next things for me to try:

  • get the linux nvidia driver running on baremetal first and try to boot that under kvm
  • examine the system bios and find where the video bios is in it and the acpi calls to get it and build up an acpi table
  • spoof a more vanilla card to the VM and see if loading my cards bios a more traditional way works
  • get GPU passthrough working on another computer (I have an older desktop with Vt-d to try plus a Metabox/Clevo P650 something with a 970m)

Also fully document what I have tried somewhere.

@softwarecreations

This comment has been minimized.

Show comment
Hide comment
@softwarecreations

softwarecreations Jul 21, 2018

Thanks @Misairu-G

I have Asus N53SV which has Geforce 540M Intel and Optimus and
i7-2670QM 45W. It has VT-x, but no VT-d.
I might upgrade to
i7-2860QM 45W which is slightly faster, and has VT-d. (same socket also)
Only problem is I don't know if my BIOS would support the faster CPU...

Seems silly to spend money on this old laptop.

softwarecreations commented Jul 21, 2018

Thanks @Misairu-G

I have Asus N53SV which has Geforce 540M Intel and Optimus and
i7-2670QM 45W. It has VT-x, but no VT-d.
I might upgrade to
i7-2860QM 45W which is slightly faster, and has VT-d. (same socket also)
Only problem is I don't know if my BIOS would support the faster CPU...

Seems silly to spend money on this old laptop.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 22, 2018

@softwarecreations VT-x and VT-d need to be supported by both CPU and motherboard, does your chipset support VT-d as well? Also check if manufacturer enabled those two in BIOS.

Owner

Misairu-G commented Jul 22, 2018

@softwarecreations VT-x and VT-d need to be supported by both CPU and motherboard, does your chipset support VT-d as well? Also check if manufacturer enabled those two in BIOS.

@KingTheGuy

This comment has been minimized.

Show comment
Hide comment
@KingTheGuy

KingTheGuy Jul 24, 2018

need some help can't find subsystem id

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106M [GeForce GTX 1060 Mobile] [10de:1c20] (rev a1)
Kernel driver in use: nvidia
Kernel modules: nouveau, nvidia_drm, nvidia

KingTheGuy commented Jul 24, 2018

need some help can't find subsystem id

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106M [GeForce GTX 1060 Mobile] [10de:1c20] (rev a1)
Kernel driver in use: nvidia
Kernel modules: nouveau, nvidia_drm, nvidia

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 25, 2018

@KingTheGuy That's your output of lspci -nnk? That would be ... so weird. Alternatively, if you have a dual boot system, you can check that ID in Windows, I don't know other ways to check that ID in Linux, sorry.

Owner

Misairu-G commented Jul 25, 2018

@KingTheGuy That's your output of lspci -nnk? That would be ... so weird. Alternatively, if you have a dual boot system, you can check that ID in Windows, I don't know other ways to check that ID in Linux, sorry.

@KingTheGuy

This comment has been minimized.

Show comment
Hide comment
@KingTheGuy

KingTheGuy Jul 25, 2018

@Misairu-G
output for lspci -nnk

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106M [GeForce GTX 1060 Mobile] [10de:1c20] (rev a1)
	Subsystem: Razer USA Ltd. GP106M [GeForce GTX 1060 Mobile] [1a58:6755]
	Kernel driver in use: nvidia
	Kernel modules: nouveau, nvidia_drm, nvidia

output for lspci -n -m -v

....
Device:	01:00.0
Class:	0300
Vendor:	10de
Device:	1c20
SVendor:	1a58
SDevice:	6755
Rev:	a1
...

so from what i can tell it is [1a58:6755] ..maybe. but it's tied to pretty much everything else and when i plug it into the xml it freezes up my system.

any other way i can get the hardware ids to show up correctly in the vm?

KingTheGuy commented Jul 25, 2018

@Misairu-G
output for lspci -nnk

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106M [GeForce GTX 1060 Mobile] [10de:1c20] (rev a1)
	Subsystem: Razer USA Ltd. GP106M [GeForce GTX 1060 Mobile] [1a58:6755]
	Kernel driver in use: nvidia
	Kernel modules: nouveau, nvidia_drm, nvidia

output for lspci -n -m -v

....
Device:	01:00.0
Class:	0300
Vendor:	10de
Device:	1c20
SVendor:	1a58
SDevice:	6755
Rev:	a1
...

so from what i can tell it is [1a58:6755] ..maybe. but it's tied to pretty much everything else and when i plug it into the xml it freezes up my system.

any other way i can get the hardware ids to show up correctly in the vm?

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 25, 2018

@KingTheGuy
"it's tied to pretty much everything else" that's just how subsystem ID works, no need to worry about that.

Can you paste your xml somewhere else? And what do you mean by "freeze", what kind of behavior?

Owner

Misairu-G commented Jul 25, 2018

@KingTheGuy
"it's tied to pretty much everything else" that's just how subsystem ID works, no need to worry about that.

Can you paste your xml somewhere else? And what do you mean by "freeze", what kind of behavior?

@KingTheGuy

This comment has been minimized.

Show comment
Hide comment
@KingTheGuy

KingTheGuy Jul 25, 2018

@Misairu-G
here is my .xml file https://gist.github.com/KingTheGuy/d8875be775bdda740b1be162d2eb404d

just want to add that, the svender & sdevice ids disappear.. when i reboot the laptop, the ids may or may not be there, once i put my laptop to sleep and back, the ids appear again.

now the "freezing" it's happened 3 times now, so currently i can't reproduce this and test it.. as i get an error https://gist.github.com/KingTheGuy/64e19c26c80040b4ec736579e2c716d3 when i put the ssid into the xml.

KingTheGuy commented Jul 25, 2018

@Misairu-G
here is my .xml file https://gist.github.com/KingTheGuy/d8875be775bdda740b1be162d2eb404d

just want to add that, the svender & sdevice ids disappear.. when i reboot the laptop, the ids may or may not be there, once i put my laptop to sleep and back, the ids appear again.

now the "freezing" it's happened 3 times now, so currently i can't reproduce this and test it.. as i get an error https://gist.github.com/KingTheGuy/64e19c26c80040b4ec736579e2c716d3 when i put the ssid into the xml.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 26, 2018

@KingTheGuy Subsystem ID cannot be used directly like that. In the output of lspci, it is a hex number, you need to either put 0x before the number to denote it is in hex form, or convert it to decimal number. In you case, it would be 0x1a58 and 0x6755 or 6744 and 26453.

And about the disappear of svender & sdevice ids, I'm not sure if you use bumblebee, or if you set vfio-pci to bind the GPU on boot, but for a correct config, no kernel module should be attached to the GPU before you bind vfio-pci and launch the VM. Meanwhile, svender & sdevice ids would only appear when you have a kernel module loaded for that device. In other words, Kernel driver in use: nvidia should not appear, and nvidia kernel module should be blacklisted.

At the end, if you are running wayland based desktop manager (current version of gdm use wayland by default), blacklist settings in /etc/modprobe.d/ folder might not work. The workaround is either specify you dm to use X11, or blacklist the module in grub setting.

Owner

Misairu-G commented Jul 26, 2018

@KingTheGuy Subsystem ID cannot be used directly like that. In the output of lspci, it is a hex number, you need to either put 0x before the number to denote it is in hex form, or convert it to decimal number. In you case, it would be 0x1a58 and 0x6755 or 6744 and 26453.

And about the disappear of svender & sdevice ids, I'm not sure if you use bumblebee, or if you set vfio-pci to bind the GPU on boot, but for a correct config, no kernel module should be attached to the GPU before you bind vfio-pci and launch the VM. Meanwhile, svender & sdevice ids would only appear when you have a kernel module loaded for that device. In other words, Kernel driver in use: nvidia should not appear, and nvidia kernel module should be blacklisted.

At the end, if you are running wayland based desktop manager (current version of gdm use wayland by default), blacklist settings in /etc/modprobe.d/ folder might not work. The workaround is either specify you dm to use X11, or blacklist the module in grub setting.

@KingTheGuy

This comment has been minimized.

Show comment
Hide comment
@KingTheGuy

KingTheGuy Jul 30, 2018

@Misairu-G
thanks for the help.. that error 43 tho, it's killing me. from my understanding i need the GPU's BIOS and pass it over, now thats where I'm lost. i've searched around for one but with no luck.. not sure how i would about getting it myself. i know that there is some program that is supposed to get it whilst you update the BIOS but i can't find that. any suggestions or links? ..would be of great help. oh and that last paragraph on your last comment really helped(tho i only just read it, but useful info).

KingTheGuy commented Jul 30, 2018

@Misairu-G
thanks for the help.. that error 43 tho, it's killing me. from my understanding i need the GPU's BIOS and pass it over, now thats where I'm lost. i've searched around for one but with no luck.. not sure how i would about getting it myself. i know that there is some program that is supposed to get it whilst you update the BIOS but i can't find that. any suggestions or links? ..would be of great help. oh and that last paragraph on your last comment really helped(tho i only just read it, but useful info).

@Al2Me6

This comment has been minimized.

Show comment
Hide comment
@Al2Me6

Al2Me6 Jul 30, 2018

@Misairu-G
This might be a bit far-fetched but do you think LookingGlass would help with a setup like this instead of remote desktop?

Al2Me6 commented Jul 30, 2018

@Misairu-G
This might be a bit far-fetched but do you think LookingGlass would help with a setup like this instead of remote desktop?

@KingTheGuy

This comment has been minimized.

Show comment
Hide comment
@KingTheGuy

KingTheGuy Jul 30, 2018

@Misairu-G
OK, so apparently my GPU does not support UEFI.. meaning I'm out. would i be correct to assume this? cuz other wise if i put the ssids into a vm with firmware set to bios it gives me an error. x-pci-sub-vendor-id ... not found

KingTheGuy commented Jul 30, 2018

@Misairu-G
OK, so apparently my GPU does not support UEFI.. meaning I'm out. would i be correct to assume this? cuz other wise if i put the ssids into a vm with firmware set to bios it gives me an error. x-pci-sub-vendor-id ... not found

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 30, 2018

@Al2Me6
The problem is LookingGlass is not fully headless, you still need to have something plugged in dGPU video output (dummy HDMI etc.), but AFAIK only a few laptop can expose video output port directly to dGPU, thus impossible to run LookingGlass.

Owner

Misairu-G commented Jul 30, 2018

@Al2Me6
The problem is LookingGlass is not fully headless, you still need to have something plugged in dGPU video output (dummy HDMI etc.), but AFAIK only a few laptop can expose video output port directly to dGPU, thus impossible to run LookingGlass.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 31, 2018

@KingTheGuy
What version of QEMU are you using? x-pci-sub-vendor-id option might not even exists for older version of QEMU (2.6 or before? I don't quite remember).

Owner

Misairu-G commented Jul 31, 2018

@KingTheGuy
What version of QEMU are you using? x-pci-sub-vendor-id option might not even exists for older version of QEMU (2.6 or before? I don't quite remember).

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 31, 2018

@KingTheGuy
Then it would be so strange to report not found. Also, I won't be of much help if you cannot provide error logs, but I respect you decision of quitting this (since your vBIOS does not support UEFI).

Owner

Misairu-G commented Jul 31, 2018

@KingTheGuy
Then it would be so strange to report not found. Also, I won't be of much help if you cannot provide error logs, but I respect you decision of quitting this (since your vBIOS does not support UEFI).

@KingTheGuy

This comment has been minimized.

Show comment
Hide comment
@KingTheGuy

KingTheGuy Jul 31, 2018

@Misairu-G
sorry didn't link an error log cuz the error log made no sense at all.. the problem was xml file order, if i had my peripherals before the GPU it would cause an error when i add the ssids.

Ok so im lost.. do i need my GPU to support uefi or can i do it without it.

im stuck at error code 43. here is my xml file https://gist.github.com/KingTheGuy/0916dc88d3f0dc4e0f35dcdf8492e3d6
maybe ive got something wrong there.

KingTheGuy commented Jul 31, 2018

@Misairu-G
sorry didn't link an error log cuz the error log made no sense at all.. the problem was xml file order, if i had my peripherals before the GPU it would cause an error when i add the ssids.

Ok so im lost.. do i need my GPU to support uefi or can i do it without it.

im stuck at error code 43. here is my xml file https://gist.github.com/KingTheGuy/0916dc88d3f0dc4e0f35dcdf8492e3d6
maybe ive got something wrong there.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Jul 31, 2018

@KingTheGuy
I think you do need a vBIOS that support UEFI, at least that's what I always been told. The other thing that does not cover in this guide are those parameters that can hide VM state in libvirt configure, which be a reason that cause Code 43, you can find it at here. I'll consider add this link to the guide later.

Owner

Misairu-G commented Jul 31, 2018

@KingTheGuy
I think you do need a vBIOS that support UEFI, at least that's what I always been told. The other thing that does not cover in this guide are those parameters that can hide VM state in libvirt configure, which be a reason that cause Code 43, you can find it at here. I'll consider add this link to the guide later.

@KingTheGuy

This comment has been minimized.

Show comment
Hide comment
@KingTheGuy

KingTheGuy Jul 31, 2018

@Misairu-G
quick question.. lets say i have an hdmi cable connected, should i get an output. even if i dont give the ssids or rom.

been searching around and seems like uefi has to be supported in order for it to work.
ive found nothing of use that can help with the error.. for others tho, if using a gtx 1000 "mobile" series they don't have uefi support. ill contact nvidia and razer for a uefi vbios... see how that goes.

KingTheGuy commented Jul 31, 2018

@Misairu-G
quick question.. lets say i have an hdmi cable connected, should i get an output. even if i dont give the ssids or rom.

been searching around and seems like uefi has to be supported in order for it to work.
ive found nothing of use that can help with the error.. for others tho, if using a gtx 1000 "mobile" series they don't have uefi support. ill contact nvidia and razer for a uefi vbios... see how that goes.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Aug 1, 2018

@KingTheGuy
"lets say i have an hdmi cable connected, should i get an output. even if i dont give the ssids or rom", that would largely depends on your laptop. Most optimus laptop have video output wired to iGPU, in that case, no video output for the VM. Some optimus laptop can choose to wire to dGPU, but I would say quite rare.

"using a gtx 1000 "mobile" series they don't have uefi support" I think that's not true. Laptop shipped with G-sync (which have dGPU connected to display and iGPU available) have to support UEFI in their vBIOS, otherwise the laptop won't boot. And several mobile GTX 1070 vBIOS I have also indicate that they support UEFI (though those vBIOS are for MXM cards).

The other thing is whether to read the rom file you gave depends on the driver. So for your graphics card, if nvidia say that it should read its vBIOS from ACPI call (which means it read its vBIOS as part of the BIOS), then the rom file you have is pretty much useless, I think I've state this in this guide. I encourage people to try, because there is no way to know. It then appears that some muxless laptop can read the rom file and not through ACPI call, but more people failed, and this error also appears as Code 43. This is just a note that you might not get what you want after all the efforts, try it wisely based on the time and energy you have.

Owner

Misairu-G commented Aug 1, 2018

@KingTheGuy
"lets say i have an hdmi cable connected, should i get an output. even if i dont give the ssids or rom", that would largely depends on your laptop. Most optimus laptop have video output wired to iGPU, in that case, no video output for the VM. Some optimus laptop can choose to wire to dGPU, but I would say quite rare.

"using a gtx 1000 "mobile" series they don't have uefi support" I think that's not true. Laptop shipped with G-sync (which have dGPU connected to display and iGPU available) have to support UEFI in their vBIOS, otherwise the laptop won't boot. And several mobile GTX 1070 vBIOS I have also indicate that they support UEFI (though those vBIOS are for MXM cards).

The other thing is whether to read the rom file you gave depends on the driver. So for your graphics card, if nvidia say that it should read its vBIOS from ACPI call (which means it read its vBIOS as part of the BIOS), then the rom file you have is pretty much useless, I think I've state this in this guide. I encourage people to try, because there is no way to know. It then appears that some muxless laptop can read the rom file and not through ACPI call, but more people failed, and this error also appears as Code 43. This is just a note that you might not get what you want after all the efforts, try it wisely based on the time and energy you have.

@Lukypie

This comment has been minimized.

Show comment
Hide comment
@Lukypie

Lukypie Aug 9, 2018

Error 43 in a Dell XPS 15 9550, no way to get the Dell gpu vbios.

Lukypie commented Aug 9, 2018

Error 43 in a Dell XPS 15 9550, no way to get the Dell gpu vbios.

@3kynox

This comment has been minimized.

Show comment
Hide comment
@3kynox

3kynox Aug 15, 2018

Back here

@Babbie It might be relevant that I was unable to extract my vROM, so I had to grab one from the internet.

Which laptop are you using? Possible to share your ROM or link found on internet?

I'll try passthrough on a Dell Inspiron 7567 soon

3kynox commented Aug 15, 2018

Back here

@Babbie It might be relevant that I was unable to extract my vROM, so I had to grab one from the internet.

Which laptop are you using? Possible to share your ROM or link found on internet?

I'll try passthrough on a Dell Inspiron 7567 soon

@d0ku

This comment has been minimized.

Show comment
Hide comment
@d0ku

d0ku Aug 19, 2018

Success

It works on ASUS G751JM (i7-4710HQ, NVIDIA gtx 860m). It wasn't completely straightforward, but following this tutorial and reading (A LOT) about it on arch wiki etc. definitely worked out.

Description

I did not have Error 43, gpu rom was not needed. GPU can be hot-swapped between linux host (Arch) and windows guest (win 10) without any problems.

miscellaneous

I can't configure nvidia gamestream at the moment, but I believe windows side is cause of the problem.

If you were messing with fan speeds (reading and writing to EC registers), remember to reset control values. 'Real' Windows does it on its own hand, but forgetting about that detail can cause gpu overheat, because vm does not restore control over gpu fans. In case you own g751 you have to write 255 (or FF in hex) to 152 register for gpu and 151 register for cpu. If you have no idea what you just read, then you are probably good to go.

Also when i run nvidia control panel i get 'You are not currently using a display attached to an NVIDIA GPU', but Unigine Heaven uses 860m and it is normally detected, so I will leave it as it is at the moment.

The last thing is I can't run furmark because it says my opengl version is too low, at the same time Unigine Heaven DirectX 11 version runs without problems. I will look at it in near future.

If you were messing with fan speeds (reading and writing to EC registers), remember to reset control values. 'Real' Windows does it on its own hand, but forgetting about that detail can cause gpu overheat, because vm does not restore control over gpu fans. In case you own g751 you have to write 255 (or FF in hex) to 152 register for gpu and 151 register for cpu. If you have no idea what you just read, then you are probably good to go.

Also when i run nvidia control panel i get 'You are not currently using a display attached to an NVIDIA GPU', but Unigine Heaven uses 860m and it is normally detected, so I will leave it as it is at the moment.

d0ku commented Aug 19, 2018

Success

It works on ASUS G751JM (i7-4710HQ, NVIDIA gtx 860m). It wasn't completely straightforward, but following this tutorial and reading (A LOT) about it on arch wiki etc. definitely worked out.

Description

I did not have Error 43, gpu rom was not needed. GPU can be hot-swapped between linux host (Arch) and windows guest (win 10) without any problems.

miscellaneous

I can't configure nvidia gamestream at the moment, but I believe windows side is cause of the problem.

If you were messing with fan speeds (reading and writing to EC registers), remember to reset control values. 'Real' Windows does it on its own hand, but forgetting about that detail can cause gpu overheat, because vm does not restore control over gpu fans. In case you own g751 you have to write 255 (or FF in hex) to 152 register for gpu and 151 register for cpu. If you have no idea what you just read, then you are probably good to go.

Also when i run nvidia control panel i get 'You are not currently using a display attached to an NVIDIA GPU', but Unigine Heaven uses 860m and it is normally detected, so I will leave it as it is at the moment.

The last thing is I can't run furmark because it says my opengl version is too low, at the same time Unigine Heaven DirectX 11 version runs without problems. I will look at it in near future.

If you were messing with fan speeds (reading and writing to EC registers), remember to reset control values. 'Real' Windows does it on its own hand, but forgetting about that detail can cause gpu overheat, because vm does not restore control over gpu fans. In case you own g751 you have to write 255 (or FF in hex) to 152 register for gpu and 151 register for cpu. If you have no idea what you just read, then you are probably good to go.

Also when i run nvidia control panel i get 'You are not currently using a display attached to an NVIDIA GPU', but Unigine Heaven uses 860m and it is normally detected, so I will leave it as it is at the moment.

@adrianf0

This comment has been minimized.

Show comment
Hide comment
@adrianf0

adrianf0 Aug 22, 2018

Failure: error 43
Thinkpad T480 with MX150

I extracted vBIOS from laptop's UEFI image using UEFITool. I extracted the UEFI image from the Lenovo UEFI update executable (Lenovo support webpage). As vBIOS was missing UEFI support it has been added with GOPupd tool. Thus the rom was provided to KVM:

-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-pci-sub-device-id=0x225e,x-pci-sub-vendor-id=0x17aa,multifunction=on,romfile=../VGA2.rom

However, Nvidia driver in Windows guest still returns error 43.

adrianf0 commented Aug 22, 2018

Failure: error 43
Thinkpad T480 with MX150

I extracted vBIOS from laptop's UEFI image using UEFITool. I extracted the UEFI image from the Lenovo UEFI update executable (Lenovo support webpage). As vBIOS was missing UEFI support it has been added with GOPupd tool. Thus the rom was provided to KVM:

-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-pci-sub-device-id=0x225e,x-pci-sub-vendor-id=0x17aa,multifunction=on,romfile=../VGA2.rom

However, Nvidia driver in Windows guest still returns error 43.

@pascalav

This comment has been minimized.

Show comment
Hide comment
@pascalav

pascalav Aug 29, 2018

Another success here: reached DirectX 3D gaming in a Win10 guest, using Looking Glass (framebuffer relay from guest to host). Still to be solved: Looking Glass gets lost when running some 3D app fullscreen (some games will launch fullscreen, some will crash Looking Glass).

@adrianf0 and at people stuck at error 43: you should try to compile a patched version of OVMF that will embed an ACPI table for your VBIOS. It solved it for me on a MUXed laptop (see my adventures at https://www.reddit.com/r/VFIO/comments/96zc7q/ubuntu_18041_lts_nvidia_defaults_hangs_on_vm_start/e4vlssn ), even if it was suggested by someone running a MUXless machine.

@Misairu-G : I believe your guide references outdated VirtIO windows-guest drivers (i.e. 2012). While installing the Looking Glass addition, I read that reference URL: https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers , leading to https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/ ). Still, I don't know the compatibility matrix between the libvirt and drivers versions. I'm using the latest ones from above (i.e. 2018).

A side-note: if you have difficulties to dump your VBIOS or face error 43, you may read about clean GPU BIOS and preventing its "shadow" copy to load in host, that may also solve this error 43 in guest (see https://passthroughpo.st/explaining-csm-efifboff-setting-boot-gpu-manually/ ). Solution is some additional GRUB param ( efifb=off ), untold in Misairu's guide, that may solve it for you if your host/flashed VBIOS is an UEFI one.

pascalav commented Aug 29, 2018

Another success here: reached DirectX 3D gaming in a Win10 guest, using Looking Glass (framebuffer relay from guest to host). Still to be solved: Looking Glass gets lost when running some 3D app fullscreen (some games will launch fullscreen, some will crash Looking Glass).

@adrianf0 and at people stuck at error 43: you should try to compile a patched version of OVMF that will embed an ACPI table for your VBIOS. It solved it for me on a MUXed laptop (see my adventures at https://www.reddit.com/r/VFIO/comments/96zc7q/ubuntu_18041_lts_nvidia_defaults_hangs_on_vm_start/e4vlssn ), even if it was suggested by someone running a MUXless machine.

@Misairu-G : I believe your guide references outdated VirtIO windows-guest drivers (i.e. 2012). While installing the Looking Glass addition, I read that reference URL: https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers , leading to https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/ ). Still, I don't know the compatibility matrix between the libvirt and drivers versions. I'm using the latest ones from above (i.e. 2018).

A side-note: if you have difficulties to dump your VBIOS or face error 43, you may read about clean GPU BIOS and preventing its "shadow" copy to load in host, that may also solve this error 43 in guest (see https://passthroughpo.st/explaining-csm-efifboff-setting-boot-gpu-manually/ ). Solution is some additional GRUB param ( efifb=off ), untold in Misairu's guide, that may solve it for you if your host/flashed VBIOS is an UEFI one.

@Misairu-G

This comment has been minimized.

Show comment
Hide comment
@Misairu-G

Misairu-G Aug 29, 2018

@pascalav Good to hear you success, also great thanks to your research and suggestion. I'll update the guide whenever I can.

Owner

Misairu-G commented Aug 29, 2018

@pascalav Good to hear you success, also great thanks to your research and suggestion. I'll update the guide whenever I can.

@adrianf0

This comment has been minimized.

Show comment
Hide comment
@adrianf0

adrianf0 Aug 29, 2018

@pascalav: Thank you for the update.
I patched, compiled and installed the OVMF package following the guides from your link. Unfortunately, it didn't solve the "Error 43" problem.

adrianf0 commented Aug 29, 2018

@pascalav: Thank you for the update.
I patched, compiled and installed the OVMF package following the guides from your link. Unfortunately, it didn't solve the "Error 43" problem.

@gachiemchiep

This comment has been minimized.

Show comment
Hide comment
@gachiemchiep

gachiemchiep Sep 7, 2018

I just want to save thank you. Keep going man

gachiemchiep commented Sep 7, 2018

I just want to save thank you. Keep going man

@samcoinc

This comment has been minimized.

Show comment
Hide comment
@samcoinc

samcoinc Oct 15, 2018

@samcoinc did you ever manage to get your W530 working? I tried but ended up with error 43...

No - I never got the W530 working. (although now that I have a working setup (gs60 2pc with gtx 860m - I could revisit it. ))

The gs60 has a dedicated video card and the video bios isn't part of the main bios. that might help a lot.., Very very happy with it. The only downside is the battery life...

http://electronicsam.com/images/KandT/testing/primus/benchmark1.png

http://electronicsam.com/images/KandT/testing/primus/passthrough.png

samcoinc commented Oct 15, 2018

@samcoinc did you ever manage to get your W530 working? I tried but ended up with error 43...

No - I never got the W530 working. (although now that I have a working setup (gs60 2pc with gtx 860m - I could revisit it. ))

The gs60 has a dedicated video card and the video bios isn't part of the main bios. that might help a lot.., Very very happy with it. The only downside is the battery life...

http://electronicsam.com/images/KandT/testing/primus/benchmark1.png

http://electronicsam.com/images/KandT/testing/primus/passthrough.png

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment