QEMU with VirtIO GPU Vulkan Support

qemu-vulkan-virtio.md

QEMU with VirtIO GPU Vulkan Support

With its latest reales qemu added the Venus patches so that virtio-gpu now support venus encapsulation for vulkan. This is one more piece to the puzzle towards full Vulkan support.

An outdated blog post on clollabora described in 2021 how to enable 3D acceleration of Vulkan applications in QEMU through the Venus experimental Vulkan driver for VirtIO-GPU with a local development environment. Following up on the outdated write up, this is how its done today.

Definitions

Let's start with the brief description of the projects mentioned in the post & extend them:

• QEMU is a machine emulator
• VirGL is an OpenGL driver for VirtIO-GPU, available in Mesa.
• Venus is an experimental Vulkan driver for VirtIO-GPU, also available in Mesa.
• Virglrenderer is a library that enables hardware acceleration to VM guests, effectively translating commands from the two drivers just mentioned to either OpenGL or Vulkan.
• libvirt is an API for managing platform virtualization
• virt-manager is a desktop user interface for managing virtual machines through libvirt

Merged Patches:

• 2024-08-14 venus: make cross-device optional merged in mesa 24.2
• 2024-11-25 KVM: Stop grabbing references to PFNMAP'd pages merged in linux 6.13
• 2024-11-12 Support blob memory and venus on qemu merged in qemu 9.2.0

Work in progress:

• libvirt Add support for more virtio-vga-gl arguments #638
• virt-manager Add support for Venus / Vulkan VirtIO-GPU driver #362

Prerequisites

Make sure you have the proper version installed on the host:

• linux kernel >= 6.13 built with CONFIG_UDMABUF
• working Vulkan and kvm setup
• qemu >= 9.2.0
• virglrenderer with enabled venus support
• mesa >= 24.2.0

You can verify this like so:

$ uname -r
6.13.0
$ ls /dev/udmabuf
/dev/udmabuf
$ ls /dev/kvm
/dev/kvm
$ qemu-system-x86_64 --version
QEMU emulator version 9.2.0
Copyright (c) 2003-2024 Fabrice Bellard and the QEMU Project developers

Check your distros package sources how they build virglrenderer.

For Vulkan to work you need the proper drivers to be installed for your graphics card.

To verfiy your setup, install vulkan-tools. Make sure mesa >= 24.2.0 and test vkcube:

$ vulkaninfo --summary | grep driverInfo
	driverInfo         = Mesa 24.2.3-1ubuntu1
	driverInfo         = Mesa 24.2.3-1ubuntu1 (LLVM 19.1.0)
...
$ vkcube
Selected GPU x: ..., type: ...

Building qemu

If your distro doesn't (yet) ship and updated version of qemu, you can build it yourself from source:

wget https://download.qemu.org/qemu-9.2.0.tar.xz
tar xvJf qemu-9.2.0.tar.xz
cd qemu-9.2.0
mkdir build && cd build
../configure --target-list=x86_64-softmmu  \
  --enable-kvm                 \
  --enable-opengl              \
  --enable-virglrenderer       \
  --enable-gtk                 \
  --enable-sdl
make -j4

The configuration step will throgh errors if packages are missing. Check the qemu wiki for further info what to install: https://wiki.qemu.org/Hosts/Linux

Create and run an image for QEMU

Create an image & fetch the distro of your choice:

Host

ISO=ubuntu-24.10-desktop-amd64.iso  
wget https://releases.ubuntu.com/oracular/ubuntu-24.10-desktop-amd64.iso  

IMG=ubuntu-24-10.qcow2
qemu-img create -f qcow2 $IMG 16G

Run a live version or install the distro

qemu-system-x86_64                                               \
    -enable-kvm                                                  \
    -M q35                                                       \
    -smp 4                                                       \
    -m 4G                                                        \
    -cpu host                                                    \
    -net nic,model=virtio                                        \
    -net user,hostfwd=tcp::2222-:22                              \
    -device virtio-vga-gl,hostmem=4G,blob=true,venus=true        \
    -vga none                                                    \
    -display gtk,gl=on,show-cursor=on                            \
    -usb -device usb-tablet                                      \
    -object memory-backend-memfd,id=mem1,size=4G                 \
    -machine memory-backend=mem1                                 \
    -hda $IMG                                                    \
    -cdrom $ISO                                                  

Adjust the parameters accordingly:

• smp: number of cpu cores
• m: RAM
• hostmem,size: VRAM

Guest

Install mesa-utilites and vulkan-tools to test the setup:

$ glxinfo -B

$ vkcube
Selected GPU x: ..., type: ...

If the deive is llvmpipe somehting is wrong. The device should be virgl (...).

Troubleshooting

• (host) add -d guest_errors to show error messages from the guest
• (guest) try installing vulkan virtio drivers and mesa
• check the original blog post

Ubuntu 24.10

This is how you do it on Ubuntu

kernel

Install mainline: https://github.com/bkw777/mainline

sudo add-apt-repository ppa:cappelikan/ppa
sudo apt update
sudo apt install mainline

find the latest kernel (>= 6.13), at the time of writing 6.13 is a release candidate, so include those:

$ mainline check --include-rc

Install kernel:

$ sudo mainline install 6.13-rc1

Verfify installed kernels:

$ mainline list-installed
mainline 1.4.10
Installed Kernels:
linux-image-6.11.0-13-generic
linux-image-generic-hwe-24.04
linux-image-unsigned-6.13.0-061300rc1-generic
mainline: done

reboot into new kernel

verify running kernel

$ uname -r
6.13.0-061300rc1-generic

virglrenderer

the ubuntu package is not compiled with the proper flags.

If installed remove it: $ sudo apt-get remove libvirglrenderer-dev

download, build & install from source with venus enabled

wget    https://gitlab.freedesktop.org/virgl/virglrenderer/-/archive/1.1.0/virglrenderer-1.1.0.tar.gz
sudo apt-get install python3-full ninja-build libvulkan-dev libva-dev
python3 -m venv venv
venv/bin/pip install meson
venv/bin/meson build -Dvideo=true -Dvenus=true
ninja -C build
ninja install

qemu

install qemu >= 9.2.0, at the time of writing ubuntu has not yet packaged it

Install build depdencies: https://wiki.qemu.org/Hosts/Linux

sudo apt-get install build-essential pip libslirp-dev slirp
sudo apt-get install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build
sudo apt-get install git-email
sudo apt-get install libaio-dev libbluetooth-dev libcapstone-dev libbrlapi-dev libbz2-dev
sudo apt-get install libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev
sudo apt-get install libibverbs-dev libjpeg8-dev libncurses5-dev libnuma-dev
sudo apt-get install librbd-dev librdmacm-dev
sudo apt-get install libsasl2-dev libsdl2-dev libseccomp-dev libsnappy-dev libssh-dev
sudo apt-get install libvde-dev libvdeplug-dev libvte-2.91-dev libxen-dev liblzo2-dev
sudo apt-get install valgrind xfslibs-dev 
sudo apt-get install libnfs-dev libiscsi-dev

build and run as described

virt-manager

-- work in progress --

Currently this is work in progress, so there is no option to add vulkan support in virt-manager. There are no fields to configure this. Also xml doesnt work, because libvirt doesn't know about these options either, so xml validation fails. There is however an option for QEMU command-line passthrough which bypasses the validation.

If you setup a default machine with 4G of memory, you can do this:

  <qemu:commandline>
    <qemu:arg value="-device"/>
    <qemu:arg value="virtio-vga-gl,hostmem=4G,blob=true,venus=true"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="memory-backend-memfd,id=mem1,size=4G"/>
    <qemu:arg value="-machine"/>
    <qemu:arg value="memory-backend=mem1"/>
    <qemu:arg value="-vga"/>
    <qemu:arg value="none"/>
  </qemu:commandline>

Which gives this error:

qemu-system-x86_64: virgl could not be initialized: -1

Changing the number from 4G to 4194304k (same as memory) leds to this error:

qemu-system-x86_64: Spice: ../spice-0.15.2/server/red-qxl.cpp:435:spice_qxl_gl_scanout: condition `qxl_state->gl_draw_cookie == GL_DRAW_COOKIE_INVALID' failed

to be further investigated.

Thanks! What about together with the prior hack (force buffer blit) and see if the mentioned erratic motion is improved?

There might exist multiple issues on the wsi path. Previously with the PAT fix, (5) working but (6) not suggested host Nvidia driver has issues dealing with linear image import.

Thanks! What about together with the prior hack (force buffer blit) and see if the mentioned erratic motion is improved?

There might exist multiple issues on the wsi path. Previously with the PAT fix, (5) working but (6) not suggested host Nvidia driver has issues dealing with linear image import.

I did guess I had misunderstood your instructions.

This time I recompiled Mesa from latest commit (09896ee79e3 as of when I pulled from FDO) with both vn-force-prime-blit and vn-fix-acquire-fence applied as patches to that commit. Then, I installed the resulting package in the guest. The host kernel is the same patched 6.14.2 Arch kernel as before. The guest kernel is a stock 6.14.2 Arch kernel as before.

In this gist logs from running the commands 1-6 are collected and numbered. Visually, 1-5 result in a spinning cube being drawn. The ones with sw in MESA_VK_WSI_DEBUG generally appear more correct but much slower. The ones without are quicker but demonstrate what appears to be abrupt changes of the spinning speed or skipped frames every so often. They are nonetheless somewhat improved over the previous case of applying only vn-force-prime-blit.

6 still does not result in a spinning cube, only a black window, and triggers the host dmesg error message from NVIDIA driver.

I have also made a screen recording of a VN_DEBUG=all VN_PERF=all vkcube run to hopefully give a sense of the cube's motion.

One more question before I go back to do more homework: do you see improvements with just vkcube (w/o any additional env vars)? as compared to the video attached on your previous #gistcomment-5537654?

The video on your latest #gistcomment-5541821 looks fine to me already. The occasional janks likely came from the compositor stack backpressure, but the out-of-order issue is gone per my visual check. Just need to confirm this ; )

One more question before I go back to do more homework: do you see improvements with just vkcube (w/o any additional env vars)? as compared to the video attached on your previous #gistcomment-5537654?

The video on your latest #gistcomment-5541821 looks fine to me already. The occasional janks likely came from the compositor stack backpressure, but the out-of-order issue is gone per my visual check. Just need to confirm this ; )

Yes, it has improved significantly when run without any environment variables. It has gone from no cube and a black window at the very beginning to a proper rendering of the cube and no glaringly erratic motion. The issues that stand out are the following:

1. vkcube --wsi wayland still stalls with the fence wait error, even though the same command runs fine on the host.
2. With XCB WSI on the guest, vkcube runs and displays fine but there is a brief display of a black background before the rendering starts. This does not happen on the host with either XCB or Wayland WSI.
3. The cube's motion, while significantly better than with just the prime blit patch, still is not smooth. As you have pointed out, it does not seem to go back and forth anymore but occasionally speeds up and slows down.

For those who encounter stuck in fence wait, adding environment variable

VN_PERF=no_fence_feedback

to /etc/environment might help.

Just in case some folks are affected by this, I happen to realize that the stock mesa driver from the stable debian bookworm is with Mesa 22.3.6 which contains an Intel ANV bug hit by Venus sync feedback optimization path. So if you see the issue with Intel iGPU setup, you can compile a separate ANV driver from the latest mesa release and the issue will be gone with optimized Venus performance.

2. With XCB WSI on the guest, vkcube runs and displays fine but there is a brief display of a black background before the rendering starts. This does not happen on the host with either XCB or Wayland WSI.

Hi @myrslint , will the black period go away with MESA_SHADER_CACHE_DISABLE=true? if so, that's due to the slow filesystem ops for shader disk cache. The initial loading occurs during VkDevice creation time.

2. With XCB WSI on the guest, vkcube runs and displays fine but there is a brief display of a black background before the rendering starts. This does not happen on the host with either XCB or Wayland WSI.

Hi @myrslint , will the black period go away with MESA_SHADER_CACHE_DISABLE=true? if so, that's due to the slow filesystem ops for shader disk cache. The initial loading occurs during VkDevice creation time.

Hello there again 🙂

Adding the environment variable MESA_SHADER_CACHE_DISABLE=true to vkcube runs does not make that period of black window display go away. The VM's storage uses virtio, relatively fast virtual storage driver, and is backed by a qcow2 file on a relatively fast SSD so I doubt disk operations would be a bottleneck anywhere.

However, as seen in the debug logs during the time a black window is displayed before the cube is shown messages indicate that a swapchain is created three times (in parallel?) and destroyed for some reason before a fourth successful one is finally created.

Hi @zhangyiwei, thanks a lot for your information in this thread!

The workaround VN_PERF=no_fence_feedback looks like to cause disordered presence of the guest images, i.e., an out-of-date image could show up again after presenting the current frame... Is this an expected behavior? IIUC, adding this environment variable should only introduce performance pushiment without any impact to the functionality. Am I wrong?

I looked through the communications on this patch set: https://lore.kernel.org/all/20240309010929.1403984-1-seanjc@google.com/, but unfortunately I am still unclear about the reason why this patch (KVM: VMX: Always honor guest PAT on CPUs that support self-snoop) is mandatory for venus to work properly with an Intel CPU. My murky understanding is that without this patch, PAT will not be honnored, leading to memory shared by host and guest being mapped as unexpected cache type (UC?) and data written by host not being visible to the guest... Could you please share more insights about the cause?

Hi @zhangyiwei, thanks a lot for your information in this thread!

The workaround VN_PERF=no_fence_feedback looks like to cause disordered presence of the guest images, i.e., an out-of-date image could show up again after presenting the current frame... Is this an expected behavior? IIUC, adding this environment variable should only introduce performance pushiment without any impact to the functionality. Am I wrong?

That's not expected behavior, and very likely it'd be broken with or without on your setup (more details?). You may give https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34516 a try to see if it helps.

I looked through the communications on this patch set: https://lore.kernel.org/all/20240309010929.1403984-1-seanjc@google.com/, but unfortunately I am still unclear about the reason why this patch (KVM: VMX: Always honor guest PAT on CPUs that support self-snoop) is mandatory for venus to work properly with an Intel CPU. My murky understanding is that without this patch, PAT will not be honnored, leading to memory shared by host and guest being mapped as unexpected cache type (UC?) and data written by host not being visible to the guest... Could you please share more insights about the cause?

No, it's the other way: guest cpu uploads end up not visible to host gpu. The missing background can be found here: https://patchwork.kernel.org/project/dri-devel/cover/20200213213036.207625-1-olvaffe@gmail.com/

The commands in the original gist for passing venus="true" to qemu no longer work in recent versions of libvirt. Below you will find an updated snippet to copy-paste. Note that it is still broken, as I'm getting the same "Display output is not active", which the original author had.

In virt-manager's "Details" view:

• Memory -> Enable shared memory
• Display Spice -> Listen type -> None
• Display Spice -> OpenGL ✔️
• Video Virtio -> 3d acceleration ✔️
• Overview -> XML tab (make sure "editable" is enabled in the preferences)

Copy paste this before the closing </domain>:

  <override xmlns="http://libvirt.org/schemas/domain/qemu/1.0">
    <device alias="video0">
      <frontend>
        <property name="blob" type="bool" value="true"/>
        <property name="venus" type="bool" value="true"/>
        <property name="hostmem" type="unsigned" value="4194304"/>
      </frontend>
    </device>
  </override>

Unfortunately I'm only getting

error: kvm run failed Bad address
RAX=0000000000000000 RBX=00007ffcca3ae490 RCX=0000000000000000 RDX=00007fd754f2dff0
...
EFER=0000000000000d01
Code=f3 41 0f 11 07 f3 0f 11 0a 4c 01 e2 f3 0f 11 0a 66 0f ef c9 <f3> 42 0f 11 04 22 8d 14 08 44 01 e8 66 0f ef c0 01 c8 01 d2 01 c0 f3 0f 2a ca 8b 54 24 50

as soon as the desktop envrionment starts :-(

Any ideas? Followed the tutorial 1:1

This is very likely transparent huge page issue via KVM that has been fixed on newer kernel (6.13 or later) which has https://lore.kernel.org/all/20241010182427.1434605-1-seanjc@google.com/. Or you can recompile your current host kernel with CONFIG_TRANSPARENT_HUGEPAGE disabled to give it another try.

I'll draft a workaround in host venus (vkr) to explicitly wait for the implicit fence before submitting to the nv driver.

@zhangyiwei Sorry if I missed it in this thread but did you ever try this workaround?

In gdb I see the virtio app issue a WaitForFences, but the vkWaitForFences issued in vkr_dispatch_vkWaitForFences returns immediately and succeeds. I'm a little confused why the app is stuck waiting in vn_update_sync_result if the server side command already completed? I'm guessing I am not understanding something about the command dispatch correctly.

I'll draft a workaround in host venus (vkr) to explicitly wait for the implicit fence before submitting to the nv driver.

@zhangyiwei Sorry if I missed it in this thread but did you ever try this workaround?

In gdb I see the virtio app issue a WaitForFences, but the vkWaitForFences issued in vkr_dispatch_vkWaitForFences returns immediately and succeeds. I'm a little confused why the app is stuck waiting in vn_update_sync_result if the server side command already completed? I'm guessing I am not understanding something about the command dispatch correctly.

It turns out no workaround needed on the renderer side. The guest virtgpu fence has the correct payload installed. So on drivers without implicit fencing support, the issue can be mitigated with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34516. I just closed that MR since I figured a better way to do it within venus. Would be siimilar but no need to touch common vk there so that review can be faster.

@amshafer you still have to force prime blit via venus. Here's the hack at this point. Will make it the default for venus on nv proprietary later.

Works like a charm with QEMU command line.
Now I need to figure out how to use this with virt manager.

EDIT:

LG GRAM 2022 - 17Z90Q-G.AD78Y 
Linux 6.14.5-300.fc42.x86_64
OS:  Aurora Version: latest-42.20250512.2
CPU: 12th Gen Intel(R) Core(TM) i7-1260P
IGPU: Intel Iris Xe Graphics @ 1.40 GHz
MESA : mesa-dri-drivers-25.0.2-1.fc42.x86_64
Virglrenderer: virglrenderer-1.1.0-2.fc42.x86_64

@myrslint anything else I forgot to mention that could be useful for others?
Regards

Works like a charm with QEMU command line. 6.14.5-300.fc42.x86_64

Now I need to figure out how to use this with virt manager.

It would be nice if any users who report a working or non-working result also included the relevant parts of their hardware and software configuration e.g., CPU make and model, GPU make and model, Linux distribution on the host and the guest, versions of Mesa and virglrenderer and their patch states.

@myrslint anything else I forgot to mention that could be useful for others? Regards

Looks comprehensive to me. Thank you!

@zhangyiwei unfortunately even with both of those (34516 and force blit) I still see the same "stuck in fence wait" others have reported. Are those known to fix that issue? I'm happy to help test whatever your venus-only follow up solution is if that helps.

@zhangyiwei unfortunately even with both of those (34516 and force blit) I still see the same "stuck in fence wait" others have reported. Are those known to fix that issue? I'm happy to help test whatever your venus-only follow up solution is if that helps.

If VN_PERF=no_fence_feedback env var can workaround that, you might be missing the host kvm bits I mentioned on https://docs.mesa3d.org/drivers/venus.html for NV, and for the final shape of the reland, see https://gitlab.freedesktop.org/mesa/mesa/-/issues/12806#note_2861587.

@zhangyiwei unfortunately even with both of those (34516 and force blit) I still see the same "stuck in fence wait" others have reported. Are those known to fix that issue? I'm happy to help test whatever your venus-only follow up solution is if that helps.

In addition, that force blit hack is for x11 only. The proper way to force it via venus for both x11 and wayland has been added in https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34965.

I'm actually reproducing this without any QEMU/VM/etc setup at all, running virgl_test_server --venus on the NVIDIA discrete GPU and vkcube --wsi wayland on virtio. Someone from this thread had pointed me at that and I was investigating to see if we have any NVIDIA bugs on our driver side, although so far I don't see anything obvious. I think because I'm not using QEMU I don't need the KVM patch you mentioned?

VN_PERF=no_fence_feedback does not help my case unfortunately. Manually turning on prime blit or using 34965 also did not help.

I'm actually reproducing this without any QEMU/VM/etc setup at all, running virgl_test_server --venus on the NVIDIA discrete GPU and vkcube --wsi wayland on virtio. Someone from this thread had pointed me at that and I was investigating to see if we have any NVIDIA bugs on our driver side, although so far I don't see anything obvious. I think because I'm not using QEMU I don't need the KVM patch you mentioned?

VN_PERF=no_fence_feedback does not help my case unfortunately. Manually turning on prime blit or using 34965 also did not help.

You better test with VM setup instead, because it's purely impl defined behavior for vtest. Venus over vtest can only work if host driver supports dma-buf export AND the driver's hook for dma-buf mmap sets the correct pat entry. The reason is dma-buf mmap uapi has explicit flush/invalidate call sequence so some implementations just map those cached non-coherent since the cache can be handled at the dma-buf flush/invalidate api boundary. e.g. Venus on Intel MTL over vtest won't work for this reason.

There's no plan to extend vtest protocol to support opaque fd export for device memory, so we are unable to steer to use Vulkan vkMapMemory to rely on what vk api guarantees for coherent memory type.

Once you have a VM setup, since you have control over the host driver, you can also try with VK_EXT_external_memory_dma_buf extension enabled/disabled from NV.

I'm actually reproducing this without any QEMU/VM/etc setup at all, running virgl_test_server --venus on the NVIDIA discrete GPU and vkcube --wsi wayland on virtio. Someone from this thread had pointed me at that and I was investigating to see if we have any NVIDIA bugs on our driver side, although so far I don't see anything obvious. I think because I'm not using QEMU I don't need the KVM patch you mentioned?

VN_PERF=no_fence_feedback does not help my case unfortunately. Manually turning on prime blit or using 34965 also did not help.

Please take the relevance of this with a grain of salt because I'm not knowledgeable at all in this field. As a data point I would like to add that, on NVIDIA, on the host itself vkcube --wsi wayland has been exiting with a segmentation fault and dumping core due to a floating point exception. This does not happen if XCB WSI is used.

My hardware configuration is just as mentioned above. My software configuration is:

1. Arch Linux, 6.14.6-arch1-1. Stock Arch kernel.
2. NVIDIA proprietary driver 575.51.02 produced and installed using the relevant AUR package.
3. wlroots-git 0.19.0.r7482.2420bfef0-1 and sway-git 1.11.r7545.8d3a52a-1

Thanks, I'll give that a go and compare

It would be nice if any users who report a working or non-working result also included the relevant parts of their hardware and software configuration e.g., CPU make and model, GPU make and model, Linux distribution on the host and the guest, versions of Mesa and virglrenderer and their patch states.

@myrslint I do BUT it did work as of now. My hardware is:
CPU: i9-10900K
GPU: RE 9800/9800 XT/ 6900 XT (hwinfo gave them all)

I'm on mint and xubuntu is my guest.

as for mesa, virglrender, and patches, I have a script built the first 2 and I need to patches:
This to stop jittery camera:
https://lore.kernel.org/qemu-devel/20250310120555.150077-1-dmitry.osipenko@collabora.com/T/#m471243e56796537ab0be33d7315dbc3c1bf99143
And this for the game to start:
torvalds/linux@377b2f3
This is the script (yha I know it's not JUST it but still):
https://pastebin.com/wpLy3c7d

Also 6.15-rc6 worked for me.

After all this all is good and I can even run DX12 games now.

If you wanna know more look here:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/12806
and here:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/12272

@myrslint I do BUT it did work as of now. My hardware is: CPU: i9-10900K GPU: RE 9800/9800 XT/ 6900 XT (hwinfo gave them all)

I'm on mint and xubuntu is my guest.
[...]

Thanks for the detailed information. I hope it helps other users as well.

@myrslint I do BUT it did work as of now. My hardware is: CPU: i9-10900K GPU: RE 9800/9800 XT/ 6900 XT (hwinfo gave them all)

I'm on mint and xubuntu is my guest.
[...]

Thanks for the detailed information. I hope it helps other users as well.

No problem! Oh and I think I should tell you something. I just noticed that I said hwinfo "list them all" but I SHOULD say it's not all the GPUs. The PC has 1 GPU but hwinfo list all of the the ones in my post. My bad if that was unclear.

Just to give an update here for folks still experiencing stuck in fence wait beyond the pat issue, it turns out qemu didn't properly implement venus fencing. This ended up with dma fences from venus context being immediately signaled most of the time, and has been the reason of issues like stuck due to host GPU hang, on screen artifacts, frame going out-of-order for heavy loaded games, etc. Btw, issues falling into this bucket won't repro via crosvm, in case useful for local debugging purpose.

Here's the QEMU patch: https://patchew.org/QEMU/20250518152651.334115-1-zzyiwei@gmail.com/

Update: the above patch has been queued to land: https://patchew.org/QEMU/20250521164250.135776-1-alex.bennee@linaro.org/

...I have uploaded a screen recording of the VM window to give a sense of the erratic motion mentioned. It also contains the terminal window showing the programs being run and the error messages.

Hi @myrslint, based on your current setup, the QEMU fix for proper Venus fencing support that has been in-queue to land will fix the erratic motion mentioned (sometimes frame going backwards). See my prior comment.

...I have uploaded a screen recording of the VM window to give a sense of the erratic motion mentioned. It also contains the terminal window showing the programs being run and the error messages.

Hi @myrslint, based on your current setup, the QEMU fix for proper Venus fencing support that has been in-queue to land will fix the erratic motion mentioned (sometimes frame going backwards). See my prior comment.

Thanks again for all the hard work and follow-up.

To summarize the findings until now, if I understand correctly the following changes from a stock Arch Linux (or other Linux distro) installation should happen for one to have working Venus on an NVIDIA GPU and Intel CPU at the moment:

1. the host kernel has to be patched and recompiled with aforementioned one-liner to enable honoring of the guest PAT (Arch Linux is currently on 6.14.7);
2. the host QEMU has to be patched and recompiled with https://patchew.org/QEMU/20250518152651.334115-1-zzyiwei@gmail.com/ (Arch Linux currently ships 10.0.0)
3. the guest Mesa has to be patched and recompiled from tip of the tree with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34965 (Arch Linux is on 25.1.1);
4. the host virglrenderer has to be built from tip of the tree (Arch Linux package for this is built from version 1.1.1 source).

Is this summary of required actions as of this moment correct?

Hi @myrslint, very precise. For guest mesa, besides the mentioned MR which has landed just now, better patch ToT mesa with the below 2 as well to be more functionally correct:

1. https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35034 (halfway through the review process)
2. https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34516 (I'll land this soon)

If things are still broken, then better steer to the sw blit path in the guest WSI support by MESA_VK_WSI_DEBUG=sw,buffer vkcube. The rendering is native as before, but the rendered WSI image will blit to a buffer cpu mapped in the guest shared with the compositor side. That's the last resolve at this stage, and I'll consider making that the default to first make things work.