Skip to content

Instantly share code, notes, and snippets.

@peppergrayxyz
Last active January 16, 2025 01:05
Show Gist options
  • Save peppergrayxyz/fdc9042760273d137dddd3e97034385f to your computer and use it in GitHub Desktop.
Save peppergrayxyz/fdc9042760273d137dddd3e97034385f to your computer and use it in GitHub Desktop.
QEMU with VirtIO GPU Vulkan Support

QEMU with VirtIO GPU Vulkan Support

With its latest reales qemu added the Venus patches so that virtio-gpu now support venus encapsulation for vulkan. This is one more piece to the puzzle towards full Vulkan support.

An outdated blog post on clollabora described in 2021 how to enable 3D acceleration of Vulkan applications in QEMU through the Venus experimental Vulkan driver for VirtIO-GPU with a local development environment. Following up on the outdated write up, this is how its done today.

Definitions

Let's start with the brief description of the projects mentioned in the post & extend them:

  • QEMU is a machine emulator
  • VirGL is an OpenGL driver for VirtIO-GPU, available in Mesa.
  • Venus is an experimental Vulkan driver for VirtIO-GPU, also available in Mesa.
  • Virglrenderer is a library that enables hardware acceleration to VM guests, effectively translating commands from the two drivers just mentioned to either OpenGL or Vulkan.
  • libvirt is an API for managing platform virtualization
  • virt-manager is a desktop user interface for managing virtual machines through libvirt

Merged Patches:

Work in progress:

Prerequisites

Make sure you have the proper version installed on the host:

  • linux kernel >= 6.13 built with CONFIG_UDMABUF
  • working Vulkan and kvm setup
  • qemu >= 9.2.0
  • virglrenderer with enabled venus support
  • mesa >= 24.2.0

You can verify this like so:

$ uname -r
6.13.0
$ ls /dev/udmabuf
/dev/udmabuf
$ ls /dev/kvm
/dev/kvm
$ qemu-system-x86_64 --version
QEMU emulator version 9.2.0
Copyright (c) 2003-2024 Fabrice Bellard and the QEMU Project developers

Check your distros package sources how they build virglrenderer.

For Vulkan to work you need the proper drivers to be installed for your graphics card.

To verfiy your setup, install vulkan-tools. Make sure mesa >= 24.2.0 and test vkcube:

$ vulkaninfo --summary | grep driverInfo
	driverInfo         = Mesa 24.2.3-1ubuntu1
	driverInfo         = Mesa 24.2.3-1ubuntu1 (LLVM 19.1.0)
...
$ vkcube
Selected GPU x: ..., type: ...

Building qemu

If your distro doesn't (yet) ship and updated version of qemu, you can build it yourself from source:

wget https://download.qemu.org/qemu-9.2.0.tar.xz
tar xvJf qemu-9.2.0.tar.xz
cd qemu-9.2.0
mkdir build && cd build
../configure --target-list=x86_64-softmmu  \
  --enable-kvm                 \
  --enable-opengl              \
  --enable-virglrenderer       \
  --enable-gtk                 \
  --enable-sdl
make -j4

The configuration step will throgh errors if packages are missing. Check the qemu wiki for further info what to install: https://wiki.qemu.org/Hosts/Linux

Create and run an image for QEMU

Create an image & fetch the distro of your choice:

Host

ISO=ubuntu-24.10-desktop-amd64.iso  
wget https://releases.ubuntu.com/oracular/ubuntu-24.10-desktop-amd64.iso  

IMG=ubuntu-24-10.qcow2
qemu-img create -f qcow2 $IMG 16G

Run a live version or install the distro

qemu-system-x86_64                                               \
    -enable-kvm                                                  \
    -M q35                                                       \
    -smp 4                                                       \
    -m 4G                                                        \
    -cpu host                                                    \
    -net nic,model=virtio                                        \
    -net user,hostfwd=tcp::2222-:22                              \
    -device virtio-vga-gl,hostmem=4G,blob=true,venus=true        \
    -vga none                                                    \
    -display gtk,gl=on,show-cursor=on                            \
    -usb -device usb-tablet                                      \
    -object memory-backend-memfd,id=mem1,size=4G                 \
    -machine memory-backend=mem1                                 \
    -hda $IMG                                                    \
    -cdrom $ISO                                                  

Adjust the parameters accordingly:

  • smp: number of cpu cores
  • m: RAM
  • hostmem,size: VRAM

Guest

Install mesa-utilites and vulkan-tools to test the setup:

$ glxinfo -B
$ vkcube
Selected GPU x: ..., type: ...

If the deive is llvmpipe somehting is wrong. The device should be virgl (...).

Troubleshooting

  • (host) add -d guest_errors to show error messages from the guest
  • (guest) try installing vulkan virtio drivers and mesa
  • check the original blog post

Ubuntu 24.10

This is how you do it on Ubuntu

kernel

Install mainline: https://github.com/bkw777/mainline

sudo add-apt-repository ppa:cappelikan/ppa
sudo apt update
sudo apt install mainline

find the latest kernel (>= 6.13), at the time of writing 6.13 is a release candidate, so include those:

$ mainline check --include-rc

Install kernel:

$ sudo mainline install 6.13-rc1

Verfify installed kernels:

$ mainline list-installed
mainline 1.4.10
Installed Kernels:
linux-image-6.11.0-13-generic
linux-image-generic-hwe-24.04
linux-image-unsigned-6.13.0-061300rc1-generic
mainline: done

reboot into new kernel

verify running kernel

$ uname -r
6.13.0-061300rc1-generic

virglrenderer

the ubuntu package is not compiled with the proper flags.

If installed remove it: $ sudo apt-get remove libvirglrenderer-dev

download, build & install from source with venus enabled

wget    https://gitlab.freedesktop.org/virgl/virglrenderer/-/archive/1.1.0/virglrenderer-1.1.0.tar.gz
sudo apt-get install python3-full ninja-build libvulkan-dev libva-dev
python3 -m venv venv
venv/bin/pip install meson
venv/bin/meson build -Dvideo=true -Dvenus=true
ninja -C build
ninja install

qemu

install qemu >= 9.2.0, at the time of writing ubuntu has not yet packaged it

Install build depdencies: https://wiki.qemu.org/Hosts/Linux

sudo apt-get install build-essential pip libslirp-dev slirp
sudo apt-get install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build
sudo apt-get install git-email
sudo apt-get install libaio-dev libbluetooth-dev libcapstone-dev libbrlapi-dev libbz2-dev
sudo apt-get install libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev
sudo apt-get install libibverbs-dev libjpeg8-dev libncurses5-dev libnuma-dev
sudo apt-get install librbd-dev librdmacm-dev
sudo apt-get install libsasl2-dev libsdl2-dev libseccomp-dev libsnappy-dev libssh-dev
sudo apt-get install libvde-dev libvdeplug-dev libvte-2.91-dev libxen-dev liblzo2-dev
sudo apt-get install valgrind xfslibs-dev 
sudo apt-get install libnfs-dev libiscsi-dev

build and run as described

virt-manager

-- work in progress --

Currently this is work in progress, so there is no option to add vulkan support in virt-manager. There are no fields to configure this. Also xml doesnt work, because libvirt doesn't know about these options either, so xml validation fails. There is however an option for QEMU command-line passthrough which bypasses the validation.

If you setup a default machine with 4G of memory, you can do this:

  <qemu:commandline>
    <qemu:arg value="-device"/>
    <qemu:arg value="virtio-vga-gl,hostmem=4G,blob=true,venus=true"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="memory-backend-memfd,id=mem1,size=4G"/>
    <qemu:arg value="-machine"/>
    <qemu:arg value="memory-backend=mem1"/>
    <qemu:arg value="-vga"/>
    <qemu:arg value="none"/>
  </qemu:commandline>

Which gives this error:

qemu-system-x86_64: virgl could not be initialized: -1

Changing the number from 4G to 4194304k (same as memory) leds to this error:

qemu-system-x86_64: Spice: ../spice-0.15.2/server/red-qxl.cpp:435:spice_qxl_gl_scanout: condition `qxl_state->gl_draw_cookie == GL_DRAW_COOKIE_INVALID' failed

to be further investigated.

@nmhn
Copy link

nmhn commented Dec 12, 2024

help needed
I tried to build a environment for my self , my system info is as below

❯ uname -a
Linux nmhn 6.13.0-rc2 #1 SMP PREEMPT_DYNAMIC Thu Dec 12 19:25:26 CST 2024 x86_64 x86_64 x86_64 GNU/Linux
❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04 LTS
Release:        24.04
Codename:       noble
❯ ls /dev/kvm
/dev/kvm
❯ ls /dev/udmabuf
/dev/udmabuf
# actually I tried wayland
> echo $XDG_SESSION_TYPE
x11
❯ lspci | grep VGA
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] (rev c1)
13:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Raphael (rev c1)
> glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: AMD (0x1002)
    Device: AMD Radeon RX 6800 XT (radeonsi, navi21, LLVM 18.1.7, DRM 3.58, 6.8.0-49-generic) (0x73bf)
    Version: 24.2.0
    Accelerated: yes
    Video memory: 16384MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.6
    Max compat profile version: 4.6
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
Memory info (GL_ATI_meminfo):
    VBO free memory - total: 16006 MB, largest block: 16006 MB
    VBO free aux. memory - total: 31640 MB, largest block: 31640 MB
    Texture free memory - total: 16006 MB, largest block: 16006 MB
    Texture free aux. memory - total: 31640 MB, largest block: 31640 MB
    Renderbuffer free memory - total: 16006 MB, largest block: 16006 MB
    Renderbuffer free aux. memory - total: 31640 MB, largest block: 31640 MB
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 16384 MB
    Total available memory: 48098 MB
    Currently available dedicated video memory: 16006 MB
OpenGL vendor string: AMD
OpenGL renderer string: AMD Radeon RX 6800 XT (radeonsi, navi21, LLVM 18.1.7, DRM 3.58, 6.8.0-49-generic)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 24.2.0-devel
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6 (Compatibility Profile) Mesa 24.2.0-devel
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 24.2.0-devel
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20


> lscpu | grep "Model name"
Model name:                           AMD Ryzen 9 7950X 16-Core Processor
>vulkaninfo --summary
WARNING: [Loader Message] Code 0 : terminator_CreateInstance: Received return code -3 from call to vkCreateInstance in ICD /usr/lib/x86_64-linux-gnu/libvulkan_virtio.so. Skipping this driver.
==========
VULKANINFO
==========

Vulkan Instance Version: 1.3.275


Instance Extensions: count = 23
-------------------------------
VK_EXT_acquire_drm_display             : extension revision 1
VK_EXT_acquire_xlib_display            : extension revision 1
VK_EXT_debug_report                    : extension revision 10
VK_EXT_debug_utils                     : extension revision 2
VK_EXT_direct_mode_display             : extension revision 1
VK_EXT_display_surface_counter         : extension revision 1
VK_EXT_surface_maintenance1            : extension revision 1
VK_EXT_swapchain_colorspace            : extension revision 4
VK_KHR_device_group_creation           : extension revision 1
VK_KHR_display                         : extension revision 23
VK_KHR_external_fence_capabilities     : extension revision 1
VK_KHR_external_memory_capabilities    : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2         : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2       : extension revision 1
VK_KHR_portability_enumeration         : extension revision 1
VK_KHR_surface                         : extension revision 25
VK_KHR_surface_protected_capabilities  : extension revision 1
VK_KHR_wayland_surface                 : extension revision 6
VK_KHR_xcb_surface                     : extension revision 6
VK_KHR_xlib_surface                    : extension revision 6
VK_LUNARG_direct_driver_loading        : extension revision 1

Instance Layers: count = 6
--------------------------
VK_LAYER_AMD_switchable_graphics_64 AMD switchable graphics layer 1.3.287  version 1
VK_LAYER_INTEL_nullhw               INTEL NULL HW                 1.1.73   version 1
VK_LAYER_MANGOHUD_overlay_x86_64    Vulkan Hud Overlay            1.3.0    version 1
VK_LAYER_MANGOHUD_overlay_x86_64    Vulkan Hud Overlay            1.3.0    version 1
VK_LAYER_MESA_device_select         Linux device selection layer  1.3.211  version 1
VK_LAYER_MESA_overlay               Mesa Overlay layer            1.3.211  version 1

Devices:
========
GPU0:
	apiVersion         = 1.3.287
	driverVersion      = 2.0.310
	vendorID           = 0x1002
	deviceID           = 0x73bf
	deviceType         = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
	deviceName         = AMD Radeon RX 6800 XT
	driverID           = DRIVER_ID_AMD_OPEN_SOURCE
	driverName         = AMD open-source driver
	driverInfo         = 2024.Q2.3 (LLPC)
	conformanceVersion = 1.3.5.2
	deviceUUID         = 00000000-0300-0000-0000-000000000000
	driverUUID         = 414d442d-4c49-4e55-582d-445256000000
GPU1:
	apiVersion         = 1.3.287
	driverVersion      = 2.0.310
	vendorID           = 0x1002
	deviceID           = 0x164e
	deviceType         = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
	deviceName         = AMD Radeon Graphics
	driverID           = DRIVER_ID_AMD_OPEN_SOURCE
	driverName         = AMD open-source driver
	driverInfo         = 2024.Q2.3 (LLPC)
	conformanceVersion = 1.3.5.2
	deviceUUID         = 00000000-1300-0000-0000-000000000000
	driverUUID         = 414d442d-4c49-4e55-582d-445256000000

>/home/nmhn/Downloads/qemu-9.2.0/build/qemu-system-x86_64 --version
QEMU emulator version 9.2.0
Copyright (c) 2003-2024 Fabrice Bellard and the QEMU Project developers

I compiled qemu with the instruction of the doc
the booting script I use is

IMG=/home/nmhn/Downloads/u2410.qcow2
ISO=/home/nmhn/Downloads/u2410.iso
  /home/nmhn/Downloads/qemu-9.2.0/build/qemu-system-x86_64   \
    -enable-kvm                                                  \
    -M q35                                                       \
    -smp 4                                                       \
    -m 4G                                                        \
    -cpu host                                                    \
    -net nic,model=virtio                                        \
    -net user,hostfwd=tcp::2222-:22                              \
    -device virtio-vga-gl,hostmem=4G,blob=true,venus=true        \
    -vga none                                                    \
    -display gtk,gl=on,show-cursor=on                            \
    -usb -device usb-tablet                                      \
    -object memory-backend-memfd,id=mem1,size=4G                 \
    -machine memory-backend=mem1                                 \
    -hda $IMG                                                    \
    -cdrom $ISO
~                          

I successfully installed the system in qcow2 disk, however ,when trying to use vkcube / vulkaninfo ,the qemu will pause and receive a strange fault
as the pic shows
Pasted image 20241212204926

@peppergrayxyz
Copy link
Author

Can u try -device virtio-vga-gl instead of -device virtio-vga-gl,hostmem=4G,blob=true,venus=true

This will start the VM with the virtio-gpu device driver, but only with OpenGL, without Vulkan. Does that work?

@nmhn
Copy link

nmhn commented Dec 12, 2024

that works, actually , with -device virtio-vga-gl,hostmem=4G,blob=true,venus=true , the glinfo -B show a virtgl backend with amd radeon6800 info


it was confusing that when i purged all my amdgpu driver and reinstall with amdgpu-install , the amdgpu-dkms failed to build.
the previous working amdgpu driver was build with linux6.8, but it seems not working in linux 6.13
The install log is like this

sudo apt install amdgpu-dkms
[sudo] password for nmhn: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
amdgpu-dkms is already the newest version (1:6.8.5.60203-2044426.24.04).
0 upgraded, 0 newly installed, 0 to remove and 190 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up amdgpu-dkms (1:6.8.5.60203-2044426.24.04) ...
Removing old amdgpu-6.8.5-2044426.24.04 DKMS files...
Deleting module amdgpu-6.8.5-2044426.24.04 completely from the DKMS tree.
Loading new amdgpu-6.8.5-2044426.24.04 DKMS files...
Building for 6.13.0-rc2
Building for architecture x86_64
Building initial module for 6.13.0-rc2
ERROR (dkms apport): kernel package linux-headers-6.13.0-rc2 is not supported
Error! Bad return status for module build on kernel: 6.13.0-rc2 (x86_64)
Consult /var/lib/dkms/amdgpu/6.8.5-2044426.24.04/build/make.log for more information.
dpkg: error processing package amdgpu-dkms (--configure):
 installed amdgpu-dkms package post-installation script subprocess returned error exit status 10
dpkg: dependency problems prevent configuration of amdgpu:
 amdgpu depends on amdgpu-dkms; however:
  Package amdgpu-dkms is not configured yet.

dpkg: error processing package amdgpu (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          Errors were encountered while processing:
 amdgpu-dkms
 amdgpu
E: Sub-process /usr/bin/dpkg returned an error code (1)

> vim /var/lib/dkms/amdgpu/6.8.5-2044426.24.04/build/make.log 
amd/amdkcl/kcl_suspend.c:32:6: warning: no previous prototype for ‘amdkcl_suspend_init’ [-Wmissing-prototypes]
   32 | void amdkcl_suspend_init(void)
      |      ^~~~~~~~~~~~~~~~~~~
  CC [M]  amd/amdkcl/kcl_memory.o
  CC [M]  amd/amdgpu/amdgpu_drv.o
In file included from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/percpu.h:5,
                 from /home/nmhn/Downloads/linux-6.13-rc2/arch/x86/include/asm/msr.h:15,
                 from /home/nmhn/Downloads/linux-6.13-rc2/arch/x86/include/asm/tsc.h:10,
                 from /home/nmhn/Downloads/linux-6.13-rc2/arch/x86/include/asm/timex.h:6,
                 from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/timex.h:67,
                 from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/time32.h:13,
                 from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/time.h:60,
                 from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/jiffies.h:10,
                 from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/ktime.h:25,
                 from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/timekeeping.h:7,
                 from /home/nmhn/Downloads/linux-6.13-rc2/include/linux/dma-fence.h:24,
                 from ././include/linux/dma-resv.h:43,
                 from amd/amdkcl/dma-buf/dma-resv.c:36:
/home/nmhn/Downloads/linux-6.13-rc2/include/linux/alloc_tag.h:229:2: error: expected identifier or ‘(’ before ‘{’ token
  229 | ({                                                                                                      \
      |  ^
/home/nmhn/Downloads/linux-6.13-rc2/include/linux/slab.h:1076:49: note: in expansion of macro ‘alloc_hooks’
 1076 | #define kvrealloc(...)                          alloc_hooks(kvrealloc_noprof(__VA_ARGS__))
      |                                                 ^~~~~~~~~~~

there was a lot of error like the error above , almost all about the ( identifier

@jon-bit
Copy link

jon-bit commented Dec 13, 2024

@peppergrayxyz Is their a way to get Direct x 12 working with this? I would love to play my favorite games on a a compartmentalized VM.

@peppergrayxyz
Copy link
Author

@jon-bit not that I'm aware of yet, but there have been initiatives based on OpenGL (virtio-win/kvm-guest-drivers-windows#841) which look promising. I'll try to get an overview myself atm, I will add the links!

@jon-bit
Copy link

jon-bit commented Dec 15, 2024

@peppergrayxyz I found these in the mason build:

option(
  'gallium-d3d12-video',
  type : 'feature',
  value : 'auto',
  deprecated: {'true': 'enabled', 'false': 'disabled'},
  description : 'build gallium d3d12 with video support.',
)

option(
  'gallium-d3d12-graphics',
  type : 'feature',
  value : 'auto',
  description : 'build gallium d3d12 with graphics pipeline support.',
)

meson
option(
  'gallium-drivers',
  type : 'array',
  value : ['auto'],
  choices : [
    'auto', 'radeonsi', 'r300', 'r600', 'nouveau', 'freedreno',
    'swrast', 'v3d', 'vc4', 'etnaviv', 'tegra', 'i915', 'svga', 'virgl',
    'panfrost', 'iris', 'lima', 'zink', 'd3d12', 'asahi', 'crocus', 'all',
    'softpipe', 'llvmpipe',
  ],
  description : 'List of gallium drivers to build. If this is set to auto ' +
                'all drivers applicable to the target OS/architecture ' +
                'will be built.'
)

(Don't know if they are releted but hey.)
and here they have RE8 and others running on venus:
https://www.youtube.com/watch?v=HmyQqrS09eo
https://www.youtube.com/watch?v=Vk6ux08UDuA
https://www.youtube.com/watch?v=icCxskFVapU

Any clue how they did it?

@kerriganx
Copy link

I need help. When I launch vkcube, it shows MESA-VIRTIO: debug: stuck in fence wait with iter at error. The cube is static and not moving.
Screenshot_20241216_212222

@niansa
Copy link

niansa commented Dec 18, 2024

Unfortunately I'm only getting

error: kvm run failed Bad address
RAX=0000000000000000 RBX=00007ffcca3ae490 RCX=0000000000000000 RDX=00007fd754f2dff0
RSI=0000000000000000 RDI=0000000000000000 RBP=0000000000000500 RSP=00007ffcca3ae240
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000010 R13=00000000000003c0 R14=000055fe4d951a80 R15=00007fd754f2dfd0
RIP=00007fd765aeefcc RFL=00000206 [-----P-] CPL=3 II=0 A20=1 SMM=0 HLT=0
ES =0000 0000000000000000 00000000 00000000
CS =0033 0000000000000000 ffffffff 00a0fb00 DPL=3 CS64 [-RA]
SS =002b 0000000000000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
DS =0000 0000000000000000 00000000 00000000
FS =0000 00007fd765f05b00 00000000 00000000
GS =0000 0000000000000000 00000000 00000000
LDT=0000 0000000000000000 00000000 00000000
TR =0040 fffffe6e8bc34000 00004087 00008b00 DPL=0 TSS64-busy
GDT=     fffffe6e8bc32000 0000007f
IDT=     fffffe0000000000 00000fff
CR0=80050033 CR2=00007fd7557f9eb0 CR3=00000001075d8000 CR4=000006f0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d01
Code=f3 41 0f 11 07 f3 0f 11 0a 4c 01 e2 f3 0f 11 0a 66 0f ef c9 <f3> 42 0f 11 04 22 8d 14 08 44 01 e8 66 0f ef c0 01 c8 01 d2 01 c0 f3 0f 2a ca 8b 54 24 50

as soon as the desktop envrionment starts :-(

Any ideas? Followed the tutorial 1:1

@peppergrayxyz
Copy link
Author

@nmhn I set up an Ubuntu machine and tested it. I needed to build virglrenderer from source, because the package from Ubuntu was not built with Vulkan support. As for the build errors of the kernel: I just installed from mainline. I added my steps to the description!

@peppergrayxyz
Copy link
Author

@kerriganx I haven't seen that error yet, but this issue #25862120 points towards virglrenderer. Try building virglrenderer from source, I added steps for ubuntu!

@peppergrayxyz
Copy link
Author

@jon-bit If you use Steam on Linux, it will use Proton to translate DirectX to Vulkan. So you can set up containerized games if you want with this setup!

@kerriganx
Copy link

kerriganx commented Dec 18, 2024

@kerriganx I haven't seen that error yet, but this issue #25862120 points towards virglrenderer. Try building virglrenderer from source, I added steps for ubuntu!

Should I do it for host or guest? On host I build virglrenderer-git from AUR since I use arch (btw). According to PKGBUILD there is support for venus -Dvenus=true. Unfortunately, it didn't help. Same error

@peppergrayxyz
Copy link
Author

On the host, nothing to do on the guest!

It works for me with linux 6.13.0-rc2-1 and virglrenderer 1.0.1-2 (extra), I'm also on Arch (btw). I'll try to spin up a fresh install later, maybe try the same!

@peppergrayxyz
Copy link
Author

@niansa nvidia gpu? Any further details on your setup?

@jon-bit
Copy link

jon-bit commented Dec 19, 2024

@peppergrayxyz Sorry for the late replay. I've had finals. Anyway, I tried RE8, Persona 5 royal, and Myst (the newer one), and none worked. P5R just hung, RE8 crashes and Myst says it needs DX12. Any idea what is wrong or if I should do something different?

@jon-bit
Copy link

jon-bit commented Dec 20, 2024

@peppergrayxyz I have an update. When I set my DXVK HUD to be used it it NOT using venus on lighter games that do start like Henry stick man. It says in the driver section it is using llvmpipe, not venus. Is their any way to select GPU and/or driver?

@rapidshmo
Copy link

@peppergrayxyz I need help. I get the same error as kerriganx when running vkcube. Im on arch linux-mainline 6.13.0-rc3-1 and virglrenderer 1.0.1-2
image

@xvim
Copy link

xvim commented Dec 23, 2024

I can only run it successfully on Intel integrated graphics card, with amd or nvidia GPU,also get error when launch vkcube

@hjolli
Copy link

hjolli commented Dec 23, 2024

I got this to work on fedora rawhide as the host and ubuntu 23.10 as the guest. Fedora rawhide has all the relevant packages in the repos now
system: AMD ryzen 7 3900x and Radeon rx 5500 gpu

@thesword53
Copy link

thesword53 commented Dec 25, 2024

@peppergrayxyz I need help. I get the same error as kerriganx when running vkcube. Im on arch linux-mainline 6.13.0-rc3-1 and virglrenderer 1.0.1-2 image

I have a similar issue but the window is black
vkcube_venus_nvidia

I'm using Nvidia proprietary drivers on host (tested with GTX 1060 and RTX 2080 SUPER)

I also have those errors on host:

[  922.039626] NVRM: GPU at PCI:0000:2d:00: GPU-85142072-d857-6520-e4b7-8d06bf8e4a0d
[  922.039632] NVRM: Xid (PCI:0000:2d:00): 69, pid='<unknown>', name=<unknown>, Class Error: ChId 00ac, Class 0000c597, Offset 00000274, Data 00000024, ErrorCode 0000009c

Note: If your are using Nvidia drivers < 565, you should use this patch https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1458 and for reasons I ignore (NVIDIA/open-gpu-kernel-modules#748) the patch is also needed for Turing GPUs even with 565 drivers.

Edit: Some other Vulkan apps such as vkgears or vkmark crash with Illegal instruction and no error on host

@Tony763
Copy link

Tony763 commented Dec 28, 2024

Hi, for me, installing kernel 6.13-rc1 fail on nvidia 565.77 dkms (Ubuntu 24.10).

Fail on /usr/bin/ld.bfd: cannot find ./nvidia/nv-kernel.o: Directory or file not found (it is symlink to binary file which exist)

unset ARCH; [ ! -h /usr/bin/cc ] && export CC=/usr/bin/gcc; env NV_VERBOSE=1 'make' -j16 NV_EXCLUDE_BUILD_MODULES='' KERNEL_UNAME=6.13.0-061300rc1-generic IGNORE_XEN_PRESENCE=1 IGNORE_CC_MISMATCH=1 SYSSRC=/lib/modules/6.13.0-061300rc1-generic/build LD=/usr/bin/ld.bfd CONFIG_X86_KERNEL_IBT= modules............(bad exit status: 2)
ERROR (dkms apport): kernel package linux-headers-6.13.0-061300rc1-generic is not supported
Error! Bad return status for module build on kernel: 6.13.0-061300rc1-generic (x86_64)
Consult /var/lib/dkms/nvidia/565.77/build/make.log for more information.

@lion328
Copy link

lion328 commented Dec 30, 2024

I got the same error on libvirt. Messing around with virgl log settings gave me failed to fork proxy server. Solved the issue by disabling seccomp_sandbox in /etc/libvirt/qemu.conf since it was blocking fork syscalls.

My setup:

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
  <memoryBacking>
    <source type="memfd"/>
  </memoryBacking>
  <devices>
    <graphics type="spice">
      <listen type="none"/>
      <image compression="off"/>
      <gl enable="yes"/>
    </graphics>
    <video>
      <model type="virtio" heads="1" primary="yes" blob="on">
        <acceleration accel3d="yes"/>
      </model>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    ...
  </devices>
  <qemu:override>
    <qemu:device alias="video0">
      <qemu:frontend>
        <qemu:property name="venus" type="bool" value="true"/>
        <qemu:property name="hostmem" type="unsigned" value="1073741824"/>
      </qemu:frontend>
    </qemu:device>
  </qemu:override>
  ...
</domain>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment