Skip to content

Instantly share code, notes, and snippets.

@OlfillasOdikno

OlfillasOdikno/Linux_GPU_PV.md

Last active January 9, 2025 13:53
Show Gist options
  • Save OlfillasOdikno/f87a4444f00984625558dad053255ace to your computer and use it in GitHub Desktop.
Save OlfillasOdikno/f87a4444f00984625558dad053255ace to your computer and use it in GitHub Desktop.
Download ZIP
Hyperv Linux Guest GPU PV
Raw
Linux_GPU_PV.md

Hyperv Linux Guest GPU PV

  • Create VM
$isopath = <iso location>
$vhdpath = <vhdx location>
$vmpath = <vm path>
$vmname = "Arch-dxgkrnl"
New-VM -Name $vmname -MemoryStartupBytes 8GB -BootDevice VHD -NewVHDPath $vhdpath -Path $vmpath -NewVHDSizeBytes 20GB -Generation 2 -Switch "Default Switch"
Set-VM -Name  $vmname -CheckpointType Disabled
Set-VMMemory $vmname -DynamicMemoryEnabled $false
Add-VMDvdDrive -VMName  $vmname -Path $isopath
Set-VMFirmware -VMName $vmname -EnableSecureBoot Off -FirstBootDevice (Get-VMDvdDrive -VMName $vmname)[0]
Add-VMGpuPartitionAdapter -VMName $vmname
Set-VMGpuPartitionAdapter -VMName $vmname -MinPartitionVRAM 1
Set-VMGpuPartitionAdapter -VMName $vmname -MaxPartitionVRAM 11
Set-VMGpuPartitionAdapter -VMName $vmname -OptimalPartitionVRAM 10
Set-VMGpuPartitionAdapter -VMName $vmname -MinPartitionEncode 1
Set-VMGpuPartitionAdapter -VMName $vmname -MaxPartitionEncode 11
Set-VMGpuPartitionAdapter -VMName $vmname -OptimalPartitionEncode 10
Set-VMGpuPartitionAdapter -VMName $vmname -MinPartitionDecode 1
Set-VMGpuPartitionAdapter -VMName $vmname -MaxPartitionDecode 11
Set-VMGpuPartitionAdapter -VMName $vmname -OptimalPartitionDecode 10
Set-VMGpuPartitionAdapter -VMName $vmname -MinPartitionCompute 1
Set-VMGpuPartitionAdapter -VMName $vmname -MaxPartitionCompute 11
Set-VMGpuPartitionAdapter -VMName $vmname -OptimalPartitionCompute 10
Set-VM -GuestControlledCacheTypes $true -VMName $vmname
Set-VM -LowMemoryMappedIoSpace 1Gb -VMName $vmname
Set-VM -HighMemoryMappedIoSpace 32GB -VMName $vmname
Start-VM -Name $vmname

  • install arch with sway the normal way

  • install dependencies

sudo pacman -S git base-devel

git clone https://aur.archlinux.org/yay.git && cd yay && makepkg -si
  • start sshd
sudo systemctl start sshd
  • for nvidia:
mkdir drivers

WSLAttachSwitch.exe "Default Switch"

do now in WSL

sudo dhclient eth1
  • copy proretary files
scp -r /usr/lib/wsl/lib arch-dxgkrnl:~/
  • for nvidia:
ssh arch-dxgkrnl sudo -S mkdir -p $(echo /usr/lib/wsl/drivers/nvle.inf_amd64_*/)
scp -r /usr/lib/wsl/drivers/nvle.inf_amd64_*/*.so* arch-dxgkrnl:~/drivers

ssh arch-dxgkrnl

sudo mv lib/* /usr/lib
sudo ln -s /lib/libd3d12core.so /lib/libD3D12Core.so
  • for nvidia:
sudo cp -r drivers/* /usr/lib/wsl/drivers/nvle.inf_amd64_*
  • install dxg dkms
cd ~
git clone https://github.com/OlfillasOdikno/dxgkrnl-dkms.git && cd dxgkrnl-dkms && makepkg -si

sudo modprobe dxgkrnl

/dev/dxg should now exist

  • install mesa-d3d12
yay -S mesa-d3d12 mesa-utils xorg-xwayland

Now in hyperv:

export MESA_LOADER_DRIVER_OVERRIDE=d3d12
export WLR_RENDERER_ALLOW_SOFTWARE=1
LIBGL_ALWAYS_SOFTWARE=1 sway

Now execute glxinfo in sway:

LIBGL_ALWAYS_SOFTWARE=0 glxinfo -B

It should show your graphics card

name of display: :0
NVD3D10: CPU cyclestats are disabled on client virtualization
NVD3D10: CPU cyclestats are disabled on client virtualization
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Microsoft Corporation (0xffffffff)
    Device: D3D12 (NVIDIA GeForce RTX 3060) (0xffffffff)       <----- here
    Version: 21.2.5
    Accelerated: yes
    Video memory: 28458MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 3.3
    Max compat profile version: 3.1
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.0
OpenGL vendor string: Microsoft Corporation
OpenGL renderer string: D3D12 (NVIDIA GeForce RTX 3060)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 21.2.5
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 3.1 Mesa 21.2.5
OpenGL shading language version string: 1.40
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.0 Mesa 21.2.5
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.00
@OlfillasOdikno
Copy link
Author

OlfillasOdikno commented Dec 7, 2021 •
edited
Loading

Was tested on Windows 11 Host Build 22000, Guest: Archlinux 5.15.6

@Skyyblaze
Copy link

Thanks for the guide! I'm a total newbie to Hyper-V and WSL(2), do I need a pre-existing Arch WSL2 installation for this to work? I only have Ubuntu installed in WSL2.

@FreddyBLtv
Copy link

Thanks for the guide! I'm a total newbie to Hyper-V and WSL(2), do I need a pre-existing Arch WSL2 installation for this to work? I only have Ubuntu installed in WSL2.

Did it work?

@FreddyBLtv
Copy link

I can't find the "nvle.inf_amd64_" folder in wsl. How did you get the file?

@OlfillasOdikno
Copy link
Author

My path is /usr/lib/wsl/drivers/nvle.inf_amd64_85ad71859ca3102f

@luc-git
Copy link

luc-git commented Apr 19, 2022

I think It can be better to just ask ask to add dxgkrnl modules to the torvalds linux kernel.
But how to do this? I wanted to do it but issues are disabled in the linux kernel repo so I don't know how to ask

@TaylorTWBrown
Copy link

Should this method more or less work for regular Hyper-V Linux guest VMs? I'm interested in sharing my Intel iGPU with an Ubuntu guest, and have been playing around with GPU-PV this week.

I'll see how far I can get and try to report back.

@max06
Copy link

max06 commented Jun 12, 2022

Highly interested in this, gonna try myself

@seflerZ
Copy link

seflerZ commented Jul 5, 2022

I'm using Ubuntu 22.04, there is no makepkg command, How can I compile the dxgkrnl?

@TylerJaacks
Copy link

I have two issues 1. /usr/lib/wsl/drivers/nvle.inf_amd64_* doesn't exist on my WSL2 vm. What WSL2 vm are u using? 2.

:: Running post-transaction hooks...
(1/2) Arming ConditionNeedsUpdate...
(2/2) Install DKMS modules
==> dkms install --no-depmod dxgkrnl/5.6.rc2.r80230.gd489414c23af -k 5.19.5-arch1-1
Error! Bad return status for module build on kernel: 5.19.5-arch1-1 (x86_64)
Consult /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/make.log for more information.
==> WARNING: `dkms install --no-depmod dxgkrnl/5.6.rc2.r80230.gd489414c23af -k 5.19.5-arch1-1' exited 10

after running git clone https://github.com/OlfillasOdikno/dxgkrnl-dkms.git && cd dxgkrnl-dkms && makepkg -si

DKMS make.log for dxgkrnl-5.6.rc2.r80230.gd489414c23af for kernel 5.19.5-arch1-1 (x86_64)
Mon Aug 29 10:29:56 PM CDT 2022
make: Entering directory '/usr/lib/modules/5.19.5-arch1-1/build'
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgmodule.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/hmgr.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/misc.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.o
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.c: In function ‘dxgallocation_destroy’:
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.c:934:66: warning: passing argument 2 of ‘vmbus_teardown_gpadl’ makes pointer from integer without a cast [-Wint-conversion]
  934 |                 vmbus_teardown_gpadl(dxgglobal_get_vmbus(), alloc->gpadl);
      |                                                             ~~~~~^~~~~~~
      |                                                                  |
      |                                                                  u32 {aka unsigned int}
In file included from /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.c:15:
./include/linux/hyperv.h:1226:58: note: expected ‘struct vmbus_gpadl *’ but argument is of type ‘u32’ {aka ‘unsigned int’}
 1226 |                                      struct vmbus_gpadl *gpadl);
      |                                      ~~~~~~~~~~~~~~~~~~~~^~~~~
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/ioctl.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.o
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c: In function ‘dxgvmbuschannel_receive’:
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:389:20: error: implicit declaration of function ‘hv_pkt_iter_first_raw’; did you mean ‘hv_pkt_iter_first’? [-Werror=implicit-function-declaration]
  389 |        for (desc = hv_pkt_iter_first_raw(channel->channel); desc; desc = hv_pkt_iter_next(channel->channel, desc)) {
      |                    ^~~~~~~~~~~~~~~~~~~~~
      |                    hv_pkt_iter_first
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:389:18: warning: assignment to ‘struct vmpacket_descriptor *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  389 |        for (desc = hv_pkt_iter_first_raw(channel->channel); desc; desc = hv_pkt_iter_next(channel->channel, desc)) {
      |                  ^
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c: In function ‘create_existing_sysmem’:
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:1425:53: error: passing argument 4 of ‘vmbus_establish_gpadl’ from incompatible pointer type [-Werror=incompatible-pointer-types]
 1425 |                                         alloc_size, &dxgalloc->gpadl);
      |                                                     ^~~~~~~~~~~~~~~~
      |                                                     |
      |                                                     u32 * {aka unsigned int *}
In file included from /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:18:
./include/linux/hyperv.h:1223:59: note: expected ‘struct vmbus_gpadl *’ but argument is of type ‘u32 *’ {aka ‘unsigned int *’}
 1223 |                                       struct vmbus_gpadl *gpadl);
      |                                       ~~~~~~~~~~~~~~~~~~~~^~~~~
cc1: some warnings being treated as errors
make[1]: *** [scripts/Makefile.build:249: /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.o] Error 1
make: *** [Makefile:1847: /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build] Error 2
make: Leaving directory '/usr/lib/modules/5.19.5-arch1-1/build'
cat: for: No such file or directory
cat: more: No such file or directory
cat: information.: No such file or directory

@FreddyBLtv
Copy link

I have two issues 1. /usr/lib/wsl/drivers/nvle.inf_amd64_* doesn't exist on my WSL2 vm. What WSL2 vm are u using? 2.

:: Running post-transaction hooks...
(1/2) Arming ConditionNeedsUpdate...
(2/2) Install DKMS modules
==> dkms install --no-depmod dxgkrnl/5.6.rc2.r80230.gd489414c23af -k 5.19.5-arch1-1
Error! Bad return status for module build on kernel: 5.19.5-arch1-1 (x86_64)
Consult /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/make.log for more information.
==> WARNING: `dkms install --no-depmod dxgkrnl/5.6.rc2.r80230.gd489414c23af -k 5.19.5-arch1-1' exited 10

after running git clone https://github.com/OlfillasOdikno/dxgkrnl-dkms.git && cd dxgkrnl-dkms && makepkg -si

DKMS make.log for dxgkrnl-5.6.rc2.r80230.gd489414c23af for kernel 5.19.5-arch1-1 (x86_64)
Mon Aug 29 10:29:56 PM CDT 2022
make: Entering directory '/usr/lib/modules/5.19.5-arch1-1/build'
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgmodule.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/hmgr.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/misc.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.o
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.c: In function ‘dxgallocation_destroy’:
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.c:934:66: warning: passing argument 2 of ‘vmbus_teardown_gpadl’ makes pointer from integer without a cast [-Wint-conversion]
  934 |                 vmbus_teardown_gpadl(dxgglobal_get_vmbus(), alloc->gpadl);
      |                                                             ~~~~~^~~~~~~
      |                                                                  |
      |                                                                  u32 {aka unsigned int}
In file included from /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgadapter.c:15:
./include/linux/hyperv.h:1226:58: note: expected ‘struct vmbus_gpadl *’ but argument is of type ‘u32’ {aka ‘unsigned int’}
 1226 |                                      struct vmbus_gpadl *gpadl);
      |                                      ~~~~~~~~~~~~~~~~~~~~^~~~~
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/ioctl.o
  CC [M]  /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.o
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c: In function ‘dxgvmbuschannel_receive’:
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:389:20: error: implicit declaration of function ‘hv_pkt_iter_first_raw’; did you mean ‘hv_pkt_iter_first’? [-Werror=implicit-function-declaration]
  389 |        for (desc = hv_pkt_iter_first_raw(channel->channel); desc; desc = hv_pkt_iter_next(channel->channel, desc)) {
      |                    ^~~~~~~~~~~~~~~~~~~~~
      |                    hv_pkt_iter_first
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:389:18: warning: assignment to ‘struct vmpacket_descriptor *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  389 |        for (desc = hv_pkt_iter_first_raw(channel->channel); desc; desc = hv_pkt_iter_next(channel->channel, desc)) {
      |                  ^
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c: In function ‘create_existing_sysmem’:
/var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:1425:53: error: passing argument 4 of ‘vmbus_establish_gpadl’ from incompatible pointer type [-Werror=incompatible-pointer-types]
 1425 |                                         alloc_size, &dxgalloc->gpadl);
      |                                                     ^~~~~~~~~~~~~~~~
      |                                                     |
      |                                                     u32 * {aka unsigned int *}
In file included from /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.c:18:
./include/linux/hyperv.h:1223:59: note: expected ‘struct vmbus_gpadl *’ but argument is of type ‘u32 *’ {aka ‘unsigned int *’}
 1223 |                                       struct vmbus_gpadl *gpadl);
      |                                       ~~~~~~~~~~~~~~~~~~~~^~~~~
cc1: some warnings being treated as errors
make[1]: *** [scripts/Makefile.build:249: /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build/dxgvmbus.o] Error 1
make: *** [Makefile:1847: /var/lib/dkms/dxgkrnl/5.6.rc2.r80230.gd489414c23af/build] Error 2
make: Leaving directory '/usr/lib/modules/5.19.5-arch1-1/build'
cat: for: No such file or directory
cat: more: No such file or directory
cat: information.: No such file or directory

Did you get this working?

@seflerZ
Copy link

seflerZ commented Dec 9, 2022

I've made an one-click installation here

@FreddyBLtv
Copy link

I got this error

"The partitionable GPU device
'pci#ven_10de&dev_1c02&subsys_37241458&rev_a1#4&1c3d25bb&0&0019' was not found on server 'WINDOWS'.
At C:\Users\Freddy\Documents\oneclick-gpu-pv\ubuntu-gpu-pv.ps1:25 char:1

  • Add-VMGpuPartitionAdapter $vmName -InstancePath "$path"
  •   + CategoryInfo          : InvalidArgument: (:) [Add-VMGpuPartitionAdapter], VirtualizationException
  + FullyQualifiedErrorId : InvalidParameter,Microsoft.HyperV.PowerShell.Commands.AddVMGpuPartitionAdapter"

@seflerZ
Copy link

seflerZ commented Dec 10, 2022

@FreddyBLtv It seems this device was not found. You can use Get-VMHostPartitionableGpu in powershell width administrator privilege to check. Look for the entry "Name". If you did found that device. You use Add-VMGpuPartitionAdapter $VM_NAME -instancePath $NAME_FOUND_IN_LAST_COMMAND to add it manually.

@FreddyBLtv
Copy link

@FreddyBLtv It seems this device was not found. You can use Get-VMHostPartitionableGpu in powershell width administrator privilege to check. Look for the entry "Name". If you did found that device. You use Add-VMGpuPartitionAdapter $VM_NAME -instancePath $NAME_FOUND_IN_LAST_COMMAND to add it manually.

Yeah I got it to work but the graphics card is not showing up in the Ubuntu VM. How do you have ubuntu setup in hyper-v?

@seflerZ
Copy link

seflerZ commented Dec 23, 2022 •
edited
Loading

@FreddyBLtv You can check this script. It's a bit complex, but it did work.

@FreddyBLtv
Copy link

FreddyBLtv commented Dec 28, 2022 •
edited
Loading

@FreddyBLtv You can check this script. It's a bit complex, but it did work.

This is what I get after running the script

Screenshot

@txj-xyz
Copy link

txj-xyz commented Jul 29, 2024

looks like wsl2 doesnt work anymore for nvle.* directory

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment