Skip to content

Instantly share code, notes, and snippets.

@Nexarian
Last active April 30, 2024 00:39
Show Gist options
  • Star 37 You must be signed in to star a gist
  • Fork 17 You must be signed in to fork a gist
  • Save Nexarian/0eb26a3284b21b55b6e1e8653ed88ec9 to your computer and use it in GitHub Desktop.
Save Nexarian/0eb26a3284b21b55b6e1e8653ed88ec9 to your computer and use it in GitHub Desktop.
Setup for XRDP using Nvidia Acceleration
#!/usr/bin/env bash
set -e
cd ~
sudo -v
# Make sure system is in a good, updated, clean, state.
sudo apt-get -y update
sudo apt-get -y dist-upgrade
sudo apt-get -y autoremove
# Install CUDA (Shamelessly copied from Nvidia's website):
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda
sudo usermod $USER -a -G video
sudo usermod $USER -a -G tty
sudo usermod $USER -a -G render
# XRDP Build Pre-reqs Part 1
sudo apt-get -y install git autoconf libtool pkg-config gcc g++ make \
libssl-dev libpam0g-dev libjpeg-dev libx11-dev libxfixes-dev libxrandr-dev \
flex bison libxml2-dev intltool xsltproc xutils-dev python3-libxml2 g++ \
xutils libfuse-dev libmp3lame-dev nasm libpixman-1-dev xserver-xorg-dev \
libjson-c-dev libsndfile1-dev libspeex-dev libspeexdsp-dev libpulse-dev \
libpulse0 autopoint \*turbojpeg\* libfdk-aac-dev libopus-dev libgbm-dev \
libx264\* libx264-dev
#XRDP Build Pre-reqs Part 2 (For some reason apt needs this to be separate)
sudo apt-get -y install libepoxy-dev
#System Pre-reqs
sudo apt-get -y install xorg xserver-xorg-legacy dbus-x11
sudo apt-get -y remove dbus-user-session
sudo tee /etc/X11/Xwrapper.config > /dev/null << EOL
# Xwrapper.config (Debian X Window System server wrapper configuration file)
#
# This file was generated by the post-installation script of the
# xserver-xorg-legacy package using values from the debconf database.
#
# See the Xwrapper.config(5) manual page for more information.
#
# This file is automatically updated on upgrades of the xserver-xorg-legacy
# package *only* if it has not been modified since the last upgrade of that
# package.
#
# If you have edited this file but would like it to be automatically updated
# again, run the following command as root:
# dpkg-reconfigure xserver-xorg-legacy
needs_root_rights=no
allowed_users=anybody
EOL
echo "gnome-session --session=ubuntu" > ~/.xsession
sudo chmod +x ~/.xsession
sudo tee ~/.xsessionrc > /dev/null << EOL
export DESKTOP_SESSION=ubuntu
export GNOME_SHELL_SESSION_MODE=ubuntu
export XDG_CURRENT_DESKTOP=ubuntu:GNOME
EOL
sudo chmod +x ~/.xsessionrc
BUILD_DIR=$(mktemp -d)
echo "Building xrdp..."
git clone https://github.com/Nexarian/xrdp.git --branch mainline_merge "$BUILD_DIR/xrdp"
cd "$BUILD_DIR/xrdp"
./bootstrap
./configure --enable-fuse --enable-rfxcodec --enable-pixman --enable-mp3lame \
--enable-sound --enable-opus --enable-fdkaac --enable-x264 --enable-nvenc
make -j $(nproc) clean all
sudo make install
echo "Building xorgxrdp..."
git clone https://github.com/Nexarian/xorgxrdp.git --branch mainline_merge "$BUILD_DIR/xorgxrdp"
cd "$BUILD_DIR/xorgxrdp"
./bootstrap
./configure --with-simd --enable-lrandr
make -j $(nproc) clean all
sudo make install
echo "Configuring Nvidia..."
BUS_ID=$(nvidia-smi --query-gpu=pci.bus --format=csv | sed -n '2 p' | xargs -I{} printf "%d\n" {})
# Alternative method to get the BUS ID that doesn't work on some systems (Such as a laptop with an Intel iGPU and an Nvidia Discrete GPU).
# BUS_ID=$(lspci | grep -e 'VGA.*NVIDIA' | cut -d: -f1 | xargs -I{} printf "%d\n" 0x{} | sed -n 1p)
# nvidia-xconfig --query-gpu-info
sudo sed -i -E 's/(BusID "PCI:)[[:digit:]]+(:0:0")/\1'$BUS_ID'\2/' /etc/X11/xrdp/xorg_nvidia.conf
echo "Starting the server..."
sudo systemctl enable xrdp
sudo service xrdp start
@Nexarian
Copy link
Author

@Lyncredible It's almost certainly the "LRandR" functionality. In order for Nvidia to work, @jsorg71 had to override the default Resize and Rotate (RandR) implementation. However, not all functions were fully implemented. This is in xorgxrdp, and some of the functions were not implemented fully yet (just the basics). Here's where the list of implemented functions is located:

https://github.com/Nexarian/xorgxrdp/blob/3d30c7a6ad4f4a582efcb919966d8f1508aa2d31/module/rdpLRandR.c#L1074C15-L1074C15

@Lyncredible
Copy link

@Nexarian Thanks for the pointer. This definitely looks super related. I will take a look in the coming weeks.

@dunkleyr
Copy link

I tried the script: Do you have any solution for this error image

@k1moradi, I get that protocol error message if I select something other than 32-bit color depth on the client side.

@bucde
Copy link

bucde commented Nov 22, 2023

Hi @Nexarian !

First: thanks for the awesome work, to get all this running! This brings me a lot more to a full remote workstation.

But I have a really strange behavior... First let me explain my setup:

  • I have two systems with the same behavior:
    1. Machine: AMD EPYC with NVIDIA RTX 3070
    2. Machine: AMD Ryzen 5950X with NVIDIA T400
  • Both running Proxmox VE
  • Both have a VM with clean Debian 12 installed with XFCE with PCIe pass-through of the named GPUs (no monitors connected to the GPUs)
  • After installing Debian, I directly ran your script with the addition to install nvidia-driver before cuda (my script version)
  • I connect to these VMs via remmina (RDP) (default settings) from a Ubuntu 20.04 and from Debian 12.
  • After rebooting the VM and checking with nvidia-smi that the card is in use nearly everything works fine!
    • Playing YouTube videos in firefox works perfectly with hardware acceleration, and some other stuff I tried like Superposition benchmark works also as expected.
  • Then I tried FreeCAD...
    • NOW things break!
      • I create a new CAD model, go to the part workbench and start a sketch.
      • When I choose e.g. a circle and cross the coordinate system axis the RDP connection immediately freezes and after maybe 5 seconds the connection closes without any message!
      • When I reconnect then everything is still there and also no (warning)message.
      • I have then directly to press ESC to abort the circle drawing to get control back via RDP.
      • This happens on both systems.
    • I also saw this behavior on some special webpages (in firefox) with a lot and fancy stuff on it.

IMO there are some 'special' calls to the 'graphics library' (I am really not an expert on this!) which causes this.

I tried to do some research via logfiles, but there was nothing noticeable for me. (again, I am not an expert on xrdp).

One (maybe) important thing I tried:
On a VM (on the same system (i)) without PCIe pass-through a GPU I installed clean Debian with the apt-version of xrdp and when I try there FreeCAD as described above, I have no problems! The session runs flawlessly.

I hope, someone can help me.
Because with this issues it is not that usable for me.

Thanks in advance!

@EduardDurech
Copy link

EduardDurech commented Nov 23, 2023

I tried the script: Do you have any solution for this error image

@k1moradi, I get that protocol error message if I select something other than 32-bit color depth on the client side.

Confirming the same error

  * Playing YouTube videos in firefox works perfectly with hardware acceleration, and some other stuff I tried like [Superposition benchmark](https://benchmark.unigine.com/superposition) works also as expected.

Interestingly, Firefox is not working for me. It works on the regular Xorg server but not with this. GPU's are working with Xorg, so thank you very much @Nexarian (and jsorg71), but I noticed you also mentioned above Firefox doesn't work due to Display, was this potentially fixed?

@Nexarian
Copy link
Author

@Nexarian
Copy link
Author

@bucde This is a very complex setup! Unfortunately, I haven't really been able to validate the GPU pass-through capabilities for video cards yet, maybe I should try replicating your setup. I usually develop on Windows systems using VS Code to SSH into a Linux system, and I haven't found a good way, using Windows as the OS host, to virtualize a GPU to a Linux VM with VirtualBox or HyperV or WSL. I should probably get a server going that can do this.

The most interesting thing to me is:
When I reconnect then everything is still there and also no (warning)message.

The special drawing commands (as I mentioned before with the LRandR problem) are likely to blame, but the fact that the use of these doesn't kill off the session is interesting. A special drawing command should kill this off. What might be happening is instead of crashing it's just failing gracefully.

Can you link to logs for ~/.xorgxrdp.10.log (Or whatever your display for the session is), ~/.local/share/xrdp/xorgxrdp_helper.10.log, and /var/log/xrdp.log -- Don't post them here, put them in a gist (or equivalent system intended to store large volumes of logging text) so we don't clutter up the thread.

@EduardDurech
Copy link

@EduardDurech Try this https://stackoverflow.com/a/784415

@Nexarian It didn't seem to work, thanks for your help though, don't want to distract you with debugging, but if you also faced this when changing DISPLAY then calling Firefox: /user.slice/user-232927.slice/session-c3.scope is not a snap cgroup, from searching online I need to diasble cgroups v2, which I did using the instructions here https://kb.nomachine.com/AR06T01163 still doesn't work though

@EduardDurech
Copy link

Oddly enough, sudo firefox works without any need to edit DISPLAY but with many errors (seemingly with permissions). This seems to be a snap problem, however, it did work without issue on the normal Xorg, not sure if there's a conflict with the Nvidia-hack

@bucde
Copy link

bucde commented Nov 25, 2023

@bucde This is a very complex setup! Unfortunately, I haven't really been able to validate the GPU pass-through capabilities for video cards yet, maybe I should try replicating your setup. I usually develop on Windows systems using VS Code to SSH into a Linux system, and I haven't found a good way, using Windows as the OS host, to virtualize a GPU to a Linux VM with VirtualBox or HyperV or WSL. I should probably get a server going that can do this.

I can try to setup a clean Debian 12 directly on one system without virtualization (Proxmox VE) and try the exact same.

The most interesting thing to me is: When I reconnect then everything is still there and also no (warning)message.

The special drawing commands (as I mentioned before with the LRandR problem) are likely to blame, but the fact that the use of these doesn't kill off the session is interesting. A special drawing command should kill this off. What might be happening is instead of crashing it's just failing gracefully.

Can you link to logs for ~/.xorgxrdp.10.log (Or whatever your display for the session is), ~/.local/share/xrdp/xorgxrdp_helper.10.log, and /var/log/xrdp.log -- Don't post them here, put them in a gist (or equivalent system intended to store large volumes of logging text) so we don't clutter up the thread.

I have closed the RDP session and deleted (moved) the actual requested logfiles via SSH.
After that I restarted the VM and connected to it via RDP.
Then I started directly FreeCAD and done the same as described above (same behavior: session closes).
I reconnected via RDP and close FreeCAD (press ESC).
After that I close the RDP session and copied the generated logfiles via SSH.

These I upload to my gitea server here (quick n dirty).

I hope there is something useful for you!

@Nexarian
Copy link
Author

The most interesting part of the logs you sent is this: https://git.bs.b-eit.de/bucde/github-xrdp-issue/src/branch/master/.xorgxrdp.10.log.test

There is no graceful shutdown. It's just... Gone. I suspect something is segfaulting (Likely Xorg itself, which still makes LRandR a likely culprit). Can you check the system logs for any segfaults that may happen at this time? The other two processes detect that the xorg server has shut down and do a graceful termination. Either use journalctl or look up syslog

@bucde
Copy link

bucde commented Nov 26, 2023

The most interesting part of the logs you sent is this: https://git.bs.b-eit.de/bucde/github-xrdp-issue/src/branch/master/.xorgxrdp.10.log.test

There is no graceful shutdown. It's just... Gone.

To be honest, I don't know exactly how you can see that.

I suspect something is segfaulting (Likely Xorg itself, which still makes LRandR a likely culprit). Can you check the system logs for any segfaults that may happen at this time? The other two processes detect that the xorg server has shut down and do a graceful termination. Either use journalctl or look up syslog

With journalctl there is nothing noticeable.

I also used htop (via SSH) to try to figure out the process(es) that are dying at this moment.
In the freeze moment no process is dying. Only when the RDP session closes, xrdp exits (see screenshot).
screenshot

I also tried sudo gdb /usr/local/sbin/xrdp <PID> with the actual running xrdp which exits with

Reading symbols from /usr/local/sbin/xrdp...
Attaching to program: /usr/local/sbin/xrdp, process 4211
[New LWP 4212]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f556871b05f in __GI___poll (fds=fds@entry=0x7ffcf29900e0, nfds=9, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
29	../sysdeps/unix/sysv/linux/poll.c: No such file or directory.
(gdb) cont
Continuing.
[Thread 0x7f5567bff6c0 (LWP 4212) exited]
[Inferior 1 (process 4211) exited normally]

I also tried to set loglevel in /etc/xrdp/xrdp.ini to

...
[Logging]
LogLevel=TRACE
...

but apart from the fact that everything was slower, there were no further findings or new error messages.

I'm really stuck on how to gather more information about the problem to even know where to look.
Can I generate more debug/log output at any point?
If how?

Thank you a lot for looking into my problem!

@EduardDurech
Copy link

@Nexarian I noticed you here neutrinolabs/xrdp#914 when I was looking for another issue, not sure if this is only for the NVIDIA-xrdp or xrdp entirely, but changing any resolution or refresh rate results in an error

:~$ xrandr
Screen 0: minimum 64 x 64, current 2560 x 1600, maximum 8192 x 8192
rdp0 connected 2560x1600+0+0 0mm x 0mm
   2560x1600     50.00*+
Changing refresh rate
:~$ xrandr --output rdp0 --rate 25
X Error of failed request:  BadLength (poly request too large or internal Xlib length error)
  Major opcode of failed request:  156 (RANDR)
  Minor opcode of failed request:  21 (RRSetCrtcConfig)
  Serial number of failed request:  21
  Current serial number in output stream:  21

It is continuously at 50Hz, even setting the registry https://learn.microsoft.com/en-us/troubleshoot/windows-server/remote/frame-rate-limited-to-30-fps to 5 decimal (20Hz) changes nothing after restart

You linked

https://github.com/Nexarian/xorgxrdp/blob/3d30c7a6ad4f4a582efcb919966d8f1508aa2d31/module/rdpLRandR.c#L1074C15-L1074C15

that not all RandR functions are supported yet, don't know if this is related

And trying to xrandr --newmode "[MODE]" [MODELINE] anything just hangs the terminal, using your comment from neutrinolabs/xrdp#448 (comment) though (I couldn't get dynamic resizing to work), using RDCMan to connect I can at least set a custom resolution then restart the server (but unfortunately this doesn't support multiple server displays)

Adding modes to /etc/X11/xrdp/xorg_nvidia.conf, /etc/X11/xrdp/xorg.conf, /usr/share/X11/xorg.conf.d/[CustomFile].conf, /etc/X11/xorg.conf doesn't seem to work either

Any advice for how I can connect and set custom resolutions per monitor and refresh rates? Normally, this would be done in the .conf per Screen-Display, but when the remote server is initiated it seems to only ever use 50Hz and the client-resolution (which also doesn't support per-display resolution in the .rdp). So right now I'm left with changing the resolution of my physical monitors, connecting to the server, and then reverting the physical monitors back, but still stuck on 50Hz 😄

@Nexarian
Copy link
Author

Nexarian commented Dec 1, 2023

@bucde I've been working on AVC444 mode, which is a lot more bandwidth heavy. When playing with this, I get a very similar behavior to your disconnect. What is the ping time to your server? If it's >> 30 ms, then the disconnect your seeing is going to be because the connection is just too slow.

@Nexarian
Copy link
Author

Nexarian commented Dec 1, 2023

@EduardDurech Unfortunately refresh rate isn't something I've been working on lately. That's not likely to be addressed until we move to wayland.

@alexanderzjs
Copy link

@Nexarian , Thanks for this awesome work, I have tested it with mainline_merge_branch for both xrdp and xorgxrdp. My testing environment is Arch Linux and I have Nvidia Quadro P400. I did not use any DE, but I have herbstluftwm. Here I will paste whatever I run on my Arch to make it success:

pacman -S nvidia: nvidia-smi shows 545.29.06 with CUDA 12.3

usermod alexander -a -G video
usermod alexander -a -G tty
echo "needs_root_rights=no" >> /etc/X11/Xwrapper.config
echo "allowed_users=anybody" >> /etc/X11/Xwrapper.config

pacman -S fuse2 nasm: need these dependencies to compile xrdp, maybe more are needed, but I only lack these two

git clone https://github.com/Nexarian/xrdp.git
cd xrdp && git checkout mainline_merge_backup
./bootstrap
./configure --enable-fuse --enable-rfxcodec --enable-pixman --enable-mp3lame --enable-sound --enable-opus --enable-fdkaac --enable-x264 --enable-nvenc
make -j8 clean all
sudo make install

pacman -S xorg-server-devel: Need this to compile xorgxrdp

git clone https://github.com/Nexarian/xorgxrdp.git
cd xorgxrdp && git checkout mainline_merge_backup
./bootstrap
XRDP_CFLAGS="-I/path/to/git/cloned/xrdp/common" ./configure --with-simd --enable-lrandr
make -j8 clean all
sudo make install

XRDP_CFLAGS is needed as it cannot find the git cloned xrdp path. Notice, the relative path is /xrdp/common, don't miss "common"

BUS_ID=$(nvidia-smi --query-gpu=pci.bus --format=csv | sed -n '2 p' | xargs -I{} printf "%d\n" {})
sudo sed -i -E 's/(BusID "PCI:)[[:digit:]]+(:0:0")/\1'$BUS_ID'\2/' /etc/X11/xrdp/xorg_nvidia.conf
sudo systemctl enable xrdp --now
sudo systemctl enable xrdp-sesman --now

Now, xrdp and xorgxrdp are installed. Next, edit the startwm.sh, in /etc/xrdp/startwm.sh, the fallback window manager is to start "xterm". You need to update it. For me, I put the exec herbstluftwm in ~/.xinitrc. So, I change xterm to sh ~/.xinitrc.

Finally, remote login and open a terminal, type in nvidia_smi, in processes part, it should show at least a process of xorgxrdp_helper

BINGO!!!

@Nexarian Only one minor issue to mention. I cannot minimize the remote desktop and then maximize it again. The screen freezes after re-maximization (The keyboard is still working but screen is not moving)

Thanks again for your efforts! @Nexarian

@Nexarian
Copy link
Author

@EduardDurech After seeing the similar behavior, I think I have an idea of what might be the cause of your issue: https://github.com/Nexarian/xrdp/blob/mainline_merge_shm/xrdp/xrdp_encoder.c#L224

Try changing the "frames in flight" number. My guess is that the connection is just too slow and too many frames are dropping. Increasing this may help, but this is kind of a guess.

@Nexarian
Copy link
Author

For all those who would want to support this work, see the announcement on supporting XRDP development through funding here: neutrinolabs/xrdp#2893

@dunkleyr
Copy link

I noticed on some machines, I had to add:

sudo chmod o+rx /usr/local/libexec
sudo chmod o+rx /usr/local/libexec/xrdp

To the end of the script. Otherwise I get the following error in xrdp-sesman.log:

[2024-01-09T21:13:20.619-0700] [ERROR] Error calling exec (excutable: /usr/local/libexec/xrdp/waitforx, arguments: /usr/local/libexec/xrdp/waitforx -d :10) returned errno: 13, description: Permission denied
[2024-01-09T21:13:20.620-0700] [ERROR] Can't run /usr/local/libexec/xrdp/waitforx - Permission denied
[2024-01-09T21:13:20.622-0700] [ERROR] An error occurred waiting for the X server

waitforx seems to have the proper permissions, but the parent directories don't.

@bcraft
Copy link

bcraft commented Jan 14, 2024

thanks for your work on this and the continued updates.
Im just curious what the correct branch to use is currently? Looks like mainline_merge has been superseded by some newer ones based on commit dates, such as mainline_merge_shm and egfx_integration_step_2. I understand the integration of changes is an ongoing process but just checking if theres a better branch to build from now.

@Nexarian
Copy link
Author

Nexarian commented Jan 15, 2024

@bcraft There are multiple things going on here:

  1. We are hard at work lifting-and-shifting the RFX Progressive pieces from mainline_merge into devel. In order to do that we need to review and test the code from that branch in a more rigorous fashion than we had when it was "just a prototype." We don't have that excuse. That means tracking down security vulnerabilities, writing good self-documenting code, and making very sure it's in compliance with the latest specs. It's taking a bit. That's where that egfx_integration branch you see comes from.

  2. Last year (2023) I spent most of my XRDP dev bandwidth on getting AVC444 implemented (mainline_merge_avc444). However, about halfway through 2023 we had to move the way shared memory worked. That broke everything in mainline_merge rather invasively meaning a rebase on devel was no longer straightforward. This is where mainline_merge_shm comes from. It's a copy of mainline_merge that has the shared memory bits from devel factored in (that can easily rebase as well). My plan was eventually to turn mainline_merge_shm into mainline_merge and delete both mainline_merge_shm and mainline_merge_avc444 but see (1), we've been pushing hard on a true production release.

So yeah, for now I won't be updating mainline_merge, but it is stable. mainline_merge_shm is what I'll be contributing to for the short term, and it will eventually have everything but updates to that are going to be slow for a bit until (1) is complete. Feel free to use mainline_merge_shm as I think it's stable, but I won't officially recommend it until AVC444 is integrated.

@Nexarian
Copy link
Author

AVC444 is mostly working except in the case of the Mac OS client. There is a graphical corruption bug where it seems the NVENC encoder outputs frames that the Microsoft Mac OS RDP client (and only the Mac OS RDP client) has issues with. I've yet to solve that, and probably need help from Microsoft at this point.

@bcraft
Copy link

bcraft commented Jan 18, 2024

Thanks for the update. I built mainline_merge for both repos without a problem.

Im having trouble creating an Xsession though, im sure im overthinking this now and the solution is simple, below is the tail end of the .xorgxrdo.log file, I can connect to the host through an RDP client, and authenticate with sessman, but x fails to start. the initial error seems to be Failed to acquire modesetting permission. To note incase its relevant, I use this machine as a desktop so it had gnome installed and gdm running, but there is no active desktop sessions when trying to use rdp.

any ideas?

[   595.856] (**) NVIDIA(0): Option "DPI" "96 x 96"
[   595.856] (**) NVIDIA(0): Option "ConnectToAcpid" "false"
[   595.856] (**) NVIDIA(0): Enabling 2D acceleration
[   595.856] (II) Loading sub module "glxserver_nvidia"
[   595.856] (II) LoadModule: "glxserver_nvidia"
[   595.856] (II) Loading /usr/lib/xorg/modules/extensions/libglxserver_nvidia.so
[   595.863] (II) Module glxserver_nvidia: vendor="NVIDIA Corporation"
[   595.863] 	compiled for 1.6.99.901, module version = 1.0.0
[   595.863] 	Module class: X.Org Server Extension
[   595.863] (II) NVIDIA GLX Module  525.147.05  Wed Oct 25 20:26:41 UTC 2023
[   595.863] (II) NVIDIA: The X server supports PRIME Render Offload.
[   595.863] (--) NVIDIA(0): Valid display device(s) on GPU-0 at PCI:1:0:0
[   595.863] (--) NVIDIA(0):     DFP-0
[   595.863] (--) NVIDIA(0):     DFP-1
[   595.863] (--) NVIDIA(0):     DFP-2 (boot)
[   595.863] (--) NVIDIA(0):     DFP-3
[   595.863] (--) NVIDIA(0):     DFP-4
[   595.863] (--) NVIDIA(0):     DFP-5
[   595.863] (--) NVIDIA(0):     DFP-6
[   595.864] (II) NVIDIA(0): NVIDIA GPU NVIDIA GeForce RTX 3070 (GA104-A) at PCI:1:0:0
[   595.864] (II) NVIDIA(0):     (GPU-0)
[   595.864] (--) NVIDIA(0): Memory: 8388608 kBytes
[   595.864] (--) NVIDIA(0): VideoBIOS: 94.04.46.00.ea
[   595.864] (II) NVIDIA(0): Detected PCI Express Link width: 16X
[   595.864] (EE) NVIDIA(GPU-0): Failed to acquire modesetting permission.
[   595.864] (EE) NVIDIA(0): Failing initialization of X screen
[   595.864] (II) UnloadModule: "nvidia"
[   595.864] (II) UnloadSubModule: "glxserver_nvidia"
[   595.864] (II) Unloading glxserver_nvidia
[   595.864] (II) UnloadSubModule: "wfb"
[   595.864] (EE) Screen(s) found, but none have a usable configuration.
[   595.864] (EE)
Fatal server error:
[   595.864] (EE) no screens found(EE)
[   595.864] (EE)

@dcarrion87
Copy link

No go with P40 on Ubuntu 22.04. Client connects then immediately disconnects. Logs:

[  2574.402] 	compiled for 1.21.1.4, module version = 1.0.0
[  2574.402] 	ABI class: X.Org ANSI C Emulation, version 0.4
[  2574.402] xorgxrdpPreInit:
[  2574.402] (==) NVIDIA(0): Depth 24, (==) framebuffer bpp 32
[  2574.402] (==) NVIDIA(0): RGB weight 888
[  2574.402] (==) NVIDIA(0): Default visual is TrueColor
[  2574.402] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
[  2574.402] (II) Applying OutputClass "nvidia" options to /dev/dri/card0
[  2574.402] (**) NVIDIA(0): Option "DPI" "96 x 96"
[  2574.402] (**) NVIDIA(0): Option "ConnectToAcpid" "false"
[  2574.402] (**) NVIDIA(0): Option "AllowEmptyInitialConfiguration"
[  2574.402] (**) NVIDIA(0): Enabling 2D acceleration
[  2574.402] (II) Loading sub module "glxserver_nvidia"
[  2574.402] (II) LoadModule: "glxserver_nvidia"
[  2574.402] (II) Loading /usr/lib/x86_64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so
[  2574.408] (II) Module glxserver_nvidia: vendor="NVIDIA Corporation"
[  2574.408] 	compiled for 1.6.99.901, module version = 1.0.0
[  2574.408] 	Module class: X.Org Server Extension
[  2574.408] (II) NVIDIA GLX Module  535.154.05  Thu Dec 28 15:57:12 UTC 2023
[  2574.408] (II) NVIDIA: The X server supports PRIME Render Offload.
[  2574.408] (--) NVIDIA(0): Valid display device(s) on GPU-0 at PCI:1:0:0
[  2574.409] (--) NVIDIA(0):     DFP-0
[  2574.409] (--) NVIDIA(0):     DFP-1
[  2574.409] (--) NVIDIA(0):     DFP-2
[  2574.409] (--) NVIDIA(0):     DFP-3
[  2574.409] (II) NVIDIA(0): NVIDIA GPU Tesla P40 (GP102GL-A) at PCI:1:0:0 (GPU-0)
[  2574.409] (--) NVIDIA(0): Memory: 25165824 kBytes
[  2574.409] (--) NVIDIA(0): VideoBIOS: 86.02.23.00.00
[  2574.409] (II) NVIDIA(0): Detected PCI Express Link width: 16X
[  2574.409] (--) NVIDIA(GPU-0): NVIDIA VGX (DFP-0): connected
[  2574.409] (--) NVIDIA(GPU-0): NVIDIA VGX (DFP-0): External TMDS
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3840 x 2400) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3840 x 2160) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (4096 x 2160) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (5120 x 2880) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (7680 x 4320) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (2880 x 1620) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (2880 x 1620) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3200 x 1800) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3200 x 1800) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3840 x 2160) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3840 x 2160) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (4096 x 2304) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (4096 x 2304) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (5120 x 2880) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (5120 x 2880) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (7680 x 4320) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3840 x 2160) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (7680 x 4320) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (3840 x 2160) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.409] (WW) NVIDIA(GPU-0): Mode (15360 x 8640) larger than per-head max resolution
[  2574.409] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.410] (WW) NVIDIA(GPU-0): Mode (7680 x 4320) larger than per-head max resolution
[  2574.410] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.410] (WW) NVIDIA(GPU-0): Mode (15360 x 8640) larger than per-head max resolution
[  2574.410] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.410] (WW) NVIDIA(GPU-0): Mode (7680 x 4320) larger than per-head max resolution
[  2574.410] (WW) NVIDIA(GPU-0):     supported (2560 x 1600)
[  2574.410] (==) NVIDIA(0): 
[  2574.410] (==) NVIDIA(0): No modes were requested; the default mode "nvidia-auto-select"
[  2574.410] (==) NVIDIA(0):     will be used as the requested mode.
[  2574.410] (==) NVIDIA(0): 
[  2574.410] (II) NVIDIA(0): Validated MetaModes:
[  2574.410] (II) NVIDIA(0):     "DFP-0:nvidia-auto-select"
[  2574.410] (**) NVIDIA(0): Virtual screen size configured to be 1920 x 1080
[  2574.411] (**) NVIDIA(0): DPI set to (96, 96); computed from "DPI" X config option
[  2574.411] xorgxrdpScreenInit:
[  2574.411] (II) NVIDIA: Reserving 24576.00 MB of virtual memory for indirect memory
[  2574.411] (II) NVIDIA:     access.
[  2574.431] (II) NVIDIA(0): Setting mode "DFP-0:nvidia-auto-select"
[  2574.434] (==) NVIDIA(0): Disabling shared memory pixmaps
[  2574.434] (==) NVIDIA(0): Backing store enabled
[  2574.434] (==) NVIDIA(0): Silken mouse enabled
[  2574.435] (==) NVIDIA(0): DPMS enabled
[  2574.435] (II) Loading sub module "dri2"
[  2574.435] (II) LoadModule: "dri2"
[  2574.435] (II) Module "dri2" already built-in
[  2574.435] (II) NVIDIA(0): [DRI2] Setup complete
[  2574.435] (II) NVIDIA(0): [DRI2]   VDPAU driver: nvidia
[  2574.435] xorgxrdpScreenInit: width 1920 height 1080
[  2574.435] rdpClientConInit: disconnect idle session after [0] sec
[  2574.435] rdpClientConInit: kill disconnected [0] timeout [0] sec
[  2574.435] rdpSimdInit: assigning yuv functions
[  2574.435] rdpSimdInit: cpuid ax 1 cx 0 return ax 0x000306a9 bx 0x06100800 cx 0x7f9ae3bf dx 0xbfebfbff
[  2574.435] rdpSimdInit: sse2 amd64 yuv functions assigned
[  2574.435] (II) Initializing extension Generic Event Extension
[  2574.435] (II) Initializing extension SHAPE
[  2574.435] (II) Initializing extension MIT-SHM
[  2574.435] (II) Initializing extension XInputExtension
[  2574.435] (II) Initializing extension XTEST
[  2574.435] (II) Initializing extension BIG-REQUESTS
[  2574.435] (II) Initializing extension SYNC
[  2574.435] (II) Initializing extension XKEYBOARD
[  2574.436] (II) Initializing extension XC-MISC
[  2574.436] (II) Initializing extension SECURITY
[  2574.436] (II) Initializing extension XFIXES
[  2574.436] (II) Initializing extension RENDER
[  2574.436] (II) Initializing extension COMPOSITE
[  2574.436] (II) Initializing extension DAMAGE
[  2574.436] (II) Initializing extension MIT-SCREEN-SAVER
[  2574.436] (II) Initializing extension DOUBLE-BUFFER
[  2574.436] (II) Initializing extension RECORD
[  2574.436] (II) Initializing extension DPMS
[  2574.436] (II) Initializing extension Present
[  2574.436] (II) Initializing extension DRI3
[  2574.436] (II) Initializing extension X-Resource
[  2574.437] (II) Initializing extension XVideo
[  2574.437] (II) Initializing extension XVideo-MotionCompensation
[  2574.437] (II) Initializing extension SELinux
[  2574.437] (II) SELinux: Disabled on system
[  2574.437] (II) Initializing extension GLX
[  2574.437] (II) Initializing extension GLX
[  2574.437] (II) Indirect GLX disabled.
[  2574.437] (II) GLX: Another vendor is already registered for screen 0
[  2574.437] (II) Initializing extension XFree86-VidModeExtension
[  2574.437] (II) Initializing extension XFree86-DGA
[  2574.437] (II) Initializing extension XFree86-DRI
[  2574.437] (II) Initializing extension DRI2
[  2574.437] (II) Initializing extension NV-GLX
[  2574.437] (II) Initializing extension NV-CONTROL
[  2574.437] (II) Initializing extension XINERAMA
[  2574.437] rdpCreateScreenResources:
[  2574.472] (II) Using input driver 'XRDPMOUSE' for 'xrdpMouse'
[  2574.472] (**) Option "CorePointer"
[  2574.472] (**) xrdpMouse: always reports core events
[  2574.472] rdpmousePreInit: drv 0x5628bf186a50 info 0x5628bf6b54e0, flags 0x0
[  2574.472] (II) XINPUT: Adding extended input device "xrdpMouse" (type: Mouse, id 6)
[  2574.472] rdpmouseControl: what 0
[  2574.472] rdpmouseDeviceInit:
[  2574.472] rdpmouseCtrl:
[  2574.472] rdpRegisterInputCallback: type 1 proc 0x7f3940d5e5f0
[  2574.472] (**) xrdpMouse: (accel) keeping acceleration scheme 1
[  2574.472] (**) xrdpMouse: (accel) acceleration profile 0
[  2574.472] (**) xrdpMouse: (accel) acceleration factor: 2.000
[  2574.472] (**) xrdpMouse: (accel) acceleration threshold: 4
[  2574.472] rdpmouseControl: what 1
[  2574.472] rdpmouseDeviceOn:
[  2574.472] (II) Using input driver 'XRDPKEYB' for 'xrdpKeyboard'
[  2574.472] (**) Option "CoreKeyboard"
[  2574.473] (**) xrdpKeyboard: always reports core events
[  2574.473] rdpkeybPreInit: drv 0x5628bf1863f0 info 0x5628bf840130, flags 0x0
[  2574.473] (II) XINPUT: Adding extended input device "xrdpKeyboard" (type: Keyboard, id 7)
[  2574.473] rdpkeybControl: what 0
[  2574.473] rdpkeybDeviceInit:
[  2574.488] rdpkeybChangeKeyboardControl:
[  2574.488] rdpkeybChangeKeyboardControl: autoRepeat on
[  2574.488] rdpRegisterInputCallback: type 0 proc 0x7f39401bcb60
[  2574.488] rdpkeybControl: what 1
[  2574.488] rdpkeybDeviceOn:
[  2574.488] rdpkeybChangeKeyboardControl:
[  2574.488] rdpkeybChangeKeyboardControl: autoRepeat on
[  2574.497] (II) config/udev: Adding input device Power Button (/dev/input/event1)
[  2574.497] (II) AutoAddDevices is off - not adding device.
[  2574.497] (II) config/udev: Adding input device Video Bus (/dev/input/event2)
[  2574.497] (II) AutoAddDevices is off - not adding device.
[  2574.497] (II) config/udev: Adding input device Power Button (/dev/input/event0)
[  2574.497] (II) AutoAddDevices is off - not adding device.
[  2574.498] (II) config/udev: Adding input device HDA Intel PCH Line (/dev/input/event6)
[  2574.498] (II) AutoAddDevices is off - not adding device.
[  2574.498] (II) config/udev: Adding input device HDA Intel PCH Line Out Front (/dev/input/event7)
[  2574.498] (II) AutoAddDevices is off - not adding device.
[  2574.499] (II) config/udev: Adding input device HDA Intel PCH Line Out Surround (/dev/input/event8)
[  2574.499] (II) AutoAddDevices is off - not adding device.
[  2574.499] (II) config/udev: Adding input device HDA Intel PCH Line Out CLFE (/dev/input/event9)
[  2574.499] (II) AutoAddDevices is off - not adding device.
[  2574.499] (II) config/udev: Adding input device HDA Intel PCH Line Out Side (/dev/input/event10)
[  2574.499] (II) AutoAddDevices is off - not adding device.
[  2574.499] (II) config/udev: Adding input device HDA Intel PCH Front Headphone (/dev/input/event11)
[  2574.499] (II) AutoAddDevices is off - not adding device.
[  2574.499] (II) config/udev: Adding input device HDA Intel PCH HDMI/DP,pcm=3 (/dev/input/event12)
[  2574.499] (II) AutoAddDevices is off - not adding device.
[  2574.500] (II) config/udev: Adding input device HDA Intel PCH HDMI/DP,pcm=7 (/dev/input/event13)
[  2574.500] (II) AutoAddDevices is off - not adding device.
[  2574.500] (II) config/udev: Adding input device HDA Intel PCH Front Mic (/dev/input/event4)
[  2574.500] (II) AutoAddDevices is off - not adding device.
[  2574.500] (II) config/udev: Adding input device HDA Intel PCH Rear Mic (/dev/input/event5)
[  2574.500] (II) AutoAddDevices is off - not adding device.
[  2574.500] (II) config/udev: Adding input device Eee PC WMI hotkeys (/dev/input/event3)
[  2574.500] (II) AutoAddDevices is off - not adding device.
[  2574.505] xorgxrdpDeferredStartup:
[  2574.505] rdpLRRInit: AddCallback ok
[  2574.505] rdpLRRInit: AddExtension ok
[  2574.505] xorgxrdpSetupDamage: DamageRegister ok
[  2574.588] rdpInDeferredRepeatCallback:
[  2574.588] rdpkeybChangeKeyboardControl:
[  2574.588] rdpkeybChangeKeyboardControl: autoRepeat off
[  2574.588] rdpInDeferredRepeatCallback:
[  2574.588] rdpkeybChangeKeyboardControl:
[  2574.588] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.427] rdpClientConGotConnection:
[  2575.427] rdpClientConGotConnection: g_sck_accept ok new_sck 9
[  2575.427] rdpClientConGetConnection: idle_disconnect_timeout set to non-positive value, idle timer turned off
[  2575.427] rdpAddClientConToDev: adding first clientCon 0x5628bf86ac70
[  2575.429] rdpClientConProcessMsgVersion: version 0 0 0 1
[  2575.429] rdpClientConProcessScreenSizeMsg: set width 2560 height 1440 bpp 32
[  2575.429] rdpClientConAllocateSharedMemory: shmemid 32769 shmemptr 0x7f33367ef000 bytes 14745600
[  2575.429] xorgxrdpRRScreenSetSize: width 2560 height 1440
[  2575.434] rdpClientConProcessScreenSizeMsg: RRScreenSizeSet ok=[1]
[  2575.434]   screen resized to 2560x1440
[  2575.434] rdpClientConProcessScreenSizeMsg: LRRScreenSizeSet ok=[1]
[  2575.434] rdpClientConProcessMsgClientInput: invalidate x 0 y 0 cx 2560 cy 1440
[  2575.435] rdpClientConProcessMsgClientInfo:
[  2575.435]   got client info bytes 7192
[  2575.435]   jpeg support 0
[  2575.435]   offscreen support 0
[  2575.435]   offscreen size 0
[  2575.435]   offscreen entries 0
[  2575.435] rdpClientConProcessMsgClientInfo: got H264 capture
[  2575.435]   cap_width 2560 cap_height 1440
[  2575.435] rdpClientConAllocateSharedMemory: shmemid 32770 shmemptr 0x7f3336ef7000 bytes 7372800
[  2575.435]   client can not do offscreen to offscreen blits
[  2575.435]   client can do new(color) cursor
[  2575.435]   client can not do multimon
[  2575.435] rdpLoadLayout: keylayout 0x00000c09 variant  display 20
[  2575.435] rdpkeybChangeKeyboardControl:
[  2575.435] rdpkeybChangeKeyboardControl: autoRepeat on
[  2575.436] rdpkeybChangeKeyboardControl:
[  2575.436] rdpkeybChangeKeyboardControl: autoRepeat on
[  2575.442] rdpClientConProcessMsgClientInfo: started helper pid 38912
[  2575.445] rdpClientConRecv: g_sck_recv failed(returned -1)
[  2575.445] rdpClientConRecvMsg: error
[  2575.445] rdpClientConCheck: rdpClientConGotData failed
[  2575.445] rdpClientConDisconnect:
[  2575.445] rdpRemoveClientConFromDev: removing clientCon 0x5628bf86ac70
[  2575.445] rdpShutdownHelper:
[  2575.535] rdpInDeferredRepeatCallback:
[  2575.535] rdpkeybChangeKeyboardControl:
[  2575.535] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.536] rdpInDeferredRepeatCallback:
[  2575.536] rdpkeybChangeKeyboardControl:
[  2575.536] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.538] ProcLRRSelectInput:                 client 0x5628bf882dc0 adding pClient to list
[  2575.560] ProcLRRSelectInput:                 client 0x5628bf889ca0 adding pClient to list
[  2575.563] ProcLRRSelectInput:                 client 0x5628bf88a0a0 adding pClient to list
[  2575.565] ProcLRRSelectInput:                 client 0x5628bf885330 adding pClient to list
[  2575.647] ProcLRRSelectInput:                 client 0x5628bf8bd1e0 adding pClient to list
[  2575.664] remove_client:                      client 0x5628bf882dc0 found pClient, removing
[  2575.734] ProcLRRSelectInput:                 client 0x5628bf8ac470 adding pClient to list
[  2575.828] ProcLRRSelectInput:                 client 0x5628bf1972a0 adding pClient to list
[  2575.829] remove_client:                      client 0x5628bf1972a0 found pClient, removing
[  2575.829] ProcLRRSelectInput:                 client 0x5628bf1972a0 adding pClient to list
[  2575.873] rdpkeybChangeKeyboardControl:
[  2575.873] rdpkeybChangeKeyboardControl: autoRepeat on
[  2575.873] rdpkeybChangeKeyboardControl:
[  2575.873] rdpkeybChangeKeyboardControl: autoRepeat on
[  2575.873] rdpkeybChangeKeyboardControl:
[  2575.873] rdpkeybChangeKeyboardControl: autoRepeat on
[  2575.873] rdpkeybChangeKeyboardControl:
[  2575.873] rdpkeybChangeKeyboardControl: autoRepeat on
[  2575.876] ProcLRRDispatch: returning Success, data 26
[  2575.876] ProcLRRDispatch: returning Success, data 30
[  2575.876] ProcLRRDispatch: returning Success, data 13
[  2575.907] ProcLRRSelectInput:                 client 0x5628bf90d7c0 adding pClient to list
[  2575.973] rdpInDeferredRepeatCallback:
[  2575.974] rdpkeybChangeKeyboardControl:
[  2575.974] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.974] rdpInDeferredRepeatCallback:
[  2575.974] rdpkeybChangeKeyboardControl:
[  2575.974] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.974] rdpInDeferredRepeatCallback:
[  2575.974] rdpkeybChangeKeyboardControl:
[  2575.974] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.974] rdpInDeferredRepeatCallback:
[  2575.974] rdpkeybChangeKeyboardControl:
[  2575.974] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.978] rdpkeybChangeKeyboardControl:
[  2575.978] rdpkeybChangeKeyboardControl: autoRepeat off
[  2575.978] rdpkeybChangeKeyboardControl:
[  2575.978] rdpkeybChangeKeyboardControl: autoRepeat off
[  2576.563] ProcLRRSelectInput:                 client 0x5628bf8be670 adding pClient to list
[  2576.570] ProcLRRSelectInput:                 client 0x5628bfa43c10 adding pClient to list
[  2576.582] ProcLRRSelectInput:                 client 0x5628bfa27c50 adding pClient to list
[  2576.600] ProcLRRSelectInput:                 client 0x5628bfa27ed0 adding pClient to list
[  2576.616] ProcLRRSelectInput:                 client 0x5628bf8bc610 adding pClient to list
[  2576.623] ProcLRRSelectInput:                 client 0x5628bfa6ba80 adding pClient to list
[  2576.666] ProcLRRSelectInput:                 client 0x5628bfa788b0 adding pClient to list
[  2576.668] rdpkeybChangeKeyboardControl:
[  2576.668] rdpkeybChangeKeyboardControl: autoRepeat off
[  2576.668] rdpkeybChangeKeyboardControl:
[  2576.668] rdpkeybChangeKeyboardControl: autoRepeat off
[  2576.699] ProcLRRSelectInput:                 client 0x5628bfa81700 adding pClient to list
[  2576.721] ProcLRRSelectInput:                 client 0x5628bfa7d000 adding pClient to list
[  2576.727] remove_client:                      client 0x5628bfa7d000 found pClient, removing
[  2576.727] remove_client:                      client 0x5628bfa788b0 found pClient, removing
[  2576.875] ProcLRRDispatch: returning Success, data 24
[  2576.889] ProcLRRSelectInput:                 client 0x5628bfa8a100 adding pClient to list
[  2576.930] remove_client:                      client 0x5628bfa8a100 found pClient, removing
[  2576.930] ProcLRRSelectInput:                 client 0x5628bfa8a100 adding pClient to list
[  2636.748] ProcLRRSelectInput:                 client 0x5628bfcb7e00 adding pClient to list
[  2636.748] ProcLRRSelectInput:                 client 0x5628bf8513b0 adding pClient to list
[  2636.775] remove_client:                      client 0x5628bf8513b0 found pClient, removing
[  2636.776] remove_client:                      client 0x5628bfcb7e00 found pClient, removing
[  2739.062] rdpClientConGotConnection:
[  2739.062] rdpClientConGotConnection: g_sck_accept ok new_sck 55
[  2739.062] rdpClientConGetConnection: idle_disconnect_timeout set to non-positive value, idle timer turned off
[  2739.062] rdpAddClientConToDev: adding first clientCon 0x5628bfcd0650
[  2739.064] rdpClientConProcessMsgVersion: version 0 0 0 1
[  2739.064] rdpClientConProcessScreenSizeMsg: set width 1920 height 1080 bpp 32
[  2739.065] rdpClientConAllocateSharedMemory: shmemid 32780 shmemptr 0x7f333a778000 bytes 8294400
[  2739.065] xorgxrdpRRScreenSetSize: width 1920 height 1080
[  2739.069] rdpClientConProcessScreenSizeMsg: RRScreenSizeSet ok=[1]
[  2739.073]   screen resized to 1920x1080
[  2739.073] rdpClientConProcessScreenSizeMsg: LRRScreenSizeSet ok=[1]
[  2739.073] rdpClientConProcessMsgClientInput: invalidate x 0 y 0 cx 1920 cy 1080
[  2739.073] rdpClientConProcessMsgClientInfo:
[  2739.073]   got client info bytes 7192
[  2739.073]   jpeg support 0
[  2739.074]   offscreen support 1
[  2739.074]   offscreen size 10485760
[  2739.074]   offscreen entries 100
[  2739.074] rdpClientConProcessMsgClientInfo: got H264 capture
[  2739.074]   cap_width 1920 cap_height 1088
[  2739.074] rdpClientConAllocateSharedMemory: shmemid 32781 shmemptr 0x7f333ab65000 bytes 4177920
[  2739.074]   client can not do offscreen to offscreen blits
[  2739.074]   client can do new(color) cursor
[  2739.074]   client can not do multimon
[  2739.075] rdpLoadLayout: keylayout 0x00000409 variant  display 20
[  2739.075] rdpkeybChangeKeyboardControl:
[  2739.075] rdpkeybChangeKeyboardControl: autoRepeat on
[  2739.076] rdpkeybChangeKeyboardControl:
[  2739.076] rdpkeybChangeKeyboardControl: autoRepeat on
[  2739.082] rdpClientConProcessMsgClientInfo: started helper pid 39526
[  2739.085] rdpClientConRecv: g_sck_recv failed(returned -1)
[  2739.085] rdpClientConRecvMsg: error
[  2739.085] rdpClientConCheck: rdpClientConGotData failed
[  2739.085] rdpClientConDisconnect:
[  2739.085] rdpRemoveClientConFromDev: removing clientCon 0x5628bfcd0650
[  2739.085] rdpShutdownHelper:
[  2739.175] rdpInDeferredRepeatCallback:
[  2739.175] rdpkeybChangeKeyboardControl:
[  2739.175] rdpkeybChangeKeyboardControl: autoRepeat off
[  2739.176] rdpInDeferredRepeatCallback:
[  2739.176] rdpkeybChangeKeyboardControl:
[  2739.176] rdpkeybChangeKeyboardControl: autoRepeat off
[  2739.255] ProcLRRDispatch: returning Success, data 24

@bcraft
Copy link

bcraft commented Feb 5, 2024

@dcarrion87 which branch did you build off out of interest?

@tokon2000
Copy link

I'm trying to run xrdp with gpu acceleration on a headless ubuntu 22.04 server.
It is a fresh installation and I have done apt install ubuntu-desktop and then run this script.
But the desktop does not load, I get "Can't create session for user testusr - X server could not be started".

What is missing, what am I doing wrong?

logs
testusr@ubuntusrv:~$ cat .xorgxrdp.10.log 
[    54.390] 
X.Org X Server 1.21.1.4
X Protocol Version 11, Revision 0
[    54.390] Current Operating System: Linux srv21318 5.15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64
[    54.390] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-102-generic root=UUID=f1aacf6e-9542-43c1-a79d-46d6cd0400d3 ro
[    54.390] xorg-server 2:21.1.4-2ubuntu1.7~22.04.10 (For technical support please see http://www.ubuntu.com/support) 
[    54.390] Current version of pixman: 0.40.0
[    54.390] 	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
[    54.390] Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[    54.390] (++) Log file: ".xorgxrdp.10.log", Time: Thu Apr 18 08:24:42 2024
[    54.390] (++) Using config file: "/etc/X11/xrdp/xorg_nvidia.conf"
[    54.390] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[    54.390] (**) Option "defaultserverlayout" "XRDP GPU Server"
[    54.390] (**) ServerLayout "XRDP GPU Server"
[    54.390] (**) |-->Screen "dGPU" (0)
[    54.390] (**) |   |-->Monitor "<default monitor>"
[    54.391] (**) |   |-->Device "dGPU"
[    54.391] (==) No monitor specified for screen "dGPU".
	Using a default monitor configuration.
[    54.391] (**) |-->Input Device "xrdpMouse"
[    54.391] (**) |-->Input Device "xrdpKeyboard"
[    54.391] (**) Option "DontVTSwitch" "on"
[    54.391] (**) Option "AutoAddDevices" "off"
[    54.391] (**) Not automatically adding devices
[    54.391] (==) Automatically enabling devices
[    54.391] (==) Automatically adding GPU devices
[    54.391] (==) Automatically binding GPU devices
[    54.391] (==) Max clients allowed: 256, resource mask: 0x1fffff
[    54.391] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist.
[    54.391] 	Entry deleted from font path.
[    54.391] (WW) The directory "/usr/share/fonts/X11/100dpi/" does not exist.
[    54.391] 	Entry deleted from font path.
[    54.391] (WW) The directory "/usr/share/fonts/X11/75dpi/" does not exist.
[    54.391] 	Entry deleted from font path.
[    54.391] (WW) The directory "/usr/share/fonts/X11/100dpi" does not exist.
[    54.391] 	Entry deleted from font path.
[    54.391] (WW) The directory "/usr/share/fonts/X11/75dpi" does not exist.
[    54.391] 	Entry deleted from font path.
[    54.391] (==) FontPath set to:
	/usr/share/fonts/X11/misc,
	/usr/share/fonts/X11/Type1,
	built-ins
[    54.391] (==) ModulePath set to "/usr/lib/xorg/modules"
[    54.391] (II) Loader magic: 0x5558fdb09020
[    54.391] (II) Module ABI versions:
[    54.391] 	X.Org ANSI C Emulation: 0.4
[    54.391] 	X.Org Video Driver: 25.2
[    54.391] 	X.Org XInput driver : 24.4
[    54.391] 	X.Org Server Extension : 10.0
[    54.391] (--) using VT number 2

[    54.391] (II) systemd-logind: logind integration requires -keeptty and -keeptty was not provided, disabling logind integration
[    54.395] (--) PCI:*(8@0:0:0) 10de:128b:1462:8c93 rev 161, Mem @ 0xf5000000/16777216, 0xe8000000/134217728, 0xf0000000/33554432, I/O @ 0x0000e000/128, BIOS @ 0x????????/524288
[    54.395] (II) "glx" will be loaded by default.
[    54.395] (II) LoadModule: "xorgxrdp"
[    54.395] (II) Loading /usr/lib/xorg/modules/libxorgxrdp.so
[    54.396] (II) Module XORGXRDP: vendor="X.Org Foundation"
[    54.396] 	compiled for 1.21.1.4, module version = 0.9.80
[    54.396] 	ABI class: X.Org Video Driver, version 25.2
[    54.396] xorgxrdpSetup:
[    54.396] (II) LoadModule: "glx"
[    54.396] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    54.396] (II) Module glx: vendor="X.Org Foundation"
[    54.396] 	compiled for 1.21.1.4, module version = 1.0.0
[    54.396] 	ABI class: X.Org Server Extension, version 10.0
[    54.396] (II) LoadModule: "nvidia"
[    54.397] (WW) Warning, couldn't open module nvidia
[    54.397] (EE) Failed to load module "nvidia" (module does not exist, 0)
[    54.514] (==) Matched nouveau as autoconfigured driver 0
[    54.514] (==) Matched modesetting as autoconfigured driver 1
[    54.514] (==) Matched fbdev as autoconfigured driver 2
[    54.514] (==) Matched vesa as autoconfigured driver 3
[    54.514] (==) Assigned the driver to the xf86ConfigLayout
[    54.514] (II) LoadModule: "nouveau"
[    54.514] (II) Loading /usr/lib/xorg/modules/drivers/nouveau_drv.so
[    54.514] (II) Module nouveau: vendor="X.Org Foundation"
[    54.514] 	compiled for 1.21.1.3, module version = 1.0.17
[    54.514] 	Module class: X.Org Video Driver
[    54.514] 	ABI class: X.Org Video Driver, version 25.2
[    54.514] (II) LoadModule: "modesetting"
[    54.515] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[    54.515] (II) Module modesetting: vendor="X.Org Foundation"
[    54.515] 	compiled for 1.21.1.4, module version = 1.21.1
[    54.515] 	Module class: X.Org Video Driver
[    54.515] 	ABI class: X.Org Video Driver, version 25.2
[    54.515] (II) LoadModule: "fbdev"
[    54.515] (II) Loading /usr/lib/xorg/modules/drivers/fbdev_drv.so
[    54.515] (II) Module fbdev: vendor="X.Org Foundation"
[    54.515] 	compiled for 1.21.1.3, module version = 0.5.0
[    54.515] 	Module class: X.Org Video Driver
[    54.515] 	ABI class: X.Org Video Driver, version 25.2
[    54.515] (II) LoadModule: "vesa"
[    54.515] (II) Loading /usr/lib/xorg/modules/drivers/vesa_drv.so
[    54.515] (II) Module vesa: vendor="X.Org Foundation"
[    54.515] 	compiled for 1.21.1.3, module version = 2.5.0
[    54.515] 	Module class: X.Org Video Driver
[    54.515] 	ABI class: X.Org Video Driver, version 25.2
[    54.515] (II) LoadModule: "xrdpmouse"
[    54.515] (II) Loading /usr/lib/xorg/modules/input/xrdpmouse_drv.so
[    54.515] (II) Module XRDPMOUSE: vendor="X.Org Foundation"
[    54.515] 	compiled for 1.21.1.4, module version = 0.9.80
[    54.515] 	Module class: X.Org XInput Driver
[    54.515] 	ABI class: X.Org XInput driver, version 24.4
[    54.516] rdpmousePlug:
[    54.516] (II) LoadModule: "xrdpkeyb"
[    54.516] (II) Loading /usr/lib/xorg/modules/input/xrdpkeyb_drv.so
[    54.516] (II) Module XRDPKEYB: vendor="X.Org Foundation"
[    54.516] 	compiled for 1.21.1.4, module version = 0.9.80
[    54.516] 	Module class: X.Org XInput Driver
[    54.516] 	ABI class: X.Org XInput driver, version 24.4
[    54.516] rdpkeybPlug:
[    54.516] (II) NOUVEAU driver Date:   Sat Jan 23 12:24:42 2021 -0500
[    54.516] (II) NOUVEAU driver for NVIDIA chipset families :
[    54.516] 	RIVA TNT            (NV04)
[    54.516] 	RIVA TNT2           (NV05)
[    54.516] 	GeForce 256         (NV10)
[    54.516] 	GeForce 2           (NV11, NV15)
[    54.516] 	GeForce 4MX         (NV17, NV18)
[    54.516] 	GeForce 3           (NV20)
[    54.516] 	GeForce 4Ti         (NV25, NV28)
[    54.516] 	GeForce FX          (NV3x)
[    54.516] 	GeForce 6           (NV4x)
[    54.516] 	GeForce 7           (G7x)
[    54.516] 	GeForce 8           (G8x)
[    54.516] 	GeForce 9           (G9x)
[    54.516] 	GeForce GTX 2xx/3xx (GT2xx)
[    54.516] 	GeForce GTX 4xx/5xx (GFxxx)
[    54.516] 	GeForce GTX 6xx/7xx (GKxxx)
[    54.516] 	GeForce GTX 9xx     (GMxxx)
[    54.516] 	GeForce GTX 10xx    (GPxxx)
[    54.516] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[    54.516] (II) FBDEV: driver for framebuffer: fbdev
[    54.516] (II) VESA: driver for VESA chipsets: vesa
[    54.517] (EE) 
Fatal server error:
[    54.517] (EE) xf86OpenConsole: Cannot open virtual console 2 (Permission denied)
[    54.517] (EE) 
[    54.517] (EE) 
Please consult the The X.Org Foundation support 
	 at http://wiki.x.org
 for help. 
[    54.517] (EE) Please also check the log file at ".xorgxrdp.10.log" for additional information.
[    54.517] (EE) 
[    54.517] (WW) xf86CloseConsole: KDSETMODE failed: Bad file descriptor
[    54.517] (WW) xf86CloseConsole: VT_GETMODE failed: Bad file descriptor
[    54.517] (EE) Server terminated with error (1). Closing log file.

testusr@ubuntusrv:~$ sudo cat /var/log/xrdp.log 
[2024-04-18T08:17:52.562+0200] [INFO ] starting xrdp with pid 50206
[2024-04-18T08:17:52.563+0200] [INFO ] address [0.0.0.0] port [3389] mode 1
[2024-04-18T08:17:52.564+0200] [INFO ] listening to port 3389 on 0.0.0.0
[2024-04-18T08:17:52.564+0200] [INFO ] xrdp_listen_pp done
[2024-04-18T08:19:28.561+0200] [INFO ] Socket 12: connection accepted from x.x.x.x:16104
[2024-04-18T08:19:28.563+0200] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[2024-04-18T08:19:28.563+0200] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[2024-04-18T08:19:28.870+0200] [INFO ] Security protocol: configured [SSL|RDP], requested [SSL|HYBRID|RDP], selected [SSL]
[2024-04-18T08:19:28.918+0200] [ERROR] SSL_accept: I/O error
[2024-04-18T08:19:28.918+0200] [ERROR] trans_set_tls_mode: ssl_tls_accept failed
[2024-04-18T08:19:28.919+0200] [ERROR] xrdp_sec_incoming: trans_set_tls_mode failed
[2024-04-18T08:19:28.920+0200] [ERROR] xrdp_rdp_incoming: xrdp_sec_incoming failed
[2024-04-18T08:19:28.920+0200] [ERROR] xrdp_process_main_loop: libxrdp_process_incoming failed
[2024-04-18T08:19:28.921+0200] [ERROR] xrdp_iso_send: trans_write_copy_s failed
[2024-04-18T08:19:28.921+0200] [ERROR] Sending [ITU T.125] DisconnectProviderUltimatum failed
[2024-04-18T08:22:26.554+0200] [INFO ] Received termination signal, stopping the server accept new connections thread
[2024-04-18T08:24:13.528+0200] [INFO ] starting xrdp with pid 1050
[2024-04-18T08:24:13.530+0200] [INFO ] address [0.0.0.0] port [3389] mode 1
[2024-04-18T08:24:13.532+0200] [INFO ] listening to port 3389 on 0.0.0.0
[2024-04-18T08:24:13.533+0200] [INFO ] xrdp_listen_pp done
[2024-04-18T08:24:35.447+0200] [INFO ] Socket 12: connection accepted from x.x.x.x:64508
[2024-04-18T08:24:35.449+0200] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[2024-04-18T08:24:35.449+0200] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[2024-04-18T08:24:35.450+0200] [INFO ] Security protocol: configured [SSL|RDP], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [SSL]
[2024-04-18T08:24:42.554+0200] [INFO ] Connected client computer name: MacBook-Pro-14
[2024-04-18T08:24:42.555+0200] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc006 is unknown (ignored)
[2024-04-18T08:24:42.556+0200] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc00a is unknown (ignored)
[2024-04-18T08:24:42.723+0200] [INFO ] xrdp_load_keyboard_layout: Keyboard information sent by the RDP client, keyboard_type:[0x04], keyboard_subtype:[0x00], keylayout:[0x00000407]
[2024-04-18T08:24:42.724+0200] [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [de] options []
[2024-04-18T08:24:42.725+0200] [INFO ] TLS connection established from x.x.x.x:64508 TLSv1.3 with cipher TLS_AES_256_GCM_SHA384
[2024-04-18T08:24:42.816+0200] [INFO ] xrdp_caps_process_pointer: client supports new(color) cursor
[2024-04-18T08:24:42.817+0200] [INFO ] xrdp_process_offscreen_bmpcache: support level 0 cache size 0 MB cache entries 0
[2024-04-18T08:24:42.818+0200] [INFO ] xrdp_caps_process_codecs: NSCodec(CA8D1BB9-000F-154F-589F-AE2D1A87E2D6), codec id [1], properties len [3]
[2024-04-18T08:24:42.819+0200] [INFO ] Client Capability: LARGE_POINTER_FLAG_96x96 supported
[2024-04-18T08:24:42.824+0200] [INFO ] Loading keymap file /etc/xrdp/km-00000407.ini
[2024-04-18T08:24:42.825+0200] [WARN ] local keymap file for 0x00000407 found and doesn't match built in keymap, using local keymap file
[2024-04-18T08:24:42.826+0200] [INFO ] Login screen monitor height is 1200 pixels over 331 mm (92 DPI)
[2024-04-18T08:24:42.844+0200] [INFO ] xrdp_egfx_create: error 0 channel_id 1
[2024-04-18T08:24:42.867+0200] [INFO ] xrdp_mm_egfx_caps_advertise:
[2024-04-18T08:24:42.868+0200] [INFO ]   version 0x00080004 flags 0x00000002 (index: 0)
[2024-04-18T08:24:42.869+0200] [INFO ]   version 0x00080105 flags 0x00000002 (index: 1)
[2024-04-18T08:24:42.869+0200] [INFO ]   version 0x000a0002 flags 0x00000022 (index: 2)
[2024-04-18T08:24:42.870+0200] [INFO ]   version 0x000a0200 flags 0x00000022 (index: 3)
[2024-04-18T08:24:42.871+0200] [INFO ]   version 0x000a0301 flags 0x00000020 (index: 4)
[2024-04-18T08:24:42.871+0200] [INFO ]   version 0x000a0400 flags 0x00000002 (index: 5)
[2024-04-18T08:24:42.872+0200] [INFO ]   version 0x000a0701 flags 0x00000082 (index: 6)
[2024-04-18T08:24:42.872+0200] [INFO ]   version 0x000b0101 flags 0x00000082 (index: 7)
[2024-04-18T08:24:42.873+0200] [INFO ]   replying version 0x000a0400 flags 0x00000002
[2024-04-18T08:24:42.873+0200] [INFO ] xrdp_mm_egfx_caps_advertise: xrdp_egfx_send_capsconfirm error 0 best_index 5
[2024-04-18T08:24:42.874+0200] [INFO ] xrdp_egfx_reset_graphics:
[2024-04-18T08:24:42.875+0200] [INFO ] xrdp_egfx_reset_graphics: width 1920 height 1200 monitorcount 1
[2024-04-18T08:24:42.875+0200] [INFO ] xrdp_mm_egfx_caps_advertise: xrdp_egfx_send_reset_graphics error 0 monitorCount 0
[2024-04-18T08:24:42.876+0200] [INFO ] xrdp_encoder_create:
[2024-04-18T08:24:42.876+0200] [INFO ] xrdp_encoder_create: starting h264 codec session gfx
[2024-04-18T08:24:42.877+0200] [INFO ] xrdp_mm_egfx_invalidate_all:
[2024-04-18T08:24:42.877+0200] [INFO ] xrdp_mm_egfx_caps_advertise: egfx created.
[2024-04-18T08:24:42.878+0200] [WARN ] Ignoring obsolete SCP port value '3350'
[2024-04-18T08:24:42.879+0200] [INFO ] xrdp_wm_log_msg: sesman connect ok
[2024-04-18T08:24:42.879+0200] [INFO ] xrdp_wm_log_msg: Performing login request for testusr
[2024-04-18T08:24:42.910+0200] [INFO ] xrdp_wm_log_msg: login was successful - creating session
[2024-04-18T08:24:52.926+0200] [INFO ] xrdp_wm_log_msg: Can't create session for user testusr - X server could not be started
[2024-04-18T08:25:00.187+0200] [INFO ] Login screen monitor height is 1200 pixels over 331 mm (92 DPI)
[2024-04-18T08:25:06.936+0200] [INFO ] proc_enc_msg: xrdp_mm term
[2024-04-18T08:25:07.936+0200] [ERROR] Attempting to send data to a channel that is not open. channel id 1, channel status unknown

testusr@ubuntusrv:~$ sudo cat /var/log/xrdp-sesman.log 
[2024-04-18T08:17:52.560+0200] [INFO ] starting xrdp-sesman with pid 50205
[2024-04-18T08:17:52.562+0200] [INFO ] Sesman now listening on /var/run/xrdp-sesman/sesman.socket
[2024-04-18T08:22:26.570+0200] [INFO ] sesman_main_loop: sesman asked to terminate
[2024-04-18T08:24:13.526+0200] [INFO ] starting xrdp-sesman with pid 1044
[2024-04-18T08:24:13.528+0200] [INFO ] Sesman now listening on /var/run/xrdp-sesman/sesman.socket
[2024-04-18T08:24:42.879+0200] [INFO ] Socket 13: connection accepted from AF_UNIX
[2024-04-18T08:24:42.880+0200] [INFO ] Received system login request from xrdp for user: testusr IP: x.x.x.x
[2024-04-18T08:24:42.884+0200] [INFO ] starting xrdp-sesexec with pid 1474
[2024-04-18T08:24:42.909+0200] [INFO ] Terminal Server Users group is disabled, allowing authentication
[2024-04-18T08:24:42.910+0200] [INFO ] Access permitted for user: testusr
[2024-04-18T08:24:42.910+0200] [INFO ] Received sys login status for testusr : logged in
[2024-04-18T08:24:42.911+0200] [INFO ] Received request from xrdp to create a session for user testusr
[2024-04-18T08:24:42.919+0200] [INFO ] Starting X server on display 10: Xorg :10 -auth .Xauthority -config xrdp/xorg_nvidia.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log 
[2024-04-18T08:24:52.924+0200] [ERROR] waitforx: Unable to open display :10
[2024-04-18T08:24:52.925+0200] [ERROR] X server failed to start
[2024-04-18T08:24:52.926+0200] [INFO ] Session on display 10 has finished.

@alexanderzjs
Copy link

I'm trying to run xrdp with gpu acceleration on a headless ubuntu 22.04 server. It is a fresh installation and I have done apt install ubuntu-desktop and then run this script. But the desktop does not load, I get "Can't create session for user testusr - X server could not be started".

What is missing, what am I doing wrong?

logs

As far as I know, if you want a headless server, you have to have a seated session and then configure your Xorg session with a virtual display card. I haven't tried yet, but I have tested it with Sunshine server. I will post the Sunshine server setup and you can adapt it to xrdp. If it works, you can also re-post a xrdp version to help others.

Setup

NOTE: The steps here only need to be performed once on your computer; after these are done, you only need to follow the steps under the "Execution" section.

  1. sudo systemctl set-default multi-user.target (original graphical.target): Set the graphical target to multi-user target so that the server will not give desktop manager on boot. After done, reboot your system.

  2. sudo usermod -a -G tty myusername: The user that will be connecting in via SSH must be in the 'tty' group on the system where Sunshine will be started. This is required because when we start Xorg, it will attempt to access the specific virtual TTY we give it to use. NOTE: You'd better reboot to take effect. Use the 'groups' command to verify that you see 'tty' before proceeding.

  3. sudo setcap CAP_SYS_RAWIO+eip /usr/lib/xorg/Xorg: You need to run the command below to give the capability 'CAP_SYS_RAWIO+eip' to the /usr/lib/xorg/Xorg executable. Note that some more research needs to be done into the implications of doing this but if you don't provide that capability than the Xorg server session will fail with an error: xf86EnableIOPorts: failed to set IOPL for I/O (Operation not permitted)

  4. sudo vim /etc/X11/Xwrapper.conf: You will need to edit the /etc/X11/Xwrapper.conf file and change the 'allowed_users' to 'anybody'. While this will have some security implications they should be managable as this should only be giving the required permission to valid and logged in users to start an Xorg server session. The default, which is typically 'console', requires that you log in with one of the virtual TTYs for the system and this is no different than physically logging into the system anyways (the exact thing we're trying to avoid). Reboot your system to make sure this takes effect before proceeding.

  5. You will need to create a custom Xorg configuration file that is suitable for use with a "headless" session (i.e., one in which we don't really use the physically attached monitor for). To do this follow the steps below:

    1. SSH into the machine where you want to run Sunshine.
    2. vim ~/xorg.conf: Create a new xorg.conf file with the following content:
         Section "ServerLayout"
             Identifier "TwinLayout"
             Screen 0 "metaScreen" 0 0
         EndSection
         
         Section "Monitor"
             Identifier "Monitor0"
             Option "Enable" "true"
         EndSection
         
         Section "Device"
             Identifier "Card0"
             Driver "nvidia"
             VendorName "NVIDIA Corporation"
             Option "MetaModes" "1920x1080"
             Option "ConnectedMonitor" "DP-0"
             Option "ModeValidation" "NoDFPNativeResolutionCheck,NoVirtualSizeCheck,NoMaxPClkCheck,NoHorizSyncCheck,NoVertRefreshCheck,NoWidthAlignmentCheck"
         EndSection
         
         Section "Screen"
             Identifier "metaScreen"
             Device "Card0"
             Monitor "Monitor0"
             DefaultDepth 24
             Option "TwinView" "True"
             SubSection "Display"
                 Modes "1920x1080"
             EndSubSection
         EndSection
      

Execution

NOTE: The steps here need to be performed each time that you want to bring up your Sunshine server. Note that the way the steps are given the entire setup (Xorg server, Sunshine, and desktop session) will be running under the context of your SSH session so once you exit the terminal all these things will exit. You will need to leave your SSH session up while you're using your Sunshine server. An alternative would be to use screen so that these processes stay up after you exit your terminal connection.

  1. SSH into the machine where you want to bring up the Sunshine server.
  2. Decide which virtual TTY (1-5) you want to tie the Xorg session to based on which are unused (we assume we use tty2). Note that TTYs 1-5 should correspond to the virtual teletype consoles that you can drop into if you were to use the key sequence [Ctrl] + [Alt] + [F1] (or [F2], etc).
  3. sudo chmod g+r /dev/tty2: Add read access to the group on its file descriptor.
  4. sudo Xorg -noreset +extension GLX +extension RANDR +extension RENDER vt2 -logfile /home/YOURUSERNAME/X_LOG.txt -config /home/YOURUSERNAME/xorg.conf :9 >> /home/YOURUSERNAME/std_stream.out 2>&1 &: Bring up an Xorg server and run it in the shell background.
  5. export DISPLAY=":9": In your shell, set the DISPLAY variable you specified when bringing up the Xorg server. This number is set in the previous command where [:x] is the number.
  6. sunshine &: Start Sunshine in the background.
  7. exec gnome-session or exec startplasma-x11: Start your desktop environment session.

Hope the above works!!!

References:
https://chadrick-kwag.medium.com/configuring-x-server-on-headless-server-a8b5c3bc7d9
https://unix.stackexchange.com/questions/503078/start-headless-x-server-as-non-root-user
https://www.howtogeek.com/428174/what-is-a-tty-on-linux-and-how-to-use-the-tty-command/
https://unix.stackexchange.com/questions/159124/xf86enableioports-failed-to-set-iopl-for-i-o-operation-not-permitted

@tokon2000
Copy link

@alexanderzjs
Thanks for your post.
I got it working on my local test machine with xserver-xorg-video-dummy and by adding a monitor to xorg_nvidia.conf.
It still does not work on the server, but I think it is a GPU/cuda/driver issue.

@alexanderzjs
Copy link

@alexanderzjs Thanks for your post. I got it working on my local test machine with xserver-xorg-video-dummy and by adding a monitor to xorg_nvidia.conf. It still does not work on the server, but I think it is a GPU/cuda/driver issue.

Great! Adding a virtual monitor is definitely the solution.

@BIGheadLL
Copy link

Hi @Nexarian, thank you for the Nvidia GPU setup. It works for me. However, I get 2 Xorg and gnome-shell processes at the same time when checking nvidia-smi. Is that normal? Is it any possible to optimize this one? Many thanks.
Screenshot 2024-04-25 at 10 24 33 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment