Skip to content

Instantly share code, notes, and snippets.

@tdcosta100
Last active March 8, 2024 07:34
Star You must be signed in to star a gist
Save tdcosta100/385636cbae39fc8cd0937139e87b1c74 to your computer and use it in GitHub Desktop.
A tutorial to use GUI in WSL2 replacing original XServer by Xvnc, allowing WSL to work like native Linux, including login screen

WSL2 with GUI using Xvnc

In this tutorial, we will setup GUI in WSL2, and access it using VNC. No additional software outside WSL (like VcXsrv) is required, except, of course, a VNC Viewer (RealVNC, TightVNC, TigerVNC, UVNC, etc, all of them might work flawlessly).

The key component we need to install is tigervnc-standalone-server.

For this setup, I will use Ubuntu 20.04 LTS (Focal Fossa, unfortunately 22.04 does not work), and install GNOME Desktop. Since the key components aren't bound to Ubuntu or GNOME, you can use your favorite distro and GUI. Check the Sample screenshots section for examples.

So let's go. First, we need a working WSL2 installation.

Then you must disable WSLg, because unfortunately it messes with the graphical internal workings of Linux. You can make them coexist, but it's painful. Do it at your own if you want to. Don't forget to reboot your computer after disabling WSLg, just shutting WSL down is not enough to bring WSLg down.

Before going to real business, let's make sure we are updated.

sudo apt update
sudo apt upgrade

If you are trying to use Debian, you also will need:

sudo apt install curl wget

Now we are ready to go.

Installing components

Installing GUI

  1. Ubuntu has a handy component installer called tasksel, but it's not installed by default. So let's install it.

    sudo apt install tasksel
    
  2. Once we have it installed, let's run it.

    sudo tasksel
    
  3. At the package list, select your favorite GUI package. Use spacebar to select the package, and Tab to confirm your choice. I selected Ubuntu Desktop. The installation will take a while, so be patient.

Installing VNC Server

Pretty easy, one command and you are done:

sudo apt install tigervnc-standalone-server

The installation process is finally done.

Configuring the environment

Creating VNC Server passwords

  1. In this setup, each user has a different VNC password. So you have to configure at least three passwords, one for the current user, other for root, and other for gdm, who whill present the login screen. If you don't configure the password, you won't able to access the login screen, or the user's desktop. First, let's configure the VNC password current user:

    vncpasswd
    
  2. Now, let's configure the VNC password for root (needed if you use LightDM instead GDM):

    sudo -H vncpasswd
    
  3. Finally, let's configure the VNC password for GDM (you need to skip this step if you didn't install GNOME):

    sudo -H -u gdm vncpasswd
    

    Tip: In Debian, the GDM user is Debian-gdm.

    You can repeat the process for other existing users.

Replacing default X by Xvnc

By default, the display manager call multiple X instances, one for each user session, including the login screen, provided by gdm. So we will replace Xorg script by a new version which calls Xvnc instead the classic X/Xorg. This IS the real magic we are trying to do.

  1. First, let's backup the original Xorg script.

    sudo mv /usr/bin/Xorg /usr/bin/Xorg_old
    
  2. Then, we create a new Xorg script.

    sudo nano /usr/bin/Xorg_new
    
  3. Paste the following content in the editor (remember, WSL shell uses CTRL + SHIFT + V for pasting, instead our traditional CTRL + V key combination):

    #!/bin/bash
    for arg do
      shift
      case $arg in
        # Xvnc doesn't support vtxx argument. So we convert to ttyxx instead
        vt*)
          set -- "$@" "${arg//vt/tty}"
          ;;
        # -keeptty is not supported at all by Xvnc
        -keeptty)
          ;;
        # -novtswitch is not supported at all by Xvnc
        -novtswitch)
          ;;
        # other arguments are kept intact
        *)
          set -- "$@" "$arg"
          ;;
      esac
    done
    
    # Here you can change or add options to fit your needs
    command=("/usr/bin/Xvnc" "-geometry" "1024x768" "-PasswordFile" "${HOME:-/root}/.vnc/passwd" "$@")
    
    systemd-cat -t /usr/bin/Xorg echo "Starting Xvnc:" "${command[@]}"
    
    exec "${command[@]}"
    

    Please note the resolution of the virtual screen. You can change that to fit your needs (1366x768, 1920x1080, etc). Also, you can change the -PasswordFile option to point to a fixed location instead the home of current user, so you don't need to have a password for each user.

  4. Finally, we set the correct permissions for the file and create a link to it:

    sudo chmod 0755 /usr/bin/Xorg_new
    sudo ln -sf Xorg_new /usr/bin/Xorg
    

    Attention: sometimes, system updates replace Xorg link with the original version. Just repeat this step if this happens, and Xvnc will work again as Xorg replacement.

Enabling systemd in your distro

WSL normally boots a minimal version of the system, without systemd. In order to enable using all the good things we are used to Linux, we must enable systemd:

  1. Create the file /etc/wsl.conf:

    sudo nano /etc/wsl.conf
    
  2. Paste the following content in the editor:

    [boot]
    systemd=true
    

Important: If you are using Debian, you need one extra step:

echo "LANG=en_US.UTF-8" | sudo tee -a /etc/default/locale

Running your distro with systemd

Now you have everything ready to start. Open a Command Prompt or a Windows Powershell window and shut down WSL:

wsl --shutdown

Then reopen your distro window.

Doing this is like booting Linux again, this time with systemd. Because of systemd, gdm will start automatically, and will create a X instance to display the login interface. We changed this process to make it create Xvnc instances, so we can access them. The first instance will listen to port 5900, the second instance will listen to port 5901, and so on.

Attention: systemd keeps running only if you have at least one distro shell Window (in Windows, not inside VNC) open. If you close all shell windows (outside VNC), your gui will shut down. If you want to prevent this from occurring, you can consider bottle-imp, but you have to compile it for Ubuntu 20.04.

Accessing the VNC screen

After a while (usually 30 seconds, but it can take more if you don't have a SSD, maybe one minute or two), you can test if it's working properly. Use your favorite VNC Viewer to connect to your localhost port 5900. Use the VNC password set for user gdm. The login screen must appear.

After logging in, the screen will be blank. This is because a new instance of Xvnc was created for user desktop, listening to port 5901. Connect to this screen now. The logged user's desktop must appear. When you log out, the screen at port 5900 will show the login interface again. This applies to GDM (which is the case if you installed Ubuntu Desktop). You can change this behavior changing the configuration file like this:

  1. sudo nano /etc/gdm3/custom.conf

  2. Uncomment and edit the following lines:

     AutomaticLoginEnable=true
     AutomaticLogin=[your username without the brackets]
    

If you are using LightDM, the desktop screen will appear in port 5900, so there's no need to connect to port 5901.

Shutting down

One important thing is: once you start systemd services, you cannot just stop your Linux instance. You must perform a standard Linux shutdown. You can do one of one alternatives below:

  • Power off option on GUI menu
  • sudo poweroff

After doing that, you can safely shut down your WSL instance, either by wsl --terminate or wsl --shutdown. Not doing the shutdown process may cause damage to your WSL instance. So be careful.

Tips and tricks

  • VNC is a very adaptive protocol, and by default tries to use the most aggressive compression available, for better networking performance. But in our case, doing this only adds unnecessary CPU extra load, since you are connecting to localhost ("infinite" bandwith, near zero ping). To remove all compression algorithms and reduce lagging, force your VNC Viewer to connect using RAW encoding. The performance gain is noticeable, specially when playing videos (not a spectacular performance in this particular scenario, though).

  • If you are trying to use Debian, you need to do the following or some apps will not launch (at least in Buster):

    echo "LANG=en_US.UTF-8" | sudo tee -a /etc/default/locale
    

Troubleshooting

  1. If it doesn't work at first, try to check your journalctl logs:

    journalctl -b -t /usr/lib/gdm3/gdm-x-session -t /usr/bin/Xorg --no-pager
    

    If you are using Debian, then the command is:

    journalctl -b -t /usr/libexec/gdm-x-session -t /usr/bin/Xorg --no-pager
    

    In the output, you must see what command line was generated for Xvnc, and which error messages appear. Of course, even if it works correctly, you can check the logs just to see what is happening, or for debugging.

  2. You must check if the custom Xorg script was not replaced by the default version of it. If it was the case, just repeat the steps of Replacing default X by Xvnc section.

  3. Check if Xorg is your default display server, not Xephyr or Wayland. If it's not, you must change it to have Xorg as your default display server.

  4. If you are using LightDM, you also need to check logs at /var/log/lightdm (you will need to use sudo to cat files in that directory). The Xvnc output will be in the file /var/log/lightdm/x-0.log.

  5. If you can connect to 59XX ports, but receive an error like Authentication failure: No password configured for VNC Auth, the file $HOME/.vnc/passwd is missing for that particular user (on port 5900, the user is gdm). Try to repeat the steps described in section Creating VNC Server passwords and try to connect again.

  6. If it still doesn't work, you can try to restart WSL. Open a Windows PowerShell command prompt, and execute the following command (don't forget to save everything that is unsaved before, because WSL will shut down completely):

    PS > wsl --terminate <your distro name>
    

    After that, open your distro shell again and repeat the steps of section Running systemd-genie.

Sample screenshots

GDM

GDM

LightDM

LightDM

GNOME

GNOME

KDE

KDE

Xfce

Xfce

Budgie Desktop

Budgie Desktop

Contributors

Thanks to this guys, whose feedback made this tutorial reach the current level of quality and completeness (and it will be more and more complete as more feedback is given).

@SathvikKS
Copy link

SathvikKS commented Mar 23, 2023

I'm getting this error

Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransSocketCreateListener: failed to bind listener
Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransMakeAllCOTSServerListeners: failed to create listener for unix
Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransmkdir: Mode of /tmp/.X11-unix should be set to 1777

WSL info

WSL version: 1.1.3.0
Kernel version: 5.15.90.1
WSLg version: 1.0.49
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.19045.2788

Ubuntu details

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal

EDIT:
I manually remounted with rw permission using

sudo mount -o remount,rw /tmp/.X11-unix

and then restarted gdm with

sudo service gdm3 restart

However, this gives new issue

Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37271]: dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]:  ComparingUpdateTracker: 0 pixels in / 0 pixels out
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]:  ComparingUpdateTracker: (1:-nan ratio)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) Backtrace:
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) 0: /usr/bin/Xvnc (OsLookupColor+0x13c) [0x562f144a339c]
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) 1: /lib/x86_64-linux-gnu/libpthread.so.0 (funlockfile+0x60) [0x7f050865d47f]
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) 2: /usr/lib/wsl/drivers/nvmiig.inf_amd64_7b850875618e1cb3/libnvwgf2umx.so (_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6__initIPmEENS_9enable_ifIXsr21__is_forward_iteratorIT_EE5valueEvE4typeES9_S9_+0xff3d4c) [0x7f04fe58f7b8]
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) Segmentation fault at address 0x7f04faafa000
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: Fatal server error:
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) Caught signal 11 (Segmentation fault). Server aborting
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)

@tdcosta100
Copy link
Author

Hi, @SathvikKS. It's complicated to use this tutorial with WSLg enabled and it demands extra steps. This tutorial is intended to be done with WSLg disabled.

@4t8dd
Copy link

4t8dd commented Sep 4, 2023

I'm getting this error

Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransSocketCreateListener: failed to bind listener
Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransMakeAllCOTSServerListeners: failed to create listener for unix
Mar 23 11:10:48 MSI /usr/lib/gdm3/gdm-x-session[821]: _XSERVTransmkdir: Mode of /tmp/.X11-unix should be set to 1777

WSL info

WSL version: 1.1.3.0
Kernel version: 5.15.90.1
WSLg version: 1.0.49
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.19045.2788

Ubuntu details

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal

EDIT: I manually remounted with rw permission using

sudo mount -o remount,rw /tmp/.X11-unix

and then restarted gdm with

sudo service gdm3 restart

However, this gives new issue

Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37271]: dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]:  ComparingUpdateTracker: 0 pixels in / 0 pixels out
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]:  ComparingUpdateTracker: (1:-nan ratio)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) Backtrace:
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) 0: /usr/bin/Xvnc (OsLookupColor+0x13c) [0x562f144a339c]
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) 1: /lib/x86_64-linux-gnu/libpthread.so.0 (funlockfile+0x60) [0x7f050865d47f]
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) 2: /usr/lib/wsl/drivers/nvmiig.inf_amd64_7b850875618e1cb3/libnvwgf2umx.so (_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6__initIPmEENS_9enable_ifIXsr21__is_forward_iteratorIT_EE5valueEvE4typeES9_S9_+0xff3d4c) [0x7f04fe58f7b8]
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) Segmentation fault at address 0x7f04faafa000
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: Fatal server error:
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE) Caught signal 11 (Segmentation fault). Server aborting
Mar 23 11:40:13 MSI /usr/lib/gdm3/gdm-x-session[37189]: (EE)

sudo umount /tmp/.X11-unix
ln -sf /mnt/wslg/.X11-unix/X0 /tmp/.X11-unix/

This fixs my issue. from here: microsoft/WSL#9303

@tdcosta100
Copy link
Author

This is happening because you are trying to make WSLg and GNOME coexist. Usually it doesn't work. If you disable WSLg, then you can use GNOME normally.

@nobody4t
Copy link

nobody4t commented Sep 6, 2023

Oh, this killed me almost.

@kchovi
Copy link

kchovi commented Oct 5, 2023

Hello im getting this screen after a few connections, however im able to use gnome normally after completely unnistaling and reinstalling the distro.
Any help is appreciated, i am absoluttely clueless

Thanks
384557394_889679089248684_955829697391718332_n

@tdcosta100
Copy link
Author

I'm sorry to getting so long to answer you. Please tell me what distro are using, and the version (for example, Ubuntu 22.04).

@kchovi
Copy link

kchovi commented Oct 9, 2023

I'm sorry to getting so long to answer you. Please tell me what distro are using, and the version (for example, Ubuntu 22.04).

no problemo, im using Debian 12

@xinxilas
Copy link

xinxilas commented Oct 13, 2023

Está dando problema com o tasksel: tasksel apt-get failed (100)
Logo depois de selecionar o Ubuntu desktop ele retorna aquilo no terminal
E se voce explicar uma forma alternativa de instalar o Ubuntu Desktop, ou só o xfce4 e algum outros exemplos?

@tdcosta100
Copy link
Author

I'm sorry to getting so long to answer you. Please tell me what distro are using, and the version (for example, Ubuntu 22.04).

no problemo, im using Debian 12

@kchovi, unfortunately Debian 12 suffers from the same problem Ubuntu 22.04 does: it is so bonded to WSLg that it's somewhat broken with the setup I propose in this tutorial. So if you want to make it work like intended in this tutorial, you must use Debian 11.

@tdcosta100
Copy link
Author

Está dando problema com o tasksel: tasksel apt-get failed (100) Logo depois de selecionar o Ubuntu desktop ele retorna aquilo no terminal E se voce explicar uma forma alternativa de instalar o Ubuntu Desktop, ou só o xfce4 e algum outros exemplos?

Olá, @xinxilas. Para suporte em português, por gentileza poderia repetir esse post nesse outro tutorial?

https://gist.github.com/tdcosta100/c5c206f8958e82e638fa0ea226351081

@osseigh
Copy link

osseigh commented Dec 5, 2023

Finished everything...what command do I need to start it...in kali win-kex there is a command but I don't see one here

@spinell
Copy link

spinell commented Dec 6, 2023

Hi @tdcosta100, awesome tuto. I give it a try on Rocky8 with lightdm + Xfce, Mate, Cinnamon and it look to work welll.
But I have a few question (I know you used ubuntu so maybe it a bit different).

  • The first think I realizes is once I'm inside a desktop environment, I'm not able o switch user and come back to lightdm.
  • If the manually start light dm systemctl start lightdm it work. But if I try to enable it at startup systemctl enable lightdm. It does not work. Lightdm does not start and I don't havelogs in /var/log/lightdm

@tdcosta100
Copy link
Author

tdcosta100 commented Dec 12, 2023

Finished everything...what command do I need to start it...in kali win-kex there is a command but I don't see one here

Hi, @osseigh. If you are using the default setup (Ubuntu 20.04), you just need to restart WSL, and open it in the Terminal. This will be enough to start everything. If you are using another Ubuntu version, or another distro, please let me know so I can assist you better.

@tdcosta100
Copy link
Author

Hi @tdcosta100, awesome tuto. I give it a try on Rocky8 with lightdm + Xfce, Mate, Cinnamon and it look to work welll. But I have a few question (I know you used ubuntu so maybe it a bit different).

  • The first think I realizes is once I'm inside a desktop environment, I'm not able o switch user and come back to lightdm.
  • If the manually start light dm systemctl start lightdm it work. But if I try to enable it at startup systemctl enable lightdm. It does not work. Lightdm does not start and I don't havelogs in /var/log/lightdm

Hi, @spinell. What image do you have installed? According to this page, there are 3 options: Base x86_64, Minimal x86_64 and UBI x86_64. I need to know which one are you using (or another) to reproduce your scenario.

@spinell
Copy link

spinell commented Dec 12, 2023

Hi @tdcosta100 , I'm using Base x86_64

@Barbaduque
Copy link

Rapaz eu to com a mesma tela to quase desistalando e instalandod e novo o sistema, ta foda

@keghan001
Copy link

keghan001 commented Mar 7, 2024

Please how do i revert back into using the original Xserver, do i have to just change the config in .wslconfig or it would conflict. So how do I revert back totally.
Or is there a way to switch inbetween them seamlessly enabeling and disabelling when needed

@keghan001
Copy link

Please, Everytime i open a gui app like Gazebo or Rviz in the 5901 port it immediately crashes and I have to login on the 5900 port again and it keeps on crashing are there any additional settings or help to resolve this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment