Ayke/cuda-wsl2-ubuntu.md

## cuda-wsl2-ubuntu.md

      
    Raw
  

              cuda-wsl2-ubuntu.md
            
          
    Prerequisites, i.e. the most important things


Time of writing: Jan 18, 2023. The following assume you're trying to install CUDA on WSL2 Ubuntu.


Check support matrix first before you install any version of CUDA, because chances are the latest CUDA does not have cuDNN support yet, then you'll have to re-install older version if you found out later.
https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html#cudnn-cuda-hardware-versions
At the time of writing, the latest cuDNN version is 8.7 and it supports CUDA 11.8.


Windows 10 must be build 20145 or later.
At the time of writing, the latest Windows 10 (22H2) Release Build is 19045.2486. If your current Windows 10 build version is older than this, then from official options it is impossible to upgrade to this version, even if you have enrolled Insider Program.
Your only shot will be to download unofficial images from uupdump or adguard then install. But I do not recommend it, because Windows 10 will find that your image has expired and you'll be receiving endless annoying alerts.
The bottom line is, if your build version is older than 20145, you'd better just upgrade to Windows 11. You'll probably need to enable TPM 2.0 from BIOS for that.


WSL must be WSL2
At the time of writing, WSL hasn't been supported by CUDA yet, and the support for WSL2 is somewhat limited too.


WSL2 Ubuntu Kernel version must be at least 4.19.121
wsl cat /proc/version to check your WSL Ubuntu's kernel version. It must be at least 4.19.121, better be 5.10.16.3 or later.
If your WSL Ubuntu kernel version is too old:
sudo apt update && sudo apt upgrade
sudo do-release-upgrade

will upgrade your kernel.


Overall procedure

The key point to installation of CUDA in WSL2 is that, you MUST match three things together:

cuDNN version
Ubuntu CUDA runtime version
Windows CUDA library version

First you should look for the latest version of cuDNN, this determines all other stuff's version (Graphics Driver, CUDA Runtime etc.)
Since Windows Nvidia Graphics Driver does not explicitly tell you which version of CUDA it includes, you can only guess its CUDA version based on CUDA release date:

Pick the latest cuDNN, then look for the range of CUDA versions it supports.
Check the release date of those CUDA versions
Search & Install a Graphics Drivers whose CUDA is supposed to be supported by cuDNN.
After installation of the Graphics Driver, in your Ubuntu bash run nvidia-smi (under /usr/lib/wsl/lib) and check its CUDA version.
Install CUDA runtime of that version based on https://docs.nvidia.com/cuda/wsl-user-guide/index.html#cuda-support-for-wsl-2
Install cuDNN in WSL2 Ubuntu (again, tar installation recommended)

Documentation

I'll skip the part where you install Windows Grphic Driver.
Check if Windows Graphic Driver is installed right

Goto C:\Windows\System32\lxss\lib and see if the folder is empty. It should contain a lot of .so files (runtime library)
Also, in Ubuntu bash, which nvidia-smi should point to /usr/lib/wsl/lib/nvidia-smi, it shall be working. Whatever it shows will be the actual CUDA version in your Windows.
Check if CUDA runtime is installed right

nvcc --version should work. which nvcc should point to somewhere like /usr/local/cuda/bin/nvcc. But you need to make sure /usr/local/cuda is pointing to the right cuda runtime (the one whose version matches Windows Nvidia Driver you just installed, not any cuda runtime you may have installed before).
nvcc --version should match whatever version shows in nvidia-smi.
Check if cuDNN is installed right

This is included in cuDNN's installation documentation.
Install CUDA

https://docs.nvidia.com/cuda/wsl-user-guide/index.html
Install the latest CUDA which has cuDNN support.
sudo apt list cuda --all-versions

To see all available cuda versions.
sudo apt-get install cuda=11.7.1-1

To install certain version of cuda.
Path

You still need to set up $PATH so that the CUDA runtime may work correctly.
Oh you should also make sure your path has included /usr/lib/wsl/lib, if it's not in your path,
probably your kernel version is not high enough.
In ~/.bashrc, add the following:
if [ -d $HOME/.local/bin ]; then
        export PATH=$HOME/.local/bin:$PATH
fi

export CUDA_HOME=/usr/local/cuda

if [ -d $CUDA_HOME/bin ]; then
        export PATH=$CUDA_HOME/bin:$PATH
fi

if [ -d $CUDA_HOME/lib64 ]; then
        export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
fi


then restart the terminal or source ~/.bashrc. The first path is actually for pip, but I included anyway.
I do not recommend manually add /usr/lib/wsl/lib into your LD_LIBRARY_PATH because it should be automatically managed under /etc/ld.so.conf.d/ld.wsl.conf which is automatically generated by WSL. If you don't have it then it's probably a sign that your WSL needs update (or re-install after uninstall)
Install cuDNN

https://developer.nvidia.com/rdp/cudnn-download
https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#install-linux
tar install is the most reliable way to install cuDNN, I personally do not recommand deb installation.
If you're installing cuDNN from deb, several useful commands:
(the reason why I don't recommend it is I've tried it)
To check all cuDNN versions in package manager:
sudo apt list libcudnn8 --all-versions

If you can't find FreeImage.h, try
sudo apt-get install libfreeimage3 libfreeimage-dev

To remove a repository
sudo dpkg -P <the deb you used to install>

And you may check /etc/apt/sources.list.d/ if your sudo apt update keeps getting blocked
Debug clues


Use pytorch's collect-env.py to verify your environment.
/usr/lib/wsl/bin is actually C:\Windows\System32\lxss\lib, your .so files got installed when you install Nvidia Graphics Driver in Windows.
Make sure your links such as /usr/local/cuda are pointing to the right positions.