Nvidia_CUDA_cuDNN_nvidia-docker2.md

Nvidia, CUDA, cuDNN and nvidia-docker2 installation on Ubuntu 22.04.1 LTS

Installation steps of nvidia-driver, CUDA_Toolkit, cuDNN and nvidia-docker2 package which enables docker containers to use host's GPU

1. Check GPU

lspci | grep -i VGA

2. Nvidia Driver installation

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
ubuntu-drivers devices  # it shows you nvidia-driver versions that you can install
sudo apt install nvidia-driver-515  # instead of 515 you can choose other version
reboot
# Make sure Secure_Boot in BIOS is disabled!
nvidia-smi  # it shows you GPU_info with CUDA version which you should install.

3. CUDA installation

From cuda-toolkit-archive, download the proper CUDA_Toolkit version which is shown on the output of nvidia-smi command and install it as shown on the website. Then add these two lines to the end of ~/.bashrc file: (If your cuda version is different from 11.7 change it below)

• export PATH=/usr/local/cuda-11.7/bin:$PATH
• export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda-11.7/lib64:$LD_LIBRARY_PATH

reboot
nvcc -V  # it shows CUDA_Toolkit info 

4. cuDNN installation

From cudnn-archive, download cuDNN which matches CUDA version.(Local Installer for Linux x86_64 (Tar))
Then run these commands from the folder that the file downloaded:

tar -xf cudnn-linux-x86_64-8.6.0.163_cuda11-archive.tar.xz
cd cudnn-linux-x86_64-8.6.0.163_cuda11-archive
sudo cp ./include/cudnn* /usr/local/cuda/include/
sudo cp ./lib/libcudnn* /usr/local/cuda/lib64/
sudo chmod a+r /usr/local/cuda/include/cudnn* /usr/local/cuda/lib64/libcudnn*
# Check it
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2  # it shows you cuDNN version

That's all! Now you can utilize Nvidia-GPU for your Machine Learning or Deep Learning projects.

If you want to also use Nvidia-GPU with docker containers then continue...

---

To use nvidia GPU with docker container:

Make sure you’ve got the NVIDIA drivers working properly on your host before you continue with your Docker configuration. You should be able to successfully run nvidia-smi and see your GPU’s name, driver version, and CUDA version on your host. To use your GPU with Docker, begin by adding the NVIDIA Container Toolkit to your host. This integrates into Docker Engine to automatically configure your containers for GPU support. Add the toolkit’s package repository to your system using the example command:

Setting up NVIDIA Container Toolkit

Setup the package repository and the GPG key:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/experimental/$distribution/libnvidia-container.list | \
         sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
         sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Next install the nvidia-docker2 package on your host:

sudo apt-get update
sudo apt-get install -y nvidia-docker2

You need to enable nvidia runtime as default by adding "default-runtime": "nvidia" in Docker Daemon settings (/etc/docker/daemon.json) as below:

{
 "default-runtime": "nvidia",
 "runtimes": {
   "nvidia": {
     "path": "nvidia-container-runtime",
     "runtimeArgs": []
   }
 }
}

Restart the Docker daemon to complete the installation:

sudo systemctl restart docker

Run nvidia-smi command in a docker container. It shows the same output as you saw on your host.
From now on you can utilize host's GPU in your docker containers!