Skip to content

Instantly share code, notes, and snippets.

@denguir
Last active April 29, 2024 19:11
Show Gist options
  • Save denguir/b21aa66ae7fb1089655dd9de8351a202 to your computer and use it in GitHub Desktop.
Save denguir/b21aa66ae7fb1089655dd9de8351a202 to your computer and use it in GitHub Desktop.
Installation procedure for CUDA & cuDNN

How to install CUDA & cuDNN on Ubuntu 22.04

Install NVIDIA drivers

Update & upgrade

sudo apt update && sudo apt upgrade

Remove previous NVIDIA installation

sudo apt autoremove nvidia* --purge

Check Ubuntu devices

ubuntu-drivers devices

You will install the NVIDIA driver whose version is tagged with recommended

Install Ubuntu drivers

sudo ubuntu-drivers autoinstall

Install NVIDIA drivers

My recommended version is 525, adapt to yours

sudo apt install nvidia-driver-525

Reboot & Check

reboot

after restart verify that the following command works

nvidia-smi

Install CUDA drivers

Update & upgrade

sudo apt update && sudo apt upgrade

Install CUDA toolkit

sudo apt install nvidia-cuda-toolkit

Check CUDA install

nvcc --version

Install cuDNN

Download cuDNN .deb file

You can download cuDNN file here. You will need an Nvidia account. Select the cuDNN version for the appropriate CUDA version, which is the version that appears when you run:

nvcc --version

Install cuDNN

sudo apt install ./<filename.deb>
sudo cp /var/cudnn-<something>.gpg /usr/share/keyrings/

My cuDNN version is 8, adapt the following to your version:

sudo apt update
sudo apt install libcudnn8
sudo apt install libcudnn8-dev
sudo apt install libcudnn8-samples

Test CUDA on Pytorch

Create a virtualenv and activate it

sudo apt-get install python3-pip
sudo pip3 install virtualenv 
virtualenv -p py3.10 venv
source venv/bin/activate

Install pytorch

pip3 install torch torchvision torchaudio

Open Python and execute a test

import torch
print(torch.cuda.is_available()) # should be True

t = torch.rand(10, 10).cuda()
print(t.device) # should be CUDA
@RedNoseJJN
Copy link

do you also find that when you run sudo apt install nvidia-cuda-toolkit your nvidia-driver-XXX is removed and nvidia-smi no longer works?

exactly, that's weird and I don't know why. I just simply restarted everything: removing everything and install them from the very beginning.

@Sh-Pe
Copy link

Sh-Pe commented Oct 1, 2023

Thanks.

@Dimacat-exe
Copy link

How to update cuda and cudnn sir?

@iukash
Copy link

iukash commented Oct 8, 2023

Thank you

@RedNoseJJN
Copy link

How to update cuda and cudnn sir?

I just simply removed everything and re-install the version I want. Uninstall Reference

@n0one42
Copy link

n0one42 commented Oct 10, 2023

I just used sudo apt install nvidia-cudnn on my ubuntu server to install cudnn
And my nextcloud instance recognized everything and it uses now my gpu

@driekus77
Copy link

Thanks!

@praetp
Copy link

praetp commented Oct 15, 2023

Thanks

@ckulkarni97
Copy link

great ! Useful ! !

@younesraoui
Copy link

Thanks

@stani3
Copy link

stani3 commented Oct 25, 2023

this guy deserves a special place in heaven.

@n0one42
Copy link

n0one42 commented Nov 4, 2023

Also maybe someone of you would also like to install a custom ffmpeg with the correct codecs. This took me quiet a while to bring it to work. It was tested on ubuntu 22.04
This was for testing so the ansible tasks are not rly perfect but works. Also you should be able to easily copy paste the commands manually in the shell.

---
# https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
# https://www.cyberciti.biz/faq/how-to-install-ffmpeg-with-nvidia-gpu-acceleration-on-linux/
# https://trac.ffmpeg.org/wiki/HWAccelIntro
# ? Infos -> ~/ffmpeg_nvidia/ffmpeg/./configure --help | more
# ? Infos -> ~/ffmpeg_nvidia/ffmpeg/./configure --help | grep -E '(libvpx|ibx264)'
# for i in encoders decoders filters; do
#     echo $i:; ./ffmpeg -hide_banner -${i} | egrep -i "npp|cuvid|nvenc|cuda|nvdec"
# done
# ffmpeg -t 300 -f lavfi -i testsrc=size=1920x1080:rate=30 -c:v h264_nvenc -preset fast output_test_5min_gpu.mp4
# wget https://file-examples.com/storage/fe1734aff46541d35a76822/2017/04/file_example_MP4_1920_18MG.mp4 -O input.mp4
# ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -c:v h264_nvenc -preset slow output.mp4
 # Full hardware transcode with NVDEC and NVENC
- name: Install dependencies
  ansible.builtin.apt:
    update_cache: true
    pkg:
      - build-essential
      - cmake
      - libass-dev
      - libgnutls28-dev
      - libmp3lame-dev
      - libvorbis-dev
      - libaom-dev
      - libdav1d-dev
      - nasm
      - libx264-dev
      - libx265-dev
      - libnuma-dev
      - libvpx-dev
      - libfdk-aac-dev
      - libopus-dev
    state: present

- name: Codecs
  ansible.builtin.shell: |
    mkdir /home/tav/nvidia/ && cd /home/tav/nvidia/ && \
    git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git && \
    cd nv-codec-headers && sudo make install && \
    cd /home/tav/nvidia/ && \
    git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg/ && \
    cd /home/tav/nvidia/ffmpeg/
  args:
    executable: /bin/bash

- name: Compile and install FFmpeg
  ansible.builtin.shell: |
    mkdir /home/tav/my_ffmpeg_build/ && cd /home/tav/my_ffmpeg_build/ && \
    /home/tav/nvidia/ffmpeg/configure \
    --extra-cflags=-I/usr/lib/cuda/include \
    --extra-ldflags=-L/usr/lib/cuda/lib64 \
    --enable-libnpp \
    --enable-nvenc \
    --enable-cuvid \
    --enable-cuda-nvcc \
    --enable-gpl \
    --enable-gnutls \
    --enable-libaom \
    --enable-libass \
    --enable-libfdk-aac \
    --enable-libfreetype \
    --enable-libmp3lame \
    --enable-libopus \
    --enable-libvorbis \
    --enable-libvpx \
    --enable-libx264 \
    --enable-libx265 \
    --enable-libdav1d \
    --enable-nonfree && \
    make -j $(nproc)
    sudo make install
  args:
    executable: /bin/bash

@Atlas3DSS
Copy link

Atlas3DSS commented Nov 6, 2023

So following the instructions works just great - all installed no problem - but i realize that i cannot seem to get it to NOT install 11.5 - which is a very old version and not compatible with flashattn2 which requires 11.6 and up - ive been banging my head against the wall all day - downloaded .run files for older versions but then nvcc won't register cause its not the right cuda tools or something. Any help would be greatly appreciated.

-- solved ended up being a PATH issue. Also needed to fully purge ALL nvidia related package handlers AND do it all while GUI was turned off.

@AFirooz
Copy link

AFirooz commented Nov 8, 2023

For people who use Poetry, I noticed I'm getting exceptions when importing torch (even though I added it to my environment).
I found out that if I force pip install torch in the virtual environment (after adding torch in poetry) solves the problem. So basically run:

$ poetry add torch torchvision torchaudio
$ poetry shell
$ pip install torch torchvision torchaudio

@moodz
Copy link

moodz commented Nov 10, 2023

I used ...
sudo apt install nvidia-driver-470

However when I install the cuda toolkit later in the process the 470 ( recommended ) drivers are uninstalled ?????

The following packages will be REMOVED:
libnvidia-compute-470 libnvidia-compute-470:i386 libnvidia-decode-470
libnvidia-decode-470:i386 libnvidia-encode-470 libnvidia-encode-470:i386
nvidia-compute-utils-470 nvidia-driver-470 nvidia-utils-470
The following NEW packages will be installed:
libcuinj64-11.5 libnvidia-compute-495 libnvidia-compute-510
libnvidia-compute-525 libnvidia-ml-dev nsight-systems nsight-systems-target
nvidia-cuda-dev nvidia-cuda-toolkit nvidia-profiler nvidia-visual-profiler

This breaks my hardware support at runtime ( unsupported HW errors ).

Any clues ?

@MilesQLi
Copy link

Is it possible to specify the CUDA version?

@cse-repon
Copy link

thanks a lot!

@sitexa
Copy link

sitexa commented Nov 22, 2023

nvidia-smi
Failed to initialize NVML: Driver/library version mismatch
NVML library version: 545.23

How to fix this problem?
I have previously installed driver-545.23, then removed it , and installed driver-535. Then failed of version mismatch.
Or, how to clean NVML library completely?

@cse-repon
Copy link

nvidia-smi
Failed to initialize NVML: Driver/library version mismatch
NVML library version: 545.23

How to fix this problem? I have previously installed driver-545.23, then removed it , and installed driver-535. Then failed of version mismatch. Or, how to clean NVML library completely?

sudo apt-get purge '.nvidia.'
sudo apt remove '.nvidia.'

then followed this gist
and installed this version

sudo apt install nvidia-driver-525

@dcerisano
Copy link

dcerisano commented Nov 23, 2023

If anyone is interested to see a clean install log of these instructions, please see log below.
I followed them to the letter and got the correct result.
Note that this was on a fresh install of Ubuntu Server 22.04, with Gnome desktop and Nvidia drivers installed immediately afterwards.
(I use Ubuntu Server to I can easily congfure my workstation NVME drives in RAID0)

Other than the WARNING: Running pip as the 'root' user these instructions cause, seems fine thanks.

kronos@kronos:~/Downloads$ sudo apt install ./cudnn-local-repo-ubuntu2204-8.9.6.50_1.0-1_amd64.deb 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'cudnn-local-repo-ubuntu2204-8.9.6.50' instead of './cudnn-local-repo-ubuntu2204-8.9.6.50_1.0-1_amd64.deb'
The following NEW packages will be installed:
  cudnn-local-repo-ubuntu2204-8.9.6.50
0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded.
Need to get 0 B/887 MB of archives.
After this operation, 887 MB of additional disk space will be used.
Get:1 /home/kronos/Downloads/cudnn-local-repo-ubuntu2204-8.9.6.50_1.0-1_amd64.deb cudnn-local-repo-ubuntu2204-8.9.6.50 amd64 1.0-1 [887 MB]
Selecting previously unselected package cudnn-local-repo-ubuntu2204-8.9.6.50.
(Reading database ... 190246 files and directories currently installed.)
Preparing to unpack .../cudnn-local-repo-ubuntu2204-8.9.6.50_1.0-1_amd64.deb ...
Unpacking cudnn-local-repo-ubuntu2204-8.9.6.50 (1.0-1) ...
Setting up cudnn-local-repo-ubuntu2204-8.9.6.50 (1.0-1) ...

The public cudnn-local-repo-ubuntu2204-8.9.6.50 GPG key does not appear to be installed.
To install the key, run this command:
sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.6.50/cudnn-local-1998375D-keyring.gpg /usr/share/keyrings/


kronos@kronos:~/Downloads$ cd /var/cudnn-local-repo-ubuntu2204-8.9.6.50/
kronos@kronos:/var/cudnn-local-repo-ubuntu2204-8.9.6.50$ ls
1998375D.pub                      libcudnn8_8.9.6.50-1+cuda12.2_amd64.deb          Local.md5      Packages.gz
cudnn-local-1998375D-keyring.gpg  libcudnn8-dev_8.9.6.50-1+cuda12.2_amd64.deb      Local.md5.gpg  Release
InRelease                         libcudnn8-samples_8.9.6.50-1+cuda12.2_amd64.deb  Packages       Release.gpg
kronos@kronos:/var/cudnn-local-repo-ubuntu2204-8.9.6.50$ sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.6.50/cudnn-local-1998375D-keyring.gpg /usr/share/keyrings/

kronos@kronos:/var/cudnn-local-repo-ubuntu2204-8.9.6.50$ sudo apt update
sudo apt install libcudnn8
sudo apt install libcudnn8-dev
sudo apt install libcudnn8-samples
Get:1 file:/var/cudnn-local-repo-ubuntu2204-8.9.6.50  InRelease [1,572 B]
Get:1 file:/var/cudnn-local-repo-ubuntu2204-8.9.6.50  InRelease [1,572 B]
Get:2 file:/var/cudnn-local-repo-ubuntu2204-8.9.6.50  Packages [946 B]                                                        
Hit:3 http://ca.archive.ubuntu.com/ubuntu jammy InRelease                                                                          
Get:4 http://ca.archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]                                                         
Get:5 https://dl.google.com/linux/chrome/deb stable InRelease [1,825 B]                                            
Hit:6 http://ca.archive.ubuntu.com/ubuntu jammy-backports InRelease                                                             
Get:7 http://ca.archive.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
Get:8 https://dl.google.com/linux/chrome/deb stable/main amd64 Packages [1,083 B]           
Get:9 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,196 kB]
Hit:10 https://ppa.launchpadcontent.net/danielrichter2007/grub-customizer/ubuntu jammy InRelease
Get:11 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [252 kB]
Get:12 http://ca.archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,159 kB]
Get:13 http://ca.archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [188 kB]
Get:14 http://ca.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,003 kB]
Hit:15 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease
Get:16 http://ca.archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [221 kB]
Get:17 http://ca.archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [985 kB]
Get:18 http://ca.archive.ubuntu.com/ubuntu jammy-security/main Translation-en [192 kB]
Get:19 http://ca.archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,134 kB]
Get:20 http://ca.archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [185 kB]
Get:21 http://ca.archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [799 kB]   
Get:22 http://ca.archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [148 kB]   
Hit:23 https://ppa.launchpadcontent.net/tatokis/ckb-next/ubuntu jammy InRelease        
Hit:24 https://ppa.launchpadcontent.net/thopiekar/openrgb/ubuntu jammy InRelease
Hit:25 https://ppa.launchpadcontent.net/ubuntuhandbook1/gdm-settings/ubuntu jammy InRelease
Fetched 7,693 kB in 1s (5,891 kB/s)             
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
41 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target Translations (main/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target Translations (main/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11 (main/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11 (main/dep11/Components-all.yml) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11-icons-small (main/dep11/icons-48x48.tar) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11-icons (main/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11-icons-hidpi (main/dep11/icons-64x64@2.tar) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target CNF (main/cnf/Commands-amd64) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target CNF (main/cnf/Commands-all) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target Translations (main/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target Translations (main/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11 (main/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11 (main/dep11/Components-all.yml) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11-icons-small (main/dep11/icons-48x48.tar) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11-icons (main/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target DEP-11-icons-hidpi (main/dep11/icons-64x64@2.tar) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target CNF (main/cnf/Commands-amd64) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
W: Target CNF (main/cnf/Commands-all) is configured multiple times in /etc/apt/sources.list.d/google-chrome-unstable.list:3 and /etc/apt/sources.list.d/google-chrome.list:3
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  libcudnn8
0 upgraded, 1 newly installed, 0 to remove and 41 not upgraded.
Need to get 0 B/444 MB of archives.
After this operation, 1,099 MB of additional disk space will be used.
Get:1 file:/var/cudnn-local-repo-ubuntu2204-8.9.6.50  libcudnn8 8.9.6.50-1+cuda12.2 [444 MB]
Selecting previously unselected package libcudnn8.
(Reading database ... 190262 files and directories currently installed.)
Preparing to unpack .../libcudnn8_8.9.6.50-1+cuda12.2_amd64.deb ...
Unpacking libcudnn8 (8.9.6.50-1+cuda12.2) ...
Setting up libcudnn8 (8.9.6.50-1+cuda12.2) ...

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  libcudnn8-dev
0 upgraded, 1 newly installed, 0 to remove and 41 not upgraded.
Need to get 0 B/440 MB of archives.
After this operation, 1,281 MB of additional disk space will be used.
Get:1 file:/var/cudnn-local-repo-ubuntu2204-8.9.6.50  libcudnn8-dev 8.9.6.50-1+cuda12.2 [440 MB]
Selecting previously unselected package libcudnn8-dev.
(Reading database ... 190279 files and directories currently installed.)
Preparing to unpack .../libcudnn8-dev_8.9.6.50-1+cuda12.2_amd64.deb ...
Unpacking libcudnn8-dev (8.9.6.50-1+cuda12.2) ...
Setting up libcudnn8-dev (8.9.6.50-1+cuda12.2) ...
update-alternatives: using /usr/include/x86_64-linux-gnu/cudnn_v8.h to provide /usr/include/cudnn.h (libcudnn) in auto mode

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  libcudnn8-samples
0 upgraded, 1 newly installed, 0 to remove and 41 not upgraded.
Need to get 0 B/1,663 kB of archives.
After this operation, 2,169 kB of additional disk space will be used.
Get:1 file:/var/cudnn-local-repo-ubuntu2204-8.9.6.50  libcudnn8-samples 8.9.6.50-1+cuda12.2 [1,663 kB]
Selecting previously unselected package libcudnn8-samples.
(Reading database ... 190312 files and directories currently installed.)
Preparing to unpack .../libcudnn8-samples_8.9.6.50-1+cuda12.2_amd64.deb ...
Unpacking libcudnn8-samples (8.9.6.50-1+cuda12.2) ...
Setting up libcudnn8-samples (8.9.6.50-1+cuda12.2) ...

kronos@kronos:/var/cudnn-local-repo-ubuntu2204-8.9.6.50$ cd
kronos@kronos:~$ sudo apt-get install python3-pip
sudo pip3 install virtualenv 
virtualenv -p py3.10 venv
source venv/bin/activate
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libexpat1-dev libpython3-dev libpython3.10-dev python3-dev python3-wheel python3.10-dev zlib1g-dev
The following NEW packages will be installed:
  libexpat1-dev libpython3-dev libpython3.10-dev python3-dev python3-pip python3-wheel python3.10-dev zlib1g-dev
0 upgraded, 8 newly installed, 0 to remove and 41 not upgraded.
Need to get 6,952 kB of archives.
After this operation, 30.4 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libexpat1-dev amd64 2.4.7-1ubuntu0.2 [147 kB]
Get:2 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-2ubuntu9.2 [164 kB]
Get:3 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpython3.10-dev amd64 3.10.12-1~22.04.2 [4,764 kB]
Get:4 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpython3-dev amd64 3.10.6-1~22.04 [7,166 B]
Get:5 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3.10-dev amd64 3.10.12-1~22.04.2 [507 kB]
Get:6 http://ca.archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-dev amd64 3.10.6-1~22.04 [26.0 kB]
Get:7 http://ca.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-wheel all 0.37.1-2ubuntu0.22.04.1 [32.0 kB]
Get:8 http://ca.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-pip all 22.0.2+dfsg-1ubuntu0.4 [1,305 kB]
Fetched 6,952 kB in 1s (11.9 MB/s)     
Selecting previously unselected package libexpat1-dev:amd64.
(Reading database ... 190378 files and directories currently installed.)
Preparing to unpack .../0-libexpat1-dev_2.4.7-1ubuntu0.2_amd64.deb ...
Unpacking libexpat1-dev:amd64 (2.4.7-1ubuntu0.2) ...
Selecting previously unselected package zlib1g-dev:amd64.
Preparing to unpack .../1-zlib1g-dev_1%3a1.2.11.dfsg-2ubuntu9.2_amd64.deb ...
Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-2ubuntu9.2) ...
Selecting previously unselected package libpython3.10-dev:amd64.
Preparing to unpack .../2-libpython3.10-dev_3.10.12-1~22.04.2_amd64.deb ...
Unpacking libpython3.10-dev:amd64 (3.10.12-1~22.04.2) ...
Selecting previously unselected package libpython3-dev:amd64.
Preparing to unpack .../3-libpython3-dev_3.10.6-1~22.04_amd64.deb ...
Unpacking libpython3-dev:amd64 (3.10.6-1~22.04) ...
Selecting previously unselected package python3.10-dev.
Preparing to unpack .../4-python3.10-dev_3.10.12-1~22.04.2_amd64.deb ...
Unpacking python3.10-dev (3.10.12-1~22.04.2) ...
Selecting previously unselected package python3-dev.
Preparing to unpack .../5-python3-dev_3.10.6-1~22.04_amd64.deb ...
Unpacking python3-dev (3.10.6-1~22.04) ...
Selecting previously unselected package python3-wheel.
Preparing to unpack .../6-python3-wheel_0.37.1-2ubuntu0.22.04.1_all.deb ...
Unpacking python3-wheel (0.37.1-2ubuntu0.22.04.1) ...
Selecting previously unselected package python3-pip.
Preparing to unpack .../7-python3-pip_22.0.2+dfsg-1ubuntu0.4_all.deb ...
Unpacking python3-pip (22.0.2+dfsg-1ubuntu0.4) ...
Setting up python3-wheel (0.37.1-2ubuntu0.22.04.1) ...
Setting up libexpat1-dev:amd64 (2.4.7-1ubuntu0.2) ...
Setting up python3-pip (22.0.2+dfsg-1ubuntu0.4) ...
Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-2ubuntu9.2) ...
Setting up libpython3.10-dev:amd64 (3.10.12-1~22.04.2) ...
Setting up python3.10-dev (3.10.12-1~22.04.2) ...
Setting up libpython3-dev:amd64 (3.10.6-1~22.04) ...
Setting up python3-dev (3.10.6-1~22.04) ...
Processing triggers for man-db (2.10.2-1) ...

Collecting virtualenv
  Downloading virtualenv-20.24.7-py3-none-any.whl (3.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.8/3.8 MB 29.0 MB/s eta 0:00:00
Collecting filelock<4,>=3.12.2
  Downloading filelock-3.13.1-py3-none-any.whl (11 kB)
Collecting distlib<1,>=0.3.7
  Downloading distlib-0.3.7-py2.py3-none-any.whl (468 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 KB 91.9 MB/s eta 0:00:00
Collecting platformdirs<5,>=3.9.1
  Downloading platformdirs-4.0.0-py3-none-any.whl (17 kB)
Installing collected packages: distlib, platformdirs, filelock, virtualenv
Successfully installed distlib-0.3.7 filelock-3.13.1 platformdirs-4.0.0 virtualenv-20.24.7
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
created virtual environment CPython3.10.12.final.0-64 in 293ms
  creator CPython3Posix(dest=/home/kronos/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/kronos/.local/share/virtualenv)
    added seed packages: pip==23.3.1, setuptools==68.2.2, wheel==0.41.3
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
(venv) kronos@kronos:~$ pip3 install torch torchvision torchaudio
Collecting torch
  Downloading torch-2.1.1-cp310-cp310-manylinux1_x86_64.whl.metadata (25 kB)
Collecting torchvision
  Downloading torchvision-0.16.1-cp310-cp310-manylinux1_x86_64.whl.metadata (6.6 kB)
Collecting torchaudio
  Downloading torchaudio-2.1.1-cp310-cp310-manylinux1_x86_64.whl.metadata (6.4 kB)
Collecting filelock (from torch)
  Downloading filelock-3.13.1-py3-none-any.whl.metadata (2.8 kB)
Collecting typing-extensions (from torch)
  Downloading typing_extensions-4.8.0-py3-none-any.whl.metadata (3.0 kB)
Collecting sympy (from torch)
  Downloading sympy-1.12-py3-none-any.whl (5.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 35.8 MB/s eta 0:00:00
Collecting networkx (from torch)
  Downloading networkx-3.2.1-py3-none-any.whl.metadata (5.2 kB)
Collecting jinja2 (from torch)
  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 26.2 MB/s eta 0:00:00
Collecting fsspec (from torch)
  Downloading fsspec-2023.10.0-py3-none-any.whl.metadata (6.8 kB)
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch)
  Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 38.0 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch)
  Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 76.7 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch)
  Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 42.0 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch)
  Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch)
  Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 14.4 MB/s eta 0:00:00
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch)
  Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 26.0 MB/s eta 0:00:00
Collecting nvidia-curand-cu12==10.3.2.106 (from torch)
  Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 34.8 MB/s eta 0:00:00
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch)
  Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 26.9 MB/s eta 0:00:00
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch)
  Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 21.6 MB/s eta 0:00:00
Collecting nvidia-nccl-cu12==2.18.1 (from torch)
  Downloading nvidia_nccl_cu12-2.18.1-py3-none-manylinux1_x86_64.whl (209.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 MB 20.5 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu12==12.1.105 (from torch)
  Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 kB 29.9 MB/s eta 0:00:00
Collecting triton==2.1.0 (from torch)
  Downloading triton-2.1.0-0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.3 kB)
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch)
  Downloading nvidia_nvjitlink_cu12-12.3.101-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting numpy (from torchvision)
  Downloading numpy-1.26.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 7.2 MB/s eta 0:00:00
Collecting requests (from torchvision)
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision)
  Downloading Pillow-10.1.0-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (9.5 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch)
  Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting charset-normalizer<4,>=2 (from requests->torchvision)
  Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests->torchvision)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 19.9 MB/s eta 0:00:00
Collecting urllib3<3,>=1.21.1 (from requests->torchvision)
  Downloading urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests->torchvision)
  Downloading certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)
Collecting mpmath>=0.19 (from sympy->torch)
  Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 78.1 MB/s eta 0:00:00
Downloading torch-2.1.1-cp310-cp310-manylinux1_x86_64.whl (670.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 670.2/670.2 MB 6.2 MB/s eta 0:00:00
Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.7/731.7 MB 5.4 MB/s eta 0:00:00
Downloading triton-2.1.0-0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 MB 27.3 MB/s eta 0:00:00
Downloading torchvision-0.16.1-cp310-cp310-manylinux1_x86_64.whl (6.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 41.8 MB/s eta 0:00:00
Downloading torchaudio-2.1.1-cp310-cp310-manylinux1_x86_64.whl (3.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 45.5 MB/s eta 0:00:00
Downloading Pillow-10.1.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 43.5 MB/s eta 0:00:00
Downloading filelock-3.13.1-py3-none-any.whl (11 kB)
Downloading fsspec-2023.10.0-py3-none-any.whl (166 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.4/166.4 kB 47.8 MB/s eta 0:00:00
Downloading networkx-3.2.1-py3-none-any.whl (1.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 38.4 MB/s eta 0:00:00
Downloading numpy-1.26.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 38.3 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 20.5 MB/s eta 0:00:00
Downloading typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Downloading certifi-2023.11.17-py3-none-any.whl (162 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 45.6 MB/s eta 0:00:00
Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.1/142.1 kB 43.9 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Downloading urllib3-2.1.0-py3-none-any.whl (104 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 33.8 MB/s eta 0:00:00
Downloading nvidia_nvjitlink_cu12-12.3.101-py3-none-manylinux1_x86_64.whl (20.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.5/20.5 MB 31.1 MB/s eta 0:00:00
Installing collected packages: mpmath, urllib3, typing-extensions, sympy, pillow, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, MarkupSafe, idna, fsspec, filelock, charset-normalizer, certifi, triton, requests, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, nvidia-cusolver-cu12, torch, torchvision, torchaudio
Successfully installed MarkupSafe-2.1.3 certifi-2023.11.17 charset-normalizer-3.3.2 filelock-3.13.1 fsspec-2023.10.0 idna-3.4 jinja2-3.1.2 mpmath-1.3.0 networkx-3.2.1 numpy-1.26.2 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.18.1 nvidia-nvjitlink-cu12-12.3.101 nvidia-nvtx-cu12-12.1.105 pillow-10.1.0 requests-2.31.0 sympy-1.12 torch-2.1.1 torchaudio-2.1.1 torchvision-0.16.1 triton-2.1.0 typing-extensions-4.8.0 urllib3-2.1.0

(venv) kronos@kronos:~$ python
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.cuda.is_available()) # should be True
True
>>> 
>>> t = torch.rand(10, 10).cuda()
>>> print(t.device) # should be CUDA
cuda:0
>>> 

@LouisJULIEN
Copy link

Worked like a charm, thanks!

The only thing I did a bit differently was the gpg key copy. It's now is a subdirectory. Here the command I used:

sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.6.50/cudnn-local-692B6C75-keyring.gpg  /usr/share/keyrings/

@LouisJULIEN
Copy link

Some tweaks are required if you're willing to use tensorflow with CUDA:

  • As of November 2023, you need cudNN v8.6 as stated by the (english) doc. Beware, the latest cudNN version is 8.9. You need to uninstall your previous cudNN to install the right version. Don't forget to uninstall then reinstall libcudnn8 libcudnn8-dev libcudnn8-samplesas well.
  • To install tensorflow with cuda, use pip install tensorflow[and-cuda]
  • Check the installation: python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))" . You should see 1 or higher if your setup is right.

@ilo-bst
Copy link

ilo-bst commented Dec 22, 2023

Hi, what does <something> stand for in sudo cp /var/cudnn-<something>.gpg /usr/share/keyrings/? My cuDNN file is cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb.

@ilo-bst
Copy link

ilo-bst commented Dec 22, 2023

Hi, what does <something> stand for in sudo cp /var/cudnn-<something>.gpg /usr/share/keyrings/? My cuDNN file is cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb.

I just went to the location /var/cudnn... in my file system and found the appropriate gpg file.

@misaelpedro
Copy link

When I was installing cuDNN and ran:

$ sudo apt install ./cudnn-local-repo-ubuntu2204-8.6.0.163_1.0-1_amd64.deb

It returned this at end of output:

The public cudnn-local-repo-ubuntu2204-8.6.0.163 GPG key does not appear to be installed. To install the key, run this command: sudo cp /var/cudnn-local-repo-ubuntu2204-8.6.0.163/<YOURS_KEY>-keyring.gpg /usr/share/keyrings/

Then I ran that command in the end of message and the others ones as it is listed above (libcudnn8, libcudnn8-dev, libcudnn8-samples)

A final tip to check if cuDNN is installed is run :
whereis cudnn.h
It should return : cudnn.h: /usr/include/cudnn.h

@aquinovo
Copy link

Thank you. This is very helpful

@MaisJamal
Copy link

Thanks, it's very helpful, worked perfectly.

@corticalstack
Copy link

@Atlas3DSS Having same issue as you reported. I've purged all nvidia (as I'm aware) and reinstalled nvidia driver, followed by cuda kit, but all I can seem to install is the 11.5 cuda kit version (on Ubuntu 22.04). Any tips to resolve you can share?

@tarreislam
Copy link

For anyone having issues on ubuntu 22.04: check this out

NVIDIA/nvidia-docker#1243 (comment)

I switched to bare metal because I thought Esxi was the issue, but after doing this, it worked

@flavves
Copy link

flavves commented Apr 8, 2024

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment