Using diffusers + pytorch + torchvision with Python 3.11 on Linux with Conda and CUDA 11.8 on Ubuntu 22.10 with Nvidia driver 525

README.md

Install latest Nvidia driver -- 525 is CUDA 12-era, but is backwards-compatible with 11.8:

apt install nvidia-driver-525 nvidia-dkms-525

Install CUDA 11.8 (use the runfile install, tell it not to replace your driver if you have a newer driver, switch to gcc 11):
https://developer.nvidia.com/cuda-11-8-0-download-archive

Install cuDNN 8 (requires registration):
https://developer.nvidia.com/cudnn

Install conda (conda-forge is how we'll get Python 3.11, because there's no deadsnakes distribution for 22.10):
https://www.anaconda.com/products/distribution

bash Anaconda-latest-Linux-x86_64.sh
eval "$(/home/birch/anaconda3/bin/conda shell.bash hook)"
conda config --set auto_activate_base false
conda init

We'll create a conda env with:

• conda-forge channel (for Python 3.11)
• pytorch channel (for magma-cuda118)

# note: Python 3.11.0 was the newest available on conda-forge at time of writing; 3.11.1 not yet available
conda create -n p311 -c nvidia/label/cuda-11.8.0 -c pytorch-nightly -c pytorch -c conda-forge -c defaults astunparse numpy ninja pyyaml setuptools cmake typing_extensions six requests dataclasses mkl mkl-include magma-cuda118 python=3.11
conda activate p311
# note: I didn't try TensorRT myself, but here's how:
pip install tensorrt
git clone --depth 1 --single-branch -b master --recursive https://github.com/pytorch/pytorch.git
# make sure you're still using gcc 11!
# I didn't try TensorRT myself, so if you have any problems with it: remove that option
# look up your CUDA arch on https://en.wikipedia.org/wiki/CUDA#GPUs_supported or delete TORCH_CUDA_ARCH_LIST to leave it to automatic detection
USE_TENSORRT=ON TORCH_CUDA_ARCH_LIST=8.9 PATH="/usr/local/cuda-11.8/bin:$PATH" LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64 python setup.py develop
# if you need to try again: git clean -dfx

Torchvision isn't quite released yet, so build-from-source:

pytorch/vision#7108
https://github.com/pmeier/vision/tree/py311

git clone --depth 1 --single-branch -b py311 https://github.com/pmeier/vision.git
cd vision
TORCH_CUDA_ARCH_LIST=8.9 PATH="/usr/local/cuda-11.8/bin:$PATH" LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64 python setup.py install

Build xformers from source:

git clone --depth 1 --single-branch -b main --recursive https://github.com/facebookresearch/xformers.git
cd xformers
pip install ninja
# make sure you're still using gcc 11!
TORCH_CUDA_ARCH_LIST=8.9 PATH="/usr/local/cuda-11.8/bin:$PATH" LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64 python setup.py install