Created
January 30, 2024 03:56
-
-
Save nekiee13/c8ec43bce5fd75d20e38b31a613fd83d to your computer and use it in GitHub Desktop.
Install Ollama under Win11 & WSL - CUDA Installation guide
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CMD prompt - verify WSL2 is installed | |
`wsl --list --verbose` | |
or | |
`wsl -l -v` | |
git clone CUDA samples - I used location at disk d:\\LLM\\Ollama , so I can find samples with ease | |
`d: && cd d:\LLM\Ollama` | |
`git clone --recursive -j6 https://github.com/NVIDIA/cuda-samples.git` | |
CUDA 12.3 was previously installed on Win11, but not under wsl. | |
So, check if Linux instance recognizes the GPU. | |
`wsl --user root -d ubuntu` | |
`nvidia-smi` | |
Then, install CUDA for wsl | |
``` | |
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/7fa2af80.pub | |
sudo apt-key add 7fa2af80.pub | |
sudo sh -c 'echo "deb https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/ /" > /etc/apt/sources.list.d/cuda.list' | |
sudo apt-get update | |
nvidia-smi | |
``` | |
Check CUDA Toolkit Installation | |
`nvcc --version` | |
`cd /mnt/d/llm/ollama/cuda-samples` | |
`make 1_Utilities/deviceQuery` | |
From root@rpc:/mnt/d/llm/ollama/cuda-samples/samples/1_Utilities/deviceQuery# execute | |
`./deviceQuery` | |
.... | |
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 12.3, CUDA Runtime Version = 12.3, NumDevs = 1 | |
Result = PASS | |
Result = PASS, indicates that CUDA is installed correctly | |
`cd /mnt/c/Users/user` | |
`curl https://ollama.ai/install.sh | sh` | |
ollama run mistral |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment