Skip to content

Instantly share code, notes, and snippets.

@nekiee13
Created January 30, 2024 03:56
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save nekiee13/c8ec43bce5fd75d20e38b31a613fd83d to your computer and use it in GitHub Desktop.
Save nekiee13/c8ec43bce5fd75d20e38b31a613fd83d to your computer and use it in GitHub Desktop.
Install Ollama under Win11 & WSL - CUDA Installation guide
CMD prompt - verify WSL2 is installed
`wsl --list --verbose`
or
`wsl -l -v`
git clone CUDA samples - I used location at disk d:\\LLM\\Ollama , so I can find samples with ease
`d: && cd d:\LLM\Ollama`
`git clone --recursive -j6 https://github.com/NVIDIA/cuda-samples.git`
CUDA 12.3 was previously installed on Win11, but not under wsl.
So, check if Linux instance recognizes the GPU.
`wsl --user root -d ubuntu`
`nvidia-smi`
Then, install CUDA for wsl
```
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/7fa2af80.pub
sudo apt-key add 7fa2af80.pub
sudo sh -c 'echo "deb https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/ /" > /etc/apt/sources.list.d/cuda.list'
sudo apt-get update
nvidia-smi
```
Check CUDA Toolkit Installation
`nvcc --version`
`cd /mnt/d/llm/ollama/cuda-samples`
`make 1_Utilities/deviceQuery`
From root@rpc:/mnt/d/llm/ollama/cuda-samples/samples/1_Utilities/deviceQuery# execute
`./deviceQuery`
....
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 12.3, CUDA Runtime Version = 12.3, NumDevs = 1
Result = PASS
Result = PASS, indicates that CUDA is installed correctly
`cd /mnt/c/Users/user`
`curl https://ollama.ai/install.sh | sh`
ollama run mistral
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment