Skip to content

Instantly share code, notes, and snippets.

@cgmb
Last active February 3, 2025 05:10
Show Gist options
  • Save cgmb/bb661fccaf041d3649f9a90560826ebc to your computer and use it in GitHub Desktop.
Save cgmb/bb661fccaf041d3649f9a90560826ebc to your computer and use it in GitHub Desktop.
How to build llama.cpp on fedora
#!/bin/sh
# Build llama.cpp on Fedora 40 and later
# Tested with `docker run -it --device=/dev/dri --device=/dev/kfd --security-opt seccomp=unconfined --volume $HOME:/mnt/home fedora:40`
dnf -y update
dnf -y install hipcc rocm-hip-devel rocblas-devel hipblas-devel git cmake g++ lld clang-tools-extra llvm
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp/
git checkout b2110
CC=clang CXX=clang++ cmake -H. -Bbuild -DLLAMA_HIPBLAS=ON -DAMDGPU_TARGETS="gfx1030;gfx1100;gfx1101;gfx1102" -DCMAKE_BUILD_TYPE=Release
make -j16 -C build
build/bin/main -ngl 32 --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -m ~/Downloads/dolphin-2.2.1-mistral-7b.Q5_K_M.gguf --prompt "Once upon a time"
@luboszima
Copy link

On fedora 40 (Asahi) I had issues with missing curl library.

This helped me to install libcurl

sudo dnf install libcurl-devel
  • add -DLLAMA_CURL=1 into params:
$ sudo CC=/usr/bin/clang CXX=/usr/bin/clang++ cmake -H. -Bbuild  -DLLAMA_CURL=1 -DLLAMA_HIPBLAS=ON -DAMDGPU_TARGETS="gfx1030;gfx1100;gfx1101;gfx1102" -DCMAKE_BUILD_TYPE=Release
$ sudo make -j16 -C build

Then you can try download some model:

llama-cli -hf bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment