As of v 1.48, just use the new Preset CLBlast NoAVX2 (Old CPU)
Keep in mind you can only use one GPU.
CLBlast is a fair enough alternative when Cublas isn't working right (only other choice really, if you want to use GPU).
(aka, Old CPU with newer GPU)
Clone latest tagged Release and open Makefile
In git bash
git clone "https://github.com/LostRuins/koboldcpp.git" && cd "$(basename "https://github.com/LostRuins/koboldcpp.git" .git)"
git checkout $(git describe --tags `git rev-list --tags --max-count=1`)
cd /d koboldcpp
notepad Makefile
In opened Makefile replace line
FULLCFLAGS += -mavx2 -msse3 -mfma -mf16c -mavx
with
FULLCFLAGS +=-msse3 -mf16c -mavx
(or whatever your cpu supports. I used coreinfo to get mine)
In w64devkit
..\w64devkit\w64devkit.exe
cd koboldcpp
make
Attempting to run
python kolboldcpp.py
without modifying the Makefile
Gives error:
OSError: [WinError -1073741795] Windows Error 0xc000001d
Due to system info not being detected properly.
In my case:
Detected | Actual | Wrong | |
---|---|---|---|
AVX | 1 | 1 | |
AVX2 | 1 | 0 | X |
AVX512 | 0 | 0 | |
AVX512_VBMI | 0 | 0 | |
AVX512_VNNI | 0 | 0 | |
FMA | 1 | 0 | X |
NEON | 0 | 0 | |
ARM_FMA | 0 | 0 | |
F16C | 1 | 1 | |
FP16_VA | 0 | 0 | |
WASM_SIMD | 0 | 0 | |
BLAS | 1 | 1 | |
SSE3 | 1 | 1 | |
VSX | 0 | 0 |
If you already ran make
in win64devkit and want to change the Makefile, run make clean
then make
I could then run with clblast with something like (disable MMAP was needed for me):