Skip to content

Instantly share code, notes, and snippets.

@TFWol
Last active November 6, 2023 03:52
Show Gist options
  • Save TFWol/f83fb5e95fea8555ef5c407bebeaf4d6 to your computer and use it in GitHub Desktop.
Save TFWol/f83fb5e95fea8555ef5c407bebeaf4d6 to your computer and use it in GitHub Desktop.
Getting koboldcpp to compile and run with clblast on PC that has CPU without AVX2 and a newer GPU (NVidia 4090 tested). No more `OSError: [WinError -1073741795] Windows Error 0xc000001d`

UPDATE

As of v 1.48, just use the new Preset CLBlast NoAVX2 (Old CPU)

Keep in mind you can only use one GPU.

CLBlast is a fair enough alternative when Cublas isn't working right (only other choice really, if you want to use GPU).

(Obsolete) Getting clblast to work on PC that has CPU without AVX2 and a newer GPU

(aka, Old CPU with newer GPU)

Clone latest tagged Release and open Makefile

In git bash

git clone "https://github.com/LostRuins/koboldcpp.git" && cd "$(basename "https://github.com/LostRuins/koboldcpp.git" .git)"
git checkout $(git describe --tags `git rev-list --tags --max-count=1`)
cd /d koboldcpp
notepad Makefile

In opened Makefile replace line

FULLCFLAGS += -mavx2 -msse3 -mfma -mf16c -mavx

with

FULLCFLAGS +=-msse3 -mf16c -mavx

(or whatever your cpu supports. I used coreinfo to get mine)

In w64devkit

..\w64devkit\w64devkit.exe
cd koboldcpp
make

Notes

Attempting to run

python kolboldcpp.py

without modifying the Makefile

Gives error:

OSError: [WinError -1073741795] Windows Error 0xc000001d

Due to system info not being detected properly.

In my case:

Detected Actual Wrong
AVX 1 1
AVX2 1 0 X
AVX512 0 0
AVX512_VBMI 0 0
AVX512_VNNI 0 0
FMA 1 0 X
NEON 0 0
ARM_FMA 0 0
F16C 1 1
FP16_VA 0 0
WASM_SIMD 0 0
BLAS 1 1
SSE3 1 1
VSX 0 0

If you already ran make in win64devkit and want to change the Makefile, run make clean then make

I could then run with clblast with something like (disable MMAP was needed for me):

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment