Skip to content

Instantly share code, notes, and snippets.

@joshuacox

joshuacox/docker-run.log Secret

Created November 9, 2023 17:32
Show Gist options
  • Save joshuacox/dc9ed633e28260f16af0ec02c69c0c74 to your computer and use it in GitHub Desktop.
Save joshuacox/dc9ed633e28260f16af0ec02c69c0c74 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
docker-run.log
go mod edit -replace github.com/go-skynet/go-llama.cpp=/build/go-llama
go mod edit -replace github.com/nomic-ai/gpt4all/gpt4all-bindings/golang=/build/gpt4all/gpt4all-bindings/golang
go mod edit -replace github.com/go-skynet/go-ggml-transformers.cpp=/build/go-ggml-transformers
go mod edit -replace github.com/donomii/go-rwkv.cpp=/build/go-rwkv
go mod edit -replace github.com/ggerganov/whisper.cpp=/build/whisper.cpp
go mod edit -replace github.com/go-skynet/go-bert.cpp=/build/go-bert
go mod edit -replace github.com/go-skynet/bloomz.cpp=/build/bloomz
go mod edit -replace github.com/mudler/go-stable-diffusion=/build/go-stable-diffusion
go mod edit -replace github.com/mudler/go-piper=/build/go-piper
go mod edit -replace github.com/mudler/go-ggllm.cpp=/build/go-ggllm
go mod download
touch prepare
mkdir -p backend-assets/grpc
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/langchain-huggingface ./cmd/grpc/langchain-huggingface/
make -C go-ggml-transformers BUILD_TYPE=cublas libtransformers.a
make[1]: Entering directory '/build/go-ggml-transformers'
I libtransformers build info:
I UNAME_S: Linux
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -I -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native
I LDFLAGS:
I CMAKE_ARGS: -DGGML_CUBLAS=ON
I CC: cc (Debian 10.2.1-6) 10.2.1 20210110
I CXX: g++ (Debian 10.2.1-6) 10.2.1 20210110
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native starcoder.cpp -o starcoder.o -c
In file included from starcoder.cpp:19:
ggml.cpp/examples/starcoder/main.cpp: In function 'int main_starcoder(int, char**)':
ggml.cpp/examples/starcoder/main.cpp:799:23: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
799 | for (int i = 0; i < embd_inp.size(); i++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/starcoder/main.cpp:821:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
821 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.cpp/examples/starcoder/main.cpp:837:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
837 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
ggml.cpp/examples/starcoder/main.cpp:859:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
859 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/starcoder/main.cpp:861:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
861 | if (embd.size() >= params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
starcoder.cpp: In function 'int starcoder_predict(void*, void*, char*)':
starcoder.cpp:80:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
80 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
starcoder.cpp:96:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
96 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
starcoder.cpp:118:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
118 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
starcoder.cpp:120:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
120 | if (embd.size() >= params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
starcoder.cpp:36:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
36 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
starcoder.cpp:47:13: warning: unused variable 't_load_us' [-Wunused-variable]
47 | int64_t t_load_us = 0;
| ^~~~~~~~~
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native falcon.cpp -o falcon.o -c
In file included from falcon.cpp:19:
implementations/falcon.cpp: In function 'bool falcon_model_load(const string&, falcon_model&, gpt_vocab&)':
implementations/falcon.cpp:187:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
187 | .mem_size = ctx_size,
| ^
implementations/falcon.cpp:188:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
188 | .mem_buffer = NULL,
| ^
implementations/falcon.cpp:189:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
189 | .no_alloc = false,
| ^
In file included from falcon.cpp:19:
implementations/falcon.cpp: In function 'bool falcon_eval(const falcon_model&, int, int, const std::vector<int>&, std::vector<float>&, size_t&)':
implementations/falcon.cpp:410:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
410 | .mem_size = buf_size,
| ^
implementations/falcon.cpp:411:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
411 | .mem_buffer = buf,
| ^
implementations/falcon.cpp:412:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
412 | .no_alloc = false,
| ^
falcon.cpp: In function 'int falcon_predict(void*, void*, char*)':
falcon.cpp:69:34: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
69 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
falcon.cpp:85:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
85 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
falcon.cpp:107:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
107 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
falcon.cpp:109:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
109 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
falcon.cpp:36:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
36 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
falcon.cpp:48:13: warning: unused variable 't_load_us' [-Wunused-variable]
48 | int64_t t_load_us = 0;
| ^~~~~~~~~
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native gptj.cpp -o gptj.o -c
In file included from gptj.cpp:19:
ggml.cpp/examples/gpt-j/main.cpp: In function 'int main_gptj(int, char**)':
ggml.cpp/examples/gpt-j/main.cpp:674:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
674 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-j/main.cpp:690:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
690 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-j/main.cpp:712:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
712 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-j/main.cpp:714:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
714 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
gptj.cpp: In function 'int gptj_predict(void*, void*, char*)':
gptj.cpp:72:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
72 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
gptj.cpp:88:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
88 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
gptj.cpp:110:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
110 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
gptj.cpp:112:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
112 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
gptj.cpp:36:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
36 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
gptj.cpp:48:13: warning: unused variable 't_load_us' [-Wunused-variable]
48 | int64_t t_load_us = 0;
| ^~~~~~~~~
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native mpt.cpp -o mpt.o -c
In file included from mpt.cpp:19:
ggml.cpp/examples/mpt/main.cpp: In function 'bool mpt_model_load(const string&, mpt_model&, gpt_vocab&)':
ggml.cpp/examples/mpt/main.cpp:246:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::__cxx11::basic_string<wchar_t>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
246 | for (int w = 0; w < word_multibytes.size(); w++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~
mpt.cpp: In function 'int mpt_predict(void*, void*, char*)':
mpt.cpp:37:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
37 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
mpt.cpp:49:13: warning: unused variable 't_load_us' [-Wunused-variable]
49 | int64_t t_load_us = 0;
| ^~~~~~~~~
mkdir build
cd build && cmake ../ggml.cpp -DGGML_CUBLAS=ON && make VERBOSE=1 ggml && cp -rf src/CMakeFiles/ggml.dir/ggml.c.o ../ggml.o
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.

-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.30.2")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Linux detected
-- Found CUDAToolkit: /usr/local/cuda/include (found version "12.1.105")
-- cuBLAS found
-- The CUDA compiler identification is NVIDIA 12.1.105
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- GGML CUDA sources found, configuring CUDA architecture
-- x86 detected
-- Linux detected
-- Configuring done (1.8s)
-- Generating done (0.1s)
-- Build files have been written to: /build/go-ggml-transformers/build
make[2]: Entering directory '/build/go-ggml-transformers/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -S/build/go-ggml-transformers/ggml.cpp -B/build/go-ggml-transformers/build --check-build-system CMakeFiles/Makefile.cmake 0
make -f CMakeFiles/Makefile2 ggml
make[3]: Entering directory '/build/go-ggml-transformers/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -S/build/go-ggml-transformers/ggml.cpp -B/build/go-ggml-transformers/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-ggml-transformers/build/CMakeFiles 3
make -f CMakeFiles/Makefile2 src/CMakeFiles/ggml.dir/all
make[4]: Entering directory '/build/go-ggml-transformers/build'
make -f src/CMakeFiles/ggml.dir/build.make src/CMakeFiles/ggml.dir/depend
make[5]: Entering directory '/build/go-ggml-transformers/build'
cd /build/go-ggml-transformers/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggml-transformers/ggml.cpp /build/go-ggml-transformers/ggml.cpp/src /build/go-ggml-transformers/build /build/go-ggml-transformers/build/src /build/go-ggml-transformers/build/src/CMakeFiles/ggml.dir/DependInfo.cmake "--color="
make[5]: Leaving directory '/build/go-ggml-transformers/build'
make -f src/CMakeFiles/ggml.dir/build.make src/CMakeFiles/ggml.dir/build
make[5]: Entering directory '/build/go-ggml-transformers/build'
[ 33%] Building C object src/CMakeFiles/ggml.dir/ggml.c.o
cd /build/go-ggml-transformers/build/src && /usr/bin/cc -DGGML_USE_CUBLAS -I/build/go-ggml-transformers/ggml.cpp/src/. -I/build/go-ggml-transformers/ggml.cpp/src/../include -I/build/go-ggml-transformers/ggml.cpp/src/../include/ggml -isystem /usr/local/cuda/include -Wall -Wextra -Wpedantic -Wshadow -Wcast-qual -Wstrict-prototypes -Wpointer-arith -Wdouble-promotion -Wno-unused-function -Werror=vla -mavx -mavx2 -mfma -mf16c -msse3 -O3 -DNDEBUG -std=gnu11 -MD -MT src/CMakeFiles/ggml.dir/ggml.c.o -MF CMakeFiles/ggml.dir/ggml.c.o.d -o CMakeFiles/ggml.dir/ggml.c.o -c /build/go-ggml-transformers/ggml.cpp/src/ggml.c
[ 66%] Building CUDA object src/CMakeFiles/ggml.dir/ggml-cuda.cu.o
cd /build/go-ggml-transformers/build/src && /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -DGGML_USE_CUBLAS --options-file CMakeFiles/ggml.dir/includes_CUDA.rsp -O3 -DNDEBUG -std=c++11 -MD -MT src/CMakeFiles/ggml.dir/ggml-cuda.cu.o -MF CMakeFiles/ggml.dir/ggml-cuda.cu.o.d -x cu -c /build/go-ggml-transformers/ggml.cpp/src/ggml-cuda.cu -o CMakeFiles/ggml.dir/ggml-cuda.cu.o
[100%] Linking CUDA static library libggml.a
cd /build/go-ggml-transformers/build/src && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/ggml.dir/cmake_clean_target.cmake
cd /build/go-ggml-transformers/build/src && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/ggml.dir/link.txt --verbose=1
/usr/bin/ar qc libggml.a CMakeFiles/ggml.dir/ggml.c.o "CMakeFiles/ggml.dir/ggml-cuda.cu.o"
/usr/bin/ranlib libggml.a
make[5]: Leaving directory '/build/go-ggml-transformers/build'
[100%] Built target ggml
make[4]: Leaving directory '/build/go-ggml-transformers/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-ggml-transformers/build/CMakeFiles 0
make[3]: Leaving directory '/build/go-ggml-transformers/build'
make[2]: Leaving directory '/build/go-ggml-transformers/build'
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native gpt2.cpp ggml.o -o gpt2.o -c
In file included from gpt2.cpp:16:
ggml.cpp/examples/gpt-2/main.cpp: In function 'int main_gpt2(int, char**)':
ggml.cpp/examples/gpt-2/main.cpp:770:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
770 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-2/main.cpp:786:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
786 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-2/main.cpp:808:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
808 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-2/main.cpp:810:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
810 | if (embd.size() >= params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
gpt2.cpp: In function 'int gpt2_predict(void*, void*, char*)':
gpt2.cpp:68:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
68 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
gpt2.cpp:84:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
84 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
gpt2.cpp:106:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
106 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
gpt2.cpp:108:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
108 | if (embd.size() >= params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
gpt2.cpp:34:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
34 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
gpt2.cpp:43:13: warning: unused variable 't_load_us' [-Wunused-variable]
43 | int64_t t_load_us = 0;
| ^~~~~~~~~
g++: warning: ggml.o: linker input file unused because linking not done
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native replit.cpp -o replit.o -c
In file included from replit.cpp:21:
ggml.cpp/examples/replit/main.cpp: In function 'std::pair<std::vector<long unsigned int>, float> encode_word(const string&, const piece_map_t&)':
ggml.cpp/examples/replit/main.cpp:54:39: warning: comparison of integer expressions of different signedness: 'int' and 'std::__cxx11::basic_string<char>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
54 | for (int start_idx = 0; start_idx < word.length(); ++start_idx) {
| ~~~~~~~~~~^~~~~~~~~~~~~~~
ggml.cpp/examples/replit/main.cpp:56:51: warning: comparison of integer expressions of different signedness: 'int' and 'std::__cxx11::basic_string<char>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
56 | for (int end_idx = start_idx + 1; end_idx <= word.length(); ++end_idx) {
| ~~~~~~~~^~~~~~~~~~~~~~~~
ggml.cpp/examples/replit/main.cpp: In function 'bool replit_tokenizer_load(replit_tokenizer&, std::istream&, int)':
ggml.cpp/examples/replit/main.cpp:94:31: warning: comparison of integer expressions of different signedness: 'std::size_t' {aka 'long unsigned int'} and 'int' [-Wsign-compare]
94 | for (std::size_t i = 0; i < max_vocab_size; i++) {
| ~~^~~~~~~~~~~~~~~~
ggml.cpp/examples/replit/main.cpp: In function 'bool replit_model_load(const string&, replit_model&, replit_tokenizer&)':
ggml.cpp/examples/replit/main.cpp:345:56: warning: format '%lld' expects argument of type 'long long int', but argument 4 has type 'long int' [-Wformat=]
345 | printf("%s: memory_size = %8.2f MB, n_mem = %lld\n", __func__, memory_size / 1024.0 / 1024.0, n_mem);
| ~~~^ ~~~~~
| | |
| long long int long int
| %ld
ggml.cpp/examples/replit/main.cpp: In function 'int main_replit(int, char**)':
ggml.cpp/examples/replit/main.cpp:704:23: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
704 | for (int i = 0; i < embd_inp.size(); i++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/replit/main.cpp:718:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
718 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.cpp/examples/replit/main.cpp:734:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
734 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
ggml.cpp/examples/replit/main.cpp:757:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
757 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/replit/main.cpp:759:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
759 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
replit.cpp: In function 'int replit_predict(void*, void*, char*)':
replit.cpp:64:21: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
64 | for (int i = 0; i < embd_inp.size(); i++) {
| ~~^~~~~~~~~~~~~~~~~
replit.cpp:65:31: warning: format '%d' expects argument of type 'int', but argument 4 has type '__gnu_cxx::__alloc_traits<std::allocator<long unsigned int>, long unsigned int>::value_type' {aka 'long unsigned int'} [-Wformat=]
65 | printf("%s: token[%d] = %6d\n", __func__, i, embd_inp[i]);
| ~~^
| |
| int
| %6ld
replit.cpp:80:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
80 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
replit.cpp:96:11: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
96 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
replit.cpp:120:25: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<long unsigned int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
120 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
replit.cpp:122:25: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
122 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
replit.cpp:39:17: warning: unused variable 't_main_start_us' [-Wunused-variable]
39 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
replit.cpp:49:11: warning: unused variable 't_load_us' [-Wunused-variable]
49 | int64_t t_load_us = 0;
| ^~~~~~~~~
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native gptneox.cpp -o gptneox.o -c
In file included from gptneox.cpp:19:
ggml.cpp/examples/gpt-neox/main.cpp: In function 'int main_gptneox(int, char**)':
ggml.cpp/examples/gpt-neox/main.cpp:728:23: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
728 | for (int i = 0; i < embd_inp.size(); i++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-neox/main.cpp:739:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
739 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-neox/main.cpp:755:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
755 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-neox/main.cpp:777:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
777 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/gpt-neox/main.cpp:779:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
779 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
gptneox.cpp: In function 'int gpt_neox_predict(void*, void*, char*)':
gptneox.cpp:71:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
71 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
gptneox.cpp:87:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
87 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
gptneox.cpp:109:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
109 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
gptneox.cpp:111:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
111 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
gptneox.cpp:36:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
36 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
gptneox.cpp:48:13: warning: unused variable 't_load_us' [-Wunused-variable]
48 | int64_t t_load_us = 0;
| ^~~~~~~~~
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native dolly.cpp -o dolly.o -c
In file included from dolly.cpp:18:
ggml.cpp/examples/dolly-v2/main.cpp: In function 'int main_dolly(int, char**)':
ggml.cpp/examples/dolly-v2/main.cpp:731:23: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
731 | for (int i = 0; i < embd_inp.size(); i++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/dolly-v2/main.cpp:744:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
744 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.cpp/examples/dolly-v2/main.cpp:760:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
760 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
ggml.cpp/examples/dolly-v2/main.cpp:783:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
783 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
ggml.cpp/examples/dolly-v2/main.cpp:785:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
785 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
dolly.cpp: In function 'int dolly_predict(void*, void*, char*)':
dolly.cpp:70:33: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
70 | for (int i = embd.size(); i < embd_inp.size() + params.n_predict; i++) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
dolly.cpp:86:15: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
86 | if (i >= embd_inp.size()) {
| ~~^~~~~~~~~~~~~~~~~~
dolly.cpp:109:31: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
109 | for (int k = i; k < embd_inp.size(); k++) {
| ~~^~~~~~~~~~~~~~~~~
dolly.cpp:111:33: warning: comparison of integer expressions of different signedness: 'std::vector<int>::size_type' {aka 'long unsigned int'} and 'int32_t' {aka 'int'} [-Wsign-compare]
111 | if (embd.size() > params.n_batch) {
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~
dolly.cpp:46:13: warning: unused variable 't_load_us' [-Wunused-variable]
46 | int64_t t_load_us = 0;
| ^~~~~~~~~
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -c ggml.cpp/examples/common-ggml.cpp -o common-ggml.o
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -c ggml.cpp/examples/common.cpp -o common.o
ar src libtransformers.a replit.o gptj.o mpt.o gptneox.o starcoder.o gpt2.o dolly.o falcon.o ggml.o common-ggml.o common.o ggml-cuda.o
make[1]: Leaving directory '/build/go-ggml-transformers'
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/falcon-ggml ./cmd/grpc/falcon-ggml/
make -C go-bert libgobert.a
make[1]: Entering directory '/build/go-bert'
I go-gpt4all-j build info:
I UNAME_S: Linux
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I. -I./bert.cpp/ggml/include/ggml/ -I./bert.cpp/ -I -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I. -I./bert.cpp/ggml/include/ggml/ -I./bert.cpp/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native
I LDFLAGS:
I CMAKEFLAGS:
I CC: cc (Debian 10.2.1-6) 10.2.1 20210110
I CXX: g++ (Debian 10.2.1-6) 10.2.1 20210110
cd bert.cpp && mkdir build
sed "s/#include <regex>/#include <regex>\n#include <unordered_map>/" bert.cpp/bert.cpp > bert.cpp/bert.tmp && mv bert.cpp/bert.tmp bert.cpp/bert.cpp
cd bert.cpp/build && cmake .. -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release && make
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
CMake Deprecation Warning at ggml/CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.

-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Linux detected
-- Configuring done (0.4s)
-- Generating done (0.0s)
-- Build files have been written to: /build/go-bert/bert.cpp/build
make[2]: Entering directory '/build/go-bert/bert.cpp/build'
make[3]: Entering directory '/build/go-bert/bert.cpp/build'
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
[ 8%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml.c.o
[ 16%] Linking C static library libggml.a
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
[ 16%] Built target ggml
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
[ 25%] Building CXX object CMakeFiles/bert.dir/bert.cpp.o
[ 33%] Linking CXX static library libbert.a
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
[ 33%] Built target bert
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
[ 41%] Building CXX object examples/CMakeFiles/server.dir/server.cpp.o
[ 50%] Linking CXX executable ../bin/server
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
[ 50%] Built target server
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
[ 58%] Building CXX object examples/CMakeFiles/main.dir/main.cpp.o
[ 66%] Linking CXX executable ../bin/main
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
[ 66%] Built target main
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
[ 75%] Building CXX object examples/CMakeFiles/test_tokenizer.dir/test_tokenizer.cpp.o
[ 83%] Linking CXX executable ../bin/test_tokenizer
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
[ 83%] Built target test_tokenizer
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
[ 91%] Building CXX object models/CMakeFiles/quantize.dir/quantize.cpp.o
[100%] Linking CXX executable ../bin/quantize
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
[100%] Built target quantize
make[3]: Leaving directory '/build/go-bert/bert.cpp/build'
make[2]: Leaving directory '/build/go-bert/bert.cpp/build'
cp bert.cpp/build/CMakeFiles/bert.dir/bert.cpp.o bert.o
g++ -I. -I./bert.cpp/ggml/include/ggml/ -I./bert.cpp/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native gobert.cpp -o gobert.o -c
In file included from gobert.cpp:6:
./bert.cpp/bert.cpp: In function 'bert_ctx* bert_load_from_file(const char*)':
./bert.cpp/bert.cpp:470:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
470 | .mem_size = model_mem_req,
| ^
./bert.cpp/bert.cpp:471:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
471 | .mem_buffer = NULL,
| ^
./bert.cpp/bert.cpp:472:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
472 | .no_alloc = false,
| ^
./bert.cpp/bert.cpp:610:89: warning: format '%lld' expects argument of type 'long long int', but argument 5 has type 'int64_t' {aka 'long int'} [-Wformat=]
610 | fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
| ~~~^
| |
| long long int
| %ld
611 | __func__, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]);
| ~~~~~~~~~~~~~
| |
| int64_t {aka long int}
./bert.cpp/bert.cpp:610:95: warning: format '%lld' expects argument of type 'long long int', but argument 6 has type 'int64_t' {aka 'long int'} [-Wformat=]
610 | fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
| ~~~^
| |
| long long int
| %ld
611 | __func__, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]);
| ~~~~~~~~~~~~~
| |
| int64_t {aka long int}
./bert.cpp/bert.cpp:610:112: warning: format '%lld' expects argument of type 'long long int', but argument 7 has type 'int64_t' {aka 'long int'} [-Wformat=]
610 | fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
| ~~~^
| |
| long long int
| %ld
611 | __func__, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]);
| ~~~~~
| |
| int64_t {aka long int}
./bert.cpp/bert.cpp:610:118: warning: format '%lld' expects argument of type 'long long int', but argument 8 has type 'int64_t' {aka 'long int'} [-Wformat=]
610 | fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
| ~~~^
| |
| long long int
| %ld
611 | __func__, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]);
| ~~~~~
| |
| int64_t {aka long int}
./bert.cpp/bert.cpp:624:37: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'int64_t' {aka 'long int'} [-Wformat=]
624 | printf("%24s - [%5lld, %5lld], type = %6s, %6.2f MB, %9zu bytes\n", name.data(), ne[0], ne[1], ftype_str[ftype], ggml_nbytes(tensor) / 1024.0 / 1024.0, ggml_nbytes(tensor));
| ~~~~^ ~~~~~
| | |
| long long int int64_t {aka long int}
| %5ld
./bert.cpp/bert.cpp:624:44: warning: format '%lld' expects argument of type 'long long int', but argument 4 has type 'int64_t' {aka 'long int'} [-Wformat=]
624 | printf("%24s - [%5lld, %5lld], type = %6s, %6.2f MB, %9zu bytes\n", name.data(), ne[0], ne[1], ftype_str[ftype], ggml_nbytes(tensor) / 1024.0 / 1024.0, ggml_nbytes(tensor));
| ~~~~^ ~~~~~
| | |
| long long int int64_t {aka long int}
| %5ld
./bert.cpp/bert.cpp:655:101: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 6 has type 'long unsigned int' [-Wformat=]
655 | fprintf(stderr, "%s: tensor '%s' has wrong size in model file: got %zu, expected %llu\n",
| ~~~^
| |
| long long unsigned int
| %lu
656 | __func__, name.data(), ggml_nbytes(tensor), nelements * bpe);
| ~~~~~~~~~~~~~~~
| |
| long unsigned int
./bert.cpp/bert.cpp:692:56: warning: format '%lld' expects argument of type 'long long int', but argument 4 has type 'int64_t' {aka 'long int'} [-Wformat=]
692 | printf("%s: mem_per_token %zd KB, mem_per_input %lld MB\n", __func__, new_bert->mem_per_token / (1 << 10), new_bert->mem_per_input / (1 << 20));
| ~~~^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| | |
| long long int int64_t {aka long int}
| %ld
./bert.cpp/bert.cpp: In function 'void bert_eval_batch(bert_ctx*, int32_t, int32_t, bert_vocab_id**, int32_t*, float**)':
./bert.cpp/bert.cpp:776:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
776 | .mem_size = buf_compute.size,
| ^
./bert.cpp/bert.cpp:777:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
777 | .mem_buffer = buf_compute.data,
| ^
./bert.cpp/bert.cpp:778:13: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
778 | .no_alloc = false,
| ^
gobert.cpp: In function 'int bert_token_embeddings(void*, void*, int*, int, float*)':
gobert.cpp:32:23: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<float>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
32 | for (int i = 0; i < embeddings.size(); i++) {
| ~~^~~~~~~~~~~~~~~~~~~
gobert.cpp:19:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
19 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
gobert.cpp:24:9: warning: unused variable 'N' [-Wunused-variable]
24 | int N = bert_n_max_tokens(bctx);
| ^
gobert.cpp: In function 'int bert_embeddings(void*, void*, float*)':
gobert.cpp:53:23: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<float>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
53 | for (int i = 0; i < embeddings.size(); i++) {
| ~~^~~~~~~~~~~~~~~~~~~
gobert.cpp:39:19: warning: unused variable 't_main_start_us' [-Wunused-variable]
39 | const int64_t t_main_start_us = ggml_time_us();
| ^~~~~~~~~~~~~~~
cd bert.cpp/build && make VERBOSE=1 ggml && cp -rf ggml/src/CMakeFiles/ggml.dir/ggml.c.o ../../ggml.o
make[2]: Entering directory '/build/go-bert/bert.cpp/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -S/build/go-bert/bert.cpp -B/build/go-bert/bert.cpp/build --check-build-system CMakeFiles/Makefile.cmake 0
make -f CMakeFiles/Makefile2 ggml
make[3]: Entering directory '/build/go-bert/bert.cpp/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -S/build/go-bert/bert.cpp -B/build/go-bert/bert.cpp/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-bert/bert.cpp/build/CMakeFiles 2
make -f CMakeFiles/Makefile2 ggml/src/CMakeFiles/ggml.dir/all
make[4]: Entering directory '/build/go-bert/bert.cpp/build'
make -f ggml/src/CMakeFiles/ggml.dir/build.make ggml/src/CMakeFiles/ggml.dir/depend
make[5]: Entering directory '/build/go-bert/bert.cpp/build'
cd /build/go-bert/bert.cpp/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-bert/bert.cpp /build/go-bert/bert.cpp/ggml/src /build/go-bert/bert.cpp/build /build/go-bert/bert.cpp/build/ggml/src /build/go-bert/bert.cpp/build/ggml/src/CMakeFiles/ggml.dir/DependInfo.cmake "--color="
Dependencies file "ggml/src/CMakeFiles/ggml.dir/ggml.c.o.d" is newer than depends file "/build/go-bert/bert.cpp/build/ggml/src/CMakeFiles/ggml.dir/compiler_depend.internal".
Consolidate compiler generated dependencies of target ggml
make[5]: Leaving directory '/build/go-bert/bert.cpp/build'
make -f ggml/src/CMakeFiles/ggml.dir/build.make ggml/src/CMakeFiles/ggml.dir/build
make[5]: Entering directory '/build/go-bert/bert.cpp/build'
make[5]: Nothing to be done for 'ggml/src/CMakeFiles/ggml.dir/build'.
make[5]: Leaving directory '/build/go-bert/bert.cpp/build'
[100%] Built target ggml
make[4]: Leaving directory '/build/go-bert/bert.cpp/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-bert/bert.cpp/build/CMakeFiles 0
make[3]: Leaving directory '/build/go-bert/bert.cpp/build'
make[2]: Leaving directory '/build/go-bert/bert.cpp/build'
ar src libgobert.a gobert.o ggml.o
make[1]: Leaving directory '/build/go-bert'
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-bert LIBRARY_PATH=/build/go-bert \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/bert-embeddings ./cmd/grpc/bert-embeddings/
make -C go-ggllm BUILD_TYPE=cublas libggllm.a
make[1]: Entering directory '/build/go-ggllm'
I ggllm.cpp build info:
I UNAME_S: Linux
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I./ggllm.cpp -I. -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I./ggllm.cpp -I. -I./ggllm.cpp/examples -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread
I CGO_LDFLAGS:
I LDFLAGS:
I BUILD_TYPE: cublas
I CMAKE_ARGS: -DLLAMA_CUBLAS=ON
I EXTRA_TARGETS: ggllm.cpp/ggml-cuda.o
I CC: cc (Debian 10.2.1-6) 10.2.1 20210110
I CXX: g++ (Debian 10.2.1-6) 10.2.1 20210110
cd ggllm.cpp && patch -p1 < ../patches/1902-cuda.patch
patching file examples/falcon_common.cpp
patching file libfalcon.cpp
patching file libfalcon.h
touch prepare
mkdir -p build
cd build && cmake ../ggllm.cpp -DLLAMA_CUBLAS=ON && VERBOSE=1 cmake --build . --config Release && cp -rf CMakeFiles/ggml.dir/ggml.c.o ../ggllm.cpp/ggml.o
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.30.2")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found CUDAToolkit: /usr/local/cuda/include (found version "12.1.105")
-- cuBLAS found
-- The CUDA compiler identification is NVIDIA 12.1.105
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CUDA found, version: 12.1.105
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- GGML CUDA sources found, configuring CUDA architecture
-- Configuring done (1.7s)
CMake Warning (dev) in CMakeLists.txt:
Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
empty CUDA_ARCHITECTURES not allowed. Run "cmake --help-policy CMP0104"
for policy details. Use the cmake_policy command to set the policy and
suppress this warning.
CUDA_ARCHITECTURES is empty for target "ggml_static".
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
empty CUDA_ARCHITECTURES not allowed. Run "cmake --help-policy CMP0104"
for policy details. Use the cmake_policy command to set the policy and
suppress this warning.
CUDA_ARCHITECTURES is empty for target "ggml_static".
This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done (0.0s)
-- Build files have been written to: /build/go-ggllm/build
Change Dir: '/build/go-ggllm/build'
Run Build Command(s): /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E env VERBOSE=1 /usr/bin/gmake -f Makefile
gmake[2]: Entering directory '/build/go-ggllm/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -S/build/go-ggllm/ggllm.cpp -B/build/go-ggllm/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-ggllm/build/CMakeFiles /build/go-ggllm/build//CMakeFiles/progress.marks
/usr/bin/gmake -f CMakeFiles/Makefile2 all
gmake[3]: Entering directory '/build/go-ggllm/build'
/usr/bin/gmake -f CMakeFiles/BUILD_INFO.dir/build.make CMakeFiles/BUILD_INFO.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp /build/go-ggllm/build /build/go-ggllm/build /build/go-ggllm/build/CMakeFiles/BUILD_INFO.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f CMakeFiles/BUILD_INFO.dir/build.make CMakeFiles/BUILD_INFO.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
gmake[4]: Nothing to be done for 'CMakeFiles/BUILD_INFO.dir/build'.
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 2%] Built target BUILD_INFO
/usr/bin/gmake -f CMakeFiles/ggml.dir/build.make CMakeFiles/ggml.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp /build/go-ggllm/build /build/go-ggllm/build /build/go-ggllm/build/CMakeFiles/ggml.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f CMakeFiles/ggml.dir/build.make CMakeFiles/ggml.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 4%] Building C object CMakeFiles/ggml.dir/ggml.c.o
/usr/bin/cc -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-ggllm/ggllm.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/ggml.dir/ggml.c.o -MF CMakeFiles/ggml.dir/ggml.c.o.d -o CMakeFiles/ggml.dir/ggml.c.o -c /build/go-ggllm/ggllm.cpp/ggml.c
/build/go-ggllm/ggllm.cpp/ggml.c: In function ‘ggml_compute_forward_mul_mat_f32’:
/build/go-ggllm/ggllm.cpp/ggml.c:10924:19: warning: unused variable ‘ne10’ [-Wunused-variable]
10924 | const int64_t ne10 = src1->ne[0];
| ^~~~
[ 6%] Building CUDA object CMakeFiles/ggml.dir/ggml-cuda.cu.o
/usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 --options-file CMakeFiles/ggml.dir/includes_CUDA.rsp -O3 -DNDEBUG -std=c++11 -mf16c -mfma -mavx -mavx2 -Xcompiler -pthread -MD -MT CMakeFiles/ggml.dir/ggml-cuda.cu.o -MF CMakeFiles/ggml.dir/ggml-cuda.cu.o.d -x cu -c /build/go-ggllm/ggllm.cpp/ggml-cuda.cu -o CMakeFiles/ggml.dir/ggml-cuda.cu.o
[ 8%] Building C object CMakeFiles/ggml.dir/k_quants.c.o
/usr/bin/cc -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-ggllm/ggllm.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/ggml.dir/k_quants.c.o -MF CMakeFiles/ggml.dir/k_quants.c.o.d -o CMakeFiles/ggml.dir/k_quants.c.o -c /build/go-ggllm/ggllm.cpp/k_quants.c
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 8%] Built target ggml
/usr/bin/gmake -f CMakeFiles/ggml_static.dir/build.make CMakeFiles/ggml_static.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp /build/go-ggllm/build /build/go-ggllm/build /build/go-ggllm/build/CMakeFiles/ggml_static.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f CMakeFiles/ggml_static.dir/build.make CMakeFiles/ggml_static.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 10%] Linking CUDA static library libggml_static.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/ggml_static.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/ggml_static.dir/link.txt --verbose=1
/usr/bin/ar qc libggml_static.a CMakeFiles/ggml.dir/ggml.c.o "CMakeFiles/ggml.dir/ggml-cuda.cu.o" CMakeFiles/ggml.dir/k_quants.c.o
/usr/bin/ranlib libggml_static.a
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 10%] Built target ggml_static
/usr/bin/gmake -f CMakeFiles/llama.dir/build.make CMakeFiles/llama.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp /build/go-ggllm/build /build/go-ggllm/build /build/go-ggllm/build/CMakeFiles/llama.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f CMakeFiles/llama.dir/build.make CMakeFiles/llama.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 12%] Building CXX object CMakeFiles/llama.dir/llama.cpp.o
/usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-ggllm/ggllm.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/llama.dir/llama.cpp.o -MF CMakeFiles/llama.dir/llama.cpp.o.d -o CMakeFiles/llama.dir/llama.cpp.o -c /build/go-ggllm/ggllm.cpp/llama.cpp
[ 14%] Linking CXX static library libllama.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama.dir/link.txt --verbose=1
/usr/bin/ar qc libllama.a CMakeFiles/llama.dir/llama.cpp.o CMakeFiles/ggml.dir/ggml.c.o "CMakeFiles/ggml.dir/ggml-cuda.cu.o" CMakeFiles/ggml.dir/k_quants.c.o
/usr/bin/ranlib libllama.a
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 14%] Built target llama
/usr/bin/gmake -f CMakeFiles/cmpnct_unicode.dir/build.make CMakeFiles/cmpnct_unicode.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp /build/go-ggllm/build /build/go-ggllm/build /build/go-ggllm/build/CMakeFiles/cmpnct_unicode.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f CMakeFiles/cmpnct_unicode.dir/build.make CMakeFiles/cmpnct_unicode.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 16%] Building CXX object CMakeFiles/cmpnct_unicode.dir/cmpnct_unicode.cpp.o
/usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT CMakeFiles/cmpnct_unicode.dir/cmpnct_unicode.cpp.o -MF CMakeFiles/cmpnct_unicode.dir/cmpnct_unicode.cpp.o.d -o CMakeFiles/cmpnct_unicode.dir/cmpnct_unicode.cpp.o -c /build/go-ggllm/ggllm.cpp/cmpnct_unicode.cpp
[ 18%] Linking CXX static library libcmpnct_unicode.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/cmpnct_unicode.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmpnct_unicode.dir/link.txt --verbose=1
/usr/bin/ar qc libcmpnct_unicode.a CMakeFiles/cmpnct_unicode.dir/cmpnct_unicode.cpp.o
/usr/bin/ranlib libcmpnct_unicode.a
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 18%] Built target cmpnct_unicode
/usr/bin/gmake -f CMakeFiles/falcon.dir/build.make CMakeFiles/falcon.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp /build/go-ggllm/build /build/go-ggllm/build /build/go-ggllm/build/CMakeFiles/falcon.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f CMakeFiles/falcon.dir/build.make CMakeFiles/falcon.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 20%] Building CXX object CMakeFiles/falcon.dir/libfalcon.cpp.o
/usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-ggllm/ggllm.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/falcon.dir/libfalcon.cpp.o -MF CMakeFiles/falcon.dir/libfalcon.cpp.o.d -o CMakeFiles/falcon.dir/libfalcon.cpp.o -c /build/go-ggllm/ggllm.cpp/libfalcon.cpp
/build/go-ggllm/ggllm.cpp/libfalcon.cpp: In function ‘bool falcon_eval_internal(falcon_context&, const falcon_token*, int, int, int, const char*, int)’:
/build/go-ggllm/ggllm.cpp/libfalcon.cpp:2076:20: warning: variable ‘offload_func_nr’ set but not used [-Wunused-but-set-variable]
2076 | offload_func_t offload_func_nr = llama_nop; // nr = non-repeating
| ^~~~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/libfalcon.cpp:2077:20: warning: variable ‘offload_func_kqv’ set but not used [-Wunused-but-set-variable]
2077 | offload_func_t offload_func_kqv = llama_nop;
| ^~~~~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/libfalcon.cpp:2316:20: warning: variable ‘offload_func’ set but not used [-Wunused-but-set-variable]
2316 | offload_func_t offload_func = llama_nop;
| ^~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/libfalcon.cpp: In function ‘size_t falcon_copy_state_data(falcon_context*, uint8_t*)’:
/build/go-ggllm/ggllm.cpp/libfalcon.cpp:4114:22: warning: unused variable ‘n_embd’ [-Wunused-variable]
4114 | const int n_embd = hparams.n_embd;
| ^~~~~~
/build/go-ggllm/ggllm.cpp/libfalcon.cpp: In function ‘size_t falcon_set_state_data(falcon_context*, uint8_t*)’:
/build/go-ggllm/ggllm.cpp/libfalcon.cpp:4230:22: warning: unused variable ‘n_embd’ [-Wunused-variable]
4230 | const int n_embd = hparams.n_embd;
| ^~~~~~
[ 22%] Linking CXX static library libfalcon.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/falcon.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/falcon.dir/link.txt --verbose=1
/usr/bin/ar qc libfalcon.a CMakeFiles/falcon.dir/libfalcon.cpp.o CMakeFiles/ggml.dir/ggml.c.o "CMakeFiles/ggml.dir/ggml-cuda.cu.o" CMakeFiles/ggml.dir/k_quants.c.o
/usr/bin/ranlib libfalcon.a
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 22%] Built target falcon
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-fns.dir/build.make tests/CMakeFiles/test-quantize-fns.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/tests /build/go-ggllm/build /build/go-ggllm/build/tests /build/go-ggllm/build/tests/CMakeFiles/test-quantize-fns.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-fns.dir/build.make tests/CMakeFiles/test-quantize-fns.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 24%] Building CXX object tests/CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o
cd /build/go-ggllm/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o -MF CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o.d -o CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o -c /build/go-ggllm/ggllm.cpp/tests/test-quantize-fns.cpp
[ 26%] Linking CXX executable ../bin/test-quantize-fns
cd /build/go-ggllm/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-quantize-fns.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o" -o ../bin/test-quantize-fns -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 26%] Built target test-quantize-fns
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-perf.dir/build.make tests/CMakeFiles/test-quantize-perf.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/tests /build/go-ggllm/build /build/go-ggllm/build/tests /build/go-ggllm/build/tests/CMakeFiles/test-quantize-perf.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-perf.dir/build.make tests/CMakeFiles/test-quantize-perf.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 28%] Building CXX object tests/CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o
cd /build/go-ggllm/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o -MF CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o.d -o CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o -c /build/go-ggllm/ggllm.cpp/tests/test-quantize-perf.cpp
[ 30%] Linking CXX executable ../bin/test-quantize-perf
cd /build/go-ggllm/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-quantize-perf.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o" -o ../bin/test-quantize-perf -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 30%] Built target test-quantize-perf
/usr/bin/gmake -f tests/CMakeFiles/test-sampling.dir/build.make tests/CMakeFiles/test-sampling.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/tests /build/go-ggllm/build /build/go-ggllm/build/tests /build/go-ggllm/build/tests/CMakeFiles/test-sampling.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f tests/CMakeFiles/test-sampling.dir/build.make tests/CMakeFiles/test-sampling.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 32%] Building CXX object tests/CMakeFiles/test-sampling.dir/test-sampling.cpp.o
cd /build/go-ggllm/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-sampling.dir/test-sampling.cpp.o -MF CMakeFiles/test-sampling.dir/test-sampling.cpp.o.d -o CMakeFiles/test-sampling.dir/test-sampling.cpp.o -c /build/go-ggllm/ggllm.cpp/tests/test-sampling.cpp
In file included from /build/go-ggllm/ggllm.cpp/tests/test-sampling.cpp:2:
/build/go-ggllm/ggllm.cpp/./libfalcon.h:252:24: warning: ‘FINETUNE_NAME’ defined but not used [-Wunused-variable]
252 | static const char *FINETUNE_NAME[6] = { "UNSPECIFIED", "NONE", "ALPACA", "OPENASSISTANT", "WIZARD", "FALCONINSTRUCT" };
| ^~~~~~~~~~~~~
[ 34%] Linking CXX executable ../bin/test-sampling
cd /build/go-ggllm/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-sampling.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-sampling.dir/test-sampling.cpp.o" -o ../bin/test-sampling -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 34%] Built target test-sampling
/usr/bin/gmake -f examples/CMakeFiles/common.dir/build.make examples/CMakeFiles/common.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples /build/go-ggllm/build /build/go-ggllm/build/examples /build/go-ggllm/build/examples/CMakeFiles/common.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/CMakeFiles/common.dir/build.make examples/CMakeFiles/common.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 36%] Building CXX object examples/CMakeFiles/common.dir/common.cpp.o
cd /build/go-ggllm/build/examples && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/CMakeFiles/common.dir/common.cpp.o -MF CMakeFiles/common.dir/common.cpp.o.d -o CMakeFiles/common.dir/common.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/common.cpp
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 36%] Built target common
/usr/bin/gmake -f examples/CMakeFiles/falcon_common.dir/build.make examples/CMakeFiles/falcon_common.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples /build/go-ggllm/build /build/go-ggllm/build/examples /build/go-ggllm/build/examples/CMakeFiles/falcon_common.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/CMakeFiles/falcon_common.dir/build.make examples/CMakeFiles/falcon_common.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 38%] Building CXX object examples/CMakeFiles/falcon_common.dir/falcon_common.cpp.o
cd /build/go-ggllm/build/examples && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/CMakeFiles/falcon_common.dir/falcon_common.cpp.o -MF CMakeFiles/falcon_common.dir/falcon_common.cpp.o.d -o CMakeFiles/falcon_common.dir/falcon_common.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/falcon_common.cpp
In file included from /build/go-ggllm/ggllm.cpp/examples/falcon_common.h:6,
from /build/go-ggllm/ggllm.cpp/examples/falcon_common.cpp:1:
/build/go-ggllm/ggllm.cpp/./libfalcon.h:252:24: warning: ‘FINETUNE_NAME’ defined but not used [-Wunused-variable]
252 | static const char *FINETUNE_NAME[6] = { "UNSPECIFIED", "NONE", "ALPACA", "OPENASSISTANT", "WIZARD", "FALCONINSTRUCT" };
| ^~~~~~~~~~~~~
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 38%] Built target falcon_common
/usr/bin/gmake -f examples/main/CMakeFiles/main.dir/build.make examples/main/CMakeFiles/main.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/main /build/go-ggllm/build /build/go-ggllm/build/examples/main /build/go-ggllm/build/examples/main/CMakeFiles/main.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/main/CMakeFiles/main.dir/build.make examples/main/CMakeFiles/main.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 40%] Building CXX object examples/main/CMakeFiles/main.dir/main.cpp.o
cd /build/go-ggllm/build/examples/main && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/main/CMakeFiles/main.dir/main.cpp.o -MF CMakeFiles/main.dir/main.cpp.o.d -o CMakeFiles/main.dir/main.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/main/main.cpp
[ 42%] Linking CXX executable ../../bin/main
cd /build/go-ggllm/build/examples/main && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/main.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/main.dir/main.cpp.o ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/main -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 42%] Built target main
/usr/bin/gmake -f examples/falcon/CMakeFiles/falcon_main.dir/build.make examples/falcon/CMakeFiles/falcon_main.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/falcon /build/go-ggllm/build /build/go-ggllm/build/examples/falcon /build/go-ggllm/build/examples/falcon/CMakeFiles/falcon_main.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/falcon/CMakeFiles/falcon_main.dir/build.make examples/falcon/CMakeFiles/falcon_main.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 44%] Building CXX object examples/falcon/CMakeFiles/falcon_main.dir/falcon_main.cpp.o
cd /build/go-ggllm/build/examples/falcon && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/falcon/CMakeFiles/falcon_main.dir/falcon_main.cpp.o -MF CMakeFiles/falcon_main.dir/falcon_main.cpp.o.d -o CMakeFiles/falcon_main.dir/falcon_main.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/falcon/falcon_main.cpp
/build/go-ggllm/ggllm.cpp/examples/falcon/falcon_main.cpp: In function ‘int main(int, char**)’:
/build/go-ggllm/ggllm.cpp/examples/falcon/falcon_main.cpp:963:27: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
963 | if (!embd.empty() && embd.back() == falcon_token_eos() || stopword_fulfilled)
| ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/examples/falcon/falcon_main.cpp:433:10: warning: unused variable ‘falcon_token_newline’ [-Wunused-variable]
433 | auto falcon_token_newline = falcon_token_nl();
| ^~~~~~~~~~~~~~~~~~~~
[ 46%] Linking CXX executable ../../bin/falcon_main
cd /build/go-ggllm/build/examples/falcon && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/falcon_main.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/falcon_main.dir/falcon_main.cpp.o ../CMakeFiles/falcon_common.dir/falcon_common.cpp.o -o ../../bin/falcon_main -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libfalcon.a -pthread ../../libcmpnct_unicode.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 46%] Built target falcon_main
/usr/bin/gmake -f examples/falcon_quantize/CMakeFiles/falcon_quantize.dir/build.make examples/falcon_quantize/CMakeFiles/falcon_quantize.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/falcon_quantize /build/go-ggllm/build /build/go-ggllm/build/examples/falcon_quantize /build/go-ggllm/build/examples/falcon_quantize/CMakeFiles/falcon_quantize.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/falcon_quantize/CMakeFiles/falcon_quantize.dir/build.make examples/falcon_quantize/CMakeFiles/falcon_quantize.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 48%] Building CXX object examples/falcon_quantize/CMakeFiles/falcon_quantize.dir/quantize.cpp.o
cd /build/go-ggllm/build/examples/falcon_quantize && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/falcon_quantize/CMakeFiles/falcon_quantize.dir/quantize.cpp.o -MF CMakeFiles/falcon_quantize.dir/quantize.cpp.o.d -o CMakeFiles/falcon_quantize.dir/quantize.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/falcon_quantize/quantize.cpp
In file included from /build/go-ggllm/ggllm.cpp/examples/falcon_quantize/quantize.cpp:3:
/build/go-ggllm/ggllm.cpp/./libfalcon.h:252:24: warning: ‘FINETUNE_NAME’ defined but not used [-Wunused-variable]
252 | static const char *FINETUNE_NAME[6] = { "UNSPECIFIED", "NONE", "ALPACA", "OPENASSISTANT", "WIZARD", "FALCONINSTRUCT" };
| ^~~~~~~~~~~~~
[ 51%] Linking CXX executable ../../bin/falcon_quantize
cd /build/go-ggllm/build/examples/falcon_quantize && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/falcon_quantize.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/falcon_quantize.dir/quantize.cpp.o -o ../../bin/falcon_quantize -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libfalcon.a -pthread ../../libcmpnct_unicode.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 51%] Built target falcon_quantize
/usr/bin/gmake -f examples/quantize/CMakeFiles/quantize.dir/build.make examples/quantize/CMakeFiles/quantize.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/quantize /build/go-ggllm/build /build/go-ggllm/build/examples/quantize /build/go-ggllm/build/examples/quantize/CMakeFiles/quantize.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/quantize/CMakeFiles/quantize.dir/build.make examples/quantize/CMakeFiles/quantize.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 53%] Building CXX object examples/quantize/CMakeFiles/quantize.dir/quantize.cpp.o
cd /build/go-ggllm/build/examples/quantize && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/quantize/CMakeFiles/quantize.dir/quantize.cpp.o -MF CMakeFiles/quantize.dir/quantize.cpp.o.d -o CMakeFiles/quantize.dir/quantize.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/quantize/quantize.cpp
[ 55%] Linking CXX executable ../../bin/quantize
cd /build/go-ggllm/build/examples/quantize && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/quantize.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/quantize.dir/quantize.cpp.o -o ../../bin/quantize -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 55%] Built target quantize
/usr/bin/gmake -f examples/quantize-stats/CMakeFiles/quantize-stats.dir/build.make examples/quantize-stats/CMakeFiles/quantize-stats.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/quantize-stats /build/go-ggllm/build /build/go-ggllm/build/examples/quantize-stats /build/go-ggllm/build/examples/quantize-stats/CMakeFiles/quantize-stats.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/quantize-stats/CMakeFiles/quantize-stats.dir/build.make examples/quantize-stats/CMakeFiles/quantize-stats.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 57%] Building CXX object examples/quantize-stats/CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o
cd /build/go-ggllm/build/examples/quantize-stats && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/quantize-stats/CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o -MF CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o.d -o CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/quantize-stats/quantize-stats.cpp
[ 59%] Linking CXX executable ../../bin/quantize-stats
cd /build/go-ggllm/build/examples/quantize-stats && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/quantize-stats.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o" -o ../../bin/quantize-stats -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 59%] Built target quantize-stats
/usr/bin/gmake -f examples/perplexity/CMakeFiles/perplexity.dir/build.make examples/perplexity/CMakeFiles/perplexity.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/perplexity /build/go-ggllm/build /build/go-ggllm/build/examples/perplexity /build/go-ggllm/build/examples/perplexity/CMakeFiles/perplexity.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/perplexity/CMakeFiles/perplexity.dir/build.make examples/perplexity/CMakeFiles/perplexity.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 61%] Building CXX object examples/perplexity/CMakeFiles/perplexity.dir/perplexity.cpp.o
cd /build/go-ggllm/build/examples/perplexity && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/perplexity/CMakeFiles/perplexity.dir/perplexity.cpp.o -MF CMakeFiles/perplexity.dir/perplexity.cpp.o.d -o CMakeFiles/perplexity.dir/perplexity.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/perplexity/perplexity.cpp
[ 63%] Linking CXX executable ../../bin/perplexity
cd /build/go-ggllm/build/examples/perplexity && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/perplexity.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/perplexity.dir/perplexity.cpp.o ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/perplexity -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 63%] Built target perplexity
/usr/bin/gmake -f examples/falcon_perplexity/CMakeFiles/falcon_perplexity.dir/build.make examples/falcon_perplexity/CMakeFiles/falcon_perplexity.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/falcon_perplexity /build/go-ggllm/build /build/go-ggllm/build/examples/falcon_perplexity /build/go-ggllm/build/examples/falcon_perplexity/CMakeFiles/falcon_perplexity.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/falcon_perplexity/CMakeFiles/falcon_perplexity.dir/build.make examples/falcon_perplexity/CMakeFiles/falcon_perplexity.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 65%] Building CXX object examples/falcon_perplexity/CMakeFiles/falcon_perplexity.dir/falcon_perplexity.cpp.o
cd /build/go-ggllm/build/examples/falcon_perplexity && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/falcon_perplexity/CMakeFiles/falcon_perplexity.dir/falcon_perplexity.cpp.o -MF CMakeFiles/falcon_perplexity.dir/falcon_perplexity.cpp.o.d -o CMakeFiles/falcon_perplexity.dir/falcon_perplexity.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/falcon_perplexity/falcon_perplexity.cpp
In file included from /build/go-ggllm/ggllm.cpp/examples/falcon_common.h:6,
from /build/go-ggllm/ggllm.cpp/examples/falcon_perplexity/falcon_perplexity.cpp:1:
/build/go-ggllm/ggllm.cpp/./libfalcon.h:252:24: warning: ‘FINETUNE_NAME’ defined but not used [-Wunused-variable]
252 | static const char *FINETUNE_NAME[6] = { "UNSPECIFIED", "NONE", "ALPACA", "OPENASSISTANT", "WIZARD", "FALCONINSTRUCT" };
| ^~~~~~~~~~~~~
[ 67%] Linking CXX executable ../../bin/falcon_perplexity
cd /build/go-ggllm/build/examples/falcon_perplexity && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/falcon_perplexity.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/falcon_perplexity.dir/falcon_perplexity.cpp.o ../CMakeFiles/falcon_common.dir/falcon_common.cpp.o -o ../../bin/falcon_perplexity -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libfalcon.a -pthread ../../libcmpnct_unicode.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 67%] Built target falcon_perplexity
/usr/bin/gmake -f examples/embedding/CMakeFiles/embedding.dir/build.make examples/embedding/CMakeFiles/embedding.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/embedding /build/go-ggllm/build /build/go-ggllm/build/examples/embedding /build/go-ggllm/build/examples/embedding/CMakeFiles/embedding.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/embedding/CMakeFiles/embedding.dir/build.make examples/embedding/CMakeFiles/embedding.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 69%] Building CXX object examples/embedding/CMakeFiles/embedding.dir/embedding.cpp.o
cd /build/go-ggllm/build/examples/embedding && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/embedding/CMakeFiles/embedding.dir/embedding.cpp.o -MF CMakeFiles/embedding.dir/embedding.cpp.o.d -o CMakeFiles/embedding.dir/embedding.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/embedding/embedding.cpp
[ 71%] Linking CXX executable ../../bin/embedding
cd /build/go-ggllm/build/examples/embedding && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/embedding.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/embedding.dir/embedding.cpp.o ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/embedding -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 71%] Built target embedding
/usr/bin/gmake -f examples/save-load-state/CMakeFiles/save-load-state.dir/build.make examples/save-load-state/CMakeFiles/save-load-state.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/save-load-state /build/go-ggllm/build /build/go-ggllm/build/examples/save-load-state /build/go-ggllm/build/examples/save-load-state/CMakeFiles/save-load-state.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/save-load-state/CMakeFiles/save-load-state.dir/build.make examples/save-load-state/CMakeFiles/save-load-state.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 73%] Building CXX object examples/save-load-state/CMakeFiles/save-load-state.dir/save-load-state.cpp.o
cd /build/go-ggllm/build/examples/save-load-state && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/save-load-state/CMakeFiles/save-load-state.dir/save-load-state.cpp.o -MF CMakeFiles/save-load-state.dir/save-load-state.cpp.o.d -o CMakeFiles/save-load-state.dir/save-load-state.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/save-load-state/save-load-state.cpp
[ 75%] Linking CXX executable ../../bin/save-load-state
cd /build/go-ggllm/build/examples/save-load-state && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/save-load-state.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/save-load-state.dir/save-load-state.cpp.o" ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/save-load-state -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 75%] Built target save-load-state
/usr/bin/gmake -f examples/benchmark/CMakeFiles/benchmark.dir/build.make examples/benchmark/CMakeFiles/benchmark.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/benchmark /build/go-ggllm/build /build/go-ggllm/build/examples/benchmark /build/go-ggllm/build/examples/benchmark/CMakeFiles/benchmark.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/benchmark/CMakeFiles/benchmark.dir/build.make examples/benchmark/CMakeFiles/benchmark.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 77%] Building CXX object examples/benchmark/CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o
cd /build/go-ggllm/build/examples/benchmark && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/benchmark/CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o -MF CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o.d -o CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/benchmark/benchmark-matmult.cpp
[ 79%] Linking CXX executable ../../bin/benchmark
cd /build/go-ggllm/build/examples/benchmark && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/benchmark.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o" ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/benchmark -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 79%] Built target benchmark
/usr/bin/gmake -f examples/baby-llama/CMakeFiles/baby-llama.dir/build.make examples/baby-llama/CMakeFiles/baby-llama.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/baby-llama /build/go-ggllm/build /build/go-ggllm/build/examples/baby-llama /build/go-ggllm/build/examples/baby-llama/CMakeFiles/baby-llama.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/baby-llama/CMakeFiles/baby-llama.dir/build.make examples/baby-llama/CMakeFiles/baby-llama.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 81%] Building CXX object examples/baby-llama/CMakeFiles/baby-llama.dir/baby-llama.cpp.o
cd /build/go-ggllm/build/examples/baby-llama && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/baby-llama/CMakeFiles/baby-llama.dir/baby-llama.cpp.o -MF CMakeFiles/baby-llama.dir/baby-llama.cpp.o.d -o CMakeFiles/baby-llama.dir/baby-llama.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/baby-llama/baby-llama.cpp
/build/go-ggllm/ggllm.cpp/examples/baby-llama/baby-llama.cpp: In function ‘int main(int, char**)’:
/build/go-ggllm/ggllm.cpp/examples/baby-llama/baby-llama.cpp:1602:32: warning: variable ‘opt_params_adam’ set but not used [-Wunused-but-set-variable]
1602 | struct ggml_opt_params opt_params_adam = ggml_opt_default_params(GGML_OPT_ADAM);
| ^~~~~~~~~~~~~~~
[ 83%] Linking CXX executable ../../bin/baby-llama
cd /build/go-ggllm/build/examples/baby-llama && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/baby-llama.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/baby-llama.dir/baby-llama.cpp.o" ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/baby-llama -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 83%] Built target baby-llama
/usr/bin/gmake -f examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/build.make examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch /build/go-ggllm/build /build/go-ggllm/build/examples/train-text-from-scratch /build/go-ggllm/build/examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/build.make examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 85%] Building CXX object examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o
cd /build/go-ggllm/build/examples/train-text-from-scratch && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o -MF CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o.d -o CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp: In function ‘void write_tensor(llama_file*, ggml_tensor*)’:
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp:2371:21: warning: suggest parentheses around ‘-’ in operand of ‘&’ [-Wparentheses]
2371 | file->seek(0-file->tell() & 31, SEEK_CUR);
| ~^~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp:2386:17: warning: suggest parentheses around ‘-’ in operand of ‘&’ [-Wparentheses]
2386 | file->seek(0-file->tell() & 31, SEEK_CUR);
| ~^~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp: In function ‘void read_tensor(llama_file*, ggml_tensor*)’:
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp:2407:17: warning: suggest parentheses around ‘-’ in operand of ‘&’ [-Wparentheses]
2407 | file->seek(0-file->tell() & 31, SEEK_CUR);
| ~^~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp: In function ‘void init_model(my_llama_model*)’:
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp:305:16: warning: ‘char* strncpy(char*, const char*, size_t)’ specified bound 64 equals destination size [-Wstringop-truncation]
305 | strncpy(layer.w1->name, (layers_i + ".feed_forward.w1.weight").c_str(), sizeof(layer.w1->name));
| ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp:306:16: warning: ‘char* strncpy(char*, const char*, size_t)’ specified bound 64 equals destination size [-Wstringop-truncation]
306 | strncpy(layer.w2->name, (layers_i + ".feed_forward.w2.weight").c_str(), sizeof(layer.w2->name));
| ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/go-ggllm/ggllm.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp:307:16: warning: ‘char* strncpy(char*, const char*, size_t)’ specified bound 64 equals destination size [-Wstringop-truncation]
307 | strncpy(layer.w3->name, (layers_i + ".feed_forward.w3.weight").c_str(), sizeof(layer.w3->name));
| ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 87%] Linking CXX executable ../../bin/train-text-from-scratch
cd /build/go-ggllm/build/examples/train-text-from-scratch && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/train-text-from-scratch.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o" ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/train-text-from-scratch -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 87%] Built target train-text-from-scratch
/usr/bin/gmake -f examples/simple/CMakeFiles/simple.dir/build.make examples/simple/CMakeFiles/simple.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/examples/simple /build/go-ggllm/build /build/go-ggllm/build/examples/simple /build/go-ggllm/build/examples/simple/CMakeFiles/simple.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f examples/simple/CMakeFiles/simple.dir/build.make examples/simple/CMakeFiles/simple.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 89%] Building CXX object examples/simple/CMakeFiles/simple.dir/simple.cpp.o
cd /build/go-ggllm/build/examples/simple && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/examples -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/simple/CMakeFiles/simple.dir/simple.cpp.o -MF CMakeFiles/simple.dir/simple.cpp.o.d -o CMakeFiles/simple.dir/simple.cpp.o -c /build/go-ggllm/ggllm.cpp/examples/simple/simple.cpp
[ 91%] Linking CXX executable ../../bin/simple
cd /build/go-ggllm/build/examples/simple && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/simple.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/simple.dir/simple.cpp.o ../CMakeFiles/common.dir/common.cpp.o -o ../../bin/simple -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 91%] Built target simple
/usr/bin/gmake -f pocs/vdot/CMakeFiles/vdot.dir/build.make pocs/vdot/CMakeFiles/vdot.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/pocs/vdot /build/go-ggllm/build /build/go-ggllm/build/pocs/vdot /build/go-ggllm/build/pocs/vdot/CMakeFiles/vdot.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f pocs/vdot/CMakeFiles/vdot.dir/build.make pocs/vdot/CMakeFiles/vdot.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 93%] Building CXX object pocs/vdot/CMakeFiles/vdot.dir/vdot.cpp.o
cd /build/go-ggllm/build/pocs/vdot && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/pocs -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT pocs/vdot/CMakeFiles/vdot.dir/vdot.cpp.o -MF CMakeFiles/vdot.dir/vdot.cpp.o.d -o CMakeFiles/vdot.dir/vdot.cpp.o -c /build/go-ggllm/ggllm.cpp/pocs/vdot/vdot.cpp
[ 95%] Linking CXX executable ../../bin/vdot
cd /build/go-ggllm/build/pocs/vdot && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/vdot.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/vdot.dir/vdot.cpp.o ../../examples/CMakeFiles/common.dir/common.cpp.o -o ../../bin/vdot -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[ 95%] Built target vdot
/usr/bin/gmake -f pocs/vdot/CMakeFiles/q8dot.dir/build.make pocs/vdot/CMakeFiles/q8dot.dir/depend
gmake[4]: Entering directory '/build/go-ggllm/build'
cd /build/go-ggllm/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-ggllm/ggllm.cpp /build/go-ggllm/ggllm.cpp/pocs/vdot /build/go-ggllm/build /build/go-ggllm/build/pocs/vdot /build/go-ggllm/build/pocs/vdot/CMakeFiles/q8dot.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-ggllm/build'
/usr/bin/gmake -f pocs/vdot/CMakeFiles/q8dot.dir/build.make pocs/vdot/CMakeFiles/q8dot.dir/build
gmake[4]: Entering directory '/build/go-ggllm/build'
[ 97%] Building CXX object pocs/vdot/CMakeFiles/q8dot.dir/q8dot.cpp.o
cd /build/go-ggllm/build/pocs/vdot && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_PERF=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/usr/local/cuda/include -I/build/go-ggllm/ggllm.cpp/pocs -I/build/go-ggllm/ggllm.cpp/examples/. -I/build/go-ggllm/ggllm.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT pocs/vdot/CMakeFiles/q8dot.dir/q8dot.cpp.o -MF CMakeFiles/q8dot.dir/q8dot.cpp.o.d -o CMakeFiles/q8dot.dir/q8dot.cpp.o -c /build/go-ggllm/ggllm.cpp/pocs/vdot/q8dot.cpp
[100%] Linking CXX executable ../../bin/q8dot
cd /build/go-ggllm/build/pocs/vdot && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/q8dot.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/q8dot.dir/q8dot.cpp.o ../../examples/CMakeFiles/common.dir/common.cpp.o -o ../../bin/q8dot -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-ggllm/build'
[100%] Built target q8dot
gmake[3]: Leaving directory '/build/go-ggllm/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-ggllm/build/CMakeFiles 0
gmake[2]: Leaving directory '/build/go-ggllm/build'
cd build && cp -rf CMakeFiles/cmpnct_unicode.dir/cmpnct_unicode.cpp.o ../ggllm.cpp/cmpnct_unicode.o
cd build && cp -rf CMakeFiles/llama.dir/llama.cpp.o ../ggllm.cpp/llama.o
cd build && cp -rf CMakeFiles/falcon.dir/libfalcon.cpp.o ../ggllm.cpp/libfalcon.o
cd build && cp -rf examples/CMakeFiles/falcon_common.dir/falcon_common.cpp.o ../ggllm.cpp/falcon_common.o
g++ -I./ggllm.cpp -I. -I./ggllm.cpp/examples -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread -I./ggllm.cpp -I./ggllm.cpp/examples falcon_binding.cpp -o falcon_binding.o -c
falcon_binding.cpp: In function 'int falcon_predict(void*, void*, char*, bool)':
falcon_binding.cpp:468:48: warning: cast from type 'const char*' to type 'char*' casts away qualifiers [-Wcast-qual]
468 | if (!returntokenCallback(state_pr, (char*)token_str)) {
| ^~~~~~~~~~~~~~~~
falcon_binding.cpp:517:27: warning: suggest parentheses around '&&' within '||' [-Wparentheses]
517 | if (!embd.empty() && embd.back() == falcon_token_eos() || stopword_fulfilled)
| ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
falcon_binding.cpp:186:10: warning: unused variable 'falcon_token_newline' [-Wunused-variable]
186 | auto falcon_token_newline = falcon_token_nl();
| ^~~~~~~~~~~~~~~~~~~~
falcon_binding.cpp:231:10: warning: variable 'input_echo' set but not used [-Wunused-but-set-variable]
231 | bool input_echo = true;
| ^~~~~~~~~~
cd build && cp -rf CMakeFiles/ggml.dir/k_quants.c.o ../ggllm.cpp/k_quants.o
cd build && cp -rf CMakeFiles/ggml.dir/ggml-cuda.cu.o ../ggllm.cpp/ggml-cuda.o
ar src libggllm.a ggllm.cpp/libfalcon.o ggllm.cpp/cmpnct_unicode.o ggllm.cpp/ggml.o ggllm.cpp/k_quants.o ggllm.cpp/ggml-cuda.o ggllm.cpp/falcon_common.o falcon_binding.o
make[1]: Leaving directory '/build/go-ggllm'
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggllm LIBRARY_PATH=/build/go-ggllm \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/falcon ./cmd/grpc/falcon/
cd bloomz && make libbloomz.a
make[1]: Entering directory '/build/bloomz'
I llama.cpp build info:
I UNAME_S: Linux
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -pthread -mavx -mavx2 -mfma -mf16c -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS:
I CC: cc (Debian 10.2.1-6) 10.2.1 20210110
I CXX: g++ (Debian 10.2.1-6) 10.2.1 20210110
cc -I. -O3 -DNDEBUG -std=c11 -fPIC -pthread -mavx -mavx2 -mfma -mf16c -msse3 -c ggml.c -o ggml.o
g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread -c utils.cpp -o utils.o
g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread bloomz.cpp ggml.o utils.o -o bloomz.o -c
g++: warning: ggml.o: linker input file unused because linking not done
g++: warning: utils.o: linker input file unused because linking not done
ar src libbloomz.a bloomz.o ggml.o utils.o
make[1]: Leaving directory '/build/bloomz'
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/bloomz LIBRARY_PATH=/build/bloomz \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/bloomz ./cmd/grpc/bloomz/
make -C go-llama BUILD_TYPE=cublas libbinding.a
make[1]: Entering directory '/build/go-llama'
I llama.cpp build info:
I UNAME_S: Linux
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I./llama.cpp -I. -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I./llama.cpp -I. -I./llama.cpp/examples -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread
I CGO_LDFLAGS:
I LDFLAGS:
I BUILD_TYPE: cublas
I CMAKE_ARGS: -DLLAMA_CUBLAS=ON
I EXTRA_TARGETS: llama.cpp/ggml-cuda.o
I CC: cc (Debian 10.2.1-6) 10.2.1 20210110
I CXX: g++ (Debian 10.2.1-6) 10.2.1 20210110
cd llama.cpp && patch -p1 < ../patches/1902-cuda.patch
patching file examples/common.cpp
patching file examples/common.h
touch prepare
mkdir -p build
cd build && cmake ../llama.cpp -DLLAMA_CUBLAS=ON && VERBOSE=1 cmake --build . --config Release && cp -rf CMakeFiles/ggml.dir/ggml.c.o ../llama.cpp/ggml.o
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.30.2")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found CUDAToolkit: /usr/local/cuda/include (found version "12.1.105")
-- cuBLAS found
-- The CUDA compiler identification is NVIDIA 12.1.105
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Using CUDA architectures: 52;61;70
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Configuring done (1.7s)
-- Generating done (0.1s)
-- Build files have been written to: /build/go-llama/build
Change Dir: '/build/go-llama/build'
Run Build Command(s): /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E env VERBOSE=1 /usr/bin/gmake -f Makefile
gmake[2]: Entering directory '/build/go-llama/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -S/build/go-llama/llama.cpp -B/build/go-llama/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-llama/build/CMakeFiles /build/go-llama/build//CMakeFiles/progress.marks
/usr/bin/gmake -f CMakeFiles/Makefile2 all
gmake[3]: Entering directory '/build/go-llama/build'
/usr/bin/gmake -f CMakeFiles/BUILD_INFO.dir/build.make CMakeFiles/BUILD_INFO.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp /build/go-llama/build /build/go-llama/build /build/go-llama/build/CMakeFiles/BUILD_INFO.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f CMakeFiles/BUILD_INFO.dir/build.make CMakeFiles/BUILD_INFO.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
gmake[4]: Nothing to be done for 'CMakeFiles/BUILD_INFO.dir/build'.
gmake[4]: Leaving directory '/build/go-llama/build'
[ 2%] Built target BUILD_INFO
/usr/bin/gmake -f CMakeFiles/ggml.dir/build.make CMakeFiles/ggml.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp /build/go-llama/build /build/go-llama/build /build/go-llama/build/CMakeFiles/ggml.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f CMakeFiles/ggml.dir/build.make CMakeFiles/ggml.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 4%] Building C object CMakeFiles/ggml.dir/ggml.c.o
/usr/bin/cc -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/ggml.dir/ggml.c.o -MF CMakeFiles/ggml.dir/ggml.c.o.d -o CMakeFiles/ggml.dir/ggml.c.o -c /build/go-llama/llama.cpp/ggml.c
[ 6%] Building C object CMakeFiles/ggml.dir/ggml-alloc.c.o
/usr/bin/cc -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/ggml.dir/ggml-alloc.c.o -MF CMakeFiles/ggml.dir/ggml-alloc.c.o.d -o CMakeFiles/ggml.dir/ggml-alloc.c.o -c /build/go-llama/llama.cpp/ggml-alloc.c
[ 8%] Building CUDA object CMakeFiles/ggml.dir/ggml-cuda.cu.o
/usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 --options-file CMakeFiles/ggml.dir/includes_CUDA.rsp -O3 -DNDEBUG -std=c++11 "--generate-code=arch=compute_52,code=[compute_52,sm_52]" "--generate-code=arch=compute_61,code=[compute_61,sm_61]" "--generate-code=arch=compute_70,code=[compute_70,sm_70]" -mf16c -mfma -mavx -mavx2 -Xcompiler -pthread -MD -MT CMakeFiles/ggml.dir/ggml-cuda.cu.o -MF CMakeFiles/ggml.dir/ggml-cuda.cu.o.d -x cu -c /build/go-llama/llama.cpp/ggml-cuda.cu -o CMakeFiles/ggml.dir/ggml-cuda.cu.o
[ 10%] Building C object CMakeFiles/ggml.dir/k_quants.c.o
/usr/bin/cc -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/ggml.dir/k_quants.c.o -MF CMakeFiles/ggml.dir/k_quants.c.o.d -o CMakeFiles/ggml.dir/k_quants.c.o -c /build/go-llama/llama.cpp/k_quants.c
gmake[4]: Leaving directory '/build/go-llama/build'
[ 10%] Built target ggml
/usr/bin/gmake -f CMakeFiles/ggml_static.dir/build.make CMakeFiles/ggml_static.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp /build/go-llama/build /build/go-llama/build /build/go-llama/build/CMakeFiles/ggml_static.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f CMakeFiles/ggml_static.dir/build.make CMakeFiles/ggml_static.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 12%] Linking CUDA static library libggml_static.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/ggml_static.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/ggml_static.dir/link.txt --verbose=1
/usr/bin/ar qc libggml_static.a CMakeFiles/ggml.dir/ggml.c.o "CMakeFiles/ggml.dir/ggml-alloc.c.o" "CMakeFiles/ggml.dir/ggml-cuda.cu.o" CMakeFiles/ggml.dir/k_quants.c.o
/usr/bin/ranlib libggml_static.a
gmake[4]: Leaving directory '/build/go-llama/build'
[ 12%] Built target ggml_static
/usr/bin/gmake -f CMakeFiles/llama.dir/build.make CMakeFiles/llama.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp /build/go-llama/build /build/go-llama/build /build/go-llama/build/CMakeFiles/llama.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f CMakeFiles/llama.dir/build.make CMakeFiles/llama.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 14%] Building CXX object CMakeFiles/llama.dir/llama.cpp.o
/usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -isystem /usr/local/cuda/include -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -pthread -MD -MT CMakeFiles/llama.dir/llama.cpp.o -MF CMakeFiles/llama.dir/llama.cpp.o.d -o CMakeFiles/llama.dir/llama.cpp.o -c /build/go-llama/llama.cpp/llama.cpp
[ 16%] Linking CXX static library libllama.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama.dir/link.txt --verbose=1
/usr/bin/ar qc libllama.a CMakeFiles/llama.dir/llama.cpp.o CMakeFiles/ggml.dir/ggml.c.o "CMakeFiles/ggml.dir/ggml-alloc.c.o" "CMakeFiles/ggml.dir/ggml-cuda.cu.o" CMakeFiles/ggml.dir/k_quants.c.o
/usr/bin/ranlib libllama.a
gmake[4]: Leaving directory '/build/go-llama/build'
[ 16%] Built target llama
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-fns.dir/build.make tests/CMakeFiles/test-quantize-fns.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/tests /build/go-llama/build /build/go-llama/build/tests /build/go-llama/build/tests/CMakeFiles/test-quantize-fns.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-fns.dir/build.make tests/CMakeFiles/test-quantize-fns.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 18%] Building CXX object tests/CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o
cd /build/go-llama/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o -MF CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o.d -o CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o -c /build/go-llama/llama.cpp/tests/test-quantize-fns.cpp
[ 20%] Linking CXX executable ../bin/test-quantize-fns
cd /build/go-llama/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-quantize-fns.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o" -o ../bin/test-quantize-fns -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 20%] Built target test-quantize-fns
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-perf.dir/build.make tests/CMakeFiles/test-quantize-perf.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/tests /build/go-llama/build /build/go-llama/build/tests /build/go-llama/build/tests/CMakeFiles/test-quantize-perf.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f tests/CMakeFiles/test-quantize-perf.dir/build.make tests/CMakeFiles/test-quantize-perf.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 22%] Building CXX object tests/CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o
cd /build/go-llama/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o -MF CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o.d -o CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o -c /build/go-llama/llama.cpp/tests/test-quantize-perf.cpp
[ 24%] Linking CXX executable ../bin/test-quantize-perf
cd /build/go-llama/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-quantize-perf.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o" -o ../bin/test-quantize-perf -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 24%] Built target test-quantize-perf
/usr/bin/gmake -f tests/CMakeFiles/test-sampling.dir/build.make tests/CMakeFiles/test-sampling.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/tests /build/go-llama/build /build/go-llama/build/tests /build/go-llama/build/tests/CMakeFiles/test-sampling.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f tests/CMakeFiles/test-sampling.dir/build.make tests/CMakeFiles/test-sampling.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 26%] Building CXX object tests/CMakeFiles/test-sampling.dir/test-sampling.cpp.o
cd /build/go-llama/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-sampling.dir/test-sampling.cpp.o -MF CMakeFiles/test-sampling.dir/test-sampling.cpp.o.d -o CMakeFiles/test-sampling.dir/test-sampling.cpp.o -c /build/go-llama/llama.cpp/tests/test-sampling.cpp
[ 28%] Linking CXX executable ../bin/test-sampling
cd /build/go-llama/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-sampling.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-sampling.dir/test-sampling.cpp.o" -o ../bin/test-sampling -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 28%] Built target test-sampling
/usr/bin/gmake -f tests/CMakeFiles/test-tokenizer-0.dir/build.make tests/CMakeFiles/test-tokenizer-0.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/tests /build/go-llama/build /build/go-llama/build/tests /build/go-llama/build/tests/CMakeFiles/test-tokenizer-0.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f tests/CMakeFiles/test-tokenizer-0.dir/build.make tests/CMakeFiles/test-tokenizer-0.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 30%] Building CXX object tests/CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o
cd /build/go-llama/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o -MF CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o.d -o CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o -c /build/go-llama/llama.cpp/tests/test-tokenizer-0.cpp
/build/go-llama/llama.cpp/tests/test-tokenizer-0.cpp:19:2: warning: extra ‘;’ [-Wpedantic]
19 | };
| ^
[ 32%] Linking CXX executable ../bin/test-tokenizer-0
cd /build/go-llama/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-tokenizer-0.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o" -o ../bin/test-tokenizer-0 -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 32%] Built target test-tokenizer-0
/usr/bin/gmake -f tests/CMakeFiles/test-grad0.dir/build.make tests/CMakeFiles/test-grad0.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/tests /build/go-llama/build /build/go-llama/build/tests /build/go-llama/build/tests/CMakeFiles/test-grad0.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f tests/CMakeFiles/test-grad0.dir/build.make tests/CMakeFiles/test-grad0.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 34%] Building CXX object tests/CMakeFiles/test-grad0.dir/test-grad0.cpp.o
cd /build/go-llama/build/tests && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT tests/CMakeFiles/test-grad0.dir/test-grad0.cpp.o -MF CMakeFiles/test-grad0.dir/test-grad0.cpp.o.d -o CMakeFiles/test-grad0.dir/test-grad0.cpp.o -c /build/go-llama/llama.cpp/tests/test-grad0.cpp
[ 36%] Linking CXX executable ../bin/test-grad0
cd /build/go-llama/build/tests && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/test-grad0.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/test-grad0.dir/test-grad0.cpp.o" -o ../bin/test-grad0 -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../libllama.a -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 36%] Built target test-grad0
/usr/bin/gmake -f examples/CMakeFiles/common.dir/build.make examples/CMakeFiles/common.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples /build/go-llama/build /build/go-llama/build/examples /build/go-llama/build/examples/CMakeFiles/common.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/CMakeFiles/common.dir/build.make examples/CMakeFiles/common.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 38%] Building CXX object examples/CMakeFiles/common.dir/common.cpp.o
cd /build/go-llama/build/examples && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/CMakeFiles/common.dir/common.cpp.o -MF CMakeFiles/common.dir/common.cpp.o.d -o CMakeFiles/common.dir/common.cpp.o -c /build/go-llama/llama.cpp/examples/common.cpp
[ 40%] Building CXX object examples/CMakeFiles/common.dir/grammar-parser.cpp.o
cd /build/go-llama/build/examples && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/CMakeFiles/common.dir/grammar-parser.cpp.o -MF CMakeFiles/common.dir/grammar-parser.cpp.o.d -o CMakeFiles/common.dir/grammar-parser.cpp.o -c /build/go-llama/llama.cpp/examples/grammar-parser.cpp
gmake[4]: Leaving directory '/build/go-llama/build'
[ 40%] Built target common
/usr/bin/gmake -f examples/main/CMakeFiles/main.dir/build.make examples/main/CMakeFiles/main.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/main /build/go-llama/build /build/go-llama/build/examples/main /build/go-llama/build/examples/main/CMakeFiles/main.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/main/CMakeFiles/main.dir/build.make examples/main/CMakeFiles/main.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 42%] Building CXX object examples/main/CMakeFiles/main.dir/main.cpp.o
cd /build/go-llama/build/examples/main && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/main/CMakeFiles/main.dir/main.cpp.o -MF CMakeFiles/main.dir/main.cpp.o.d -o CMakeFiles/main.dir/main.cpp.o -c /build/go-llama/llama.cpp/examples/main/main.cpp
[ 44%] Linking CXX executable ../../bin/main
cd /build/go-llama/build/examples/main && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/main.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/main.dir/main.cpp.o ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/main -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 44%] Built target main
/usr/bin/gmake -f examples/quantize/CMakeFiles/quantize.dir/build.make examples/quantize/CMakeFiles/quantize.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/quantize /build/go-llama/build /build/go-llama/build/examples/quantize /build/go-llama/build/examples/quantize/CMakeFiles/quantize.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/quantize/CMakeFiles/quantize.dir/build.make examples/quantize/CMakeFiles/quantize.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 46%] Building CXX object examples/quantize/CMakeFiles/quantize.dir/quantize.cpp.o
cd /build/go-llama/build/examples/quantize && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/quantize/CMakeFiles/quantize.dir/quantize.cpp.o -MF CMakeFiles/quantize.dir/quantize.cpp.o.d -o CMakeFiles/quantize.dir/quantize.cpp.o -c /build/go-llama/llama.cpp/examples/quantize/quantize.cpp
[ 48%] Linking CXX executable ../../bin/quantize
cd /build/go-llama/build/examples/quantize && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/quantize.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/quantize.dir/quantize.cpp.o -o ../../bin/quantize -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 48%] Built target quantize
/usr/bin/gmake -f examples/quantize-stats/CMakeFiles/quantize-stats.dir/build.make examples/quantize-stats/CMakeFiles/quantize-stats.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/quantize-stats /build/go-llama/build /build/go-llama/build/examples/quantize-stats /build/go-llama/build/examples/quantize-stats/CMakeFiles/quantize-stats.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/quantize-stats/CMakeFiles/quantize-stats.dir/build.make examples/quantize-stats/CMakeFiles/quantize-stats.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 50%] Building CXX object examples/quantize-stats/CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o
cd /build/go-llama/build/examples/quantize-stats && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/quantize-stats/CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o -MF CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o.d -o CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o -c /build/go-llama/llama.cpp/examples/quantize-stats/quantize-stats.cpp
[ 52%] Linking CXX executable ../../bin/quantize-stats
cd /build/go-llama/build/examples/quantize-stats && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/quantize-stats.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o" -o ../../bin/quantize-stats -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 52%] Built target quantize-stats
/usr/bin/gmake -f examples/perplexity/CMakeFiles/perplexity.dir/build.make examples/perplexity/CMakeFiles/perplexity.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/perplexity /build/go-llama/build /build/go-llama/build/examples/perplexity /build/go-llama/build/examples/perplexity/CMakeFiles/perplexity.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/perplexity/CMakeFiles/perplexity.dir/build.make examples/perplexity/CMakeFiles/perplexity.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 54%] Building CXX object examples/perplexity/CMakeFiles/perplexity.dir/perplexity.cpp.o
cd /build/go-llama/build/examples/perplexity && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/perplexity/CMakeFiles/perplexity.dir/perplexity.cpp.o -MF CMakeFiles/perplexity.dir/perplexity.cpp.o.d -o CMakeFiles/perplexity.dir/perplexity.cpp.o -c /build/go-llama/llama.cpp/examples/perplexity/perplexity.cpp
[ 56%] Linking CXX executable ../../bin/perplexity
cd /build/go-llama/build/examples/perplexity && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/perplexity.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/perplexity.dir/perplexity.cpp.o ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/perplexity -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 56%] Built target perplexity
/usr/bin/gmake -f examples/embedding/CMakeFiles/embedding.dir/build.make examples/embedding/CMakeFiles/embedding.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/embedding /build/go-llama/build /build/go-llama/build/examples/embedding /build/go-llama/build/examples/embedding/CMakeFiles/embedding.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/embedding/CMakeFiles/embedding.dir/build.make examples/embedding/CMakeFiles/embedding.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 58%] Building CXX object examples/embedding/CMakeFiles/embedding.dir/embedding.cpp.o
cd /build/go-llama/build/examples/embedding && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/embedding/CMakeFiles/embedding.dir/embedding.cpp.o -MF CMakeFiles/embedding.dir/embedding.cpp.o.d -o CMakeFiles/embedding.dir/embedding.cpp.o -c /build/go-llama/llama.cpp/examples/embedding/embedding.cpp
[ 60%] Linking CXX executable ../../bin/embedding
cd /build/go-llama/build/examples/embedding && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/embedding.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/embedding.dir/embedding.cpp.o ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/embedding -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 60%] Built target embedding
/usr/bin/gmake -f examples/save-load-state/CMakeFiles/save-load-state.dir/build.make examples/save-load-state/CMakeFiles/save-load-state.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/save-load-state /build/go-llama/build /build/go-llama/build/examples/save-load-state /build/go-llama/build/examples/save-load-state/CMakeFiles/save-load-state.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/save-load-state/CMakeFiles/save-load-state.dir/build.make examples/save-load-state/CMakeFiles/save-load-state.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 62%] Building CXX object examples/save-load-state/CMakeFiles/save-load-state.dir/save-load-state.cpp.o
cd /build/go-llama/build/examples/save-load-state && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/save-load-state/CMakeFiles/save-load-state.dir/save-load-state.cpp.o -MF CMakeFiles/save-load-state.dir/save-load-state.cpp.o.d -o CMakeFiles/save-load-state.dir/save-load-state.cpp.o -c /build/go-llama/llama.cpp/examples/save-load-state/save-load-state.cpp
[ 64%] Linking CXX executable ../../bin/save-load-state
cd /build/go-llama/build/examples/save-load-state && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/save-load-state.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/save-load-state.dir/save-load-state.cpp.o" ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/save-load-state -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 64%] Built target save-load-state
/usr/bin/gmake -f examples/benchmark/CMakeFiles/benchmark.dir/build.make examples/benchmark/CMakeFiles/benchmark.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/benchmark /build/go-llama/build /build/go-llama/build/examples/benchmark /build/go-llama/build/examples/benchmark/CMakeFiles/benchmark.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/benchmark/CMakeFiles/benchmark.dir/build.make examples/benchmark/CMakeFiles/benchmark.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 66%] Building CXX object examples/benchmark/CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o
cd /build/go-llama/build/examples/benchmark && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/benchmark/CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o -MF CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o.d -o CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o -c /build/go-llama/llama.cpp/examples/benchmark/benchmark-matmult.cpp
[ 68%] Linking CXX executable ../../bin/benchmark
cd /build/go-llama/build/examples/benchmark && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/benchmark.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o" ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/benchmark -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 68%] Built target benchmark
/usr/bin/gmake -f examples/baby-llama/CMakeFiles/baby-llama.dir/build.make examples/baby-llama/CMakeFiles/baby-llama.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/baby-llama /build/go-llama/build /build/go-llama/build/examples/baby-llama /build/go-llama/build/examples/baby-llama/CMakeFiles/baby-llama.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/baby-llama/CMakeFiles/baby-llama.dir/build.make examples/baby-llama/CMakeFiles/baby-llama.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 70%] Building CXX object examples/baby-llama/CMakeFiles/baby-llama.dir/baby-llama.cpp.o
cd /build/go-llama/build/examples/baby-llama && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/baby-llama/CMakeFiles/baby-llama.dir/baby-llama.cpp.o -MF CMakeFiles/baby-llama.dir/baby-llama.cpp.o.d -o CMakeFiles/baby-llama.dir/baby-llama.cpp.o -c /build/go-llama/llama.cpp/examples/baby-llama/baby-llama.cpp
/build/go-llama/llama.cpp/examples/baby-llama/baby-llama.cpp: In function ‘int main(int, char**)’:
/build/go-llama/llama.cpp/examples/baby-llama/baby-llama.cpp:1620:32: warning: variable ‘opt_params_adam’ set but not used [-Wunused-but-set-variable]
1620 | struct ggml_opt_params opt_params_adam = ggml_opt_default_params(GGML_OPT_ADAM);
| ^~~~~~~~~~~~~~~
[ 72%] Linking CXX executable ../../bin/baby-llama
cd /build/go-llama/build/examples/baby-llama && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/baby-llama.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/baby-llama.dir/baby-llama.cpp.o" ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/baby-llama -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 72%] Built target baby-llama
/usr/bin/gmake -f examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/build.make examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/train-text-from-scratch /build/go-llama/build /build/go-llama/build/examples/train-text-from-scratch /build/go-llama/build/examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/build.make examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 74%] Building CXX object examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o
cd /build/go-llama/build/examples/train-text-from-scratch && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o -MF CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o.d -o CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o -c /build/go-llama/llama.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp
[ 76%] Linking CXX executable ../../bin/train-text-from-scratch
cd /build/go-llama/build/examples/train-text-from-scratch && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/train-text-from-scratch.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o" ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/train-text-from-scratch -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 76%] Built target train-text-from-scratch
/usr/bin/gmake -f examples/simple/CMakeFiles/simple.dir/build.make examples/simple/CMakeFiles/simple.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/simple /build/go-llama/build /build/go-llama/build/examples/simple /build/go-llama/build/examples/simple/CMakeFiles/simple.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/simple/CMakeFiles/simple.dir/build.make examples/simple/CMakeFiles/simple.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 78%] Building CXX object examples/simple/CMakeFiles/simple.dir/simple.cpp.o
cd /build/go-llama/build/examples/simple && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/simple/CMakeFiles/simple.dir/simple.cpp.o -MF CMakeFiles/simple.dir/simple.cpp.o.d -o CMakeFiles/simple.dir/simple.cpp.o -c /build/go-llama/llama.cpp/examples/simple/simple.cpp
[ 80%] Linking CXX executable ../../bin/simple
cd /build/go-llama/build/examples/simple && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/simple.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/simple.dir/simple.cpp.o ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/simple -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 80%] Built target simple
/usr/bin/gmake -f examples/embd-input/CMakeFiles/embdinput.dir/build.make examples/embd-input/CMakeFiles/embdinput.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/embd-input /build/go-llama/build /build/go-llama/build/examples/embd-input /build/go-llama/build/examples/embd-input/CMakeFiles/embdinput.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/embd-input/CMakeFiles/embdinput.dir/build.make examples/embd-input/CMakeFiles/embdinput.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 82%] Building CXX object examples/embd-input/CMakeFiles/embdinput.dir/embd-input-lib.cpp.o
cd /build/go-llama/build/examples/embd-input && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/embd-input/CMakeFiles/embdinput.dir/embd-input-lib.cpp.o -MF CMakeFiles/embdinput.dir/embd-input-lib.cpp.o.d -o CMakeFiles/embdinput.dir/embd-input-lib.cpp.o -c /build/go-llama/llama.cpp/examples/embd-input/embd-input-lib.cpp
[ 84%] Linking CXX static library libembdinput.a
cd /build/go-llama/build/examples/embd-input && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/embdinput.dir/cmake_clean_target.cmake
cd /build/go-llama/build/examples/embd-input && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/embdinput.dir/link.txt --verbose=1
/usr/bin/ar qc libembdinput.a "CMakeFiles/embdinput.dir/embd-input-lib.cpp.o" ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o"
/usr/bin/ranlib libembdinput.a
gmake[4]: Leaving directory '/build/go-llama/build'
[ 84%] Built target embdinput
/usr/bin/gmake -f examples/embd-input/CMakeFiles/embd-input-test.dir/build.make examples/embd-input/CMakeFiles/embd-input-test.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/embd-input /build/go-llama/build /build/go-llama/build/examples/embd-input /build/go-llama/build/examples/embd-input/CMakeFiles/embd-input-test.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/embd-input/CMakeFiles/embd-input-test.dir/build.make examples/embd-input/CMakeFiles/embd-input-test.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 86%] Building CXX object examples/embd-input/CMakeFiles/embd-input-test.dir/embd-input-test.cpp.o
cd /build/go-llama/build/examples/embd-input && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/embd-input/CMakeFiles/embd-input-test.dir/embd-input-test.cpp.o -MF CMakeFiles/embd-input-test.dir/embd-input-test.cpp.o.d -o CMakeFiles/embd-input-test.dir/embd-input-test.cpp.o -c /build/go-llama/llama.cpp/examples/embd-input/embd-input-test.cpp
[ 88%] Linking CXX executable ../../bin/embd-input-test
cd /build/go-llama/build/examples/embd-input && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/embd-input-test.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG "CMakeFiles/embd-input-test.dir/embd-input-test.cpp.o" ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/embd-input-test -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a libembdinput.a -pthread ../../libllama.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -pthread -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 88%] Built target embd-input-test
/usr/bin/gmake -f examples/server/CMakeFiles/server.dir/build.make examples/server/CMakeFiles/server.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/examples/server /build/go-llama/build /build/go-llama/build/examples/server /build/go-llama/build/examples/server/CMakeFiles/server.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f examples/server/CMakeFiles/server.dir/build.make examples/server/CMakeFiles/server.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 90%] Building CXX object examples/server/CMakeFiles/server.dir/server.cpp.o
cd /build/go-llama/build/examples/server && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -DSERVER_VERBOSE=1 -I/build/go-llama/llama.cpp/examples -I/build/go-llama/llama.cpp/examples/server -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT examples/server/CMakeFiles/server.dir/server.cpp.o -MF CMakeFiles/server.dir/server.cpp.o.d -o CMakeFiles/server.dir/server.cpp.o -c /build/go-llama/llama.cpp/examples/server/server.cpp
[ 92%] Linking CXX executable ../../bin/server
cd /build/go-llama/build/examples/server && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/server.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/server.dir/server.cpp.o ../CMakeFiles/common.dir/common.cpp.o "../CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/server -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib: ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 92%] Built target server
/usr/bin/gmake -f pocs/vdot/CMakeFiles/vdot.dir/build.make pocs/vdot/CMakeFiles/vdot.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/pocs/vdot /build/go-llama/build /build/go-llama/build/pocs/vdot /build/go-llama/build/pocs/vdot/CMakeFiles/vdot.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f pocs/vdot/CMakeFiles/vdot.dir/build.make pocs/vdot/CMakeFiles/vdot.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 94%] Building CXX object pocs/vdot/CMakeFiles/vdot.dir/vdot.cpp.o
cd /build/go-llama/build/pocs/vdot && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/pocs -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT pocs/vdot/CMakeFiles/vdot.dir/vdot.cpp.o -MF CMakeFiles/vdot.dir/vdot.cpp.o.d -o CMakeFiles/vdot.dir/vdot.cpp.o -c /build/go-llama/llama.cpp/pocs/vdot/vdot.cpp
[ 96%] Linking CXX executable ../../bin/vdot
cd /build/go-llama/build/pocs/vdot && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/vdot.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/vdot.dir/vdot.cpp.o ../../examples/CMakeFiles/common.dir/common.cpp.o "../../examples/CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/vdot -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[ 96%] Built target vdot
/usr/bin/gmake -f pocs/vdot/CMakeFiles/q8dot.dir/build.make pocs/vdot/CMakeFiles/q8dot.dir/depend
gmake[4]: Entering directory '/build/go-llama/build'
cd /build/go-llama/build && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/go-llama/llama.cpp /build/go-llama/llama.cpp/pocs/vdot /build/go-llama/build /build/go-llama/build/pocs/vdot /build/go-llama/build/pocs/vdot/CMakeFiles/q8dot.dir/DependInfo.cmake "--color="
gmake[4]: Leaving directory '/build/go-llama/build'
/usr/bin/gmake -f pocs/vdot/CMakeFiles/q8dot.dir/build.make pocs/vdot/CMakeFiles/q8dot.dir/build
gmake[4]: Entering directory '/build/go-llama/build'
[ 98%] Building CXX object pocs/vdot/CMakeFiles/q8dot.dir/q8dot.cpp.o
cd /build/go-llama/build/pocs/vdot && /usr/bin/c++ -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMQ_Y=64 -DGGML_CUDA_MMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -I/build/go-llama/llama.cpp/pocs -I/build/go-llama/llama.cpp/examples/. -I/build/go-llama/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -mf16c -mfma -mavx -mavx2 -MD -MT pocs/vdot/CMakeFiles/q8dot.dir/q8dot.cpp.o -MF CMakeFiles/q8dot.dir/q8dot.cpp.o.d -o CMakeFiles/q8dot.dir/q8dot.cpp.o -c /build/go-llama/llama.cpp/pocs/vdot/q8dot.cpp
[100%] Linking CXX executable ../../bin/q8dot
cd /build/go-llama/build/pocs/vdot && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/q8dot.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/q8dot.dir/q8dot.cpp.o ../../examples/CMakeFiles/common.dir/common.cpp.o "../../examples/CMakeFiles/common.dir/grammar-parser.cpp.o" -o ../../bin/q8dot -L/usr/local/cuda/targets/x86_64-linux/lib/stubs -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/usr/local/cuda-12.1/targets/x86_64-linux/lib ../../libllama.a -pthread -pthread /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so /usr/local/cuda-12.1/targets/x86_64-linux/lib/libculibos.a /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[4]: Leaving directory '/build/go-llama/build'
[100%] Built target q8dot
gmake[3]: Leaving directory '/build/go-llama/build'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/go-llama/build/CMakeFiles 0
gmake[2]: Leaving directory '/build/go-llama/build'
cd build && cp -rf CMakeFiles/llama.dir/llama.cpp.o ../llama.cpp/llama.o
cd build && cp -rf examples/CMakeFiles/common.dir/common.cpp.o ../llama.cpp/common.o
cd build && cp -rf examples/CMakeFiles/common.dir/grammar-parser.cpp.o ../llama.cpp/grammar-parser.o
cd build && cp -rf CMakeFiles/ggml.dir/ggml-alloc.c.o ../llama.cpp/ggml-alloc.o
g++ -I./llama.cpp -I. -I./llama.cpp/examples -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread -I./llama.cpp -I./llama.cpp/examples binding.cpp -o binding.o -c
binding.cpp: In function 'int llama_predict(void*, void*, char*, bool)':
binding.cpp:533:42: warning: cast from type 'const char*' to type 'char*' casts away qualifiers [-Wcast-qual]
533 | if (!tokenCallback(state_pr, (char*)token_str)) {
| ^~~~~~~~~~~~~~~~
binding.cpp:591:1: warning: label 'end' defined but not used [-Wunused-label]
591 | end:
| ^~~
binding.cpp: In function 'void llama_binding_free_model(void*)':
binding.cpp:613:5: warning: possible problem detected in invocation of 'operator delete' [-Wdelete-incomplete]
613 | delete ctx->model;
| ^~~~~~~~~~~~~~~~~
binding.cpp:613:17: warning: invalid use of incomplete type 'struct llama_model'
613 | delete ctx->model;
| ~~~~~^~~~~
In file included from ./llama.cpp/examples/common.h:5,
from binding.cpp:1:
./llama.cpp/llama.h:70:12: note: forward declaration of 'struct llama_model'
70 | struct llama_model;
| ^~~~~~~~~~~
binding.cpp:613:5: note: neither the destructor nor the class-specific 'operator delete' will be called, even if they are declared when the class is defined
613 | delete ctx->model;
| ^~~~~~~~~~~~~~~~~
cd build && cp -rf CMakeFiles/ggml.dir/k_quants.c.o ../llama.cpp/k_quants.o
cd build && cp -rf CMakeFiles/ggml.dir/ggml-cuda.cu.o ../llama.cpp/ggml-cuda.o
ar src libbinding.a llama.cpp/ggml.o llama.cpp/k_quants.o llama.cpp/ggml-cuda.o llama.cpp/ggml-alloc.o llama.cpp/common.o llama.cpp/grammar-parser.o llama.cpp/llama.o binding.o
make[1]: Leaving directory '/build/go-llama'
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-llama LIBRARY_PATH=/build/go-llama \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/llama ./cmd/grpc/llama/
# github.com/go-skynet/go-llama.cpp
binding.cpp: In function 'void llama_binding_free_model(void*)':
binding.cpp:613:5: warning: possible problem detected in invocation of 'operator delete' [-Wdelete-incomplete]
613 | delete ctx->model;
| ^~~~~~~~~~~~~~~~~
binding.cpp:613:17: warning: invalid use of incomplete type 'struct llama_model'
613 | delete ctx->model;
| ~~~~~^~~~~
In file included from go-llama/llama.cpp/examples/common.h:5,
from binding.cpp:1:
go-llama/llama.cpp/llama.h:70:12: note: forward declaration of 'struct llama_model'
70 | struct llama_model;
| ^~~~~~~~~~~
binding.cpp:613:5: note: neither the destructor nor the class-specific 'operator delete' will be called, even if they are declared when the class is defined
613 | delete ctx->model;
| ^~~~~~~~~~~~~~~~~
make -C gpt4all/gpt4all-bindings/golang/ libgpt4all.a
make[1]: Entering directory '/build/gpt4all/gpt4all-bindings/golang'
I go-gpt4all build info:
I UNAME_S: Linux
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I. -I../../gpt4all-backend/llama.cpp -I../../gpt4all-backend -I -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I. -I../../gpt4all-backend/llama.cpp -I../../gpt4all-backend -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native
I LDFLAGS:
I CMAKEFLAGS:
I CC: cc (Debian 10.2.1-6) 10.2.1 20210110
I CXX: g++ (Debian 10.2.1-6) 10.2.1 20210110
g++ -I. -I../../gpt4all-backend/llama.cpp -I../../gpt4all-backend -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native binding.cpp -o binding.o -c
binding.cpp: In lambda function:
binding.cpp:46:33: warning: unused parameter 'token_id' [-Wunused-parameter]
46 | auto lambda_prompt = [](int token_id) {
| ~~~~^~~~~~~~
binding.cpp: In lambda function:
binding.cpp:54:20: warning: cast from type 'const char*' to type 'char*' casts away qualifiers [-Wcast-qual]
54 | res.append((char*)responsechars);
| ^~~~~~~~~~~~~~~~~~~~
binding.cpp:55:39: warning: cast from type 'const char*' to type 'char*' casts away qualifiers [-Wcast-qual]
55 | return !!getTokenCallback(mm, (char*)responsechars);
| ^~~~~~~~~~~~~~~~~~~~
binding.cpp:53:35: warning: unused parameter 'token_id' [-Wunused-parameter]
53 | auto lambda_response = [](int token_id, const char *responsechars) {
| ~~~~^~~~~~~~
binding.cpp: In function 'void model_prompt(const char*, void*, char*, int, float, int, int, int, float, float, int, float)':
binding.cpp:64:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
64 | .logits = NULL,
| ^
binding.cpp:65:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
65 | .logits_size = 0,
| ^
binding.cpp:66:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
66 | .tokens = NULL,
| ^
binding.cpp:67:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
67 | .tokens_size = 0,
| ^
binding.cpp:68:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
68 | .n_past = 0,
| ^
binding.cpp:69:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
69 | .n_ctx = 1024,
| ^
binding.cpp:70:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
70 | .n_predict = 50,
| ^
binding.cpp:71:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
71 | .top_k = 10,
| ^
binding.cpp:72:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
72 | .top_p = 0.9,
| ^
binding.cpp:73:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
73 | .temp = 1.0,
| ^
binding.cpp:74:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
74 | .n_batch = 1,
| ^
binding.cpp:75:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
75 | .repeat_penalty = 1.2,
| ^
binding.cpp:76:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
76 | .repeat_last_n = 10,
| ^
binding.cpp:77:9: warning: C++ designated initializers only available with '-std=c++2a' or '-std=gnu++2a' [-Wpedantic]
77 | .context_erase = 0.5
| ^
mkdir buildllm
cd buildllm && cmake ../../../gpt4all-backend/ && make
-- The CXX compiler identification is GNU 10.2.1
-- The C compiler identification is GNU 10.2.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Interprocedural optimization support detected
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Configuring ggml implementation target llama-mainline-default in /build/gpt4all/gpt4all-backend/llama.cpp-mainline
-- x86 detected
-- Configuring ggml implementation target llama-230511-default in /build/gpt4all/gpt4all-backend/llama.cpp-230511
-- x86 detected
-- Configuring ggml implementation target llama-230519-default in /build/gpt4all/gpt4all-backend/llama.cpp-230519
-- x86 detected
-- Configuring model implementation target llamamodel-mainline-default
-- Configuring model implementation target replit-mainline-default
-- Configuring model implementation target llamamodel-230519-default
-- Configuring model implementation target llamamodel-230511-default
-- Configuring model implementation target gptj-default
-- Configuring model implementation target falcon-default
-- Configuring model implementation target mpt-default
-- Configuring model implementation target bert-default
-- Configuring model implementation target starcoder-default
-- Configuring ggml implementation target llama-mainline-avxonly in /build/gpt4all/gpt4all-backend/llama.cpp-mainline
-- x86 detected
-- Configuring ggml implementation target llama-230511-avxonly in /build/gpt4all/gpt4all-backend/llama.cpp-230511
-- x86 detected
-- Configuring ggml implementation target llama-230519-avxonly in /build/gpt4all/gpt4all-backend/llama.cpp-230519
-- x86 detected
-- Configuring model implementation target llamamodel-mainline-avxonly
-- Configuring model implementation target replit-mainline-avxonly
-- Configuring model implementation target llamamodel-230519-avxonly
-- Configuring model implementation target llamamodel-230511-avxonly
-- Configuring model implementation target gptj-avxonly
-- Configuring model implementation target falcon-avxonly
-- Configuring model implementation target mpt-avxonly
-- Configuring model implementation target bert-avxonly
-- Configuring model implementation target starcoder-avxonly
-- Configuring done (0.7s)
-- Generating done (0.0s)
-- Build files have been written to: /build/gpt4all/gpt4all-bindings/golang/buildllm
make[2]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -S/build/gpt4all/gpt4all-backend -B/build/gpt4all/gpt4all-bindings/golang/buildllm --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles /build/gpt4all/gpt4all-bindings/golang/buildllm//CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[3]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/ggml-mainline-default.dir/build.make CMakeFiles/ggml-mainline-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/ggml-mainline-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/ggml-mainline-default.dir/build.make CMakeFiles/ggml-mainline-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 1%] Building C object CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/ggml.c.o
/usr/bin/cc -DGGML_USE_K_QUANTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -mf16c -mfma -mavx2 -pthread -MD -MT CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/ggml.c.o -MF CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/ggml.c.o.d -o CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/ggml.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-mainline/ggml.c
[ 2%] Building C object CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/k_quants.c.o
/usr/bin/cc -DGGML_USE_K_QUANTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -mf16c -mfma -mavx2 -pthread -MD -MT CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/k_quants.c.o -MF CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/k_quants.c.o.d -o CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/k_quants.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-mainline/k_quants.c
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 2%] Built target ggml-mainline-default
make -f CMakeFiles/llama-mainline-default.dir/build.make CMakeFiles/llama-mainline-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llama-mainline-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llama-mainline-default.dir/build.make CMakeFiles/llama-mainline-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 3%] Building CXX object CMakeFiles/llama-mainline-default.dir/llama.cpp-mainline/llama.cpp.o
/usr/bin/c++ -DGGML_USE_K_QUANTS -DLLAMA_BUILD -DLLAMA_SHARED -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/llama-mainline-default.dir/llama.cpp-mainline/llama.cpp.o -MF CMakeFiles/llama-mainline-default.dir/llama.cpp-mainline/llama.cpp.o.d -o CMakeFiles/llama-mainline-default.dir/llama.cpp-mainline/llama.cpp.o -c /build/gpt4all/gpt4all-backend/llama.cpp-mainline/llama.cpp
[ 4%] Linking CXX static library libllama-mainline-default.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama-mainline-default.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama-mainline-default.dir/link.txt --verbose=1
/usr/bin/ar qc libllama-mainline-default.a "CMakeFiles/llama-mainline-default.dir/llama.cpp-mainline/llama.cpp.o" "CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/ggml.c.o" "CMakeFiles/ggml-mainline-default.dir/llama.cpp-mainline/k_quants.c.o"
/usr/bin/ranlib libllama-mainline-default.a
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 4%] Built target llama-mainline-default
make -f CMakeFiles/ggml-230511-default.dir/build.make CMakeFiles/ggml-230511-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/ggml-230511-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/ggml-230511-default.dir/build.make CMakeFiles/ggml-230511-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 5%] Building C object CMakeFiles/ggml-230511-default.dir/llama.cpp-230511/ggml.c.o
/usr/bin/cc -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -mf16c -mfma -mavx2 -pthread -MD -MT CMakeFiles/ggml-230511-default.dir/llama.cpp-230511/ggml.c.o -MF CMakeFiles/ggml-230511-default.dir/llama.cpp-230511/ggml.c.o.d -o CMakeFiles/ggml-230511-default.dir/llama.cpp-230511/ggml.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'ggml_compute_forward_alibi_f32':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:9357:15: warning: unused variable 'ne2_ne3' [-Wunused-variable]
9357 | const int ne2_ne3 = n/ne1; // ne2*ne3
| ^~~~~~~
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'ggml_compute_forward_alibi_f16':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:9419:15: warning: unused variable 'ne2' [-Wunused-variable]
9419 | const int ne2 = src0->ne[2]; // n_head -> this is k
| ^~~
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'ggml_compute_forward_alibi':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:9468:5: warning: enumeration value 'GGML_TYPE_Q4_3' not handled in switch [-Wswitch]
9468 | switch (src0->type) {
| ^~~~~~
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 5%] Built target ggml-230511-default
make -f CMakeFiles/llama-230511-default.dir/build.make CMakeFiles/llama-230511-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llama-230511-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llama-230511-default.dir/build.make CMakeFiles/llama-230511-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 6%] Building CXX object CMakeFiles/llama-230511-default.dir/llama.cpp-230511/llama.cpp.o
/usr/bin/c++ -DLLAMA_BUILD -DLLAMA_SHARED -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/llama-230511-default.dir/llama.cpp-230511/llama.cpp.o -MF CMakeFiles/llama-230511-default.dir/llama.cpp-230511/llama.cpp.o.d -o CMakeFiles/llama-230511-default.dir/llama.cpp-230511/llama.cpp.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230511/llama.cpp
[ 7%] Linking CXX static library libllama-230511-default.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama-230511-default.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama-230511-default.dir/link.txt --verbose=1
/usr/bin/ar qc libllama-230511-default.a "CMakeFiles/llama-230511-default.dir/llama.cpp-230511/llama.cpp.o" "CMakeFiles/ggml-230511-default.dir/llama.cpp-230511/ggml.c.o"
/usr/bin/ranlib libllama-230511-default.a
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 7%] Built target llama-230511-default
make -f CMakeFiles/ggml-230519-default.dir/build.make CMakeFiles/ggml-230519-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/ggml-230519-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/ggml-230519-default.dir/build.make CMakeFiles/ggml-230519-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 8%] Building C object CMakeFiles/ggml-230519-default.dir/llama.cpp-230519/ggml.c.o
/usr/bin/cc -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -mf16c -mfma -mavx2 -pthread -MD -MT CMakeFiles/ggml-230519-default.dir/llama.cpp-230519/ggml.c.o -MF CMakeFiles/ggml-230519-default.dir/llama.cpp-230519/ggml.c.o.d -o CMakeFiles/ggml-230519-default.dir/llama.cpp-230519/ggml.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230519/ggml.c
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 8%] Built target ggml-230519-default
make -f CMakeFiles/llama-230519-default.dir/build.make CMakeFiles/llama-230519-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llama-230519-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llama-230519-default.dir/build.make CMakeFiles/llama-230519-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 10%] Building CXX object CMakeFiles/llama-230519-default.dir/llama.cpp-230519/llama.cpp.o
/usr/bin/c++ -DLLAMA_BUILD -DLLAMA_SHARED -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/llama-230519-default.dir/llama.cpp-230519/llama.cpp.o -MF CMakeFiles/llama-230519-default.dir/llama.cpp-230519/llama.cpp.o.d -o CMakeFiles/llama-230519-default.dir/llama.cpp-230519/llama.cpp.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp
/build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp: In function 'size_t llama_set_state_data(llama_context*, const uint8_t*)':
/build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp:2685:27: warning: cast from type 'const uint8_t*' {aka 'const unsigned char*'} to type 'void*' casts away qualifiers [-Wcast-qual]
2685 | kin3d->data = (void *) inp;
| ^~~~~~~~~~~~
/build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp:2689:27: warning: cast from type 'const uint8_t*' {aka 'const unsigned char*'} to type 'void*' casts away qualifiers [-Wcast-qual]
2689 | vin3d->data = (void *) inp;
| ^~~~~~~~~~~~
[ 11%] Linking CXX static library libllama-230519-default.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama-230519-default.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama-230519-default.dir/link.txt --verbose=1
/usr/bin/ar qc libllama-230519-default.a "CMakeFiles/llama-230519-default.dir/llama.cpp-230519/llama.cpp.o" "CMakeFiles/ggml-230519-default.dir/llama.cpp-230519/ggml.c.o"
/usr/bin/ranlib libllama-230519-default.a
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 11%] Built target llama-230519-default
make -f CMakeFiles/llamamodel-mainline-default.dir/build.make CMakeFiles/llamamodel-mainline-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llamamodel-mainline-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llamamodel-mainline-default.dir/build.make CMakeFiles/llamamodel-mainline-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 12%] Building CXX object CMakeFiles/llamamodel-mainline-default.dir/llamamodel.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -DLLAMA_DATE=999999 -DLLAMA_VERSIONS=">=3" -Dllamamodel_mainline_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-mainline-default.dir/llamamodel.cpp.o -MF CMakeFiles/llamamodel-mainline-default.dir/llamamodel.cpp.o.d -o CMakeFiles/llamamodel-mainline-default.dir/llamamodel.cpp.o -c /build/gpt4all/gpt4all-backend/llamamodel.cpp
/build/gpt4all/gpt4all-backend/llamamodel.cpp: In member function 'virtual bool LLamaModel::loadModel(const string&)':
/build/gpt4all/gpt4all-backend/llamamodel.cpp:159:71: warning: 'llama_context* llama_init_from_file(const char*, llama_context_params)' is deprecated: please use llama_load_model_from_file combined with llama_new_context_with_model instead [-Wdeprecated-declarations]
159 | d_ptr->ctx = llama_init_from_file(modelPath.c_str(), d_ptr->params);
| ^
In file included from /build/gpt4all/gpt4all-backend/llamamodel.cpp:28:
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/llama.h:161:49: note: declared here
161 | LLAMA_API DEPRECATED(struct llama_context * llama_init_from_file(
| ^~~~~~~~~~~~~~~~~~~~
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/llama.h:30:36: note: in definition of macro 'DEPRECATED'
30 | # define DEPRECATED(func, hint) func __attribute__((deprecated(hint)))
| ^~~~
[ 13%] Building CXX object CMakeFiles/llamamodel-mainline-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -DLLAMA_DATE=999999 -DLLAMA_VERSIONS=">=3" -Dllamamodel_mainline_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-mainline-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/llamamodel-mainline-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/llamamodel-mainline-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 14%] Linking CXX shared library libllamamodel-mainline-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llamamodel-mainline-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libllamamodel-mainline-default.so -o libllamamodel-mainline-default.so "CMakeFiles/llamamodel-mainline-default.dir/llamamodel.cpp.o" "CMakeFiles/llamamodel-mainline-default.dir/llmodel_shared.cpp.o" libllama-mainline-default.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 14%] Built target llamamodel-mainline-default
make -f CMakeFiles/replit-mainline-default.dir/build.make CMakeFiles/replit-mainline-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/replit-mainline-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/replit-mainline-default.dir/build.make CMakeFiles/replit-mainline-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 15%] Building CXX object CMakeFiles/replit-mainline-default.dir/replit.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dreplit_mainline_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/replit-mainline-default.dir/replit.cpp.o -MF CMakeFiles/replit-mainline-default.dir/replit.cpp.o.d -o CMakeFiles/replit-mainline-default.dir/replit.cpp.o -c /build/gpt4all/gpt4all-backend/replit.cpp
/build/gpt4all/gpt4all-backend/replit.cpp: In function 'bool replit_eval(const replit_model&, int, int, const std::vector<int>&, std::vector<float>&, size_t&)':
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::n_nodes' [-Wmissing-field-initializers]
555 | struct ggml_cgraph gf = {.n_threads = n_threads};
| ^
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::n_leafs' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::work_size' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::work' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::nodes' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::grads' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::leafs' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::perf_runs' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::perf_cycles' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::perf_time_us' [-Wmissing-field-initializers]
[ 16%] Building CXX object CMakeFiles/replit-mainline-default.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dreplit_mainline_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/replit-mainline-default.dir/utils.cpp.o -MF CMakeFiles/replit-mainline-default.dir/utils.cpp.o.d -o CMakeFiles/replit-mainline-default.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 17%] Building CXX object CMakeFiles/replit-mainline-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dreplit_mainline_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/replit-mainline-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/replit-mainline-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/replit-mainline-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 18%] Linking CXX shared library libreplit-mainline-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/replit-mainline-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libreplit-mainline-default.so -o libreplit-mainline-default.so "CMakeFiles/replit-mainline-default.dir/replit.cpp.o" "CMakeFiles/replit-mainline-default.dir/utils.cpp.o" "CMakeFiles/replit-mainline-default.dir/llmodel_shared.cpp.o" libllama-mainline-default.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 18%] Built target replit-mainline-default
make -f CMakeFiles/llamamodel-230519-default.dir/build.make CMakeFiles/llamamodel-230519-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llamamodel-230519-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llamamodel-230519-default.dir/build.make CMakeFiles/llamamodel-230519-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 20%] Building CXX object CMakeFiles/llamamodel-230519-default.dir/llamamodel.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -DLLAMA_DATE=230519 -DLLAMA_VERSIONS===2 -Dllamamodel_230519_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230519-default.dir/llamamodel.cpp.o -MF CMakeFiles/llamamodel-230519-default.dir/llamamodel.cpp.o.d -o CMakeFiles/llamamodel-230519-default.dir/llamamodel.cpp.o -c /build/gpt4all/gpt4all-backend/llamamodel.cpp
[ 21%] Building CXX object CMakeFiles/llamamodel-230519-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -DLLAMA_DATE=230519 -DLLAMA_VERSIONS===2 -Dllamamodel_230519_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230519-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/llamamodel-230519-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/llamamodel-230519-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 22%] Linking CXX shared library libllamamodel-230519-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llamamodel-230519-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libllamamodel-230519-default.so -o libllamamodel-230519-default.so "CMakeFiles/llamamodel-230519-default.dir/llamamodel.cpp.o" "CMakeFiles/llamamodel-230519-default.dir/llmodel_shared.cpp.o" libllama-230519-default.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 22%] Built target llamamodel-230519-default
make -f CMakeFiles/llamamodel-230511-default.dir/build.make CMakeFiles/llamamodel-230511-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llamamodel-230511-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llamamodel-230511-default.dir/build.make CMakeFiles/llamamodel-230511-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 23%] Building CXX object CMakeFiles/llamamodel-230511-default.dir/llamamodel.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -DLLAMA_DATE=230511 -DLLAMA_VERSIONS="<=1" -Dllamamodel_230511_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230511-default.dir/llamamodel.cpp.o -MF CMakeFiles/llamamodel-230511-default.dir/llamamodel.cpp.o.d -o CMakeFiles/llamamodel-230511-default.dir/llamamodel.cpp.o -c /build/gpt4all/gpt4all-backend/llamamodel.cpp
[ 24%] Building CXX object CMakeFiles/llamamodel-230511-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -DLLAMA_DATE=230511 -DLLAMA_VERSIONS="<=1" -Dllamamodel_230511_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230511-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/llamamodel-230511-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/llamamodel-230511-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 25%] Linking CXX shared library libllamamodel-230511-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llamamodel-230511-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libllamamodel-230511-default.so -o libllamamodel-230511-default.so "CMakeFiles/llamamodel-230511-default.dir/llamamodel.cpp.o" "CMakeFiles/llamamodel-230511-default.dir/llmodel_shared.cpp.o" libllama-230511-default.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 25%] Built target llamamodel-230511-default
make -f CMakeFiles/gptj-default.dir/build.make CMakeFiles/gptj-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/gptj-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/gptj-default.dir/build.make CMakeFiles/gptj-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 26%] Building CXX object CMakeFiles/gptj-default.dir/gptj.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dgptj_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/gptj-default.dir/gptj.cpp.o -MF CMakeFiles/gptj-default.dir/gptj.cpp.o.d -o CMakeFiles/gptj-default.dir/gptj.cpp.o -c /build/gpt4all/gpt4all-backend/gptj.cpp
[ 27%] Building CXX object CMakeFiles/gptj-default.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dgptj_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/gptj-default.dir/utils.cpp.o -MF CMakeFiles/gptj-default.dir/utils.cpp.o.d -o CMakeFiles/gptj-default.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 28%] Building CXX object CMakeFiles/gptj-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dgptj_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/gptj-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/gptj-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/gptj-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 30%] Linking CXX shared library libgptj-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/gptj-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libgptj-default.so -o libgptj-default.so "CMakeFiles/gptj-default.dir/gptj.cpp.o" "CMakeFiles/gptj-default.dir/utils.cpp.o" "CMakeFiles/gptj-default.dir/llmodel_shared.cpp.o" "CMakeFiles/ggml-230511-default.dir/llama.cpp-230511/ggml.c.o" -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 30%] Built target gptj-default
make -f CMakeFiles/falcon-default.dir/build.make CMakeFiles/falcon-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/falcon-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/falcon-default.dir/build.make CMakeFiles/falcon-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 31%] Building CXX object CMakeFiles/falcon-default.dir/falcon.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dfalcon_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/falcon-default.dir/falcon.cpp.o -MF CMakeFiles/falcon-default.dir/falcon.cpp.o.d -o CMakeFiles/falcon-default.dir/falcon.cpp.o -c /build/gpt4all/gpt4all-backend/falcon.cpp
/build/gpt4all/gpt4all-backend/falcon.cpp: In function 'bool falcon_model_load(const string&, falcon_model&, gpt_vocab&, size_t*)':
/build/gpt4all/gpt4all-backend/falcon.cpp:199:19: warning: unused variable 'n_ctx' [-Wunused-variable]
199 | const int n_ctx = hparams.n_ctx;
| ^~~~~
/build/gpt4all/gpt4all-backend/falcon.cpp:340:19: warning: unused variable 'n_head_kv' [-Wunused-variable]
340 | const int n_head_kv = hparams.n_head_kv;
| ^~~~~~~~~
/build/gpt4all/gpt4all-backend/falcon.cpp:344:23: warning: unused variable 'n_elements' [-Wunused-variable]
344 | const int64_t n_elements = head_dim*n_mem;
| ^~~~~~~~~~
/build/gpt4all/gpt4all-backend/falcon.cpp: In function 'bool falcon_eval(const falcon_model&, int, int, const std::vector<int>&, std::vector<float>&, size_t&)':
/build/gpt4all/gpt4all-backend/falcon.cpp:465:15: warning: unused variable 'version' [-Wunused-variable]
465 | const int version = hparams.falcon_version;
| ^~~~~~~
[ 32%] Building CXX object CMakeFiles/falcon-default.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dfalcon_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/falcon-default.dir/utils.cpp.o -MF CMakeFiles/falcon-default.dir/utils.cpp.o.d -o CMakeFiles/falcon-default.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 33%] Building CXX object CMakeFiles/falcon-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dfalcon_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/falcon-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/falcon-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/falcon-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 34%] Linking CXX shared library libfalcon-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/falcon-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libfalcon-default.so -o libfalcon-default.so "CMakeFiles/falcon-default.dir/falcon.cpp.o" "CMakeFiles/falcon-default.dir/utils.cpp.o" "CMakeFiles/falcon-default.dir/llmodel_shared.cpp.o" libllama-mainline-default.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 34%] Built target falcon-default
make -f CMakeFiles/mpt-default.dir/build.make CMakeFiles/mpt-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/mpt-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/mpt-default.dir/build.make CMakeFiles/mpt-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 35%] Building CXX object CMakeFiles/mpt-default.dir/mpt.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dmpt_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/mpt-default.dir/mpt.cpp.o -MF CMakeFiles/mpt-default.dir/mpt.cpp.o.d -o CMakeFiles/mpt-default.dir/mpt.cpp.o -c /build/gpt4all/gpt4all-backend/mpt.cpp
[ 36%] Building CXX object CMakeFiles/mpt-default.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dmpt_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/mpt-default.dir/utils.cpp.o -MF CMakeFiles/mpt-default.dir/utils.cpp.o.d -o CMakeFiles/mpt-default.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 37%] Building CXX object CMakeFiles/mpt-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dmpt_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/mpt-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/mpt-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/mpt-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 38%] Linking CXX shared library libmpt-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/mpt-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libmpt-default.so -o libmpt-default.so "CMakeFiles/mpt-default.dir/mpt.cpp.o" "CMakeFiles/mpt-default.dir/utils.cpp.o" "CMakeFiles/mpt-default.dir/llmodel_shared.cpp.o" "CMakeFiles/ggml-230511-default.dir/llama.cpp-230511/ggml.c.o" -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 38%] Built target mpt-default
make -f CMakeFiles/bert-default.dir/build.make CMakeFiles/bert-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/bert-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/bert-default.dir/build.make CMakeFiles/bert-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 40%] Building CXX object CMakeFiles/bert-default.dir/bert.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dbert_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/bert-default.dir/bert.cpp.o -MF CMakeFiles/bert-default.dir/bert.cpp.o.d -o CMakeFiles/bert-default.dir/bert.cpp.o -c /build/gpt4all/gpt4all-backend/bert.cpp
[ 41%] Building CXX object CMakeFiles/bert-default.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dbert_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/bert-default.dir/utils.cpp.o -MF CMakeFiles/bert-default.dir/utils.cpp.o.d -o CMakeFiles/bert-default.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 42%] Building CXX object CMakeFiles/bert-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dbert_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/bert-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/bert-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/bert-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 43%] Linking CXX shared library libbert-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/bert-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libbert-default.so -o libbert-default.so "CMakeFiles/bert-default.dir/bert.cpp.o" "CMakeFiles/bert-default.dir/utils.cpp.o" "CMakeFiles/bert-default.dir/llmodel_shared.cpp.o" libllama-mainline-default.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 43%] Built target bert-default
make -f CMakeFiles/starcoder-default.dir/build.make CMakeFiles/starcoder-default.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/starcoder-default.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/starcoder-default.dir/build.make CMakeFiles/starcoder-default.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 44%] Building CXX object CMakeFiles/starcoder-default.dir/starcoder.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dstarcoder_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/starcoder-default.dir/starcoder.cpp.o -MF CMakeFiles/starcoder-default.dir/starcoder.cpp.o.d -o CMakeFiles/starcoder-default.dir/starcoder.cpp.o -c /build/gpt4all/gpt4all-backend/starcoder.cpp
/build/gpt4all/gpt4all-backend/starcoder.cpp: In function 'bool starcoder_eval(const starcoder_model&, int, int, const std::vector<int>&, std::vector<float>&, size_t&)':
/build/gpt4all/gpt4all-backend/starcoder.cpp:470:18: warning: unused variable 'head_dim' [-Wunused-variable]
470 | const size_t head_dim = n_embd / n_head;
| ^~~~~~~~
[ 45%] Building CXX object CMakeFiles/starcoder-default.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dstarcoder_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/starcoder-default.dir/utils.cpp.o -MF CMakeFiles/starcoder-default.dir/utils.cpp.o.d -o CMakeFiles/starcoder-default.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 46%] Building CXX object CMakeFiles/starcoder-default.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"default\" -Dstarcoder_default_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/starcoder-default.dir/llmodel_shared.cpp.o -MF CMakeFiles/starcoder-default.dir/llmodel_shared.cpp.o.d -o CMakeFiles/starcoder-default.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 47%] Linking CXX shared library libstarcoder-default.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/starcoder-default.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libstarcoder-default.so -o libstarcoder-default.so "CMakeFiles/starcoder-default.dir/starcoder.cpp.o" "CMakeFiles/starcoder-default.dir/utils.cpp.o" "CMakeFiles/starcoder-default.dir/llmodel_shared.cpp.o" libllama-mainline-default.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 47%] Built target starcoder-default
make -f CMakeFiles/ggml-mainline-avxonly.dir/build.make CMakeFiles/ggml-mainline-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/ggml-mainline-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/ggml-mainline-avxonly.dir/build.make CMakeFiles/ggml-mainline-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 48%] Building C object CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/ggml.c.o
/usr/bin/cc -DGGML_USE_K_QUANTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -MD -MT CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/ggml.c.o -MF CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/ggml.c.o.d -o CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/ggml.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-mainline/ggml.c
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/ggml.c: In function 'quantize_row_q8_0':
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/ggml.c:1096:15: warning: unused variable 'nb' [-Wunused-variable]
1096 | const int nb = k / QK8_0;
| ^~
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/ggml.c: In function 'quantize_row_q8_1':
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/ggml.c:1291:15: warning: unused variable 'nb' [-Wunused-variable]
1291 | const int nb = k / QK8_1;
| ^~
[ 50%] Building C object CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/k_quants.c.o
/usr/bin/cc -DGGML_USE_K_QUANTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -MD -MT CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/k_quants.c.o -MF CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/k_quants.c.o.d -o CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/k_quants.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-mainline/k_quants.c
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 50%] Built target ggml-mainline-avxonly
make -f CMakeFiles/llama-mainline-avxonly.dir/build.make CMakeFiles/llama-mainline-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llama-mainline-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llama-mainline-avxonly.dir/build.make CMakeFiles/llama-mainline-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 51%] Building CXX object CMakeFiles/llama-mainline-avxonly.dir/llama.cpp-mainline/llama.cpp.o
/usr/bin/c++ -DGGML_USE_K_QUANTS -DLLAMA_BUILD -DLLAMA_SHARED -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/llama-mainline-avxonly.dir/llama.cpp-mainline/llama.cpp.o -MF CMakeFiles/llama-mainline-avxonly.dir/llama.cpp-mainline/llama.cpp.o.d -o CMakeFiles/llama-mainline-avxonly.dir/llama.cpp-mainline/llama.cpp.o -c /build/gpt4all/gpt4all-backend/llama.cpp-mainline/llama.cpp
[ 52%] Linking CXX static library libllama-mainline-avxonly.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama-mainline-avxonly.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama-mainline-avxonly.dir/link.txt --verbose=1
/usr/bin/ar qc libllama-mainline-avxonly.a "CMakeFiles/llama-mainline-avxonly.dir/llama.cpp-mainline/llama.cpp.o" "CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/ggml.c.o" "CMakeFiles/ggml-mainline-avxonly.dir/llama.cpp-mainline/k_quants.c.o"
/usr/bin/ranlib libllama-mainline-avxonly.a
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 52%] Built target llama-mainline-avxonly
make -f CMakeFiles/ggml-230511-avxonly.dir/build.make CMakeFiles/ggml-230511-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/ggml-230511-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/ggml-230511-avxonly.dir/build.make CMakeFiles/ggml-230511-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 53%] Building C object CMakeFiles/ggml-230511-avxonly.dir/llama.cpp-230511/ggml.c.o
/usr/bin/cc -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -MD -MT CMakeFiles/ggml-230511-avxonly.dir/llama.cpp-230511/ggml.c.o -MF CMakeFiles/ggml-230511-avxonly.dir/llama.cpp-230511/ggml.c.o.d -o CMakeFiles/ggml-230511-avxonly.dir/llama.cpp-230511/ggml.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'quantize_row_q4_0':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:781:15: warning: unused variable 'nb' [-Wunused-variable]
781 | const int nb = k / QK4_0;
| ^~
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'quantize_row_q4_1':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:1129:27: warning: unused variable 'y' [-Wunused-variable]
1129 | block_q4_1 * restrict y = vy;
| ^
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:1127:15: warning: unused variable 'nb' [-Wunused-variable]
1127 | const int nb = k / QK4_1;
| ^~
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'quantize_row_q8_1':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:1507:15: warning: unused variable 'nb' [-Wunused-variable]
1507 | const int nb = k / QK8_1;
| ^~
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'ggml_compute_forward_alibi_f32':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:9357:15: warning: unused variable 'ne2_ne3' [-Wunused-variable]
9357 | const int ne2_ne3 = n/ne1; // ne2*ne3
| ^~~~~~~
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'ggml_compute_forward_alibi_f16':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:9419:15: warning: unused variable 'ne2' [-Wunused-variable]
9419 | const int ne2 = src0->ne[2]; // n_head -> this is k
| ^~~
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c: In function 'ggml_compute_forward_alibi':
/build/gpt4all/gpt4all-backend/llama.cpp-230511/ggml.c:9468:5: warning: enumeration value 'GGML_TYPE_Q4_3' not handled in switch [-Wswitch]
9468 | switch (src0->type) {
| ^~~~~~
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 53%] Built target ggml-230511-avxonly
make -f CMakeFiles/llama-230511-avxonly.dir/build.make CMakeFiles/llama-230511-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llama-230511-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llama-230511-avxonly.dir/build.make CMakeFiles/llama-230511-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 54%] Building CXX object CMakeFiles/llama-230511-avxonly.dir/llama.cpp-230511/llama.cpp.o
/usr/bin/c++ -DLLAMA_BUILD -DLLAMA_SHARED -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/llama-230511-avxonly.dir/llama.cpp-230511/llama.cpp.o -MF CMakeFiles/llama-230511-avxonly.dir/llama.cpp-230511/llama.cpp.o.d -o CMakeFiles/llama-230511-avxonly.dir/llama.cpp-230511/llama.cpp.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230511/llama.cpp
[ 55%] Linking CXX static library libllama-230511-avxonly.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama-230511-avxonly.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama-230511-avxonly.dir/link.txt --verbose=1
/usr/bin/ar qc libllama-230511-avxonly.a "CMakeFiles/llama-230511-avxonly.dir/llama.cpp-230511/llama.cpp.o" "CMakeFiles/ggml-230511-avxonly.dir/llama.cpp-230511/ggml.c.o"
/usr/bin/ranlib libllama-230511-avxonly.a
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 55%] Built target llama-230511-avxonly
make -f CMakeFiles/ggml-230519-avxonly.dir/build.make CMakeFiles/ggml-230519-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/ggml-230519-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/ggml-230519-avxonly.dir/build.make CMakeFiles/ggml-230519-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 56%] Building C object CMakeFiles/ggml-230519-avxonly.dir/llama.cpp-230519/ggml.c.o
/usr/bin/cc -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -MD -MT CMakeFiles/ggml-230519-avxonly.dir/llama.cpp-230519/ggml.c.o -MF CMakeFiles/ggml-230519-avxonly.dir/llama.cpp-230519/ggml.c.o.d -o CMakeFiles/ggml-230519-avxonly.dir/llama.cpp-230519/ggml.c.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230519/ggml.c
/build/gpt4all/gpt4all-backend/llama.cpp-230519/ggml.c: In function 'quantize_row_q8_0':
/build/gpt4all/gpt4all-backend/llama.cpp-230519/ggml.c:1025:15: warning: unused variable 'nb' [-Wunused-variable]
1025 | const int nb = k / QK8_0;
| ^~
/build/gpt4all/gpt4all-backend/llama.cpp-230519/ggml.c: In function 'quantize_row_q8_1':
/build/gpt4all/gpt4all-backend/llama.cpp-230519/ggml.c:1187:15: warning: unused variable 'nb' [-Wunused-variable]
1187 | const int nb = k / QK8_1;
| ^~
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 56%] Built target ggml-230519-avxonly
make -f CMakeFiles/llama-230519-avxonly.dir/build.make CMakeFiles/llama-230519-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llama-230519-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llama-230519-avxonly.dir/build.make CMakeFiles/llama-230519-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 57%] Building CXX object CMakeFiles/llama-230519-avxonly.dir/llama.cpp-230519/llama.cpp.o
/usr/bin/c++ -DLLAMA_BUILD -DLLAMA_SHARED -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/llama-230519-avxonly.dir/llama.cpp-230519/llama.cpp.o -MF CMakeFiles/llama-230519-avxonly.dir/llama.cpp-230519/llama.cpp.o.d -o CMakeFiles/llama-230519-avxonly.dir/llama.cpp-230519/llama.cpp.o -c /build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp
/build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp: In function 'size_t llama_set_state_data(llama_context*, const uint8_t*)':
/build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp:2685:27: warning: cast from type 'const uint8_t*' {aka 'const unsigned char*'} to type 'void*' casts away qualifiers [-Wcast-qual]
2685 | kin3d->data = (void *) inp;
| ^~~~~~~~~~~~
/build/gpt4all/gpt4all-backend/llama.cpp-230519/llama.cpp:2689:27: warning: cast from type 'const uint8_t*' {aka 'const unsigned char*'} to type 'void*' casts away qualifiers [-Wcast-qual]
2689 | vin3d->data = (void *) inp;
| ^~~~~~~~~~~~
[ 58%] Linking CXX static library libllama-230519-avxonly.a
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -P CMakeFiles/llama-230519-avxonly.dir/cmake_clean_target.cmake
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llama-230519-avxonly.dir/link.txt --verbose=1
/usr/bin/ar qc libllama-230519-avxonly.a "CMakeFiles/llama-230519-avxonly.dir/llama.cpp-230519/llama.cpp.o" "CMakeFiles/ggml-230519-avxonly.dir/llama.cpp-230519/ggml.c.o"
/usr/bin/ranlib libllama-230519-avxonly.a
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 58%] Built target llama-230519-avxonly
make -f CMakeFiles/llamamodel-mainline-avxonly.dir/build.make CMakeFiles/llamamodel-mainline-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llamamodel-mainline-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llamamodel-mainline-avxonly.dir/build.make CMakeFiles/llamamodel-mainline-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 60%] Building CXX object CMakeFiles/llamamodel-mainline-avxonly.dir/llamamodel.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -DLLAMA_DATE=999999 -DLLAMA_VERSIONS=">=3" -Dllamamodel_mainline_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-mainline-avxonly.dir/llamamodel.cpp.o -MF CMakeFiles/llamamodel-mainline-avxonly.dir/llamamodel.cpp.o.d -o CMakeFiles/llamamodel-mainline-avxonly.dir/llamamodel.cpp.o -c /build/gpt4all/gpt4all-backend/llamamodel.cpp
/build/gpt4all/gpt4all-backend/llamamodel.cpp: In member function 'virtual bool LLamaModel::loadModel(const string&)':
/build/gpt4all/gpt4all-backend/llamamodel.cpp:159:71: warning: 'llama_context* llama_init_from_file(const char*, llama_context_params)' is deprecated: please use llama_load_model_from_file combined with llama_new_context_with_model instead [-Wdeprecated-declarations]
159 | d_ptr->ctx = llama_init_from_file(modelPath.c_str(), d_ptr->params);
| ^
In file included from /build/gpt4all/gpt4all-backend/llamamodel.cpp:28:
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/llama.h:161:49: note: declared here
161 | LLAMA_API DEPRECATED(struct llama_context * llama_init_from_file(
| ^~~~~~~~~~~~~~~~~~~~
/build/gpt4all/gpt4all-backend/llama.cpp-mainline/llama.h:30:36: note: in definition of macro 'DEPRECATED'
30 | # define DEPRECATED(func, hint) func __attribute__((deprecated(hint)))
| ^~~~
[ 61%] Building CXX object CMakeFiles/llamamodel-mainline-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -DLLAMA_DATE=999999 -DLLAMA_VERSIONS=">=3" -Dllamamodel_mainline_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-mainline-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/llamamodel-mainline-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/llamamodel-mainline-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 62%] Linking CXX shared library libllamamodel-mainline-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llamamodel-mainline-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libllamamodel-mainline-avxonly.so -o libllamamodel-mainline-avxonly.so "CMakeFiles/llamamodel-mainline-avxonly.dir/llamamodel.cpp.o" "CMakeFiles/llamamodel-mainline-avxonly.dir/llmodel_shared.cpp.o" libllama-mainline-avxonly.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 62%] Built target llamamodel-mainline-avxonly
make -f CMakeFiles/replit-mainline-avxonly.dir/build.make CMakeFiles/replit-mainline-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/replit-mainline-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/replit-mainline-avxonly.dir/build.make CMakeFiles/replit-mainline-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 63%] Building CXX object CMakeFiles/replit-mainline-avxonly.dir/replit.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dreplit_mainline_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/replit-mainline-avxonly.dir/replit.cpp.o -MF CMakeFiles/replit-mainline-avxonly.dir/replit.cpp.o.d -o CMakeFiles/replit-mainline-avxonly.dir/replit.cpp.o -c /build/gpt4all/gpt4all-backend/replit.cpp
/build/gpt4all/gpt4all-backend/replit.cpp: In function 'bool replit_eval(const replit_model&, int, int, const std::vector<int>&, std::vector<float>&, size_t&)':
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::n_nodes' [-Wmissing-field-initializers]
555 | struct ggml_cgraph gf = {.n_threads = n_threads};
| ^
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::n_leafs' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::work_size' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::work' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::nodes' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::grads' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::leafs' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::perf_runs' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::perf_cycles' [-Wmissing-field-initializers]
/build/gpt4all/gpt4all-backend/replit.cpp:555:52: warning: missing initializer for member 'ggml_cgraph::perf_time_us' [-Wmissing-field-initializers]
[ 64%] Building CXX object CMakeFiles/replit-mainline-avxonly.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dreplit_mainline_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/replit-mainline-avxonly.dir/utils.cpp.o -MF CMakeFiles/replit-mainline-avxonly.dir/utils.cpp.o.d -o CMakeFiles/replit-mainline-avxonly.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 65%] Building CXX object CMakeFiles/replit-mainline-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dreplit_mainline_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/replit-mainline-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/replit-mainline-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/replit-mainline-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 66%] Linking CXX shared library libreplit-mainline-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/replit-mainline-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libreplit-mainline-avxonly.so -o libreplit-mainline-avxonly.so "CMakeFiles/replit-mainline-avxonly.dir/replit.cpp.o" "CMakeFiles/replit-mainline-avxonly.dir/utils.cpp.o" "CMakeFiles/replit-mainline-avxonly.dir/llmodel_shared.cpp.o" libllama-mainline-avxonly.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 66%] Built target replit-mainline-avxonly
make -f CMakeFiles/llamamodel-230519-avxonly.dir/build.make CMakeFiles/llamamodel-230519-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llamamodel-230519-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llamamodel-230519-avxonly.dir/build.make CMakeFiles/llamamodel-230519-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 67%] Building CXX object CMakeFiles/llamamodel-230519-avxonly.dir/llamamodel.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -DLLAMA_DATE=230519 -DLLAMA_VERSIONS===2 -Dllamamodel_230519_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230519-avxonly.dir/llamamodel.cpp.o -MF CMakeFiles/llamamodel-230519-avxonly.dir/llamamodel.cpp.o.d -o CMakeFiles/llamamodel-230519-avxonly.dir/llamamodel.cpp.o -c /build/gpt4all/gpt4all-backend/llamamodel.cpp
[ 68%] Building CXX object CMakeFiles/llamamodel-230519-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -DLLAMA_DATE=230519 -DLLAMA_VERSIONS===2 -Dllamamodel_230519_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230519 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230519-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/llamamodel-230519-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/llamamodel-230519-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 70%] Linking CXX shared library libllamamodel-230519-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llamamodel-230519-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libllamamodel-230519-avxonly.so -o libllamamodel-230519-avxonly.so "CMakeFiles/llamamodel-230519-avxonly.dir/llamamodel.cpp.o" "CMakeFiles/llamamodel-230519-avxonly.dir/llmodel_shared.cpp.o" libllama-230519-avxonly.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 70%] Built target llamamodel-230519-avxonly
make -f CMakeFiles/llamamodel-230511-avxonly.dir/build.make CMakeFiles/llamamodel-230511-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llamamodel-230511-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llamamodel-230511-avxonly.dir/build.make CMakeFiles/llamamodel-230511-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 71%] Building CXX object CMakeFiles/llamamodel-230511-avxonly.dir/llamamodel.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -DLLAMA_DATE=230511 -DLLAMA_VERSIONS="<=1" -Dllamamodel_230511_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230511-avxonly.dir/llamamodel.cpp.o -MF CMakeFiles/llamamodel-230511-avxonly.dir/llamamodel.cpp.o.d -o CMakeFiles/llamamodel-230511-avxonly.dir/llamamodel.cpp.o -c /build/gpt4all/gpt4all-backend/llamamodel.cpp
[ 72%] Building CXX object CMakeFiles/llamamodel-230511-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -DLLAMA_DATE=230511 -DLLAMA_VERSIONS="<=1" -Dllamamodel_230511_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llamamodel-230511-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/llamamodel-230511-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/llamamodel-230511-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 73%] Linking CXX shared library libllamamodel-230511-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llamamodel-230511-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libllamamodel-230511-avxonly.so -o libllamamodel-230511-avxonly.so "CMakeFiles/llamamodel-230511-avxonly.dir/llamamodel.cpp.o" "CMakeFiles/llamamodel-230511-avxonly.dir/llmodel_shared.cpp.o" libllama-230511-avxonly.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 73%] Built target llamamodel-230511-avxonly
make -f CMakeFiles/gptj-avxonly.dir/build.make CMakeFiles/gptj-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/gptj-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/gptj-avxonly.dir/build.make CMakeFiles/gptj-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 74%] Building CXX object CMakeFiles/gptj-avxonly.dir/gptj.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dgptj_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/gptj-avxonly.dir/gptj.cpp.o -MF CMakeFiles/gptj-avxonly.dir/gptj.cpp.o.d -o CMakeFiles/gptj-avxonly.dir/gptj.cpp.o -c /build/gpt4all/gpt4all-backend/gptj.cpp
[ 75%] Building CXX object CMakeFiles/gptj-avxonly.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dgptj_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/gptj-avxonly.dir/utils.cpp.o -MF CMakeFiles/gptj-avxonly.dir/utils.cpp.o.d -o CMakeFiles/gptj-avxonly.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 76%] Building CXX object CMakeFiles/gptj-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dgptj_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/gptj-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/gptj-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/gptj-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 77%] Linking CXX shared library libgptj-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/gptj-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libgptj-avxonly.so -o libgptj-avxonly.so "CMakeFiles/gptj-avxonly.dir/gptj.cpp.o" "CMakeFiles/gptj-avxonly.dir/utils.cpp.o" "CMakeFiles/gptj-avxonly.dir/llmodel_shared.cpp.o" "CMakeFiles/ggml-230511-avxonly.dir/llama.cpp-230511/ggml.c.o" -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 77%] Built target gptj-avxonly
make -f CMakeFiles/falcon-avxonly.dir/build.make CMakeFiles/falcon-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/falcon-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/falcon-avxonly.dir/build.make CMakeFiles/falcon-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 78%] Building CXX object CMakeFiles/falcon-avxonly.dir/falcon.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dfalcon_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/falcon-avxonly.dir/falcon.cpp.o -MF CMakeFiles/falcon-avxonly.dir/falcon.cpp.o.d -o CMakeFiles/falcon-avxonly.dir/falcon.cpp.o -c /build/gpt4all/gpt4all-backend/falcon.cpp
/build/gpt4all/gpt4all-backend/falcon.cpp: In function 'bool falcon_model_load(const string&, falcon_model&, gpt_vocab&, size_t*)':
/build/gpt4all/gpt4all-backend/falcon.cpp:199:19: warning: unused variable 'n_ctx' [-Wunused-variable]
199 | const int n_ctx = hparams.n_ctx;
| ^~~~~
/build/gpt4all/gpt4all-backend/falcon.cpp:340:19: warning: unused variable 'n_head_kv' [-Wunused-variable]
340 | const int n_head_kv = hparams.n_head_kv;
| ^~~~~~~~~
/build/gpt4all/gpt4all-backend/falcon.cpp:344:23: warning: unused variable 'n_elements' [-Wunused-variable]
344 | const int64_t n_elements = head_dim*n_mem;
| ^~~~~~~~~~
/build/gpt4all/gpt4all-backend/falcon.cpp: In function 'bool falcon_eval(const falcon_model&, int, int, const std::vector<int>&, std::vector<float>&, size_t&)':
/build/gpt4all/gpt4all-backend/falcon.cpp:465:15: warning: unused variable 'version' [-Wunused-variable]
465 | const int version = hparams.falcon_version;
| ^~~~~~~
[ 80%] Building CXX object CMakeFiles/falcon-avxonly.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dfalcon_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/falcon-avxonly.dir/utils.cpp.o -MF CMakeFiles/falcon-avxonly.dir/utils.cpp.o.d -o CMakeFiles/falcon-avxonly.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 81%] Building CXX object CMakeFiles/falcon-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dfalcon_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/falcon-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/falcon-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/falcon-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 82%] Linking CXX shared library libfalcon-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/falcon-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libfalcon-avxonly.so -o libfalcon-avxonly.so "CMakeFiles/falcon-avxonly.dir/falcon.cpp.o" "CMakeFiles/falcon-avxonly.dir/utils.cpp.o" "CMakeFiles/falcon-avxonly.dir/llmodel_shared.cpp.o" libllama-mainline-avxonly.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 82%] Built target falcon-avxonly
make -f CMakeFiles/mpt-avxonly.dir/build.make CMakeFiles/mpt-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/mpt-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/mpt-avxonly.dir/build.make CMakeFiles/mpt-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 83%] Building CXX object CMakeFiles/mpt-avxonly.dir/mpt.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dmpt_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/mpt-avxonly.dir/mpt.cpp.o -MF CMakeFiles/mpt-avxonly.dir/mpt.cpp.o.d -o CMakeFiles/mpt-avxonly.dir/mpt.cpp.o -c /build/gpt4all/gpt4all-backend/mpt.cpp
[ 84%] Building CXX object CMakeFiles/mpt-avxonly.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dmpt_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/mpt-avxonly.dir/utils.cpp.o -MF CMakeFiles/mpt-avxonly.dir/utils.cpp.o.d -o CMakeFiles/mpt-avxonly.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 85%] Building CXX object CMakeFiles/mpt-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dmpt_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-230511 -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -MD -MT CMakeFiles/mpt-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/mpt-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/mpt-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 86%] Linking CXX shared library libmpt-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/mpt-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libmpt-avxonly.so -o libmpt-avxonly.so "CMakeFiles/mpt-avxonly.dir/mpt.cpp.o" "CMakeFiles/mpt-avxonly.dir/utils.cpp.o" "CMakeFiles/mpt-avxonly.dir/llmodel_shared.cpp.o" "CMakeFiles/ggml-230511-avxonly.dir/llama.cpp-230511/ggml.c.o" -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 86%] Built target mpt-avxonly
make -f CMakeFiles/bert-avxonly.dir/build.make CMakeFiles/bert-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/bert-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/bert-avxonly.dir/build.make CMakeFiles/bert-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 87%] Building CXX object CMakeFiles/bert-avxonly.dir/bert.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dbert_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/bert-avxonly.dir/bert.cpp.o -MF CMakeFiles/bert-avxonly.dir/bert.cpp.o.d -o CMakeFiles/bert-avxonly.dir/bert.cpp.o -c /build/gpt4all/gpt4all-backend/bert.cpp
[ 88%] Building CXX object CMakeFiles/bert-avxonly.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dbert_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/bert-avxonly.dir/utils.cpp.o -MF CMakeFiles/bert-avxonly.dir/utils.cpp.o.d -o CMakeFiles/bert-avxonly.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 90%] Building CXX object CMakeFiles/bert-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dbert_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/bert-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/bert-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/bert-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 91%] Linking CXX shared library libbert-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/bert-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libbert-avxonly.so -o libbert-avxonly.so "CMakeFiles/bert-avxonly.dir/bert.cpp.o" "CMakeFiles/bert-avxonly.dir/utils.cpp.o" "CMakeFiles/bert-avxonly.dir/llmodel_shared.cpp.o" libllama-mainline-avxonly.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 91%] Built target bert-avxonly
make -f CMakeFiles/starcoder-avxonly.dir/build.make CMakeFiles/starcoder-avxonly.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/starcoder-avxonly.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/starcoder-avxonly.dir/build.make CMakeFiles/starcoder-avxonly.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 92%] Building CXX object CMakeFiles/starcoder-avxonly.dir/starcoder.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dstarcoder_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/starcoder-avxonly.dir/starcoder.cpp.o -MF CMakeFiles/starcoder-avxonly.dir/starcoder.cpp.o.d -o CMakeFiles/starcoder-avxonly.dir/starcoder.cpp.o -c /build/gpt4all/gpt4all-backend/starcoder.cpp
/build/gpt4all/gpt4all-backend/starcoder.cpp: In function 'bool starcoder_eval(const starcoder_model&, int, int, const std::vector<int>&, std::vector<float>&, size_t&)':
/build/gpt4all/gpt4all-backend/starcoder.cpp:470:18: warning: unused variable 'head_dim' [-Wunused-variable]
470 | const size_t head_dim = n_embd / n_head;
| ^~~~~~~~
[ 93%] Building CXX object CMakeFiles/starcoder-avxonly.dir/utils.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dstarcoder_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/starcoder-avxonly.dir/utils.cpp.o -MF CMakeFiles/starcoder-avxonly.dir/utils.cpp.o.d -o CMakeFiles/starcoder-avxonly.dir/utils.cpp.o -c /build/gpt4all/gpt4all-backend/utils.cpp
[ 94%] Building CXX object CMakeFiles/starcoder-avxonly.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DGGML_BUILD_VARIANT=\"avxonly\" -Dstarcoder_avxonly_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -I/build/gpt4all/gpt4all-backend/llama.cpp-mainline -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/starcoder-avxonly.dir/llmodel_shared.cpp.o -MF CMakeFiles/starcoder-avxonly.dir/llmodel_shared.cpp.o.d -o CMakeFiles/starcoder-avxonly.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 95%] Linking CXX shared library libstarcoder-avxonly.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/starcoder-avxonly.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libstarcoder-avxonly.so -o libstarcoder-avxonly.so "CMakeFiles/starcoder-avxonly.dir/starcoder.cpp.o" "CMakeFiles/starcoder-avxonly.dir/utils.cpp.o" "CMakeFiles/starcoder-avxonly.dir/llmodel_shared.cpp.o" libllama-mainline-avxonly.a -pthread
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 95%] Built target starcoder-avxonly
make -f CMakeFiles/llmodel.dir/build.make CMakeFiles/llmodel.dir/depend
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd /build/gpt4all/gpt4all-bindings/golang/buildllm && /usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-backend /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles/llmodel.dir/DependInfo.cmake "--color="
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
make -f CMakeFiles/llmodel.dir/build.make CMakeFiles/llmodel.dir/build
make[4]: Entering directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[ 96%] Building CXX object CMakeFiles/llmodel.dir/llmodel.cpp.o
/usr/bin/c++ -DLIB_FILE_EXT=\".so\" -Dllmodel_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llmodel.dir/llmodel.cpp.o -MF CMakeFiles/llmodel.dir/llmodel.cpp.o.d -o CMakeFiles/llmodel.dir/llmodel.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel.cpp
[ 97%] Building CXX object CMakeFiles/llmodel.dir/llmodel_shared.cpp.o
/usr/bin/c++ -DLIB_FILE_EXT=\".so\" -Dllmodel_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llmodel.dir/llmodel_shared.cpp.o -MF CMakeFiles/llmodel.dir/llmodel_shared.cpp.o.d -o CMakeFiles/llmodel.dir/llmodel_shared.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_shared.cpp
[ 98%] Building CXX object CMakeFiles/llmodel.dir/llmodel_c.cpp.o
/usr/bin/c++ -DLIB_FILE_EXT=\".so\" -Dllmodel_EXPORTS -I/build/gpt4all/gpt4all-bindings/golang/buildllm -O3 -DNDEBUG -std=gnu++2a -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -MD -MT CMakeFiles/llmodel.dir/llmodel_c.cpp.o -MF CMakeFiles/llmodel.dir/llmodel_c.cpp.o.d -o CMakeFiles/llmodel.dir/llmodel_c.cpp.o -c /build/gpt4all/gpt4all-backend/llmodel_c.cpp
[100%] Linking CXX shared library libllmodel.so
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/llmodel.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libllmodel.so.0 -o libllmodel.so.0.3.0 CMakeFiles/llmodel.dir/llmodel.cpp.o CMakeFiles/llmodel.dir/llmodel_shared.cpp.o CMakeFiles/llmodel.dir/llmodel_c.cpp.o
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_symlink_library libllmodel.so.0.3.0 libllmodel.so.0 libllmodel.so
make[4]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
[100%] Built target llmodel
make[3]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
/usr/local/lib/python3.9/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /build/gpt4all/gpt4all-bindings/golang/buildllm/CMakeFiles 0
make[2]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang/buildllm'
cd buildllm && cp -rf CMakeFiles/llmodel.dir/llmodel_c.cpp.o ../llmodel_c.o
cd buildllm && cp -rf CMakeFiles/llmodel.dir/llmodel.cpp.o ../llmodel.o
ar src libgpt4all.a llmodel.o binding.o
make[1]: Leaving directory '/build/gpt4all/gpt4all-bindings/golang'
mkdir -p backend-assets/gpt4all
cp: cannot stat 'gpt4all/gpt4all-bindings/golang/buildllm/*.dylib': No such file or directory
cp: cannot stat 'gpt4all/gpt4all-bindings/golang/buildllm/*.dll': No such file or directory
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/gpt4all/gpt4all-bindings/golang/ LIBRARY_PATH=/build/gpt4all/gpt4all-bindings/golang/ \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/gpt4all ./cmd/grpc/gpt4all/
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/dolly ./cmd/grpc/dolly/
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/gpt2 ./cmd/grpc/gpt2/
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/gptj ./cmd/grpc/gptj/
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/gptneox ./cmd/grpc/gptneox/
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/mpt ./cmd/grpc/mpt/
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/replit ./cmd/grpc/replit/
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-ggml-transformers LIBRARY_PATH=/build/go-ggml-transformers \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/starcoder ./cmd/grpc/starcoder/
cd go-rwkv && cd rwkv.cpp && cmake . -DRWKV_BUILD_SHARED_LIBRARY=OFF && cmake --build . && cp librwkv.a ..
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Configuring done (0.4s)
-- Generating done (0.0s)
-- Build files have been written to: /build/go-rwkv/rwkv.cpp
gmake[1]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[2]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
[ 7%] Building C object CMakeFiles/ggml.dir/ggml/src/ggml.c.o
/build/go-rwkv/rwkv.cpp/ggml/src/ggml.c: In function ‘ggml_compute_forward_win_part_f32’:
/build/go-rwkv/rwkv.cpp/ggml/src/ggml.c:13064:19: warning: unused variable ‘ne3’ [-Wunused-variable]
13064 | const int64_t ne3 = dst->ne[3];
| ^~~
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
[ 7%] Built target ggml
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
[ 15%] Building CXX object CMakeFiles/rwkv.dir/rwkv.cpp.o
/build/go-rwkv/rwkv.cpp/rwkv.cpp: In function ‘bool rwkv_fread_string(FILE*, size_t, std::string&)’:
/build/go-rwkv/rwkv.cpp/rwkv.cpp:149:18: warning: cast from type ‘const char*’ to type ‘void*’ casts away qualifiers [-Wcast-qual]
149 | return fread((void *) dest.data(), length, 1, file) == 1;
| ^~~~~~~~~~~~~~~~~~~~
/build/go-rwkv/rwkv.cpp/rwkv.cpp: At global scope:
/build/go-rwkv/rwkv.cpp/rwkv.cpp:223:21: warning: ‘rwkv_type_to_string’ initialized and declared ‘extern’
223 | extern const char * rwkv_type_to_string[TYPE_COUNT + 1] = {"float32", "float16", "Q4_0", "Q4_1", "Q4_1_O", "Q4_2", "Q4_3", "Q5_0", "Q5_1", "Q8_0", "unknown"};
| ^~~~~~~~~~~~~~~~~~~
/build/go-rwkv/rwkv.cpp/rwkv.cpp: In function ‘bool rwkv_gpu_offload_layers(const rwkv_context*, uint32_t)’:
/build/go-rwkv/rwkv.cpp/rwkv.cpp:1280:58: warning: unused parameter ‘ctx’ [-Wunused-parameter]
1280 | bool rwkv_gpu_offload_layers(const struct rwkv_context * ctx, const uint32_t n_gpu_layers) {
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/build/go-rwkv/rwkv.cpp/rwkv.cpp:1280:78: warning: unused parameter ‘n_gpu_layers’ [-Wunused-parameter]
1280 | bool rwkv_gpu_offload_layers(const struct rwkv_context * ctx, const uint32_t n_gpu_layers) {
| ~~~~~~~~~~~~~~~^~~~~~~~~~~~
[ 23%] Linking CXX static library librwkv.a
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
[ 23%] Built target rwkv
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
[ 30%] Building C object tests/CMakeFiles/test_ggml_basics.dir/test_ggml_basics.c.o
[ 38%] Linking CXX executable ../bin/test_ggml_basics
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
[ 38%] Built target test_ggml_basics
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
[ 46%] Building C object tests/CMakeFiles/test_tiny_rwkv.dir/test_tiny_rwkv.c.o
/build/go-rwkv/rwkv.cpp/tests/test_tiny_rwkv.c: In function ‘test_model’:
/build/go-rwkv/rwkv.cpp/tests/test_tiny_rwkv.c:60:45: warning: implicit conversion from ‘float’ to ‘double’ when passing argument to function [-Wdouble-promotion]
60 | fprintf(stderr, "Difference sum: %f\n", diff_sum);
| ^~~~~~~~
/build/go-rwkv/rwkv.cpp/tests/test_tiny_rwkv.c:74:54: warning: implicit conversion from ‘float’ to ‘double’ when passing argument to function [-Wdouble-promotion]
74 | fprintf(stderr, "Sequence difference sum: %f\n", diff_sum);
| ^~~~~~~~
[ 53%] Linking CXX executable ../bin/test_tiny_rwkv
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
[ 53%] Built target test_tiny_rwkv
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
[ 61%] Building C object tests/CMakeFiles/test_context_cloning.dir/test_context_cloning.c.o
/build/go-rwkv/rwkv.cpp/tests/test_context_cloning.c:7:5: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
7 | int main() {
| ^~~~
[ 69%] Linking CXX executable ../bin/test_context_cloning
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
[ 69%] Built target test_context_cloning
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
[ 76%] Building C object extras/CMakeFiles/rwkv_cpu_info.dir/cpu_info.c.o
/build/go-rwkv/rwkv.cpp/extras/cpu_info.c:5:5: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
5 | int main() {
| ^~~~
[ 84%] Linking CXX executable ../bin/rwkv_cpu_info
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
[ 84%] Built target rwkv_cpu_info
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[3]: Entering directory '/build/go-rwkv/rwkv.cpp'
[ 92%] Building C object extras/CMakeFiles/rwkv_quantize.dir/quantize.c.o
[100%] Linking CXX executable ../bin/rwkv_quantize
gmake[3]: Leaving directory '/build/go-rwkv/rwkv.cpp'
[100%] Built target rwkv_quantize
gmake[2]: Leaving directory '/build/go-rwkv/rwkv.cpp'
gmake[1]: Leaving directory '/build/go-rwkv/rwkv.cpp'
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/go-rwkv LIBRARY_PATH=/build/go-rwkv \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/rwkv ./cmd/grpc/rwkv/
cd whisper.cpp && make libwhisper.a
make[1]: Entering directory '/build/whisper.cpp'
I whisper.cpp build info:
I UNAME_S: Linux
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -pthread -mavx2 -mfma -mf16c -mavx -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -pthread
I LDFLAGS:
I CC: cc (Debian 10.2.1-6) 10.2.1 20210110
I CXX: g++ (Debian 10.2.1-6) 10.2.1 20210110
cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -pthread -mavx2 -mfma -mf16c -mavx -msse3 -c ggml.c -o ggml.o
g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -pthread -c whisper.cpp -o whisper.o
ar rcs libwhisper.a ggml.o whisper.o
make[1]: Leaving directory '/build/whisper.cpp'
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" C_INCLUDE_PATH=/build/whisper.cpp LIBRARY_PATH=/build/whisper.cpp \
go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o backend-assets/grpc/whisper ./cmd/grpc/whisper/
I local-ai build info:(B
I BUILD_TYPE: cublas(B
I GO_TAGS: (B
I LD_FLAGS: -X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"(B
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v1.23.2" -X "github.com/go-skynet/LocalAI/internal.Commit=acd829a7a0e1623c0871c8b34c36c76afd4feac8"" -tags "" -o local-ai ./
5:19PM DBG no galleries to load
5:19PM INF Starting LocalAI using 1 threads, with models path: /models
5:19PM INF LocalAI version: v1.23.2 (acd829a7a0e1623c0871c8b34c36c76afd4feac8)
5:19PM DBG Model: gpt-3.5-turbo (config: {PredictionOptions:{Model:open-llama-7b-q4_0.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.2 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0} Name:gpt-3.5-turbo StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 F16:true NUMA:false Threads:0 Debug:false Roles:map[] Embeddings:false Backend:llama TemplateConfig:{Chat:openllama-chat ChatMessage: Completion:openllama-completion Edit: Functions:} MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:35 MMap:true MMlock:false LowVRAM:false TensorSplit: MainGPU: ImageGenerationAssets: PromptCachePath: PromptCacheAll:false PromptCacheRO:false Grammar: PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} SystemPrompt: RMSNormEps:0 NGQA:0})
5:19PM DBG Extracting backend assets files to /tmp/localai/backend_data
5:19PM DBG Config overrides map[batch:512 f16:true gpu_layers:35 mmap:true]
5:19PM DBG Checking "open-llama-7b-q4_0.bin" exists and matches SHA
5:20PM DBG File "open-llama-7b-q4_0.bin" already exists and matches the SHA. Skipping download
5:20PM DBG Prompt template "openllama-completion" written
5:20PM DBG Prompt template "openllama-chat" written
5:20PM DBG Written config file /models/gpt-3.5-turbo.yaml
 ┌───────────────────────────────────────────────────┐
 │ Fiber v2.48.0 │
 │ http://127.0.0.1:8080 │
 │ (bound on host 0.0.0.0 and port 8080) │
 │ │
 │ Handlers ............ 32 Processes ........... 1 │
 │ Prefork ....... Disabled PID ............. 11147 │
 └───────────────────────────────────────────────────┘
[127.0.0.1]:53244  200  -  GET  /readyz
[127.0.0.1]:51742  200  -  GET  /readyz
[127.0.0.1]:51422  200  -  GET  /readyz
[127.0.0.1]:52380  200  -  GET  /readyz
[127.0.0.1]:50440  200  -  GET  /readyz
[127.0.0.1]:45034  200  -  GET  /readyz
[127.0.0.1]:58848  200  -  GET  /readyz
[127.0.0.1]:39190  200  -  GET  /readyz
[127.0.0.1]:51318  200  -  GET  /readyz
[127.0.0.1]:40980  200  -  GET  /readyz
[127.0.0.1]:52126  200  -  GET  /readyz
[172.17.0.1]:58314  200  -  GET  /v1/models
5:31PM DBG Request received:
5:31PM DBG Configuration read: &{PredictionOptions:{Model:luna-ai-llama2 Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:512 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0} Name: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:512 F16:false NUMA:false Threads:1 Debug:true Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false TensorSplit: MainGPU: ImageGenerationAssets: PromptCachePath: PromptCacheAll:false PromptCacheRO:false Grammar: PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} SystemPrompt: RMSNormEps:0 NGQA:0}
5:31PM DBG Parameters: &{PredictionOptions:{Model:luna-ai-llama2 Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:512 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0} Name: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:512 F16:false NUMA:false Threads:1 Debug:true Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false TensorSplit: MainGPU: ImageGenerationAssets: PromptCachePath: PromptCacheAll:false PromptCacheRO:false Grammar: PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} SystemPrompt: RMSNormEps:0 NGQA:0}
5:31PM DBG Prompt (before templating): How are you?
5:31PM DBG Template failed loading: failed loading a template for luna-ai-llama2
5:31PM DBG Prompt (after templating): How are you?
5:31PM DBG Loading model 'luna-ai-llama2' greedly from all the available backends: llama, gpt4all, falcon, gptneox, bert-embeddings, falcon-ggml, gptj, gpt2, dolly, mpt, replit, starcoder, bloomz, rwkv, whisper, stablediffusion, piper, /build/extra/grpc/huggingface/huggingface.py
5:31PM DBG [llama] Attempting to load
5:31PM DBG Loading model llama from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model llama: {backendString:llama modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/llama
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:33457'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager3620426260
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:33457: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr 2023/11/09 17:31:16 gRPC Server listening at 127.0.0.1:33457
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr create_gpt_params: loading model /models/luna-ai-llama2
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr ggml_init_cublas: found 1 CUDA devices:
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr Device 0: NVIDIA GeForce RTX 3070 Ti Laptop GPU, compute capability 8.6
5:31PM DBG [llama] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [gpt4all] Attempting to load
5:31PM DBG Loading model gpt4all from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model gpt4all: {backendString:gpt4all modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/gpt4all
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:36555'
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr llama.cpp: loading model from /models/luna-ai-llama2
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr error loading model: unknown (magic, version) combination: 46554747, 00000002; is this really a GGML file?
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr llama_load_model_from_file: failed to load model
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr llama_init_from_gpt_params: error: failed to load model '/models/luna-ai-llama2'
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33457): stderr load_binding_model: error: unable to load model
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager1874264123
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:36555: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:36555): stderr 2023/11/09 17:31:17 gRPC Server listening at 127.0.0.1:36555
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:36555): stderr load_model: error 'Model format not supported (no matching implementation found)'
5:31PM DBG [gpt4all] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [falcon] Attempting to load
5:31PM DBG Loading model falcon from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model falcon: {backendString:falcon modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/falcon
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:34157'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager2617030949
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:34157: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:34157): stderr 2023/11/09 17:31:18 gRPC Server listening at 127.0.0.1:34157
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:34157): stderr falcon.cpp: loading model from /models/luna-ai-llama2
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:34157): stderr error loading model: unknown (magic, version) combination: 46554747, 00000002; is this really a GGML file?
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:34157): stderr falcon_init_from_file: failed to load model
5:31PM DBG [falcon] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [gptneox] Attempting to load
5:31PM DBG Loading model gptneox from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model gptneox: {backendString:gptneox modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/gptneox
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:39909'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager3713018119
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:39909: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:39909): stderr 2023/11/09 17:31:20 gRPC Server listening at 127.0.0.1:39909
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG [gptneox] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:39909): stderr gpt_neox_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG [bert-embeddings] Attempting to load
5:31PM DBG Loading model bert-embeddings from luna-ai-llama2
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:39909): stderr gpt_neox_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model bert-embeddings: {backendString:bert-embeddings modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/bert-embeddings
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:46105'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager3859495679
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:46105: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46105): stderr 2023/11/09 17:31:21 gRPC Server listening at 127.0.0.1:46105
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46105): stderr bert_load_from_file: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG [bert-embeddings] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [falcon-ggml] Attempting to load
5:31PM DBG Loading model falcon-ggml from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model falcon-ggml: {backendString:falcon-ggml modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46105): stderr bert_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/falcon-ggml
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:40631'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager2904493726
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:40631: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:40631): stderr 2023/11/09 17:31:22 gRPC Server listening at 127.0.0.1:40631
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:40631): stderr falcon_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:40631): stderr falcon_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG [falcon-ggml] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [gptj] Attempting to load
5:31PM DBG Loading model gptj from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model gptj: {backendString:gptj modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/gptj
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:37217'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager2121419443
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:37217: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:37217): stderr 2023/11/09 17:31:23 gRPC Server listening at 127.0.0.1:37217
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG [gptj] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [gpt2] Attempting to load
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:37217): stderr gptj_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG Loading model gpt2 from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model gpt2: {backendString:gpt2 modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:37217): stderr gptj_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/gpt2
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:46395'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager927256336
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:46395: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46395): stderr 2023/11/09 17:31:24 gRPC Server listening at 127.0.0.1:46395
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG [gpt2] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [dolly] Attempting to load
5:31PM DBG Loading model dolly from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model dolly: {backendString:dolly modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46395): stderr gpt2_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46395): stderr gpt2_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/dolly
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:46733'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager3702163237
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:46733: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46733): stderr 2023/11/09 17:31:25 gRPC Server listening at 127.0.0.1:46733
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46733): stderr dollyv2_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:46733): stderr dolly_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG [dolly] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [mpt] Attempting to load
5:31PM DBG Loading model mpt from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model mpt: {backendString:mpt modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/mpt
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:38383'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager371655322
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:38383: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:38383): stderr 2023/11/09 17:31:26 gRPC Server listening at 127.0.0.1:38383
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG [mpt] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [replit] Attempting to load
5:31PM DBG Loading model replit from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model replit: {backendString:replit modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:38383): stderr mpt_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:38383): stderr mpt_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/replit
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:33943'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager847316612
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:33943: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33943): stderr 2023/11/09 17:31:27 gRPC Server listening at 127.0.0.1:33943
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33943): stderr replit_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:33943): stderr replit_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG [replit] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [starcoder] Attempting to load
5:31PM DBG Loading model starcoder from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model starcoder: {backendString:starcoder modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/starcoder
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:38617'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager1188628188
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:38617: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:38617): stderr 2023/11/09 17:31:28 gRPC Server listening at 127.0.0.1:38617
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:38617): stderr starcoder_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:38617): stderr starcoder_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG [starcoder] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [bloomz] Attempting to load
5:31PM DBG Loading model bloomz from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model bloomz: {backendString:bloomz modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/bloomz
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:41523'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager1643108513
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:41523: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:41523): stderr 2023/11/09 17:31:29 gRPC Server listening at 127.0.0.1:41523
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:41523): stderr bloom_model_load: invalid model file '/models/luna-ai-llama2' (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:41523): stderr bloomz_bootstrap: failed to load model from '/models/luna-ai-llama2'
5:31PM DBG [bloomz] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
5:31PM DBG [rwkv] Attempting to load
5:31PM DBG Loading model rwkv from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model rwkv: {backendString:rwkv modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/rwkv
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:42799'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager880251401
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:42799: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr 2023/11/09 17:31:30 gRPC Server listening at 127.0.0.1:42799
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/go-rwkv/rwkv.cpp/rwkv.cpp:250: header.magic == 0x67676d66
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr Invalid file header
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/go-rwkv/rwkv.cpp/rwkv.cpp:1132: rwkv_fread_file_header(file.file, model.header)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/go-rwkv/rwkv.cpp/rwkv.cpp:1266: rwkv_instance_from_file(file_path, *instance.get())
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr panic: runtime error: invalid memory address or nil pointer dereference
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x524c54]
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr goroutine 50 [running]:
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr github.com/donomii/go-rwkv%2ecpp.(*Context).GetStateBufferElementCount.func1(0xc0002041d0?)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/go-rwkv/wrapper.go:63 +0x14
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr github.com/donomii/go-rwkv%2ecpp.(*Context).GetStateBufferElementCount(0xc000214030?)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/go-rwkv/wrapper.go:63 +0x19
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr github.com/donomii/go-rwkv%2ecpp.LoadFiles({0xc000214030?, 0xc000214038?}, {0xc000266030, 0x25}, 0x53?)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/go-rwkv/wrapper.go:131 +0x5d
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr github.com/go-skynet/LocalAI/pkg/grpc/llm/rwkv.(*LLM).Load(0xc0000d8290, 0xc00022c140)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/pkg/grpc/llm/rwkv/rwkv.go:25 +0xcf
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr github.com/go-skynet/LocalAI/pkg/grpc.(*server).LoadModel(0x910940?, {0xc00022c140?, 0x5cb826?}, 0x0?)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/pkg/grpc/server.go:42 +0x28
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr github.com/go-skynet/LocalAI/pkg/grpc/proto._Backend_LoadModel_Handler({0x8ee3c0?, 0xc0000b9d10}, {0x9d1c10, 0xc0002003c0}, 0xc000208150, 0x0)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /build/pkg/grpc/proto/backend_grpc.pb.go:236 +0x170
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001741e0, {0x9d4898, 0xc0002b8340}, 0xc000232000, 0xc00017c9f0, 0xc86530, 0x0)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:1360 +0xe23
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr google.golang.org/grpc.(*Server).handleStream(0xc0001741e0, {0x9d4898, 0xc0002b8340}, 0xc000232000, 0x0)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:1737 +0xa36
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr google.golang.org/grpc.(*Server).serveStreams.func1.1()
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:982 +0x98
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr created by google.golang.org/grpc.(*Server).serveStreams.func1
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:42799): stderr /go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:980 +0x18c
5:31PM DBG [rwkv] Fails: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF
5:31PM DBG [whisper] Attempting to load
5:31PM DBG Loading model whisper from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model whisper: {backendString:whisper modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/whisper
5:31PM DBG GRPC Service for luna-ai-llama2 will be running at: '127.0.0.1:35467'
5:31PM DBG GRPC Service state dir: /tmp/go-processmanager3358758835
5:31PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:35467: connect: connection refused"
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:35467): stderr 2023/11/09 17:31:31 gRPC Server listening at 127.0.0.1:35467
5:31PM DBG GRPC Service Ready
5:31PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:/models/luna-ai-llama2 ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:1 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0}
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:35467): stderr whisper_init_from_file_no_state: loading model from '/models/luna-ai-llama2'
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:35467): stderr whisper_model_load: loading model
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:35467): stderr whisper_model_load: invalid model data (bad magic)
5:31PM DBG GRPC(luna-ai-llama2-127.0.0.1:35467): stderr whisper_init_no_state: failed to load model
5:31PM DBG [whisper] Fails: could not load model: rpc error: code = Unknown desc = unable to load model
5:31PM DBG [stablediffusion] Attempting to load
5:31PM DBG Loading model stablediffusion from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model stablediffusion: {backendString:stablediffusion modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG [stablediffusion] Fails: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/stablediffusion. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
5:31PM DBG [piper] Attempting to load
5:31PM DBG Loading model piper from luna-ai-llama2
5:31PM DBG Loading model in memory from file: /models/luna-ai-llama2
5:31PM DBG Loading GRPC Model piper: {backendString:piper modelFile:luna-ai-llama2 threads:1 assetDir:/tmp/localai/backend_data context:0xc00003e0b0 gRPCOptions:0xc0006c40a0 externalBackends:map[huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py]}
5:31PM DBG [piper] Fails: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/piper. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
5:31PM DBG [/build/extra/grpc/huggingface/huggingface.py] Attempting to load
5:31PM DBG Loading model /build/extra/grpc/huggingface/huggingface.py from luna-ai-llama2
5:31PM DBG [/build/extra/grpc/huggingface/huggingface.py] Fails: backend unsupported: /build/extra/grpc/huggingface/huggingface.py
[172.17.0.1]:58330  500  -  POST  /v1/chat/completions
[127.0.0.1]:34174  200  -  GET  /readyz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment