Skip to content

Instantly share code, notes, and snippets.

@dadeba

dadeba/memo.txt

Created May 22, 2023 07:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dadeba/b5aaa2f4eae111bab39f3e3e275e1260 to your computer and use it in GitHub Desktop.
Save dadeba/b5aaa2f4eae111bab39f3e3e275e1260 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
memo.txt
# build
git clone https://github.com/dadeba/ggml
cd ggml
git checkout gpt-neox
mkdir build
cd build
cmake ..
make
cd ..
# convert
mkdir open-calm-small
python3 ./examples/gpt-neox/convert-h5-to-ggml.py cyberagent/open-calm-small 1 open-calm-small
./build/bin/gpt-neox-quantize open-calm-small/ggml-model-f16.bin open-calm-small/ggml-model-q4.bin 3
./build/bin/gpt-neox -m open-calm-small/ggml-model-q4.bin -p "hello world"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment