Skip to content

Instantly share code, notes, and snippets.

@zoff99

zoff99/ll2

Created September 29, 2023 08:18
Show Gist options
  • Save zoff99/80246aca892d43ef323455036aedb08a to your computer and use it in GitHub Desktop.
Save zoff99/80246aca892d43ef323455036aedb08a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
ll2
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp/
make -j$(nproc)
cd models/
wget https://huggingface.co/TheBloke/Uncensored-Frank-13b-GGUF/resolve/main/uncensored-frank-13b.Q2_K.gguf
cd ../
# until here no big thing
##############
# now start the interactive prompt, it will use RAM and CPU (about 10 GByte RAM, and all CPU you have)!!
##############
./main -m models/uncensored-frank-13b.Q2_K.gguf --log-disable --ctx_size 4096 -n -1 -ins -b 256 --top_k 10000 --temp 0.2 --repeat_penalty 1.1 -t 8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment