Skip to content

Instantly share code, notes, and snippets.

View pdtgct's full-sized avatar

Pete Tanski pdtgct

View GitHub Profile
pdtgct /
Created April 28, 2023 15:27
Convert HF to GGML

The LLaMA model weights may be converted from Huggingface PyTorch format back to GGML in two steps:

  1. download from decapoda-research/llama-7b-hf and save as pytorch .pth
  2. use the ggerganov/llama.cpp script, to convert from pytorch .pth to GGML

This process will result in ggml model with float16 (fp16) precision.