Skip to content

Instantly share code, notes, and snippets.

@fakerybakery

fakerybakery/README.md

Created April 10, 2024 02:05
Show Gist options
  • Save fakerybakery/321ffcbfbcce1bafe7ba815476849119 to your computer and use it in GitHub Desktop.
Save fakerybakery/321ffcbfbcce1bafe7ba815476849119 to your computer and use it in GitHub Desktop.
Download ZIP
How to convert Mixtral 8x22B to HF
Raw
README.md

How to convert Mixtral 8x22B to HF

You'll need:

  • 600GB disk space (the model itself is ~280, then you'll convert it to HF which will double it)
  • High bandwidth

First, download the Mixtral model using a torrent client:

magnet:?xt=urn:btih:9238b09245d0d8cd915be09927769d5f7584c1c9&dn=mixtral-8x22b&tr=udp%3A%2F%http://2Fopen.demonii.com%3A1337%2Fannounce&tr=http%3A%2F%http://2Ftracker.opentrackr.org%3A1337%2Fannounce

Then, run:

git clone https://github.com/huggingface/transformers
cd transformers
pip install -e .

Now, convert the Mixtral model to HF. Move the downloaded Mixtral model to the mixtral directory in the transformers directory, modify src/transformers/models/mixtral/convert_mixtral_weights_to_hf.py line 228 (replace 7B w/ 22B) and run:

python src/transformers/models/mixtral/convert_mixtral_weights_to_hf.py --input_dir mixtral --model_size 22B --output_dir hf_mixtral

Then upload to HF!

NOTE: This has not been tested

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment