push bnb 4 bit models on the hub

bnb-serialization.py

# pip install -U bitsandbytes
# pip install -U git+https://github.com/huggingface/transformers.git
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
path_to_hub = XXX
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)
model.push_to_hub(path_to_hub)
tokenizer.push_to_hub(path_to_hub)

What version of transformers are you using?
I tried to push lite llama 4bit but throwing the same error as 4bit models cannot be pushed to hub

I ran into this problem then fixed it by running these (running colab nb):

!pip install git+https://github.com/huggingface/transformers.git -q -U # transformers version:  4.37.0
!pip install git+https://github.com/huggingface/accelerate.git -q -U # accelerate version:  0.27.0
!pip install bitsandbytes # bitsandbytes version:  0.42.0
!pip install git+https://github.com/huggingface/peft.git -q -U # peft version: 0.7.2

hope this helps

Hey Thanks a lot it worked

Facing this issue
RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM:
size mismatch for model.layers.0.mlp.gate_proj.weight: copying a param with shape torch.Size([2097152, 1]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).