Skip to content

Instantly share code, notes, and snippets.

@zucchini-nlp

zucchini-nlp/update_blip.py

Last active June 10, 2024 07:54
Show Gist options
  • Save zucchini-nlp/e9f20b054fa322f84ac9311d9ab67042 to your computer and use it in GitHub Desktop.
Save zucchini-nlp/e9f20b054fa322f84ac9311d9ab67042 to your computer and use it in GitHub Desktop.
Download ZIP
Update BLIP-2 model for new version
Raw
update_blip.py
# Load your model and processor and run the following to update BLIP-2 model
# It will update file in your repo by adding new args in configs and resizing embedding layer
# Then you'll be able to run BLIP-2 without warnings/errors
processor.num_query_tokens = model.config.num_query_tokens
model.resize_token_embeddings(processor.tokenizer.vocab_size, pad_to_multiple_of=64) # pad for efficient computation
model.config.image_token_index = processor.tokenizer.vocab_size
model.push_to_hub("YOUR-REPO")
processor.push_to_hub("YOUR-REPO")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment