MLX Fine-tuning Google Gemma

mlx_finetuning_gemma.ipynb

This was super helpful!

One change might be needed:

!python -m mlx_lm.generate --model "google/gemma-7b-it"
--adapter-file checkpoints/600_adapters.npz
--max-tokens 256
--prompt "Why is the sky blue?"
--seed 69

@johngroves glad to hear!
You're right, thanks for pointing that out. Just updated that.

In my M1 with 8 Gb I had to change some things, the first one to go for a smaller model, for example the training should be like this

!TOKENIZERS_PARALLELISM=false python -m mlx_lm.lora
--model "google/gemma-3-1b-it"
--train
--iters 200
--data data
--adapter-path ./checkpoints
--save-every 100
--batch-size 1
--grad-checkpoint

and then loading the latest checkpoint should be like this

# Load the fine-tuned model with LoRA weights

model_lora, _ = load(
"google/gemma-3-1b-it",
adapter_path="./checkpoints", # adapters.npz is the final checkpoint saved at the end of training
)