Skip to content

Instantly share code, notes, and snippets.

@dyborg-one
dyborg-one / distill_122b.py
Last active March 12, 2026 14:37
Distill Opus reasoning into Qwen3.5-122B-A10B (MoE) using Unsloth. 2x B200 (384GB VRAM). Uses Jackrong's published datasets.
#!/usr/bin/env python3
"""
Distill Opus reasoning into Qwen3.5-122B-A10B (MoE) using Unsloth.
Hardware: 4x H200 (143GB each = 575GB total)
Data: Jackrong's published distillation datasets
Method: bf16 LoRA + train_on_responses_only (matching Jackrong's 27B recipe)
Usage:
pip install --upgrade unsloth unsloth_zoo