Skip to content

Instantly share code, notes, and snippets.

@Quentin-Anthony

Quentin-Anthony/megatron-checks.md

Last active June 7, 2022 05:30
Show Gist options
  • Save Quentin-Anthony/ed2082202b79451a15e946db174b4dbc to your computer and use it in GitHub Desktop.
Save Quentin-Anthony/ed2082202b79451a15e946db174b4dbc to your computer and use it in GitHub Desktop.
Download ZIP
Megatron Spreadsheet
Raw
megatron-checks.md
ZeRO Stage Data-Parallel MP PP MP+PP MoE MoE+MP
1 ✓ ✓ ✓ ✓ ✓ ✓
2 ✓ ✓ N/A N/A ✓ ✓
3 ✓ ✓ N/A N/A N/A N/A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment