Skip to content

Instantly share code, notes, and snippets.

@antferdom

antferdom/transformer_parameters.py

Last active September 15, 2023 09:28
Show Gist options
  • Save antferdom/d2edec70a769aa5ec23e1a1f72c25525 to your computer and use it in GitHub Desktop.
Save antferdom/d2edec70a769aa5ec23e1a1f72c25525 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
transformer_parameters.py
import torch
model_size = sum(
p.numel() * p.element_size() for p in model.parameters() if p.requires_grad
)
model_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment