Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
# optimizer from hugging face transformers
from transformers import AdamW
# define the optimizer
optimizer = AdamW(model.parameters(),
lr = 1e-5) # learning rate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment