Skip to content

Instantly share code, notes, and snippets.

@shahules786

shahules786/llrd.py

Created September 24, 2021 04:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shahules786/1cf3bd3f987f91a012edcf2c10c32bf9 to your computer and use it in GitHub Desktop.
Save shahules786/1cf3bd3f987f91a012edcf2c10c32bf9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
llrd.py
def llrd(model,peak_lr,multiplicative factor):
"""sets learning rate for each layer and returns the parameters"""
parameters = get_model_parameters()
return parameters
model = DefinedModel()
layerwise_ll = llrd(model,1e-5,0.95)
optimizer = Adam(layerwise_ll)
dataloaders = get_train_valid_data()
train_fn(model,dataloaders,optimizer)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment