Skip to content

Instantly share code, notes, and snippets.

@ronykroy

ronykroy/UlMFIT_Classfier1.py

Last active October 22, 2019 09:53
Show Gist options
  • Save ronykroy/1b2f00858322ca4d88f4bbb1d08878e7 to your computer and use it in GitHub Desktop.
Save ronykroy/1b2f00858322ca4d88f4bbb1d08878e7 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
UlMFIT_Classfier1.py
data_clas = TextClasDataBunch.from_df(path = path_d, train_df = df_trn, valid_df = df_val, test_df=df_test, vocab=data_lm.train_ds.vocab, bs=bs)
learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.5)
learn.load_encoder('fine_tuned_enc') # load th encoder from the LM
learn.lr_find() # find an LR
learn.recorder.plot()
# Training
learn.fit_one_cycle(3, 1e-2, moms=(0.8,0.7)) # you can of course train more, Jeremy promises its hard to over fit here :D
learn.freeze_to(-2) # unfreeze last 2 layers
learn.fit_one_cycle(2, slice(1e-2/(2.6**4),1e-2), moms=(0.8,0.7))
learn.freeze_to(-3) # unfreeze last 3 layers
learn.fit_one_cycle(2, slice(5e-3/(2.6**4),5e-3), moms=(0.8,0.7))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment