Skip to content

Instantly share code, notes, and snippets.

@yaroslavvb2

yaroslavvb2/optimizer_step.py

Last active October 22, 2017 20:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save yaroslavvb2/383dd9620476733de6eef5762282e4d4 to your computer and use it in GitHub Desktop.
Save yaroslavvb2/383dd9620476733de6eef5762282e4d4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
optimizer_step.py
mode = 'capture'
loss.backward()
# invert
for i in range(n):
As_inv.append(regularized_inverse(As[i] @ As[i].t()))
Bs_inv.append(regularized_inverse(Bs[i] @ Bs[i].t()))
mode = 'kfac'
loss.backward()
optimizer.step()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment