Skip to content

Instantly share code, notes, and snippets.

@chauhan-utk

chauhan-utk/Gradients_hooks.md

Last active February 10, 2018 05:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chauhan-utk/ed9d5b91d2532251e9afbb7640c1e7ad to your computer and use it in GitHub Desktop.
Save chauhan-utk/ed9d5b91d2532251e9afbb7640c1e7ad to your computer and use it in GitHub Desktop.
Download ZIP
Pytorch important
Raw
Gradients_hooks.md

How to modify the gradients in the backward pass? Using a register_hook for saving the gradients of a Variable: link When we want to get more crazy with the gradients and want only one gradient computation to be done from two heads

    zz1
   /   
yy
   \
     zz2

Want gradient to flow backward from only zz1 part: link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment