Skip to content

Instantly share code, notes, and snippets.

@JoshVarty
Last active February 27, 2018 05:36
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save JoshVarty/0d8ad5f19a8ae7b2736c602ed563e7bf to your computer and use it in GitHub Desktop.
Save JoshVarty/0d8ad5f19a8ae7b2736c602ed563e7bf to your computer and use it in GitHub Desktop.
velocity = 0 #No initial velocity. (Defined outside of optimization loop)
...
momentum = 0.9
learning_rate = 0.01 #Some human-chosen learning rate
gradient_for_weight_1 = ... #Compute gradient
velocity = (momentum * velocity) - (gradient_for_weight_1 * learning_rate) #Maintain a velocity that keeps increasing if we don't change direction
weight_1 = weight_1 + velocity
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment