Skip to content

Instantly share code, notes, and snippets.

Last active Feb 27, 2018
What would you like to do?
velocity = 0 #No initial velocity. (Defined outside of optimization loop)
momentum = 0.9
learning_rate = 0.01 #Some human-chosen learning rate
gradient_for_weight_1 = ... #Compute gradient
velocity = (momentum * velocity) - (gradient_for_weight_1 * learning_rate) #Maintain a velocity that keeps increasing if we don't change direction
weight_1 = weight_1 + velocity
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment