Skip to content

Instantly share code, notes, and snippets.

@markddesimone

markddesimone/Simple-Train-of-New-Dense-Layers.ipynb Secret

Last active Dec 9, 2017
Star
Embed
What would you like to do?
Learn more about clone URLs
Download ZIP
Simple-Train-of-New-Dense-Layers.ipynb not training!
Raw
Simple-Train-of-New-Dense-Layers.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@markddesimone
Copy link
Author

markddesimone commented Dec 9, 2017

Solved

So the problem was the learning rate. It was too high. I had to reduce the learning rate to 0.001 or better still 0.0001 to get good results
Here are the 50 iterations with lr=0.001
image

note the convergence is jumping around
0.0001 give a better result and is overfitting the sample data set as expected:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment