Skip to content

Instantly share code, notes, and snippets.

@markddesimone

markddesimone/Simple-Train-of-New-Dense-Layers.ipynb Secret

Last active December 9, 2017 19:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save markddesimone/3501f9811f00562eb1ba831ced2c90f5 to your computer and use it in GitHub Desktop.
Save markddesimone/3501f9811f00562eb1ba831ced2c90f5 to your computer and use it in GitHub Desktop.
Download ZIP
Simple-Train-of-New-Dense-Layers.ipynb not training!
Display the source blob
Display the rendered blob
Raw
Simple-Train-of-New-Dense-Layers.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@markddesimone
Copy link
Author

Solved

So the problem was the learning rate. It was too high. I had to reduce the learning rate to 0.001 or better still 0.0001 to get good results
Here are the 50 iterations with lr=0.001
image

note the convergence is jumping around
0.0001 give a better result and is overfitting the sample data set as expected:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment