Last active
January 19, 2018 06:23
-
-
Save infinite-Joy9l/404041c90d489ea66d8f524e643d8170 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"cells": [ | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"This function creates the gradient component for each Theta value. The gradient is the partial derivative by Theta of the current value of theta, minus a \"learning speed factor alpha\" times the average of all the cost functions for that theta. For each Theta there is a cost function calculated for each member of the dataset." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 12, | |
"metadata": { | |
"collapsed": true | |
}, | |
"outputs": [], | |
"source": [ | |
"def Cost_Function_Derivative(X,Y,theta,j,m,alpha):\n", | |
" sumErrors = 0\n", | |
" for i in range(m):\n", | |
" xi = X[i]\n", | |
" xij = xi[j]\n", | |
" hi = Hypothesis(theta,X[i])\n", | |
" error = (hi - Y[i])*xij\n", | |
" sumErrors += error\n", | |
" m = len(Y)\n", | |
" constant = float(alpha)/float(m)\n", | |
" J = constant * sumErrors\n", | |
" return J" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"For each theta, the partial differential. The gradient, or vector from the current point in Theta-space (each theta value is its own dimension) to the more accurate point, is the vector with each dimensional component being the partial differential for each theta value" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 13, | |
"metadata": { | |
"collapsed": true | |
}, | |
"outputs": [], | |
"source": [ | |
"def Gradient_Descent(X,Y,theta,m,alpha):\n", | |
" new_theta = []\n", | |
" constant = alpha/m\n", | |
" for j in range(len(theta)):\n", | |
" CFDerivative = Cost_Function_Derivative(X,Y,theta,j,m,alpha)\n", | |
" new_theta_value = theta[j] - CFDerivative\n", | |
" new_theta.append(new_theta_value)\n", | |
" return new_theta" | |
] | |
} | |
], | |
"metadata": { | |
"kernelspec": { | |
"display_name": "Python 3", | |
"language": "python", | |
"name": "python3" | |
}, | |
"language_info": { | |
"codemirror_mode": { | |
"name": "ipython", | |
"version": 3 | |
}, | |
"file_extension": ".py", | |
"mimetype": "text/x-python", | |
"name": "python", | |
"nbconvert_exporter": "python", | |
"pygments_lexer": "ipython3", | |
"version": "3.6.1" | |
} | |
}, | |
"nbformat": 4, | |
"nbformat_minor": 2 | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment