{ | |
"cells": [ | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"This function creates the gradient component for each Theta value. The gradient is the partial derivative by Theta of the current value of theta, minus a \"learning speed factor alpha\" times the average of all the cost functions for that theta. For each Theta there is a cost function calculated for each member of the dataset." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 12, | |
"metadata": { | |
"collapsed": true | |
}, | |
"outputs": [], | |
"source": [ | |
"def Cost_Function_Derivative(X,Y,theta,j,m,alpha):\n", | |
" sumErrors = 0\n", | |
" for i in range(m):\n", | |
" xi = X[i]\n", | |
" xij = xi[j]\n", | |
" hi = Hypothesis(theta,X[i])\n", | |
" error = (hi - Y[i])*xij\n", | |
" sumErrors += error\n", | |
" m = len(Y)\n", | |
" constant = float(alpha)/float(m)\n", | |
" J = constant * sumErrors\n", | |
" return J" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"For each theta, the partial differential. The gradient, or vector from the current point in Theta-space (each theta value is its own dimension) to the more accurate point, is the vector with each dimensional component being the partial differential for each theta value" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 13, | |
"metadata": { | |
"collapsed": true | |
}, | |
"outputs": [], | |
"source": [ | |
"def Gradient_Descent(X,Y,theta,m,alpha):\n", | |
" new_theta = []\n", | |
" constant = alpha/m\n", | |
" for j in range(len(theta)):\n", | |
" CFDerivative = Cost_Function_Derivative(X,Y,theta,j,m,alpha)\n", | |
" new_theta_value = theta[j] - CFDerivative\n", | |
" new_theta.append(new_theta_value)\n", | |
" return new_theta" | |
] | |
} | |
], | |
"metadata": { | |
"kernelspec": { | |
"display_name": "Python 3", | |
"language": "python", | |
"name": "python3" | |
}, | |
"language_info": { | |
"codemirror_mode": { | |
"name": "ipython", | |
"version": 3 | |
}, | |
"file_extension": ".py", | |
"mimetype": "text/x-python", | |
"name": "python", | |
"nbconvert_exporter": "python", | |
"pygments_lexer": "ipython3", | |
"version": "3.6.1" | |
} | |
}, | |
"nbformat": 4, | |
"nbformat_minor": 2 | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment