Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This function creates the gradient component for each Theta value. The gradient is the partial derivative by Theta of the current value of theta, minus a \"learning speed factor alpha\" times the average of all the cost functions for that theta. For each Theta there is a cost function calculated for each member of the dataset."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def Cost_Function_Derivative(X,Y,theta,j,m,alpha):\n",
" sumErrors = 0\n",
" for i in range(m):\n",
" xi = X[i]\n",
" xij = xi[j]\n",
" hi = Hypothesis(theta,X[i])\n",
" error = (hi - Y[i])*xij\n",
" sumErrors += error\n",
" m = len(Y)\n",
" constant = float(alpha)/float(m)\n",
" J = constant * sumErrors\n",
" return J"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For each theta, the partial differential. The gradient, or vector from the current point in Theta-space (each theta value is its own dimension) to the more accurate point, is the vector with each dimensional component being the partial differential for each theta value"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def Gradient_Descent(X,Y,theta,m,alpha):\n",
" new_theta = []\n",
" constant = alpha/m\n",
" for j in range(len(theta)):\n",
" CFDerivative = Cost_Function_Derivative(X,Y,theta,j,m,alpha)\n",
" new_theta_value = theta[j] - CFDerivative\n",
" new_theta.append(new_theta_value)\n",
" return new_theta"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.