Simple Linear Regression using Gradient Descent algorithm (plain Python no lib required)

linear-regerssion-gradient-descent.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "**Simple Linear Regression using Gradient Descent algorithm**<br>\n",
    "Plain Python (vanilla version) i.e. no third party extension (or any library) needed\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "Hypothesis\n",
    "> $h_\\theta(x) = \\theta_0 + \\theta_1x$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def hypothesis(x):\n",
    "    return theta[0] + theta[1] * x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Loss (cost) function\n",
    "> $J(\\theta_0, \\theta_1) = \\frac{1}{2m}\\sum(h_\\theta(x^{(i)}) - y^{(i)})^2$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def loss(X, y):\n",
    "    m = len(X)\n",
    "    return sum([(X[i] - y[i])**2 for i in range(m)]) / (2 * m)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Gradient Descent algorithm \n",
    "> repeat until converge \n",
    "> {\n",
    "\n",
    ">>    $\\theta_0 = \\theta_0 - \\alpha \\frac{1}{m} \\sum_{i=1}^{m} (h_\\theta(x^{(i)}) - y^{(i)} )$\n",
    "\n",
    ">>    $\\theta_1 = \\theta_1 - \\alpha \\frac{1}{m} \\sum_{i=1}^{m} (h_\\theta(x^{(i)}) - y^{(i)}) x^{(i)})$\n",
    "\n",
    "> }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def gradientDescent(X, y, theta, num_iter=1000, alpha=0.01):\n",
    "\n",
    "    m = len(X)\n",
    "    \n",
    "    for j in range(num_iter):\n",
    "        \n",
    "        # predict\n",
    "        h = list(map(hypothesis, X))\n",
    "\n",
    "        # compute slope, aka derivative with current params (theta)\n",
    "        deri_th0 = sum([(h[i] - y[i]) for i in range(m)]) / m\n",
    "        deri_th1 = sum([(h[i] - y[i]) * X[i] for i in range(m)]) / m\n",
    "\n",
    "        # update parameters (moving against the gradient 'derivative')\n",
    "        theta[0] = theta[0] - alpha * deri_th0\n",
    "        theta[1] = theta[1] - alpha * deri_th1\n",
    "\n",
    "        # report\n",
    "        if j % 200 == 0:\n",
    "            print('loss: {}'.format(loss(h, y)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "**Test Run**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Sample Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# data (top 20 samples from ex1data1.txt)\n",
    "X = [6.1101, 5.5277, 8.5186, 7.0032, 5.8598, 8.3829, 7.4764, 8.5781, 6.4862, 5.0546,\n",
    "     5.7107, 14.164, 5.734, 8.4084, 5.6407, 5.3794, 6.3654, 5.1301, 6.4296, 7.0708]\n",
    "y = [17.592, 9.1302, 13.662, 11.854, 6.8233, 11.886, 4.3483, 12.0, 6.5987, 3.8166,\n",
    "     3.2522, 15.505, 3.1551, 7.2258, 0.71618, 3.5129, 5.3048, 0.56077, 3.6518, 5.3893]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss: 38.1423900131325\n",
      "loss: 7.444891107784142\n",
      "loss: 7.3770168292129155\n",
      "loss: 7.326846499294364\n",
      "loss: 7.289762318701628\n",
      "\n",
      "y = -1.464 + 1.276*X\n"
     ]
    }
   ],
   "source": [
    "theta = [0, 0] # initial values\n",
    "gradientDescent(X, y, theta) # run to optimize thetas\n",
    "print('\\ny = {:.3f} + {:.3f}*X'.format(theta[0], theta[1])) # print solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Plot result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgcAAAFkCAYAAAC0KZhSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xt0XXWd9/H3j4AXUGpEgUe5N0nB8QKtl8JAixJJTBVQ\nHJ20jY63R2ZkwM7j6LC8zKCzHMUFCsyIo8/jBSMH8YLANG0kKC2IHTQRcBQ5JwEERCuSWhDklv6e\nP3bCyQ5Jm+Rc9jkn79daWSa/fXL2N9uG/cl3//ZvhxgjkiRJE3bLugBJklRbDAeSJCnFcCBJklIM\nB5IkKcVwIEmSUgwHkiQpxXAgSZJSDAeSJCnFcCBJklIMB5IkKWXe4SCEcFYI4cYQwgMhhK0hhMtD\nCG1TXvP0EMJ/hBD+EEJ4MITw7RDCvqWXLUmSKqWUzsFxwIXAq4B2YA/g+yGEZ056zeeAVcCpwArg\nBcB3StinJEmqsFCuBy+FEJ4H/B5YEWO8PoSwN3Af8NcxxsvHX7MEuBVYHmO8sSw7liRJZVXOOQfP\nASIwOv71MmB34JqJF8QYbwPuAo4u434lSVIZ7V6ONwkhBJJLCNfHGH85Prw/8FiM8YEpL986vm26\n99kH6ADuBB4pR22SJC0QzwAOAfpjjPeX8kZlCQfA54EXAcfO4rWBpMMwnQ7gG2WqSZKkhWgNcEkp\nb1ByOAgh/DvQBRwXY7x30qbfAU8LIew9pXuwL0n3YDp3AvT29nLEEUeUWtqCsW7dOj772c9mXUbd\n8bjNncdsfjxuc+cxm7tbb72VtWvXwvi5tBQlhYPxYHAysDLGeNeUzYPAE8AJwMSExDbgIODHM7zl\nIwBHHHEES5cuLaW0BWXRokUer3nwuM2dx2x+PG5z5zErScmX5ecdDkIInwe6gZOAh0II+41v2h5j\nfCTG+EAI4f8B54UQtgEPAhcAP/JOBUmSalcpnYPTSOYOXDtl/B3AxeOfrwPGgG8DTwc2Au8rYZ+S\nJKnC5h0OYoy7vA0yxvgo8PfjH5IkqQ74bIUG0N3dnXUJdcnjNnces/nxuM2dxyxbZVshsRxCCEuB\nwcHBQSeiSJI0B0NDQyxbtgxgWYxxqJT3snMgSZJSDAeSJCnFcCBJklIMB5IkKcVwIEmSUgwHkiQp\nxXAgSZJSDAeSJCnFcCBJklIMB5IkKcVwIEmSUgwHkiQpxXAgSZJSDAeSJCnFcCBJklIMB5IkKcVw\nIEmSUgwHkiQpZfesC1D55fN5RkZGaGlpobW1NetyJEl1xs5BAxkdHaWzcxVLliyhq6uLtrY2OjtX\nsW3btqxLkyTVEcNBA1m9uoeBgS1AL3AX0MvAwBa6u9dmXJkkqZ54WaFB5PN5+vv7SILBmvHRNYyN\nRfr7eygUCl5ikCTNip2DBjEyMjL+2YopW1YCMDw8XNV6JEn1y3DQIBYvXjz+2eYpWzYB0NLSUtV6\nJEn1y3DQINra2ujo6KKp6QySSwt3A700NZ1JR0eXlxQkSbNmOGgguVwv7e3LgR7gIKCH9vbl5HK9\nGVcmSaonTkhsIM3NzWzcuJ5CocDw8LDrHEiS5sVw0IBaW1sNBZKkefOygiRJSjEcSJKkFMOBJElK\nMRxIkqQUw4EkSUoxHEiS6sbtt8Pjj2ddReMzHEiSatpDD8Gpp0IIsHgxXH551hU1Ptc5kCTVpE2b\n4Pjj02MnnQRvelMm5Swodg4kSTXjscfgb/826RJMDgaXXQYxwhVXwO7+WVtxHmJJUuZuvhmOOQYe\nfrg4dvTR8L3vwb77ZlfXQmXnQJKUiR074GMfS7oERx5ZDAYXXZR0CW64wWCQFTsHkqSquv12eM1r\n4Ne/Lo61tMDVV8Mhh2RWliaxcyBJqrgY4cILi3ccTASDs8+GsTEoFAwGtcTOgSSpYrZuhde/Hn76\n0+LY3nvD9dfDS16SXV3aOTsHkqSyu/TSpEuw//7FYHD66cndCNu3GwxqnZ0DSVJZPPAAdHdDX196\n/Lrr4Nhjs6lJ82PnQJJUkquvTroEixYVg8Fb35qsbBhjeYNBPp9nw4YNFAqF8r2pnsJwIEmas0cf\nhXe8IwkFJ55YHL/yyiQQXHop7Lln+fY3OjpKZ+cqlixZQldXF21tbXR2rmLbtm3l24meZDiQJM3a\nT34CTU3wjGfAV7+ajL361XD//UkoeMMbKrPf1at7GBjYAvQCdwG9DAxsobt7bWV2uMAZDiRJOzU2\nBh/8YNIleOUrk8WLAL785SQQ/OAH8NznVm7/+Xye/v4+xsYuANYABwJrGBs7n/7+Pi8xVIATEiVJ\n07rtNli5MrkdccKLXwwbNsABB1SvjpGRkfHPVkzZshKA4eFhWltbq1fQAjDvzkEI4bgQwpUhhN+E\nEHaEEE6asv0r4+OTP/pmej9JUvZihM98JukSHH54MRicc07SMfj5z6sbDAAWL148/tnmKVs2AdDS\n0lLVehaCUjoHewE3AV8GvjPDazYAfwOE8a8fLWF/kqQK+c1voLMT/ud/imP77ps8Nvnww7OrC6Ct\nrY2Oji4GBs5gbCySdAw20dR0Ju3tXXYNKmDenYMY48YY48dijN+jePKf6tEY430xxt+Pf2yf7/4k\nSeX31a8mXYIDDigGgw98AJ54IukaZB0MJuRyvbS3Lwd6gIOAHtrbl5PL9WZcWWOq9JyD40MIW4Ft\nwA+Aj8QYRyu8T0nSToyOwpvfDD/8YXEsBNiyJZlwWIuam5vZuHE9hUKB4eFhWlpa7BhUUCXDwQaS\nyw13AIuBfwP6QghHxxhjBfcrSZrGVVfBSSelx97+dvjCF5JbE+tBa2uroaAKKhYOYoyXTfryFyGE\nnwMjwPHAD6f9pnHr1q1j0aJFqbHu7m66u7vLXaYkNbSHH4Z3vhO++c30eH9/evEi1ZdcLkcul0uN\nbd9eviv3oRx/xIcQdgCnxBiv3MXrfg98OMb4pRm2LwUGBwcHWbp0acl1SdJCdf31cNxx6bGuLrjk\nkmSZYzWeoaEhli1bBrAsxjhUyntVbRGkEMIBwD7Ab6u1T0laSB5/HP7+75P5A5ODwSWXJLcorl9v\nMNDszPuyQghhL6CF4p0Kh4UQXgaMjn/8M8mcg9+Nv+7TQB7oL6VgSVLaz3+ePNzogQeKY694RfKc\ng/33z64u1a9SOgcvB34GDAIROBcYAs4GxoCXAlcAtwFfAn4CrIgxPl5KwZKkZEGij3886RK89KXF\nYHDhhcm2G280GGj+5t05iDFuYufhonO+7y1Jmt6dd0J7Ozy5ojBw8MFwzTXw5EKCUol88JIk1YGL\nLkq6BIceWgwGH/1o8lCkO+80GKi8fPCSJNWo++5L1iXYsqU4tuee8KMfwZFHZleXGp+dA0mqMd/+\ndtIl2HffYjA47TR49FF46CGDgSrPzoEk1YAHH4S1a5M7DCa79trksclSNdk5kKQM/eAHSZdg772L\nweBNb0rCQowGA2XDcCBJVfboo/Dudyeh4IQTiuPf/W4SCL7zHXjWs7KrT/KygiRVydAQLF+erGQ4\n4bjjklDwvOdlV5c0lZ0DqQzy+TwbNmygUChkXYpqzNgYnHVW0iVYtqwYDL74xaRLsHmzwUC1x86B\nVILR0VFWr+6hv7/vybGOji5yuV6am5szrExZKxTg+OPh3nuLY4cfnjwN8aCDMitLmhU7B1IJVq/u\nYWBgC9AL3AX0MjCwhe7utRlXpizECOedl3QJ2tqKweCTn0yWNL71VoOB6oOdA2me8vn8eMegF1gz\nPrqGsbFIf38PhUKB1tbWDCtUtdx7b/I45JtvLo4997lw3XXwohdlV5c0X3YOpHkaeXJx+xVTtiT3\nng0PD1e1HlVfb2/SJXjhC4vB4P3vT+YV3H+/wUD1y86BNE+Ln1zMfjPFzgHAJgBaWlqqXZKq4I9/\nhLe8Ba6+Oj1+ww1w9NHZ1CSVm50DaZ7a2tro6OiiqekMkksLdwO9NDWdSUdHl5cUGsyGDUmXoLm5\nGAzWrIGHH07mGhgM1EgMB1IJcrle2tuXAz3AQUAP7e3LyeV6M65M5fDnPydLGoeQzCmYsH59Egh6\ne+GZz8yuPqlSvKwglaC5uZmNG9dTKBQYHh6mpaXFjkED+PGP4Zhj0mMnngjf/CY85znZ1CRVk+FA\nKoPW1lZDQZ174gn4x3+Ez30uPX7xxdDTk01NUlYMB5IWtF/+MlnCeHS0OHbUUfBf/wUveEF2dUlZ\ncs6BpAUnxmRhohDgL/6iGAzOOy9ZrGhoyGCghc3OgaQF4667krkDt91WHHvhC+GHPwSvCklFdg4k\nNbwvfSnpEhx8cDEYnHVWMs/gnnsMBtJUdg4kNaQ//AHe+Ea4/vri2B57JHciLFuWXV1SPbBzIKmh\nXH550iV4/vOLweBd74JHHoHHHjMYSLNh50BS3XvoIXjb2+C7302PDwzACSdkU5NUzwwHkurWpk1w\n/PHpsZNOSlYufPazMylJagheVpBUVx57DE47Lbl0MDkYXHZZcoviFVcYDKRS2TmQVBduuilZ0vjP\nfy6OHX10Egae//zs6pIakZ0DSTVrxw746EeTLsFRRxWDwUUXJV2CG24wGEiVYOdAKqN8Ps/IyIgP\nYCrRyAi85jXJokUTWlqSRyUfckhmZUkLhp0DVUw+n2fDhg0UCoWsSymr6X6u0dFROjtXsWTJErq6\numhra6OzcxXbtm3LsNL6EiNccEHSJWhpKQaDs8+GsTEoFAwGUrUYDlR2jXqi3NnPtXp1DwMDW4Be\n4C6gl4GBLXR3r8246tq3dSu84hWw225w5pnJ2N57wy23JIHhYx9LtkmqHn/lVHaNeqKc6ec66aRT\n6O/vY2zsAmANcCCwhrGx8+nv72u4zkm5XHpp0iXYf3/46U+TsdNPT+5G2L4dXvKSbOuTFjLnHKis\n8vk8/f19JCfQNeOjaxgbi/T391AoFOryWvzOfq7rr+8Z/3rFlO9aCcDw8HBd/syVsH07dHfDhg3p\n8euug2OPzaYmSU9l50BlNTIyMv7ZzCfKerSrnyuxecq2TQC0tLRUqKr68f3vJ12C5zynGAze+tZk\nZcMYDQZSrTEcqKwWL148/lljnSh39XMdd9xKmprOIOks3A300tR0Jh0dXQu2a/DII/D2tyehoKOj\nOH7llUkguPRS2HPP7OqTNDPDgcqqra2Njo6uhjtR7urnuuKKy2lvXw70AAcBPbS3LyeX682y7Ezc\neGMSCJ75TLj44mTsNa+B++9PQsEb3pBtfZJ2zXCgssvlehvyRLmzn6u5uZmNG9eTz+fp6+sjn8+z\nceN6mpubM666OsbG4AMfSELBq15VHP/yl5NAcM018NznZlefpLlxQqLKbuJEWSgUGB4ebpgFgWbz\nc7W2tjbEzzpbv/oVrFgB991XHHvxi5N5BQcckF1dkkpjOFDFNOqJslF/rtmKET7zGfjQh9Lj55xT\n7B5Iqm+GA0mzcs890NkJv/hFcWzffZPHJh9+eHZ1SSo/5xxI2qmvfCXpBhx4YDEYfOAD8MQTyeqG\nBgOp8dg5kPQUo6Pw5jfDD39YHAsBtmyBV74yu7okVYedA0lPuuqqJATss08xGLz97cmjknfsMBhI\nC4WdA2mBe/hheMc74LLL0uP9/XDiidnUJClbhgNpgbr+ejjuuPRYVxdccgksWpRNTZJqg5cVpAXk\n8ceTJx+GkA4GuVxyi+L69QYDSXYOpAXh5z+Hv/xLePDB4tgrXpHMMdhvv+zqklSb7BxIDWrHDjj7\n7KRL8NKXFoPBhRcm22680WAgaXp2DqQGc+edcMIJcPvtxbGDD06eb/DkwyUlaSfsHEgN4qKLki7B\noYcWg8FHP5o8FOnOOw0GkmbPzoFUx37/ezj55GRxogl77gk/+hEceWR2dUmqb/PuHIQQjgshXBlC\n+E0IYUcI4aRpXvPxEMK9IYSHQwhXhxBaSitXEsC3vpV0CfbbrxgMTjsNHn0UHnrIYCCpNKVcVtgL\nuAl4HxCnbgwhfAg4HXgv8ErgIaA/hPC0EvYpLVgPPggnnZSEgre8pTh+7bXJbYgXXQRP87dLUhnM\n+7JCjHEjsBEghGkf0nom8IkY41Xjr3kbsBU4BbhsmtdLmsY110B7e3rs1FPha1+DvfbKpiZJja0i\nExJDCIcC+wPXTIzFGB8A/hs4uhL7lBrJo4/Cu96VdAkmB4PLL0+6BN/+tsFAUuVUakLi/iSXGrZO\nGd86vk3SNAYHYfny5HHIE1asgO9+N3kYkiRVQ7XvVghMMz9hqnXr1rFoyhqu3d3ddHd3V6ouKTNj\nY/DhD8OnP50e/+IX4T3vyaYmSbUtl8uRy+VSY9u3by/b+4cYd3mu3vWbhLADOCXGeOX414cCI8CR\nMcZbJr3uWuBnMcZ1M7zPUmBwcHCQpUuXllyXVMsKBVi5En772+LY4YcnT0M86KDs6pJUn4aGhli2\nbBnAshjjUCnvVZE5BzHGO4DfASdMjIUQ9gZeBdxQiX1K9SBGOPfcZC5BW1sxGHzyk8mSxrfeajCQ\nlL15X1YIIewFtJBcKgA4LITwMmA0xng38DngIyGEYeBO4BPAPcAVJVUs1aF7700eh3zzzcWx5z4X\nrrsOXvSi7OqSpOmU0jl4OfAzYJBkHsG5wBBwNkCM8RzgQuA/Se5SeCbwuhjjY6UULNWTr3896RK8\n8IXFYPD+9yePTr7/foOBpNpUyjoHm9hFuIgx/gvwL/Pdh1SP/vhH+Ku/goGB9PgNN8DR3sgrqQ74\n4CWpTE4/PekSNDcXg8GaNfDww8lcA4OBpHrhg5cyks/nGRkZoaWlhdbW1qzL0Tzde29yyWCqvj54\n3euqX48klYOdgyobHR2ls3MVS5Ysoauri7a2Njo7V7Ft27asS9McvPrVxbkEk91xR9IlMBhIqmeG\ngypbvbqHgYEtQC9wF9DLwMAWurvXZlyZdmV0NAkEISQPO5osxuTjkEOyqEySystwUEX5fJ7+/j7G\nxi4A1gAHAmsYGzuf/v4+CoVCxhVqOi95SRIIpi5ffNllxVAgSY3EOQdVNDIyMv7ZiilbVgIwPDzs\n/IMa8cQTsMce02/bsSMJC5LUqOwcVNHixYvHP9s8ZcsmAFpaWqpaj57qfe9LTvxTg0FnZ7FLYDCQ\n1OjsHFRRW1sbHR1dDAycwdhYJOkYbKKp6Uza27vsGmRophP+n/7ko5ElLTx2Dqosl+ulvX050AMc\nBPTQ3r6cXK4348oWnq98pTjBcLLddy92CQwGkhYiOwdV1tzczMaN6ykUCgwPD7vOQQZm6hLceScc\nfHBVS5GkmmQ4yEhra6uhoIp+/GM45pjpt3m3gSSleVlBDW3issHUYHDttd6GKEkzsXOghnPPPXDg\ngdNvMwxI0q7ZOVDDaG5OugRTg8HnP2+XQJLmws6B6tqf/wx77jn9NsOAJM2PnQPVpVNPTboEU4PB\nO95hl0CSSmXnQHUjRththjj76KPwtKdVtx5JalR2DlTzLr446RJMDQaHHFLsEhgMJKl87ByoZs20\nWNHWrbDvvtWtRZIWEjsHqik33DD9ksbNzcUugcFAkirLzoFqwkxdgl/9CpYsqW4tkrTQGQ6UmV//\nOpk3MB3vNpCk7HhZQVW3ZEnSKZgaDL7/fW9DlKRaYOdAVfHAA7Bo0fTbDAOSVFvsHKii3vCGpEsw\nNRhceKFdAkmqVXYOVHY7dkBT0/TbxsZmXsionuTzeUZGRmhpafHR25IaTgP8Z1q14hOfSLoEU4PB\nKacUuwT1HgxGR0fp7FzFkiVL6Orqoq2tjc7OVWzbti3r0iSpbOwcqGQz3Yb4wAPw7GdXt5ZKW726\nh4GBLUAvsALYzMDAGXR3r2XjxvUZVydJ5VHnf8cpK9/73q4XK2q0YJDP5+nv72Ns7AJgDXAgsIax\nsfPp7++jUChkXKEklYedA83JTF2C22+HQw+tbi3VNjIyMv7ZiilbVgIwPDzs/ANJDcHOQR3J5/Ns\n2LCh6n+h3nLL9F0CKHYJGj0YACxevHj8s81TtmwCoKWlpar1SFKlGA7qQFaT4CYCwctelh6/9tqF\neRtiW1sbHR1dNDWdQTLn4G6gl6amM+no6LJrIKlhGA7qQHoS3F1ALwMDW+juXlv2fd133667BCtX\nln23dSOX66W9fTnQAxwE9NDevpxcrjfjyiSpfJxzUOMmJsElwWDN+OgaxsYi/f09FAqFsvzF+vKX\nw+DgU8e/8AV473tLfvuG0dzczMaN6ykUCgwPD7vOgaSGZDiocZWcBPfYY/D0p0+/7cQTV3Hppb00\nNzfP670bXWtrq6FAUsPyskKNq8QkuJNPTi4bPDUYFEiuo3+Da66pzGULSVLts3NQ4yYmwQ0MnMHY\nWCTpGGyiqelM2ttnPwlu56sTPh34MpW8bCFJqh92DupAKZPgzjkn6RJMFwxihL6+DcBj7OyyhSRp\nYbFzUAfmMwlupsWK7roLDjyw+HX6ssWaSa/03n1JWqgMB3VkV5PgrroKTjpp+m0zrUlQrssWkqTG\n4WWFBjCxLsHUYLB58+wWK/LefUnSZHYO6tRtt8Hhh0+/ba4rF3rvviRpMsNBnZlpLsG558I//ENp\n7+29+5IkMBzUhT/9aebHHy+05xtIkirPOQc17LWvTToFU4PBKacszAcfSZKqw85BjdnZYkWPPw67\n+/+YJKnC7BzUiC9+cfrFil72smKXwGAgSaoGTzcZm2mC4fbtsPfe1a1FkiSwc5CJm24qrk0w2SGH\nFLsEjRYM8vk8GzZsoFAoZF2KJGkXDAdVdNhhSSA46qj0+MhIEgjuuKO69VTjhD06Okpn5yqWLFlC\nV1cXbW1tdHauYtu2bRXbpySpNIaDChsdLXYJJp/8W1qKXYLDDqt2TdU7Ya9e3cPAwBagF7gL6GVg\nwMdBS1Itq2g4CCH8cwhhx5SPX1Zyn7Xi3HOTQLDPPunxm25KAkGW3fVqnbDz+Tz9/X2MjV1A8lCn\nA0keB30+/f19XmKQpBpVjQmJ/wOcAExcYX+iCvvMxCOPwDOf+dTxo46CoaHq1zOdiRN2EgwmnsK4\nhrGxSH9/D4VCoWyrJI6MjIx/NvPjoF2RUZJqTzUuKzwRY7wvxvj78Y/RKuyzqi67LOkSTA0Gg4NJ\nl6BWggHM7oRdLunHQU/m46AlqZZVIxy0hhB+E0IYCSH0hhAOrMI+Ky5GaGtLQsFb31ocP+gg2LEj\n2b50aXb1zaSaJ+yJx0E3NZ1B0qm4G+ilqelMOjp8HLQk1apKh4MtwN8AHcBpwKHA5hDCXhXeb8Xc\ndRcsWZIsVjT5knlfXxIIfv3rmdcuqAXVPmH7OGhJqj8hVnGB/hDCIuDXwLoY41em2b4UGFyxYgWL\nFi1Kbevu7qa7u7s6he7EbrsVn2nwwQ/Cv/4r7LFHtjXN1bZt2+juXjs+9yDR0dFFLtdLc3NzRfbp\n46AlqXxyuRy5XC41tn37djZv3gywLMZY0gXtqoYDgBDCjcDVMcYPT7NtKTA4ODjI0lrsyQPDw/Dg\ng09dq6AeecKWpMYxNDTEsmXLoAzhoKrLJ4cQngUsBi6u5n7LqZHm0LW2thoKJElPUel1Dj4TQlgR\nQjg4hHAMcDnJrYy5XXyrJEnKSKU7BwcAlwD7APcB1wPLY4z3V3i/kiRpnioaDmKM2c8gVEPI5/OM\njIw4P0KSqsBnK6im+eAmSao+w4Fqmg9ukqTqq+rdCtJcVPM5EJKkIjsHqlnVfA6EJKnIcKCa5YOb\nJCkbhoM5yufzbNiwgcLkByuoInxwkyRlw3AwS86az4YPbpKk6nNC4iylZ82vADYzMHAG3d1r2bhx\nfcbVlaaW1xBobm5m48b1PgdCkqrIcDALjTprfnR0lNWre6r6dMb58jkQklQ9XlaYhVJmzdfyHAXX\nEJAkTcdwMAvzmTVf63MUJrohY2MXkHRDDiTphpxPf39fTYYZSVJ1GA5mYT6z5mv9r3LXEJAkzcRw\nMEtzmTVf6l/l1bgU4RoCkqSZOCFxluYya342f5VP973TTRA89tiVXHnl5WWfIDjRDRkYOIOxsThe\n2yaams6kvd01BCRpIbNzMEetra287nWv2+nJc75/la9e3cPVV98AHPnk2PXXb6K19YiKzFVwDQFJ\n0nQMBxUwnzkKE5ciduw4hIk5ChP/e//9j3DyyW8se50T3ZB8Pk9fXx/5fJ6NG9fX3G2MkqTq8rJC\nheRyvXR3r6W/v+fJsfb2rhn/Ki9eiriJqespQOS66yq3noJrCEiSJjMcVMhcV/YrXoqAuc5VkCSp\nnLysUGGzmaMAyaWIY4+dCAXeQSBJyo7hoIZceeX32Gef/YD34VMIJUlZMRzUkObmZgqFWznuuCPx\nDgJJUlacc1Bjmpub2bz5Wp9CKEnKjOGgRnkHgSQpK15WkCRJKYYDSZKUYjiQJEkphgNJkpRiOJAk\nSSmGA0mSlGI4kCRJKYYDSZKUYjiQJEkphgNJkpRiOJAkSSmGA0mSlGI4kCRJKYYDSZKUYjiQJEkp\nhgNJkpRiOJAkSSmGA0mSlGI4kCRJKYYDSZKUYjiQJEkpu2ddgIry+TwjIyO0tLTQ2tqadTmSpAXK\nzkEJ8vk8GzZsoFAolPQ+o6OjdHauYsmSJXR1ddHW1kZn5yq2bdtWpkolSZo9w8E8lPtkvnp1DwMD\nW4Be4C6gl4GBLXR3ry1n2ZIkzYrhYB7KeTLP5/P09/cxNnYBsAY4EFjD2Nj59Pf3ldyVkCRprgwH\nc1Tuk/nIyMj4ZyumbFkJwPDwcIkVS5I0N4aDOSr3yXzx4sXjn22esmUTAC0tLXN6P0mSSmU4mKNy\nn8zb2tro6OiiqekMkssUdwO9NDWdSUdHl3ctSJKqruLhIITwvhDCHSGEP4cQtoQQXlHpfVZSJU7m\nuVwv7e3LgR7gIKCH9vbl5HK9Za1dkqTZqOg6ByGEtwLnAv8buBFYB/SHENpijH+o5L4rKZfrpbt7\nLf39PU+Otbd3zftk3tzczMaN6ykUCgwPD7vOgSQpU5VeBGkd8J8xxosBQginAauAdwLnVHjfFVOp\nk3lra6uhQJKUuYqFgxDCHsAy4JMTYzHGGEIYAI6u1H6ryZO5JKkRVXLOwfOAJmDrlPGtwP4V3K8k\nSSpBFs+2nGaAAAAIdUlEQVRWCEDc2QvWrVvHokWLUmPd3d10d3dXsi5JkupCLpcjl8ulxrZv3162\n9w8x7vQ8Pf83Ti4rPAycGmO8ctL4V4FFMcY3TvM9S4HBwcFBli5dWpG6JElqRENDQyxbtgxgWYxx\nqJT3qthlhRjj48AgcMLEWAghjH99Q6X2K0mSSlPpywrnAV8LIQxSvJVxT+CrFd6vJEmap4qGgxjj\nZSGE5wEfB/YDbgI6Yoz3VXK/kiRp/io+ITHG+Hng85XejyRJKg+frSBJklIMB5IkKcVwIEmSUgwH\nkiQpxXAgSZJSDAeSJCnFcCBJklIMB5IkKcVwIEmSUgwHkiQpxXAgSZJSDAeSJCnFcCBJklIMB5Ik\nKcVwIEmSUgwHkiQpxXAgSZJSDAeSJCnFcCBJklIMB5IkKcVwIEmSUgwHkiQpxXAgSZJSDAeSJCnF\ncCBJklIMB5IkKcVwIEmSUgwHkiQpxXAgSZJSDAeSJCnFcCBJklIMB5IkKcVwIEmSUgwHkiQpxXAg\nSZJSDAeSJCnFcCBJklIMB5IkKcVwIEmSUgwHkiQpxXAgSZJSDAeSJCnFcCBJklIMB5IkKcVwIEmS\nUgwHkiQpxXAgSZJSDAeSJCnFcCBJklIMB5IkKcVw0AByuVzWJdQlj9vceczmx+M2dx6zbFUsHIQQ\n7gwh7Jj0MRZC+GCl9reQ+Us0Px63ufOYzY/Hbe48ZtnavYLvHYGPAF8CwvjYgxXcnyRJKoNKhgOA\nP8UY76vwPiRJUhlVes7BP4UQ/hBCGAohfCCE0FTh/UmSpBJVsnNwPjAEjALHAJ8C9gc+sJPveQbA\nrbfeWsGyGs/27dsZGhrKuoy643GbO4/Z/Hjc5s5jNneTzp3PKPW9Qoxx9i8O4d+AD+3kJRE4IsaY\nn+Z73wF8AXhWjPHxGd5/NfCNWRckSZKmWhNjvKSUN5hrONgH2GcXL7s9xvjENN/7IuDnwOExxsJO\n3r8DuBN4ZNaFSZKkZwCHAP0xxvtLeaM5hYOSdhTCGuCrwPNijNurslNJkjRnFZlzEEJYDrwK+CHJ\n7YvHAOcBXzcYSJJU2yrSOQghHAV8HlgCPB24A7gY+OxM8w0kSVJtqNplBUmSVB98toIkSUoxHEiS\npJSaCwchhLPGH9R0Xta11LoQwgtCCF8fX4Xy4RDCzSGEpVnXVatCCLuFED4RQrh9/HgNhxA+knVd\ntSaEcFwI4coQwm/GfxdPmuY1Hw8h3Dt+HK8OIbRkUWst2dlxCyHsHkL4dAjhlhDCn8Zf87UQwv/K\nsuaszebf2qTX/uf4a86oZo21aJa/o0eEEK4IIfxx/N/cf4cQDpjtPmoqHIQQXgG8B7g561pqXQjh\nOcCPgEdJ1oY4Avg/wLYs66px/wS8F/g74HDgg8AHQwinZ1pV7dkLuAl4H8nCZikhhA8Bp5Mcy1cC\nDwH9IYSnVbPIGrSz47YncCRwNnAU8EaSCdtXVLPAGrTTf2sTQginkPxb+02V6qp1u/odXQxcB/wS\nWAG8BPgEc1g/qGYmJIYQngUMAn8LfBT4WYzxH7KtqnaFED4FHB1jXJl1LfUihHAV8LsY43smjX0b\neDjG+LbsKqtdIYQdwCkxxisnjd0LfCbG+Nnxr/cGtgJvjzFelk2ltWW64zbNa14O/DdwcIzxnqoV\nV6NmOmYhhBcCPyb5I6iP5K63CzIosSbN8DuaAx6LMb59vu9bS52D/wCuijH+IOtC6sQbgJ+GEC4L\nIWwdf7jVu7MuqsbdAJwQQmgFCCG8DPhLkv/gaBZCCIeSPCPlmomxGOMDJCe5o7Oqq049h+Svvj9m\nXUitCiEEktvgz4kx+tCdWRg/ZquAQghh4/j5YUsI4eS5vE9NhIMQwl+TtNzOyrqWOnIYSZflNuBE\nkudWXBBCWJtpVbXtU8A3gV+FEB4j6VR9LsZ4abZl1ZX9SU5oW6eMbx3fplkIITyd5N/jJTHGP2Vd\nTw37J5K/gP8960LqyL7As0ieg9QHvBa4HPhuCOG42b5JJZ/KOCvjEyQ+B7zWBZLmZDfgxhjjR8e/\nvjmE8BckgaE3u7Jq2luB1cBfk1yLOxI4P4Rwb4zx65lWVv8CO7lmrKIQwu7At0iO199lXE7NCiEs\nA84gmaOh2Zv4o/97ky6/3BJCOAY4jWQuwqzfJEvLgOcDgyGEx0MIjwMrgTNDCI+Nt0j0VL8FprbZ\nbgUOyqCWenEO8G8xxm/FGH8RY/wG8FnsWM3F70iCwH5Txvflqd0ETTEpGBwInGjXYKeOJTk33D3p\n3HAwcF4I4fZsS6tpfwCeoMTzQ+adA2CAZCblZF8l+UE+FWtlxmTt+RHJbOfJlgC/zqCWerEnT/3r\ndge1EZLrQozxjhDC74ATgFvgyQmJryKZN6QZTAoGhwGvjjF6Z9HOXQxcPWXs++PjX6l+OfUhxvh4\nCOEnPPX80MYczg+Zh4MY40MkLd4nhRAeAu53AspOfRb4UQjhLOAykv84v5vkVlBN7yrgwyGEu4Ff\nAEuBdcD/zbSqGhNC2AtoIekQABw2PnlzNMZ4N8llwI+EEIZJHq/+CeAeFvhteTs7bsC9wHdILmW9\nHtgjhDDRfRldqJdUZ/FvbduU1z9OcsdRobqV1pZZHLfPAJeGEK4jeQDi60j+3c3+7rYYY819AD8A\nzsu6jlr/ALpI/np7mORk986sa6rlD5J7g88jeRDYQ0CB5L7z3bOurZY+xv8DsgMYm/Lx5Umv+ReS\nE97DQD/QknXdWX/s7LiRtMOnbpv4ekXWtdfiMZvh9bcDZ2Rdd9Yfs/wd/RsgP/7fuiHg9XPZR82s\ncyBJkmqD11olSVKK4UCSJKUYDiRJUorhQJIkpRgOJElSiuFAkiSlGA4kSVKK4UCSJKUYDiRJUorh\nQJIkpRgOJElSyv8HfVE/ILNYHEIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x104901748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(X, y)\n",
    "plt.plot(X, list(map(hypothesis, X)))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "To browse this notebook as reveal.js slides, run this command:<br>\n",
    "`$ jupyter nbconvert --to slides --post serve linear-regression-GradDes.ipynb`<br>\n",
    "then go to: http://127.0.0.1:8000/\n",
    "\n",
    "> <sub>Aziz Alto, Thu Sep 15 04:40:25 EDT 2016</sub>"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}