Skip to content

Instantly share code, notes, and snippets.

@CalvinTChi

CalvinTChi/Hebb's Rule.ipynb

Created June 7, 2016 02:40
Show Gist options
  • Save CalvinTChi/36c5f78a6cbae35a67da1b5277b695ec to your computer and use it in GitHub Desktop.
Save CalvinTChi/36c5f78a6cbae35a67da1b5277b695ec to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
Hebb's Rule.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Hebb's Rule\n",
"### Author: _Calvin Chi_"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Hebb's Rule is a simple binary classificaiton algorithm that finds a decision boundary resulting from the subtraction of two class centroids. Before we dive in, let as first understand what a centroid is and how a boundary can be constructed from the substraction of class two centroids. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Math Prerequisites\n",
"A centroid can be understood as the average in multiple dimensions, or the geometric center of $n$-dimensional space. Assume that $x^{(i)}$ is the $i$th sample with $n$ features, or $x \\in \\mathbb{R}^{1xn}$, then the centroid of $m$ such samples is computed as:\n",
"\n",
"$$x_{c} = \\frac{\\sum_{i = 1}^{m}x^{(i)}}{m} \\in \\mathbb{R}^{1xn}$$\n",
"\n",
"To visualize this:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAEACAYAAAAp/xTFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHhJJREFUeJzt3X2Q1NWd7/H3d+IMNQQxDkMRImEgmuju3QRHolFxpV0z\nMz4SEUtN4uOthLKicRdJLipGhjhoyF41t7ZSe0PWm8AmVHYjUZGbzMAKYxXxcVHQGE1MFHDDlcjD\nJuXDAsL3/tE9w8zQ3fPr6V//Hro/r6ouuqd/3ef8+uHLOed7zmlzd0REal1d3BUQEUkCBUMRERQM\nRUQABUMREUDBUEQEUDAUEQFCDIZmVmdmz5nZ6rCeU0QkKmG2DP8W+HWIzyciEplQgqGZTQIuAP4p\njOcTEYlaWC3D+4GvA1rOIiKpVHYwNLMLgZ3uvhmw3EVEJFWs3LXJZnY3cBXwPtAIHA38zN2vGXKc\nWo0iEgt3H7aRVnbL0N1vd/fJ7v4x4Epg/dBAOODY2C6LFi1S+TVYtspX+UFpnqGICHBUmE/m7o8D\nj4f5nCIiUaiZlmEmk1H5NVi2ylf5QZWdQAlckJlHVZaISB8zwwMkUELtJovUuilTprBt27a4q1GT\nWlpa2Lp164gfr5ahSIhyrZC4q1GTCr32QVuGNTNmKCJSjIKhiAgKhiLx2r8/nGOkbAqGIpXiDgcP\nFr7/nXfgr/4Knnii8DF798KZZ0Jvb+jVS5p77rmHuXPnFrx/6tSprF+/vmLlKxiKVMpPfgJz5hRu\n2X3wg3D99TBzZv6AuHcvnHEGnHxy9phCdu2Cxx4LXK2VK1dy6qmncvTRR3Pcccdx4YUX8stf/jLw\n4/NZvHgx11yTdxVuYLfddhvLli0r6znKoWAoqdTT00N7+xza2+fQ09MTd3XymzMn2zq8/PL8AXHD\nBrjvPvjmN+GSS+Dppw/ft3cvtLXBxInZ4954I38Zu3bBZz8buOV43333ccstt3DHHXfwxz/+ke3b\nt3PjjTfy6KOPln5+JUp8lj3CxdIuEobu7m5vbJzg8EOHH3pj4wTv7u6Ou1ru7n7E53zfPvdZs9w/\n97ns9T7r17s3N7tv2JC9vWaN+/jx7k895b5nj/v06e7z5rkfOuR+333uH/uY+7Ztg5/7rbfcp01z\nv/XW7HHD+NOf/uRjxozxVatW5b3/0KFDfs899/jxxx/vzc3NfsUVV/jevXvd3X3r1q1uZr58+XKf\nPHmyjx8/3pcsWeLu2fejoaHBGxoafMyYMX7yySe7u3smk/GFCxf6jBkzfPTo0f773//ed+zY4bNm\nzfKmpib/+Mc/7t///vf7y+/s7PSrrrqq//aKFSu8paXFm5ubfcmSJT5lyhR/7LHHCp5foRiT+/vw\nMSrIQWFcFAwlLG1tl+YCoecuP/S2tkvjrpa7F/hCDg2IQwNhnzVr3OvrsyfVFwj7DA2IJQZC92zQ\nqq+v94MHD+a9/zvf+Y6fccYZvmPHDt+/f7/fcMMN/vnPf97dDwfDuXPn+r59+3zLli0+atQof+WV\nV9w9G8iuvvrqQc+XyWS8paXFX375ZT948KAfOHDAzz77bL/pppt8//79vnnzZh8/frxvyL0OA5/j\npZde8jFjxvjGjRt9//79fsstt3h9fX1Fg6G6yZJ4qegSF9PQAD/9aTZujx8Ps2dnbw9ds3vmmXDg\nQPb6FVeADZgnPG8e3HQTnHMOPP98tmt8/vlw992Djyti9+7dNDc3U1eX/2v/ve99jyVLljBx4kTq\n6+u58847efDBBzl06BCQnbzc2dlJQ0MDn/rUp5g2bRpbtmwpWuZ1113HSSedRF1dHW+++SZPPPEE\nS5cupb6+nmnTpvGlL32JFStWHPG4VatWcfHFFzNjxgzq6+u56667sIDnOVJajieJ1tPTw+zZ1/Le\ne0sB2LjxWhYu/CobNy7gvfeyxzQ2LmD+/OUx1jKAhgb4yldg9Wo45phs4Buob4xw3jw491y4+GJ4\n9FH4zGcOHzNvHuzeDaecAl/8YkmBEGDcuHHs2rWLQ4cO5Q2I27ZtY/bs2f33uTv19fXs3Lmz/5gJ\nEyb0Xx89ejRvv/120TI/+tGP9l/fsWMHTU1NjB49uv9vLS0tbNq06YjH7dixY9BjR48ezbhx4wKc\n5cipZSiJdu+9y3KB8FogGxQff/w5HnpoOW1tq2lrW81DDy2no6Mj7qoWt2EDXHUV9PRkM8MDkyp9\ngfDss+Hee+HCC+EHP8gGxIFJlV27YM0aGDUKnnyycFKlgDPOOINRo0bx8MMP571/8uTJ/OIXv2DP\nnj3s2bOHvXv38s477zBx4sRhn7tQq23g3z/ykY+wZ88e3nnnnf6/bd++neOOO+6Ix02cOJE3Bpzf\nu+++y+7du4etRzkUDCWVOjo6WLt2FWvXrkpHILz88mzXuL39cJf58sth587BgbAveAwNiH1Z4/PP\nh/feO9xl3r49cDXGjh3L4sWLufHGG3nkkUd47733eP/99+nu7mbBggXccMMN3H777WzPPedbb73F\n6tWHfwbdi2SDJ0yYwNatW4seM2nSJM4880xuu+029u3bxwsvvMADDzzA1VdffcSxl112GWvWrOGJ\nJ57gwIED3HnnnZXPRgcZWAzjghIoMgJBMsfd3d3e1napt7VdGntW+YjPeaFkyb597hdc4H7MMe43\n31w4CbJmjfu4ce6f+MSRyZJCWeZhrFy50j/96U/7mDFjfOLEiX7RRRf5k08+6e7u999/v5944ok+\nduxYP+GEE3zhwoXunk2g1NXVDUq+nHPOOf7AAw+4u/vu3bv9rLPO8mOPPdanT59+xP19/vCHP/hF\nF13kTU1NfsIJJ/iyZcv67xuahFmxYoVPnjzZm5ub/e677/apU6cqmyy1rViwS9o0m0Gf88cfzx8I\n+/zXf7mfdlo20/z++/mPeest9+OPd/+7v8sfMEcYEKtRucFQW3hJqrW3z2HdullkxxQBsmOJa9eu\niqU+g7aRev317Lje2WcXfsD+/fCzn8GVV+a//5ZbsmOExZIl998PmzbBj35UXuVTrtwtvJRNFqmU\nqVOzl2IaGgoHQoClS+Goo4pnjefNOzwlR0ZMwVBSbf78uWzceG26ptmUor4+3OOkIHWTJfV6enq4\n997sAv/58+fGml3WTtfxKbebrGAoEiIFw/jEvu2/mY0ys6fN7Hkze9HMFpX7nCJhSP0yPolWkJTz\ncBdgdO7fDwBPAaflOaastLlIKcKechN0LmNLS4sDusRwaWlpyfueEMc8Q2A08O/AqXnuG9mnUCSP\n4YJTmDvbJG0uo5QmaDAMJZtsZnXAJuB44Lvu/mwYzyuST77NGyq5Pnnw+ujsarh7712W/GWAUpJQ\ngqG7HwJazWws8LCZ/aW7/3rocZ2dnf3XM5kMmaFbGIkEECQ4Vf2UGymot7eX3hH8Zkzo2WQz+wbw\njrvfN+TvHnZZUpuCrjoJa8rN0JZoY+OCdOyUI0CEK1DMrBk44O5/MrNGoA34VrnPK1JIkFZfmHMP\nOzo6eOih5QOeT4GwGpXdMjSzTwLLyU7TqQP+xd2X5DlOLUMJTbFgp5acDKRJ1xKZQoEprpUhSdu8\nQeKljRokEoUyu0CkGV+RcikYSlkKZXaz1+OZjqJMsoyEtv2XxBrpcrq+hEeqfiNF4hdkZnYYF7QC\npSoVWp1R7qqNpK/6SNJPDVSLSr2maNt/iUqhD3E5H+6wltNV4guW9ECdRpV8TRUMJdXCCIaV+oKF\nue5Zsir5mgYNhkqgSCIMnYYTRhJEa4qlFAqGErtC03OSuupD2erwJeI1DdJ8DOOCuslSQKW6SEO7\nyXV1x3pXV1cINVYCpRKUQJHQJfGLWqxOlRwv6urq8rq6cQ6nO8xXsqMGKRjWqCRmOoerUyXrrGSH\nBA2GGjOsMklMGgxXJ+0KI0mgYCiJ0NHRUZEAmIiBeUkFBcMqk8Qvf5x1UqtTgtIWXlUoST+q3ieJ\ndYLk1kvCo/0MC9CHv3YNfe+Bmt0Etpa+B0GDYU1lk5OYaZVo5HvvW1tn1mSmuda+ByibfKQkZlol\nGvne+23b7oq3UjHR9yA/7WdYI0a6N2A1a2n5MI2NC8j+hM/yXGJnbtzVkrgEaT6GcUHd5NiEfd5J\nXOEynGL7LgY9lzSedz619j1AK1Dyq5YPdCnCXIWR5i9SOe99ms87n1r6HigYSr8wg2EtLG/r7u72\n1tYZ3tR0vLe2zuwPHJXabDaswFRLAa4UCobSL8xWTbUHw+7ubm9o+JBDc//r1dAw3ltbZ1Rks9mu\nrq5Q3ptqa7mGKbJgCEwC1gMvAS8CNxc4LoLTlkLCbH1U85cuG+xPPyLwtbbOLPu88/1H0tR0fCj/\nuVT7f1LlCBoMw5ha8z5wi7tvNrMxwCYzW+vur4Tw3BKSsNb+xrm8Lc6Jws3N47Ssr9oFiZilXICH\ngXPz/L2y4V9CkdRxp6hapIW6yWGUFWY3eej7VO0t9nIQx5ghMAXYCozJc1/FT1rKk+QvVJTdwHwJ\nlDCfu9wEShjThGpJ0GAY2trkXBe5F7jL3R/Jc78vWrSo/3YmkyGTyYRStoSjvX0O69bNom9lAmR/\niH3t2lVxVgtIdt2ipteiuN7eXnp7e/tvL168GA+wNjmU5XhmdhTwIPDP+QJhn87OzjCKkxqUxK3J\nqkE1btgwtKG1ePHiYA8M0nwc7gKsAO4b5phKtoSlgFJXWCS1m+ye3PHMqIX1PiX9/Q4LEU6tmQEc\nBDYDzwPPAeflOS6K85YBRvJhV8ApT1SvXxjljGQcNo2fj8iCYdCLgmH0NPcsfMWCQdpaWqV+PtJ2\nfn0UDEXBMGTDBYO0vd6lBre0nV+foMGwpvYzrDVKOoSr2vYB1O/DDKZgWMX0YY82W5rG/3xKWZmU\nxvMrSZDmYxgX1E2WiFViH8fhni+NCYZSpPH8iHrS9XCS8oNQUv36WoObNm1hz55vEObk5Gqclxe2\npL1GQX8QSt1kqSo9PT0DfvFuR+jPX6kfu68Wg19/2Ljx2tT84qCCoVSVwUmODwNX9d9X7hhX0lo8\nSZTmJJOCoVSxDuBampruYvr0aWUlkNLc4pGAggwshnFBCRSJQKUmBqd1jl3UkjgxG80zlFqk6UTx\nSvPrr2xyABorkqHd5MbGBWV1k/WZik7QbLK6ycNIYrNf4qHfkUknNM8wHNpIU8JWqc+UWpv5BW0Z\n1kVRGZFa1NPTQ3v7HNrb59DT01PxsmbPvpZ162axbt0sZs++tuJlVp0gzccwLqibLHF57bVwjilB\nsc9NJT5TynYXhrLJ4UhzdqyWDOwizpx5Co8//hwAd3xxFmd/7Wvwr/8K55yT/8EbNsDll8Mzz8DU\nqaHUp9jkY32mEipIxAzjQkpbhpJ8Q1taMNZhfn+r65mlS92bm93Xrz/ywevXF76vDFG31NSDKQxt\n7iq1Il/ggUsHB6F8Qa9CgdB9YHCa73C619WN866urtDLGVpm2naUiYKCYUKk+QM60rpHfc6BgqH7\n4OBXwUDYp6ury+vqjlVrLWYKhgmQ5q7LSOsexzkP100eVP769d4fNSsYCN2V1EiKoMFQCZQKSvMO\nHiOtexznPDQhMXPm/8glUF5PXXJCcwXjo2AokavEF37oPoMLF+Y5qC9rvH599vbllw/KModdr1K3\nyS93ZxwF0jIFaT6GcUHdZHWT89zf0PAhb22dWfnxxWESKJV6r0oZPy2nWx20/mkewx4pohwzBB4A\ndgIvFDmm4iedRGn+8FUigTL4C9/t0Fz5/yyKJUty931t+l/HPr5XTjAM8tg0/+dcjqiD4VnAyQqG\n6Vfp4D34SxtBguG11/IGwoHn+czSpb63vsGn8PexBsNyglWQYFirCZ2gwTCUMUN332hmLWE8l8Sn\n0JgVENpY1OBxtPB/o+QIU6cesbLkyPNcwLe/ehM7//F/wnvjgXh+BrOclSlV/zOeUQgSMYNcgBbU\nMky1fC2H1tYZoXet+lplra0zvKFhfOTdtkItpDQPabgP36pXNzlBU2s6Ozv7r2cyGTKZTJTFywhs\n2/Zm6FNlBmZ+B2dA450Gk/Zfvhuu/rWyJrq3t5fe3t7SHxgkYga5oJZh6uVrObS2zqy6caZSW0hJ\naTEmpR5pQ9QrUIApwItF7q/0OUsIhn7h4uxaVfLLH/S5h9uKK6rgVKtd3DBEGgyBlWRHw/cB24Hr\n8xwTwWlLJcTRIknKl7/Y+GKU9avVTHAYggbDsLLJXwjjeSSZ4hhLi3Mp48BxzF27dieuflIZ2vZf\nEiHqLfILlTV0+/yXXtpCQ8PXgeXA8tyUlbmR123+/Lk0Ni6IrB6VEOV7PCJBmo9hXFA3WQrI1+Xs\n6uqq2PK4Ys9baHrR0GGCSnSTgyxlTGsCJc5hD7SFl6RFlPP+8ge7mcPWJZ+w61fN44JxnlvQYKhd\nayRBeoBlwA527fpAZGOVW7b8ip6eHjo6OkpayZH2eYkyRJCIGcYFtQylgO7ubm9o+JAP3LShoWF8\nRbpRXV1dDsf0lwPZrfkHtlLKafGV+9gkZNArQd1kBUMJKOzJ3fmC0uEv5EkOp3t2o4ju0LpsYXzh\n0zwuOJy4zk3BUFIlzDGlQkHpcBnduRZhuK2UsM6hmgNiHIIGQ40ZSiKEuetKoTmAh3WQnaLSSVPT\nW6xcmZw1uuXudi0jp2AoiRDFJgJHBtzXQw2EYQR0TeaOUZDmYxgX1E2WiMS5nrjc56/EFJRa73aj\nMUOpZWkNAGFnXas5Qx1U0GBo2WMrz8w8qrJE0qhvTXR2PfT7NDdPKHtn8fb2OaxbN4u+bjcsp61t\nNWvXrgqjyqlgZri7DXecxgxFEmBo4qSxcQEPPfQNjRVGSMFQJAEqlTjRb6MEp11rREqQ+J1XhujL\n0re1raatbbWm6RQTZGAxjAtKoEgFRZEwqWQyQomOykEJFKkV+cfbwm8BVToZMfjHscpLnMhhSqBI\nzaiWicraBSdeCoaSWn0tqU2btgBThz2+XEpGVDd1kyWVhnaN4Wbgy8AnK9ZN7itXXdl0CdpNVjCU\nxMsXgPKN3zU13cX06dMUpGQQjRlKVSi0i0s+06dPq6mVFRIuBUNJtELJEY3fSdhCmXRtZueZ2Stm\n9lszWxDGc4oUo8nEErayxwzNrA74LXAusAN4FrjS3V8ZcpzGDKVkUc0hDFIPJU7SKbIEipmdDixy\n9/Nzt28lO+N76ZDjFAxlROIOREkJyDIyUQbDOUCHu8/N3b4KOM3dbx5ynIKhpJK2wUq3RGaTOzs7\n+69nMhkymUyUxUsNiLoVGXerVY7U29tLb29v6Q8MsoC52AU4HegecPtWYEGe40JYci1SWKU2Oyj0\nvEnaXCGtO3tHgai2/Qc+APwOaAEagM3AX+Q5LorzlhoW9u+HDAwwXV1dRwSbSvxeyUjrmZSgnERB\ng2HZ3WR3P2hmNwFryU7VecDdXy73eUXidORk7+QmTaplo4q4hTJm6O7dwIlhPJdIEPnG6qL47eWB\nAUYTv6uLVqBI6hT7ofVK//byQFGXV4iCcji0UYOkThRTXdI2t1BZ7cISObVGJC2S0uoLShvDlk8t\nQ0mdtLXaJF7az1BCl6SuWJLqIsmmYCihUmtM0krBUEKl9bmSVkGDoX5EXkQEBUMJaP78uTQ2LgCW\nA8tzc9nmRlJ2T08P7e1zaG+fQ09PTyRlRqnazy81gqzZC+OC1ianXhybAaRl3e1IX5u0nF+aEdVG\nDUEvCoYyEknZDKGYcgJaGs4v7YIGQ026FimTNkqoDgqGkmjVvu622s8vTTS1RhIv6ROsy52DmfTz\nSzvNMxSJkAJacikYSmIoUEicFAwlEbSMT+KmFSiSCIMzrdmg+IUv3KjJxZI4CoYSuT17xjN79rUK\niJIo6iZLRQ3tJkPfkr43tdGDRELdZEmEvh2jm5ruAv432UCo8UJJHgVDqbiOjg5WrvwujY2vA29S\nzkYP2tRAKkXdZIlMuVNslJmWkYhkao2ZXQZ0An8BnOruzxU5VsFQyqINZmUkohozfBGYDTxe5vOI\niMSqrGDo7r9x91eBYaOuFKexsOHFucGsVL9QxgzNbAMwX93kkdFYWHBa2ielCu1H5M1sHTBh4J8A\nBxa6+6OlVKqzs7P/eiaTIZPJlPLwqqX98ILTj6XLcHp7e+nt7S35ccMGQ3dvG0mF8hkYDEVEKmFo\nQ2vx4sWBHhfm5q4aNxwhbfApEr9yp9ZcAvwD0Az8J7DZ3c8vcKzGDIvQWJhIZWgLLxERtDZZRKQk\nCoYiIigYiogACoYiIoCCoYgIoGAoIgIoGIqIAAqGkkLa4UcqQZOuJVW0w4+USitQpCppt2splVag\niIiUQMFQUiWO3a41Rlkb1E2W1Ilyhx+NUaafxgxFQqAxyvTTmKGISAnC3OlapOpoF/LaoW6yyDC0\nC3m6acxQRASNGYqIlETBUEQEBUMREUDBUCKgFRySBkqgSEVpBYfELZJsspl9G7gY2Af8Hrje3f9c\n4FgFwxqkFRwSt6iyyWuB/+buJwOvAreV+XwiIrEoawWKu//bgJtPAXPKq45UG63gkLQIbczQzFYD\nP3H3lQXuVze5RmkFh8QpaDd52Jahma0DJgz8E+DAQnd/NHfMQuBAoUDYp7Ozs/96JpMhk8kMV7xU\ngY6ODgVAiUxvby+9vb0lP67slqGZXQd8Gfgbd99X5Di1DEUkcqG1DIcp5Dzg68DZxQKhiEjSlTu1\n5lWgAdid+9NT7v6VAseqZSgikdOuNSIiaNcaEZGSKBiKiKBgKCICKBiKlEQ78FQvJVBEAtIOPOmk\nbLJIyLQDTzopmywiUgL9brJIQNqBp7qpmyxSAu3Akz4aMxQRQWOGIiIlUTAUEUHBUEQEUDAUEQEU\nDEVEAAVDERFAwVBEBFAwFBEBFAxFRAAFQxERQMFQRARQMBQRAcoMhmb2TTPbYmbPm1m3mX04rIqJ\niESp3Jbht919mru3Av8XWBRCnSqit7dX5ddg2Spf5QdVVjB097cH3PwgcKi86lRO3G9ILZdfy+eu\n8uMvP6iyd7o2sy7gGuA/gXPKrpGISAyGbRma2Toze2HA5cXcvxcDuPsd7j4Z+DHw1UpXWESkEkLb\n6drMPgr83N0/WeB+bXMtIrEIstN1Wd1kMzvB3X+Xu3kJ8HI5lRERiUtZLUMzexD4BNnEyTbgBnf/\nfyHVTUQkMpH9IJSISJJFugIl7knaZvZtM3vZzDab2SozGxth2ZeZ2a/M7KCZnRJhueeZ2Stm9lsz\nWxBVubmyHzCznWb2QpTlDih/kpmtN7OXcom/myMuf5SZPZ37vL9oZpHPwzWzOjN7zsxWR112rvyt\nA77zz0Rc9jFm9tPcd/4lM/tM0Qe4e2QXYMyA618F/jHi8j8L1OWufwu4J8KyTwQ+DqwHTomozDrg\nd0ALUA9sBk6K8JzPAk4GXojyfR5Q/oeBk3PXxwC/ifL8c+WOzv37AeAp4LSIy58H/AhYHdN78Bpw\nbExl/xC4Pnf9KGBsseMjbRl6zJO03f3f3L2vzKeASRGW/Rt3fxWIMpF0GvCqu29z9wPAT4DPRVW4\nu28E9kZVXp7y33T3zbnrb5NN8B0XcR3ezV0dRfYLGdm4lJlNAi4A/imqMvNVgxj2QMj1+v7a3X8A\n4O7vu/ufiz0mjkp2mdl24AvAnVGXP8B/B34RY/lROA54Y8Dt/yDiYJAUZjaFbCv16YjLrTOz54E3\ngXXu/myExd8PfJ0IA3AeDqwzs2fN7MsRljsV2GVmP8gNEywzs8ZiDwg9GMY9SXu48nPHLAQOuPvK\nqMuW6JnZGOBB4G+H9E4qzt0PeXbt/iTgM2b2l1GUa2YXAjtzLWMj2h7JQDPc/RSyLdQbzeysiMo9\nCjgF+G6u/HeBW4d7QKjcvS3goSuBnwOdUZZvZteRfWP+Jsxyg5Qdgz8AkwfcnpT7W80ws6PIBsJ/\ndvdH4qqHu//ZzDYA5wG/jqDIGcAsM7sAaASONrMV7n5NBGX389xUO3d/y8weIjt0szGCov8DeMPd\n/z13+0GgaAIx6mzyCQNuFp2kXaHyzyPbbZjl7vuiLHtoVSIq51ngBDNrMbMG4Eog6qxinK0SgP8D\n/Nrd/1fUBZtZs5kdk7veCLQBr0RRtrvf7u6T3f1jZN/39VEHQjMbnWuVY2YfBNqBX0VRtrvvBN4w\ns0/k/nQuw/wnFHrLcBjfylWuf5J2xOX/A9BAdgwD4Cl3/0oUBZvZJbnym4E1ZrbZ3c+vZJnuftDM\nbgLWkv2P7wF3j+w/IDNbCWSAcblx4kV9A9oRlT8D+CLwYm7czoHb3b07oipMBJabWR3Z1/9f3P3n\nEZWdBBOAh3JLcY8CfuzuayMs/2bgx2ZWTzarfX2xgzXpWkQEbfsvIgIoGIqIAAqGIiKAgqGICKBg\nKCICKBiKiAAKhiIigIKhiAgA/x9ENcd5oymt1AAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x105d648d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"\n",
"mu = [2, 1]\n",
"cov = np.array([[2, 0], [0, 2]])\n",
"points = np.random.multivariate_normal(mu, cov, 100)\n",
"centroid = np.mean(points, axis = 0)\n",
"plt.scatter(points[:, 0], points[:, 1])\n",
"plt.scatter(centroid[0], centroid[1], color=\"red\", label=\"Centroid\", marker=\"x\", s = 100)\n",
"plt.legend(loc=\"upper right\")\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To see how a plane can be contructed from a vector, imagine that we have a 2-dimensional weight vector $w \\in \\mathbb{R}^{2 x 1}$. The plane perpendicular to this vector is defined as:\n",
"\n",
"$$w^{T}x = w_{1}x_{1} + w_{2}x_{2} = 0$$\n",
"\n",
"Rearranging:\n",
"\n",
"$$x_{2} = -\\frac{w_{1}}{w_{2}} x_{1}$$ \n",
"\n",
"To visualize this, assume our $w$ is:\n",
"\n",
"$$ w = \\begin{bmatrix}\n",
"1\\\\\n",
"1\\\\\n",
"\\end{bmatrix}$$\n",
"\n",
"And contruct the line perpendicular to this weight vector."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQMAAAEACAYAAAC3RRNlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAERRJREFUeJzt3X+MXWVex/H3pxQIFNh0raGbVrYuiAaFFGiICUytqzMi\nSwBt/1jUkHVl47rhR50JbAR0Wn4oIWk3xERJbN24mgYJIAIBbAnUtpqysLRQSguVdlmKUHEZNZVk\nOzJf/7j3ssN0ftw79zk/7+eVNNxpT59zh8685zln7jyPIgIzszlFPwEzKwfHwMwAx8DMmhwDMwMc\nAzNrcgzMDIC5KQaR9H3gv4ExYDQiLkkxrpnlJ0kMaERgRUSMJBrPzHKW6jJBCccyswKk+gQOYIuk\nFyV9LdGYZpajVJcJl0bEe5J+kkYU9kXEjkRjm1kOksQgIt5r/vcDSf8AXAJ8KgaS/EMQZgWJCM10\nTNeXCZJOlXRa8/E8YAB4bbJjL774YlavXs3Y2BgRkfzX8PBwJuPmeY46vA8+R3nGj2j/a3CKewZn\nAjsk7QJ2Ak9ExObJDtyyZQvbt29ncHCwoydpZtnrOgYRcSgilkbEhRFxfkTcO9Wx8+fPdxDMSir3\nbwdmGYQVK1YkG6uoc9ThffA5yjN+J5TXV2dJMf5cIyMj9Pf309fXx/r165FmvL9hZrMgicjjBuJs\n+ZLBrFwKfdWgg2BWHoW/hNhBMCuHwmMADoJZGZQiBuAgmBWtNDEAB8GsSKWKATgIZkUpXQzAQTAr\nQiljAA6CWd5KGwNwEMzyVOoYgINglpfSxwAcBLM8VCIG4CCYZa0yMQAHwSxLyWIgaY6klyU9nmrM\nyTgIZtlIOTO4GXg94XhTchDM0ksSA0mLgSuADSnGa4eDYJZWqpnBt4BbaGymkhsHwSydrvdNkPQl\n4EhE7Ja0gsZWa5Nas2bNJ49XrFiRZP23VhD6+/sZHBz0EmrW87Zu3crWrVs7/ntdr4Eo6U+B3wH+\nDzgFOB14NCKum3BcZPmVu7Wm4vLly1m3bp2DYNbU7hqISRdElfRLwFBEXDXJn2UaA3AQzCZT+gVR\ns9C6ZNi2bRtDQ0O+h2DWgcKWSs+SZwhmP9aTM4MWzxDMOlfLGICDYNap2sYAHASzTtQ6BuAgmLWr\n9jEAB8GsHT0RA3AQzGbSMzEAB8FsOj0VA3AQzKbSczEAB8FsMj0ZA3AQzCbq2RiAg2A2Xk/HABwE\ns5aejwE4CGbgGHzCQbBe5xiM4yBYL3MMJnAQrFd1HQNJJ0t6QdIuSXskDad4YkVyEKwXJVnpSNKp\nEfGRpBOAfwFuiojvTjgmt5WOUvGKSVYHua50FBEfNR+eTGP59Wp91k/BMwTrJal2VJojaRfwPrAl\nIl5MMW4ZOAjWK7reRAUgIsaACyWdATwm6byIOG7fxSw2UcnD+I1ahoaGfMlgpVbYJirHDSj9MfC/\nEbF+wu9X7p7BRL6HYFWU2z0DSQskfab5+BSgH9jf7bhl5EsGq7MU9ww+BzwvaTfwAvBPEfFUgnFL\nyUGwuqrlJip58CWDVUVPb6KSB88QrG4cgy44CFYnjkGXHASrC8cgAQfB6sAxSMRBsKpzDBJyEKzK\nHIPEHASrKscgAw6CVZFjkBEHwarGMciQg2BV4hhkzEGwqnAMcuAgWBU4BjlxEKzsHIMcOQhWZo5B\nzhwEKyvHoAAOgpVRimXPFkt6TtLe5iYqN6V4YnXnIFjZdL3SkaSFwMKI2C3pNOB7wNURsX/CcbVa\n6SgVr5hkWcttpaOIeD8idjcfHwX2AYu6HbdXeIZgZZH0noGkJcBSGgujWpvGB2FwcNBBsEIk2UQF\noHmJ8DBwc3OGcJyqbqKSh/EbtQwODrJ+/XpfMtisFLqJiqS5wJPA0xFx/xTH+J5BG1r3EPr6+hwE\nS6LdewapYvAd4D8jYnCaYxyDNjkIllJuMZB0KbAN2ENj9+UAbouIZyYc5xh0wEGwVHKdGbTDMeic\ng2ApeBOVGmjdVNy+fbu/y2CZcwxKzkGwvDgGFeAgWB4cg4pwECxrjkGFOAiWJcegYhwEy4pjUEEO\ngmXBMagoB8FScwwqzEGwlByDinMQLBXHoAYcBEvBMagJB8G65RjUiINg3XAMasZBsNlyDGrIQbDZ\nSBIDSRslHZH0aorxrHsOgnUq1czg28CvJRrLEnEQrBNJYhARO4CRFGNZWg6Ctcv3DHqAg2DtcAx6\nhINgM0m2iUo7vIlKsVpBGBgYqPRGLYcOHWLv3r1ceeWVRT+VUip0ExX4ZGu1JyLi/Cn+3Ksjl8TI\nyAgDAwNcdtlllQrCBx98wD333MPGjRvZs2cPS5YsKfopVUKuqyNL2gT8K3CupB9I+t0U41o25s+f\nz+bNm9mxY0clLhmOHj3KXXfdxdlnn83999/P9ddf7xBkwPsm9LCyzxBGR0fZsGEDa9eu5ciRIwCc\nfvrpHDx4kAULFhT87KrD+ybYjMo+Q3jsscd44IEHPgkBwK233uoQZMQzAyvtDGF0dJRrr72WRx55\nBIAzzzyTt956i3nz5hX8zKrFMwNrWxlnCONDsGrVKi644ALWrFnjEGQpInL51TiVldmHH34Yy5Yt\ni9WrV8fY2Fhhz+PYsWOxcuXKAGLVqlVx7NixeOihh+LYsWOFPacqa37uzfg56ssE+5SiLxkmzgg2\nbdrEiSeemOtzqBtfJtisFHnJ4BAUyzGw4xQRBIegeI6BTSrPIDgE5eAY2JTyCIJDUB6OgU0ryyA4\nBOXiGNiMsgiCQ1A+joG1JWUQHIJycgysbSmC4BCUl2NgHekmCA5BuTkG1rHZBMEhKD/HwGalkyA4\nBNWQaqWjyyXtl/SmpG+mGNPKr50gOATV0fUPKkmaA7wJ/Arw78CLwJcjYv+E4/yDSjU11Q83OQTl\nkOcPKl0CHIiItyNiFHgQuDrBuFYRk80QHILqSbFU+iLgnXFvH6YRCOshrSAMDAywevVqDh8+zKOP\nPuoQVEiu+yZYvbWCsGzZMg4ePMjKlSsdggpJEYN3gbPGvb24+XvH8SYq9Td//nxeeuklli1bxqJF\ni5g7119v8lbYJiqSTgDeoHED8T3gu8C1EbFvwnG+gdhDRkZG6O/vp6+vr1SLrPai3G4gRsTHwA3A\nZmAv8ODEEFjv8d6O1eM1EC1TniEUz2sgWil4hlAdjoFlzkGoBsfAcuEglJ9jYLlxEMrNMbBcOQjl\n5RhY7hyEcnIMrBAOQvk4BlYYB6FcHAMrlINQHo6BFc5BKAfHwErBQSieY2Cl4SAUyzGwUnEQiuMY\nWOk4CMVwDKyUHIT8OQZWWg5CvrqKgaRVkl6T9LGki1I9KbMWByE/3c4M9gC/AfxzgudiNikHIR9d\nxSAi3oiIA4DXsrJMOQjZ8z0DqwwHIVszLmovaQtw5vjfAgK4PSKe6ORk3jfButUKQn9/P4ODg15k\ndRKF7ZsAIOl5YCgiXp7mGK+ObMm0Vl1evnw569atcxCmUcTqyP7XsNy0Zgjbtm1jaGjIlwwJdPut\nxWskvQP8IvCkpKfTPC2zmTkIaXkTFas8XzJMz5uoWM/wDCENx8BqwUHonmNgteEgdMcxsFpxEGbP\nMbDacRBmxzGwWnIQOucYWG05CJ1xDKzWHIT2OQZWew5CexwD6wkOwswcA+sZDsL0HAPrKQ7C1BwD\n6zkOwuQcA+tJDsLxHAPrWQ7CpzkG1tMchB/rdqWj+yTtk7Rb0iOSzkj1xMzy4iA0dDsz2Az8fEQs\nBQ4Af9T9UzLLn4PQ/SYqz0bEWPPNncDi7p+SWTF6PQgp7xl8FfCCqFZpvRyEJJuoSLodGI2ITdON\n5U1UrArGb9QyNDRUuUVWC9tERdJXgK8BX4yIH01znFdHtkqpy6rLuayOLOly4BbgqulCYFZFvXbJ\n0NXMQNIB4CTgh83f2hkR35jiWM8MrJKqPkNod2bgTVTM2lDlIHgTFbOEeuGSwTEwa1Pdg+AYmHWg\nzkFwDMw6VNcgOAZms1DHIDgGZrNUtyA4BmZdqFMQHAOzLtUlCI6BWQJ1CIJjYJZI1YPgGJglVOUg\nOAZmiVU1CI6BWQaqGATHwCwjVQuCY2CWoSoFwTEwy1hVgtDtsmd3SnpF0i5Jz0hamOqJmdVJFYLQ\n7bJnp0XE0ebjG4HzIuIPpjjWKx1ZzytixaRcVjpqhaBpHjA21bFmVu4ZQoql0u8GrgP+C/jliPjh\nFMd5ZmDW1Joh9PX1sX79+kxnCMkWRG1nE5Xmcd8ETomINVOME8PDw5+87U1UrNdlFYSJm6isXbs2\n39WRJf0U8FREnD/Fn3tmYDZBHjOEvDZROWfcm9cA+7oZz6zXtO4hbN++ncHBwULvIXT73YSHgXNp\n3Dh8G/h6RLw3xbGeGZhNIcsZgjdRMauYrILgTVTMKqboSwbHwKxEigyCY2BWMkUFwTEwK6EiguAY\nmJVU3kFwDMxKLM8gOAZmJZdXEBwDswrIIwiOgVlFZB0Ex8CsQrIMgmNgVjFZBcExMKugLILgGJhV\nVOogOAZmFZYyCI6BWcWlCkKSGEgakjQm6bMpxjOzzqQIQtcxkLQY6Kex0lGhxi8CWdVz1OF98DmK\nGb/bIKSYGXwLuCXBOF3zB4fPUbVzpB6/myB0uyDqVcA7EbGnm3HMLJ2JQWjX3JkOmGbfhDuA22hc\nIoz/MzMrWCsIAwMDbf+dWS+IKukXgGeBj2hEYDHwLnBJRPzHJMd7NVSzguS9icoh4KKIGEkyoJnl\nKuXrDAJfJphVVm77JphZueX6CkRJw5IOS3q5+evyjM6T2YugJN0p6RVJuyQ9I2lhBue4T9I+Sbsl\nPSLpjAzOsUrSa5I+lnRRwnEvl7Rf0pvNzXiTk7RR0hFJr2Y0/mJJz0naK2mPpJsyOMfJkl5ofhzt\nkTQ889+a1XnmND/XHp/x4IjI7RcwDAxmfI7FwDPAIeCzGYx/2rjHNwJ/mcE5fhWY03x8L/BnGZzj\nZ4GfAZ6jca8nxZhzgH8DPg+cCOwGfi6D534ZsBR4NaOPoYXA0ta/N/BGRu/Hqc3/ngDspHHzPfU5\n/hD4O+DxmY4t4mcTsr6vkOmLoCLi6Lg359HYZzL1OZ6NiNa4O2kELvU53oiIA6T997gEOBARb0fE\nKPAgcHXC8QGIiB1AZjeqI+L9iNjdfHyUxobCizI4z0fNhyfT+DZ/0mv25quDrwA2tHN8ETG4oTn9\n3SDpMykHzutFUJLulvQD4LeAP8nyXMBXgaczPkcqi4B3xr19mAw+ifIkaQmNWcgLGYw9R9Iu4H1g\nS0S8mPgUrS+MbUVmxhcddWqaFyndDvwFcGdEhKS7gfXA7yUaP9mLoKZ7HyLiiYi4A7ijeU18I7Am\n9Tmax9wOjEbEpizej9mM2UsknQY8DNw8YUaYRHP2d2HzntBjks6LiNdTjC3pS8CRiNgtaQVtfC4k\nj0FE9M98FAB/BXT8ATnV+M0XQS0BXlFj+9rFwPckTfoiqNmcYxKbgKeYRQxmOoekr9CY4n2x07Hb\nPUcG3gXOGvd264VolSNpLo0Q/G1E/GOW54qI/5H0PHA5kCQGwKXAVZKuAE4BTpf0nYi4bqq/kPd3\nE8bfef9N4LVUY0fEaxGxMCK+EBE/TWOKemGnIZiJpHPGvXkNjevJpJrfZbkFuCoifpR6/MlOmWic\nF4FzJH1e0knAl4GZ72LPjsj2/tNfA69HxP1ZDC5pQesyWdIpNGa0+1ONHxG3RcRZEfEFGv8Oz00X\nAshgZjCD+yQtpXHT7fvA72d4rqxeBHWvpHNpvA9vA1/P4Bx/DpwEbGlMctgZEd9IeQJJ1zTPswB4\nUtLuiPj1bsaMiI8l3QBspvGFZmNEZBHLTcAK4Cea926GI+LbCce/FPhtYE/zmj6A2yLimVTnAD4H\n/I2kOTT+X/19RDyVcPyO+UVHZgZ42TMza3IMzAxwDMysyTEwM8AxMLMmx8DMAMfAzJocAzMD4P8B\nt9FQXFkNVhEAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x104125e80>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig, ax = plt.subplots()\n",
"ax.quiver((0), (0), (1), (1), angles='xy', scale_units='xy', scale=1, color='black')\n",
"x = np.array(range(-5, 5))\n",
"y = -x\n",
"plt.plot(x, y, color=\"black\")\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Derivation\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The decision boundary determined by Hebb's rule is the plane perpendicular to the vector resulting from the subtraction of two class centroids. Assume that we have $m_{1}$ samples of class $1$ and $m_{2}$ samples of class $2$:\n",
"\n",
"$$w = \\frac{\\sum_{i = 1}^{m_{1}}x^{(i)}}{m_{1}} - \\frac{\\sum_{i = 1}^{m_{2}}x^{(i)}}{m_{2}}$$\n",
"\n",
"Notice that if we cleverely denote one class as $+1$ and the other as $-1$, the above equation becomes:\n",
"\n",
"$$w = \\frac{\\sum_{i = 1}^{m_{1}}y^{(i)}x^{(i)}}{m_{1}} + \\frac{\\sum_{i = 1}^{m_{2}}y^{(i)}x^{(i)}}{m_{2}}$$\n",
"\n",
"Remember from above that the decision boundary is defined as:\n",
"\n",
"$$w^{T}x = 0$$\n",
"\n",
"Thus the denominators $m_{1}$ and $m_{2}$ do not matter, and the equation for $w$ is reduced to:\n",
"\n",
"$$w = \\sum_{i=1}^{T}y^{(i)}x^{(i)}$$\n",
"\n",
"Where $T = m_{1} + m_{2}$. Once we have $w$, then we can construct our decision boundary. The classifer is:\n",
"\n",
"$$f(x) = w^{T}x$$\n",
"\n",
"When $f(x^{(i)}) > 0$, then sample $i$ is classified as class $+1$, and if $f(x^{(i)}) < 0$, then the sample $i$ is classified as class $-1$. If $f(x^{(i)}) = 0$, then the classificaiton is ambiguous because it lies on the decision boundary."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Implementation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us visualize samples drawn from two different distributions belonging to different classes in two dimensions and the decision boundary that results from applying Hebb's rule."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 86.24049496 78.85898632]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQMAAAEACAYAAAC3RRNlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHilJREFUeJzt3Xl4VPW9x/H3N4QYlitm4eEqi2irteXyWHGrJYEhkEwS\nIBgCLshuEL1WsaU8KlaBXlu3a3Gnj0JpRRRlk0AgEwQGEh69rlDc6kKlYFsI0CqoiJLf/WMmGCAh\ny5w5vzMz39fznMeZycmZD8h857edc8QYg1JKJdkOoJTyBi0GSilAi4FSKkyLgVIK0GKglArTYqCU\nAhwqBiLSSUQWi8h7IvKOiFzqxHGVUu5Jdug4DwOrjTEjRSQZaO/QcZVSLpFIFx2JyKnAW8aY7zkT\nSSllgxPdhLOAvSIyX0TeFJEnRaSdA8dVSrnIiWKQDPQBHjfG9AG+BG5z4LhKKRc5MWawC9hpjHk9\n/HwJcOvxO4mIngShlCXGGGlqn4hbBsaY3cBOETk3/NJA4N1G9vXUNmPGDOsZYiGTV3NppuZtzeXU\nbMLNwEIRaQtsByY4dFyllEscKQbGmK3AxU4cSyllR0KvQPT5fLYjnMCLmcCbuTSTsyJeZ9DsNxIx\nbr2XUuo7IoJxYwBRKRUftBgopYAYLAYHDx60HUGpuBRTxeDrr7/mggsuYN26dbajKBV3YqoYnHLK\nKcydO5err76aN954w3YcpeJKTBUDgP79+/Pkk08yZMgQPvjgA9txlIobTq1AdNXll1/Ovn378Pv9\nbN68mTPOOMN2JKViXkwWA4Brr72Wmpoa/H4/mzZtIi0tzXYkpWJaTC86MsYwdepUXn31VSorK2nf\nXi+wpNTxmrvoKKaLAUBtbS3jxo3jX//6F8uXL6dt27aOv4dSsSxhViAmJSXxhz/8AWMMpaWl1NbW\n2o6kVEyK+WIA0LZtWxYvXsxHH33EtGnTWnQOt1IqJC6KAUD79u1ZuXIlgUCABx54wHYcpWJOzM4m\nNCQ9PZ1AIEDfvn3JzMxk4sSJtiMpFTPiqhgAdO3alUAggM/nIzMzk6KiItuRlIoJMT+b0JjXX3+d\nwsJClixZQr9+/Vx7X6W8JmFmExpz0UUX8eyzzzJixAi2bt1qO45Snhe3xQBg0KBBPP744wwePJjt\n27fbjqOUp8XdmMHxRo4cyd69e8nLy2Pz5s106dLFdiSlPCnuiwHADTfcQE1NDfn5+QSDQTp16mQ7\nklKeE7cDiMczxnDTTTfxzjvvsGbNGlJTU61lUcpNCXNuQkvU1tYyatQoDh8+zOLFi2nTpo3VPEq5\nIeFnExqSlJTE008/zcGDB7nhhht02bJS9SRUMQBISUlh6dKlbNmyhV/96le246goCQQgLy+0BQK2\n08SGhOom1Ld3716ysrK4/vrrueWWW2zHUQ4KBKC4GL76KvS8XTtYvhz8fru5bNFuQhMyMzMJBAI8\n+OCDLFy40HYc5aAHH/yuEEDo8YMP2ssTKxybWhSRJOB1YJcxJiZOCDjzzDOpqKggJyeH9PR0CgoK\nbEdSyhonWwZTgHcdPJ4revXqxYsvvsi4ceN4+eWXbcdRDpg6NdQ1qNOuXeg1dXKOFAMR6QYUAnOd\nOJ7bLrvsMv70pz9RXFzMO++8YzuOipDfHxojyM0NbYk8XtASjgwgishi4DdAJ2BqQ90Erw0gNmTh\nwoXcfvvtVFdX06NHD9txlHJEcwcQIx4zEJHBwG5jzBYR8QFNvqlXXXPNNUfPY6iuriYzM9N2JKVc\n48QAYl+gSEQKgXbAf4jI08aYscfvOHPmzKOPfT4fPp/Pgbd31pQpU9izZw+FhYWsX7+ejh072o6k\nVIsEg0GCwWCLf8/RdQYi0p8Y7ibUMcZw3XXXsWPHDlatWkVKSortSEq1mq4ziICIMGfOHDp27MjY\nsWP18usqISTsCsTmOHToEPn5+fTu3ZtHHnkEkZgdDlEJTFsGDkhNTWXFihVUV1dz9913246jVFQl\nxMVNItGpUycqKiro27cvnTt35vrrr7cdSamo0GLQDF26dKGyspLs7GwyMjIYOXKk7UhKOU6LQTOd\nffbZrF69mtzcXNLS0hg0aJDtSEo5SscMWuD8889nyZIljBo1itdff912HKUcpcWghfr168dTTz3F\n0KFD+eCDD2zHUcox2k1ohWHDhrF//378fj/V1dV07drVdiSlIqbFoJUmTJhATU0Nfr+fTZs2kZ6e\nbjuSUhHRRUcRMMYwbdo0Xn75ZdauXUv79u1tR1LqBHqpdJfU1tYyYcIE9u3bx/Lly2nbtq3tSEod\nQ1cguiQpKYm5c0PXdLn22mv1PAYVs7QYOKBt27a88MILfPzxx0ybNk3vx6BikhYDh7Rv355Vq1ZR\nWVnJAw88YDuOUi2mswkOSktLo6KigqysLDIzM5k4caLtSEo1m7YMHNa1a1cCgQB33HEHL774ou04\nMcfGnZD07kthxhhXttBbJY7XXnvNdO7c2WzcuNF2lJhRUWFMu3bGQGhr1y70Wry9p9vCn70mP6Pa\nMoiSiy66iOeee44RI0awdetW23FiQkN3Qrr9dmeO3di3v9596TtaDKJo4MCBPP744xQWFrJ9+3bb\ncWLS1q2RN93r7r24dm1oKy5O8O5AY5rTfHBiI8G6CfXNmTPHfO973zP/+Mc/bEfxtIoKY5KSvmuy\n1225uZEdNze38WNqN0G7Ca66/vrrGTduHAUFBXz22We243iW3w/nn+/+e+rdl0J0ObJLjDHcfPPN\nbNu2jYqKClJTU21H8qRo3E490W/RrucmeFBtbS3XXHMNhw4dYvHixSQn6zKPhgQC3w3iTZ3qzIc2\nGseMFVoMPOrw4cMMHTqU7t2789RTT+nl1z0qnoqHFgMPO3jwIAMHDmTgwIH89re/tR1HHSfeuhV6\n1qKHdezYkfLycpYtW8bs2bNtx1HHSdS1B9pptSQzM5PKykqysrLo3Lkzo0ePth1JJThtGVjUo0cP\nKioqmDp1KqtXr7Ydx1VePh9g6tRQ16BOu3ah1+Kdjhl4wMsvv0xRURFlZWVcdtlltuNEXSz0yXUA\nMYq0GJzcmjVrGD9+POvXr6dXr16240RVXl5oWXB9ublQWWknT7xzbQBRRLqJyHoReUdEtonIzZEe\nMxEVFBTwu9/9jvz8fHbs2GE7juPqdwv27o38GF7rWsSF5qxZPtkG/Cfw4/DjjsBfgPMa2M+5xdZx\n7KGHHjLnnnuu2bNnj+0ojjl+/X9KSmhryfkAiXAOQbTg1rkJxph/GmO2hB8fBN4D9K4irTRlyhRG\njBhBYWEhBw4csB3HEcdP1R0+DL16tex8gESd7nOTo1OLItIT+DHwf04eN9Hcfffd1NTUUFxcTHl5\nOaeccortSI7LzNQxAq9xbABRRDoCQeB/jDErGvi5mTFjxtHnPp8Pn8/nyHvHoyNHjnDFFVfQpk0b\nnnvuOdq0aWM7Uqs5MXvg5RkIr808BINBgsHg0eezZs1ybzZBRJKBVcAaY8zDjexjnCo8ieLQoUMU\nFBTwox/9iMceeyymz2Nw4gPjtQ8deLtI1XF1alFEngb2GmN+cZJ9tBi0wueff47P52PYsGHUb1kp\nb4iFaVI3pxb7AtcAOSLyloi8KSL5kR5XhZx66qmsWbOGZ555hieeeMJ2HBXHIh5ANMZsBmK3QxsD\nunTpQiAQoF+/fmRmZnLFFVfYjhQxLzb5W2PqVKiuPrabEKtLl3UFYgz585//zKBBg3j22WcZNGiQ\n7TitFgv97JbwemHT5chxqqqqipKSEsrLy7n44ottx2mVWOhnxxO9nkGcys7OZt68eRQVFfGXv/zF\ndhzHtWTJsS5Pdlhzlik6saHLkR01f/58c+aZZ5pdu3bZjtJid9994qXL7767ZUuOdXly86GXSo9v\n48eP58YbbyQvL4/9+/fbjtMiGzc2/FpLlhw3tO+oUdpCiIQWgxg2bdo0CgsLGTJkCF988YXtOFHR\nkjMc9+939m5JCdcNaU7zwYkN7SZExZEjR8y4ceNMQUGBOXz4sO04zdJYE7+i4tizGevOcGyo+X/8\nMZy8A9PJMsYitJuQGJKSknjqqado06YNEydOpLa21nakJjV2FyO/P3Q2Y32HDzfcVag7Rnp6dDIm\n4lmSWgziQNu2bXn++ef561//ytSpU+taYp7m94emEisrj52Xz8xs2TGefTYxr1cYDVoM4kT79u1Z\nuXIl69at47777rMdp9VaejHSaN0rMREviqqLjuLM3//+d7Kyspg+fTqlpaW247SKV1b0eSVHpHQF\nYgL78MMP6d+/P0888QSXX3657TgxJ16KQB0tBgnuzTffJD8/n8WLF9O/f3/bcWJGvJ03AbocOeH1\n6dOHRYsWMXLkSLZs2WI7jnXNXTOQiLMIdbQYxLGcnBzmzJnD4MGD+fjjj23Hsabu237t2tDm5MKk\neKLFIM6VlJRw11134ff7+ec//2k7jhUt+bZPxFmEOloMEsDkyZMZP348+fn5fPbZZ7bjeMIbbzTc\nOojWVGUs0AHEBGGMYcqUKWzdupVAIEBqaqrtSK45flCwTjwMDjaHziaoE9TW1jJ69Gi+/PJLlixZ\nQnKyo7fN8LRAIHRW4/EneCbCRVV0NkGdICkpiT/+8Y989dVXTJ48OSaWLTvF74cLL7Sdwtu0GCSY\nlJQUli5dyttvv8306dNtx3FVIg8ONod2ExLU3r17yc7OZtKkSfziF43e7iLuxNvqwubQMQPVpL/9\n7W9kZWXxm9/8hjFjxtiOo6KkucUgcUaQ1Al69OhBRUUFOTk5pKenM3jwYNuRlEXaMlC88sorDB06\nlBUrVvDTn/7UdhzlMJ1NUM32k5/8hAULFlBcXMzbb79tO46yRIuBAiA/P5/Zs2dTUFDAjh07bMdR\nFuiYgTpq1KhR7N27l7y8PKqrq+ncubPtSMpFjrQMRCRfRN4XkQ9E5FYnjqnsuPnmmxk5ciSFhYUc\nOHDAdhzloogHEEUkCfgAGAj8HXgNuMoY8/5x++kAYowwxjB58mS2b99OeXk5p5xyiu1IKgJuDiBe\nAnxojNlhjPkGWAQMc+C4yhIRYc6cOXTq1IkxY8Zw5MgR25GUC5woBl2BnfWe7wq/pmJYmzZtWLhw\nITU1Ndx0000JdR5DonJ1AHHmzJlHH/t8Pnw+n5tvr1ooNTWVFStW4PP5mDVr1jH//5R3BYNBgsFg\ni3/PiTGDnwAzjTH54ee3Ebqd033H7adjBjFq9+7dZGVlccstt3DjjTfajqNayM3lyK8B3xeRM4F/\nAFcBVztwXOURXbp0obKykuzsbDIzM7nyyittR1JREHExMMYcEZGfAZWExiDmGWPeiziZ8pSzzjqL\n1atXk5ubS3p6Orm5ubYjKYfpuQmqRaqqqigpKaG8vJyLL77YdhzVDHpugoqK7Oxs5s2bR1FREe+/\n/37Tv6BihhYD1WJDhw7lnnvuIT8/n127dtmOoxyi5yaoVhk/fjw1NTX4/X6qqqpIT0+3HUlFSMcM\nVESmTZtGdXU1L730Eh06dLAdRzVAxwwSxc6dUFX13fOqqtBrLrn//vv5wQ9+wIgRI/jmm29ce1/l\nPC0Gse6TT2D4cAgGQ9vw4aHXXCIizJ07l+TkZCZMmEBtba1r762cpd2EeBAMwoABoccbNoCFZd5f\nfvklfr+fCy+8kNmzZyPSZKtUuUS7CcpV7du3p6ysjPXr13PvvffajqNaQWcTYl1VFYwcGWoRQOjx\nsmWQne16lLS0NCoqKsjKyqJz586Ulpa6nkG1nhaDWNez57Ef/mXLQq9ZcsYZZxAIBOjfvz8ZGRkU\nFxdby6JaRscMVFS8+eab5Ofn88ILL+ip6pbpmEEssDwtGE19+vRh0aJFXHHFFbz11lu246hm0GJg\nk+VpwWjLyclhzpw5DB48mI8++sh2HNUEHTOwKTsbFi8+dlrQwsBfNJWUlLBv3z7y8vLYvHkzp59+\nuu1IqhFaDFTUXXfdddTU1JCfn8/GjRs57bTTbEdSDdBugk31pwU3bAg9rj+GEEdjCtOnT8fn81FU\nVMRXX31lO45qgM4m2LRzZ2iMoGfP78YK6h7X/Xf48FBXAqyuIXBCbW0to0eP5osvvmDp0qUkJ2vD\n1A3NnU3QYuAFVVWNf+g9sNTYSYcPH6aoqIgzzjiDefPm6bJlF+jUYiypP5A4YEDocYx++zclJSWF\npUuX8u6773L77bfbjqPq0WLgZU2NKcSoDh06UF5eTllZGQ8++KDtOCpMO21e0Nj5BR5bauykjIwM\nAoHA0fMYxo4daztSwtMxAy+oG0is+9BXVYU+9N27R+f3POS9995jwIABzJs3j8GDB9uOE5d0ADER\nnGzgMYa8+uqrDBkyhOXLl9O3b1/bceKOFgOviPa3d5zMNlRWVjJmzBheeuklevfubTtOXNHZBK9o\nyfkHcbTIqKXy8vJ46KGHKCgo4JM4Oj8jphhjXNlCb+WQv/3NmE2bvnu+aVPoNa/asMEYCG0bNjSe\nf9MmYzIzQ/ts2GBMWpoxjz564n71n9ffPzPz2OPGoEceecScc845Zvfu3bajxI3wZ6/pz2hzdnJi\nc7QYxNqH4PhicLL89fd9+OGT/zkbKiqvvBJbhbIBd955p+nTp4/57LPPbEeJC/FdDIw58QPmVY19\n8BvLf/zrzz9/YiE52Yc7kkLpkRZXbW2tmTx5ssnJyTGHDh1y/f3jTXOLQURjBiJyv4i8JyJbRGSp\niJwaYa8l/tStFfD5QtvJ1go0tMho8+bvfr5lS9PXPIhkNaNHrq8gIjz++OOkpaUxevRojhw54nqG\nhNScitHYBgwCksKP7wXuOcm+zpW6WOsmHK+x/Md/Mz/6qDHp6cbMnn1s16EpkbSaPNTiOnTokMnJ\nyTGTJ082tbW1VrPEMtzuJgCXAwtO8nPn/nQeac62Okdzf69uv/of0OefP/mxIy2UHioGxhjz+eef\nmwsvvNDcddddtqPELBvFoAwYdZKfR/vP7D43WigtfY9ICqVHW1y7d+8255xzjnm0/syKarbmFoMm\nFx2JyFqgS/2XAAPcYYxZGd7nDqCPMabkJMcxM2bMOPrc5/PFx1Vzo73ox80lxx5e3vzJJ5+QnZ3N\nAw88wFVXXWU7jqcFg0GCweDR57NmzcK4sQJRRMYDk4AcY8zXJ9nPRPpeURPJhyBOVgDGgm3btjFo\n0CAWLFhAXl6e7Tgxw5UViCKSD0wDik5WCDyvtaPocXqKsVf17t2bZcuWMXr0aF599VXbceJORC0D\nEfkQSAH2hV96xRjz343s692WAbTuG97Dzep4tmrVKkpLSwkGg5x33nm243hec1sGEV3PwBhzTiS/\nH/O6dz/2gx9jZwvGqiFDhnD//ffj9/uprq6muxZfR+jFTcBTNy9VzTN27Fhqamrw+/1UVVWRkZFh\nO1LM01OYQZv7MezWW29l48aNrFu3jg4dOtiO40l6PQOVEIwxlJaW8umnn1JWVkZKSortSJ6jxUAl\njG+//ZaSkhI6duzIggULSErSy3TUpxc3UQkjOTmZRYsWsXPnTn7+85+jXzqto8VAxYV27dpRVlZG\nMBjknnvusR0nJulsgoobp512GhUVFUcvvz5p0iTbkWKKFgMVV04//XQCgQD9+vUjIyOD4cOH244U\nM7QYqLjz/e9/n/Lycvx+P2lpaQyoW1mqTkrHDLwiga+MHA0XXHABL7zwAldeeSVvvfWW7TgxQYuB\nV3jkkmPxxOfz8fvf/57Bgwfz0Ucf2Y7jedpNcFtjqx3rX7sQQkujdTl0xIYPH86+ffvIy8tj8+bN\nnH766bYjeZa2DNzmdgtAux9MmjSJ0tJS8vPz+fe//207jnc153JITmzE42XPWquh6wxG65JjHr2U\nmdtqa2vNlClTTFZWlvnyyy9tx3EVTl32zCm6HLmehq6dEM2TpfRqTADU1tYyZswYDhw4wLJly0hO\nToxesi5H9qrGro7UvfuxYwTZ2XrWpMOSkpKYP38+33zzDZMmTdJly8fRloHb3D5dOk5u2+6kL774\ngkGDBtGvXz/uu+8+23GiTs9aVCF6rYYG7d+/n+zsbCZMmMAvf/lL23GiSouBUk3YtWsXWVlZzJo1\ni3HjxtmOEzWuXANRqVjWrVs3KioqGDBgABkZGQwZMsR2JKt0AFEltPPOO48VK1YwceJEqqurbcex\nSouBSniXXHIJCxcupKSkhG3bttmOY40Wg4boqr2Ek5ubyyOPPEJBQQGfJOg5IVoMGmL7pCEtRlZc\neeWV3HbbbeTl5bFnzx7bcVynswmNsblqT9cGWDVjxgxWrVrFhg0bOPXUU23HiZjOJsQyPYPRqpkz\nZ7Jnzx6Ki4spLy8nNTXVdiRXaDehIXpD1YQmIjz22GOkp6czevRojhw5YjuSK7Sb0BDbq/a0m+AJ\nX3/9NYWFhZxzzjnMmTMHkSZb2p7k6gpEEZkKPABkGmP2N7JP7BQD22wXI3XUgQMHGDBgAIWFhfz6\n17+2HadVXCsGItINmAv8ALhQi4GKNzU1NWRlZfGzn/2Mm266yXacFnNzAHE2MA0oc+BYSnlO586d\nCQQCZGdnk5mZydVXX207UlREVAxEpAjYaYzZFqv9KaWao2fPnqxZs4aBAweSnp6O3++3HclxTXYT\nRGQt0KX+S4ABfgVMB3KNMQdE5K/ARcaYfY0cR7sJKuZt3ryZ4uJiVq5cyaWXXmo7TrM41k0wxuQ2\n8gb/BfQEtkqoWdANeENELjHGNLh8a+bMmUcf+3w+fAl6+S0Vu/r27cv8+fMZNmwYGzZs4Ic//KHt\nSCcIBoMEg8EW/55jU4vhlkEfY8y/Gvm5tgxU3Hj66ae58847qa6uprvHZ3lsrEA0hLoQSsW9sWPH\nsnfvXvx+P1VVVWRkZNiOFDFddKRUBG677TaCwSDr1q2jQ4cOtuM0SC97ppQLjDGUlpby6aefUlZW\nRkpKiu1IJ9BioJRLvv32W0pKSujQoQPPPPMMSUneOuVH75uglEuSk5NZtGgRu3bt4sknn7Qdp9W0\nZaCUQz7//HNSU1M911XQboJSCtBuglKqhbQYKKUALQZKqTAtBkopQIuBUipMi4FSCtBioJQK02Kg\nlAK0GCilwrQYKKUALQZKqTAtBkopQIuBUipMi4FSCtBioJQK02KglAK0GCilwrQYKKUALQZKqTAt\nBkopQIuBUipMi4FSCtBioJQKi7gYiMhNIvKeiGwTkXudCKWUcl9ExUBEfMBQoLcxpjfwv06Ecksw\nGLQd4QRezATezKWZnBVpy+AG4F5jzLcAxpi9kUdyjxf/x3kxE3gzl2ZyVqTF4Fygn4i8IiIbROQi\nJ0IppdyX3NQOIrIW6FL/JcAAvwr/fpox5icicjHwAnB2NIIqpaIrohuvishq4D5jzMbw84+AS40x\n+xrYV++6qpQlzbnxapMtgya8COQAG0XkXKBtQ4WguWGUUvZEWgzmA38QkW3A18DYyCMppWyIqJug\nlIofrq9A9OoiJRGZKiK1IpLugSz3h/+OtojIUhE51WKWfBF5X0Q+EJFbbeWol6ebiKwXkXfC/4Zu\ntp2pjogkicibIlJmO0sdEekkIovD/57eEZFLG9vX1WLg1UVKItINyAV22M4SVgn0Msb8GPgQuN1G\nCBFJAh4D/EAv4GoROc9Glnq+BX5hjOkFXAbc6IFMdaYA79oOcZyHgdXGmB8C5wPvNbaj2y0Dry5S\nmg1Msx2ijjHmJWNMbfjpK0A3S1EuAT40xuwwxnwDLAKGWcoCgDHmn8aYLeHHBwn94+5qMxMc/UIp\nBObazlIn3KLMNsbMBzDGfGuM+byx/d0uBp5bpCQiRcBOY8w221kaMRFYY+m9uwI76z3fhQc+eHVE\npCfwY+D/7CYBvvtC8dIg3FnAXhGZH+6+PCki7RrbOdLZhBN4cZFSE5mmE+oi1P9Z1J0k0x3GmJXh\nfe4AvjHGPOtGplgiIh2BJcCUcAvBZpbBwG5jzJZwV9gr0+jJQB/gRmPM6yLyEHAbMKOxnR1ljMlt\n7Gcicj2wLLzfa+EBu4zG1iZEO5OI/BfQE9gqIkKoOf6GiFxijNljI1O9bOMJNTtzopmjCZ8CPeo9\n7xZ+zSoRSSZUCBYYY1bYzgP0BYpEpBBoB/yHiDxtjLE91b6LUKv39fDzJUCjg8BudxPqFinR1CIl\nNxhj3jbG/Kcx5mxjzFmE/vIuiHYhaIqI5BNqchYZY762GOU14PsicqaIpABXAV4YKf8D8K4x5mHb\nQQCMMdONMT2MMWcT+jta74FCgDFmN7Az/FkDGMhJBjgdbxk0weuLlAzeaOI9CqQAa0MNFl4xxvy3\n2yGMMUdE5GeEZjeSgHnGmEZHo90gIn2Ba4BtIvIWof9n040xFTZzedjNwEIRaQtsByY0tqMuOlJK\nAXrZM6VUmBYDpRSgxUApFabFQCkFaDFQSoVpMVBKAVoMlFJhWgyUUgD8P82zhRDbFODTAAAAAElF\nTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x105d7dcc0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"mu1 = [2, 2]\n",
"cov1 = np.array([[1, 0], [0, 1]])\n",
"class1 = np.random.multivariate_normal(mu1, cov1, 20)\n",
"\n",
"mu2 = [-2, -2]\n",
"cov2 = np.array([[1, 0], [0, 1]])\n",
"class2 = np.random.multivariate_normal(mu2, cov2, 20)\n",
"\n",
"w = np.sum(class1, axis = 0) + -np.sum(class2, axis = 0)\n",
"print(w)\n",
"\n",
"def line(w, x):\n",
" slope = -w[0]/w[1] \n",
" return slope*x\n",
"\n",
"plt.scatter(class1[:, 0], class1[:, 1], color=\"blue\")\n",
"plt.scatter(class2[:, 0], class2[:, 1], color=\"red\", marker=\"x\")\n",
"x = np.array(range(-5, 5))\n",
"y = line(w, x)\n",
"plt.plot(x, y, color=\"black\")\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It can be seen that Hebb's rule works quite well in this example dataset. In general, Hebb's rule is robust to outliers because every sample \"weighs in\" in the construction of the decision boundary. This concludes the Hebb's rule tutorial!"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment