CalvinTChi/Whitening_tutorial.ipynb

## Whitening_tutorial.ipynb
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Whitening Tutorial\n",
    "### Author: _Calvin Chi_\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Motivation\n",
    "Whitening is a common preprocessing step in machine learning that aims to transform the data such that it has the identity covariance  matrix. Once a data set is whitened, its covariance matrix is a diagonal matrix with all entries equal to one. \n",
    "\n",
    "Why perform whitening on your data? There are two main reasons for doing so: \n",
    "\n",
    "1. Data with independent dimensions are often easier to model statistically.\n",
    "2. Forcing all dimensions to have equal unit variance treats each dimension with equal importance.\n",
    "\n",
    "Let us graphically illustrate why whitening might be important with a contour plot. In a contour plot, every point on the contour is the same \"distance\" from the mean. In the contour plot plotting the real distribution of our hypothetical data below, not only are the two dimensions not independent, but the variance in each dimension is different. If we are blind to the contours, then we might conclude that a point in the y-direction is closer to the mean than a point in the x-direction, even though in reality, they are on the same contour. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALcAAADDCAYAAADA3YBIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl4FEXCh9/iBsMpIiCiIIqKF3ggioiLIKeCuOwqeC64\n64XyrXitq6yugqKrrq6KhAUVRTl1kUOiqAiKilxyyX1JIEBCDnLP/L4/uhOGZDKZkJkcTb3PM8/0\nUV1VPXlTXV1dVW0kYbF4kSrlnQGLJVpYuS2excpt8SxWbotnsXJbPIuV2+JZrNylwBgzyRjzbIj9\nqcaY08suR0Xmo6sxZld556OsqbRyG2O2G2PSXYH2GmMmGmNOKONsyP0E3ynVlbQdiv9HKE+MMX5j\nTJr7Wx4wxnxhjBlUguMr5D9PpZUbR6q+kuoCHYBLgCcLBjLGVCvrjFVSLnB/y7OAScAbxpinyjdL\npaMyy52PpD3AfKAd5JdE9xpjNgG/utv6GmNWGmOSjDFLjDHnB4vLGFPLGJNhjGnkrv/NGJNjjIlx\n1581xrwScEgjY8xnxpgUY8xSY0zrgLj8xpgzjDF3A7cAj7il46fu/ubGmBnGmARjzFZjzAMBx44y\nxkw1xrzrxr3GGHNxwP5Qx9Z2rxSJxpi1wKUl+C0TJU0G7gEeN8Y0dOO80xizzs3LFveccK+W84Dm\n7rmlGGOaGmMuM8Z87/7ee4wxrxtjqoebj4ggqVJ+gG1AN3f5VGAN8A933Q98DjQAagLtgX04f2QD\n3OYeX6OIuL8BbnSXFwCbgJ7u+iLgBnd5EnAA56pRFZgMTAmIxw+0dpcnAs8E7KsC/IxztakGtAK2\nAD3c/aOADKCnm+fnge/DPHaMew4NgBbub7MzxG+Zn8+AbdWBHOA6d7030Mpd7gIcBtq761cDuwoc\n3wG4zM3racA64MEydaS8JS2F3NuBVCDJXX4DqBnwx+oaEPatQLHcbRuALkXE/QzwmitsPPAAMBqo\nBaQDDQOEfSfguF7A+hByPxuwryOwo0C6jwP/DZB7QcC+c4H0MI/NF91dH1ZQvuLkdrfHAzcXccws\nYLi73DVU/G6Yh4CZZelIZa6PCqcEXVjE/sAbnNOA2wIv3TglUzNjzC3AOHfbIkl9cEq9f+GUPr8A\nXwATcKTaLCkpIJ59AcsZQEyY+T8N51IeGFdVnCtDsLjTgVrGmCphHNuco89/Z5h5ysetQpwEJLrr\nvYCngTNxSuM6wOoQx5+F8xte7IatBiwraT5KQ2WWuzgCWzF2As9Jer6IsB8WWP8eaAsMAL6WtN4Y\n0xLn0vx1BPKTl6dtks4KM3wgu4o5Nh5oCax311uGncsj3ADkAj8aY2oCM4AhwKeSfMaYWTjVpaLy\n+hZO1ekPkg4bYx4CBh5DPo4ZT9xQhsF44C/uTY4xxpxgjOmTd5NYEEnpOH+Y+3BKcYDvgL8ErMOR\nP2447ANaB6z/CKQaYx5xbwCrGmPOM8ZcEkbcxR07FedmsIExpgVOtao4DIAxppExZjBONW+Me5Wq\n4X4OAH63FO9R4NxONMbUC9gWg1NtTDfGnI1zg1qmeFXuo0oSST/j1DvfwLnMbsK5qQzFNzhXth8D\n1mM4utoQrJ1bRSxPAM51Ww9mSvIDfYGLgK3AfuAdoF7AsUHjluQr5th/ADtwbprnA+8Fiasgq4wx\nqTi/zV3AQ5JGuemlAsNx/mkSgZuBT/MzJW0ApgBb3RaapsDDOC1EKW7ePgojDxHFuJV9i8VzeLXk\ntlis3BbvYuW2eBYrt8WzRL2d2xhj71gtUUVS0GbTMim5j/Xx6dNPP11uj/fLK217ziX7hMJWSyye\nxcpt8SwVWu6uXbsed2nbc44cUX9CaYxRtNOwHL8YY1B53lBaLOWBldviWazcFs9i5bZ4lojI7XaW\nX2GMmR2J+CyWSBCpkvtBnNHNtlnEUmEotdzuMKbeQCwlG3ZlsUSVSJTcrwAjcaYHsFgqDKWS2xjT\nF0iQtAJbalsqGKXt8noFcL0xpjfOhDX1jDHvSTpq8O2oUaPyl7t27Vquj5gtlZuvv/6ar7/+Oqyw\nEXv8boy5GnhYUr8C2+3jd0vUKMvH79ZiS4XBdpyyVGpsxynLcYmV2+JZrNwWz2LltngWK7fFs1i5\nLZ7Fym3xLFZui2excls8i5Xb4lms3BbPYuW2eBYrt8WzWLktnsXKbfEsVm6LZ7FyWzyLldviWazc\nFs9i5bZ4Fiu3xbNYuS2eJRITYZ5qjPnKGLPWGLPGGDM8EhmzWEpLqectMcY0BZpKWmmMiQF+BvpL\nWu/ut/OWWKJGVOctkbRX0kp3OQ1YDzQvbbwWS2mJaJ3bGHM60B74IZLxWizHQmlnec3HrZJMBx50\nS/B87CyvlkhR5rO8GmOqA58B8yS9WmCfrXNXcHw+PwcPZgBQs2ZVatWqRo0aVTGm4k+5HqrOHYkb\nSgO8CxyUNCLIfit3OZGZmcvevWnEx6e6385yfHzaUev796dTv35NqlQxZGbmkpXlIzvbR82aValZ\ns1q+8DVrVnO/j6znLQ8d2oHevc8s83OMttydgUXAao5MYfy4pPnufit3lNm1K5m4uK0sWrSDXbtS\n8oVOS8umadMYmjaNoVmzujRrFkOzZgXX69KkyQnUqFH1qDj9fpGd7XNld4TPW877B8hbPnAgnYcf\njmPevMFccknZtiVEVe4wErdyR5iUlCy++mobcXFbiYvbSmJiBtde25quXU+jVauG+dI2alSbKlXK\npmrxyScbuP/+uSxdOpQWLeqVSZpg5a705OT4+PHH3/JlXr16H5df3oLu3VvTvXtrLrywaZlJHIoX\nX1zClClr+PbbO4mJqVEmaVq5KxmS2LjxYL7M33yznVatGubL3LlzS2rXrl7e2SyEJIYO/R8HDmQw\nc+YgqlaNfu8OK3clYdOmg7zwwhIWLNiCRL7M3bq1pkmTE0ocX64PUtKdT3J64eXAbYcz4cYroM8l\nUJpGkuxsHz16vM+llzZn7Ngexx5RmFi5KziSiI1dzuOPf8mIEZczcOC5tG17YthNcTm5MGURxC6A\nhOQj0mblQN3aUP8EqJf3XQfq13G+A7dVrQJvzXP2jbkdOp977Odz8GA6nTpN4JFHrmTo0A7HHlEY\nWLkrMPv3H2bYsNns2JHM5MkDaNeuSdjHpmfBhDh4aRa0aQYjbnC+86StU7NkpbDPBx8ugqc+gHYt\n4blb4cJWx3BSwMaNB7nqqol89NFArrnmGCMJg1ByIymqHycJSzDmzt2oZs1e0iOPLFBmZk7YxyWm\nSv/8WGoyROr/nLR0Q2TzlZkt/Xu2dPKt0i0vSZv3HFs8X365VU2ajNWvvx6IbAYDcP0K7l5ROyL1\nsXIX5vDhbN133xy1bPmKvvpqW9jH7Tkojfyv1OgW6fZXpHU7o5dHSUo5LD0zRTrxFunet6T4xJLH\nMX78z2rT5t86cOBw5DOo0HLbakkZs3x5PEOGzOSii5ry5pt9aNCgVrHHbN4DY2fBtCVw6zXw1/7Q\n8qTi0xLCB/gQue7H527LW87bLqA1taga5C3nB1Jg9DSYtBD+0hNGDoAGMeGf88iRC1i2LJ7PPx9S\n6GFRabF17gqAz+fnpZe+4+WXv+fVV3tyyy3nF3vMyq0wZgZ8uQru6QUP9IWT6hcOl0gO77OfmRwk\nDb8rsCNxVaAqhqoYqrvfVYFq+cuGakA2ohZVeIjmXEM9TBDJd+6Hf0yB2T85gt/fB2rXDO/cb7xx\nKo0b1yY29vqI9lmxde5yZvv2JHXpMlFXXz1R27cnhQzr90vf/CL1GiU1v10aO9OpHgRjtzL1rHaq\no1ZplHZqizKUplxlyqcc+eWXP+w8+uXXQh3S9Vqnm/WrflJqkWHX7ZRufF465Q7pnflSTm7x8aem\nZumii97Wiy8uDjtP4YCtc5c9qalZ+uijX3TjjR+rfv3ReuGFxcrN9RUZftNv0vNTpQsekNrc7UiT\nmV04XLp8mq2Dulub1VGr9JJ2K0FBAh4jufJrkvbpHC3XeoWuJy/dINUdJD33cXhxz5q1XtWqPaOU\nlMwI5NQhlNwR689tcR6Tz5mzicmTVxMXt5VOnVowaFA7xo/vR6NGtQuF3xLv1KOnLYHdB2DgFfDa\nMOjSDqoEPNzzIX4gldkksZBkLqQO/WjEK5xOHSJbh00ml5kc5C+cTFsK5zkPvx9i45ymwuH9io93\n3rxNDBs2mxkzBlG3bhh1mQhg69wRYPPmRCZMWM6kSato06YRt99+IQMGnM2JJ9YpFHbbXkfmqUuc\nOuzATjCosyN01QBPhdhABrNJYg6JNKE6/WhELxpyEtF59J5IDneymW7U5wGaBa13gyP2n9+EDbth\n3tMQU/T/AACTJq3k0Ue/YNasP3DFFadGNM/2hjIKZGbmMmvWesaPX86aNQncdtuF/OlP7TnnnMLN\nGDsSXKEXw/YEuLET/P5KuPo8qFag4I0nmzkkMZtE0vDRj0b0oxFnUHyrSmmIhtiSGD16Me+88zPz\n5w/h7LMbRzzfVu4IsmZNArGxy/ngg19o374pQ4d24IYb2lKz5tE1vJ37Ydpip4TeuhcGXO6U0F3P\nLyx0Erl8ziHmkcSvZHAdDehHIzpwAlWKkCyS+BG3sJFO1GV4hMT2+fwMHz6PxYt3MW/eYJo3rxuV\nvIeS29a5wyAtLZuPP15DbOwKdu5M5q67LuLHH4fSqlXDQmFnfue0SW/aAwM6wT8HO0JXL/BLH8bH\nlyQzhyRWkEZn6nErJ9GFetQo47mSPiWRKpiIiZ2ZmcvgwTNJSspg0aI7qF8/uledorAldxFIYtmy\nPcTGLmfatHVcddVpDBvWgZ4921CtWmH59ifD/eNg1TZ4+S7o0b6w0Fn4+YYU5pLEd6RwCTH0piHX\nUJ8TInxjGC6H8dGH9bxGKy4keM/DkoidlJRB//4f07RpDO+917/QFS3S2JK7hEyZ8gsvvLCElJQs\nhg7twJo194a8rE5fAg+8A7d2hUkPHv1gIwexlFTmksRXJHMOtelNQ0ZxKg0K/Pw+UklnDVCNKtTA\nUCPguybmqG2R+dNNYB+XERMRsXfvTqFnz8l0796al1++rtwHUFi5A8jJ8TFixOfExW3ljTd60a1b\n65B/oP3JcN/bsHo7zHocLj/b2e5HLOcwc0liAYdoQQ360JARNKdJgZYOP9mk8C1JzCaF76hNWwBE\nNn6ykfvxk+Uu5+AnC8AVvnq+/NVpwhm8Q9UiRC3IHrKZwgFmcnbQ/SURe+3aBHr1+oDhwzvy1792\nqhgj54tqAI/Uh0ryECchIU1XXz1RvXt/oKSkjGLDT/1Wanqb9MhEKd19JuGTX+9pn67RL7pe6/S2\n4rVThR9Y+OVTqpZph57WKnXSRt2q/ZqqHB0KO79+5ShXh5WjJGVrnzK1S5t1rxL0QdhxPKxt+reC\nd/nz+aShr0udH5VS00PHs3jxDjVpMlaTJ68KO+1IgX1CGZqff96j0057RU888UXIp4iStC9Jumm0\ndPY90vfrj2zfpgwN1q8arF+1rognexnarN/0qtboWq1TX8VrnLK0O2LnkaqftFa95Vfoc5CklUpT\nV/2iwyr87LwkYn/zzXY1bvyi5s/fdKzZLhVRlRvoCWwANgGPBtlfRqd5bHz44Wo1bvyipk5dU2zY\nj791+jg/MlHKyHK2+eTXu9qnTlql97RPuQX6c2QrQfs0Ses1UKvVRbv1gg5rXeF+HzlZ0spPpLnP\nSfHrdSz45dd69VeyFhUb7g/aoE9UuJ/1sYgdF7flmPIbCaImN06ns83A6UB1YCVwjiqB3Lm5Po0c\nuUCtWr2qlSvjQ4YNLK0DBwYEltbbdKQqk6s0HdSn2qShWqWO2q7Hlawl8hcsJf1+aetS6cN7pYcb\nSy93kabcJz3aTBp9qbTw31JKQonO64BmaLPuDhlmtg7q99ogX4F/sMomthRduTsB8wPWHwMeUwWX\n++DBdPXo8b66dXs3ZCd6v1/6aJFTWj866Uhpnde5KK+0zpPEpwzt1ktapcu0WfcoUXPlUxBLDmyT\n5jwrPXWW9NSZzvL+rUf25+ZIa+dL/x0sjagv/aeftGyqlF38vYBPmVqtzsrQtqD70+XTNfpFywr0\n+quMYkvRlfsmYHzA+hDgdVVguX/5ZZ/OOOM1jRgxXzk5RddN9yZKA0dL59wbvLQeol+1PeBmMU0r\ntFa9tVUPKVtBStvDSdK346WXrpL+eqJTWm9d6vwHhSIjRfpukvTK76T/ayhNHiZtXOTYWAS/6VXt\n1LNB972peD2krYW2PzCu8oktRVfugZVJ7ri4LWrc+EW9++7KkOE2/uaU1o8FlNaStFQp6qRVel8J\nR13SD+lLrdaVStS84BEunyHdX1N6oqX083Snfh2EvatX68N+/YrO2MGd0oyR0j1GerqtlB286+gh\nfamVuqRQNehrHVInrdKugH9Kv196e57Tdzy5mJFgn332a4USWwotd2nbuX8DArt5nQrsLhioIkxh\n/MUXW7nllhnMmDGILl1OKzJcWgbc/qozlGvkjUe25yL+yW6epSXdaHDUMTU4BUM1GtAteKRtfwf9\nR8MP78P0h2DHT9DxVmje7qhgjc8+m77jxhU+PicL1sxxjt/4FVz8B+h0J1Q/uuuonyzi+Q+JzKIl\nz7hjbpxurGP4jeWk8RqtaIFz3M79MPR1SEqDBc84UzwEQxLPP/8tb721jP/974906hTZnn0loSRT\nGJe25K4GbMG5oaxBBb2hjIvbopNOelGLFm0PGS413bk0D3298FX/QyXoTm0scnTLRt2uRM0pPjO7\nf5FmPiI9dor0XAfpi1ek5L2Fw/n90uYl0gd/caox/+oqLZkgpScXCuq0yLyrteqlLRqubO3P37dQ\nh9RVv+hZ7VSaW5L7/dK4eVLjwc4AiVAjaVJTszRw4Mfq2HG8fvstpfjzK2OIclNgL+BXnFaTx4Ps\nL6PTDE4kxD6kHHXWam0IdnPokqQ4bdAfw8+YL1daFydNvE0a0UB6o7f000fSb2uk2aOkv58hjTrb\naRo8UDjvR1pkhrktMo8pRd/l//MdUo4e1Tb10Br9oCNSbt8nXfukdMkIac2O0FncsiVR55//pu68\n8xNlZIQ/9URZElW5i/uUp9yREFuSxmiXnlJoE/zK1RpdqzQdw1O6zDRp6fvSaz2cZsCPHpC2/1To\nZtOvHB3SN9qmh90Wmb8oUXMKtcgs1CFd7ZbWh4OU1qOnFT/uccGCzWrSZKxef/0H+Yu76S1Hjku5\nIyX2NmWok1ZpfxjjFPfqv9qmkcea5aD45VeaVmuXntNqddYG/VEJmqxsHSwUNkk5esQtrX8MUlpf\n+n/Fl9Z+v18vvbREJ588tkRzqpQXx53c06evjYjYknSftihWQerEQcjRIa1Sx+BNgSUkUzu0R//R\nWvXUWl2nPXpDmSp8Pn75tVppGqNd6qzV+qd2HVVav+2W1mOmF19ap6dna8iQmWrf/u1iR+lXFELJ\n7blegW+88SOjRy/m88+H0L59syLDpWVAr3/A2S1g3L1HD8jNYympbCSDf3F6WGlXoz4nchNr6UFd\nLi/Ua+9Il9XqGGrmd2cN7NqaywGSmEsWO2lAT05jDHW4oNAggk1kMJck5pKEwdCHhrzLmbR2h6Pt\nSIA/vQ6pGfDN83Buy9B537kzmQEDPqZt2xNZvPgu6tSpeFMklxTPDFaQxJNPLmT69PXMnz846CiZ\nPMIR24e4iQ3cQzN6FGj6K45UfsBPRkA31SPdV4Nvy0ZkIXKoQh0acB31uAJToHvsbrLyhU7GRy8a\n0puGtKN2vvwSjJsPf/8AHh7gNGkWHNYW+Jt9//1uxo9fziefbOCJJzrz8MNXVIzuqmHi+TGUOTk+\n7r77M9at289nn93MSScV3Z85HLEBpnGA2STxLm2KHHpVFuwnh/kkMYckdpFNDxrQh4ZBx1du3+eU\n1mmZMHF40aX1gQPpvP/+KmJjV5Cb62fYsA7cdtuFxzQHeHnj6ZE4aWnZDBo0DWMMCxfexgknFP26\ninDFTsPH68TzFmeUi9jJ5BLHIeaQxHoy6Eo97qUpnahH9QL5OZwJc5c5I+u/XA2PDYT/C1Ja+/1i\n4cJtxMYuZ/78zVx/fVveeqsPV13VslKV1CWhUpfcCQmH6dPnQy64oAnjxvULOrYxj3DFFuI5dpOO\nn+cp+klmpMnEz1ck8xmJ/EQaV1CP3jSkC/WoVWDAcHrWEaE/XwGXt4VBV0L/y+HEAu9a2rMnlUmT\nVjJhwgrq1q3BsGEduOWW82nYsJjJRioJnqyWbN2axHXXTeaPf2zHM89cE7L0KYnYrxPPlyQziTNp\nGOULmw/xE2nMJpEvSeY86tCXhlxLA2IKDBjOyIJ5PztTRcxfDped6UwV0f9yaFxA6NxcP/PmbSI2\ndgWLFu1g0KBzGTq0A5dc0txzpbTn5P755z306zeFv/+9C/fcc2nIsMci9kTa0ChKszoBbCSD2STy\nGUk0pBr9aEgfGhUaX5mR5Yg8dTHMWw6XtHFK6AGdgs/2um1bEhMmrGDixJW0bFmfYcM6MGhQuzJ7\ns1h54Cm5FyzYwuDBM3nnnb4MGHBOyLAVSewEcviMRGaTRDK59KUh/WjEmQXm48vMdoSetgTmLIOL\nz3BK6AGXQ5MgjTa7diUzffo6pk5dx6ZNBxky5AKGDu3AeeeF//qRyoxn5J4zZyN33fU/ZswYROfO\noRtuJfjdk3BSPfhoZNFiA/yXfXzAfqbRNqJi+xELSWYKB1hLOtdSn3404lJiCrV0rNoGr/4PPlkK\nF7V2SugbO8HJRbRoTpu2lldeWcqvvx6kf/+2/P737ejWrRXVq5fP/CflhWdaS+rVc7pqhttkde2F\n8K9P4aFY+NvvixblLGpTDcOj7OAhmtOOIvp+hkk2fuaQxAT2UYeq3EET/kPrQjeGAN+udSaYX7EV\nHuwH6/4DzRoVHfehQ5ncf/9cli3bw8sv96B79zMi/rYCr1CpSm6A2NjlvPDCEpYu/VPQWVQLknAI\nnpsGk7+G+3o7DzaC9VvOxs9MDvIWe2lLbbpQP/+1G0e/ckPkQsBy4VdzLCONVtRiKCdzOTGFmhMl\np7Vj9HTYmwSP3Ai3/Q5qFVM1/uab7dx++yf06XMmY8f28MRTxNLimWpJHiNHLuCnn/awYMGtYZda\n2/fBqClOi8OjA+He3sFlSseX376c90oN59sEfBfelheuKoY21OLcIKV/rs+5ORwz3akmPTYQbrqy\n6CeIeWRn+3jqqa94771VjB/fjz59zgrrnI8HPCd3ad6xsmYHPDkZlm+BUTc7JWZxcpWWzGyY+IUz\nQeapjeGxm6Bnh/DeEbl+/X4GD55Jixb1iI29vlI+RYwmnnwnTmnfsfLdeqnLY850DTOWFD9O91g4\nlOb0nW56m9T3GWnJuvCP9fv9euONH3TiiS9o3LhlFbpPdXmCV7u87tqVrFNOeVmzZh3jJDZ+ad4y\n6aLhTl/nL0OPGw6bvYnS4+86728c/JK0elvJjo+PT1WvXpN16aXvRPUFpV4glNyVsloSyLJle+jV\n6wM+/3wIHToU3cU1FH6/Uxd+cjK0bgrP3woXt3GqEynpR96lHrhc8DtvOfkwrN8NN3eBh/tDq6Yl\ny8unn27gz3/+jGHDOvDUU1cfd017JcVzde6CzJixjoce+pylS//EKafUK/6AIsjJdd6l/szHsO8Q\n1KjmvEe9Xh3nXep571QvuK1eHahf58i2tqcEf4IYjEOHMvn66+3ExW0hLm4rfr94993+XHllMR2w\nLcBxIDfA6NHfMn36ehYtuiNkz8Bw8Pudlo0aUWhpy8nxsXTpbuLithIXt5U1axLo1KkF3bu3pnv3\nM7jggpPLfV7rysRxIbck7rzzU1JSspg+fVCFEUQSGzYcyJd50aIdtGnTiGuvbUX37mfQuXNLatWq\nVM/SKhTHhdwAWVm5dO/+PldccSpjxlxbJmkGIyHhMF984cj8xRdbqVLFuCVza7p1a03jxqV7Amo5\nQtTkNsaMBfoC2TiT89wpKblAmDKTG5xRJh07xvLkk1dx553tIxp3ZmYu+/alER+fRnx8Knv3HlmO\nj0/LXz98OJuuXU/Pr2qceWYjz3U1rShEU+7uwJeS/MaYMQCSHisQpkzlBtiw4QAdO8ZywgnVqVWr\nGjVrVnO/qxaxXvWobTVrViM1NcsVN0/aVA4fzuHkk0+gadMYmjWrS7NmMTRrFlNgvS5Nm8aEHDhh\niRxlUi0xxgwABkoaUmB7mcsNkJGRQ2JiBpmZuWRl+dzvo5dD7YuJqZEvap60jRrVrjB1eYtDWfUK\nvAuYEsH4SkXt2tU55RTbseh4pli5jTFxQLBHEU9Imu2G+RuQLenDYHFUhFleLd6gJLO8lrpaYoy5\nAxgGdJOUGWR/uVRLLMcHUauWGGN6AiOBq4OJbbGUJ6VtLdmEMy93orvpe0n3FghjS25L1DhuHuJY\njj9CyW0bYy2excpt8SxWbotnsXJbPIuV2+JZrNwWz2LltngWK7fFs1i5LZ7Fym3xLFZui2excls8\ni5Xb4lms3BbPYuW2eBYrt8WzWLktnsXKbfEsVm6LZ7FyWzyLldviWUottzHmr8YYvzEmxKtBLZay\np1RyG2NOBboDOyKTHYslcpS25P4X8EgkMmKxRJpjltsYcwOwW9LqCObHYokYIecKDDHD69+Ax4Ee\ngcEjmC+LpdSElFtS92DbjTHnAa2AVe7rMFoAPxtjLpOUUDC8ncLYEinKdApjAGPMNuBiSYlB9tm5\nAi1RoyzmCrT2WiocdpZXS6XGzvJqOS6xcls8i5Xb4lms3BbPYuW2eBYrt8WzWLktnqVCyx3uY1Yv\npW3POXJYuStY2vacI0eFlttiKQ1WbotnKZO+JVFNwHLcU26vx7ZYygtbLbF4Fiu3xbNUGrnLY34U\nY8xYY8x6Y8wqY8xMY0z9KKfX0xizwRizyRjzaDTTCkjzVGPMV8aYtcaYNcaY4WWRboE8VDXGrDDG\nzI5kvJVC7nKcH2UB0E7ShcBGnEHRUcEYUxV4A+gJnAvcbIw5J1rpBZADjJDUDrgcuK+M0g3kQWAd\nER7RVSmrcVjoAAABaklEQVTkppzmR5EUJ8nvrv6AMxA6WlwGbJa0XVIO8BFwQxTTA0DSXkkr3eU0\nYD3QPNrp5mGMaQH0BmKJ8AwKFV7uCjQ/yl3A3CjGfwqwK2B9t7utzDDGnA60x/lHLiteAUYC/uIC\nlpSQUzuUFeU5P0qItJ+QNNsN8zcgW9KHkUy7AOXaJmuMiQGmAw+6JXhZpNkXSJC0whjTNdLxVwi5\nIzU/SiTTDsjDHTiXzW6RSC8EvwGnBqyfilN6Rx1jTHVgBjBZ0idlkabLFcD1xpjeQC2gnjHmPUm3\nRSR2SZXmA2wDGpVhej2BtUDjMkirGrAFOB2oAawEzimDdA3wHvBKOf9trwZmRzLOCl/nLkBZX7pf\nB2KAOLep6s1oJSQpF7gf+Byn5eBjSeujlV4AVwJDgGvcc1xhjOlZBukGI6J/X/v43eJZKlvJbbGE\njZXb4lms3BbPYuW2eBYrt8WzWLktnsXKbfEsVm6LZ/l/B3zcRZsnimUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x878c730>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALcAAADDCAYAAADA3YBIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHY1JREFUeJztnXl8VNXd/99fVi0RWQxEFjekKloTwBU3cCui1T51gz7W\n4vM81Vb7aN1arc/LtVUUrbu1rb9WqUVQWZTKIiAqIIqiCRoChD0BshCWQIBs8/n9cSfJzGSyT7Y7\n5/16zes199yz3cl7Ts4995wzJgmHw490aO0KOBzNhZPb4Vuc3A7f4uR2+BYnt8O3OLkdvsXJHQUz\nC5jZcTWc+08zm9fSdYqGmb1uZo/VM+4xweuKm7+57y/UzO43s9kRYZk1hF1XV36S/iXphyHpavwi\ntAAKvmKKmY00s6xY59vS+F5u4BNghJkZgJkdCXQCUipasWDYIODTRpZhsahoOyy7TRMPcn8FdAZS\ngsfnAYuAtRFh6yXlhKS7xMzWmtkuM3upItDMxpvZ4uD7ii9DmpntNbNrg+FXmFlqMO1SM/tBSPpN\nZna3maWZ2W4zm2JmXUPO15Z2qJl9bWaFZjYFOKSmizazDmb2tJnlm9l64PKI8zeZ2apgXuvN7OZg\neDdgDtAveE2FZpZkZmeY2bJgvbaZ2Ytm1rnuj78VkeT7F/AR8Jvg+5eAm4A/RIS9FhI/ALwPdAcG\nAnnAD4PnxgOLI+IeF3I8FMgFTsdrVW8ENgKdg+c3Ap8DSUBPYBVwS11pgS7AZuAOoCNwNVACPFrD\nNf8SyAD6B8tZBJQDHYLnxwDHBt+fDxQBQ4PHFwBZEfkNA87AaxCPDtb7jtb+29b2ioeWG7yuyfnB\n9+fidT8Wh4SdF4wTygRJhZKy8MRIoX7cDPxF0pfymAQUA2eFxHlBUo6kXcCskLxrSnt2MH0nSc9L\nKpc0DfiylnpcBzwraWuwnMcJ6cJImi1pY/D9p8CHwc8BonR1JH0tabmkgKTNwF/xvgRtlniR+1Pg\nXDPrCSRKWg8sw+uL9wROpnp/O7SLsh/oVs+yjgbuDv773mVmu4ABQL8a8j4AJNSR9shg+q0RZW2m\n5j73kUDoTeGW0JNmdpmZfW5mBcFyxgC9a7ooM/u+mf3bzLab2R7gj7XFbwvEi9yfA4cDvwCWAkgq\nBLbhtZbbgq1RLNgC/FFSz5BXgqSptaRRPdJux+tihHI0NY+WbAeOCjmufB/s408DngL6SOoJzKbq\nixItzz/jdUWOl3Q48ABt3J82XblYIekA3o3lXYS30EuCYZFdkkiMmlvIXLyRlgr+BvwyeANmZtbN\nzC43s4Qa0lfkX1faz4AyM7vdzDqb2U/w+uY18TZwu5n1D/53ui/kXJfgawcQMLPLgEsjrqm3mXUP\nCUsA9gL7zexE4Fe1lN0miAu5g3wCJOIJXcFi4Aiqd0kiW67Q8eTIseWHgTeC3YhrJK3A+w/xErAT\nyMS7Maypha3Mr5a0SCoFfoJ3Q1uA16eeVsv1/g2YB6ThfbGnhZSzF7gd7wuwExgHvFdZIWk18Baw\nwcx2mlkScA/wU6AQr789pZZrahOY3GIFh0+Jp5bbEWc4uR2+xcnt8C1Obodv6dTcBZiZu2N1NCuS\nog7TtkjL3di5AQ899FCrzUtorbLdNTfsVRuuW+LwLU5uh29p03KPHDky7sp21xw7mv0JpZmpuctw\nxC9mhlrzhtLhaA2c3A7f4uR2+BYnt8O3xERuM+toZt+Y2axY5OdwxIJYtdx34C1BcsMijjZDk+U2\nswF4i0tfw20Q42hDxKLlfha4F2//DoejzdAkuc3sCiBP0je4VtvRxmjqlNcRwJVmNgZva6/uZjZJ\n0o2hkR5++OHK9yNHjmzVR8yO9s3HH3/Mxx9/XK+4MXv8bmYXAPdI+lFEuHv87mg2WvLxu7PY0WZw\nE6cc7Ro3ccoRlzi5Hb7Fye3wLU5uh29xcjt8i5Pb4Vuc3A7f4uR2+BYnt8O3OLkdvsXJ7fAtTm6H\nb3FyO3yLk9vhW5zcDt/i5Hb4Fie3w7c4uR2+xcnt8C1ObodvcXI7fIuT2+FbYrER5kAzW2Rm6Wb2\nnZndHouKORxNpcn7lphZEpAkKdXMEoAVwI8lZQTPu31LHM1Gs+5bIilHUmrw/T4gA+jX1HzjkY0b\ndzFjRgbFxWWtXRVfENMdp8zsGOAT4OSg6K7lroO9e4tZujSLP//5K5Yu3cKQIYmsXVvAzTcP56c/\n/QGDB/eiY0d3a1QTtbXcTd3lNbSQBOBd4I4KsStwu7yCJLKyCklLyyE1NYfU1FzS0nLYvn0fycl9\nGT8+hcmTf0K3bl1YvXoHL720nMsvn0xu7j5OOaUPKSlJJCf3JSUliR/8oC8JCV1a+5JahRbf5dXM\nOgP/BuZIei7iXNy13CUl5WRk5AclziEtLZfU1By6du0UJmlKSlKdLfOePQdZuTK3Mo+0tFxWrcqn\nf//DKvOoyK9fv8Mwi69t0mtruWNxQ2nAG0CBpDujnI8buSXx+98v5Pnnv+DYY3uGSZyc3Je+fRNi\nUk5ZWYA1a3ZUCl/x6tq1E++9N5Zhw46MSTntgdrkRlKTXsC5eD8Zkgp8E3yNDjmveCAQCOiee+Zp\n6NBXlZ9f1OD05eXSp19Kr06RtuY2rvxp01apT5+JWrFiW8MzaKcE/YrqptvCOAZI4re/nc/ChRtZ\nsOBGevU6tJ7pYEU6TJkNU+dCr8NhyCCYtwSST4Sxl8E1P4TePepfl5kzV3PLLf9mzpz/jIsWvFm7\nJfUo3NdyN0bsVevgrdme1BKMuxzGjYEhx3vnDxbD3MUwZQ7MWQznDPXiXHUhdK9HzyaeBHdyNxMN\nEXtjtifzlDlQsBuuH+0JO/xkqO0ecF8RzPrY+zJ88iVcMsL7Iow5Hw49pOZ08SJ4s/a563rh0z53\naB+7oGB/1Djb8qTnJklnjZUSz5F+9Yj0yZde/zoaW0qkmYVSaSD6+YJd0mvvShfdJPU4U/rZ76TZ\nn0glJdHjz5iR4fs+OLX0uZ3cjaA2sffuk/76tjRqvCfgz++X5i6uWUBJyi6Rbtsm9Votnb5eOj5T\nmrSrZsklaXue9MI/pRE/lY4YId3ykLT4q+rx/C64kzuG1Cb2rj3SaddKl/9Smj5fOnCw9ry2lUi3\nb/ekvidHyi31wj/aJ523Ufp+pvSv3VJZLZJL0sZs6cnXpIGjpGffqH7ez4I7uWNEfcS+43EpUIeM\nOaXSXUGp79wubS+NVpY0f680YoN0UqY0ZbdUXke+m7dKx10aX4I7uWNALMTOK5XuzfGkvn2713KH\nsk/7tF4bVK6qTnkgIM3dK525QTplnfTOntoljzfBndxNpKli7yiV7gtKfes2KStC6iIVaZ7m6zE9\nrj/peb2ol5WuVQqoKsNAQPqgUBq+Xjp1nTRjT83lxZPgTu4m0BSxC8qkB3I9qW/ZJm2OkHq/9mu+\nFugxPa7pmqmd2qWAAkrXKr2gl/WS/qwMra4m+fuFUsp6aeh67320suNFcCd3I2ms2LvKpAdzpd6r\npf/eKm0sDj9/QAe0QB/pD3pC0zRDBdpZrexyles7pQfb8Ve1WmuqST59j9eKn7bea9Uj6xEPgju5\nG8l7763WkCEvRx3HHnePdOuj1YVac1Dqv0Yany2tK66WTHnK15OaqHc0TTu0I/xkICCVh4ternKt\n1Ld6Vs/rXU0P649LXv/7nT3eTeeN2dEFHzgq+jDhtGmrNGDAnxSo6w64DePkbiT3379AjzzycbXw\nPXul7qd7D1UiGbNZenpH9XDJE3uCJuorrah+MhCQdt8qbT9MKv6s2uliFetv+n96R9OqCS5JReWe\n4LMLq2c94W/eOHj1IgNKTHxKW7dGSdROqE1ut8SjFlJTc0hO7lst/P2P4PzToFfEhKbP98O3B+HX\nPavnlc8O/s7rXMyFDGdY+EkJCn8NpV/D4a/BrqugZFlYlC504UZuYDe7mc5MAgTCzn+vAzzSBx7M\n97IL5frLYNp8KC0NDzczkpOTSE3NqfVzaK84uWshLS2XlJSkauFvzfbmd0TycD48kAhdIz7Veovd\nax4ceh0cPqlRgl99GBQLPghbBwXH9IfBR8OC8OwASEnpS1qakzuuyMsrYv/+Uo466vCw8ILdsORr\nuHJUePyl+2F1CdwU0Zo3SOwO3b3wQ0Y3SvAOBo8kRm+9x17mTdqKJCUlidTU3Do/j/aIk7sG0tK8\nLknksq1pH8LocyGhW3j8h/Lh/46ALiHRGyx2STFsSvfON1LwHx8GAt7bG17cdaPh/UVw4GB4uOuW\nxCE1dUmmzIGxEV2ST4tgQwn8PKTVbpTYf7wG/ncYLJnmxWuE4BZsvR/Oh0BI652UCMNO8uaHh3LC\nCb3JytpDUVFJgz6f9oCTuwZSU3Oqyb0tD1Iz4LLzwuM+lA8PJkLnYKvdaLE7dYGnl8DLtzVJ8B8l\nQCeDGRGt97jL4a0PwsM6d+7IkCGJfPttXoM/o7aOk7sGoo2UvD3XWw1zSNeqsEVFkF0GNwS75gUU\nNF7s+6fACafDY3MaJfgM3kOoxtb7JxfDh59BYcQNZ3JyX192TZzcUTh4sIwNG3YxZEhiWPiU2eFd\nEinYah/htZQAHzCHszmzcWJ36uzFPX5oowTPIotM1gEwJgG+Z/BuYVW8Xj28Icz3PwqvWkpKki9H\nTJzcUUhPz2Pw4N507Vq1Z9GGLNiQDRedVRVvYRHklcG4YKudRTY55HA2Z4VnWE+xD+zaxdoPgv2G\nRgh+IaNYyEeVrfejfbzWuzyk9R43xhvKDMWvIyax2OV1tJmtNrNMM/tdLCrV2kTrkkydA9dcCp2C\nvle22olVrfZCPuICzqcznasSNkDsf158MdPGjeOzZ57x0jZQ8FM4mRJKWcNaAC7tBj06wtshrfeV\no7yhzILdVWGnntqXb7/Npbw8fNy8vdMkuc2sI/ASMBoYAowzs5NiUbHWJNrNZOSDmw+LYFc5XB8c\nmt7CFvLJD++ONFDso0eO5Nb0dL565ZVGCd6BDlzEKBayqKr1Dva9y4Ktd0I3byhz2odV1Tz88EPo\n06cbmZk7Y/URtg1qei5fnxdwNjA35Pg+4L6IOC00yyA2FBeXKSnpaaWn51WGfbdWGjCqamFvICCd\nsV6aursq3d/1upbry6qAirki+WdJ5XuCmR+UHrxCevQnUqk3/3X/zp36y7BhmnvXXZUTmHZv2aLn\njztOS59+uiq/zK+lsX2lxe9WhR2YI+Ukhs1FKVe5XtDLWqWMymqct9Fbk1nB9PneGs9QbrvtAz34\n4EcN+7DaADTXxCngGuBvIcc3AC+qHcv95ptpuvjiSWFh//e8dPeTVccfFHqrYipWxGzUJk3UMypT\nmRfQBLEraIrg6VqlF/Vy5RTZj/Z5i44rFhwfOCj1PDN8Z6tVq/LUt+9EHTwYZc1bG6Y55b7ab3Jf\ndNEbmjLl27CwH1wlLQmZyHfJJun1kJZwsqZoqZZVBex7Xso9rkpsSXrldumucyvFlqR/jRmj6Tfc\nUOOU091btujxww7T2g8+qApcvVy6oou08buqsP1TvdmEwemyAQX0Jz2v9dpQGeX09d62ERWMvVv6\ny9Tw8s499++aMSMjal3aKrXJ3dQtjLcCA0OOBwLZkZHa0xbGw4cfyfLlW7n++lMqw4YMgu8y4Zxg\nd3rEofDJ/qonkkcxkE1sYkTFKEnXK6HoWTjwT+h2mxd24Q3wyRT4ai6c9SOvrFtuYdYvfsH2FSvo\nd9ppYfWQxOfPPUfvwYMZcPbZXmBZKbw9AYZdCv2C21MF9nhlHfozMK9Cu9nNfvaThHdTnFcG60th\neMgmPt9lwm3jqo6LikrIyMiP+lS2LdGQLYyb2nJ3AtYDxwBd8DbDPEntuOXetGmXevV6Unv3Vq00\nmDFfGvnzqji7yqQjVkuZwSglKtETekpbFbKqpXSjlHuMtO+lqrDVy6Xr+0jL3q8Mypg5UxP79NHW\nL6v664FAQHPvukt/GTZM+3cGFy+Ulnhdmgev8Lo4klS+2+v67L41bJXCdM3Uh5pfeXz3dunXIVX7\nNuIeQpJeffVLXXXVW/X+nNoKNOdiBeAyYA2wDrg/yvkWuszYccUVk/Xmm2mVxweLvT5qdk5VnEfz\nvJUvFSzVMk3Sv8IzaoTgTRW7QAX6g55QkbydZreXems4t4as33zgOemep8KretZZr2nu3Mx6f0Zt\nhWaVu65Xe5R7woTFuvPOuWFhN/0+fC3injIpcbW0OuhaiUo0QROVra3hmTVQ8H9ccEGjxZakdzVd\n87Ww8vg3271XBYGANOhSaUV6SBVLy/W97/1RhYV17CLUBnFyN5C5czM1atTrYWHzlkhnXBce7495\n0k+zqo6X6XO9oX9Wz7Cegq9+7z29fe21jRY7Xzv0Bz2h/fLWfG4tkXpmhG/6s3ylNHh0eNL09Dwd\nf/wLtX4mbRUndwPJydmrXr2eDBvFKC2V+pwrrdtcFa+wTOqzWkoPeleqUj2pidqiLdUzrafgVfEb\nJrYkva13tVBVY9W/3ub1t0O5c4L0YITHkyev1DXXvB3to2jz1Ca3m1sShb59E+jcuQPZ2VXPrTt1\n8h6/Tw1ZzXJYR7irNzyaH4xDJy7mIt5hOoUUhmfa6RjotQiKnoail72wE06HR/4Nz/0PfD6rKm5Z\nKTwxFspK4IF3oUtXb1Rk52joPAy6v1Rt3+MVfM1GNjICb2QluxQmF8Jvj6iKEwh49Y+cj17TWtH2\njpO7BryZcuGTiaJNOrqtF3xc5C0MBhjGUIYzlNf4R+MEb6TYC/iImxjPIXjjfY/vgP/pAX1CBnsX\nr4DEnnDSoPBq1bQwo73j5K4Bb6Zc+DTQEUNhz15vjLiChA5w7xHwSH5V2AWc3zjBn/1vuLpHo8T+\nL8aTiNdMby6BqYVwb+/woqfM9hYsRBJtLo0fcHLXQLQJ/B06eNskTIlovX/VE5YegLSQ9YmNEvyx\n2XDjY00SG7xW+5YecERIq11aCu9+6NU/lJycfZSWBujf/7CGfDztg5o647F60Q5vKCVvrkW0EYQV\n6d4WZZH3c8/ukH4c5T7yY32iZ/Sc9mhP9ZPRbjIrqOPm8Sut0ARNVJ7yw8I3FHvj2jsipojM/kQ6\ne1z1YubOzdSFF0bZb62dgLuhbDjf/35vtm3by969xWHhQ0+Cjh3gq+/C49/SE5YfgK8PhIc3uAWH\nRrfYAH/YAbf2hN4REyumzPG2d4jE65L472YSXLekRjp27MDJJyeycmX4TaVZ9BvLQzvAfUd4c6cj\naZDgTRB7fYm3pcNdEX3tAwe9bR2uG129bqmpuSQn+6+/DU7uWok2YgLeUNrUOd7QWii/6AHfHIQv\nD1RLUk/Bn4LcHo0SG+CxfPjfXtCzY3j4nMXetg5JidWSkJbmz5tJcHLXSkpKEkuXZlULP2mQN6Q2\nb0l4+CEdvHWLP9vqdVEiCRV8CxH5djoGen0KvRZUE7uUUj5lcY1iFwfg6R2woAju6BWerQR/nx59\nlGTHjv1s2rSbE0+s/kXxBTV1xmP1op3eUErek8rjj39BTz21pNq5OZ96P7/3RVr1dH/fJQ1YI12x\nWfoqyq/4rdDXelJP63VNUpayq0cIUqpSLdMXmqCJekP/rLblcXFA+nOBV9blm73tk0MJBLz9w0+7\n1vuVtVAKCvZr6NBXdd9989WewT1+bzxZWXtqFHzWopoFP1AuvVgg9VsjXbVF+uZA+PlQcSfpzbDp\nsqUq1RdaXuMXoCQg/XWndNRaafQm6YsoX6BQsXdFDNRUiH3PPfPa9d7ckpO7yTRWcEnaX+4NEyat\nka7eIq2MkLxEJVqqZXpCT+lNTdaHmq+Jekb/0BvaHDFHpTTg/Vc4dq23GuizouhlxovYkpM7JjRF\ncMnbHP7pHVLf1dJ1WVWTrSooUYmWaKmma6Y2aVPYudKA9MYuadBa6cKN0uIapJbiS2zJyR0zmiq4\nJO0rl57M92YTjsuSMmqZQl0WkN7cLQ3OlM7fKC3aV3NcKf7ElpzcMaU+gk+ZLZWV1Z5PYZn0eL63\n4OFn2dLaEMnLA96Pqp6YKZ2zQVqwt+4fbi3Y5f1GTzyJLTm5Y05tgi9ZIaX8h9TvAuk3T3gteW0+\n7SmTHsvz1mSOz/b65yev835UdV4dUhfuk9583/s57u6nez9CFU9iS7XLbd755sPM1NxltAbZ2YWM\nGvUGN988jHvvPafa+Yz13oOet2ZDWbn36Hvc5XDK4Oj57S6H5wogrRhu7gGjE6o9wwHgYDHM/tSb\nvDVvKZw33HtieuWFcFjEhvg7dx7g4osncdFFx/LUU5dU20jfD5gZkqJemJO7CdQlOHgPUb7J8GSc\nMhu6J3gyjh0Dg46qXzmlpbDwc++L8v4i72nj2DFw9SXVf3SqgngQG5zczUp9BK8gEIBlqZ6k78yD\no/t5ol83Gvr3rR53ydfeZvHT5sOggV7ca0fDkVEeo4cSL2KDk7vZaYjgFZSVwaLlnrwzF8KpJ3hd\nl1MGw4wFMHUu9O7hCX39ZXDsgPrVJZ7EhmaU28wmAlcAJXib89wkaU9EHN/LDY0TvILiEpi72JuW\numq99+sNYy+DIcc3rA7xJjY0r9yXAAslBcxsAoCk+yLixIXcUCX4GWf0Z8SIASQnJ3HqqX3p3r1r\n3YkbgSQ2b95DamoOaWk5vP32KsaMOT5uxIYW6paY2X8AV0u6ISI8buQGyM3dx/TpGaSm5pCamst3\n3+Vx5JEJJCcnkZLSl5SUJJKTkxg4sHuDBCwuLiM9PZ+0tJzKvFeuzKVbt86VeZ955gB+9KPvx43Y\n0HJyzwLekjQ5Ijyu5I6kvDxAZubOoJA5pKXlkpqaQ3FxWVD0vgwf3o8f//hEEhK6VKZbtSqfOXMy\nSU3NJS0th3XrdjJoUC9SUjyRk5O9tImJ3Wop3f80SW4zmw9Em83+e0mzgnEeAIZJujpKej300EOV\nx219l9eWIjd3X6XoS5Zs4bPPsrjxxmROOKE377yzilWr8rn66pMYOvRIUlKSGDIkkUMOaeqmvO2f\nyF1eH3nkkeZruc1sPPAL4CJJB6Ocj+uWu75s3rybV175kjVrCrjuupO55pohdOnSse6EcU5z3lCO\nBp4BLpC0o4Y4Tm5Hs9Gccmfi7ctd8UtByyTdGhHHye1oNtxDHIdvqU1ut0DY4Vuc3A7f4uR2+BYn\nt8O3OLkdvsXJ7fAtTm6Hb3FyO3yLk9vhW5zcDt/i5Hb4Fie3w7c4uR2+xcnt8C1ObodvcXI7fIuT\n2+FbnNwO3+LkdvgWJ7fDtzi5Hb6lyXKb2d1mFjCzXnXHdjhajibJbWYDgUuAzbGpjsMRO5racv8J\n+G0sKuJwxJpGy21mVwHZklbGsD4OR8yoddvQWnZ4fQC4H7g0NHoM6+VwNJla5ZZ0SbRwMzsFOBZI\nC250PgBYYWZnSMqLjP/www9XvndbGDuaQuQWxrURk70CzWwjMFzSzijn3F6BjmajJfYKdPY62hxu\nl1dHu8bt8uqIS5zcDt/i5Hb4Fie3w7c4uR2+xcnt8C1ObodvadNy1/cxq5/KdtccO5zcbaxsd82x\no03L7XA0BSe3w7e0yNySZi3AEfe02s9jOxytheuWOHyLk9vhW9qN3K2xP4qZTTSzDDNLM7PpZnZ4\nM5c32sxWm1mmmf2uOcsKKXOgmS0ys3Qz+87Mbm+JciPq0NHMvjGzWbHMt13I3Yr7o3wInCwpGViL\ntyi6WTCzjsBLwGhgCDDOzE5qrvJCKAXulHQycBZwWwuVG8odwCpivKKrXchNK+2PImm+pEDw8Au8\nhdDNxRnAOkmbJJUCU4CrmrE8ACTlSEoNvt8HZAD9mrvcCsxsADAGeI0Y76DQ5uVuQ/uj/Bcwuxnz\n7w9khRxnB8NaDDM7BhiK90VuKZ4F7gUCdUVsKLVu7dBStOb+KLWU/XtJs4JxHgBKJE2OZdkRtOqY\nrJklAO8CdwRb8JYo8wogT9I3ZjYy1vm3CbljtT9KLMsOqcN4vH+bF8WivFrYCgwMOR6I13o3O2bW\nGZgGvClpZkuUGWQEcKWZjQEOAbqb2SRJN8Ykd0nt5gVsBHq1YHmjgXTgiBYoqxOwHjgG6AKkAie1\nQLkGTAKebeW/7QXArFjm2eb73BG09L/uF4EEYH5wqOqV5ipIUhnwa2Ae3sjBVEkZzVVeCOcANwCj\ngtf4jZmNboFyoxHTv697/O7wLe2t5XY46o2T2+FbnNwO3+LkdvgWJ7fDtzi5Hb7Fye3wLU5uh2/5\n/2t/n/hxReOSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8ac0710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from matplotlib.mlab import bivariate_normal\n",
    "\n",
    "x = np.arange(-5, 6, 1)\n",
    "y = np.arange(-5, 6, 1)\n",
    "X, Y = np.meshgrid(x, y)\n",
    "Z = bivariate_normal(X, Y, sigmax=2, sigmay=1, mux=0, muy=0, sigmaxy=1.5)\n",
    "plt.subplot(121)\n",
    "plt.contour(X, Y, Z)\n",
    "plt.title(\"Pre-whitened Data\")\n",
    "plt.gca().set_aspect('equal', adjustable='box')\n",
    "plt.show()\n",
    "\n",
    "Z = bivariate_normal(X, Y, sigmax=1, sigmay=1, mux=0, muy=0, sigmaxy=0)\n",
    "plt.subplot(122)\n",
    "plt.contour(X, Y, Z)\n",
    "plt.title(\"Whitened data\")\n",
    "plt.gca().set_aspect('equal', adjustable='box')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, if we whiten our data so that the contour plot looks spherical, then one unit in the y-direction will be equal to one unit in the x-direction. \n",
    "\n",
    "---\n",
    "\n",
    "# Deriving the method\n",
    "Let $X \\in \\mathbb{R}^{nxd}$ be our data matrix, where $n$ = number of samples and $d$ = number of features. Let us first set our goal to find $W_{D} \\in \\mathbb{R}^{dxd}$ such that $Y = XW_{D}$, where $Y$ is our transformed data with diagonal covariance matrix. Before diving in, let us also state an equivalence that we know from linear algebra. \n",
    "\n",
    "Let $\\Sigma$ be the covariance matrix of our data matrix $X$. Since a covariance matrix is by definition symmetric, from linear algebra we know that $\\Sigma$ can be decomposed into: \n",
    "\n",
    "$$\\Sigma = EDE^{-1} = EDE^{T}$$\n",
    "$$D = E^{-1}\\Sigma E = E^{T}\\Sigma E$$\n",
    "\n",
    "Where E is an orthogonal matrix and D is a diagonal matrix containing eigenvalues. For the purposes of the derivation, we assume that $X$ is centered by column, so that we can state: \n",
    "\n",
    "$$\\Sigma = \\frac{X^{T}X}{n}$$\n",
    "\n",
    "The goal is to find $W_{D}$ such that $Y$ has covariance matrix equal to $D$, or stated mathematically: \n",
    "\n",
    "$$D = \\frac{Y^{T}Y}{n}$$\n",
    "\n",
    "## Method 1\n",
    "We start our derivation with what the covariance matrix of the transformed data $Y$ should be: \n",
    "\n",
    "$$D = \\frac{Y^{T}Y}{n} = \\frac{(XW_{D})^{T}(XW_{D})}{n} = \\frac{W_{D}^{T}X^{T}XW_{D}}{n} = W_{D}^{T}\\Sigma W_{D}$$\n",
    "\n",
    "$$\\text{Since we know that }D = E^{-1}\\Sigma E$$\n",
    "\n",
    "$$W_{D}^{T}\\Sigma W_{D} = E^{-1}\\Sigma E$$\n",
    "\n",
    "$$\\text{Hence, }$$\n",
    "\n",
    "$$W_{D}^{T} = E^{-1} \\text{ or } W_{D} = E$$\n",
    "\n",
    "Thus, multiplying $X$ by $E$ will tranform our data such that it has diagonal covariance matrix. In whitening though, not only are we making the covariance matrix of the transformed data diagonal, but also an identity matrix. To achieve this, we only need to re-state our goal as: \n",
    "\n",
    "$$I = \\frac{Y^{T}Y}{n}$$\n",
    "\n",
    "Before we dive into deriving what $W_{W}$ should be such that $XW_{W} = I$, let us state a few linear algebra manipulations that we will be using: \n",
    "\n",
    "$$I = D^{-1}D = D^{-\\frac{1}{2}}DD^{-\\frac{1}{2}} = D^{-\\frac{1}{2}}E^{-1}\\Sigma ED^{-\\frac{1}{2}}$$\n",
    "\n",
    "$$I = \\frac{Y^{T}Y}{n} = \\frac{(XW_{W})^{T}(XW_{W})}{n} = \\frac{W_{W}^{T}X^{T}XW_{W}}{n} = W_{W}^{T}\\Sigma W_{W}$$\n",
    "\n",
    "$$W_{W}^{T}\\Sigma W_{W} = D^{-\\frac{1}{2}}E^{-1}\\Sigma ED^{-\\frac{1}{2}}$$\n",
    "\n",
    "$$\\text{Hence, }$$\n",
    "\n",
    "$$W_{W}^{T} = D^{-\\frac{1}{2}}E^{T}$$\n",
    "$$W_{W} = ED^{-\\frac{1}{2}}$$\n",
    "\n",
    "## Method 2\n",
    "Is there a simpler, yet equivalent way of expressing $W_{W} = ED^{-\\frac{1}{2}}$? It turns out there is. Let us derive that! \n",
    "\n",
    "$$\\Sigma^{1/2} = ED^{1/2}E^{-1}$$\n",
    "\n",
    "$$\\Sigma^{-\\frac{1}{2}} = (ED^{1/2}E^{-1})^{-1} = ED^{-\\frac{1}{2}}E^{-1}$$\n",
    "\n",
    "$$\\Sigma^{-1} = \\Sigma^{-\\frac{1}{2}}\\Sigma^{-\\frac{1}{2}} = ED^{-\\frac{1}{2}}E^{-1}ED^{-\\frac{1}{2}}E^{-1} = ED^{-\\frac{1}{2}}D^{-\\frac{1}{2}}E^{-1}$$\n",
    "\n",
    "So $W_{W} = ED^{-\\frac{1}{2}} = \\Sigma^{-1/2}$ !\n",
    "\n",
    "---\n",
    "\n",
    "# Code Demonstration\n",
    "To demonstrate this, let us generate 50 data points drawn from a multivariate normal distribution with origin as the mean and the covariance matrix: \n",
    "\n",
    "$$\\Sigma = \\begin{bmatrix}\n",
    "2&1.5\\\\\n",
    "1.5&2\\\\\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "We plot the 50 data points:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPAAAAEZCAYAAACpaahsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFnVJREFUeJzt3XuUXWV9xvHvM8DISLgIkTs0rUWuFhIVaYUmXaxJsCoU\n0LKs1YC1dEEVpVMMmK5CNREspl7bpVSUiHIRLJZUzRApAaOWJYYQCLfAAgRcBFHkUmIDzK9/7D1m\nMjkz58zMOXvvd5/ns9ZeOefMPud9ZzK/ee/vq4jAzNLUU3YGzGzyHMBmCXMAmyXMAWyWMAewWcIc\nwGYJcwCbJcwB3AUkPSzpBUnPSnpa0g8l/Y0ktfDeGZKGJPl3pYL8n9IdAnhbROwE7A9cBCwALp3A\nZzQNdiueA7jLRMRzEbEMOAWYL+lQSW+VdLukZyT9TNL5I95yS/7vryU9J+lNkl4j6b8lPSXpF5K+\nLmnn4r8bcwB3qYj4CfAYcAzwPPCXEbEz8FbgDEkn5Lcek/+7c0TsGBG35s8XA3sBBwP7ARcUlXfb\nzAHc3X4OvCoibo6IdQARcSdwFTA7v2erqnNEPBgRN0bEixHxFPDpEfdbgbYtOwNWqn2AX0l6E1m7\n+FCgF3gF8M2x3iRpD+CzwNHAjmQFwa86nlvbikvgLiXpjWQB/EPgCuDbwL4RsQvwRTb/bjRarvYJ\n4GXgsLza/R78u1QK/9C7hwAk7STpbcCVwOURcRcwDXg6IjZJOhL4CzYH7i+AIeA1Iz5rGvC/wLOS\n9gHOKeh7sFHk9cD1J+khYA/gJbJgXAd8HfhiRISkk4ElwK7AzcBDwC4R8d78/f8EnEHW5DqOrNPr\na8CBwPr8sz4cEfsX+X1ZiQEsaXuyX5ZXkLW7/jMizislM2aJKrUElvTKiHhB0rbAKuDvI2JVaRky\nS0ypbeCIeCF/2Atsg3syzSak1ACW1CNpDbABuCki7i4zP2apKbsEHoqII4B9gT+WNKfM/JilphIT\nOSLiGUnfAd4ArBx+XZK7yK2rRcS4i0hKK4ElTZe0S/64D+gHbh99X0SUdp1//vlO3+mXdrWizBJ4\nL2Bpvs60h2xSwY0l5scsOaUFcGST5meVlb5ZHXgq5TjmzJnj9J1+pVV6KqWkqHL+zDpJElHVTiwz\nmzoHsFnCHMBmCXMAmyXMAWyWMAewWcIcwGYJcwCbJcwBbJYwB7BZwhzAZglzAJslzAFsljAHsFnC\nHMBmCXMAmyXMAWyWMAewWcIcwGYJcwCbJcwBbJawsg8320/STZLWSbpL0lll5scsNWWXwC8CZ0fE\nocBRwN9KOrjkPFnBBgcHmTv3ZObOPZnBwcGys5OUUg83i4gngCfyx89LugfYG7inzHxZcQYHBznx\nxPls3PhJAFatms911y1l3rx5JecsDZXZ2F3SDOBm4NCIeD5/zRu719zcuSezYsXxwPz8laX091/P\nDTd8q8xsVUIrG7tX4nhRSdOAa4EPDQfvsAsuuOC3j+fMmZPEcRdmk7Fy5UpWrlw5ofeUXgJL2g74\nL+B7EfGZUV9zCVxzo6vQfX0LXIXOtVIClxrAkgQsBX4ZEWc3+LoDuAsMDg6yZMklAAwMnO7gzaUQ\nwEcDtwBrgeGMnBcRy/OvO4ArwAFWjsoHcDMO4PK5ilseB7BNmXuJy+PjRc1qrhLDSFZdAwOns2rV\nfDZuzJ739S1gYGBpuZmy33IV2ppyJ1Y5PwO3ga0QdQ/wsjryWglgIqKyV5Y9q7Lly5dHX98eAZcF\nXBZ9fXvE8uXLC02/v/+k6O8/qWPp9veflH9/kV+XRX//SR1Ja6T893/cGHEb2KZkyZJL8pIp66Xe\nuDF7rYhS2Ash3IllCSvqj0eVO/IcwDYlVf7lbpd58+Zx3XVLR7Tzq1PKuxPLpqysTqy6zxJzL7TV\nXp17wB3AZgnzVEqzmnMAmyXMAWxdo467X7oNbF0hxR5rd2KZ5VJc1+xOLLOa80ws6wp1nTHmKrS1\nVZUnVlQ5b424DWyFSrGjqMocwFaoFDuKqsydWFYpdRyHLV2zFf+dvICvABuAO8f4+qR2MrByjLc7\nR9k7d6SIFnbkKPtkhmOA54GvRcTrGnw9ysyfTdxYHUWuXk9c5U8njIgf5MeKWk3MmzfPnVYF8jiw\nFaKu47Blq3wA+3zgeqjytjRVker5wDOAZW4Dd6/UJlgUpfJtYDNvDTs1pY4DS7oS+BHwWkmPSjqt\nzPxYpsjx2i23hs0Cebg0tubK7oV+V5npFy2FqqJLxMQ0Gygu86JGEzlSmchQ9DEiqfxcykALEzk8\nlbIgrio2Ntw73d9/Pf39129V2nv65fjciWVbKGO8dqzJH67Ot6BZEV3mhavQpSjixL9WlHUqYFXg\n0wmrI6WJDJ4OmY7SJ3KMxxM5yldmz3m3bxDgBf02YSMDdvbsWSxe/PlSAyiFobdOcQDbhIwu8Xp6\nzmZo6H3Ap/I7vASwSJ5KaRMy+sDsoSGAL5aZJWvC48A2rp6e9cBSYGk+pHT6mPcWPWbrMWI8jGSb\nNRrqWrRoUUtDSkUPk6U0LDdZtDCMVHqQjps5B/CYOjVWO9nPLXrMthvGiFsJYLeBE9TJGUoeA05M\nswgv88IlcENVLH1chW4/XAJbUYqeaZbSzLZO8jhwgrp9hlK38ESOGuvmGUrdwker1Ni8efO44YZv\nMTBwOkuWXNJ0LLTMMVOP13ZQs0ZymRfuxBpXqx05ne7wGW/oqRs6mzoFjwPX23i90SODaubMN3es\n17pZgLaax1aDuiprlYvQSgC7F7qGGi1KgDs7ktbo+dMbN2avNWuTT2Ys2zt0NNAswsu8cAk8rrFK\nv0alXk/Pbh2pxjYbk55IHpvVCqo4/t1JeFO7emnUGXTQQQex664fZ+bMfx+3NDr88MPG3DhuKgYG\nTqevbwFjLXhotmmdTVGzCI8tS8RtgJ0m8p4mn3cccC+wHljQ4Oud+uOWnNElWW/vLtHb++pKnMU7\n2bbsRPPYbR1itKMTC7gS2AnYAbgbeBz4SLP3tfC52wAPADOA7YA1wMGj7unsTyghW1cfj2pada16\nZ487scbXSgC30ol1SEQ8K+ndwPeAc4HVwD9PsfA/EnggIh4GkHQVcAJwzxQ/10hjUcJk8pjC91Wk\nVgJ4W0nbAX8G/GtEvCipHdOj9gEeHfH8MeBNbfjcWhq9X3Nv773AOWzalD0fvX+zZ2p1h1YC+EvA\nw8Ba4Jb8ONBn2pB2S38EfD5wZuvJ+1cBNJzM7+GWNBVyPrAkAdtExEsTeuPWn3MUcEFEHJc/Pw8Y\niohPjrgnJpo/g7lzT2bFiuMZHpv1ZnRpastcaEl7SrpU0vL8pYPZ/JsxFbcBB0iaIakXOAW4vg2f\na9Y1WhkHvgy4Adg7f74eOHuqCecl+AeAQbLe7asjwh1YbdBsbLYsXtTQAc26qYHb8n9vH/Hammbv\na8eFh5EmrdFwS5lDMIsWLYqenld1zRhuO9CmceCVwG7DAQwcBdzc7H3tuBzA7VPmJIjly5ePmMrZ\neNzattZKALfSCz0ALAN+T9KPgFcD72hbFcAKMdlFB+1Ke2jogI6n042aBnBE/FTSbODA/KX7IuLF\nzmbL6uSppzYAv2Zk10lPz9kMDFxZWp7qomkAS5pPNmY73J09K+/e/lpHc2ZtVcbB3ZB1XK1bdz9w\nMdmSxr9DCj72sQGPS7dB03FgSV9g86SL7YFjgdUR0fFqtMeB26uM2VmNxqRnzvwqq1ev7HjaqWvL\n4WYR8YFRH7oLcPUU82YlaNc84qn+IZg+fbcp58FyzXq5Rl9AL3D/RN83mQv3QlfORHuzu20JYDvR\nQi90K1XoZSOe9gCHAN+MiAWd+ZOyRdrRLH9WrMlM0/TCislp1/nAS0Y8fgl4JCIeHetms9G8BLBz\nWmkDrywgHzZFRZVyZfVm2xjGqlsDzwPPjXE926xu3o4Lt4FbMpH9odsxlbKbdsUoE+1oA5fJbeDW\ntNIuHb1GuLf3wxx66OFMn76b26UV1a428PCH7U42DgxARPxsCnmzgm05lXKQTZu25fbbTwO84D9l\nrawHPl7SeuAh4Gay3Tm+1+F8Jams5XITXz54CfApsmDOSuXh9rOlpZX1wIuAPyQb+/1dsplYt3Y0\nVwkarqKuWHE8K1Ycz4knzi80iA866Pfz/aG/2rA03TLIf15YvqzDmjWSgZ/m/95BtpUOwNpm72vH\nRUKdWGWdGjCRiRLDnU8zZ755zD2lrTpo03LCpyXtCPwA+IakJ8l6qJNVp4kFE1kmOHI8dsufgdu/\nyWoW4WQbum9Dtvn6qcBZwG7N3teOiw6UwJ2a2lfWlMFuOy+om9CmHTkGgH2a3deJqxMB3Mlf+DLG\nRz3XuL5aCeBW5kJfALwTeBq4CrgmIja0vSrQOO1olr+JquOWqyOrw7Nnz+Lmm1cD6TcPul0r48AT\nKQ0PBxYD9wE3tvq+qVwkVIWugjp/b92Ids7EkrQX2V5Y7wKmRcQfTPpPS4s6NROrTp1YI9WxdtHN\n2jITS9KZwJ8DuwPXAO+PiLvbk8VyFLU6pq5/KKxCmhXRwIXAEc3um8hF1qZeB7wMzBrnvvbXSwpS\nRnXWVeh6oaqLGSQdBAyRHZw2EBGrx7gvyshfO5RVnXWpXx9tXczQThFxL2QZtPby4vnuUkoAdwMv\nfLcitNKJdRZweUQ8PZEPlrQC2LPBlz4aEcsavN5QqucDb32er6cr2vg6cj6wpMVkR3+uBr4CDLar\nYSrpJmraBjabqracDxwRC4HXkgXvqcB6SZ+Q9Jq25HLziQ9mNkGtrAcmIoaAJ4ANZEM/rwKulXTx\nZBKVdKKkR8lOOvyOJG8QYDYJrVShPwS8F/gl8GXguoh4UVIPsD4i2lUSN0rbVWjrWu0aRtoVOCki\nHhn5YkQMSXr7VDJoZlPjXSnNKqotnVhmVl0OYLOEOYDNEuYANkuYA9g6qqzN7ruFe6GtY0afx9TX\nt8BHuEyAe6E7zKXL+Lbcs9pHuHSCA3iSij5KxX8srKFmW3aUeVHhLXWK3FA91a1yUs13VdCmo1Ws\nZBM5PqVKvCa68xzAk+QdN1rjLX46y73QU1DUBnLuze1OrfRCO4AT4d0mu48D2CxhHgc2qzkHsFnC\nahXAnuxg3aY2bWD31FrddFUnlo/WtLpxJ5ZZzdVmJpZnRlk3Kq0KnW8K/zZgE/AgcFpEPDPqngmN\nA3uyg9VJpdvAkvqBGyPbX/oigIg4d9Q9nshhXavSbeCIWBHZkS0AtwL7lpUXs1RVpRPrfcB3y85E\n1Xhc25rpaCdWK2cES1oIbIqIKxp9RqrnA0/V6HHtVavme1y75jpyPnAnSToV+Gvg2Ij4TYOvd20b\n2OPa1q7DzTpC0nHAOcDsRsFrZs2VOQ78eaAXWCEJ4McRcWaJ+akUj2tbK2ozlbKOPK7d3So9DtyK\nbg9g626VHge26vLwVTpcAtsWvCyzOlyFtgnz8FV1uAptVnO1WU5o7eHhq7S4Cm1b8fBVNbgN3Eb+\npbaiOYDbxD2zVgYHcJu4Z9bK4F5os5pzL3QL3DNrVeUqdIvciWVFcxvYLGFuA0+RJ/Vb1bkEHoOH\njqxsrkJPgYeOrGyuQpvVnIeRxuChI0uBq9Dj8NCRlcltYLOEuQ1sVnMOYLOElRLAkj4u6Q5JayTd\nKGm/MvKRAk8msfGU0gaWtGNEPJc//iBweES8v8F9Xd0G9mSS7lbZs5GGgzc3DXiqjHxU3ZIll+TB\nm00m2bgxe80BbMPKPNxsMfAe4AXgqLLyYZayjgVws7OBI2IhsFDSucCngdMafU63ng8MnkzSbZI7\nHxhA0v7AdyPisAZf6+o2MHgySTer7EQOSQdExPr88QeBIyPiPQ3u6/oAtu5V2U4s4EJJBwIvAw8C\nZ5SUD7OklV6FHo9LYOtmnkppVnMOYLOEOYDNEuYANkuYA9gsYQ5gs4Q5gM0S5gA2S5gD2CxhDmCz\nhDmAzRLW1QHs/aYsdV27mMH7TVnVVXY9cKs6GcA+vMyqzquRzGquaw83835TVgddW4UG7zdl1eY2\nsFnC3AY2qzkHsFnCHMBmCXMAmyWs1ACWNCBpSNKuZebDLFWlBXB+JnA/8EhZeWhmoufUOH2nX7Qy\nS+B/AT5SYvpNlf0f6PS7O/1WlBLAkk4AHouItWWkb1YXZRwvuhA4D5g78vZO5cOszgqfiSXpMOBG\nsoO9AfYFHic7ofDJUfd6GpZ1tcpPpZT0EPD6iPhVqRkxS1AVxoFdyppNUuklsJlNXhVK4HFJ+rik\nOyStkXRjPn5cZPoXS7onz8N/SNq54PTfKWmdpJclzSoozeMk3StpvaQFRaQ5Kv2vSNog6c4S0t5P\n0k35z/wuSWcVnP72km7Nf9/vlnThuG+IiEpfwI4jHn8Q+HLB6fcDPfnji4CLCk7/IOC1wE3ArALS\n2wZ4AJgBbAesAQ4u+Hs+BpgJ3FlkunnaewJH5I+nAfeV8P2/Mv93W+B/gKPHurfyJXBEPDfi6TTg\nqYLTXxERQ/nTW8l6zYtM/96IuL/AJI8EHoiIhyPiReAq4IQC0ycifgA8XWSaI9J+IiLW5I+fB+4B\n9i44D8MjNL1kf1DH7OCtfAADSFos6WdkO9BdVGJW3gd8t8T0i7AP8OiI54/lr3UdSTPIagK3Fpxu\nj6Q1wAbgpoi4e6x7K7En1jiTPj4aEcsiYiGwUNK5wKeB04pMP79nIbApIq5oZ9qtpl8g92oCkqYB\n1wIfykviwuQ1viPy/pZBSXMiYmWjeysRwBHR3+KtV9CBErBZ+pJOBf4UOLbdabeSfsEeB0Z2FO5H\nVgp3DUnbAd8Cvh4R3y4rHxHxjKTvAG8AVja6p/JVaEkHjHh6AnB7wekfB5wDnBARvyky7UbZKSCN\n24ADJM2Q1AucAlxfQLqVIEnApcDdEfGZEtKfLmmX/HEfWSfqmL/zlR8HlnQtcCDwMvAgcEaMmnLZ\n4fTXk3UmDHck/Dgiziww/ROBzwHTgWeA2yPiLR1O8y3AZ8g6UC6NiPGHMtqf/pXAbGA34EngHyPi\nqwWlfTRwC7CWzc2J8yJieUHpvw5YSla49gCXR8TFY95f9QA2s7FVvgptZmNzAJslzAFsljAHsFnC\nHMBmCXMAmyXMAWzjklToNEKbGAewNeOJAhXmAK4JSW/MNx14haQd8sXoh4y650JJZ454fkF+OsYO\nkr4v6aeS1ko6vsHnz5G0bMTzL0ianz9+vaSVkm6TtFzSnvnrZ+UL4+/IZ1dZm1ViMYNNXUT8RNL1\nwCKgj2wK3uhlaFeTTZH8t/z5O8m29/0NcGJEPCdpOvBjms9/DiDyif+fB94eEb+UdAqwGPgrYAEw\nIyJelLTT1L9LG80BXC8fI1uMsJFs95ItRMQaSbtL2gvYHXg6Ih7Pg/BCSccAQ8DeknZvYc65yOap\nHwp8P1sHwDbAz/OvrwWukPRtoLRVPXXmAK6X6cAOZEHUx+a9t0e6BngH2frjq/LX3p2/d1ZEvJxv\n9bv9qPe9xJZNrpFfXxcRf9QgrbcCfwy8nWw99+si4uWJfUs2HreB6+VLwD+QrZv+5Bj3XA28iyyI\nr8lf2wl4Mg/ePwF+p8H7HgEOkdSbL3c7lqwafR/waklHQbaWVtIh+bK8/fOF6OcCO5P9cbE2cglc\nE5LeC/xfRFwlqQf4UaOdHCLi7ny3icciYkP+8jeAZZLWklXB7xn5lvx9j0r6JnAX8BCwOn/9RUnv\nAD6X7yCxLdmuKfcDl+evCfhsRDzbkW++i3k5oVnCXIU2S5gD2CxhDmCzhDmAzRLmADZLmAPYLGEO\nYLOEOYDNEvb/fjTG6veEbtcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x56e1950>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from numpy.linalg import inv as inv\n",
    "\n",
    "np.random.seed(1)\n",
    "\n",
    "mu_vec = np.array([0,0])\n",
    "cov_mat = np.array([[2,1.5],[1.5,2]])\n",
    "data = np.random.multivariate_normal(mu_vec, cov_mat, 50)\n",
    "\n",
    "plt.scatter(data[:,0], data[:, 1])\n",
    "plt.title('Data')\n",
    "plt.xlabel('x values')\n",
    "plt.ylabel('y values')\n",
    "plt.gca().set_aspect('equal', adjustable='box')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Method 1\n",
    "In method 1 derived above, we find $W_{W} = ED^{-1/2}$ such that: \n",
    "\n",
    "$$X_{Whitened} = XW_{W}$$\n",
    "\n",
    "We implement method 1 to see that it works. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Covariance matrix of Whitened Data\n",
      "[[  1.00000000e+00   1.94544980e-16]\n",
      " [  1.94544980e-16   1.00000000e+00]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQ8AAAEZCAYAAACJoKC4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGUpJREFUeJzt3X2UZHV95/H3Z4CRDgMIDIqoiFESAiEMo7iQA1p7SE9j\nDIMDYk7ikYHs7mzWZFG3zY6EzdpuYtB45qyim7MSUUZ84EGEDEd32talkFHDQZgBBJSHIEEUPDws\nahgFme/+cW8zNTXVXVW36/bv3qrP65x7purWvVXfqp761u/5KiIwM+vXktQBmFk9OXmYWSFOHmZW\niJOHmRXi5GFmhTh5mFkhTh5DTNIOSb8+x2NvkzS92DF1IulSSX+dOg7rj5NHTUg6X9JX2vbdO8e+\nt3Z7voj4XERMtJw3Z6JZBJFvu5F0jqTnJP0s3/5Z0qckHdHrkzs5lcPJoz5uAH5XkgAkvQTYE1gh\naUnLvlcB3yj4GhpEoCW89jcjYl9gP+D3gO3ALZKOXpTIrCMnj/r4DrAXsCK/fzJwPXBP2777I+KR\nlvPGJd0j6UlJH5/dmf+i35jfnk02t+W/7mfl+/9A0rb83G9KOqbl/B9ImpR0m6T/J+lySS9oeXy+\nc4+TdKukn0q6HNi7y3sXQGT+OSL+jCyZTrU851WSfpzHcoOko/L964A/Bv5r/t7+Md//Xkn35THc\nKenNXWKwNk4eNRERzwA3AW/Id70euBHYkt+e3XdD26lvAl4L/A7wVkkTbY8TEbPn/05E7BsRV0k6\nDrgE+A/AgcAngE2S9po9DTgLmABemT//OZAlh7nOlbQUuBbYCBwAXAWcyRzVlnl8iSxZzvoy8Grg\nYOBW4HP5e7s4v/2h/L2dnh9/H3BSROwHvB/4rKRD+oxhpDl51MsN7EwUJ5FVT25s2XcyuyePD0bE\nTyPiIbKSygp6sw74RETcnP/ifwb4JXBCyzEXRcQjEfEkcF3Lc8917on5+XtGxEcj4rmIuBq4uceY\nWv2YLDEBEBGXRsS/RsSzZMngWEn7thy/S7UoIr44W0KLiCuBe4HXFYhjZDl51Ms3gJMkHQAcHBH3\nA98maws5ADia3ds7WqswTwP79PharwAm82rHk5KeBF4GHDrHc28HlnU59yX5+Q+3vdaD9N/e8lLg\nCQBJe0j6YF4NeQp4ID9m+VwnSzpb0taW+H4bOKjPGEaak0e9/BOwP1l14JsAEfFT4Edkv/Y/iogH\nB/Ra/wJ8ICIOaNmWRcQV85wTPZz7Y7IvfqtX0H+1ZQ07E+UfA6uBUyJif7JqFOxMSLs8t6RXABcD\nfwYcGBEHAN8lbYNx7Th51EhEbCdrOP0v7FrC2JLva6+ytBNzf0EeJeupmfUPwJ9Kep0y+0h6k6Rl\nc5w/+/zdzv0W8CtJ5+VtIGcAx3eJO3vyrITxSkkfI6uqvT9/aBlZtegJSfsAf9vhvbV2Q+9DllAe\nA5ZIOpes5GF9cPKonxvIGgW3tOy7kayI3l5laf81bx1P0T62YgrYmBfj3xIRt5CVcD5OVj24Fzi7\nw3Pu9tzznEveJnEGWePq48Bbgavneb8BnCjpZ8BTZO02y4DjI+LO/JjPkFV9HiYrQXy7Lc5LgKPy\n9/aliLgL2JAf9whZ4mj9PK0HSrUYkKS9yb4ILwCWAv8YEecnCcbM+pYseQBI+rWIeFrSnmSZ/z0R\n4V8AsxpIWm2JiKfzm0uBPchbz82s+pImD0lLJG0ja9C6Pq+LmlkNpC557IiIFWRjAF4vqZEyHjPr\n3Z6pAwCIiKckfZlsGHVzdr8kL+1ulkhEzDvuJVnJQ9JySS/Mb48B48DW9uMiolLb+973vuQx1CGm\nqsblmHrbepGy5PESsnEFS8iS2GUR8fWE8ZhZH5Ilj4i4A1iZ6vXNbGE8wrRPjUYjdQi7qWJMUM24\nHNPgJB0k1o2kqHJ8ZsNKElHVBlMzqzcnDzMrxMnDzApx8jCzQpw8zKwQJw8zK8TJw8wKcfIws0Kc\nPMysECcPMyvEycPMCnHyMLNCnDzMrBAnDzMrxMnDzApx8jCzQpw8rHTT09OsWnUmq1adyfT0dOpw\nbEC8kpiVanp6mjVr1rJ9+4cAGBtbzzXXbGRiYiJxZDafXlYSc/KwUq1adSYzM6uBtfmejYyPb+Kr\nX706ZVjWhZchNLPSVOKKcTa8JifXsWXLWrZvz+6Pja1ncnJj2qBsIFxtsdJNT0+zYcPFQJZM3N5R\nfW7zMLNC3OZhZqVx8jCzQpw8zKyQZMlD0sslXS/pTknflXReqljMZnk0bO+SNZhKOgQ4JCK2SVoG\n3AK8OSLubjnGDaa2aDwadqdKN5hGxCMRsS2//XPgbuDQVPGYbdhwcZ441gJZEpntYrbdVaLNQ9Lh\nwHHATWkjMbNeJR9hmldZvgi8My+BmCXh0bD9SZo8JO0FXA18NiKu7XTM1NTU87cbjQaNRmNRYrPR\nMzExwTXXbGwZDTs67R3NZpNms9nXOSkbTAVsBB6PiHfPcYwbTM0SqPTwdEknAd8Abgdmgzg/Ija3\nHOPkYZZApZNHL5w8zNKodFetmdWbk4eZFeLkYWaFOHmYWSFOHmZWiJOHmRXi5GFmhTh5mFkhTh5m\nVoiTh5kV4uRRc142z1Lx3JYa87J5VhZPjBtyvoi0lcUT48ysNMmXIbTivGyepeRqS835ItL1U4e/\nmds8zCqmLo3cTh5mFVOXRm43mJpZaZw8bKhVbRDd5OQ6xsbWk104YGPeyL0udViFuNpipUrZOFjV\n9gU3mC4CJ496S/3lrUv7QhX1kjw8zsNKs+uFo2H79mxfFX9prX9OHja0PIiuXK62WGlSV1tmY6h6\n+0IVuc3DkvOXt56cPMysEA8SGwFVG8dgo8PJo8Zm2xRmZlYzM7OaNWvW1iqBOPHVXEQk24BPAY8C\nd8zxeNjcxsfPCLg0IPLt0hgfPyN1WD3ZvHlzjI29OI//0hgbe3Fs3rw5dViWy797835/U5c8Pg2c\nmjgGWwTtpYxdx4BkPTKzDatWD0nHeUTEjZIOTxlDndVlHEN7l+2WLWs58shXJ47KFsqDxGpsYmKC\na67Z2NIVmn7eRiedRprCpxkbW1/5xGdzq3zymJqaev52o9Gg0Wgki6WKJiYmKpkwulm+/KBaJL5R\n0Ww2aTabfZ2TfJxHXm25LiKO6fBYpI7PFq692rJ06bs4+uhjWb78IA8cqyiP87BKmK1ejY9v4rjj\n/gHYi61bz03Svezu4QHq1h1T5gZ8AfgR8EvgIeDctscH3gVVRZs3b47x8TNifPyMoe+uTNm97O7h\n3tFDV23q3pY/Svn6VdCpJ6IKC9YMIy8RMFiutiRWxfEOZRbth2kZvlFX+d4WW1xll4RSdi/XZVxM\nbXSr16TcGIE2j6rVw+s85L0Xo9S+tBBUvc3D6jPQayGqtKZHXcfFVFK37JJyYwRKHlUz6JJQ1UpW\n1ht6KHkkHyQ2Hw8SS2OhJYXW8x977HG2bj0Xr2BeL1493QpZSNG+vcF1yZLJQYaWTJWqXlXh5GED\n1T6WYseOO1iy5N3s2JE9XsceDo/F6czJw0p2DMceexTLl28C6tkg7MFlnTl5WE96LbZ3Gktx4YX1\nSxjWg24tqik33NuyIIMa09Bvj8l8r1vHcRaj2GNED70tyRPEvME5eRQ2yP/wgxo4VucvYR2T3kIM\nPHkAewD79XPOQjYnj+IG8YWf/cIceOCrBpI8hn306jDpJXl0bfOQ9AXgPwLPATcD+0v6aET8XRnV\nKKuGXXsYXgmc9/xjdewxsRJ0yy7Abfm/bwM2AHsxx6USBr3hkkdhC60i7F5KmIwDD3zVgortda62\njBoGNLdlT0l7AW8G/ldEPCvJwz4rbvBzZo7hNa95YEEjQ0dhHs8o6To8XdJ5wHrgduBNwGHAZRFx\ncunBeXh6MlW4wr2lU8qFriUJ2CMifrWQ4Hp8LSePhDwke3QNJHlIOgT4APDSiDhV0lHAiRFxyeBC\nnfO1nTzMEhjU6umXAl8FDs3v3wu8e2GhmVnd9ZI8lkfEFWRdtUTEs0DpVRYzXyah2nrpbfm5pINm\n70g6AXiqvJDMPJO1DnopeUwC1wG/LulbwGW0jhgy61MvJYoqripvu+pa8oiIWyS9AfjNfNf386qL\nVVhVe0pcohgi3UaRkaX+s5n9Cchun93tvEFseIRpIVUeydnr/JYqv4dRwIBGmB4PzPaX7g2cAtwK\nfGawacwGZRgWr/Fo1Orrpdry5633Jb0QuKK0iGyo9XPhpfa1VKtaFRtZ3Yom7RuwFLin3/OKbLja\nUkjVi/xF1sao+nsaNgzi0guSrmu5uwQ4CrgyItYvNHFJOhX4CNk6IZ+MiA+1PR7d4rPOhu1XetWq\nM5mZWY0v4bA4BnXphQ0tt38FPBgRDy0oMkDSHsDHgd8DHgZulrQpIu5e6HObr4xm5eulzaNZ0mu/\nDrgvIn4AIOly4HTAycN244tUV8+cyUPSz9nZy9IuImK/Bb72S4HWEswPgX+zwOe0IeXel+qZM3lE\nxLKSX7unxoypqannbzcaDRqNRknh2KyqtpcMW1WsSp9zs9mk2Wz2d1K3FtXY2fPxIrKFgA4DDuv1\nvHme7wRgc8v984H1bccMtAXZunOvxuKo+ufMIFZPB1aTTcP/V+ABYAdwZ7fzenjePYH7gcPJun+3\nAb/Vdky5n5DtpuornA/LJRCq/jn3kjx6mRj3N8CJZGM7Xkk2wvSm/so3u4tsJbI/B6aBu4ArYoR7\nWjz9vPtnMDsvZmZmNTMzq1mzZu3IflaV0C27ALfk/95GtvwgwO3dzhvExoiUPKpUhE0VSy+vW/Vf\n637M9X6rUrJiQNWWrwH7ko3JuBy4CPhWt/MGsY1K8qjalyLFf+BePoOqfU4L1f45V+lHpJfk0csg\nsdOBX5AtPfg2YD/g/YMq+Vj1VLVXY9jGerR/zqtWnVmrCY29tHn8KXBIRDwbEZdGxEUR8XjZgY2S\nycl1jI2tBzYCG/MvxboFP2+d2lEmJ9exdOm7yJrXTmTp0nft9hnMjvUYH9/E+PgmrwOSWreiCTAF\n3AlsIWvgfHG3cwa1MSLVlojBVxXKKAKXWZ3ZvHlzLF168PPxLl16cK17U4qoW7Wlny/ysWSXYPg+\n8PVez1vINkrJY9AG3T5Q9n/sYWvPKKpODaa9tHnM+gnwCPA4cPAgSj1WH8OwwFAdVLW9qZOuyUPS\nO4C3ko0wvQr49xFxV9mB2cLUrXGxbvEaPbV5XAis6HZcGRuutizIIIvAi1Efr0qR3XqrtvR9rdrF\n5MWAqqVKE7kWapjeSxlKudD1YnLysDK0X/5hbGy9u33bOHmYdeAlDbsbyIWuJZ0n6YDBhWVmw6CX\nEaYvJltf9EpJp0qaNxvZ8KjTCNV+lDWid9T0VG2RtARYBZwDvBa4ErgkIu4vNThXW5IZ9nYBN5jO\nb6BtHpJWAOcCpwL/l2wlsK9FxF8sNNB5XtPJIxG3C4y2gVx6QdI7ya5P+zjwSeA9EfFsXhq5Fygt\neZhZdfUyPP1A4IyIeLB1Z0TskHRaOWFZah7xad24q3ZIDaJOX5XnsMXXS7Vl0Yec97Ph4emFVGVq\nd1XisP4xyCn5KTYnj2KqMr29KnGMuiJzhnpJHv1MyTezmmnvct+yZe3guty7ZZeUGyNU8qjbDNg6\nxTEIdZ3xW7T0h6st9VC3JQPrGMdC1DkJOnkMuaq1DQzDF36Qqvb36UfRxNdL8nCbh+2i1DqyLbrZ\nFed3dpcP8G/ZLbuk3BiRkkeVisV1/pUtS5X+PosFlzzqodRfB1sw/3068whT28Wwz6a13lR2JTFJ\nZ5FdTOpI4PiIuHWO45w8EvCQcqty8jgS2AF8Aph08jCrloEsQ1iGiPheRNyT4rVtMKanp1m5ssFB\nB72alStPGqqVxqw3SZKH1dv09DSrV7+drVvP5Ykn/oqtW+/mjW88i5UrG04iI6S03hZJM8AhHR76\ny4i4rtfnmZqaev52o9Gg0WgsOLayDXubwYYNF/PMMx9m5ypjEPG/2br1XNas8biQOmo2mzSbzf5O\n6taXW+YGXA+snOfxAfRYL65RGBPQaSwInOFxIUOEmozzGKrV2EfhgtCTk+u44Ya388wzs3veA3w2\nYUSWQpI2D0lrJD1EtojylyX9nxRxjIIyLp8wMTHBpk2Xcdxxn2bfff8K6RfAI/gyBiOmW9Ek5Yar\nLbWIxRPphg++0HUaVWkw9eUTrKiBXHrB+jcxMTFUbRxmnTh5DDFfPsHK5GrLkKtKFcrqpbJzW3rl\n5GGWRmXntlg1lNGNa6PDJY8R5XU7bD6uttic3I1r83G1xcxK467aEeVuXFsoV1tGmLtxbS5u8zCz\nQtzmYWalcfIws0KcPMysECcPMyvEycPMCnHyMLNCnDzMFsEwTkL0OA+zktVxEqIHiZlVQB0nIXqQ\nmJmVxhPjzEo2rJMQXW0xWwR1m4ToNg8zK8RtHmZWGicPMyvEycPMCkmSPCR9WNLdkm6T9CVJ+6eI\nw8yKS1Xy+CpwdEQcC9wDnJ8oDjMrKEnyiIiZiNiR370JeFmKOMysuCq0efwJ8JXUQZhZf0pLHpJm\nJN3RYTut5ZgLgGci4vNlxWH1NYwzUYdJacPTI2J8vsclnQP8PnDKfMdNTU09f7vRaNBoNBYenFVe\n+0zULVvWVn4map01m02azWZf5yQZYSrpVGAD8IaIeGye4zzCdETVcSbqMKnyCNOPAcuAGUlbJf19\nojjMrKAks2oj4ogUr2v1sVgzUes2Ya1KPDHOKqvsL3YdV/haLJ5VazYPt6vMrcptHmZWc15JzEbW\nsK7wtVhcbbGR5gbTztzmYWaFuM3DzErj5GFmhTh5mFkhTh5mVoiTh5kV4uRhZoU4eZhZIU4eZlaI\nk4eZFeLkYWaFOHmYWSFOHmZWiJOHmRXi5GFmhTh5mFkhTh5mVoiTh5kV4uRhZoU4eZhZIU4eZlaI\nk4eZFeLkYWaFJEkekv5a0m2Stkn6uqSXp4jDzIpLVfL4u4g4NiJWANcC70sUR9+azWbqEHZTxZig\nmnE5psFJkjwi4mctd5cBj6WIo4gq/qGrGBNUMy7HNDjJrlUr6QPA24GngRNSxWFmxZRW8pA0I+mO\nDttpABFxQUQcBlwK/M+y4jCzciS/Vq2kw4CvRMRvd3jMF6o1S6TbtWqTVFskHRER9+Z3Twe2djqu\nW/Bmlk6SkoekLwK/CTwH3A/8p4j4yaIHYmaFJa+2mFk9VX6EaRUHlEn6sKS787i+JGn/CsR0lqQ7\nJT0naWXiWE6V9D1J90panzKWWZI+JelRSXekjmWWpJdLuj7/u31X0nkViGlvSTfl37e7JF0458ER\nUekN2Lfl9n8GPlmBmMaBJfntDwIfrEBMRwK/AVwPrEwYxx7AfcDhwF7ANuC3KvD5nAwcB9yROpaW\nmA4BVuS3lwHfr8hn9Wv5v3sC/wSc1Om4ypc8ooIDyiJiJiJ25HdvAl6WMh6AiPheRNyTOg7gdcB9\nEfGDiHgWuJysUTypiLgReDJ1HK0i4pGI2Jbf/jlwN3Bo2qggIp7Oby4l+zF4otNxlU8ekA0ok/Qv\nwFqyX/oq+RPgK6mDqJCXAg+13P9hvs/mIelwspLRTWkjAUlLJG0DHgWuj4i7Oh2XbIRpK0kzZEW4\ndn8ZEddFxAXABZLeSzag7NzUMeXHXAA8ExGfLzueXmOqALfA90nSMuCLwDvzEkhSeal6Rd6WNy2p\nERHN9uMqkTwiYrzHQz/PIv3Kd4tJ0jnA7wOnLEY80NfnlNLDQGuj9svJSh/WgaS9gKuBz0bEtanj\naRURT0n6MvBaoNn+eOWrLZKOaLk754CyxSTpVOAvgNMj4hep4+kg5eC67wBHSDpc0lLgD4FNCeOp\nLEkCLgHuioiPpI4HQNJySS/Mb4+RdQ50/M5VfpxHFQeUSbqXrDFptiHp2xHxjoQhIWkNcBGwHHgK\n2BoRb0wUyxuBj5A1tl0SEXN39y0SSV8A3gAcBPwE+O8R8enEMZ0EfAO4nZ3VvfMjYnPCmI4BNpIV\nLJYAl0XEhzseW/XkYWbVVPlqi5lVk5OHmRXi5GFmhTh5mFkhTh5mVoiTh5kV4uRhpZKUfLi1lcPJ\nw8rmgURDysnDAJB0fL640Qsk7ZMvTnNU2zEXSnpHy/0pSZP58V+TdIuk2yWt7vD8DUnXtdz/uKS1\n+e3XSGpK+o6kzZIOyfefly+Uc1s+QtQqpBIT4yy9iLhZ0ibgb4AxsmHJ7VOxryAbdv73+f2zgFXA\nL4A1EfEzScuBb9N9PksAkU8M+xhwWkQ8LukPgQ8A/w5YDxweEc9K2m/h79IGycnDWv0Psolt28lW\nbdtFRGyT9CJJLwFeBDwZEQ/nCeBCSScDO4BDJb2ohzlIIpu3dDTwtWyeGHsAP8ofvx34vKRryS5L\nahXi5GGtlgP7kH2Bx8iu5tfuKuAtZOuKXJ7ve1t+7sqIeE7SA8Debef9il2rya2P3xkRv9vhtd4E\nvB44jWw9l2Mi4rn+3pKVxW0e1uoTwH8jWzflQ3MccwXwR2QJ5Kp8337AT/LE8W+BV3Q470HgKElL\n8ynfp5BVXb4PHCzpBMjWt5B0VD5d/bB8EZr3AvuTJTarCJc8DABJZwO/jIjLJS0BvtVpBamIuCtf\n+eqHEfFovvtzwHWSbier9tzdekp+3kOSrgS+CzwA3Jrvf1bSW4CL8pWr9iRbLe4e4LJ8n4CPRsRP\nS3nzVoin5JtZIa62mFkhTh5mVoiTh5kV4uRhZoU4eZhZIU4eZlaIk4eZFeLkYWaF/H/B06BqPPFO\nqAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x575b330>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from scipy.linalg import sqrtm\n",
    "\n",
    "#Find the covariance matrix of the data\n",
    "cov_mat = np.cov(data.T)\n",
    "#Call the SVD function from linalg module to perform spectral decomposition on the covariance matrix\n",
    "U, S, V = np.linalg.svd(cov_mat)\n",
    "S = np.diag(S)\n",
    "SInvSr = np.sqrt(inv(S))\n",
    "\n",
    "W = U.dot(SInvSr)\n",
    "\n",
    "transformedData = data.dot(W)\n",
    "\n",
    "cov = np.cov(transformedData.T)\n",
    "print \"Covariance matrix of Whitened Data\"\n",
    "print cov\n",
    "\n",
    "plt.scatter(transformedData[:,0], transformedData[:, 1])\n",
    "plt.title('Whitened Data')\n",
    "plt.xlabel('x values')\n",
    "plt.ylabel('y values')\n",
    "plt.ylim([-3, 3])\n",
    "plt.xlim([-3, 3])\n",
    "plt.gca().set_aspect('equal', adjustable='box')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indeed, the transformed data has identity covariance matrix and when plotted, appears spherical!\n",
    "##Method 2\n",
    "In method 2 derived above, we find $W_{W} = \\Sigma^{-1/2}$ such that: \n",
    "\n",
    "$$X_{Whitened} = XW_{W}$$\n",
    "\n",
    "We implement method 2 to see that it works. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Covariance matrix of Whitened Data\n",
      "[[  1.00000000e+00  -8.85540810e-17]\n",
      " [ -8.85540810e-17   1.00000000e+00]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQ8AAAEZCAYAAACJoKC4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGPBJREFUeJzt3X+wZGV95/H35zJcvWFAJoKCiuAaNytGBRItTKHTKfbO\nxRjRGcTUxloHkyyVNbWYzWgGZDdcNuOKsabiD2prJYvO+BP54RAo9I6jRQOjhFJgBAF1QKOIMtQg\nAQxjIMx3/zjnDj09fW93n9unn3O6P6+qU3SfPqf723fobz/P9zzP04oIzMz6NZE6ADOrJycPMyvE\nycPMCnHyMLNCnDzMrBAnDzMrxMljhEnaK+nfLfDYOyRtHXZMnUjaJOlvUsdh/XHyqAlJ50n6ctu+\nnQvse3u354uIz0XETMt5CyaaIYh8O4CksyQ9LenxfPuhpE9KelmvT+7kVA4nj/q4AfhdSQKQdDSw\nDDhB0kTLvpcCNxZ8DQ0i0BJe+xsRcShwGPAfgT3ArZJeMZTIrCMnj/r4NnAwcEJ+//XA9cAP2vbd\nFxEPtpw3LekHkh6RdPH8zvwb/ab89nyy+U7+7X5mvv8PJO3Iz/2GpFe2nP9PktZJ+o6kf5Z0maRn\ntTy+2LknSrpN0mOSLgOe3eW9CyAyP4yIPydLprMtz3mFpJ/nsdwg6fh8/9nAHwF/lb+3f8j3nyvp\n3jyGuyS9tUsM1sbJoyYi4kngFmBlvusNwE3A9vz2/L4b2k59E/A7wKuAt0uaaXuciJg//1URcWhE\nXCHpROBS4L8Avw58ArhG0sHzpwFnAjPAS/LnPwuy5LDQuZImgauBzcAK4ArgDBbotiziS2TJct51\nwG8ARwK3AZ/L39sl+e0P5e/tLfnx9wKnRMRhwIXAZyUd1WcMY83Jo15u4JlEcQpZ9+Smln2v58Dk\ncVFEPBYR95O1VE6gN2cDn4iIb+Xf+J8G/hU4ueWYj0XEgxHxCHBty3MvdO7r8vOXRcRHI+LpiLgK\n+FaPMbX6OVliAiAiNkXEv0TEU2TJ4NWSDm05fr9uUURcOd9Ci4jLgZ3AawvEMbacPOrlRuAUSSuA\nIyPiPuBmslrICuAVHFjvaO3CPAEc0uNrHQusy7sdj0h6BHgR8IIFnnsPsLzLuUfn5z/Q9lo/pv96\nywuBXwBIOkjSRXk35FHgR/kxRyx0sqR3Srq9Jb7fAp7bZwxjzcmjXv4ReA5Zd+AbABHxGPAzsm/7\nn0XEjwf0Wj8BPhARK1q25RHxxUXOiR7O/TnZB7/VsfTfbVnNM4nyj4DTgVMj4jlk3Sh4JiHt99yS\njgUuAf4c+PWIWAF8l7QF49px8qiRiNhDVjj9S/ZvYWzP97V3WdqJhT8gu8iu1Mz7e+DPJL1WmUMk\nvUnS8gXOn3/+bud+E/g3SefkNZA1wGu6xJ09edbCeImkj5N11S7MH1pO1i36haRDgP/d4b21XoY+\nhCyh7AYmJL2LrOVhfXDyqJ8byIqC21v23UTWRG/vsrR/m7eOp2gfWzELbM6b8W+LiFvJWjgXk3UP\ndgLv7PCcBzz3IueS1yTWkBVXHwbeDly1yPsN4HWSHgceJavbLAdeExF35cd8mqzr8wBZC+Lmtjgv\nBY7P39uXIuJuYGN+3INkiaP172k9UKrFgCQ9m+yD8CxgEviHiDgvSTBm1rdkyQNA0q9FxBOSlpFl\n/vdGhL8BzGogabclIp7Ib04CB5FXz82s+pImD0kTknaQFbSuz/uiZlYDqVseeyPiBLIxAG+Q1EgZ\nj5n1blnqAAAi4lFJ15ENo27O75fkpd3NEomIRce9JGt5SDpC0uH57SlgGri9/biIqNR2wQUXJI+h\nDjFVNS7H1NvWi5Qtj6PJxhVMkCWxz0TE1xPGY2Z9SJY8IuJO4KRUr29mS+MRpn1qNBqpQzhAFWOC\nasblmAYn6SCxbiRFleMzG1WSiKoWTM2s3pw8zKwQJw8zK8TJw8wKcfIws0KcPMysECcPMyvEycPM\nCnHyMLNCnDzMrBAnDzMrxMnDzApx8jCzQpw8zKwQJw8zK8TJw8wKcfIws0KcPMysECcPMyvEycPM\nCnHyMLNCnDzMrBAnDzMrxMnDzApx8jCzQpw8zKyQZMlD0jGSrpd0l6TvSjonVSxm1r9kv1Ur6Sjg\nqIjYIWk5cCvw1oi4p+UY/1atWQKV/q3aiHgwInbkt38J3AO8IFU8ZtafStQ8JB0HnAjckjYSM+tV\n8uSRd1muBN6Tt0DMrAaWpXxxSQcDVwGfjYirOx0zOzu773aj0aDRaAwlNrNx0mw2aTabfZ2TsmAq\nYDPwcET89wWOccHULIFeCqYpk8cpwI3AHcB8EOdFxFzLMU4eZglUOnn0wsnDLI1KX6o1s3pz8jCz\nQpw8zKwQJ48xtXXrVlatOoNVq85g69atqcOxGnLBdAxt3bqV1avXsmfPhwCYmlrPli2bmZmZSRyZ\nVYWvtlhHq1adwbZtpwNr8z2bmZ6+hq9+9aqUYVmF+GqLmZXGyWMMrVt3NlNT68kG+G5mamo969ad\nPZTXdq1ldLjbMqa2bt3Kxo2XAFkyGUa9w7WW+nDNwyrFtZb6cM3DzEqTdEq+jZd1685m+/a17NmT\n3c9qLZvTBmWFudtiQ5Wi1mL9c83DzApxzcMGypdZrZVbHtYTX2YdL+622MD4Mut4cbelRG7C27jz\npdoC2pvw27evHfkmvC+zWjt3WwoY1ya8L7OOj166LW55WM9mZmacMGwfJ48C3IQ3c7elMDfhbZT5\nUq2ZFeJLtWZWGicPMyvEycPMCnHyMLNCnDzMrJCkyUPSJyXtknRnyjjMrH+pWx6fAk5LHIOZFZA0\neUTETcAjKWOw0ebZz+VJ3fKwEZb6gzs/+3nbttPZtu10Vq9e6wQyQJWf2zI7O7vvdqPRoNFoJIvF\neleFZQs2brwkf/1s9vOePdk+TyU4ULPZpNls9nVOrZKH1Yc/uPXS/sV84YUXdj2n8snDrCjPfi5X\n0olxkr4ArASeCzwE/HVEfKrlcU+Mq6mqLJjs2c/FeFatLclSP3j+4NaXk4cVVpWWg6XhKflW2P4F\nzyyJzLciypL60q71xwVTq4QqXNq1/rjlUVNlf0uvW3c2U1Prgc3A5vxKxdkDf515KVo6wzZyLauI\nqOyWhWft5ubmYmrq+QGbAjbF1NTzY25urpTXmZ5eE9PTa0p5/lbT02vy9xP5timmp9eU+prDNKx/\ns0HJP3uLfz67HZByc/LobBQ/aHX7cPWrbv9mvSQP1zysEmZmZtiyZXPLpV3XOyqvW3ZJueGWR0ej\n/i1dRUvtwtXt3wx3W0bXMOsRC71mihhSGNQHv05/r16ShweJWU/aL6VOTr4PeIonn/wIMNqDyMbx\nt4n9W7U2MO2zZJ98EuD/4lmz48vjPEbMyI0lWMAw3+ewx7zURrd+TexfgzgIOKyfc5ay4ZpHX8os\nyrU/9+TkkTE5eXiSAmCK4mOd6hWDwCAKpsAXgMOAQ4C7gQeAv+p23iA2J4/+lD2WoCoF07qNmaij\nXpJHLzWP4yPiMUnvAL4CnAvcBvztQJtA1rf2Ke9lm5mZOaCm4RrH+Oql5rFM0sHAW4FrI+IpwJdA\nEuu0uO/KlSflffP3Aq9jYmIdK1ee1NNz1alO4hpERXRrmgDnkHVVvkKWbI4Dbup23iA23G1Z0EJN\n9w0bNsTExIpF6wGt3Y0NGzbUavDSvHGrQQwbZQwSAwQs6/e8IpuTx8IWSh7d6gHtxcYs0axz/cD2\n00vy6NptkXSUpEslzeW7Xs4zo2UskaJN9/ap73v3/h3wjXKDtZHUS8F0E9nPQp6f398JXA5cWlJM\n1oPFJpL1u2L4xMRO9u7d3PPxg+R1TmusW9ME+Hb+39tb9u3odt4gNkrotoxDX3mx99hpjMSGDRuS\n/E3qNllsnDCgcR5Nsp9GuD2/fzJwQ7fzBrENOnn4f9ZMVRLosMdrVOV918GgksdvA98EHs3/uxN4\ndbfzBrENOnl4cFG1DPPfw18c/ekleXSteUTErZJWAr+Z7/p+ZGM9zJZkmL/o5p+/HLyuyUPSWrJB\nYfPTc0/Kp+t+utTISuCfH+wsVdHSq4fVXLemCXAx8PF8+3vgh8CV3c4bxIYLpqUbl+b8uLzPQaGM\nxYAkHQ58MSJK/4rwYkDlG6eFbnxZuHdlLQb0BPCSYiGZpdNpYp8V18sI02tbtuuA7wNbBvHikk6T\n9D1JOyWtH8RzVlGVJ55VaZJZlf9O1kG3fg3QaNlOAY7pdk4vG9nCQveSTbQ7GNgBvLztmLK6dENT\nh752FepAdfg7jROqvHo68DpgruX+ucC5bceU85cZIo8t6U2vf6cqJLpx0EvyWLDbIumXkh5fYHts\nAI2eFwL3t9z/ab7PKqYq3YlOa5i4e5POggXTiFhe8mv3dBlldnZ23+1Go0Gj0SgpnHLUfWzJsH69\nvpe/kwd6lafZbNJsNvs7qVvTJJ7pQjwPePH81ut5izzfyezfbTkPWN92TGnNsmGqc1N72EPIF/s7\npegC1vnfbikY0NyW08nms/wL8CNgL3BXt/N6eN5lwH1kBdNJRrRgWndVqtkMu6g6zkXcQSWPO4Aj\neGZW7e8Bn+x2Xi8b8EayS7/3Aud1eLzEP4/1omofoGG2BKqUOIetl+TRyyCxpyJit6QJSQdFxPWS\nPtpf56iziPgK2dqoVlFVm3/S60AvjyYdgm7ZBfgacCjZHJfLgI8B3+x23iA23PKwAgb5w9RVanUN\nE4OY2yLpEOBXZKNR30H2A1Cfi4iHy0poLa8d3eIzazfI+Trj2oIZ1NyWPwMui4gHyNYztRE0rh+S\nbjwfZmG9JI9Dga9KeoSs23JFROwqNywbpmGN5RiWuo+tqYuep+RLejXwduBtwE8j4tQyA8tfc6y6\nLam+/UdxWr5bUksz6Cn5DwEPAg8DRy4lMDvQqH37p+buRvl6mZL/bklN4Otk4z3+NCJeVXZg46b9\nx5j27PnQvm/OslVpWn4VVWVuT9X00vI4BviLiNhRdjCWRutYjt27HwZ+Y1/i6vbtPerdA7cIF9Ht\nWm7KjTEa51GFMQX9xlCFmMs2rqNMGdAIUxuCKozk7HfWqme5jjcnjwoZhSLf7t27WLXqDGBp3ZhB\ndIcG8Ry+7LuIbk0T4BxgRbfjytgYo25LFSy12zI5eXhMTh5ZiWHhGzZsiImJFQPpUo3jtHwGNKv2\nA2SzXi8HTiMfGzKMzclj+Pr9oLQef+KJKwdSH1hqnWFubi4mJp47lrWKQeklefTyc5PnS/qfwCrg\nLOBiSZcDl0bEfYNuCVla/XadWo+f766ktnHjJezde3TqMEZeTzWPiNgr6UFgF/A0sAK4UtLXIuJ9\nZQZo1dBL/WBQ9YGlPs/u3buAnwDv3bdP+gvWrbus71hsEd2aJsB7gFuBr5INTz843z8B3Nft/KVs\nuNtSCf3UIAZVH1jK8zzTfZoLWBNwcrz0pa8sHMs4YkA1jwuBYxd47Phu5y9lc/KohrqNdahbvFXU\nS/LopeZxwSKP3V28zWNWDl9eHY6uc1tGmecs9KZuc1/mB9xNT1/D9PQ1Hk5ekp6n5KdQ5pT89jkL\nU1Pr/T/ZIkZ9Dovtr5cp+WObPKq0hoU/mFY1g17Pw0rgWZtWV2ObPKpSVPPkMqursU0eVZjFalZn\nY1vzqAoXbq2KXDCtCRdMrWqcPMyskF6Sx1gPEjOz4pIkD0lnSrpL0tOSTkoRg5ktTaqWx53AauDG\nRK9vZkuU5FJtRHwPsn6VmdWTax5mVkhpLQ9J24CjOjz0/oi4ttfnmZ2d3Xe70WjQaDSWHJuZ7a/Z\nbNJsNvs6J+mlWknXA+si4rYFHvelWrME6nKp1oUPsxpKdal2taT7gZOB6yR9JUUctjgvlmSL8QhT\n68hzbsabh6dbYVVaLMmGry41DzOrobFdz8MWV5XFkqy63G2xBXmpgPHlmoeZFeKah5mVxsnDzApx\n8jCzQpw8zKwQJw8zK8TJw8wKcfIws0KcPMysECcPMyvEycPMCnHyqIGqLMpTlTisIiKislsW3nib\nm5uLqannB2wK2BRTU8+Pubm5sY3DhiP/7C3++ex2QMrNySNienpN/oGNfNsU09NrxjaOYZmbm4vp\n6TUxPb1mLJNkL8nD63mYtWlfgnH79rVegrGTbtkl5YZbHpXpLlQljmEYt1ZWJ7jlUX8zMzNs2bK5\nZVGeNN+AVYnDqsOLAZm18crxXknMrLBxX4LRyWNEjPv/yDZ8Th4jwE1oS8HJYwT4x5csBS+AbGal\n8aXaivOPL1lVJem2SPow8AfAk8B9wLsi4tEOx419twVcMLXhq2zNQ9I08PWI2CvpIoCIOLfDcU4e\nZglUtuYREdsiYm9+9xbgRSniMLPiqlAw/WPgy6mDMLP+lFYwlbQNOKrDQ++PiGvzY84HnoyIz5cV\nh5mVo7TkERHTiz0u6Szg94FTFztudnZ23+1Go0Gj0Vh6cGa2n2azSbPZ7OucVAXT04CNwMqI2L3I\ncS6YmiVQ5astO4FJ4Bf5rpsj4t0djnPyMEugssmjV04eZmlU9lKtmdWfk4eZFeLkYSPPvzdTDtc8\nbKR5PZRiXDC1sef1UIpxwdTMSuP1PGykeT2U8rjbYiPP66H0zzUPMyvENQ8zK42Th5kV4uRhZoU4\nedjQeKTnaHHB1IbCIz3rxQXTEVTXb++NGy/JE8daIEsi85dPrZ48SKxG2r+9t29f629vS8bJo0b2\n//aGPXuyfXVIHh7pOXqcPGwoZmZm2LJlc8tIT7eY6s4F0xpx0dGGxcPTR5DnadgwOHmYWSG+VGtm\npXHyMLNCnDzMrBAnDzMrxMnDzApx8jCzQpw8zKyQJMlD0t9I+o6kHZK+LumYFHGYWXGpWh5/GxGv\njogTgKuBCxLF0bdms5k6hANUMSaoZlyOaXCSJI+IeLzl7nJgd4o4iqjiP3QVY4JqxuWYBifZrFpJ\nHwD+M/AEcHKqOMysmNJaHpK2Sbqzw/ZmgIg4PyJeDGwC/q6sOMysHMknxkl6MfDliPitDo95VpxZ\nIt0mxiXptkh6WUTszO++Bbi903HdgjezdJK0PCRdCfwm8DRwH/BfI+KhoQdiZoUl77aYWT1VfoRp\nFQeUSfqwpHvyuL4k6TkViOlMSXdJelrSSYljOU3S9yTtlLQ+ZSzzJH1S0i5Jd6aOZZ6kYyRdn/+7\nfVfSORWI6dmSbsk/b3dL+uCCB0dEpTfg0Jbb/w34fxWIaRqYyG9fBFxUgZj+A/DvgeuBkxLGcRBw\nL3AccDCwA3h5Bf4+rwdOBO5MHUtLTEcBJ+S3lwPfr8jf6tfy/y4D/hE4pdNxlW95RAUHlEXEtojY\nm9+9BXhRyngAIuJ7EfGD1HEArwXujYh/ioingMvIiuJJRcRNwCOp42gVEQ9GxI789i+Be4AXpI0K\nIuKJ/OYk2ZfBLzodV/nkAdmAMkk/IfvBkotSx9Pmj4Evpw6iQl4I3N9y/6f5PluEpOPIWka3pI0E\nJE1I2gHsAq6PiLs7HVeJ322RtI2sCdfu/RFxbUScD5wv6VyyAWXvSh1Tfsz5wJMR8fmy4+k1pgpw\nBb5PkpYDVwLvyVsgSeWt6hPyWt5WSY2IaLYfV4nkERHTPR76eYb0Ld8tJklnAb8PnDqMeKCvv1NK\nDwCtRe1jyFof1oGkg4GrgM9GxNWp42kVEY9Kug74HaDZ/njluy2SXtZyd8EBZcMk6TTgfcBbIuJX\nqePpIOXgum8DL5N0nKRJ4A+BaxLGU1mSBFwK3B0RH0kdD4CkIyQdnt+eIrs40PEzV/lxHlUcUCZp\nJ1kxab6QdHNEvDthSEhaDXwMOAJ4FLg9It6YKJY3Ah8hK7ZdGhELX+4bEklfAFYCzwUeAv46Ij6V\nOKZTgBuBO3imu3deRMwljOmVwGayhsUE8JmI+HDHY6uePMysmirfbTGzanLyMLNCnDzMrBAnDzMr\nxMnDzApx8jCzQpw8rFSSkg+3tnI4eVjZPJBoRDl5GACSXpMvbvQsSYfki9Mc33bMByW9u+X+rKR1\n+fFfk3SrpDsknd7h+RuSrm25f7Gktfnt35bUlPRtSXOSjsr3n5MvlPOdfISoVUglJsZZehHxLUnX\nABuAKbJhye1Tsb9INuz8/+T3zwRWAb8CVkfE45KOAG6m+3yWACKfGPZx4M0R8bCkPwQ+APwJsB44\nLiKeknTY0t+lDZKTh7X6X2QT2/aQrdq2n4jYIel5ko4Gngc8EhEP5Angg5JeD+wFXiDpeT3MQRLZ\nvKVXAF/L5olxEPCz/PE7gM9LuprsZ0mtQpw8rNURwCFkH+Apsl/za3cF8DaydUUuy/e9Iz/3pIh4\nWtKPgGe3nfdv7N9Nbn38roj43Q6v9SbgDcCbydZzeWVEPN3fW7KyuOZhrT4B/A+ydVM+tMAxXwT+\nE1kCuSLfdxjwUJ44fg84tsN5PwaOlzSZT/k+lazr8n3gSEknQ7a+haTj8+nqL84XoTkXeA5ZYrOK\ncMvDAJD0TuBfI+IySRPANzutIBURd+crX/00Inbluz8HXCvpDrJuzz2tp+Tn3S/pcuC7wI+A2/L9\nT0l6G/CxfOWqZWSrxf0A+Ey+T8BHI+KxUt68FeIp+WZWiLstZlaIk4eZFeLkYWaFOHmYWSFOHmZW\niJOHmRXi5GFmhTh5mFkh/x+EZMy+f+2gWwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x57c0470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Find the inverse of the covariance matrix\n",
    "covInv = inv(cov_mat)\n",
    "#Find the square root of the inverse covariance matrix\n",
    "covInvSr = sqrtm(covInv)\n",
    "\n",
    "transformedData = data.dot(covInvSr)\n",
    "\n",
    "cov = np.cov(transformedData.T)\n",
    "print \"Covariance matrix of Whitened Data\"\n",
    "print cov\n",
    "\n",
    "plt.scatter(transformedData[:,0], transformedData[:, 1])\n",
    "plt.title('Whitened Data')\n",
    "plt.xlabel('x values')\n",
    "plt.ylabel('y values')\n",
    "plt.ylim([-3, 3])\n",
    "plt.xlim([-3, 3])\n",
    "plt.gca().set_aspect('equal', adjustable='box')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indeed, the transformed data has identity covariance matrix and when plotted, appears spherical! To a very close appromixation, method 1 and method 2 give the same transformation. \n",
    "\n",
    "This concludes the whitening tutorial."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}