Skip to content

Instantly share code, notes, and snippets.

@CalvinTChi
Created March 23, 2016 09:09
Show Gist options
  • Save CalvinTChi/dab3843882cbcc5a0cee to your computer and use it in GitHub Desktop.
Save CalvinTChi/dab3843882cbcc5a0cee to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Principal Component Analysis (PCA) Tutorial\n",
"### Author: _Calvin Chi_"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Why PCA works\n",
"The purpose of principal component analysis (PCA) is to perform dimensionality reduction by projecting data points onto new dimensions that capture maximum variance. PCA could be used as a visualization technique for understanding the hidden and simplified structure that underlie data. The new dimensions (axes) that PCA projects data points to are the eigenvectors. How do you project data points to another dimension (axis)? Before we answer that, let us review a linear algebra concept. Let $A$ be a matrix where each column is treated as a vector and let $X$ be a matrix of data points. \n",
"$$A = \\begin{bmatrix}\n",
"1&1\\\\\n",
"1&-1\\\\\n",
"\\end{bmatrix}\n",
"X = \\begin{bmatrix}\n",
"1&1\\\\\n",
"2&2\\\\\n",
"3&3\\\\\n",
"4&4\\\\\n",
"2&3\\\\\n",
"3&2\\\\\n",
"\\end{bmatrix}$$\n",
"\n",
"Plotting the vectors and data points: "
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAK0AAAEKCAYAAABgwdWhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADvdJREFUeJzt3XuQlfV9x/H3B5SKwUsslqSKg5PaDDWppNOig1qxkUuM\nxkGmY9uJoWHGdgxJjTWpY7CVTIemJENM81fUaJSYaDu2EGk6rHghAdtQmYIhoBaNtppa74mgdLzw\n7R/nAQ64u5zd5/p7zuc1s7OX8+z5/XZ98/js5XxXEYFZSsbUvQGzkXK0lhxHa8lxtJYcR2vJcbSW\nHEdryXG0OUl6StLrkl6V9IqkByX9qST18L5TJO2R5P8OI+BPVn4BXBARRwMnAX8LXA3cPIL7OGTg\ntp+jLVBE7IyI1cAlwAJJp0r6qKTNkn4h6b8lXdf1Lj/Mnv9c0k5Jp0t6n6T7Jb0o6QVJt0s6pvqP\nprkcbQki4iHgGeBsYBfw8Yg4BvgocLmki7JDz86eHxMRR0XExuz1pcB7ganAZGBJVXtPgaMtz/8A\n746IH0TENoCI2ArcCZyTHfOOy4KIeCIi7ouINyPiReD6ruMNOKzuDbTYCcDLkk6nc517KjAO+CXg\nH4Z6J0mTgL8DzgKOonNiebn03SbEZ9oSSPodOtE+CHwXWAWcGBHHAt9g/+d9sF+x+xvgbeAD2SXF\npfi/0wH8ySiGACQdLekC4A7g2xHxE2AC8EpEvCFpOvBH7I/1BWAP8L6u+5oAvAa8KukE4PMVfQzJ\nkH+fNh9JTwKTgLfoBLgNuB34RkSEpPnAcuA44AfAk8CxEfGJ7P2/CFxO51JtLp0v3FYA7wd2ZPf1\n2Yg4qcqPq8lyRyvpKeBVOv9LezMiphewL7MhFfGFWAAzI8JfLFglirqm9U90rDJFRBvAvZI2Sbqs\ngPszG1YRlwdnRsSzko4H1kp6NCLWF3C/ZoPKHW1EPJs9f0HSSmA6sC9aSf72hPUsIg55qZnr8kDS\nkZKOyl5+FzAb2DrIRip7uu6667xeouv1Ku+ZdhKwMvvV0cOA70TEPTnv02xYuaKNiCeBaQXtxawn\nrfsx7syZM71ewuv1ovQf40qKstewdpBElP2FmFkdHK0lx9FachytJcfRWnIcrSXH0VpyHK0lx9Fa\nchytJcfRWnIcrSXH0VpyHK0lx9FachytJcfRWnIKiVbS2GxE++oi7s9sOEWdaa8AtjP4vFWzQuWO\nVtKJwPnAN/FMr9YYGBhg9uz5zJ49n4GBgbq3c4AixiJdT2fw79EF3Jc1wMDAAPPmLWD37mUAbNiw\ngJUrb2POnDk176wj74SZC4DnI2IzPsu2xvLlN2bBLgA68S5ffmPd29on75l2BvAxSecDRwBHS1oR\n2ZTrvZYsWbLv5ZkzZzbysfRWvXXr1rFu3bqRv2OBc5jOAVYP8vawtKxZsybGj58UcGvArTF+/KRY\ns2ZN6etmrRyytaL/JJO/e9ACc+bMYeXK2/ZdElx1VXOuZ8ETZqxBPGHGWsvRWnIcrSXH0VpyHK0l\nx9FachytJcfRWnIcrSXH0VpyHK0lx9FachytJcfRWnIcrSXH0VpyHK0lx9FachytJSfv3IMjJG2U\ntEXSdklfKmpjZkPJFW1E/B9wbkRMA34TOFfSWYXsrOGqHhvU5DFFlevlcea9PAFHAg8Bv3HQ20t9\nrHwdqp4LUNccgqrR49yDImIdA2wBdgJfHuT2Kj7eSs2adXEWUGRPt8asWRe3Zr269Bpt7mEdEbEH\nmCbpGGBA0syIWNd9jMci2WBqH4vU+YfCXwKfO+htZf8DrZwvD8pBj2faXBNmJE0E3oqIn0saDwwA\nX4yI+7qOiTxrNNXAwEDX2KA/KX1sUNXr1aHXCTN5o/0gcBud69oxwLcj4isHHdPKaK14lUTb40Yc\nrfXEs7ystRytJcfRWnIcrSXH0VpyHK0lx9FachytJcfRWnIcrSXH0VpyHK0lx9FachytJcfRWnIc\nrSXH0VpyHG3RnngCbrih7l20mqMtynPPwWc+A1OnwrRpde+m1XLNPZA0GVgB/AoQwI0R8fUiNpaU\nVavg0kth1y6YPx9OP73uHbVa3jPtm8CVEXEqcAawSNLU/Ntqvu7ZWht/+lMYOxYOPxyWLi19Pc/y\nKnZYxyrgwwe9rfChDnXrHp4xg8XxEorN114bsWJF6et5WEcBs7xif5xTgP8CJkTLo907W2sGG+Il\n3h0Xs8izvArQa7S5Z3kBSJoA3AVcERG7Dr69jbO8ZrCD1VzJZdzEP7GLWdxd95aSU9ssL+BwOuOQ\nPjvE7eX/E63Yj5Yvj5dQXMwiz/IqEBXN8hKdsUgvRcSVQxwTedZonAcfhAsvZMuiRfzFxu2AZ3kV\npapZXmcBPwR+TOdbXgDXRMSarmPaE20WLDfd1PnWlhXKs7yK5mBL51leRXKwjeJoD8XBNo6jHY6D\nbSRHOxQH21iOdjAOttEc7cEcbOM52m4ONgmOdi8HmwxHCw42MY7WwSanv6N1sEnq32gdbLL6M1oH\nm7T+i9bBJq+/onWwrdA/0TrY1uiPaB1sq7Q/WgfbOu2O1sG2Uu5oJd0i6TlJW4vYUGEcbGsVcab9\nFjC3gPvJ5YDZWl/9qoNtsdwTZiJivaQp+bcyegMDA8ybt4Ddu5cxgx2csvZzbLl2MdMcbCsVMhap\nbsuX35gF+2vZqKJPsXPjdu6pe2NWikqirWKWl2drpae2WV6xf2Li1iFuK2Hq04Gqnq1l5aCKWV57\nZde0qyPig4PcFkWsMaSaZmtZ8SobiyTpDuAc4JeB54G/iohvdd1eXrT+tlartH+Wl4NtnXbP8nKw\nfS29aB1s30srWgdrpBStg7VMGtE6WOvS/GgdrB2k2dE6WBtEc6N1sDaEZkbrYG0YzYvWwdohNCta\nB2s9aE60DtZ61IxoHayNQP3ROlgboXqjdbA2CvVF62BtlOqJ1sFaDtVH62Atp2qjdbBWgCJmec2V\n9KikHZKuHvJAB2sFyfXARkljgceA84CfAQ8BfxgRj3QdE7Fhg4O1Q6rqgY3Tgccj4qmIeBO4E7jo\nHUc5WCtQ3mhPAJ7uev2Z7G0HcrBWoLyzvHq6tliydi1s7YyvLWuWl6VntLO88l7TngEsiYi52evX\nAHsiYlnXMRETJ8K998Jpp416LWu/qq5pNwGnSJoiaRxwCQwyrnDZMjjvPHj44ZzLmeW8PIiItyR9\nGhgAxgI3d3/nYJ+FCzvPzzvPZ1zLrdpZXrfcAldf7XBtUL1eHlQ7CdxnXCtA9ePrHa7lVM/fXHC4\nlkN9fyjE4doo1fvXbRyujUL9f5LJ4doI1R8tOFwbkWZECw7XetacaMHhWk+aFS04XDuk5kULDteG\n1cxoweHakJobLThcG1SzowWHa+/Q/GjB4doB0ogWHK7tk0604HANSC1acLiWYLTgcPvcqB+NK+n3\nJW2T9Lak3ypyUz1ZuNCP8u1Tec60W4F5wA0F7WXkfMbtS6OONiIehc4jKGvlcPtOmte0B3O4fWXY\naCWtBd4zyE1fiIjVvS6yZMmSfS+XNsvL4SanllleAJIeAK6KiP8Y4vYoeyDIATwQJFlVD+uo+cK2\ni8+4rTfqaCXNA74OTAS+L2lzRHyksJ3l4XBbrdpZXlXzpUJSmjnLq2o+47ZSu6MFh9tC7Y8WHG7L\n9Ee04HBbpH+iBYfbEv0VLTjcFui/aMHhJq4/owWHm7D+jRYcbqL6O1pwuAlytOBwE+No93K4yXC0\n3RxuEhztwRxu4znawTjcRnO0Q3G4jeVoh+NwG8nRHorDbZw8jxH7CnAB8AbwBPDJiPhFURtrFIfb\nKKOe5QXcA5waEacB/wlcU8yWGsqzwxojz1iktV2vbgTm599Ow/mM2whFXdMuBO4o6L6azeHWLvdY\nJEmLgTci4rsl7K+ZHG6tho02ImYNd7ukPwbOBz483HGVzPKqmsPNrfJZXpLmAsuBcyLixWGOq29Y\nRxW6B4Ls3g1nnFH3jpLV67COPNHuAMYBL2dv+reI+NQgx7U7WtgfLsD27XD88fXuJ1GlT5iJiFNG\n+76tc9xx8NprnTPt0qXwta/VvaNWa/csryqtX985227aBI89BiefXPeOklP65cEINtIf0QJEwPe+\nB9u2weLFde8mOY62Tnv2wJg8P2zsT71G689sGRxsqfzZteQ4WkuOo7XkOFpLjqO15DhaS46jteQ4\nWkuOo7XkOFpLjqO15DhaS46jteQ4WkuOo7XkjDpaSX8t6WFJWyTdJ2lykRszG0qeM+2XI+K0iJgG\nrAKuK2hPuYzmcfRerznr9WLU0UbEzq5XJwBDzj6oUtv/o7Z9vV7kmuUlaSlwKfA64CkVVolhz7SS\n1kraOsjThQARsTgiTgJuBa6vYL9mxTwaV9JJwL9ExAcGua3PHopreZQ6YUbSKRGxI3v1ImDzaDdh\nNhJ5ZnndBbwfeJvO+PrLI+L5AvdmNqjSh3WYFa20n4hJmivpUUk7JF1d1jpd690i6TlJWytYa7Kk\nByRtk/QTSX9W8npHSNqY/SBnu6Qvlble17pjJW2WtLqCtZ6S9ONsvX8f9uCIKPwJGAs8DkwBDge2\nAFPLWKtrzbOBDwFby1wnW+s9wLTs5QnAYxV8fEdmzw8DfgScVcHH+efAd4C7K1jrSeC4Xo4t60w7\nHXg8Ip6KiDeBO+l8sVaaiFgPvFLmGl1r/W9EbMle3gU8AvxqyWu+nr04js5J4eVhDs9N0ol0prx/\nE6jqi+me1ikr2hOAp7tefyZ7W+tImkLnDL+x5HXGSNoCPAc8EBHby1yPzvfdPw/sKXmdvQK4V9Im\nSZcNd2BZ0fbFV3eSJgB3AVdkZ9zSRMSe6Pyex4nA70qaWdZaki4Ano+IzVR3lj0zIj4EfARYJOns\noQ4sK9qfAd2/9TWZztm2NSQdDvwjcHtErKpq3ej8VczvA79d4jIzgI9JepLOn9r6PUkrSlyPiHg2\ne/4CsJLOJeagyop2E3CKpCmSxgGXAHeXtFblJAm4GdgeEaXPqpc0UdKx2cvjgVkM8cOcIkTEFyJi\nckScDPwBcH9EfKKs9SQdKemo7OV3AbOBIb8LVEq0EfEW8GlgANgO/H1EPFLGWntJugP4V+DXJT0t\n6ZMlLncm8HHg3OxbNJuzv/ZTlvcC92fXtBuB1RFxX4nrHazsy71JwPquj++fI+KeoQ72DxcsOX64\njSXH0VpyHK0lx9FachytJcfRWnIcrSXH0Vpy/h/ZP4NZmS3E3wAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x55bc7d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"x = np.array([[1, 1], [2, 2], [3, 3], [4, 4], [2, 3], [3, 2]])\n",
"A = np.array([[1, 1], [1, -1]])\n",
"fig, ax = plt.subplots()\n",
"plt.scatter(x[:, 0], x[:, 1])\n",
"plt.ylim([-3, 5])\n",
"plt.xlim([0, 5])\n",
"plt.title(\"Data\")\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"ax.quiver((0, 0), (0, 0), (1, 1), (1, -1), scale_units = 'xy', scale = (0.45), color = 'r')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The operation $XA$ rotates the data points in $X$ such that they are plotted against the vectors in $A$ as the axes. In other words, $A$ is a rotation matrix. Let us illustrate that: "
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAACxCAYAAAAoJG0qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEk1JREFUeJzt3X+QXlV9x/H3x0SdIFJEMAhEUSFTsGqhmqYwDhmUJPxo\nkMEfUK0RO9ChVdFiBZQRmGo11owIDNQoQvwFUiAYNGQJVhRGBTEBEUgFJBJ+BTD8iBBbYj79455l\nl82zm+xzd/duuJ/XzDP73HvPc8957ux+99xz7/0e2SYiItrhBU03ICIixk6CfkREiyToR0S0SIJ+\nRESLJOhHRLRIgn5ERIsk6Ed0QdIMSaubbkfEcCXox7ghaZWkpyWtk/SQpG9K2m4Ynz1wGHVdKOnf\num/tZve/UdIfynd5VNI1kt49jM/nn0qMigT9GE8MHGb7pcCbgDcApw7jsxqthnXpjeW7TAUuBM6R\n9OlmmxRtl6Af45LtNcDVwOt710maI+k2SY9J+pGkPy/rvwm8Criy9Kw/Xtb/l6QHJT0u6ceS9i7r\njwP+DvhEKf+9sn4XSZdJeljSbyV9uF/dk8rZwVpJtwFvGcZ3WWv7W8DxwCmSXlb2eYyk2yU9Kenu\n0i4kvQS4CtiltO9JSTtLmibpZ+X7PyDpbEkv7PogRysl6Md4IwBJuwGzgRvK8lTgO8BHgB2BJVRB\nfqLtvwfupZwl2P5i2dcPgD2AnYDlwLcBbC8o7+eV8odLegFwJbAC2AV4G/BRSTPLvk4DXgO8FpgF\nzKU6uxiOxcBEYFpZXgMcans74BjgS5L2sf1U+e4PlPZtZ/shYANwAvBy4G9KG/9pmG2IlkvQj/FE\nwBWSnqQK4ncDnynb3gN83/YPbf8J+CIwCdhvsJ3ZvtD2U7afAc4A3iTppQPq6/UWYEfbn7G9wfY9\nwNeAo8r2dwGftf247fuALzPM4aTSjkeBHcryklIPtn9CdWbz1g5t6/38cts32t5o+3fAAuCA4bQh\nIkE/xhMDh5ee7wzgQODNZdsrqf4RVAWrTIGrgV077UjSCyR9XtJdkp4A7imbdhyk7ldTDac81vsC\nTgFeUbbvUurrde/AHWxOGYrZCVhblg+W9HNJvy/1HULVix/s81Mlfb8MWT0BfHao8hGdJOjHuFR6\nvmcD88qqB6gCMwCSBEwB7u/9yIBdvBeYA7zN9p9RDc1AXw96YPl7gXtsv6zfazvbh5XtD1JdN+j1\nKobvcKohmhslvRi4DPgC8ArbL6MashqsfQDnAbcDe5Tv9CnyNxzDlF+YGM/OBKZJ+mvgEuBQSQeW\nHvOJwB+Bn5aya4DX9fvstsD/AmvLhdF/H7DvNVTj871uBNZJ+kS5aDtB0l9I6j3TuITqIuz25XrD\nh9m83usTO0h6L3AO8HnbjwEvKq9HgY2SDgZm9vvsGuDlA25Z3RZYBzxdLmIfvwVtiHiOBP0Yt2w/\nCiwETrL9G+B9VL3/R4BDgb+1vaEU/xxwahma+RfgG8DvqM4Efg38jOf2ns8H9i7lL7e9ETgM+Evg\nt6WOBUBv0D2j7O8eYGnZ/+Yu5N4iaR1wJ/BB4KO2Ty/fbR3VRelLqIZ7jga+1++7rwQuAn5b7hja\nGfg41V1HT5a2XbwFbYh4DtWZREXSFKpf/ldQ/fItsH1Wh3JnAQcDTwMfsL2i60ojIqJrE2t+/hng\nY7ZvlrQt8EtJy2zf0VtA0iFUY5B7ltP084DpNeuNiIgu1Bresf2Q7ZvL+z8Ad1Dd5dDfHKpTdGzf\nAGwvaXKdeiMiojsjNqYvaXdgH8rDNP3synNvdbsP2G2k6o2IiC03IkG/DO1cCpxQevybFBmwnItP\nERENqDum3/vAyWXAt2xf0aHI/VT3U/fajb57q/vvJ/8IIiK6YHuLnw6v1dMvD8icD9xu+8xBii0G\n3l/KTwceL8m0NmE7L5vTTjut8TaMl1eORY5FjsXQr+Gq29Pfn+re6V9J6r0N85OUpxVtf8X2EkmH\nSLoLeIoqsVRERDSgVtC3fT1bcLZg+0N16omIiJGRJ3LHoRkzZjTdhHEjx6JPjkWfHIvu1XoidyRJ\n8nhpS0TE1kISHqsLuRERsXVJ0I+IaJEE/YiIFknQj4hokQT9iIgWSdCPiGiR2kFf0tclrZF06yDb\nZ0h6QtKK8jq1bp0REdGd2gnXgAuoprD7xhBlfmx7zgjUFRERNdTu6du+DnhsM8W2+MGBiIgYPWMx\npm9gP0m3SFoiae8xqDMiIjoYieGdzVkOTLH9tKSDgSuAqWNQb0REDDDqQd/2un7vr5J0rqQdbK8d\nWPb0009/9v2MGTOSVCkiYoBrr72Wa6+9tuvPj0jCtTI/7pW239Bh22TgYduWNA24xPbuHcol4VpE\nxDANN+HaSEyXeBFwALCjpNXAacALoZpEBXgncLykDcDTwFF164yIiO4ktXJExFYsqZUjImJQCfoR\nES2SoB8R0SIJ+hERLZKgHxHRIgn6EREtkqAfEdEiCfoRES0y6pOolDJnSbqzZNrcp26dERHRnZHo\n6V8AzB5so6RDgD1s7wkcB5w3AnVGjLqenh5mzjySmTOPpKenp+nmRIyI2rl3bF9XEq4NZg6wsJS9\nQdL2kibbXlO37ojR0tPTwxFHzGX9+nkAXH/9XBYtWsisWbMabllEPWMxpr8rsLrf8n3AbmNQb0TX\n5s9fUAL+XKAK/vPnL2i6WRG1jcUkKrDpdIkdM6sln35ExNC2hnz6/wlca/visrwSOGDg8E6ybMZ4\nMnB4Z9KkkzK8E+PSeMyyuRh4P4Ck6cDjGc+P8W7WrFksWrSQgw5azEEHLU7Aj+eN2j39/pOoAGvY\ndBIVJJ1DdYfPU8Axtpd32E96+hERwzTcnn4mUYmI2IqNx+GdiIgYJxL0IyJaJEE/IqJFEvQjIlok\nQT8iokUS9CMiWiRBPyKiRUYin/5sSStLvvyTOmyfIekJSSvK69S6dUZERHdqJVyTNAE4B3g7cD/w\nC0mLbd8xoOiPbc+pU1dERNRXt6c/DbjL9irbzwAXA4d3KLfFT4tFRMToqRv0O+XK33VAGQP7lakS\nl0jau2adERHRpbr59LckWc5yYIrtpyUdDFwBTK1Zb0REdKFu0L8fmNJveQpVb/9Zttf1e3+VpHMl\n7WB77cCdZRKViIihNTqJiqSJwP8AbwMeAG4Eju5/IVfSZOBh25Y0DbjE9u4d9pUsmxERwzTcLJu1\nevq2N0j6ENADTADOt32HpH8s278CvBM4XtIG4GngqDp1RkRE95JPPyJiK5Z8+hERMagE/YiIFknQ\nj4hokQT9iIgWSdCPiGiRBP2IiBZJ0I+IaJEE/YiIFhn1SVRKmbPK9lsk7VO3zoiI6E6toN9vEpXZ\nwN7A0ZL2GlDmEGAP23sCxwHn1akzIiK6NxaTqMwBFgLYvgHYviRhiwF6enqYOfNIZs48kp6enqab\n06gciz45Fn1yLEaA7a5fVMnUvtpv+X3A2QPKXAns12/5GuCvOuzLbbZ06VJPmjTZcKHhQk+aNNlL\nly5tulmNyLHok2PRJ8eisxI7tzhuj8UkKrDpdInJrDbA/PkLWL9+HjAXgPXrq3WzZs1qtmENyLHo\nk2PRJ8diZIz6JCodyuxW1m0ik6hERAyt7iQqdYd3JgJ3A7sDLwJuBvYaUOYQYEl5Px34+SD7Gt1z\noHEup659ciz65Fj0ybHojGEO79TOp1/mvT2TvklUPjdgEhUk9d7h8xRwjO3lHfbjum3Z2vX09DB/\n/gIATjzxuFaftuZY9Mmx6JNjsanh5tPPJCoREVuxTKISERGDStCPiGiRBP2IiBZJ0I+IaJEE/YiI\nFknQj4hokQT9iIgWSdCPiGiRrnPvSNoB+C7wamAV8G7bj3cotwp4EvgT8Iztad3WGRER9dTp6Z8M\nLLM9FfhhWe7EwAzb+yTgR0Q0q07Qf3ZylPLzHUOU3eJHhCMiYvTUCfqTba8p79cAg82GZeAaSTdJ\nOrZGfRERUdOQY/qSlgE7d9j0qf4Lti1psGxp+9t+UNJOwDJJK21f111zIyKijiGDvu2DBtsmaY2k\nnW0/JOmVwMOD7OPB8vMRSYuo5tXtGPQziUpExNDqTqLSdWplSV8Afm97nqSTge1tnzygzDbABNvr\nJL0EuBo4w/bVHfaX1MoREcM0Zvn0yy2blwCvot8tm5J2oZos/VBJrwUuLx+ZCHzb9ucG2V+CfkTE\nMGUSlYiIFskkKhERMagE/YiIFknQj4hokQT9iIgWSdCPiGiRBP2IiBZJ0I+IaJEE/YiIFuk66Et6\nl6TbJP1J0r5DlJstaaWkOyWd1G19ERFRX52e/q3AEcBPBisgaQJwDjAb2Bs4WtJeNeqMGDM9PT3M\nnHkkM2ceSU9PT9PNiRgRXU+XaHslVI8AD2EacJftVaXsxcDhwB3d1hsxFnp6ejjiiLmsXz8PgOuv\nn8uiRQuZNWtWwy2LqGe0x/R3BVb3W76vrIsY1+bPX1AC/lygCv7z5y9oulkRtXU7iconbV+5Bfsf\nVga15NOPiBhaY/n0n92B9CPgRNvLO2ybDpxue3ZZPgXYaHteh7LJshnjxsDhnUmTTsrwToxLw82y\n2fWY/sB6B1l/E7CnpN2BB4D3AEePUJ0Ro2bWrFksWrTw2SGdE09MwI/nhzqTqBwBnAXsCDwBrLB9\ncP9JVEq5g4EzgQnA+ZlEJSJi5GQSlYiIFskkKhERMagE/YiIFknQj4hokQT9iIgWSdCPiGiRBP2I\niBZJ0I+IaJGxyKe/StKvJK2QdGO39UVERH110jD05tP/ymbKGZhhe22NuiIiYgSMdj79Xlv8tFhE\nRIyesRjTN3CNpJskHTsG9UVExCBGO58+wP62H5S0E7BM0krb1w23oRERUd+QQd/2QXUrsP1g+fmI\npEVUUyh2DPqZRCUiYmjjZRKVj9v+ZYdt2wATbK+T9BLgauAM21d3KJssmxERwzRmWTYlHSFpNTAd\n+IGkq8r6XST9oBTbGbhO0s3ADcD3OwX8iIgYG8mnHxGxFUs+/YiIGFSCfkREiyToR0S0SIJ+RESL\nJOhHRLRIgn5ERIsk6EdEtEiCfkREi9R5Ivc/JN0h6RZJl0v6s0HKzZa0UtKdkk7qvqkREVFXnZ7+\n1cDrbb8J+A1wysACkiYA5wCzgb2BoyXtVaPOVqiTTOn5JseiT45FnxyL7nUd9G0vs72xLN4A7Nah\n2DTgLturbD8DXAwc3m2dbZFf6D45Fn1yLPrkWHRvpMb0Pwgs6bB+V2B1v+X7yrqIiGhA7UlUJH0K\n+D/b3+lQLhnUIiLGkVpZNiV9ADgWeJvtP3bYPh043fbssnwKsNH2vA5l8w8iIqILw8my2fXE6JJm\nA/8KHNAp4Bc3AXtK2h14AHgPcHSngsNpdEREdKfOmP7ZwLZU896ukHQuPHcSFdsbgA8BPcDtwHdt\n31GzzRER0aVxM4lKRESMvsafyM3DWxVJUyT9SNJtkn4t6SNNt6lpkiaUs8grm25LkyRtL+nS8jDk\n7eVaWStJOqX8jdwq6TuSXtx0m8aKpK9LWiPp1n7rdpC0TNJvJF0tafvN7afRoJ+Ht57jGeBjtl9P\nNe/wP7f4WPQ6gWpYsO2no18GltjeC3gj0Moh0nJt8FhgX9tvACYARzXZpjF2AVWs7O9kYJntqcAP\ny/KQmu7p5+GtwvZDtm8u7/9A9Ye9S7Otao6k3YBDgK8Brb3IX9KbvNX216G6Tmb7iYab1ZQnqTpH\n20iaCGwD3N9sk8aO7euAxwasngMsLO8XAu/Y3H6aDvp5eKuD0qPZh+pJ57b6EtXdYRs3V/B57jXA\nI5IukLRc0lclbdN0o5pgey0wH7iX6m7Ax21f02yrGjfZ9pryfg0weXMfaDrot/20fROStgUuBU4o\nPf7WkXQY8LDtFbS4l19MBPYFzrW9L/AUW3AK/3wk6XXAR4Hdqc6Ct5X03kYbNY64uitnszG16aB/\nPzCl3/IUqt5+K0l6IXAZ8C3bVzTdngbtB8yRdA9wEXCgpG803Kam3AfcZ/sXZflSqn8CbfRm4Ke2\nf19uB7+c6nelzdZI2hlA0iuBhzf3gaaD/rMPb0l6EdXDW4sbblMjJAk4H7jd9plNt6dJtj9pe4rt\n11BdqPtv2+9vul1NsP0QsFrS1LLq7cBtDTapSSuB6ZImlb+Xt1Nd6G+zxcDc8n4usNnOYtdP5I4E\n2xsk9T68NQE4v8UPb+0PvA/4laQVZd0ptpc22Kbxou3DgB8Gvl06RncDxzTcnkbYvqWc8d1Eda1n\nObCg2VaNHUkXAQcAO0paDXwa+DxwiaR/AFYB797sfvJwVkREezQ9vBMREWMoQT8iokUS9CMiWiRB\nPyKiRRL0IyJaJEE/IqJFEvQjIlokQT8iokX+H3tYaHEByMwZAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x56751d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"rotated = x.dot(A)\n",
"plt.scatter(rotated[:, 0], rotated[:, 1])\n",
"plt.ylim([-2, 2])\n",
"plt.xlim([0, 10])\n",
"plt.title(\"Rotated Data\")\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"PCA performs operations similar to the one just illustrated. To iterate, PCA plots data points against axes that capture maximum variance. We will later see that these axes are just eigenvectors, and they can be found either through singular value decomposition (SVD) of the column-centered data matrix or through finding the eigenvectors of the covariance matrix. Assume $X \\in \\mathbb{R}^{nxd}$ is our data matrix and we have already found these eigenvectors contained as columns in the matrix called $U \\in \\mathbb{R}^{dxk}$. Let us assume $n$ = number of data points and $d$ = features. To plot the data against one eigenvector, we would pick a column $e$ from $U$ and perform the following operation: \n",
"$\\hat{X} = e^TX^T$, where $e^T \\in \\mathbb{R}^{1xd}$ and $\\hat{X} \\in \\mathbb{R}^{1xn}$. We can see that after this matrix multiplication, each data point is now represented by one feature instead of $k$ features, thus achieving dimensionality reduction. Now we show the proof that eigenvectors are the dimenions that maximize variance. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$X \\in \\mathbb{R}^{nxd}$ = items by features data matrix (centered)\n",
"\n",
"$e$ = dimension(vector) that data points are projected to\n",
"$$\\frac{1}{n}\\sum_{i=1}^{n}\\Big(\\sum_{j=1}^{d}x_{ij}e_j - \\mu\\Big)^2 = \\frac{1}{n}\\sum_{i=1}^{n}\\Big(\\sum_{j=1}^{d}x_{ij}e_j\\Big)^2\\\\\n",
"V = \\frac{1}{n}\\sum_{i=1}^{n}\\Big(\\sum_{j=1}^{d}x_{ij}e_j\\Big)^2 - \\lambda\\bigg(\\Big(\\sum_{k=1}^{d}e_j^2\\Big)-1\\bigg)\\\\\n",
"\\frac{\\partial V}{\\partial e_{a}} = \\frac{2}{n}\\sum_{i=1}^{n}\\Big(\\sum_{j=1}^{d}x_{ij}e_j\\Big)x_{ia} - 2\\lambda e_a = 0\\\\\n",
"2\\sum_{i=1}^{d}e_j\\Big(\\frac{1}{n}\\sum_{i=1}^{n}x_{ia}x_{ij}\\Big) = 2\\lambda e_a\\\\\n",
"\\begin{Bmatrix}\n",
"\\sum_{j=1}^{d}cov(1, \\textit{j})e_j = \\lambda e_1\\\\\n",
"\\vdots\\\\\n",
"\\sum_{j=1}^{d}cov(d, \\textit{j})e_j = \\lambda e_d\\\\\n",
"\\end{Bmatrix}\\\\\n",
"\\Sigma e = \\lambda e$$\n",
"\n",
"The first equation comes from the definition of variance of points plotted against a vector. Since the data is assumed to be centered by column(features), the mean of the projected points $\\mu$ is equal to zero. The second equation applies the method of Lagrange multiplers (constraint: $\\|e\\|_{2} = 1$) to solve for $e$ that would maximize variance. We can observe from the end result that $e$ must be an eigenvector and $\\lambda$ must be an eigenvalue from definition of eigenvector and eigenvalue. The last equation rewrites the result in matrix notation, where $\\Sigma$ is the feature covariance matrix. Let us verify the result in the last line through an example calculation: "
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Toy data:\n",
"[[ 0.5 0.51706651]\n",
" [ 1. 1.07165469]\n",
" [ 2. 1.73323622]\n",
" [ 3. 2.87334892]\n",
" [ 4. 4.19179732]]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPkAAAEKCAYAAADQPvOTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADNVJREFUeJzt3X+s3XV9x/HnCyrjIr+2uABCDYuZi3MmEB0uA/S4rbdM\nWbXpH85FafxDFrNFtl1J4/7h+sd0/NGwJfvDEHEpsuGMTQnMZKcVOEo1qWLaCBSNWyBDRcSBIKPL\nYH3vj3vAjtF7zuWec7/3fPp8JM09597vOX23vc9+f5zvPd9UFZLadVLXA0iaLiOXGmfkUuOMXGqc\nkUuNM3KpcUYuNc7IG5Lk4STPJnk6yZNJvpbkj5NkjMdemORoEr8nGuM/aFsKuLKqzgReB/w1sAO4\naQXPMfI/BM0WI29UVf2squ4A3gdsT/KmJO9OcjDJU0n+Pcl1xzzkq8OPP03ysyRvS/L6JHcl+UmS\nx5PckuSstf/TaDWMvHFV9U3g+8DlwDPAB6rqLODdwEeSvGe46OXDj2dV1RlVdWB4/6+A84A3AhuB\nxbWaXZNh5CeGHwK/WFVfqaoHAKrqPuDzwDuGy/y/zfSq+requrOqnquqnwA3HLO8ZsSGrgfQmjgf\neCLJ21jaT38TcArwC8AXjvegJOcAfwtcBpzB0krhialPq4lyTd64JL/JUuRfA/4RuA24oKrOBj7N\nz78HXu7HET8J/A/wG8NN/A/i98zM8R+sPQFIcmaSK4Fbgc9V1f3A6cCTVfXfSS4B/oifx/04cBR4\n/THPdTrwn8DTSc4Hrl2jP4MmKP48eTuSPAScAzzPUrAPALcAn66qSrIN2An8EvAV4CHg7Kq6avj4\nTwAfYWk37gqWDtTdDPwa8L3hc/1ZVb1uLf9cWp2xIk/yMPA0S5tuz1XVJVOeS9KEjHvgrYBeVXnQ\nRZoxK9kn90woaQaNG3kBX05yb5IPT3MgSZM17ub6pVX1aJJfBvYl+U5V3TPNwSRNxliRV9Wjw4+P\nJ9kDXALcA5DEw/NSh6pq2V3pkZvrSU5Lcsbw9quBeeC+l/wmM/Xruuuu63yG1meetXlndeZxjLMm\nPwfYM/yR5A3AP1TV3rGeXVLnRkZeVQ8BF63BLJKm4IQ8rbXX63U9worN2syzNi/M5szjWPVprUlq\ntc8h6ZVJQq32wJuk2WbkUuOMXGqckUuNM3KpcUYuNc7IpcYZudQ4I5caZ+RS44xcapyRS40zcqlx\nRi41zsilxhm51Dgjlxpn5FLjjFxqnJFLjTNyqXFGLjXOyKXGGbnUOCOXGmfkUuOMXGqckeuE1+/3\nmZ/fxvz8Nvr9ftfjTJwXPNQJrd/vs3Xrdo4cuR6Aubkd7Nmzi82bN3c82XjGueChkeuENj+/jX37\ntgDbh5/ZxaZNt7N37+4uxxqbVzWVxIauB5C6tLBwNfv3b+fIkaX7c3M7WFjY1e1QE+bmuk54/X6f\nnTtvBJain5X9cXCfXGrexPbJk5yc5GCSOyYzmqS1Mu6Bt2uAw4CrbGnGjIw8yQXAu4DPAMtuFkha\nf8ZZk98AXAscnfIskqZg2ZfQklwJ/LiqDibpHW+5xcXFF2/3ej16veMuKmkVBoMBg8FgRY9Z9uh6\nkk8CHwSeB04FzgR2V9VVxyzj0XWpIxN9CS3JO4CPVdUfvOTzRi51ZBqntVqzNGM8GUaaYf6AiiQj\nl1pn5FLjjFxqnJFLjTNyqXFGLjXOyKXGGbnUOCOXGmfkUuOMXGqckUuNM3KpcUYuNc7IpcYZudQ4\nI5caZ+RS44xcapyRS40zcqlxRi41zsilxhm51Dgjlxpn5FLjjFxqnJFLjTNyqXFGLjXOyKXGGbnU\nOCOXGmfkUuNGRp7k1CQHkhxKcjjJp9ZiMEmTsWHUAlX1X0neWVXPJtkA7E9yWVXtX4P5JK3SWJvr\nVfXs8OYpwMnAE1ObSNJEjRV5kpOSHAIeA+6uqsPTHUvSpIy7Jj9aVRcBFwBvT9Kb6lSSJmbkPvmx\nquqpJF8C3goMXvj84uLii8v0ej16vd5kppP0fwwGAwaDwYoek6pafoHkNcDzVfXTJHNAH/hEVd05\n/HqNeg5J05GEqspyy4yzJj8P2JXkJJY27z/3QuCS1r+Ra/KRT+CaXMfo9/vs3HkjAAsLV7N58+aO\nJ2rbOGtyI9fE9Pt9tm7dzpEj1wMwN7eDPXt2GfoUGbnW1Pz8Nvbt2wJsH35mF5s23c7evbu7HKtp\n40TuuetS41b0Epq0nIWFq9m/fztHjizdn5vbwcLCrm6HkpvrmiwPvK0t98mlxrlPLsnIpdYZudQ4\nI5caZ+RS44xcapyRS40zcqlxRi41zsilxhm51Dgjlxpn5FLjjFxqnJFLjTNyqXFGLjXOyKXGGbnU\nOCOXGmfkUuOMXGqckUuNM3KpcUYuNc7IpcYZudQ4I5caZ+RS44xcatzIyJNsTHJ3kgeS3J/ko2sx\nmKTJGHl98iTnAudW1aEkpwPfAt5bVQ8Ov+71yaWOTOT65FX1o6o6NLz9DPAg8NrJjChp2la0T57k\nQuBi4MA0hpE0eWNHPtxU/yJwzXCNLmkGbBhnoSSvAnYDt1TVbS/9+uLi4ou3e70evV5vQuNJOtZg\nMGAwGKzoMeMceAuwC/iPqvrzl/m6B96kjoxz4G2cyC8Dvgp8G3hh4Y9X1b8Mv27kUkcmEvkYv4mR\nSx2ZyEtokmabkUuNM3KpcUYuNc7I17l+v8/8/Dbm57fR7/e7HkczyKPr61i/32fr1u0cOXI9AHNz\nO9izZxebN2/ueDKtF76ENuPm57exb98WYPvwM7vYtOl29u7d3eVYWkd8CU3SeOeuqxsLC1ezf/92\njhxZuj83t4OFhV3dDqWZ4+b6Otfv99m580ZgKXr3x3Us98mlxrlPLsnIpdYZudQ4I5caZ+RS44xc\napyRS40zcqlxRi41zsilxhm51Dgjlxpn5FLjjFxqnJFLjTNyqXFGLjXOyKXGGbnUOCOXGmfkUuOM\nXGqckUuNO6Ei9wqhOhGdMBdX8AqhatFELq6Q5LNJHkty3+RGW3s7d944DHw7sBT7C5cfklo2zub6\n3wNXTHsQSdMx8qqmVXVPkgunP8p0eYVQnajG2icfRn5HVb35Zb42E/vk4BVC1Z5x9skncn3yxcXF\nF2/3ej16vd4knnbiNm/ebNiaaYPBgMFgsKLHnFBrcqk1XrpY0lgvod0KfB14Q5JHknxo+mNJmpQT\n5mQYqUVurksycql1Ri41zsilxhm51Dgjlxpn5FLjjFxqnJFLjTNyqXFGLjXOyKXGGbnUOCOXGrcu\nI/ciCNLkrLufJ/ciCNL4xvl58nUX+fz8Nvbt28LSRRAAdrFp0+3s3bt7Yr+H1ArfNELSZN6SeZK8\nCII0Wetucx28CII0rpncJ5c0PvfJJRm51Dojlxpn5FLjjFxqnJFLjTNyqXFGLjXOyKXGGbnUOCOX\nGmfkUuOMXGqckUuNGxl5kiuSfCfJ95LsWIuhJE3OspEnORn4O+AK4NeB9yd541oMNk2DwaDrEVZs\n1maetXlhNmcex6g1+SXAv1bVw1X1HPB54D3TH2u6ZvEfc9ZmnrV5YTZnHseoyM8HHjnm/veHn5M0\nI0ZF7vs6STNu2fd4S/JbwGJVXTG8/3HgaFVdf8wy/kcgdWhVb+SYZAPwXeB3gR8C3wDeX1UPTnJI\nSdOz7PuuV9XzSf4U6AMnAzcZuDRbVv2WzJLWt1Wd8TZrJ8ok+WySx5Lc1/Us40iyMcndSR5Icn+S\nj3Y90yhJTk1yIMmhJIeTfKrrmcaR5OQkB5Pc0fUs40jycJJvD2f+xrLLvtI1+fBEme8Cvwf8APgm\n63x/PcnlwDPAzVX15q7nGSXJucC5VXUoyenAt4D3rue/Y4Akp1XVs8NjOvuBj1XV/q7nWk6SvwDe\nApxRVVu6nmeUJA8Bb6mqJ0Ytu5o1+cydKFNV9wBPdj3HuKrqR1V1aHj7GeBB4LXdTjVaVT07vHkK\nS8dyRn4jdinJBcC7gM8Ayx6pXmfGmnU1kXuizBpKciFwMXCg20lGS3JSkkPAY8DdVXW465lGuAG4\nFjja9SArUMCXk9yb5MPLLbiayD1it0aGm+pfBK4ZrtHXtao6WlUXARcAb0/S63ik40pyJfDjqjrI\nbK3FL62qi4HfB/5kuCv6slYT+Q+Ajcfc38jS2lwTlORVwG7glqq6ret5VqKqngK+BLy161mW8dvA\nluE+7q3A7yS5ueOZRqqqR4cfHwf2sLT7/LJWE/m9wK8muTDJKcD7gNtX8Xx6iSQBbgIOV9XfdD3P\nOJK8JsnZw9tzwCbgYLdTHV9V/WVVbayqXwH+ELirqq7qeq7lJDktyRnD268G5oHjvmL0iiOvqueB\nF06UOQz80wwc9b0V+DrwhiSPJPlQ1zONcCnwAeCdw5dKDia5ouuhRjgPuGu4T34AuKOq7ux4ppWY\nhd3Qc4B7jvk7/ueq2nu8hT0ZRmqcb/8kNc7IpcYZudQ4I5caZ+RS44xcapyRS40zcqlx/wtjpgRW\nM0UhsAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x58ad650>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import random\n",
"random.seed(1)\n",
"noise = np.random.normal(0, 0.2, 5)\n",
"x = np.array([[0.5, 0.5 + noise[0]], [1, 1 + noise[1]], [2, 2 + noise[2]], [3, 3 + noise[3]], [4, 4 + noise[4]]])\n",
"print \"Toy data:\"\n",
"print x\n",
"plt.scatter(x[:, 0], x[:, 1])\n",
"plt.ylim([0, 5])\n",
"plt.xlim([0, 5])\n",
"plt.title(\"Data\")\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Feature covariance matrix: \n",
"[[ 1.64 1.67419575]\n",
" [ 1.67419575 1.73376473]]\n"
]
}
],
"source": [
"x_centered = x - x.mean(axis=0)\n",
"XTX = x_centered.T.dot(x_centered)\n",
"feature_cov = XTX/5\n",
"print \"Feature covariance matrix: \"\n",
"print feature_cov"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"U, S, V = np.linalg.svd(feature_cov)\n",
"#U = eigenvectors to project points to produce reduced item feature vector\n",
"#S = eigenvalues associated with eigenvectors\n",
"left = feature_cov.dot(U[:, 0])\n",
"right = S[0]*U[:, 0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$\\Sigma e$ is: "
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([-2.34359916, -2.41014547])"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"left "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$\\lambda e$ is: "
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([-2.34359916, -2.41014547])"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"right"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Which verifies the equivalence of the result in the last line. So far we have talked about the role of eigenvectors in PCA, but not the eigenvalues. Let us talk about the meaning of eigenvalues $\\lambda$. In PCA, the eigenvalue $\\lambda$ of an eigenvector is the variance of the projected points against that eigenvector. Let us proof this statement by starting with the variance of projected points: \n",
"$$\\sigma^2 = \\frac{1}{n}\\sum_{i=1}^{n}\\Big(\\sum_{j=1}^{d}x_{ij}e_{j} - \\mu \\Big)^2 = \\frac{1}{n}\\sum_{i=1}^{n}\\Big(\\sum_{j=1}^{d}x_{ij}e_{j}\\Big)^2$$\n",
"$$=\\frac{1}{n}\\sum_{i=1}^{n}\\Big(\\sum_{j=1}^{d}x_{ij}e_{j}\\Big)(\\sum_{a=1}^{d}x_{ia}e_{a}\\Big)$$\n",
"$$=\\sum_{a=1}^{d}\\sum_{j=1}^{d}\\Big(\\frac{1}{n}\\sum_{i=1}^{n}x_{ia}x_{ij}\\Big)e_{j}e_{a}$$\n",
"$$=\\sum_{a=1}^{d}\\Big(\\sum_{j=1}^{d}cov(a, j)e_{j}\\Big)e_{a}$$\n",
"$$=\\sum_{a=1}^{d}\\Big(\\lambda e_{a}\\Big)e_{a}$$\n",
"$$=\\lambda \\|e\\|^2 = \\lambda$$\n",
"Let us verify this via an example calculation: "
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Variance of reduced item vector: \n",
"3.36173440585\n",
"Eigenvalue of eigenvector projected to: \n",
"3.36173440585\n"
]
}
],
"source": [
"reduced_item = U[:, 0].T.dot(x_centered.T)\n",
"variance = reduced_item.dot(reduced_item.T)/5\n",
"print \"Variance of reduced item vector: \"\n",
"print variance\n",
"print \"Eigenvalue of eigenvector projected to: \"\n",
"print S[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In PCA terminology, the first principle component refers to the dimension capturing the maximum variance, the second principle component refers to the dimension capturing the second most variance, and so on. In PCA, one would project the data onto dimensions that capture maximum variance to capture the most important structure of the data. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"# How to perform PCA\n",
"Let us now talk about how to perform PCA, and there are many ways one can perform PCA. The first step in PCA is to compute the eigenvectors and eigenvalues of your data matrix $X$. There are two methods of computing the eigenvectors and eigenvalues. In the first method, one could directly perform singular value decomposition (SVD) on the data matrix $X$.\n",
"\n",
"$$X = USV^T$$\n",
"\n",
"$$U \\in \\mathbb{R}^{nxk}$$\n",
"$$S \\in \\mathbb{R}^{kxk}$$\n",
"$$V^T \\in \\mathbb{R}^{kxd}$$\n",
"\n",
"$U$ and $V$ are orthonormal matrices containing eigenvectors, and $S$ is a diagonal matrix containing eigenvalues, which are ordered in decreasing value as you transverse down the diagonal from top left to bottom right. Each column in $U$ is an eigenvector, and it is associated with an eigenvalue in $S$. Specifically, the eigenvector $U_1$ would correspond to the eigenvalue $S_{11}$. In contrast, each row in $V^T$ is an eigenvector, and it too is associated with an eigenvalue. For example, the eigenvector $V_1$ would correspond to the eigenvalue $S_{11}$. \n",
"\n",
"In the second method, one would compute a rough covariance matrices of the data matrix $X$ first, then perform SVD on each covariance matrix to solve for $U$ and $V$. I used the term \"rough\" to describe the covariance matrices because in the equations below, I do not divide the $XX^T$ by _d_ or $X^TX$ by _n_, simply because this ensures that $S_1 = S_2 = S^2$. However, if I were to calculate the proper covariance matrices, the eigenvectors I get from their SVD would still be the same. Concretely: \n",
"\n",
"$$Cov1 = XX^T\\\\\n",
"Cov2 = X^{T}X\\\\\n",
"U_1S_1V_{1}^T = Cov1\\\\\n",
"U_2S_2V_{2}^T = Cov2\\\\$$\n",
"\n",
"We can show that $U_1 = U$ and $U_2 = V$: \n",
"\n",
"$$XX^T = USV^TVSU^T = US^2U^T\\\\ \n",
"X^TX = VSU^TUSV^T = VS^2V^T\\\\$$\n",
"\n",
"Which means that: \n",
"$$S_1 = S_2\\\\\n",
"S = S_1S^{-1}\\\\$$\n",
"\n",
"\n",
"Let us run some actual code to demonstrate their equivalence: "
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Method 1 U\n",
"[[-0.54492313 -0.24183089]\n",
" [-0.36292236 -0.3566404 ]\n",
" [-0.07719133 0.68601659]\n",
" [ 0.29222002 0.36846187]\n",
" [ 0.69281679 -0.45600717]]\n",
"Method 2 U\n",
"[[-0.54492313 -0.24183089]\n",
" [-0.36292236 -0.3566404 ]\n",
" [-0.07719133 0.68601659]\n",
" [ 0.29222002 0.36846187]\n",
" [ 0.69281679 -0.45600717]]\n",
"Method 1 V\n",
"[[ 0.69713989 0.71693512]\n",
" [ 0.71693512 -0.69713989]]\n",
"Method 2 V\n",
"[[-0.69713989 -0.71693512]\n",
" [-0.71693512 0.69713989]]\n",
"Method 1 S\n",
"[[ 4.09983805 0. ]\n",
" [ 0. 0.24525827]]\n",
"Method 2 S\n",
"[[ 4.09983805 0. ]\n",
" [ 0. 0.24525827]]\n"
]
}
],
"source": [
"#Method 1\n",
"U, S, V = np.linalg.svd(x_centered, full_matrices=0)\n",
"S = np.diag(S)\n",
"Sinv = np.linalg.inv(S)\n",
"\n",
"#Method 2\n",
"XXT = np.dot(x_centered, x_centered.T)\n",
"U1, S1, V1 = np.linalg.svd(XXT)\n",
"XTX = np.dot(x_centered.T, x_centered)\n",
"U2, S2, V2 = np.linalg.svd(XTX)\n",
"S2 = np.diag(S2)\n",
"print \"Method 1 U\"\n",
"print U\n",
"print \"Method 2 U\"\n",
"print U1[:, :2]\n",
"print \"Method 1 V\"\n",
"print V\n",
"print \"Method 2 V\"\n",
"print U2\n",
"print \"Method 1 S\"\n",
"print S\n",
"print \"Method 2 S\"\n",
"print S2.dot(Sinv)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once we have our eigenvectors and eigenvalues, the next step will be to project our data onto the eigenvectors. There are two different ways of projecting the data to $\\textit{k}$ principle components to arrive at the following result: \n",
"$$features = \\mathbb{R}^{kxd}\\\\\n",
"items = \\mathbb{R}^{kxn}$$\n",
"1. Multiplying the appropriate eigenvector-containing matrices by the data: \n",
"$$features = \\hat{U}^TX\\\\\n",
"items = \\hat{V}^TX^T\\\\$$\n",
"2. Multiplying $S$ by $\\hat{V}^T$ or $S$ by $\\hat{U}^T$:\n",
"$$U^TX = U^TUSV^T = SV^T\\\\\n",
"features = S\\hat{V}^T\\\\\n",
"V^TX^T = V^TVSU^T = SU^T\\\\\n",
"items = S\\hat{U}^T\\\\$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The hat symbol above a matrices indicates matrices with the top $k$ eigenvectors chosen, thus $\\hat{U} \\subseteq U$. "
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Method 1 feature vector: \n",
"[[ 2.85816065 2.9393179 ]]\n",
"Method 2 feature vector: \n",
"[[ 2.85816065 2.9393179 ]]\n",
"Method 1 item vector: \n",
"[[-2.23409657 -1.48792288 -0.31647196 1.19805478 2.84043664]]\n",
"Method 2 item vector: \n",
"[[-2.23409657 -1.48792288 -0.31647196 1.19805478 2.84043664]]\n"
]
}
],
"source": [
"#Choose only one eigenvector (represent data in one dimension)\n",
"\n",
"Ureduced = np.reshape(U[:, 0], (5, 1))\n",
"Vreduced = np.reshape(V.T[:, 0].T, (1, 2))\n",
"Sreduced = np.array(S[0, 0])\n",
"\n",
"#Method 1\n",
"features1 = (Ureduced.T).dot(x_centered)\n",
"items1 = Vreduced.dot(x_centered.T)\n",
"\n",
"#Method 2\n",
"features2 = Sreduced.dot(Vreduced)\n",
"items2 = Sreduced.dot(Ureduced.T)\n",
"\n",
"print \"Method 1 feature vector: \"\n",
"print features1\n",
"print \"Method 2 feature vector: \"\n",
"print features2\n",
"print \"Method 1 item vector: \"\n",
"print items1\n",
"print \"Method 2 item vector: \"\n",
"print items2"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"Now that we have our reduced feature and item vectors, we can reconstruct our original data matrix $M$.\n",
"\n",
"$$M = items^TS^{-1}features = USS^{-1}SV^T$$\n",
"\n",
"Let us see how closely the reconstructed matrix resembles the original data matrix: "
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Reconstructed matrix: \n",
"[[ 0.54252216 0.47571843]\n",
" [ 1.0627096 1.01067656]\n",
" [ 1.87937477 1.85053087]\n",
" [ 2.93521178 2.93634828]\n",
" [ 4.08018169 4.11382953]]\n",
"Orignal matrix: \n",
"[[ 0.5 0.51706651]\n",
" [ 1. 1.07165469]\n",
" [ 2. 1.73323622]\n",
" [ 3. 2.87334892]\n",
" [ 4. 4.19179732]]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPkAAAEKCAYAAADQPvOTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FFXWx/HvCWsCCQGCBMI2MjJuKKMOKipEBEQQFBkV\nEVxQcVDQGUFRRwUR3EYGfd1ABhgURUUEAZUAgaCi4oiiCAqC7KASlrBkI8l5/7ghJASSDnS6ujvn\n8zx50p2urj6dzi9169atW6KqGGPCV4TXBRhjypeF3JgwZyE3JsxZyI0JcxZyY8KchdyYMGchNybM\nWcjDiIhsEJF0EdkrIrtFZImI3Cki4sNzm4lInojY30SYsQ80vChwparGAE2Ap4GhwIQyrKPUfwgm\ntFjIw5Sq7lPV2cD1wM0icoaIdBWRb0UkTUQ2iciwQk/5JP/7HhHZJyLni0hzEVkoIqkiskNEpohI\nrcC/G3MiLORhTlX/B2wBLgH2A31UtRbQFRggIlflL3pJ/vdaqhqtqkvz748CGgCnAY2B4YGq3fiH\nhbxi2AbUVtXFqroSQFVXAG8D7fKXKdZMV9V1qpqsqgdVNRUYU2h5EyIqe12ACYgEYJeInI/bTz8D\nqApUA9491pNEpD7wAnAxEI3bKOwq92qNX9mWPMyJyF9wIV8CvAXMBBqpaiwwlsN/A0c7HfFJIBc4\nM7+J3xf7mwk59oGFHwEQkRgRuRKYCryhqj8ANYHdqpotIq2B3hwO9w4gD2heaF01gQPAXhFJAO4P\n0HswfiR2Pnn4EJH1QH0gBxfYlcAUYKyqqoj0BEYDdYDFwHogVlVvyn/+48AA3G5cZ1xH3evAn4Cf\n89f1d1VtEsj3ZU6MTyEXkQ3AXlzT7aCqti7nuowxfuJrx5sCiapqnS7GhJiy7JPbSChjQpCvIVdg\ngYh8LSJ3lGdBxhj/8rW5fpGqbheResB8EflJVT8tz8KMMf7hU8hVdXv+9x0iMgNoDXwKICLWPW+M\nh1S1xF3pUpvrIhIlItH5t2sAnYAVR7xISH0NGzbM8xrCveZQqzdUa/aFL1vy+sCM/FOSKwNvquo8\nn9ZujPFcqSFX1fVAqwDUYowpBxVyWGtiYqLXJZRZqNUcavVCaNbsixMe1ioieqLrMMYcHxFBS+l4\nK7dTTX2YVsyEOfvnHxzK9Xxy+5ArLvsnHzwq5D65MRWJhdwYL+TmQmpqQF7KQm5MoM2fDwHsybeQ\nl8FTTz3FHXf4dn5OWZYtTUREBL/88otf1mU8tHIldOkCnTrBZZdBXFxgXtcPw+r0aI7182AyadIk\nPfPMMzUqKkrj4+N1wIABumfPHq/LKkZEdN26dV6XUSah8PkHzPbtqv37q0ZEqIJqnTqqaWl+WXX+\n77nEjAbnljwjA15+GR57DBYuLJeXGD16NA8++CCjR49m7969fPnll2zcuJGOHTty8ODBYsvn5uaW\nSx0mjOXkwKhRcMop8NprkJfnfv7ggxATE7g6SvsvUNoXx7Ml//ln1TFjVF99VXXnzqKPZWSotmyp\nGhmpKqIaFaX68stl/xdXgrS0NK1Zs6ZOmzatyM/379+v9erV04kTJ+qwYcO0Z8+e2qdPH42JidH/\n/Oc/OmzYMO3Tp0/B8pMnT9YmTZpo3bp19YknntCmTZtqcnKyqmqRZdevX68iUrB8XFycjho1qmA9\nS5cu1QsuuEBjY2O1QYMGOnDgQM3Ozi543LbkIWzzZtV27dwWHFTj41UPHPDb6gnKLflXX0GrVjB0\nKNx3H5xxBvz+++HHZ86E9evd1lwV0tPh/vvd7UN27nT7NTVqQOPGriOjDD7//HMyMzO55pprivy8\nRo0adOnShfnz5yMizJo1i2uvvZa0tDRuvPHGIsd+V61axd13383UqVPZvn07aWlpbNu2reDxox0n\nXrJkCWvWrCE5OZkRI0awevVqACpXrswLL7zAzp07+eKLL0hOTuaVV14p03syQWrOHFi8GE46yd1/\n5BGIigpoCYEP+aBBcOAAZGe7IKemwr/+dfjxvXsPN2sOycoq+rOrr4aUFPcPYMsWd3/NGp9LSE1N\nJS4ujoiI4m+/QYMGpOYf2mjTpg3du3cHoHr16kUG97z33nt0796dNm3aUKVKFUaMGFEk2IWXPWTY\nsGFUq1aNs846i7PPPpvly5cDcM4559C6dWsiIiJo2rQp/fv3Z/HixT6/HxOkJk6EAQOgeXNYtgza\ntQM/dcaWReCvoHLkscGcHPjtt8P3L7206ONVq8LFF0OlSoeX//zz4v8IFi+GFi18KiEuLo7U1FTy\n8vKKBX3btm3E5fd6NmrU6Jjr2LZtW5HHIyMjqVu3bomvGx8fX3A7KiqKAwcOALBmzRruu+8+li1b\nRnp6Ojk5OZx33nk+vRcTpN58E26/HZo2df1KjRrBjBnu7znAAr8l794dIiMP34+KgquuOnz/lFNc\nE+fkk13nxOWXw/Tphx+vVAmqVSu6zogIiI31uYQLL7yQatWqMb3weoH9+/czd+5cOnToAJQ8NLNh\nw4Zs2bKl4H5GRgY7d+70uYbCBgwYwOmnn87atWtJS0tj1KhR5B35T8yEjmnT4KabICHBBbxJ/jT1\ntWt7Uk7gQ/7MM3D99S7oMTHw+OPQs2fRZS69FNatg7Q0mDWraIBFYMwY98+hUiX3/U9/KvqPohS1\natVi2LBhDBo0iKSkJA4ePMiGDRu47rrraNy4MX369Cl13H3Pnj2ZPXs2X3zxBdnZ2QwfPvy4x+rv\n37+f6OhooqKi+Omnn3j11VePaz0mCHzwAfTu7fbBFy50GyuPBT7kVavCpElufzotDYYMKfs67rwT\n5s51/yBeeAGWLClzM+j+++/nySefZMiQIdSqVYsLLriApk2bkpycTNWqVRGRYlvywj8744wzePHF\nF+nVqxcNGzYkOjqak046iWr5rYwjn19Sq+C5557jrbfeIiYmhv79+9OrVy+fn2uCyEcfwbXXui12\ncrJrlQaBcjufPP881xNadyjZv38/tWvXZu3atTRt2tTrcjxX0T5/5s+Hbt3cEZ9Fi+CsswLysr6c\nTx6cg2FCxOzZs0lPT+fAgQMMGTKEs846ywJeES1e7HYXq1d3YQ9QwH1lIT8Bs2bNIiEhgYSEBNat\nW8fbb7/tdUkm0JYsga5doXJlSEqCc87xuqJirLluykWF+Py/+go6dHCHc+fOdYd6A8zT6Z+MCWvf\nfOMO7x486DrcPAi4ryzkxpTVihXQsaM7QjR7dvEBXEHGQm5MWfz4ozsXfN8+N4KtUyevKyqVhdwY\nX/38swv4rl1uVFvXrl5X5BMLuTG+WL8e2rd351lMnQo9enhdkc/sEJo5bomJiUyYMMHrMsrfpk1u\nv3vrVpg8Ga67zuuKyqRChrxZs2ZERUURHR1NfHw8ffv2Ze/evV6XVUx5zu22YcMGIiIiTuhEmKMN\n/Q0727a5LfjGjfCf/0CfPl5XVGZBGfLynv1JRJgzZw779u3ju+++Y8WKFYwcOdL/L+QHJR1rzsnJ\nKdf1V3i//eb2wdetg1degX79vK7ouHgS8rVr4fnnYexY14dRWGYmnH++mwxm5Eg3HLg8J0mpX78+\nnTp1YuXKlQB8+eWXtGnThtq1a9OqVasikzfs2rWLW2+9lYSEBOrUqUOPQvtl48eP55RTTqFu3bpc\nddVVbN++veCxiIgIxo0bR4sWLahduzYDBw4seGzt2rW0a9eO2NhY6tWrxw033ABA27ZtATj77LOJ\njo5m2rRppKSk0KhRI5599lkaNGhAv379mDx5MpdcckmR91S4BZCRkcHgwYNp1qwZsbGxtG3blszM\nzIL1x8bGEh0dzdKlSwGYOHEip59+OnXq1KFz585s2rSpYL3z58/n1FNPJTY2lkGDBpXpGtlBLSnJ\n9ZJ36uRug5v3oEMH+Okn98c6YIC3NZ6I0uaHKu2LMs7xtnSpao0aqlWrumnc4uNVf/vt8ONTp6rW\nrHl4Sixw07zl5R1eJjVVtWNH9/NGjVTnzStlIqwjNGvWTBcsWKCqqps3b9aWLVvq448/rlu2bNG6\ndevqxx9/rKqq8+fP17p162pqaqqqqnbp0kV79eqle/bs0YMHD+onn3yiqqrJyckaFxen3377rWZl\nZemgQYO0bdu2Ba8nItqtWzdNS0vTTZs2ab169TQpKUlVVXv16qVPPvmkqqpmZWXpkiVLijyv8Nxu\nixYt0sqVK+uDDz6o2dnZmpGRoZMmTdKLL764yPsr/Ly77rpLL730Ut22bZvm5ubqF198oVlZWbph\nwwYVEc3NzS143syZM/WPf/yj/vTTT5qbm6sjR47UNm3aqKrqjh07NDo6WqdPn645OTk6ZswYrVy5\nsk6YMOGov+Njff5BZ+5c94d46I8tMlJ12jTVVq3c/Wee8brCEuHDHG8BD3nr1kUDXLmy6pAhhx8f\nN86Ft/AylSqp5uQcXubii1WrVCn6T2D1at9/MU2bNtWaNWtqdHS0ioheffXVmpOTo08//bT27du3\nyLKXX365Tp48Wbdt26YRERFHnbK5X79+OnTo0IL7+/fv1ypVqujGjRtV1YWucHivu+46fSb/j+em\nm27S/v3765YtW4qt92ghr1q1qmZlZRX8rKSQ5+bmamRkpH7//ffF1n1ocsnCIe/cuXOR0Obm5mpU\nVJRu3LhRJ0+erBdeeGGRdTRq1Cj0Q96xY9E/NlCNiXHfR4zwurpS+RLygDfXj2f2p3btis/+dOSs\nyWWZEk1E+OCDD9i7dy8pKSksXLiQZcuWsXHjRqZNm0bt2rULvpYsWcKvv/7K5s2bqVOnDrVq1Sq2\nvu3btxc5+6xGjRrUrVuXrVu3FvzsyKmf9u3bB8Czzz6LqtK6dWvOPPNMJk2aVGLt9erVo6qP586n\npqaSmZlJ8+bNfVp+48aN3HvvvQXv/dB0Vlu3bmX79u3FpsNq3LixT+sNOXv38tYfHqbTJ48UtN5D\nWcBDHgSzPxXRtm1bBg0axNChQ2nSpAl9+/Zl9+7dBV/79u3jgQceoHHjxuzatYu0tLRi62jYsCEb\nNmwouH/gwAF27txJQkJCqa9fv359XnvtNbZu3cq4ceO46667SuxRP7I3u0aNGqSnpxfc//XXXwtu\nx8XFUb16ddauXVvqegCaNGnCa6+9VuT9HzhwgAsvvJAGDRqwefPmgmVVtcj9kDV4cNE/SOD5iH9w\n4/qRzF8g9OhB6Ae9tE19aV+UsbmelaV6yy1u1ycmRvVf/yp7E2XsWNdEr1TJfT/3XLdeXzVr1qxg\nfnRVt78ZFRWln376qcbHx2tSUpLm5ORoRkaGLlq0qKAp3bVrV+3du7fu3r1bs7OzdfHixaqqumDB\nAq1Xr54uX75cMzMz9Z577tFLLrmkYP1HNrtvvvlmfeSRR1RV9d1339XNmzerquoPP/ygkZGRun79\nelVVjY+P13mFOhwWLVqkjRo1KvJeVq9erdWqVdPly5drRkaG3nnnnUVe7+6779bLLrtMt23bpjk5\nOfr5559rVlaWHjhwQCtVqqRr1qwpWNeMGTP0zDPP1JUrV6qq6p49e/Tdd98t+B1FR0fr+++/rwcP\nHtTnn38+PPbJVVU/+MBd1QR0Rr3bFfKKtN47dvS6wGMjGPfJ/eWTT1RHjlQdP141M7Nszz0y5Kqq\nAwYM0B49euhXX32l7dq10zp16mi9evX0yiuv1E2bNqmq6q5du/Tmm2/W+vXra+3atbVnz54Fzx87\ndqw2b95c69Spo926ddOtW7cWPBYREVEk5Lfccos++uijqqr6wAMPaEJCgtasWVObN2+u48ePL7LO\nBg0aaGxsrE6bNk1TUlK0cePGxd7PqFGjNC4uTps0aaJTpkwp8noZGRn697//XRMSErRWrVrarl07\nzcz/hT322GNar149jY2N1aVLl6qq6htvvKEtW7bUmJgYbdy4sd52220FrzN37lxt0aKF1qpVSwcO\nHKiJiYmhH/LMTNXOnV0U+vfXjh3yiu2ih3rIfTqfXEQqAV8DW1S12xGP6dHWUSHOJzbHFBKff3a2\nm5Nt1iy45RaYMIGk+RH06OHGaoBryc+Y4XYbg5Ev55P7GvL7gHOBaFXtfsRjFnJTTNB//jk50KuX\n6/Dp3Rtef72gdzcpCUaPdosNHhy8AQc/hVxEGgH/BUYB99mW3PgiqD//3Fzo29edaPLXv7rvlUPz\nXC1/TeQ4BrgfsNn+TejLy3NXNpk61R3qeeutkA24r0p8dyJyJfC7qn4rIonHWm748OEFtxMTE0lM\nPOaixnhH1Q1P/e9/4Yor4N13oUoVr6sqk5SUFFJSUsr0nBKb6yLyJNAXyAGqAzHAdFW9qdAy1lw3\nxQTd568K99wDL73kxqTPmlXs+Hgo8lvHW/7K2gFDbJ/c+CKoPn9Vd8bT6NHQti18/HHALx9cXspj\nttYyfWphf66xCQ2PPuoCfuGFbjhlmATcV+U277oxQeGJJ9zEBH/5i7u6yVHOPQhlfm2ul/AiFnIT\nnJ59FoYOhVat3OwjHl06uDzZtdBMxfX88y7gZ5zhtuBhGHBfWchN+Hn1VfjHP9x165OTIS7O64o8\nZSE34WXCBLjrLmje3AW8fn2vK/KchdyEjzfegDvugGbN3D64D+fzVwQWchMe3nnHnUmWkOAC3qSJ\n1xUFDQu5CX0zZsCNN7qm+cKF8Ic/eF1RULGQm9A2Zw5cfz3Urev2wU85xeuKgo6F3ISuefOgZ083\nGeCCBXDaaV5XFJQs5CY0LVrkZgCNinLHwVu29LqioGUhN6Hns8/gyivdaaJJSfDnP3tdUVCzkJvQ\nsnQpdOkCIjB3LrRu7XVFQS+8p8Qw4eWbb9yEazk57nTRNm28rigkWMhNaPj+e+jY0V0Rc84cd1kd\n4xMLuQl+q1a52Vz27YOZM91t4zMLuQlua9a4a4Tv3g3vvef2x02ZWMhN8PrlF2jfHn7/Hd5+u+hF\n84zPLOQmOG3c6AK+bZs78eTaa72uKGRZyE3w2brVBXzjRnfq6I03el1RSLPj5Ca4/PqrC/gvv7jJ\nH/r187qikGchN8Fjxw7XybZmDbzwAvztb15XFBYs5CY47NrljoOvWuUmYLznHq8rChsWcuO9PXug\nUyf47js3hfL993tdUVixkBtv7dvnrku2bBk88oj7Mn5lITfeOXDADW758ku39R4xwuuKwpKF3Hgj\nI8NdOvizz+Dee+GZZ9yZZcbvLOQm8LKyoEcPNx/b3/4GY8ZYwMuRhdwEVna2G72WlOSOgb/8sgW8\nnFnITeDk5EDv3jB7NvTpA6+9BhH2J1je7DdsAiM3F/r2henT3ZZ80iSoVMnrqioEC7kpf3l5cNtt\nh88ke/NNqGynTQSKhdyUr7w817k2ebI7XPbOO24CRhMwFnJTflTd8NTx491sLtOnQ7VqXldV4VjI\nTflQhSFDXO95u3bwwQdQvbrXVVVIFnLjf6rwz3/Cv//tZlSdM8ddBMF4otSQi0h1EVkqIstFZJWI\nPBWIwkwIGzECnnoK/vIX+OgjqFnT64oqNFHV0hcSiVLVdBGpDHwGDFHVz/IfU1/WYSqIp5+Ghx5y\nVzVJTobatb2uKKyJCKpa4mgin5rrqpqef7MqUAnYdYK1mXA0ZowLeMuW7mKEFvCg4FPIRSRCRJYD\nvwGLVHVV+ZZlQs7LL8N998Gpp7orjMbFeV2RyefrljxPVVsBjYC2IpJYrlWZ0DJ+PAwcCH/8o2ui\nn3SS1xWZQso07EhV00TkQ+A8IOXQz4cPH16wTGJiIomJif6pzgS/yZPhzjtJj/8DdzRYyI5bGjJ4\nsLtkmfG/lJQUUlJSyvScUjveRCQOyFHVPSISCSQBj6tqcv7j1vFWUb39Ntx4Ixl1E/jz3k9YndUM\ngMhImDHDgh4I/up4awAszN8nXwrMPhRwU4G9/747kyw+ngEtFhYEHNx8EKNHe1eaKarU5rqqrgDO\nCUAtJlTMmQO9ernOteRktt3zx+LLLFsGSam2OQ8CNuLNlE1SEvTsCTExrhf91FMZPNg10Q+JJJ3B\nux52s78kJXlXqwF8HAxT4gpsn7ziWLgQunZ1Y9AXLnQDXvIlJcHo3stg104GM5rLmece6NjRHTM3\n5cKXfXI7qdf45tNPoVs3qFrVhbZQwMG1yi8/9yGYP9+jAs2xWHPdlO7LL9254CLw8cduTPrRFGu3\nR7qfGU9Zc92UbNkyd32y7GyYOxfati15+aSkw13rdsC83PnSXLeQm2P77ju49FJIT3c96h06eF2R\nOYLtk5vjt3KlC/WBAzBzpgU8hFnITXGrV7sm+p49bsqmK67wuiJzAizkpqh166B9e0hNdcNWu3f3\nuiJzgizk5rCNG13At2+HKVPgr3/1uiLjBxZy42zZ4jrZNm1yFz7o3dvrioyf2HFy47bc7dvD+vUw\nbhzccovXFRk/spBXdDt2uJ7zn3+GF1+E/v29rsj4mYW8Itu50wV81Sp47jk3u4sJOxbyimrPHujU\nCb7/HkaNsuGnYcxCXhHt3QudO8M338Bjj8HDD3tdkSlHFvKKZv9+d7ro0qXwwANQaH4+E54s5BVJ\nerob3PLZZ3Dvve5CCFLisGcTBizkFUVmppupZdEiGDDAXQjBAl4hWMgrguxsN3pt3jy47TZ46SUL\neAViIQ93Bw+6SRc//BD69nWDXSLsY69I7NMOZzk5LtgzZsB118HEiVCpktdVmQCzkIer3Fzo1w/e\necfti0+ZApXtVIWKyEIejvLy4M474Y033OGyt9+GKlW8rsp4xEIeblTd8NQJE9yItvfeczOsmgrL\nQh5OVN3lg1991Z02OmOGmyPdVGgW8nChCg89BM8/DxddBLNmQVSU11WZIGAhDxePPw7PPAPnnw8f\nfQQ1a3pdkQkSFvJw8NRTLuTnnOPmRo+J8boiE0Rs3vVQ9+9/u9NEW7Z0Q1br1vW6IhNA/ro+uQlW\nL73kAn7aae4KoxZwcxQW8lD12mswaBCccgokJ8NJJ3ldkQlS1lwPRZMnw623QrNm8Mkn0KiR1xUZ\nj1hzPRxNneqGqzZu7K4RbgE3pbCQh5Lp090JJ/HxLuDNmnldkQkBFvJQMWuWO2U0Ls4FvHlzrysy\nIaLUkItIYxFZJCIrReQHEbknEIWZQubOhWuvhdhY18n2pz95XZEJIaV2vIlIPBCvqstFpCawDLha\nVX/Mf9w63spTcrI7kywqyh0HP/tsrysyQcQvHW+q+quqLs+/vR/4EWjonxJNiT75BLp1g2rV3NRN\nFnBzHMq0Ty4izYA/A0vLoxhTyBdfuC14pUqQlATnned1RSZE+TxVSH5T/T3g3vwtuikvX3/tLn6Q\nl+f2xy+4wOuKTAjzKeQiUgWYDkxR1ZlHPj680AT9iYmJJCYm+qm8Cmj5cjfZQ3a2m3zxkku8rsgE\nkZSUFFJSUsr0HF863gSYDOxU1X8c5XHrePOXH35wkz3s3esOmV1+udcVmSDnS8ebLyG/GPgE+B44\ntPBDqjo3/3ELuT+sXg3t2rkrjb7/vutwM6YUvoS81Oa6qn6GDZopX2vXknlRe6rsSmVUy3c4v2o3\nbBtu/MVOUPHahg1ktG5L1R1b6cMU3uYGIiPd9GzWWjelsRNUgt3mzdC+PdV2bKEfE3mbGwDIyIDR\noz2uzYQNC7lXtm+Hyy6D9et54bRxvM7NXldkwpSF3Au//+4C/vPP8NJLnD7mDiIjDz8cGekmfDHG\nHyzkgbZzJ3ToAD/+6Nrkd9/N5Ze7ffCOHd1Xkf3xpCR33LxTJ3fbmDKyjrdA2r3bbcG//RaefNLN\nk16SpCR3HbOMDHffeuTMEazjLZjs3euGqn77LQwbVnrAwW3pDwUcrEfOHBcLeSDs3w9XXAFffQUP\nPuhCbkyAWMjLW3q6G732+efwj3+4ZrqU2Lo6bPBgrEfOnCjbJy9PmZnQvTvMnw933w0vvuh7wA9J\nSjrcRB882PbHTRF+Gbvuw4tYyI8mO9t1mn30Edx+O4wbBxHWcDL+ZR1vXjl4EK6/3gX8ppss4MZT\n9pfnbzk50KcPzJzpZledONECbjxlf33+lJvrrmzy7rtwzTXw+utu+iZjPGQh95e8POjfH6ZMgSuv\ndFc6qVLF66qMsZD7hSoMHOia5p06wbRpULWq11UZA1jIT5yqO/796qtu6qaZM6F6da+rMqaAhfxE\nqLoRbC+8ABdfDLNnFx28YkwQsJCfiGHD4Nln4fzz3cyqNWp4XZExxVjIj9eoUfDEE3DuuW5u9JgY\nrysy5qhsxNvxeO45uP9+OOssd32yOnW8rshUUDbirTz83/+5gJ9+OixYYAE3Qc9CXhbjxsG990KL\nFu5qo/XqeV2RMaWykPtq0iT429/g5JNh4UKIj/e6ImN8YiH3xZtvwm23QZMmLuAJCV5XZIzPLOSl\nmTbNnUnWsKHrZGva1OuKjCkTC3lJPvgAeveGk05yW/CTT/a6ImPKzEJ+LB9/DNdeC7Vru062Fi28\nrsiY42IhP5oFC9ysLtHR7vbpp3tdkTHHzUJ+pMWL3bxs1avDvHluwIsxIcxCXtjnn0PXrm6ih7lz\n3ZBVY0JcqdcnrzD+9z83N7qqC/gFF3hdkTF+YSEHd1WTTp3cDKsffuhOGzUmTFjIV6xwVxlMT4dZ\ns6B9e68rMsavKtY++ZFXCP3pJ3eF0b17Yfp0u3CBCUsV51TTI68QWq2am+QhLc2NauvRw9v6jDkO\nfjnVVEQmishvIrLCf6V54MgrhGZlwa5dbly6BdyEMV+a65OAzuVdSKAp8EyNx+k04XqSkryuxpjy\n41NzXUSaAbNVteVRHgud5vrVV7uLEAIDeIWxDADc3IszZtguuQk9NjNMYa1aFUzy8GLU0IKAg2vF\nH7pwqDHhxi+H0IYPH15wOzExkcTERH+s1n9SU10v+ubNMGYMsz/6O8z3uihjyi4lJYWUlJQyPSf8\nm+u7d7tj38uXw9NPw9ChxTrarbluQpU119PSXHKXL4fHH4ehQwH3oxkz3BiYjh0t4Ca8lbolF5Gp\nQDugLvA78JiqTir0eHBuyfftg86d3UknDz8MI0eClPgPz5iQ48uWPDwHw6SnQ5cu7rTR++5z86Rb\nwE0YqphFDMSsAAAGWUlEQVTN9cxMuOoqF/CBAy3gpsILr5BnZcE117jZXO64w12I0AJuKrjwCfnB\ng3D99W5utptvhrFjISJ83p4xxys8UpCT42ZV/eADuOEGmDDBAm5MvtBPQm6u23K/9x707Amvv+6m\nbzLGAKEe8rw8uP12eOst6NbNfa9s82AYU1johlwV7roL/vtfdzx82jSoWtXrqowJOqEZclV3ddFx\n4+Cyy+D9990kEMaYYkIv5KrwwAPw4otwySWusy0y0uuqjAlaoRfyxx5zA1wuvNDNrFqjhtcVGRPU\nQivkI0e6r/POc8fDo6O9rsiYoBc6If/Xv+DRR93kD0lJUKuW1xUZExJCI+QvvOD2w884w12frE4d\nrysyJmQE/1loY8fCgAHu0sGLF0N8fPm9ljEhxpez0IJz5EhSkpt0betWWLUKmjeHhQst4MYch+AL\n+ZFzM4nAsGGQkOBtXcaEqODbJy90EYQMqnGTTqLT30+3udGNOU7BtyX//XcAsqhKa77iB86CXfBZ\nD5uLzZjjEVxb8g8/hJUrAbibl1zA89nc6MYcn+AJ+bx57lTRmBh45RU21TnH64qMCQvB0VxPSXHz\nskVGwvz5cM45DD7ZNdELz40+eLCnVRoTkrw/Tr5kidvRjohwc7O1bl3w0KEjaeACbvvjxhQV/FMy\nL13qrm6Ql+cSfdFFJ1SLMRVNcA+G+eYbt2nOyYGPPrKAG1NOvAn5ihVuC56ZCbNnQ7BdINGYMBL4\nkP/4o5vNZd8+mDnThd0YU24CG/Kff3YB373bzcnWpUtAX96YiihwIV+/3l1C+LffYOpUuPrqgL20\nMRVZYEK+aRNceqk7q+z11+G66wLyssaYQIR861a3Bd+40V3ZpE+fcn9JY8xh5Tus9ddf3T74unXw\nyivQr1+5vpwxprjyC/mOHdChA6xeDc8/72Z3McYEXPmEfNcud2hs5Up45hl3IQRjjCf8H/K0NDeS\n7bvvYMQINwGjMcYz/g35vn3uumRffw3//KebQtkY46lSQy4inUXkJxH5WUSGHnPBAwega1f48ksY\nMgSeeMKvhRpjjk+JIReRSsBLQGfgdOAGETmt2IIZGdC9O3z6KdxzDzz7rJuAMUilpKR4XUKZhVrN\noVYvhGbNvihtS94aWKuqG1T1IPA2cFWxpa65xk2ZfOedric9iAMOoflhhlrNoVYvhGbNvigt5AnA\n5kL3t+T/rKi5c+HWW92x8CAPuDEVTWkh9202iBtvhPHj3ewuxpigUuLMMCJyATBcVTvn338IyFPV\nZwotU47XSDLGlOaEpn8SkcrAauAyYBvwFXCDqv7ozyKNMeWnxBNUVDVHRAYCSUAlYIIF3JjQcsIT\nORpjgtsJ9ZT5PFAmSIjIRBH5TURWeF2LL0SksYgsEpGVIvKDiNzjdU2lEZHqIrJURJaLyCoRecrr\nmnwhIpVE5FsRme11Lb4QkQ0i8n1+zV+VuOzxbsnzB8qsBjoAW4H/EeT76yJyCbAfeF1VW3pdT2lE\nJB6IV9XlIlITWAZcHcy/YwARiVLV9Pw+nc+AIar6mdd1lURE7gPOBaJVtbvX9ZRGRNYD56rqrtKW\nPZEtuW8DZYKIqn4K7Pa6Dl+p6q+qujz/9n7gR6Cht1WVTlXT829WxfXllPqH6CURaQR0Af4DhNJA\nD59qPZGQ+zZQxviFiDQD/gws9baS0olIhIgsB34DFqnqKq9rKsUY4H4gz+tCykCBBSLytYjcUdKC\nJxJy67ELkPym+nvAvflb9KCmqnmq2gpoBLQVkUSPSzomEbkS+F1VvyW0tuIXqeqfgSuAu/N3RY/q\nREK+FWhc6H5j3Nbc+JGIVAGmA1NUdabX9ZSFqqYBHwLneV1LCdoA3fP3cacC7UXkdY9rKpWqbs//\nvgOYgdt9PqoTCfnXwCki0kxEqgLXA7NOYH3mCCIiwARglao+73U9vhCROBGJzb8dCXQEvvW2qmNT\n1YdVtbGq/gHoBSxU1Zu8rqskIhIlItH5t2sAnYBjHjE67pCrag5waKDMKuCdEOj1nQp8DrQQkc0i\ncqvXNZXiIqAPcGn+oZJvRaSz10WVogGwMH+ffCkwW1WTPa6pLEJhN7Q+8Gmh3/EcVZ13rIVtMIwx\nYc5OGzMmzFnIjQlzFnJjwpyF3JgwZyE3JsxZyI0JcxZyY8KchdyYMPf/RlEHi9Xp5YYAAAAASUVO\nRK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x2de5910>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"M = items1.T.dot(1/(Sreduced)).dot(features1) + x.mean(axis=0)\n",
"print \"Reconstructed matrix: \"\n",
"print M\n",
"print \"Orignal matrix: \"\n",
"print x\n",
"fig, ax = plt.subplots()\n",
"p1 = plt.scatter(x[:, 0], x[:, 1], color='red')\n",
"p2 = plt.scatter(M[:, 0], M[:, 1], color='blue')\n",
"plt.ylim([0, 5])\n",
"plt.xlim([0, 5])\n",
"plt.legend([p1, p2], [\"Original\", \"Reconstructed\"], loc='upper left')\n",
"plt.title(\"Data\")\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"ax.quiver((0), (0), (Vreduced[:, 0][0]), (Vreduced[:, 1][0]), scale_units = 'xy', scale = (0.15), color = 'r')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatively, the data reconstruction could be performed much more straightfowardly by taking advantage of SVD: \n",
"\n",
"$$\\hat{U}\\hat{S}\\hat{V}^T$$\n",
"\n",
"Where _m_ eigenvectors and eigenvalues are chosen from $U$, $S$, and $V^T$. Let us see that this gives us the same result as above: "
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Directly using SVD: \n",
"[[ 0.54252216 0.47571843]\n",
" [ 1.0627096 1.01067656]\n",
" [ 1.87937477 1.85053087]\n",
" [ 2.93521178 2.93634828]\n",
" [ 4.08018169 4.11382953]]\n",
"Using reduced feature and item vectors: \n",
"[[ 0.54252216 0.47571843]\n",
" [ 1.0627096 1.01067656]\n",
" [ 1.87937477 1.85053087]\n",
" [ 2.93521178 2.93634828]\n",
" [ 4.08018169 4.11382953]]\n"
]
}
],
"source": [
"M2 = Ureduced.dot(Sreduced).dot(Vreduced) + x.mean(axis=0)\n",
"print \"Directly using SVD: \"\n",
"print M2\n",
"print \"Using reduced feature and item vectors: \"\n",
"print M"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Code Implementation vs Python's PCA package\n",
"Let us try PCA on three-dimensional data using the methods introduced above as well as practice using the PCA package from python. First, let us generate some three-dimensional data and visualize it! In this data sample, we will have 20 points belonging to one class and 20 points from the other. The points will be drawn from two multivariate normal distributions, one with mean (0, 0, 0) and the other with mean (1, 1, 1). The variances are 1 in each dimension and all covariances are 0. "
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAHMCAYAAABY25iGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXmYFOW1/7+9d89Mz7CLgohREwXZvEYiLiQKKkYFryZq\nrmtMjEmuV0x+JooaUBHcEWNcsmmMSxJvjIAoCghGSIQYAUU0LlcUUOMCznR39Vbd9ftjPOXbNVXV\ntVd19/t5Hh91uqvq7VreU+e853xPSJIkcDgcDofD0Sfs9wA4HA6Hw2kEuMHkcDgcDscA3GByOBwO\nh2MAbjA5HA6HwzEAN5gcDofD4RggWudznkLL4XA4nFYjpPZH7mFyOBwOh2MAbjA5HA6HwzEAN5gc\nDofD4RiAG0wOh8PhcAxQL+mHw+FwOE1IKKSa19KSGJWI5QaTw+FwWhSuJW7uxYGHZDkcDofDMQA3\nmBwOh8PhGIAbTA6Hw+FwDMDXMDkcDocDQRBwzTW/woYNH6NYjCCRqGDChEGYPfsCpFIpx7dTY86c\nOUin0/jxj39s9+cAAL797W9j6dKlGDJkCF5++WXb++MGk8PhcFocQRAwZcql+PvffwpghPz3p59+\nF3/96//DypU3qxo/q9tp4XTm7nnnnYeLLroIZ599tiP74yFZDofDaXGuueZXfYxeLyPw97//FHPm\n3OPodsT999+PcePGYfz48X2M2q9+9SsccsghGD9+PE499VTk83kAwCOPPIIxY8Zg/PjxmDx5MgDg\nlVdewcSJEzFhwgSMGzcOb775JgDgiCOOQP/+/Y2cAkNwg8nhcDgtzoYNH6Ov0SNGYOPGjx3dDug1\nctdddx1WrVqFjRs3YuHChTWfn3LKKVi/fj02btyIAw44AL/5zW8AANdeey2efvppbNy4EUuWLAEA\n3HPPPbj44ouxYcMG/POf/8Tw4cN1fq11uMHkcDicFqdYjNT5XH31zup2APDMM8/gm9/8JgYMGAAA\nfTzBl19+GUcccQTGjh2LBx98EFu2bAEAHHbYYTjnnHPw61//GqIoAgAOPfRQzJs3DzfeeCO2bt2K\nZDKpOy6rcIPJ4XA4LU4iUanzuejodkDveqWacAKtY5577rm488478dJLL2H27NlySPauu+7C3Llz\nsW3bNvzHf/wHdu7ciTPOOANLlixBKpXC8ccfj1WrVumOyyrcYHI4HE6LM2HCIADvanz6DsaPH+To\ndgBw1FFH4ZFHHsHOnTsBQP43GdFsNouhQ4eiXC7jgQcekLd76623cMghh+Dqq6/G4MGDsX37drz9\n9tsYOXIkLrroIkyfPt2RjFg1uMHkcDicFmf27Atw6KE3oK/xexeHHnoj5sz5nqPbAcCoUaNwxRVX\nYPLkyRg/frxcSkIe5rXXXouJEyfi8MMPxwEHHCD//Sc/+QnGjh2LMWPG4LDDDsPYsWPxpz/9CWPG\njMGECRPwyiuvyAlEZ5xxBiZNmoTXX38de+65J+69916zp6aGUB0tQS40yOFwOE2IMiSaz+cxZ849\n2LjxYxSLUSQSIsaPH4Q5c76nWxpidbugoBEaVq1v4QaTw+FwWhCtNcRWw4zB5CFZDofD4XAMwA0m\nh8PhcDgG4AaT0zDMmTMHZ511luP7Pe+88zBgwAB85StfcXzfLOeeey6uuuoqV48RBFavXo0999zT\n9HZuXV8Oxym4weTUZc2aNZg0aRL69euHgQMH4vDDD8cLL7zg+Tjc6BD/3HPPYcWKFXjvvffw/PPP\nO75/llAo5GmX+wsuuAD7778/IpEIfve733l2XKt4eW4++ugjnHHGGRg2bBj69euHww8/HOvXr/fs\n+JzGhBtMji49PT044YQTcPHFF2PXrl3YsWMHZs+ejUQi4flY3EhQeOeddzBy5EhLyiCkMmIGL5Ms\nxo8fjzvvvBMHHXSQp8bIKl6em2w2i4kTJ+LFF1/Erl27cM455+DrX/86crmcZ2MIMlavRbMnEXGD\nydHl9ddfRygUwmmnnYZQKIRkMompU6dizJgxAHqLiI866igMGjQIgwcPxplnnonu7m55+5EjR+Lm\nm2/G2LFjkU6ncf755+Pf//43pk2bhq6uLkydOhWffvopAGDr1q0Ih8P41a9+hWHDhmGPPfbALbfc\nojm2559/HpMmTUL//v0xfvx4PPvss/Jn9913H/bZZx90dnbiC1/4Ah566KE+2//mN7/Bd7/7Xfz9\n739HOp3G1VdfDaBX9Hm//fbDwIEDMX36dLz//vvyNuFwGHfeeSf2228/fOlLX1IdF3nk/fv3x4gR\nI3D//ff3+c6uXbtwwgknYMiQIRgwYABOPPFE7Nixo+7433zzTUyePBn9+vXD4MGDcfrpp2uenx/8\n4Ac46qijDL0MLF26FBMmTEBXVxdGjBghnwvg8+ty//33Y6+99sLgwYMxb948+fN8Po9zzz0XAwYM\nwOjRo/GPf/xD91ivvPIKpk6dioEDB2Lo0KGYP3++6ve+8Y1vYPfdd0e/fv0wefJkWRoNAJ544gmM\nHj0anZ2dGD58uHyffPzxxzjhhBPQv39/DBw4EEceeaTqJL733ntj5syZ2G233RAKhfDd734XpVIJ\nr7/+et1z1exIkoQffec7po2f1e1Y5syZo/vMm2Hbtm342te+htGjR+PAAw/E7bffbn+nkiTp/cNp\ncXp6eqSBAwdK55xzjvTkk09KO3furPn8zTfflFasWCGVSiXpo48+ko488khp5syZ8ucjR46UDj30\nUOnDDz+UduzYIQ0ZMkSaMGGCtHHjRqlQKEhHHXWUdPXVV0uSJElvv/22FAqFpG9961uSIAjSyy+/\nLA0ePFhasWKFJEmSNHv2bOnMM8+UJEmStm/fLg0cOFB68sknJUmSpOXLl0sDBw6UPv74YymbzUqd\nnZ3S66+/LkmSJH3wwQfSK6+8ovr77rvvPunwww+X/3/lypXSoEGDpA0bNkjFYlG66KKLpCOPPFL+\nPBQKScccc4y0a9cuqVAo9Nnf1q1bpXQ6Lf3hD3+QRFGUPvnkE2njxo2SJEnSueeeK1155ZWSJEnS\nJ598Ij366KNSPp+XMpmM9I1vfEOaMWOGJEmS7vhPP/10ad68eZIkSVKxWJTWrl1b5wpK0uGHHy79\n7ne/0/3O6tWrpc2bN0uSJEkvvfSStNtuu0mPPfaYJEmfX5cLLrhAKhQK0qZNm6REIiG99tprkiRJ\n0k9/+lPpyCOPlHbt2iVt27ZNGj16tLTnnnuqHqenp0caOnSodOutt0rFYlHKZDLSunXrJEmqvb6S\nJEn33nuvlM1mpVKpJM2cOVMaP368/NnQoUOlNWvWSJIkSZ9++qn04osvSpIkSZdddpl04YUXSqIo\nSqIoyt+px4YNG6RkMin19PQY+n4z0Dv99+XJRx6RZqbT0rL//V9T+7O6HcucOXOkm2++2fL2LO+/\n/760YcMGSZIkKZPJSF/84helLVu29PmexnlQtYncw+Tokk6nsWbNGvktfMiQIZg+fTo+/PBDAMA+\n++yDo48+GrFYDIMGDcIll1xS4+kBwEUXXYTBgwdjjz32wBFHHIFDDz0U48aNQyKRwMknn4wNGzbU\nfH/27NlIpVI48MADcd555+Hhhx/uM64HHngAxx9/PI477jgAwJQpU3DwwQdj6dKlCIVCCIfDePnl\nl5HP57Hbbrth1KhRqr9PUrwNP/jggzj//PMxfvx4xONxzJ8/H3//+9/x7rufK5lcfvnl6Nevn2pY\n+qGHHsLUqVNx2mmnIRKJYMCAARg3blyf7w0YMAAnn3wykskkOjo6MGvWrJrzpjX+eDyOrVu3YseO\nHYjH45g0aZLq7zLL5MmTMXr0aADAmDFjcPrpp/e5jhSKHzt2LMaNG4dNmzYB6G23dMUVV6Bfv34Y\nPnw4Lr74Yk0v4/HHH8cee+yBSy65BPF4HB0dHTjkkENUv3vuueeivb0dsVgMs2fPxqZNm5DJZOTz\n8Morr6CnpwddXV2YMGGC/Pf3338fW7duRSQSwWGHHVb3t/f09OCss86Smxe3MpIk4ambb8atmQyW\n3XSTYW/R6nZutvcaOnQoxo8fDwDo6OjAAQccgPfee8/oqVCFG0xOXfbff3/ce++92LZtGzZv3oz3\n3nsPM2fOBAD8+9//xumnn47hw4ejq6sLZ511Fj755JOa7XfbbTf5v1OpVM3/J5NJZLPZmu+zGZYj\nRoxQvcnfeecdPPLII+jfv7/8z9q1a/HBBx+gra0Nf/zjH3H33Xdjjz32wAknnIB//etfffZBb40s\n77//Pvbaay/5/9vb2zFw4MCacKleBuj27dvxhS98QfNzQhAEfO9738PIkSPR1dWFyZMno7u7G5Ik\nob29XXP8N954IyRJwiGHHIIDDzzQttQXsW7dOnzta1/DkCFD0K9fP9xzzz19ruPQoUPl/25ra5Ov\n23vvvdfnmmmxbds2Q+enUqngsssuw7777ouuri7svffeCIVC+Pjj3nZRf/7zn/HEE09g5MiR+OpX\nvyonbF166aXYd999ccwxx2CfffbBDTfcoHucfD6PE088EZMmTcJPf/rTuuNqdp76859x3MsvIwTg\n2JdfxtOPPuradl6299q6dSs2bNiAiRMnGvo9WnCDyTHFl770JZxzzjnYvHkzAGDWrFmIRCLYvHkz\nuru78fvf/x7ValV3H/XePllv7t1338WwYcP6fGfEiBE466yzsGvXLvmfTCaDn/zkJwCAY445Bk8/\n/TQ++OAD7L///vjud7/bZwylUgn5fB7VahWiKEKSJOyxxx7YunWr/L1cLodPPvmkZgx6CTR77rkn\n3nrrLc3PadtbbrkFr7/+OtavX4/u7m48++yzNQZca/y77bYbfvnLX2LHjh2455578IMf/AD/93//\np3k8o3zrW9/CjBkzsH37dnz66ae48MIL615HYvfdd+9zzbQYMWKEofE+9NBDWLx4MVauXInu7m68\n/fbbNefn4IMPxmOPPYaPPvoIM2bMwDe/+U0AvZ7EzTffjLfeeguLFy/GrbfeimeeeUb1GMViETNm\nzMCIESNwzz36jY5bAfISjxEEAMCxgmDIW7S6nVftvbLZLE499VQsXLgQHR0dJs5IX7jB5Ojyr3/9\nC7feeqvsYW3btg0PP/wwDj30UAC9N2N7ezs6OzuxY8cO3HTTTbaPOXfuXOTzebzyyiu47777cNpp\np/X5zplnnoklS5bg6aefRqVSQaFQwOrVq7Fjxw58+OGHWLRoEXK5HGKxGNrb2xGJfN63r1qtolgs\nykayWq0im83KiTj33nsv/vnPf6JQKGDWrFn4yle+ous1sfzXf/0XVqxYgUceeQSiKOKTTz6RQ5fs\nhJ/NZpFKpdDV1YWdO3fWJNnojf+RRx7B9u3bAQD9+vWTw89qlMtlFAoFVKtVlEolFAoFzUksm82i\nf//+iMfjWL9+PR566CHDmbXf/OY3MX/+fHz66afYvn07fv7zn2t+94QTTsD777+PhQsXolgsIpPJ\nqJZzZLNZJBIJDBgwALlcDrNmzar5XQ8++CC6u7sRiUSQTqfl8/P444/jzTffhCRJ6OzsRCQSqbn2\n7D5OPfVUtLW14b777jP0O5sd1ksEYNhbtLqdF+29yuUyTjnlFJx55pmYMWOG0VOhCTeYHF3S6TTW\nrVuHiRMnoqOjA4ceeijGjh0rZ7LNnj0bL774Irq6unDiiSfilFNOqTvRsp+r1SZOnjwZ++67L6ZM\nmYJLL70UU6ZM6fPd4cOHY9GiRZg3bx6GDBmCESNG4JZbbpEN4IIFCzBs2DAMHDgQzz33HO666y5I\nkgRRFLFr167eBfxwWP6HJtVJkybh8ssvxze+8Q3sscceeOONN3DffffJxrXeb9tzzz3xxBNP4JZb\nbsHAgQMxYcIEvPTSS33GP3PmTOTzeQwaNAiTJk3CtGnT5M+0xg8AL7zwAr7yla8gnU5j+vTpuP32\n2zFy5EjVsUydOhVtbW14/vnnccEFF6CtrQ3PPfec6nfvvPNO/OxnP0NnZyeuvfbaPi8per979uzZ\n2GuvvbD33nvjuOOOw9lnn635/Y6ODixfvhxLlizB7rvvji9+8YtYvXp1n/Nz9tlnY6+99sKwYcNw\n4IEH4tBDD63Z5wMPPIC9994bXV1d+OUvf4kHH3wQQG8W8dSpU5FOpzFp0iT88Ic/lNe5WP72t79h\n6dKlWL58Ofr164d0Oo10Oo21a9dq/s5mRuklEvW8RavbAd609zr//PMxatQoeQnJLlx8nRMYtm7d\nii984QsQRVHTa7IKhWCr1So+/fRT9O/fX/4bHUsURZTLZbnDApsdR0QiEcRiMUSjUUQikYaob+Rw\n1GA9vGX/+78InXMOjlUYPgBY1taG0P3349hTTun7mcXtiPvvvx833XQTIpEIJkyYgJEjRyKdTuNH\nP/oR7r77btx4440YPHgwJk6ciGw2i9/+9rc45ZRT8MYbb0CSJEyZMgULFizADTfcgN///veIxWLY\nfffd8dBDD2Hz5s048sgjMXbsWPk5nT9/vpwoqHYe2D+rnjNuMDlBwS2DSSFJ8hB37dqF/v37o1wu\no1gsIhqNIhwOo1Kp1BhMJdyAcpoJ1lBcdt55SPzf/6nev5IkofiFL+B6lQQzq9sFCW4wOQ3J1q1b\nsc8++6BcLjtiMCkEK4piTbhv586dSCaTKBaLCIfDqFarcohWkiQkk0mEw+G6xo+MZ7Valb/LDSin\nUeDtvXrhBpPT8lSrVZTLZdmYkeGqVCro7u5GNBpFW1ub7M3S91m5u3A4XGP4uAHlNBPcYPZixmBG\n3R8Oh+MdkiTJoVWgNpGkVCrJWqHt7e01xosSf6rVKlKpFKrVKiqVCiqVCkqlEgDIGZdaBpT+Rt4x\nGc98Pl9jQKPRKGKxGDegHE6DwQ0mp2nQCsFKkgRBEFAul5FOp5HJZOoaKsqejcViAOCYARVFEblc\nTq4Ti0aj8j/cgHI4wYYbTE5ToBWCJQMViUTQ2dlZd21UK0TFGlAKveoZULXjsMYzEonUeMM0Xtb7\n5AaU4zb8/jIHN5ichkZpdFhvrlgsIp/Po62tDfF4XHVyUNaEGoH1HO0aUOWYK5VKzToqGVDK5OUT\nHMcp+PqlebjB5DQsbG0l61VWq1UIgoBKpSKrvaht65TxMWNAKRNXbz/sGMmAUhYvm0TEDSiH4y3c\nYHIaEpKzq1QqSKVSNSHYbDaLWCyGzs7Oul6lG6gZUHYNFIAcJjbigRJKDzQUCtWsgXIDyuG4CzeY\nnIaCTeyh+klKCy8UCigUCmhvb0c8Hvd7qDKhUEg2jOT9JpNJ2fgVi0X5O3rJP2oGlNSJ6HNuQDkc\n9+AGk9Mw6IVgqdWUVgg2SLAGFECNB0qC6RR+1Uv+0TOg5XIZ0WgU8XicG1AOxyG4weQ0BEpPijWW\n3d3dSCQSNaHZRsINA0pSgGQ86XPugXI41uEGkxNo9GoryRik02m5XtLMfp0WeHcKJw0o621rGVAq\nY+EGlMPRhxtMTmDRk7ejEGwsFjNtLBvNKFgxoHr7IbgB5XDMwQ0mJ3Ao5e1YT7BYLEIQBLmjCFuz\n6NSxg44RA8pixwNldXC5AeW0OtxgcgIFTdqVSqVPCDaXy0EURaTTaUSjURSLRUeP3ajGQM2AUnd6\nSpKyGsItlUpyFi9rQKPRqCFBeg6nmeAGkxMYlH0rlbWV0WgUXV1d8t95twV1lMZNKaLghAFlP08k\nEtyAcloCbjA5vqNM7NGSt0skEq4cu9kneTY7FoCqAVWKKBgxoFTOwyZk0Zqy0ZZoHE4jwQ0mx1dI\ncCCbzSKdTteUi+RyOVSrVcdrK1nPtBUndD0DWiwWTRlQALIXC/SGgEkKkPqJcgPKaRa4weT4BqvY\nQ2uWAFAul5HL5RCLxdDR0cEnWZdxwoCy/T5pH4C+AQ1qWQ+HowU3mBzP0QvBmpG342uY7mDGgNLn\n3IByWgFuMDmeolVbKUkSMpkMAKCrq4tPnjZx8kVCy4BSSY+akDw3oJxmhBtMjifo1VbShBmLxZBM\nJnkINuCQAY1EIhBFEW1tbbIHWi6XIUmSJQNKWbjcgHKCCjeYHNfRq60UBEE2oiRG4DZ6oVwe4jVP\nOByWW5kBqGllpjSgWuUnakLybBlLuVxGMpnsI6TA4XgJN5gcV9GqrSR5u3A4jHQ6je7ubtMlHk6v\nYXrl2XpllP3y1PUMKAkqKGtA9QworW1TmRHVgdIxuAHleAU3mBxX0EvsKZVKsrxdIpFoqRBsM/1W\noy84egaUwq/1DCjthz220oBGIhE5fMsNKMcNuMHkOI5W30o1eTtO68EaUDJ8egYU6PuioRbCrVar\nso6uJEk1658UCuZw7MBnLI6jsOtWavJ2sVisRt6O09rQPVLPgNI6uJbnqGdA6V4kD5RCuPwe5JiF\nG0yOI9CE1t3djY6ODtXaSj15O1qP9GINk9dvBhc1A1qpVFAoFDQ9ULMGlOAGlGMWbjA5tmFrK2nN\nkv7ulrwdpzUg4xkKhZBMJmtamYmiKHdS4QaU4wXcYHIso6ytZCeYcrmMbDaLRCIRWHk7mnD1mi5z\ngoVeL1AnDOinn36KeDwud3PhBpTDwg0mxxJatZUAIAgCSqUSOjo65MzIengZJmUbJUej0ZpQHwDd\ntTJOsDBiQI22MmPDweFwGNVqFfl8vkZkgRvQ1oYbTI5p9Gor6d9eyNtZMbLkFQNAOp2WDT5NtPl8\nvo+nwidIfzGztq1nQMvlMgqFgq4BZbvYKMuhuAHlcIPJMQwbgmUnE6BX3i6XywEA2tvbA+mdUaYu\nACQSCUQiEdl4sr0ek8kkAJiaaDnBxKwB1TLO3IByAG4wOQbRq60kebt0Oi0bJKvHcANlI2o2MUkL\nu54KJ5jUu64AUCgUDIdw9QwoCdZzA9o8cIPJqYtWCFYURblTRWdnpy2v0q3JhMQSKpWKnKlLXTbM\noDbRsrWCRpsuNxNmy4CCiPK6Uq0wrXMbfTFSM6BsNAbgBrQZ4AaTo4mevB3rscXjcV8e/nprmKxY\nQmdnp+ZEZ/XYdpsuc4IJm/Cl9mLkhAGVJAnhcBjJZJJHKBoIbjA5qmj1raxWqxAEocZjYwmCKIDS\noGuJJTiJ0oAqO3YAfSdiPkEGE/a66L0Y2TGgoijKKkQESflFo1H+ghVQuMHk1EChqHw+L3uOavJ2\nWh6bVZwytGohWK3vuYmW4DiFgwVBqFsryOklSC8XRgyo0cgCm2imbMoNcAMaRLjB5MhQCJYMJmWL\nsvJ27e3tiMfjdffjB0YNuh8TDxnQaDSKXC6HVCqlWmzP17caC6dC82pCCqwBZY/DDah/cIPJAdA3\nsYf9O2W+GpG38/IhZsPEpVLJUgjWL+9F2bGDZ+D6jxMvekYNKHtMowaUXmaVx+EG1Du4wWxx1Gor\nq9UqgM9rKxOJBFKplKsPpJ2QbKPr1fISlmDh9FKDmgEllSzKMrfigXID6j3cYLYwWrWV9JkgCKbk\n7QBvk37YcJVTa6o0fj8nG17C0ryQYaN7LJFIOBbCJblH9jjcgDoLN5gtip68HRuCDWIyCpsFC/Qq\nC5mdDPzO5DWDlXUyTrBhnztldjV1/aG+skYNqDLUSwaU1KySyaTcUJsbUGtwg9liaNVWAkCxWIQg\nCEgmkzWKJV5g1DNVZsH29PTY7qPpt0dpFiMlLHSdaVJupN+npNGujx3Y5DCg77W1YkArlYpcJqb0\nQLkBNQc3mC2EnrxdLpeDKIpIp9OIRqOy92YWN0OyTpe10DloJG9TDbUSlnw+L/8bqN9w2QrNZsiC\n+Hu0ypOU9b3s2rZZD5Q+Z3VwuQFVhxvMFkGZIKCsrYxGo+jq6grkQ+KHEEEjQ5Md9XVkSxR4CUsw\nsPqSpmdAqUWd0oCqvQioGdBSqYRisSgfhwxoNBpt+CiFU3CD2eQoQ7CsV6lnhPxIflGbRIwKEXDU\nYRVmeAlLsHDiPBsxoPQ9teQ+dizcgNaHG8wmRk/ezq1SDKshTrWHz6gWrJXj+RWG9XuS4SUszY2y\nvpdejCVJMhWeV6oQAZANKM0lsVhMXgNtFQPKDWYTQiG4TCYDAGhra5M/K5fLyOVyiMVi6Ojo0FXD\n8cuouB2CDcKDHZT1Ml7C8jlBuSZOwSZ8kWeovL5AfQNK50TNgNI+KFGp2Q0oN5hNBi3mVyqVGqNn\nVt7O7hjsbOt3CLbRk4DsYKaEhT53m2YzZADkbiVeohaeZ5ds2PVtqwa0UCigUqkgkUj0ycJtBprj\nV3AA9IZai8WibCzJYFarVWQyGZTLZXR1dRkylk6GVo1uV6lU0N3dLQsR+GEsm21itgsZ0EQigba2\nNrS1tSEajdaE+wuFgvzfrfyyEVT05PfC4TDi8ThSqRTa29uRTCYRDochiiIEQUAul6u5vmrQXKNm\nQHO5HL7//e/jrbfecu8Hegj3MJsAvdpKMkLJZBLJZDKQBoEdf3t7u6kQbDOUhTQS7BqZKIoolUqI\nRCKmQnycYKK3vs16oJRdXa+VGe3no48+khs5NDrcYDY4erWVpVJJrq00I29nB7MGjA3BJhIJXjLS\nYLDJHxTia9QSFi9Dv40QZraSIKb27AuCUJNH0chwg9nAaNVWsvJ2NJmZxQvPjc2C9dJQcq/UHXgJ\nSzBxyjgbMaB0D4iiCEmSEIvFkM/n0d7ebvv4QO/cdvDBB2P48OFYsmSJI/s0AzeYDYhebWWpVIIg\nCHJ3EbYWKyioZcHm83luxBqMehOxUyUs/L4IJmrXlxLDyuUyTjvtNOzcuROpVArLli3D5MmT0dXV\nZeuYCxcuxKhRo+QKAK/hCwwNBommqxnLXC6HfD6PdDpte73STtKP3nY0zmKxiM7OTtueJfcWGwea\nYNkkE0pAowSRfD6PUqmESqVSc12bzRNtxvAvW8KSSqXwpz/9Cddffz1KpRJuu+02DBs2DIcccggW\nLlxoaf/bt2/HE088ge985zu+PfPcw2wQ2D56AGqSKdjQplLeLkjGRE+IIBQKaWbhuQk3uP5htISF\nSjDcnvgbYV2xkUgmkzjssMOQSqWwcuVKFItFrFu3Tl4uMssll1yCm266CT09PQ6P1DjcYDYAytpK\n1quk2kooXPzgAAAgAElEQVQteTurOGlIuBas9zTiS4CWAaWIiiiKfTI0uYELFlr1paFQCMlkEpMn\nT7a038cffxxDhgzBhAkTsHr1apujtA43mAGnUqlAEIQ++o1G5e28njiVhjYIQgRK2DFS7WBQvXI7\nNLoxIQNaqVRkQ2pWpSaoNGNIVgsnjv23v/0NixcvxhNPPIFCoYCenh6cffbZuP/++x0YoXEa5w5r\nMcirLBQKyOVyNZJk5XIZ3d3diEQiukbI7wlTFEXXhQjsaMnm83nkcjkIggBBEORJuFkMZrNB2bfJ\nZBJtbW1IpVKIRCJ9iuwpQ5PjPWrG2YlrMW/ePGzbtg1vv/02/vCHP+Coo47y3FgC3MMMJGxtJa3d\n0N/z+TyKxSI6OjoMlYtYvVnthGRJ/cVMCNbLtUQK9UmShPb29prsTVJLEkVRrh/koT9/UQvzuVHC\n4rcn5gZ+vziUy2VXasD9uk7cYAYMtrM6e1NUKhXkcjmEQiF0dXUZCj/5dVNR0kZQQrAs5I2Q1yuK\nIqrVqhzyo/8Gen+Hmw2YOc7hVAmLV3htnP0K/+ZyOcdqMInJkydbXgu1CzeYAUFL3o7eEHt6eizJ\n23npYSqNUZDe1tnEo3g8LisjqUFrZ3rqNfXkwTj+omZA63Vh4TgP1YQ3C9xgBgCtvpWSJEEQBAAw\nHIJl8fKtkoxRIpGQjb4Z3Kr7pPGxiUeVSgWFQsHw/o2G/qLRaFO3v2pkjJSw0LWuVCpNcx299mSV\nx6NlmWaBG0wfUdZWssZSFEXkcjn5DdlqaNNuqy0j32GNUbVahSiKlo/pNCQTSAlSdus99TwXZfsr\n3j8yuKgZ0EKhUPNvpQauU7/V73VFLxEEwfGQrJ9wg+kTerWVbM1iPB6XDapZ3K7DVBMi8EN8QAtS\nj0mlUkgkEq5M7lqeC6vzy0N/wYeiCKFQSA7Zu13C4sXLhtfGWfkS1UzC6wA3mL5A8nZ0c7G1lYIg\n9KlZtJtB6rQnoCdE4GZo1cz4BEFAuVxGOp2WjZkXKA2o2sQLQC7Cb1QPzUv88GTZNmbsOjZFEthI\nQ9Cvo59j4waTYxm9vpV6snFWsethqnmLQRMiUBraarWKbDYrJx757dEpJ95qtYp8Pi8nEAUpc5Oj\njt46tpXr2Kwh2WZv7QVwg+kZbG2llrwdK0bNEhS9UzeMupOUy2Vks1lD2cRa59PNc01eCQC5oS6t\n+WplbgbtHDczWrJuSpwoYWlmlR9l0g9fw+SYolKpIJPJIB6P10yC5A0BcF0Jx07WqhktWDsGx852\n+XwehULBUDYxey78lgwzkkDkRuIJxznMlrC0ErlcjnuYHGOwIVgquaAJjxJSEomE3LtSCz89TK11\nVaexagjImIfDYcOCDkHFyPonTyAKPkZKWIDeOaCZIglqL+b5fB6DBg3yaUTOww2mS+jVVlLPPzO1\nlX5J3PX09JgKwXpp3CmUGY1GkU6nG2rSMeL1qyWe0AsYX/9sHJQGlK4fzRFul7D4eV/wOkyOLlq1\nlVQQLQgCwuGwqYQUr294dmJub28PXDsuNkQci8UCn6WoxMpY6R6iNW523YzVHaZJNwhr3k7h1aTv\n1XHoWrLr2M0QSdDyMPkaJkcVZRYse/NIkoRsNmupJtCO12Z2WwrBUtmDV8bS6DiVWbrFYtGD0QUP\ndt0sHo/3CfsBQLFY5ALyDYCbJSx+e5h8DZOjilZtJU3w1BkjaN4aC5sF29bWZlg+jsXNkCwlT0Wj\nUVtZusox+j2pOIEy7EfqRlTGAjjvtTTDefMDvfPmdAmLl2h5mNxgcmTYEKxWbSWFyaxOUm57mGpZ\nsFbVhdyiWCzKNV1OCCU0O2RAqT0cF5BvTJwoYfETHpLlyOjVVioNUE9Pj8+jVSco6kJax/NTtadZ\n0PNauID85zTCi5fZEhZa2/YCtWeea8lyAOjL2+VyOVSrVccMkFvbBk1dSAkJp5tNkrJDK3isRus/\nW1FAodGSi4yUsNCx/LiWXOmnxdGTtyuXy8jlcojFYujo6OhzYwZlIjYjROAXVKdqpQcoxxxak25Q\nBOT5WqlxlNcyn88jFAr5VsLCQ7ItjF15O6s46WEaFSLw2tOi41GdarFYNKzaY7ZDSit4kXYwKqBg\nVEqO4x8UTaDnyOsSlkql0lTqRtxgGoC8SlLmUYZgSd5OT2kmCJO0V1qwVuX4ACCTyQDQP5ccb1ET\nkKf1T5p8g5p0EkT89JjNlLCYNaBav6uZ7gduMOtAxrJcLiOfzyOVSsmfeRU2tOJFKbctFAqmQrBe\nG3hqOh2NRutKBTpJMz3MXsBOqHR/RKPRphCQb+TQr9bY/SxhaeTzqQV/hdeBEntovRKA/FaWy+Ug\nCAI6OjoMTfB+Jf1QGLlYLKKzs9OT9Uoz46VwNnnpXhpLjn3IgCYSCbS1taG9vV2efIvFInK5HPL5\nvCwT6XeUpRmRJAmzfvhD2+eWrmU8HkcqlZLrxkOhkJyfIQgCisUiRFHsczzuYbYoerWVlUoFuVzO\nUuam15MFecY0Vq/UhYyiVO3p7u42vY8ghLo5n2N0/bPRZN+cxGnPa8WiRYguWoSVxx6LKdOnO3Ys\nsyUsWmVhzUTr3a11II+MjKXyZstkMkgkEujo6DD1sHuZ9ENeG6nixOPxwL3liaKI7u5uhEIhzxtR\ncyPrDEYF5GOxGJLJJNra2pBKpRAOhyGKIgRB0PVYzBynVZEkCatvvx0LMhmsWrjQ1fuaXoa0ognk\nZJRKJbz22ms1kTk7FAoFTJw4EePHj8eoUaNw+eWXO/BrrMENJkO1Wq2pXWLDsBQy7OjosLRe6dUk\nTXWgFIKNRqO2BQjMUu+3FotFZDIZOezDJ8PWgKI1aiE/ygcQBAGlUgmVSsXzl5pGNMwrFi3C8Vu2\nIARg2pYtWLl4sWfHVhpQWu8sl8s4++yzse++++Lf//437rzzTrz22muWr2cymcSqVauwceNGvPTS\nS1i1ahXWrFnj8K8xBjeY6H1QyuWyLFrNJiqwnpAyPOsVRo2tKIro6elxxGtzY+KgEGw+n0c6ne6z\nnhoEzy8IY2gV2DUz8lhISJ5d/wTQVOufThlm8i6PFQQAwHGC0MfL9PIlgK5ne3s71q9fjyeffBID\nBgzAP/7xDxxzzDEYPnw4/vjHP1raN4kf0MvUgAEDnBy6YVp+DdNIbSVlluqFjerh5kSsJ0RgJ8PW\nSVjVnq6uLkeVTux60M0yETc6euuf1Aig1dc/WVjvEkCNl6lcy/QCpXHu7OzEiBEjcO+990KSJLz1\n1luWkw6r1SoOOuggvPXWW/j+97+PUaNGOTVsU7T0HSeKIgqFQh9jSbWVpVLJscxSt7JklSFYJ7Ng\nrY5ZuV2pVEJPT4+89huUsJcoishkMsjn8/JaWjN5Mo0OrX8CvdnTqVQKkUjE1PqnURrtmiu9S0Lp\nZfoZZmZVfkKhEPbdd1/sueeelvYVDoexceNGbN++HX/961+xevVqB0dqYhy+HNVnKATLZsHSTVUu\nl9Hd3Y1IJNInrBm0cJ2REKyfYybhdCq/qbf269VYaQy0jppKpWR1JqpB46UQwYDOPVsvWK/kwc76\npxfGxQkjpvQuCT/WMgnl73JDR7arqwtf//rX8cILLzi6X6O0XEiWNBXVQrD1JNn8qqVUbuuVFqyd\nMVcqFXn9ySvhdCOQEQeAdDotT7hsc15Kl1eWQrD6mxxvUZ5zvZKHVhCQX/vUU0hOmAC11BcJQGHZ\nMhx90kleD6sGp5pHf/zxx4hGo+jXrx/y+TyWL1+O2bNnOzBC87SMwWS9SpKHIqi2MhQKuSrJ5oSn\nYlQLlvDDw6SXj0Qi4boQgZnfV61Wkclk5OtL7Y/U9hmLxRq6FVarecXK9c+gCcg7zZy77qr7HdY7\n9wKlh+lU8+j3338f55xzDqrVKqrVKs466ywcffTRtvdrhZYwmGQsc7kcJEmqUc83I29n10u0Ch3X\nKy1Yq1CiVKVSkevuzOCmcS+Xy8hms/J13rVrl+ExGfVkyEMNynUJyjj8wKiAAkUNvKRVXmac6oU5\nZswYvPjiiw6MyD5NbzApfEmdFSqVivx3s42J/QrJAr3jzWQylkKwXoyZ7QMai8UC06GAzXZWC7Vb\nqadVejKiKNaEoJUGlOM/egLyFDUAevMCvAi7e7VW6hVq3myz9cIEWsBg0hs//UMTXC6XkxN7ghye\nIUMEwFJtpRcPJuv5dnR0yOP1G6X0nvLcOTGhKMO3dH85IV4ddBqx0B9QjxrQ9Wp0AXklfo7bKQ8z\nSDS9wQRqPaVKpSJ7amYl47z2MFlDBMCyYXdzzMViUX6TdCL5yOxYtcZI11krfK113e3UrdJLGWXc\nkiej7ORBYd0ghW+DihdGmQxoKBRCW1ubZtjdjabLzYLadSoUCthtt918GpE7tIzBpJu/Uqmgq6vL\nUsjQibCq0e8ps2BJhcgsbj3YbEjbqfIbp8ZK69KpVArJZNKRfVpBb/2zVCqhVCo1VSJKs2B0/dPq\ndfPKM/c7AiAIQk07xGagJQxmuVxGJpOpucmt4nbSj1YWLBkhrx8ALe8tm80iEok4qtpjF8rOLZVK\nhtelvYQm4nA4LNcQOtV7kGMPvWdLrelyq4TdjaJ2/nhItkEhebtwOGxrfc0p/Uet/biRBet0Zi/r\nvdGk7yRWx0rqTECw6j710EpEabZ1tGZCK+zOXrdwOCyHb/28bn5n4zpVVhIkWsJgptNp+aa2cxO5\nVVZiRIjAb5UhM96blyFZWnPs6emRu2AY3Y/fISsWNnxLAuRcRzX46F03rbIjrwXRvYB7mE0EXUgn\njI4TBpe9sdhyDDf6QjqRqETeWygUCpz3ViqV5Npaeus3QlAMpRbsOloikZBf+HgYMNjorX9S2RHQ\nu0xE4flmhZeVtDh2JyWl8VKWY7glmmCHSqUCQRAMCTuwuD1WNumIDZE1K2bDt81CoyfIqK1/Uo5C\nqVRy9cXH7wgKK77eLLSUwbRrdJwyWla1YJ3oHGIUCi1RCFZNW9fpYxrdjm0V1tHRIa9dtgr1wrd0\nDlvBi2kkWOOVSqUaVnZRDR6SbULogtp587JrcCkL1mwI1u6DY+Y3s2HiZDJpyli6jVLijt7a7dAo\nk5IWauFb1ouhz3n4NliYkV30O4HICjzpp0FhbzI7i+5OGK1sNot4PG66L6RXiTRsmDgWi9lKxHES\nLYk7u3WxaoIGfmcX2oU8SqpBVfNinMji9Dvk12wo1z/tCsh7eX3UjkUZw81ESxhMp7AT3qQ3Riui\n5F6gFiYWFM1p3Ubr/NaTuLN6LC/w26gY9WJYA9qKBHGt1KiAQlDbzjWjIlLLPR1ey9tRhmmxWKy5\n+b04ttFtySAVi0V0dna61l/TCpVKBd3d3XKGblBE3Y0gSRJm/fCHgfJYaRJOJBJoa2tDW1sbotGo\nnMWZy+VQKBQgimKgxs2BnDxEL93JZBLhcLhPA226dn56mM1677SEwVQLyXqBKIro6elBOBwOZDsu\noNcg9fT0AOgr7u528k49SqUSenp6kEwm0d7ernr+ghxCXbFoEaKLFmHl4sV+D0UTo5Ow3Rpmq/jt\noQcVihxQ7XF7e7v8okviItRFh18752i5kKwXHqZWFqzX3m29bd1W7bECvRl7KXHnxoMtSRJW3347\nFmYyuHjhQhx90kmBOL96mElCiUajgX1RCTJuGRG1a0cZ0/RvNwXkJUlqiXB+8/9CF9CbKNgQrFp4\nMwiTDIVgBUFAOp3WrK/02nuj42UyGYiiiM7OTk/0YN2YwFYsWoTjt2xBCMC0LVsC7WVqoRW+rVar\nyOfzkCQJpVKpKcK3zeYNkUGMxWJob2/vE3oXBAGFQgHlctnxBD2geQ1o8/2iOrglbwf0DcEq19vs\nPJBOeZjVahWZTEYua3HDIFkda7Valdd60+m0qw+cmxJl5F0e+1nS1HGCgFULFza8UVGGb0OhUKDC\ntxxtlNculUohHA5DFEUIgtBn/dMsymepWCw2pZhIS4RknVzDVJtojQoR+L3epqxhDNIbNTXvjcfj\nprKInaitdRrWuwRQ42VOmT7dz6GZQu+ckgdDIT4jGqpBuT5+4uXzr3X96Fq4KSDfjLJ4APcwbW9f\nLwTrFHbHXSgUkM1m0dHRYVig3M4xjW5H4eF8Po9EItHwYRyld0k0mpdpNsNXGb5tb2+vCd+yIcBG\nOQduEbQXBzaBiK4dKUgVi0X5+SRjqlX6xf4ubjA5fagXglVi1+hZDXPS22NXV5cnqj1GJwTK0K1W\nq+jq6gqEsbR7jZTepbxfNNZapt0MX1o/UwsBqpVAaBHE+shmp97atZH1z2ZU+QFaJCTL4oSHWa1W\nUS6XTWvBAu43oGYh1Z5QKCSXC5g9plveQJDCw05OlmufegrJCROwRu04AArLlmHS1KmOHMstjGb4\nGj1vWiFAURR5708Xceq+VhOQV3bOoZdyuna5XI4bzEbF6TpMyhA0W0jvVdIPu6ba3t6OYrHo6QSk\nN1YtiTvazkrGnpUEHnaMTp6bOXfdVfc7XisomUUtw9fJtVet8hU1CbhmC982uidLLz/Kzjn5fB6i\nKOJf//oXLr74YkyYMKEmL8Eq27Ztw9lnn40PP/wQoVAIF1xwAf7nf/7HwV9kDv9jYD5g9SGkQmCr\nqjNeJP0oVXvi8bjvAgTKsZVKJXR2dgZC1J00VnlmZy9+ZPhSCJAEKlKpFCKRiOyBkifTDOUrXuKF\ncaaXH6C3A8t+++2HWbNmoVgsYsWKFRg0aBC+/vWvY8GCBfj4449N7z8Wi2HBggV45ZVX8Pzzz+MX\nv/gFXn31Vad/hmFawsME7JURsB5bJBLxpcjfiPGitleRSCRwykKVSgWZTAaxWCwwY2NDSuTZhMNh\nz2XFgkQQMnzZEGCpVJJfUt0M3zZr3aDXJJNJTJkyBYIgYPTo0Tj//POxatUqrFy5ErlcDoMGDTK1\nv6FDh2Lo0KEAgI6ODhxwwAF47733cMABB7gx/Lq0jMEkzHpNbKurzs5OORzrxbHNUCwW5cw05Zqq\nXwIEhFFFIa/GyYYA29vbZeNIa9OVSgW5XK5hexNaRV67VMnw9UutiF3/VOv9CZjr4OE3zfgipvbM\n0lw0cOBAnHrqqTj11FNtH2fr1q3YsGEDJk6caHtfVgn23eUCZiZls1mwRnDa2LJlGel02tGyFiey\neqkomsb2kwsvdNwoml3fpRegVCpVExam2rNwOCyn1gOoSa13ShkliKHFIGb4Kg0Mm8HJhm8rlYoj\nBfjNhNfGWVlW4mTz6Gw2i1NPPRULFy5ER0eHY/s1S8t4mOykWu9B0hMicFMpyCwUgiWDrvV27Ydg\nAkncAZDH9vSjjyL86KNYcdxxmHryyZ6OB6gNWdfr9UkTs15rpWZrymwkwzdowgvKDE5lAT7PvvWH\nQqGA3Xff3ZF9lctlnHLKKTjzzDMxY8YMR/ZplZYxmES9B0YZglWTt/MjJKsV5nSzLMPqeCmBJhqN\nyiIJkiRhxW234bZMBjMXLMCUGTM0SxXcQFnGksvlTG2vNTFTU2YzE3NQJ20jGb5Bhs2+VQvf0j2p\nFr716oXSy7pSr1D7TbQE48S+zz//fIwaNQozZ860vT+78JAsgxshWKehMGculzOs2uOlh0nhSwCy\n3igALP/LXzBt82aEABy3eTNWPPaY6jidhspYjKocGTlXrDIKtVYiQ1ooFLiqTUBQhm/b2tpqwreU\nTU7h26C+yNjBr99EJW12Wbt2LR544AGsWrUKEyZMwIQJE7Bs2TIHRmiNlvQwlZOYUS1Yre3tHNvM\ntiScDsATZRyza4OCIKBcLiOdTss9NumzFbfdhtuYUgU9L9OpcdJ6ZaVScfUFiA3fJhIJ2ftks3DZ\n5CGOP+hFCUj4n8K4PHxrHLWXDacM5uGHH+5KNxWrtMzTy15QO1qwfhlMNsxptpOH2x6mUuJOaZhY\n7xKArpfp9JiAvo2x3YYmZWVjX/K+yZgGaSKwghdemVv3rTJKQAlfSv1Up9tftYrUX7Mq/bSMwSTY\nm6hRQrCFQkE1zBkEyuUyenp6EI/H0dHR0Wds5F2qiZEvX7CgZkJ0yrCzY2pvb1c9X16FqWliZkXJ\nKVqQz+f7hAU5ffHqfmevk7J/ZC6XQ6FQ4NdJBS0PkxvMJoAmykKhgEwmI3sBRh9KLz1MVrUnnU5b\nOqaV4xrZTpIk5PN53bVBSZL6eJfyvuG8l8muV1LJQZBeLgDIsmLUGYIStqinJHWF4MpD3sPeK8r+\nkaTF3Ci9P/0eUz6f97X8wy1abg2TbiQKwfohb2ckXELC6aSMEyQkSUI2m9XNJCb+umwZkgcdhL8p\nfm+lUsE7736AV340HzfeuQWJRAVjxvTDzJmnmf695LGRpxbUaIESLU1Vu0X5bmjktjJ618ls708v\nQ6VeHUcrS7YZPcyWMZihUEg2QgBMrwM6NQYjqKn2sDWkVuX9rMIek5W4UwvBKpn7y1/2+ZsgCJg2\nbRbWbX8YwAjg/d6/r1jxDp57bi6WL7/RVEo6JRwFURLQDEaTh5qp9rMR0avRzefzACAbz1a9ToVC\nwZGykqDRMiFZNgRr5wa262HWC3NqqfbYHbMT25VKJfT09BgKY+v9zuuu+y3WrbsMwAjFJ3vhhReu\nwNy5vzY8RpKyi0ajhgx4I6GVPEQ1uGxTX45/KMO3lETkZ/jWS09W61jNmBHeMh4mhTYjkUgg9WBZ\n1Z6uri7dRBWvPUwKd5ZKJaTTafnN2iqbNu0EsJfGp3t99nl9CoUC8vk8wuGwJUF8v9d5zKAWFqTu\nOUqvppkmqkbLKmW1b2m/yvAtveTUC982Os34u1rGYFLKOOCPVByhdmyj4uR2jmkHytDVk98zQ7Go\nv75YLOrflmzNZ2dnJwRBMH09tc6Jn/eGGUKhEGKxmGpNYaVSAdB7XzVDTWGjj50N3+ZyOUSjUTlL\nGkCf9c9GQ/myIUlSQzxDVmgZg+kUToZkKdO0VCr1aabs9LGtbCeKIoDeB9psOYveWBOJiu62iYSo\n+RnVzYZCIU1PvNVQep/ValV+iWjEjh7NDFv/SYZFFEXH16n9rsMEGvtFR4uWNJh+iQ+w0MQPmPPc\nrJaHmKVYLOLjjz/Grbc+jNdey6FUiiKRqGDcuAG48srzbS3ojxs3ACtWvAP1sOxWjBs3QHU7StpK\nJBKO6uc224NNvyeRSPDkoYDBGjK18G21WpUbZzeKeLwk9e0lGsRxOgE3mBaw62GSYHcikTCVhOTV\nWo4gCOju7sa3vnUj1q2bBTY5Z8WKd7BmzeVYtmy+rtHUO8dXXnk+1qy5/LPEH9ZovoODD74OV155\nU59tKHOYbbvFMYaaJJzapGxmTc3NkJsf3lFQPDK18hVRFOUG50YjBX7+niCcS7doGYPJXkC7HqZV\naLIqFAqWJn43BAhY2MSjX/ziUaxbdznUMlnXrbsMc+f+Gtddd5HpsQBAKpXCsmXzMXfur7Fp004U\ni1EkEiLGjOmPiy6aVWOIleuVavWVVs9Ls66z6GEmeciI9+n0xChJEmb98IeY94tfNO2kaxSt8pWg\nRwrIGWhGWsZgOgXdlGbfoqhtmCRJSKVSgfOSlO2vnMpk1SKVSvUxuNVqFd3d3TX/T+uVTiUcEY2S\n3OM2eslDhUJB1lj1KiS4YtEiRBctwspjj8WU6dObzluxc8+Z6f3p5b2tvEZON48OEi25+u+UWo9R\nWM1aO5lwXkrc2c1ktXuO6ZxRfSVPVHEfNiGFaj8pOYUEyaltmRu1n5IkYfXtt2NBJoNVCxc27QuN\nU+UrdK1Io5htMUfZ0m5dKz0EQWhK0QKAG0zL2xulWCxa0qz1ApK4K5VK6OzsrMnStZPJahW6LnTO\nSATb7XPWrBOzXSgkyAqSUz9JCt86KRy/YtEiHL9lC0IApm3ZgpWLF6t+z+nr1QxSguy1am9vl1/M\ntXp/Oomah9mMsnhACxlMp9YwjW7PqvawbcP8yNBV265SqaC7u1uzU0tvpuo7GnvUzmS1A42RlI6M\nhq15eNUblIo2QO+5V1MeMns9yLs8lumZquZl0hpnI15vr8ccjUaRTCbR3t7eR+RfEATXRP65weTU\nUG+Cpl6MkiSp9of0CxqzEYm7K688HxMnXo++RvMdTJx4A6688ju6xzJrxJRlNnbVhIygNj52jZqj\nDZ0nZUiQCvIFQTDVDov1LgHIXubqpUv7fC+6aJGm99kIeO3Juh1qb6U1zJZK+mGl5dyK69dT7fHL\nwwRqhRLqSdxRJutVV92JLVt6UC7HkUiIn9Vh6peUmIXqK+PxOERR9GRCaeTwWxBhMzrZgnwjyUPk\nXS5U6Zl60Z13YuqMGbXfy2Rw8cKFOPqkk/h1tIBW9i0lEAGQr5WV7Ntm9jBbymASboRkjRojP8OH\nZoUSUqkU5sz5HmKxmGtp4sr6SlKm8QqqO2y2bEw/MaKnyk7IKxcvrvEu5f0AOP7VV7H68ccx9eST\nVdc4p0yfbnu8zXjtzfwmrexbo5nSymM1a/NogBtMR7Z3s/xB77hGsSNxZ5V6YzXj7do9Fsdf9Dya\nYrGIZ5cuRfu4cXhORTShUqmg+NRTmDJjRo0XepwgNJyX2QiGWUs8Qdn7kzWgSnhItklwY2JV1i8a\nKfS2Ew42O37q6AHAdmszp9CTBfTD26Pr4dUxvTLwbp5HO/tWejRX3323XJCvVB7K5/NIJpOaa5xO\neZnNhlPX3kj4Fuh9KafMXEEQ0K9fP1vH/fa3v42lS5diyJAhePnll23ty0laMunHCQ+TkhuU9YtG\n8EJliLJ0C4WCbJTcVAkyuh1bX+lHE28WUhGif/L5vCxB1ug0SjapVj0hPV+UmLJq4UI5g5bQyqTl\nuIcyUzqZTALofa7/8Ic/4Ktf/SrWr1+Pd99919azdN5552HZsmVODdsxuMG0CPWHVNYvGjm2VcxI\n3PX09KBarQYqS5etSfUqNKxHuVyGKIpob29HW1sbotGo3BqLmv66UbfmBauXLnUkm9Tr304eDU3I\noeIUTpsAACAASURBVFAIzy5diuNffVV1jVOvXtMoXkU0GiEkawZaqw6FQkilUpgxYwauuuoqCIKA\n+++/H4MGDcL06dNxxx13yMtCRjniiCPQv39/l0ZunZYKyRJ2DCaJVpOHZPYBcDscpxUidluHVg8n\n1yvVMDtGKrwPhUJIp9PymzBJxFFHFLbZLxsmDLrqkCRJWHv33bjDZjap37quNBk/v3IlUhMmYC0+\nN+CUiYtQCIUnnsBRJ54Y+OviJX4Y51QqhaOPPhpr1qzBzJkzMXr0aKxYsQLr168PzEu7XVrKYNq5\ngSg0lM/nEYvFfGn2Wk/irlAooFAoGOqt6TY0VlZD10hClFcvFHR+tK4hGUeqV1MKlNtJu3eblYsW\n4YTXXrO9zqfUdfUDSZIw+847Ve8bdj1NEATZOw3qdWlG1AwzlZUMHjwYZ5xxBs444wyfRuc8LflK\nZnZSpvXAYrEoF9V7XUtZb3xaEnd2j2tnvNVqVdbQDcJ6ZaFQkNec9V4olL85FAr1WbdxSuHGaSRJ\nwuqf/xzH1VHMMbQfn3Vd6x2TXU9j1WyU18UNNRsreOn1+Rn+beY6TG4w60DrgQA020uZxUljW0/i\nzi8oDT0IGrqU3EMvPHa873pJKmYVbpxGL5vU6n6cWCc0i9mkJb3rUigU5OtSLpdVa6i5N2oNteuT\nz+ebtqykJQ0mUe9hJAk5EjS2ux5I2zqFEYk79rhelTIIggBRFBGPx00LHlgZp9421WoVmUwG1Wq1\n5oXCqfOhTFJJJpMIh8OyZqeXXo5Sj5Uw62Ua1XV1EwoHP6uQxjOK8rqkUimEw2GIoihrqRaLxcB4\nn07iR6IWixN1mGeccQYmTZqE119/HXvuuSfuvfdeW/tzipZcw6xntGjSL5fLqkkqfsjbsdu6nUSj\nPKbRulF2vTIIDWRZyT0valC1ir5JIg7oXRutVquuhKeV3qU8Lphby9TyUlcsWiTL1LkJK4F30V13\n4bhTT7W1PyPKQ0Dv+rZWMb4TeO3J+uU1O6H08/DDDzs0GmdpKYPJolUg75Vqj1XIYwKMS9wB7nuY\nlUoFmUwGsVgMbW1tKBQKnvfhY6F1rLa2NsPGWxAE3HDD7/Diix9DFONIJCoYN64/LrvsXEvauVr6\nqlT0TcXeTjVnXvvUU0hOmIA16L0ebHheAlBYtqyuwdTTdT35kktw9Eknuf5MsAb7+NdewzOLF2Pq\nySc7tn9lMX65XJajAE5oqbYSanNooVBo2n6YLW8wWYyq9vjlYVarVVSrVc88JsDYeNWMk9Wx2TXs\nVr1vQRAwffocrF8/C8AI+e/PPPMO1q79GZYsucbWJMB6OeRhhsNhVCoVWT+XLV2xcv7m3HUXgM+T\n1Do6OkzvQ89LPe+TT7DwqqtwyXXXueYtKQ32NEHAxT//OabMmOHq/R6JRJBMJmu0VJUlRU692LhN\nENZkg+ZoOEVLGUz2JmInZrMlGX4YTJK4C4VClsIdbmXnehUa1oMNVWezWcMlLOz2N9xwP9avvxys\nsexlL6xffznmz78P11zzfUfHTF4O1XyKoghRFFEsFusKXrsF66WyvL1pE/bJZvHXhx7CzLlzXTu+\n3xJ4bFidSooofGv3xSYIhsxp1H5Ts60Js7SUwVSDXXfr6uoK3JsRu57a3t4OQREqcxstQ1vPOHmV\nZMSOp6enB5FIBB0dHYYmJnZ8L720C8BeGt/cCy+9tNOZgWoQDodV19ic8j6NQl4qy/LHHsPhF16I\n4wA8mc9j5eLF+NoJJzh+bL1wsF9C62ovNrQuTS82rKBFsxlEszSzsQRaOEs2FArJuqaRSMRUnaAT\nWbJWJO6CUv/JlrK4UV9pdpzUikiZzayH8jvFon45TrHo3bslTdL0e1KpFCKRiC8Znl5mzBpJWvIb\nqv2kzHRagqBGzKRH7Of6vd/1nrT80Iy0pIdJ6xRUL0Rv9kbxwntSW08Nwtsb2yCbhJedxOyDRqF0\nKiGwSiJRqfO5OS1MJ1F299DrLen0y4tWiHTVkiU47NhjHT2WWjiYkpfYpCWnDYLV/Smzous1Yg7C\n8+sFzfw7W8pgktHJ5XKoVqtIpVKmjaWT41B7SPXWU51IiLE6Vi/XeY3Ahqrb2tpsdxkZO7Y/nnnm\nHaiHZbdi7NgBtvbvFMoMT60QIWDf09ALkf7Pz3+OScccY/2HqKAMB9OzykYNSNDAL31bPeo1Yg6F\nQnKil5vhW6+XQtiXNLdKpoJC8/4yFVjVnlgsZvnC2jUG9dYF63VBsWr4rELjKpfL6Orq8l2nVilG\n4MQDetll5+CQQ+YDeEfxyTs45JDrcfnl59o+hhtohQgB2A4R6oVIj9+yxbKogFnYe1fWtw1AeFYP\nVnmIrg212FMqD7kRvvXrZaKZS0qAFvMww+Gw7FVSoo8V3PCe2DpGraQVuw+BlTHTW3I0GvWkJVe9\nc+ukGAF7nFQqhSVLrsH8+fdhw4aPIIoJJBIixo4dgMsvt1dS4hU0SZPSUFtbW034lrxPo/WFWhmz\nwGcvUcuX4/hvftOdH6NxTBI08CsJyCrkXbKlRc2QPKSMYjSzjizQggaTQrBOGD07ax/ssa0U2Vs5\npllKpZLcBcKs1JUbLxVOnie185FKpXDNNd+HIAhIJBKB0eU1C513tRAhtaej+kK2dEWJWsYsQUbY\nS9T0bZ0oNfEqhMmGL7WuDa1L17s29Y7jl8Gl57NZaSmDqcRPPVirEnd6659Gjmn0e7Re2dbWJre0\n8gt2PE5LFQqCgGuv/TU2bdqJYjGCRKKCAw7owBVXfNtS4X9QUMtcVMr2sapDQLDVbZTrqU6Xmvj5\nexv52ijnonw+3xDRGKu0nMFkDY4T65BWPUyrEndWMTpOWq+kUha/kozYfeRyOVQqFcfrZAVBwOmn\n34B162oFC5555h2sXz8Hjz9+bdM+/KFQSG6Yraduo9Uw2+tMSL8FDbxE69pQ8pBfohZGaOZOJUCL\nJf2w+FmmQRme0WjUdB2jnXHX246SopzQ0XXiIVa2VnP6peKWWx7CunWXQU3d5x//mIX58+9z9HhB\nhU1QUWtZRr1glS3L3J6oWSUuJ7qw+I2VF2y15CFSINJKHvKzDjObzTZ1SJYbTI+3LxQKqFQqSCQS\nlpJorB633nHK5bLjrczsQOOJx+Ou9dPcvPlT+KnuE1SodEWvYTYZT7fvjVAo1BCCBl6hJ2ohCILc\nWs+vZuZOdCoJMi0XknUKs4aEvEpRFBGLxXxJKNEqZTFTX2kUO4a9XC6bEpWwcqxQKFRXvcdNdZ9G\n8Yq0tFXJoxEEoSZBxY0XG91sXRjrwqKHspawkVBLHqLEoVwu10dS0W3Ps9lDsi1nMJ1awwSMT3qV\nSgXZbBaRSASdnZ0QBMHzkhYtkQRaH2SbKztxPCtIkiRPxFrjcZJ66j1uqfsEac3JLOThAL3XK5FI\n1KyvudHZQy9bt5FwO1TKvtzQOiglD1HintMvN8rfxLNkmxQnQrJGMNoyzAvY36s04n5P4tSHlCZh\nLzzw0aO7sHp18NV9gopadqeaaLyb3idHG7eTh9Tmz3w+j4EDBzr1EwJHY8YhHMDtNUwqGclms+jo\n6KgpsrdzbCc8TK31Sj2shDzNeOAkgu+litD/+3//hYkTr4eaus+Xvzw/sOo+QYVdX2tra0MqlZJF\nFEg0nho1N0pIuhHREkTXSx7K5XKWlYd4WQnHEFoPPVua4UVo0Sj0cOTzecPrlW57BUoxgnw+76px\nZkmlUli2bD6uvvqXeOmlnSgWo0gkRBxwQBqzZs1u6gffbWi9zE7LMq+yPZvtOEZgNYn12paZrf0U\nBKGh65fr0XIG06nsT60byKjEnR+yfJQQEAQj7laykVHoPKZSKVx99YWy/B/Q+5bst15us2Fkgg5q\nbWGjYcUwG1GFUiYPqR2HS+M1KW6EZI1Kt5FwgVfQeiUAS+uVVkQa9M5vvWQjL8N1xWJRTsKiSd2L\ncolWx0jLMj9rpVsZrbVpURRrkodITJ6l2ctK+BqmDdjCaqqBSqfTrunBAubHzdYz0vZ+Uq1Wa8QI\nlMbSy4JrWmem9TZKyiJPPAjNgIOKk4ZMufbZ1taGSCQir3V62TDbTYIUkjWDsi6X1qZFsTeLXBAE\nvPjii1i/fj0EQXCkrGTZsmXYf//9sd9+++GGG26wvT+naDkPU3nD2pG3Az7P7gSMq9HYNdZGtpUk\nSZ70Ozo6EIlELItl21UXonNFnUYSiQSSySTy+Tyuu+63NTquo0d34tJLzzT10FmtiQV6rxkVedO6\nDRueVXb6aLRuEm7i1jkg7zMcDqNQKCCRSNQIkzvdMLtRDZkWbv4edm06EomgUCggHo9jy5YtuO22\n2/DBBx9g586d+M///E8ce+yx2GOPPUwfo1Kp4L//+7+xYsUKDBs2DF/+8pdx0kkn4YADDnDhF5mj\n5QwmYfeGCoVCcnanE62mzBy3HmohT6/DjMpxUuiTsvMEQcC0abP66LiuWPEO/va3a/D00ze4knTD\nltMAvZOz0oOksStT8p3qJsExjjI8SGufWi8ytA3HfeglMxqN4swzz8SZZ56JU089FUcffTSefPJJ\n/PjHP8bRRx+NRx55xNR+169fj3333RcjR44EAJx++ulYtGhRIAxmSz/pdjwnEqqmEJJT63t2t60X\n8vQ6pEXeXD6fRzqdlkPD1133W10d17lzf+34WERRlF9wzKyz0KTNhgyj0SgqlYocim+GkGHQUPOU\naN0zmUzWNMwuFovIZrP4yYUXyokqQcLL+8JPj1kURXzve9/Dn/70J3z00UdYsGCB6X3s2LEDe+65\np/z/w4cPx44dO5wcpmVa1sMErBkuMgDlchmxWMyQdJsTxzWCnkiCnQfIzni1wtWbNu2Eno5r7+fO\nofRw7Zz/er0MyfPUKpfwk2YKPyq9z6cffRSJJUuwcupUHHHccZZLI9wkCGNwCrV7qVgsynNiJBLB\n8OHDTe83yOeo5TxMOxeDQrBUjuDHhVUzXlSikc1mZUFmv2+6SqUCoNe4qHVkKRb1y1rM6LjWy8hV\n83CV21tFzfskMeygFetLkoRZP/yh7+Mwi9E1+2fvuAO3ZbNYc+edcqZ6KFQrGq+WxNVMLxF+44Qu\n77Bhw7Bt2zb5/7dt22bJ8LpByxlMFjOek7J7hlpKtRvHrQebRdjZ2anr8dpRCbKSmQtAM1ydSFR0\n9+GEjiut5YqiiM7OTkMNuu1CnqdeKya/OkmsWLQI0UWLGqq7h1Ejz3Y0mbZlC55ZsqRPyzIKo+u1\nLHPzd3hFowsxHHzwwXjjjTewdetWlEol/PGPf8RJJ53k+HGswA1mnRtZT+LODk4YW1qvdEpRSBAE\nXHHFHTjhhGswdep1OOGEa3DFFXfItVf1YD3djo4O3ZeKceMGoK8kHbH1s8+tw67lmu056hRsuQR5\n/pRkVC6XPfU+Jam3p+SCTKahekiufvzxukaefhv1y1Trk0m6qmoty4Detnu09unWuWl2L9YpAxqN\nRnHHHXfg2GOPxahRo3DaaacFIuEH4GuYdfVgSeKuq6urZtK1m7hjFyui7npj1staffbZuVi6dC66\nuro0902eLnlz9Yz3lVeejzVrLv8s8Yddy3wHBx88D1deeWPd36OFsnxF79x4aTjC4XDN+imVsHgh\nVK70wFYuXmyrJZYXSJKE5+66Cz/PZHDxwoU4+qSTVM+Lsl9mvd/Irn3G43Fks1lEo1G5YTaAmtKV\nRjJ0fnuyThnNadOmYdq0abb34zQtZzCNXsx6End+ydtROQutV1pJOlJDL2v1hReuwPz59+H66y9W\n3ZZqUUOhkGElIdJxnTv315/VYfbquB54YD9ccsmVlkpKJEmShb7rnRunJ0FBEHDDDb/Dpk275JrS\nceP647LLzlX9LaxUHBlRURT7tMlyoo8heWALGQ9MzwAFhZWLF+Prr76qawCVv40w+xtjsZj8XCq7\nenjdU9IJGmGMjUjLGUwWLcNlVOLOayjkWalU0NXVZToEq2eo62WtvvSSetYqeXNqtaj1XgxSqRSu\nu+6imr9R82gz0DHz+TxKpRLS6bQn65WEIAg46aTZWL9+FtgXjmeeeQdr1/4MS5Zco/sCwBaDA+pS\nZHY8HrMemBG8CCH/9Y47cHsdI6/8bYTV32hUFs5sJKAVEouoPrmZ4QaTefBpvdLIpOu1h8kqCrEP\ntFNYyVoNyosFnctyuWxYbUkPs9fnhht+h/Xra0PZveyF9esvx/z59+Gaa75v6vjkfbIej5oQtvK3\nCoKA+fPvw8aNn0AU44jHRfR79SH82aYHpjVOt1ixaBGO/8y7BLQN4NqnnkJywgSsUdmHBKCwbJmt\n0LPyWmhFAoIiGu+lYVZmxAqC0PQdflrOYKrF3AHzEndOadEaubnZNblIJGLaAyP0xmwma9XMi4WV\nc2RmG/a6+ZXcs2nTLljxzo2gXG+jCZuyPYHPvc9isYjp0+fUeLpt+F/cj9sd88BYBEHAvHm/NRyG\nNnq/mwmzzrnrLktjt4JWJIA3zO6l2TuVAC1oMFmUGqdeSdyZ2b+y4J4Ej51m3LgBWLHiHahP/Fsx\nZkw/AJ8nQkmSVPfFwo5GrxHY62b1vDgRWnSyprQelO3JiiaQ9zl37m/6eLrtWIrb8WX8HEUMH/Yh\nRo78XNvTjgcmCAJOOWUuXnjhShgJQ1N5yLxf/KLuNXY6zKoHXX+r96qyZRnrfbIty6rVakOXehg5\nFvcwmxTytEKhkHxzm02gsethsmNQg1UUYrNO3QoF62WtfvnL83HZZVfX6LBq9fr0CmU42IqwvFPj\n96KmVA2l9/nqq1koX3g+wr346LP/PupLl+Gux65w5Ng33/wgXnjhChgNQ8s1oMceW9fYUZj1uc9C\noOxLmRNhVregLGigr/fJzjdBVIBygnw+70inkiDTkgYT6L2haU3ITg2jG290yqxTL8KMWlmr48YN\nwI9+NBuxWAw9PT1IpVKygko93JAApMSnQqHgeXKPFuPG9cczz2h752PH2qspNUo9TzefD6NcLjsi\nGP/yy8bD0HKItU55CEFh1lKpBEmSApV4ZxSl91kqlVAul2saZjd69xs1D5OHZJsQKhkBPl//MYvd\nG1zLmNQLD7uZbKSWtQoAPT098nplLBazdGyj1JO502s87ReXXXYu1q792Wfh0Frv/JBDrsfll1/j\nyTiMeLpOTdj1wszs517UgAY9C5WiAclkssb7dLplmZ/ngRvMJkSSJGQyGcTjcYRCIVtrgvXCqka2\nZVGuV/oNhYUrlQoSiYTrxlKPerWeftXFAr0vGkuWXIP58+/DSy997p2PHTsAl1+uXlLiRllGPU93\n/PhBSKVSuhO20XBhvTAzfe5FDaiZ9VG1bZ1S7jK6H9b7BD5vWdZo3qeah8lDsk1GKBRCv3795JCs\nE+uQdmHXK4NSzsIaKKshMafOD0UE3EjKoofe7j5TqZSp0hE3MOrpGpmw2V6faudmzJj+WLWqfhja\njRpQJWbWR93AjsEGjHe/CXrv1VbwMIN79l2Ebmo31tjMjIEejkwmI6+lGl2Tc3Pc1DcyGo26ktyj\nN3blNSmVSvLaqdm+o/Xw683dreOSp3vxxQ9j8uRLcdhhV+Kooy7DzJl/1BVPMCIYrxQpv/TSM3Hw\nwdehrx4wGedz+2i8Empar1YJgkauEVF7ox4ohW6V3W+UvVe1ROP9zJLN5/NNbzBbzsNU4qeHSS2g\nzHhOdh+GeuNVEyMIhUKONeSVJAk/ufBC3Hj33bq/RZIkFItF5PN5dHR0eBoO9vNFyi6pVApXX/19\nyxmL9UolyNNJJBL485+vxIIFf9AMQy9/7DFb5SFGroHfGrlmE5rMouV9siIWbDTATwRBwODBg30d\ng9u0pMFkPUy7+7E6sVLxP9s13m3qGSins0/Vzs/yv/wF4UcfxYrjjsPUk0/WHIsZIXetY3Hso1Yq\nIYq9yUOxWAyzZp2nWajvhApPvXvW7vqoXY/MS4OtlO2jUDqtRZPBtJNbYRS1Z417mE2OH5MsGcpK\npSKXaJjFzgOhFcah7FNlVxb2eHaRJAkrbrsNt2UymLlgAabMmNHnN5AnSyHqoCY8BBm37mnW+ywU\nCvLar5pIeTgcdl2Fx4v1UT3MGGw3DJia90meZy6X05VQdIpWS/ppyTVMwinxAaPQeiW9nVu9ia2O\nW+2BrVQqct9IN2o+2XEu/8tfMG3zZoQAHLd5M1Y89lifsZDMnd/CCBx9WJk4WvuMxWJyiyy3GzR7\nsT5aDz2D7TWs9xmNRuWG2W5dD7UXAJ700wJ4ZTDZRBrSPPUjhMges1wuo6enB/F4HO3t7Y4bKHZ/\n5F2yTX6XL1ggj4fGkkwm+2zrFs0cxvX6ZYO8T7UGzblcDvl8HuVy2bG1cCPyeW4SBIOtNS56mdFq\nmE3Xw+mG2Vzpp0lRrmG6He+n+kplVw+v6wbZ32im5tMJw8J6lwBqvMwjpk2Tk3ui0SgEQTB9TZrZ\n+DUaSsk+SfpcML5UKtV8blWk3KkuJVaffbN6t0qJPy9Rux52W5a1qofZkgaTcDvph9Yr1bp6+BVu\npPUNpUatG9D5kdcuVd7G/+fmm3HwV7/qq3JPsVhEqVSyrPrE0Ye8HbOZnnrGzMsuJWq43VbMKkYM\nM7sWTddDrWG22ZZlrbCG2dIGE3BerYcgwyRJ6l09/FKmEUURkUjE1HqlXe9N6V3K+wUw7dVXsX7V\nKhz7n/9pef9WocxgURQRjUblbEMKaVUqlUArrTiB11JqRjI9yYAGWe7Ob4PtFMrrwXqf5XIZgHHv\nk4dkOXXR04ONxWKOF9tbhfonhkIhpNNpT8f012XLkDzoIPyNCYFXKhWEQiGEw2EUli3z3GCyb9Yd\nHR3yeKj+s1qtNn2PQ7sKNU5QT+WGnq9G7/ARZOPPovQ+KZyubFmmBjeYTQp74zrt6akV/mttazUB\nwuyYaUzxeNxyXz6ra6bVahVzf/lL+W/lchnZbBapVEpO8FHbzs01TMpWDoVCSKVSCIfDqFQq8n7o\nbTuZTGoW7ruZqu82dG79lpRTovR26AXPjGSfWRrFkBnFyd/DZkLTvikaQNm2hUIBb7zxBoYNG4Zi\nseiowMgjjzyCOXPm4LXXXsM//vEPHHTQQY7t2yqN+cQ7iFMGk4rtBUFAOp2uW1/pRZIKraHmcjlZ\nLcepchQrFItFZLNZdHR0aBpLtxFFEd3d3YjH4/KkK0mS3HpJ+RJDhft+lE64AXmV1WrVd0m5epAB\n1ZPsK5fLgRx7M0LeZyKRQDwel19cHnzwQRx44IF47733MG/ePLzwwguOZEOPGTMGf/nLX3DkkUc6\nMHpn4AbTAYNJQuWkTON2j0YjYyYxglKphM7OTsRiMd/epOllIp/Py2Pxg1KphEwmg/b2dlmKkDxI\nCvnR2zOFa9kHv17pRCNM4ORV3nbVVX0Uaszgx9onTdZ0/SKRiCwvKQgCSqUSKpVKoM+/23h5Xcj7\nvP766/HWW29h8ODB2LlzJ8466ywMHToUd9xxh63977///vjiF7/o0GidoSUNppoKh1UkSUJPTw/C\n4bBcX2l0DHaPq0W1Wq0RI7Cb+WmnjEX5MuGHzB15JblcDul0WvZUgF6vV5IkRKNROfRUrVZlT5JK\nIZTeJ3k/8Xi8RiQ7yBM41Q4uyGSw8be/xTFMTWwQvUy9yd+KYLyfNHvoN5lMIhaL4dZbb8Wrr76K\n9evXY8qUKT6O0B1acg2Txc5NTKnYXurBAvpjFkURmUwGyWRS9oDY7bycQMh7I6Pi5oSh9dvUdGlp\nLSaRSMjrY4IgyGISqVRK9oJZT5PWbwDIhp9ekJTJK/RdtcQhv2BrB8/P5bAcwLHwXlLOadhEFbbu\nU23dmdbliGYzZH5D53LkyJGGvj916lR88MEHff4+b948nHjiiU4OzRG4wbRgRGhtsFgsyp0bvDiu\ncgxKjIgRCIKAG2/8PTZt2oliMYJEooJx4wbgyivP12z/ZGWsNFmFQiHfMoXVmk6TMZMkqaZ8gRpl\nRyIRCIIgfxaLxeSJtlqtyv+Q4WRLT1jxa72OHzRpV6tVzxKHlLqnXwfwIwDH4DMRCRcaO/uBVqIK\nW6TP9pdsNrx6AVA7jpXjLl++3KkheQI3mCaNgSRJyGazkCQJbW1tKBaLnhxXua1yTFoCCSz5fB7f\n+Mb/Z+/Lw6Mos+5PdSeddGcDBEGWT3AFkdVPEFFHFESWLMoMIKig4hfREUV/IwrowAwBHEb4VBzc\nQR1HhYQlSMgD4gRFDAwuoKMMqJ8IyKKyJJ2k9/79EW/xdqWqu/bqpOs8zzwjSaf7ra7u97z33nPP\nLcGuXY8D+C/+5++9dwDbtj2GysoFkqSpBETcGRkZfLuGUmiNhMmXNi0tjSdsijyAM5FhJBLho0tq\nt6FNNhgM8q5DRJz0P2p5EJInESdLhMKJH36/P6bNh1XdGrXZifmeDgewCS0jypSCVJM+mSaYdXhp\n6ZEsHRyNQrKk1u0apgKEw2GcPn2a31wpvWc2WLJVIjhasGAFdu2aDZYsG3Eudux4FPPmvSz5t3Ku\nkxX3aBkRpnVjId9eGsLLinnYSJAlVXYeKW2ybrcb2dnZyMrKgsPhgN/vR01NDe+URA42mZmZvGqQ\nXisYDPI1TGHtk9oj2NYjv99vmHCIokuh7+lwAC9lZ2PWlVdi9uDB2NavH7ZVVur2uskGYd05KyuL\n/w43V9WzEFZFmNRKpyfWrFmDLl26oLq6GqNGjcKIESN0fX41sCNMmZEeKSHZ/kGr3HoI4XAYtbW1\nsg0S9uw5AeBcid+ei927T0iuNRHYyJtchMxUjNL7SfeJFYIQabH1q1AohPr6ep7s4j0vOzSZbUEh\nVyA2+qRohX1dIDZ1K/bcwJl6qd59h/F8T++KROCbOrVFRZVyQZ8Hqn0Ko89kGs6c7DBiFuZNN92E\nmyRm5lqFlCVMtjk+3qZOyjufz8f3MuoFLWRLUVQ8AwAhfL74NRu/P/7HQer0SkYATqfTsrFcTx2k\ncwAAIABJREFULJFRdEspOCFZBgIB+Hw+eDwexVEw1cfETKxJbUu1TyERsvVPWpswdStXOKTkPU5W\n39NEMCtaos+GlGG80LJPzeHFjGsxMyoWXk8qGK8DKUyYBIoIxEBRUyQS0X2wstovDxFDMBhETk6O\nIgLPzAzH/X1GRkj052pVuWqhpq5MaTS6T6y4hzZEqh+SslmPdhsiSHIFonvT0NAAp9PJR590n6h+\n6XK5YlS38YRD7AauJvppKb6nZoIyB1KWfckcfVpxYLUJM8XB1rekoiazU7JkRhAOh/kvsxL06XMW\n3nvvAMTTst+jT582ip4vkSrXjDYWquECjb1g8ciyoaEBkUiEr0nqDYfDgYyMDD51GwqF+L5MWkc4\nHI4Z7iuWuhUKh1jlp5zoxww01xqfGkilzvWKPpsjUjXCTK6jkYlgBR7CL38gEGgiGokHowwIWFDa\nMxqNqo7kZs++C5ddNg/AAcFvDmDgwCcxe/YUyb9l3yciHxL3JJqnaRTC4TBqamr4zYrWRtEmbV40\nOQaAYWQpBEUoJBxyOBx8Craurg51dXW8CMnlcsUIh+jaKFoVEw6R2IjtASbhUCAQQDQaNZzUUoEY\nxEBpc6n3Xyjcov83IyVr1T1JBeN1wI4wmxCBknqllg+nkr+l6SfkaapWTON2u7Fy5WN47rlV2LPn\nBPz+NGRkhH7tw5TXUsJGuUpGhCmBnMhUaOJO7R9EQvT+hsNh1NfXIz09HRkZGaZvKHS4iEajfNsK\n25dJqVuxnk82+qTNUCx1K4x+iGTr6uoMMSxvSaCDlVrIEW6xo7Na6vtvhEo2GWETJndmyDERgVi9\nMtHfa6lJxvtbqbSn2ujB4/Fg3rz7VG0SFKnRPM1E12xUSlb4ntD9o/l95JsrVwlrFNgeTzZTQX2Z\nrHBIqufT6XTG9HyywiGpnk+a7el2u5sIh9im/Za6eVsJMeEW9f6yhhjN/f0X7ltGqGSTESlPmMAZ\nP1i5RKAX4r1OPDMCPdZXX1+PkpJXFTn+6C3uUQrKAPj9/iZKWJYkaW5fJBKJsbkzE3IjW1bcQ3/H\n2vWxqluhXZ9wQxZuwkLhUHNrm2juERm9/zQ+zu12i86W1Cv6t/L9SpUaZsoSJpuyC4fDfBO50g+c\nEcIfsZ5GPV+zrq4Oo0c/jh07HoMcxx+qidGEDiWvpWadYn8nlgpmxT0kuHG5XPD5fAgEAkhPT+dT\n7MJeSSOhJbLVq+dT2LZidNuEjcQQOj7p1TZkBcQizA4dOli4InOQsoRJERx5e6qdz6h32lHM0k1v\nLFiwAjt2PIp4jj8lJffHRHTxJq0bjUSesGJKWJocQ8QRDAZjojYiHr0jK6pL6hHZqun5pAiGohrW\nLJ691nhtE9FotFlt3lpgVVQm1TYkxzBeCmYpl8Vep76+XhdbzWRHyhImqQlzcnJQW1tryRqEZMsK\nWRKl8bREmLt3/4JEjj/CiM6q94jcjEjwRNcu5QnLcRyysrJiVNBCUYaU4EZrZOX3++H3+3Xp8RRC\nTs+n0+lEMBjkFZxEhESc8Xo+xYQrWjZvG2eQiJTZtiF6fDzD+ETPZRaEbSW2SrYFg9SVWk9leqVk\nfT4fGhoadHcTEkMiRx+fz4na2tqYiE7NdWp5bygdySphAYja3FG9kMgk3qYhFNywvZIAVKVuKRIP\nhUJ8C4nREPZ8ElkD4Ou4FHmqMYsXE66wm7cZbSupCvZwxN4ztvZsdt9tIthtJS0caWlpfJSi5Yuv\nNSVLkZxZA5Y5jpN09CE4nb4mhuRmgsQ77AGCVYmyZEn1QiIPpa9DaUmK2oSCm0Sp22g0yqtbrbIF\nJAcgsvqj6NPn8yEcDsdcB12vUDgkx3GI3bwjkQhvVJ7MwqHmjni1Z2H0Sd8NoyEWMduinxSDVbUM\nkpqbqc7t3bsNtmyRdvzp27dt3A+/UoWtkveWNgSqQwo3ajZtSp6wetULWcENkScRN5u69fl8WLjw\nNeze/Qvq6zlkZITRr19bzJp1p+l1HDFfXLkpaLlm8WKp21AoxL/nRgmHzPhOmvUaekGs9iwcVxYM\nBk0/wNhtJSkCNt2o5oujNtqjzTg9PV1xZKI1wnz00UnYvv2Pvwp/WNI8gIEDF2LOnIWSr1lfX48R\nI2bKUtgqfT8p2o5EIsjMzOTJklXC0jr8fj8CgYAh9UJAulfy559/xtix85vMFK2qOoDt22dhw4YS\nU0hT7nugpucTOJP6TtTzSX65rHCoOao+jYYR1y+MPgOBgCnKZ6kIMzs7W5fnT2akLGHq9eFRQ140\ngoo+6GrXopbk3W43KisXYN68l/H557+gocGB9PQA+vc/G088sTDuhl9S8qosha1SkBJWTHovpYQ1\nq17IpiVLSl6NM1P0MZSUvIp58+5DfX09FixY0SQKnznzDs2EqrZmqqXnkxUP0eed6mn03GIRuphw\nyE7dGge2JUloGM8Kh/S+B3aEmULQuzVECsLGe5LwK4UWsqdrdbvd+POf7+PbNRJFufR3jTMzlc/U\njAehEpbaJoRKWIpAhUpYM5FIYfzpp8dx4sQJjBkzDzt3zgRLrFu2HMBHH2mLQvWsmQp7PongEvV8\nUvqP/psiT2H0qYfq04Y6SCmfhZZ9aqJPqQgzFUQ/9lEP5kwdoc0+EAggNzdXl55GLSRPxuXxprGI\nwe9XNlMz0fsTDAb5uZ7Ud0oKWUoFAmciUKfTaVh/aiL4/X40NMT/yoRCGVi06O/YuTM2Zd2IM1Go\nGpA1oREHBiJIj8eDnJwcntAbGhpQW1uL+vp6Pt0XDAZ5ow82WqU2F0rlss9NCmYyv+A4js+0NDQ0\n8H9nNlpKnTTR65Dq2e12x9hsShnGKwW1M7V02BHmrzCSMMmMQCju0UrUav+OlKW06cn9u2g0iowM\ndTM1xUCpQKESljZhn8/H1y7D4TAyMjJUG0xoAZsC9Xji36/MzDD+/e8axI/Cf1G8BiJLOa0zWiHV\n80mpPXbyCht9sjVPILFwiLXrYyMfO21rLISpeal7IGXZJ0bMqdKfm7KfSvbm6qHmkwJFUC6XSzQq\nMDqyFb4WCQKys7MVt2EA+HVmpnA8GKHpTE2xdVJasaGhAbm5uU0Uf06nE5mZmcjJyUFGRga/8fr9\nfni9Xn7jNgNUM6U5ln36nIV419+7d5uEUXhDgwO1tbU8Ccs9cLHmDWaCXi8ajSIrK4sn0bq6Oni9\nXr6mTCTocrn4HlDWBIHUzyzEIh8SNAHgW1fsnk/jIHUPfD4f6uvrZX1OU+X+pCxhsjAq0qMNPisr\nS3SjM3PjI5KiSE1t+mT27LswcOBCqJmpSevwer0xfadsSwOrhKV6b3Z2NnJycpCbm8sTqNfrVUQ6\naiA2R3PmzDswcOACietfiFmz7kwYhbvdjabwRMaU8hRLiZGpQmZmpqoDjlbQfQgEAsjOzuZFQW63\nGzk5OXx63OfzoaamJmbOJ/W30pxPjuN48gwEAqJzPtPS0vg5tEDjZh4MBvnULdVPmxOSISUrF+w9\noH1LeA+E/ev0/6kQYdopWehfwyRyCgaDcc0IKJWl1+tKgfVidblcqtJdrFiIFLaNClD5MzVpCLbT\n6eTrpvGUsOFwOEYFKuxBE/NWZfsLtUAqBep2u7FhQwlKSl6NmSnau3cbzJrVKObp0yd+n2ufPmfJ\nUqsCjYcutsfSTLCKZLHB22LCEqne1Xg9n6FQiCdU4WskEg5pbZmwqv+6uUBMvEWHnXA4jOXLl+Pc\nc8/VtX75hz/8Ae+++y5cLhfOP/98LF++HHl5ebo9vxZwCTbdFhtnk8IPAG92rqY+5vf7EQwG+R4k\nlpzENhkW5MSiRl12+vRpZGVlJdxISYFKgg7abJRKwMmnVam6k9YJoElaUUoJW19fz69R7mZGdTba\nsNkWCaUHBC3uQUCjUGbUqFm/9qoK+1wXxFXJkuCJrRcS4ZipKFV7H9i/Z1OxUgcatm2F3YscDgfv\n9CT23ETO9Lf0/ig9LFEGyMj3lab9GJ0haGho4N9fI0FRvsvlwrJly/Duu+/iX//6F6699lqMHDkS\nI0eOxIUXXqj6+Tdv3ozrr78eDocDjz76KABg4cKmveEGQ/QDkbKECYCvk1BPpBrCDAQCfJuImFF4\notdnyVYJSFka72Qn5sVK0ZhSwlT7d6dPn0ZGRgbfp0WbhpgnrF7CFiIdIk82Mk0Ujeg1baShoUEi\nCo3vBhQrMPLEXIveUbQU2MHXetVMKTok9bPYgYYVDRHZZmRkNBEOia2Xnp/q3XLnTJpBmNSmY/QQ\nc7MIU3g9wWAQ+fn5ePjhh1FRUYGKigosXLgQt99+u+bXWrNmDcrKyvD3v/9d83MphOgHwk7JQp+U\nLEnk1ShPjUA8M3czC/SRSKSJJ6zX68X8+csZMgmjV69WmDbtd2jVqpXmkzh9meW427AbJRmY65EC\ndbvdmDfvPkV/I5UClWM0oJeq1Cg1rtyeT0rv+v1+/vVZ9yEpxyGhWXxznTPZHFFXV4fc3FzcfPPN\nuPnmm2MyR1rx6quv4pZbbtHlufRAShMmEZYW4qIvaF1dnSmTRghSa05UP9WyYSh5j2jzJ2UlS5aj\nRs0Wberfvn0+Kirmq16fGOS621CKzyirvURgU6BSEY/c4dJqiYE+x2pT0XIRrxZN5EjDwIWpW5YQ\ngfhm8VJzJoXvUUshUTPFReyBRejyQ/c3HoYNG4ajR482+fn8+fORn58PACgpKYHL5cKECRN0Wrl2\npDRhslBDmJQ+i0ajaNWqleJTvt4RJkUH0WgUubm5cSdsKEF9fT3mzn0Re/acQCiUIcts3ev1IhqN\n8mkx2ujmz18u2dS/c+dM3lrOKAjt2yjKoU0gGAwCgCYhiVKoieqkomi1Aiiq25Kq1SywBEd1S1JC\n19TU8F61Yo5DbO2TSFNInmzqkMiTWqvofRGa+usNIcG0NKiZVLJ58+a4v1+xYgUqKiqwZcsWLUvT\nHTZhQt0JkxSfieorRkFIttRqkZaWFlekofRalZitA02VsLW1tTFK2D174lvrNf7eHHDcmckObreb\nJ1AtszGVgqI6l8sVd2h4PEhF0YFAQFbqVq+6rRZQOjw7O5uP8IngqLWGvU7WLJ5EQInM4sUiW2pd\n0iIcShZY1b6i92ivyspKLFq0CFu3brXEqCQeUpow1aZkQ6EQamtr+dN4TU2NptfX+rdi4h69oMRs\nPRQKwev18o48dMIn2yyn05lweHWi3+sFsaiOxCJCd5v6+voY8lRyOIpnwp6enm5IVKekXkipSitb\nV1jxmxTBid0T4ZxPQDx1S5kOYfRJxOzxeBS73dg4A719ZO+//34EAgEMGzYMADBo0CD87W9/0+35\ntSClCZOghLjoy0qOGFqaqPVIyQrt5fR+Tblm62KiJ3LtCQQCCAQCSE9Ph8sVjPt6Sqz11CJRrU6q\nv5CdKSkUqYihvr4eo0fPxo4dTeu127bNxNtvP4o2bdoYGtXJqRdmZmZaVrclRXCiFiw9ez7ZyJOI\nMJFwSItZfEvr9TQ6wty/f79uz6U3bML8FYlIhEQsgUCAH2wMmDfpRAzUDxXPHEEr5JitCxW5bIqM\n6myRSASBQAA9euTg/felm/p7924j8nP9oCb9KJwpKZYmpAia3UgWLFjRJJXdiMZ67dNPv43585WP\nQlMLWit9VgKBAFwuFz/DMhAIYPHit/Dvf5/m1ct6jSQTglUEq5m6kmjOJ0uewokdbBRKn1V6TqCp\ncEg4pJkVDiVTbdIqYtabMJMZNmEicV1PjphGzYdVLdmSQhJAXHGPHq+ZyObN6fTB5/PF2NzRBsOm\nsiidNnv2XfjXv+b8KvyJbeofMGAhZs0qkb02pQgEAprTj1JpQlZsQ5t1ouj8yy9Pqb4WtaCoTuii\nVFdXh6KiOdi5cxb0HkkmtoZEimAlkKuEZud8hsNh1NfXw+l0KjKLFwqHjBrSnMwQ7nWpMgsTSHHC\nZGXlUiQidMoRfiHM/oKQuIc2CaNPuH36tMF770lHhJde2oonbVbcw3rCkrkDEVVFRay1XHp6AD17\n5uGhh2bw0Y6efXNsnUzPthF2M2XJkwj0V06QhFn1WgIRFbX6sO/twoWvNWn1aUTjSLI5c17AvHn3\naRbERKONY+70NEUQIlENNy0tDcFgEGlpabynL3vQA6R7PoUpbqkhzez7ZEbkZ1WWC0idWZhAihMm\nQYowqS6XSEzDiof0eF0psOIeMzxogUaz9W3bHvtV+BMbEV5++QL86U8Lm5AliX3YGhUbzUg19Qtr\nbHo420iZARgBh8PB10Wj0Sjc7vj3yIx6LYElKrGDX6Jo+N//Pi1bdSsFchCiA4YZh00hwVH7DMdx\n/OeMTamnp6cr6vlMNKQ5LS2NJ2QzYJVKtnXr1oa/bjLAJkwRUEQi5ZQjhNY6phyyFYqNKK1mNMhs\nfe7cF/H55z8jFMqA0+lDnz5nYc6chbx9WzxPWLk1Kja1lpmZKZpaI/KUs1ETSXCc/gOXE4HjOPTt\n2zZuvfbSS/NMiT7k9HkmqlUHg43j6aQiNrEartga0tPTVbfPaAUd4EjFLcwIyOn5BMRTt4C4cIgs\nDf1+Py+Ca2mOQ6QJSAXYhIlYwqNNNhwOGyqmoddNBCmxkZbXVEruFBGSGQE7sT2eJyz1N6rdHKSc\nbYSqSLHakZkDl6Xw6KOT8MEHM7Fr12yIRefTp89ETU2NIRZ3BLlEJXcweDzVrbCGyxoDmOEgFA9i\naxBmBBL1fAKxwiFKx1KkKUzd0t9SqxUJ34wQDpnZgykEqfRTASlNmMIaJjtpJDc3V/YHUI9+SrHX\nYh1zhOIeM9W5dEImRS6lmWjTYMmS3ZhYazOt4DhOtlKVSELvNSgBpfLWrZuLv/71zRgT9h49suFw\ndMNdd70Av98JlyuInj3zMH36eHg8HtlG8XLWIJeoEo0kE1MvCzMCYhEbtRW53W5THYRY0PsQr99V\nbc8naxYfL3VL6dmWIhwysq0kmZHShCnE6dOnkZmZqTgi0SMlK4TY7Eg5f5cIStdKqdVgMMhvkFJK\nWLMcY+IpVSltRuRqxeYjtJlj67VSvZnvv38AO3cuwLp1c2PEOWrdhmgNcu/FzJl34KOPpEaSyVMv\nCyM2cu8BwKckE6Vu9YYayz8lPZ/031KpW7HrjCccIpMFpaI3M1tKhK9jq2RTDDQXU8mkEb0g9iFn\nnYSkyNus9IvX6wXQWIekaFeohAX0nfShBOzmRl6k1DtndLpTDIlaV+L1Zu7Y8RiefPJ1zJt3X5ON\nVEkNlw4PSu6FnMHYSkDWfLQGOalbvaH00CAFLT2fVMOk1K0c4ZDQLJ6NPq2G2GG7oaHBVsmmCurr\n63nCVPul0iMlSyBlLlsn1PM15f6d0JuWPUHLUcKaCaG9mtCLVGgLp0e6Uwx0aIjXupJIjUpeuuxG\nSvUvoduQWA2XCFtN+4yakWRiECPsRKlbOc5JWtegB5T0fHJcoy2k2+3m+z3jmcUDZ8gZaKoaB7Q5\nDhkFu60kReD3+xEKhZCbm4uamhrdSE8NiHhoGLUVvp4EinDdbneMUIQESLS5salDNW4teiBe24iY\nQEXOXEw1a5B7aJDjnCSGeG5DAPhrFB4azIYcwhYT2yhV3caDUWQpBqmeT9ZSj9Szaszi6fPJlkHE\nhENmin6Er2PXMFME7DR3M0U0LEi6Tm0icp17tESYgHTNQyzCpZNxdnZ2zGkaaNww4k1HMRJKHGPE\nNh89zNWV9nnKVaPGg7CGS56nJDrx+/2qrkUr5ETYQojV84hwyAhASepWS4StFXQtFFlmZmbynw9h\nGlqYumWjSQCSZvHkOCTs+WRFd2YLh+yUbIqAVXdqfR61ZEubflpamiJlrt6gKMnn8/ERrlAJy6r5\nKF0LgBcnselOo6GlbSRe3YhNESa6FjV9nmrUqIlAJYXc3NyYKMeodKcQwpS42vsv975IqUmtJEsC\nrTUrK4v/fmjp+QyFQk3IE2ja8xkIBHiXLACqhENyIHbQDofDlmbEzERqXKUE2BuvZx1SLkha7nK5\nFDfW69nKQqQdCoWQl5fHO/ckUsIKB/OaVSvUu3VFLN0pNQ5L2GuqlLD1UKMSxCJssfYbPdOdYmug\n6FZvJyUlpvdkv5gMZCmWCo7X8wnEzl6V6vmkDIJYzycdZomczRQOmanQtRopTZgszE7JUuqTTppW\nfeDEek+VKmHNqhUCZzYlvWdIEsSuRajudDqdvGOMUtcavdSoZDMXz5NVzrXoaTto5GdYTiuRVQYV\nQGxkmYiwtfR8sn2fAEQPpEYKh1KJHMVgE+avMCvCFKY+KYVi5GtK/a2YsXwiJWyiDUFurVDNIUGP\naSNKwF4L1QppDQD4g4HSk7tWNaqa6FZ4LXRf1PrDKqkf6w1K3VImJBgMxowpS5S61RtaUsFSaWg5\nPZ9s1Em1z0gkklA4RC0/UmbxSsDuE6kAmzB1ANUfEoFqXuFwmE99WiU2YnvUyFhe2C9GP2MnXCgd\nJRavJiV3k6YaWSAQsDTlRiTj8XjgdDqbbGxmbNJ62cwlUqqKpaEJ9Dk2cuJIIrCpYLZumih1q/da\n9a6bKu35TE9P5++by+XSxSyeDiNi75UdYaYw9KphyoGU7Z4WtatWoREpYYXiHrZOR2k/PZSwcmqF\nwronm/azqs8TEI9u1QyV1gI1rjVyoCR1S2SpRmylF+KlgqXSnUSuatXQYjBaZMRGh4B4z6fD4eAN\nImjSipjjkFyzeFJcA/KFQ1Yc+K1CShMmCyNTsqFQCF6vFy6XS/cTuZITH53KKVqMR5YUybhcLkOm\nS0jVPYUiCCJTs9N+BFYBKrUxxquv6VErBMy1HRSmoYWtROSJatX9UNJKpEV1Gw9qWmi0QtjzyVoP\nUsmEDmnxhENEnFKpW+GBlhUOCVO+wWDQ0M9jsiHlCZNVjBpBmCTukbLd0xJhKgGbDmbrP2JKWKMi\nGSmI1deotgY0kmcoFNJFNKQESnssAemh0lpmSZpdu2XBKiy9Xi+/OVJKVsxtyCiwZKkm4yHH/EGO\nOI3KA1ZmPKiensh6UCgcIvKkaDIajYr2fJLaGogVDlHJJhqN4uuvv0anTp107cF8/PHHUV5eDo7j\ncNZZZ2HFihXo0qWLbs+vFVyCzbrFx9qBQCCmuViNYwVJ2nNycvif0Qkw0UxNdkNWipMnT/K10Hhg\n08HZ2dm8Ty2RJhtZWrk5E9jolsiS/qeGcNRA6+Ys9Zx0ag+FQrIIx+hIpr6+HgsWrMDu3Sfg9zuR\nkRFGnz5tMHPmHbxiV2zaB5sVIL9UPdXQQhhZN2XFadTqxWYF2M+Zz+fjsw1WkWUij1w2kqY+TmEk\nLZa6JVW8MPpk0dDQAIfDgVAohJEjR+L//u//cM4552D27Nm48cYb0aaN8j5iFrW1tfw++uyzz2L3\n7t14+eWXNT2nSoh+wFI+wiTIFe5I/S178GD7GhPN1DT6VE5KWGE6mFIzrBI2UerRDIhFt2IzMVmR\ngt4RjpyWDTVQ0n4DQBczgHiQmpyyZcsBfPTRLGzYUIL09HTRzVlKfal22Hc8GC0ykpO6JXWqlZ7J\ngDxDeSWRtFjqlhUO0e+Fo8qysrKwdetWfPzxx/jTn/6ElStXYurUqejVqxdeeukl9OjRQ9X1sUGH\n1+tF27ZtVT2PUUh5wtQ7Jat0pqaRtdNgMAiv19tECet0OpvUcGjepZUn50TRLaWJhApCPYU2cgcu\na0Wi9ht6XSPvR6LJKX/+80t49NHJCbMNLOFIGcWrrRVaMQhcjHDIqo/juBjrQTNLBGqmr0jV15X0\nfAJnhENCzURWVhZ69eqFZcuWwefzYevWrejYsaOm65w1axbeeOMNeDweVFdXa3ouvZHyhEnQQyUr\nFc1ZAfpCUDqYFfdQOwH1Y5EqjqI4s6XjatpGpPoKtQht9GrZUAqWcCiaovSY1+s1zHYw0eSUzz//\nRVVqXolDT7x7Y9bhJRGoxzE7OxsAJAnHyIOmHqPK5PZ8sr2ZbM9nKBSKIVGHw8HrM4BG44jhw4cn\nXMewYcNw9OjRJj+fP38+8vPzUVJSgpKSEixcuBDTp0/H8uXLVV2vEbAJUwdQOrempkbxTE29I0yq\nxwYCAUlPWNp4HA4Hb82XlpbWJG1jVO+a2HqF/XRKICW0UbKp6TU7UQvEUo9KeySVINHklFAoQ3Md\nO16bRzxzdSJLUmlbAWGvJ60vUepW7xKBUZ9NqZ5PsUOnw+GA3+/nrxFoPGDu378fx44dU3TI3rx5\ns6zHTZgwASNHjlR9fUbAJsxfoYW4SDiUk5Oj6gOtVx8Tq4SlqSdSZClWKzSyLUJqvSSs0XM8mLAp\nn+qeUmbkySB0kko9KumRVPr+JZqckpkZ//dKIbfNw+FoHARudqTPIl6vJ0GKcJSqbuPBrIOcVM8n\nqbtp78jMzOQf8+233+Kll17CwoULdfvu7t+/HxdeeCEAYN26dejXr58uz6sXUp4w2Y1JKXHRhh8M\nBgGoG0Ct5YMmVjt1OByinrBylbB6RGtyQAThdDoNTV8L657C9CDHcbxxuJWqYLINjJd6TNQjqfTe\nGDE5RQnEUrc0dYOyNuy0DrMghyyFECsRaBVBEVla2U7kcrn4A4DD4cC7776LuXPn4tprr8WHH36I\nt956C3369NHtdR977DH85z//gdPpxPnnn49ly5bp9tx6IOXbSkhGTuYCrVq1kvV3rLgnKysLp06d\nUiWpps07Ly9P8d/W1NQgMzMTTqezSe1UjCxZJSzZuymBlFxdaW3NaFMEOYhEImhoaODbO4yMpONB\nr7ppvFYCqfvc0NCAUaOkJqcswIYN8s3g9QCb9aDWhWAwaOq9EbaY6fFabK0wGAzKSt1aSZYEsdaq\ncDiMTZs24dlnn8WxY8dw/Phx3HjjjRg9ejTGjx9vqW5DZ4heiE2YvxImCXbkEKaYafmJEyfQunVr\nxR8YLYRJcyhpgghtuHRaB87IwdlTs8fj0UXqz27QbOowXv3GbFMEMYhFEHRoCgaDcftueKI9AAAg\nAElEQVTw9IRR6TY2kg4Gg3GFNvX19Zg790V88cVJhMOZzOSUOy0hS7H3gu2RNLIX14jeW7HXSNS/\nmqxkCQBHjhzBhAkT8Pzzz6Nfv344fPgwKioq8OWXX+Lpp5+2ZK0GwSZMMRBhRiIRnD59Gq1bt477\neLFWDQCqCVMJUQtx+vRphMNhvnYqVa80qq+QwG4CbHpauEEnQ62Q6rwcx0luinpF0vEQb3ainmBb\nVoTRGrUXAcYRhBwoeS+kDmpaRVBmkKXYa7KfNXLhCofDMfNmzYbUe3H06FFMmDABzzzzDAYMGGDJ\n2kyEbVwgBiU1TJ/PJ+ncw/ZzKn19NbVTUpbSiTyeJ6yc+pgWyGnxoI0uOzvb0mkjcnr61AyUVgKj\nTbtZxKtJk3kFiaOsIEylBwcxEZRWgRoRBMdxpraDCftXycSC5q0GAgFTrQeBM3sLEEuWx48fx8SJ\nE7FkyZJUIEtJpDxhCiG2cbDiHinnHj36OOWur66uDpFIhDfAVqKENRrCDTocDvPkDjQeOszoWxNC\nbd000QatRAmppt9Ub9B0ikAgwK/daOckKWg9OEgpO5UIbYx2EZILUj6T+CyRE5RR6WKx+u0vv/yC\niRMn4i9/+QsGDRqk++s2J9iE+SukPoDRaBRerxfRaJRv1ZD6ezWEqeTvIpEIX7fMycnhv0hKlbBm\ngfrYOK7R9YiN1tg2AqN9YfU6OCTaoFnHFLFDF02UsNJaTcwMIF5bhFG9uEZ45Iq5DYkNYmbn0CYL\nWQprlnocBpRAShl88uRJ3HLLLSgpKcHVV1+t+XWaO1KeMNkviTCtGg6H4fV6kZaWZnhdI1FKTGpE\nGDt9hFXCJsOwZWHdVKzFI9E8TK0wciyWHDs4Ik92pqdVG3M8Ra5UWl04R1LsMKAUZozGStQjmZaW\nxn93ko0sxSAc7UU1XDqQavnuSJHlqVOncMstt2DOnDm49tprtVxmi0HKEyYLNtpjxT1yUnhaIsxE\noLUIlbBU6wgGg/xmR8IOq0cPJUp/SqU62fST1uiGNmYzoux4dU+yuTNTdSqEkiibTasD0gYDSjMD\nbFuTmZ9P4WGAok4A/PfFijKBWuGX8OAp/O4oqeNS5kNIljU1NZgwYQJmzpyJoUOHarrOlgSbMEVA\naQ8asiwHWmqY8QRDQqERW69MS0tDTk4OPz+SNgGqS5m9AQDq0p/shsaqOtUKOdiN2Yoomw4DTqeT\nT/mRIlWvw4ASaG1f0UMExaakrTT4p3WQq5VUmUCNUbwS6KWSlpO6lWrBYa3/WLL0er2YMGEC/t//\n+3+48cYb1V9kC0TKt5XQ5gw0piCIbHJychRttLW1tcjIyFBVIxOba0lpEvKEJWPueErYtLQ03lTd\nyJYIKRhRN2VbIuT0R+rdb6oWYrVC9jBARtZSXqp6wcj2FTk9hfQ42piT7Z6wUNK/qgVmthSx/avs\n4Ya8YYVkWVdXh1tuuQX33XcfbrrpJsPW1gxg92GKIRqN8l6wp06dgsPhQE5OjuIvNU2jV+PWcurU\nqRiCZoVGlLpKpIQVq0sJe9ZYFaSeURe9h0bXpRL1R1rRSycGue49Rjfkmy38EjN/oANoNBqVbTNn\nBFiyZPunpWDU4cYsshRCeLihfYTeC5fLhYaGBkyYMAFTpkzB7373O9PWlqSw+zClQOIeALwtl1Jo\nbSthPWFJCUsCkURKWKlUm1SdkBr39Wr4Nkv9mSg1SHXdZBBxyElJC03i2evR2uJhhrBGCKEIiq3h\nOp1Ovq/Q7BS5msknYnVcynRImfgnglVkCZxJ3ZIzWCAQ4H1i+/fvj0svvRQ+nw/jx4+3yTIOUj7C\nDIVC+Pnnn5GZmYlgMKi69YCMxOWcXoU4ffo0srKyADRGqhkZGXxjvRhZalXC0nOyXp1q6mrJFtE5\nHA5EIhEA5o0nY6GXIlfonKQkNSis31pZK6TPhtvtjvm8GZXqFAN5Pus5+USN25CVZMmCRILsZ+P4\n8eP4wx/+gGPHjuGrr75Cp06dkJ+fj6lTp6JTp06WrdVi2BGmGKinkWTmWoU7av+WJPxKPGG1zI8U\nTr0QzihMJLIx2m5PLoTpTzaVpuR6tELP9KdUi0ciEVQyCWuE/Y1kmMBOWTF6fJxRA8HFMjfxrieZ\nyTIQCGD69OkYPXo07rzzTkQiEXz88cd499134ff7LVtrsiLlI0wA/AdDS5TIOmQoxalTp/h6ZTyb\nO9qIOE7aB1Ur5IhszLDbkwM5EZ1S0ZAamJn+FKt7UqTm9/sVjaQyAvQZpe9RonVIXY/W+0NkabbJ\nv/B6KOthpTcsIE6WwWAQd911F4YOHYri4mLLPjNJClv0IwUS/bB+kkqhhjDpNalXkGTuUgbqcnxQ\n9YSYyIZqUVZvAGoiOlIN6qUgtjr9KbweAMjIyNBd1CUXWj+jrKpT7hgsMVhFlkJQq5fT6eS/z3ro\nBpSCLd/QZzQUCuF//ud/cNVVV+G+++6zybIpbMKUgh6EyfYzyQGrhAXAt6QoVcKaBUr3BQIBAIjZ\nzMzcnPVS5ErVoeRuzsnSvsJmHVwulyV1QiBxy4ZSCOu4gLy6dLKQpTANK7cFR2+IkWU4HMa9996L\n/v3748EHH9TttX0+H37zm9/wr1lYWIgFCxY0edy0adOwceNGeDwerFixAv369dPl9XWGXcNMBD3M\nB+SAlLBkuVdXVyephDXS2k0uiKRYhxYjFLdy1qGXIleL0xAraLG6VUJYRza7Tkjr0FtYE8+qT6rF\nw6j5okohVrOUYzBgVKmA/a6Ew2FMmzYNvXr10pUsgcYOg3/+85/weDwIhUK46qqrsG3bNlx11VX8\nYyoqKvDNN99g//792LFjB6ZOnYrq6mrd1mA0bMJkwHEcr7I0CuQJyyphgcYPssPhiPGENaO3MRGk\nSEooGmI3ZyMUqkRSVOvV84su5TQkJhoCkBSG3fHSn1JkY8TmbJSwhkUiqz66jkAgAI/HkxRkmeg7\nq9QoXinEyDISieChhx7CBRdcgD/84Q+GfHapJBUIBBAOh9GmTZuY35eXl2PSpEkAgIEDB+LUqVM4\nduwY2rdvr/tajIBNmIi1pjMywgwEAqirq4ux3GM9YdmRS4FAQJMSVg/IISk9FLdy1kEkZXT7ilT/\nXSAQ4KNK2uiSRRmcCGy/p5QvrJr+SKvSn8J+XCIH4ExpxOw6ISCfLIVIZBSvNHVLB20hWT7yyCPo\n2LEjZs6cadj7EolE0L9/f3z77beYOnUqLrnkkpjfHz58GF26dOH/3blzZxw6dMgmzFRDPMKkL3VD\nQwPfwsKKeygio82ZNub09HR+yK/ZmzOb7lNCUmJkoyWysULsxILIJi0tjVd/Ao39slrIRi20RnR6\nDcdOlvRnOByO6UfWMq9UC9SSpRCJsgPs9Yh9h9gZoyxZzpo1C61atcKcOXMMfR8cDgc+//xznD59\nGsOHD0dVVVWTSSfCfbI5CY5swmSg1a1HDBSlhUIhfvi0lBKWerZoQ5M7a1Fv6NU2IieyiadQZcmB\nhmVbAbFISops9B5PxkLvgeCJ+gmlyCZZyFKsVmh2nZBdh96lk0SpaOGBTWwgdyQSwZw5c5Ceno55\n8+aZ9h3Ky8vDqFGjsGvXrhjC7NSpEw4ePMj/+9ChQ83KHMEmTAZ6p2RJCQuA96dV4gkrNWvRqC8+\nuw6902zxIhsxj1uj1qEUUuSgRTSkBkaLv8RS62JkA8DyweSAPOccOfNKtWYHjCJLMYh9h0KhEJ+6\njUajcLvdMSYnJSUlCAaDWLJkieGlnZ9//hlpaWlo1aoVGhoasHnzZvzxj3+MeUxBQQGWLl2K8ePH\no7q6Gq1atWo26VjAJkwAiInw9CJMseHTapWwwi+KmKelHobdZilypciGFLcOhwOhUMjyXk+5Di1K\nRENqyNMKpxgxsiEhB6mkSaRmNtSQVDyyUduCYyZZCiHMaNDIMr/fjxEjRqBLly58Wvall14y5T4d\nOXIEkyZN4gOC2267Dddffz1eeOEFAEBxcTFGjhyJiooKXHDBBcjKysLy5csNX5eesPsw0RhF0Im6\nrq4OeXl5ip8jHA6jtrYWrVq1QigUQm1tbczwaSlPWC1KWLFpJGpPzVaYdQvB9nrSpmWFJyyg3/uh\n1clGLM1mBVijCNZgwIxUtNg69Ho/xHyV5RxwkuW+iJH2gQMHsHjxYlRVVeHYsWMYOHAgCgoKMGbM\nGHTs2NGytTYz2H2YiaBHhCmlhBXzhNXaUyiM1ISnZjkbmZnTRuKBhFE0i5Tt9TTTE1bo3qN1MxRO\nJCGikZMWTIZDjNQ64qWijRLZGEFSalpwkpkso9Eo1q1bh2AwiL179/Kp0fLycnTt2tUmTI2wI0yc\nEQdEIhGcPn0arVu3VvwckUgEp06dAsdxokpYNgVr9JQPudMu2LYRq/1HE7nmmOEJa6Z5eTynIY7j\neKMIK03UAfF+PjGwqWgjhmNbQVJiVn0cx/Gfj2Sr4UajUbzwwgv47LPPsGLFCkvJvAXAtsaTAhFm\nNBrFyZMnmzTbJgL1CQYCAeTl5cVVwprdJsFuZOwoLzLrtnp+pJrDQ6JB0mrXQaRt9uFBaJtG5hlU\nS7bi3mj1yRXeI5Y8lT5XMkTayVQukCLLV155Bdu3b8ff//53S8m8hUD0hlp3dE1iKEnLks0d/U08\nJSwJgVwul2k9hSRNz8zMRE5ODu91S+QAgD8smA06PHCcsukrJODIyspCbm4uMjIy+PfW6/Xyzety\nYXWkTRkAGl5OadxAIIDa2lrU19fzhx0zQOSgJcIV3iOXyyV6jxJdExvhWp3+JGtI+h5xHAefz8ff\nI/KkNhKUzheS5euvv44PP/wQb7zxhq5kefDgQQwZMgQ9e/bEpZdeimeeeabJY6qqqpCXl4d+/fqh\nX79+mDdvnm6vn2ywjyEMlG6UJPRJT0+Hx+PBqVOnEIlEEI1Gk9ITFjiTZsrMzER6erpovaY59Xom\nUtwmasRPlrmebKTNuioJnYaMbCmidRCZ6WVBGO8eAdKRGpG2lbV1QDwdLGbQwbZ9qY2m44FanIRk\n+Y9//AObNm3CO++8o/vekp6ejiVLlqBv377wer247LLLMGzYMPTo0SPmcb/5zW9QXl6u62snI2zC\nRCxRsjZ58RAMBuH1euF2u2PmZ7LOPPQcdEpOlt41lrSljAXM6PU0YrCvEo9bvSdsqAVr/SckbS2i\nITXrMDotLSWyYU3i09LS+HtndQ1XTu1UeI+MMLQQI0sAWLlyJcrLy1FaWmpIC1aHDh3QoUMHAI0H\nuR49euDHH39sQphWZKisgE2YAshRylJEJlTCOhwOfgOmLwgpP60+JScibbFeT+HG3Nx6PeN53Dqd\nToRCId4E3yooqWlzHKeLrZ0YrJjAwjrZCCeSkMcy1T+t+O6oERrJMbRQqiKWIsuysjKsWrUKq1ev\nNmXs3/fff4/PPvsMAwcOjPk5x3HYvn07+vTpg06dOuGvf/1rEw/ZlgKbMBWA0lV+v19UCcv2qNFJ\nneM4y1N9SttG4m3MYq48cqFm6LNeYFNoNNiXDjThcNjQNKcUtES4UhuzmokxRqu25YKifo7jkJWV\nFZMh0GOChxLoocoVM7RQatXHOk2x35ny8nL8/e9/x5o1a0w58Hm9Xvz2t7/F008/jezs7Jjf9e/f\nHwcPHoTH48HGjRtRVFSEffv2Gb4mK2CrZHGmBgEAp0+fFpWMU9osHA4ntLmjjdDhcPCnZLP6CIVr\n1nPIsVi7ipyohhSXrEm2VRBGuEYobuXAqLFYUu0dUp+7eOlgM8HWToURLttjbMZwbDNaWKQ+d2x6\nXcqWccOGDXjhhRewdu3aJuRlBILBIEaPHo0RI0bgwQcfTPj4bt264ZNPPlHcbZBksI0LpCBWw2RB\nA3I5jkNubi7/GCklrJhhuNjUASPJkxWz6JViU1ojBJLHGAEQj3CVetzqASPHYrFpTiC+aAgA6uvr\n+ZRoMvThin1W2Whaqu6p13fJrH7PRFNjnE4ngsFgE7LctGkTli1bZhpZRqNR3HXXXbjkkkskyfLY\nsWM4++yzwXEcdu7ciWg02tzJUhJ2hAnwDj0AUFNTE7ORkRLW5XLxJ3C1nrAEKbs0vdSpydDrScRK\n76uVxgiA8l4+tdF0Ilg56UPYiA+A78O1sr9Ri9BIT0OLZHDwYUs69F6QStXlcuGZZ55BeXm5KvtO\nNdi2bRuuueYa9O7dm1/P/Pnz8cMPPwBo9Id97rnnsGzZMt43e/HixbjiiitMWZ+BsI0LpMASZm1t\nLR8diilhpchSrRJWrMFbSz3NKAWqEkQiEd4jF2gcueRyuUxLRbPQw71HzG9UTdN6srQWUcaEda7R\n60CgBESW0WhUl9qpnDSnFJKBLIEz2Qf6jITDYTz33HNYu3YtPvvsM1x33XUYM2YM8vPzm9WUj2YI\n27hACmIpWb/fD6/Xi+zsbJ4s6QsJgBce0Jc+EAggOztbsZhF2OCdnp7Om7d7vV74/X7eYCARSI1H\npu9WguZ60nsSCARQU1PDOyLJvSYtoHtDdTG16WDWWECqaT2RsQAJjTwej6VkSeYBGRkZyMrKgsfj\nQU5ODtxuN/9+1dbWoqGhwVBDC9YsQi+hkZihBWVbamtr+YOT8JqSjSypRxpozABcdtllcDqd+Oqr\nr3DbbbfhvffeQ/fu3bFq1SrL1pqqsCPMX0HRkNfr5WuTtNkn8oQ1wiFG6DVKk0ikxCjJ0usZT8xC\n6SY97NISwSzlp5yUYDJYuwHyhEZKRUNqYLYqVyxDQNcUiUSS6t4I69o7duzArFmzsG7dOrRr147/\nOY1ac7vdViw3FWCnZOOBop7Tp08jGo0iLy8voRLWLIeYeEbdNAsvGZq8lQx9Zq+JjK31asK3Svkp\nlhKklKfV1m5qhUZax5MJkQwtLOxwbOr3pJKBFd8fqXuza9cuzJgxA2vWrOHNA2yYBpsw44HSa5FI\nBC6Xi++pjKeEdblcpjvECMUoVEulPi2rhDVahhyLtQ2odUcxW/AUbx2U1gQQ00doNnHqJTQSioaU\nHnKSpYUFiM3IsIcCs9qKCFRPFpLl559/joceegirV6/WdSTXwYMHcfvtt+P48ePgOA7/8z//g2nT\npjV53LRp07Bx40Z4PB6sWLEC/fr1020NzQR2W4kUotEoampqYjaTREpYI1oC5IDqadTfSb2e5I5i\nxQQFrSlHYduA0A9Wbs+dlQcZFiQ0ikajyMnJAcdxij1u9YKeqlwxQwvh/FWpa0pGsmTbnIxwT0oE\nIksSGRK++OILTJ8+HWVlZbrPr5TjDVtRUYFvvvkG+/fvx44dOzB16lRUV1fruo7mCpsw0bgRkBkB\nNU8TWbKnzGSpE4pFUWztSY3bixqQOEpPg+xEXqNS12Rkb6MSSLVJKO1f1QNaov5EEDvkCK+JrhkA\n6urqLO/3BOLP95TjnqTXcGyWLNl68ldffYX7778fK1euROfOnTW9hhjkeMOWl5dj0qRJAICBAwfi\n1KlTOHbsmK3KhU2YPNLS0vhJI5RuYicCsHXCZKhFCb9oQl9OoXeqEcINlhiMSF/JvSYASdGuIceP\nVcz8wYj7ZCRZCiF1TWTSQffRyqgfkD8MG4i9JgBNronukZreaTrwCr/De/fuxdSpU/H222+ja9eu\niq9PKaS8YQ8fPowuXbrw/+7cuTMOHTpkEyZswuRBJ0qqYfj9fn5qB81WtFpUo6SPT8ztRS+XISuM\nugHxa2LN1KPRM2PVzIbalGO8+6RWYGN1mwRdE0VRJJ6rra01fDyZFJSQpRiE90k42Ueu2pvIksoG\nhP3796O4uBj/+Mc/cP755yten1LE84YFmk4fsfKgk0ywCfNXfPHFF7jwwgv5aAYAP2EdOBNlmukF\ny0KLcTk7fkhqU5Z7Uk6W+ZG0MVFLD5u6NXtT1ktoJLxP7KYsV4ySLC0sQmKgsoGR48mkoJUshUhk\nayclWBOa7RO+++47TJkyBW+88QYuvPBCzetLhGAwiDFjxuDWW29FUVFRk9936tQJBw8e5P996NAh\ndOrUyfB1NQfYKlk0foDuuece7NmzB4MHD0ZhYSE8Hg8mTpyIsrIyXHDBBTE9hOx0C73s7KTA1gk9\nHo+uG4tSlyG9hj5rRbz3RErJaZTq0YyZmmI9uWKKW9bg3spMCPueSE3SkFJG6y2w0Zss40GoYAcQ\ncxil7w77nvzwww+47bbbsGLFCvTs2dPQ9dEaJ02ahLPOOgtLliwRfUxFRQWWLl2KiooKVFdX48EH\nH0xF0Y/dVpIIwWAQH3zwAZYsWYKqqir87ne/w4QJE3DFFVfEbExSRKM3eeo9bSTRa7GmAkKiUdJj\naSSUWN3JJRq1MGriSDyIedySuUYyGNxLpRzjQcp6UKvAxsoDhNBfORKJ8JGp3+9Hbm4uDh8+jIkT\nJ+Lll19G7969TVmXHG9YAPj973+PyspKZGVlYfny5ejfv78p60si2IQpB6+88gpmzZqFt956CxzH\nobS0FDt27ED//v1RVFSEwYMHx6RExdxr9EgHUk2M4zjTG7yFpgIOhwORSASZmZmWWu5pMeqWMlNX\nO18xGQ4QdE3s7FUr2ooIUspPpWDJU202J9mibdoP9u7dixtuuAGXX345jh49iiVLlmD48OGWrc+G\nJGzCTIRvv/0Wo0ePxrp163DRRRfxPw+Hw6iurkZpaSm2bduG3r17o7CwENdcc03MZqkXeSZL8z3Q\nuPH4fD5e/CTclM2Cng4xLHmSt6gSorFy4ggL4QHCaEu7eDAq2lbjNJQsZBmNRuH1ept8j/fv34+Z\nM2ciFAph586duOiii1BYWIipU6eidevWlq3XRgxswpQD+lJKIRKJYNeuXSgtLUVVVRW6d++OoqIi\nDBkyJGajkLJ+S1RLsyLNJwaqE7JDn/WO0uTCSKGR2GiyeOlAM9s14iFRtK23pV08mNUDK8dpKJnI\nUqz39KeffsL48eOxePFiDBo0CIFAAB988AHWrVuHefPmmTa2y0ZC2ISpNyKRCHbv3o1Vq1bh/fff\nR7du3VBUVIShQ4fGmCLLJc9kGf8kp04oVUvTOx1ohqiGBZsOFEZp1MaSDGSpJNpOVJ/WAqsMI8RE\nQxzH8QcIKxXCUmT5yy+/YNy4cXjyySdx9dVXW7Y+G7JgE6aRiEaj+PLLL1FaWorNmzejU6dOKCws\nxPDhw5GVlRXzOLEpJNFo40zOZNmMlUxgkYrStJKn1dG2MEoDwK/FSoWwltS0nkKoZHJXohYw6vnU\n05VH6VrEyPLkyZMYN24c/vznP2PIkCGmrceGatiEaRai0Sj+85//oLS0FJWVlWjXrh0KCwtx4403\nIjc3N+ZxoVCIb75nR3hZcULWK/UpNIdXU0tLBlENEJuazsjIiElxGjWaLN5a9PRjlUqxy2ntSJY6\nLtDYLx0MBvlsiB6iITWQuj+nTp3C+PHj8fjjj2PYsGG6vuadd96JDRs24Oyzz8YXX3zR5PdVVVUo\nLCzEeeedBwAYM2YMZs+eresaWihswrQC0WgU3377LcrKyrBhwwbk5eWhoKAAo0aNgtvtxoMPPojb\nbrsNAwcOjEmdmVUfJBglNBJK6+WQZ7KnptXWp7WADjNG+bFKtXaIZQmSmSyFMKuWK0WWNTU1GD9+\nPGbMmIERI0bo9nqEDz/8ENnZ2bj99tslCXPx4sUoLy/X/bVbOOxpJVaA4zhccMEFmDFjBh555BEc\nOHAAZWVlGDt2LI4ePYrOnTujS5cuPCkK/TjZSRBGkaeRqc94LkNiJ38tjkZ6ghXVZGdnx7znQpNu\nuU4vamGGalquxy2QHL69QGKyBGI/f0Y5DVGaXEiWXq8XEyZMwMMPP2wIWQLA1Vdfje+//z7h+mzo\nAzvCtAAHDx7EyJEjcdlll6Fv375499134XA4kJ+fj/z8fLRr147/0glbIADoKq6xKvUpZv5A67F6\n2LLaOqGWFKcUzBY9Sa0hGAzyQ9atHrgMyCPLeNDLaYg+KxzHxZBlXV0dbrnlFtx77724+eabFa9P\nCb7//nvk5+eLRphbt27FzTffjM6dO6NTp07461//iksuucTQ9bQQ2CnZZEBdXR169uyJ+++/Hw89\n9BDvsXns2DGsWbMGa9euRTAYRH5+PgoKCtChQ4cY8tRTXJMs0RylG8nk3or6IEGvOqGSFKcUrBY9\nsWDT5GykZvbAZUA7WQqh1mlI6mDV0NCACRMmYMqUKfjd736neX2JEI8wa2tr4XQ64fF4sHHjRjzw\nwAPYt2+f4WtqAbAJM1nwzTff4IILLhD9XTQaxc8//4y1a9dizZo1aGhowIgRI1BQUIAuXbroRp7J\nYtIttP/jOM70+iDByH5PNksgp5abLApUQLr31GjrQTHoTZZikCMakiJLn8+HW2+9FbfeeismTJhg\nyPqEiEeYQnTr1g2ffPIJ2rRpY8LKmjVswmyOOHHiBMrLy7F69WqcOnUKN954IwoLC9G1a9cmA5Tl\nNN8r8WI1GolSn2ZuyGa6K7EHHbENuTmQpRBGpKOFMIMshRArHVBPrtC20u/3Y9KkSfjtb3+L2267\nzbT0eTzCPHbsGM4++2xwHIedO3di7NixCWueNgDYhNn8cfr0aaxfvx6rV6/G8ePHMWzYMBQWFuLC\nCy9sMkZITJnqdDp5EYfRZu6JoDT1aaTLkJzpGkZBuCE7nU6Ew2G43W7LyVLtXE090tFCWEGWQlAq\n2ufzIRqNwul0Yvfu3TjrrLNw3nnn4Y477sDo0aNx5513mkaWt9xyC7Zu3Yqff/4Z7du3x9y5c/lJ\nKcXFxXjuueewbNkypKWlwePxYPHixbjiiitMWVszh02YLQm1tbWoqKhAaWkpDh8+jOuuuw5FRUXo\n0aOHJHmSF2xmZqbhPWnxoDWa0zOaSbY6IbWOsMPMrajl6jmEWmk6mgU7yi0ZMiINDQ2IRqNwu90I\nh8NYunQpli5dCqfTiZ49e2Lu3LkYMGCApeu0oQtswmypqK+vR2VlJUpLS/Hddyy44MAAACAASURB\nVN/h2muvxU033YSePXvC4XDgyJEj2LVrF66//no4nc6YVKBZ5twElqBcLpduzfdqoplkMUcAmqY+\npRyhjB62DOhLlkIo6YtMRrIUeveGQiEUFxfzJZJ169bh5MmTuP/++/HYY49Ztl4bmmETZirA5/Nh\n06ZNKCsrw9dff41+/fph48aNmDx5MmbMmMF/0c2a6cnCaIJSYqSeTM33idTKUi0QRvTlmikGi+dx\ny3Fc0pNlOBzGvffei379+mH69On8z/ft24djx47ZfrHNGzZhphq2bduGgoICXHnllTh+/DiuvPJK\nFBYW4vLLL4/ZgMwgTysISspInYRPVrfTAMqjOWE6GoBuMzCJLK0YQi2MqAlWH2josxIOh5uQ5bRp\n09C9e3c88sgjlo7gs2EIbMJMJWzYsAGTJ0/GihUrMGrUKIRCIWzduhWrVq3CJ598goEDB6KgoACD\nBg2K2aiNGIidDP2ewub7tLQ0vvneqs1OazSndDRZPCSDqAY4Q1DBYBBpaWkIhUKGKG6VrEVIlpFI\nBNOnT8e5556LWbNm6bqeRN6wADBt2jRs3LgRHo8HK1asQL9+/XR7fRs8bMJMJbz77rto164dBg4c\n2OR34XAYH374IcrKylBdXY3+/fujqKgIgwcPluyzU9sTmSz9nuxahM33ZqSjhTCCoOKNJovXfJ9M\nqU/hWoxQ3MpdixRZPvLII2jbti3mzp2r++sn8oatqKjA0qVLUVFRgR07duCBBx5AdXW1rmuwAcAm\nTGDVqlWYM2cO9u7di3/961/o37+/6OO6du2K3NxcniB27txp8krNQzgcRnV1NUpLS7Ft2zb07t0b\nhYWFuOaaa2LqjErJM9k3YvZ3ekfUateiJ4TqaLFDQbLdIzn9wVoUt0rXIkaWs2bNgsfjwfz58w0j\n63h9lffccw+GDBmCcePGAQC6d++OrVu3on379oasJYVhm6/36tULa9asQXFxcdzHcRyHqqqqlHDD\ncDqdGDx4MAYPHoxIJIJdu3ahtLQU8+bNQ/fu3VFUVIQhQ4YgIyND0nCcHUtGUUEymSPEWwvHcXC5\nXHC5XJLm3Hq1dZj5vghN7+l+NTQ08CnbcDjMk0Iy3yMWTqeTz1Swafb6+npdDjt0iKC1sGQ5d+5c\npKWloaSkxLI0/uHDh9GlSxf+3507d8ahQ4dswjQJKUWY3bt3l/3YVHT4dzgcGDBgAAYMGIBIJII9\ne/Zg1apV+Mtf/oJu3bqhqKgIQ4cO5YUYUtM6CMIpH2Yj3sQRMQjJM96hwOi16AmHw9HkUOD3+3kj\n9WAwaEmvJ6DtECE2iYQOBWoPO2IRdzQaxfz58+H3+/G///u/lvdYCvcmW3BkHlKKMOWC4zgMHToU\nTqcTxcXFuPvuu61ekulwOBzo27cv+vbti3nz5uHLL79EaWkpnn76aXTs2BFFRUUYPnw4srKy+I3J\n6/UiEonwz+H1ek2d6cmCtd1jIwW5EBvhFQqF4PV6Fbd1SLUlWAUS0uTk5PApTuFoMjPqzXpG3IkO\nO3IsFcXqytFoFH/5y19w8uRJPPfcc5aTZadOnXDw4EH+34cOHUKnTp0sXFFqocUR5rBhw3D06NEm\nP58/fz7y8/NlPcdHH32Ec845Bz/99BOGDRuG7t27p3RPFcdx6NWrF3r16oU5c+bgP//5D0pLS3HT\nTTehXbt2KCwsxGWXXYZbb70VDz/8MMaMGQPgTL3JjJmeLPSaOEJg1650VqlW4tYTYsRNkTPbrsJe\nl1HKVLZOqHfELTzsyLkuKbJcsmQJDh8+jBdffNFysgSAgoICLF26FOPHj0d1dTVatWplp2NNREqJ\nfghDhgzBU089JSn6YTF37lxkZ2fj4YcfNmFlzQvRaBTffvstXnrpJTz//PO46qqrUFhYiNGjR6NV\nq1YxjzNypicLmjjidDoNN1EXXpdQwQlAdFaiFWBt3RLN+DRamSolqjEaUtdFESnbfxqNRvHss89i\n7969eOWVV0xTeCfyhgWA3//+96isrERWVhaWL18uax+zoRi26IeF1EGB5jLm5OSgrq4OmzZtwh//\n+EeTV9c8wHEc6uvr8eabb2LevHnIz89HWVkZJk6cCLfbjfz8fIwePRpt2rThhSZs7yBt4HptxmYP\nW+Y4Lu51ATBkVJhSJJoKIwR7XSQaYq9LizLVKrIEYq+LMgWUEgYax+598cUXuOGGG/Dmm2/iyy+/\nxIoVK0xth3rrrbcSPmbp0qUmrMSGGFIqwlyzZg2mTZuGn3/+GXl5ebxt3I8//oi7774bGzZswHff\nfcdPSA+FQpg4caLtCSmBaDSKQYMG4cEHH8T48eNjfn748GGsXr0a5eXlcDgcyM/PR35+Ptq1a2fI\nQOxkMlEn4qbh4FpJRguUkmUiiI0mU2KkbhVZioF1NgKAXbt24Y9//CM+/fRTtGrVCnPnzkVhYSHO\nOussS9dpwxLYfZhmQG6vZ2VlJR588EGEw2FMmTIFM2bMMHml+iAQCMT1hY1Gozh27BjWrFmDtWvX\nIhgMYvTo0SgsLESHDh1UzfQUIpnmR4pFuVpIRgv0JkshlBqpJytZsmnY119/He+99x5Gjx6N9evX\nY8uWLbjsssuwZs0a5OXlWbpmG6bCJkwzsHfvXjgcDhQXF0vWScPhMC6++GK899576NSpEy6//HK8\n9dZb6NGjhwUrNg/RaBQ///wz1q5di7Vr16K+vh4jRoxAQUEBunTpImumZzKbqMuZqylFMnq7DOkt\nfJLzevGM1JOJLMmqUUiW//jHP7BhwwasXLmSP3jV19dj69atuPHGGy1ftw1TYROmmYgnLPr4448x\nd+5cVFZWAgAWLlwIAHj00UdNXaPVOHHiBMrLy7F69WqcOnUKN954IwoLC/lRSQQp8qSoJRnIUk1K\nWMr0XqvLkNlkKfb6rCsUvX4y2CNKmd2/8847WL16NVatWmX6EHEbSQnRL431OukUhJhbx+HDhy1c\nkTVo06YNJk+ejPLycqxfvx7nnnsuHn/8cdxwww1YtGgR9u3bh2g0yjeoZ2dnIzs7G06nEz6fDw0N\nDTERglVgU8JK6qdkKJCVlYXc3Fykp6cjGAyitrYWXq+XNxdQAiJLp9NpmdiIWjfoIENim7q6OtTW\n1vLRptn3TIosV69ejZUrV2LlypWGkGVlZSW6d++OCy+8EE8++WST31dVVfGain79+mHevHm6r8GG\nPkhZlawWaO31tFM7TZGXl4eJEydi4sSJqK2tRUVFBUpKSnD48GFcd911KCoqQo8ePeBwOPDWW2+h\nb9++6NmzJ6LRaBNrNDNN1PWa8amHy5CZLTWJQNE/62wk1hNpVHuREFJkWV5ejtdffx1r166F2+3W\n/XXD4TB+//vfx5RfCgoKmpRffvOb36C8vFz317ehL2zCVIHNmzdr+nuhW8fBgwfRuXNnrctqMcjJ\nycG4ceMwbtw41NfXo7KyEosXL8Z3332Hjh07YteuXVi3bh2fhnW5XKJ+qWYNxNY7JSzmMpTIjYfq\np9QykQxkKaxZCts6jGovEkKKLCsqKvDyyy9j7dq18Hg8ur0ei507d+KCCy5A165dAQDjx4/HunXr\nmhBmKlpxNkfYKVkDIfUl+O///m/s378f33//PQKBAN555x0UFBSYvLrmAY/Hg5tvvhlvvvkmrr76\nauzatQtDhgxBcXExnnjiCXz66aeIRCKS6c2amhrU1dXxczD1QjAYNEVsRATp8XiQk5PDjyZj05vB\nYJC3IUwGsmxoaEgo8OE4jo+Ec3Jy+Mf6fD7U1taivr6eV0trQTAYFCXLzZs347nnnsPq1av5thIj\nIKf8wnEctm/fjj59+mDkyJH46quvDFuPDW2wI0ydwfZ6jho1SrTXMy0tDUuXLsXw4cMRDodx1113\ntXiFrBZEIhHce++9+PTTT/HZZ5/hrLPOQiAQwPvvv48VK1Zg9+7duPLKK1FUVITLL79c1Gxczwkk\nFBWZPRBbrPE+EAjA7/cDOONkY+aQZRZaPHPFppD4/X5NqXa6T0Ky/Oc//4nFixdj3bp1yM3NlX+B\nKiBnvf3798fBgwfh8XiwceNGFBUVYd++fYauy4Y62CrZZooTJ05g3LhxOHDgALp27YqVK1fG2NER\nWspsz5dffhnjxo1DTk5Ok9+FQiFs3boVq1atwieffIKBAweioKAAgwYNitko9RiITek9s8lSDGwb\nC0XUZg5ZZmGUwbxaJbHUoeaDDz5ASUkJ1q1bZ8r4vurqasyZM4dXxC9YsAAOhyNu33W3bt3wySef\npMR4wSSG3VbSkkBT3x955BE8+eSTOHnyJN+ewiLVvnzhcBjbtm1DaWkpqqur0b9/fxQVFWHw4MEx\nGydLnqFQSJawRqoWZgUikQi8Xq9oG4vQAMJolyGzprGw2YJ4Bx4psvzoo48wZ84crFu3Dm3btjVk\njUKEQiFcfPHF2LJlCzp27IgBAwY06bk+duwYzj77bHAch507d2Ls2LH4/vvvTVmfDUnYhNmSwE5a\nP3r0KK699lrs3bu3yeO6deuGXbt2paS9VzgcRnV1NcrKyrBt2zb06tULhYWFuOaaa2IUrWLkKRwH\nRc4wyUCWSno+5RpAqIVVo8uk7pnD4RDNAOzYsQOzZs3C2rVrcfbZZ5uyRsLGjRt5V6+77roLjz32\nGF544QUAjYbqzz33HJYtW4a0tDR4PB4sXrwYV1xxhalrtNEENmG2JLRu3RonT54E0Lh5tGnThv83\ni/POOw95eXkpPdsTaCSOXbt2obS0FFVVVejevTsKCwtx3XXXxZAO2/pATfdkb8c6w1gFLTaAQos+\nrUriZJnzSffM7/fzsz7D4TC+/fZb9O/fH59++ilmzJiBNWvWoEOHDpas0Uazgz2tpLlBqt+zpKQk\n5t+0qYvBnu3ZCIfDgQEDBmDAgAGIRCLYs2cPVq1ahUWLFqFbt24oKirC0KFD4Xa7eWGNy+VCXV0d\nIpEIOI6LqReaPRAb0O6ZSwYQNIFErA1HrstQspAlcEZYEw6Hec/cL7/8EpMnT0YgEEBaWhqWLFmC\ndu3aWbZGGy0DdoTZTNG9e3dUVVWhQ4cOOHLkCIYMGSKakmVhz/Zsimg0ii+//BKlpaXYvHkzOnbs\nyJPn448/Drfbjfnz5/NRCxt5GjlgWQgjDeaFPrB0YJCq5yYTWQJn+mGFadg9e/Zg9uzZ6NOnD95/\n/338+OOPKCwsxBNPPGH3PdtIBNsaryWhoKAAr732GgDgtddeQ1FRUZPH1NfXo7a2FgD42Z69evUy\ndZ3JDo7j0KtXL8ydOxcfffQR5s2bh++++w6XX345PvroI1x66aXwer18S4fb7UZOTg7f6N7Q0IDa\n2lo0NDTwQ6T1RigUQl1dHdxutyHTWMhliHpYXS4XwuGwqEVfspIlZQYIX331Fe6//368/PLLeOqp\np/DZZ5/h448/xsUXX2yYSYGNlg87wmymOHHiBMaOHYsffvghpq3Enu2pDcFgELfeeit++eUXLF68\nGBs3bsSGDRuQl5eHgoICjBo1KqZ9R8+ZnmKwchqLUFjDOvYkG1my783evXtRXFyMt99+G+eff76F\nK7TRjGGLfmzYSITHHnsM//73v2OMuKPRKA4cOICysjK8++67yMzMREFBAUaPHo02bdoknOmpljyT\nbXRZfX09H2myFn1W1nOF783+/fsxZcoUvPnmm7jooot0f105c2ynTZuGjRs3wuPxYMWKFejXr5/u\n67BhOGzCtCEPqbwpnD59Gh6PR5KgotEoDh8+jNWrV6O8vBwOhwP5+fnIz89Hu3btmpAnRWhKWzqk\n6nJWgNKw0WiUT2cmQz1XSJbfffcd7rjjDrz++uuGOGfJmWNbUVGBpUuXoqKiAjt27MADDzyA6upq\n3ddiw3DYNUwbiUHTFSorK/HVV1/hrbfewtdffx3zmIqKCnzzzTfYv38/XnzxRUydOtWi1eqPvLy8\nuNEcx3Ho3Lkzpk2bhs2bN+ONN95AWloaiouLkZ+fj+effx5HjhxBNBqF0+lsMpbM7/cn9Eoln9pk\nJEtSZFtVz5Uiyx9++AF33HEHli9fbpjNJGuknp6ezhupsygvL8ekSZMAAAMHDsSpU6dw7NgxQ9Zj\nw3zYhGkjBvamIB8cx6F9+/aYOnUqKisrsXLlSuTm5mLatGkYPXo0li5dih9++EFypqff70dNTU0M\neVrlUysGMbIUgjVRz87O5mubLHnqYaIOxCqFWbI8fPgwbr/9drz88su49NJLNb+OFOQYqYs95tCh\nQ4atyYa5sPswbcRA7Au/Y8eOhI85dOgQ2rdvb9o6kw0cx6Ft27aYMmUKpkyZghMnTqC8vByPPPII\nTp06heHDh6OwsBDdunUT7YekmZ4AkJGRYbmbkByyFILIkwiUUtJ+v79Jr6fStK1UW82RI0dw6623\n4vnnn0efPn0UX6cSyF2z8HBgtTjKhn6wI0wbMbA3BX3Qpk0bTJ48GeXl5Vi/fj26du2KJ554AsOH\nD8eiRYuwb98+PvJ0uVz44osvADSSZTgcjhlLZvasxGg0ivr6ekVkKQaplLQwqk4EKbI8evQoJk6c\niGeeeQb9+/dXtUYlkDPHVviYQ4cOoVOnToavzYY5sCNMGzGwNwX9kZeXh4kTJ2LixInwer2oqKjA\n/PnzcejQIVx33XXweDx45pln8MEHHyAvLw9ArJmAGicetSCyBKCJLIWIF1XHs+ijiSxCsvzpp58w\nceJELF68GAMHDtRljYnAzrHt2LEj3nnnHbz11lsxjykoKMDSpUsxfvx4VFdXo1WrVimdeWlpsCNM\nGzGQM9y6oKAAr7/+OgDYm4JCZGdnY+zYsVi5ciXee+89/PLLL1i4cCF69+6NV199FXv27OGt+IQD\nsUOhkKiZgF4wiiyFkDvsm53IwpLlL7/8gltuuQVPPvkkrrzySkPWKAZ2ju0ll1yCcePGoUePHnjh\nhRd4M/WRI0fivPPOwwUXXIDi4mL87W9/M219NoyH3VZiowkSTVcAwCtps7KysHz5clNSYi0NS5cu\nxaJFi7BlyxZ06dIFmzZtQmlpKb7++mtcc801KCoqQt++fWMiSj1meorBLLJMtAb22gDwaly6tpMn\nT2LcuHH485//jCFDhpi+RhspA7sP04aNZIHX60VBQQFeeeUVdOvWLeZ3gUAA77//PkpLS7F7925c\neeWVKCoqwuWXXy5JnnJneoohGciSBUWWpBKuqKjA008/jREjRmDLli3405/+hGHDhlm6RhstHjZh\n2kguJDJIqKqqQmFhIc477zwAwJgxYzB79mwrlmoZQqEQtm7dilWrVuGTTz7BwIEDUVBQgEGDBsUo\naeXM9BRDMpJlXV0dXC4XP3YtEAigsrISy5YtwxdffIH/+q//wpgxY3DzzTfjkksusXzNNlokbMK0\nkTyQ45pSVVWFxYsXo7y83MKVJg/C4TC2bduG0tJSVFdXo3///igqKsLgwYNjejbFZnqyFn3s45KR\nLNPT03lbQqAxGh8/fjymT5+OkSNHYvv27SgrK8PatWvx8ccf45xzzrFw1TZaKGzCtJE8+PjjjzF3\n7lxUVlYCABYuXAgAePTRR/nHVFVV4amnnsL69estWWMyIxwOo7q6GmVlZfjwww/Rq1cvFBUV4Zpr\nrokRyEiRZ1paGnw+HziOS2qyrKurw4QJEzB16lR+kAAhGo1avm4bLRa2NZ6N5IEc1xSO47B9+3b0\n6dMHI0eOxFdffWX2MpMWTqcTgwcPxuLFi7Fjxw7cc8892LZtG4YPH4577rkHGzduhN/vb2Jj53a7\nEY1GUVdXh3A4DIfDgXA4bHqvJwspsmxoaMCtt96Ku+++uwlZAnbvrw3zYfdh2rAEcja7/v374+DB\ng/B4PNi4cSOKioqwb98+E1bXvOBwODBgwAAMGDAAkUgEe/bswapVq7Bo0SJ069aNH4jtdrsRCoXw\n4osvYvLkycjKykIoFOIdffQcSyYXLFlSzRIAfD4fbr/9dtx+++0YO3asoWs4ceIExo0bhwMHDsSM\nyhOia9euyM3N5ZXJO3fuNHRdNpIPdoRpwxLIMUhgjb1HjBiBYDCIEydOmLrO5gaHw4G+ffuipKQE\nH3/8MWbNmoWvv/4a+fn5mDhxIgoKCrB9+3Z+4HI8D1ijDNQJFOmmpaUhIyODJ2m/34/Jkydj7Nix\nmDBhgmGvT1i4cCGGDRuGffv24frrr+fLA0JwHIeqqip89tlnNlmmKOwapg1LEAqFcPHFF2PLli3o\n2LEjBgwY0ET0c+zYMZx99tngOA47d+7E2LFj8f3331u36GaMhoYGDB8+HLW1tcjOzkbbtm1RUFCA\nESNGIDc3N+axes70lAKRJfnO0vMGAgHceeedGDlyJO666y5TIt3u3btj69ataN++PY4ePYprr70W\ne/fubfK4bt26YdeuXTjrrLMMX5MNyyH6wbNTsjYsAeuaQgYJ5JoCNBoklJaWYtmyZUhLS4PH48Hb\nb79t8aqbJ3w+H26++Wacc845eP/99+F0OvHtt9+irKwM48aNQ25uLgoKCjBq1Ci0atVK1EDd5/Mp\nnukpBSmyDAaDuPvuuzF06FDTyBJoPJiRU1X79u0lJ+9wHIehQ4fC6XSiuLgYd999tynrs5E8sCNM\nGy0ed955JzZs2ICzzz6bNzkXoqUOxAaA48eP46mnnkJJSUmTkWHRaBQ//PADysrKsH79emRmZqKg\noACjR49GmzZtYkgrEonwkada8pQiy1AohOLiYgwaNAj333+/7mQ5bNgwHD16tMnPS0pKMGnSJJw8\neZL/WZs2bURT/0eOHME555yDn376CcOGDcOzzz6Lq6++Wtd12kga2G0lNlITH374IbKzs3H77beL\nEmZFRQWWLl2KiooK7NixAw888ACqq6stWKm1iEajOHz4MFavXo3169eD4zjk5+cjPz8f7dq1kyTP\ncDjMp23jkSeRpcPhgNvt5h8XDodx7733om/fvnjooYdMV792794dVVVV6NChA44cOYIhQ4aIpmRZ\nzJ07F9nZ2Xj44YdNWqUNk2G3ldhITVx99dVo3bq15O/tgdiN4DgOnTt3xrRp07Bp0ya88cYbSEtL\nQ3FxMfLz8/H888/jyJEjTQZi5+TkwOl0IhAISI4li0eWDzzwAHr27GkJWQKNwwRee+01AMBrr72G\noqKiJo+pr69HbW0tgMbe0E2bNqFXr16mrtOG9bAJ00bKQ2ogdiqD4zi0b98eU6dORWVlJVauXInc\n3FxMmzYNo0aNwtKlS/HDDz/EkGdWVhZycnKaTB/x+/2iZBmJRPDQQw+hW7dumDFjhmV9lY8++ig2\nb96Miy66CO+//z5vnvHjjz9i1KhRABpnb1599dXo27cvBg4ciNGjR+OGG26wZL02rIMt+rFhA/ZA\n7HjgOA5t27bFlClTMGXKFJw8eRLr1q3DjBkzcPLkSQwfPhyFhYXo1q0bP7rL5XIhGo0iEAjA7/fz\nrjy7d+9Gx44d0bZtWzzyyCM455xzMHv2bEvf7zZt2uC9995r8vOOHTtiw4YNAIDzzjsPn3/+udlL\ns5FksCNMGykPeyC2MrRu3RqTJ0/GunXrsH79enTt2hVPPPEEhg8fjkWLFmHfvn08Wc78/+3dXUiT\nfRjH8e+2RDDFCGRJCBYulubLKvKgzIIM5nQlRhj2AoWJEavjB0+EEDqIMIROgiA6KHoZDtIwiXVQ\nrEF5FpEJomO0kqi2olLacxDPyMep89HNZ/r7HIn73/eus2v//31f1/XXX3z58iW287x37x5lZWXs\n3LmT4eFh2tra9ONE0oZ2mLLiOZ1Ouru7aWpq0kDsecrNzaW5uZnm5mYikQi9vb10dnYyOjqKwWAg\nJyeHvLy82M6zo6MDg8FAIBDAaDRSXFxMWVlZrAWeyP+Z3pKVZe/IkSM8efKE8fFxzGYzHR0dsQHF\nGoi9+CYnJzl8+DCBQACLxcLw8DDV1dU0NDTg8XgIh8N0dXVhNBr5/v07AwMDjIyMcPbs2aUOXeQf\nKisRkeSanJzk2LFjfP78GbfbTWZmJj9+/KC/v59r164RDocZGBiY14BrkSWgshKRZDp58iRms3nG\ncgOv10tubi42mw2bzcaFCxdSHGFqbN26lfv378eaqWdmZlJfX09PTw+PHz9OarK8c+cOJSUlmEwm\nXr58OeO6hw8fYrVasVgsXLx4MWnxyPKiHabIIpmrQYIGYiff69evMRqNtLa2cunSpbhH64kML5cV\nTztMkWSaq0ECTC9fkcVltVrZtGnTrGv8fj9FRUUUFhaSkZFBU1MTPT09KYpQ0pkSpkiKaCD2/0Mi\nw8tF4lFZiUiKaCD24pipkXpnZyf19fVzXq+6T/mvlDBFUiQnJyf2t91u58yZM3z8+JG1a9cuYVTp\n59GjRwu6PpHh5SLx6EhWJEVCoVDsGabf7ycajSpZJtFMz4u3b9/O0NAQIyMj/Pz5k9u3b+N0OlMc\nnaQj7TBFFsmfDRIKCgqmNUjQQOzkc7vduFwuxsfHcTgc2Gw2+vr6CAaDtLS08ODBgxmHl4vMRWUl\nIiIiU6msRGS5GBsbY+/evZSUlLBlyxauXLkSd53L5cJisVBeXs7g4GCKoxRZXnQkK5KGMjIyuHz5\nMhUVFUQiEbZt20ZNTc2Uo8Xe3l7evn3L0NAQz58/p62tDZ/Pt4RRi6Q37TBF0tC6deuoqKgAIDs7\nm82bNxMMBqes8Xg8nDhxAoDKyko+ffpEKBRKeawiy4USpkiaGxkZYXBwkMrKyin/j1egHwgEUh2e\nyLKhhCmSxiKRCIcOHaKrq4vs7Oxpn//7pT4V7Yv8d0qYImlqYmKCxsZGjh49ysGDB6d9/u8C/UAg\nwPr161MZ4oIkOnmksLCQsrIybDYbO3bsSGGEstIoYYqkoWg0yqlTpyguLub8+fNx1zidTm7cuAGA\nz+djzZo1mM3mVIa5IKWlpbjdbnbv3j3rOoPBgNfrZXBwEL/fn6LoZCXSqSspBQAAAghJREFUW7Ii\naejp06fcvHkztrOC371UR0dHgd+NEmpra+nt7aWoqIjVq1dz/fr1pQx53qxWa8JrNQVGUkEJUyQN\n7dq1i1+/fs25rru7OwXRLC2DwcC+ffswmUy0trbS0tKy1CHJMqWEKSKzGhsb4/jx47x//x6DwcDp\n06dxuVxT1ni9Xg4cOMDGjRsBaGxspL29fc57L3TyCPzebefn5/PhwwdqamqwWq1UVVUldK3IfChh\nisisEmmSAFBdXY3H45nXvRc6eQQgPz8fgLy8PBoaGvD7/UqYkhR66UdEZpVIkwRI7nPEme797ds3\nwuEwAF+/fqW/v5/S0tKkxSErmxKmiCRspiYJBoOBZ8+eUV5eTm1tLa9evVrwd7ndbgoKCvD5fDgc\nDux2OwDBYBCHwwHAu3fvqKqqoqKigsrKSurq6ti/f/+Cv1skHk0rEZGERCIR9uzZQ3t7+7S6z3A4\njMlkIisri76+Ps6dO8ebN2+WKFKRBYvb4UMJU0TmNDExQV1dHXa7fca6zz9t2LCBFy9eaEC2pCuN\n9xKR+UukSUIoFIo9Z/T7/USjUSVLWXb0lqyIzCqRJgl3797l6tWrrFq1iqysLG7durWUIYskhY5k\nRUREpop7JDvXDlOjDURERNAzTBERkYQoYYqIiCRACVNERCQBSpgiIiIJUMIUERFJgBKmiIhIAv4G\nQj7Ty5/aXIsAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7a01d50>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from mpl_toolkits.mplot3d import Axes3D\n",
"from mpl_toolkits.mplot3d import proj3d\n",
"import pandas as pd\n",
"\n",
"np.random.seed(1)\n",
"\n",
"#Class 1\n",
"mu_vec1 = np.array([0,0,0])\n",
"cov_mat1 = np.array([[1,0,0],[0,1,0],[0,0,1]])\n",
"class1_sample = np.random.multivariate_normal(mu_vec1, cov_mat1, 20).T\n",
"\n",
"#Class 2\n",
"mu_vec2 = np.array([1,1,1])\n",
"cov_mat2 = np.array([[1,0,0],[0,1,0],[0,0,1]])\n",
"class2_sample = np.random.multivariate_normal(mu_vec2, cov_mat2, 20).T\n",
"\n",
"fig = plt.figure(figsize=(8,8))\n",
"ax = fig.add_subplot(111, projection='3d') #1x1 grid, 1st subplot. 3D plot\n",
"plt.rcParams['legend.fontsize'] = 10\n",
"#alpha = measure of transparency\n",
"ax.plot(class1_sample[0,:], class1_sample[1,:], class1_sample[2,:],\n",
" 'o', markersize=8, color='blue', alpha=1, label='class1')\n",
"ax.plot(class2_sample[0,:], class2_sample[1,:], class2_sample[2,:],\n",
" '^', markersize=8, alpha=1, color='red', label='class2')\n",
"\n",
"plt.title('Samples for class 1 and class 2')\n",
"ax.legend(loc='upper right')\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"PCA is an unsupervised learning algorithm, therefore we can disgard information concerning labels and merge the two numpy arrays. Once we have the data matrix, we need to center each feature before performing SVD to find the eigenvectors and eigenvalues. In SVD, the eigenvectors in $V$ corresponds to feature dimensions while the eigenvectors in $U$ corresonds to sample dimensions. "
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Dimensions of U:\n",
"(40, 3)\n",
"Dimensions of V:\n",
"(3, 3)\n"
]
}
],
"source": [
"all_samples = np.concatenate((class1_sample, class2_sample), axis=1).T\n",
"df = pd.DataFrame({1:all_samples[:,0], 2:all_samples[:,1], 3:all_samples[:,2]})\n",
"df_means = df.mean(axis=0)\n",
"\n",
"#Centering the data: \n",
"df_centered = df - df_means\n",
"\n",
"#Getting eigenvectors and eigenvalues\n",
"U, S, V = np.linalg.svd(df_centered, full_matrices=0)\n",
"S = np.diag(S)\n",
"print \"Dimensions of U:\"\n",
"print U.shape\n",
"print \"Dimensions of V:\"\n",
"print V.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us reduce all our item vectors from 3 dimensions to 2 dimensions by projecting the data onto feature eigenvectors corresponding to the top 2 eigenvalues, and then plotting the transformed samples in 2-dimensions."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAARIAAAEaCAYAAADdZNhOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYFNX18PHvAdnCRIVgxCCLYdCAshnFBRGUl0XFGI3x\nFxNjEDUbAhoRlSEwAyqiGA0ivAm4gqISzQK4wCvDYkADRgQBX2QfRDEEgg4BRpjz+6Nqhp6me7q6\nq6ure+Z8nqef6a6uqnt7mdO3bt06V1QVY4zxo07YFTDG5D4LJMYY3yyQGGN8s0BijPHNAokxxjcL\nJMYY3yyQxCAiJ4vIEhH5QkQeDrs+kUSkjYiUi0jWfnYi8oyIjAup7NdE5KfVPJ+WumXyNSbzmYtI\nLxEpSbGclLc9LpWNgiQipUDF4JbGwEHgiPv456o6KwPV+Dnwuaoen4GyaiLl6GeY2YJVL6+4LyID\ngZtVtUfkKqSnbqG9xmyUdYFEVfMq7ovIFpwvwsLo9UTkOFU9HFA1WgPrU9kw4HrlEgm7AhlQG16j\nJ1nbPI7mNrt2iMgIEfkUeFJEThSRuSLyuYjsEZE5ItIiYptFIjJWRN52D1PeFJFvuM81FJGZIrJb\nRPaKyD9E5Jsi8gxwIzBCRL4UkUtFpL6IPCYin7i3R0Wkfpx6PSUiY0RktojMcMtdLSLtROReEdkl\nIttEpE9EPU8QkSdFZKe7r3EVzVgRqSMiE0XkXyKyCbgiwft0t7uPL0TkIxG51F3eTUSWu691p4g8\nLiL1IrYrF5FficjH7rZjRaStu81/ROTFivUjXvO9br22iMiPq6nTABFZ5Zb9dxHpmKi+UdufJiJ7\nIx5PE5FdEY9niMiwiM/8ZhH5DvB/gQvcz3FPxC6but+bL0TkHRH5djV1v0hElrl13y4iN8ZYp0mC\n7+FAEdnklre54r0SkXwRWey+v/8SkRfj1SOqvJtEZJ27v00i8vMY68T8bESkgft92iYin4nIVBFp\nGKechJ9NJVXN2huwBbjUvd8L+AoYD9QDGgJNgavd+3nAy8CfI7ZfBHwM5LvrFAPj3ed+AfzNXS5A\nV+Dr7nNPA2Mj9jMWWAY0c29/r3g+Tr0KgQNAH6Au8CywFbjXfXwLsDli/38GpgKNgJOAd3EO4wB+\nidM6agE0cV/DEaBOjPfrDGA70Nx93Ar4tnv/bKAbzo9Ha2AdMCxi23K3HnlAB+AQsBBoAxwPrAVu\njHrNE93XfDFQCrSLeP/Gufe7AruAc933+Ub3c61XXX1jvLZtQFf3/v8HNgLfiXius3u/GBjk3v8Z\nsDRqP88Au4Fz3M9iJjArTpmtgS+A/3HXbRpRTuRrjPs9xDk83xfx3pwMdHDvzwLude/XBy6MU482\n7udTx318OXCae/9iYH/EexPvszndff5R4C/AiW5d/wY8ELFtSaLvUsw6hh0skgwkh4D61azfBdgT\n8bgYGBnx+FfA6+79m3ACQscY+6n8kriPNwL9Ix73BbbEqxdOIHkz4vGVwJeAuI+/7n4xjne/WAeB\nhhHrXw8sdO8vxA0q7uM+kV+qqHrn4/zT9gbqJXhvbwdejXhcDlwQ8XglcFfE44nAo1Ff1kYRz78E\njIp4/yoC7VQigrK77CP3C942ifo+B9wBNHe3fxDnx+A0YG/UZ14RSAZybCB5GvhjxOPLgPVxyrwX\neCXOc1W+I/G+hziBZC9wTeT75T73LPAHoEWC194m3mfuPv9nYGiizwYnkJcSERCAC3B/1KgaSDx/\nl1Q1dw5tXP9S1bKKByLyNRH5g4hsFZF9wGLgBBGJPHb9LOL+AZwoDDADeBN4UZzDlQkiEq/P6Fs4\nv3oVtrvLYtbL9XlUubvV/YTcx7h1aY3zy/Gp23zei9MkP8ld5xQgsid9e5w6oqobcQJEIbBLRGaJ\nyCkAInK62/z+1H2v7ge+EbWLXRH3D0Q9PsjR9w6cf94DEY+3uXWN1hq4s+K1ua/vVOAUVd0Ur74x\nLMb5ovcAlriPe+IEpKVxtokn+nXmxVnvVGBzop1V9z1U1f04LZpfAjvdz+AMd9MROP/c/xCRD0Xk\nJi+VF5HL3EOyf7vv5+VU/SzjfTbNgK8B70V8Fq+7y6uo7rsUS64Fkuhe8juB04FuqnoCzhdL8NAJ\npqqHVXWsqp4JXAgMwGl2x7IT51ehQit3Wbx6JdObX4LTovmGqjZxbyeoakU/wqdueZFlx6Wqs9Q5\nS9HarccE96mpOIcz+e57VUByn3/0a2oiIl+LeNyaqu9Jhe3A/RGvrYmq5qnqSwnqG20xThDphXPI\n+jbQHeczX+SxzskqwWk1xVOx/2q/h6o6X1X7crQ1Nc1dvktVf66qLXBaV1Oq668Bp48DeAV4CPim\nqjYBXqPqdz7eZ7MbJ3B2iPgsTtQ4ZyeT+GxyLpBEy8N5Y/aJSFNgTIx1YgYVEblERDqKSF2cw46v\nOHqaOXqbWcAoEWkmIs2A0Tgtmng89+ar6qfAfOB3IvJ1cTpX24rIxe4qLwNDRaSFiDQB7olbqNPq\nuNT9sh2i6qnzPPd1/tftiPyVh+pJnPsVikSknoj0wOkEnh2xbsX604BfitPZKyLSWESuEJG8BPWt\nwv2FPAjcACxW1S9xWn0/wAkysewCTpWITuU4ryOe54H/IyI/FJHjROQbItI5xmuM+z0UpwP/KhFp\njPMd21/xGt39nuqu+h+cf9byBHWq7952A+UichnOoXa0Yz4bt0U8DXhMRE5y69BCRI7ZPpnPBnIv\nkET/wjyG00G5G6cz9PUY62jU/YrHJ+N88ffh/FIv4mhw0Kjt7sPpM1jt3la6y+LVK3r7eOtUuBHn\ny7EO2OPWq7n73DScQ7AP3HJfibGvCg1wOn3/hdOSaYZznA8wHPgxTufhH4EXOfa9iRbvvQPnkHEv\nzi/dDOAXqrohel1VfQ+4FZjsvraPOdryq66+sSzCOUT8JOIxwD/jrP8WTifxZyJScajp5bPBrXsJ\nzmHDncC/gfeBTjH2U933sA5O384n7j56cDSInwO8IyJfAn/F6efYGue1VLyfXwJDcX5g9uD0p/01\nat1Pif/Z3I3T5/eOexi2AKc1Ff1eJPXZVHT+hcptFawEdqjqlWHXx1RPRHoBM1S1Zdh1MdkhW1ok\nw3B+jcOPasaYpIUeSNxjxMuB6dhIwVxiQd9UCj2Q4AyQuYvEnUwmS6jqIlWt9uyRqV1CDSQiMgDn\n4rj3sdaIMTkr1M5WEXkA+ClwGGd48fE4IwlvjFjHmtDGhERVPf3Ah9oiUdWRqtpSVU8DfoQzLPyY\nQWGJhuem8zZmzBgrz8rLyvIy/dqSkQ19JJGs9WFMDsqafCSqupj4IxSNMVks21okoevVq5eVZ+Vl\nZXmZfm3JyIqRrdVxLqDM7jqa3FX1QvHaLfr/TERQj52tWXNoY0xY7IfKf0C1QxtjjG8WSIwxvlkg\nMSbLFBYW8sgjj6Rtf4MGDeLkk0+mY8eOiVdOkQUSY6LMm7eEfv1G0atXIf36jWLevCWBbhct3R3A\nN910E2+88UZa9xnNOluNiTBv3hKGDXuTTZvur1y2efNoVq7cwJgxt6R9O4DnnnuORx55BBGhU6dO\ntG17NLvjtGnTmDZtGmVlZeTn5zNjxgwaNWrE7NmzGTt2LHXr1uWEE05g8eLFrF27lkGDBlFWVkZ5\neTmvvPIK+fn59OjRg61bt6b+pniRySG3KQ7TVWOCEv396tu3QEGPueXlzdfCwmlx95Pqdh9++KGe\nfvrp+u9//1tVVffs2aOFhYU6ceJEVdXK5aqqo0aN0scff1xVVTt27Kg7d+5UVdV9+/apquqQIUP0\n+eefV1XVr776Sg8cOFC57ZYtW/Sss87y/D5ELPP0f2qHNsZEOHQodiO9tLQPS5fGTeCf8nYLFy7k\nuuuuo2nTpgA0adKkyvNr1qyhR48edOrUieeff55169YB0L17d372s58xffp0Dh92Jna84IILeOCB\nB3jooYfYunUrDRvGnPcqEBZIjInQoEHs2Vbz8hbQo0f8FCypbucO+oq5HGDgwIFMmTKF1atXM2bM\nGA4ccGaZmDp1Kvfddx8lJSV897vfZc+ePVx//fXMmTOHRo0acfnll1NcXBy33HSzQGJMhKFD+5Kf\nP7rKsnbtihg+fFu1fR2pbnfppZcye/Zs9uxxZhSt+FsRXEpLS2nevDlfffUVM2fOrNxu06ZNdOvW\njaKiIk466SR27NjBli1baNOmDUOGDOGqq65izZo1yb14HyyQGBPhiisu5oYbWpGXtwBwWhQ/+UmL\nhB2mqW7XoUMHCgoK6NmzJ126dOHOO+8EjrZIxo0bx3nnncdFF11E+/btK5ePGDGCTp060bFjR7p3\n706nTp14+eWX6dixI127dmXt2rXceKOTkeP666/nwgsvZMOGDbRs2ZKnn3469TcoDrvWxtRq8Q4t\nioqms3Tpdnr0aJUwGKRju7DFeh+SudbGAomp1eIFktrGbyCxQxtjjG8WSIwxvlkgMcb4ZoHEGONb\n2PPaNBSRd0VklYisE5HxYdbHGJOasKejOAhcoqpdcGZ5v0RELgqzTsZUSPVsjt+zQOlMI1BSUsIl\nl1zCmWeeyVlnncWkSZPSst9ooR/aqOp/3bv1gbrAnhCrY0ylO2+9lfLy5GeSTXW7CulMI1CvXj0e\nffRR1q5dyzvvvMMTTzzB+vXr07b/CqEHEhGpIyKrgF1AsaquC7tOxqxcvpzy2bOZ9cQTgW/33HPP\n0blzZ7p06VI5GrXCtGnT6NatG126dOHaa6+tvNZm9uzZdOzYkS5dutCzZ08A1q5dy3nnnUfXrl3p\n3LkzGzdupHnz5nTp0gWAvLw82rdvz86dO5N6TZ54vUw46BtwAvAO0CtqedxLn43xK973694rr9Qy\n0Lvat9fS0lLP+0t2u0ylEVB1Ugm0atVKv/zyy2PqEet9IIk0AlmT2EhV94nIPOAcYFHkc4WFhZX3\ne/XqldXze5jct2vXLpqsWkU9YMj69Txx992MmDw5kO28pBEYNWoU+/bto7S0lP79+wNH0whcd911\nXHPNNYCTRuD+++9nx44dXHPNNeTn51fup7S0lGuvvZbf//735OXlxazLokWLWLRoUcLXGZPXiBPE\nDWgGnOjebwQsAXpHrXNMpDQmXWJ9vyYMHqy7IrITPdiypW7fti3hvlLZ7vHHH9eCgoIqywoLC/WR\nRx5RVdU2bdro6tWrVVX1mWee0YEDB1au9+677+ro0aO1TZs2lS2XzZs366RJk7Rdu3a6cOFCVVUt\nKyvTvn376qOPPprU+0AOJTY6BVjo9pG8C8xR1bdCrpOpxcrKythbXMw3I5bdVlLC5MGDA9kuE2kE\nbr75Zjp06MDtt9/u8V1IXtinf9eo6tmq2kVVO6nqw2HWx5hXn3ySazdsqLKsMdBp2TKWV5MoKNXt\ngk4j8PbbbzNz5kyKi4vp2rUrXbt2DSQRtF39a2q16KteRw4aRP3Nm49Zr1yVQ23bMuGpp2LuJ9Xt\nsoWlETDGB0sj4LA0AsaY0FkgMcb4ZoHEGOObBRJjjG9ZM7LVmLCke67d2sgCianV7IxNetihjTHG\nNwskxhjfLJAYY3yzQGKM8c0CiTHGNwskxhjfLJAYY3yzQGKM8c0CiTHGNwskxhjfLJAYY3wLe+7f\nliJSLCJrReRDERkaZn2MMakJNdWiiDQHmqvqKhHJA94Dvq+q6yPWsVSLxoQgZ1ItqupnqrrKvV8K\nrAe+FWadjDHJy5o+EhFpA3TFmd/GGJNDsiIfiXtY8ydgmNsyqcKm7DQmeH6m7Ax9OgoRqQfMBV5X\n1cdiPG99JMaEIGfmtREnx92zwL9V9Y4461ggMSYEuRRILsKZOHw1UFGRe1X1jYh1LJAYE4KcCSRe\nWCAxJhw5c/rXGFMzWCAxxvhmgcQY45sFEmOMbxZIjDG+ZcXIVmOy1bx5S5g0aT6HDh1HgwaHGTq0\nL1dccXHY1co6FkiMiWPevCUMG/YmmzbdX7ls8+bRrFy5gTFjbgmxZtnHxpEYE0e/fqOYP/++Y5bn\n5S1g+PBtNT6Y2DgSY9Lg0KHYDfbS0j4sXbo9w7XJbhZIjImjQYPDMZfn5S2gR49WGa5NdrNAYjyp\njYeXQ4f2JT9/dJVl7doV1YrDmmRZIDGe3HnrrZSXl4ddjYy64oqLueGGVuTlLQCclshPftLCgkgM\ndtbGJLRy+XLKZ89mVufO/GTIkLCrk1FO0JjO0qWj6dGjlQWROOysTQ2lqjjpXvxvM/J736NozhwK\n2rdnzIoVNG7cOF3VNFnMztqYlA5FYm2za9cumqxaRT1gyPr1PHH33WmspakpLJDUQJWHIk884Xub\nZ8aO5WclJQC0BPRvf6Nku536NFVZIKmBXh0/noe/+IIPpk5l//79KW9TVlbG3Oefp1nEereVlDB5\n8OAAam1ymQWSGiaVQ5F42zxSUECXffuYFbFuY6DTsmUsLy4OovomR9lZmxrm2XHjGBh9KDJiBC1b\nxR9AFW+bf7z0Ei8DP//a11h79tnUr1sXgHJVVs+YwQWXXBLwqzG5IvRAIiJPAVcAn6tqx7Drk8vK\nysrYW1zMNyOW3VZSwtjBg5kwZ05S2xTccgsXAvWAsf/9L7M6d2bE5MkB1t7ksmw4tHka6B92JWqC\nV598kms3bKiyLNGhSLxtzlqyhA5Z1sk6b94S+vUbRa9ehfTrN4p585aEWh9zVFaMI3Gn65wTq0Vi\n40i8GzloEPU3bz5mebkqh9q2ZcJTT3napry8nM0rV9LiwAEmuMv2A2MHDIjbsglarEv68/NHc8MN\nuT9ILFtzniQzjgRVDf0GtAHWxHlOTWbNmjJFVx53nCpUuc1s2lSXLVyY1rLKy8s9rde3b0F0dRRU\n8/Lma2HhtLTWKZPmzl2sbduOrPKa8vN/mxWvyf3f8/Q/HHofiRc2929mrV6xgo+6dye67RFEJ+ud\nt97KxD/+kTp1qj/Krv6S/tExn8sFkybNr9LKAti4cSwTJy4Apme0teVn7t/QWyNqLZJaa8WyZTrs\n+ON15qRJCdetqS2Snj3HxHxdoNq7929DrRtJtEiyobPV1FLJDJzzc0l/NnfS1picJ14jTlA3YBaw\nEzgElAA3RT0fRLA1KfDan+HFZ599pg+1bKkKuh10wuDBCbcpLJymeXnzk2qJxOqDOOWUO7Vt2x9p\nz55jtG/fAp07d3E6XlJK5s5drPn5v61Sv3btCrOilUUSLZLQA0nCClog8S1dAeCOm2/WI0eOpGVf\nEwYP1l0R/z0Ptmyp27dtS7hdYeE07d3be2dkvEMimK8wLSs6N1MJkJlggcRUkY4AkEx/RiKHDh3S\nezp0qPKfXQo6YsAA3/uOVl0fBBxtCYT9D5xsgMwECySmUroCwL1XXqlloHe1b6+lpaW+9pXJ08te\nWiTZ0rmZbZIJJDlx+tekrqJDs2DqVL4/aFBKSYliXdTnZ7h8Jk8vDx3al82bR7Nx49iIpUVAC+Bo\nJ23OdW5mG68RJ6wb1iJJWSodmrGk2p+RLSL7IBo2fE2bNPlRVnZuZhvSffpXRB4WkeNFpJ6IvCUi\nu0Xkp4FGOOPbs+PG+U5KFO+ivlzKSTJmzC0MH76N3r1Hc889nzBsWG9L6JxuXqIN8IH792rgSeAE\nYLXXaOXnhrVIUpKuDs1ZU6boP+rWzUh/RiZlY+dmtiGAPpKK9QYAf1LVfSJiV9JlsURXAnvth5j3\n6hx21/sWB458u3JZo4ZbOCmvAafkcE4Sa4Gkl9dAMkdEPgIOAr8SkW+6902WSleH5ud1zmb+wdeq\nLjwIeXsWMLz1tvRU1uQ8T4FEVe8RkYeAfap6RET2A1cFWzXjxwMxUgakIp0Xy2Xr5fLGP0+BREQa\nA4OBVsCtwLeAM4C5wVXNZAOv14KoVj+PTqx8Ips3j2blyg12mFEDeL1o72mgDLjQfbwTuD/+6iZI\nTj9YZni9WC7RPDrxL5dvTVHR9PRW2mSc10DSVlUn4AQTVNXbHAcmEJmch9fL/Lde5tGp/hDJ5snJ\ndV4DySERaVTxQETa4lytazIslcmv/IochxHrsn0v6QBqzOXyJjYv54iBvsBi4F/AC8A24BKv55j9\n3LBxJFWk85qXdPA6ejabL5c3sZHuka2qOh/4AXCTG0i+q6o2Q1KGBTEPr/rsb/E6etbLIVIysjlZ\nUa3kJdoAPYGL3b8V9y/2Gq383LAWSaUgrnnxk2IgldGz6RhRms0Jk2sS0p1GAOc07xz3tgDYByz0\nWoifmwUSRxA5PPymGMhkOoBINTV/a7ZJJpB4HZA2IPKxiLQEfu+7OWQ8S9eQ9yr79JliIJPpACLV\n1IzyuSzVfCQ7gPbprIipXrr/adORYyRdo2eTZWeAso+nmfZE5PGIh3WALsAWVb3BdwVE+gOPAXWB\n6eqMV4l8Xr3U0STnodtuY+ATT1SmB5jQsiU/fvvtaicbzxbz5i3h9tv/X5VkRe3aFVk6gDRLZqY9\nr+NI3ou4LQfuTlMQqQtMxpn7twNwvYhYSydguZ5jJN1ngIx/XvtIngmo/G7ARlXdCiAiL+JcDLg+\nmZ2oVn+dh6kqXf0tYb7vTtCYztKlo+nRI/fn/8111QYSEVlTzdOqqp18lt8CZy6bCjuA85Ldiddp\nH40jXf0tYb/vFjyyR6IWyZUBl++p86O6uX8rh4x37sxPhgxJd/1qJD+dpBWpAPZ+/imd1sxkVN3G\nPPCH1E7gVexr585SPv30U5o3P5EWLU6y9AIhydm5f4HzgTciHt+L0//ieRxJtg0Zz2WJJtKKHAh2\nJs77/v16zXTkyMlJlxVrUJkzz8w0G1yWJQhgQNoFwApgP/AVUA584bWQavZ7HLAJZxLx+sAqoH3U\nOnFfaLqypNd0XmfaSzTK9ehAsM/0Npz3fSvot4/7XtL/+Inmm7HBZeFLJpB4PbidDPwY2AA0BG4G\npqTWBjpKVQ8DtwFvAuuAl1TVc0drOrKk1wSa4PS4l7QDyaQCaMk4fut2bbUG+h9+nwXzVydV53iD\nyqAPsN3SC+QYz71kqvoxUFdVj6jq0zinbH1T1ddV9QxVzVfV8V63y/VTmOlUXaDwmnbAeyqAMrpQ\n9X1/iBJO2r0oqTrHG1TmXIHRygaX5RivgWS/iDQAPhCRh0TkN0Co51sTncKsLRIFCi8BwutVxUOH\n9qVds+8zkmPf92t3f5LU+x4r85ozA9422rX7JGbeE5O9vI5sbQ18jtOPcQdwPDBFVTcGW734I1tH\nDhpE/c2bj1lersqhtm2ZENLwbb80ybEZI7/3PYrmzKGgfXvGrFhR5XqZXbt28dy553JXSQklwKzB\ng2MOg09mlOtlXbtTtvpLjpQ3pW6dPbRseYg2bU5J6X0vKprOxImtKS3tA7wOfEJeXuusDSK1LXl1\nMiNbvXaK/gBo4LXjJZ03atnVv3fcfLMePnzY07qJOpu9pB0IKxVA9L569fplVk9YVRtTFxDAWZtn\ngO3ADJxJso7zWoDfW20KJBWX9V95/vmecoRUFyi8BoiwUgHkmkSpC+bOXax9+xZoz55jtG/fAp07\nd3HYVfYtmUDidYj8QBGpD1wGXA9MEZEFqnqz94aSSeTV8eP50Rdf8Mw//sEzv/sdg4YPj7tuvM7m\nsYMHM2HOHM/D4MNKBZBrqktd8Oc/D2HGjC21eqoNT30klSs7waQfMAgnQ9o3gqpYRJmaTB1zVUV/\nxt6SEoqAXzRpwuMlJXFzhLw4dSrthg7lu4ernv14vmlTvv2nPzFnxowa2YcUln79RjF//n3HLM/L\nW8DJJz/Fpk2zYj6Xrf09XiTTR+J1gqzLgeuAS4BFwDTgh6lW0Bzr2XHjuLykhNeBekDh3r08+Otf\nM+7ZZ2Oun6glYYEivYYO7cvmzaNjpi4oLj6DTZuO3aZWJVrycvwDzAK+DzT0esyUrhu1oI+koj9j\nAlTp87inYUPfOVlN+hQWOiNuo9M61tTUj6S7szXhTmB5OvYTZ9/pfn+yzqwpU3R53bp6T9Q3sRT0\n1vPPD7t6JkKsM1Y1daqNZAJJuq7/bpim/dRKq1esYPJpp3F11PLGwDn//GetGmCXrSqmvygu3kHd\nuuWcc87plc9ZoqUkO1vj7kTkfVXtmob6xNq3pqOO2a6mDrCrCWJNgJ6fP5obbqiaUKmoaDpLl26v\nMYmWkulstUBiApfrI0KrO2OTy2dlEgnirM1QYIaq7vVVM1PrxPo1z7UxFjb9RWJe+0hOBlaIyMsi\n0l+OvRjkxjTXy2RI0K29SZPmVwkiABs3jmXixNYUFU0PtOx0sekvEvM6928BcDrwFDAQ+FhEHhCR\ntu7z1eV2NVnMS64SP6r/Nc+NfCOxrlRu166oRh/WJCuZfCTlwGfALuAI0AT4k4g8HFDdTMC85irx\noyb8mttZGQ+8nCMGhuHMaTMfZ4RrPXd5HWCT13PNqdyoBeNIwpKJfLc1aYxFOq96zgUEcPVvEdA6\nznMdvBaWys0CSTAyme823ojQdPKal9Z4l0wg8dpHMkZVt8V5bl3q7SETlkzmux0z5haGD99G796j\nA+tXCLqvx1QvLeNIUipY5IdAIfAd4FxV/Wec9TSsOtZUZWVljOnalfHrjv4G7AfGDhjAhDnRlwFm\nv5XLlzOzf3/Ove8+33Mb5fqYl3RKe4a0IG44AeR0oBg4u5r10tdWM6pa85IZpauvpzZmQasO6U5s\nFARV/QiwOXtDUJOSGcVKXB0rL60X8ce8LACm21maaoQWSEx4/EzZmW2eHTeOgdF9PSNGxExcnYiN\nYE1doIFERBYAzWM8NVJVPR+MVzf3r6m9EqWbTFZNGPPiR87O/eschlkfiUlNuvt6atKYl3QgF/pI\nolhHiUlauvt6rrjiYlau3MDEiQsoLe1jI1iTEObp36uBSUAzYB/wvqpeFmM9DauOpnaqaXlFUpXx\nfCRBskBikmHjQNIn7flIjAlSuv75a0Luk5zltTMlrBvW2VqjpXMQWE3N5h4WQkj+bExK0pn4qCbk\nPslVFkhMqNL5z1/bx4GEyQKJCVU6//ktk1l4LJCYUKXzn98ymYXHTv+a0BUVTWfixNaVg8D8tiBs\nHEh62DhMg/aVAAAI1UlEQVQSk3Psnz/7WCAxxviWTCCxPhJjjG8WSIwxvlkgMcb4ZoHEGOObBRJj\njG8WSIwvdkbNgAUS45NNTGXAAonxIROTkJvcYIHEpOzV8eN5+Isv+GDqVPbv3x92dUyILJCYlMSa\nmMrUXqGlWhSRh4EBQBmwCbhJVfeFVR9Tveh0iG3rraPQ48RUlke1FvCaSi3dN6APUMe9/yDwYJz1\n0pI2zqTu2HSIh/QH9ZpVyWdYCjpiwAAP29bu+XRzCbmQalFVF6hqRXf/u8CpYdXFVC86HeIJPMnw\nr/5TZZ3GQKdly1heXFzttpB6KkWTvbIli/wgYFbYlTCxRadDbMgK7qE7AE1O3ELnzqcBsSemsvl0\na4fQ5/4VkQKgTFVfiLcfm/s3XNHpEHfxFLtwMpANv736JESWRzV35Ozcv8BA4O9Aw2rWSe+Bn0ma\nnzlxbT7d3EUu9JGISH/gLuAqVT0YVj1MYn5yoVoe1dohzLl/PwbqA3vcRctV9dcx1tOw6miq8pMO\n0VIp5h5LtWiM8c1SLRpjMsoCiTHGNwskxhjfLJAYY3yzQGKM8c0CiTHGNwskxhjfLJAYY3yzQGKM\n8c0CiTHGNwskxhjfLJAYY3yzQGKM8c0CiTHGNwskxhjfLJAYY3yzQGKM8c0CiTHGtzCTP48TkQ9E\nZJWIvCUiLcOqizHGnzCTP39dVb907w8BOqvqMVmBLWerMeHIiZytFUHElQfsDqsuxhh/Qp2yU0Tu\nB34K/Bc4P8y6GGNSF+ihjZcpO9317gHOUNWbYuzDDm2MCUEyhzaBtkhUtY/HVV8AXov3pM39a0zw\n/Mz9G2ZnaztV/di9PwTopqo/jbGetUiMCUHWtEgSGC8iZwBHgE3Ar0KsizHGB5uy0xgTU06c/jXG\n1BwWSIwxvlkgMcb4ZoHEGOObBRJjjG8WSIwxvlkgMcb4ZoHEGOObBRJjjG8WSIwxvlkgMcb4ZoHE\nGOObBRJjjG8WSIwxvlkgMcb4ZoHEGOObBRJjjG8WSIwxvlkgMcb4FnogEZE7RaRcRJqGXRdjTGpC\nDSTuxOF9gG1h1iNSqvN6WHlWXk0qK1lht0h+B4wIuQ5V1OQvopWX2+VZIIlBRK4Cdqjq6rDqYIxJ\nj0AnyKpm7t8C4F6gb+TqQdbFGBOcUCbIEpGzgLeA/7qLTgU+wZm28/OodW12LGNC4nWCrKyYaU9E\ntgDfVdU9YdfFGJO8sDtbK4QfzYwxKcuKFokxJrdlS4vEk0wNXhORcSLygYisEpG33PEuQZb3sIis\nd8t8VUROCLCsH4rIWhE5IiJnB1hOfxH5SEQ+FpG7gyrHLespEdklImuCLCeivJYiUuy+jx+KyNCA\ny2soIu+638d1IjI+yPIiyq0rIu+LyJxE6+ZMIMnw4LWHVLWzqnYB/gKMCbi8+cCZqtoZ2IBzRiso\na4CrgSVBFSAidYHJQH+gA3C9iLQPqjzgabesTPkKuENVzwTOBwYH+fpU9SBwift97ARcIiIXBVVe\nhGHAOjx0PeRMICGDg9dU9cuIh3nA7oDLW6Cq5e7Dd3HOYgVV1kequiGo/bu6ARtVdauqfgW8CFwV\nVGGquhTYG9T+Y5T3maqucu+XAuuBbwVcZsUZzvpAXSDQExMicipwOTAdD0MzciKQhDF4TUTuF5Ht\nwM+ABzNVLjAIeC2D5QWhBVAS8XiHu6zGEZE2QFecH4Agy6kjIquAXUCxqq4LsjzgUeAuoDzRihDw\ngLRkZHrwWjXljVTVOapaABSIyD04b+pNQZbnrlMAlKnqC0GXFbBa0YMvInnAn4BhbsskMG6LtYvb\nf/amiPRS1UVBlCUiA4DPVfV9EenlZZusCSSq2ifWcnfw2mnAByICTrP/PRE5ZvBaOsqL4QXS0EJI\nVJ6IDMRpSvYOuqwM+ASI7KBuidMqqTFEpB7wCjBTVf+SqXJVdZ+IzAPOARYFVMyFwPdE5HKgIXC8\niDynqjfG2yDrD21U9UNVPVlVT1PV03C+kGf7CSKJiEi7iIdXAe8HVZZbXn+cZuRVbsdapgR1WcJK\noJ2ItBGR+sD/AH8LqKyME+cX7Ulgnao+loHymonIie79RjgnHQL7TqrqSFVt6f6//QhYWF0QgRwI\nJDFkotk8XkTWuMekvYA7Ay7vcZxO3QXu6bYpQRUkIleLSAnO2YZ5IvJ6ustQ1cPAbcCbOL3+L6nq\n+nSXU0FEZgHLgNNFpEREfB2GetAduAHn7Mn77i3Is0anAAvd7+O7wBxVfSvA8qIl/J+zAWnGGN9y\nsUVijMkyFkiMMb5ZIDHG+GaBxBjjmwUSY4xvFkiMMb5ZIDHG+GaBxATKHd2akTwhJjwWSIwxvlkg\nMQCIyLluhrYGItLYzfzVIcZ6s9yLuSoePyMiPxCR1iKyRETec28XxNh2oIg8HvF4roj0dO/3FZFl\n7rYvi0hjd/mDbiayD0Tk4WBevfEra67+NeFS1RUi8jfgPqARMCNOzouXgOuA19wL8i4FfoHzo9RH\nVQ+5Fz2+AJybqFhARaQZTrqI3qp6wE3N+BsReQL4vqp+B0BEjvf/Sk0QLJCYSGNxrtw9AAyJs84b\nwO/dIHIZsNgNHicAk0WkM3AEON1jmYJzAWEHYJmbKqI+zkV4+4CDIvIkMNe9mSxkgcREagY0xknl\n14ijE5hVUtWDIrII6IfTMpnlPnUH8Kmq/tTN2RorHcJhqh5ON4y4v0BVfxy9gYh0w8nRci3OFcW+\n87WY9LM+EhPpD8AonMOSCdWs9xJOSsgeOC0UgOOBz9z7N+IEo2hbcbJ8iZvMuxvO4c07QHcRaQvg\n9tG0c/tJTlTV14HfAJ19vDYTIGuRGABE5EbgkKq+KCJ1cA4z4qXzmw/MAP7i5h4BmAK84u7nDSAy\n9aACqOrb4syquA4nYfJ77vLdboa4WSLSwN2mAPgS+KuINMQ5BLojbS/YpJXlIzHG+GaHNsYY3+zQ\nxsQkIh2B56IWH1TVY8aHGGOHNsYY3+zQxhjjmwUSY4xvFkiMMb5ZIDHG+GaBxBjj2/8CA0PWeVMD\nWj4AAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1090c090>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"samples_reduced = V[0:2, :].dot(df_centered.T).T\n",
"plt.plot(samples_reduced[0:20,0], samples_reduced[0:20, 1],\n",
" 'o', markersize=7, color='blue', alpha=1, label='class1')\n",
"plt.plot(samples_reduced[20:40, 0], samples_reduced[20:40, 1],\n",
" '^', markersize=7, color='red', alpha=1, label='class2')\n",
"plt.xlabel('x_values')\n",
"plt.ylabel('y_values')\n",
"plt.xlim([-4,4])\n",
"plt.ylim([-4,4])\n",
"plt.title('Transformed samples with class labels')\n",
"plt.legend()\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Observe that with 2 dimensions, one is still able to capture the spatial separation between the two classes. Let us use the *PCA* package from sklearn to verify that our results!"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWMAAAEaCAYAAADE7zboAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucFNWVwPHfQREIIwajER8gBjAB5ZVE1Cgy6ApGURNj\n3BgREWXdLIIa8AUEBlDxgTFBkU3AJyoqahIBH6AwggsayYoQwEVAYfAxPiDgqDDAnP3j3hlqmu6e\nnpnurpru8/186jNdz3urpvr0rVu3bomqYowxJlyNws6AMcYYC8bGGBMJFoyNMSYCLBgbY0wEWDA2\nxpgIsGBsjDEREPlgLCKHicgiEdkuIneFnZ8gEWkrIhUiEtnjKCIPi8iEsPMBICJTRWR0ltMsFpEr\najsvTA3hvEqHms4HEblZRKZlM0/pIiJXicg9/vNhIrJaRA5Itk7Sf7aIlInIl36oEJGvA+MXpzPz\nSfwH8KmqtlDV67OUZi5RP4ROVX+jqrdkO1kS739kjk0+Cp4PIlIoIiUx8yeq6uB0pysiRSKyy8ex\nrSLyPyJyUmD+4SLygIh85AuBa/w63wosIyKyQURWxdn+AcAo4E6/H6XAQlwsSyhpMFbVAlU9UFUP\nBDYC/SrHVXVmIPH9UzsMdXI0sKYuK2Y4Xw2JhJ2BfCIi+4WdB5OUAjN9XDsUeB14DkBEDgaWAk2A\nk1S1BXAmcBDQLrCN0/wyh4rIj2O2fz6wRlU/Dkx7HLgqWabqdBnkf8U2i8gNIvIx8ICIfFtE5ojI\npyKyRURmi8iRgXWKRWS8iLzuf21eFpHv+HlNReQxEfnc/1L9XUS+KyIPAwOAG/yv2OkicoCI/EFE\nPvTDPZXF/zj5elBExorILBGZ4dNdISId/CVQqYhsFJEzA/k8KPCruFlEJlReLopIIxGZJCKfich6\n4JwajtONfhvbReRdETndT+8hIkv9vn4kIveKSOPAehUi8hsRec+vO15E2vl1/iUiT1YuH9jnm32+\n3heRXyfJUz8RWR4oEXSuKb9xtlHt8l5EBorIYv9Z/P+kVES2+ePdyc+rqjIJ5Pu3ftmPRGRgYJvf\n8efQNn8+3FKZRpz8xDt/Do2z3OE+P8MTbGeQuMvJLSLykoi0Ccz7o4hs8vlZJiKnBuYVicgz/hzb\nBgz0x2iCxDnf46Sb9LxKdk76+YN9vreLyCoR6e6nd/T52Coi/xSRcwPrPCwi94vIC+K+W4tFpJXf\nz63iSoPdAst/ICI3+e1vEZEHRaRJTB7eE5EvRORvInJ4YF7S80FcifNF4Aifl+3+f1UkIjMC2znP\np79VRBaKyA9i8jdcRN6Rvd+RqvzFHnI/oKq7gUeBVv7/81tgm6r2V9VNfpnNqnqdqq4MbOMy4Fng\nb/5z0E+B12Km/R34noi0TpAnUNWUBuB94HT/uRDYBUwEGgNNgYOBn/vPBcDTwF8C6xcD7wHt/TIL\ngYl+3lXA8366AN2BA/28h4Dxge2MB5YAh/jhfyrnJ8hXEfAN7tdtP+AR4APgZj9+JbAhsP2/AFOB\nZrhfzTeB//Dz/hNXSj8SaOn3YQ/QKM7x+j6wCWjlx9sA3/Offwj0wP0YHg2sBq4JrFvh81EAdAJ2\nAguAtkALYBUwIGafJ/l9Pg0oAzoEjt8E/7k7UAqc4I/zAP9/bZwsv3H2bSEwKDA+EFjsP/cFlgEt\nAsehVez/MpDvIv9/+CnwFXCQn/8k8IT/H3b0eVuUID/Jzp+FwCDgGOD/gCvj7QeuNPOez28j3GXm\n/wSWvcT/zxvhvrAfAwf4eUVAOXCeH29KkvM9Tv6TnlckPyd/CWwGfuTH2/n/XWNgHXATsD/QG9gO\nHOuXexj4zB+rJsCruO9Ff38MJwALAnn8AFgRyOPr7D2vTvfb6gYcAEwGXqvl+dALKIk5LmOBGf7z\nsbjz+gzc+XK9P777B+LTG0Arn7/VwFUJjndRYLtNgLuAD/z4G8DYGmLht4BtwCm4uPIZ0Dgw/+/A\nL+Ks9w5wbsLt1iMY78SfjAmW7wZsiTnxRwbGfwO86D9fjguqneNspyqY+PF1wFmB8T7A+4ny5Q/8\ny4Hxc4EvAfHjB+KCXwvgMGAH0DSw/MWVJyUuIP5HYN6Zft14wbg9LvCdEfxHJThW1wLPBcYrgJMD\n48uA6wPjk4B7YoJas8D8p4DRcU74qQR+2Py0d3EBvF0t8pssGJ+OC3onxh4Xqv8wFAJfB5fx6ffA\nfdnK8T8oft6EyjTi5CfZ+bMQuBt3/v57ov3AlcyC+9QI9+PQOkGaWyrT8+dYcZxtxz3f42wr4XmV\nwjn5MjA0zjZ7Ah/HTHsCH2hwwfhPgXlXA6sC452BrTHf/2Aefwqs858fAG4PzGvu/39tcD8CqZ4P\nscG4iL1B83fAk4F5gvsROi2Qv18H5t8BTE1wvItwcWKrP+deAbr7eWuD+5lg/f6VefXn6mfAzwLz\n1wJ94qz3OtA/0Xbrc7f2M1UtrxwRkW+JyJ/85cI2XDH9IBEJ1ld+Evj8Da7kBzADd1I9Ka7q4Q5J\nXN97BK7+utImPy1uvrxPY9L9XP3R8eP4vByNK1F87C+FtgL/jSuNABwOBG8ybEqQR1R1HS7IFgGl\nIjKz8tJNRI4VV6XzsT9WtwKxl7ClMXkOju9g77ED96X5JjC+0ec11tHA8Mp98/t3FHC4qq5PlN/a\nUNUFwH3AFL+dP4nIgQkW/0JVKwLjX/v9OhRXmgse681Jkk12/giuVLsZd1mZyNHAHwPH5Qs//UgA\nERnhqwL+5ecfhLsyS5a/ROd7rGTnVU3n5FHA+jjbPCJmm+DOi8rvilL9e7GDfb8nsfmNzWPltg4n\n8J1U1a9wx+9IVV1I6udDMkcQOC7++1uC//94qR5vgKdUtaWqHqaq/6aqb/vpX1A9nsRzGb6OWVX3\nAH+lelXFVlzhLtaBwL8SbbQ+wVhjxofjLiV6qOpBuMuOqrqZpBtS3a2q41X1OOAnQD/cJXQ8H+Eu\n1yu18dMS5St2PJkS3C/md/w/qqWqHqSqlfWqH/v0gmknpKozVbUn7guluF9rcCXU1UB7f6xGUbv/\nRew+tZTAnV6f3kfsaxNwa2DfWqq7SftUDfmN9RWu9FOpVbXMqd6rqj/GVbEci7ukTJT3eD4DdgPB\n+rWEdW01nD+Ku9z9AnhCEjcX24QrEQWPTXNVfUNEevp9+KWqfltVW+IuU4Pndm3Os1jJzquazskS\n3FVYrI+A1jGFoaOBD+uRz9g8Vm6r2ndSRJrjChcfQsrnQ03H70Nc/ivTENw5kWh/km1PSRyXXgF+\nHnPcqojIUbirv8t8Yepj4CLgbHE3/8BV5xwbs97+uP/TO4kylc52jAW4X6NtPlNj4yyTaAd7i0hn\ncXehv8Rddu9JsM5MYLSIHCIihwBjcCWjRFJuSaDu7uc84PcicqC/sdJORE7zizwNDBORI0WkJa4+\nLn6irvR7ur+JsBNX8qjcpwK/n1/7mxC/SSF7kuBzpXEi0tgHjnOAWYFlK5efBvynuBuIIiLNReQc\nESmoIb+xlgMXiEgzEWkPXIE/+UXkxyJyorgbjF/HbCfVH+c9uJJHkU/jB8ClJPiCibsZmOj8wY//\nEvcD8miCL9p/AyMDN5cOEpFf+nkH4n4cPhd3A3kM8Us++2QthWUgyXmVwjk5HRghIj/0/9P24m48\nvoE7/jf486IQ9yP1ZC3zFtyX//J5PBhXgHjKz5sJXC4iXf35cxvwhqpuqsX5UAp8R0QSHddZwDn+\nHG2MK/ztwN0/SpTfZPuSyO9x/9tH/HHE7/Pd4m52X4qr2jsW6OqHY3FXRpU3zl/AFUaDeuDqpWOv\nVqqks2T8B9wNhs9xB+jFOMtozOfK8cNwB3sbrsRYzN4AqzHr3YKrQ13hh2V+WqJ8xa6faJlKA3A3\nIVbj6gVnsbfkNw13OfyOT/fZONuq1AR3I/EzXMnnENxNQ4ARuH/cduDPuC9I7LGJlejYgbs824or\noczA3bhYG7usqv4DGIy7bNyCuwFSWYJMlt9Y9+DqBEtx9X6PBea18Pu0BXfT53PcDZJ4+U5Werka\nVxXwCe6m60yfZjytSHz+uIRUdwEX4M61B2IDsqr+FXcl8KSvOlqJu/kE8JIf1vp9+obqVQnxzrHY\n/au2jLhWA6f40ZrOq4TnpKo+g6vmegJ3Pj0HtPT7ey6ubvcz3P/80njnRZJ9iJ3/BO6HYT3u3LnF\n5+FVXJ3us7hz8BjgV369lM4HVX0X9z/eIK61xuEx8/8PV1d7r9+fc3A3w3YTX9W6ItLGH++jkuwr\nPp2tuKurXcCbIrIdV1r+F+5+1QDgflX9NDCU4n7MK79Lc4AfxFTzXYK7Ik6o8iZWXvAlp2XAZlU9\nt6blGwJf4pmhqombzOQAEbkD+K6qXh52XvKRiLwPXOHvCZgaiMhgoJOqXici38UVELrFuZ9VJd8e\nirgGV7qoyw0Ek0Ui8n1caX0lrineIFx1iDGRp6rTAp8/xdWXJ5XTz74H+UuUs3F1bLn2RFouXt4c\niLvsLcNV40xS1efDzZIxmZM31RQiMgt3Y6EFMCJXqimMMbkhL0rGItIP19nQ2+ReqdgYkwPyomQs\nIrfhmqTsxj2a2gJ4VlUHBJbJ/QNhTESpat4XkvKiZKyqI1W1tapWNrlZEAzEgeWyNowdO9bSs/Qs\nPbUyUKW8CMZx2BlgjImUfGvahqq+xr7d2xljTKjytWQcusLCQkvP0rP0TJW8uIGXChFROxYmkxL0\nPZN3Yr9nIoLaDbz8q6YwJkz5/oNvP0iJWTWFMcZEgAVjY4yJAAvGxuSxoqIi7r777rRtb9CgQRx2\n2GF07ty55oVNNRaMjQnR3LmL6Nt3NIWFRfTtO5q5cxdldL1Y6a7Dvfzyy3nppZfSus18YTfwjAnJ\n3LmLuOaal1m//taqaRs2jGHZsrWMHXtl2tcDePTRR7n77rsREbp06UK7du2q5k2bNo1p06ZRXl5O\n+/btmTFjBs2aNWPWrFmMHz+e/fbbj4MOOojXXnuNVatWMWjQIMrLy6moqODZZ5+lffv29OzZkw8+\n+KDuByWfZfMRyygP7lAYkzmx51ifPqMUdJ+hoGCeFhVNS7iduq73z3/+U4899lj94osvVFV1y5Yt\nWlRUpJMmTVJVrZquqjp69Gi99957VVW1c+fO+tFHH6mq6rZt21RVdejQofr444+rququXbv0m2++\nqVr3/fff1+OPPz6lYxCYFnoMCHuwagpjQrJzZ/wL07KyM1m8OOGLx+u83oIFC7jooos4+GD33syW\nLVtWm79y5Up69uxJly5dePzxx1m9ejUAp5xyCpdddhnTp09n9273lqOTTz6Z2267jTvvvJMPPviA\npk2bJt5RkxILxsaEpEmT+K9vKyiYT8+eiV88Xtf1/MMVcacDDBw4kPvvv58VK1YwduxYvvnmGwCm\nTp3KLbfcQklJCT/60Y/YsmULF198MbNnz6ZZs2acffbZLFy4MGG6JjUWjI0JybBhfWjffky1aR06\njGPEiI1J637rut7pp5/OrFmz2LJlC0DV38oAXVZWRqtWrdi1axePPbb3HbPr16+nR48ejBs3jkMP\nPZTNmzfz/vvv07ZtW4YOHcr555/PypUra7fzZh8WjI0JyTnnnEb//m0oKJgPuJLtJZccWeNNuLqu\n16lTJ0aNGkWvXr3o1q0bw4cPB/aWjCdMmMCJJ57IqaeeSseOHaum33DDDXTp0oXOnTtzyimn0KVL\nF55++mk6d+5M9+7dWbVqFQMGuB5pL774Yn7yk5+wdu1aWrduzUMPPVT3A5RnrG8Kz/qmMJmWqJpg\n3LjpLF68iZ4929QYUNOxXpjiHQPrm8KxYOxZMDaZligY5xMLxolZNYUxxkSABWNjjIkAC8bGGBMB\nFoyNMSYC8iYYi0hTEXlTRJaLyGoRmRh2nowxplLeBGNV3QH0VtVuQBegt4icGnK2jAHq/gaQ+rbO\nSGcXmiUlJfTu3ZvjjjuO448/nsmTJ6dlu/kib4IxgKp+7T8eAOwHbAkxO8ZUGT54MBUVFVlbr1I6\nu9Bs3Lgx99xzD6tWreKNN95gypQprFmzJm3bz3V5FYxFpJGILAdKgYWqujrsPBmzbOlSKmbNYuaU\nKRlf79FHH6Vr165069at6qm5StOmTaNHjx5069aNCy+8sKpvilmzZtG5c2e6detGr169AFi1ahUn\nnngi3bt3p2vXrqxbt45WrVrRrVs3AAoKCujYsSMfffRRrfYpr4XdbVwYA3AQ8AZQGJimxmRSonPs\n5nPP1XLQ6zt21LKyspS3V9v1stWFpqrrRrNNmzb65ZdfVpse7xhgXWiiqvnZubyqbhORucCPgeLK\n6UVFRVXLFBYWUlhYmO2smTxTWlpKy+XLaQwMXbOGKTfeyA333ZeR9VLpQnP06NFs27aNsrIyzjrr\nLGBvF5oXXXQRF1xwAeC60Lz11lvZvHkzF1xwAe3bt6/aTllZGRdeeCF//OMfKSgo2CcfxcXFFBcX\n17iPeSfsX4NsDcAhwLf952bAIuCMwHw1JpPinWN3DBmipYEe4m9v3Vo3bdxY47bqst69996ro0aN\nqjatqKhI7777blVVbdu2ra5YsUJVVR9++GEdOHBg1XJvvvmmjhkzRtu2bVtVgt6wYYNOnjxZO3To\noAsWLFBV1fLycu3Tp4/ec889KR8DrGSMan51Ln84sMDXGb8JzFbVV0POk8lj5eXlbF24kO8Gpl1d\nUsJ9Q4ZkZL1sdKF5xRVX0KlTJ6699toUj4KplDfBWFVXquoPVbWbqnZR1bvCzpPJb8898AAXrl1b\nbVpzoMuSJSxN0ll7XdfLdBear7/+Oo899hgLFy6ke/fudO/e3V5OWgvWa5tnvbaZTIvtsWzkoEEc\nsGHDPstVqLKzXTvuePDBuNup63pRYL22JWbB2LNgbDLNutC0YJxM3lRTGGNMlFkwNsaYCLBgbIwx\nEWDB2BhjIiAvn8AzJizp7JjH5BYLxsZkSb63pDDJWTWFMcZEgAVjY4yJAAvGxhgTARaMjTEmAiwY\nG2NMBFgwNsaYCLBgbIwxEWDB2BhjIsCCsTHGRIAFY2OMiQALxsYYEwF5E4xFpLWILBSRVSLyTxEZ\nFnaejDGmUt68dklEWgGtVHW5iBQA/wB+pqpr/Hx77ZIxIbDXLjl5UzJW1U9Udbn/XAasAY4IN1fG\nGOPkTTAOEpG2QHfgzXBzYowxTt71Z+yrKJ4BrvEl5CpFRUVVnwsLCyksLMxq3ozJB8XFxRQXF4ed\njcjJmzpjABFpDMwBXlTVP8TMszpjY0JgdcZO3gRjce+7eQT4QlWvizPfgrExIbBg7ORTMD4VWASs\nACp3+mZVfcnPt2BsTAgsGDt5E4xrYsHYmHBYMHbysjWFMcZEjQVjY4yJAAvGxhgTARaMjTEmAiwY\nG2NMBOTdE3jGRNXcuYuYPHkeO3fuT5Mmuxk2rA/nnHNa2NkyWWLB2JgImDt3Eddc8zLr199aNW3D\nhjEsW7aWsWOvDDFnJlusnbFn7YxNmPr2Hc28ebfsM72gYD4jRmzM6YBs7YwdqzM2JgJ27ox/kVpW\ndiaLF2/Kcm5MGCwYGxMBTZrsjju9oGA+PXu2yXJuTBgsGJtIydeqomHD+tC+/Zhq0zp0GJfzVRRm\nLwvGJlKGDx5MRUVF2NnIunPOOY3+/dtQUDAfcCXiSy450gJxHrHWFCYyli1dSsWsWczs2pVLhg4N\nOztZ5wLvdBYvHkPPnm0sEOcZa03hWWuKulFVXFfR9V9n5HnnMW72bEZ17MjYt96iefPm6cqmiTBr\nTeFYNYWpl7pUK8Rbp7S0lJbLl9MYGLpmDVNuvDGNuTQm+iwYmzqrqlaYMqXe6zwyYQIDSkoAaA3o\n889TssmadJn8YcHY1NlzEydy1/btvDN1Kl999VWd1ykvL2frwoXcAVSWl68uKeG+IUMyk3FjIsiC\nsamTulQrJFrnuQce4Lh336UCmOmXbQ50WbKEpQsXZmgPjIkWC8amTh6ZMIHLalmtkGidFW+9xV9a\ntuQu4JVvfYuRp55KUa9e/N/xx/PXGTMyuyPGRIWq5sUAPAiUAisTzFeTmp07d+pNnTqpQtVQBnpD\nv351WueTTz7RO1u3VgXdBHrHkCFZ3BsTNv/dCz1GhD3kU8n4IeCssDORC5574AEuXLu22rSaqhWS\nrXPrVVfVupSdaXPnLqJv39EUFhbRt+9o5s5dFGp+TO7Lq3bGItIWmK2qnePM03w6FvUxctAgDtiw\nYZ/pFarsbNeOOx58MOV1du/Zw9KVK3l127aqaV8B4/v1447Zs9Oa71TF686yffsx9O/fsB/EiGp/\nydbO2Au7aJ7NAWiLVVNEysz779dl+++vweoLBX3s4IN1yYIFaU2roqIipeX69BkVmx0F1YKCeVpU\nNC2tecqWOXNe03btRlbbn/btfxeJ/cGqKVBVexw6qKioqOpzYWEhhYWFoeUlX6x46y3ePeUUYsvA\nFaqsmDGDk3v3TltawwcPZtKf/0yjRslr55J3Zzkm7ryomzx5XrWSPsC6deOZNGk+MD2rJf7i4mKK\ni4uzll5DYcE4IBiMTXbcFqdKIxNq0+9FLnZnGaUfmNiCzrhx47KaflTl0w08k8dq84BKXbuzjPJN\nv1z8gck5YdeTZGvAPU/wEbATKAEuj5mvJhpSrdtNVV2azhUVTdOCgnkp1xXHq5M9/PDh2q7dr7RX\nr7Hap88onTPntXTtUq3NmfOatm//u2r569ChyOqMIzSEnoGoDBaM6y9dQfS6K67QPXv2pGVbqqp3\nDBmipYEodHvr1rpp48Ya1ysqmqZnnJHaTa5EN/1gnsK0SNwwq+0PTLZYMPYxKOwMRGWwYFx/6Qii\nby1Zote0aKGPTZ6cljzV5QGVuujVa2yCYKwKe0ukYQfB2vzAZIsFYzdYnbFJi7r04BZPXTofSrq9\nOjygUheJ6mRhPrC3TjbsF4yOHXslr7wyvkG3l85V1prCpEVlEB01dSo/GzSoTh3Dx+tI6Ib77qtX\nvrLVdG7YsD5s2DCGdevGB6aOA44E9gY+u2FmEgq7aB6VAaumqLN09S1R17rdqAjWyTZt+oK2bPmr\nSN4wixqsmgLViFVTiMhdItJCRBqLyKsi8rmIXBp2vkxydenBLVZln8bfDUxraH0ajx17JSNGbOSM\nM8Zw000fcs01Z9gLRk3KItU3hYi8o6pdReTnQD/gt8BiVe2ShbQ1SseioSgvL2ds9+5MXL26alpd\n+pZ4cupU2g0dygl79lSb/vjBB/O9Z55J65N42TRu3HQWL95kLxhNwvqmcKJWZ1yZn37AM6q6TUQs\nQkZYTTfIUg2ic5+bzeeNj+CbPd+rmtas6fscWtCEw9P8WHQ2WQA2qYpaMJ4tIu8CO4DfiMh3/WcT\nUem6QfZpox8yb8cL1SfugIIt8xlx9Mb0ZNaYCItUMFbVm0TkTmCbqu4Rka+A88POl0ksXX1LpLPv\nhKh2FWlMMpEKxiLSHBiCa5g5GDgC+D4wJ8x8mcxLte8EVUUkcfVivL6IN2wYw7Jla63KwERapFpT\n4N7GUQ78xI9/BNyaeHGTSdm8oZlq5zzDBw+moqIidvUqibuKPJpx46anN9PGpFHUgnE7Vb0DF5BR\n1fo/gmXqrKbAl07nnHMa/fu3SdoULJWn/JJXd4T7KidjkolaMN4pIs0qR0SkHa6XNZNl6Xq8uTaC\n7XTjdVeZyqPS1lWkabDCfuokOAB9gNeAz4AngI1A7yylrWavm889V8tBr+/YUcvKysLOTspP+UW5\nq0gTH/YEHqoRewJPVecBvwAuxwXjH6lq+npzMSmJ10dEOrjvXd2k+pRfKtUdtRHlDuNNjgn71yA4\nAL2A0/zfys+nZSltNU6m+oioaxebdekGMx1dRUb5JZ65BCsZuxgUdgaqZcY1YZvth/nANmBBltJW\nk7n+f+vTT3E23yAdlItviY4iC8ZuiFQ7Y1XtFxwXkdbAH0PKTl5K1+PN+2y3Hl1sZvMN0kFReomn\nyX2RCsZxbAY6hp2JfJKJwFfffoqz9QbpWNYyw2RT1Hptuzcw2gjoBryvqv3TsO2zgD8A+wHT1bVn\nDs7XKB2LXHLn1VczcMqUqu4x72jdml+//jqt20Q7oM2du4hrr32lWofxHTqMs64w08x6bXMi1ZoC\n+EdgWArcmKZAvB9wH3AW0Am4WESsxJ0FDbmf4nS3zDAmmUhVU6jqwxnadA9gnap+ACAiT+I6IFpT\nm42oJu8XwewrHXXQYR53F3ins3jxGOuT2GRUJIKxiKxMMlu1/p3LHwmUBMY3AyfWdiPDBw9m0p//\nTKNGUbugiK501EGHfdwtAJtsiEQwBs7N8PZTqgwuKiqq+lxYWEhhYWHVeNXjwV27csnQoenOX86q\n6823ym4wt376MV1WPsbo/Zpz25/q1rAm2KXm9u2fAuW0aHGUda8ZkuLiYoqLi8PORvSE3bYuGwNw\nEvBSYPxmXH10cBlNJmqPBzdkFRUVSecHH7Y4Dnfcf9b4EB058r5apxXvwQ34ncI0e4gjIrB2xi4G\nhZ2BapmBk4G3cK9R2wVUANvTsN39gfVAW+AAYDnQMWYZTSRdbz/OBzUFWtWan8Tb+7DFJ3o17rh/\nAPq9/c+rdeBM9OAGzKsKyPYQR7gsGLshapWf9wG/BtYCTYErgPvru1FV3Q1cDbwMrAaeUtWUb96l\n4+3HucB9b5KrqdvN2nSD2ZoJ/M5X9R8NnLX7bebPW1GrPCd6cAPOBNz/0LrXNFEQtWCMqr4H7Keq\ne1T1IVxztHRs90VV/b6qtlfViamu15CbZqVbOgJt6t1gltON6sf9Tko49PPiWuU50YMb7ml7187Z\nHuIwURC1YPyViDQB3hGRO0Xkt0CobclqapqVL9IRaFPtDW7YsD50OORnjGTf437h5x/W6rjHe4MI\njMP1znpl3LeJGBOGqD2BdzTwKa5e9zqgBXC/qq7LQtoa71iMHDSIAzZs2Gd6hSo727XjjpAe1a0v\n1dq13R153nmMmz2bUR07Mvatt/bpW6K0tJRHTziB60tKKAFmDhmyzyPPtXkS76fdT6F8xZfsqTiY\n/RptoXU5EkOfAAAPnUlEQVTrnbRte3idjvu4cdOZNOloysrOpHHjucBidu26nYKC+ZENxPn0UlV7\nAs8Lu9I6OOD6Mm4SUtqaT6674grdvXt3SsumcgOzpm43w+oGM9620rndTMi3rjuxG3guBoWdgWqZ\ngYdxd1VmAP2A/bOYtuaLyu4szz3ppJT6F05HoA2rG8yGKFnXnb/+9Sjt02eU9uo1Vvv0GaVz5rwW\ndnbrzYKxG6Ly0AcAqjpQRA4AfgpcDNwvIvNV9YqQs5ZTnps4kV9t387Df/87D//+9wwaMSLhsolu\nYI4fMoQ7Zrvn6lJ55DmsbjAbomRddz733EPs2PFE1bQNG8awbNnaSFa1mNqJVJ1xJR+Q+wKDcG/6\n+E4W0tQoHot0q6zb3VpSwjjgqpYtubekJGH/wk9OnUqHYcP40e7qrRIeP/hgvvfMM5zcu3fO1quH\npW/f0cybd8s+0xs1mk1FRSlQPfBGue47FVZn7EQqGIvI2cBFQG+gGHgKmKeunXCm086LYHzn1Vdz\n9pQpvAhcj6sTmjZgABMeeSTu8hZosy9R153l5WvZuPHxuOucccYYXnllfNx5UWfB2IlUNQVwKS4A\n/6eq7gg7M7mmssrhBWCgn9YG2P3005RMmBC3VUNYHbvns3POOY1ly9YyadJ8ysrOrOq6c8mSXWzc\nuO/y1k46N0SqZFwTEVmqqidnaNs5XzJ+cupU2g4dyt/27CH41MtXwHUnncSfly4NK2smjnHjprN4\n8aaqrjtztbN7Kxk7UXvooyZNw85AQ7birbe475hj+HnM9ObAj//3f/PqIZaomjt3EX37jqawsIgl\nSz7guuv+rSrQWmf3ua2hlYzfVtXuGdp2zpeMweqAo2zu3EVcc83LrF9/a9W09u3H0L9/9U7tY0vM\nDZ2VjB0Lxnu3nRfBOJc19KfWErWiaOitJWpiwdiJ1A08ERkGzFDVrWHnxTQs8UqVDa0NbrL2xYsX\nx/avYXJN1OqMDwPeEpGnReQs2bfzhAFhZMrUX6avOiZPnlctEAOsWzeeSZOOZty46RlNO10S9TBn\nrSXyQ6SCsaqOAo4FHsS1vnpPRG4TkXZ+frJ35ZkIq6n7zfpKXqpsGH0Vx+thznqVyx+RCsYAqloB\nfAKUAnuAlsAzInJXqBkzdZZK95v1lQulSmstkd8idQNPRK7BVUV8AUwH/qKqu0SkEfCeqrbLYNp2\nAy9Daup+Mx1yqQ1urrWWqIndwHOiFozHAQ+q6j7PGYlIJ1VdncG0LRhnQCr9HKdLsN/iTLVAUK1d\nP9CmZhaMnUhVU6jq2HiB2M/LWCA2mZPN9weOHXslI0Zs5IwzxmSsnjXTdd8mf0WqZJwpIvJLoAj4\nAXCCqv5vnGWsZJxm5eXljO3enYmr9/6OfgWM79evqvvNhmTZ0qU8dtZZnHDLLVwydGi9ttXQ20Sn\nk5WMvbA7VM7GgAvCxwILgR8mWEZNeuVah/I3n3uuloNe37GjlpWV1Xk7+fYmj5pgncujGrHO5TNF\nVd8FrK4vy3KpQ/l4L1Ota9134jbR84HpeXHTzuwrL4KxCUcudb/5yIQJDIyt+77hhrjdjtbEnrQz\n8eRMMBaR+UCrOLNGqmpKFZRFRUVVnwsLCyksLExL3kzDlsqrp2ojF9pE10dxcTHFxcVhZyNy8uIG\nXiURWQgMV7uBZ2ohlVdP1UYutYlOB7uB5+RMybgW8v6fbmon3XXfid7kkY+B2OyVFyVjEfk5MBk4\nBNgGvK2qP41ZxkrGJqvy7Um7RKxk7ORFME6FBWNTW9ZWOD0sGDv5WE1h8li6Amgu9J9sosVKxp6V\njHNfqq81SkW+vpUjE6xk7ESqbwpjMimdHdDnQv/JJlosGJu8kc4Amu9thU36WTA2eSOdAdTeymHS\nzYKxyRvpDKD2Vg6TbnYDz7MbePkh3R3QW1vh+rMbeI4FY8+Ccf6wABotFowdC8aeBWNjwmHB2LE6\nY2OMiQALxsYYEwEWjI0xJgIsGBtjTARYMDbGmAiwYGwaDGvtYnKZBWPTYAwfPJiKioqws2FMRlgw\nNg3CsqVLqZg1i5lTpoSdFWMywoKxaRCemziRu7Zv552pU/nqq6/Czo4xaWfB2EReaWkpLZcvpzEw\ndM0aptx4Y9hZMibt8uK1SyJyF9APKAfWA5er6rZwc2USiX01UrvGqykqKQGgNaDPP0/JDTfQuk2b\npOvZO+lMg6KqOT8AZwKN/OfbgdvjLKMmfHPmvKbt2o1UUD/s1F80PkQDE7QM9IZ+/WpYT7V9+99p\nUdG0kPbEpMp/90KPE2EPeVFNoarzVbXyNvybwFFh5sckFvtqpIN4gBG7/lVtmeZAlyVLWLpwYcL1\noO6vVDImDHlRTRFjEDAz7EyY+GJfjdSUt7iJUwBo+e336dr1GAAqVFkxYwYn9+4dd71K7pVKY+LO\nMyZKciYYi8h8oFWcWSNVdbZfZhRQrqpPxNtGUVFR1efCwkIKCwvTn1GTVOyrkUp5kFL8W5evTdwR\nvL2TruEoLi6muLg47GxETt70ZywiA4HBwBmquiPOfM2XYxFlc+cu4tprX2HduvFV0zp0GFfjK43q\nup4Jn/Vn7ORFnbGInAVcD5wfLxCb6Kjru+XsnXSmocuLkrGIvAccAGzxk5aq6n/FLGMl4wip66uR\n7JVKDY+VjJ28CMapsGBsTDgsGDt5UU1hjDFRZ8HYGGMiwIKxMcZEgAVjY4yJAAvGxhgTARaMjTEm\nAiwYG2NMBFgwNsaYCLBgbIwxEWDB2BhjIsCCsTHGRIAFY2OMiQALxsYYEwEWjI0xJgIsGBtjTARY\nMDbGmAiwYGyMMRFgwdgYYyIgL4KxiEwQkXdEZLmIvCoircPOkzHGBOXFO/BE5EBV/dJ/Hgp0VdUr\nY5axd+AZEwJ7B56TFyXjykDsFQCfh5UXY4yJZ/+wM5AtInIrcCnwNXBSyNkxxphqcqaaQkTmA63i\nzBqpqrMDy90EfF9VL49Z36opjAmBVVM4OVMyVtUzU1z0CeCFeDOKioqqPhcWFlJYWFjvfBljqisu\nLqa4uDjsbEROzpSMkxGRDqr6nv88FOihqpfGLGMlY2NCYCVjJ2dKxjWYKCLfB/YA64HfhJwfY4yp\nJi9KxqmwkrEx4bCSsZMXTduMMSbqLBgbY0wEWDA2xpgIsGBsjDERYMHYGGMiwIKxMcZEgAVjY4yJ\nAAvGxhgTARaMjTEmAiwYG2NMBFgwNsaYCLBgbIwxEWDB2BhjIsCCsTHGRIAFY2OMiQALxsYYEwEW\njI0xJgIsGBtjTARYMDbGmAjIm2AsIsNFpEJEDg47L8YYEysvgrGItAbOBDaGnZdKxcXFlp6lZ+mZ\nKnkRjIHfAzeEnYmgXP9yWXqWnqmdnA/GInI+sFlVV4SdF2OMSWT/sDOQDiIyH2gVZ9Yo4GagT3Dx\nrGTKGGNqQVQ17DxkjIgcD7wKfO0nHQV8CPRQ1U9jls3dA2FMxKlq3heScjoYxxKR94EfqeqWsPNi\njDFBOV9nHCN/fnmMMQ1KXpWMjTEmqvKtZJySbD0gIiITROQdEVkuIq/69tCZTO8uEVnj03xORA7K\ncHq/FJFVIrJHRH6YoTTOEpF3ReQ9EbkxE2nEpPegiJSKyMospNVaRBb6Y/hPERmW4fSaisib/nxc\nLSITM5leIN39RORtEZmdjfSiyoJxjCw/IHKnqnZV1W7AX4GxGU5vHnCcqnYF1uJammTSSuDnwKJM\nbFxE9gPuA84COgEXi0jHTKQV8JBPLxt2Adep6nHAScCQTO6fqu4AevvzsQvQW0ROzVR6AdcAq8nz\nakQLxvvK2gMiqvplYLQA+DzD6c1X1Qo/+iaudUkm03tXVddmMIkewDpV/UBVdwFPAudnMD1UdTGw\nNZNpBNL6RFWX+89lwBrgiAynWdny6ABgPyCjN7tF5CjgbGA6ed7s1IJxQBgPiIjIrSKyCbgMuD1b\n6QKDgBeymF4mHAmUBMY3+2k5R0TaAt1xP6KZTKeRiCwHSoGFqro6k+kB9wDXAxU1LZjrcuKhj9rI\n9gMiSdIbqaqzVXUUMEpEbsKdmJdnMj2/zCigXFWfqE9aqaaXQXlxWSsiBcAzwDW+hJwx/sqpm7+f\n8LKIFKpqcSbSEpF+wKeq+raIFGYijYYk74Kxqp4Zb7p/QOQY4B0RAXcJ/w8R2ecBkXSkF8cTpKGk\nWlN6IjIQd1l4Rn3TSiW9DPsQCN70bI0rHecMEWkMPAs8pqp/zVa6qrpNROYCPwaKM5TMT4DzRORs\noCnQQkQeVdUBGUov0qyawlPVf6rqYap6jKoeg/tS/7A+gbgmItIhMHo+8Ham0vLpnYW7JDzf36zJ\npkzUBy4DOohIWxE5APh34PkMpBMKcaWCB4DVqvqHLKR3iIh8239uhruRnbFzUlVHqmpr/337FbAg\nXwMxWDBOJhuXwBNFZKWvoysEhmc4vXtxNwrn+6ZE92cyMRH5uYiU4FoCzBWRF9O5fVXdDVwNvIy7\nG/+Uqq5JZxqxRGQmsAQ4VkRKRKRe1Uo1OAXoj2vV8LYfMtmS43BggT8f3wRmq+qrGUwvVl5UOyVi\nD30YY0wEWMnYGGMiwIKxMcZEgAVjY4yJAAvGxhgTARaMjTEmAiwYG2NMBFgwNsaYCLBgbHKGfxIv\n4/0MG5MJFoyNMSYCLBibrBORE/zbRpqISHP/FotOcZab6TuRqRx/WER+ISJHi8giEfmHH06Os+5A\nEbk3MD5HRHr5z31EZIlf92kRae6n3+7fqvGOiNyVmb03Jr6867XNhE9V3xKR54FbgGbAjAT95j4F\nXAS84DsCOh24CleIOFNVd/rOlp4ATqgpWUBF5BBcd6lnqOo3/lVNvxWRKcDPVPUHACLSov57akzq\nLBibsIzH9br2DTA0wTIvAX/0gfinwGs+AB8E3CciXYE9wLEppim4Tos6AUt8V6kH4Dr+2QbsEJEH\ngDl+MCZrLBibsBwCNMe92qcZ8HXsAqq6Q0SKgb64EvJMP+s64GNVvdS/By9ed6C7qV4N1zTweb6q\n/jp2BRHpgevn+UJcb3Bp6fPZmFRYnbEJy5+A0bgqhjuSLPcU7hVRPXElZYAWwCf+8wBcQI/1Ae6N\nFeJfMtsDV1XxBnCKiLQD8HXWHXy98bdV9UXgt0DXeuybMbVmJWOTdSIyANipqk+KSCNclUGi1/vM\nA2YAf/X9FwPcDzzrt/MSEHwVkQKo6usi8j6un+M1wD/89M/9205mikgTv84o4EvgbyLSFFedcV3a\ndtiYFFh/xsYYEwFWTWGMMRFg1RQmdCLSGXg0ZvIOVd2n/bAxucqqKYwxJgKsmsIYYyLAgrExxkSA\nBWNjjIkAC8bGGBMBFoyNMSYC/h/ppPB8pp/zvwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x6eab750>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from sklearn.decomposition import PCA as sklearnPCA\n",
"\n",
"sklearn_pca = sklearnPCA(n_components=2) #Number of eigenvectors to project to\n",
"sklearn_transf = sklearn_pca.fit_transform(df_centered)\n",
"\n",
"plt.plot(sklearn_transf[0:20,0],sklearn_transf[0:20,1],\n",
" 'o', markersize=7, color='blue', alpha=1, label='class1')\n",
"plt.plot(sklearn_transf[20:40,0], sklearn_transf[20:40,1],\n",
" '^', markersize=7, color='red', alpha=1, label='class2')\n",
"\n",
"plt.xlabel('x_values')\n",
"plt.ylabel('y_values')\n",
"plt.xlim([-4,4])\n",
"plt.ylim([-4,4])\n",
"plt.legend()\n",
"plt.title('Transformed samples using sklearn.decomposition.PCA()')\n",
"plt.gca().set_aspect('equal', adjustable='box')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can observe that the two results are the same! This concludes the PCA tutorial! "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.11"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment