Skip to content

Instantly share code, notes, and snippets.

@fabgoos
Last active January 20, 2024 15:00
Show Gist options
  • Save fabgoos/6788818 to your computer and use it in GitHub Desktop.
Save fabgoos/6788818 to your computer and use it in GitHub Desktop.
An IPython notebook about bias variance tradeoff in machine learning.
Display the source blob
Display the rendered blob
Raw
{
"metadata": {
"name": "Bias and Variance"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Bias and Variance\n",
"================\n",
"\n",
"[*Machine learning class*](https://sites.google.com/a/unal.edu.co/machine-learning-2013-2/) *support material, Universidad Nacional de Colombia, 2013*\n",
"\n",
"The purpose of this notebook is to illustrate the bias-variance trade-off when learning regression models from data. We will use and example based on non-linear regression presented in Chapter 4 of [[Alpaydin10]](#biblio).\n",
"\n",
"Training data generation\n",
"------------------------\n",
"\n",
"First we will write a function to generate a random sample. The data generation model is the following:\n",
"\n",
"$$r(x) = f(x) + \\epsilon$$\n",
"\n",
"with $\\epsilon\\sim\\mathcal{N}(0,1)$"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import numpy as np\n",
"import pylab as pl\n",
"\n",
"def f(size):\n",
" '''\n",
" Returns a sample with 'size' instances without noise.\n",
" '''\n",
" x = np.linspace(0, 4.5, size)\n",
" y = 2 * np.sin(x * 1.5)\n",
" return (x,y)\n",
"\n",
"def sample(size):\n",
" '''\n",
" Returns a sample with 'size' instances.\n",
" '''\n",
" x = np.linspace(0, 4.5, size)\n",
" y = 2 * np.sin(x * 1.5) + pl.randn(x.size)\n",
" return (x,y)\n",
" \n",
"pl.clf()\n",
"f_x, f_y = f(50)\n",
"pl.plot(f_x, f_y)\n",
"x, y = sample(50)\n",
"pl.plot(x, y, 'k.')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 5,
"text": [
"[<matplotlib.lines.Line2D at 0x10731a5d0>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD9CAYAAABDaefJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtcVHX6B/APeEdN1NJKyBuad0BDVlMbI9dAKVelVUst\nc6e2LX/Wb+u3rrpqJVpmZbnrZrbmplioKXjB1bLBCyrhtdYLlTdQMU0wkZTb+f3xBF4AnZlz5pwz\nZz7v14uXCsOcp2/wzJnv9/k+Xz9FURQQEZEl+BsdABERaYdJnYjIQpjUiYgshEmdiMhCmNSJiCyE\nSZ2IyEJUJfXLly8jMjISYWFh6NChAyZMmKBVXERE5AY/tXXqBQUFCAgIQHFxMXr16oW33noLvXr1\n0io+IiJygerpl4CAAABAYWEhSkpK0KhRI9VBERGRe1Qn9dLSUoSFhaFp06bo27cvOnTooEVcRETk\nhupqn8Df3x979+7FhQsX0L9/fzgcDthstvKv+/n5qb0EEZFPcmd2XLPqlwYNGmDAgAHIyMio8DVF\nUUz1MWXKFMNj8Ja4GBNj8oW4zBiTu1Ql9XPnziEvLw8A8Msvv2Djxo0IDw9X85RERKSCqumX06dP\nY/To0SgtLUVpaSlGjhyJqKgorWIjIiIXqUrqnTt3xu7du7WKRTfXzvmbiRnjYkzOYUzOM2NcZozJ\nXarr1G95AT8/VfNDRES+yN3cyTYBREQWwqRORGQhTOpERBbCpE5EZCFM6iZht9ths9kQExNTXvtP\nROQqJnWTyMzMRGpqKlJSUmC3240Oh4i8FJO6SZR1u4yIiMD8+fOd/j7e4RPRtZjUTSIhIQFxcXHY\nsGEDAgMDnf4+3uET0bVUd2kkbQQGBiIxMdHl73P3Dp+IrIk7Sr1cXl4e7HY75s+f79IdPhGZm7u5\nk0mdiMiE2CbAS3Bhk4g8iUldZ2ZZ2OSLC5E1ManrzCwLm2Z5cSEibTGp68zd0kWtmeXFhYi0xYVS\nH8WqGSJzY/ULEZGFsPqFiIjMmdRZmUFE5B5TJnVWZhARuceUSZ2VGURE7jHlQikrM4jI1xlW/ZKV\nlYVRo0bhxx9/hJ+fH+x2O8aNG6c6MCIiX2ZYUs/JyUFOTg7CwsKQn5+Pbt26YdWqVWjfvr2qwIiI\nfJlhJY133nknwsLCAAD16tVD+/btcerUKbVP65aqqmZYTUNEvkLTQzKOHTuGPXv2IDIy8rrPT506\ntfzvNpsNNptNy8uWK6uaASSRlx06UdXnjVJcDPzwA3DwIHDokPx58CBw/Djg7w/UqAFUry5/lv29\nYUMgLAzo2hUIDwfatZPPE5E1OBwOOBwO1c+j2UJpfn4+bDYbJk2ahEGDBl29gI7TLzExMUhJSUFE\nRMR1vVWq+rxeFAXYvx9YtgxYvRrIzATuvhto316Sc/v28tGihTy+qEgSf3Hx1b+fPQvs2QPs3i1/\nZmcDnTpJku/bF3jkEaB2bV3/s4jIgwxtE1BUVISBAwciOjoa48eP1yQwd1RVNWNENU1ZIk9MlGRe\nWAg89hgweDAQGgrUqaPu+X/+Gdi3T5L8mjXArl3AkCHAqFHA/ffLHT8ReS/DkrqiKBg9ejQaN26M\nd955R7PAvNVPPwFz5wJLlshddlycfNx3H+Dn57nrZmcDCQnAv/8NFBQATzwBjBwJtGnjuWsSkecY\nltS3bt2KPn36oEuXLvD7NWvNmDEDDz/8sKrAvM1PPwHvvAPMmyd343a75xN5ZRQF2LtXkvvSpRJD\nfDzQpYu+cRCROuzSaJBrk/mQIcBf/wrEx9uRmZmJgIAAJCQkGLaB6soV4IMPJKk/9BDw6qtAq1aG\nhEJELmKXRp2dPw9MmgS0bQv8+KPMac+fL4udZuldU6sWMG4c8N13Mg0TEQE8/zxw5oxhIRGRhzGp\nu0hRgE8/BTp0kOR4bTIvY7beNfXrA1OmSPlkjRoS++TJwKVLRkdGRFrj9IsLTp8G/vjHsjtfO/Ly\nKp9iMXvvmuPHgQkTgIwMYNEioEcPoyMiohtx+sWDFEWSX2io1Ibv3g3k5VU9xRIYGIjExERTJnQA\naN5cKmVmzAB+9ztg4kQpuSQi78ekfgtZWcCAAbIY+p//AK+/LnPVZpticceQIVIp8803QGSk/ElE\n3o1J/SaWLJEdmz16AOnpsj2/TEJCAuLi4gzZoaqlO+8EkpKAF14AHnwQmDULKCkxOioichfn1CtR\nVAS8/LLs1FyxQqZdfMGxY8CTTwKlpcBnnwF33WV0RES+i3PqGjl3DujfXypFvv7adxI6IBU8mzYB\n/frJdMzu3Z6/JjtoEmmLSf0ae/ZILXf37sDatdIZ0df4+0u547vvAg8/LH1rPMksNf1EVsHmrb9a\nulQ26sydC/z+90ZHY7zBg2X36aOPAgcOAH/7m2daHlhhwZnITHx+Tr24WGq2V6wAVq70rekWZ+Tk\nSNnjPfcACxcCv+ZgzZi9pp/IKOz94oYrV4B777XjwoVMdOsWgOXLjevTYmaXL0uDsv/+VyplgoKM\njojI+rhQ6qL8fKk/LyjIRF5eKr78knO6ValdWzZfVa9uR0iIDX36cFGTyKx8MqmfPy9dC1u0ALp1\n45yuM/z8gDp1MnHlSiq2bEnBY4/xBZDIjHwuqZ8+DTzwANCrF/Dhh8DSpdbYRKSHskXN1q0jsGfP\nfOzcaXBARFSBTyX1o0clmQ8bJjsn/fzM36cFME8td9ku2oyMDVi0KBCxscCv53kTkYbUTAX7zELp\nf/8rm4omTAD+9Cejo3GNzWZD6q/ZMy4uDomJiQZHJDZtkvLPxYtlbIlIG2W/8+7kTp+oU9+7VzbS\nzJ4NPP640dG4zqy13A8+KNUwgwYBYWF2FBYaf9oTkRUEqKgdtvyd+sGDknzee08OgPZGZq/l3rUL\n6NHDhqIi872bIPJGeXl5aNiwIevUb/TDD7IoGh8PjBplSAg+o3fvGGzdmoKQkAh8/TUXnYnUYp36\nDU6ckLLFSZOY0PWwenUCHnooDrm5G7BrFxM6kVEseaeekwP06QM8+yzw0ku6XtrnpaYCQ4cCq1cD\nv/mN0dEQeS/eqf/q3Dm5Qx85kgndCA88AHz8sTQC27/f6GiIfI/qpD5mzBg0bdoUnTt31iIeVS5c\nkNK6gQNl2oWMMWAAMGcOEB0NfP+9+89jlvp8Im+iOqk/9dRTWL9+vRaxqFJQAMTEAPffLwcqe6JN\nrLcxMikOGwZMmSIHbmRnu/cc7LVO5DrVSb13795oaPBpEiUlwIgR0v/73XeZ0MsYnRTtdtno1a8f\ncPas699/s/p83sUTVU6XzUdTp04t/7vNZoPNZtPsuRUFGD8euHgRSEyUk3tImGHT0p//DOTmypSM\nw+FaP/aEhIQq6/PLXrAASfCsiydv53A44HA4VD+PJtUvx44dQ2xsLL755puKF/Bw9ctbb0lb2K1b\ngQYNPHYZr2SWTUuKAoweLe2Oly0DqlVT/5wxMTFISUlBREQEm7GRJflk9ctnn8mC3Lp1TOiVMUuz\nMj8/YMECuWN/+WVtnrOsuRgTOllRUZH73+u1vV82bwZeeAHYuBEIDjY6GrqVmjWBzz8HevYEWraU\n/3dqlL1gEVmNosgeG3epvlMfPnw4evbsiczMTAQHB2PhwoVqn/KWDh6UPi5LlvBMUW/SsKG8q5ox\nA0hONjoaInOaPVv6KbnL63aU5uQAPXpIudyTT2r2tKSj9HRZOF2/HujWzehoiMwjKQl47jlgxw7g\nnnt8YE69oEA2Fj31FBO6N+veHZg/H3jkEeD4caOjITKHPXuAsWOBVavUTSl7zZ26osiGlho1gE8+\nYS26Fbz7riygbt0KcK2TfNmpU9Irafbsqy3C3c2dXpPUX3sNWLNGGkbVrq1BYGQ4RZEF0++/B9au\n1abUkcjbFBRIz6RBg4CJE69+3tJJfcUK4MUXgZ07gbvu0igwMoXiYjmVKixM9hwQ+ZLSUuCxx4A6\ndYB///v6GQhT16mr2cq9d6+U96xcyYRuRdWry36DVavkh5rIl0yeLMUfCxZoN6WsS1J3t/fImTPS\nwvXvf2eVhJU1biyr/n/+s6z6E/mCJUuApUvlhrVWLe2eV5fpF3e2cl+5AvTtK82gpk3zYIBkGqtX\ny7uy9HSgWTOjoyHynF27ZNpx0yagqq7lpp5Tz83NdSmhKwowZozxTbrsdjsyMzMREBCAhIQEbkfX\nwcyZsoayebPMMxJZzZkzQEQE8M47wJAhVT/O1End1Uu8/baULW7dCtSt66HAnGCz2co7AcbFxXFb\nug4UBXj8cXkhZ+kqWU1RERAVJdUur71288eaeqHUFV98AcyaJXOsRiZ0wByta32Nnx/w0UfAoUPy\nc0BkJS++KM0HPTmlbKo79WPHpAB/6VKZTzeaWVrX+qLsbCAyUqoCoqONjoZIvY8+khuVnTud6yrr\n9dMvv/wiR9GNHCmvZkTbtgGDBwNpaUDr1kZHQ+S+7dulkm/LFuDee537Hq9O6mWHKJSUAIsXcx6V\nrvr734EPPpBfCqOn44jcceqU9Dv65z+ld5WzvDqpv/++vDVJS3PtuDOyPkWRBm6FhVLXyxd8MqOq\nKuWuXAFsNulKOmmSa8/pdUm9bBAuXw7AkSMJ2LkzEC1bejIS8la//AL06iVTc+PHGx0NUUVVVco9\n+yzw44/A8uWul2a7m9QNO/no2oODe/e2o2VLlgtS5erUkVOTIiOlR4yG55YTaaKySrmFC+Ww9fR0\nfffaGFbSWKuWDEJQUASSk50vF7Tb7bDZbKr6yZD3ad5c1ltGjJDKGCIzufHM3F27gFdekZuR227T\nNxbDpl9Gj87Dpk127N8/Hw0bOl8uyA1Bvu3NN6/uONWyXwaRVn76CbjvPvlZLeuN7g6v2nz00UfA\n118H4sCBRJcSOsANQb7u5ZflVJjnnzc6EqKKSkrk3eTQoeoSuhq636mXNbLZsgVo18715+OGILp4\nUebXX3pJjv8iMotJk6SKb8MGaSuthldUv5S9LZk1S17JiNx16BDQuzcPrybzSE6Wd5AZGUCTJuqf\nz/RJvaREajU7d2ZPD9LG8uUyHZORIT3Z1WBHTlLju+9kR3xysrQ60YLp59RffVUK8WfM0OuKZBae\nqlgaOlQ+Hn9c5jLVKCuxdfdAF/Jdly5JO4tXX9UuoauhOqmvX78e7dq1Q5s2bfDGG29U+pg1a4B/\n/Qv49FP180zkfTyZMGfMAC5fll8oNbgAT+5QFMBulynAZ54xOppfKSoUFxcrrVu3Vo4ePaoUFhYq\noaGhyoEDB657DADljjsUZds2NVcibxYdHa0AUCIiIpTc3FzNnz8nR1GCghRlzRr3nyM3N1eJi4vz\nSHxkXe+/ryhhYYpSUKD9c7ubnlXdqaenpyMkJAQtWrRAjRo1MGzYMCQlJVV43N/+BvTsqeZK5M1u\n3JihtaZN5fDqMWOAI0fce47AwEAkJiZyLt2LGL0RMS1NDrpYsaLqU7qMiFHVZMjJkycRHBxc/u+g\noCDs3LmzwuPOnZuKqVPl7zabDTbu8/YpZQnTk3r2lHKysla9bAxnfde2GrHb7bpuRDxzBvj972XP\nTatWVT/uZjHeuDi/d+9eOBwO1bGpSup+TrbMm1qW0Yk86PnnpUXvc89J3w12dLQ2o9ZBiouBYcOA\nJ5+8dSvdm8VYWcK/9oZ3mpvHI6mafmnWrBmysrLK/52VlYWgoCA1T0nkNj8/4MMPZYMb1zqtz9PT\nelWZOBGoWRNw5l71ZjF66kVJVZ16cXEx7r33Xnz55Ze4++670b17dyxduhTt27e/egE3ay2J3JWZ\nKa16V6+WnadEWvn8c9nJvGuX+r0Rt9odb9jmo5SUFIwfPx4lJSV4+umnMWHCBE0CI1IjKQl44QXt\ndvcRHT4su5jXrgUiIjx/PdPvKCXS28SJMseuRR8O8m35+bKxaNw4qUvXA5M60Q1KSqR5XLduwMyZ\n+l+frQesQVFkYbRePWDBAv0W4L3u5CMiT6tWDVi6VJJ69+5S7qgnI0vuSDtvvy37H7Zs8Y6KKiZ1\nsrTbb5fGXwMGAB07Avfeq9+12XrA+331lTQgTE8Hatc2OhrncPqFfMKCBcA77wA7d8rbaD2w9793\ny8qSd3iLFwNRUfpfn3PqpBmrzgWPHSsHbHz6qXe8jSbjXL4M9OkjXUBfecWYGEzfepe8h1Xb0M6d\nCxw9yn7+dGvjxgH33CP9+r0N59SpAqvOBdeuLZtHuncHwsKA3/7W6IjIjBYskEXR9HTvfEfH6Req\nwOpzwZs3y6HAaWlA69ZGR0Nm8vXXsqi+ebN7ZyhriXPqRC6YO1f6w2zfDtSta3Q0ZAY5OfIubs4c\n4He/MzoaJnUilyiK9F8vKODCKclRmw8+CPTr51yjLj0wqRO5yAwVDkazaqWTK8qOpDt3Tg688DdJ\n+Qh3lBK5qHZt+SWOjPSOhVNPJGDuegXmzZNpuO3bzZPQ1bDAfwKR+4KDZfpl5Ejghx+MjubmPFFq\natVKJ2elpgLTpklXz/r1b/5Yo4/PcxaTOvm8Pn3kHN1HHwV+/tnoaKrmiQRs1EETZnD8uDTqWrLE\nuSoob9m/wTl1Isi86p/+JL/oycnSDMxsrF5qqqdLl+QgldGjgfHjnfuemJgYpKSkICIiQpcXQS6U\nEqlUVARERwOhocDs2UZHQ55S1kq3Th3XzrLV+0WVSZ1IA+fPy2EIr7wivWLIeqZPl3djqanm7rzI\n6hciDTRqJGeb9ukDhIQA1xzuThbw2WdXN52ZOaGrwTt1okp88QXwxBPAtm1sJWAVaWnAoEHy/7ZL\nF6OjuTV2aSTS0EMPSUXMwIHAhQtGR0NqHTkCDBkCLFrkHQldDU6/EFXhueeAAweATp3saNUqE3Xr\n+u6uS2+WmwvExACTJ8tCuNVx+oXoJoqLgSZNbMjNlV2XcXFxPrnr0lsVFsrh46GhcvKVN+H0C5EH\nVK8O3HefbPpp1sw3d116K0UBnn1Wdoq+9ZbR0ejH7aS+bNkydOzYEdWqVcPu3bu1jInII9zd5p2Y\nmICBA+NQrdoGrFrFqRdvMXMmsHev7Bg142YyT3F7Tr1z585YuXIlnnnmGS3jIfIYd5tXBQYGYvXq\nRBw6BDzwANC0qfXnZr29e2NCwtVGXXodNG4Wbt+pt2vXDm3bttUyFiKPUts7pV07YOVKYNQoICND\n6+jMxVv6nFQmJQV48UVg7VqgWTOjo9GfLtUvU6/pOm+z2WDjjg4yQEJCgupt3j17Ah9+CDzyiJxj\nadUadm/t3piWJi+6yclA585GR+Mah8MBh8Oh+nluWv3Sr18/5OTkVPh8fHw8YmNjAQB9+/bF7Nmz\n0bVr18ovwOoXsqB//lP6w6SlAXfcYXQ02vPG5mHffCP7CxYtkooXb+eRNgEbN250OyAiK3v2WSA7\nWw4p3rTJevO2gYGBXlW6eeSIrHO8+641EroampQ08k6cfNFrr8nuxNhYOeuUjJGTI6dW/fWvwPDh\nRkdjPLeT+sqVKxEcHIwdO3ZgwIABiLZ6OQDRDfz8gA8+kNOTBg2SM09JX3l5cmc+apTsACbuKCVS\nrbhYmn9dvAh8/jlQq5bREfmGggJJ6GFhwJw5zvdF9xbsp05koKIiOXihuBhYtgyoWdPoiKwtP1+m\nve65Rw66cOXAaG+pwWebACID1agBLF0qW9NHjJDkTp5x8aIsirZqBfzrX64ldMC7a/CdwaROlmLk\nie81a8pd+qVLMsdbUqLr5X3ChQuyKNqxo+wXcGf7v7fW4DuLSZ0sxei7sFq1ZF797FlgzBj9EruR\nL2Z6yc2VOvT77pMWAK7eoZdJSEhAXFycLodHG4FJnSzFDHdhdeoASUnAqVPAY4/pUxVj9IuZp/30\nExAVBfTuDbz3nrpF0bIafCsmdIBJnSzGLHdhAQHAmjXSuvfhhz1/epIZXsw85exZoG9fmXaZPdt6\nVS5aY/ULkQeVlgL/8z/SJyYlBbjrLs9UX3jjtn5nnDghi6KDBwOvvupbCZ0ljUQmpSjA9OlSevef\n/wBjx9rKWwDzJKWqZWQAjz4KvPQS8L//a3Q0+vNI7xciUs/PD5g0CWjSBOjTB2jVyrpTJVpZtQr4\nwx+kwmXQIP2u6y017DfDO3UiHX3+OWC356FTJztWrbLWVIkWFAV4+205T3TVKql00ZPNZp53Udx8\nROQFBg8GVqwIxMGDiVi0KBC837mqqAj44x+ldW5amv4JHbDGgjPv1IkMcPQoMGQI0LYtsGCB9Vr3\nuurCBSn/9PcHPvsMuO02Y+Iw04Iz79SJvEjLlsC2bUDdukD37sChQ567ltk3Ju3fD/ToAYSEAKtX\nG5fQAWvUsDOpExmkTh3go4+ksqN3b2D5cs9cx6wbkxRFNhJFRQH/93/A3LlS10/qcAiJDPb009I+\nduhQYPt2YOZMaRCmFTPOE585Azz1FHDunPw3h4QYHZF18E6dyAS6dQN27QIOHpS79m+/1e65zbLL\ntsy6dUB4uHxs28aErjUulBKZSGmpLJxOmgSMHQtMnizTNFZw+bJMs6xcCXzyCfDAA0ZHZG5cKCWy\nAH9/wG6XxcMjR4BOnQBvP/9dUaQPTliYNDnbu5cJ3ZN4p05kYikpcvZmz56yIadJE6Mjcs2+fbIQ\nfPKkNOOKjvat/i1q8E6dyIKio2V+/e675a59zhw5m9PsTp+W6aPf/lY2XO3fD8TEMKHrgUmdyOTq\n1gVmzZJpmM2bpcb9tdeA8+eNjqyiMWPsaNnShubNYxAQkIfDh+WdhpbVPHRzTOpEXiI0FFixAkhN\nlR2pISEytZGdbXRk8m5i3Djgk08ycexYKoqKUpCTY4cJim18DpM6kZdp104OXN63TxYhu3SRWve0\nNH3PRS0okD4tPXsC/fsDgYFAr17u1cSbfderN1G1UPryyy9jzZo1qFmzJlq3bo2FCxeiQYMG11+A\nC6VEHvXTT3Jm57JlUl0SEwPExsp8ttZb7i9eBHbulOP6EhKA3/xGqnUGDJDdoO72TjFTd0SzMOSQ\njI0bNyIqKgr+/v74y1/+AgCYOXOmJoERkeuOH5fywTVrZGNPZCQwcKDczbdoAQQFOT+/rSjAsWPy\nDmDbNvnz+++Brl1la/+TTwLNm2sTd0xMDFJSUhAREWGaTVJGM/zko5UrV2LFihVYvHixJoERkTr5\n+bK4mpICHD4sCfr0aTlSr0UL+bj7buDKFbkDL/v4+Wf5MydH6ubvv1+mWHr2lF2gNWtqH6uZuiOa\nheFJPTY2FsOHD8eIESMqBDZlypTyf9tsNthsNi0uSUQuKiqShdVjx+Tj5EnZsVq//vUft90G3HEH\n0KwZyxD14nA44HA4yv89bdo0zyT1fv36IScnp8Ln4+PjERsbCwCYPn06du/ejRUrVlS8AO/UiYhc\nZtid+scff4wPP/wQX375JWrXrq1ZYEREvsyQg6fXr1+PWbNmITU1tdKETkRE+lJ1p96mTRsUFhai\nUaNGAIAePXrgH//4x/UX4J06EZHLDF8orfICTOpERC5jQy8iImJSJyKyEiZ1IiILYVInIrIQJnUi\nIgthUicishAmdSIiC2FSJyJT4wEarmFSJyJTy8zMRGpqKlJSUmC3240Ox/SY1InI1AIC3Dsiz1ex\nTQARmZqvHqDB3i9ERL+y2+3IzMxEQEAAEhISvPLFgL1fiIh+5cvz8EzqRGQ5vjwPz+kXIrIcK8zD\nc06diMhCOKdORERM6kREVsKkTkRkIUzqRCqwLwmZDZM6kQq+XA9N5sSkTqSCL9dDkzmxpJFIBSvU\nQ5M56V6nPnnyZCQnJ8PPzw+NGzfGxx9/jODgYM0CIyLyZbon9YsXL6J+/foAgPfffx/79u3DggUL\nNAuMiMiX6b75qCyhA0B+fj5uv/12d5+KiIg0Ul3NN0+cOBGffPIJAgICsGPHjiofN3Xq1PK/22w2\n2Gw2NZclIrIch8MBh8Oh+nluOv3Sr18/5OTkVPh8fHw8YmNjy/89c+ZMHD58GAsXLqx4AU6/EBG5\nzNCGXidOnEBMTAy+/fZbzQIjIvJlus+pf/fdd+V/T0pKQnh4uLtPRUREGnH7Tn3o0KE4fPgwqlWr\nhtatW2PevHlo0qRJxQvwTp2IyGXsp05EZCHsp05EpsbmZ/pgUiciXbD5mT6Y1IlIF2x+pg/OqROR\nLtj8zDVcKCUishAulBJZABcTSS0mdSIT4WIiqcWkTmQiXEwktTinTmQi3rCYaLfbkZmZiYCAACQk\nJJg2Tm/HhVIi0oXNZkNqaioAIC4uDomJiQZHZE1cKCUiXXCKyNx4p05ELvGGKSIr4PQLEZGFcPqF\niIiY1ImIrIRJnYjIQpjUiYgshEmdiMhCmNSJiCyESZ2IyEKY1ImILIRJnYjIQlQn9dmzZ8Pf3x/n\nz5/XIh5dOBwOo0OolBnjYkzOYUzOM2NcZozJXaqSelZWFjZu3IjmzZtrFY8uzPo/0IxxMSbnMCbn\nmTEuM8bkLlVJ/aWXXsKbb76pVSxERKSS20k9KSkJQUFB6NKli5bxEBGRCjft0tivXz/k5ORU+Pz0\n6dMRHx+PDRs24LbbbkPLli2RkZGBxo0bV7yAn5+2ERMR+QjdWu9+++23iIqKKm+Wn52djWbNmiE9\nPR1NmjRxOQgiItKGJv3UW7ZsiV27dqFRo0ZaxERERG7SpE6dUyxEROagSVI/cuQI0tPT0a5dO7Rp\n0wZvvPFGpY8bN24c2rRpg9DQUOzZs0eLS9/U+vXrbxqTw+FAgwYNEB4ejvDwcLz++usej2nMmDFo\n2rQpOnfuXOVj9B6nW8VkxDhlZWWhb9++6NixIzp16oT33nuv0sfpOVbOxKT3WF2+fBmRkZEICwtD\nhw4dMGHChEofp+c4OROTET9TAFBSUoLw8HDExsZW+nW9f/ecicvlsVI0UFxcrLRu3Vo5evSoUlhY\nqISGhioHDhy47jFr165VoqOjFUVRlB07diiRkZFaXFpVTF999ZUSGxvr0ThutHnzZmX37t1Kp06d\nKv263uPkTExGjNPp06eVPXv2KIqiKBcvXlTatm1r+M+UMzEZMVaXLl1SFEVRioqKlMjISGXLli3X\nfd2In6mFVCZDAAADnklEQVRbxWTEOCmKosyePVsZMWJEpdc2YpycicvVsdLkTj09PR0hISFo0aIF\natSogWHDhiEpKem6xyQnJ2P06NEAgMjISOTl5eHMmTNaXN7tmAD3VpfV6N27Nxo2bFjl1/UeJ2di\nAvQfpzvvvBNhYWEAgHr16qF9+/Y4derUdY/Re6yciQnQf6zKChYKCwtRUlJSYW3LiJ+pW8UE6D9O\n2dnZWLduHcaOHVvptY0YJ2fiAlwbK02S+smTJxEcHFz+76CgIJw8efKWj8nOztbi8m7H5Ofnh7S0\nNISGhiImJgYHDhzwWDzO0nucnGH0OB07dgx79uxBZGTkdZ83cqyqismIsSotLUVYWBiaNm2Kvn37\nokOHDtd93YhxulVMRozTiy++iFmzZsHfv/K0Z9TP063icnWsdF0ovfHVxpMLrM48d9euXZGVlYV9\n+/bhhRdewKBBgzwWjyv0HCdnGDlO+fn5GDp0KObMmYN69epV+LoRY3WzmIwYK39/f+zduxfZ2dnY\nvHlzpVve9R6nW8Wk9zitWbMGTZo0QXh4+E3vevUeJ2ficnWsNEnqzZo1Q1ZWVvm/s7KyEBQUdNPH\nlNW2e4ozMdWvX7/8bWJ0dDSKiooMb0ym9zg5w6hxKioqwpAhQ/DEE09U+oNsxFjdKiYjf6YaNGiA\nAQMGICMj47rPG/kzVVVMeo9TWloakpOT0bJlSwwfPhybNm3CqFGjrnuMEePkTFwuj5W66X1RVFSk\ntGrVSjl69Khy5cqVWy6Ubt++3eOLEM7ElJOTo5SWliqKoig7d+5Umjdv7tGYyhw9etSphVI9xsmZ\nmIwYp9LSUmXkyJHK+PHjq3yM3mPlTEx6j9XZs2eV3NxcRVEUpaCgQOndu7fyxRdfXPcYvcfJmZiM\n+t1TFEVxOBzKwIEDK3zeqN+9W8Xl6lhV1+LVpnr16pg7dy769++PkpISPP3002jfvj0++OADAMAz\nzzyDmJgYrFu3DiEhIahbty4WLlyoxaVVxbR8+XLMmzcP1atXR0BAAD799FOPxgQAw4cPR2pqKs6d\nO4fg4GBMmzYNRUVF5THpPU7OxGTEOG3btg2LFy9Gly5dEB4eDgCIj4/HiRMnyuPSe6yciUnvsTp9\n+jRGjx6N0tJSlJaWYuTIkYiKijL0d8+ZmIz4mbpW2bSKkePkbFyujpUmO0qJiMgcePIREZGFMKkT\nEVkIkzoRkYUwqRMRWQiTOhGRhTCpExFZyP8DnN2h6DBVmEMAAAAASUVORK5CYII=\n"
}
],
"prompt_number": 5
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Model fitting\n",
"=============\n",
"\n",
"We will use least square regression (LSR) to fit a polynomial to the data. Actually, we will use multivariate linear regression, over a dataset built in the following way:\n",
"\n",
"For each sample $x_{i}$ we build a vector $(1 , x_{i} , x_{i}^{2} , \\dots , x_{i}^{n})$ and we use LSR to fit a function $g:\\mathbb{R}^{n+1}\\rightarrow\\mathbb{R}$ to the training data."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"# This illustrates how vander function works:\n",
"x1 = np.array([1,2,3])\n",
"print np.vander(x1, 4)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"[[ 1 1 1 1]\n",
" [ 8 4 2 1]\n",
" [27 9 3 1]]\n"
]
}
],
"prompt_number": 6
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"from sklearn.linear_model import LinearRegression\n",
"\n",
"def fit_polynomial(x, y, degree):\n",
" '''\n",
" Fits a polynomial to the input sample.\n",
" (x,y): input sample\n",
" degree: polynomial degree\n",
" '''\n",
" model = LinearRegression()\n",
" model.fit(np.vander(x, degree + 1), y)\n",
" return model\n",
"\n",
"def apply_polynomial(model, x):\n",
" '''\n",
" Evaluates a linear regression model in an input sample\n",
" model: linear regression model\n",
" x: input sample\n",
" '''\n",
" degree = model.coef_.size - 1\n",
" y = model.predict(np.vander(x, degree + 1))\n",
" return y\n",
"\n",
"model = fit_polynomial(x, y, 8)\n",
"p_y = apply_polynomial(model, x)\n",
"pl.plot(f_x, f_y)\n",
"pl.plot(x, y, 'k.')\n",
"pl.plot(x, p_y)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 7,
"text": [
"[<matplotlib.lines.Line2D at 0x1075d9bd0>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD9CAYAAABDaefJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XdclXX7wPEP4MSBA0eJE/cCLMTtMSc4cmHu3bGnrLRf\nlj1amU+paUtNfRw5UnHnDnIkuDdo5sAUFQfmAAWUeb6/P+5HykSFszlc79eLF3DOfe776tvxOjff\ncX2dlFIKIYQQDsHZ1gEIIYQwH0nqQgjhQCSpCyGEA5GkLoQQDkSSuhBCOBBJ6kII4UBMSupJSUn4\n+fnh7e1N7dq1+eijj8wVlxBCCCM4mTpP/cGDB7i6upKWlkazZs346quvaNasmbniE0IIkQ0md7+4\nuroCkJKSQnp6OiVKlDA5KCGEEMYxOakbDAa8vb0pU6YMrVq1onbt2uaISwghhBHymHoCZ2dnIiIi\nuHfvHu3btyc0NBSdTpfxvJOTk6mXEEKIXMmY3nGzzX5xc3OjY8eOHD169InnlFJ29fXpp5/aPIac\nEpfEJDHlhrjsMSZjmZTUb9++TVxcHAAPHz5k+/bt+Pj4mHJKIYQQJjCp++XGjRsMGjQIg8GAwWBg\nwIABtG7d2lyxCSGEyCaTknq9evU4fvy4uWKxmr/3+dsTe4xLYsoaiSnr7DEue4zJWCbPU3/uBZyc\nTOofEkKI3MjY3CllAoQQwoFIUhdCCAciSV0IIRyIJHUhhHAgktTthF6vR6fTERAQkDH3XwghskuS\nup2IjIwkLCyM4OBg9Hq9rcMRQuRQktTtxKNql76+vsybNy/Lr5M7fCHE30lStxNBQUEEBgaybds2\nihUrluXXyR2+EOLvTK7SKMyjWLFirF69OtuvM/YOXwjhmGRFaQ4XFxeHXq9n3rx52brDF0LYN2Nz\npyR1IYSwQ1ImIIeQgU0hhCVJUrcyexnYlA8XIRyTJHUrs5eBTXv5cBFCmJckdSszduqiudnLh4sQ\nwrxkoDSXklkzQtg3mf0ihBAORGa/CCGEsM+kLjMzhBDCOHaZ1GVmhhBCGMcuk7rMzBBCCOPY5UCp\nzMwQQuR2Npv9Eh0dzcCBA/nzzz9xcnJCr9fzzjvvmByYEELkZjZL6jExMcTExODt7U1CQgIvvfQS\nGzZsoFatWiYFJoQQuZnNpjSWLVsWb29vAAoXLkytWrW4fv26qac1ytNmzchsGiFEbmHWTTIuXbpE\neHg4fn5+jz0+YcKEjJ91Oh06nc6cl83waNYMaIn80aYTT3vcVtLS4MIFOHMGzp7Vvp85A5cvg7Mz\n5M0LefJo3x/9XLw4eHtDgwbg4wM1a2qPCyEcQ2hoKKGhoSafx2wDpQkJCeh0OsaPH0/Xrl3/uoAV\nu18CAgIIDg7G19f3sdoqT3vcWpSCkydhzRrYvBkiI+HFF6FWLS0516qlfVWqpB2fmqol/rS0v36+\ndQvCw+H4ce371atQt66W5Fu1gi5doEABq/5nCSEsyKZlAlJTU+nUqRP+/v6MGjXKLIEZ42mzZmwx\nm+ZRIl+9WkvmKSnQqxd07w5eXlCwoGnnv38fTpzQkvyWLXDsGPToAQMHQtOm2h2/ECLnsllSV0ox\naNAgSpYsybfffmu2wHKqO3fg++9h+XLtLjswUPt6+WVwctKOUUpx9+FdLsZe5ELsBS7cvcDFuItc\nj7+Os5MzeZzzPPaV1zkvxQsWx7uMNw1eaECtUrXI4/x438vVqxAUBD/+CA8eQP/+MGAAVKtmg0YQ\nQpjMZkl97969tGjRgvr16+P0v6w1efJkOnToYFJgOc2dO/DttzBnjnY3rtf/lcjjkuLYfXk3oZdC\n2XtlL5F3IlEoPIt74lnCE8/inlQpXoVyRcoBkGZII82QRqohNePnW4m3CI8J5/iN40Tfj6ZOqTr4\nvOBDg7INeKXyK1QrqWVvpSAiQkvuK1ZoMUyaBPXr27J1hBDZJVUabeTvybxHD/j3v2HC5MEcu3WM\nxFKJuHm78UfcHzTyaISuoo6WlVpSy70WJQqWyPgQzK6ElAROxJwgPCacYzeOEXw+mGolqzHUeyiB\ndQIpnK8wAMnJMHeultTbtIGJE6FKFXP+1wshLEWSupXdvQvffPNXMh/5fhzhDzay5vQaQs6EkB6d\nDpdAV0lHyIIQ8ufJb7FYUtNT+fn8zyyMWMjuy7vpXqs7Q72H0qR8E5ycnIiP12KdMQP69IGPP4Yy\nZSwWjhDCDKT0rpUoBStXQu3acPnPu4xdtYhruo40/6kiG85toE/dPrwS8QosAd8Hvqz/br1FEzpA\nXpe8vFrzVTb23siZt85Qo2QNhm0aRs1ZNVkYvpBChQ18+qk2fTJvXi32jz+GxESLhiWEsAG5U8+G\nGzdA/+ZDwh9uIq32GG4Xuo77fXc+7/s5r3m/RpH8RQD7qF2jlGLPlT2M3TGW5PRkpneYTrMKzQBt\nPvxHH8HRo7BkCTRubJMQhRDPIN0vFpRuMPDx/L18++uPUOsnmlZ+mWs/R3N2w1lIgcDAQJsvaHoa\npRQrTq1g7I6xNC7fmKltplKxWEUA1q2Dt96CYcPg008hXz4bByuEyCDdLxZw/s55Rm38hCLjPPk2\n8k1G9KzOH+/9xo5B26gcXxlS7L88sJOTE33r9eXsyLPUdq9Ng3kN+HjXxySmJNKjhzZT5rffwM9P\n+y6EyNnkTv0f4pLiWHVqFUtOLOH3GxdIOd6bwd6D+O5DH/Ln/2u2ij10sRgj+l40Y3eOJexSGAtf\nXUg7z3YoBYsWwYcfwgcfwHvvgYuLrSMVIneT7hcTpBnS+OWPX1hyYgm/XPiFtpXbkXp0EKc2tOen\ntXnx8rJ1hOa3K2oX/df3Z6jPUCa0nICLswuXLsHgwWAwwKpV8MILto5SiNxLkno2KaU4fuM4y35b\nxspTK6lUrBKDvAbRpuxr6AcWJ18+bfFO8eK2jtRybibcpO9PfVFKEdQjiLKFy2IwwBdfwPz5sGGD\nVlvGkvR6PZGRkbi6uhIUFJSj/uoRwpKkTz2LomKj+Hz359SeXZtea3tRNH9RwgaHcWDYAfxc3qBt\n8+I0bAhbtzp2QgcoU7gM2/pvo0XFFjSY24Bfo37F2Vmb7vjdd9Chg1a3xpJkP1ohzExZmBUu8Vy3\nE2+r2Ydnq6Y/NFXuU93VW1vfUvuv7FcGgyHjmKAgpdzdlVq50oaB2tC2P7apsl+VVRNDJ6q09DSl\nlFLh4UpVqKDUhAlK/a2pzMrf318BytfXV8XGxlrmIkLkQMbmToftfklOS2br+a0sPbmUX6N+xb+q\nP/3r96e9Z3vyuuTNOC4tTZuzvW4drF+PQ/afZ9X1+Ov0WdeHAnkKsLrnatwKuBETA926QYUK2mDq\n//YEN5ucOuAshKVJnzpaP/n+6P0sPbmUtafXUq9MPQbUH0CPWj1wK+D2xPHJyVCjhp579yJ56SVX\n1q6VPt00QxqjfxnNnst7COkfQtnCZUlK0gqU/f47bNwIHh62jlIIx5erk/r95PssDF/IrCOzyOOc\nhwH1B9CvXr+MRTaZSUiArl3h5Ekdt25puyLZ8yIia1JK8fnuz1l8YjHb+m/Ds4QnSkGjRnpOnIik\nYUNXNm2SD0AhLMnY3JmjN0SLio1ixuEZLIlYQlvPtvzY9UcaeTR6bvXDu3chIEDbOShvXldCQux/\nEZE1OTk58XHLj3F3dafF4hZs7bsV77LeFCwYSXJyGHv2QK9eerZtkw9AIexNjkvqSin2XtnLtwe/\nZffl3Qz1GUrEGxFUcKuQpdffuAHt2kH79jBtGty7FyR9uk/xL99/4e7qTrul7Vjbay2u/+tQ9/T0\nJTx8HocOaStRhRD2I0cl9d9u/saILSO4/eA27/q9y4/dfsyoHZ4VUVFaXfGhQ7W6505OUKxYMbvv\ncrHlXO7AOoEUL1icHqt7MOPzGRQuXJh58+axf38xOnfWpjy2bGm1cITIFUya3mvqtJvnMcclktOS\n1YRdE5T7VHc17+g8lW5Iz/Y5Tp1Sqlw5pb7/3uRwrK5ly5YKUIAKDAy0SQyHrx5WZb8qqxaFL8p4\nbOdObRpoSIhNQhLCYT36N28Mu79TP3b9GEM2DqG8W3nCR4TjUTT7Uy8iIrSFNF9/Df36WSBIC3vU\n7WHLfn/fcr7sGrSLtkvb4uzkzECvgbzyijYbpmtX8PbWk5IiK0OFMAdXU+YOm/kD5gnGXuJh6kM1\ndsdYVXpaabX0xNLHFgplx+nTSpUtq9Tq1Ua93C7ExsaqwMBAu1icc/rP06rsV2XV2t/XZjx29KhS\nefPa/q8JIRxFbGysYy0+OnLtCAPWD6BO6TrMCphF2cJljbr2hQtaf++kSTBwoFGnEJmIiImg/bL2\nLHp1EQHVAgBo3jyAvXuDqVrVlyNHtsmduhAmcpjaLxExEQQEBfCZ7jPW9VpndEK/ckUbFB0/XhK6\nuXmX9WZj740M2jCIXVG7ANi8OYg2bQKJjd3GsWOS0IWwFbu6U78Ud4lmC5vxTftv6FWnl9HXjImB\nFi3gjTe02uDCMkIvhRK4JpBNvTfRuLy2J15YGPTsCZs3Q6NGNg5QiBwsx9+p335wm/bL2vNB0w9M\nSui3b2t36AMGSEK3NF0lHUu6LqHrqq6E3wgHtO6uxYvh1Vfh5EnbxidEbmRyUh86dChlypShXr16\nRp8jMSWRTkGd6FazG+/4vWP0ee7d0xYVdeqkdbsIywuoFsDsgNkEBAVw+tZpADp2hOnTwd8f/vjD\n+HPr9Xp0Oh0BAQHExcWZKWIhHJvJSX3IkCGEhIQY/fo0QxqvrX2NGu41mNx6stHnefBAW/rftClM\nnqwtLMrtrJUUe9TuwdQ2U+mwrAPX7l8DoHdvbTPrtm3h6lXjziu11oXIPpOTevPmzSlu5G4SSilG\nbBlBmiGNBZ0XPLdmy9Okp0PfvlClira5gyR0jTWT4gCvAbzp+yYdgzoSnxwPaJUd33pLS+y3bmX/\nnM+any938UJkziqLjyZMmJDxs06nQ6fTAfBp6KecvHmSXYN2PVbjPDuUglGjID4eVq8GZ7sZJbA9\nay9a+rDph0TFRdFrbS829d5EXpe8vP8+xMZqXTKhodmrxx4U9PS6PI8+sEBL8PZe6kGI5wkNDSU0\nNBTQKs8azRwT5aOiolTdunUzfe5pl1h2YpmqOqOquplw06RrT5umVN26SsXFmXQah2SLRUup6anK\nf5m/en3T6xkLxgwGpQYMUKpbN6XS0sxzHdkxSTgqg8Gg2vzYxujFRza5r01KS2LszrH82PVHShcq\nbfR5Vq3SBuR+/hncntwDI9d7VKzMmguB8jjnYVXPVRy9fpQpe6cAWnfYggXaHfuYMea5TlBQEIGB\ngWzbJgudhGNZHLGYOw/uGv16myT1+cfm41XGK2NuszF274a334YtW6B8eTMGJ0xWJH8RtvTdwn+P\n/Zeg34IAyJcPfvoJgoNh5kzTr2GLDywhLC0mIYYPd3xIhfAfjD6HyUm9T58+NGnShMjISMqXL8+i\nRYueefyD1AdM3juZia0mGn3NM2cgMBCWL8/de4rasxeLvMiWPlsYFTKK3Zd3A1C8uPZX1eTJsGmT\njQMUwg69Hfw29VKHc+mgt9HnsPqK0q/2f8WBqwdY12udUeeLiYHGjbXpcoMHmylIYTE7Lu6g30/9\nCBscRk33mgAcPqwNnIaEwEsv2ThAIezEhrMbeGvDh6jZJzi0rwAVKuSAPUrjk+OpOrMqOwfupG7p\nutk+14MH2vL/Ll3gk0/MHamwlEXhi/hizxccGn6Ikq4lAVi/HkaOhP37oeLTt5IVIleIS4qjxvS6\nJAcFsX1BC3x9c0iZgBmHZtC6cmujErpSMGQI1KwJH39sgeCExQzxGULXml3ptbYXqempAHTrpg2a\nduwIMs1c5HZvb/yQxPBOzB+vJXRTWO1OPS4pjmozq7F3yF5quNfI9nn+8x9tUDQsDAoUsECgwqLS\nDel0XtGZKsWr8H3A94D2Qf3221opga1bwcXFxkEKYQO/nAuj86L+fFj4FP8Z/9c0Pru/U//mwDd0\nqt7JqIS+bh3Mnw8bNkhCz6lcnF1Y0WMFO6N28t+j/wW0qY7ffQdpafDhhzYOUAgbSEx+SI8lw2kS\nO4uJ48wzL9sqK0rbdGnD8abHOaY/lu3XRkRoJXRDQuCFFywQnLAatwJubOq9iWaLmlHTvSa6Sjry\n5NHWG/j5Qf36Uvte5C5tv/iMArE+/DKji9nKm1jlTn3nw50Uv16cysUrZ+t1N29qJVxnzZJZEo6i\nWslqBHUPovfa3lyMvQhAyZLaXqfvvw8HD9o4QCGsZPLCkxxKWUjY2Jnkz2++81olqbs0dGHj+xuz\n9ZrkZG0wbfBg6GV8eXVhh1pXac34FuPpsqJLRo2LOnXghx+gRw+4ds3GAQphYceOwYTQCbzv92/q\nVCxj1nNbZaBUv17P3K5zs/wapWDoUNsX6dLr9URGRuLq6kpQUJCsXjQjpRRvbH2D6/HX2fDaBlyc\ntVHSKVO0MZTdu6FgQRsHKYQF3LwJ3m1/J+m11lz74CKueTOvcmfsQKlVkvr1+9d5oUjWO8S/+QaW\nLoW9e6FQIQsG9xw6nS6jEmBgYKBUAjSzlPQU2i1tR+PyjTNq6SsF/fppH+RLl0oZZeFYUlOhdWuI\nbdWPfm3rMbbZ2Kcea9ezX7KT0HfsgGnTtD5WWyZ0sH7p2twmn0s+1gSuYcVvK1j9u/aB6eSkdcOc\nPau9D4RwJKNHQ57SfxBTeBtv+r5pkWvYVfXxS5egf38ICoIKFWwdjVQCtIZShUqxofcG3vr5LU7E\nnAC0bpcNG7QKnMHBNg5QCDP54QftptWj9xTe9H2TovmLWuQ6Vq/98jQPH2pb0Q0YoH2aidxl5amV\nfLTzI468fgR3V3cA9u2D7t21UgKenjYOUAgTHDigzeRbFXKFnjt8iBwZmVEy42nsuk/9eZdQCgYN\n0ralW7ZM+lFzqw93fMiRa0f4pf8vGTthzZoFc+dq/yhs3R0nhDGuX4eGDeG//4VfXN6mYJ6CTG07\n9bmvy9FJfeZM7U+T/fuzt92ZcCzphnQ6rehE9ZLVmd5hOvBXzZ+UFK3UsnzgC3v0tJlyycmg02k1\njoaPiqH2rNqcfus0ZQuXfe45c1xSf9QISUmuXLwYxKFDxaicvbVJwgHFPoyl4YKGjGs+jsHegwGt\na65ZM61rbtQo28YnRGaeNlPujTfgzz9h7VoYu/MDktKSmOE/I0vnNDapW6VMQGb+vnFw8+Z6KleW\n6YICihcszsbeG2m5uCW1S9WmYbmGFCyo7Zrk5wfe3tqdjxD2JLOZcosWaZutHz4MsUl3WHB8ASfe\nOGHxWGw2+yV/fq0RPDx82bQp69MF9Xo9Op2OgIAA4qRmq0OqXao2CzovoMfqHtyIvwFoNdeXLYO+\nfeHqVRsHKMQ//HOm3LFj8MEH2s1I0aIw4/AMetTuQXk3K+y9adR21dnwtEsMHBirPDwC1d272dsJ\nvmXLlgpQgAoMDDRHiMJOTQydqBotaKSSUpMyHvvyS6UaNlQqKekZLxTChm7fVqpSJaVWr9Z+v5d0\nT5X8sqQ6f+d8ts5jbHq2yZ36Dz/AkSPFOH16NcWLZ2/+tywIyj3GtRhHuSLl+NfWf2X0LY4Zo200\nPnKkjYMTIhPp6dpfkz17avsoA8w+MpsOVTtQtURV6wRh1EdBNvzzEkePKuXurtSZM8adLzY2VgUG\nBqrY2Ozd4YucKT45XtWfU19NPzg947H795WqVUup+fNtGJgQmRg3TqlWrZRKTdV+T0xJVGWmlVGn\nbp7K9rmMTc9Wnf1y5w68/LK2/LtnT0teVTiSqNgoGv/QmOXdl9O6SmtAKyPQvLlsXi3sx6ZN2l+Q\nR49C6dLaY7OPzGbbhW1s6L0h2+ez69ovoP1Z0q+flswloYvsqFy8Mit7rqTfT/0yarDXrAlz5mjv\npTt3TL+GDMALU5w/D8OHa1VlHyX0dEM63xz4hjFNxlg1Fqsl9YkTtYn4kydb64rCXpgjYeoq6fi4\nxcd0WdGF+OR44K8bhH79tJsGUzyaYhscHIxerzftZCJXSUzUyllMnAiNGv31+IazGyhdqDRNKzS1\najwmJ/WQkBBq1qxJtWrV+PLLLzM9ZssWWLgQVq6EPDabGS9sxVwJ803fN2lcvjEDNwzEoAyAdpOQ\nlKT9gzKFDMALYygFer3WBThixN8fV0zbP433m7xvi6CMl5aWpjw9PVVUVJRKSUlRXl5e6vTp0090\n9pcqpdS+faZcSeRk/v7+ClC+vr4mD3AnpyWrpj80VZ/s+iTjsZgYpTw8lNqyxfjzygC8MMbMmUp5\neyv14MHjj++9vFd5TvdUaelpRp/b2PRsUlLfv3+/at++fcbvkydPVpMnT34isJkzTbmKyOnMnTBj\n4mNUhW8rqFWnVmU8tm+fUqVLK3XhglkuIXKA119/XbVs2VL5+/vb5MP4We+5riu7qlmHZ5kUo7FJ\n3aTOkGvXrlG+/F8rpDw8PDh06NATx92+PYEJE7SfdTodOlnnnasUK1bMrLtGlSlcho29N9J2aVsq\nFatEw3INadIExo//q1SvFIZzfH8vNaLX6626M9nNm/Daa9qamypV/hHXnUj2XdnH8u7LCYgMeGqM\n/ywCFhERQWhoqMmxmZTUnbJYMm/Co4wuhJl4l/Xmhy4/0G1VNw4OO0h5t/KMHKmV6H3zTa3uhlR0\ndGy2GgdJS4PevWHwYOjU6cnnvznwDW+8/AaueV2fGWNmH0p/v+H97LPPjIrPpIHScuXKER0dnfF7\ndHQ0Hh4eppxSiCzrUqMLoxuNpvOKziSkJODkBPPnazu1y1in47PVzmTjxkG+fJDZveqfiX+y6vdV\njGw48rkxWuxDyahOm/9JTU1VVapUUVFRUSo5OfmpA6VCWIrBYFDDNg5TXVZ0yRiUOndOqVKllDp4\n0MbBCYezbp1SFStq9V0y8+muT5V+sz5L53reWJOxudPkFaXBwcGMGjWK9PR0hg0bxkcfffTY88au\nihIiq1LSU2i3tB0NyzXM2FFm40Z4++3HV/cJYYpz57RVzFu3gq/vk88/SH1Ape8qsWfIHmq41zD5\nejlukwwhzOnOgzs0+qERY5uOZViDYYD2Z/KBA7Btm6yPEKZJSNAWFr3zjjYvPTNzjswh5EIIG3tv\nNMs1JamLXO/c7XO0WNyCVT1XoaukIz0dOnTQFoZMmWL9eJ62xZnIWZTSBkYLF4YFCzIfgE83pFNz\nVk0WdllI84rNzXJdu6/9IoSl1XCvQVD3IF5b+xpnbp3BxQVWrNC+fvrJ+vFI6QHH8M03cPGitgn6\n02ZUbTq3iRIFS9CsQjPrBpcJSerCobSu0pppbafhv9yf6/HXcXfX9od84w2tT9SapPRAzrdrl1ZV\ndt06KFDg6cd9deArxjQZk+Vp3pYkSV04nIFeAxnx0gj8l/tzL+kevr4waZK2MCkhwXpx2GrKnTCP\n6Ghtw4vly6FChacft/vybm4m3KRbzW7WC+4ZpE9dPMER+oKVUowMHsmZW2cI7hdM/jz5GT4c4uO1\nwnJ2cEMl7FhSErRooVUB/eCDZx/bfll7AmsHMrzBcLPGIH3qwmwcoS/YycmJGR1mUKxAMQZvHIxB\nGfj+e4iK0v6cFuJZ3nlHuzsf85xS6EeuHeH0rdMM9BponcCyQJK6eIKj9AW7OLuwvPtyou9F88H2\nDyhQQBsw/e47bZqjEJlZsAD27MlaqYlJeycxpskY8rnks05wWSDdL+IJcXFx6PV65s2blyO7Xv7p\n7sO7NFvYjNcbvM7oxqPZvVvbFHj/fvD0tHV0wp4cOQIdO8Lu3druWs9y6s9TtPmxDRffvYhrXvNX\nkJN56kI8w5V7V2i6sCnT2k6jd93efP+9Vh/mwAEoVMjW0Ql7EBMDDRvC9OnQLQtjnv1+6ke90vUY\n22ysReKRpC7Ec5y8eZK2S9uyoPMCOlXvzNCh8OCBDJwKbavNV16Btm0zL9T1T3/c/YNGCxpx8d2L\nFM1f1CIxSVIXIguOXDtCx6COLO22lJYe7bM8w8FROcJMJ1M92pLu9m1tPrpzFkYa9Zv1lC1clomt\nTNxH8RkkqQuRRfuu7KPrqq6sCVyDp4sOPz9YvBjatbN1ZM9miQSs0+kyanoHBgZadaMJezF7tvZ1\n4AAUKfL846/ev4rXf72IHBlJSdeSFotLpjQKkUVNKzRldc/VBK4JJJr9rFwJAwbAhQu2juzZLDHV\n1FFmOhkrLAw++0yr6vm8hK7X69HpdDT7oBl9a/W1aEI3hSR1kSu1qtyKpd2W0nVlV1yrHuWTT+DV\nV+H+fVtH9nSWSMC5edXr5ctaoa7ly7M2CyoyMpKwo2FcLnaZSysuWTo8o0n3i8jVNp7diH6Lnl/6\nbWPef7y4fBk2bQIXF1tH9iRHm2pqS4mJ0KwZDBoEo0Zl7TUBAQEEpwRTqnwpIr+NtPj/A+lTF8JI\na35fwzsh7xDSZyf/N7A2Xl7w9de2jkpYyqNSugULZm8v28s3L1P9++ocef0I9SvUt2yQGJ87ZesA\nkesF1gkkJT2FDitaE/T9VkZ0aUCtWjDcvKU8hJ2YNAkuXdL607MzlXXpuaX0eamPVRK6KSSpCwH0\nq98P17yuvLa5A98uXMd7PZpTtSr8bXN34QBWrfpr0dmzSun+U+zDWGYcmsGeIXssF5yZSPeLEH+z\n4+IO+qzrw+hKPzLjbX/27ZNSAo5i/37o2hV27ID62bzZHrN9DPeT7zO301zLBJcJ6VMXwkwORB+g\n66qudHKeyf75vTh4ENzcbB2VMMXFi9C0KSxcCP7+2XvtpbhLvDzvZU69eYqyhctaJsBMSFIXwoxO\n3jyJ/3J/alz/jPOrDlOlSiSFCuXeVZc5WWwsNG6sldN9883sv77vur7UKFmDT3Wfmj+4Z5CkLoSZ\nnb9znrZL2/LnZhce/noRyL2rLnOqlBRt83EvL/j22+y//uj1o7y68lXOjTxH4XyFzR/gM0hSF8IC\nrt6/Ss0vapIYkciLp1/m99+2y516DqEUDBsGd+5odfSzu/ZAKUWrJa3oV68fr7/0umWCfAarlwlY\ns2YNderq899wAAAVf0lEQVTUwcXFhePHjxt7GiGs5tEy74CAAOLi4rL0Go+iHvw2+jdK1i3N3XYl\nWPGTLMLOKaZMgYgIbcWoMYvJtkRu4faD2wzxGWL+4CzI6HdovXr1WL9+PS1atDBnPEJYjLG1UyqX\nrcyNqVfp2qoKb0c0YdGGKAtGaR+M+QC0J0FBMGcObN4MhY3oNUkzpPHBjg+Y2nYqeZxz1sxvo6Ot\n+bxtQYSwM6bUTsnrkpegfrN5Ic8shu1vQprral5v19wSYdqFRx+AoCX4nDSOEBwMo0drUxfLlTPu\nHAuOL+DFIi/iXzWbU2XsgFU+gib8req8TqdDJys6hA0EBQWZVDvFycmJb3qPpGhqdd7Y2ZO7TOHD\ndjnrT/OsyqnVG/fvh4EDtfo99eoZd4745Hg+C/uMrX234mTF3VNCQ0MJDQ01+TzPHCht27YtMTEx\nTzw+adIkOnfuDECrVq34+uuvadCgQeYXkIFS4YAmfH+WSVGdeb3Fq8zo/CUuznZYAcwEObF42G+/\nQZs2sGSJNuPFWJ/s+oSouCiWdltqvuCMYLPZL5LURW71f+PvsiDuNer7pLIicBkeRT1sHVKudfEi\ntGgB06ZBnz7Gn+d6/HXqzanHcf1xKharaL4AjWDTTTIkaYvc6Kv/lKBHUgjXdrfjpbkvs+ncJluH\nlCvFxGi7Vv3736YldICPdn7E8AbDbZ7QTWF0Ul+/fj3ly5fn4MGDdOzYEf/srr0VIodzcoL5c11o\npv5NxYM/8fbP7/BuyLskpyXbOrRcIy5O62oZONC41aJ/tyVyC7sv72Z88/HmCc5GZPGRECZKS4P+\n/eHug1gK9RnOpfsXWdljJTXca9g6NIf24IGW0L29Yfr07JXR/ac7D+5Q/7/1CeoeRMtKLc0XpAlk\nj1IhbCRPHli6FIrkLU76yrUM9dLTbFEzFkcslhsaC0lIgI4doXJl+O677CX0zObgvx38NoG1A+0m\noZtC7tSFMJOUFOjZE/Llg3EzfmPgxr5UcKvAnI5zqOBWwdbhOYz4eAgIgOrVtdro2V0tqtPpMubg\nBwYG0mtCL8b9Oo7wEeG45nW1QMTGkTt1IbDtSsh8+WDNGm3/y2nv1+PwsGM09mhMg7kNmHFoBumG\ndKvG44ju3dMGRevUgfnzjVv+//c5+F989wUjfx7Jkq5L7Cqhm0Lu1IVD+eddmC1WQj58CF26wIsv\navW7z8eeRb9ZT0p6CvM7z6deGSNXxTyDXq8nMjISV1fHLQ8cG6sl9EaNYMYM4/vQH83Bnzt3LkO3\nDaWme00mt55s3mDNwOjcqSzMCpcQIoO/v78ClK+vr4qNjbVZHImJSrVpo1T37ko9fKhUuiFdzT06\nV7lPdVfjdo5TD1MfmvV6LVu2VIACVGBgoFnPbQ9u31bKx0ep0aOVMhjMc86lJ5aqurPrqqTUJPOc\n0MyMzZ3S/SIcSlBQEIGBgWzbts2md6uurrBlizaI2qEDxN93Rv+SnpNvnOTs7bPUm1OP9WfWm+2v\n2Jy6rD8rbt2CVq20u/SvvzZtlssj1+5f471f3mNJ1yXkz5Pf9BPaEel+EcKCDAZ4913Ys0crNPXC\nC1pXycHbB7lU7RJ1qtbhO//v8PPwM+k6OXFZf1ZcuaJtP9e9O0ycaJ6ErpQiICiARuUaWX03o+yQ\ngVIh7JCzs9b/27MnNGsGf/yhVUD8bf1vxE+LJ+1IGt1Xd6fPuj5ExRpf0rdYsWKsXr3aoRL60aPa\nNnRDh8J//mOehA4w68gs/kz8k383/7d5TmhnJKkLYWFOTjB+PHz4oVafJC3tf10lL/uyfep2IkdG\nUsu9Fi/Pf5n3t71P7MNYG0dsexs2aHfos2bB//2f+c67+dxmvtjzBSt7rCSvS94nns/pdeQBGSgV\nwprWrVOqZMlY1bJl4BMDudfvX1fDNw1XJb8sqcb/Ol7dSrxloyhtx2BQ6quvlCpXTqkjR8x77gPR\nB5T7VHd16Oqhpx5jTwPOxuZOuVMXwoq6d4d164px5sxqliwpxt+7TF8o8gLzO8/n0PBD3Ey4SfWZ\n1RmzfQwxCU+Wv3ZEqanwr39ppXP374eXXzbfuc/dPkfXlV1Z0nUJDcs1fOpxjjDgLAOlQthAVBT0\n6KGtilywIPMt16LvRTNt/zSWnVxG//r9GdNkDOXdyls/WCu4dw969dLGIFatgqJFzXfuG/E3aLqw\nKeNbjGeoz9BnHmtPA84yUCpEDlK5MuzbB4UKQcOGcPbsk8eUdyvPDP8ZnH7rNAXyFMB7rjdDNg4h\n/EZ4tq5l7/3EJ09qA6JVq2p7ipozod9Pvk9AUABDvIc8N6GDgww4m7ELKFNWuIQQOdqCBUq5uyu1\nZs2zj7udeFtN3jNZeXzjoZotbKZWnVqlUtJSnnt+e+on/juDQanp07X/9sWLzbeo6JHktGTV5sc2\nasTmEcpg7pNbgbG5U7pfhLADx45p0x67d4cpUyDvkxMzMqQZ0thwdgMzD8/kwt0LvOn7Jq83eJ1S\nhUplenxAQADBwcH4+vrafFHWIzdvwpAhcPs2BAVpd+nmZFAGBq4fSEJKAut6rcuR2w1K94sQOdhL\nL2mJ/cwZaN4cTp16+rF5nPPQs3ZPwgaHsaXvFi7EXqDazGr0+6kfv0b9ikEZHjveXlbZPvLzz+Dj\no33t22f+hJ6Ykkj/n/pzKe4SQT2CcmRCN4XcqQthRwwGbeB0/HgYPhw+/hgKFnz+6+48uMPy35bz\nQ/gP3E++zxDvIQz2HmxXJX+TkrS5+uvXa/XnW1qgdPn5O+fpvro7L7/4MrMDZlMwbxYaz07JnboQ\nDsDZGfR6bfDw4kWoWxe2b3/+60q6luQdv3eIGBHBul7r+DPxT3zm+tBuaTtWnlrJw9SHlg/+KZTS\n6uB4e8P16xARYZmEvvncZpoubMpbvm+xsMvCHJ3QTSF36kLYseBgbe/NJk3g22+hdOmsvzYpLYkN\nZzewKGIRh64eolP1TvSu25t2nu3I55LPckH/zYkT2orQa9e0Ylz+/uZb7v9IuiGdCWETWByxmDWB\na2jk0ci8F7ARY3OnJHUh7FxiIkyYoC3KGTcOXn9dqwKZHTcTbrLuzDpWnlrJ77d+p1vNbvSu2xtd\nJR15nPOYPeYbN7Suo82b4dNPtZifNfhrrLsP79J3XV+S0pJY1XMVZQqXMf9FbESSuhAO7sQJrVLh\n3r0wciS89RaUKJH980Tfi2bN6TWsPLWSy/cu06l6J7pU70KbKm0olK+QSTEOHapn165Irl1z5Y03\ngpg4sRiWGJs1KAPrz6zn/e3v071Wd75s86VFPpxsSZK6ELnE2bMwdapW9GrIEBg9Gjw8jDvXxdiL\nbD63mc2Rmzl87TAtK7WkS/UudKreiReKvJDl85w6pe0XOmeOjrQ0y+08lW5IZ83pNXy++3Nc87oy\nsdVEOlTtYNZr2AtJ6kLkMtHRWj/74sXQrRsMGwZ+fsbt2wkQlxRHyB8hbDq3iZA/QvAs4ckrlV+h\nVaVWNKvQjML5Hq9l8OCBtifr3Llw+bJ2/T17AggNzf6c+Odtx5dmSGPFbyv4Ys8XlChYgk9afkJ7\nz/Y4mbuD3o7YJKmPGTOGLVu2kC9fPjw9PVm0aBFubm5mCUwIkTV37sCcOVqCvX4dAgKgc2dtpyBj\nl9ynpqdy4OoBdkXtYtelXRy9fpT6ZerT9MVWlEpsxcWwJqwJcqVRI222TseO2i5PxtZOedresjEJ\nMWw+t5kv931JuaLl+KTFJ7xS+RWHTuaP2CSpb9++ndatW+Ps7MzYsWMBmDJlilkCE0Jk3+XL2vTB\nLVu0hT1+ftCpE9SvD5Uqad00WR2wVAouXdIqJobte8jO8/uJdtlFwVq7eFgsnApuFXnZwwuvMl54\nldW+v1jkRaMS7qNVr17NvHhr2lscvX2U0Euh3Eq8ha6Sjnf93qVlJQvMg7RjNu9+Wb9+PevWrWPZ\nsmVmCUwIYZqEBG2Oe3AwnDunJegbN7Qt9SpV0r5efBGSkyE+/q+v+/e17zEx2rz5pk21KZVNmmir\nQPPl0+7kz94+S0RMBCduntC+Yk5gUAaqlqhK6UKlKV2oNKUKlaK0q/a9lGspFIp7Sfe4l3yP+8n3\nuZd8j3tJ94i5F0PwyWDyueWjZaWWtKzYEl0lHfXK1MPZKXcup7F5Uu/cuTN9+vShb9++TwT26ad/\n7QOo0+nQ6XTmuKQQIptSU+HqVS3BX7qkzR8vWBCKFHn8q2hRKFUKypXL+rxypRQxCTFExUXxZ+Kf\n3Eq8pX1/8Nd3Zydn3PK74VbATfue342i+YtSrEAxGrzQgDql6+TaJB4aGkpoaGjG75999pllknrb\ntm2JiXmySP+kSZPo3LkzAF988QXHjx9n3bp1T15A7tSFECLbbHanvnjxYubPn8/OnTspUKCA2QIT\nQojczNjcadJs/ZCQEKZNm0ZYWFimCV0IIYR1mXSnXq1aNVJSUijxv2VtjRs3Zvbs2Y9fQO7UhRAi\n22w+UPrUC0hSF0KIbJPSu0IIISSpCyGEI5GkLoQQDkSSuhBCOBBJ6kII4UAkqQshhAORpC6EEA5E\nkroQwq7p9Xp0Oh0BAQHExcXZOhy7J0ldCGHXIiMjCQsLIzg4GL1eb+tw7J4kdSGEXXN1dQXA19eX\nefPm2Tga+ydlAoQQds3YLfJyOqn9IoQQ//O8jaxzAqn9IoQQ/5Ob++ElqQshHE5u7oeX7hchhMNx\nhH546VMXQggHIn3qQgghJKkLIYQjkaQuhBAORJK6ECaQuiTC3khSF8IEuXk+tLBPktSFMEFung8t\n7JNMaRTCBI4wH1rYJ6vPU//444/ZtGkTTk5OlCxZksWLF1O+fHmzBSaEELmZ1ZN6fHw8RYoUAWDm\nzJmcOHGCBQsWmC0wIYTIzay++OhRQgdISEjA3d3d2FMJIYQwkzymvHjcuHEsXboUV1dXDh48+NTj\nJkyYkPGzTqdDp9OZclkhhHA4oaGhhIaGmnyeZ3a/tG3blpiYmCcenzRpEp07d874fcqUKZw7d45F\nixY9eQHpfhFCiGyzaUGvK1euEBAQwKlTp8wWmBBC5GZW71M/f/58xs8bN27Ex8fH2FMJIYQwE6Pv\n1Hv27Mm5c+dwcXHB09OTOXPmULp06ScvIHfqQgiRbVJPXQghHIjUUxdC2DUpfmYdktSFEFYhxc+s\nQ5K6EMIqpPiZdUifuhDCKqT4WfbIQKkQQjgQGSgVwgHIYKIwlSR1IeyIDCYKU0lSF8KOyGCiMJX0\nqQthR3LCYKJerycyMhJXV1eCgoLsNs6cTgZKhRBWodPpCAsLAyAwMJDVq1fbOCLHJAOlQgirkC4i\n+yZ36kKIbMkJXUSOQLpfhBDCgUj3ixBCCEnqQgjhSCSpCyGEA5GkLoQQDkSSuhBCOBBJ6kII4UAk\nqQshhAORpC6EEA5EkroQQjgQk5P6119/jbOzM3fv3jVHPFYRGhpq6xAyZY9xSUxZIzFlnT3GZY8x\nGcukpB4dHc327dupWLGiueKxCnv9H2iPcUlMWSMxZZ09xmWPMRnLpKT+3nvvMXXqVHPFIoQQwkRG\nJ/WNGzfi4eFB/fr1zRmPEEIIEzyzSmPbtm2JiYl54vEvvviCSZMmsW3bNooWLUrlypU5evQoJUuW\nfPICTk7mjVgIIXIJq5XePXXqFK1bt84oln/16lXKlSvH4cOHKV26dLaDEEIIYR5mqadeuXJljh07\nRokSJcwRkxBCCCOZZZ66dLEIIYR9MEtSv3jxIocPH6ZmzZpUq1aNL7/8MtPj3nnnHapVq4aXlxfh\n4eHmuPQzhYSEPDOm0NBQ3Nzc8PHxwcfHh88//9ziMQ0dOpQyZcpQr169px5j7XZ6Xky2aKfo6Gha\ntWpFnTp1qFu3LjNmzMj0OGu2VVZisnZbJSUl4efnh7e3N7Vr1+ajjz7K9DhrtlNWYrLFewogPT0d\nHx8fOnfunOnz1v63l5W4st1WygzS0tKUp6enioqKUikpKcrLy0udPn36sWO2bt2q/P39lVJKHTx4\nUPn5+Znj0ibFtGvXLtW5c2eLxvFPu3fvVsePH1d169bN9Hlrt1NWYrJFO924cUOFh4crpZSKj49X\n1atXt/l7Kisx2aKtEhMTlVJKpaamKj8/P7Vnz57HnrfFe+p5MdminZRS6uuvv1Z9+/bN9Nq2aKes\nxJXdtjLLnfrhw4epWrUqlSpVIm/evPTu3ZuNGzc+dsymTZsYNGgQAH5+fsTFxXHz5k1zXN7omMC4\n0WVTNG/enOLFiz/1eWu3U1ZiAuu3U9myZfH29gagcOHC1KpVi+vXrz92jLXbKisxgfXb6tGEhZSU\nFNLT058Y27LFe+p5MYH12+nq1av8/PPPDB8+PNNr26KdshIXZK+tzJLUr127Rvny5TN+9/Dw4Nq1\na8895urVq+a4vNExOTk5sX//fry8vAgICOD06dMWiyerrN1OWWHrdrp06RLh4eH4+fk99rgt2+pp\nMdmirQwGA97e3pQpU4ZWrVpRu3btx563RTs9LyZbtNPo0aOZNm0azs6Zpz1bvZ+eF1d228qqA6X/\n/LSx5ABrVs7doEEDoqOjOXHiBG+//TZdu3a1WDzZYc12ygpbtlNCQgI9e/Zk+vTpFC5c+InnbdFW\nz4rJFm3l7OxMREQEV69eZffu3Zkuebd2Oz0vJmu305YtWyhdujQ+Pj7PvOu1djtlJa7stpVZknq5\ncuWIjo7O+D06OhoPD49nHvNobrulZCWmIkWKZPyZ6O/vT2pqqs0Lk1m7nbLCVu2UmppKjx496N+/\nf6ZvZFu01fNisuV7ys3NjY4dO3L06NHHHrfle+ppMVm7nfbv38+mTZuoXLkyffr04ddff2XgwIGP\nHWOLdspKXNluK9O69zWpqamqSpUqKioqSiUnJz93oPTAgQMWH4TISkwxMTHKYDAopZQ6dOiQqlix\nokVjeiQqKipLA6XWaKesxGSLdjIYDGrAgAFq1KhRTz3G2m2VlZis3Va3bt1SsbGxSimlHjx4oJo3\nb6527Njx2DHWbqesxGSrf3tKKRUaGqo6der0xOO2+rf3vLiy21Z5zPFpkydPHr7//nvat29Peno6\nw4YNo1atWsydOxeAESNGEBAQwM8//0zVqlUpVKgQixYtMselTYpp7dq1zJkzhzx58uDq6srKlSst\nGhNAnz59CAsL4/bt25QvX57PPvuM1NTUjJis3U5ZickW7bRv3z6WLVtG/fr18fHxAWDSpElcuXIl\nIy5rt1VWYrJ2W924cYNBgwZhMBgwGAwMGDCA1q1b2/TfXlZissV76u8edavYsp2yGld228osK0qF\nEELYB9n5SAghHIgkdSGEcCCS1IUQwoFIUhdCCAciSV0IIRyIJHUhhHAg/w8FViBNWVHF0AAAAABJ\nRU5ErkJggg==\n"
}
],
"prompt_number": 7
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Model averaging\n",
"---------------\n",
"\n",
"The following code generates a set of samples of the same size and fits a poynomial to each sample. Then the average model is calculated. All the models, including the average model, are plotted."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"degree = 4\n",
"n_samples = 20\n",
"n_models = 5\n",
"avg_y = np.zeros(n_samples)\n",
"for i in xrange(n_models):\n",
" (x,y) = sample(n_samples)\n",
" model = fit_polynomial(x, y, degree)\n",
" p_y = apply_polynomial(model, x)\n",
" avg_y = avg_y + p_y\n",
" pl.plot(x, p_y, 'k-')\n",
"avg_y = avg_y / n_models\n",
"pl.plot(x, avg_y, 'b--')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 8,
"text": [
"[<matplotlib.lines.Line2D at 0x1079cfbd0>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD9CAYAAABDaefJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdYVEf3B/BjiZpoUGSb9I6ggGIXBcQCijUaBA2i2HvF\nyquxgCV2o7HEmqiosQtqBCT2TrAbQRQURTD0vtzv7w/C/sTdhd1lkeJ8nofn1Xvv3jmbV87Ozp05\nUwMAiGEYhqkWalZ0AAzDMIz6sKTOMAxTjbCkzjAMU42wpM4wDFONsKTOMAxTjbCkzjAMU42oJakX\nFBRQy5YtqU+fPuq4HcMwDKMitST1DRs2kJWVFdWoUUMdt2MYhmFUVOak/vr1awoODqZRo0YRW8fE\nMAxTscqc1KdPn04//fQT1azJhucZhmEqWu2yvPjMmTMkEAioZcuWFB4eLvMaNiTDMAyjGlVGP8rU\nvb527RqdOnWKjIyMyNPTk8LCwmjYsGEyA6tMP4sWLarwGKpCTJU1LhYTi+lLiEtVZUrqAQEBFBcX\nRzExMRQYGEjOzs60b9++stySYRiGKQO1DoSzoRaGYZiKVaYx9Y85OjqSo6Ojum5XrpycnCo6BCmV\nMSaiyhkXi0kxLCbFVda4VFEDZRm8UaSBGjXKND7EMAzzJVI1d6qtp/6lCAoKIiKiLl260DfffFPB\n0TAMwxTHeuoKSk5OpgkTJlBERASJRCK6d+8ederUidzc3KhXr15kZGRU0SEyDFONqJo72YohBYSF\nhZGtrS3x+XyKiIig8PBwio2NpeHDh9Pt27epffv2ZGVlRb6+vnTx4kXKz8+v6JAZhvlCsZ56CXJy\ncmj+/Pl0+PBh2rlzJ7m4uMi8juM4unv3LgUFBVFQUBBFRUVRt27dqFevXtSzZ08SiUSfOXKGYao6\nVXMnS+py3L9/n4YOHUpNmzalrVu3kpaWlsKvTUhIoLNnz1JQUBCFhISQqakp9erVi9zc3Kh169as\npALDMKViSV1NOI6jtWvX0sqVK2nNmjXk5eVVpvn3+fn5dPXqVQoKCqLg4GBKTEyknj170sCBA6lv\n375qjJxhmOqEJXU1iI2NJW9vbxKLxfTbb7+RoaGh2tuIiYmhs2fP0oYNG8jd3Z2WLFnCFm0xDCOF\nPSgtAwC0f/9+at26Nbm4uFB4eHi5JHQiIiMjI5owYQJdvnyZTp06Rb6+vlXmQ49hmMrvi++pJycn\n0/jx4+n+/fu0f/9+atmy5Wdr+99//yUXFxdq164dbdy4kY21MwwjwXrqKggNDSUbGxsSCoV09+7d\nz5rQiYgaN25MISEhFBERQWPGjKGCgoLP2j7DMNXPF9lTz8nJoXnz5tGRI0do9+7d1L179wqNJyMj\ng/r06UM6Ojq0Z88eql2bLfRlmC8d66krKDIyklq3bk1v3ryh+/fvV3hCJyJq0KABBQUFUWJiInl6\nerLFSwzDqOyLSeocx9GqVauoW7duNGfOHDp06BA1bty4osOS+Oabb+jkyZOUm5tLgwYNotzc3IoO\niWGYKuiLGX7ZvHkz7dq1i44dO0YGBgYVHY5ceXl5NGTIEMrIyKBjx46xomEM84Vi89RLkJubS6am\npnTs2DFq06ZNhcaiCLFYTMOHD6c3b97Q6dOnqUGDBhUdEsMwnxkbUy/B3r17qVmzZlUioRMR1a5d\nm/bu3UsmJibk4uJCqampFR0SwzBVRLXvqefn55O5uTn9/vvvZG9vX2FxqILjOJo8eTLdunWLzp8/\nX6meATAMU75YT12O/fv3k5GRUZVL6ERENWvWpJ9//pkcHBzI2dmZEhMTKzokhmEquWrdUxeLxWRp\naUnbt2+nLl26VEgM6gCA/ve//9Hx48cpJCSEmjRpUtEhMQxTzth2djIcOnSIhEJhld9UtkaNGrRs\n2TKqV68eOTo6UmhoKOnp6VV0WAzDVELVNqlzHEf+/v60bt26alMF0c/Pj77++mtJYmdb6DEM86lq\nm9SPHj1KDRo0oB49elR0KGo1c+ZMSY89JCSEzM3NKzokhmEqkTIl9ZycHHJ0dKTc3FzKy8ujfv36\n0fLly9UVm8o4jqNly5bRsmXLqk0v/WMTJ06kevXqUZcuXejChQtkZWVV0SExDFNJlCmp16tXjy5e\nvEjffPMNicVi6tSpE125coU6deqkrvhUcvr0aapZsyb17t27QuMoTyNHjqSaNWtSv3796N69e/Tt\nt99WdEgMw1QCZZ7SWLSMPS8vjwoKCip8LjUAWrp0Kfn5+VXLXvrHRowYQV26dKHx48dX+KpdhmEq\nhzKPqXMcR3Z2dhQdHU3jx4+XORTw448/Sv7s5ORUrrNRzp07Rzk5OTRgwIBya6MyWb9+PbVp04b2\n7dtH3t7eFR0OwzAqCg8Pp/Dw8DLfR23z1FNTU8nFxYVWrFhRLGl/znnqAMje3p4mT55Mnp6en6XN\nyuDBgwfk7OxMV69eZQ9OGaaaqPAVpQ0bNiQ3Nze6c+eOum6ptLCwMPrw4QO5u7tXWAwVwdrampYs\nWUIeHh6sZC/DfOHKlNSTkpIoJSWFiIiys7PpwoULn31LuI8tXbqU5s+fT7Vq1aqwGCrKuHHjyMjI\niObMmVPRoTAMU4HKNKb+9u1b8vb2Jo7jiOM48vLyoq5du6orNqVcvnyZYmNjaciQIRXSfkWrUaMG\n/frrr9SyZUvq2rUr9enTp6JDYhimAlSb2i89evSg77//nkaPHl3ubVVmV69epYEDB9Ldu3dJR0en\nosNhGEZFX/QmGTdv3qTvv/+eoqKiqE6dOuXaVlXg7+9PFy5coNDQ0C9yKIphqoMKf1BakZYuXUpz\n5sxhCf0/c+fOpZo1a5K/v39Fh8IwzGdW5Xvq9+7doz59+lB0dDTVq1ev3NqpauLj48nOzo6OHDlC\nnTt3ruhwGIZR0hfbU1+2bBn5+vqyhP4JbW1t2rlzJ/3www/077//VnQ4DMN8JlW6p/7gwQPq3r07\nvXjxQlKugCluxowZFBMTQ8eOHav2ZRMYpjr5Invq/v7+NGPGDJbQS7B8+XKKjY2lLVu2VHQoDMN8\nBlW2p/706VNycHCg6OhoVqGwFM+fP6eOHTtSSEgI2draVnQ4DMMo4IvrqQcEBNCUKVNYQleAmZkZ\nrVu3jjw8PCgzM7Oiw2EYphxVyZ56dHQ0tWvXjqKjo6lhw4ZqvXd15u3tTbVr16adO3dWdCgMw5Ti\ni+qpL1++nCZMmMASupI2b95Mly9fpsDAwIoOhWGYclLleuqvXr0iOzs7ev78eYVvyFEV3bt3j1xd\nXenGjRtkbGxc0eEwDCPHF9NTX7lyJY0ePZoldBXZ2dnRggULyNPTk/Ly8io6HIZh1KxK9dTfvHlD\n1tbW9PTpUxIIBGq555cIAPXt25csLS1p1apVFR0OwzAyfBEFvaZNm0Y1a9aktWvXquV+VVlSUhI9\nfPiQ6tSpQ+3atVO6cFdSUhK1bNmSfv31V3JxcSmnKBmGUVW1T+oJCQlkaWlJjx49oiZNmqghsqoh\nPT2dHj16RA8fPiz2k5OTQ82bN6fU1FRKTEykvn370oABA8jZ2Znq1q2r0L3Dw8NpyJAhdO/ePRKJ\nROX8ThiGUUa1T+qzZ8+mrKws+vnnn9UQVeWTk5NDT58+lUreiYmJZGlpSc2bNy/2o6OjI1n2HxUV\nRSdOnKDjx4/To0ePyNXVlQYMGEA9e/YkDQ2NEttduHAh3bhxg86dO0c1a1a5RywMU21V66SelJRE\n5ubmFBkZSXp6emqKrOJwHEenT5+miIgISfJ+9eoVmZiYSCVvIyMjpYZW3r17R6dOnaITJ07QlStX\nqFOnTjRgwADq27cvCYVCqevFYjE5OjqSh4cHTZ48WZ1vk2GYMqjWSd3Pz4/ev39P27dvV1NUFeft\n27c0bNgwSk5OJldXV0nyNjc3V3s9+LS0NAoODqbjx4/T+fPnqXnz5jRgwAAaMGBAsemMz549I3t7\ne7pz5w4ZGhqqNQaGYVRTbZO6WCwmoVBIt27dIhMTEzVG9vmdPXuWfHx8aOzYseTn50e1a5dpi1il\n5ObmUmhoKJ04cYJOnjxJQqFQkuBtbW1p5cqVFBYWRufPn2fVHBmmEqi2Sf3mzZs0atQoevDggRqj\nKl9iMVF8PNH790SJiURv34pp795gun//Jp040YMcHR0rNL6CggK6fv26ZBweAK1cuZJWrFhBkyZN\nohEjRlRofAzDVOOkHhAQQImJibRu3To1RqW6x4+J7t4tTNi1axNNnSr7GhcXIj6fqEGDTHr0KJw0\nNcU0cWJXmj69gdT19+8TJScTOTgQfe5OMgC6cuUKDR8+nOzs7Oivv/6iyMjIL2qGEcNURtV2RWlI\nSAh17dq1osMgsZho2TKiLl2Izp0jev26MKnLYmVFFBdHNH36b/TkiSEtWfKSnj/vKzOhExXea8IE\nIgsLohUriN6+Lcc38okaNWpQ586dKSIigurWrUsA6Icffij3zcIZhiknKGdlaSIzMxP169dHamqq\nGiNSjZcX0K0b8Pp16dempaXBy8sLTZs2xd9//63Q/TkOuH4dGDkSaNQI6NsXePmyjEGrYOfOnahV\nqxZGjBgBjuM+fwAMwwBQPXeWqaceFxdHXbp0oWbNmlHz5s1p48aN6vmk+c/Vq1fJ1ta21LnWn8OK\nFUTnzxPp6JR83d27d8nOzo7q1q1Ld+7ckdqUAgBlZWVRRkZGseM1ahC1b0/066+Fvfx+/Yg0NdX9\nLkrn4+NDBw8epN9//5169uxJSUlJnz8IhmFUVqYx9Xfv3tG7d++oRYsWlJGRQa1ataITJ06QpaXl\n/zdQhjH1uXPnUp06dWjJkiWqhqh2WVlZFBcXR2lpacV+UlJS6Ny5c3Tp0iXq0KEDCQQCSktLo9TU\nVKlra9euTbVq1aKRI0fSwoULSUtLS+H28/MLf8p7B78pU6ZQaGgopaam0t69eyvFEBjDfElUzZ1l\nmlMnEokky8sbNGhAlpaWFB8fXyypl0VISEiF1HnhOCJZiyufPn1KLi4uVKdOHWrYsCFpaGhQw4YN\n6auvvqJbt26RWCymSZMmkaGhoeSchoaG1E+dOnUoMTGRFi9eTE2bNqV58+bRpEmTFJqnfvUq0cCB\nRIMHE40aRWRnVw7/AaiwZr2NjQ2NHz+ehg0bRsOGDaMlS5bQV199VT4NMgyjHuoa/4mJiYG+vj7S\n09OLHSciLFq0SPJz8eJFhe734cMHfPvtt8jNzVVXiKXKzAQmTADmz5c+d+PGDQiFQuzZs6fY8QsX\nLkBbWxvz5s1DXl6e0m0+fvwYbm5uMDExwdGjRxUax371Cli8GDAwALp3B968UbpZhYSGhkJPTw/R\n0dFwc3ND27ZtERUVVT6NMcwX7uLFi8VyparpWS1JPT09Ha1atcLx48elG1AxsD/++AOurq5lDU1h\nd+4AFhbA0KFAcnLxc+fOnQOPx8Pp06clx/Ly8jB37lxoa2sjJCSkzO3/+eefsLa2hoODA+7cuaPQ\na/LzC5O7UFh+D1VHjx6NsWPHguM4bNiwAXw+H7///nv5NMYwjESFJfW8vDz06NED69atk92AioGN\nHz8eP/30U1lCU4hYDPj7A3w+cPCg9PkDBw5AIBDgypUrkmMvXrxAu3bt0KtXL7x//16NsYixY8cO\nNGnSBMOGDUNcXJxCr3vwoHD2THlISUmBrq6u5BvW33//jaZNm8LLywtpaWnl0yjDMBWT1DmOg5eX\nF6ZNmya/ARUDMzc3R0REhKqhKWzpUsDZGYiNlT63YcMG6Orq4sGDB5JjJ06cAJ/Px9q1a1FQUFAu\nMaWlpWHBggVo3Lgx/ve//0kNaX1up06dgqmpKTIzMwEAGRkZGD16NExMTHDr1q0KjY1hqqsKSeqX\nL19GjRo1YGtrixYtWqBFixY4e/ZsmQOLjY0Fj8crt6T5scxM4NNmOI6Dn58fzMzMEBMTIzl+//59\n8Hg83Lx5s9zjAoBXr15h6NCh0NbWxs6dOyEWiz9Lu7J4enpi5syZxY4dOXIEAoEAK1as+Cz/XzHM\nl6RCx9RLbECFwHbv3g13d/dyiKZ0YrEYY8aMQatWrZCQkCA5npGRgaZNm2Lv3r2fPaabN2/C3t4e\ntra2CA0NVeg10dGAjw+QkqKeGN6/fw+hUCj1gfbq1St07twZ3bp1Q3x8vHoaYximYhYflZfyKg2Q\nnV3y+ZycHHJ3d6fo6Gi6ePFisX1QJ0+eTO3ataNhw4apPa7StG3bli5fvkx+fn40atQo6tu3Lz17\n9qzE1zRpQlS3LlHLlkQ3bpQ9Bj6fT+vXrycfHx/Kzc2VHNfX16ewsDDq3LkztWnThqKiosreGMMw\nqlPzh4sUZZvgOA4ikQjR0dFqjeO334BeveSfT01NhZOTE9zd3ZGTk1Ps3L59+2BhYVHhY9sAkJOT\ng59++gk8Hg+TJ09GUlJSidcfOwYIBEBAgPQwk7I4jkOfPn2waNEimee3b98OAwMDvKyI+gYMU82o\nmp4rXVJ/+PAhDA0N1RpDbGzh7BZ5z13fvXuHli1bYsKECVLj1k+fPgWPx0NkZKTa4uE4rszj4+/f\nv8e4ceNgamqKV69elXjtq1dA586FH2plnSXz+vVr8Hg83L9/X+b59evXw9TUlA3FMEwZqZrUP98u\nDQoKDQ2lbt26qe1+QOHKyylTiFq0kD7/4sUL6tGjB3l5edHChQuLbRCRnZ1N7u7utGzZMmrWrBlF\nRETQ5cuXKSEhgXJycig3N5dycnKU/nNubi7VqlWLTE1NydbWlmxtbcnGxoZsbW1JV1dXoU0q+Hw+\n/fLLL7R+/XpydHSk0NDQYrsZfUxfnygsrLBkcFlL++ro6NDy5cvJx8eHrl+/LrXRx9SpUykzM5O6\ndetG4eHhxOfzy9Ygw3yBNmzYoPJrK1099b59+9KQIUPIw8NDLe1v21ZYJOv6delSuZGRkdSrVy/y\n8/Oj8ePHFzuXn59PHh4eFBUVRXp6enT16lUSiUTk4OBA+vr6VLduXapXrx7Vq1dP8mdljonFYnr6\n9Cndv3+fIiMjJT/5+fmSBF+U7Js1a0Zff/213Pf4yy+/0PLly+nChQtkYWGhlv9uJQFA3bt3JxcX\nF/L19ZV5zYIFC+js2bMUFhZGjRo1KveYGKa64DiORCIRJSYmqlY3S31fFmRTpon8/Hw0bNiw2KyT\nsnj/HuDxgEePpM/99ddf4PP5OHz4MIDCseorV67A398fPXr0wNdff406depgzJgxOHz4MN69e6eW\nmErz7t07nD9/HqtWrcIPP/wAa2tr1KtXD5aWlvDw8EBAQACCgoLw+vXrYiUFdu3aBW1tbTx8+PCz\nxBkdHQ0tLS08e/ZM5nmO4zB16lS0b9+eLVJiGCXcu3cP5ubmKg+/VKqe+vXr12ncuHEUGRmptvZj\nYoiMjIofO3nyJI0aNYrmzp1LGRkZ9Ndff9Ht27fJ3NycHB0dycLCghYsWEDnzp2j1q1bqy0WVeXm\n5tLTp08pMjKyWM+e4zhydHSkESNGkKurKx0+fJhmzpxJZ8+epRayxppkOHqUKC2NaPhw5Ydm1q9f\nT8ePH6eLFy9STRkV0ADQ2LFj6fnz5xQcHFzitw2GYQqtWLGC4uPjadOmTVW/p7506VLMmDGjXOLI\nzc3F+fPn0bt3b3z11Vf4+uuv0a5dO8yePRtBQUFI+W9Cd25uLtq0aSO37EFlwXEc4uPjsWPHDnTo\n0AFNmjTBnDlzsGHDBggEAoUXSD14AFhaArNmKf8QVSwWo0OHDtiyZUuJ1wwdOhSurq5Ss4oYhpHW\npUsXnD59unrMfnF0dERQUJBa209OTsbKlSuho6MDfX19NGzYELt27ZI7PXHGjBno27dvldv15/Hj\nx/D19YVQKISlpSUaNGiA8+fPK/TaDx+Atm2BsWMLa+Eo2y6PxytxBk5+fj6+++47DBgwQKVKlgzz\npUhPT0eDBg2Qnp5e9ZN60dZ16poL/urVK8yYMQONGzfG0KFD4e/vj6ZNm+JNCXVqT506BX19fXz4\n8EEtMVSEvLw8nDx5Eh07dkSNGjXg4uKCS5culfohlZYGODoCQ4YAyubdZcuWwdXVtcQ2cnNz0bNn\nT3h6elZouQOGqczOnDkDJycnANVgnvr58+fRqVOnMrcXFvY3hgwZgsaNG2PmzJmIjY2FWCyGubl5\niSVyY2NjpaoxVnXHjh1D/fr1oa+vD1NTUwQEBOB1CZusZmUBHh7AkyfKtZOXl4cWLVqUWkIhKysL\nXbp0wciRI1mtGIaRYfLkyVi+fDlycnKqflL39fWVu1KxNBzHITg4GG3bjkTNmm/h779OMkYOAAcP\nHkTHjh3l9iTz8/Nhb2+P5cuXq9R+ZXbt2jXweDysWrUKY8aMgaamJnr16oUjR46odYz77t274PP5\npS46Sk9PR4cOHTB58uQqN8TFMOXNwsIC169fR+vWrat+Urezs8Ply5eVundOTg527doFKysrWFu3\ngoHBB2zfnl/smoKCAlhZWUlVj/zY/Pnz0aNHD7m9x4yMDJw4cQKjRo2CtrY2+Hw+bG1t0atXL4wa\nNQqLFi3C9u3bcebMGURERCAhIaFS9UTv3LkDoVCIQ4cOITMzE/v27YOTkxN4PB6mTp2qttWyCxYs\nQP/+/UtN1snJybCzs8PcuXNZYmeY/7x8+RJ8Ph9Tp05F//79q/aUxg8fPpCRkRElJSUptE9ncnIy\nbd26lTZt2kQ2NjY0a9YsunSpK0VE1KBTp4pPzfvjjz9o1apVdPPmTZkrNS9cuEDDhw+ne/fukVAo\nlBx/9eoVBQUF0ZkzZ+jKlSvUtm1b6tOnD/Xq1Ys0NDTozZs39ObNG4qPjy/2v0V/Tk9PJ5FIRDo6\nOqStrU06OjrF/mxkZESGhoYK/3csq/v375OrqyutWLFCUpQsOjqa9uzZQ7t376ZWrVrRqlWryrR4\nKTc3l1q1akV+fn6lLh5LSkoiJycn8vDwID8/P5XbZJjqYseOHbR//356+fIlRUREUOPGjavulMYj\nR46gV0nVtv7z4sULTJkyBZqamvD29pb0MG/dKixa9ek3/4KCAtjY2BTbhu5jb9++RZMmTRAaGgqx\nWIxr165h3rx5sLa2Bp/Ph7e3N44cOVJsKEdR2dnZePHiBS5fvoxDhw5h7dq1mDlzJjw9PeHg4ACB\nQICOHTti7969yMrKUvr+qnj8+DF0dHSwbds2qVhXrVoFLS0tmUXCDh0CFNyECbdu3YJQKFRosVZ8\nfDzMzMywdu1ahd8Dw1RXvXr1goaGBq5evQqgij8oHTt2LNasWSP3/J07d+Du7g4tLS3MmTOn2MM+\njgPat5e9Fd2JEyfQsmVLmV/xxWIxHBwcMGjQIAwbNgw8Hg82NjaYP38+rl27Vu4zNPLz83H8+HG4\nurpCS0sLU6dOxSNZS1/V7Pnz5zAwMMDGjRulzr1//x4TJ04Ej8fD6tWrJWPuq1cDhoaAontOz5kz\nB4MGDVLo2levXsHAwABbt25V+D0wTHWTnZ2N2rVrY+7cuZJjVTqpm5qayhzXTU1Nxbhx46CtrY21\na9fKXW7+/r30whmO42BnZ4djx44VO/78+XOsW7cOxsbGqFWrFlxdXbF58+YKLRcbExOD+fPnQyQS\noVOnTvjtt9+QnZ1dbu29fPkSxsbGWLlypczzjx8/Ru/evWFsbIwjR46A4zhs3Qro6ACKVCHIzs5G\n06ZNJSUYShMVFQUdHR3s27dPmbfBMNXGiBEj0KBBA8mzuKioqKqb1IseDnz6YPHUqVPQ1dXF6NGj\nkZycrHS7QUFBsLa2ltz35MmTsLCwgEgkgpubGxo1aoR//vlH6fuWp7y8PBw9ehQ9evQAj8fD9OnT\n8UTZ+YUKev36NSwsLLB48WK5DytDQkJga2sLe3t73Lx5E/v3A0IhcPt26fe/fv06RCKRwhtzP3r0\nCCKRCEeOHFHmbTBMlRcWFoYGDRpg/PjxAAo7pBYWFlU3qe/cuRMeHh6SvyckJGDw4MEwMTFBWFiY\nSm1yHId27drh0KFDKCgowOLFi6Grq4uwsDAkJCRAV1cXwcHBKt37c4mOjsbcuXMhFArh6OiIAwcO\nqH2Z/du3b9GsWbMSZ6GIxWLs3LkT2traGDJkCLZvT0DTpootUJo5cyYGDx6scDwREREQCAQ4d+6c\nwq9hmKrs/fv30NHRgaWlJS5cuAAAuHjxIqysrKpuUvf09MSOHTvAcRz27dsHgUAAX19fyc71qjh/\n/jwsLS2RkpKC7777Dh06dMDbt29RUFAANzc3+Pr6qnzvzy03NxeHDx9G165dwefzMWvWLLV+w0hM\nTIStrW2pc/TT09OxcOHC/xZ1+SE1NbXUe2dlZcHc3BxHjx5VOJ5Lly5BIBCofecrhqlsCgoK0LNn\nT0ybNg3ffvutZMh18ODB2LhxY9VM6hzHQSgU4vLly3BxcYGtrS3u3LlT6j1jY+UXn+I4Dvb29li3\nbh2sra0xcuTIjx74rUb79u2rbP2R58+fw9fXF3w+H87OzggMDERubm6Z7/v69Wvo6uoqlHzj4uLg\n7e0NkUiEX375Bfn5+SVef+XKFTRp0qTUbfc+tnHjRtja2pbpg51hKrvVq1ejQ4cOCAwMhKurK4DC\nkYqGDRsiOTm5aib1v//+G1paWtDS0kJAQIBCyTYlBdDXB65fl30+LCwMOjo6EAgE+PnnnyXDChER\nEeDz+YiJiVHlbVQqOTk5CAwMhJOTE0xMTNQylHT37l3weDyFPlSLrndycoKVlRWCg4NLXEQ0bdo0\nDB06VOFYOI7D0KFDMXToULY4iamWbt68CYFAgJcvX2LMmDGSqrArVqzAiBEjAFTB2S+PHj2CoaEh\nRCIRnj59qvD9fHyAMWNkn+M4DqamptDQ0MDFixeLnevduzc2bdqkcDtVxdmzZ2FiYoIBAwaUuldp\naY4dOwYdHR3EKTgpneM4nDhxAmZmZmjffozcfUszMzNhYmKCkydPKhxLZmYmbG1tZU69ZJiqLCUl\nBUZGRjh69Cg4joOBgQEePXqEgoICGBsb48aNGwAqMKmPGDECAoEAzZs3l93AJ4Hl5uZi8eLF0NLS\nQvPmzXF8d6DoAAAgAElEQVRQ1gRzOU6fLpwvLWtmY3Z2Nnr27Ik6depIjTnfuXMH2tra5TpNsCJl\nZ2dL/psuX768TEMyK1euRIsWLZSqlpmcnAseLxn16y/BihUrZM7xDw8Ph7a2Nv7991+F7xsdHQ2B\nQKB0+QiGqaw4joO7uzsmTJgAAHj27Bl0dXXBcRz+/PNPtGjRQvLttMKS+qVLl3Dv3j2FkvrNmzfR\nvHlzuLm5ITo6GhoaGkhMTFSonaQkQFsbCA+XPvfmzRu0a9cOQqEQP//8s9T5fv36Yf369Yq9oSos\nOjoabm5uaNq0aZlmDvn4+KBfv35KLcCKjweMjfNgbLwJjo6OMr81TJo0Cd7e3krFExwcDG1t7VIL\nhTFMVbB9+3bY2NhIOpgbN26Ej48PAOC7777DL7/8Irm2QodfYmJiSkzqGRkZmD59OoRCIQ4cOACO\n43D16lW0aNFC4Tb8/ICpU6WP37hxAzo6Ohg7diz09fWleqkRERFo0qTJZ1uKX9GKhkQMDAzg6emp\nUjLMzc2Fo6Oj0rOEYmMBQ0MOAwYEg8/nS30LS09Ph5GREc6cOaPUfZcsWQJ7e3u1PBRmmIry4MED\n8Hi8YmtPevfujUOHDuHNmzdo1KhRsQWWlTqpN2rUCDY2NvD19ZWMdS9evBizZs1SuI38fODT0ZPd\nu3eDz+fj9OnT6NmzZ7FPuSIDBgz4ImuLZGZmYv78+eDxeFi3bl2ps1Q+lZSUBDMzM+zYsUOp1z1/\nXrjyNCAgGhYWFvjhhx+K1c4JDQ2Frq6uUgvKCgoK0KdPH0yaNEmpWBimssjMzISVlRX27NkjOZab\nmwsNDQ0kJSVh6dKl6N27NxYtWiT5qdRJXdbsDAcHhxLL4ZYkPz8fU6dOhZmZGR4/fozbt29DR0dH\nanFOZGQkRCLRFz017smTJ+jatStsbGyU3gDk2bNnEAgESg/lPHwIPHpUWLJ43LhxMDQ0LDYuPm7c\nOMlXTkUlJyfD1NSUlRJgqqRRo0bhhx9+KDab6+LFi2jbti3EYjH09fVx7969Yq+p1En9UxkZGahf\nvz4yMjKUbispKQnOzs5wdXWV9Pb69u0rc5bEoEGD8NNPPyndRnXDcRwCAwOho6OD4cOHIyEhQeHX\nhoaGQiAQ4NmzZyq3f+rUKQiFQixYsAB5eXlIS0uDgYGB0itHi76+fvqPn2Eqs4MHD8LMzEyqdtXc\nuXPh5+eHM2fOoE2bNgAKi+YVTRKrUkn97NmzcHBwULqd+/fvw9jYGLNnz5Y8xJM3Zv7gwQMIhUKV\nPjiqq7S0NMyYMQN8Ph9btmxR+EHojh07YGpqqtQCok+9ffsWrq6uaNOmDf755x/8+eef0NPTU2hl\n6scCAwNhZGRUpfeRZb4cUVFRcjsiRRsD9enTBzt37gTHAT17AitWAH/88UfFJXUPDw80adIEderU\nga6uLnbt2lW8ARmBzZw5E4sXL1aqnaNHj4LP52P//v3Fjg8cOFBm2V53d3e5VQi/dPfv30enTp3Q\nunVr3Lp1S6HXzJo1C46OjmV6WMlxHDZt2gQtLS1s374dI0eOxBh5iw5KMGPGDLi4uLANrJlKLTc3\nF61bt5Y5ilC0cjQ6OhqNGzdGRkYGjh0DLC2B8+cvQiAQVK3FRy1atJAUgi/J3LlAWFjh7Ad9fX2p\n1Y7yeuOPHj0Cn89Xaq71l4bjOOzduxcikQhjx44tdSMQsViMvn37wsfHR6VVnlu2/H/Z3ocPH8LW\n1hZubm7Q1taWFDJSVH5+PpycnODn56d0HAzzucyYMQP9+vWT+fuyf/9+9O/fHwsXLsTEiRORkQHo\n6QEHD76FUCjEhQsXqk5Sf//+PTQ0NEotCZCcDDRqBPz66xmYmprK3Eln8ODBMnvjnp6eCAgIKFvg\nX4jk5GSMHj262Eo2edLT09GiRQusWrVK6Xb27y+cFfP8eeHfc3JyMGvWLGhpaUEgEMitlS9PQkIC\n9PT0lFqlyjCfy5kzZ6Cvry93mHDYsGHYtGkTtLW1cf/+fcyZAwwenA9bW1tJyYAqk9QPHTqE3r17\nl/q6DRuA/v2zIBQKcV1GoZcnT56Az+dLJYMnT56Ax+MpnSS+dEePHoVAIEBAQECJwxpxcXHQ0dHB\n8ePHlW5j27bCFcGxsf9/LDQ0FN988w2sra2VXvF748YN8Pn8Mj3EZRh1e/36taRQoSwcx0kK4nXs\n2BG5uUDHjhz69BkFb2/vil9RWmoDnwT2cfEaeTgOsLDg0KGDLxYsWCDzmh9++AHLli2TeXzp0qWq\nB/wFi42NhYODA5ydnfHmzRu5192+fRs8Hg93795Vuo01awBzc+DjL14vXrzA119/DSMjI5k7YJVk\n27ZtsLKyYkNtTKUgFovh6OgoMzcViYyMhImJCVxcXCRTdP39A9CmTZtiHZsqk9SNjY3x4MGDEl8T\nGgro6HxAixYtZT6Y++eff6ClpSU1Dvzs2TPweDyVNopmConFYixZsgRCoRCnTp2Se90ff/wBXV3d\nYvvFKurHH4H/Ko1KnD59Gnw+HzweD1u2bFF43L6orIG7uzur6MhUuB9//BHOzs4lftv96aefMHTo\nUGhpaSErKwtnzpyBtra21O9SlUjqL168gFAoLPWXb/r0D6hff6bcjZhHjBiBRYsWSR0fNmyY0rNq\nGNmuXLkCAwMDTJo0Se6wSEBAAOzs7JSeNspxgKySP15eXvDy8kKzZs0wZswYhWfaZGdno3Xr1li9\nerVScTCMOoWHh0MkEpVamqNbt24YOHAgpk+fjqdPn4LP58ucOFIlkvqOHTswZMiQEq8Xi8Wwt7fH\nmjWyl/a/ePECjRs3lqr29/z5c2hpaam0nykjW3JyMr7//ntYW1vL/IDlOA7e3t7o37+/1B6zqvjw\n4QO0tbURHByMfv36wd7eXuYDcllevXoFkUikciEzhimLxMRE6OrqlrpKPjMzE/Xr14dAIMCtW7dg\nYWGBX3/9Vea1VSKpe3h4YOfOnSVev2LFCnTp0kVukhg9erTMcfYRI0Zg4cKFZQuWkcJxHH799Vfw\neDxs27ZN6ltWTk4OOnfujDlz5qilvaCgIOjp6SEhIQELFy6Enp6ewht3hISEQCQSIfbjJ7EMU844\njoObmxtmz55d6rVnz55F06ZN0a6dO1xcvsPEiRPlXlvpk3pBQQH4fH6JGzn8/fff4PF4cq959eoV\nNDU1pcr1Fk3gV6ZWN6OcJ0+eoEWLFvjuu++kpmklJibCxMREauGZsoo+L3x9fdGzZ08UFBTgjz/+\nAI/Hk1p0Js/KlSvRtm1btW/SzTDyrF27Fu3atVNo57bp06fDyMgEmppPYWHhX+JrKn1Sj4yMhKmp\nqdzrcnJyYG1tXayK2acmTJgg89Nw1KhRbCHKZ5CTk4Np06ZBT08Pf/31V7FzRVNMFVlUJsvvvwPD\nhgEFBUBeXh46dOggWYMQGRkJIyOjYuUh5OE4DgMHDsTo0aNVioNhlHH79m3w+Xy8ePFCoetNTExQ\nt+4E1KlzB+/evS/x2kqf1NesWYNx48bJvc7X1xcDBgyQ+xD19evX0NTUlCpGFRMTg8aNG5epLgmj\nnDNnzkAkEmHRokXFSvoGBQXJfIqviMxMoFMnYMKEwh77q1evIBAIJJUlExMT0aVLF/Ts2bPU5yZp\naWmwsLBQuHfPMKpITU2FiYkJDh8+rND1cXFxqFNHG0TvEBhY+haelT6p9+rVC0eOHJF5zV9//QUe\nrzOmTJFfInfq1KmYPn261PExY8Zg3rx56gmWUVh8fDy6desGe3t7vHz5UnI8IEB6vq2iUlKAVq0K\ny0MAhdUd9fT0JB/YeXl5mDJlCszNzYttNCDLvXv3lOpBMYwyOI6Dp6cnxo4dq/BrVq9eDaLtcHCI\nUOj6Sp3Uc3Nz8e2338rsTaempsLQ0BC9e7+AvBGUt2/fQlNTU2qqkLwxdubzKCgowKpVqyAQCCQf\n2EV7MH68Mk4ZiYmAlRWwfHnh32fOnAk3N7diD8537twJPp9f6g5Ka9asQYcOHZTeIIRhSrNz5040\nb95c4R3V8vLyUL++HWrWjIeiy2gqdVK/fPky7OzsZJ738fGBt/dEaGoWXz7+sZkzZ2Ly5MlSx8eP\nH6/QE2emfN26dQsmJiYYPXo0srKykJGRAVtbW5X3hX3zBnB2LtxgPC8vD+3bt5eqi3/t2jVoa2tj\n+fLlcj88CgoK0KNHDzYrilGrR48egcfjyV1HI8vEiRNRo0YNrFq1W+HXVOqkvmjRIpnJ98SJEzA2\nNsaGDdno31/263NyctCwYUOpaWpxcXHQ1NTE+/clP2xgPo+0tDQMHjwYLVq0QFRUFGJiYiAUChES\nElLme798+RICgQDXrl0rdvz169do06YNBg8eLHd3q7dv30IkEuHSpUtljoNhsrKy0Lx5c7lzy2XZ\nuXMnRCIRateurdAMmSKVOql36tQJ58+fL3Y8ISHhv1+2y7CxAf78U/brQ0JC0L59e6njkyZNUmqP\nU6b8FdVL5/P5OH78OMLCwiAUCtUyrn3y5EmZVe+ys7Ph5eWFli1byp0KGxQUBH19fTbllSmzcePG\nwdPTU+GhxevXr4PP58PW1hZt27ZVqq1KndTr169frCfFcRz69euHOXPm4J9/gObNC6eyyTJ9+nQs\nWbKk2LE3b95AU1NT4dWGzOd148YN6OvrY9asWVi7di1sbGzUsgPVjBkz0KdPH6lfKI7jsGbNGohE\nIqmplkWmTJmCQYMGsfowjMoOHz4MExMThXfrevPmDXR0dPDHH3+gdu3a2Lt3r1LtVeqk7uTkVOzY\nrl27YGNjI1kgUtJzLAsLC6kVhVOmTJE5E4apPJKSkuDq6opOnTrB3d0d33//fZkSKscB0dG5aNu2\nrcydrgDg/PnzEAgE+OWXX6TOZWdnw9rautQVzQwjy4sXL8Dn83H79m2Frs/Ozka7du2wZEkAtmzZ\nglq1askdIpSnUif1j0vhxsTEgMfjKVRiNSoqCkKhsNjMh/j4eJkzYZjKp6CgAEuWLIFIJELTpk3h\n7++v8r2ePAEEAiA4+DUEAoHMGvtAYQ0gS0tLjBs3Tmr88uHDh+DxeHj6tPQ5wgxTJC8vD+3atcPa\ntbLrUX2K4zgMHz4cAwd6wNaWg4FBT7Ro0ULpdit1Ui/6BRSLxXBwcFB479BNmzZh+PDhxY5Nnz4d\nU6ZMUXucTPm5cOEC+Hw+NDQ0SiznW5pDh4AmTYDNm/+EgYGB3F1lUlNT4ebmhu7du0t9Vd6yZQvs\n7OzKtNcq82WZPXs23NzcFP6muWHDBtjY2GDp0lx06pSCb76pjxUrVijdbqVO6kXzhFevXo3OnTsr\nvGFwz549i63WevfuHTQ1NUvcwIGpnOLi4tC8eXPUqVOn1G3zSrJjB2BgAPj4/Ii+ffvK/UXLz8/H\nhAkTYG1tXWzmFMdx6Nu3L3x9fVWOgflynDt3Drq6ugqvhTl37hyEQiGuXImFlhbg6ekHTU1N3L9/\nX+m2K3VSBwo3iebxeArPhMjMzESDBg2KLQmfOXMmJk2aVC5xMuUvLy8P3bt3x1dffYWLFy+qfJ81\nawAzMw4tWvQocRctjuOwevVq6OrqIiLi/1fxJSYmQkdHR+kNr5kvS3x8PEQiEcLDwxW6/uLFi+Dx\neLhy5QoGDAAWLCicji0QCFR6nlSpk3pubi5sbW2Lze1csgQoqURIUFAQHBwcJH9PSEiApqYm4uLi\nyjNc5jNwcXFBnTp1sHnzZpUfnm7aBFy58gp8Ph83b94s8dojR46Az+cjODhYciwkJAQ6OjpsnQMj\nk1gshrOzs8zNeGS5cuUK+Hw+wsLCEBQEmJoCmzfvhK2tLby9vVWKoVIn9Xnz5hWbihYVBfB4QEkr\nbCdOnFhs7H327NkYP358eYfLfAZ5eXlo06YN+Hw+hg4dWqbpjseOHYOhoWGpc9CvXbsGkUiErVu3\nSo7Nnj1b5hRJhlm2bBkcHR0VGiq+efMm+Hy+ZC1ObCxw+zZgZ2eH9u3bq1xYrsKS+tmzZ2FhYQFT\nU1OZDwOICCKRqNiccl9foKR1QxzHwdDQULKXaWJiIjQ1NUusxc5ULQkJCdDT04OjoyOsrKxKLdBV\nkilTpqB///6lJufnz5/DzMwMs2fPRkFBAXJzc9G6dWv8/PPPKrfNVD+XL1+GUChUaFTg3r17EAgE\nOH36dLHjt2/fhqGhITQ0NFT+NlghSV0sFsPExAQxMTHIy8uDra0tHj9+LBXY8ePHJX/PyirspUdF\nyb/v48ePoaenJ/klnTdvnlLV0Jiq4e7du+DxeFi0aBF4PB4CAwNVuk9OTg5atWqFDRs2lHptUlIS\n7O3t4e7ujuzsbPzzzz/g8XilbobOfBk+fPgAfX19qSQty4MHDyASiXD06FGpcyNHjsSYMWPk1rxS\nRIUk9WvXrsHFxUXy9+XLl2N5UXk9OYHt3Su9k/ynVq9eLUnimZmZ0NTURExMTFlCZSqpAwcOwMjI\nCGFhYTAxMcHkyZNVmm4YFRWNb78diRs3bpV6bXZ2Ntzd3WFvb4/ExETs3r1bqYp7TPXEcRz69++v\n0MLGJ0+eoEmTJjh48KDUuZSUFDRq1AjTpk0rU1lwVZN6bSqDN2/ekJ6enuTvurq6dPPmTanrfvzx\nR8mfz5xxokWLnEq8b3BwME2dOpWIiE6dOkVt27YlQ0PDsoTKVFKenp4UERFBy5Ytoxs3btDo0aOp\nU6dOFBgYSMbGxgrfR0fHmHR0VlCPHkH08qUZaWo2knttvXr16ODBgzR//nzq2LEjBQUF0blz52j2\n7Nm0adMmdbwtpgravHkzxcXFUWBgYInXRUVFUbdu3Wj58uXk4eFBeXlEtWoV/hAR/f7779S9e3e6\nevUqrVq1SuH2w8PDKTw8vAzv4D8qf4wA+OOPPzBq1CjJ33/77TepKYefNpGRAZT07CE1NRUNGjRA\neno6gMK56r///ntZwmQqObFYDBcXF0ybNg0cx2H9+vXg8/lyN1WRJyUF4PNj0bTpYYUffm7duhVC\noRDnzp2DgYGBQl+7meqnqPDW8+fPS7zu5cuXMDAwwLZt2yTHJkwAli0r/DPHcbC2tsaxY8egoaFR\npkVuqqbnMiX169evFxt+CQgIkHpYqmxgx44dQ/fu3QEUlk1t1KiRWopBMZXbv//+C1NTU0nRo9u3\nb8PExATjx49Xahel2Ngc1K37Av36XVH4NcHBweDxeFiyZAmEQiErQfGFKSq8Vdpq57i4OBgbG2Pj\nxo2SYwcOcDA0FOPPP29i3759GD58OHg8Hnbs2IHevXuXKa4KSer5+fkwNjZGTEyMZC66rAelyhg1\napRkQcnatWtVnuPJVD1FtVmK5p2npKTA3d0dNjY2StVruXw5BjVrxsDX96HCr4mIiICOjg66d++O\nbt26Fas3xFRfRYW3lhV1tWXgOA4RERHQ1dWFu7s75s+fD3d3d1hZDUCNGolo0KAz2rRpAw8PD2hr\na6NDhw6oW7cuunbtqtJ+vUUqJKkDhb0cc3NzmJiYICAgoEyBcRwHbW1t/PPPPwCAli1bqmWTBabq\nOHXqFLS1tSUPxjmOw7Zt28Dj8bBv3z6F73PkyE00bmwl2bhaEbGxsbC2toZIJFK4PhFTdRUV3ioq\nyfz27VtcvHgRO3bswJw5czBw4EDY2trim2++Qa1ataCnpwcvLy8sXrwYu3YFwtQ0E2vW/P8owrZt\n29CxY0eIxWIIhUJ4e3tDU1MTI0aMkOrsKqLCknqpDSgRWEREBExNTQEUThfS1dVVuE4MU31s2LAB\nlpaWxUpEREZGomnTphg+fLjCw3Hnzp0Dn8/H3bt3FW47NTUVnTp1Qp06ddhuSdVcUeGty5cv47vv\nvoOmpiY6d+6MESNGwN/fH4cOHUJYWBiaN28Ov082UF61ChgypLAkNAC8f/8efD4fkZGRePToEQwN\nDcFxHJKSkrB48WIIBAL0799fbnVRWSp1Ur95EwgKKv1af39/SQXG2bNnY86cOeUcHVNZTZ06Fc7O\nzsUeNKWnp8Pb2xuWlpYKF0g6evQoRCKRUvtJ5uXlwcnJCXXr1pV8a2Sql9DQUGhqasLJyQna2tpY\nt26dVGchJSUFrVq1wqxZs6QevOfnAx+XR/fx8cG0adMAFA4bf7quJjMzE5s2bYKhoSEcHBwQFBRU\n6sP8Sp3UBwwAPlqdLZe9vT3OnTsHsVgMbW1tPHyo+JgoU72IxWL069cP3t7eUv/49+zZAx6Ph+3b\ntys0y2Xfvn3Q1dVFVFS0wu1zHIeWLVuifv36KlXYYyonjuPw22+/4auvvkKTJk2wdetWyWY9H0tL\nS0OHDh0wefLkUv+NXb16FTo6OpIyz66urjIXJAGFzyH3798PW1tbWFtb47fffpO7b2mlTuqamsB/\nMxTl+vDhA7799ltkZ2fjwoULZVqJxVQPGRkZaN26tdR2hkDh4g9ra2t4eHgotL3Yli1boKm5CjNm\npCjcflpaGoRCITQ0NKT22GWqFo7jcOrUKbRp0wZ169bF0KFD5SbTjIwMODg4YMyYMaUm9Pz8fNjY\n2EgWIWVnZ+Pbb78tNnQoL56zZ8/CyckJ+vr62LBhg9Q3hUqd1BWplnvw4EHJFCAvLy+sX7++nCNj\nqoK3b9/C0NAQv/32m9S5rKwsjB07FqamplJbHsryv//9jDp1nmHGjAwoOI0dN2/eRKNGjSTT1Jiq\nRSwWIzAwEDY2NrC1tUX79u1lfvsrkpWVha5du8Lb21uhGVDr16+Hs7Oz5H7nz59Hx44dlYrx5s2b\n+O6778Dn87Fo0SJJ7fZKndQVGc708vLCli1bkJ6ejoYNGyIhIaG8Q2OqiIcPH4LP58uta33o0CHw\n+Xxs2LCh1J7V9OnLUa/eM8yYkaVwYvf390fbtm1hYmKCuXPnsumOVUBeXh52794Nc3NztG/fHmfO\nnMHSpUvRrl07mcMtQGENoV69esHDw0Nqgsbp08Cn+9y/efMGWlpaxYrReXp6qtwhffr0KUaNGgVN\nTU1Mnjy5cif10hQUFIDP5+Ply5fYt28f3NzcyjsspooJCQmBQCCQW80xKioKrVq1Qv/+/eVucwcU\nfu0dM2Y+vvnmH0yfnqtQYheLxXB0dMSCBQvQsWNHDB48WKkFUcznk52djS1btsDAwADOzs4IDQ2V\nDL3o6OjI3TUtKysL/fr1w3fffSc1LHPvXmERwk+XSnh6ehar7ZKQkIBGjRqVOvRSmvj4eMyePbtq\nJ/UbN26gWbNmAIDu3burXK2Pqd527doFY2Njud/icnJyMG3aNBgYGODatWty78NxHIYOnQKh8AIS\nExVLzrGxseDz+bh8+TLc3d3RsWNHhbc4Y8pfRkYG1qxZA21tbbi5uRX7///x48fg8/lypxO+evUK\ndnZ2GDJkiNSy/pQUwMQEOHCg+GtCQkJgYGCAzI+mwAQEBGDkyJFqe09VOqkvXLgQvr6+eP36NTQ1\nNVm1PEYuPz8/tG/fvsR/IydPnoRAIMCCBQvkftUWi8Xw8PBA79695T4w+9Thw4dhamqK1NRUzJ07\nF6ampmzKYwXLzc3FqlWrwOfzMWjQINy7d6/Y+eTkZJiZmWHXrl0yXx8eHg6RSITVq1dLDd1xHDBo\nEPDp3jy5ubmwsLDAiRMnJMfEYjEMDAwUerajqCqd1Fu3bo2LFy9i1apVav2kY6ofjuMwZMgQDBw4\nsMSx7fj4ePTv3x9WVlZyN7rOy8tDnz59MHjwYIUXuY0YMQIjRowAAGzfvh1CoRCXL19W/o0wZXbh\nwgVYWFigV69eMtchiMViuLq6Sta+fIzjOGzatAlCoVDuXrUbNwJ2dsCnI20BAQHo3bt3sQ+B06dP\no23btmV7Q5+oskn93bt3aNiwIXJzc9G8eXP89ddf5R0SU8Xl5OSgc+fOmFXS9lko/MUNDAyEUCjE\nrFmzZPbus7Oz4ezsDB8fH4UegKanp8PU1BSHDx8GUDjbgc/n48Cn38+ZchMXF4fvv/8ehoaGOHny\npNyH43PmzEGXLl2kvollZ2djxIgRsLa2RnS0/LULy5dLb+bz8uVLaGlp4cWLF8WO9+rVC7t371bp\n/chTZZP6nj17MHDgQERERMDAwIDNLGAU8uHDB5ibm2PLli2lXvv+/XsMHjwYZmZmMnvV6enp/y00\nmYp9+ziU9k/w1q1b4PP5ku0V79+/D319fSxbtoztd1qOcnNzsWLFCmhpaWHhwoUlDsEVbb7y6XOP\n169fo23btvj+++9Vqv7av39/LF26tNixFy9egMfjqX3YuMomdXd3d+zcuRPTp0+Xqq/AMCWJioqC\nSCRCkCI1KFBY1llbWxtTpkyR+oVOTk6GtXV76Om9xOjRKDWxBwQEwMHBQTJs8+bNG7Rs2RI+Pj4K\nj9Ezivt4qKW0muf37t0Dj8dDZGRkseNXrlyBtrY2VqxYodKH75kzZ2BmZib1nGbOnDmYMWOG0vcr\nTZVM6vn5+dDU1ERsbCxEIhGePXtW3uEw1cy1a9fA5/MRERGh0PUfPnyAl5cXjI2NERYWVuzc+/fv\nYWZmByOjWPj4lJzYi6Y5+vv7S46lp6ejd+/e6Nq1a5mntTGFFB1qKZKQkAADAwPJ8FiRrVu3gs/n\nIzg4WKU4srKyYGRkJLWyOCcnB3w+v1wemFfJpH7p0iXY2dkhODgY7dq1K+9QmGrq8OHD0NXVVWj3\n9yJnzpyBrq4uxo0bV6zMwOvXr2Fg0AxmZq/h7V3yLl1F0xw/fhArFosxadIkNGvWDC9fvlTl7TBQ\nbqilSF5eHhwcHLBgwYJi9xkzZgysrKxKTbwlfV7873//w/fffy91/Pfff5ds6qNuVTKpz5kzB35+\nfvD09MTPP/9c3qEw1diqVatgY2OjUB2YIikpKRg1ahT09fVx7tw5yfHo6Ghoa5vB0jIeMsrOFHPk\nyBGYmJggLS2t2PH169dDW1sbt2/fVup9MMWHWqI+fVJZggkTJqB3796S53Lx8fHo2LEj+vfvL/X/\nz5U1XYIAACAASURBVKeio4E2bQBZX7D++ecfaGlpyew0dOzYEceOHVM4RmVUyaRubW2NCxcuoGHD\nhmwhB1MmHMdh7NixcHFxUXpM+88//4SBgQFGjBiBf//9F0DhghWh0BDbt8uutvcxHx8fmTt0HT9+\nHDwer9h8ZkY+ZYdaPrZ9+3Y0bdoUKSmFBdtu3LgBXV1dLFmypNTJF//8A+jpAbKeuXMchx49emD1\n6tVS5yIjI6Grq4v8/HyF41RGlUvqsbGx0NLSwq+//op+/fqVdxjMFyA/Px+urq4KVdf7VFpaGiZO\nnFhsr8qIiAgYGhpi7NixJc6USE9Ph5mZmcyV0Ldu3YK2tjZWrlzJZsbIocpQy8euXLkCPp8veSa3\na9cu8Pl8nDx5stTXPn4M6OgAv/4q+/yRI0fQrFkzmR2FcePGYfHixUrFqowql9S3bduGIUOGwMnJ\nSW7tYYZRVlpaGmxtbVXeji48PBwmJiYYMmQIkpKSkJKSAi8vL5ibm5c4lHL79m1J/aJPxcbGok2b\nNhg0aFCpwwBfGlWHWorcvn0bAoEAZ8+eRV5eHiZOnAhzc3O5NYI+9uAB0KQJ8N9e51LS0tKgq6sr\ncwes1NRUNGrUSG4tGXWockm9X79+WL9+PRo3bix3KTfDqCIuLg76+vrYvHmzSq/PzMzE9OnTIRKJ\ncODAARQUFCAwMBB8Ph+LF/vjwIECmQ/VVqxYgc6dO8tcnZqdnY1Ro0bByspKqU20q6sHDx6gT58+\nKg21FLl79y4EAgFOnDiBhIQEODg4wM3NTTIEU5qdO4H9++WfnzVrFoYNGybz3ObNmzFo0CClY1ZG\nlUrqOTk50NDQwIIFCzBu3LjyDoH5Ar148QJGRkZYs2aNyve4du0aWrdujZYtWyI4OBivXr2CvX0/\n1K//DIMGpeHTvkhBQQG6dOkitTjlY9u3bwefz/9ix9lfvnwJb29vCAQCrF27VuVql/fu3YNAIMCx\nY8dw/fp16Ovrw8/PT22LFx88eAA+n493n9bbReE4e7NmzaSmxKpblUrqFy5cQLt27WBhYYGrV6+W\ndwjMFyo2NhZmZmZYtmyZyvfgOA5Hjx6FpaUlOnfujL/++gv+/utQp85pmJu/Q0JC8R5mXFwcBAJB\niRsM37hxA3p6evDz8/tiNlZPSkrCjBkz0LhxY/j5+Sncm5bl77//hlAoxPbt2zFs2DCIRCK1DuFy\nHAcHBwe53/QuXbqEpk2blvszkiqV1KdNm4axY8fCxMSEPTxiylV8fDysrKywYMGCMv1bE4vF2LNn\nDwwMDNCzZ08cOBAIHm8r6td/h6tXiyeoo0ePwtjYuMTple/evYOjoyNcXV1LrP9e1WVkZGDZsmXQ\n0tLC+PHjER8fX6b7RUZGQiAQYNCgQWjcuDHmz5+v9ucU+/btQ6tWreR+4Hp4eGDDhg1qbVOWKpXU\nzc3NMXjwYPz444/l3TzD4P3797C1tcWMGTPK3InIycnBpk2bIBKJMHDgQNjbb8U335xAaGhosetG\njRoFLy+vEu+Vl5eH6dOnw9jYGH///XeZ4qps8vLy8Msvv0BbWxvu7u5qWXEZGRmJhg0bQlNTEx4e\nHkot7goPL9zsojTJyckQiUS4deuWzPPv3r1Ty0YYivjsSf3w4cOwsrJCzZo1cffuXYUDe/78OYRC\nIfh8fokV0hhGnT58+IA2bdpgwoQJahl3zcjIQEBAAHg8HlxcXCEUCjFz5kzJQ/+MjAyYm5srVL3x\nwIED4PF42F/SU7sqguM4HDp0CGZmZujWrZva6ovv3r0bX331FUxNTUvcAEWWCxcKdy5SZAh84sSJ\nGDt2rNzz/v7+n608+GdP6k+ePMGzZ8/g5OSkVFLfuHEjunbtCnt7e1WbZhiVpKSkoGPHjvDx8VHb\nWPa///6LefPmoVGjRjAxMYGVlRUePHgAALhz5w54PB5iYmJKvU9kZCRMTEwwderUKlsQLCQkBK1b\nt4adnR3+/PNPtdzz+fPn6NatG2rVqoWJEycq/U3r7FmAzwdkzEqUcufOHQiFQrnDYWKxGPr6+iXm\nO3WqsOEXZZO6q6sr2rdvj23btpW1aYZRWnp6Orp06YIhQ4aodSXg27dvMXHiRNSvXx9ff/01li9f\njoKCAqxcuRL29vYKtfXvv/+iV69ecHBwkDnrorK6e/cuunfvDhMTEwQGBqrlm1BycjJmzJiBRo0a\nQUNDA7/KWx1UglOnChO6Ih37rKwstG7dWu4OSYX3O6X2jTBKUqmT+qJFi7Bo0SLMnz8f9erVg4aG\nhmQ5NsN8bllZWXBxccHAgQOl9qQsqxcvXqB//4GoUWM3DA1d8Pz5c3Tt2hXz589X6PUFBQVYtGgR\ndHV1S5xBUxlERUXBw8MDIpEImzdvVst/y7y8PGzatAkCgQDu7u5o0qQJ9uzZo/R94uIAkQiQMzRe\nTHp6OpydneHp6VniB1LPnj1VikVRFy9elOTKRYsWlU9S79atG5o3by71U7SMGlCup15Uj7i8J+0z\nTGlycnLQt29f9O7dW+W50iX58cdXqFUrCTVq9ICXlxd0dHQUWrZe5NSpU+Dz+di6dWulmiFWUFCA\nS5cuYfT/tXfvYTXl+x/A36WhiYoppYuEapIuuzQ6EiLlNtSZXIpuaDIOORwnNMaccUsjxl2n4UQh\nl4dUT9OvjKiMkVvFkKM0GYUkQlHatT6/P4yOpttu39aefF/Ps5+n9l5rfd/7a+9Py7p8v59/Tlpa\nWrRmzRqqqqqSeLscx1FycjKZm5vT2LFjKSkpiQwNDdvcc26PKOcyRT0kV1RUJJOJMNqi0Hvqb/3t\nb39rvIOMYfhWV1dH06ZNI1dX1yazwktLZiaRhsZL+uCDxaSsrEwqKiq0ceNGkY/nFxQU0ODBg2nu\n3Lky+cMjqreFfOHChaSnp0fW1ta0du1aKi8vl8r2r1+/Tq6urmRubk7JyclUUFBAffv2FeuQS0c8\nfvyY7OzsaOHChe0eMlq2bBktXbpUpnn+iNei3tYZ7rfBOI4jfX19+uijj6T+X16GEZdQKCRfX18a\nNWqUTMZlKSoisrDgyMHhHhkZDSIApKKiQsOHD6fjx4+3uxdeVVVF06dPJ3t7+8bp8+Shvr6eMjMz\nGwu5jY0NrVu3TmoT2XAcR9evX6egoCDS0dGhnTt3Ul1dHd25c4eMjIxkfs7t4cOHNHjwYFq+fHm7\n/wY1NTUymwijLXIv6vHx8WRoaEiqqqqkq6tL48ePbzPYzZs3SVNTkxYuXChukwwjEw0NDfT555/T\nsGHDJLrTsTWvXhF9/z1RQwNHXl5eNHz4cLK3t6cuXbpQ165dydnZuc0ZeTiOo02bNpGOjg7FxsbK\n7HDM20K+YMEC6tOnj9QLeUNDA124cIFCQkLIxMSE+vXrR6GhoY3n13799VcyMjKiyMjIDm+7IyN3\nv73TeO3atSL15YEDB8jNza3DmSSl8Dcfbdy4kTQ0NFq9qJ9h+MRxHAUHB9OQIUOooqJCZu1UV1eT\npaUl7d69m4RCIUVFRZFAICBlZWVSVVUlNzc3Sk9Pb7HYXL58mQQCAbm6ukrtHo/6+nrKyMhoUsjX\nr18vtUIuFAopPT2dFixYQPr6+mRhYUFfffUV5eTkNHmPxcXFZGxs3OFB2OrqiNatI7KxaX9eWaI3\nJ3eNjY3pu+++E7kNR0dHOnnyZIdySYPCF3U7OzsyMDBQqJM+DPMujuNo2bJlZGVlRY8ePZJZOwUF\nBY3T4L39OtTV1dHWrVtp8ODBpKSkRGpqajRp0iTKyspq8p2pq6ujb7/9lrS0tCgiIkKsyzL/WMgF\nAgGtX79eaocXampqKCkpiQICAkhLS4vs7e0pLCys1eFw7969S/3796cdO3Z0qJ3s7DfF3NWVSJQj\nUzdv3iQDA4MOHdrJy8uT6UQYbRG3qCv9vrLMKCkp4fnz59DS0sLKlSvxzTffyLI5hpEIEWH16tU4\nevQo0tPToa+vL5N2EhISsHDhV9DQyMO336pg8uT/vVZTU4Nt27YhOjoad+7cgZqaGlxdXTF//nx0\n794dtbW1KC4uxvbt21FVVQU/Pz/o6uqitrYWr1+/Rm1tbbOf3/09Ly8Pffr0wfTp0zF16lSYmppK\n/H5evHiBlJQUnDx5EmlpaRAIBPjss8/g4eEBIyOjVtcrKSmBs7MzFi1ahL///e8itfX8OfDll0B8\nPPDdd4CXF6Ck1PY6eXl5mDBhAiIiIuDj4yPy+/riiy9gYGCAVatWibyOtCgpKUGc8iyXon7kyBH4\n+vqisLAQ/fr1k2VzDCMV4eHh+M9//oP09PQ2i5IkQkNDkZYmRHV1BCwtlbB9O2Bo2HSZZ8+eYevW\nrYiJicG9e/egqqqK7t27Q11dHZqamnj9+jWKiopgbm4OJycnqKurQ1VVFd26dYOqqmqTx9vnTE1N\nYWJiInH+iooKJCUlIT4+HllZWRgxYgQ+++wzTJkyBb179253/cLCQkyYMAELFizAkiVLRG43JweI\nigLCw4Fevdpf/uLFi5gyZQp2794NT09Pkdt58eIF+vXrh/z8fOjp6Ym8nrQodFF3cXHBb7/9hsLC\nQlk2xTBStW3bNoSHhyMqKgpTpkyR+vbr6+sxbtw4DBkyHGpqa7BzJ7BqFbBwIdClS/Plq6urcefO\nHRQUFDQ+CgsLcevWLbx69QrKysoYPnw4Ro0aBTMzM5iamsLU1BQaGhoiZyIivHz5EhUVFaioqMDj\nx49b/PnBgwf473//Czc3N/z1r3/FxIkToampKXIb0dHRWLFiBdavX4+goCCR83VUZmYmpk2bhv37\n92PixIkdWnfXrl3IzMzEsWPHZJSubQpd1Lt164Z//etfCA0NlWVTDCN1P/30E/z8/DBmzBhs2bIF\n6urqUt1+eXk57O3tsWPHDpibuyM4GNi+HTA379h2njx5gpiYGGzYsAF9+vTBgAEDGnekNDQ0YGZm\n1ljoe/To0VicWyrcXbp0gba2Nnr37g1tbe3Gx7u/6+jowN7eHh9++GGHcwYFBaGoqAhxcXGwsLDo\n2BvtgNTUVPj5+eHIkSMYM2ZMh9YlIlhaWmLnzp0YPXq0jBK2TaGLurKyMiorKzu0x8AwiqKqqgpL\nlizBmTNnEBsbCycnJ6luPzs7G1OmTMH58+clPr5dXV2Nr7/+GnFxcYiIiMDMmTPx8OFDFBYWoqCg\nALdv38arV68aC/S7hbp3797Q0tKCmpqalN5ZU+np6QgICMD06dMRFhaGbt26tbl8djaQng6sXNnx\nthISEjBv3jycPHkSjo6OHV4/KysL8+bNQ35+PpTaO2AvI+IWdblc/WJmZibrZhhG5hITE0lPT4+W\nLVsm9Xl1d+3aRVZWVlRdXS2V7V26dIlsbGzIzc2Nfv31V6lsU1y1tbUUEhJCBgYGIo3eWFlJNH/+\nm7FbRBi5uJm4uDjS1dWVaDTFGTNmyGUijLaIW57lUtQlmU6MYRRJeXk5eXh4kLW1NV2/fl1q2+U4\njnx8fMjHx6fFy363bBHtsr131dXVUXh4uESXP0rq1q1bZGtrS+7u7vS4nTuEOI7o6FEifX2ioCAi\nccb827t3L+nr6zcOfywOeU6E0RaFLurSGPCHYRQFx3EUHR1N2traFBERIbWx2V++fEnW1tbNbsDh\nOKI1a4h69iTy9CSKj6dmk163pbCwkMaMGUN2dnZyGwuc4ziKjIwkbW1tioqKEun+lJ07iQYPJvrp\nJ/Ha3LZtGxkZGUl8vf369espMDBQom1Ig0IXdYbpjIqLi2nkyJE0cuRIkSbCEEVhYSH17t27xWF3\nKyuJ9uwhGjWK6KOPiFauFH27HMfRvn37SEdHhwIDA+ny5csyuxGwvLycpkyZQnZ2dq3ecNSS6moi\ncYeF2rBhA5mYmHRoiruWyHsijLawos4wPKivr6eIiAjS1tamffv2SaVQJiYmkqGhYZt3tf72G9Hp\n0x3fdnl5Oa1bt46MjY1JIBDQrl27pDreTVpaGunr69Py5cvbHLhPWn9PcnJyyN/fnywsLCSe1Jro\nzZDHDg4OUkgmOVbUGYZH169fJ2tra/Lw8JDKkLRffvkljRkzRqzj4Dk5RO3Vt4aGBjp16hRNnTqV\nNDU1KSAggM6fPy/2H6WamhpavHgxGRoa0pkWJgN9/pwoIeHNCdABA4h++EGsZojozRjou3fvJjs7\nO+rXrx+tWbOm1SnoOmr8+PEUExMjlW1JihV1huFZbW0tLV++nPT09JpMJCOO+vp6Gjt2LK1YsaLD\n665f/+b4u6srUUwMUXsjCj969Ig2btxIpqamZGFhQVu2bOnQoGY3btwga2tr8vT0bFZcExKIRo4k\n6tGDyMWFaONGomvXOr6nznEcnTt3jvz9/alnz540bdo0SktLk8rUeW/duXOHtLW1eR27/l2sqDOM\ngjh37hz179+fAgMDJRqjvby8nIyMjMQaIfDVqzdXkkyeTKSpSeTt/WZvuS0cx1FGRgbNmjWLNDU1\nydvbm86cOdPq3jvHcbRjxw7S1tam6OjoFpfLziZKSSESdw6S8vJyioiIoI8//pjMzc1p06ZNUpuc\n449CQkLkPhFGW8StnXK5+UjGTTCMwnl7w9LZs2cRExMj9g1Lly5dwqeffirRjUkVFUBSEjB7dvOB\nr6qrgchIQEOj6aNLl+e4fHk/9uzZg9evXyMwMBABAQHQ1dUFADx69Aj+/vNQXGyAkSPXoWfPXoiI\nECteMxzH4ccff8TevXtx+vRpeHh4IDAwEI6OjjK7EejJkycwNzfHhQsXpDIujjQo9B2lrKgz76uk\npCR88cUXGDp0KHx9fTFp0iSoqqp2aBv//ve/sWvXLmRnZ6N79+5SzVdZCYSFAS9eNH306AH83/+9\nuV0+Ozsbe/bsQXx8PAYNckF+fjhevnyKLl1sYGvbFePHK2PSJOCTTyTLUlJSgn379iE6Ohra2toI\nDAyEt7e3yGPKiIuI4OnpiYEDByJCWn+ZpIAVdYZRUC9evMCJEydw8OBB5OXlwdPTEz4+PnBycoKy\nsnK76xMRAgICIBQKcejQIbnftn7nzh0cOHAAMTExeP0aaGiwAMd9iMGDG6ClpQwNDQ1oaGhAU1Oz\n8efWfldXV4eKikrjtoVCIZKTk7F3715kZ2fDy8sLgYGBsLW1ldv727NnD3bv3o3s7Ox2hy6QJ1bU\nGeZPoKSkBIcPH8aBAwdQVVUFHx8f+Pj4wLydEbxevXqFsWPH4pNPPsHWrVtlXtifPn2KY8eOITY2\nFkVFRZg5cyb8/PwgEAigpKSE/Px83L9/Hy9evGh8PH/+vMnvrT2nqqraWOgrKythbm6Ozz//HJ6e\nnjIbd6Y1t2/fhpOTE7KysjBo0CC5tt0eVtQZ5k+EiHDt2jUcPHgQcXFxMDAwgI+PD7y9vaGjo9Pi\nOs+ePYOLiwvc3NwQFhYm9cJeV1eH1NRUxMbG4scff8T48ePh5+cHNzc3fPDBB1Jpg34f2vfdAm9s\nbCyVbXdUXV0dHB0dMXfuXMyfP5+XDG1hRZ1h/qQaGhpw5swZHDhwAElJSXB0dISvry/c3d2b7bk+\nefIEzs7OmDFjBr766iuJ2yYiXLlyBbGxsTh69CjMzc3h5+eHqVOnomfPnhJvX5GFhobixo0bSEpK\n4m0kxrawos4wncDLly+RkJCAgwcPIjs7G+7u7vD19YWzszO6/D5zRllZGUaNGoWgoCAsXbpUrHbu\n3buHQ4cOITY2FkKhEH5+fvDx8cGAAQOk+XYUVkZGBmbOnIm8vLxW/2fEN1bUGaaTKSsrw+HDh3Hw\n4EE8evQI7u7u6NGjB4A3l0zGxcVh6NChHTqpSES4evUq8vLyMH36dPj6+mLYsGEKuacqK5WVlbCx\nscH333+P8ePH8x2nVayoM0wnlp+fj9TUVAiFwsbnnj59iqioKLi5uWHIkCEib8vExESsSys7AyLC\njBkzoKenh23btvEdp01yL+ohISFITk5G165dMXDgQOzbt6/F60lZUWcY2SkoKMDo0aOxefNmeHl5\n8R1H4cXExGDTpk24fPmywv9RE7d2tn+RbCvc3Nxw8+ZNXLt2DWZmZtiwYYO4m2IYRkxmZmZIS0vD\n4sWLkZCQwHcchVZUVIR//vOfiIuLU/iCLgmxi7qrq2vjjRMODg4oLS2VWiiGYURnaWmJlJQUzJs3\nD6mpqXzHUUhCoRCzZs3CqlWrYGVlxXccmVJpf5H2RUdHw9vbu9XXv/nmm8afnZ2d4ezsLI1mGYb5\nnZ2dHRISEuDu7o5jx46x79gfrF27Fr169UJwcDDfUVqVkZGBjIwMibfT5jF1V1dXlJWVNXs+LCwM\nkydPBgCsX78eOTk5OHHiRMsNsGPqDCM3GRkZmD59OhITEzFs2DC+4yiEn376CdOmTUNubi769OnD\ndxyR8XL1y/79b0ZxS09Pb/UYFSvqDCNfaWlp8PPzQ0pKSoeuiumMnj9/DoFAgO3btzfuiP5ZyL2o\np6amYunSpcjMzIS2trbUgzEMI77ExETMmzcPp0+fhqWlJd9xeOPj4wMNDQ3s3r2b7ygdJm7tFPuY\nenBwMOrq6uDq6goAGDZs2J+y4ximM3J3d0dtbS3GjRuHM2fO4OOPP+Y7ktwdOnQIOTk5uHLlCt9R\n5Ersol5YWCjNHAzDSNmMGTNQU1MDV1dXZGZmon///nxHkpu7d+9iyZIlOHXqlNxHfuSbVK5+YRhG\nMQUEBKCmpgZjx45FZmYmDA0N+Y4kc/X19fDx8cHy5cshEAj4jiN3rKgzTCc3f/581NTUwMXFBVlZ\nWY1T0nVW4eHhUFVVxZIlS/iOwgtW1BnmPfCPf/yjcaKNs2fPtnlxw5/ZxYsXsXPnTly9elWkWaU6\no/fzXTPMe2jlypXw8PCAvb09Tp06xXccqauqqsKsWbMQGRkJAwMDvuPwho3SyDDvmVOnTiEoKAgu\nLi7YvHlzp5kMY/bs2VBRUcGePXv4jiIVch/Qi2GYPyc3Nzf88ssv6NatGywtLZGcnMx3JIkdO3YM\n58+fx5YtW/iOwju2p84w77GMjAzMnTsXjo6O2Lp1K7S0tPiO1GElJSUYMmQIfvjhB3zyySd8x5Ea\ntqfOMEyHOTs74/r169DS0oKVlRXi4+P5jiQyjuNw6NAhODk5ISQkpFMVdEmwPXWGYQAA58+fx5w5\ncyAQCLBjxw6FnbsTALKysrB06VIoKytj8+bNcHJy4juS1LE9dYZhJDJ8+HDk5eXB2NgY1tbWOHLk\niMLtkBUUFMDDwwP+/v5YunQpLly40CkLuiTYnjrDMM1cunQJc+bMgYmJCSIjI6Gnp8drnsePH2PN\nmjU4cuQIli1bhuDg4E49exHA9tQZhpGioUOH4urVq7CysoKNjQ1iYmJ42Tmrra3Fxo0bYWFhASUl\nJdy6dQshISGdvqBLgu2pMwzTptzcXMyePRv6+vqIiopC3759Zd4mx3E4evQoQkNDYWdnh/DwcJiZ\nmcm8XUXC9tQZhpEJW1tbXL58GcOGDYOdnR327Nkj0x21c+fO4S9/+Qu2bNmC2NhYxMfHv3cFXRJs\nT51hGJHduHEDs2fPhrq6Ojw8PNC/f//GR/fu3SXadkFBAVasWIGcnBxs2LABM2bMeG/HbwF4ms5O\npAZYUWeYTqW+vh779+9Hbm4uiouLUVxcjLt370JdXb1JkX/3YWRkhK5du7a4vYqKCqxZswaHDx9G\nSEgIFi1axI6ZgxV1hmF4REQoKytrLPJ/fDx48AC6urpNCr2xsTEePnyIzZs3w8vLC19//TV69+7N\n91tRGKyoMwyjsOrr61FaWtqs2CsrK2PlypXv5XR77WFFnWEYphNhV78wDMMwrKgzDMN0JqyoMwzD\ndCJiF/VVq1bBxsYGAoEALi4uKCkpkWYumcrIyOA7QjOKmAlQzFwsk2hYJtEpai5xiF3Uly1bhmvX\nriEvLw8eHh5YvXq1NHPJlCL+AypiJkAxc7FMomGZRKeoucQhdlFXV1dv/Lm6urrTzk7OMAzzZ6Ii\nycorV67EgQMHoKamhuzsbGllYhiGYcTU5nXqrq6uKCsra/Z8WFgYJk+e3Ph7eHg4bt++jX379jVv\nQElJSlEZhmHeL7zdfHTv3j1MnDgRN27ckHRTDMMwjATEPqZeWFjY+HNiYiJsbW2lEohhGIYRn9h7\n6lOnTsXt27fRpUsXDBw4EJGRkQo9US3DMMz7QOw99ePHj+OXX35BXl4eTpw4gZycHJibm8PU1BTf\nfvtti+ssWrQIpqamsLGxQW5urtihRZWamtpmpoyMDGhqasLW1ha2trZYt26dTPPMmTMHurq6sLKy\nanUZefeRKLnk3U8AUFJSgtGjR2Pw4MGwtLTE9u3bW1xOnv0lSiZ591VtbS0cHBwgEAhgYWGB0NDQ\nFpeTZz+JkomPzxQANDQ0wNbWtsk5wXfx8f1rK5NY/URSUF9fTwMHDqTi4mKqq6sjGxsbys/Pb7LM\nDz/8QBMmTCAiouzsbHJwcJBG0xJlOnv2LE2ePFmmOd6VlZVFOTk5ZGlp2eLr8u4jUXPJu5+IiB4+\nfEi5ublERFRVVUVmZma8f6ZEycRHX718+ZKIiIRCITk4ONC5c+eavM7H56q9THz0ExHR5s2baebM\nmS22zdf3r61M4vSTVIYJuHTpEkxMTGBsbIwPPvgAXl5eSExMbLJMUlIS/P39AQAODg549uwZHj16\nJI3mxc4EiHd2WVwjRoxAr169Wn1d3n0kai5Avv0EAH369IFAIAAA9OjRA4MGDcKDBw+aLCPv/hIl\nEyD/vlJTUwMA1NXVoaGhAR999FGT1/n4XLWXCZB/P5WWliIlJQWBgYEtts1HP7WXCeh4P0mlqN+/\nf7/JZLSGhoa4f/9+u8uUlpZKo3mxMykpKeHnn3+GjY0NJk6ciPz8fJnlEYW8+0hUfPfT3bt3kZub\nCwcHhybP89lfrWXio684joNAIICuri5Gjx4NCwuLJq/z0U/tZeKjn5YsWYKIiIhWp8jjo5/aB1w1\nRgAAAm5JREFUyyROP0mlqIt6Lfof/+LI8hp2UbZtZ2eHkpISXLt2DcHBwfDw8JBZHlHJs49ExWc/\nVVdXY+rUqdi2bRt69OjR7HU++qutTHz0lbKyMvLy8lBaWoqsrKwWb3mXdz+1l0ne/ZScnAwdHR3Y\n2tq2uecrz34SJZM4/SSVom5gYNBkQK+SkhIYGhq2uUxpaSkMDAyk0bzYmdTV1Rv/mzhhwgQIhUI8\nffpUZpnaI+8+EhVf/SQUCuHp6QkfH58WP8x89Fd7mfj8TGlqamLSpEm4cuVKk+f5/Fy1lkne/fTz\nzz8jKSkJ/fv3h7e3N86cOQM/P78my8i7n0TJJFY/iX94/3+EQiENGDCAiouL6fXr1+2eKL1w4YLM\nT0KIkqmsrIw4jiMioosXL1K/fv1kmomIqLi4WKQTpfLoI1Fz8dFPHMeRr68vLV68uNVl5N1fomSS\nd189fvyYKisriYjo1atXNGLECDp9+nSTZeTdT6Jk4uMz9VZGRgZ9+umnzZ7n8/vXWiZx+kmisV/e\nUlFRwc6dOzFu3Dg0NDRg7ty5GDRoEKKiogAA8+bNw8SJE5GSkgITExN07969xSEFpEmUTMePH0dk\nZCRUVFSgpqaGI0eOyDSTt7c3MjMzUVFRgb59+2L16tUQCoWNeeTdR6Lmknc/AcD58+dx8OBBWFtb\nN97YFhYWhnv37jXmknd/iZJJ3n318OFD+Pv7g+M4cBwHX19fuLi48PrdEyUTH5+pd709rMJnP4mS\nSZx+kvkcpQzDMIz8sJmPGIZhOhFW1BmGYToRVtQZhmE6EVbUGYZhOhFW1BmGYToRVtQZhmE6kf8H\nXNGMzVqLBIoAAAAASUVORK5CYII=\n"
}
],
"prompt_number": 8
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Calculating bias and variance\n",
"-----------------------------\n",
"\n",
"Same as previous example, we generate several samples and fit a polynomial to each one. We calculate bias an variance among models for different polynomial degrees. Bias, variance and error are plotted against different degree values."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"from numpy.linalg import norm\n",
"n_samples = 20\n",
"f_x, f_y = f(n_samples)\n",
"n_models = 100\n",
"max_degree = 15\n",
"var_vals =[]\n",
"bias_vals = []\n",
"error_vals = []\n",
"for degree in xrange(1, max_degree):\n",
" avg_y = np.zeros(n_samples)\n",
" models = []\n",
" for i in xrange(n_models):\n",
" (x,y) = sample(n_samples)\n",
" model = fit_polynomial(x, y, degree)\n",
" p_y = apply_polynomial(model, x)\n",
" avg_y = avg_y + p_y\n",
" models.append(p_y)\n",
" avg_y = avg_y / n_models\n",
" bias_2 = norm(avg_y - f_y)/f_y.size\n",
" bias_vals.append(bias_2)\n",
" variance = 0\n",
" for p_y in models:\n",
" variance += norm(avg_y - p_y)\n",
" variance /= f_y.size * n_models\n",
" var_vals.append(variance)\n",
" error_vals.append(variance + bias_2)\n",
"pl.plot(range(1, max_degree), bias_vals, label='bias')\n",
"pl.plot(range(1, max_degree), var_vals, label='variance')\n",
"pl.plot(range(1, max_degree), error_vals, label='error')\n",
"pl.legend()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 9,
"text": [
"<matplotlib.legend.Legend at 0x107b64f50>"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD9CAYAAABdoNd6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtclGX6+PHPIJiJJ1BQnFEBIQFNJUEDs1hPJJV5qljt\nZGpkudXWbm3f2m+6bZa1h2q1siwPravWt19qhsiqkaYpKZqaqaCAnDyCR5TD8Pz+uGMERZzBGZ6Z\n4Xq/Xs+LOTz3zPWgc83NfTRomqYhhBDCrXnoHYAQQgjHk2QvhBBNgCR7IYRoAiTZCyFEEyDJXggh\nmgBJ9kII0QRcM9mnpKQQFhZGaGgos2bNuuL5FStW0KdPHyIjI+nXrx/r16+3PBcYGEjv3r2JjIyk\nf//+9o1cCCGE1Qz1jbM3m8306NGDtWvXYjQaiY6OZsmSJYSHh1vOOX/+PN7e3gDs3r2b0aNHk5WV\nBUBQUBDbt2/H19fXwZchhBCiPvXW7NPT0wkJCSEwMBAvLy8SExNZsWJFrXOqEz3AuXPn6NChQ63n\nZc6WEELor95kX1BQQJcuXSz3TSYTBQUFV5y3fPlywsPDGTFiBO+9957lcYPBwNChQ4mKiuLjjz+2\nY9hCCCFs4VnfkwaDwaoXGTVqFKNGjWLjxo089NBD7N+/H4BNmzYREBDA8ePHGTZsGGFhYQwaNKhB\n7yGEEKI2W1pO6q3ZG41G8vLyLPfz8vIwmUxXPX/QoEFUVlZy8uRJAAICAgDw8/Nj9OjRpKenXzVg\nVz1effVV3WOQ+PWPoynG78qxu0P8tqo32UdFRZGZmUlOTg7l5eUsW7aMkSNH1jrn4MGDljfOyMgA\noH379pSWlnL27FlAdeKmpqZy88032xygEEKI61dvM46npyezZ88mPj4es9nMpEmTCA8PZ+7cuQAk\nJSXx5ZdfsmjRIry8vGjVqhVLly4F4MiRI4wZMwaAyspKJkyYwPDhwx18OUIIIepS79DLRgnAYGjQ\nnyTOIi0tjbi4OL3DaDCJX1+uHL8rxw6uH7+tuVOSvRBCuCBbc2e9zThCCFHN19eXkpISvcNocnx8\nfCguLr7u15GavRDCKvJZ1cfVfu+2/nvIQmhCCNEESLIXQogmQJK9EEI0AZLshRAuLTAwkHXr1l3x\n+MaNGwkLC9MhIuckyV4I4dIMBkOda2wNGjSIffv26RCRc5JkL4QQTYAkeyGEy0tPT6dnz574+vry\n2GOPUVZWRlpaWq0l2t98801CQkJo06YNPXv2ZPny5ZbnsrKyuOOOO2jXrh1+fn4kJibqcRkOJcle\nCOHSNE3jP//5D6mpqRw8eJADBw7w17/+9YqmnZCQEL7//nvOnDnDq6++yoMPPsjRo0cB+POf/8yd\nd97JqVOnKCgo4Omnn9bjUhxKkr0Qwi4MBvsctr+vgWnTpmE0GvHx8eHll19myZIlV5w3btw4OnXq\nBMD9999PaGioZdn15s2bk5OTQ0FBAc2bNyc2Nva6fhfOSJJ9Q50+DQsWwFXW6BeiqdE0+xwNUbO5\npmvXrhQWFl5xzqJFi4iMjMTHxwcfHx/27NnDiRMnAHjrrbfQNI3+/fvTq1cv5s+f37BAnJisjWOL\nsjJITobFi+G//4XAQAgLg2XL9I5MiCbt8OHDtW537ty51vO5ubk8/vjjrF+/npiYGAwGA5GRkZbl\nBjp27MhHH30EqB32hg4dyh133EFwcHDjXYSDSc3+Wqqq4NtvYfJk6NwZ3nsP7rwTcnLgyy/hhx/0\njlCIJk3TNObMmUNBQQHFxcW8/vrrV3Swnj9/HoPBQIcOHaiqqmL+/Pns2bPH8vwXX3xBfn4+AO3a\ntcNgMODh4V7p0b2uxl40DXbsgD/8Abp2heeeUzX4n366lPh9fKB7d7hwAWps3SiEaFwGg8GyOVL3\n7t0JDQ3llVdeQdM0SydtREQEzz//PDExMXTq1Ik9e/Zw2223WV5j27Zt3HrrrbRu3Zp7772X9957\nj8DAQJ2uyDFk1cuaDh2CJUtUM82FCzBhAowfDxERVy9z773qvPvvb7w4hdCBU31WmxB7rXopbfbH\nj8Pnn6sEn5UF990H8+ZBTIx1QwNiYlRTjiR7IYQTa5rJ/tw5WLFCJfjNm+Guu+CVV2DYMPDysu21\nYmPhj390TJxCCGEnTacZp6ICUlNVgk9OhoEDVRPNvfdCq1YNf93SUvDzgxMn4MYb7RevEE5GmnH0\nIc041tA0VXNfvBi++AJCQ1X7+rvvqgRtDy1bqjb97duhRoePEEI4E/dO9osWwaxZKsFv3QqOGjMb\nG6u+VCTZCyGclHs345jN4OHRsDnYtli6VE2s+uorx76PEDqSZhx9NNoetCkpKYSFhREaGsqsWbOu\neH7FihX06dOHyMhI+vXrx/r1660u63DNmjk+0cOlmr18EIQQTqremr3ZbKZHjx6sXbsWo9FIdHQ0\nS5YsITw83HLO+fPn8fb2BmD37t2MHj2arKwsq8qCm9QWNA26dIENGxzXVCSEztzis+qCGqVmn56e\nTkhICIGBgXh5eZGYmMiKFStqnVOd6AHOnTtHhw4drC7rNgwGNd5+82a9IxFCXAd33sqw3g7agoKC\nWqvJmUwmtm7desV5y5cv56WXXqKoqIjU1FSbygJMnz7dcjsuLo64uDhbrsE5xMaqyVUPPqh3JEKI\nBnLmrQzT0tJIS0trcPl6k31d+zrWZdSoUYwaNYqNGzfy0EMP2fzLqpnsXVZMjBr9I4RwSZWVlXh6\nOu8AxcsrwjNmzLCpfL3NOEajkbwai3zl5eVhMpmuev6gQYOorKykuLgYk8lkU1mXFxkJBw7A2bN6\nRyJEkzJr1izuu+++Wo8988wzPPPMMyxYsICIiAjatGlD9+7dLcsYg6opm0wm3nrrLQICApg0aZJN\nWxkuWLCA2267jT/+8Y/4+voSHBxMSkqK5fni4mImTpyI0WjE19eX0aNHW55btWoVffv2xcfHh4ED\nB7J7925H/Gpq0+pRUVGhBQcHa9nZ2VpZWZnWp08fbe/evbXOycrK0qqqqjRN07Tt27drwcHBVpf9\ntXO4vhBcS2yspq1bp3cUQjiEs35Wc3NztZYtW2pnz57VNE3TKisrtYCAAG3r1q3aN998ox06dEjT\nNE377rvvtJYtW2oZGRmapmnat99+q3l6emp/+tOftPLycu3ChQvat99+q5lMJstrf/HFF1pRUZGm\naZq2bNkyzdvbWzty5IimaZo2f/58zcvLS5s3b55WVVWlffDBB1rnzp0tZRMSErTExETt1KlTWkVF\nhbZhwwZN0zQtIyND8/f319LT07Wqqipt4cKFWmBgoFZWVlbn9V3t927rv0e9f7N4enoye/Zs4uPj\nMZvNTJo0ifDwcObOnQtAUlISX375JYsWLcLLy4tWrVqxdOnSesu6teohmIMH6x2JEI3OMMM+w5y1\nV20b8dO1a1duueUWvvrqKx566CHWr19Py5Yt6d+/f63zbr/9doYPH87GjRuJjIwEwMPDgxkzZuDl\n5YVXHetijRs3znL7/vvv54033mDr1q2MHDkSgG7dujFp0iQAHn74YZ588kmOHTuG2WwmJSWF4uJi\n2rZtC6iWD4CPPvqIpKQkoqOjLeVmzpzJli1buP322226dltcs4FqxIgRjBgxotZjSUlJltsvvPAC\nL7zwgtVl3VpsrFoxU4gmyNYkbU/jx49nyZIlPPTQQ/znP/9hwoQJAKxevZoZM2aQmZlJVVUVpaWl\n9O7d21LOz8+P5s2bX/V1Fy1axD//+U9ycnIANeLw5MmTluer97QFaNmypeWcEydO4Ovra0n0NeXm\n5rJo0SL+9a9/WR6rqKigqKioYRdvJdm8xJ6qlzuuqtI7EiGalHHjxpGWlkZBQQHLly9n/PjxlJWV\nMXbsWF544QWOHTtGSUkJCQkJtcam1zcIpXorwzlz5lBcXExJSQm9evWyamx7ly5dKC4u5vTp01c8\n17VrV15++WVKSkosx7lz53jggQcadvFWkmRvT506Qbt2sH+/3pEI0aT4+fkRFxfHo48+SnBwMD16\n9KC8vJzy8nI6dOiAh4cHq1evtgwNt8a1tjKsT0BAACNGjODJJ5/k1KlTVFRUsGHDBgCmTJnChx9+\nSHp6Opqmcf78eb755hvOnTvXoGu3liR7e6uu3QshGtX48eNZt24d48ePB6B169a899573H///fj6\n+rJkyRLuvffeWmXqqtlbu5WhwWC4onzN+5999hleXl6EhYXRsWNH3nvvPQD69evHxx9/zLRp0/D1\n9SU0NJRFjTBs270XQtPDnDlq/1ppuxduxu0+qy6i0RZCEzaqnkkrhBBORGr29lZZCT4+cPiw+imE\nm3C7z6qLkJq9s/L0hOhotVmKEEI4CUn2jiArYAohnIwke0eonkkrhBBOQtrsHeHkSQgKgpIStVuW\nEG7ALT+rLkDa7J1Z+/bQuTNYOQFDCCEcTZK9o8gQTCGEE5Fk7yjSSSuEcCKS7B1FOmmFEE5Ekr2j\nhIfDiRNw7JjekQghfmU2m2vd1zTNpk5OW893JpLsHcXDA269VdrthWgEhYWFjB07Fn9/f4KDgy1r\nxU+fPp1x48bx0EMP0bZtWxYsWEBcXBwvv/wyAwcOxNvbm+zsbDZv3kx0dDTt2rWjf//+/FDjcxsX\nF8crr7xS63xXJMnekaSTVgiHq6qq4p577iEyMpLCwkLWrVvHO++8Y1nOeOXKldx3332cPn3asqnJ\n4sWLmTdvHufOncPb25u77rqLZ599luLiYp577jnuuusuSkpKLO/x73//23J+165ddbnO6yXJ3pGk\nk1Y0JQaDfQ4b/fjjj5w4cYJXXnkFT09PgoKCmDx5MkuXLsVgMBAbG2vZRrBFixYYDAYeffRRwsPD\n8fDwIDU1lR49ejBhwgQ8PDxITEwkLCyMlStX/npZtc/39LzmBn9OyTWjdhUDBkBGBpSXQz1bnwnh\nFnRqy87NzaWwsBCfGgsPms1mbr/9drp164bJZLqiTJcuXSy3CwsLr6itd+vWjcLCwjrPd1VSs3ek\nNm0gOBh++knvSIRwW127diUoKKjWNn9nzpxh1apVQP0blAAYjUZyc3NrPZ+bm4vRaKzzfFclyd7R\nZAimEA7Vv39/WrduzVtvvcWFCxcwm83s2bOHbdu2XbVMzRE1CQkJHDhwgCVLllBZWcmyZcvYt28f\nd999d53nuypJ9o4m2xQK4VAeHh6sWrWKnTt3EhwcjJ+fH48//rhls+9r1ex9fX1ZtWoVf//73+nQ\noQN/+9vfWLVqFb6+vnWe76pkITRHy8yEIUPUZiZCuDC3/6w6qUZbCC0lJYWwsDBCQ0OZNWvWFc8v\nXryYPn360Lt3bwYOHMiuXbsszwUGBtK7d28iIyPp37+/1UG5lZAQuHAB8vP1jkQI0YTVOxrHbDYz\nbdo01q5di9FoJDo6mpEjRxIeHm45Jzg4mA0bNtC2bVtSUlJ4/PHH2bJlC6C+edLS0mr9OdTkGAyX\nmnLuu0/vaIQQTVS9Nfv09HRCQkIIDAzEy8uLxMREVqxYUeucmJgY2rZtC8CAAQPIv6wGK3/2IZ20\nQgjd1VuzLygoqDW+1GQysbWevVU/+eQTEhISLPcNBgNDhw6lWbNmJCUlMWXKlDrLTZ8+3XI7Li6O\nuLg4K8N3ETEx8OKLekchxHXx8fFxi45KV1M9fyAtLY20tLQGv069yd6Wf9hvv/2WTz/9lE2bNlke\n27RpEwEBARw/fpxhw4YRFhbGoEGDrihbM9nbU2Gh2jSqV68GTcyzn+ho2L0bLl6EFi10DESIhisu\nLtY7hCbt8orwjBkzbCpfbzOO0WgkLy/Pcj8vL6/O2Wi7du1iypQprFy5stYstoCAAAD8/PwYPXo0\n6enpNgV3vX76Ce69F0wmeOwxWLZMJf9G17IlRETA9u06vLkQQlwj2UdFRZGZmUlOTg7l5eUsW7bM\nssZEtcOHDzNmzBj+/e9/ExISYnm8tLSUs2fPAnD+/HlSU1O5+eabHXAJVzdiBBw8CGlpEBkJn32m\ntoa99VZ49VXVjF5Z2UjByDo5Qggd1duM4+npyezZs4mPj8dsNjNp0iTCw8OZO3cuAElJSfzlL3+h\npKSEqVOnAuDl5UV6ejpHjhxhzJgxAFRWVjJhwgSGDx/u4Mu5ksEAoaHq+N3voKwMNm2CNWtg6lTI\ny4PBg+HOOyE+Hhy2BEZsrPrTQgghdNDkJ1UVFUFqqkr+qang76+Sfnw83HEH3Hijnd4oN1ctjFZU\npHMHghDCHdiaO5t8sq/JbFaLVK5Zo46dO1WFvDr5R0RcR57WNNV5sHGjWhxNCCGugyR7Ozp9Gtav\nh5QUlfzNZhg+XDX5DB0KNfqirTNuHIweDb9uoCCEEA1l9+USmrK2bVVunjsXsrNh7Vro0wcWLIBu\n3VSfq03DXmVylRBCJ1Kzb6CLF+Ef/1BNPZ9/bmWhLVvgySdVW5EQQlwHacZpRPv3qyYdq/cfLisD\nX184ehRatXJobEII9ybNOI0oNBSKi+H4cSsL3HAD9O0LjTy5TAghJNlfBw8P6NfPxomxspmJEEIH\nkuyvU3Q0/PijDQWkk1YIoQNJ9tfJ5mQfE6M6aquqHBaTEEJcTpL9dYqKUsne6n6SgABo0wYOHHBo\nXEIIUZMk++vUrZtaTK2w0IZC0pQjhGhkkuyvk8HQwKYc6aQVQjQiSfZ2IJ20QghnJ8neDqrb7a3W\nuzccPgynTjksJiGEqEmSvR1ER8O2bTZ00np6qm+ILVscGpcQws0cParWaenb1+aikuztoFMn8PaG\nQ4dsKCRNOUIIa1y8qBbguusu6NEDdu2Cf/7T5peRZG8nNjflSCetEOJqNE1VBpOSwGiEjz6CxEQo\nKFDL7v7mNza/ZL3bEgrrVTflJCZaWeDWW9W69mYzNGvm0NiEEC4iJ0dtlr1okcoLjzyilta1w36p\nUrO3E5tH5HTooNp/fv7ZYTEJIVzA2bMwf76qrUdFwZEj8O9/wy+/wEsv2W1jbKnZ20lUlFqm3qaK\nenW7fe/eDo1NCOFkzGa1Dd6iRfD112rD62nT4O671eq4DiA1ezvx8YGOHWHfPhsKxcZKu70QTUl1\nbT0wEP70J9UkcOAArFgBY8c6LNGDJHu7qm63t1pMjIzIEcLdnTwJc+ZA//4wZIhaX2X1arU2+tNP\ng79/o4Qhyd6ObG63j4hQO58cO+awmIQQOjh9WtXWx4yB4GD4/nv4y1/UZMq334ZevRo9pGsm+5SU\nFMLCwggNDWXWrFlXPL948WL69OlD7969GThwILt27bK6rLuxefilhwcMGCCTq4RwJaWlak/StWtV\nx+pf/gJTpqg9Snv2VKvaGo3w979DQoJK8EuWqOc99esmrXcPWrPZTI8ePVi7di1Go5Ho6GiWLFlC\neHi45ZwffviBiIgI2rZtS0pKCtOnT2fLli1WlQXX3oP2cufOqXb7khJo3tzKQjNmwIUL8OabDo1N\nCGGF8nLIz4e8vKsf58+DyaRGyVztaNdOrZLoQLbmznq/ZtLT0wkJCSEwMBCAxMREVqxYUSthx8TE\nWG4PGDCA/Px8q8u6m1atICgI9uyBW26xslBsLPz1rw6NSwhxmT17YM2aKxP5yZNqz4maiTs8HIYP\nv3Tfz8/hidwR6k32BQUFdKkxxtNkMrF169arnv/JJ5+QkJBgc9np06dbbsfFxREXF2dN7E6put3e\n6mQ/YIDqqKmoAC8vh8YmRJNWUQFffaU6S7OyVHt6UBAMHHgpkXfq5LSTHNPS0khLS2tw+XqTvcGG\nb69vv/2WTz/9lE2bNtlctmayd3XV7fZJSVYWaNNGdeDs3Km+KYQQ9lVYqJYb+PhjCA1V49lHjXK5\nytXlFeEZM2bYVL7eDlqj0UheXp7lfl5eHiaT6Yrzdu3axZQpU1i5ciU+Pj42lXU3Ng+/BFknRwh7\n0zTYsAHuv191mh49qppt0tLgvvtcLtHbhVaPiooKLTg4WMvOztbKysq0Pn36aHv37q11Tm5urta9\ne3fthx9+sLnsr53D9YXgci5e1LQbb9S08+dtKLRggaY98IDDYhKiyTh7VtM++EDTevXStLAwTXvv\nPU07dUrvqBzC1txZbzOOp6cns2fPJj4+HrPZzKRJkwgPD2fu3LkAJCUl8Ze//IWSkhKmTp0KgJeX\nF+np6Vct6+5uuEH15+zcqfperRITA//7vw6NSwi3tm8fvP8+LF6slh545x0YPNglO1Lro2kah0oO\nsfHwRpvL1jv0sjG409DLak88oeZLPf20lQU0TfXw79yphnQJIa6tslKtKzNnjhpdM3my6iyz08Jh\nzqBKq+LnYz+z8fBGNuRuYOPhjWiaxqBug/j8vs/tN/RSNEx0tGoatJrBcGmdnPvuc1RYQriHY8dU\nZ+vcuSqxP/WUw9eVaSwV5goyijIsiX1T3iZ8b/RlUNdBxHeP5/XBrxPsE4zBYOBzPrfptaVm7wC7\ndql+IZsWRXvjDbV0wj/+4bC4hHBZmqZmms+ZA998o5L7U09BZKTekV2X0opStuZvtST3rQVbCfYJ\nZlDXQdze7XYGdR1EQOuAOsvamjsl2TtAZaWaQFdQAG3bWlnou+/gxRdl6QQhaiotVUsNzJkDZ87A\nk0/CxIlqmVkXdOriKTYd3sSGwxvYmLuRn47+RO+OvS2JfWCXgfjcaN21SbJ3EgMHqomxVu8edv68\nWv3u5Elo0cKhsQnRaDRNLUFQVmb7sWePWu89JkbV4ocPV+tJuZCis0VsPLzR0uZ+qOQQA4wDLDX3\nAaYBtPRq2aDXtutyCaLhqmfSWp3svb3VMJ7t29U3hRDOStNg6VL45BNVSbk8SV+8eOl2ebka037D\nDdYfLVqon0aj+hAFBel9xVY7dv4Y67PXsy57HWk5aZwsPcltXW/j9m6389HdH3FLwC14NdNnjL8k\neweJilIrnNqkupNWkr1wVps2wXPPqZ2WXn5Zrfx3tWR9ww1qRUAXq43b4kzZGb7L+Y512etYn72e\nw6cPc0fgHQwJGsLT/Z+mp39PPAzOcf2S7B0kOhr+/GcbC8XEwBdfOCQeIa7LwYNqZ6WtW2HmTBg/\n3q2T+NVcrLzID3k/sC57Heuy17Hn2B4GGAcwJGgIH9/zMf0698PTwznTqrTZO0hVFfj6QmamGkJv\nldxctTBaUZHbTQYRLqqkRHU+LVyoavS//z3ceKPeUTUac5WZ7UXbWXdIJfetBVvp6deTIcFDGBI0\nhNgusbTw1KePTdrsnYSHB/Trp9bJGTHCykJdu6qCOTku1U4p3FB5OXzwAbz+OoweDT//rJps3Jym\naew9vtfSLPNd7neY2pgYEjSEZwY8w+3dbqdtC2uH2DkXSfYOVL0CptXJvnpy1ebNkuyFPjRNdTa9\n8AJ07w7r1+uyhV5jyj2Va2mWWZ+9nhs9b2RI8BASeyUy9+65dGzlHl9ykuwdKDpajRyzSfUKmBMm\nOCQmIa5q2zZ4/nkoLobZs9VQRzdQWVXJ8fPHOXb+WK3jlxO/sC57HefKzzE4aDBDgobw19/8lSAf\n96xoSZu9A+Xmwq23quW0rW6C/+EHNaY4I8OhsQlhkZcH//M/sG6d2ibzscecdgMPUE0tp8tOX5G8\nr3acLjtN+xvb4+/tX+sI9glmcNBgevr1tGn/DWchk6qciKapeVI7dtiwvllZmerZPXpU7XMohKOc\nPav2Pv7wQzUz9YUXoHVr3cI5X36egrMFFJwpIP9MPkXniupM3sdLj9PCs8UVydvf2x//lpdud2zV\nEX9vf3xa+NDMw3m/vBpKOmidiMFwaXKV1cn+hhugTx8bZ2QJYYPKSjUhavp01VTz008OXW21Sqvi\n+PnjlkRecLagVlKvvl1mLsPY2oixjRFjayOdW3emo3dHbva/uVZC9/P2020EjCuTZO9g1TtXjR5t\nQ6HqTlpJ9sLeUlLgD39Q44G/+caGzZLrdrHyIoVnCy1JvGbyrv5ZdK6I1s1bY2pjsiRyY2sjsV1i\nLcnd1MaETwsfl2xOcRWS7B0sOhree8/GQjExMH++Q+IRTdTu3SrJ5+TA22/DPfc0eC5Hzqkc/rnl\nnyzds5RTF08R0CrAksRNbUyYWpsYYBxgSeSdW3eWmrgTkDZ7BztyRG1kcvKkDZ+twkK4+WY4cUIm\nV4nrc+SImsq9ciW88oraWaeB+69mFGXw9ua3ST2YyuRbJvNEvyfo1q6b0ywH0NRIm72T6dRJrXF2\n6JAatmyVzp2hTRs4cAB69HBofKIRVVWpmvXeveo4eFCtMePhoUa/NGvW8Nt1PZadrbbqe+wx2L9f\nrbttI03TSD2Yytub32b/yf08M+AZ5t49lzY3tLH/70c4lCT7RlDdSWt1sgfVlLN5syR7V1RZqRL5\n3r3wyy+Xkvv+/dChg/pTLzwc+vZVtWyzWR1VVVfevvyxigrrz23VSv3HCw62+RIqzBUs+3kZb29+\nmyqtij/G/pHEXok0b9bcAb8w0Rgk2TeC6pm0iYk2FKrupJ040WFxietUVqYWP6qZ0Pfuhaws9ddZ\nRIQ6hg2DZ56BsDBdhzZa42zZWT7O+Jh3trxDaPtQZg2dRXz3eOk4dQOS7BtBdDS89pqNhWJi1Phn\nob8LF1StvGZC37tXNckEBl5K6vfeCy+9BDfdBC0btiGFXgrPFvLe1veYlzGPocFD+eqBr+jXuZ/e\nYQk7kg7aRlBSotY4O3XKhomJFRVqclVeXoPaWsV12rJF7Qe8fbvqMA8JUU0v1Yk9IgJCQ11+k+u9\nx/fyt81/Y/m+5TzY+0F+f+vv3Xa5AHcjHbROyMdHddTu2wc9e1pZyMtLLZu5cKFaiKqiQh2VlZdu\nN+SoWb6yUnXe3X23Q6/fpfzwg5pstG+f2hP4tddUm3cDR7A4I03T2Hh4I29vfpsfC37kqeinyPxd\nJu1bttc7NOFA10z2KSkpPPvss5jNZiZPnsyLL75Y6/l9+/YxceJEduzYweuvv87zzz9veS4wMJA2\nbdrQrFkzvLy8SE9Pt/8VuIjqdnurkz3AI4/Ap5+qVQi9vNTh6Xnp9rWOli1r37+8bFkZPP642nHo\nqaccdu1FEsoxAAAc8ElEQVQuYdMmtS7MgQNqnZhHH1W7LLkRc5WZ5fuW89bmtyi5UMLzMc/z+bjP\nudGr6axP35TV24xjNpvp0aMHa9euxWg0Eh0dzZIlSwgPD7ecc/z4cXJzc1m+fDk+Pj61kn1QUBDb\nt2/H19f36gE0gWYcUC0Chw6pxQSdyqFDag3me+6Bt95qersPff+9SvJZWepL7+GH3S7JX6i4wIKd\nC/jHln/QoWUHXoh9gZE9RrrlejFNia25s95Pdnp6OiEhIQQGBuLl5UViYiIrLttY1c/Pj6ioKLyu\n8mduU0jk1qgeful0goPVqJ+tW9VwoYsX9Y6ocWzcCEOHwkMPqes+cAAmT3arRH+i9AQz0mYQ+G4g\nKQdTmH/vfDY/tpnR4aMl0TdB9TbjFBQU0KVLF8t9k8nE1q1brX5xg8HA0KFDadasGUlJSUyZMqXO\n86ZPn265HRcXR1xcnNXv4SoiI9WM9fJyJ8wn7dvDf/+rmo2GDlXNRu3dtP12wwZVk8/OVjNKH3rI\nZdrjy83lnCw9yYnSE1ceF658rPhCMQ/0fIC0R9II9wu/9hsIp5aWlkZaWlqDy9eb7K93bO2mTZsI\nCAjg+PHjDBs2jLCwMAYNGnTFeTWTvbtq1UpVonfvVv2uTqdFC1iyRG0qHRsLyck2zgJzct99p5J8\nbq5K8g8+qHuSL7lQwtHzR+tO3nUc5yvO0/7G9nRo2aHW4eftR3C7YPp37l/rcX9vf2mPdyOXV4Rn\nzJhhU/l6k73RaCQvL89yPy8vD5MNS6EGBAQAqqln9OjRpKen15nsm4rqFTCdMtmDaq9/6y01dvy2\n22D5crUBuitLS1Oja/LzVZKfMKFRk7y5ykz2qWz2ndh3xVFRVUGnVp2uSN6dWnWil3+vKx5ve0Nb\nmdwkGqzeZB8VFUVmZiY5OTl07tyZZcuWsWTJkjrPvbxtvrS0FLPZTOvWrTl//jypqam8+uqr9ovc\nBVW32ycl6R3JNTz5JHTpooZkzpunJgu5Ek1TSX7GDCgouJTkPR030vhc+Tn2n9ivEvnJSwk9qziL\nTq06EdYhjLAOYUR3juah3g8R1iEMf29/Sd6i0dT7v9/T05PZs2cTHx+P2Wxm0qRJhIeHM3fuXACS\nkpI4cuQI0dHRnDlzBg8PD95991327t3LsWPHGDNmDACVlZVMmDCB4W6yp2VDRUXBr78653fPPaop\n59574fBh+N3v9I7o2jQNvv1WJfmiIrXa429/a7ckr2kaReeKLIn8lxO/WG6fLD3JTe1vIqxDGOEd\nwhkXPo6wDmGEtg+lpZdrzaYV7klm0DaisjI1werECReaTZ+drYZm3n238w7N1DRYv1411xw7ppJ8\nYuJ1JfkKcwWpB1P56ehPtZpeWnq1tNTSax5d23aVpX5Fo5I9aJ1cVJTazCQ2Vu9IbFBcDKNGQceO\nsGgR3OgknX6apjbJnj5dfYNWJ/nr2Cw773QeH2V8xCcZn9DdtzuxXWIJa38pqfvc6GO/+IW4DpLs\nndwTT6glVp55Ru9IbHTxolqB8/BhNTSzQwf9Yikvhy+/VN+ap06pJP/AAw1O8lVaFakHU/lg2wd8\nf/h7Jtw8gSeiniDCL8LOgQthP7I2jpOLjlbNyi6nRQtYvFgtJRAbC6tXN/7QzPx81ekxb55ad+KF\nF2DkyAYn+ePnjzN/53zmbp9LuxbtmBo1lf+M+Q/ezb3tHLgQ+pNk38iio9UWoC7JwwPefBO6dWu8\noZnVI2vmzFHt8hMmqJ/hDZskpGkam/I28eG2D/km8xtGhY1iydglRHeOlpExwq1JM04jq6xUKxYX\nFEDbtnpHcx1WrVLNOh9/rNrz7e3sWfjsM5XkAaZNUxOhGrj5x5myM/x717/5cNuHlJnLeKLfEzzS\n9xF8b7z6uk1CODNpxnFynp7Qp49aJn3wYL2juQ53362ackaOVO34Tz9tn9f95ReV4P/zHxgyRN2+\n444Gb7z+05Gf+GDbB3z+8+cMCR7CO3e+w28CfyO1eNHkSLLXQfVMWpdO9qCGFm3erIZm5uTA3/7W\nsKGZlZWwcqVK7Hv3wpQpsGsX2DBbu6aLlRf54ucv+GDbB+SdyePxWx7n5yd/JqB1QINeTwh3IM04\nOli8WDV3f/GF3pHYSUmJasrx81NNL9YOzTx6VHW2zp2rtvJ66ikYO7bBK8VlFWfx4bYPWfjTQvoF\n9GNq1FTuuukuPD2kTiPcj12XOBaOUb2Ridvw8YHUVLXmzJAhasz71Wia2g1qwgS1AXdOjqrVf/+9\nmu1qY6KvrKrkq1++Yvhnw4n9JBYPgwdbJm0h5cEU7g27VxK9EL+Smr0OqqrU9rKZmaoy7DaqqtQG\nIF9+qZZaCAm59FxpqVpVc84cOHNG1eIffVR9Udj6NloV2wq38fWBr5m/Yz6B7QKZGjWVsRFjaeHZ\nwn7XI4QTkw5aF+DhoVa+3LZNNXe7DQ8PeOMNtWrmoEHw1Vfq2+yDD2DBAoiJgZkzYfhwm9v2iy8U\nk3owleTMZFKyUvDz9mNEyAiSJyTTu2Nvh1yOEO5Ekr1OqlfAdKtkXy0pSXWuJiSoCU8TJ6qLDQqy\n+iWqtCp2HtlJcmYyq7NWs/vobuIC40gITeC137xGt3bdHHgBQrgfacbRyf/9HyxcCF9/rXckDnTs\nmBoXb2WH7amLp/jvwf+yOms1q7NW0+aGNiSEJjAiZAS3d7tdmmiEqEHWxnERublq8mlRUYOHkLs8\nTdPYfWy3pfaeUZTBoK6DLAm+u68b7ZQlhJ1JsncRmqYWkczIaPBwcpd0tuwsaw+tJTkrmdWZq2ne\nrDl33XQXI0JGEBcYJ2u/C2El6aB1EQbDpSGY7pzsNU3jlxO/kJyZTHJmMj8W/kiMKYYRISP4Q8wf\nuKn9TTKbVYhGIMleR9WdtKNH6x2JfZ0pO8P67PWsObiG1Zmr0dBICE3g2VufZXDQYFo1b6V3iEI0\nOZLsdRQdrZZkd3VVWhUZRRmsyVrDmoNr2HFkB7eabiW+ezzfjP+GCL8Iqb0LoTNps9fRkSMQEQEn\nT7peJ+2Rc0dIPZjKmoNrSD2YSoeWHYjvHk9893juCLxD2t6FcDDpoHUxXbqozUxqTjZ1RmWVZWzK\n28Sag2tYk7WG3NO5DA4abEnwMu5diMYlHbQuprrd3tmSvaZpZBVnsebgGlKyUtiQu4Fwv3Diu8cz\nJ2EOA0wDZN0ZIVyI1Ox19sYbat2wv/9d70hqd6yuyVrDxcqLxIfEc2f3OxkaPJT2LdvrHaIQ4ldS\ns3cxUVHw2mv6vHd9HasrElfQy7+XdKwK4SauWbNPSUnh2WefxWw2M3nyZF588cVaz+/bt4+JEyey\nY8cOXn/9dZ5//nmry4LU7EtK1FLup041eN/sOlWYKzhy7gj5Z/IpOFtAwZkCCs4WXHG/W9tultq7\ndKwK4Trs2kFrNpvp0aMHa9euxWg0Eh0dzZIlSwivsdnz8ePHyc3NZfny5fj4+FiSvTVlGxKwOwoN\nVZuZ9Oxp3fnnys9RcKZ24s4/m18roZ8oPYG/tz/G1kaMbYwYWxsxtTFZ7lff9m7u7diLE0I4hF2b\ncdLT0wkJCSEwMBCAxMREVqxYUSth+/n54efnxzfffGNzWaFUd9LWTPaVVZWszlxNemG6Suo1EnmF\nucKSwKsTd4/2PRgcOFgl8TZGOrXqJB2oQgiLerNBQUEBXbp0sdw3mUxs3brVqhe2pez06dMtt+Pi\n4oiLi7PqPdxF9bIJjz4KRWeLmJcxj48yPqJLmy4M7z6cgV0G1qqV+7TwkbZ0IZqYtLQ00tLSGly+\n3mR/PQnFlrI1k31TFBWl8dF/07jvi/dZe2gt9/e8n69/+zV9O/XVOzQhhJO4vCI8Y8YMm8rXm+yN\nRiN5eXmW+3l5eZisXLXreso2FacunmLhzoW8/9OHZHZvxlTTVD4Z+Qltbmijd2hCCDdT795wUVFR\nZGZmkpOTQ3l5OcuWLWPkyJF1nnt5R4EtZZuabYXbmLRyEkHvBrG1YCvzRn5EeNpubmv+lCR6IYRD\n1Fuz9/T0ZPbs2cTHx2M2m5k0aRLh4eHMnTsXgKSkJI4cOUJ0dDRnzpzBw8ODd999l71799KqVas6\nyzZVpRWlLNuzjPe3vc+J0hMk9Uti/7T9+Hv7A9D/107afv10DlQI4ZZkBq2D7T+xnw+3f8hnP33G\nraZbmRo1lTtD7qSZR+1B9e+/rzYymTdPp0CFEC5FZtA6gQpzBSv3r+T9be/z87GfeSzyMbY9vo3A\ndoFXLRMdDb/+wSSEEHYnNXs7yj+Tz8cZHzMvYx4hviFMjZrKmPAxNG/W/Jply8rAx0etk9NSJrEK\nIa5BavaNrEqrYu2htXyw7QO+y/mO8TePZ82Da+jl38um17nhBrW2/Y4dMHCgg4IVQjRZkuwbSNM0\nFuxcwMzvZ9KqeSumRk3ls9GfXdeWe9HRsG2bJHshhP1Jsm+A/Sf2k7QqidKKUhaOWkiMKcYuM1qj\no2H9ejsEKIQQl6l3nL2ordxczmvfvcbATwcyJnwMP0z6gdgusXZbuqB62QQhhLA3qdlbadPhTTy+\n6nGCfYLJSMqga9uudn+PiAgoKIDTp6FtW7u/vBCiCZNkfw2nLp7ipXUvsXL/St69813Gho912CJk\nnp7Qty9s3w6DBzvkLYQQTZQ041yFpml8ufdLer7fE03T+PnJnxkXMc7hq01KU44QwhGkZl+HvNN5\nTFs9jcyTmSwbt4zbut7WaO8dHQ1ffdVobyeEaCKkZl+DucrMv7b+i8i5kfQL6MeOpB2Nmujh0vBL\nIYSwJ6nZ/2rX0V1M+XoKLTxb8P1j3xPWIUyXOEJC1H60x4+Dn58uIQgh3FCTr9lfqLjAS+teYuii\noTx+y+N8+8i3uiV6AA8PtfKltNsLIeypSSf7tYfWcvMHN5Ndks2uqbuYdMskPAz6/0qq96QVQgh7\naZLNOCdKT/DcmufYkLuB9+96n4TQBL1DqiU6GhYs0DsKIYQ70b8a24g0TWPRT4vo9X4v/Lz92PPk\nHqdL9HBp+KWbLAYqhHACTaZmn1WcxROrnqD4QjHfjP+Gfp2dd0uorl2hqgry86FLF72jEUK4A7ev\n2VeYK3jz+ze5dd6tjAgZQfqUdKdO9AAGgwzBFELYl1vX7Hce2ckjyx+hc+vO/DjlR4J8gvQOyWrV\nnbSjR+sdiRDCHbh1zd7D4MGfBv6J5PHJLpXoQZZNEELYl2xL6KSOHoWwMCguVs06QghRk625061r\n9q6sY0do3RoOHtQ7EiGEO5CavRMbM0YtmRAXB/7+6gvA3x/at4dmzfSOTgihJ1tz5zWTfUpKCs8+\n+yxms5nJkyfz4osvXnHO008/zerVq2nZsiULFiwgMjISgMDAQNq0aUOzZs3w8vIiPT39ugNuSrZv\nh4UL4dgx1axT/fP0afD1rf0FUPP25Y/deKPeVyKEsDdbc2e9o3HMZjPTpk1j7dq1GI1GoqOjGTly\nJOHh4ZZzkpOTycrKIjMzk61btzJ16lS2bNliCSYtLQ1fX98GXk7T1q+fOi5XWQknTlz6Aqj5ZZCZ\neeXjXl51fxl07Aj33ad+CiHcW73JPj09nZCQEAIDAwFITExkxYoVtZL9ypUreeSRRwAYMGAAp06d\n4ujRo3T8NYNIrd3+PD2hUyd1XIumwdmztb8Qqm9v3AiffAIbNqj+ASGE+6o32RcUFNClxhROk8nE\n1q1br3lOQUEBHTt2xGAwMHToUJo1a0ZSUhJTpkyp832mT59uuR0XF0dcXFwDLkXUxWCANm3UERJS\n+zlNgyeegAcegJUr1ZeIEMI5paWlkZaW1uDy9X68rd2C72q19++//57OnTtz/Phxhg0bRlhYGIMG\nDbrivJrJXjQegwHmzIF77oFp0+CDD2SYpxDO6vKK8IwZM2wqX+/QS6PRSF5enuV+Xl4eJpOp3nPy\n8/MxGo0AdO7cGQA/Pz9Gjx5dZwet0JenJ3z+OWzZAm+/rXc0QghHqTfZR0VFkZmZSU5ODuXl5Sxb\ntoyRI0fWOmfkyJEsWrQIgC1bttCuXTs6duxIaWkpZ8+eBeD8+fOkpqZy8803O+gyxPVo3RpWrYLZ\ns1XiF0K4n3qbcTw9PZk9ezbx8fGYzWYmTZpEeHg4c+fOBSApKYmEhASSk5MJCQnB29ub+fPnA3Dk\nyBHGjBkDQGVlJRMmTGD48OEOvhzRUCYTfP01DBsGRiMMHKh3REIIe5JJVaKWNWvgkUfUSJ3QUL2j\nEUJcjSyXIK5LfDy89hokJKix/EII9yA1e1Gn//kfSEuDdetkBq4QzsjuyyU4miR751RVBRMmgNkM\nS5eCh/wNKIRTkWYcYRceHjB/PhQVwUsv6R2NEOJ6SbIXV9WiBSxfro4PP9Q7GiHE9ZAJ8qJe7dtD\ncjLcdpvaCD0hQe+IhBANITV7cU3du8P/+3/w6KOwY4fe0QghGkKSvbBKTIxaO+eee6DG6hhCCBch\nzTjCamPHQk6Oasr5/nto21bviISw3ZEj8OWXau+HkBC46SZ1dOniGjvAVVZCfr7t5WTopbCJpqkV\nMjMz4Ztv1MYoQji7oiKV4L/4Anbtgrvugj594NAhOHBAHSdOQHDwpeRffYSGqg1+GmtFWE2D48dV\nbNnZl47q+wUFavOh/HwZZy8crLISRo1Sm6d8/LEsiyycU2HhpQS/ezfcfbfamW34cDXS7HKlpZCV\ndSn5HzigKjUHDkBZWd1fAjfd1LC/cM+erZ3IaybznBwVX1DQpSM4+NLtrl3hhhtkUpVoJOfOwR13\nqE3RX35Z72iEUAoKLiX4PXtUH1N1gr/hhoa/bnHxpcRf/bP68Pa+8gvgpptUwq4rmWdnw/nzV0/m\nQUFqs6FrkWQvGk1REdx6K8ycqWbbCqGHggL4v/9TCX7v3ksJftiw60vw1tA01QdQM/lXfyFcuHD1\nZG6PZiFJ9qJR7dkDgwerD9odd+gdjWgq8vMvJfhffoGRI1WCHzrU8QneWUiyF41u7VpVs//uOwgL\n0zsa+6iqgtOnoaJC3a4+NM1+9w0GNWnN3x98fZ2376N60/rCwtrHqVPg4wMdOtQ+/PxUO7a9r6dm\ngt+3r3aCb97cvu/lCiTZC13Mn6+WRt6yRSUvZ1RRAceOwdGj1/55/Di0bKlqiR4eKnF5eFw6Lr9v\nzTmX36+qUiNAjh5VnYN+furPe39/9bP6uPx+hw722xz+/Pkrk3jNo6hI/fTwgM6d1REQoH62awcl\nJeoaLj/On1dfZJd/EdT8Qrj8sZYtr/yCyMu7lOD374d771UJfsiQppnga5JkL3Tzv/8Lqamwfr36\n4DqapqmkcvSodQn87FmVZKqTZ30//f0bN5mUlV0Z7+XXVX0UF6sa9dW+DKrvt22rvrTqS+bl5Wpn\nsupEXtcREKC2rrRFeTmcPFn3F0HN4/jxSz+hdvI/cwYOHryU4AcPlgRfkyR7oRtNU7tcnTunamL2\nmKBSWlp7NEP1iIZDh9QQtaqqKxPe1ZK4j497LNVsNl/6i6C+L4bTp9W115fIHdHc0lClpbW/CDw9\nYdAgmctxNZLsha7Ky9VuV5GR8I9/XPt8s1mNprg8kVf/PH0aAgMvjWioHtUQHKwel1m8oqmSZC90\nV1ICsbHw5JNqtm1JSd2JPDsbDh9WTSs1k3jNxN6pk3vUxoWwN0n2wilkZ8PAgepPc027MolX3+7W\nre7ZjEKI+kmyF07j9GnVTOPj4zztwkK4C7tvS5iSkkJYWBihoaHMmjWrznOefvppQkND6dOnDztq\nLHhuTVlXl5aWpncI18WR8bdt6/jx4/L7148rxw6uH7+t6k32ZrOZadOmkZKSwt69e1myZAm//PJL\nrXOSk5PJysoiMzOTjz76iKlTp1pd1h24+n8YiV9frhy/K8cOrh+/repN9unp6YSEhBAYGIiXlxeJ\niYmsWLGi1jkrV67kkUceAWDAgAGcOnWKI0eOWFVWCCFE46g32RcUFNClSxfLfZPJREFBgVXnFBYW\nXrOsEEKIxlHvpGuDlY2t19vBau37OKsZM2boHcJ1kfj15crxu3Ls4Prx26LeZG80GsmrseFoXl4e\nJpOp3nPy8/MxmUxUVFRcsyxc/xeFEEKIa6u3GScqKorMzExycnIoLy9n2bJljBw5stY5I0eOZNGi\nRQBs2bKFdu3a0bFjR6vKCiGEaBz11uw9PT2ZPXs28fHxmM1mJk2aRHh4OHPnzgUgKSmJhIQEkpOT\nCQkJwdvbm/nz59dbVgghhA40Ha1evVrr0aOHFhISor355pt6hmKzw4cPa3FxcVpERITWs2dP7d13\n39U7JJtVVlZqffv21e6++269Q7FZSUmJNnbsWC0sLEwLDw/XfvjhB71DssnMmTO1iIgIrVevXtpv\nf/tb7eLFi3qHVK+JEydq/v7+Wq9evSyPnTx5Uhs6dKgWGhqqDRs2TCspKdExwvrVFf8f/vAHLSws\nTOvdu7c2evRo7dSpUzpGWL+64q/2t7/9TTMYDNrJkyfrfQ3dVh1x9XH4Xl5e/POf/+Tnn39my5Yt\nzJkzx6XiB3j33XeJiIhwyQ7yZ555hoSEBH755Rd27drlUn815uTk8PHHH5ORkcHu3bsxm80sXbpU\n77DqNXHiRFJSUmo99uabbzJs2DAOHDjAkCFDePPNN3WK7trqin/48OH8/PPP/PTTT9x000288cYb\nOkV3bXXFD6ov9L///S/dunW75mvoluxdfRx+p06d6Nu3LwCtWrUiPDycwsJCnaOyXn5+PsnJyUye\nPNnlOslPnz7Nxo0beeyxxwDVZNjWhZa/bNOmDV5eXpSWllJZWUlpaSlGo1HvsOo1aNAgfHx8aj1W\nc47NI488wvLly/UIzSp1xT9s2DA8fl1lb8CAAeTn5+sRmlXqih/gueee46233rLqNXRL9taM4XcV\nOTk57NixgwEDBugditV+//vf8/bbb1v+s7uS7Oxs/Pz8mDhxIrfccgtTpkyhtLRU77Cs5uvry/PP\nP0/Xrl3p3Lkz7dq1Y+jQoXqHZbOjR4/SsWNHADp27MjRo0d1jqjhPv30UxISEvQOwyYrVqzAZDLR\nu3dvq87X7ZPuik0HdTl37hzjxo3j3XffpVWrVnqHY5VVq1bh7+9PZGSky9XqASorK8nIyODJJ58k\nIyMDb29vp25CuNzBgwd55513yMnJobCwkHPnzrF48WK9w7ouBoPBZT/Tr7/+Os2bN2f8+PF6h2K1\n0tJSZs6cWWuewLU+y7ole2vG8Du7iooKxo4dy4MPPsioUaP0DsdqmzdvZuXKlQQFBfHb3/6W9evX\n8/DDD+sdltVMJhMmk4no6GgAxo0bR0ZGhs5RWW/btm3ExsbSvn17PD09GTNmDJs3b9Y7LJt17NiR\nI0eOAFBUVIS/s24+XI8FCxaQnJzscl+2Bw8eJCcnhz59+hAUFER+fj79+vXj2LFjVy2jW7J39XH4\nmqYxadIkIiIiePbZZ/UOxyYzZ84kLy+P7Oxsli5dyuDBgy1zJVxBp06d6NKlCwcOHABg7dq19OzZ\nU+eorBcWFsaWLVu4cOECmqaxdu1aIiIi9A7LZiNHjmThwoUALFy40KUqPKBW5X377bdZsWIFLVxs\nU4Wbb76Zo0ePkp2dTXZ2NiaTiYyMjPq/cB0wSshqycnJ2k033aR1795dmzlzpp6h2Gzjxo2awWDQ\n+vTpo/Xt21fr27evtnr1ar3DsllaWpp2zz336B2GzXbu3KlFRUW5xLC5usyaNcsy9PLhhx/WysvL\n9Q6pXomJiVpAQIDm5eWlmUwm7dNPP9VOnjypDRkyxCWGXl4e/yeffKKFhIRoXbt2tXx+p06dqneY\nV1Udf/PmzS2//5qCgoKuOfRS981LhBBCOJ7rDcUQQghhM0n2QgjRBEiyF0KIJkCSvRBCNAGS7IUQ\nogmQZC+EEE3A/wdyKUYaLoJ3rQAAAABJRU5ErkJggg==\n"
}
],
"prompt_number": 9
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Cross Validation\n",
"----------------\n",
"\n",
"Since in a real setup we don't have access to the real $f$ function. We cannot exactly calculate the error, hoevere we can approximate it using cross validation. We generate to samples, a training sample and a validation sample. The validation sample is use to calculate an estimation of the error."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"n_samples = 20\n",
"# train sample\n",
"train_x, train_y = sample(n_samples)\n",
"# validation sample\n",
"test_x, test_y = sample(n_samples)\n",
"max_degree = 20\n",
"test_error_vals = []\n",
"train_error_vals = []\n",
"for degree in xrange(1, max_degree):\n",
" model = fit_polynomial(train_x, train_y, degree)\n",
" p_y = apply_polynomial(model, train_x)\n",
" train_error_vals.append(pl.norm(train_y - p_y)**2)\n",
" p_y = apply_polynomial(model, test_x)\n",
" test_error_vals.append(pl.norm(test_y - p_y)**2)\n",
"pl.plot(range(1, max_degree), test_error_vals, label='test error')\n",
"pl.plot(range(1, max_degree), train_error_vals, label='train error')\n",
"pl.legend()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 10,
"text": [
"<matplotlib.legend.Legend at 0x107b98250>"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD9CAYAAAChtfywAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVXX+x/HXRRBUFkHhgiCiJiqLirmMrVdZ0kqiLLPF\nsGyxqWYcaxzLccKaCtscW2amKRvJZtxqRtHU0gxz8ldaarmCuSLCRWQTFJHL+f1x5MYq272cu3ye\nj8d53MvlcO7H671vDt/zXXSKoigIIYSwWy5aFyCEEKJ9JMiFEMLOSZALIYSdkyAXQgg7J0EuhBB2\nToJcCCHsXLNBnpmZSUxMjHnz8fHhrbfeorCwkPj4eMLDw0lISKC4uLgj6hVCCFGPrjX9yKurqwkO\nDmbHjh28/fbb9OzZk9mzZ7NgwQKKiopITU21Zq1CCCEa0aqmlc2bN3PVVVfRu3dv0tPTSU5OBiA5\nOZnVq1dbpUAhhBBX1qogX758Offccw8ARqMRvV4PgF6vx2g0Wr46IYQQzWpx00plZSXBwcEcOHAA\nf39/fH19KSoqMn/fz8+PwsLCugfX6SxbrRBCOInWzJ7S4jPyDRs2cPXVV+Pv7w+oZ+F5eXkA5Obm\nEhAQ0GQxsrV/e/755zWvwZE2eT3l9bTlrbVaHOTLli0zN6sAJCYmkpaWBkBaWhpJSUmtfnIhhBDt\n16IgLy8vZ/Pmzdxxxx3mx+bMmcOmTZsIDw9ny5YtzJkzx2pFCiGEaJprS3bq1q0bBQUFdR7z8/Nj\n8+bNVilKNGQwGLQuwaHI62lZ8npqq1X9yFt9cJ2uTe09QgjhzFqbnS06IxdC2B8/P786PcuE7fH1\n9W3Q268t5IxcCAclnz/b19T/UWv/72x60qzcXKiu1roKIYSwbTZ7Rl5VBX36wNmz0Ls3hIVB377q\nbe37ej242PSvIyG0IWfkts9SZ+Q2G+Q1LlyAkyfh2DE4flzdat8vLYXQ0MZDPiwMAgJABpgKZyRB\nbvucJsibU14OJ040HvJHj8K4cbBqlVVLEMImSZDbPgnyFsjNhehoOHNGzsqF89H683clYWFhfPjh\nh4wbN65dx1myZAmLFy9m27ZtFqqsYznFxc72CgxUb2ViRiFsi63+kjGZTA0eq25lj4vW7m8JDh3k\nOh1ERcG+fVpXIoSoMXXqVE6ePMnEiRPx8vLi9ddfB+Dbb7/lmmuuwdfXl2HDhrF161bzzyxZsoT+\n/fvj7e1Nv379+Pe//82hQ4eYMWMG//d//4eXlxd+fn6NPl9JSQnTp0+nV69ehISEMG/ePHPYLlmy\nhGuvvZZZs2bRs2dPUlJSePDBB3n88ce5+eab8fT0JCMjg4MHD2IwGPD19SUqKoq1a9eajz9t2rQG\n+3c4xYqsfPgWeeIJRVm4UOsqhOh4tvD5a0pYWJjy5Zdfmr8+deqU0qNHD2XDhg2KoijKpk2blB49\neigFBQVKWVmZ4u3trWRlZSmKoih5eXnK/v37FUVRlCVLlijXXXfdFZ8rKSlJmTFjhnL+/HklPz9f\nGTVqlPLee+8piqIo//znPxVXV1flnXfeUUwmk3LhwgUlOTlZ8fHxUbZv364oiqKUlpYq/fv3V155\n5RXl0qVLypYtWxQvLy8lMzNTURSlwf4VFRUtfh2a+j9q7f+dQ5+Rg9pGvnev1lUIYXt0OstslvDx\nxx9z8803M378eADi4uIYMWIEn332GTqdDhcXF/bu3cuFCxfQ6/VEREQAzc/ZbTQa2bBhAwsXLqRL\nly74+/szc+ZMli9fbt6nV69ePPHEE7i4uODh4YFOpyMpKYkxY8YAsGfPHsrLy5kzZw6urq6MHTuW\nW2+9lWXLlpmPUXt/d3d3y7woreDwQS5NK0I0TlEss1nCiRMnWLVqFb6+vubtm2++IS8vj65du7Ji\nxQr+/ve/06tXL2699VYyMzNbfNxLly4RFBRkPu6MGTM4c+aMeZ/evXs3+LmQkBDz/dOnTzfYp0+f\nPpw+fRpQ2/sbO0ZHcvi5ViIjYf9+dYSoDBwSwjbUXz0sNDSUqVOn8o9//KPR/RMSEkhISODixYvM\nnTuXRx55hK+//rrZVch69+6Nu7s7Z8+exaWJAGjsGLUf69WrF9nZ2SiKYn78xIkTDBo06IrP3ZEc\nPtq6dwdfX7VfuRDCNuj1eo4cOWL++v7772ft2rV88cUXmEwmKioqyMjIICcnh/z8fNasWUN5eTlu\nbm5069aNTp06mY9z6tQpLl261OjzBAUFkZCQwKxZszh37hzV1dUcOXKEr7/+usna6jfX/OpXv6Jr\n1668+uqrXLp0iYyMDNatW8eUKVMa3V8LDh/koLaTS/OKELbj2Wef5c9//jO+vr68+eabhISEsGbN\nGl5++WUCAgIIDQ3ljTfeQFEUqqurWbhwIcHBwfTo0YNt27bxt7/9DYDY2FgiIyMJDAxscrnJjz76\niMrKSiIiIvDz8+Ouu+4yL1Op0+kanJHXf8zNzY21a9eyYcMG/P39efLJJ1m6dCnh4eFNHqOjOfSA\noBqzZ6tn5s89p3UlQnQcW/n8iabJgKBWiIqSnitCCMflFEEuTStCCEfmFE0rFy6Anx+UlEDnzlpX\nI0THsJXPn2iaNK20Qpcu6lS3WVlaVyKEEJbnFEEO0rwihHBcThPkMsJTCOGoWhTkxcXF3HnnnQwe\nPJiIiAi+++47CgsLiY+PJzw8nISEBIqLi61da7tIzxUhhKNqUZD/9re/5eabb+bgwYP89NNPDBo0\niNTUVOLj48nKyiI2NpbU1FRr19ou0rQihHBUzfZaKSkpISYmhqNHj9Z5fNCgQWzduhW9Xk9eXh4G\ng4FDhw7VPbgNXTWvqgJvb3W1oG7dtK5GCOuzpc+fNTz++OMEBwfzxz/+UetS2sxSvVaanTTr2LFj\n+Pv78+CDD/Ljjz9y9dVX85e//AWj0YherwfU+Q6MTSzDk5KSYr5vMBgwGAwtLs6SXF1h4EB1Aq1R\nozQpQQhxmSWWeqsZpu8IMjIy2rUgRbNn5N9//z1jxoxh+/btjBw5kpkzZ+Ll5cU777xDUVGReT8/\nPz8KCwvrHtzGzggeeAAMBnjoIa0rEcL6bO3zV1vfvn354IMPiI2NbfT7VVVVuLraxuSsJpPJPElX\njerq6iZnU2xMU/t3WD/ykJAQQkJCGDlyJAB33nknu3btIjAw0DzxTG5ubpMT1tgS6bkihPYaW+rt\n+PHjuLi48OGHH9KnTx/i4uIAuOuuuwgKCqJ79+7ceOONHDhwwHycadOmMW/ePEA9ow0JCeHNN99E\nr9fTq1cvlixZ0mQNjrb8W7NBHhgYSO/evcm6PJpm8+bNREZGMnHiRNLS0gBIS0sjKSnJqoVagvRc\nEUJ7S5cuJTQ0lHXr1nHu3DmeeeYZ8/e+/vprDh06xOeffw7ALbfcws8//8yZM2cYPnw49913n3nf\n+rMOGo1GSktLOX36NIsXL+aJJ56gpKSk0RqmTZtG586dOXLkCLt37+aLL77ggw8+MH9/x44d9O/f\nn/z8fObOnYuiKCxbtox58+ZRVlbGyJEjmThxIuPHj+fMmTO8/fbb3HfffeacBOrsf+2111rs9WtM\ni/52qSmysrKS/v37889//hOTycTkyZNZvHgxYWFhrFy50qqFWoL0XBHiF7r5lpl6VXnecs03KSkp\ndOnSxfz1tGnTzPeff/55Fi1axLlz5/Dy8lKfu1bzg5ubG3/6059wcXFhwoQJeHp6kpmZyah6F8Vq\nln8rLi7Gw8ODLl26MHPmTN5//30effRR4Jfl34Bml38D6iz/9vzzzwMdu/xbi4J86NCh7Ny5s8Hj\nmzdvtnhB1hQSAufPQ0EB9OypdTVCaMuSAWwptZdMq66u5rnnnuOTTz7hzJkz5jbmgoICc5DX1qNH\njzrt0F27dqWsrKzBfrWXf6v9XKGhoY3WUaO1y7/V3t/abONqQgfR6X5pJ9eo84wQgsaXV6v/+L/+\n9S/S09P58ssv6dOnD8XFxfj5+dU5C2/Lgg6OuPyb0wzRryHNK0Jor/5Sb40pKyvD3d0dPz8/ysvL\nea7eyjCKorSpV44jLv/mdEEuPVeE0F79pd6g4VnwAw88QJ8+fQgODiYqKooxY8bU2af+xc7WnJ07\n2vJvTjEfeW0ZGfDHP8L//qd1JcKeXe6pRiu6Enc4W/z8ibos1Y/c6YK8oACuugqKitQ2cyFa68gR\nuOMOOHBAvWju7w8BAc1v3bp17HvOFj9/oq4OG6LvaHr2VBeaOHUKGrkwLcQVbdwIycnwpz/Bo4+q\nJwb5+Q23I0fqfl0zg0XtYA8MhHvvhbFj5aRCtI/TBTn80k4uQS5aqroaXnkF/vpX+OQTuP569fGg\nIHVrifJyNdTPnPkl7H/9a3Uyt9mz4fbbod5IcCFaxOmaVgB+9zv1wzd7ttaVCHtQWqqeheflqSEe\nHGy5Y1dXQ3o6LFgAZ8/C738PU6eCh0f7j+3n51dnPiRhe3x9fRvMUQWyZmeLSBdE0VKZmTB6NOj1\n6oVyS4Y4qBdLk5Jg+3b44ANYvRr69VODvYnR5S1WWFho7qInm21ujYV4m95HFjmKnZEuiKIl1qxR\nm1Cefhr+/new5ihrnQ5uuAE++ww+/1x9f/brp/7VeHmwoBBNcsogj4iAQ4fUxSaEqK+6Wr2Y+dRT\nsHYtPPxwxz5/dDQsXQo//AAVFeqJx8MPq38dCNEYp2wjB+jfH9avVxebEKJGcTHcdx+UlcHKlWqT\nitYKCuCdd9QLrdddB3/4g9rc0xqVlWpPrZMnf9lOnFBvz50DT8+6W7duDR+70ta5s/S8sSTpR95C\nt92mLjQxaZLWlQhbsW+f2nPkllvgtdfAzU3riuoqL4fFi+GNN35pdhk/Xv1eYWHdkK4d1CdPqr8M\nevWC0FDo00e9rdm8vdVjl5W1bqv5mXPnQFFaHvpeXo0/HhCg/uL09JRfChLkLTR3rvpBrbUSnXBi\nq1apXQEXLoT779e6miu7dAlWrPilp0tJiXpGXDuca7aa0A4Ksm7XxsrKusHe2l8KpaVqt8zLo+QJ\nDFRDvfZtY/drzXjrUCTIW2j5crUr2SefaF2J0FJVlfpLfeVK+PRTGD5c64paTlHg55/VQPP21roa\nyykrUwPdaFRva9+v/5i7+y/h3qsXhIer28CB6mavr4sEeQvt2wd33qle9BTO6exZuDxZHcuWyRz1\n9kZR1L9GakL91CnIylIvCmdmwuHDajNO7WCvud+3r+01ndUmQd5ClZXg46O2LTrqn2eiabt3q/Ol\nTJ4ML70ENrLOr7AgRYGcHDXUawd8ZqbapbNPn4YBHxkJPXpoXbkEeatER8NHH0FMjNaViI705Zfq\nmfi776pBLpxPRYU6RUL9kN+3T/3LbORIGDFCvR0+vOObaCTIW+Gee+Dmm9Uh0cI5FBbCkCGwZAlc\nXqhdCLPqajXYd+6E779Xb3/8Ub1gXDvchw2z7l/yEuSt8NJLahvbq69qXYnoKPfeq047u2iR1pUI\ne3Hpkjplce1wP3hQbY6pHe7R0ZZrd5cgb4X0dHXo9fr1WlciOsKKFfD887BrF3TtqnU1wp5VVMBP\nP6mhXhPwx46po3AXLoRrrmnf8SXIW+HoUXUR5pMnta5EWNvp0+q1kHXr1LMnISytrEy9iD5ggNod\nsj0kyFuhulq9iHHqFHTvrnU1wloURb0WMnq0DAAT9sEq09iGhYUxZMgQYmJiGDVqFKBOkRkfH094\neDgJCQkUFxe3rWINubio3Y3279e6EmFN772nDlGfO1frSoSwjhYFuU6nIyMjg927d7Njxw4AUlNT\niY+PJysri9jYWFJTU61aqLXIlLaO7eef1cW2P/rItgeACNEeLZ7Gtv5pfnp6OsnJyQAkJyezevVq\ny1bWQaKiYO9erasQ1mAyqROjzZsHgwdrXY0Q1tOi8Ww6nY64uDg6derEY489xiOPPILRaER/eY5P\nvV6PsWZ12XpSajVKGgwGDAZDu4u2pOhodQEB4XhefVXt6/vUU1pXIsSVZWRkkJGR0eafb9HFztzc\nXIKCgjhz5gzx8fG8/fbbJCYm1lkP0M/Pr8GyRbZ+sRPUeRqiotSZ15x96kxHsmcPxMerizOEhmpd\njRCtY5WLnUGXlwn39/fn9ttvZ8eOHej1evIuzzmZm5tLQEBAG8rVXs3CATXTZwr7V1GhjtZ94w0J\nceEcmg3y8+fPc+7cOQDKy8v54osviI6OJjExkbS0NADS0tJISkqybqVWotPJYsyO5k9/UkfdydQL\nwlk020ZuNBq5/fbbAaiqquK+++4jISGBESNGMHnyZBYvXkxYWBgrV660erHWUtNzJT5e60pEe23b\nBh9/rM6PIU1lwlk49YCgGu+9B999Bx9+qHUloj3OnYOhQ9V5VCZO1LoaIdrOKm3kjk6aVhzDrFkw\nbpyEuHA+Mp0+6ujOAwfUIfsu8qvNLq1bB5s3q00qQjgbiS3UlYL8/NTZy4T9OXMGHn0U0tLsd41G\nIdpDgvwyGapvnxQFZsyA++6DG27QuhohtCFBfpm0k9unjz9Wl+h68UWtKxFCOxLkl8mcK/YnO1u9\nwLl0KXh4aF2NENqRIL9MmlbsS3U1PPgg/O53sni2ENKP/LKKCvD1Vdfw7NxZ62pEc956C5YtUwcA\nuUrfK+FgpB95G3l4QJ8+anursG2HDsELL6hzjEuICyH9yOuoaV6Jjta6Esd04QJkZalrG5pMUFXV\nttslS9SLmwMGaP0vEsI2SJDXIj1XLKOyUg3sffvUZfRqbrOzoV8/td++qyt06tS223vvVbscCiFU\nEuS1REWpg0pEy5hMcOSIGtS1Q/voUbWZKipKHTV7773q7YABstyaENYgFztrycyECRPUIHI0iqK2\nLVdWqgHckq2mKaP2lpPzS2hnZkJgoBrYNaEdFQUDB0p3QCHao7XZKUFei8kEXl6Qnw+enlpXY1mv\nvw6vvaYupNGpU8u3miaNmk2vV5ugIiPVdTAd7XUSwhZIkLfT8OHwt7/B6NFaV2I5RUXqQgtffy2L\nEAthD6T7YTs54sCgBQsgKUlCXAhHJRc763G0niunTsH778NPP2ldiRDCWuSMvB5Hm3MlJQUeeQSC\ng7WuRAhhLXJGXo8jNa0cPAhr1qh9uoUQjkvOyOsJCVHnXTlzRutK2u+55+APf1DnkBFCOC4J8np0\nOsc4K9++HX74AZ58UutKhBDWJkHeCHsPckWBOXNg/nwZmCOEM5Agb4S991z57DMoLIQHHtC6EiFE\nR2hRkJtMJmJiYpg4cSIAhYWFxMfHEx4eTkJCAsXFxVYtsqPZc88Vk0k9G3/5ZXUkphDC8bUoyBct\nWkRERAQ6nQ6A1NRU4uPjycrKIjY2ltTUVKsW2dFqmlbsbFAqoK5h2b07XP6dK4RwAs0G+alTp1i/\nfj0PP/ywechoeno6ycnJACQnJ7N69WrrVtnBevRQ5xDJzta6ktapqIB589SRnJd/5wohnECz/ch/\n97vf8dprr1FaWmp+zGg0otfrAdDr9RiNxiZ/PiUlxXzfYDBgMBjaXm0HqmleCQ3VupKWe/dddf3K\na6/VuhIhRGtkZGSQkZHR5p+/YpCvW7eOgIAAYmJimnwSnU5nbnJpTO0gtyc1zSu33KJ1JS1TXKye\niX/1ldaVCCFaq/5J7vz581v181cM8u3bt5Oens769eupqKigtLSUqVOnotfrycvLIzAwkNzcXAIC\nAtpUfHMURbniLwlrio6GLVs0eeo2efVVtV08MlLrSoQQHe2KbeQvv/wy2dnZHDt2jOXLlzNu3DiW\nLl1KYmIiaZeX0klLSyMpKcnihZVXljPy/ZEcKTxi8WO3hD31XDl9Gt57T51XRQjhfFrVj7zm7HjO\nnDls2rSJ8PBwtmzZwpw5cyxeWLfO3Xhk+CPc9PFNGMuaboO3logIdQWcqqoOf+pWmz8fpk+H3r21\nrkQIoQWbX1jiha0vsPrQajKmZeDt7m2hylqmf391cM2gQR36tK1y6BBcf736S8fPT+tqhBCW4HAL\nS8y7YR5jeo8haXkSFVUVHfrc0dG237wydy4884yEuBDOzOaDXKfT8db4t+jRtQdT/zsVU7Wpw57b\n1udc+fZb+O47+M1vtK5ECKElmw9ygE4unfj49o8pvFDIUxue6rB1QNsT5NXV6uyDCxaovUn+8Q91\n+LylKIo6Re38+dCli+WOK4SwP3YR5ADuru789+7/8u2pb3lh6wsd8pytbVo5elQN7MmTISAA7r8f\ncnLg7rvVofMjR8K2bZapbcMGyM+HywNshRBOzOYvdtZnLDNy7YfX8sw1zzBjxAyLHru+ykrw9lZX\noW/srPfsWbWv+ebN6lZeDnFxEB8PsbHqIhU1FAVWroTf/14defnqq23vZWIyqSM4X3hBXVRZCOFY\nHO5iZ316Tz1fTP2CF79+kU8PfGrV5+rcGQYMUJdMA3Uuk82b1dkFR4yAvn1hyRK1V8uaNZCbq555\nJyfXDXFQ5z65+261l0l4uBrEL74IFy60vq5//1udC+a229r9TxRCOAC7OyOvsSdvDwlLE1hx5wrG\n9h1rlecAuPde9cy8pES9uBgdrZ51x8XBr36lhn1bHD+unp3v3Amvvw6TJrVsoquLF2HgQFi6VO12\nKIRwPK3NTrsNcoCvjn3F3Z/czef3f05MUIxVnuOzz2DjRjW4DQbw8bHs8b/6Cn77W3XGxUWLYMiQ\nK+//l7/Al1/C2rWWrUMIYTucKsgBPj3wKb/Z+Bu+nvY1/f36W/W5rKWqSr1ImpICd92ltn336NFw\nv5IStVnmyy/VHjVCCMfk8G3k9U2KmMS8G+ZpNpTfElxd4de/VtvPdToYPFidkrb+9ACvvQYTJkiI\nCyHqsvsz8hrzM+azJnONJkP5LW3vXrW55cwZtbll3Dj1QmpUFOzebV9zpAshWs/pmlZqKIrCE+uf\nIPNsJuvvXY+7q3uHPK+1KAr85z/q8Pvhw8HNTe0J8/rrWlcmhLA2pw1yAFO1iSmfTgFg+aTldHKx\n/9WHL1yAN95Qe6ls395427kQwrE4dZADXKy6yIR/TWCw/2DemfCOZgtTCCFEWzndxc763F3dWT1l\nNduzt/Pi1y9qXY4QQlidwwU5gLe7Nxvu28BHP37EB7s+0LocIYSwKodrWqltd+5uklYkcfy3x6WJ\nRQhhN5y+aaW2YYHDMFWbOFx4WOtShBDCahw6yHU6HXH94th8dLPWpQghhNU4dJADxPeLZ9PRTVqX\nIYQQVuPwQR7XL46M4xlUVVc1v7MQQtghhw9yvaeeUJ9Qdubs1LoUIYSwCocPcpDmFSGEY7tikFdU\nVDB69GiGDRtGREQEzz77LACFhYXEx8cTHh5OQkICxcXFHVJsW0mQCyEcWbP9yM+fP0/Xrl2pqqri\nuuuu4/XXXyc9PZ2ePXsye/ZsFixYQFFREampqQ0PrnE/8hrnL51H/7qe07NO4+XupXU5QghxRRbv\nR961a1cAKisrMZlM+Pr6kp6eTvLl5duTk5NZvXp1G8vtGF3dujIqeBRbT2zVuhQhhLA41+Z2qK6u\nZvjw4Rw5coTHH3+cyMhIjEYjer0eAL1ej9HY9IIOKSkp5vsGgwGDwdDuotsirm8cm45u4tbwWzV5\nfiGEaEpGRgYZGRlt/vkWD9EvKSnhpptu4pVXXuGOO+6gqKjI/D0/Pz8KCwsbHtxGmlYAvj/9PQ/8\n9wEOPHFA61KEEOKKrDZE38fHh1tuuYUffvgBvV5PXl4eALm5uQQEBLS+0g4WExhDfnk+p0pPaV2K\nEEJY1BWDvKCgwNwj5cKFC2zatImYmBgSExNJS0sDIC0tjaSkJOtX2k6dXDoxru84Ga4vhHA4V2wj\nz83NJTk5merqaqqrq5k6dSqxsbHExMQwefJkFi9eTFhYGCtXruyoetulphvitGHTtC5FCCEsxqGn\nsa3vePFxRn8wmtync3HROcVYKCGEHZJpbK8grHsY3u7e7DXu1boUIYSwGKcKckCmtRVCOBynC3IZ\nri+EcDROF+Tj+o7jm+xvqKiq0LoUIYSwCKcL8u4e3Yn0j2R79natSxFCCItwuiAHiO8vzStCCMfh\nnEHeL55NRyTIhRCOwSmD/Fchv+Jw4WHOnj+rdSlCCNFuThnknTt15vrQ6/ny2JdalyKEEO3mlEEO\n0p9cCOE4nDbIa/qT29IUAkII0RZOG+QR/hFUmio5UnRE61KEEKJdnDbIdTodcf3ipPeKEMLuOW2Q\ngwzXF0I4BqcO8rh+cXx1/Cuqqqu0LkUIIdrMqYM80DOQ3t69+f7091qXIoQQbebUQQ7SDVEIYf+c\nPsilnVwIYe+cPshv6HMDu3J3UVZZpnUpQgjRJk4f5N06d2NErxFsPb5V61KEEKJNnD7IQZpXhBD2\nTYIcCXIhhH2TIAeGBw0nryyPnNIcrUsRQohWazbIs7OzGTt2LJGRkURFRfHWW28BUFhYSHx8POHh\n4SQkJFBcXGz1Yq2lk0snxvUdJ90QhRB2qdkgd3NzY+HChezfv59vv/2Wd999l4MHD5Kamkp8fDxZ\nWVnExsaSmpraEfVaTVzfODYfkyAXQtifZoM8MDCQYcOGAeDp6cngwYPJyckhPT2d5ORkAJKTk1m9\nerV1K7Wy+P7xbD66Waa1FULYHdfW7Hz8+HF2797N6NGjMRqN6PV6APR6PUajsdGfSUlJMd83GAwY\nDIY2F2tN/Xz70dWtK/vy9xGtj9a6HCGEE8nIyCAjI6PNP69TWngKWlZWxo033si8efNISkrC19eX\noqIi8/f9/PwoLCyse3Cdzq7OcGesm0F4j3BmjZmldSlCCCfW2uxsUa+VS5cuMWnSJKZOnUpSUhKg\nnoXn5eUBkJubS0BAQBvKtS3SDVEIYY+aDXJFUZg+fToRERHMnDnT/HhiYiJpaWkApKWlmQPeno3r\nO45vTn7DxaqLWpcihBAt1mzTyv/+9z9uuOEGhgwZgk6nA+CVV15h1KhRTJ48mZMnTxIWFsbKlSvp\n3r173YPbWdMKwOgPRpMam8rYvmO1LkUI4aRam50tbiPviGJswdwtcwF4adxLGlcihHBWVmkjdybx\n/eJlHU8ZgsRNAAANqElEQVQhhF2RIK9nTMgYDhUcovBCYfM7CyGEDZAgr8fd1Z3rQq9jy7EtWpci\nhBAtIkHeCOmGKISwJxLkjYjvL+3kQgj7IUHeiEj/SCqqKjhSeETrUoQQolkS5I3Q6XTE9YuT5hUh\nhF2QIG9CXL84mZ9cCGEXJMibENcvji3HtmCqNmldihBCXJEEeRN6efWil1cvfsj9QetShBDiiiTI\nr0B6rwgh7IEE+RVIf3IhhD2QIL+CG/vcyA+5P1BWWaZ1KUII0SQJ8ivo1rkbVwddzYbDG+xuFkch\nhPOQaWyb8e+9/2bmxpm46FwYFTyKkb1GMjJ4JCN7jaRH1x5alyeEcEAyH7kVKIrCyZKT7Dy9kx05\nO9h5eic/nP6Bnl17mkN9VPAohgcNx7Ozp9blCiHsnAR5B6lWqsksyGTn6Z3qlrOTn4w/0c+3X51w\nH6IfQudOnbUuVwhhRyTINVRpqmSvcW+dcD9SdIQI/wiCvYLp7tEdHw8f9dbdBx93nwaP1Xwt4S+E\n85IgtzHlleXsydtDfnk+xRXFlFwsMd+WVJTUfayixHzfzcUNH49fwj6gWwC3ht/KHYPvoGfXnlr/\ns4QQViRB7gAUReFC1YU64X6i+AT/OfQfNv68kWt6X8OUyCkkDUrCx8NH63KFEBYmQe7gyirLWJu5\nlhX7V7Dl2BbG9R3H3ZF3M3HgRLnQKoSDkCB3IsUVxaw5tIbl+5ezPXs7468az92RdzPhqgl0ceui\ndXlCiDaSIHdSBecL+M/B/7Bi/wp25e7i1vBbmRI5hfj+8XLhVAg7Y/Egf+ihh/jss88ICAhg7969\nABQWFnL33Xdz4sQJwsLCWLlyJd27d293McIy8sry+OTAJyzft5xDBYdIGpTElKgpGMIMuLq4al2e\nEKIZFg/ybdu24enpyQMPPGAO8tmzZ9OzZ09mz57NggULKCoqIjU1td3FCMvLLslm5f6VLN+/nO9P\nf9/u43l19qKXVy+CvILUW896t5cfl/Z6IdrOKk0rx48fZ+LEieYgHzRoEFu3bkWv15OXl4fBYODQ\noUPtLkZYV3v/LxQUSi+Wknsul9PnTpNbVu+21uMuOpcGAR/kqW7uru51jqtD1+C5dDpdk/v4dfHj\nhj43NNhHCEfR2uxs09/ZRqMRvV4PgF6vx2g0NrlvSkqK+b7BYMBgMLTlKYUFtDf4dOjo7tGd7h7d\nGew/uMn9FEXhXOW5BuF++txpduXuotJU+cu+KA1+tsHx6u2zL38f1/S+hr/e/Fe6de7Wrn+TELYg\nIyODjIyMNv98m87IfX19KSoqMn/fz8+PwsLChgeXM3JhBeWV5fx6/a/ZmbOTVXetIjIgUuuShLCo\n1mZnm6axrWlSAcjNzSUgIKAthxGiTbp17saS25bwzDXPcOOSG0nbk6Z1SUJoqk1BnpiYSFqa+uFJ\nS0sjKSnJokUJ0RydTsdDMQ/xVfJXvPK/V5iePp3zl85rXZYQmmi2aeWee+5h69atFBQUoNfreeGF\nF7jtttuYPHkyJ0+elO6HQnNllWXMWDeDH40/suquVQzqOUjrkoRoFxkQJJySoih8sOsDntvyHIvG\nL+Le6Hu1LkmINpMgF07tx7wfuWvVXRjCDCwav0imKhB2qUMudgphq4YGDuX7R7+n9GIpYxaPIets\nltYlCWF1EuTC4Xi7e7Ns0jIeu/oxrv3wWlbsW6F1SUJYlTStCIe2K3cXk1dNJqF/Am/e9CYerh5a\nlyREs6RpRYhahgcN54dHfyC/PJ9rFl/DkcIjWpckhMVJkAuH5+Phw6q7VvFQzEOMWTyGTw98qnVJ\nQliUNK0Ip/L96e+ZvGoyQ/RDGB08mqiAKKL10fTx6SOTcAmbId0PhWhGzcpK+87sY69xL/vy91F6\nsZTIgEiiA6LVcA+IJlofLQtdC01IkAvRBoUXCtmXv499+fvYm6+G+17jXjxcPYjWR9cJ+Aj/CJl1\nUViVBLkQFqIoCjnncthr3PtLuOfvJbMgk15evQj0DMTD1QN3V3f1tpN7nfvm73VSb+s85upBN7du\njOs7rsH87EJIkAthZVXVVfxc+DMF5wuoqKrgYtVF9dZ0scHXF6suUmFqfJ+8sjyyS7NJuTGFqUOn\nyjJ8wkyCXAg78s3Jb3huy3Pkl+fz57F/5o7Bd8hFVyFBLoS9URSFz498znNfPkcnl068PO5l4vrF\nSaA7MQlyIexUtVLNJwc+Yd5X8wj2CuaV2FcYHTJa67KEBiTIhbBzVdVVLNmzhPlb53N10NW8NO4l\nWc7OycgQfSHsnKuLKw8Pf5jDTx3mhj43MO6jcSSvTuZ48XGtSxM2SoJcCBvl4erBrDGzOPzUYcK6\nh3H1P67mqQ1PYSwzal2asDES5ELYOG93b+Yb5nPoiUO4urgS8dcI5m6ZS3FFsdalCRshbeRC2JmT\nJSeZv3U+azPXMn34dHp26Uknl0500nXCRedivl/71kXn0uRjwd7BRAdESy+ZNiq9WMr+/P3sP7Of\nffn7eGDoAwwPGt6uY8rFTiGcxKGCQ3z040dUVFVgUkxUK9WYqk2YFJP5tv5j1Up1g+9nnc3ikukS\niQMTmRg+EUOYQUabNuL8pfMcPHOQffn7zKG9/8x+Cs4XEOEfQVRAFJH+kdwZcSdh3cPa9VwS5EKI\nVlEUhYMFB1mbuZb0rHT25+8nrl8ciQMTuXnAzU43cdjFqotknc1S5945s4/9+Wponz53mgE9BhAV\nEEWUfxSRAZFEBUQR1j0MF51lW6klyIUQ7ZJfns/6w+tZm7WWzUc3M0Q/hMTwRCYOnMjAHgPtvgmm\nqrqK3HO5nCw5SXZpNidLTprvHz57mGPFx+jbva8a1P5R6pl2QCRX+V3VYdModGiQb9y4kZkzZ2Iy\nmXj44Yf5wx/+0K5iRNMyMjIwGAxal+Ew5PVsmYqqCjKOZ5CemU56Zjpd3boyceBEEsMTuTb0WnOw\n2crrqSgKRRVFajCXXA7p0lr3S05iLDfSs2tPQn1CzVtv796E+oTSz7cfA3sM1LxpqbXZ2eZfLyaT\niSeffJLNmzcTHBzMyJEjSUxMZPDgwW09pLgCW/mgOAp5PVvGw9WD8VeNZ/xV43n35nfZk7eH9Mx0\nnv7iaY4VH2PCVROYGD6RLWu34DPQp85F1JoLry352kXnwoVLFyirLGuwlV8qb/Tx+t8rrigmuyQb\nVxfXXwLapzeh3qEMGTBEve8TSrBXMG6d3LR+aS2qzUG+Y8cOrrrqKsLCwgCYMmUKa9askSAXwkHp\ndDpigmKICYrhecPz5JTmsC5rHUt/WsqurF18t+a7OhdYay6s1r7g2tT3qpVqurp1xbOzZ5Nbt87d\n8Ozsib6bnv6+/Rt838fDhxDvELzdvbV+qTpcm4M8JyeH3r17m78OCQnhu+++s0hRQgjbF+wdzGMj\nHuOxEY+RkpVCyowUrUtyWm0O8pZe8LD3CyO2ZP78+VqX4FDk9bQseT210+YgDw4OJjs72/x1dnY2\nISEhdfaRC51CCGF9be78OGLECA4fPszx48eprKxkxYoVJCYmWrI2IYQQLdDmM3JXV1feeecdbrrp\nJkwmE9OnT5cLnUIIoYF2DUeaMGECmZmZ/Pzzzzz77LPmxzdu3MigQYMYMGAACxYsaHeRzi4sLIwh\nQ4YQExPDqFGjtC7H7jz00EPo9Xqio6PNjxUWFhIfH094eDgJCQkUF8sEVC3R2GuZkpJCSEgIMTEx\nxMTEsHHjRg0rtC/Z2dmMHTuWyMhIoqKieOutt4DWvz8tPvthTf/yjRs3cuDAAZYtW8bBgwct/TRO\nRafTkZGRwe7du9mxY4fW5didBx98sEG4pKamEh8fT1ZWFrGxsaSmpmpUnX1p7LXU6XTMmjWL3bt3\ns3v3bsaPH69RdfbHzc2NhQsXsn//fr799lveffddDh482Or3p8WDvHb/cjc3N3P/ctE+cuG47a6/\n/np8fX3rPJaenk5ycjIAycnJrF69WovS7E5jryXI+7OtAgMDGTZsGACenp4MHjyYnJycVr8/LR7k\njfUvz8nJsfTTOBWdTkdcXBwjRozg/fff17och2A0GtHr9QDo9XqMRlmsoT3efvtthg4dyvTp06WZ\nqo2OHz/O7t27GT16dKvfnxYPcuk3bnnffPMNu3fvZsOGDbz77rts27ZN65Icik6nk/dtOzz++OMc\nO3aMPXv2EBQUxNNPP611SXanrKyMSZMmsWjRIry8vOp8ryXvT4sHeUv6l4vWCQoKAsDf35/bb79d\n2sktQK/Xk5eXB0Bubi4BAQEaV2S/AgICzGHz8MMPy/uzlS5dusSkSZOYOnUqSUlJQOvfnxYPculf\nblnnz5/n3LlzAJSXl/PFF1/U6TEg2iYxMZG0tDQA0tLSzB8g0Xq5ubnm+//973/l/dkKiqIwffp0\nIiIimDlzpvnxVr8/FStYv369Eh4ervTv3195+eWXrfEUTuPo0aPK0KFDlaFDhyqRkZHyerbBlClT\nlKCgIMXNzU0JCQlRPvzwQ+Xs2bNKbGysMmDAACU+Pl4pKirSuky7UP+1XLx4sTJ16lQlOjpaGTJk\niHLbbbcpeXl5WpdpN7Zt26bodDpl6NChyrBhw5Rhw4YpGzZsaPX706oLSwghhLA+izetCCGE6FgS\n5EIIYeckyIUQws5JkAshhJ2TIBdCCDsnQS6EEHbu/wE0DIuruZHvTwAAAABJRU5ErkJggg==\n"
}
],
"prompt_number": 10
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Regularization\n",
"--------------\n",
"\n",
"Another way to deal with the model complexity is using regularization. A regularizer is a term that penalizes the model complexity and is part of the loss function. the next portion of code shows how the norm of the coefficients of the linear regression model increased when the complexity of the model (polynomial degree) increases."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"n_samples = 20\n",
"train_x, train_y = sample(n_samples)\n",
"max_degree = 9\n",
"w_norm = []\n",
"for degree in xrange(1, max_degree):\n",
" model = fit_polynomial(train_x, train_y, degree)\n",
" w_norm.append(pl.norm(model.coef_))\n",
"pl.plot(range(1, max_degree), w_norm, label='||w||')\n",
"pl.legend()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 11,
"text": [
"<matplotlib.legend.Legend at 0x1079d6b10>"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD9CAYAAABz5fboAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xtc1HW+x/EXpp1uXqBgNMWlNUnxkpiXLipjClYmoZmt\nmnEs2902z8keu9va42ybdfYk1u6j0tp2tzyFppWnzMjUlGyINDOTsJtipYIK09qAF9AQmPPHV0BQ\nZAZn5jfDvJ+PxzziMvObz7L15svn971EuN1uNyIiElLaWF2AiIh4T+EtIhKCFN4iIiFI4S0iEoIU\n3iIiIUjhLSISgpoN7x07dpCYmFj36NixI/Pnz8flcpGcnEx8fDwpKSmUlZUFol4REQEivJnnXVNT\nQ9euXdm8eTMLFizgkksu4cEHH2TevHmUlpaSkZHhz1pFROQEr9om2dnZXH755cTGxpKVlUV6ejoA\n6enprFixwi8FiojIqbwK79dee43JkycD4HQ6sdlsANhsNpxOp++rExGR0/K4bVJZWUnXrl35+uuv\niY6OJjIyktLS0rrvR0VF4XK5Gl48IsK31YqIhInmotnjkffq1au56qqriI6OBsxou6SkBIDi4mJi\nYmKaLCBUH4888ojlNYRj7arf+ofqt/bhCY/D+9VXX61rmQCkpqaSmZkJQGZmJmlpaZ5eSkREzpJH\n4V1eXk52djYTJkyo+9rs2bNZt24d8fHxrF+/ntmzZ/utSBERaaitJ0+68MILOXDgQIOvRUVFkZ2d\n7ZeigoXdbre6hBYL5dpB9VtN9Qc/r+Z5e33xiAiP+zciIgJuN7Rp03x2ejTyFhHxlaioqAYz1cJZ\nZGRkg1l6VVUwaJBnr1V4i0hAlZaW6i/yExpPp167Fs47z7PXamMqEZEgsXgxTJvm2XPV8xaRgFIu\n1Dv5Z3HoEMTGwnffQXR08z8jjbxFRILAm2+C3Q6XXOLZ8xXeIiInzJkzh0cfffSUj73x7//+73UL\nGE/3eVO8aZmAwltEpE5ERETdTcSW7s3U+HUnX7MpRUWQnw833+z5+yi8RURO4o9+fHPXXLIEJk70\nfKYJKLxFRJr10ksvkZqaWvd5z549mTRpUt3nsbGxbNu2rUXXdru9b5mAwltEpFlJSUnk5uYCsH//\nfo4fP86mTZsA+P777ykvL6d///4tunZeHhw9Ctdd593rtEhHRIKOr44C8FUH5Oc//znt27cnLy+P\nHTt2MGbMGPLz89mxYwcbN25kxIgRLb724sVwxx3e/29WeItI0AnGaeBJSUk4HA6+/fZbkpKS6NSp\nEzk5OXz88cckJSW1+LqvvgonBvVeUdtERMQDSUlJfPDBB+Tm5mK32+vCPCcn56zCOy4Oevb0/nUK\nbxERD9SG97Fjx7j00ksZNmwYa9asweVykZiY2OLrenujspbaJiIiHujZsyft27dn+PDhAHTo0IEe\nPXoQExNzVuf13n57y16n8BYR8dD+/fsbfP7pp5+e9TU9XQ7fmNomIiKnEeybZym8RUROcvLyeF8u\nkfc1bQkrIgGlXKjX1M/Ck5+RRt4iIiFI4S0iEoIU3iIiIcij8C4rK2PixIn07t2bhIQEPvnkE1wu\nF8nJycTHx5OSkkJZWZm/axWRViAyMrLuZmC4PyIjI1v8c/QovO+//35uuukmvvnmG7Zt20avXr3I\nyMggOTmZgoICRo0aRUZGRouLEJHw4XK5cLvdYflYtcrNkCH1n7tcrhb/HJudbXLw4EESExP5/vvv\nG3y9V69e5OTkYLPZKCkpwW63s3379oYX111lEZE6U6bAtdfCzJlnfp4n2dnsCstdu3YRHR3N9OnT\nyc/P56qrruLpp5/G6XRis9kAsNlsOJ3O075+zpw5dR/b7Xbsdntzbyki0uocPgzvvgvz55/6PYfD\ngcPh8Op6zY68t2zZwjXXXMPGjRsZPHgws2bNon379jz77LOUlpbWPS8qKuqUPwE08hYRMV5+GZYv\nh6ys5p/rk3ne3bp1o1u3bgwePBiAiRMnsnXrVjp37kxJSQkAxcXFxMTENF+RiEiYaslRZ2fSbHh3\n7tyZ2NhYCgoKAMjOzqZPnz6MGzeu7jj7zMxM0tLSfFeViEgrsnevOe5s3DjfXdOj5fH5+fnMmDGD\nyspKevTowUsvvUR1dTWTJk2isLCQuLg4li1bRqdOnRpeXG0TERHmzYNvv4UXXvDs+Z5kp/Y2ERHx\nI7cb+vWDv/0NPD3qUnubiIhY7PPP4cgRGDbMt9dVeIuI+FHt6fBtfJy2apuIiPhJVRXExoLDAVdc\n4fnr1DYREbFQdrYJb2+C21MKbxERP/H13O6TqW0iIuIHhw+bUffOnRAd7d1r1TYREbHI8uVmaqC3\nwe0phbeIiB/4s2UCapuIiPjc3r3Qvz/s3w/nnef969U2ERGxwNKlcOutLQtuTym8RUR8yO32f8sE\nFN4iIj6Vn29mmvh6OXxjCm8RER/y13L4xnTDUkTER1q6HL4x3bAUEQmg99/333L4xhTeIiI+Eogb\nlbXUNhER8YEjR6Bbt5Yth29MbRMRkQBZvhyGD/ffcvjGFN4iIj4QyJYJqG0iInLW9u0z51Tu2wfn\nn3/211PbREQkAJYuhQkTfBPcnlJ4i4icBbcbFi0KbMsEFN4iImclPx8OHTI3KwOprSdPiouLo0OH\nDpxzzjm0a9eOzZs343K5uP3229mzZw9xcXEsW7aMTp06+bteEZGgEqjl8I159HYRERE4HA7y8vLY\nvHkzABkZGSQnJ1NQUMCoUaPIyMjwa6EiIsGmqsr0uwPdMgEv2iaN73xmZWWRnp4OQHp6OitWrPBt\nZSIiQe79983CnF69Av/eHrVNIiIiGD16NOeccw6/+tWvuOeee3A6ndhsNgBsNhtOp/O0r50zZ07d\nx3a7HbvdftZFi4gEA1/N7XY4HDgcDq9e49E87+LiYrp06cK//vUvkpOTWbBgAampqZSWltY9Jyoq\nCpfL1fDimuctIq1U7XL4ggKIifHttX02z7tLly4AREdHM378eDZv3ozNZqOkpAQw4R7j6+pFRILY\n8uXmwAWroq/Z8K6oqODw4cMAlJeXs3btWvr160dqaiqZmZkAZGZmkpaW5t9KRUSCSKCXwzfWbNtk\n165djB8/HoCqqiqmTp3KQw89hMvlYtKkSRQWFjY5VVBtExFpjfbtg759zenw/lhV6Ul2am8TEREv\nPfkkbN8OCxf65/ra20RExA+sbpmAwltExCv5+XDwIIwYYW0dCm8RES8sXgxTpwZ+OXxj6nmLiHio\nutocMPz++9C7t//eRz1vEREfev996NrVv8HtKYW3iIiHguFGZS21TUREPODP5fCNqW0iIuIjb71l\n7XL4xhTeIiIeCKaWCahtIiLSrP37zXJ4X50O3xy1TUREfGDpUhg/PrCnwzdH4S0i0oxga5mAwltE\n5Iy2bYOyMuuXwzem8BYROYNgWQ7fmG5Yiog0oboauneH7OzArqrUDUsRkbOwfj106RIcy+EbU3iL\niDQhGG9U1lLbRETkNMrLzSZUgVgO35jaJiIiLfTWW3DddcGzHL4xhbeIyGkEc8sE1DYRETlFcTEk\nJPjvdPjmqG0iItICwbgcvjGFt4hII8HeMgEPw7u6uprExETGjRsHgMvlIjk5mfj4eFJSUigrK/Nr\nkSIigbJtG7hckJRkdSVn5lF4P/PMMyQkJBAREQFARkYGycnJFBQUMGrUKDIyMvxapIhIoATrcvjG\nmi1v7969rFq1ihkzZtQ10LOyskhPTwcgPT2dFStW+LdKEZEAqK42/e5gb5kAtG3uCQ888ABPPvkk\nhw4dqvua0+nEZrMBYLPZcDqdTb5+zpw5dR/b7XbsdnvLqxUR8aP166FzZzPTJJAcDgcOh8Or15wx\nvFeuXElMTAyJiYlNXjgiIqKunXI6J4e3iEgws+pGZeOB7aOPPtrsa84Y3hs3biQrK4tVq1Zx7Ngx\nDh06xLRp07DZbJSUlNC5c2eKi4uJCdYlSCIiHiovh6wsePJJqyvxzBl73o8//jhFRUXs2rWL1157\njeuvv57FixeTmppKZmYmAJmZmaSlpQWkWBERf3nrLbj2WjjREQ56Xt1PrW2PzJ49m3Xr1hEfH8/6\n9euZPXu2X4oTEQmUUJjbfTItjxeRsFe7HH7fPrjgAqur0fJ4ERGPLF0KaWnBEdyeUniLSNgLtZYJ\nKLxFJMx98QX8+COE2hIUhbeIhLXFi+GOO4J/OXxjumEpImGr9nT4desCv6ryTHTDUkTkDD74wJrl\n8L6g8BaRsBWKNyprqW0iImGpvBy6dYPt24NvVaXaJiIiTVixAq65JviC21MKbxEJS6HcMgG1TUQk\nDAXbcvjG1DYRETmNV18NveXwjSm8RSTshHrLBBTeIhJmvvwSDhwIveXwjSm8RSSshMrp8M3RDUsR\nCRvV1fCzn8F770GfPlZX0zTdsBQROYnDATExwR3cnlJ4i0jYaA03KmupbSIiYaGiArp2hW++MZtR\nBTO1TURETlixAq6+OviD21MKbxEJC62pZQJqm4hIGCgpgd69g3c5fGNqm4iIYJbD33JLaAS3p84Y\n3seOHWPo0KEMGDCAhIQEHnroIQBcLhfJycnEx8eTkpJCWVlZQIoVEWmJRYtaV8sEPGibVFRUcMEF\nF1BVVcWwYcP4y1/+QlZWFpdccgkPPvgg8+bNo7S0lIyMjFMvrraJiFjsyy/hhhtgzx445xyrq/GM\nT9omF5z4O6OyspLq6moiIyPJysoiPT0dgPT0dFasWOGDckVEfK92OXyoBLen2jb3hJqaGgYOHMh3\n333HvffeS58+fXA6ndhOHD9hs9lwOp1Nvn7OnDl1H9vtduyhvhuMiISM6mpYsgTWrLG6kjNzOBw4\nHA6vXuPxbJODBw8yZswY5s6dy4QJEygtLa37XlRUFC6X69SLq20iIhZ6/3343e8gL8/qSrzj09km\nHTt2ZOzYsXz22WfYbDZKSkoAKC4uJiYm5uwqFRHxg9Y2t/tkZwzvAwcO1M0kOXr0KOvWrSMxMZHU\n1FQyMzMByMzMJC0tzf+Vioh4oaIC3n4bJk+2uhL/OGPPu7i4mPT0dGpqaqipqWHatGmMGjWKxMRE\nJk2axMKFC4mLi2PZsmWBqldExCMrVsDQodCli9WV+IdWWIpIq3TjjXDHHWamSajxJDsV3iLS6pSU\nQK9eZjn8hRdaXY33tDxeRMJS7enwoRjcnlJ4i0ir05pnmdRSeItIq/LVV/DDD6F/OnxzFN4i0qq0\n1uXwjTW7PF5EJFRs3AgLF8KGDVZX4n8aeYtIq/DFFzB+vBl5x8dbXY3/KbxFJOTt2mXmdT/9tNn+\nNRwovEUkpDmdkJICDz3UepfCn47CW0RC1sGDZqR9xx1w331WVxNYWmEpIiHp6FET3P37w/z5EBFh\ndUW+o+XxItIqVVXBrbeaFZSvvAJtWlkPwZPs1FRBEQkpNTUwYwZUVsL//V/rC25PKbxFJGS43fD7\n30NBAaxbB+eea3VF1lF4i0jImDcP1q6FnJzWvemUJxTeIhISXngB/vlP+OgjiIqyuhrrKbxFJOi9\n+SY88ogZcV96qdXVBAeFt4gEtfffh3vvhffeg549ra4meITpfVoRCQVbtphVk2+8AYmJVlcTXBTe\nIhKUtm+HcePgxRdhxAirqwk+Cm8RCTpFRTBmDGRkQGqq1dUEJ4W3iASVAwfMRlP33w/p6VZXE7y0\nPF5EgsbhwzBqlHnMnWt1NdbxyenxRUVFjBw5kj59+tC3b1/mz58PgMvlIjk5mfj4eFJSUigrK/NN\n1SIh7q9/NYcClJRYXUlo+eknmDABrrwSHn/c6mqCX7Ph3a5dO5566im++uorNm3axHPPPcc333xD\nRkYGycnJFBQUMGrUKDIyMgJRr0hQW7fOhPfll8OAAbBihdUVhYbqarOta4cO8Pe/t64dAv3F67ZJ\nWloaM2fOZObMmeTk5GCz2SgpKcFut7N9+/aGF1fbRMJIUREMGQJLl8LIkeY8xWnT4Prr4amn4KKL\nrK4wOLnd8Otfw3ffwbvvwr/9m9UVWc/nuwru3r2bvLw8hg4ditPpxGazAWCz2XA6nad9zZw5c+o+\nttvt2O12b95SJCT89BPcdhs88IAJboBrr4XPPzc33hITzdmKV19tbZ3B6OGHYetWWL8+fIPb4XDg\ncDi8eo3HI+8jR46QlJTEww8/TFpaGpGRkZSWltZ9PyoqCpfL1fDiGnlLmJg5E/btg+XLT/8n//Ll\n8JvfmBHmf/0XtGsX+BqD0VNPwT/+Abm5EB1tdTXBwyc3LAGOHz/OrbfeyrRp00hLSwOoa5cAFBcX\nExMTc5blioSmJUvM0u2XX266VzthghldfvwxDB8OO3cGtMSgtGiRCe+1axXcLdFseLvdbu6++24S\nEhKYNWtW3ddTU1PJzMwEIDMzsy7URcLJl1/CrFlm46SOHc/83EsvhdWrYepUuOYas0teuP5h+s47\n8OCD5pde9+5WVxOamm2bfPTRR4wYMYL+/fsTcWJYMXfuXIYMGcKkSZMoLCwkLi6OZcuW0alTp4YX\nV9tEWrFDh2DQINOznTbNu9d+9ZWZXdG9u1n+HU4jz9xcc4TZypXmBq+cSmdYiviJ2w0TJ0JMDDz/\nfMuu8dNPZpvTRYtMgN90k29rDEaff25WTy5dCqNHW11N8FJ4i/jJX/4Cy5aZUeTZzpDIyYE774Sx\nY811L7jANzUGm2+/haQkeOYZ84tPmuazG5YiUi8nx4TsG2/4ZmpbUhLk55s2zMCBZhvU1qa42Iy4\nH3lEwe0rCm8RLxQXw5QpkJnp2xttnTrBK6/AnDmmffL442bVYWtQWmp2CJwxA375S6uraT3UNhHx\n0PHjZrVkcjL86U/+e5+iIrObXmWlWdhz2WX+ey9/q6gwP6+hQ822AVr27hm1TUR8aPZsaN8e/vhH\n/75PbCxkZ5vNrYYMMaP8UBwDHT9uWiSXX27aTApu39LIW8QDb7wBv/89fPZZYE8u37bNzAvv1cts\n2HTxxYF777NRU2Nuwh48aFaXakWpdzTyFvGBHTvMAbhvvBHY4Abo3x8+/dSMxq+80qxGDHZut1m4\nVFhoZuQouP1DI2+RMzhyxPRrZ82Ce+6xtpbsbJg+3SxwmTsXzj/f2nqa8t//bVacOhzmRqx4TyNv\nkbPgdpvZEUOHmpkSVhs92kwp3L8fBg82Hweb5583Pfo1axTc/qbwFmnCc8/B11+bfwbLzbaoKHj9\ndfjDH0yYP/mk6S8Hg9dfh//5H9Pa6dzZ6mpaP7VNRE7j44/hllvMP3v0sLqa09u92+yp0rat7+ed\ne+u998wNynXrTJ9ezo7aJiIt8MMPcPvtsHBh8AY3QFyc6SuPGWM2yFq61Jo6Nm0ym2wtX67gDiSN\nvEVOUl1tlnFffbVpAYSKrVvNlMLERPjb3wLXb/7qK3PS+//+b3hsrBUoGnmLeOlPfzL97cces7oS\n7wwcWD8H/cor4YMP/P+eu3fDDTeYlZMK7sDTyFvkhKwsc5zZli1mq9dQtXq1mR0zdaqZtuePcyF/\n+AGGDTM/r//8T99fP9xp5C3ioe++M4G3bFloBzfAjTeafbN37jTTHL/80rfXP3TIjLgnT1ZwW0nh\nLWHv6FGz8OVPf2o9p7tHR5sbiP/xH2C3mz20fTGl8NgxMwvnmmvMDohiHbVNJKy53XDXXeZUmyVL\ngmc+ty99+62ZUti+Pbz0EnTt2rLrVFXBbbeZNsySJXDOOb6tU+qpbSLSjBdfhM2b4Z//bJ3BDWZX\nv9xc06MeONDs0eKt2tWmFRXm2DYFt/U08pawtWWL6Q9/9BFccYXV1QTGJ5+YOdnXXQfz50OHDp69\n7sEHzS+A7Gy48EL/1igaeYs0yeUyLYDnnw+f4AZzAzMvD849FwYMML+4mvPEE/Duu+ah4A4eGnlL\n2KmpgZtvht69zRzlcJWVZVohd99tzpY899xTn7NwIfz5zybkW9orF+9p5C1yGn/+s9nqNSPD6kqs\nlZpqphR+/jlcey1s397w+2+9BQ8/bPYtUXAHn2bD+6677sJms9GvX7+6r7lcLpKTk4mPjyclJYWy\nsjK/FiniK++9B//4h9kBT4cEmN3/Vq40c9yHDTNtJLfbrND81a/M9+Ljra5STqfZ8J4+fTpr1qxp\n8LWMjAySk5MpKChg1KhRZIT7EEZCwp495mDfV1+FLl2sriZ4RETAr38NGzaYNklKitmYa9kyMztF\ngpNHPe/du3czbtw4vvjiCwB69epFTk4ONpuNkpIS7HY72xv/zYV63hI8fvoJhg+HSZPgd7+zuprg\ndfw4zJtnNrgaO9bqasKXJ9nZtiUXdjqd2Gw2AGw2G06ns8nnzjlpGZbdbsdut7fkLUXOyqxZ5hzI\n3/7W6kqCW7t28Mc/Wl1F+HE4HDgcDq9e06KRd2RkJKWlpXXfj4qKwuVynXpxjbwlCCxaZLZ3/fRT\nz+c1i1jJb7NNatslAMXFxcSE+k4+0mpt22ZG22++qeCW1qVF4Z2amkpmZiYAmZmZpKWl+bQoEV8o\nKzMbTj39NPTta3U1Ir7VbNtk8uTJ5OTkcODAAWw2G4899hi33HILkyZNorCwkLi4OJYtW0an0xzd\nobaJWMXthvHjoVs3ePZZq6sR8Y4n2akVltIqPfGE2RI1J8c/hxGI+JPfZpuIBLMPPoCnnjK7BSq4\npbXS8nhpVfbtM8d/LV5spgaKtFYKb2k1jh83i3Duuw9Gj7a6GhH/Us9bWo1Zs8xZlG+/DW00LJEQ\npp63hI3XX4d33jEHLCi4JRxo5C0h75tvYMQIWLvW7MkhEuq0n3eYcjrN5vlHj1pdif8dPgwTJtRv\npiQSLjTyDnFut+nz5uaawM7NhX/9C37+c9i5E5KSzKkxY8eaBSutidsNv/iFORX9xRetrkbEd7RI\npxWqrjb7ddQGdW6uOcl7+HCzmf7w4WYpeJs2UFpqDh9YuRJWrzZT52qDfMiQ0D8B/JlnzKZTGzbA\needZXY2I7yi8W4Fjx8xueLVB/fHH5vST4cPrH3FxZkP9M6mqMieHr1xpHk6nOTl97FgYMwY6dgzI\n/xyf2bDBtEs2bYLLLrO6GhHfUniHoLIy2LixPqzz8iAhoT6ohw2D6Oizf589e8xp4CtXmlH8oEH1\no/L4+OZ/GVjJ6YSrrjLHmenAAGmNFN4hYP/++qD+6CPTvx48uD6sr74aLrrIvzWUl8P69SbI330X\nzj+/PshHjDj9qeJWqaqC5GTzs3nsMaurEfEPhXeQcbuhoKDhzcWysvpe9bBh5sxAK8PS7Yb8/Pog\n/+Ybs1rx5ptNm+XEAUqWmT0btm41PfxQ79mLNEXhbbGqKhOEJ4+szzuv4c3F3r2De1HJDz+YoFy5\nEtatgyuuqB+VJyYGtr2yYgXcfz989hlcckng3lck0BTeAXb0qLkpWBvWmzaZGR4n31zs3t3qKluu\nstL8AqodlR85YkJ87FgzOr/wQv+9986dcN11ZhXl0KH+ex+RYKDw9jOXy8x6qB1V5+dDv371QX3d\ndXDxxVZX6T8FBSbE333XbL963XUmyG++2cyA8ZWKCtP7v/de8xBp7RTePlZU1LAFsmePGQXWtkGG\nDvXv6DOYHTxo2irvvgurVpkZMbVBfs010LaFu+i43ZCebv65aFFwz4IR8RWF91lwu83NupMXw5SX\nN+xXDxgA7dpZXWnwqakxc9NrpyLu2WPmkt98M9xwA0RFeX6tv/8dnnvOtKDC9RejhB+F9xlUVUFx\nMRQWmhF1YWHDj3ftgsjIhv3qK67QyK8l9u0zo/GVK80pN1deaYL85pvNHPamfqabN5vnbNgAPXsG\ntmYRK4VteLvdZmn46UK59uPiYvOnfffu5hEb2/Djn/1MMxr84dgxcDjqV3pCfZDb7fXL3A8cMAtx\nnn7aHCQsEk5abXgfOwZ79zYdzIWFpsd6ulCu/fjSS4Nr8Uk4crvhq6/q2yv5+TBypAnyN94wI/Qn\nnrC6SpHAC8nwrqkxy58bh/LJn5eVQdeuTY+aY2NDb68OgR9/rN9Iq00bePnllt/oFAllfg/vNWvW\nMGvWLKqrq5kxYwZ/+MMfmi3g0KGmR8uFhaY/2qnTmUfNNltgFrY4HA7sdrv/38gPQrl2UP1WU/3W\n8usxaNXV1cycOZPs7Gy6du3K4MGDSU1NpXfv3g2ed++9DUP6+PH6EK4N5euvr/+8W7fg2d4zlP8F\nCOXaQfVbTfUHvxaH9+bNm7n88suJO7Ea4xe/+AVvv/32KeHdty/cdFP9yDkyUjM2RETOVovDe9++\nfcTGxtZ93q1bNz755JNTnnfffS19BxERaUqLe95vvvkma9as4YUXXgDglVde4ZNPPmHBggX1F9cQ\nW0SkRfzW8+7atStFRUV1nxcVFdGt0SGJwbpAR0Qk1LV4zsagQYPYuXMnu3fvprKyktdff53U1FRf\n1iYiIk1o8ci7bdu2PPvss4wZM4bq6mruvvvuU25WioiIf5zVbOkbb7yRHTt28O233/LQQw/Vff2u\nu+7CZrPRr1+/sy4w0IqKihg5ciR9+vShb9++zJ8/3+qSvHLs2DGGDh3KgAEDSEhIaPD/Syiprq4m\nMTGRcePGWV2K1+Li4ujfvz+JiYkMGTLE6nK8UlZWxsSJE+nduzcJCQls2rTJ6pI8tmPHDhITE+se\nHTt2DLn/fufOnUufPn3o168fU6ZM4aeffmr6yW4/+PDDD91bt2519+3b1x+X96vi4mJ3Xl6e2+12\nuw8fPuyOj493f/311xZX5Z3y8nK32+12Hz9+3D106FB3bm6uxRV5769//at7ypQp7nHjxllditfi\n4uLcP/74o9VltMidd97pXrhwodvtNv/+lJWVWVxRy1RXV7s7d+7sLiwstLoUj+3atct92WWXuY8d\nO+Z2u93uSZMmuV9++eUmn++XdYrDhw8nMjLSH5f2u86dOzNgwAAALrroInr37s3+/fstrso7F1xw\nAQCVlZVUV1cT5c0erEFg7969rFq1ihkzZoTsTe9QrPvgwYPk5uZy1113AaY12jFE95nIzs6mR48e\nDaYzB7sxgWiIAAADEElEQVQOHTrQrl07KioqqKqqoqKigq5duzb5/CA+PdF6u3fvJi8vj6Ehdu5W\nTU0NAwYMwGazMXLkSBISEqwuySsPPPAATz75JG2C+XDPM4iIiGD06NEMGjSobiptKNi1axfR0dFM\nnz6dgQMHcs8991BRUWF1WS3y2muvMWXKFKvL8EpUVBS//e1v6d69O5deeimdOnVi9OjRTT4/NP/r\nCIAjR44wceJEnnnmGS666CKry/FKmzZt+Pzzz9m7dy8ffvghDofD6pI8tnLlSmJiYkhMTAzJ0SvA\nhg0byMvLY/Xq1Tz33HPk5uZaXZJHqqqq2Lp1K7/5zW/YunUrF154IRkZGVaX5bXKykreeecdbrvt\nNqtL8cp3333H008/ze7du9m/fz9HjhxhyZIlTT5f4X0ax48f59Zbb+WOO+4gLS3N6nJarGPHjowd\nO5YtW7ZYXYrHNm7cSFZWFpdddhmTJ09m/fr13HnnnVaX5ZUuXboAEB0dzfjx49m8ebPFFXmmW7du\ndOvWjcGDBwMwceJEtm7danFV3lu9ejVXXXUV0dHRVpfilS1btnDttddy8cUX07ZtWyZMmMDGjRub\nfL7CuxG3283dd99NQkICs2bNsrocrx04cICysjIAjh49yrp160hMTLS4Ks89/vjjFBUVsWvXLl57\n7TWuv/56Fi1aZHVZHquoqODw4cMAlJeXs3bt2pCZddW5c2diY2MpKCgATN+4T58+FlflvVdffZXJ\nkydbXYbXevXqxaZNmzh69Chut5vs7Owztjz9slvy5MmTycnJ4ccffyQ2NpbHHnuM6dOn++OtfG7D\nhg288sordVO9wEzfueGGGyyuzDPFxcWkp6dTU1NDTU0N06ZNY9SoUVaX1WKhtsWC0+lk/Imjf6qq\nqpg6dSopKSkWV+W5BQsWMHXqVCorK+nRowcvvfSS1SV5pby8nOzs7JC611Dryiuv5M4772TQoEG0\nadOGgQMH8stf/rLJ5/v1MAYREfEPtU1EREKQwltEJAQpvEVEQpDCW0QkBCm8RURCkMJbRCQE/T8m\nU41RNkK2IwAAAABJRU5ErkJggg==\n"
}
],
"prompt_number": 11
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The above result suggests that we can control the complexity by penalizing the norm of the model's weights, $||w||$. This idea is implemented by the *Ridge Regression* method.\n",
"\n",
"### Ridge regression ###\n",
"\n",
"Ridge regression finds a regression model by minimizing the following loss function:\n",
"\n",
"$$ \\min_{W}\\left\\Vert WX-Y\\right\\Vert ^{2}+\\alpha\\left\\Vert W\\right\\Vert ^{2} $$\n",
"\n",
"where $X$ and $Y$ are the input matrix and the output vector respectively. The parameter $\\alpha$ controls the amount of regularization. You can find more information in the documentation of [scikit-learn ridge regresion implementation](http://scikit-learn.org/stable/modules/linear_model.html#ridge-regression). \n",
"\n",
"\n",
"#### Exercise ####\n",
"\n",
"Repeat the cross validation experiment using ridge regression. Use a fixed polynomial degree (e.g. 10) and vary the $\\alpha$ parameter.\n",
"\n",
"___________________\n",
"\n",
"<h2 id=\"biblio\"> References </h2>\n",
"\n",
"* [Alpaydin10] Alpaydin, E. 2010 [Introduction to Machine Learning][alp10], 2Ed. The MIT Press.\n",
"\n",
"[alp10]: http://www.cmpe.boun.edu.tr/~ethem/i2ml2e/\n"
]
}
],
"metadata": {}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment