LSTM

LSTM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stock price prediction using LSTM\n",
    "\n",
    "In this example, stock prices are forecasted using LSTM cells. This is intended as an exercise to understand LSTM and its implementation using Tensorflow. The dataset consists of the prices of S&P 500 index and all the stocks that form the index, for 41266 minutes of data from April to August 2017 (downloaded from http://files.statworx.com/sp500.zip). \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data preparation\n",
    "The first part in implementing any machine learning algorithm is data preparation. This involes reading the data, cleaning it up, normalizing if necessary, and splitting it into train and test portions. In this example, the data is stored as a .csv file which is first read into a pandas dataframe object. The 'DATE' section of the data refers to the timestamp for each price, and this can be safely ignored as LSTM's do not use time as explicitly in the their computatation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " No. of rows is  41266  and no. of cols is  501\n"
     ]
    }
   ],
   "source": [
    "# Import data\n",
    "data = pd.read_csv('data_stocks.csv')\n",
    "\n",
    "# Drop date variable\n",
    "data = data.drop(['DATE'], 1)\n",
    "\n",
    "# Know the shape of the data\n",
    "print(\" No. of rows is \",data.shape[0],\" and no. of cols is \",data.shape[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SP500</th>\n",
       "      <th>NASDAQ.AAL</th>\n",
       "      <th>NASDAQ.AAPL</th>\n",
       "      <th>NASDAQ.ADBE</th>\n",
       "      <th>NASDAQ.ADI</th>\n",
       "      <th>NASDAQ.ADP</th>\n",
       "      <th>NASDAQ.ADSK</th>\n",
       "      <th>NASDAQ.AKAM</th>\n",
       "      <th>NASDAQ.ALXN</th>\n",
       "      <th>NASDAQ.AMAT</th>\n",
       "      <th>...</th>\n",
       "      <th>NYSE.WYN</th>\n",
       "      <th>NYSE.XEC</th>\n",
       "      <th>NYSE.XEL</th>\n",
       "      <th>NYSE.XL</th>\n",
       "      <th>NYSE.XOM</th>\n",
       "      <th>NYSE.XRX</th>\n",
       "      <th>NYSE.XYL</th>\n",
       "      <th>NYSE.YUM</th>\n",
       "      <th>NYSE.ZBH</th>\n",
       "      <th>NYSE.ZTS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2363.6101</td>\n",
       "      <td>42.3300</td>\n",
       "      <td>143.6800</td>\n",
       "      <td>129.6300</td>\n",
       "      <td>82.040</td>\n",
       "      <td>102.2300</td>\n",
       "      <td>85.2200</td>\n",
       "      <td>59.760</td>\n",
       "      <td>121.52</td>\n",
       "      <td>38.99</td>\n",
       "      <td>...</td>\n",
       "      <td>84.370</td>\n",
       "      <td>119.035</td>\n",
       "      <td>44.40</td>\n",
       "      <td>39.88</td>\n",
       "      <td>82.03</td>\n",
       "      <td>7.36</td>\n",
       "      <td>50.22</td>\n",
       "      <td>63.86</td>\n",
       "      <td>122.000</td>\n",
       "      <td>53.350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2364.1001</td>\n",
       "      <td>42.3600</td>\n",
       "      <td>143.7000</td>\n",
       "      <td>130.3200</td>\n",
       "      <td>82.080</td>\n",
       "      <td>102.1400</td>\n",
       "      <td>85.6500</td>\n",
       "      <td>59.840</td>\n",
       "      <td>121.48</td>\n",
       "      <td>39.01</td>\n",
       "      <td>...</td>\n",
       "      <td>84.370</td>\n",
       "      <td>119.035</td>\n",
       "      <td>44.11</td>\n",
       "      <td>39.88</td>\n",
       "      <td>82.03</td>\n",
       "      <td>7.38</td>\n",
       "      <td>50.22</td>\n",
       "      <td>63.74</td>\n",
       "      <td>121.770</td>\n",
       "      <td>53.350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2362.6799</td>\n",
       "      <td>42.3100</td>\n",
       "      <td>143.6901</td>\n",
       "      <td>130.2250</td>\n",
       "      <td>82.030</td>\n",
       "      <td>102.2125</td>\n",
       "      <td>85.5100</td>\n",
       "      <td>59.795</td>\n",
       "      <td>121.93</td>\n",
       "      <td>38.91</td>\n",
       "      <td>...</td>\n",
       "      <td>84.585</td>\n",
       "      <td>119.260</td>\n",
       "      <td>44.09</td>\n",
       "      <td>39.98</td>\n",
       "      <td>82.02</td>\n",
       "      <td>7.36</td>\n",
       "      <td>50.12</td>\n",
       "      <td>63.75</td>\n",
       "      <td>121.700</td>\n",
       "      <td>53.365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2364.3101</td>\n",
       "      <td>42.3700</td>\n",
       "      <td>143.6400</td>\n",
       "      <td>130.0729</td>\n",
       "      <td>82.000</td>\n",
       "      <td>102.1400</td>\n",
       "      <td>85.4872</td>\n",
       "      <td>59.620</td>\n",
       "      <td>121.44</td>\n",
       "      <td>38.84</td>\n",
       "      <td>...</td>\n",
       "      <td>84.460</td>\n",
       "      <td>119.260</td>\n",
       "      <td>44.25</td>\n",
       "      <td>39.99</td>\n",
       "      <td>82.02</td>\n",
       "      <td>7.35</td>\n",
       "      <td>50.16</td>\n",
       "      <td>63.88</td>\n",
       "      <td>121.700</td>\n",
       "      <td>53.380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2364.8501</td>\n",
       "      <td>42.5378</td>\n",
       "      <td>143.6600</td>\n",
       "      <td>129.8800</td>\n",
       "      <td>82.035</td>\n",
       "      <td>102.0600</td>\n",
       "      <td>85.7001</td>\n",
       "      <td>59.620</td>\n",
       "      <td>121.60</td>\n",
       "      <td>38.93</td>\n",
       "      <td>...</td>\n",
       "      <td>84.470</td>\n",
       "      <td>119.610</td>\n",
       "      <td>44.11</td>\n",
       "      <td>39.96</td>\n",
       "      <td>82.03</td>\n",
       "      <td>7.36</td>\n",
       "      <td>50.20</td>\n",
       "      <td>63.91</td>\n",
       "      <td>121.695</td>\n",
       "      <td>53.240</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 501 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       SP500  NASDAQ.AAL  NASDAQ.AAPL  NASDAQ.ADBE  NASDAQ.ADI  NASDAQ.ADP  \\\n",
       "0  2363.6101     42.3300     143.6800     129.6300      82.040    102.2300   \n",
       "1  2364.1001     42.3600     143.7000     130.3200      82.080    102.1400   \n",
       "2  2362.6799     42.3100     143.6901     130.2250      82.030    102.2125   \n",
       "3  2364.3101     42.3700     143.6400     130.0729      82.000    102.1400   \n",
       "4  2364.8501     42.5378     143.6600     129.8800      82.035    102.0600   \n",
       "\n",
       "   NASDAQ.ADSK  NASDAQ.AKAM  NASDAQ.ALXN  NASDAQ.AMAT    ...     NYSE.WYN  \\\n",
       "0      85.2200       59.760       121.52        38.99    ...       84.370   \n",
       "1      85.6500       59.840       121.48        39.01    ...       84.370   \n",
       "2      85.5100       59.795       121.93        38.91    ...       84.585   \n",
       "3      85.4872       59.620       121.44        38.84    ...       84.460   \n",
       "4      85.7001       59.620       121.60        38.93    ...       84.470   \n",
       "\n",
       "   NYSE.XEC  NYSE.XEL  NYSE.XL  NYSE.XOM  NYSE.XRX  NYSE.XYL  NYSE.YUM  \\\n",
       "0   119.035     44.40    39.88     82.03      7.36     50.22     63.86   \n",
       "1   119.035     44.11    39.88     82.03      7.38     50.22     63.74   \n",
       "2   119.260     44.09    39.98     82.02      7.36     50.12     63.75   \n",
       "3   119.260     44.25    39.99     82.02      7.35     50.16     63.88   \n",
       "4   119.610     44.11    39.96     82.03      7.36     50.20     63.91   \n",
       "\n",
       "   NYSE.ZBH  NYSE.ZTS  \n",
       "0   122.000    53.350  \n",
       "1   121.770    53.350  \n",
       "2   121.700    53.365  \n",
       "3   121.700    53.380  \n",
       "4   121.695    53.240  \n",
       "\n",
       "[5 rows x 501 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the dataset contains price history of S&P 500 as well as each of the 500 stocks that form the index. In this example, I will only try to use the SP500 price as the sequential data, as a simple start. Ofcourse one can even think of using the other stock prices to create a different problem. \n",
    "\n",
    "A good step in any data analysis is data visualization, done using matplotlib here. This helps us to understand the complexity of the data we are trying to model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XecFPX5wPHPc4Xe4agH3oEUUUDw\nBBRUFAQUWywp5ofGmGCiJhg1iqKYaOyJmqIxRk2iMaKxxAKKiCiiSBVEPJWO9F6kH/f8/pjZvdnd\n2Xa3d3t797xfr3s5+51ysyM335lveR5RVYwxxtQ+Wek+AWOMMelhFYAxxtRSVgEYY0wtZRWAMcbU\nUlYBGGNMLWUVgDHG1FJWARhjTC1lFYAxxtRSVgEYY0wtlZPuE4ilVatWWlBQkO7TMMaYjDJ//vyt\nqpoXb7tqXQEUFBQwb968dJ+GMcZkFBFZnch21gRkjDG1lFUAxhhTS1kFYIwxtZRVAMYYU0tZBWCM\nMbWUVQDGGFNLWQVgjDG1lFUAxpgaYfveQ0z6bEO6TyOjWAVgjKkRfv7v+VzznwVs3n0g3aeSMawC\nMMbUCGu27wPgYElpms8kc1gFYIypETbscp78Zy3fluYzyRxWARhjapT3v96c7lPIGFYBGGNqlKNb\nN073KWQMqwCMMTVKl7yG6T6FjGEVgDGmRrl38pfpPoWMYRWAMaZG2WjDQBNmFYAxJuMdKdV0n0JG\nsgrAGFMlvtm+j4JxkygYN4m9B0tSeuzDR0LH/u/cdyilx6+p4lYAItJRRKaLSLGILBGRsWHrbxQR\nFZFW7uemIvKGiCxyt7/Cs+3lIrLU/bk89V/HGFNd/eqFhcHlwKStVDkUVgEE5gSY2BJ5AygBblDV\nY4CBwDUi0hOcygE4E1jj2f4a4AtV7QMMAf4gInVEpAVwBzAA6A/cISLNU/ZNjDHVxqzl2ygYN4n7\n3/6SgnGTWLdzP/NW7wiub1Q3tenIn50VmgL3kM0GTkjcCkBVN6jqAnd5D1AMdHBXPwzcBHgb4BRo\nLCICNAK241QiI4CpqrpdVXcAU4GRqfoixpjq4wd//wSAv76/HIBB970Xsj68yaYiNu85wINTvgop\n23sotU1MNVVSfQAiUgD0BWaLyHnAOlVdFLbZX4BjgPXAYmCsqpbiVBrfeLZbS1lF4v0dY0RknojM\n27JlSzKnZ4ypBtbuiN+8U5JAp+32vYcoDdtu464DFIybxK79hwFQVfrfPS24PjdbAKe/wcSXcAUg\nIo2Al4HrcJ7oxwMTfDYdASwE2gPHA38RkSaA+Gwb8a9AVZ9Q1SJVLcrLy0v09Iwx1cR3H58Vd5uX\n5q/1LX/2k9Ws37mfTbsP0O+uqfxl+rKQ9QPvdW72wx/+gIMlRyi8ZXLIenFvMze/vLg8p17rJFQB\niEguzs3/OVV9BegCFAKLRGQVkA8sEJG2wBXAK+pYBqwEeuA88Xf0HDYf5y3BGJNBdu6LfDL3Wp9A\nB2zxht3B5QenfMkrC9aya99hbv/f5/zfk7PZ6B7j3eJNvvtv2n2QAfdMiyi/6IR8AC5x/2tiS2QU\nkABPAcWq+hCAqi5W1daqWqCqBTg3936quhGnQ3iou28boDuwApgCDBeR5m7n73C3zBiTIbZ+e5Dj\n75zKVf+ez5AHp3PGH95HNfkx+D3bNQkuPzp9Ode/uIgs9260Yutezn/0IwA+W7uLe98qpmDcJFZs\n+ZYGdbKD++3cdzjiuFcMKgCgY4sGSZ9TbZTIG8AgYDRwhogsdH/OjrH9XcDJIrIYmAbcrKpbVXW7\nu26u+3OnW2aMyRDbvnXG10/9YhOrtu1jxZa9EU/87ZvWAyC/ef1g2YtXnRSyTd9OkQMAs8SvlRj+\n9sEKAD5evo0m9XJjnl+XvEYAlJajUqqN4o7FUtWZ+Lffe7cp8Cyvx3m699vuaeDp5E7RGFNd5GRH\n3gq2f3uIDs3KbvYtG9Vl/a4DdG3diOd/OpAt3x6kX9gN/2f/ns/ogUdx1wXHBcvijQya/uVmmtbP\njQj1MHHMQA4cPsJ7X24mO8s5v32HjiT93aqTnfsOUScniwZ1UjtcNpzNBDbGJCwnK7ICyM0JLTu1\nWysAfnVmNzq2aBC8+V9zepeQtvlnPwkduz/mmfkxf/e0Lzf7TiAb2LklQ7q35s7zyyqTJ2Y4bw1b\nvz3IgcOJVwb7Dx2hYNwkLnn844T3qQzH3zmVs//4YaX/HqsAjDEVImENBI3qOs00XcPi8v96RA/G\nDusa9ThzVpW1CA8obOG7zf4kbuYARb97lx63v53w9rNXOtnE5q7aEWfLyrdqW+UPZa3c9wtjTI0S\nPuMWoKQ0tOnm/redcMx+Tfq52Yk9c85fvYPe+U1ZvG4Xfs35R7VswKCjW3H2ce2iHuMn/5qb0O/y\n+tE/kt8nk1kFYIxJ2JMzV0aWfbiSVz9dB8CsW86IuX94BRAtaFvP9k3YsOuA780foH5uNvd8p1fM\n3/VucWamhlywpurePqwJyBiTkPDhni0a1gEI3vwB3v+qbPa+30ic3LBO5L9+sNz3d/XOb8qWPQej\nnstvzzs2/gknKbwyKs/w1lSYt6rqBkdaBWCMScjDU78O+ewdkx9wyytlM3Dr5USuzw7rRA4M8fQ6\nt097fndB7Kf7Ph2bxVxfHje//FnI59cXVe08VVVl6aY9PDx1aZX9TqsAjDEJ+dN7oWEZbhvVM+q2\nD3+vD1k+I4bq50ZWCgE/OrmAGb8+nUe+d3zEupX3ns3ye8qmH9WLcZzymrIkdNbxkx9GNndVpolz\nv+HMh2ck3dFdEVYBGGPi+nzdroiy9Tv3R91+5LH+nbMSZbIXwKbdB+jUskHEW0Jgv+ws4eQuLRM4\n28SpKs9+spr9PvMGFvt858oUHsDuzV8MrvTfaZ3Axpi4zvnzzODylYMLmbJkI/uihFz+8aBC6vs0\nD0UzsHMLPlmxnT98t0/U4wU88+P+CUUSjefNz9Zz7+QvuWF4N27/3+e85unH8Np/6EhS36Uiwt9q\nUp0zwY+9ARhjknL7OT2ZefMZeCfu/uenA4LLvx7RPanjTRxzEqvuGxUx67VOjnN7un54t2BZTnZW\nSpp/rn9hEet27uf6F51o9t5kNV7b9kbviE61h8L6WApaNaz032kVgDEmYZcO6BRc7tK67AY1sLBl\ncJZw3ZzU3FbOOq4tEDlyKBXCU0h6ff/EsqDF2/emJ7fwqvtGVcnvsSYgY0zCbh7RI7h8areyfB1Z\nWcLSu88CYrfzJ+PBi/tw88ge1PUZTZSMo1s3Smr75u7w1trA3gCMMQlrXK/smbGh22QTiP4pIknf\n/J+8rCjqujo5WbT3BJlL1pd3jaRZg1w6VSA09ObdVdcE1KEC37W8rAIwxsSVmy1cPaRLyNDO7Czh\n43Fn8N6NQ8p1zDd/MZhhPduk6Awj1cvNplOLBr4T0prUi974cVG/sky1N4XNDahMo3o7I6daVuEb\niDUBGWOCbnhxEUO659GheX2Oa9802BFbqv7x+ivyhH5ch6bl3jdRIuIbTmL3gehJ44/2BLH72Wmd\nK+O0fB0pVRrWyWb+7WdW2e9MJCNYRxGZLiLFIrJERMaGrb9RRFREWnnKhriJY5aIyAee8pEi8pWI\nLBORcan9KsaYinp5wVp+8fynXPjYx8GgbuCEdUhR036VypLYyWH+/IO+IZ8X3RGayqRxnAQ0qXSk\nVH0nz1WmRN4ASoAbVHWBiDQG5ovIVFX9QkQ6AmfipIEEQESaAY8BI1V1jYi0dsuzgUfd7dcCc0Xk\ndVX9IsXfyRhTDuFx85+auZLbz3Fm+6qmrnO3st15/rEs2/wtAOt27CfWrIFz+7RnypKNLN+yl8tO\nOoqm9UNv+CVxktSkUqmq7yS4ypRIRrANwAZ3eY+IFAMdgC+Ah4GbgNc8u1yKkxR+jbtPICRff2CZ\nqq4AEJGJwPnucYwxabZ2h3/8+UBQtpVb91bl6ZTbZScVBJc3xwgoF/CXS/tFXZeKSWeJWrBmR9S0\nmJUlqU5gESkA+gKzReQ8YJ2qLgrbrBvQXETeF5H5InKZW94B+Maz3Vq3zBhTDQx7aEZE2Tfb93Hn\nm84z2hspCo721x/249WrT07JsSqif5SkM1A2B6GqmoDOf/QjPl+3u8rnHSTcCSwijYCXgetwmoXG\n45/7Nwc4ARgK1Admicgn+OcVjqheRWQMMAagU6dOETsYY6rOKQ9MT/kxz+oVPYlLVRoYowK4blg3\n3vp8I4u+2cnFnjSWlWXRNzsr/Xf4SegNQERycW7+z6nqK0AXoBBYJCKrgHxggYi0xXmyf1tV96rq\nVmAG0Mct7+g5bD4Q8Uihqk+oapGqFuXl5YWvNsZUEu/ww1aNIocizrz59Ko8ncoXo7kl0BYfnre4\npklkFJAATwHFqvoQgKouVtXWqlqgqgU4N/d+qroRpz/gFBHJEZEGwACgGJgLdBWRQhGpA3wfeL1S\nvpUxJmnbPM0Pr18bGYkyv3n5J1Sl24dLyxLVBBK9xGptr+rO2HRJ5A1gEDAaOMMd2rlQRM6OtrGq\nFgNvA58Bc4AnVfVzVS0BrgWm4FQIL6rqkgp/A2NMhXmzb/1qWLcKje+vjjbsOhBcDowKjdXfml3F\nnbHd2iQXriJVEhkFNJPYlSXuW4D384PAgz7bTQYmJ3eKxpjK9sHXZU/IY4d1jVj/q2HdIsoyyazl\n2/hukdMCHeh4jDXiJquKYyTsiTExrTJZKAhjDIWtnOaduy44znf9GT1aV+XppNyrn67jYIkzz6G0\nEpqAXpq/NiKhSzKsAjDGpM3Bw86Ep64+kTPbNa1Hr/zKD9tQ2brf9jaQWBNQMuPxj5QqN/53Ed95\n7KOIdYdKSjkSZy7BkVLl24NWARiTcaYs2UjBuEkcf+c77E3TH3EqrHXTO7bwCUR24/DkErxUd+o2\nAsWa2exdNWfl9pjHC7xZbP02cgx/t9veYugf3o+5/4otzqzlH51cwMfjzoi5bapZBWBMOakqD7jx\ncnbuO8zaHdFz5FZnry1cx00vOVEvWzeuG7G+po2IiREayLNR2eK7xZuibwe8MPebmOtXbYvdNHTm\nw84EvDkrt1d557tFAzWmnPr89p2QqJKpyoRV1cZOXBhc9sbCadmwDtv2HqryAGWVST13/1jNPN6M\nYXmNIitFr9++kZpoNiOObZuS4yQjM//FGlMNhIcUzqmE1IVVzdssMqCzM1M2k+//r187KOTzxt0H\nyjqBY3yvlg3LbvrhAeLi2frtwWCzUDJ+ckph0vtUlFUAxqRIQk0LwD2Tixl033uVezLl0Kdjs5DP\nVw7uTJ3sLAZ2bpmmM6q43vmh3yk7qyw/QKx6rX6dbJ663MlWlt88drNM5zwnN/KFbiKZot+9yzXP\nLQjZZvyri+l319SIfb1vJA3rVn2DjDUBGZMiiVYAT8xYUbknkoSvN+0JLv/9shNC1p1wVHO+dvP8\n1hRHSjXYvB9voE8rt+ln5rKtnHx0K99tzn/0I1ZscaKkfrRsK7NXbAPg3eLNIds9N9uJmK+qIW9Z\ns5ZvS/YrpJS9ARiTIrESj1RHz89Zw/CHyyKAtm5cL41nUzU27jrgCQURuwZY546Meuz95VG38QZx\n27T7IN974pPgZ79moOfnhHYYX/rk7PgnXYmsAjAmRYY+9EH8jTyWbd4Tf6NKsv/QEW55ZXHafn+6\nfOexj9m02wkLEe8NwG9ORDL8Qjs/MOVLny3TxyoAY8ohMLknr3Fdio5qHlKWqE274ycrqSwn3Tct\nbb+7quWGdc4H8h7Ey3DWtY2TG7i8o7uWb45MoNPEk1/A2/7/xOgTIratClYBmFqntFQpGDeJgnGT\nyn2Mw+4wwUv7d+Lq07uU6xh+k66qgqqyc9/hkLJ03YCqQrR6OdHBTQdLypcWcsGaHRFlazzhIrzH\nHZCmjnarAEyts9TNF1sRgY6/P05bSr3c7GD5z/89v8LHrmwzlm6NKBt2TJs0nEl6JRPw0689P/AQ\nUCfb/zb60NSvfcsDT/63/e/zYFl9z7+hqmQVgKl1vH/4Ws6O23mry8IDeF/r3/p8I6u3JZY7947X\nl5RrvHhFFIybxOVPzwkp+/p3Z9WoyV7hXhgz0LfcGyI6Hr8wDzPdivRQkonjH52+jDcWreel+WuD\nZckeI1WsAjC1TsmRspv+3kPluwH36+S0+996dg+O6xAaKO3WVxPrXJ2zcjuD7pvOgcNVUwk8+aH/\n8NM6GTqDOVFFBS04PmyOAyQ3HPc9n3AQV/xzbrnO5/fvfM0vnv80+LlNk7o0SsMcALAKwNRCz8xa\nFVzeFyWA2yWPf8zdk75g1da9bN4T+aQYCCPQqYUTRnnO+KHBdR8tS3xs99ZvDwbHiAOs3Lo32D+R\n6giRv5tUnNLjZZJ6uRW71d3+WsVyV/Vs1yTquk9uGRp1XWVLJCVkRxGZLiLFIrJERMaGrb9RRFRE\nWoWVnygiR0TkYk/Z5SKy1P25PHVfw5jEdW/bOLi8L8obwNxVO/j7hysZ8vv36X935IgZz3QigJB+\nAIB3lmwMLn+4dEsw49Zhn1f9GZ5kLKf//v3g8vf+Niv2F0lSTX/Sj2VI98rLZzDvtmHB5b/+sJ/v\nNs9e2T/q/vFGI1WmRP5FlAA3qOoxwEDgGhHpCU7lAJwJrPHuICLZwP046R8DZS2AO3ByBPcH7hCR\n5qn4EsYko13TsglPX22KHIu/+8DhiLJw4THlw1MIjnnW6Qx+af5aRj81hxPvfpd3v9hE1/FvRRzL\nm43La8n63XHPIxnn9GoX8vnqIV348KYalug9ivJG2czziY4arlWjuhS0bMCYUztzVtg1BidURMtG\ndYMzi72G90xv53vcCkBVN6jqAnd5D04+3w7u6oeBmwgJngrAL4CXAe986BHAVFXdrqo7gKnAyIqd\nvjHOU/WTH67gUILD9Q4cLtsuMCnIKzyOi59ABRBoCgp/Awi48b+Lgss/eWZe1OMt8xmZ1LheatuF\nh3pG+jSul8NNI3vQsUXmJnpPht+1HNU78mYdrr3nYeG52avZtb/s4cAbJvv9X5/OrWcf43uMwIgx\n75tCwJ3n+2dgqypJvROKSAHQF5gtIucB61R1Udg2HYDvAI+H7d4B8M6DXktZReLdf4yIzBOReVu2\n+D8ZGRNwqKSUruPf4neTiul221s8mMBMS+/Imwlhbbvb9x7iQ59hkuEdtcGkIu7n7Cxh1X2jkjz7\nMsN8ZhFPOKdnRNn0rzazJ4E3FD/eUBVPXlZUrmNkKr+Qzqd29Y/v4zV+VNn/g/Gvfs6tntnTJ3Vu\nSb9OkZ3LyWjWILlIo6mWcAUgIo1wnuqvw2kWGg9M8Nn0EeBmVQ1vXPVr6IoYg6eqT6hqkaoW5eXl\nJXp6ppYKH3L56PTlFIybRKnP7J8Nu/ZTMG4SN7/sP0rn8JFSXlu4znfdDS+GPOcklFYwWet3hiaU\nCY8ttH7nfq74x1x6/eadch0/cLR3rz81bROP0sX7hhaIetqzXfw0l/0LW9C4bg493H6jOavKhv8q\n6tt+P+jo6Nf2+jO7hYxIivbmWFUSescUkVycm/9zqvqKiPQCCoFF7gXIBxaISH+gCJjolrcCzhaR\nEpwn/iGew+YD76fma5jaym+2JcBri9bxnb75IWXewF1+zv3zTL7c6B+fx/uHDyQUUfLeycmNujk5\nLER0eH/x/goOFw0GQUtjp2O6tGpUNuv68f/rx39mr+G4DtFH5njtOVgS/HcR6MwH5yHA70pecXJh\nyEiwyb88Jbj8y6Fdad+sPgvj/FusKnErAHH+tTwFFKvqQwCquhho7dlmFVCkqltxKoZA+T+BN1X1\nf24n8D2ejt/hwC0p+h6mlor2NL91T+TEHW/bv59oN3+AnLCJUoncTP/mjjPv0bZx8NgdmtUPRpk8\nvXse07+K3sy5envo2837nm3nrdpOUUGLqPv6SSQOfk3VrEEdZt58Om2a1CM3O4sbUpDnWNX/AaCD\nmz+ga+tGTL3+tAr/nsqUSBPQIGA0cIaILHR/zk72F6nqduAuYK77c6dbZkzKnVjYgutfXMht/3Mq\niL0HS3jlU//mnUSEZ/sqjXIzDSQF8fIOO72gb/vg8sPfO973dwVGhvztg9CJShPnlA22K88cgUSS\noddk+c0bkBslbEN5RGsCCuRVvuiE/Ih1AEvTGAU2XCKjgGaqqqhqb1U93v2ZHLZNgfv0H77vj1T1\nJc/np1X1aPfnH6n5Cqa2ihXG4fCRUl5ZsI5/f7KG0lLlmv8sCBlvH8+4s3qEfM7NCv9T8b+Z/uGS\nPhHHem3h+uDyjK/L/kyaNQgNBvfLM44G4K4L/EeGnNqtrE+sPCOEykYuJb2rcXkD+JVGaQJq2agu\nX941kqtO7ex7jLc/3+hbng61d2aIyXjb3HjrP/XkUn30UmciziWPl02iuvb5BSHNJxC/8/b/Bh7F\nd4vyOaqlM0wy/A0gWnNKvKfrhnWjd/r96b1lMc/N24dxpBS63fYWBeMmsXNfZHOXn7K3FqsByisk\nxn+UJiBwOnej/VtoUKf6JGK0CsBkrMBM2f/OX8sNZ3bjzV8MpnnDyGF1kxdHPnH5vTx4/14b1c3h\ngYv7sHqbE773YEkpz81eHbzhBnbPSrI55YpB5U/8fUrXsjeAPQcOB+c9HH9nZK5ZP5pAMnSTOEXL\nVZlWNCxFKlWfMzEmScvdCTY79x3mF0O7clyHphEzchMRuDE2d5tkurdpHLHN6m37GP+qE7739UXr\ng8NM/X5deAKR+y/qFVw+pWsrnv/pQD77zfCo59O0vv/YcG8ohyv/FX1SWTSVMXS1NprwmvPvIFon\ncDzV6fJbBWAynrfNPMmkXEDZ6J8Sd9xlvD/qF+Z+w+J1uwB8OxXnemZ8Djq6Jd87sVPwc72cbE7q\n0jIYQnrKdaeGDFEEqJtT1kzknc9w/9vJpxNcu2NfMApobe8ELq9z+7QP+fzMrNWA0wtUnkvZ1p1d\n/L2ijhU9tQqzCsBkvLOOaxtc9gu2Fs+vXlgIlMWL+bFPM01zz4zNJet3ByNr+qUL9OYHCIQHCPyx\nh8fd7962MaMHFkQ9t137D7Nr3+Fy5S34cOkWBt8/nd9NKmbLnoPWCVxO2VGul2r5moACTukWfyZy\nZbMKwGS8lp6RGSWlyVcAgTeARnVzGNi5Bd89sezJrHNeQwB2H/Afdtmgjn+nbr47FvxoN7H4fRf1\nYtndZ/lue9EJkUNHAzbsOkCfO9/hyQ9XRt2mk088n0MlpYx+qizxi6LWCVxO2REjwODmlz5j+Za9\nvonf4wnkowifW5IO1ac72phy8jZpBO7/+c3rs3bH/ih7+Ju3ekdwyn9A+6b1WbFlb9SE79Gm8s/4\n9emUlGqw3V5EIkYSBWTHuBFMXrwBgLtjzCru2CIy0mX4TFNVJxSGcy5RD2V8+N2oX5jnhDXzBodL\nVElpoAJI//N3+s/AmHIoidLUE7hNh9/IwzWM8uQePhv4tG6x41HVj3KcrCxJOP5+rJFEX26MHxK6\na+vI7/rdsFwCh0pK+XOcYabGX6DZrkM5Q0qHC1QA2dHalqqQVQAmIx2IEvo5MM6+dZN6vPmLwSHr\nTurckj/9oC8AvfObMXZo1+C6Xr9xUleEx2z/ySnRh232aNvYN8Z7svwqgMDIocb1okeL7NOxGU3r\n5ybUP3Dnm18El60JKDmzVzpxfdbt3E8Xt0mwIi7s6zT5xXtIqQrWBGQy0hJ3FE64kzq35PeX9GFU\nr3bUr5PNF3eOoOcE5+b+vJscvG2TevRo15j5q8oCye1x2/i3fnsw5HixRsy8NfaUqOuS4W0CCiQg\nCTQPxJo4dv9FvfjBE5+EhNRV1ZBctz/o34nn56xh6hdlOW3tDSA5gXj+ACu2hsZniveG6OeCvh24\noG/0fp+qZBWAyUjfe+IT33IR4WJPDBa/WZf9C1u425b/9//npwNSNpzS28QciDYZ6C+okx29AsjJ\nyiJLJCRsdP97poVErDytWx7PzwlJ2BcRZtokLvzSfbg0s3OWWBOQyTifrS3r4PSOAKpKqZzO73c/\n3nvQCf28be/ByJWu3GxBRELmPnhv/oFtwlkTUOqUZ95JdWJvACbjeP/ozj8+/qv0G9cOppFP8LSK\n/O0mmn4yEX4dyX+c9jUQGkguXE52FiKxg+KFVwiQWJ5bUzvYG4DJOPs8oZD/b2CnGFs6euU3pbBV\nYp13FxzfPuq6v40+IbhcP4WZnPyGkn7/xPjfKzdLyJKyNwi/MenhSe9/PaLicfBNzWEVgMk4e9wK\n4G+jT6BzXqNyH8cvbeSEc4+Nur131Eai2aQSFX5jXhWW6hIig4jVzckmS4TF63ZRMG4SE+euidhn\nQGFo0phLivxj1Jvy+dHJBek+hQqJWwGISEcRmS4ixSKyRETGhq2/UURURFq5n38oIp+5Px+LSB/P\ntiNF5CsRWSYi41L/dUxtEAh/XNFhdCU+FYBfm3nA4SOlvH3dKfx6RPeUx9MZcayTBCYw89jv6BPH\nnMTCCWcGP9fNzUJwQlMAPPD2V4Azygmczu6Rx7ULOUadFCZEqS3aNPFvMnv2yv7ccnYP33WZIpE+\ngBLgBlVdICKNgfkiMlVVvxCRjsCZgPfRYyVwmqruEJGzgCeAASKSDTzqbr8WmCsir6vqFxiThEAa\nyIo2w/i1nccad3/gcCnHdWhKj7apffqHsnADgRnHfp2LffKbIiJMue5U3i3eFDXm/F0XHMfaHfuC\nzUidWzUMDl/MsQogaX3ym/HOF5sY2qM1077cHCz3hufOVIlkBNugqgvc5T1AMRDoeXsYuAlPf5qq\nfqyqgQHWn+AkfwfoDyxT1RWqegiYCJyfkm9haqWKdmYmO4IjWtiHVAiEGwjEiQkPavfAxb2DN/vu\nbRtzzelO9rD1uyLDXdTPzeaKQYXBzuU9nj6TegnOTjZlAg8Flw6I3y+TaZIaBSQiBUBfYLaInAes\nU9VFMV6HrwTecpc7AN941q0FBiTz+43xqmgzjHc8/JWDC/nx4NjJWgKB3SpDYDJY4K0kfJTRwSij\njrwvMYEn/bphfQXekUD2BpC8Cef2pLBVA07v3jrdp5JyCVcAItIIeBm4DqdZaDwQNauFiJyOUwEE\n5uP7/bVGPIOJyBhgDECnTjVSIThCAAAfZ0lEQVSvxjXRHT5SypFSjfukfWJB85Qk9w40t5zTux23\nn9OzwseriHZN63Hl4EK+50YiPadPu5Dmhot8ks2HCzTz+IWoNuXXtH4u157RNaSsb6dmaTqb1Ero\nX4qI5OLc/J9T1VeALkAhsEhEVuE08ywQkbbu9r2BJ4HzVXWbe5i1gDcDQj4QMchZVZ9Q1SJVLcrL\ny/w2NpO4i//6MT1ufzvmNgu/2cncVTso3hA/SFo8gafnWNE4ITIhSGUQEW4/pyfd3Gxkp3ULfdpM\nZuJZtCiT493cBKb8AsOOz+ld+f8mqkLcf1XivGc/BRSr6kMAqroYaO3ZZhVQpKpbRaQT8AowWlW/\n9hxqLtBVRAqBdcD3gUtT9UVM5lq2+Vs+XbODRWv94/t4XfDoRwDs2Jd8GN5wgSageA1Jf/5BX/7s\nBpGrKrFGI8UTPnu4TZO6bNp9kBHHto2yh0lUu6ZORNDwLG6ZKpHHikHAaGCxiCx0y25V1clRtp8A\ntAQec9toS9wn+hIRuRaYAmQDT6vqkoqdvqkJhj30QcjnyYs3cHavdlG2Tp3mbhiJ/OaRCVXSzTsa\nqXd+06T2LWgZOukt8EZgQeAq7qpTO5PfvD7nVcFbYVWIWwGo6kziPCSpaoFn+SfAT6JsNxmIVnEY\nA8DVzy0ILs8ZP5TnPlnDjn2HuPP842Lslbwh3fJ4/P/6MfSYNik9bqpdckJyk7c6hmUIC3QsWwVQ\ncTnZWQmFH8kU1ltk0ipeLPv+d0/jj9OWBhNxp5KIMPK4dinpUK4MgfAVLy1YV6HjBK6wJYM34arn\nv3xTa2z9NvmcqgA/H9IlxWdS/ax0R/UsCkvvGItff3agr6MapKA11YxVACajBCZ//ezUml8BJGJA\nYQtaeEJi+7VNjzurB9lZErKdMWDhoE2aTf9qc/yNPLbsOcix7ZvQtEH0kA01Tawb9wtXnQRAwbhJ\nAEz/KjJByXf65vOdvhYEzkSyNwCTVt6InIEk6p/9JnR+YSCiZWDbQPCzmq6z2wfwgpvKMpZAwvJ+\nNWSCkqkaVgGYtOrihle4eWQPvvjtCFbdN4omniGQN4/sERym+d2/zUrLOaZLoM82kb7bi9yRQn5v\nAMZEYxWASavDboybvp2a+capyckS1mx3OkPnrXZiDNaWGa2BUTuJpPBdsi7+JDpjwlkFYNLqsNus\nEz7zdexQJ/ZKfvP6EVE7/zv/G2qTRIKWDuzcEkh9ohpTs1kFYNLqSKnzBhAev2bs0K4895MBjDyu\nLV9vDE1ruG5HZAjkmihQJSbyBhDIFtYn3/oATOKsAjBpFYh/Hx6QLStLGHR0K0SEnu1Dn2q7VGJY\n5uok0PavCbwDnOHOZk4kl7AxATYM1KRVIC1jTozgZ7NXbg/5/ORlRZV6TtVFllsDlPqnAgjRoVl9\nVt03qpLPyNQ09gZg0mrfoSNA9BDGflq7OW9ruhMLnOGvzWrRnAdTtawCMEnZte8wM5duTdnxbvzv\nIgC+9aQtNI7bz+nJO786lfbuGH9jUs2agGqx0lIlK8EAMZt3H+BwqTLovvcA+ODXQzgqLOxwRWzd\nczDuNtef2Y1WjSqWBziT1MnJCiaIMaYy2BtALfWrFxYy+P73Et6+/z3Tgjd/gD9NW5bS8xl0dKuo\n6zrnORXNjwcX1sjE3Maki70B1FKvfuqEGFbVcoUJfnnBWh64uHfcdIqxLPREuaxfJ3oe4Km/Oo1v\nD5TQqK79czUmleK+AYhIRxGZLiLFIrJERMaGrb9RRFREWrmfRUT+JCLLROQzEenn2fZyEVnq/lye\n+q9jkvXHaUvjbrPvkH/7/J1vlD+h2/5DR/hqY2IxfbKzpFYFfzOmqiTySFUC3KCqC0SkMTBfRKaq\n6hci0hE4E1jj2f4soKv7MwD4KzBARFoAdwBFOJMb54vI66q6I4Xfx8Tw0vy1tG5cl1O75QXLHnl3\nKdcN6xZzvwWr/ePR/2vWaurkZDF+VM+Ez+HhqV9zVMsGXP/iooT3McZUjrhvAKq6QVUXuMt7gGIg\nkBPtYeAmQmernw88o45PgGYi0g4YAUxV1e3uTX8qMDJ1X8XEc+N/F3HZ03Oiri8tVd8MXQ3qRm+e\n+fuHK5M6hz9OW2o3f2OqiaQ6gUWkAOgLzBaR84B1qhr+19wB8AZrWeuWRSsP/x1jRGSeiMzbssUi\nG1aGhWEZpu6dXMw/P1pJ51snU3jLZDbsCg21EAjY1qtDcsnJw63Y8m2F9jfGpFbCFYCINAJeBq7D\naRYaD0zw29SnTGOUhxaoPqGqRapalJeX57OLqagLHv0o5PPfZqzgN298Efz8/Ow1IesDk7VuHtmj\n3L9TVbn3rS991/3sNMvuZUw6JFQBiEguzs3/OVV9BegCFAKLRGQVkA8sEJG2OE/2HT275wPrY5Qb\nj/2HjlAwbhIF4yYx4+v0vAFlh83KDVQAeY3rsuiO4X67xDVrxTamfrEppKxNk7o0a5DLD21opzFp\nkcgoIAGeAopV9SEAVV2sqq1VtUBVC3Bu7v1UdSPwOnCZOxpoILBLVTcAU4DhItJcRJoDw90y47Fr\n/+Hg8hX/nJuWcygN6wcoCUTszJaIsM3HtCt/+OEbh3dn4YThdGzRoNzHMMaUXyKjgAYBo4HFIrLQ\nLbtVVSdH2X4ycDawDNgHXAGgqttF5C4gcFe7U1W3+x+i9vKOqz8SHgi/ApJpfz+mXejs08B5ZIsE\nY/M3b5BLUUEL1iYYmrluTtmzxjWnd6Ft0/pcUtQxxh7GmMqWyCigmaoqqtpbVY93fyaHbVOgqlvd\nZVXVa1S1i6r2UtV5nu2eVtWj3Z9/pP7rZL7wp+8dew+l5Lhn/OGDkM+jerVj0i8Hh5T940cnArD/\nsNMMddFfP0ZVgxE7s7OERnVzuOWsHrx69SAA31FDftbvPBBc/vWIHoweeFS5v4sxJjUsFEQ1E/7U\nP3eV/0vS3oMlnP7799nvts9Hc+urixn38mcR5Y/+sB/Htm/KMDeOPEDrJk6cne17nWao+at3sGrb\nvmAy9sDbyVWndaGgVUPfXn0/L879Jhj0zRhTfVgFUM1M+mxDyOdo7eMX/fVjVm7dyzET3o55vP/M\nXsPEuaEpFIf3LLvpH+tJtlI3xxnv723mF+CI+idtSdRNL3/GwZIEgtobY6qUBVepZh559+uQz+FN\nQgGbE4ie6Wf6jUNo64mnn9+8LNRw4Ab/zCerg2UiRLwBJONQ2I3/0Uv7RdnSGFPV7A2gGpk4Zw17\nw5p05q/ewe+nfBWx7fizjwkuPzFjOcs274nYxk9hq4Yhgdcu6pfPT08p5OkfFRG4v6/Ysje4/rQH\n3+fAYecmnl2OoHHdbnsr5PNZx7VN+hjGmMphFUA1MuG1yOBqE15bwl+mL+M3ry/hwGGncti0+wA3\neNrU75n8JcMemsHXm5xKYPqXm/nbB8sT+p1ZWcL4UT05o0ebYArCcG8v2RjctiJEKn4MY0zqWBNQ\nNbBgzQ5mLt3KoSPR28n/+fEqROCOc49lwD3TfLcZ/vAMVt03Kjh/INrM22iiNfHMX70j5vpEfXXX\nWRXa3xiTWlYBVAMXPvZxRFnX1o1Yujl07P66BMfcRzPvtmEx10d7Awiokx36wphsi1CdHHvhNKY6\nsb/IauqOc4+NKHvni028/9Xmch1v0YThcdMpeiNA9OvULGJ9+CzgeEY/NTu4fO3pRye1rzGm8lkF\nUE3VzfX/X3Ozz5h+rz0HDvuWJ5JQxdvJu3N/5HGSyRw2f/V2PvQkjw+kdTTGVB/WBFTN/O+aQRzX\nvgnFG/xH9ZQciT3z9qmZycXn9/I2AXlHAv18SBeWbU4ulPNFf50V8rmwlVUAxlQ3VgFUM+2b1iMn\nO4sOnvH5XtvihIZ45N3IFI/Xnxk741eAd4TOr4Z14+xebTlYUspxFcwDANCzffmDxhljKodVANXI\nlOtOpbU7SatFwzpxt3/xqpN478vNjD7pKAbd917U7cac2jmh3+8d5DN2WNeE9kkkFNDPh3QJzjI2\nxlQfVgFUA33ym7Jz/2G6t20cf2OP/oUt6F/YIu52udmJdfUkO8xTEowGFG02szEmvawTuBpoWDeH\n1o1jj9A5vmPoqJwL+4Vm06wbY4hlRcfvJ+vj5VtDC+z+b0y1ZBVANVCq6jvC5sbhZW334cMoL+0f\nmkXrgYt7+x772CTa3usl2Uyz+8BhlvqEoHhseugs5Hj9FsaY9EgkI1hHEZkuIsUiskRExrrld4nI\nZyKyUETeEZH2bnlTEXlDRBa521/hOdblIrLU/bm88r5WZinV0Pb3gKvcXLmndcvjjB6t+YHnpl9U\nENr0c16f9iGfG7jxfs4NK48l2TANHy/fhl/OmpnLQt8AXl9omT+NqY4S6QMoAW5Q1QUi0hiYLyJT\ngQdV9XYAEfklToL4nwHXAF+o6rkikgd8JSLPAY2AO4AinEaB+SLyuqruSP3XyiyqSlZWZF2cm53F\nsrvPIjtLEBHuvbAXJxY0Z0j31hHbet8gFk44kzHPzmfOyu0RTUfpECvEhTEmfeJWAG4+3w3u8h4R\nKQY6qOoXns0aUtbSq0BjN5dwI2A7TiUyApgaSAPpViIjgedT9F0ylmr0MAw5YR24F/bLj3u8Zg3q\nBEM45yT5VL/qvlEJb9uyYR1r3jEmgyXVByAiBUBfYLb7+W4R+Qb4Ic4bAMBfgGOA9cBiYKyqlgId\nAG9mkrVuWa3n9AGk9pjXDetGk3o5SY8sSsb3+3eMWcF48w4YY6qfhCsAEWkEvAxcp6q7AVR1vKp2\nBJ4DrnU3HQEsBNoDxwN/EZEm4DtmMKIFWUTGiMg8EZm3ZcuWpL5MpiqN8QaQjFvP7sHEMQMBGNy1\nFZ/9ZgSN68UPAVFeWSLBbGF+/vXj/gC8ce3gqNsYY9InoQpARHJxbv7PqeorPpv8B7jIXb4CeMVN\nDr8MWAn0wHni7+jZJx/nLSGEqj6hqkWqWpSXl5f4N0nQG4vWM3HOGh6e+nX8jauIqvp2AidrzKld\nGNi5ZcUPlCDBab6Klhi+e9vGrLpvFL3yKz6T2BiTeomMAhLgKaBYVR/ylHunip4HBILPrwGGutu0\nAboDK4ApwHARaS4izYHhblmV+sXznzLulcX8cdrSiHSF6bJk/e50n0K5/Om9ZQCsrWCYamNMeiQy\nCmgQMBpYLCIL3bJbgStFpDtQCqzGGQEEcBfwTxFZjPOQeLOqbgVn6Cgw193uzkCHcLrMWbmdwV1b\npfMUmLdqOyWlyvSvMre5a8qSjfzklLJwE8e2b0K7ptb+b0x1l8gooJn4t99PjrL9epyne791TwNP\nJ3OCqRRIqRig1WCK6sWPz4q/UTX3u0nFXDm4EBHhm+37WLJ+Nx2a+QezM8ZUH7VqJvCOfaFDFp/8\nsPyhkytqw679jIsT2z+T3P+2k7h+xCMzAMhJMnmMMabq1aoKIDtspM0HX6ev2eWke99j4txv4m+Y\nIR53k9DvO+S8ZaViVJMxpnLVqgrgsF/cAlNuV50WPcz0R2HhIIwx1U+tqgBKwkISFLRskKYziTQ/\nTsL26mjYMW2irtuxzz81pTGm+qhdFUDYG8APBxyVpjOJ1DJOwvbqqEclzjI2xlS+2lUBhOXTjTWL\ntbJ5M351bJGZI2aqOs+AMSa1alUFcDisCehIGvsEentmx/7v6kFpO4+KsArAmMxWqyqA8Bt++BtB\numRi8w9EjqoyxmSW2lUBuE0+gTy6+w6VpPN0AHj2yv7pPoVyC38DeOz9ZWk6E2NMedSqCiAQtCyQ\nXvFvM1ak83Tok9+UU7qmPuBdVQlPY/mAOxnMGJMZalUFEOgCiDdJadnmPVEjXJpQH950Ot3b2Ggg\nYzJRja0A3v58A+t3hkapLHVv6j7ZF4NmLt3KsIdm8Mi7Syvz9GqMji0ahIxoMsZkjhpZAagqP/v3\nAk6+772Q8kCaxFhvAI+86+QJ+OM0qwASNWvFtoiyuy44Lg1nYoxJRo2sAPYeOuJbHhgE5O28PFRS\nGhIldN7qWp+jvsIu7NuB0QOrzyQ7Y4y/GlkBbP/WP1F5YBSQd/BKt9veosftbwMw8J5pIduPnfgp\nuyykQdL2HEz/6CpjTHyJZATrKCLTRaRYRJaIyFi3/C4R+UxEForIOyLS3rPPELd8iYh84CkfKSJf\nicgyERlXOV8p+gzfYB9AlCagjbsPhHx+beF6Hpjype+2JrqpX2xK9ykYYxKQyBtACXCDqh4DDASu\nEZGewIOq2ltVjwfeBCYAiEgz4DHgPFU9FrjELc8GHgXOAnoCP3CPk3KFrRr6lsfqAzhY4t9slM7Z\nwsYYU5niVgCqukFVF7jLe4BioIOqehPZNoRgeq1LcZLCr3H32eyW9weWqeoKVT0ETATOT83XSEzg\nZu4XwuCNRRt89wkf615ez8xaxQl3TQ1+rkmjTHt1sKTvxmSipPoARKQA6AvMdj/fLSLfAD/EfQMA\nugHNReR9EZkvIpe55R0AbwaUtW5ZpZo4Zw1H3zqZDbv2M+bZ+UBZU5DXjf9d5Lv/5rBmofKa8NoS\ntu09xAxvEpoaEkqhTZPMDGVhTG2XcAUgIo2Al4HrAk//qjpeVTsCzwHXupvmACcAo4ARwO0i0g3/\nvMIRd2IRGSMi80Rk3pYtFc/YNe6VxZSUKifdWzYkdPG6XQnvP+3LzfE3SsJlT89J6fGqg9N7tE73\nKRhjyiGhCkBEcnFu/s+p6is+m/wHuMhdXgu8rap7VXUrMAPo45Z39OyTD6wPP5CqPqGqRapalJdX\nOWESovURmPK5tH+nkM8r7jk7TWdijElGIqOABHgKKFbVhzzlXT2bnQcEhsu8BpwiIjki0gAYgNNv\nMBfoKiKFIlIH+D7wemq+RnJOLGiRjl9bY3n7Sa4b1pUsCxNtTEbISWCbQcBoYLGILHTLbgWuFJHu\nQCmwGvgZgKoWi8jbwGfuuidV9XMAEbkWmAJkA0+r6pJUfhmvM3q05r0ozTe52TVy+kO1cN2wbuk+\nBWNMguJWAKo6E//2+8kx9nkQeNCnfHKs/VLp+I7NolYAVc070xiwQHPGmGqhxj4KR2uFeGL0CVV7\nIsBrC9eFfN6932bKGmPSr+ZWAFFqgOHHto2537vXnxpcrp+bzcldWgLOJLLyPrnf/PLikM8X/vWj\nch3HGGNSqcZWAH7pCm8bdUzc/Y5u3ZiurRsBkJMtlKpz4+9862R++sy8cp1Lq0ah4ZKXb9kbOf7V\nGGOqWCKdwBnJb7bvT07pHFH2g/6deH7OGgDO7uW8HQRuznWys9h/6AiFtzjdFu8WJ96ncLDkCHVz\nsgHY6hOcbsbXW2pUHP2nLi+ifp3sdJ+GMSYJNfYN4NM1OxPednjPNgCc18eZmBzotG1QN5tFaxOf\nNBawetteut/2Ni/PXxtS3rheaH27fa9/1NJMNPSYNpzcpVW6T8MYk4QaWwFMWuwf2yfcp2t20NJt\nogncoAOx7Ns0rleu3/3srNUA3BAWXuL5nw4s1/GMMaYy1NgKINzfLyvyLf9uUUduP6cn913YK9jh\ne9VpXVh13yjf5DCJPLU/OXNlyOcrBhUA0LVNoyTP2hhjKk+tqAD+ccWJnOk28wR8tygfgAZ1smlQ\nJ4fv9++UUOTPP777NXNXbfddF2g66tSiQbBs854D/OOjVQDkxEpGbIwxVaxW3JFO7x4ZrOzqIUdz\nbPsmjDwu9rDQcP+atZpLHp8VUf7xsq30uP1tZq/YRk52WUWye39ZRjGLkGCMqU5qbAUQGNETTUGr\nhkz65Sk0a1C+kTgF4yaxc19Zc9C/Zq0C4Mp/zaOBZzTMoZKyAZ8iwmvXDCrX7zPGmFSrsRXAQ989\nvtJ/x/F3liV4mbLESYP47cESLhtYECz/9JvQfoQ+HZsFl1fea1EzjTHpU2MrgHq56RuTnptT1tYz\n/tXPo26XqmxjxhhTHjW2Aki1BjbJyRhTw9ToCuCu849l0i8Hp+RYr18b2XY/oNA/r8CE/8WPcn1h\n30rPhmmMMTHV2FAQAKNPKkjZsY5u3TiirHUT/4liew7Gjva56r5RKTknY4ypiBr9BlDZDpU44/5D\nEr1HcVLnlpV9OsYYk5REUkJ2FJHpIlIsIktEZKxbfpeIfCYiC0XkHRFpH7bfiSJyREQu9pRdLiJL\n3Z/LU/91qtaUJZtYsn5XQone9x2yHADGmOolkTeAEuAGVT0GGAhcIyI9gQdVtbeqHg+8CUwI7CAi\n2cD9OOkfA2UtgDtwcgT3B+4QkeYp+yZpct9bX0Zd5w0DPfSYNlG3M8aYdIhbAajqBlVd4C7vwUnw\n3kFVd3s2awghIe5/AbwMeOMnjwCmqup2Vd0BTAVGVvD8q9SDF/fm6R+FxhT6cOnWqNtfdEJ+cPmq\n0yJDURtjTDol1QcgIgVAX2C2+/luEfkG+CHuG4CIdAC+AzwetnsH4BvP57VuWbXVq0PTkM+XFHVk\n8NF5Ce171amdOadXWatYIDeAMcZUFwlXACLSCOep/rrA07+qjlfVjsBzwLXupo8AN6vqkfBD+Bw2\nIjGWiIwRkXkiMm/Llvidq5WpXdPIUT45UQL6DOwcOiT0ppE96NEucuSQMcZUFwlVACKSi3Pzf05V\nX/HZ5D/ARe5yETBRRFYBFwOPicgFOE/8HT375APrww+kqk+oapGqFuXlJfa0XVku7Bf5ghIt1/DE\nMSeFhHnIzhLftJTGGFNdJDIKSICngGJVfchT3tWz2XnAlwCqWqiqBapaALwEXK2q/8PpEB4uIs3d\nzt/heDqJq6Ohx7ThikEFzL9tWEj5R+PO8N3+3u/0CvkcrbIwxpjqIJGJYIOA0cBiEVnolt0KXCki\n3YFSYDXws1gHUdXtInIXMNctulNV/QPrVxO52Vncce6xEeUdmtX33d4vD7ExxlRXcSsAVZ2Jf/v9\n5AT2/VHY56eBpxM9uUwTSAhjjDGZwGYCl9MDF/WOKDtYUgrACUeVTW9o1iC3ys7JGGOSUaNjAVWm\nfkdFzmHr2trJ+fuz07oEy6ZdfxrbEsgjbIwxVc0qgHI6unVkgvfmDetEBHpr2aguLRvVrarTMsaY\nhFkTUAW0shu7MSaDWQVQAXWybdSPMSZzWQVQATnZdvmMMZnL7mAV0NIT7dMYYzKNVQAVMHaoMxn6\nO5be0RiTgWwUUAUMProVVw4u5MrBhek+FWOMSZpVABWQk53F7ef0TPdpGGNMuVgTkDHG1FJWARhj\nTC1lFYAxxtRSVgEYY0wtZRWAMcbUUlYBGGNMLWUVgDHG1FJWARhjTC0lqpruc4hKRLbg5Bsur1bA\n1hSdTk1l1ygxdp0SY9cpMZV9nY5S1bx4G1XrCqCiRGSeqhal+zyqM7tGibHrlBi7TompLtfJmoCM\nMaaWsgrAGGNqqZpeATyR7hPIAHaNEmPXKTF2nRJTLa5Tje4DMMYYE11NfwMwxhgTRY2sAERkpIh8\nJSLLRGRcus+nKojI0yKyWUQ+95S1EJGpIrLU/W9zt1xE5E/u9flMRPp59rnc3X6piFzuKT9BRBa7\n+/xJRKRqv2HFiUhHEZkuIsUiskRExrrldp08RKSeiMwRkUXudfqtW14oIrPd7/yCiNRxy+u6n5e5\n6ws8x7rFLf9KREZ4ymvM36iIZIvIpyLypvs5c66TqtaoHyAbWA50BuoAi4Ce6T6vKvjepwL9gM89\nZQ8A49zlccD97vLZwFuAAAOB2W55C2CF+9/m7nJzd90c4CR3n7eAs9L9nctxjdoB/dzlxsDXQE+7\nThHXSYBG7nIuMNv9/i8C33fLHwd+7i5fDTzuLn8feMFd7un+/dUFCt2/y+ya9jcKXA/8B3jT/Zwx\n16kmvgH0B5ap6gpVPQRMBM5P8zlVOlWdAWwPKz4f+Je7/C/gAk/5M+r4BGgmIu2AEcBUVd2uqjuA\nqcBId10TVZ2lzr/YZzzHyhiqukFVF7jLe4BioAN2nUK43/db92Ou+6PAGcBLbnn4dQpcv5eAoe6b\nz/nARFU9qKorgWU4f5815m9URPKBUcCT7mchg65TTawAOgDfeD6vdctqozaqugGcmx/Q2i2Pdo1i\nla/1Kc9Y7ut3X5ynW7tOYdxmjYXAZpwKbjmwU1VL3E283y14Pdz1u4CWJH/9MtEjwE1Aqfu5JRl0\nnWpiBeDX5mpDnUJFu0bJlmckEWkEvAxcp6q7Y23qU1YrrpOqHlHV44F8nCfRY/w2c/9bK6+TiJwD\nbFbV+d5in02r7XWqiRXAWqCj53M+sD5N55Jum9xmCdz/bnbLo12jWOX5PuUZR0RycW7+z6nqK26x\nXacoVHUn8D5OH0AzEclxV3m/W/B6uOub4jRHJnv9Ms0g4DwRWYXTPHMGzhtB5lyndHegpPoHyMHp\nlCukrOPk2HSfVxV99wJCO4EfJLRz8wF3eRShnZtz3PIWwEqcjs3m7nILd91cd9tA5+bZ6f6+5bg+\ngtMu/0hYuV2n0OuRBzRzl+sDHwLnAP8ltHPzanf5GkI7N190l48ltHNzBU7HZo37GwWGUNYJnDHX\nKe0XrpL+Z5yNM8JjOTA+3edTRd/5eWADcBjnyeFKnPbFacBS97+Bm5QAj7rXZzFQ5DnOj3E6oZYB\nV3jKi4DP3X3+gjuJMJN+gME4r9CfAQvdn7PtOkVcp97Ap+51+hyY4JZ3xhnltMy9ydV1y+u5n5e5\n6zt7jjXevRZf4RkRVdP+RsMqgIy5TjYT2Bhjaqma2AdgjDEmAVYBGGNMLWUVgDHG1FJWARhjTC1l\nFYAxxtRSVgEYY0wtZRWAMcbUUlYBGGNMLfX/K0B8cHtXdgQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11ad58a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(data['SP500'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data is rich of frequencies as is evident from the plot. Also, one can observe that the data has quite large values (mean around 2400). This can usually be problematic for machine learning algorithms , especially in the gradient descent step. Hence it is important to normalize the dataset, so that the values are sufficiently small. \n",
    "\n",
    "But before this step, we need to first formulate our problem, and decide what the inputs and the outputs are going to be. As a simple start, let us try to predict the next minute data from the previous ten minutes of data. This is a very standard problem in dynamical systems or stochastic systems where the state is estimated using the current and previous state. Another usual assumption in many stochastic systems is the Markovian property, which means that the current state alone contains all the information needed to estimate the next state. But here, we are using a sequence of previous states just as an example to implement LSTM. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Obtain the number of training samples and decide split factor for train-test split\n",
    "nt = data.shape[0]\n",
    "split = 0.8\n",
    "ntrain = int(split*nt)\n",
    "ntest = nt - ntrain\n",
    "window = 10\n",
    "\n",
    "# Only store the SP500 prices\n",
    "data = data['SP500']\n",
    "data = data.values\n",
    "\n",
    "# Form the train and test input data\n",
    "X_train = np.zeros((ntrain-window-1,window))\n",
    "Y_train = np.zeros((ntrain-window-1,1))\n",
    "X_test = np.zeros((ntest-window-1,window))\n",
    "Y_test = np.zeros((ntest-window-1,1))\n",
    "for i in range(ntrain-window-1):\n",
    "    X_train[i,:] = data[i:i+window]\n",
    "    Y_train[i,:] = data[i+window]\n",
    "\n",
    "# Form train and test output data    \n",
    "for i in range(ntest-window-1):\n",
    "    X_test[i,:] = data[i+ntrain:i+ntrain+window]\n",
    "    Y_test[i,:] = data[i+ntrain+window]    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(33001, 10) (33001, 1) (8243, 10) (8243, 1)\n"
     ]
    }
   ],
   "source": [
    "# Just see the new training and test size\n",
    "print(X_train.shape,Y_train.shape,X_test.shape,Y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the input and output data nicely divided as test and train, we can proceed to apply normalization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# normalize the training and use this parameters to normalize test\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "Normalize = StandardScaler()\n",
    "\n",
    "# Fit transform on training\n",
    "X_train = Normalize.fit_transform(X_train)\n",
    "Y_train = Normalize.fit_transform(Y_train)\n",
    "\n",
    "# Transform on test\n",
    "X_test = Normalize.transform(X_test)\n",
    "Y_test = Normalize.transform(Y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnWeYFFXWgN8zOQJDlJxFQAXJZlRE\nMIGuAQyYWQOucc0R3V3DuqbVdVFR9DOjmBVREVYFBBRUQJScYUjDkCb1/X5UTXd1mu6Z6ek0532e\nfrrq3ltVp6urTt0699xzxBiDoiiKUn9IibUAiqIoSnRRxa8oilLPUMWvKIpSz1DFryiKUs9Qxa8o\nilLPUMWvKIpSz1DFryiKUs9Qxa8oilLPUMWvKIpSz0iLtQCBaNq0qenQoUOsxVAURUkY5s+fv9UY\n0yyctnGp+Dt06MC8efNiLYaiKErCICKrw22rph5FUZR6hip+RVGUeoYqfkVRlHpGSMUvIm1FZLqI\nLBGRRSJyXYA2IiJPicgyEflZRPo46i4SkT/sz0WR/gGKoihK9QhncLccuMkY86OI5APzRWSaMWax\no81woKv9GQj8BxgoIo2Be4F+gLG3/dAYsyOiv0JRFEUJm5A9fmPMRmPMj/ZyMbAEaO3TbATwirGY\nDTQSkZbAScA0Y8x2W9lPA4ZF9BcoiqIo1aJaNn4R6QAcBszxqWoNrHWsr7PLgpUriqIoMSJsxS8i\necC7wPXGmF2+1QE2MVWUB9r/WBGZJyLzCgsLwxUrKVm31vDh9V/Dnj2xFkVRlCQkLMUvIulYSv81\nY8x7AZqsA9o61tsAG6oo98MYM8EY088Y069Zs7AmnyUt3Q50MeLJ46k4/YxYi6IoShISjlePAC8C\nS4wx/wrS7ENgjO3dMwgoMsZsBKYCQ0WkQEQKgKF2mVIFe/enArDy6xUxlkRRlGQkHK+eI4ELgV9E\nZIFddgfQDsAY8xzwKXAysAzYC1xi120XkQeAufZ2440x2yMnfnKzlG50ibUQiqIkHSEVvzHmWwLb\n6p1tDHBNkLqJwMQaSVfPyWhVv01eiqLUDTpzN84oKfEsv7BheOwEURQlaVHFH2fscvhLvc25MGlS\n7IRRFCUpUcUfZ2zd6lPw/vsxkUNRlORFFX+csWaN93r5vrLYCKIoStKiij/OKdqXEWsRFEVJMlTx\nxxmVNv6WOTsBeHNmyxhKoyhKMqKKP8445xzr+9q9DwPQsm16DKVRFCUZUcUfp5zY0Jrztnl3Towl\nURQl2VDFH0e4XNZ3Q3bScLrlzXP1jr/HUCJFUZIRVfxxRKoVooer+A+tu+UB0J5VsRNIUZSkRBV/\nnLBokWd5O43JyYFWeUWsoR0fnv2K53VAURSllqjijxP27fMsN8Ly6NmwuyGGFEZMHgO//x4jyRRF\nSTZU8ccJs2Z5lu/5arB/g+LiqMmiKEpyo4o/TjjoIOt75hG3kXv8QP8GZTqDV1GUyKCKP06o1OsZ\nBzR2l6WlWnb9PIpV8SuKEjFU8ccJ5eXWd3qGJ/XBCYdbhv/d5HP54D90gFdRlIigij9OqOzQp2V4\n/pLJn+XS44BtALzI5dyfej9m3fpYiKcoShKhij9OqFT8zh5/Xh607+RJknYf9/Nd23OjLZqiKElG\nOMnWJ4rIFhH5NUj9X0Vkgf35VUQqRKSxXbdKRH6x6+ZFWvhk4pefLFtPum9oHvHOejmGV6IkkaIo\nyUo4Pf6XgWHBKo0xjxpjehtjegO3AzN8EqofZ9f3q52oyc3fH7F69g0/+j+v8s07vJ8E3VkSNZkU\nRUlOQip+Y8xMYHuodjajgTdqJVE9p9GY073Wf1yc7bX+Kad4z/ZSFEWpJhGz8YtIDtabwbuOYgN8\nISLzRWRsiO3Hisg8EZlXWFgYKbEShvzMElqygfSLzw9Y/zTjPCu//RYlqRRFSUYiObh7GvCdj5nn\nSGNMH2A4cI2IHBNsY2PMBGNMP2NMv2bNmkVQrPjk7zdvRwQOkt9g7lyKSzLZSCvIzPRqN+2TUk5t\n8j3XzLvUXVby+LPRFldRlCQikop/FD5mHmPMBvt7CzAFGBDB4yU0dz5mTdRaykHs+8/LAPRhPrRt\n69VuyMkZfLT1CKRvH1o2s1x/thWloSiKUlMiovhFpCFwLPCBoyxXRPIrl4GhQEDPoPrAZxM3sm/O\nz+zZA8O7LfeqG/rlLQA0pMjPi8fJ409bA72b5YC6E1RRlKQnZNdRRN4ABgNNRWQdcC+QDmCMec5u\ndgbwhTFmj2PTFsAUsRRZGvC6MebzyImeGOzaBQ0bArQkheZYc287e7X5dm17AKZzfJX7qpzd2+eD\nezGRFlRRlHpDSMVvjBkdRpuXsdw+nWUrgF41FSwWVHa2zR/LoEuXWu+veJdh8IC9QC4ALlKrbN+H\n+UDfoPXdujl3Xgz5+bWWUVGU+ofO3A3ECy/UehebNkGDhsJPS3ODtjkt50uv9WOZUeU+W7XyLJdf\nfmWt5FMUpf6iij8QzZtXe5Nlv7t46bxpsGULAC1betd/+eJq9/KuaXNY+/UfDL5tkFebRx6u+hgF\nBZ7lfRt3VltGRVEUUMUfkH1N2/LaLQvDDohmDHTtlsKlb5xI4dlXe+VMGckUVj39Eb1Ot+z4d/EA\n+UMG0ua4rojPQG7aMUdUeZzsbLjiAmvyVumQk6vxixRFUTyoX6BNUZFnucElZ1LuSqXji6dxxLaP\nQm77wQee5eYzJ3Nq6xnAsQxiFlPMGe66wtV7adzweve6+D52Bw0iFH36p8L/QVmpDu8qilIztMdv\nc/vtnuVylzUIm7I9vBnED97lHULh4+JjAbiS57zKm7bLIaWhZ0C2osLT49868vKwjpWeZf1lZSUa\nm19RlJqRdIq/c7MijpUZlh9lNVi50r8skxL+e8tyNrw2vcpt5y/KDljekKKA5ZX885ks93KTe64J\nLSSQnmk9lEr3VYTVXlEUxZekMvUYAyu2NmQFx0LRWmjQIOxtS0v9y/rwEzwKV9IZM2QztGhR5T4a\nspMiGrnXT1/0UJXtN212PHd9QjUEozJev5p6FEWpKUnV4581y7GSWrXPvC+hwgP92uMcwPL1F4FV\n598JwP79njb/xwXu5Q2jbyKlx0HhC3BQeG0zMqxvNfUoilJTkkrxe2Gq1yM+N0Riq0O2z+DPJ691\nr7/2uvW9xzFXuUsHy/zy8YDxtHz9sZDHnPGNQ8aU8P6KykQt2uNXFKWmJJWpp1IJp1NabcUfDhM+\n8wRQK80pwOWC/XsqgFTSKKPb509iugHcE9b+jjlW+OmLQnp02g+0DdkePIq/tEQVv6IoNSNpevx7\n9sA1l1t2lzIycK1cjTHw/mPLcK1YFfHjjd97M6mp0Ka9ZVJ6tddjSLcDq72f3ic2I6NzeEofYN06\n6/vNKRnVPpaiKAokkeLPyTb8scbjJdPxmDakpMAZN3fhvM6zQ24f7AVh39Y9gSt8yM8qC6tdbanM\nwfIk1zPj9NDmJMUbY+DnT9bC7t2xFkVR3GzcCHlp+/j50ieicrykUfyS4j0Ldg3t3ctvMarKbVf8\nVsryaSsC1mU1yaVTJ89TYc6t73HPrSV+7XbNWVwdcWuM0/nngo/Oicoxk4nLLyyh16ltmdHx4liL\noihuzjvPsKcim3+8VLXnYKRIGsVfFefyZpX1nbtncMtznbzKnj3qdda/+jUACxZYD5Wph9zMgIfO\nZP1Wf9fLFKLjZZPuyL2+g4LgDYPwxI1rmHrepAhKlFhMfM367wZvnVwn40CKUhNaFFgWgx5EpwOZ\nVIr/gb8EnmnbgMCTuc45cUfQvCdX3deCVhdY8fHz8y0dMfTnfwJw2mlWm7/jme571vjoRKBu0sSz\nLDWIyn/D4+0Y9sZF8OWXoRsnGUW+8+m8/H8VJXY0yrLGJwvYEZXjJZXiv+vJZrwzaa9f+TK6QEkJ\n29buxVW8h4oKyxf/nS+r6DFnBB88HTEC1izcwe177mZ3sWHXsi2k3n1HJH5CSC68MEI7mjw5QjuK\nf+6/uRgR+Pbm970rNm+OjUCK4sP+XdYM0hLCm8hZW5JK8QOcNSbHr2w6x7Nw7DM0bZdDaoNc9hSV\nh96RHV45GG0PLYCcHHLzhPzO1Q/jXFOcNv5APf7fF5cz847PoSJESIf58yMsWfxy32NWfKSRL5wC\nwKCUHwAoK9TQ1kp8MOmTpgDMYWBUjhdS8YvIRBHZIiIB8+WKyGARKRKRBfbnHkfdMBFZKiLLROS2\nSApeXXq/cqN7+fUm40JvMGRIHUpTc9KqmHmxezd065nGsf8YBhMm+NV7xSO6+urICxeHOM345VbG\nUJq5NgGwa8v+QJsoSlT5cYYnjvs7RMdhI5we/8vAsBBt/meM6W1/xgOISCrwDDAc6AGMFpEetRG2\nJlx5uX/v/io7auafUt/3q3OTlRW8LoY4B3d9B5SdmRjNqtX4snatY6UGyWYSkb3+lj/OfORwAIoK\nAwRoUpQoYgz0HRz9FKohFb8xZiawvQb7HgAsM8asMMaUAm8CI2qwnxrTlEJOHB68i9yvwt+/P59i\nbsz8t7eGjSOcg9FX8HzQdpMe2eRXFioeUTIy6nRvzf+PZo/RqKt1Iq57qhPsiM5gmqIEYk9404Qi\nTqRs/IeLyEIR+UxEetplrQFnH3OdXRY10iivUn8fziw+e7uYXgVr3GULht3GY/vHhR07J5bkNPD+\ncS0KPPMLLuFlv/ZePf56wIKfDB9/7T3mU1pYRMOG1vLHnAYzqs5zrCh1yUeh8zzVCZHQbj8C7Y0x\nvYCngUr7SSBHyaD+hyIyVkTmici8wsLwEqCEojcLGDYMrh/2m7ts/Yxl7uVjP76FYWfnc/kdHrNH\n2np/E0nc4uOL2rapd0KY8+R1tsz2TEybO9dRmeQ+7DM/3c1hffwvwV4s9Frf5ipg4QtzQw+GK0od\nUM0gwhGj1orfGLPLGLPbXv4USBeRplg9fGcQmjbAhir2M8EY088Y069ZLW0Sy/4wDCj4g/c/TCU9\nHR7/7CBeeWg931z1FgX9u3gaDh4MQEqqR0G0fnF8rY4dTYyBy87cjgiYKe/TqpG3WeMNzqPF4Z3g\n228BOOSQWEgZG449JS9geU8WkZvrWR9w6cH0vqI/PKbhL5ToEyoqcF1Ra8UvIgeInTVcRAbY+9wG\nzAW6ikhHEckARgEf1vZ44dC5izBne1cyTxvqLrvw1tYc++y5ZGfDZ5O2sP6NmVRqgGnTPeMAqf37\nREPEiGCAiVMaA3Dkmc0p2VtBf37wbzfNmqy1v546sZzJu+7lRuykf39P3Yoia0acmf9jtMVS6iE7\nt5azZ6W/q/glTIyqHOG4c74BzAK6icg6EblMRK4UkSvtJmcBv4rIQuApYJSxKAfGAVOBJcDbxphF\ndfMzqsewMc1pNeoY9/r7H8XofauGfP/RNgCM8bypzOIISjbvJJMSmjT0Dhi3evzLAIwaldzmnWB0\nZCULFxju6/sRTTcvRgSuG72Fhnj8+L9/u54NgCh1yo8/WpbYGU3O9CovaJZGXqfmfqlhKztw0SJk\nPH5jzOgQ9f8G/h2k7lPg05qJFn1asAk4INZihOTwU5uQKv426RVbG3Agm7hu7H7uedQz8PseZ3Jd\nhfeDItlt/E7u/GAgBb2EQ+ed5i6b8k0jivDMzp5PX46MhXBKUtK3r/U9ePt77oFN55DcXQ2f4kFz\nF2kpFYx2vQYjx/DJJ7B0aXTki3/XlSgwYID11zzFX2IsSfV4pPhKr/U1tCeNcm67ybvHfxP/YsGC\naEoWXxScfrRf2ZqN3iE5WjUM4PCvKJGgxD+a79+4CxEod6XSPtMKHXLyyXDDDdERSRU/8MokGNHu\nR06fGsaM3jhBMJThH0/oc4aTnpfJ+nWGIQd5zBfOUA/ZJLeSq8nLTFmjejjJQYkOxcWsXx+8ellJ\nm+jJYqOKH+h2kPD+6j5kDT0mdOM4IVhUUQAyM2nVWnjjf5ZTVQOK2LvbM8s3E/8eSDyydKn1O/8h\nt4du7OB5x7y2zWddE9Y21665GRHreCXPvFCt4ymKL6f0sRwYj2YmFBezyX8+pRsT0PO9blHFn6CU\nuaoYnrED+jRtCoe2KuRwZvHU/dYM1UMzllhtYmTj/3leKZ9f8xGUhw6U969/Wd938I9qHeObrzzj\nH80fD++hsc144l1njbuc4hffrtYxFcXJ/v2WMi8mn2bdmzCon2V+fYpr/doGCxtfl6jiTzK68rvX\n+s8bmjGVYbz2uaXY8kq3x6SHUUmv/hkMf/Y0eOedkG2d7qemOLxUiVM/KeeNtx1eWkFmYE95u+pU\nmYuveSas4ylKIPbZin8Bh7G1pIE7QOAxzKS42LvtqW0W+m5e56jiTzJ+OrDqNJP5FFdZHy3M9h38\nqe9KPj312aBtXnnFs/xIgwfD2u+wU33ehIIo/pNHVB2LKVYzKpXEZ/16+H5FYO/Ag3sY8vJgz84y\nKvaWULy1hNNXRCfPrhNV/ElGbvd2VddjR4WKsTvnuLsb8d6PHTnlk6spW7iYopWeOIDlpS5mf+ft\nrvpZyACxQQgyGJKRATnpwaNzpuyNjwekkni0qWKsNvV/3wCQ0zCdlOxM8ppkxiQgpCr+JOAtZwzv\nF1/0qvthtnfo5my84/nEimd3nOdezujdg0adGjP5sL9xdKNfSM9M4fCjvLvc4eQ03l8cwHxTRbC9\nPaUZfq/dldQkraWiVMXfuR0aR3eiVjBU8ScBzXFMAXcm5QX6D/T+ixsNPCimNv6qOHvBnXxbFDig\nUKXi/+S9EvpnLMAsXsLKxfso+nE5AIsWQXaDAD2nEFFWgyW2iddzpCQunVgRulGUUMWfBJTbE7DP\n5c0q293FA6R3rtoUFK98xRBwuTj1T5nMK+tNv5576dQzm0Z9O0NxsXfkUSdV+r16v2U/wl/p08vy\nNspvFx89MyV5aEhRrEVwo4o/CchmH3t3lfPa7pEB62dMKyFH9jL2wfaewhjZ+HPSap716o17PeG1\nf6Svp2LVKrKzPasPcqdnJUSP3zmI+9f78hh3rdU+/YxTayynojjJscfV8jPiJ+ObKv4k4IiHR5Kd\nn0ZqbuB0kccMyWSPK4e2d44BiZ0Zo7wc9pZ7ZhsvwpOJ83Q+CLn9eQ8Gydy5YgWjHM5MXr8vRI/f\nC5cLSVETjxJZOjSzZsrnZYWeuxItVPEnAXLy8PDb1qEcodi2zXu9+91nu5cfvr6KqY1VILgwOz2v\n0CfwJTdNPcnToDqZ1ByB+mdMXB5zzyclMenefCsHsJGyX37DGCjJaQRATnb8XE+q+BOcVX3OhIMP\nDrv98nUZFNMAs2x5HUoVmN32HKyXOt5PRWkFMv5+7rl2BxekvM5BD5zPR8+u5fc35pMiHg8ekeA3\nSxO2Ykhh1BWepCtfruhM9lBHUDanDSgUWVnuF4SLi5+GJ58Mf1tFsSmvEAbzDWkHdgJgn+1Il5Ud\nP2+TqvgTnPxG1Ztp9OEMq/ex6q8BI2nXKRdfZCnxhSsbkJJuyX3/UwW8WnEe5OVx6lVt6TqqLy5j\nXZan8hFTPwxuFy21g9S9XeaIed6xIwC/zS1m67e/BXfbCUSfPl6moY03PBz+topi4zJCKhXuAaRX\nJsHR+QtoOfFvMZbMgyr+BCcjvWavj3vIDd0ogmzcCN9+ZynVAwjPrHMKn3DiqZlB6286Zp7X+kim\nuJe79cunyZEHhXWc4l2GrQvWwVFHeQ0JrCExPaCU2OJyieV+bJsZTxiWzsxdvUk7zj88eKxQxZ/g\npKfVTPFP4qIIS1I1rVp5lv+c+1pY2zRlq1+Za+Vq93JRXmuvurFMqJFseflCk17+0y2bX3xKjfan\n1G9cxp53Uh3HgihTjfdgJR6pqeJvx5oISxI+edeE99DpPcC/ty8Fjbj6rM1kTf2QlC7e+xk0JL/W\nsjnvVVdeg1rvT6l/uIyQEuczv8PJuTtRRLaIyK9B6s8XkZ/tz/ci0stRt0pEfhGRBSIyL9D2Su1I\nSavZS1ssZxGm/X18lfWzpu7ih/s/o8vs//OvbNCAZ95pwWO7rqAi1eMaOuGIlyj4+NVay7Z5s2fZ\nVRHfN68Sn7iMVOmUEA+EozVehiojZK0EjjXGHAo8AH7v28cZY3obY/rVTESlSqrjruiggtiEn2zG\nlpChLwcNbUD/e4a7u99jRjrilTu65JUOO/9lLFd8d4l3mrEaMm2aZ7nCP62xooQkKXr8xpiZwPYq\n6r83xuywV2cD0c8jVg95+M5dNGErjKtZushoKv69jkyPl1P97FYTJzegX+uNvNL7X17ld9wB95+5\ngEt+uLq2Irr56ivPsvb4lZpgDKQkQY+/OlwGfOZYN8AXIjJfRMZWtaGIjBWReSIyr7CwMMJiJR+3\nPNiAraYpHFO9dJETnrVmD1Z0Cc/jJRKsW+dZfnDdJdXePjUV5q5ryYU/3ehVnpsL97zbm/T+vWsr\nopv2jqgWrvLQEUEVxReXkfqj+EXkOCzFf6uj+EhjTB9gOHCNiATVUsaYCcaYfsaYfs2aaeLruuLI\nY63x/EeWneGZUVXHOBV/SuNGUTlmTennMEhWqN5XakC9UfwicijwAjDCGOOemG+M2WB/bwGmAAMi\ncTyl5lSa1+cywDvFVR1y222OlYyMoO3iAWcqYJfa+JUaUC8Uv4i0A94DLjTG/O4ozxWR/MplYCgQ\n0DNIiR5e46pRyvyzaJHjJojznIbOAV218SvhMvfzbRR+aeXOTQrFLyJvALOAbiKyTkQuE5ErReRK\nu8k9QBPgWR+3zRbAtyKyEPgB+MQY83kd/AalGngNn1QnnEEt2LvX8sT5iuOjcrzacMUVnuVv3qpZ\n4Dil/vDMI7s5P2syA4Y34ZQTS8CYhFD8Ie98Y8zoEPWXA5cHKF8B9PLfQoklXi6KUc712YoNUT1e\nTTjlFJgwAcaOhZt5jJtiLZAS14y7NQ84C7DMp7temYLLDCclJb4Vv4ZsqGd06+ZYiZLZ5cJTLW/g\ng566JirHqy0//BBrCZREYNtWf+Xe7eJBSTOBS0kivHI9RymWSOP8chpQBG3bRuV4keT3X0tZ8Ph0\njc2v+NG0mf/9s4mW7C3PTHxTj5JceE30jZIyMwYEE9dBq5w4n0/dDskAjsN0/xyGVTWBXVE8xLvi\n1x5/PcOpe01pWfSOm0CKP9A0kpI3p/gXKkoQ4j2Dpyr+esi5Q60IG4f/pT/35T8GJSV1erxEs5L0\nsl0SBM8Mrs8nqYePEpiPznvDr0x7/Erc0WeAZeGbs6s79+++CR5/vE6PZ4xJqB7/EUdY35fwkrts\nPa2DtFbqK/3abWYYn3HqE0PYU+zi1EGe/BGyd08MJQuNKv56SHqWz9/uDFBTBySajR+gfcMdTOc4\n9/o1PBtDaZR4Y+tWmLemBZ8zHHJzyclLYdInTd31j1VcF0PpQqODu/WQjCwfN86Cgjo/ZqIpfhcp\nrKZTrMVQ4pQVznQWdnxwp8fcPnKiK1A10R5/PSTdR/Gb5XWblMUkQbCzBhTFWgQlDnn/kLsDdmiG\nMjUG0oSPKv56iK/if27cL3V6vEQ09YiPrFekvRwbQZS4ZPI71uDtnl+WB6x/77a50RSn2qjir4dk\n+Nj4r+Y/dep6k5CK3+fOKE2pfXYvJXl49J/WtTy46aKA9TmN4jsKrSr+ekjAED379tXZ8RLNqycQ\npRXxHVVUiS6dDrDSyrW854qA9ZKdFU1xqo0q/npIQMVfVLc27ERT/L6mnhUVdev5FG2mPL6K3bPq\n1sSXzKzYlEMGJciA/oEbZKniV+IMZ0asSs5r5R+PZudOePXaH2DjxlodL9EmcAHg84yaxlD49NPY\nyBJhliyBM2/swOVHBDZTKFWz3c5AXkomNGgQuJEqfiXeaB1gLtIbnAd7vCedXHFpBWP+PYCFR9Uu\nqqZxJV6P32UC3BoPPhh9QeqA0lLrezE9YitIgrLVnqd1DDN8wt3CmJM2czQz4fj4zj2hir8esnVr\nkAofO//mzdb3zlU7a3W8RBzcXbMt169sx6DhMZAk8lSa+kqI/oD17t2wdvqyBH0NtKh8Ab57yGyf\nqIcw6fMWzDTHQJs2MZAsfFTx10Py8z3LqTiSzO7d69UuO8u6Ofe6aq8gEk3xB6Lx43fDd9/FWoxa\n87//Wd+/0w3274/qsQf3K6bd8V3g+eejetxIsmG9dV+0bBfdREaRJCzFLyITRWSLiATMmSsWT4nI\nMhH5WUT6OOouEpE/7M9FkRJcqTnOTsrwlC88Kz6mnvQ0a+ZVeS0neCdw586fXxM/bXTDho6V3buj\neuz5S61ex/Q/+wc2i3e+n17CmZ0WcN75VgemdcvEnZkYbo//ZaCqYOTDga72ZyzwHwARaQzcCwwE\nBgD3ikjdxwdQqsTZ8c7q6QhL4BOl85MvLF/kCmrnypiIpp6gBLWTJQarVsFoZzLVSoN/lDme6VAW\nvbDgkeDI4zOZsrK3e73RsYmbWTYsxW+MmQlsr6LJCOAVYzEbaCQiLYGTgGnGmO3GmB3ANKp+gChR\noFL/jmQK6d06eyqCvPafwfuWi08NSXQ//p44evl33RU7QWrJSf230bGjT+Hbb8dEFsDPtBjP/DY/\nQLTNE06IviARIlI2/tbAWsf6OrssWLkSQ1z2G2oa5aRmOcw4n3wSfKPFi2t8vETs8Tdu6Bn7OI2P\n3MvmsstjIU5E+GJeE//C5YFDDkQD89TTMTt2dbj/9n107+c92D+rx2V+A7uJRKQkD3RHmyrK/Xcg\nMlZE5onIvMLCwgiJpQSiZ0/r+7TG31Na6vmLKv72EB++Vkyn7A3c1uxF741mzqzVMRNN8ac5rFsd\nLvX07CoGHhEDaeqQESNidujSex6I2bGrw30PZXutp1PKoF9fiJE0kSFSin8d4Myk3QbYUEW5H8aY\nCcaYfsaYfs0C5b5TIsbBB8POTfsZs+Ehr4HXNMoZcUE+K/e34uGtl3ltM+/2yTU+XiIO7t56vTXe\nUfLCq8jAAe7yirLEHdBzcmbDr2jPKsiIXUyZh7gtZseuDRMa3ZpQnZhARErxfwiMsb17BgFFxpiN\nwFRgqIgU2IO6Q+0yJcY0bJEFmZnk5YXXvpCaP4wTcQLXjXfn4qowZFx2IUce6SmvKE/Apxj+D9/8\nvgfGRhAHK/EdcIhPzrFTlQJ7AfLHAAAgAElEQVQcws9cvOWRGEoTGcJ153wDmAV0E5F1InKZiFwp\nIlfaTT4FVgDLgOeBqwGMMduBB4C59me8XabECf/6V3jtXLXoIySijR9A7IzZPXvCo/cWA+AqT8we\nv9ML9ZN2V0FazV10/33zKn4ZX/vk82WE9oMv3lqC2RfduQa+NM33eD6N554gwa4Si7D+fWPM6BD1\nBgg4r98YMxGYWH3RlGjQqFF47fbWMqNQIip+J5UhHCpWBwh0lACccYZn+eTV/+HtkzZikBrZ4a59\nrAPQAXNP9eVo12Ana3ZZF103llbZtrAQmjfP5AHu4i4Tm3AZu3fDs++2AGDOXycz4I6XYyJHpEnc\nYWklYpz1p9A3fzi9s2Akoo3fl4eesgb4nn8qcVwQndx7r/X9OScBNX8G1/a/LHOlcIa8D0DjdvlV\ntt20yfp+k1G1O2gtcM5yH3Bai/B7SnGOKn6FFycG1wJDG/0AwLXU3PUuUU09TnbstG6Vv/LPqIc5\nqAmTn1zPnH98jYh12ifeZNl6ut90Sq3266qlpausIoX8VMsnfn9p1eqn8nIxAZ0DY0Bm8iTjUcWv\n0KABHD8w8NT9nAxrduV2AviAh0kyKH4ni7rEzgUyHMrK4OzrWzPoDk+EyG8KDwYgp4GPddfRhV+x\n3NA2aws/XP+6X/gO575rJVtFCnkpVjDA/WXhqZ/F9OShM+ZQNm9h7Q4eJr/M2s2CJ75h8yaf1xtV\n/EqycfO9gd17um63evxCzbt6yaD433CElvl5fePYCRIGCxYEr3OnBBT/nnTnLsK6kuYMfPI8Npxz\nfcDtp71R85AVO3dCUWkO2aU7SREXJWXBQ4Fcc+oqDjnEs377+wN5pH90ZhkfekQeh90wmHMGrfau\nSIJB3UpU8SsArFkTuHxw+TQA/sqjtdp/oit+Zwf4RsJ0hYoRN1xXEbQuq8TKtBbqn1j06Sq/sr17\n4fRLm3oKqmnwv+Zqq/0MjsVlUti521vx//uRvYjA57dO59lPOvhtfxd/q9bxasvM1T4y1NbOFUeo\n4leA4GNWuewhRVxkNPBkFLr6shJE4OfW4cWnT4bB3YsccWU30TJ2goTBd7OC96RTxOfPsP+c1T6d\n228Y7LdtZX4GN+Xlfm2qYuYM61jzsNIV/tt1DRQXu+uvvdXyHBv+yHHBd1LHyrfKmHXJcCHbqOJX\nAOjlCDSYhseQm8V+0qSCsgrPpfKfiZat8+UNJ4a170QP0gbebu+H833sBAlBZax9J+sbO2wmF1xg\nfTtMPe8+V8gJfbyn17Tv5O/pfdZpPoPa06Yx77NCds2t2i2zknUb/NVN2ZXXhrWtmzpOf7llS+Dy\n9//0qjXlPUlQxa8AcNBBnuUG7HIvZ7Gf9BQXZRVWL9KZpOtxbgy4r00bDfOf/t7dO0sGGz94BjaP\nz/shtoJUwccf+5e1fP0xSvYbSrbthgMOALz/irOuasby7d7jFg+sOM+rd71/P/y4yDuPbGlmPv1P\nbsbIAetrLO/+tz6o3gZeyQQiz/WX7fIr+/bIWxkx+cKEv36dqOJX/JjAWC78k2XU7vzkdaSnVlDm\nsi6Vk4/yuTEq/O3JvbqX0O8vR8ATTwDJo/jT0iAnrYT9FRl88K/luFasirVIfjTIKvErk+OPIyNT\nyGjsPYC/jrbI8YHNKutoCytXutezs/3btDvdek38jiP9K6tgNe3cy6VHWwHwinf4m436NV/tV1ab\nGceh2LwZ3v3CO3l6f37gyG8frrNjxgpV/IofhzOLVybnYgzk/eVSrx7/Nz963xhs8I+5t6XI6hma\nH39ylyWD4gfYW57JY/uuZuRNnXm28z9jLY4fednWg3jbA8/iqjC4KkwQb5TA/8Wfec69PLbLV0Hd\nOgE277WuhXDDeTTNs0xF7RZ4wlyX9ejFWxP30KCxv0KfPK0RuRk+RnefvNCRZNUq/7LRJF6msHBQ\nxa/40aqHd5K0tFQX5a7Al8rHna8Lup9Zr1mx3pNoTMyLDbSKtQh+LFxoKfTU/XuQFHHHG/JFfAd5\nbf7ziuet4HnGsqfvMbx2/7Iqj1lOelhx/ft03MEgZkF2Ni/813pAvTi7J6Mu88S6P4e32Li6lG1z\nV9D+0Ib8vMQneui+fVwweC0vHvJExAd6t23zL9sY5wP5NUUVv+KPj4tPeqqLMpMaUIOvKvMov8mP\nr2X9G564/cVY892TxdTjyyYOiLUIfrw5xVKUWSuX1Gh7yfG26fxz6alccF+XkNvt7NI3jL17rp+M\nLEv13PXjmV4tbuafHNAug8b9rJSgnTp5VTNzfi6vzWjL5b9eH1hT14LCzdaDZGqf291ltQlVEs+o\n4lf8udF70DY91WXdAAHs+RMYy7b3ZtCj1Q7OvrEtbc47xl33b8YByaX4+7fz+DS+xKWUzpgVQ2m8\nWbOinO5NLLeUzIuqjm9TXhH81r/pJs9yajPPjO2/8ghjxgR+U9hPVsByJ8bY/78I6RmBr4XOIw+t\nch9zP3UkaaplcqBKju68nld6/4stq6w4TIeP7sDDtlk/jeq5rCYKqvgVfw70jtXuVvwB5uv/wqE0\n/dOxLNlY4Ff3P452LyeL4p+7poXX+rWDf46RJP6075zGjxts00SI2AqTPm8RuKKwkMGDPav7cz2K\nvyFFdO/u+Q/zHd5fFed7JjpUxgdi40afndtuvQTO/2L27qPxlBf9yg872PNbbp5ztnu57KzIBG/7\ndkVrLlp4I7eMt8xceW0aufs4qQSfDJfIqOJX3OzYbtg6bxVec+WB3wqb8g7nBJ/eG4TrM/4DwIbC\n5HxdBpjAn2MtQmB27AjdJhCLF3PqqfDi01bv92+rzndX3dh7OlmOjn0xnoH+8hb+qbTNw94JS5w9\n/oBzvwK5DgEFQSJkZFDGdRnPBnwTDYdlfxi6N/J3TpD8PPeDqYAansc4RxW/4qZRgdCkb4eg9eag\ng+jfOXy7anlmLsuXw+xlzfiVQ5Kixx+v/DrD53/JCm16cXJF6kSe7vkcPPmktb8V/vkXsk88Kuhu\ny0v9B1p3pPgH9qvs8fvq6mF8FlS2rJzgauqpsqur9Dx6+4kNLBj/YcC6IwaU81tRgAH6Jk0YNw4e\nOOcXrv/unKD7TmRU8Sth8xGnkekKz50uBWu2r5eLnCr+OmH5cjhksI+SPf30au3jvytOZNyvV7r/\noyY+u+vGb3DnnUFzs5eX+dv+V/8RPP6Br+J//5PguX+zc6tWU+aLaUHrzr2hFaff2ztgXeFO/zfR\npxkHgwaRmQl3vXUImUeEM2ideISbenGYiCwVkWUi4pchWUQeF5EF9ud3EdnpqKtw1AV+9CoJwQg+\n5NuVbYLWt8aTnSo3rYSyihRycx0NklTxF06q2zACoSgqClBYjSTqZ/M20q6tV9m553q3eZULoWFD\nWraEZ5/wKPRDsMY4Ain+Ph+P94rF43YKE/EagujDfDJPPiGofE2bVn3dvHn25IDllQ+XtY4JYwDF\nRS52rwgcm2Hc7uSbrBWIkIpfRFKBZ4DhQA9gtIj0cLYxxtxgjOltjOkNPA2856jeV1lnjKleN0SJ\nS5qwlTl3e8cGOICNrJq7lVce3sgXF79OcXkOj5dcTdnLr8VIyujR7uIqgorVMfv3wynHeWcF+/G0\ne6u1j3T8B4K7dPH23u2/5FX3clq6RxH372CFaS4vC+xTv+s471eESlPPAQ5P2GYUUhWPPgpXHhc8\nHtBSunmtu1yw7tedHNwt8BtHg0Yp5Hdu7le+/a1pePdUkpdwevwDgGXGmBXGmFLgTaCqTBSjIUmn\nuykADOAHelw6yKvsGwaT1q83F97SkhNfOs9dfsl/B3oaJUDmqpqwn8CDktFgyhTYtMtjj59LPw77\n8P5q7eOCg8NIcOKw/aSme9RGQa6lXAP1+AE+nO+xobtbiDDcEdg1VIat/Hx49qtuXHnK2oD1LbBc\nbLdsgdQUF6mp0PaQRvy2PPy3nsV0p+Cc8IIOJgPhKP7WgPOMr7PL/BCR9kBH4GtHcZaIzBOR2SIy\nssaSKnFDOmVk5Hrso4OYxYE/vxuw7XIck3+SQPHfPC6+f0P3v55WrfYLOZThvzwStP73hftY88FP\n0KyZu2zmdw61sdTqiZc7XhoaZ3kGWy/k/6iogCHd1vL14pbuHj9YVqC+B6zj4T97YgIFQwQenNQ2\nYN2P9AFg7FhwmeAqrbQU/nLi4oB1HY8MqNKSlnAUf6DHcbBJ+KOAycYY59BNO2NMP+A84AkR6Rzw\nICJj7QfEvMLCql/9lOjy5VTvkbgPGUF6rqc3NWvCr8ghYYSsrcM4K9Hi4Sez2L15D6Ul3rfA6uFX\nxkSe8zwvV5gnniT3keqZebp1q7q33fXQbNqefphX2bvverbpfpB1Hip7/HO+3sP2/d7mkul3fcVX\nvzuUtj3Wk5cH8za2ofdz4Z07p7fnP27ZQQ+xlPgLXMG4U1Yif/wecLuzeAewIjo//aWXlZo3Ot3J\niQ3mkPlE/bDtVxKO4l8HOB+1bQB/51eLUfiYeYwxG+zvFcA3wGH+m4ExZoIxpp8xpl8zR+9CiT39\nBvon9pBMx2t0uBETk0Dxp6RAbvNcv5mnT38eOqxBndOjR+g2PoSa4RuIMWM8y3m2rXzih00QgUEn\nWEr/Wp5ytznxoeADt9XB6Up628MFLHJZv/dwvueZTzvy/uIDA26XgzUGEmiawKjCp/miaCDSLzm9\nd4IRjuKfC3QVkY4ikoGl3P28c0SkG1AAzHKUFYhIpr3cFDgSCPyupcQtOf4u3ZDqeBiEkYv0IW6F\nk06KnFBxxg78Zy5HkyP4Dk4M30a97HcX0276HG6+udrH+rNjzlppqqVNJ3KZV5un+UvAbSsInh0s\nFCkpcO8Fy5l7w+vusrYNdoYdSC010KE/+aTG8iQyIbtqxphyERkHTAVSgYnGmEUiMh6YZ4ypfAiM\nBt40xiuSV3fgvyLiwnrIPGSMUcWfYPjq9aYUAs04uX8hv8/dCUccEXIft/63c5AnSHIQ6xmeXz00\nD6oRF79z1xQ6/3NYjY7l/BsXFlYvQulecmrl1nvfq50Bj7V47a5GgHdQwTFMouPph/L6x/mUZeRi\n9lvH++St3YAn+ujsS/4LR8fpzOu6xhgTd5++ffsaJb6wnPusz6cMC9n+998qvLYxBQV1L2SUcf6+\nh7glJjIc1WWjOZ4vjVm7NmrH3LLF87vvHDjN6zxUfi7hxYDlXVlqzMqVEZMl0DHKvvnWXd+x0XZz\nIZOMcbm82jzIHRGTIV7A6oiHpWN15q5SbToS2gujazefSyvge3byECo65QcvbuWLS9+MeHKC8go7\ngmQdZqbyxTkEd1fpPX71Y5jExOLAoQ5q2+OvisE5P/DVn54h7Vj/N5/Zs73XF9GzTmRIFFTxK9Um\nI7UGCTDeeSfygsSYh2/yhGhuTpAs3TYjL2/KSS+Ngt9+i6gM5RUSdcXvJCvDxQF5xV5lPVlkuewE\nYC+RNfc1b+hJNTn9t5YcP/magO18Q/df1ObrgO3qC6r4lWqT3qV9WO16dbcm9+SyG69Yv0nCLf9s\n4Q5Yeh/3hbdRhLNGxVrxk5bGpt35XkUnDAneo99BYyjxzwtcU6Z+kwlAY7ZB28B+/gbxqnqRSxm6\nIPjchfqAKn6l2mRnhBcG99wLLZfPcfy7LsWJKZUWrC20CM+ME3FTT2wU/6Cediz+l17yKq+YPZe+\nXzwEQGaaFXv55FY/cWLBPE+j3bsjJkfPnnDBwD/49q6pAesrU0w6YwNdeE8npEmQWM/1hBh1E5RE\nJjM9vF5r5du+BJ3vl/g4Y87w/fdwZNWeNRtPuICWmxdE7PixUvyzfq2Mxd/VXZZBCSkHdnHb8Ddu\nTqX8jxU0G3gYX34J0yq9TXv1ipgc6enw6uyuXnIEwppgJnx81D9Iv/+uiB0/UdEev1JtMk8Nz1/8\niivg5uGLuOPj8N0ME40Uxx1Usb/qrFcAv22JrL9/eUVKbE09Dv7X6WIo8Py+gsZCs4FW0tymTa2y\n0bwek4H+ysQvGUFSPtY3VPErYfHgX63YvxvGv0DGvbeHaG2RlQWPftqT/FOOCd04genW0jo3Fw1Z\nF6IlTOKikG2qQ7nL7vHHgddU0YqtQet694Zfv9nK/20eGkWJPFSaepyRReszqviVsLjzkYYYAy3v\nvty7m6uwdGNDAF7jgpBtDyeyydnLK1JIk4q4yHUQKLyzk57HNiWledMoSePBIGzaZC2nmdBvZfUB\nvYMVJZKEyP+aRqBkszVnXXFDfjKBM0xFmx5xGI2l8nk4+nxL1ckXn8dQmvhBFb+iRJIQroqX8yJs\n3BjRQ/5khyWONc1HD4m1CCHZQ/1ItBIKVfyKEknCyTnw/fcRPeQZXgnvos+YEXbux2rm+Y0WThNc\npCeQJSqq+BUlkoSj+J9/PmKHy0gp48DcYFHSo8NL7zWktLgERlU/xHO0OeL8TrEWIS5Qxa8oteTW\nWx0rAXIO7NrlvV4xdVrEjm0Q9ySlWJGSAul5mTGVIRjLt3smag1kNi3uHhtDaeIHVfyKUktGOhOK\nlvon+F7wk7dijmSOXmMCp8hT/JnDIMiMzwdUtFHFryi1xJGH3Ds2AHDfdTs4drC3al5Lm4gd2+rx\nR2x3yU9W1VFU6wuq+BWlljRy5gEp93bXvP8p/5m63YlchE6DkBJjU09CoT1+QBW/otSaZs2gQZZt\n4glg6glIhIK1uUxKzG38CYUqfkAVv6JEhF37rUikFaODz94d1HyFZyXERK/qoJaeapCREWsJ4oKw\nFL+IDBORpSKyTERuC1B/sYgUisgC+3O5o+4iEfnD/kQ2UImixBnLV3nfUs7e+PV9ZgIwgDkRiUlf\n+dKgNv5qEAcxjeKBkIpfRFKBZ4DhQA9gtIj0CND0LWNMb/vzgr1tY+BeYCAwALhXRCIbnlBR4giX\nzy3ltL/3ZgENs0r4gYGk5mUhAl3SVrL/vU9rdCyP4ldTTzgMYZo+JW3C6fEPAJYZY1YYY0qBN4ER\nYe7/JGCaMWa7MWYHMA0YVjNRFSX+eZHLgtZlz/qaov2WjdmF1fNcXtGRVVc+VKNjuRV/jbauf1yT\nNiHWIsQN4Sj+1sBax/o6u8yXP4nIzyIyWUQqE52Fuy0iMlZE5onIvMLCwjDEUpT4431Geq1XuDy3\nWHbrwFmfwh0P9kVNPeHzLFcxYknNHrDJSDiKP9Bl5ftu+RHQwRhzKPAlMKka21qFxkwwxvQzxvRr\n1qxZGGIpSvwwcKB1We9zTM7y8ewku23gkMT7y2tmd65U/OrOGZqrnu+LdOkcazHihnAU/zrAmcW4\nDeAVHMQYs80YUzla9TzQN9xtFSUZGDTI6uPsJs9dNn68d5vsPt0Dbjt5z3AqNgdPYhKMyrzt2uMP\ng6GxSQATr4Sj+OcCXUWko4hkAKOAD50NRKSlY/V0YIm9PBUYKiIF9qDuULtMUZKKVq2sb2eP/4EH\nPPU/3jGZ1AfvD7jto9zCwANWVfuYOrhbDTR5kBchz4YxphwYh6WwlwBvG2MWich4EamMw/oXEVkk\nIguBvwAX29tuBx7AenjMBcbbZYqSVFx8sfV9Nc8GrD/s/pGQksL3MwJngJpPv2ofUwd3q4Eqfi/C\nytBsjPkU+NSn7B7H8u1AwESsxpiJwMRayKgocU/lsNR2Ag/gViZD7zsovVr7nf3VHvZ/NI3Bj4/w\ns+no4G41UMXvhZ4NRYkAlcq3ccYev7p2rHYvpzv0funU6SH3e/iQXI57ciR8+aVf3ZYt1vfeLcXV\nE7Y+ok9HL1TxK0qEaJS512sC12WnWBm+V730jbusUv+MZArpQ49zl1/MS1XvvKjIr+jaa63vZ7m6\nZgLXJ1TxexGWqUdRlNCkiMHl8iiYigpDW9YgHdp7tStcs48G6cd6b4ur6p0HCC62aqULSKEBuwB1\nga4SNfV4oYpfUSJEihhcxqH4yyGNcrd9v5KmbbPBJxmLBJje4gzgaXYW+Q3i/rrIUmapRC7gW9Ki\nit8LPRuKEiFSxFBhPLdUebkJqPid/LHImv5STpqXpt+9G1654Sf3evaYs2C7xyHO+VAI+bagqOL3\nQc+GokSI1BTDT65DEYG5w+9hzuJ8ltGlyoiQXXpYJpz/cbRXqOYrx1Zw8ZOHuddLyGL/kSe417/9\n1rMPVfzBSUup4GhmQk5OrEWJK1TxK0qESBHDD2YAABM+b8uKLfkYUrx66sFYQWdYtsy9vnSJvzJf\nvs2T6ss51nsNz9RC6uSmrCKVmeYYjcPvgyp+RYkQzpg5L3CFp8JVdY+8Q5NiDmCjV7t5Czx+n4ey\nEICDC6e7fTjzPJEhuKb0idqIrdRDVPErSoRYu6tR4IoQCb4P7ribTbTkl7veCli/h1z38vctvKN/\nfj30Ie/JAYoSBqr4FaWuad++yuqVWyzFPnLKGACmTPGuHz/SM8h7JN8DMPt76+0ga7vGPFSqjyp+\nRalDhvMpdOpUZRtj34Yr6Azr1zPREeBk18TJjH7vbO8N9u3j9jutbXbO+yOi8ir1A1X8ilKHtGNN\nyDYVDt9/Nm/m4489q/l9uiICZ4702P+bNvWMJRx/1UERkVOpX6jiV5Q6JI3ykG2Wrs0NXmm/LYy5\n2HOrbtvrcU3MGNC75sIp9RZV/IpSh4Sj+IPRkJ2Qnw/AiCBZriVVb2Gl+uhVoygRIiUlQNiFMKLl\nd+scOEb/R91vDX1Q9U9XaoDG6lGUCOEM0FZJOIq/cYFzA8Px3TdSsmQ5R38chuIPELxNUUKhPX5F\nqUPCUfyS4mlTtL2Cr5e0JIv9fv7/a1cHmAhWRRwgRQlGWIpfRIaJyFIRWSYitwWov1FEFovIzyLy\nlYi0d9RViMgC+/Oh77aKksy4wrjFGjT0LA++oisAXzHErzffpl0Ke3b5ROIsKam1jEr9I+RVKSKp\nwDPAcKAHMFpEevg0+wnoZ4w5FJgMPOKo22eM6W1/TkdR6hEr6RiyzcOPWkHcGlDEgtUOu0+AGb85\n+ansLnIo/549ay2jUv8Ip8c/AFhmjFlhjCkF3gS8fAyMMdONMXvt1dlAm8iKqSiJyWecHLLNob2E\na87aTDo+g7xB7Pe5DVJp3XA3R/E/aN06EmIq9YxwFH9rYK1jfZ1dFozLgM8c61kiMk9EZovIyGAb\nKUqis8ERPaFXj9JqbZuRKZTi46FThf1+3c48/meOdrt7Kkp1CEfxBxqd8vdbA0TkAqAf8KijuJ0x\nph9wHvCEiHQOsu1Y+wExr7CwMAyxFCW+aNnSszzmrH0ANGZbWNtu3plJMQ3qQixF8SMcxb8OaOtY\nbwP4RYYSkSHAncDpxhj3iJMxZoP9vQL4BjjMd1u7foIxpp8xpl+zZpo/VElsLjjH6vH/o+DREC0t\nXv+kodd6CzZFXCZFqSQcxT8X6CoiHUUkAxgFeHnniMhhwH+xlP4WR3mBiGTay02BI4HFkRJeUeKV\n5gekYAyM3f5QtbfNYQ9LvttRB1IpikVIxW+MKQfGAVOBJcDbxphFIjJeRCq9dB4F8oB3fNw2uwPz\nRGQhMB14yBijil9JfqqZ4/Xiizw++mfzDgVHdI+0RIriJqzZH8aYT4FPfcrucSwPCbLd98AhtRFQ\nURISCT1xy0nXAz0PiklczMsRFkdRnOjMXUWJML1YANnZ1domRHZGRYkoOt9bUSLI/r0uUsu7VTuG\njip+JZpoj19RIkhmdgpp+dXr7QMYh4P05TwfQYkUxR9V/IoSBzgV/9U8GztBlHqBKn5FiQPGjfMs\n53TRMAxK3aKKX1HigKZNPcs5zapIxagoEUAVv6LEGTk5odsoSm1Qxa8ocUb28l9jLYKS5KjiV5Q4\nI2egznlU6hb141eUOGH2zFIWPPARPK/unErdoopfUeKEgUdnMPCLP8VaDKUeoKYeRVGUeoYqfkVR\nlHqGKn5FUZR6hip+RVGUeoYqfkVRlHqGKn5FUZR6hip+RVGUeoYqfkVRlHqGGGcg8DhBRAqB1TXc\nvCmwNYLiJCt6nsJDz1No9ByFR12fp/bGmGbhNIxLxV8bRGSeMaZfrOWId/Q8hYeep9DoOQqPeDpP\naupRFEWpZ6jiVxRFqWcko+KfEGsBEgQ9T+Gh5yk0eo7CI27OU9LZ+BVFUZSqScYev6IoilIFSaP4\nRWSYiCwVkWUiclus5Yk2ItJWRKaLyBIRWSQi19nljUVkmoj8YX8X2OUiIk/Z5+tnEenj2NdFdvs/\nROSiWP2mukJEUkXkJxH52F7vKCJz7N/7lohk2OWZ9voyu76DYx+32+VLReSk2PySukVEGonIZBH5\nzb6uDtfryRsRucG+334VkTdEJCshridjTMJ/gFRgOdAJyAAWAj1iLVeUz0FLoI+9nA/8DvQAHgFu\ns8tvAx62l08GPgMEGATMscsbAyvs7wJ7uSDWvy/C5+pG4HXgY3v9bWCUvfwccJW9fDXwnL08CnjL\nXu5hX2OZQEf72kuN9e+qg/M0CbjcXs4AGun15HV+WgMrgWzHdXRxIlxPydLjHwAsM8asMMaUAm8C\nI2IsU1Qxxmw0xvxoLxcDS7AuzBFYNzD290h7eQTwirGYDTQSkZbAScA0Y8x2Y8wOYBowLIo/pU4R\nkTbAKcAL9roAxwOT7Sa+56jy3E0GTrDbjwDeNMaUGGNWAsuwrsGkQUQaAMcALwIYY0qNMTvR68mX\nNCBbRNKAHGAjCXA9JYvibw2sdayvs8vqJfYr5GHAHKCFMWYjWA8HoLndLNg5S/Zz+QRwC+Cy15sA\nO40x5fa68/e6z4VdX2S3T/ZzBNbbcyHwkm0We0FEctHryY0xZj3wT2ANlsIvAuaTANdTsih+CVBW\nL92VRCQPeBe43hizq6qmAcpMFeUJj4icCmwxxsx3FgdoakLUJe05cpAG9AH+Y4w5DNiDZdoJRr07\nV/b4xggs80wrIBcYHqBp3F1PyaL41wFtHettgA0xkiVmiEg6ltJ/zRjznl282X7lxv7eYpcHO2fJ\nfC6PBE4XkVVY5sDjsdo6dmIAAAGGSURBVN4AGtmv6uD9e93nwq5vCGwnuc9RJeuAdcaYOfb6ZKwH\ngV5PHoYAK40xhcaYMuA94AgS4HpKFsU/F+hqj6ZnYA2cfBhjmaKKbSt8EVhijPmXo+pDoNKT4iLg\nA0f5GNsbYxBQZL+6TwWGikiB3aMZapclPMaY240xbYwxHbCuka+NMecD04Gz7Ga+56jy3J1ltzd2\n+SjbS6Mj0BX4IUo/IyoYYzYBa0Wkm110ArAYvZ6crAEGiUiOff9VnqP4v55iPTIeqQ+WV8HvWCPi\nd8Zanhj8/qOwXg9/BhbYn5OxbIhfAX/Y343t9gI8Y5+vX4B+jn1dijXAtAy4JNa/rY7O12A8Xj2d\n7BttGfAOkGmXZ9nry+z6To7t77TP3VJgeKx/Tx2do97APPuaeh/LK0evJ+9zdD/wG/Ar8CqWZ07c\nX086c1dRFKWekSymHkVRFCVMVPEriqLUM1TxK4qi1DNU8SuKotQzVPEriqLUM1TxK4qi1DNU8SuK\notQzVPEriqLUM/4fmvy9YnI26vgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1267fcd30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Jut plot the test and trainng tests to see if the normalization has been properly implemented.\n",
    "plt.plot(X_test[:,0],'r',Y_test[:,0],'b')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# This reshaping is required for Tensorflow, \n",
    "# which needs the input to be (number of samples, length of sequence, number of features)\n",
    "ntrain = X_train.shape[0]\n",
    "ntest = X_test.shape[0]\n",
    "\n",
    "X_train_shaped = np.zeros((ntrain,window,1))\n",
    "X_test_shaped = np.zeros((ntest,window,1))\n",
    "X_train_shaped[:,:,0] = X_train[:,:]\n",
    "X_test_shaped[:,:,0] = X_test[:,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensorflow computational graph\n",
    "The next step is the implementation of LSTM using Tensorflow API. We will use the tf.nn.rnn_cell.LSTMCell function to create a single LSTM cell. I strongly advise you to check out other cells and read about them. Here we will only use a single layer of LSTM cell, followed by a linear transformation of the output. But ofcourse one can add multiple layers and this is another hyperparameter in the implementation of neural networks. \n",
    "\n",
    "The num_hidden layer required for the LSTMCell is a quite tricky parameter. It does not represent the number of hidden layers, but the size of the output of the LSTM cell. The key idea is to map an input to a higher dimensional cell output, and then look for relationships in this cell output space, followed by a transformation (i.e. weights*output + bias) to the actual output (referred as target). This is similar to the Kernel ideas, if you are familiar with it. \n",
    "\n",
    "Briefly speaking, tensorflow creates a computational graph, which is evaluated only when sess.run() is called. For recurrent neural nets, the network has to be unrolled to create the computational graph, which is achieved by using tf.nn.dynamic_rnn(). The output of the unrolled network is a sequence of outputs, with the length of sequence equal to the length of the input, and the size of each element of the sequence equal to the num_hidden parameter.\n",
    "\n",
    "The last steps are creating a loss function, and mean square error seems to be a good indicator of the loss. The optimization can be solved using different algorithms, and here I have used AdamOptimizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:96: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.\n",
      "  \"Converting sparse IndexedSlices to a dense Tensor of unknown shape. \"\n"
     ]
    }
   ],
   "source": [
    "# Placeholders for input (data) and output(target)\n",
    "data = tf.placeholder(tf.float32, [None, window,1])\n",
    "target = tf.placeholder(tf.float32, [None, 1])\n",
    "\n",
    "# Specify the number of hidden cells and create one LSTM cell\n",
    "num_hidden = 24\n",
    "cell = tf.nn.rnn_cell.LSTMCell(num_hidden)\n",
    "\n",
    "# Unroll the cell using dynamic_rnn\n",
    "val, state = tf.nn.dynamic_rnn(cell, data, dtype=tf.float32)\n",
    "\n",
    "val = tf.transpose(val, [1, 0, 2])\n",
    "last = tf.gather(val, int(val.get_shape()[0]) - 1)\n",
    "\n",
    "weight = tf.Variable(tf.truncated_normal([num_hidden, int(target.get_shape()[1])]))\n",
    "bias = tf.Variable(tf.constant(0.1, shape=[target.get_shape()[1]]))\n",
    "\n",
    "prediction = tf.matmul(last, weight) + bias\n",
    "cross_entropy = tf.losses.mean_squared_error(prediction,target)\n",
    "\n",
    "optimizer = tf.train.AdamOptimizer()\n",
    "minimize = optimizer.minimize(cross_entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After creating the computational graph, the input and output is fed as a dictionary using feed_dict parameter in a batch fashion. I have been using this template for implementing neural networks in a rather quick manner, and also in a more clean and intuitive ways (rather than use lot of unnecassary functions and classes)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch -  0  loss -  0.536583\n",
      "Epoch -  50  loss -  0.000792924\n",
      "Test loss - 0.000609993\n"
     ]
    }
   ],
   "source": [
    "init_op = tf.global_variables_initializer()\n",
    "sess = tf.Session()\n",
    "sess.run(init_op)\n",
    "\n",
    "batch_size = 1000\n",
    "no_of_batches = int(ntrain/batch_size)\n",
    "epoch = 100\n",
    "for i in range(epoch):\n",
    "    ptr = 0\n",
    "    for j in range(no_of_batches):\n",
    "        inp, out = X_train_shaped[ptr:ptr+batch_size,:,:], Y_train[ptr:ptr+batch_size,:]\n",
    "        ptr+=batch_size\n",
    "        loss,_ = sess.run([cross_entropy,minimize],feed_dict={data: inp, target: out})\n",
    "    if(i%50 == 0):\n",
    "        print(\"Epoch - \",str(i),\" loss - \",loss)\n",
    "test_loss,Y_pred = sess.run([cross_entropy,prediction],feed_dict={data:X_test_shaped, target: Y_test})\n",
    "print('Test loss -',test_loss)\n",
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Voila, the loss for the test set is very low which means that LSTM is very efficient in predicting the next minute of stock data. Note that the variables you want to print on screen or use later must be included in the sess.run() input as a list. Otherwise they will be computed but not accesible!\n",
    "\n",
    "Don't be very happy, as this is not useful in practical scenaries, where one might be interesting in predicting prices in a bit larger timestep and modelling trends over larger time periods, rather than just a minute ahead. But this is intended as toy problem to learn implmentation of LSTM in tensorflow.\n",
    "\n",
    "Let us go ahead and compare the predictions with the actual test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnWd4VNXWgN89M+mFJCR0pAuCIkjo\nICiiiApYroIKXBVQwYbXghVFrr19CKioqFhQr4ogotgFpQYEpAjSCS0hIT2TzGT29+NMTSaZSTLJ\nlOz3eebJObucvZLMWWeftddeS0gpUSgUCkXDQedvARQKhUJRvyjFr1AoFA0MpfgVCoWigaEUv0Kh\nUDQwlOJXKBSKBoZS/AqFQtHAUIpfoVAoGhhK8SsUCkUDQyl+hUKhaGAY/C2AO5KTk2Xbtm39LYZC\noVAEDZs2bTolpUzxpm1AKv62bduSlpbmbzEUCoUiaBBCHPK2rTL1KBQKRQNDKX6FQqFoYCjFr1Ao\nFA0Mj4pfCNFaCPGLEGKXEGKHEOJuN22EEGKOEGKvEGKbEOI8p7qJQoh/rJ+Jvv4FFAqFQlE9vFnc\nNQP/kVJuFkLEAZuEED9IKXc6tbkU6GT99AVeB/oKIZKAmUAqIK19l0kpT/v0t1AoFAqF13ic8Usp\nj0spN1uP84FdQMtyzUYDi6TGOiBBCNEcuAT4QUqZbVX2PwAjfPobKBQKhaJaVMvGL4RoC/QE1per\nagkccTpPt5ZVVq5QKBQKP+G14hdCxAJfAPdIKfPKV7vpIqsod3f9KUKINCFEWmZmprdihSbp6bBs\nmb+lUCgUIYpXil8IEYam9D+SUn7ppkk60NrpvBVwrIryCkgpF0gpU6WUqSkpXm0+C106d4bRo6Gs\nzN+SKBSKEMQbrx4BvAPsklK+XEmzZcAEq3dPPyBXSnkcWAlcLIRIFEIkAhdbyxRVUVSk/TxwwL9y\nKBSKkMQbr56BwHjgLyHEFmvZw8AZAFLKN4AVwEhgL1AE3GStyxZCPAVstPabJaXM9p34Ic7u3dCx\no7+lUCgUIYZHxS+l/B33tnrnNhKYVkndQmBhjaRr6ISH+1sChUIRgqidu4FGSQkS2M2Z8Pbb/pZG\noVCEIErxBxp5eaxgJF3YzbeflXeeUigUitqjFH+gceoUH3M9AIdo42dhFApFKKIUf6Bx+DDFRAGQ\nRzyYzX4WSKFQhBpK8QcgOSQAcJSWkJvrZ2kUCkWooRR/oJGXRzqtAJjD3cjFn/hZIIVCEWooxR9g\nyGuvtSt+gKxYZedXKBS+RSn+ACObJIqJZnDPfACO7Cn2s0QKhSLUUIo/kLBY7LP9fr1MAKQ/s8if\nEikUihBEKf5AQq9nM1rysr4XxACQnnSuPyVSKBQhiFL8gcKOHQAsYxRNo3K57KoI9JhJz45WIZoV\nCoVPUYo/UCjWbPlr6U+fFkeJjIQWHCOdllqIZoVCofAR3kTnVNQHa9fyJlM4STOOxWv5CFqR7uLh\no1AoFL5AzfgDhS5deJl7AZg0RQ9AS47yM8OYxlx/SqZQKEIMpfgDBZOJLBpz65Unue02rShJ5AAw\n333Ea4VCoagRSvEHCNJkJpdGJCU4UhIfSz7H0WDSJD9IpVAoQhGl+AOEogILZsJIaORQ/A8v7m4/\nLnvnXXjySZBuc9UrFAqF1yjFHyDYYrE1auQo6z8smhc6a8lY0kiFJ56AP/6of+EUCkVI4U2y9YVC\niAwhxPZK6u8XQmyxfrYLIcqEEEnWuoNCiL+sdWm+Fj6UyNlyEICERNcsl7EGIwD9WM93XAITJtS3\naAqFIsTwZsb/HjCiskop5QtSyh5Syh7AQ8Bv5RKqX2CtT62dqKFN7lufAtCoWaRLeVzhCfvxpXzH\n10kT61UuhUIRenhU/FLKVUC2p3ZWxgGLayVRA8UWgz+hZaxLedTBnS7nyzc1qzeZFApFaOIzG78Q\nIhrtzeALp2IJfC+E2CSEmOKh/xQhRJoQIi0zM9NXYgUNhyI6A9DiDNc9dYXEuJzv4qx6k0mhUIQm\nvlzcvQL4o5yZZ6CU8jzgUmCaEOL8yjpLKRdIKVOllKkpKSk+FCtAefppEAK6dAHgcEkTDJhoVW6j\n7sWfTaZPwm7SNkpu5AMO0A5KSvwgsEKhCBV8qfjHUs7MI6U8Zv2ZASwB+vhwvODmkUewIDi+OxeK\nizlOc5qGZaMr9x9p+q/zWX+6M71SBR1iMzhKS4rTs/wjs0KhCAl8oviFEI2AIcBSp7IYIUSc7Ri4\nGHDrGdQg+PZbLRBbYSFceikAdzGHFhwn78IxHKc5zSNPV3mJsyf1Q6Lj7w159SGxQqEIUTwGaRNC\nLAaGAslCiHRgJhAGIKV8w9rsSuB7KWWhU9emwBIhhG2cj6WU3/lO9CAhLw8aNUICQqcDi4XTJPAX\ng5nHHVqTdTtII5UB+WuALpVeqnWClpXr6PX30XPc8noQXqFQhCIeFb+UcpwXbd5Dc/t0LtsPBFcW\nEWH1offV7tj8fBg6lDR60Zs0nrQ8zjB+YhCum7B+YwhZJLPRMKDKy7VMbQ7Aftr7Rj6FQtEgUTt3\nncgnlv20883FTpyA+Hh2/1lIb7S9azOZxZ28VqFpGtoWh+GdDlR5yaZna4vedzMHzGbfyKlQKBoc\nSvE7cRtv0IH9lJXVoPPevfDuu47z5s15ndvowm4AEmK1HLp/ch5vvGpkzqzT/G/C1wAcog0A89f0\nrHKIsCaJ3MgHAOSeUEnYFQpFzVCK34mPuQGA4/uL4aOPvDf5SElpp66k3TwPMjMhP5+VXMxUXgfg\nnpvzOL5LC8ZzVnIGt94dyZ2PJZIUo7llLuEq4sgjNsGD5S0qikuGlgKQcUzN+BUKRc1QGbhs5OYC\nWoS0/LP6QNl2aNcOBlRtdwdg6VJe4j88zDP80WQAAy5vzDM8pF0rH2Jj4wHY8ftpzjgn2d4tKtzx\nanEmewDPUS1iOreEX6EwryavJQqFQqFm/A4eesh+mF8WxY8Mo4JTfWXMns3DPAPAu9xE8fIf2UAf\n7h6+k1inCAxdByYSG++4ZqSu1H789P/O9Gqo2GjtLUQpfoVCUVNCT/F36ABDhlS/3wHHwmpfNjCc\nH1n7dyK8+SYcO1ZlV9Omrfbjt5lMNMUUE02PLsYq+0V9sMB+POSKeK/EjLFGcCjIVYpfoVDUjNBS\n/FLy4/52rFwV6blteUpLKxQV33Q7R26bTXHLDlV2fZxZAAw3/OxSPurxHlX2iz+1z34cEeGdmNHR\nVtkKLd51UCgUinKEluJfu5bh/MgIVla7a1njJhXK3uEWzuAIN/ARbLduOhaCXNEIDh7Uzo1GnrXa\n8+fOdcTSP7A+g6Tkqv+8TTlZbTkjY7RE7MYipfgVCkXNCC3FXwtMV4+tUGbz8lnCVRw+ZyTceivf\ncQkJ5PL+f7ZpjQoLOZPdtOUAZw5pTgYpFHz+HW37VHyQlEf/2y/cxussYrzXckZGaQ+XkmKl+BUK\nRc0IKa8eU25RjfuWmqt+BrbhMHsWdGIymjnn571nMNFiAaORPOK5XKyAzjeTIqsRUvr883n9z3jo\neovXXSKi1YxfoVDUjtCZ8RcWcnzqU45zi0Xzw//qK+3YA6Yy93+KKbxpPz6Tf0inNQD7t+WzWj+E\nN1vNIpMUmo7u5wj5UB169IDwcK+bR2YeAcC4bkv1x1IoFApCSfFHR3PXwen2U9m2Heh0/HXlYxRe\ne5PH7qWmikr7bl7lzeKJLsof4EGe5XcGcz6ruY03KcNAyxb1MwOP3K9l5DKuWg+//VYvY4YUUsK2\nbf6WQqFw5fhxiI2tt+9myCh+Y4lgKWPs50uP9OQ4zejOX9z1RaX5XzT276d03xGXorX041WmQ2Qk\nD7f+EIBwStj5yTbG3Ny4wiU6NjpV+1/CCyJiNOuckUi48cZ6GTOUkLdMYt25UzD/pB6aisAh51+T\niS7MYOVd39TLeCGj+CMiYD6328//w0u0Q/PNXxk1prJuAPzS4RYefsY1123izLvh6FEA2uxYwVym\nsXbuZs66rjvdTA4ziw7Nn75j80LqA124pvhf5R5kdtXx+93y6quwsvpeT6HC8ncz6M86Zl/0i79F\nUSjs/BnWh2KimX3ghnoZL2QUvxBw+1MtOUBbAPbTgRI0f/7ESPcbqTJGTWK+mMpYPrF78NhImjYO\nWrTQTuLimCbncd60/trp1RfzLSP45z9vsIKRTOQ92tx2ad38YuVprL1t5JDIXaYXq99/+nQYMcLH\nQgUJublcxZcAPMkTPou+rVDUlvwILZRLVHQ9DSilDLhPr169ZI353/9kN/6SmjFX+ySHndbqTp2S\nsqxMSrNZ5hErO/CPSzvnT2mph3EOH9Z+FhRImZdXc3mrS3a2i5zV5T6el2dw0PdyBTAlj86S7zFB\nmpZ+4/K3y8jwt2QKhcb8vu9JkPKKrntrfA0gTXqpY0Nmxm/nmmtohBYJM5pCZvMIp0wJFK3bxkfJ\nd/Gb/gIoLGQE37GPjpVeJizMwzitNe8eYmIgLs5HwnuBpy2+e/bAqlWVVr/I/Ry2hoFuKMyZncu/\neZ9nrlwPwMA4LcTG8cMmf4qlUNg5ul5bY9Tn1E8+bY+KXwixUAiRIYRwmy9XCDFUCJErhNhi/Tzu\nVDdCCLFbCLFXCDHDl4JXRSwFAPRgC03IAODl/p9xIx8xlN+wfPgxaxjotu/z3M/PXFBfolYfg2Pr\nxW1hb7vWFRRg7NydXUNudd/3QNWJXkISKbkfzST2X4u2w/ryltoaTcYhldNAEQBs3sw2ugOQc6ye\n1gq9aPMe4MkovFpK2cP6mQUghNAD84BLga7AOCFE19oI6y3fcwkAYZgYOlZLV/gYs+31EdNuIZwS\n7um4nOa4BmDrzUYu4Nf6ELNmhIUxjo+BitsGTsW1JQojXdnF8WNuDNhHjlQsC3WKHJv6bGs+YyZo\nAfFOHqo6iJ5CUedIyQe9XuFrRgGQRUWPwbrAo+KXUq4Csmtw7T7AXinlfillKfAJMLoG16kxvzGU\ndlf1JBLXmZ2ZMEqJIKltvD1uvo3waAPce299ilk9hOBjbqA5xzB1Psel6kuush9vfuGnCl1lcorj\nuIEsbOZf47qH41/n7KJpa23D3Nv/PeEPkRQKB4WFTLBm1QM4RXIVjX2Hr2z8/YUQW4UQ3wohulnL\nWgLOU8x0a1mdU0I4w/iRoyk9MEQaKMaxVL4ZR3rDy66NYeK34/ij40R7mW7R+/DSS/UhZq0wYMas\nc93xa4lLsB//+eqvFfoU/uN4u6lReslgY8sWZnznGqK7c2ImCS2078OvWd39IZVCYUcu+9p+PIjV\nvMD99TKuLxT/ZqCNlPJc4DXgK2u5u/gFlc4zhRBThBBpQoi0zMxqxLtxQzgmfmQ4LXo111wX77mH\n95nAdF6m51HHBonzJveCESOIvGqkvezMTsExFQ7DhMni+u8ratTcfvwYs9l7xXTIyLCX5a91LNOE\nvOJftYqSnn2ZzzQAerMBgDN7xriayLKyYOtWNxdQKOqeE/kx9uNJvM0NVjNuXVNrxS+lzJNSFliP\nVwBhQohktBl+a6emrYBKM5pIKRdIKVOllKkpKSmVNfOOvXuhTx8tTk9YGLzyChMWDeflX3tBYiK7\nOdNl5h8V4Qi3kNStubsrBhxhmDCX6eCWWzRjv5TkxzQFoBE5AAxcPgOaNrX3yTvjbPtxyCv+IUN4\nkfscp2g7dVulNoOYGKYyD4DS3gO1eEkKhR/YeJvmoPE89zOBRfU2bq0VvxCimRDaHEoI0cd6zSxg\nI9BJCNFOCBEOjAWW1XY8r+jQAdavd3V9HD9ey8wVFcWZ385xmflHrvsV0EIyoNfXi4i1xYBZm/Ev\nXIgZPSX9h5JnjCBaV8xjaMHqMmhKKhvJztLeYvJyHA+4UFf8EvgLbQ3kmpZreJxZvMdEhlwcAb17\n04l/ADh1II93uBlpCY43PUWQk5MDhQ7PndUMJgIjd47PdWsiqSu8cedcDKwFOgsh0oUQtwghbhNC\n3GZtcg2wXQixFZgDjLXuJzADdwArgV3AZ1LKHXXza1STESMcu3KBZj8sIoks3mKyH4WqBmvWaKae\nMh07OYswzESu/42C0nBi9cX0i99lb7qJVNK+0RK+5D3ynL3cbK53qeuVZ5nBp2g5Ft67ejlxW/9g\n4hPt0TVJBiGIHdYPgJe5l0m8w4qXdlV1OYWiemzeTJZojPF71z019yW+zfRYR8rVg7SlLQeJXLSg\n/BXqFI/x+KWU4zzUzwXmVlK3AlhRM9HqjyiMZJHsYhYJaPr3x8BGzBYdn3ONvbggt4xofQkD7+vP\nR49fb7cXHvlmG9yQwjSnf1Ooz/jfwZHjIOaJ+yExEbo7FnMjtqwDxvI+2sJ+etoJNK9jhaL2rO51\nN+eTRb9L1rLW+jK5SfTiJTYBMPnW/6Prm3dzguY0b2p9E//mG9i9u17kC72duzWhTx/t55w5/pWj\nGoRhxrT/CAYcU/e0orOI0ptgxgyuZzEZaGslf322E7ZsYTdd7G1DXfEPte7FeJMpmtIvR1mWtg5y\nyvo3Ko2t2EahqCkb0HTKOvrby2bzqP2424K7eUfcwu8Mollj6z08cqQWS6seUIofYNEiGD0aRo3y\ntyReE4YJs9STg8OFcw+d2VV4hragffQoKRf1oC/r2M7ZFUI9hLTil5KWaJFVJ/OW2yblF9KKi+vT\nwqoIdcxWY4ptg6hMP8pXXOnSZhLvAJCfU/92V6X4ATp31jyAIiP9LYnXGISZP+lJGqnuG7RoAYsX\n05aDHNK1w1JQhI4y+xcxKGz8u3drHkvPPFO9fm+9xZ/0JAIj4qT7hPY6JBE4du6+/1kkA8QanhEP\nYSkuqY3UCgU5HbT70mTQ9oyc3qO5qF/JlxVCwojKvdzrDKX4g5QwWUouCfzChS7l/8ddjpPkZFom\nl3JMtGT3fz/Hgp5mkVoAO7/N+Ldtg+++867tyy9rPx9+uFpD/PxJBl8zSgvR0KTypPe2EA4AO8u6\nsJYBPMwzvBd9O+TnV2tMhcKZHKP23cori+bZmKdYOFxbb7vh5kgu4Fce4Dk+5VoWchOfXPxuvcsX\nUsnWGxJhuI8sObj1IZfzFqe2UsQEui5/HoDmkaf50+hHxX/uudpPb2JGGI0IJFOZxzwpvcppXPDV\njwz75VGP7ViyhAuu/LnCgxNgJk9y886d0Lev5+soFG7IKdEUf6kM56Gix+zlqde0hf/L5znniL5X\nLq1n6dSMP2gJp9R+fDvz7cfdt7vu/EvENUtXXKTWz1+Kfymj6MlmSoySj/u8wpePbKq0bdkiLeXl\nfKbB8897df3vr5zvuRHAyJH8yEX8xdkYyj1E02lNqUXNiRQ15OhRck65t6W2ubizllu3sFC7CfPz\n/bK2qBR/kBLpZJ/uy3rOYxPnsgV9fIxLO1vGKRuXtdKSOfvLxv8YT7GFnuyY8Bw3bJzO1U/3Qpaa\n+Hujk2nFbIZ161jqFNPv4XnehXnaw5neCRIeji46irPZQVsOVqj+YUMj766jUJTjp1YT+I6KGfnO\nY5Njg2h0NOh02kPADyjFH6Q4K/6kByaxiVS2OIWhsJGw4QfGWz1YruILoqM0E4u/ZvxlaF/8Xv9z\npGcYGLGRs/rE8eX9a/j+7Hu5IexTXu//Plc7PbSeOeJFYnmjkZ10JY4874QpLIT8fCzW2+Aa/se1\nfApA85RgWP1WBCIv8R+35Qnt6yfksjcoxR8CNGlTRaLO3r3tLo2P8RT62CjAf4q/hZtwTWsZAMDV\nLw7gkh0v8zE3MJXX3fbP+XQlDzZ/nzKzZP9vR9i+1vqmsGMHz0XN5AMmcB6bvRfIYGA/HQAYzGrG\nW0PkWixVdVIoKicO944BCcmBEw5GKf4gxTZzvprP6Xum1Y5/3XVu2w7mdySCHo9egT5M+5f7e8Zv\nwxZQzltuH5vN8ycmMr3Fp3QY2ppzBmiLZGXr05iBFpKiI3u9v6BTjs2Bd6US1rEtAKVhMZV0UCiq\nJsH6nX6Z6UgE87kdcLuP0G8oxR+k2BToZXwDUVFapqmPPnLf+LffNJvilCkYhKbx/WXjPy0cr7tr\n6E+udQPaOG/C0S5ezCdoEUReyxxrLy4thT05joiuPS5vXaFrpTgF5Wv+4ATCrrwcAJNUi7uKmqGn\njBQymM6rAFwTtYL+rOGRW9zvKfEHSvEHKTbFH4YJBgzQlH9lkUXPP1+zZ7dujV7nRxu/2cwpmcRZ\n7GQE39Jj00L7hrI3LnUEbrXZ6JPI4hYceYXF9e7DRmVnw3//oyWJe5qHmNJrE53Yw5BqptBs0gTC\nDZqNp9SkdvIqaoYZg4u7dUq7WNYwkHZnhlXRq35Rij9IsSl+PWVe+bfb8KfiL8vI4jjNuYov+ZaR\nRPXswg8MZzFjiX/5CbJIwkgEA/kDgC78zVzuoI0brxuA9uwDID9P8hHa4u+MCzcS/uB09tCZX8vt\nkKyMJYzhVt7AYIAwvab4TX9u99BLoXCPuVEyBp0Ek0nbr1Ji3QkeXcVaXD2jFH+QYvNE0c+tXmA5\nw3EtG6bZVP/bxE8eLKYMA60mDNOePELQ7fFrGHtjGHTpQtLXi4jYs50YtATpbTlIpChljD2pmysD\nw9MA+OjKz4kjjzuZg/jpR+3tpxqMYSlvWO2w4WHa3+WyVy6itLSqXgqFe0wWPYaoMDBYzYXF1pzf\nARQSRin+IKUxWQBEJlTvyxS5RkvCbjxU//bG9KlPA9AyZ4fmwwzw5JPwgTXZ9OWXQ6dOrORiAJLb\nxsHKlURbHwQAeTh2PA7T/aJdYue/yCfe1R//77/h1Klqy2ib8QMcWJ9RRUuFwj1mqcOgc3ILW7QI\nBg+G5oGT3U8p/iBlFo/zGLO4dGT1bNE2/39jXj1PZ48f58Gtmo2+Tbuqv3b9WAdY02AOH06+k7KP\nowCBdlN1G3+eS7+OfZMdJ507Q2Mv/abz8+0PiTCD402o6EDgLMYpggezRY9B52RLHTYMVq1yvAEE\nAErxBylNyGQWMwmLrt6CkV3x/7i6LsSqlN0thtpt7t0eu6rKtiu5hA+5gfvGpgNQhJNt1GJhJ11Z\nxWCiSxzhKP7FZ4yaWXEDm1fExtofErbFXYDi6MDZcKMIHsxS7zrjD0CU4g92wmqo+OPrN9vY91bz\nDYC+UdXb1HVIbuBjYvppOXNfYTod2MtDPA1C0GXqMAbf24cmsZoJaA538hnXQb9+tZbTecZvNFbR\nUKGoBLPUExbgit/ju4cQYiFwOZAhpTzbTf0NwIPW0wLgdinlVmvdQSAfKAPMUspKgscraoyues9u\nu+JPaFYX0lSK0RoCOYYCMHiIT7J2reaa2rEjAPHks5dO1sqHYd48AJLvvZdSwgjDDAsW+GSHTHTu\ncftxcZFKwK6oPiZpcDX1BCDeaI33gBFV1B8AhkgpuwNPAeWzBl8gpeyhlH5gEIXmYWCs51wjJ9He\nMP5J8mJW3q8f9O7tOJ8wwX27qChN6b/5Jkye7AMpIeH35fZjmzOGQlEdQsLUI6VcBWRXUb9GSmkz\ntq4DWvlINkVVPPec94uXTthn/MZ63KBUVMReOnIGh2h+aw1C0C5cCKmpmneEMw8/rHkF3XSTb+QE\ndD//yPtoDxql+BU1ISQUfzW5BfjW6VwC3wshNgkhplTVUQgxRQiRJoRIy8zM9LFYIcgDD9TMXfHN\neQgsGEvqUfGnp7OUMRymDcyeXf3+ej1s3Ajjx7uWx8TA449Xe52jStq0YRiay6tS/IqaYEavbeAK\nYHym+IUQF6Ap/gedigdKKc8DLgWmCSHOr6y/lHKBlDJVSpmakpJSWTNFLRGDBhKJEeMflSdA8TWF\n/2hhGTrzd7XXJOqd1FSHOUwt7ipqQIOZ8QshugNvA6OllFm2cinlMevPDGAJ0McX4ylqgV6PQHL8\nRP0NefIhLVjVDJ6tv0FritlsN4cVK8WvqAHa4m6IK34hxBnAl8B4KeUep/IYIUSc7Ri4GFABUPyN\nXk8RMXzIeM9tfUTGHi1MbROCYCdsWZlD8RerQG0K79i3ZBtHt2lzXrM0uOwAD0S8cedcDAwFkoUQ\n6cBMIAxASvkG8DjQGJgvtGBhNrfNpsASa5kB+FhK+V0d/A6K6pCZCWhukiaTb83jlZFVosW2bzx3\nVt0PVlsmT0a3dCkRGCn+Jx1o4W+JFAHM3ic+5Odvjdy6YRJ6zJil1cavD+wMbh4Vv5TSfSxcR/0k\nYJKb8v3AuTUXTVEnOIXlzM+HpKS6H9Js/ZqFNQuCnbCXXQYLFhA1pRjjr2tR1klFVVz1ZHf+ojsA\nZRjYtqYAszRg0AW2nTDAV9oUPqdzZ/thvvsMcT5HDtFCNeg6tKufAWvLhg3EkU8uKuG6ogqystjD\nmS5FU4butnr1BLapRyn+hkZSEh9YY9fXl7uiJTYeCHyHHmeSOUUWjWHPHtiyxd/iKAIQS3IKJbhG\nx11v6sV+2R5DgNv4g+hWVPgEnc4e5ri+4s3bEpcHjeJv3ZpIjJQQwUednyS1p9lvOYoVgYtz/Ckd\nrl+QBuPHrwgShCACLV5DibF+vpwWqXnHBI3iT0khDBOlhDODZ9lEKscO1HOMC0XAs5/2AOzmTIqI\n5l98Zq8LdK+eYLkVFT4kYkh/AP65fiZpU8qHVvI9Qaf4zz2XcEopJRyJJvuuD+tvw5siOEinFQZM\ndPjqZSIo5UY+tNcZ9GrGrwgwwrtoM5Ub9s2i91tVRtLwCUFn6hkwQFP8jVvQHC1a58E9Kg+jwpUj\nST1oEZmNfvTlUFjIqMslQ/gVQNn4FYFHRKTrxiRZx5OToJvxA+FRBkrzjISjKfzsxWoLisKJU6dY\nl92Jc4wbtfPoaHj/fXu4j8hNa/wonGeC6FZU+IqIKNd/e25u3Y4XlIqfUkpNjj0I2dTDhgdF0FC6\n+wB76URvNjoKk5LIQ/Nga2k+6B/BvCRwkkAq6o3wCNcZf2aGJCGh7sITBKXiFyZ208V+nh1Wv4lr\nFIHNsUxty3uraWNcym0ThFY9AjvQZBDdigpfUX7Gn7lgSZ2OF3Q2fjTF70x2mxrm81WEJPsWbwCg\nTbxrqpIcEgBo9dqDFfoEEkF0Kyp8RXnF/8tLdeuxEqwzfmeyiyMraaloiBz8bD0AHUa5ZqM9jZb+\ns1XHwP6+BNGtqPAVEdF6l/PLa2j3AAAgAElEQVRH+W+djhcSM/7iKD9JoghEjiR0R2ChZc8mLuUL\nuZkzOERKk8CO7BpEt6LCV4RH1u+/3UIIzPgLI/wkSR2xZAkUFPhbiqDln5xkmnGiwnrZ9SzmEG0D\n/rse4OIp6oKIjCMVyo6NmVqxYU4OfPBBrcezWIJP8UcKx07dSIrJtoaWDgl27YKrroJJFYLqKrwh\nO5ufuZDzWeVvSWpMEN2KCl8RfobDQ+UF7gOg5dL5FRtOngwTJsDWrbUaLxht/AYcM/4zOEwx0aGT\ng7e0lIv4gfmrz/G3JEFJ0ZEsTtCcc9q6CW87YQIMHlz/QlWTILoVFb5Cl+1I0h5PXuUNT54kl3ht\n5l8LgtHG77y5oTXaG9LpA7X7OwQKpTKMn7iIacceqf/BCwrgSMU3zmDi4Bbte9B+XL+Kle+/D6sC\n/00gmG5Fha+Ii3McisJKm6WVdieBXFaujq7VcMFo43cmFs0WvqSbHxRlHZD1vR/jDg0dCmec4b/x\nfcD2zZoZsHPP2t0X/sSrW1EIsVAIkSGEcJszV2jMEULsFUJsE0Kc51Q3UQjxj/Uz0VeCK2qBTkcZ\nOiwIwnqeXWmzn3N7AfDj5sRaDReMNv4yHJ5Pg/gdgDuY5y9xfEqWSPbf4JusD51ffvGfDDVlzRpO\nXz6e6+YMAuDsIUGQUa4SvL0V3wNGVFF/KdDJ+pkCvA4ghEhCy9HbFy2H3UwhRO20iKL2CIHOGnfS\nEubwVrGUiytl+Xu31ryWwXyC0cZv88e+h1cYxk/28qIif0nkG8x7D9L9gUv8KsN7TOSLC+f6VYaa\nsHHg3SR943B2CG+S4EdpaodXt6KUchWQXUWT0cAiqbEOSBBCNAcuAX6QUmZLKU8DP1D1A0RRHwir\nC9qYMaQ2OWwvLu/dp0N7EsivvqrVcBbpOmwwcB8vAnAvL5PYybH9Ppg9IDf3n8Y9nZYj/WjhXcVg\nbuI9ruELv8lQE4r+3M29vGw/38R5VbQOfHz1DWgJOK/YpFvLKitX+BPb1N5goH2jLOZzOwB55dZ5\nP+Z6APbRoZbDBd+Mv01SARJBa9JpMcaRcL24KLDjrFdFn3X/xzzusJ8ncLreZXCOWV/nYWF9xZNP\nEnNeZ35H89Zpw0HOW+vGCy6I8NWt6G4uJ6sor3gBIaYIIdKEEGmZmZk+Ekvhlm7dtJ9XXAGlpTQm\nC4C802WwbBm0bw8zZrCVHgAs4apaDReUi7sGR/zC8A6t+ZhxABQXBG8OxjKnmIw3sRCLH2b+41hs\nP87JCI4cB9uecH07+cVwMfRz49ETRPjqP58OtHY6bwUcq6K8AlLKBVLKVCllakpKYEe2C3rOPltz\n0ZwwAaS0u3Qu7P4KcvRo5h4YSbfnxtOPtfYuOb/8WePhZBDa+HnQGmSrpASEsMdZLy4M7AQb3pLY\nMZkyUf/BeUtwrCl9O215vY9fE2zrPQA92Uy7tx72ozS+wVe34jJggtW7px+QK6U8DqwELhZCJFoX\ndS+2lin8TaNG2s/YWBKtr/wvcR/vchN3MpeddCMDRxySY3tqbty22fiDSvHfe69mEgsPh4EDg1/x\nS0mc9QG/mkHoIwwunkv1RREOF8hNu4LDHTJv0EgAOrGHtzs+B//+t38F8gHeunMuBtYCnYUQ6UKI\nW4QQtwkhbrM2WQHsB/YCbwFTAaSU2cBTwEbrZ5a1TBEovPwyZ7HLfnoLC+3HB2hHPNpGpuOnwmo8\nRDDa+AHHanS3bkTdfhMQvDZ+05/bySeeJ5jJoG8eRi8slMka/kPmzoW//qpR1yKi6cBeIilGJ70w\nm+Xn+30tIC+yKQDfcBnnvTDOr7L4Cq/e9aSUVf62UkoJTKukbiE4aRNFYJGQQDxutp4DEh1tOcg2\nziXrVM1vPpuNP5i8esoTpdfs0cE64z911RRgLU3IgJEj0c9YVnMb/513aj9roJCLopKJLi4iimKK\n4ppU3TgzE5o0gaeegkcfrYGgPqCggLwf1wP/Jn7Fp3BpcHvz2Ai2OZiiLrjmGsIpcVvVkqMA5OTX\n/KtisQh0BO+iKEDUojcBKF7+k4eWgcmpKZpdOvnhWwHQ66TLYq/X1HL2XWSJIDoxkmiKKNbFVt34\nxAnt5yef1GrM2nA8rhNTtW1JxA8NDaUPSvErAN55hz2c6baqldX59vTHNU82bpHCvicgWInK05TQ\n/g//8LMk3rHi4d9Z+/YODoszmCvu4IM3tNAcyec0B0AnNAVeftOeR6rdwZWiskiiI8xEUUyx0cMr\noBCY0SMt/jP1bMDhyhsZ2LlVqoVS/AqIj6fR4HPtpx3Yaz9uElNIGKWcLq55PHpN8QenbdxGI+ta\nxwyegx07/CyNB0wmLntmEAMmd6MNh7mTubxwZCwAiU3DAdALTYGXOb+I7d/PiRbnseHdHZVP7E0m\n3mIS3zO8RqIVyUiiI8o0U4/Rs/oJw8zwXf8Hzz4LJpPH9r6geON2jn6/A06eZDed7eXBbKosj1L8\nCgDiHrjdftyJf+zHsc1iaEQuudR8e7pFgk4E94y/8ceOEAPL3ztVRUv/k//HtkrrEptr01Z3in9l\nh9tpfnwzfW/uxm9fVPI7/vADU3iLS/i++oLl5FBYFkl0hIVYCigsqdyr6MdRczjbGjX6Jy7C9NBj\n8Pzz1R+zBkzus4VWl3TjyKhpPEj9jFnfKMWvAEB/9DAx1iiUNrs+QPRZbYjESMk5vWp8bYvUBb2p\nRxQVMtUapG3t61v8LE3VHL73VZfzMBwbpRKaam9uep32/3C23Czncvvx0bTjFS9cVMSqUS/UXLBp\n0ygimuisI8SRT36Rq+IvfXU+y8XlWFZ8x/Cv72IHjgCCf3EOZY8+XvOxq8FH3AjA3Ruut5etZlC9\njF1fKMWv0EhIIBotApmz4s81RROJEaPFydQzdar23rut8pmlM6Fg42fiRF5hOgCRhYE940//M8Pl\nfB2OXabxCdotr7fa+O0z/kOHXNZ5jqx3s8/y5EmG1CLrVPrPezhMG/ZlNdIU/0nXiHcvTz/MFSxn\n2mUHKvTtxWZ7lNQ6pdTxkHTesT6I4Fjb8Ral+BUa555r31zTUneCaWimjWZNJBGilBKT46tS8Poi\nBrGaLc9797pvkcK+mBi0GAyEYyKJLE407eFvaSpn9WpGlNsjedbe5ZQQTh5xdju1ztnU88UX5Awd\nw/doUTujKeS4rkWFS2ePubnieGlpFYM8VcKyE70B2EIPfmMIO+lGYY7Dbr8VbZ3pDWvsqJk84dJ/\nHf29Gqc2yJMZFcp+5gKoZaDCQEMpfoVGly4UornXtYw+zRzuYiUXc9M1+USKEow2xV9czAb68AeD\nuP2jge6vdeKEI+46Vht/sM/4AUwmmnKSk4mdPbf1E5mfOeLcz2Mq73AzUe2bE15SQFxJlr3Oecb/\n2zVzuOTgG/a6VqRzaE25Gb/RyJJt7V3LSkuhd28YM8Yr2WxvlJs5j3Zos/oThxxuxJm4hmqZyny2\n082ra/uKPZNcbfqX8B0X/D4bRo+uVznqGqX4FRVoOXMSuvE3cjE/YOjcgQhhosRktceOHInZ6v99\ntLJAq+eeC6mp9tNQ8OoBwGCgkS6f/OIwWLq01q6NdUGmTttl+hHXM5XXuZl3NbNceLj2sWJb3LUk\np3ARP7KBvgC8yRT6s5afjf1dPHu+jbqSSbzjOpgtk9Yf3plBbG+U8Wm/8IB10bQoz6xV5uezjw5E\nWR8OzThOSq82LrvK65yTJ+ny/RyXosQkAQMrmeAEMUrxKyrQ8pKzYdEibbNObCyRuhKMZqvi//VX\nstAyD1Wm+PdmxPEfXrSHN7BIXdB79diIseRTcOiUNsudH3iheW0BxZI/f0N7MFXycLIt7t7Km5hx\nhOM476q29ORP8olnylmr7eUvcH+FaxhP5iCQ3GJ6o0KdO0wxmmxh7VoRg7avoDCvDMviTxHxcRyk\nHf/hJSSCY7RAfPE5uhjXeD6ldRjQs+hvR26KibwHQGSHVnU3oB9Ril9hZ6w1ZG7jbs1cyiN0JkrM\n2izfSAS/Wz0cLOjJ/+zbCteZyx28zH/4ce7fWrtQWNy1EksBhcSQTSJl6W48X/zM6Z2aTAmJOm2m\nX4nzuc3UUz7kdtLzD9Ge/QC8vXswmQcL4aOP7G955/Obve2/+B8AC+VNXslmbtkG0CJexzyg5QUo\n/OI7xl7vkLFD78Zw/DgiKwvatIFt2+jDent9fj58d+Hz/HTfCq/GrA6HdmlvGx9wIwNYA0B6Rs1j\nVAUySvEr7CxiAqdoXCGYWqTOZJ/xv8h9zHcKyzT9VUfibMtnnyPTj3KQtgBkZ2iv8ZoffwiYeoAY\nCskkhWRO8eiKAf4WpwKnf9M8rWwbtSrDNuMvT1JjQUenDXx/PvYl3Hgje+nITSx0SaQinMx32Qdy\nPcpmCwpnMEBMjFb207uH+B/X2tu0uelCaNYMkpK0gvbtOQPHTDzvxw1c+ssDXPTSSI/jVZej+7X1\nhjNevNv+8DOXhdCuLSeU4lfYCcNMYzcZNiP0JkrKtBnfeqst2Ea7/T9DVha7OlxOx+vOo1vrXAxo\nCv/4N5uBEJvxp0RxjJZIdLz7V6+6tT1Ul8OHOd20C+BZ8ess7mMnxcdDp+lX0NG6iS/XFE1PNnOc\nFnQY1IL4wQ6PJtv/GSDruOe/g9mq+PV6iI7RFOozaDGEBrGarxjN0GsrBm6z5YsAyF3/t/24zFfh\nnwYPhkWLOHFE8zBqdsFZdHjwXwBc03m7jwYJLJTiV3gkUm/CaNZeeTuwz6VOf/IoJCfTdf9yDtCe\nXXTlT3oCcGK/9uqsbeAKkRl/5kH7cSwFjkiVAcDKNpOZnT4RgAQPG631f/zmch5JMeGUoNOB4cLz\n+ZkLAcjN17HF+v/s2zGL9ufE2Psc0zlyLJUUOx7sx0QLFouKAX3LpPbWaDBATKzrTDqCEkbLpYjG\nSRX6GRo7fpmdrzhiRu3f7aMQDr//DhMncuCTdQA06xBDu0bZZJHE1F7rPXQOTpTiVzg4fRpOVdyc\nFJGbQUmxNr1ynuUBPMSzLq/8APutOXoLumlvB5ai4pBZ3I3FkZBmHx2RCxb4URpXRrCSTJoQT65z\n5ki36MtFSy0imhKsUcguv5xGc58GYPKKK+1ths65it6dcriD1wA4anGsBRmLHP/f+3iR61nMieOu\n3wuzRVM3Oh3EGFyjwSZT+aa4iCjHQ+IGPrYf7+52Jdx9d+W/pCf27uVEp8Fcz0d05B8e5ykA4uI1\nL6gkTiOSEj1cJDhRil/hICEBGjeuUByJUUuZJyXGJppN/2surzSUM4COMopL9bBvH6WZuUiTudK2\nwYTNG8XGKJb5SZJybHeYJFpzxGPz/Th88gUWxOTJ8Npr9rK4va6pNl/nNgxxURAZyWA0b590Wtuz\nejkr/j/Q3B/3bXHN82CWevSYEQJiw1y/O/93wdJKZb2rg/uF3P/wEi/N8fCE++wz2FJJiI0BAzhn\n75cs5nr20dFeLARwxx1aHoB77qn6+kGKUvwKj+ynPbkkIJd9jVFE05J0LuebKhN5dGUnRaV6OHiQ\nj7mBI5xRadtgovwbz3Ku8JMkTuzbx5/njLeftnCf1toFk5MLZ1NOwptvasrOikhuzJc4ZvuiqbY/\ngNGjicRoL+/MbsA1QU1jtI1iR7Y6NoyBtrhrENqbRoTOYaZpw0GafvtepbJ2TM6hhIprFnvozH28\nVHWOgOuug1Gj3FZZMk9xqtymsdPPWd/gIiK05C8RNY9KG8h4m3pxhBBitxBirxBihpv6V4QQW6yf\nPUKIHKe6Mqe6AJkeKarD32gLhi+NWUXxyVz7je/s/92WA+Q278JOzmIVg4nSlVJc6uS+ESJsxjUZ\nhwGTlinKj5hO5XIejhn6G9xWRWuNaczjCpYxi8dYNeKZim6f113HlTjCFNy8xPqAa96cyDsm28tb\nxmuzemcbv03xr3jIsQ8AtMVd28YxTCbSrftAnmzzbtUKNjmZcEx8ztVuq9fO+sF9v7IyYijg8iPl\n9lvk50NBActwPBDacJCDtCHhgSmVyxFCeEzBI4TQA/OA4UA6sFEIsUxKudPWRko53an9nWBdDdIo\nllIGcHAThSfmcgeX8D0GzBiJJFJvgjXrcXbwWZQ0nfjD24lfvBiaNSPq4gKKDxVgKq7/hN51yW28\nwcfcYD83E8aBVoNpV/J3Fb3qEKORo5ffipbSGj5mHO03f+GxWxMyWYY1DEHS9RUbdOyozaStz4Ow\n/o6d2FFRjmYpzQyQByVGx6zb5sH1ARN47UgejVrHA9rirm3GT7NmtOQYEgFdLqla2BdeACEY/cZb\n9qJ27OeA1Vz122qdaxQfiwWOHWP7+VMpYhnfOEUdBTTXJeBKp7Wpm1lIm+zAjrrqS7yZ8fcB9kop\n90spS4FPgKoCV4wD604gRUhwLlsBCKeUEiKIiDFA1672+mH8SN9fn9PcNcaPh+HDWcUQVjGEz8Z+\n6S+x64TB/I5E8LdTgo5hpb7fTOQ1S5bw4qmJ9tMLvn0QevasooMbbryx0qqNpGpBypyIi3YsDDdJ\n1Nw4jcUOJVqI4y0vbf4G+7HZydTDpZc6LugpnWNcHMyfj+E2x5vGOCcVUxqlKXIyMjima8Va/UBE\n61acc8C9geEwrfkAx+88n9t5cMv1kBiaC7nu8EbxtwSX1aJ0a1kFhBBtgHbAz07FkUKINCHEOiGE\nd9GcFAGFzZNlKaPZQB8i9GUucV8+7fUC4We7T934Q0b3epGx3rjvPgA6s4ceVvPKIdr4TZzjpyOZ\nh2ab/5vONBvcqXoX2LrVVQmXI3XPYi44vMilLGG7IzxySpS22O2s+Iv0cXRFy1J25NkPoayMtD5T\nOZSud5h6QDO59OoFzz3nWU4hYPZs++mTzGQXXYgjj9MHtc1jxbfcQUuZzgDWunTtibafhNJSuOsu\n2nCYCXwAwCLGcztvENG5rWcZQghvFL+7rWuVPaLHAp9LKZ19xc6QUqYC1wOvCiE6uB1EiCnWB0Ra\npp9tpgpXon9YhsDC91xCJk2IOH0cwhz2/cQNKyvYiG+03lj5xAFapMiQ4LnnoKAASrW3H7C6Rh46\n5BdxTk1zJCdpa9xd/TWVzh4ijXbqBK1buxQ1+u5T+3FKx0aAk6ln/XqKyiJoy0HAGnHzl1/ovXE+\nXzPKMeMHiI3Vwjr38NISHBXF29zCz1yA4ZnZdOmqJ4lssrcf48jERzlQ1NRtt5bNrQ+bFSt45jXX\nv0+vF8bB8OEhu4hbGd4o/nTA+T/fCip1GxhLOTOPlPKY9ed+4Fdc7f/O7RZIKVOllKkpKSnumij8\nhOid6uK/nkAOCMHHjGMC71cI8QDwLlr8Fltyj578WbFRMKLTaco1LMyu+E2Eu7hC1ic2t8zr+ahm\nuqsGneLHOyy9tlvV+HsaCMHCfm+ymy4kdW5CBEYySaFsuMOGX37/QLWIjOQWFnIBv8KMGbBjB4mc\n5lijszhj0Wy6/ez+f2DRaUuZpYZoHuYZl7qOd42E778PrYS6XuCN4t8IdBJCtBNChKMp9wrGMyFE\nZyARHO9ZQohEIUSE9TgZGAjsLN9XEeBER5NPvP30TPYAMI5PeJ9/u+1isN7g29ESpybe6yaJR5Bj\nU/wA/9vs9kW2zklDW3R9ObWay2p798IPlXjDeEB/u8PzpUm0NiHI37CTGAq4hYUArNzdhhQyeYEH\n7N8FwHXGX110Opg5EzZutBclRRSxOd/VvDWeRbzFJGLJpzWHsVhDReSVuD7kLmO5s8WyQeFR8Usp\nzcAdwEpgF/CZlHKHEGKWEMLZQXYc8ImULis1ZwFpQoitwC/As87eQIogIcw1QmGbGKt/9siRmveH\nFyQ+MNlzoyDjOhwmj3cPDvWLDCbCiMBI09WfV69jhw5w0UU1GzTaESq58eE/ac8+fmA4RU6LulEU\nk8jpCl0NspYb+Z54wiXXQ2mJhWyL66LsOzf+yqSZrcjv1IuWEacok9psPne5w730AZ5j+bqGa1nw\nyo9fSrlCSnmmlLKDlPK/1rLHpZTLnNo8IaWcUa7fGinlOVLKc60/3yl/bUVw0MEpYmOze8ZqB998\nA//8477Dnj08xNP201B0mHieB9hl3eNwfueTfpHB3LINBr2EyMj6G7RRI/thfLwWbuEPp2Tk7dnH\njhuf5S8qLuzvLfXtRr4MXIO6DeUXwt5doD0g9uxBj8Ux43/P4eZqGXQ+9HUNONiQUDt3FV5xi1P2\npbh2yZ47dOrE1ThutFB8pdZjoTO7EVgoLPJwKy1dqtmSfYy5TFc780lNSElhPJqnT/xT97uEsRjD\nEn67+jViP3i9XkR5m0n241973MOXX+lxDlSkw4JFCli3zr7uBBCd43l3cyijFL/CK5yDaIlw75JT\nNEWbBTcix0PLIOW55xBAHPnk6z2EwxwzBi7xsFGpBpjLRP0rfmAhN5NLPCIqkpgIh/nmc66hVR8t\nUftgVtW5HP1T9tGSdO7jBYb8+SqJo893qdcJi5YHICuL17gLgJF8w/QnPPy/Qhyl+BVekeCkvOMT\nvPvaNDlHi974SNgLdSKT33ngATh8WFP8a/0Tt91s0WGoJKlKXWKgjHi0cA0xJY4cDnosMGwYoO37\n+JFhPMMM2lqTq/tcju9XcJC2vJD0rNt6vW3G7+SS+tn8LBKuurBO5AkWlOJXeEUjHBmW+vT3LgxD\n+LirkQjuv7f+Z6T1hl5PI3LJNkZ5blsHmC3CdVNUfdGvX4WiBE5r2VF69QIgMaKYYfzMjJF/sePC\nu+pGjm7dMNw4Toup7wadsGBBgMlELPlM52ViJo1rcO6b5VGKX+EV3dFS+i1gsve+37Gx2s9Qvsma\nNaMD+9iHl+6cx32bp9dc5p8ZP2vX2kMtGK1x/F/Uz8BlU8fx45CRAd98Q/RDd9OUE9zP876VIywM\nPvgAzjrLbbUOzdQjTWaKiCb6uisqeKk1RDwGaVMoAJpxEjN67VU+Yp53nSZPhsOH4eGH61Y4f6LT\n0YwTbKCPNtvVu38byqER9/Iy//1jP82vae6z4f1l6nGm1BoyOfG/97lWOLtyJSdznOYwtmJmrrrE\nZuoxGcuwoCcqKoQnIdVAzfgV3jF7tqb0jx3z3kUnMlKLrBgXV7ey+ZnkJMkpkpETJlba5klm8i43\n885/T/h07EBQ/LYZf3RZfuWNevRAbN+O+OjDytvUATqhKf6iAu1vFB0VGilAa4tS/ArveOQR7dW+\nue9mq6FCSvbflGEg5+NvKm3zKlrk8thOvv37maX/Fb8tDENktAd10q0bbuN71CE2U0/RYc0rzWnv\nWYNGKX6FopbYXF0H4X6B0ZlCk283NJiLTRiMhZ4b1iFfcDWvcjdDxrXwqxzu0Ntm/NM0M1RUpJrx\ng1L8CkWtSUGLJruTbpW2ucAaqfz0V7/6dOwy9BiMVZhY6oF48rmbOYimTTw3rmd01p27y63JWCzF\nleeJbkgoxa9Q1BLnzW2VYbODH6StT8c2Y8CQ6Oc1lAkT/Dt+FeiQlOXmc4B2AHRr6vl/1RBQil+h\nqCVJZHtsY1P8X3CNT8c2i3AM0X6Oh/Huu1qSkwBEL8qwoLPvIj/nuq4eejQMlOJXKGpJ4t2Ve/MA\nkJdnV/wAJqPvNrSZ0ft9cRedLmB943UFeeymC88ygwhRQmQb98laGhpK8SsUtaTRtR5i8GzZQjGO\nnb1Zx3xnZzZLAwadWrCsDFvi93ziiZJFfpYmcFCKX6GoJbqUxjzIs0RgrFj5xBMwZAgHaUcSWh6D\n7J2+8+U3Y/D/jD9IyHNKJtTQUYpfoagtCQlEU0QJkZSVs+L8/eQnjGEJAI2tin/DFbN8NrQZPQa9\nUvyVkYsjd4AF72JMNQSU4lcoaktKCtERmvItLnat+j/uZiljABjAGgBu4j2fDa1m/FWTQ8MOv1wZ\nSvErFD4g2hqauCjfdcrv7L7Zr10GAG046LNxNcWvbPyV4Tzj78oOP0oSWHil+IUQI4QQu4UQe4UQ\nM9zU/1sIkSmE2GL9THKqmyiE+Mf68eD+oFAEJ9FoC4eFOw+5lEscQcFSxgzkQn6idayPEtNIqSl+\nZeqplNM4AsU5Z+tq6HhU/EIIPTAPuBToCowTQrhzhv1UStnD+nnb2jcJmAn0BfoAM4UQIZh9VdHQ\nsaUfLCh0jf7ofBbX7QxiDSXkF6BF8RRCS1ZvdLMo7A02xa9MPZViM/XsoRP9L4r1szSBgzcz/j7A\nXinlfillKfAJMNrL618C/CClzJZSngZ+AEbUTFSFInCJJw+A/E9cA7UJHGaYuKQw4szZbKUHP1mG\nMo+psG8fHDxYs0Htil+ZeirDbI08n8hpmDbNz9IEDt4o/pbAEafzdGtZea4WQmwTQnwuhLDlOfO2\nL0KIKUKINCFEWmZmphdiKRSBgy015elft5arcSjl2KRw1tIfgIv4iTuYxyBWk3mihhu6lKnHI48y\nG4D4Of+F0d7OV0MfbxS/u8wF5acYXwNtpZTdgR+B96vRVyuUcoGUMlVKmZqSkuKFWApF4NDkXC0y\nZWaxkznBbMbidIs1bhFhn4Ha+INBfLi0hiYINeP3yCM8jUQQfuetoZ0Jrpp4o/jTgdZO562AY84N\npJRZUkrbdsS3gF7e9lUoQoHkAWcCkGl0UuKzZlFALN3ZymFa06J9JLN5tEJf46adVNgA4A0WC2YM\n6JXiV1QTbxT/RqCTEKKdECIcGAssc24ghHDOLjEK2GU9XglcLIRItC7qXmwtUyhCitg2jdFjJqc0\nxl527Km3+YNBxFJA683LQK9nPBUzUJlWr4W+fas/qN3UoxS/onp4VPxSSjNwB5rC3gV8JqXcIYSY\nJYQYZW12lxBihxBiK3AX8G9r32zgKbSHx0ZglrVMoQgpxE3/phG55Jw9yF72FpMBWMNA6NlTK1yz\npkLfmcxi9aaoCuUeUV49ihriVbJ1KeUKYEW5ssedjh8CHqqk70JgYS1kVCgCn5QUojiKschhsunE\nP1oVGYA1SUmvXsxlGvemy/0AAAwySURBVJ9xLYuYQFs0v//zWe1+8WvdOs3dc+jQinVqxq+oIWrn\nrkLhC4QgghJKDA5Tjy0U86rmYx3twsKYxnx+YyhtSvfai6Nxnz5xf//rWX3BY+7HzMjQMnCVqaxS\niuqhFL9C4SPCdWZKzY5AYEX9hwHQeMtPjkY2z5IxY1xi2DePK3B7zSv4WnsbcDOpL7vjbkyEY9yk\nQhEoqodS/AqFj4gQpZSUORR/sUmzpEZFl3MjzMyETz91KYoPd79715bH9/TpinWfb+sEwCvG22sq\nsqKBohS/QuEjwoWJErNj2cyu+Muv2yYnQ7iWLrGEcJpwkrgwN4rfaZp/5HDFKf/JI4GZ7lAR+CjF\nr1D4iAhRSmmZ45YqLjUQRin6KsLAh/+zk15soshULnVhQQEsWmQ/3d/7Otd6KTGh9fmqxdRay65o\nWCjFr1D4iAhhoiSvRLPjb9xI8dFsoiiuulPHjkRTRFGe2bX8ttvg3/+2n15l/gxjjtNbwe+/8zHX\nAzAq/lff/AKhiMEAgwf7W4qAQyl+hcJHhAsTpQVW88uCBZTkGYnAs8dNNEV2s5AN8997WcYVLmX/\n/JLuOMnNZbN1g7y4QwUfqxSTCVat8rcUAYdS/AqFj4jQlVJChHby9tuUEEE4nu3wUbEGinSu8Xre\n2JTKaOsG+d5R2wHYetUT9npzVBwAD/G0ijqpqDZK8SsUPiKiOIdiohBIHuJpSgn3bsafEkOxJQL+\n+stelkVj+3Eja+TP8XyI5Xdt5292nvaG0OLOa3z5KygaCErxKxQ+IpxS9qK5WD7LQ5QQ4ZXij8rP\noIhozbcfYMkSlzj+77/gCFOeNvgeAE6t2glAY8fzQaHwGqX4FQofUV7JH6AdEXERHvtFi2LMhLF4\nfx8A5DsL+dpq3y85lU+LqWN4ndsA6MsGAPJffRtwJIBRKKqDUvwKhY8or/g3kUpElOcY8DbPn+tZ\nDMCyb3Sk0RuA8MZxIAQXDnda/E1OphAtNERM/+6+EF3RwFCKX6HwEe4WcsO8yI4lMk+6nO+mc4U2\nZ04bzgX8TEf+4ZGs6QzjZwBiEsIqtFUoPKEUv0LhI9zZ88uoYvdWJW30aBE+f4hxShU4ejRnsoe9\ndOJpHrEXx8SqrFKK6qMUv0LhIwxUzKJVZvGsmMuSm7mc56O5al6w/D6X8r84p0LfmJgKRQqFR5Ti\nVyh8xA66VigzWzzP+G0++TbyW3cjRleEfqjrjtOHeKZCX6X4FTVBKX6FwkfoqGjPL5OeZ/wtwx3u\nmpbTueQdySHOUDHUw+VH3uARZruUKcWvqAleKX4hxAghxG4hxF4hxAw39fcKIXYKIbYJIX4SQrRx\nqisTQmyxfpaV76tQhAoxTslUlqD55Jstnm+xiR3/YCC/A1A8ZASFxBBb6iZDaatWzC68l5VcbC+K\njKyl0IoGicdvpRBCD8wDLgW6AuOEEOXfaf8EUqWU3YHPgeed6oqllD2sn1EoFCHKhVZPG4AE625b\nS4m5suZ2dM8/y7X/3979x1Z11nEcf3+ltqXA1kJbRMoGLESHhlnSbCjETUYYzIXuj/1BJgOdZomG\nxB9/LCxEEzUxIsYsJsZJtphpdIzhdD/CspCNJZumuG6yjcGAUnSUohTn0EE2XPz6x3l6e1pu29Pu\n3p7bcz6v5OY+57nPbZ/nm+d+7znPOfdedgHwz9dO8RC3c5o5xRvX1bH6nd/zMq08yMbC77qIjEWS\nPf5rgS5373b3i8BOoD3ewN33ufuFsNkBtJS2myKVbyMDX6PcRLR8c+F8gh9CX7KEqTcsA+DbfB+A\n80wfvv20abTO7WPjihPj76zkWpLEPxc4GdvuCXXD+TLwVGy71sw6zazDzG4dRx9FJgXr7S2UGxfP\nBsBJtkteWxN9RcNxrkr2z3p64Pnnx9ZBkSBJ4i82c4v8AiiY2QagDdgeq77C3duA24F7zazozDaz\nu8IbRGdfX1+xJiKVbc7A8kzznbdwN9t44rINiZ5ae/4sAH9kBQBW5ESxSKkkSfw9wLzYdgvQO7SR\nma0CtgLr3L3wSRZ37w333cBzQGuxf+LuO9y9zd3bmpqaEg9ApBLZHRvYxhaWbL8jUfupL+wF4Gqi\nL1/rbbymbH0TqRq9CS8Ci8xsAXAKWA/hp38CM2sFfgGscfczsfoG4IK7v2dmjcByBp/4Fcmm5uZB\nv5k7mlqiX9c6SyOfsEN85Kh+PETKZ9Q9fnd/H9gMPA0cBna5++tm9j0z679KZzswHXhkyGWbVwOd\nZvYKsA/4obsfKvkoRCrEc1zPCywf8/Nq164EoI9mps+qgYaGUndNpCDJHj/uvgfYM6TuO7HyqmGe\n9yco8jlzkYy6nvHtqdfOH/jahhlnuyHpSV6RcdAnd0VK7Zqxr89Xf2jgev+DfLKUvRG5RKI9fhFJ\n6N13Ycro388zVH31hUL5ItWl7JHIJbTHL1JKNTVQNfb9qSsuP1co/2D5nhFainxwSvwilcCdz4Wv\nfJjVviLlzkjWKfGLVILNmwtLPDObtQIr5aXEL1IJGhsLib+uXmv8Ul5K/CIV4iqOA1A/W4lfykuJ\nX6RC7OAuHmMdH19Sk3ZXJOOU+EUqxAzeYR1PQF1d2l2RjNNZJJFK0dEBBw6k3QvJASV+kUpx3XXR\nTaTMtNQjIpIzSvwiIjmjxC8ikjNK/CIiOaPELyKSM0r8IiI5o8QvIpIzSvwiIjlj7p52Hy5hZn3A\n38b59EbgbAm7k1WKUzKK0+gUo2TKHacr3b0pScOKTPwfhJl1untb2v2odIpTMorT6BSjZCopTlrq\nERHJGSV+EZGcyWLi35F2ByYJxSkZxWl0ilEyFROnzK3xi4jIyLK4xy8iIiPITOI3szVmdsTMusxs\nS9r9mWhmNs/M9pnZYTN73cy+HupnmtleMzsW7htCvZnZT0O8XjWzpbG/tSm0P2Zmm9IaU7mY2RQz\n+4uZPRm2F5jZ/jDeh82sOtTXhO2u8Pj82N+4J9QfMbOb0hlJeZlZvZntNrM3wrz6tObTYGb2zfB6\nO2hmD5lZ7aSYT+4+6W/AFOA4sBCoBl4BFqfdrwmOwRxgaSjPAI4Ci4EfAVtC/RZgWyjfDDwFGLAM\n2B/qZwLd4b4hlBvSHl+JY/Ut4LfAk2F7F7A+lO8DvhrKXwPuC+X1wMOhvDjMsRpgQZh7U9IeVxni\n9CDwlVCuBuo1nwbFZy5wApgam0dfnAzzKSt7/NcCXe7e7e4XgZ1Ae8p9mlDuftrdXw7l/wCHiSZm\nO9ELmHB/ayi3A7/ySAdQb2ZzgJuAve7+lrv/C9gLrJnAoZSVmbUAnwfuD9sGrAR2hyZDY9Qfu93A\njaF9O7DT3d9z9xNAF9EczAwzuwz4LPAAgLtfdPe30XwaqgqYamZVQB1wmkkwn7KS+OcCJ2PbPaEu\nl8IhZCuwH5jt7qchenMAmkOz4WKW9VjeC9wN/C9szwLedvf3w3Z8vIVYhMfPhfZZjxFER899wC/D\nstj9ZjYNzacCdz8F/Bh4kyjhnwNeYhLMp6wkfitSl8vLlcxsOvA74Bvu/u+Rmhap8xHqJz0zuwU4\n4+4vxauLNPVRHstsjGKqgKXAz929FThPtLQznNzFKpzfaCdanvkoMA1YW6Rpxc2nrCT+HmBebLsF\n6E2pL6kxsw8TJf3fuPujofof4ZCbcH8m1A8XsyzHcjmwzsz+SrQcuJLoCKA+HKrD4PEWYhEevxx4\ni2zHqF8P0OPu+8P2bqI3As2nAauAE+7e5+7/BR4FPsMkmE9ZSfwvAovC2fRqohMnj6fcpwkV1gof\nAA67+09iDz0O9F9JsQl4LFa/MVyNsQw4Fw7dnwZWm1lD2KNZHeomPXe/x91b3H0+0Rx51t2/AOwD\nbgvNhsaoP3a3hfYe6teHqzQWAIuAP0/QMCaEu/8dOGlmHwtVNwKH0HyKexNYZmZ14fXXH6PKn09p\nnxkv1Y3oqoKjRGfEt6bdnxTGv4Lo8PBV4EC43Uy0hvgMcCzczwztDfhZiNdrQFvsb91JdIKpC/hS\n2mMrU7xuYOCqnoXhhdYFPALUhPrasN0VHl8Ye/7WELsjwNq0x1OmGH0K6Axz6g9EV+VoPg2O0XeB\nN4CDwK+Jrsyp+PmkT+6KiORMVpZ6REQkISV+EZGcUeIXEckZJX4RkZxR4hcRyRklfhGRnFHiFxHJ\nGSV+EZGc+T+NZS52NqQ8DgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1261cedd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(Y_test,'r',Y_pred,'b')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}