Implementation of a Basic Neural Network from Scratch

Gradient Descent Demonstration.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implement a Basic Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The focus of this notebook is to delve deep into the working of neural networks internally. So, this does not implement many of the features used in full-scale frameworks like Mini-batching and different kinds of optimisation techniques (like *RMSProp* and *Adam*), but focuses on the core elements of deep learning which is **Vanilla Gradient Descent** and **Backpropagation**.\n",
    "\n",
    "The dataset used here is an extremely simple dataset containing weights of an animal's *body weight* and *brain mass*. It has only one feature to work with and one target feature. There are only 62 data points to work with. As specified before, this is a toy dataset used only to elucidate the internal working of a neural network.\n",
    "\n",
    "This is an example extending from [this document on Explaining Backpropagation](https://drive.google.com/open?id=1MCy8Ct-m9uPI22JMOylYPDH3boQOi67p)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing relevant libraries\n",
    "\n",
    "As you can see, we are not using any of the ML frameworks and aim to devise the techniques entirely from scratch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = \"data/x01.csv\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Implementing the cost function and activation function (also their derivatives)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section implements many of the utility functions that the network will use. This includes the cost function (**Squared Error**) and activation functions (**ReLU** and **Leaky ReLU**) essential for the forward propagation. This section also implements the derivatives of these functions which are required in the backpropagation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Cost Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def squared_error(output, target):\n",
    "    return 0.5*((target - output)**2)\n",
    "    \n",
    "def squared_error_deriv(output, target):\n",
    "    return -(target-output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### ReLU Activation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def activation_relu(value):\n",
    "    out = 0\n",
    "    if value > out:\n",
    "        return value\n",
    "    return out\n",
    "\n",
    "def activation_relu_deriv(value):\n",
    "    out = 0\n",
    "    if value > out:\n",
    "        return 1\n",
    "    return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Leaky ReLU Activation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def activation_lrelu(value):\n",
    "    out = 0\n",
    "    if value > out:\n",
    "        return value\n",
    "    else:\n",
    "        return value*0.01\n",
    "    return out\n",
    "\n",
    "def activation_lrelu_deriv(value):\n",
    "    out = 0\n",
    "    if value > out:\n",
    "        return 1\n",
    "    else:\n",
    "        return 0.01\n",
    "    return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Forward and Backward Propagation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The aim is to build a small Neural Network. The structure of the network in provided in the following image.\n",
    "\n",
    "![Neural Network to be Implemented](BackProp.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def forward_prop(w, x):\n",
    "    # These are the list of parameters which are held through each iteration through the network.\n",
    "    # Please refer to the diagram to understand the variables better.\n",
    "    inputs = []      # Lists the inputs to each layer\n",
    "    net = []         # Lists the intermediate state of each layer\n",
    "    outputs = []     # Lists the output of each layer\n",
    "    \n",
    "    # Propagating through Layer 1\n",
    "    inputs.append(x)\n",
    "    net.append(w[0]*inputs[0])\n",
    "    outputs.append(activation_lrelu(net[0]))\n",
    "    \n",
    "    # Propagating through layer 2\n",
    "    inputs.append(outputs[-1])\n",
    "    net.append(w[1]*inputs[1])\n",
    "    outputs.append(activation_lrelu(net[1]))\n",
    "    \n",
    "    return inputs, net, outputs\n",
    "        \n",
    "    \n",
    "def backprop(weights, inputs, net, outputs, target):\n",
    "    # Setting gradients to zero\n",
    "    grad_j = np.ones(len(weights))\n",
    "    # Back propagating through Layer 2\n",
    "    grad_j[1] = squared_error_deriv(outputs[1], target)*activation_lrelu_deriv(net[1])\n",
    "    # Back propagating through Layer 1\n",
    "    grad_j[0] = grad_j[1]*weights[1]*activation_lrelu_deriv(net[0])\n",
    "    \n",
    "    for i in range(len(grad_j)):\n",
    "        grad_j[i] = grad_j[i]*inputs[i]\n",
    "    return grad_j"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Gradient Descent\n",
    "\n",
    "Updating weights based on the $\\nabla J$ calculated using backpropagation. This uses the following formula:\n",
    "\n",
    "$$W = W - \\eta \\cdot \\nabla J$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradient_descent(weights, grad, learning_rate=0.01):\n",
    "    for i in range(len(weights)):\n",
    "        weights[i] = weights[i] - learning_rate*grad[i]\n",
    "    return weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_nn_output(w, value):\n",
    "    return forward_prop(w, value)[-1][-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reading the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Brain</th>\n",
       "      <th>Body</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3.385</td>\n",
       "      <td>44.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.480</td>\n",
       "      <td>15.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.350</td>\n",
       "      <td>8.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>465.000</td>\n",
       "      <td>423.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>36.330</td>\n",
       "      <td>119.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>27.660</td>\n",
       "      <td>115.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>14.830</td>\n",
       "      <td>98.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1.040</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>4.190</td>\n",
       "      <td>58.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.425</td>\n",
       "      <td>6.4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Brain   Body\n",
       "0    3.385   44.5\n",
       "1    0.480   15.5\n",
       "2    1.350    8.1\n",
       "3  465.000  423.0\n",
       "4   36.330  119.5\n",
       "5   27.660  115.0\n",
       "6   14.830   98.2\n",
       "7    1.040    5.5\n",
       "8    4.190   58.0\n",
       "9    0.425    6.4"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(dataset)\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we are working only to test the technique, we are using the whole dataset for training the model. In real world scenarios, you will be required to break the dataset into *Training*, *Validation* and *Test* sets. In this case, we are using the whole dataset for training.\n",
    "\n",
    "In this problem, we are trying to predict the **Body Weight** from the **Brain Mass**. Hence, we are loading the brain mass data into **X** and the body weight into **Y**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.array(df[\"Brain\"].tolist()) # Inputs\n",
    "Y = np.array(df[\"Body\"].tolist()) # Targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Training the network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 : \t MSE =  6016891.0202 \t ∇ J =  [-413656.158552 -239480.570309]\n",
      "Epoch 2 : \t MSE =  36647186.8484 \t ∇ J =  [6049400.841672 7839796.868979]\n",
      "Epoch 3 : \t MSE =  2663897.4235 \t ∇ J =  [-2914167.301695 -2170051.360963]\n",
      "Epoch 4 : \t MSE =  1162093.8962 \t ∇ J =  [-551350.403567 -275161.311856]\n",
      "Epoch 5 : \t MSE =  598898.4039 \t ∇ J =  [-340187.589273 -141669.458237]\n",
      "Epoch 6 : \t MSE =  342311.1243 \t ∇ J =  [-231358.895595  -82988.066715]\n",
      "Epoch 7 : \t MSE =  213564.7293 \t ∇ J =  [-164855.377781  -52190.037117]\n",
      "Epoch 8 : \t MSE =  145281.4682 \t ∇ J =  [-120587.211586  -34397.315766]\n",
      "Epoch 9 : \t MSE =  107797.6637 \t ∇ J =  [-89633.128637 -23455.034004]\n",
      "Epoch 10 : \t MSE =  86753.3803 \t ∇ J =  [-67320.190103 -16414.231083]\n",
      "Epoch 11 : \t MSE =  74757.5131 \t ∇ J =  [-50916.264006 -11722.732784]\n",
      "Epoch 12 : \t MSE =  67846.7589 \t ∇ J =  [-38696.390888  -8507.873923]\n",
      "Epoch 13 : \t MSE =  63835.4512 \t ∇ J =  [-29510.393249  -6253.992524]\n",
      "Epoch 14 : \t MSE =  61494.4156 \t ∇ J =  [-22560.837503  -4643.970749]\n",
      "Epoch 15 : \t MSE =  60122.7114 \t ∇ J =  [-17279.163149  -3476.129152]\n",
      "Epoch 16 : \t MSE =  59316.6009 \t ∇ J =  [-13251.737264  -2618.411279]\n",
      "Epoch 17 : \t MSE =  58841.8279 \t ∇ J =  [-10173.207441  -1982.099857]\n",
      "Epoch 18 : \t MSE =  58561.7379 \t ∇ J =  [-7815.737561 -1506.227212]\n",
      "Epoch 19 : \t MSE =  58396.2931 \t ∇ J =  [-6007.989603 -1148.055044]\n",
      "Epoch 20 : \t MSE =  58298.4743 \t ∇ J =  [-4620.361465  -877.104394]\n",
      "Epoch 21 : \t MSE =  58240.5973 \t ∇ J =  [-3554.394044  -671.318254]\n",
      "Epoch 22 : \t MSE =  58206.3339 \t ∇ J =  [-2735.043518  -514.53688 ]\n",
      "Epoch 23 : \t MSE =  58186.0413 \t ∇ J =  [-2104.972018  -394.800026]\n",
      "Epoch 24 : \t MSE =  58174.019 \t ∇ J =  [-1620.288396  -303.181782]\n",
      "Epoch 25 : \t MSE =  58166.8947 \t ∇ J =  [-1247.347144  -232.975958]\n",
      "Epoch 26 : \t MSE =  58162.6721 \t ∇ J =  [-960.328875 -179.117   ]\n",
      "Epoch 27 : \t MSE =  58160.1689 \t ∇ J =  [-739.403585 -137.762318]\n",
      "Epoch 28 : \t MSE =  58158.6849 \t ∇ J =  [-569.331671 -105.987236]\n",
      "Epoch 29 : \t MSE =  58157.805 \t ∇ J =  [-438.395642  -81.559868]\n",
      "Epoch 30 : \t MSE =  58157.2833 \t ∇ J =  [-337.582762  -62.773508]\n",
      "Epoch 31 : \t MSE =  58156.9739 \t ∇ J =  [-259.958734  -48.320961]\n",
      "Epoch 32 : \t MSE =  58156.7904 \t ∇ J =  [-200.187222  -37.199783]\n",
      "Epoch 33 : \t MSE =  58156.6816 \t ∇ J =  [-154.160917  -28.64049 ]\n",
      "Epoch 34 : \t MSE =  58156.6171 \t ∇ J =  [-118.718072  -22.05198 ]\n",
      "Epoch 35 : \t MSE =  58156.5789 \t ∇ J =  [-91.424571 -16.979918]\n",
      "Epoch 36 : \t MSE =  58156.5562 \t ∇ J =  [-70.40634 -13.07494]\n",
      "Epoch 37 : \t MSE =  58156.5427 \t ∇ J =  [-54.220399 -10.068301]\n",
      "Epoch 38 : \t MSE =  58156.5347 \t ∇ J =  [-41.755652  -7.753222]\n",
      "Epoch 39 : \t MSE =  58156.53 \t ∇ J =  [-32.156523  -5.970567]\n",
      "Epoch 40 : \t MSE =  58156.5272 \t ∇ J =  [-24.764175  -4.597848]\n",
      "Epoch 41 : \t MSE =  58156.5255 \t ∇ J =  [-19.071259  -3.540772]\n",
      "Epoch 42 : \t MSE =  58156.5245 \t ∇ J =  [-14.68708   -2.726747]\n",
      "Epoch 43 : \t MSE =  58156.5239 \t ∇ J =  [-11.310765  -2.099878]\n",
      "Epoch 44 : \t MSE =  58156.5236 \t ∇ J =  [-8.710615 -1.617132]\n",
      "Epoch 45 : \t MSE =  58156.5234 \t ∇ J =  [-6.708199 -1.24537 ]\n",
      "Epoch 46 : \t MSE =  58156.5233 \t ∇ J =  [-5.166106 -0.959074]\n",
      "Epoch 47 : \t MSE =  58156.5232 \t ∇ J =  [-3.978513 -0.738597]\n",
      "Epoch 48 : \t MSE =  58156.5231 \t ∇ J =  [-3.063927 -0.568804]\n",
      "Epoch 49 : \t MSE =  58156.5231 \t ∇ J =  [-2.359587 -0.438045]\n",
      "Epoch 50 : \t MSE =  58156.5231 \t ∇ J =  [-1.817163 -0.337346]\n"
     ]
    }
   ],
   "source": [
    "err_dict_for_viz = {}\n",
    "\n",
    "np.random.seed(1)\n",
    "weights = np.random.uniform(size=2)\n",
    "epochs = 51\n",
    "\n",
    "\n",
    "for e in range(1, epochs):\n",
    "    ## Setting gradient of J(w) to 0\n",
    "    grad = np.zeros(2)\n",
    "    \n",
    "    ## Looping through all the data points\n",
    "    for ind in range(len(X)):\n",
    "        i, n, o = forward_prop(weights, X[ind])\n",
    "        temp = backprop(weights, i, n, o, Y[ind])\n",
    "        for i in range(2):\n",
    "            grad[i] += temp[i]\n",
    "    ## Accummulating the gradient and then taking the mean\n",
    "    grad = grad/len(X)\n",
    "    \n",
    "    ## Using gradient of J(w) to perform gradient descent\n",
    "    weights = gradient_descent(weights, grad, 0.000005)\n",
    "    \n",
    "    ## Calculating MSE for all the samples with the new weight and taking the mean\n",
    "    net_error = 0\n",
    "    for index in range(len(X)):\n",
    "        net_error += squared_error(get_nn_output(weights, X[index]), Y[index])\n",
    "    net_error /= len(X)\n",
    "    \n",
    "    if e>8 and e<20:\n",
    "        err_dict_for_viz[\"e\"+str(e)] = (np.mean(Y)-((net_error*2)**0.5), net_error)\n",
    "    \n",
    "    print(\"Epoch \"+str(e)+\" : \\t MSE = \", np.around(net_error, decimals=4), \"\\t \\u2207 J = \", np.around(grad, decimals=6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualising the gradient descent\n",
    "\n",
    "Herein, a visualisation of the gradient descent process is provided on the MSE error function. The MSE at every 10th epoch is selected to be plotted here which can show the route which gradient descent plots on the gradient function.\n",
    "\n",
    "In this visualisation, we show the descent between epoch 9 and 19 (including both). The tags represent the epoch for which the point is plotted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Mapping values of the network predictions and errors for plotting\n",
    "x = []\n",
    "y = []\n",
    "label = []\n",
    "\n",
    "for k, v in err_dict_for_viz.items():\n",
    "    label.append(k)\n",
    "    x.append(v[0])\n",
    "    y.append(v[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABCgAAAHSCAYAAAAuWfokAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXRV1cHG4Xffe0MG5iHRSAgogxggBAghqGgVUKxMWhAkCgpKHFoHFASplq8FLdZqRQZFUIaggFgFq2I1KgoCIZEwyCxjEJkSCJBApv39kYtlFjKdDL9nrax7s8/ge+yya/Gyz97GWisAAAAAAAAnuZwOAAAAAAAAQEEBAAAAAAAcR0EBAAAAAAAcR0EBAAAAAAAcR0EBAAAAAAAcR0EBAAAAAAAc53E6QFGrU6eObdCggdMxAAAAAADAGZKSkg5YawPPdazcFRQNGjRQYmKi0zEAAAAAAMAZjDE7zneMVzwAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKCgAAAAAAIDjKChKgePZufr5UKbTMQAAAAAAcAwFRSnw5Jxk3f3WMh08esLpKAAAAAAAOIKCohR48Iar9Mvh4xo0PVGZWblOxwEAAAAAoMRRUJQCrUNr6rW+rbQq5ZAem71SuXnW6UgAAAAAAJQoCopSokvzy/WXrmH6Yt1e/fXjH2UtJQUAAAAAoOLwOB0A/3PfdVdq96FMvfXdNtWt6a/BNzR0OhIAAAAAACWCgqKUGXHbNfr58HG98OkGBVf3V7eWVzgdCQAAAACAYkdBUcq4XEb/7N1S+9NP6Km5qxRU1VftrqrtdCwAAAAAAIoVa1CUQn4+bk3u30b1avnrwRmJ2rz3iNORAAAAAAAoVhQUpVSNgEqadn+UKnncuu+dFdqXftzpSAAAAAAAFBsKilKsXq0AvXNfW6VlZOn+aSt09ESO05EAAAAAACgWFBSlXIuQ6prQr7U2/HJEj876Qdm5eU5HAgAAAACgyFFQlAE3NQ3SmJ7NtWjTfv35w7Wy1jodCQAAAACAIsUuHmVE36hQ7T6Uqde/2qK6Nf31WMfGTkcCAAAAAKDIUFCUIUM6N9HuQ5l65YtNuqKGv3q1CXE6EgAAAAAARYKCogwxxujvd4ZrX/oJDf9gtS6r5qsOjQOdjgUAAAAAQKGxBkUZU8nj0sR7WqtRUBU9HPeD1v2c7nQkAAAAAAAKjYKiDKrm56N37m+rqn4e3fdOgnalZjgdCQAAAACAQqGgKKOCq/tr+sAoHc/O1YB3EpR2LMvpSAAAAAAAFBgFRRnW5LKqmnpfW6WkZWrg9BXKzMp1OhIAAAAAAAVCQVHGtW1QS+P6ttKqXYf0x3d/UE5untORAAAAAAC4ZBQU5UCX5pfrrz2aK37DPo38cK2stU5HAgAAAADgkrDNaDlxT3R97U0/rte/2qLLqvtpSOcmTkcCAAAAAOCiUVCUI0M6N9He9OMaF79ZQVV9dU90facjAQAAAABwUSgoyhFjjF64o4UOHs3S8/PXKrCqr25tdrnTsQAAAAAA+E2sQVHOeNwuje/XWi3r1dBj763Uiu2pTkcCAAAAAOA3/WZBYYx52xizzxiz9pSxfxhjNhhjVhtjPjTG1Djl2AhjzBZjzEZjzK2njLcxxqzxHhtnjDHecV9jzBzv+HJjTINTrhlgjNns/RlQVA9d3vlXcmvqgLaqW8Nfg6at0Ka9R5yOBAAAAADABV3MDIppkrqcMfaFpObW2nBJmySNkCRjTJikvpKaea+ZaIxxe6+ZJGmwpMben5P3HCQpzVrbSNKrksZ671VL0l8ktZMUJekvxpial/6IFVOtypU0fWCUfH3cGvB2gvYcznQ6EgAAAAAA5/WbBYW19ltJqWeM/ddam+P9dZmkEO/3HpJmW2tPWGu3SdoiKcoYEyypmrV2qc3fA3OGpJ6nXDPd+32epI7e2RW3SvrCWptqrU1TfilyZlGCC6hXK0DT74/S0eM5GvB2gg5nZDsdCQAAAACAcyqKNSgGSvrM+72upF2nHEvxjtX1fj9z/LRrvKXHYUm1L3AvXIKwK6rpzXvbaNuBY3pwRqKOZ+c6HQkAAAAAgLMUqqAwxoyUlCNp1smhc5xmLzBe0GvOzDHYGJNojEncv3//hUNXQNc2qqNX7opQwvZUPTE7Wbl55/zXCAAAAACAYwpcUHgXrewqKcb72oaUP8uh3imnhUj62Tseco7x064xxngkVVf+KyXnu9dZrLWTrbWR1trIwMDAgj5Sudat5RV6vmuYFv74i0Yt+FH/+58MAAAAAADnFaigMMZ0kfSMpO7W2oxTDi2Q1Ne7M8eVyl8MM8Fau0fSEWNMtHd9if6S5p9yzckdOnpJ+spbeHwu6RZjTE3v4pi3eMdQQAOvv1KxN16lmct2aFz8FqfjAAAAAADwK89vnWCMeU/S7yTVMcakKH9njRGSfCV94d0tdJm19iFr7Y/GmLmS1in/1Y9HrbUnFz14WPk7gvgrf82Kk+tWTJU00xizRfkzJ/pKkrU21RjzN0krvOf91Vp72mKduHTDuzTVwaNZevXLTapVpZLuja7vdCQAAAAAAGTK21T/yMhIm5iY6HSMUi0nN08PxSUpfsM+vX53K3UNv8LpSAAAAACACsAYk2StjTzXsaLYxQNljMft0vh+rRVZv6aenJOs7zazsCgAAAAAwFkUFBWUn49bUwa0VcPAKoqdmaRVuw45HQkAAAAAUIFRUFRg1f19NGNglGpXqaT73knQln1HnY4EAAAAAKigKCgquKBqfpo5sJ3cLqP+U5drz+FMpyMBAAAAACogCgqoQZ3KmnZ/lI4cz9G9UxOUdizL6UgAAAAAgAqGggKSpOZ1q2ty/0jtTM3QwOkrlJGV43QkAAAAAEAFQkGBX7VvWFvj+rbSql2H9FDcD8rKyXM6EgAAAACggqCgwGm6NL9cL9zRQt9u2q+n31+lvDzrdCQAAAAAQAXgcToASp++UaFKzcjSSws3qlblSvpLtzAZY5yOBQAAAAAoxygocE4P39hQB49maeribapTpZL+eHNjpyMBAAAAAMoxCgqckzFGI39/jdKOZenl/25SzcqVFNOuvtOxAAAAAADlFAUFzsvlMhrbK1xpGVn680drVcO/km4PD3Y6FgAAAACgHGKRTFyQj9uliTFt1Ca0pp6Ys1KLNu13OhIAAAAAoByioMBv8q/k1tT72qpxUFXFzkxU4vZUpyMBAAAAAMoZCgpclOr+Ppo+MErB1f11/7QV+vHnw05HAgAAAACUIxQUuGiBVX0V90A7VfH1aMDbCdq6/6jTkQAAAAAA5QQFBS5J3Rr+mjmonayV7p2aoJ8PZTodCQAAAABQDlBQ4JI1Cqqi6QOjlJ6ZrXumLteBoyecjgQAAAAAKOMoKFAgzetW19T72mp3WqYGvJ2g9OPZTkcCAAAAAJRhFBQosKgra+mNe9to4y9H9MC0RGVm5TodCQAAAABQRlFQoFBuujpIr/aJ0IodqXp4VpKycvKcjgQAAAAAKIMoKFBo3VpeoRfuaKFvNu7XkLnJys2zTkcCAAAAAJQxHqcDoHy4OypU6ZnZevGzDarq56MX7mguY4zTsQAAAAAAZQQFBYpM7I0NdTgzWxO/+UnV/D0acds1TkcCAAAAAJQRvOKBIjX01qt1T3So3ly0VRO/2XLJ18+ZM0fh4eFq1qyZhg0bVgwJAQAAAAClEQUFipQxRn/t3lw9Iq7QSws3auayHRd97cGDBzV06FDFx8frxx9/1N69exUfH1+MaQEAAAAApQUFBYqcy2X0cu+W6tg0SM/PX6sPV6ac87y4uDhFRUUpIiJCsbGx2rJli5o0aaLAwEBJUqdOnfTBBx+UZHQAAAAAgEMoKFAsfNwuTYhprfZX1dZTc1fpszV7Tju+fv16zZkzR0uWLFFycrLcbrc2bNigDRs2aPv27crJydFHH32kXbt2OfQEAAAAAICSxCKZKDZ+Pm691T9S905drsdmr9RkH7duahokSYqPj1dSUpLatm0rScrMzFRQUJAmTZqkPn36yOVy6dprr9XWrVudfAQAAAAAQAmhoECxquzr0Tv3RylmyjI9ND1B78S/pmsT42Vr1NCA66/XiwsWnHVNt27dJEmTJ0+W2+0u6cgAAAAAAAfwigeKXXV/H82otkuhB1L0wPWxSgq+Wh3T0jTvP//RvokTJUmpqanasWOH9u3bJ0lKS0vTxIkT9cADDzgZHQAAAABQQigoUCJq/eVZzXrvWQUdTdN9vUcpL+gqjbZWtzzxhMLDw9W5c2ft2bNHjz/+uMLCwnTddddp+PDhatKkidPRAQAAAAAlwFhrnc5QpCIjI21iYqLTMXAml0uyVrurBuqumLHK9PHVnHeHq3FqipSX53Q6AAAAAEAJMMYkWWsjz3WMGRQoGaGhkqS6R/Zr1uyRcuflKqbPaG0Pa+NwMAAAAABAaUBBgZIxZowUECBJanBoj2bN+bOyPT6KueN57T6U6XA4AAAAAIDTKChQMmJipMmTpfr1JWPUpLLRzBZG6e5KinlrmfalH3c6IQAAAADAQaxBAUcl7UjTvVOXK6Smv2YPbq9alSs5HQkAAAAAUExYgwKlVpv6NTVlQKR2HMxQ/7eXK/14ttORAAAAAAAOoKCA465tWEdv3NNGG385ovvfWaFjJ3KcjgQAAAAAKGEUFCgVbmoapHF9W2nlzjQ9OCNRx7NznY4EAAAAAChBFBQoNW5rEayXe7fU0q0H9XBckk7kUFIAAAAAQEVBQYFS5c7WIXrhjhb6euN+/fHdlcrOzXM6EgAAAACgBFBQoNS5OypUf+3RTF+s26vHZ69UDiUFAAAAAJR7HqcDAOfSv30DZeXkafQn6+VxrdKrfSLkdhmnYwEAAAAAigkFBUqtBzpcpexcq7ELN8jjNnq5V0u5KCkAAAAAoFyioECp9vDvGio7N0+vfLFJldwuvXBHC0oKAAAAACiHKChQ6j3WsbGyc/P0+ldb5HEb/a1HcxlDSQEAAAAA5QkFBcqEIZ2bKCs3T28u2ioft0vPdw2jpAAAAACAcoSCAmWCMUbDuzRVdo7V20u2ycft0ojbmlJSAAAAAEA5QUGBMsMYo+e6XqPs3DxN/narfNxGT99yNSUFAAAAAJQDFBQoU4wx+r/uzZSTl6cJX/+kSm63Hu/U2OlYAAAAAIBCoqBAmeNyGY3p2ULZuVavfrlJHrfRozc1cjoWAAAAAKAQXL91gjHmbWPMPmPM2lPGahljvjDGbPZ+1jzl2AhjzBZjzEZjzK2njLcxxqzxHhtnvPPyjTG+xpg53vHlxpgGp1wzwPvP2GyMGVBUD42yz+UyGvuHcPWIuEL/+Hyj3vp2q9ORAAAAAACF8JsFhaRpkrqcMTZcUry1trGkeO/vMsaESeorqZn3monGGLf3mkmSBktq7P05ec9BktKstY0kvSpprPdetST9RVI7SVGS/nJqEQK4XUb/7N1St7cI1phP12vakm1ORwIAAAAAFNBvFhTW2m8lpZ4x3EPSdO/36ZJ6njI+21p7wlq7TdIWSVHGmGBJ1ay1S621VtKMM645ea95kjp6Z1fcKukLa22qtTZN0hc6uyhBBedxu/SvvhG6Jewyjfp4nWYu3e50JAAAAABAAVzMDIpzucxau0eSvJ9B3vG6knadcl6Kd6yu9/uZ46ddY63NkXRYUu0L3As4jY/bpfH9WqvTNZfpufk/UlIAAAAAQBlU0ILifM6136O9wHhBrzn9H2rMYGNMojEmcf/+/RcVFOVLJY9LE2Naq9M1QfklxbIdTkcCAAAAAFyCghYUe72vbcj7uc87niKp3innhUj62Tseco7x064xxngkVVf+KyXnu9dZrLWTrbWR1trIwMDAAj4Syrr8kqJNfknx0VpKCgAAAAAoQwpaUCyQdHJXjQGS5p8y3te7M8eVyl8MM8H7GsgRY0y0d32J/mdcc/JevSR95V2n4nNJtxhjanoXx7zFOwacVyWPSxNOzqT4aK3iKCkAAAAAoEzw/NYJxpj3JP1OUh1jTIryd9b4u6S5xphBknZK6i1J1tofjTFzJa2TlCPpUWttrvdWDyt/RxB/SZ95fyRpqqSZxpgtyp850dd7r1RjzN8krfCe91dr7ZmLdQJn8fW4NSGmtR6J+0F//ih/d9x7ous7nAoAAAAAcCEmf7JC+REZGWkTExOdjoFS4EROrh6J+0HxG/ZpzB3NFdOOkgIAAAAAnGSMSbLWRp7rWFEvkgmUGr4etybe01o3Nw3SyA/XatZyXvcAAAAAgNKKggLlmq/HrUmUFAAAAABQ6lFQoNw7s6R4d/lOpyMBAAAAAM5AQYEK4dSS4tkP11BSAAAAAEApQ0GBCuNkSXHT1YGUFAAAAABQylBQoELx9bj1xr1tfi0p3kugpAAAAACA0oCCAhVO/kyK/JJixL+ZSQEAAAAApQEFBSokP5//lRTPfrhGM5dudzoSAAAAAFRoFBSosPx88l/36HRNkJ6b/6PeXrzN6UgAAAAAUGFRUKBC8/W4NTGmjbo0u1x//c86Tf72J6cjAQAAAECFREGBCq+Sx6XX+7XS7eHBeuHTDZrw9RanIwEAAABAheNxOgBQGvi4XXqtT4Q8LqN/fL5ROblWj3dq7HQsAAAAAKgwKCgAL4/bpVfuipDH5dKrX25STl6ehnRuImOM09EAAAAAoNyjoABO4XYZ/aNXuHzcRq9/tUXZuVbPdLmakgIAAAAAihkFBXAGl8vohTtayOM2emPRT8rOzdOfb7+GkgIAAAAAihEFBXAOLpfR33o0l8fl0tTF25STm6dR3ZtRUgAAAABAMaGgAM7DGKO/dAuTj9vore+2KTvPanSP5nK5KCkAAAAAoKhRUAAXYIzRs7+/Rh63S5O++Uk5uXl68c5wuSkpAAAAAKBIUVAAv8EYo2G3Xi0ft0vj4jcrJ9fqH71bUlIAAAAAQBGioAAugjFGQzo3kcdl9MoXm5STZ/XKXS3lcbucjgYAAAAA5QIFBXAJHuvYWD5ul8Yu3KCsnDyNu7uVKnkoKQAAAACgsPiTFXCJHv5dQz3fNUwLf/xFg2cm6nh2rtORAAAAAKDMo6AACmDg9VfqxTtbaNGm/brvnQQdPZHjdCQAAAAAKNMoKIACujsqVK/eFaEV29N079TlOpyZ7XQkAAAAACizKCiAQujZqq4m9GuttbsPq99by5R6LMvpSAAAAABQJlFQAIXUpfnlmtw/Ulv2HVWfN5dqX/pxpyMBAAAAQJlDQQEUgZuuDtK0+6O0+1Cmer+5VClpGU5HAgAAAIAyhYICKCLtG9ZW3APtlHosS3e9sVTbDxxzOhIAAAAAlBkUFEARah1aU+89GK3jOXnq/eZSbdp75JLvMXLkSNWrV09VqlQ5bfzEiRPq06ePGjVqpHbt2mn79u1FlBoAAAAAnEdBARSx5nWra87gaBlJfd5cqrW7D1/S9d26dVNCQsJZ41OnTlXNmjW1ZcsWPfnkk3rmmWeKKDEAAAAAOI+CAigGjS+rqrmx7RVQyaO731qmpB1p5zwvLi5OUVFRioiIUGxsrHJzcxUdHa3g4OCzzp0/f74GDBggSerVq5fi4+NlrS3W5wAAAACAkkJBARSTBnUqa+5D7VW7ciXdO3W5vv/pwGnH169frzlz5mjJkiVKTk6W2+3WrFmzznu/3bt3q169epIkj8ej6tWr6+DBg8X6DAAAAABQUjxOBwDKs7o1/DU3tr1ipizX/e+s0Bsh6brp789IO3cqvkYNJVmrtm3bSpIyMzMVFBR03nuda7aEMabYsgMAAABASWIGBVDMgqr5aU5sezX2ZGnwT3762D9UslY2LU0Djh5V8tChSk5O1saNGzVq1Kjz3ickJES7du2SJOXk5Ojw4cOqVatWCT0FAAAAABQvCgqgBNSqXEnvznpGET9v1GPdh+q9lreqo6R5OTnaN3y4JCk1NVU7duw47z26d++u6dOnS5LmzZunm2++mRkUAAAAAMoNCgqghFT7aZNmzH1eN279QSO6/EnfRv1BoyXdkpKi8PBwde7cWXv27NGwYcMUEhKijIwMhYSE/DqrYtCgQTp48KAaNWqkV155RX//+98dfR4AAAAAKEqmvO0CEBkZaRMTE52OAZytQQNpxw5luTwa0nWI/nPNDXp46fsatuMbme3bnU4HAAAAAMXOGJNkrY081zEWyQRKypgx0uDBqpSRodc+fllVTxzTpPa9lX57d/0tz8rl4nUNAAAAABUXr3gAJSUmRpo8WapfX25ZvbDhP3oo8LhmpfrqiTnJys7NczohAAAAADiGGRRASYqJyf+RZCQNl1Ttmy16aeFGHT2Ro4kxreXn43Y0IgAAAAA4gRkUgMMe+V0jje7ZXF9v3Kf+byfoyPFspyMBAAAAQImjoABKgXui6+tffSL0w4403f3WMh08esLpSAAAAABQoigogFKiR0RdTe7fRpv3HtVdby7VnsOZTkcCAAAAgBJDQQGUIjc3vUwzB7XTvvQT6jVpqbYdOOZ0JAAAAAAoERQUQCkTdWUtvTc4WpnZuer9xvda93O605EAAAAAoNhRUAClUPO61TU3tr183C71mbxUK7anOh0JAAAAAIoVBQVQSjUKqqL3H2qvwCq+umfKcn25bq/TkQAAAACg2FBQAKVYSM0Avf9Qe119eVXFxiVpbuIupyMBAAAAQLGgoABKudpVfPXug9Fqf1VtDZu3Wm8s+knWWqdjAQAAAECRoqAAyoAqvh69fV9bdQ0P1t8/26Axn6xXXh4lBQAAAIDyw+N0AAAXp5LHpXF9W6l25UqasnibDh7L0ku9wuXjpmcEAAAAUPZRUABliMtlNKp7M9Wp4qt/frFJaRlZmhjTWgGV+E8ZAAAAQNlWqL96NcY8aYz50Riz1hjznjHGzxhTyxjzhTFms/ez5innjzDGbDHGbDTG3HrKeBtjzBrvsXHGGOMd9zXGzPGOLzfGNChMXqA8MMboTx0b64U7WujbTfsVM2W5DmVkOR0LAAAAAAqlwAWFMaaupMckRVprm0tyS+orabikeGttY0nx3t9ljAnzHm8mqYukicYYt/d2kyQNltTY+9PFOz5IUpq1tpGkVyWNLWheoLzp1y5UE2Na68ef09X7jaXaczjT6UgAAAAAUGCFfXndI8nfGOORFCDpZ0k9JE33Hp8uqaf3ew9Js621J6y12yRtkRRljAmWVM1au9Tmb00w44xrTt5rnqSOJ2dXAJC6NA/W9Puj9Mvh4/rDxO+1Zd8RpyMBAAAAQIEUuKCw1u6W9LKknZL2SDpsrf2vpMustXu85+yRFOS9pK6kXafcIsU7Vtf7/czx066x1uZIOiypdkEzA+VR+4a1NTs2Wlm5Vr3eWKqVO9OcjgQAAAAAl6wwr3jUVP4MhyslXSGpsjHmngtdco4xe4HxC11zZpbBxphEY0zi/v37LxwcKIeaXVFdHzzcXtX8fNTvreX6ZuM+pyMBAAAAwCUpzCsenSRts9but9ZmS/q3pGsl7fW+tiHv58k/KaVIqnfK9SHKfyUkxfv9zPHTrvG+RlJdUuqZQay1k621kdbayMDAwEI8ElB21a9dWfMebq8r61TWA9MT9dHK3U5HAgAAAICLVpiCYqekaGNMgHddiI6S1ktaIGmA95wBkuZ7vy+Q1Ne7M8eVyl8MM8H7GsgRY0y09z79z7jm5L16SfrKu04FgHMIquqn2bHRimxQU0/MSdabi34S/8kAAAAAKAs8Bb3QWrvcGDNP0g+SciStlDRZUhVJc40xg5RfYvT2nv+jMWaupHXe8x+11uZ6b/ewpGmS/CV95v2RpKmSZhpjtih/5kTfguYFKopqfj6aPjBKQ+au0oufbdCew8f1XNcwuV2sLwsAAACg9DLl7W9XIyMjbWJiotMxAMfl5VmN+XS9pi7eptuaX65X+0TIz8f92xcCAAAAQDExxiRZayPPdazAMygAlG4ul9FzXcMUXN1Poz9Zr4NHEzS5fxvVCKjkdDQAAAAAOEth1qAAUAY80OEqvX53KyXvOqRebyzV7kOZTkcCAAAAgLNQUAAVQLeWV2j6wCjtTT+uOycu0bqf052OBAAAAACnoaAAKoj2DWtr3kPXysjorjeXasmWAwW6z8iRI1WvXj1VqVLltPFvv/1WrVu3lsfj0bx584oiMgAAAIAKhIICqECuvryqPnz0WtWt4a/73knQ/OTdl3yPbt26KSEh4azx0NBQTZs2Tf369SuKqAAAAAAqGAoKoIIJru6vuQ+1V+vQmnp8drLeXPSTzrebT1xcnKKiohQREaHY2Fjl5uYqOjpawcHBZ53boEEDhYeHy+Xi/1YAAAAAXDr+JAFUQNX9fTRjUJRuDw/Wi59t0P99vE65eaeXFOvXr9ecOXO0ZMkSJScny+12a9asWQ4lBgAAAFDesc0oUEH5etx6vW8rBVfz05TF27Q3/bhe7RMhPx+3JCk+Pl5JSUlq27atJCkzM1NBQUFORgYAAABQjlFQABWYy2X0565hury6n8Z8ul4Hpi7XW/7bVeMvz8ru2KEB1arpxaFDpZgYp6MCAAAAKOd4xQOAHuhwlV6/u5VW7UjVnd8f1c5Dx9VR0rz0dO178EFp1iylpqZqx44dTkcFAAAAUE5RUACQJHUNv0JxX76qVL+quuPef+pEcBONlnRLZqbC779fnTt31p49ezRs2DCFhIQoIyNDISEhGjVqlCRpxYoVCgkJ0fvvv6/Y2Fg1a9bM0ecBAAAAULaY863eX1ZFRkbaxMREp2MAZZPLpa01gnV/71H6pUptvfafl9Vl01LJGCkvz+l0AAAAAMo4Y0yStTbyXMeYQQHgf0JDdVXaz/r3zKfVbN9WPdxzhKa07SkbGup0MgAAAADlHAUFgP8ZM0YKCFDtzHS9O3ukbtv4vUbf/ID+8sgrysllBgUAAACA4kNBAeB/YmKkyZOl+vXllyIYI3YAACAASURBVJut8cmzFVvnuGak+ip2ZpKOnchxOiEAAACAcoo1KAD8prhlO/T8/LW6Jria3r6vrS6r5ud0JAAAAABlEGtQACiUe6Lra+p9bbX9wDHdMWGJNvyS7nQkAAAAAOUMBQWAi3LT1UGa+1B75Vqr3pOWavHmA05HAgAAAFCOUFAAuGjNrqiujx69TnVr+uu+dxI0d8UupyMBAAAAKCcoKABckuDq/nr/ofa6tlEdDftgtV7+fKPK21o2AAAAAEoeBQWAS1bVz0dTB0Tq7qh6Gv/1Fj0xJ1nHs3OdjgUAAACgDPM4HQBA2eTjdumFO1ootFZljV24QbtSMzS5f6TqVPF1OhoAAACAMogZFAAKzBijh3/XUJNiWmvdnnT1nLBEG3854nQsAAAAAGUQBQWAQrutRbDej71WWTl5+sOk7/X1xn1ORwIAAABQxlBQACgSLUKqa/4fr1P92gEaNG2Fpi3ZxuKZAAAAAC4aBQWAInNyh49O11ymUR+v03Pz1yo7N++S7zNy5EjVq1dPVapUOW38lVdeUVhYmMLDw9WxY0ft2LGjqKIDAAAAcBgFBYAiFVDJozfuaaOHbmyouGU7NXDaCh3OzL6ke3Tr1k0JCQlnjbdq1UqJiYlavXq1evXqpWHDhhVVbAAAAAAOo6AAUORcLqPhtzXVS73CtWzrQd05cYl2HDx2znPj4uIUFRWliIgIxcbGKjc3V9HR0QoODj7r3JtuukkBAQGSpOjoaKWkpBTrcwAAAAAoORQUAIrNXZH1NHNQOx08lqWeE5YoYVvqacfXr1+vOXPmaMmSJUpOTpbb7dasWbMu6t5Tp07VbbfdVhyxAQAAADjA43QAAOVb9FW19dEj12ng9BWKmbJMLwYfVa+Xh0k7dyq+Rg0lWau2bdtKkjIzMxUUFPSb94yLi1NiYqIWLVpU3PEBAAAAlBAKCgDFrkGdyvrw4ev0yD8/0dMplbW1/o16esdM2bQ0DfB49OLQoVJMzEXd68svv9SYMWO0aNEi+fr6FnNyAAAAACWFVzwAlIjqAT6a9vYQ9Vv5mSa2v0uP9Byuaz0+mpeTo33Dh0uSUlNTL7gzx8qVKxUbG6sFCxZc1EwLAAAAAGUHBQWAEuOzY7vG/HeCnoufrP82jtbzMf/QEL+quiUlReHh4ercubP27NmjYcOGKSQkRBkZGQoJCdGoUaMkSUOHDtXRo0fVu3dvRUREqHv37s4+EAAAAIAiY6y1TmcoUpGRkTYxMdHpGADOpUEDyTtD4uurIvVY96HyzcnSm99PVZukr53NBgAAAKDYGWOSrLWR5zrGDAoAJWfMGMm7TehNWxP14cynVCX7uPre8pTmrtjlcDgAAAAATqKgAFByYmKkyZOl+vUlY9Soilvzrw1QdKNADftgtf768Trl5OY5nRIAAACAA3jFA4DjcnLz9MKnG/T2km3q0LiOxt/dWtUDfJyOBQAAAKCI8YoHgFLN43bp+W5heukP4Vq29aB6TFisLfuOFOheI0eOVL169VSlSpXTxt944w21aNFCERERuv7667Vu3bqiiA4AAACgiDCDAkCpkrg9VQ/FJelEdp7G3d1KNzW9tO1Ely1bpvr166tx48Y6evTor+Pp6emqVq2aJGnBggWaOHGiFi5cWKTZAQAAAFwYMygAlBmRDWpp/h+vV71aARo4fYUmf/uTzlekxsXFKSoqShEREYqNjVVubq6io6MVHBx81rknywlJOnbsmIwxxfYMAAAAAC4dBQWAUqduDX/Ne7i9ft88WC98ukFPzV2l49m5p52zfv16zZkzR0uWLFFycrLcbrdmzZp1wftOmDBBDRs21LBhwzRu3LjifAQAAAAAl4iCAkCpFFDJo/H9Wumpzk3075W71WfyMu1NP/7r8fj4eCUlJalt27aKiIhQfHy8tm7desF7Pvroo/rpp580duxYjR49urgfAQAAAMAlYA0KAKXewrW/aMjcZFXNy9bkT/+hlsmL9XqNGvr5+uv14oIF57ymSpUqp61Bcaq8vDzVrFlThw8fLs7YAAAAAM7AGhQAyrQuzS/Xv+unyefgAfW++Ul9EHaTOqalad5//qN9EydKklJTU7Vjx47z3mPz5s2/fv/kk0/UuHHjYs8NAAAA4OJRUAAoE5r+bYTmT39CbXav11Ndh2hux8EaJemWJ55QeHi4OnfurD179mjYsGEKCQlRRkaGQkJCNGrUKEnS+PHj1axZM0VEROiPf/yjUlJSztqK9KR58+bJGCNmYwEAAAAlh1c8AJQNLpdkrXKMSy/eNFBT2/ZU1M41mrhgrOocTbukW51vK1JJOnLkiG6//XZlZWVp/Pjxiow85+wzAAAAAAXAKx4Ayr7QUEmSx+bpua+m6F8fv6zVwY3VbeDrWrXr0Hkvu5StSCXpueee07Bhw+Tn51csjwEAAADg3CgoAJQNY8ZIAQG//tpz3TeaN+95uapXV+83l+r9xF1nXXKpW5GuXLlSu3btUteuXYvlEQAAAACcHwUFgLIhJkaaPFmqX18yRqpfX81feFYfD79VbRvU1NB5q/X8/LXKzs379ZJL2Yr02WefVfv27bVw4cLTxqdNm6bAwEBFREQoIiJCU6ZMKdbHBAAAACoq1qAAUObl5Obppc83avK3WxUVkKMJs/6swM0/XtJWpF9++aXuuusuHTp0SKGhofrll19Uq1YtPfjggzp48KDGjx9fko8EAAAAlEusQQGgXPO4XXr299fotXrHtPpwrrp1elorL2983q1IT65LkZmZ+eu6FJ06dVJqaqoCAgK0fft2RUdHa8GCBbryyisdfjoAAACgYqCgAFBu9PjHMP175tPy5OaoT7+xWtuis0Zbe9pWpEuXLtWoUaO0e/duWWs1a9Ys9erV67T7jBw5UsuWLdMNN9wgSfrggw8UHh6uXr16aeLEiQoLC1OzZs3Ur18/Jx4TAAAAKJcoKACUHzt3Kmz/Nn08/Um127VGw37/uNZ0flgJedLq1auVlJSkAwcOKCMjQ4GBgQoPD1fdunXVsmXL027TrVs3bdu2TTk5ORo5cqSOHDmi1atXKzw8XM8884xuvPFG+fj4aPXq1WrSpIlq1Kjh0AMDAAAA5UehCgpjTA1jzDxjzAZjzHpjTHtjTC1jzBfGmM3ez5qnnD/CGLPFGLPRGHPrKeNtjDFrvMfGGWOMd9zXGDPHO77cGNOgMHkBlHPerUhrHj+id94fpdhl8xTX+nb1G/Cy9k17V2rQQPZPf9KAY8eUPHSokpOT9dxzz+nTTz/9dRtSa+2v25C63W716tVLx44dk8vl0ujRo5WTk6NJkyZpwYIFqlq1qnbu3KmjR48qPDxcKSkpDv8LAAAAAMquws6geE3SQmttU0ktJa2XNFxSvLW2saR47+8yxoRJ6iupmaQukiYaY9ze+0ySNFhSY+9PF+/4IElp1tpGkl6VNLaQeQGUZ6dsReqxeRqxaJpeX/iqfqxTX7cn5Skht4o6SpqXnq59Dz6o9S+9pLi4OL377rt6+umn9dFHH522LoXb7Vbnzp3l6+srPz8/5ebmqmbNmrruuusUGRmp/fv3KyQkRG+88Yb27t2rESNGOPv8AAAAQBlW4ILCGFNN0g2SpkqStTbLWntIUg9J072nTZfU0/u9h6TZ1toT1tptkrZIijLGBEuqZq1davO3FJlxxjUn7zVPUseTsysA4Czn2Iq029D79eHCsapyIkN33/2Cvo/sob9JuiUzUx2feUZfff65OkVHa9CgQdq/f7+stZo8ebL8/f114sQJfffdd8rKytKJEyeUl5ensLAwffPNN+rUqZO2bdumw4cPq2nTprrqqqs0f/58p/8NAAAAAGVWYWZQXCVpv6R3jDErjTFTjDGVJV1mrd0jSd7PIO/5dSXtOuX6FO9YXe/3M8dPu8ZamyPpsKTaZwYxxgw2xiQaYxL3799fiEcCUObFxEjbt0t5efmfMTFqumaZ5k9/Up02L9fojg/qux7PaHElf42Q9JSkp9PTVc3XVwEBAapataoaN26sli1bKi8vT82aNVNAQID8/f3l8Xh0zz33yMfHR//85z8lSYcPH1a3bt1055136siRIzp48KCDDw8AAACUXYUpKDySWkuaZK1tJemYvK9znMe5Zj7YC4xf6JrTB6ydbK2NtNZGBgYGXjg1gIonNFTVsjL0xkcvaMTXb+uzJteqR/9X1Kj65ZonKT07W62OHJErN1eZmZnatWuX1qxZo7y8PG3duvXX23g8Hn399deSpClTpshaq7Zt2yowMFAvvvii6tatK4/H49BDAgAAAGVbYQqKFEkp1trl3t/nKb+w2Ot9bUPez32nnF/vlOtDJP3sHQ85x/hp1xhjPJKqS0otRGYAFZF3bQojKTbh35o1+8867FdFTw18XT3rNtUMSV9JsseP625JD3TooBo1asgYoylTpigrK0uS5Ha7Vbt2bYWFhWn06NGqUaOGFi9erE2bNqlatWrKy8tT9erVHXxQAAAAoOwqcEFhrf1F0i5jzNXeoY6S1klaIGmAd2yApJMvZS+Q1Ne7M8eVyl8MM8H7GsgRY0y0d32J/mdcc/JevSR95V2nAgAu3qlrU0hqv2uNPpn2uML2bdX797ysvh0H6wUZnZA0OydHi774QtMHDNC9996rnJwc5eTkKCMjQ1lZWXrllVe0bt06hYaGqk2bNrLWav369Tpw4IAGDRrk7HMCAAAAZZgpzJ/3jTERkqZIqiRpq6T7lV96zJUUKmmnpN7W2lTv+SMlDZSUI+kJa+1n3vFISdMk+Uv6TNKfrLXWGOMnaaakVsqfOdHXWvu/+dbnEBkZaRMTEwv8TADKuVmzpMGDpYwMZbvcevF39+vttj3VJmWdbpn3V71w4mj+tC9jVL9JE0VGRmr+/Pk6evSoJKlq1aoaMmSI7rrrLt1xxx3avn27JOnmm2/WRx99JF9fX8ceDQAAACjtjDFJ1trIcx4rbxMSKCgA/KZZs6QBA6TcXEnSx0076JnbHlNA9nGNW/CSrt25Jn8XkLw8h4MCAAAA5cuFCorCrEEBAGVTTIw0fXp+CSGp24bvtGDGk6p+/Kju6TNak9r9QTY01OGQAAAAQMVCQQGgYoqJkR566NdfGx1M0fwZQ3TbxiUa+7v7FTv4VaUfz3YwIAAAAFCxUFAAqLgmTpTi4qTatSVJVbIyNX7JVD0XnKmvjvqq++uLte7ndIdDAgAAABUDa1AAwDms2J6qP777g9IysjWqWzPdHVVPxvtKCAAAAICCYQ0KALhEbRvU0iePdVC7K2vp2Q/X6Ik5yTp2IsfpWAAAAEC5RUEBAOdRp4qvpt8fpadvaaKPV/2sbuMXa8MvvPIBAAAAFAcKCgC4AJfL6I83N9asB6J15HiOeoxfojkrdqq8vR4HAAAAOI2CAgAuQvuGtfXpYx0U2aCmnvlgjZ6au0oZWbzyAQAAABQVCgoAuEiBVX01Y2A7PdmpiT5M3q3u45do094jTscCAAAAygUKCgC4BG6X0eOdGmvWoHY6lJGt7uMX6/3EXU7HAgAAAMo8CgoAKIBrG9XRp49fr1b1amrovNV6+v1VyszKdToWAAAAUGZRUABAAQVV9VPcA+30WMfG+uCHFPWYsFhb9vHKBwAAAFAQFBQAUAhul9GQzk00Y2CUDh7NUrfXl+iDpBSnYwEAAABlDgUFABSBDo0D9enjHdQipLqeen+VhsxJ1tET7PIBAAAAXCwKCgAoIpdV89N7D0briU6N9VHybnUd953WpBx2OhYAAABQJlBQAEARcruMnujURO89GK0TOXm6c9ISTfluq6y1TkcDAAAASjUKCgAoBu2uqq1PH+ug310dpNGfrNfAaSt08OgJp2MBAAAApRYFBQAUk5qVK2nyvW301x7NtOSng7rtte/0/ZYDTscCAAAASiUKCgAoRsYY9W/fQB89cp2q+nkUM3W5/vH5BmXn5jkdDQAAAChVKCgAoASEXVFNH//pet3Vpp4mfP2T+ry5VLtSM5yOBQAAAJQaFBQAUEICKnk0tle4xt3dSpv3HtXvx32nT9fscToWAAAAUCpQUABACeve8gp98lgHXRVYRY/M+kEj/r1GmVm5TscCAAAAHEVBAQAOCK0doHkPtddDNzbUewk71X38Yq3fk+50LAAAAMAxFBQA4BAft0vDb2uqmYOilJaRrR7jl2jKd1uVl2edjgYAAACUOAoKAHBYh8aB+vyJDrqhSaBGf7Je/d9O0N70407HAgAAAEoUBQUAlAK1q/jqrf5tNOaO5krckapb//WtFq5lAU0AAABUHBQUAFBKGGMU066+Pnmsg+rVDNBDcT9o2LxVOnYix+loAAAAQLGjoACAUqZhYBV98PC1euR3DfV+UopuH/edkncdcjoWAAAAUKwoKACgFKrkcWlYl6aa/WC0snOt/jDpe42L36yc3DynowEAAADFgoICAEqxdlfV1qePd1DX8GC98sUm9Z28TLtSM5yOBQAAABQ5CgoAKOWq+/votb6t9K8+Edr4yxHd9tp3+iApRdayHSkAAADKDwoKACgjeraqq08f76BrgqvqqfdX6U/vrdThjGynYwEAAABFgoICAMqQerUCNHtwew299WotXPuLurz2rb7bvN/pWAAAAEChUVAAQBnjdhk9elMj/fuRaxVQya17pybo+flrlZHFdqQAAAAouygoAKCMCg+poU8e66CB112pGUt36PevfaekHWlOxwIAAAAKhIICAMowPx+3nu8Wpve825H2fuN7vbRwg7Jy2I4UAAAAZQsFBQCUA+0b1tbCJzqoV5sQTfzmJ/WYsETr96Q7HQsAAAC4aBQUAFBOVPXz0Uu9WmpK/0jtP3JC3ccv1qRvflJuHtuRAgAAoPSjoACAcqZT2GX675M3qHPYZRq7cIPuenOpth845nQsAAAA4IIoKACgHKpVuZIm9Gut1/pGaPPeI7rtte80c+l2WctsCgAAAJROFBQAUE4ZY9Qjoq4+f/IGRTaoqefm/6j+bydoz+FMp6MBAAAAZ6GgAIByLri6v2YMjNLfejZX4vY03frqt/ogKYXZFAAAAChVKCgAoAIwxuje6Pr67PEOanJZVT31/ioNmp6oXw4fdzoaAAAAIImCAgAqlAZ1KmtObHs93zVM3/90QJ1fXaS5ibuYTQEAAADHUVAAQAXjdhkNvP5KLXz8Bl0TXE3D5q3Wfe+s0M+HWJsCAAAAzqGgAIAKqkGdypr9YLT+r3szJWxL1S2vfqvZCTuZTQEAAABHUFAAQAXmchkNuLaBPn/iBjWvW03D/71G/d9O0G5mUwAAAKCEUVAAABRaO0DvPhCtv/Vsrh92pOmWVxZp1vIdzKYAAABAiaGgAABIyp9NcW90fS184gZFhNbQyA/XKmbKcu1KzXA6GgAAACoACgoAwGnq1QpQ3KB2euGOFlqdcli3/utbzVy6XXl5zKYAAABA8aGgAACcxRijfu1C9fmTN6hN/Zp6bv6PuvutZdp24JjT0QAAAFBOUVAAAM6rbg1/zRgYpbF/aKF1e9J167++1YSvtyg7N8/paAAAAChnCl1QGGPcxpiVxpj/eH+vZYz5whiz2ftZ85RzRxhjthhjNhpjbj1lvI0xZo332DhjjPGO+xpj5njHlxtjGhQ2LwDg0hhj1KdtqOKH3KiOTYP0j883qtvri7Vq1yGnowEAAKAcKYoZFI9LWn/K78MlxVtrG0uK9/4uY0yYpL6SmknqImmiMcbtvWaSpMGSGnt/unjHB0lKs9Y2kvSqpLFFkBcAUABB1fw06Z42evPeNkrLyNIdE5do9H/WKSMrx+loAAAAKAcKVVAYY0Ik3S5pyinDPSRN936fLqnnKeOzrbUnrLXbJG2RFGWMCZZUzVq71ObvZzfjjGtO3muepI4nZ1cAAJxxa7PL9cWQG3V3VKimLN6mW179Vos27Xc6FgAAAMq4ws6g+JekYZJOfRn5MmvtHknyfgZ5x+tK2nXKeSnesbre72eOn3aNtTZH0mFJtQuZGQBQSNX8fDTmjhaaG9telTwuDXg7QUPmJCv1WJbT0QAAAFBGFbigMMZ0lbTPWpt0sZecY8xeYPxC15yZZbAxJtEYk7h/P3+LBwAlJerKWvr0sQ76082NtGDVz+r0yiLNT96t/AlxAAAAwMUrzAyK6yR1N8Zs///27jxM6urO9/jn1Nb7vq900+wgmw0iIIsbalyTODGJo9HJNcmYicl4M9dM7s1k7txnbmaSjGYzM97EqNmMM5pI3BUUFFlsQGigWZqt933fu6vO/aPKtkVAUeDX1fV+PU899avzq1/x7YdD0fWps0h6XNKlxpjfSGoMTdtQ6L4p9PwaSQVjrs+XVBdqzz9J+3uuMcZ4JCVJajuxEGvtQ9baUmttaUZGxsf4kQAAZyra69a9V07XM19brsLUWN3z+Nu645G3VNPe53RpAAAACCMfOaCw1n7LWptvrS1ScPHL9dbaWyWtlXR76Gm3S3o6dLxW0i2hnTmKFVwMc1toGki3MWZJaH2J20645p3X+nToz+BrOQAYh2ZkJ+rJryzVP1w3S9uOtunK+zfq4TeOyh/gbRsAAAAf7Gzs4nGi70m6whhzSNIVocey1u6V9ISkfZJekHS3tdYfuuYrCi60WSnpsKTnQ+2/lJRmjKmU9LcK7QgCABif3C6jO5YV66VvrNDi4lT972f26aYHN2lPbafTpQEAAGCcMxNtQEJpaaktKytzugwAiHjWWq3dVad/eqZCbb2Duu3iIt175TQlRHudLg0AAAAOMcZst9aWnuzcuRhBAQCAjDG6YX6e1t27Up+/aJIe3XxMl/1wg57dXc8imgAAAHgfAgoAwDmVFOPVP904R3/862XKSIjS3b/boS/86i1VtbKIJgAAAN5FQAEAOC/mFyTr6buX6TvXzlLZsTZdcf8G/ezVSg2NBJwuDQAAAOMAAQUA4LzxuF26c3mx1t27SpfNzNT3Xzyga378urYcaXW6NAAAADiMgAIAcN5lJ0Xrwc9fqF99YZEGR/y65aEtuveJXWrtGXS6NAAAADiEgAIA4JjVMzL10tdX6u7VJVq7q1aX/nCDHt9WpUCARTQBAAAiDQEFAMBRMT63vrlmhp772iWanp2g+54q183/sVl76zqdLg0AAADnEQEFAGBcmJqVoD/ctUTf//RcHWvp1XU/eUPfeXqPOvqGnC4NAAAA5wEBBQBg3DDG6ObSAq2/d5Vuu7hIv9lynGkfAAAAEYKAAgAw7iTFevXd62frmb+5RFMy4nXfU+W66cFNeru6w+nSAAAAcI4QUAAAxq1ZuYn6w5eW6IHPzFdd54BuenCT7ntyN7t9AAAATEAEFACAcc0YoxsX5Gn9vSv1xeXF+q/tNVr9g9f02OZjGvEHnC4PAAAAZwkBBQAgLCREe/XtT8zS8/dcogvyk/Sdp/fqup9uUtmxNqdLAwAAwFlAQAEACCtTsxL0m7+6SA9+fqE6+4b06X/frL/9w9tq6hpwujQAAAB8DAQUAICwY4zRNRfk6JV7V+ru1SV6Zne9Lv3hBj208bCGRpj2AQAAEI4IKAAAYSvW59E318zQi99YoUVFKfrn5/bryvs36KW9DbKWbUkBAADCCQEFACDsFafH6Vd3LNYjdyySx+3SXb/erlt/uVX7G7qcLg0AAAAfEgEFAGDCWDU9U8/fc4m+e90s7ant0jU/el3f/mM525ICAACEAQIKAMCE4nW79IVlxdrwzVW67eIiPf5WtVb94DX94vUjrE8BAAAwjhFQAAAmpORYn757/Wy9+PVLtLAwRf/n2QqteWCjXtnXyPoUAAAA4xABBQBgQpuSmaBH71ysX92xSC4jffGxMv3lL7fpQEO306UBAABgDAIKAEBEWD09Uy98fYX+4bpZKq/t1NU/2qj/+adytfUOOV0aAAAAREABAIggXrdLdywr1mv/fZX+cskk/X5btVZ+/1X9+4bDGhj2O10eAABARCOgAABEnJQ4n/7xhjl64Z5LVDopRd97fr8u++EG/XFnjQIB1qcAAABwAgEFACBiTc1K0K/uWKzfffEipcR59Y0/7NJ1P31DmypbnC4NAAAg4hBQAAAi3tIp6Vp793I98Jn56ugb1ud/sVVf+BULaQIAAJxPBBQAAEhyuYxuXJCndfeu1N9fM0M7jrfr6h9t1N/91y41dA44XR4AAMCEZybaXvClpaW2rKzM6TIAAGGuvXdIP321Uo9tPia3y+iLyyfrSysnKyHa63RpAAAAYcsYs91aW3rScwQUAACcWnVbn/71xQP68646pcX59PXLp+qWxYXyuhmECAAAcKZOF1Dw2xUAAKdRkBqrn3x2gZ6+e5mmZMbrfz29V2vu36jny+s10UJ+AAAAJxFQAADwIcwrSNbjdy3RL28vlctl9JXf7tANP9ukNw6x4wcAAMDZQEABAMCHZIzRZTOz9MI9l+hfPz1XrT1DuvWXW/W5/7dFO6vanS4PAAAgrLEGBQAAH9HgiF+/3VKln71aqdbeIV0xK0v//crpmp6d4HRpAAAA4xKLZAIAcA71Do7o4TeO6qGNR9QzNKKb5ufpG1dMU0FqrNOlAQAAjCsEFAAAnAftvUP69w2H9cibxxSwVp9dXKivXjpFmQnRTpcGAAAwLhBQAABwHjV0DujH6w/pD29Vy+d26Y5lRfrSihIlxXqdLg0AAMBRBBQAADjgWEuv/u3lg1q7q06J0R59aWWJvrC0SHFRHqdLAwAAcAQBBQAADtpX16UfvHRA6/c3KTXOpy+vnKxbl0xSrI+gAgAARBYCCgAAxoHtx9v1wCsH9fqhFqXH+/TllSX6/EWTFONzO10aAADAeUFAAQDAOFJ2rE33v3JQmypblR4fpa+sKtHnLypUtJegAgAATGwEFAAAjEPbjrbp/pcPavORVmUmBIOKzy4mqAAAABMXAQUAAOPYSDLcPwAAHupJREFUliOtuv/lg9p6tE1ZiVG6e/UUfWZRgaI8BBUAAGBiIaAAACAMvHm4RQ+8fEjbjrUpJylaf716iv6iNJ+gAgAATBgEFAAAhAlrrTZVtur+Vw5q+/F25YaCik9fmM/UDwAAEPYIKAAACDPWWr1R2aL7Xz6oHVUdykyI0l0rJutzFxWyPSkAAAhbBBQAAIQpa63ePNyqn66v1OYjrUqN8+nOZUW6bWmREqO9TpcHAABwRggoAACYALYfb9NP11fq1QPNSoj26PaLi3Tn8mKlxvmcLg0AAOBDIaAAAGAC2VPbqQdfq9TzexoU7XHrcxcV6q4Vk5WVGO10aQAAAKdFQAEAwARU2dStB189rKd31cltjG4uzdeXV5aoIDXW6dIAAABOioACAIAJrKq1Tz/fcFhPbq+R31rdOD9Pf726RCUZ8U6XBgAA8B4EFAAARICGzgE9tPGIfrftuAZHAlozK1t3rZyshYUpTpcGAAAgiYACAICI0tIzqF9tOqpfbz6uroERLS5K1V0rJuvSGZlyuYzT5QEAgAh2uoDC9TFetMAY86oxpsIYs9cYc0+oPdUY87Ix5lDoPmXMNd8yxlQaYw4YY9aMab/QGFMeOvdjY4wJtUcZY/4Qat9qjCn6qPUCABAp0uOj9M01M7T5W5fpO9fOUm1Hv774WJmufGCjnnirWoMjfqdLBAAAeJ+PHFBIGpF0r7V2pqQlku42xsySdJ+kddbaqZLWhR4rdO4WSbMlXSXpQWOMO/RaP5d0l6SpodtVofa/ktRurZ0i6X5J//Ix6gUAIKLERXl05/JivfbNVXrgM/Pldbv0d0/u1iX/8qp+/tphdfYPO10iAADAqI8cUFhr6621O0LH3ZIqJOVJukHSo6GnPSrpxtDxDZIet9YOWmuPSqqUtNgYkyMp0Vq72Qbnmzx2wjXvvNZ/SbrsndEVAADgw/G6XbpxQZ6e+9pyPXbnYk3LStC/vLBfy763Xv/8XIXqO/udLhEAAECes/EioakXCyRtlZRlra2XgiGGMSYz9LQ8SVvGXFYTahsOHZ/Y/s411aHXGjHGdEpKk9RyNuoGACCSGGO0YlqGVkzL0J7aTv3HxiP6xetH9PAbR3XD/DzdtWKypmcnOF0mAACIUB9niockyRgTL+lJSV+31nad7qknabOnaT/dNSfWcJcxpswYU9bc3PxBJQMAEPHm5CXpJ59doA3fXK1bl0zSc+X1WvPARt3+8DZtPNisibaINgAAGP8+VkBhjPEqGE781lr7VKi5MTRtQ6H7plB7jaSCMZfnS6oLteefpP091xhjPJKSJLWdWIe19iFrbam1tjQjI+Pj/EgAAESUgtRYfff62Xrzvkv1t1dM0966Lt328DZdef9G/X5blQaGWVATAACcHx9nFw8j6ZeSKqy1/zbm1FpJt4eOb5f09Jj2W0I7cxQruBjmttB0kG5jzJLQa952wjXvvNanJa23fKUDAMBZlxLn09cum6pN963WD26eJ6/bpW89Va6L/+86/eDFA2rsGnC6RAAAMMGZj/p53xizXNLrksolBULNf6/gOhRPSCqUVCXpZmttW+iab0u6U8EdQL5urX0+1F4q6RFJMZKel/Q31lprjImW9GsF17dok3SLtfbI6eoqLS21ZWVlH+lnAgAAQdZabT3apoffOKqXKxrlcRldOzdXdy4r1gX5SU6XBwAAwpQxZru1tvSk5ybagAQCCgAAzq7jrb165M1jeuKtavUO+bW4KFV3Li/SFbOy5XaxuRYAAPjwCCgAAMDH1jUwrCfeqtYjbx5TTXu/8pJjdOuSSfrMogKlxvmcLg8AAIQBAgoAAHDW+ANWL+9r1GObj+nNw63yeVy6bm6ubl86SXPzk50uDwAAjGMEFAAA4Jw42NitX28+rid31KhvyK/5Bcm67eJJ+sTcHEV53E6XBwAAxhkCCgAAcE51DQzrqe01emzLcR1p7lVanE+fWVSgzy+ZpLzkGKfLAwAA4wQBBQAAOC+stdpU2apHNx/TuopGSdIVs7J065JJWlaSLheLagIAENFOF1B4zncxAABg4jLGaPnUdC2fmq6a9j79dmuVHt9WpRf3NmpSWqxuWVSom0vzlR4f5XSpAABgnGEEBQAAOKcGhv16cW+Dfru1StuOtsnrNlozO1ufu6hQF09OkzGMqgAAIFIwxQMAAIwLlU3d+t3Waj25o0ad/cOanB6nzy4u1KcuzGerUgAAIgABBQAAGFcGhv16rrxev9tapbLj7fK5Xbr6gmx9bnGhFhenMqoCAIAJioACAACMWwcauvX7bVV6ckeNugdGNDk9TjeXFuhTC/OUmRjtdHkAAOAsIqAAAADjXv+QX8+W1+uJt6q17Vib3C6j1dMzdHNpgS6dkSmv2+V0iQAA4GMioAAAAGHlSHOP/nN7jZ7cXqOm7kGlx/v0yYX5+ovSfE3JTHC6PAAA8BERUAAAgLA04g9ow8FmPVFWrXUVTRoJWC0sTNZnFhXoE3NzFR/FjukAgMgUCFi5XOG3ZhMBBQAACHvN3YP6085a/aGsWpVNPYrxunX1nGx9cmG+Li5JkzsMf0kDAOBM+ANWmw+36skdNTrU1K0/f3V52C0sfbqAgq8dAABAWMhIiNJ/WzFZX7ykWDurO/SfZTV6dnedntpZq6zEKN24IE+fWpivaVlMAQEATCyVTT16ckeN/rSzVvWdA0qI9ui6ebkaGA4oxud2uryzhhEUAAAgbA0M+7V+f5Oe2lGj1w40ayRgNScvUZ9ckK/r5+cqPT7K6RIBAPhImrsH9czuOv3p7Trtqu6Q22W0Ymq6PnVhvi6fmaVob3gGE0zxAAAAE15Lz6D+vKtOT+2oVXltp9wuo5XTMvTJhXlh/YscACBy9A6O6KV9DfrTzjq9Udkif8BqVk6iblqQpxsW5CozIfy33yagAAAAEeVQY7ee2lmrP+6oVUPXgBKiPFozJ1vXz8vV0pI0ediyFAAwTgz7A3rjUIv+9HatXtrbqP5hv/KSY3TD/FzduCBvwk1dJKAAAAARyR+w2nKkVX/cWasX9zSoe3BE6fE+XXNBjm6Yn6uFhSlht7gYACD8WWu1s7pDT++s1TO769XaO6SkGK8+MTdHNy3I04WFKWG5Q8eHQUABAAAi3sCwX68daNbaXbVaV9GkwZGA8pJjdN28XF0/L1czcxIIKwAA54y1Vvvqu/TM7no9u7teVW198nlcumJmlm6Yn6tV0zPl80z8EX4EFAAAAGN0Dwzr5X2NWrurTq8fCs7xnZIZr+vn5eq6ebkqTo9zukQAwARxoKFbz+6u0zO763WkpVdul9HSkjRdNy9XV83JVmK01+kSzysCCgAAgFNo6x3Sc+X1Wvt2nbYda5MkzcxJ1CcuyNY1F+Rocka8wxUCAMLN4eYePbOrXs/srtOhph65jLRkcpqunZurNbOzlBbBu0wRUAAAAHwIdR39eq68Xs+V12tHVYckaUZ2gq65IEfXXJCjKZmEFQCAkzva0qvnyuv151112t/QLWOkRZNSde28HF01J3tC7MBxNhBQAAAAnKH6zn49X96g5/fUq+x4u6yVpmXFj4YVE21VdQDAmbHWan9Dt17Y06AX9zZof0O3JGlhYbKunZuray7IUXYSocSJCCgAAAA+hsauAb2wp0HPltfrrWNtslaakhmvq+dk68pZ2ZqTl8gCmwAQAQIBq7drOvTinga9sLdBx1v7giMlilJ11exsrZmTrbzkGKfLHNcIKAAAAM6Spu4BvRgKK7YdbVPASrlJ0bpiVpaunJ2txcWp8ron/irsABApRvwBbTvaphf2BkdKNHYNyus2WlqSrqvmZOvymVnKSIjcNSXOFAEFAADAOdDWO6T1+5v00t4GbTzUrIHhgBKjPbp0RqaunJ2tldMyFBflcbpMAMAZ6hkc0esHm/VKRZPW729Ue9+wor0urZqWqavmZGv1jEwlxUTW7htnCwEFAADAOdY/5Nfrh5r10r5GrasI/jLr87i0fEq6rpyVpUtnZrJAGgCMY7Ud/VpX0ahXKpq05XCrhvwBJcV4demMTK0Jhc4xPrfTZYa90wUURPoAAABnQYzPrStnZ+vK2dka8QdUdrxdL+1t1Ev7GrR+f5MkaW5+klZPz9SlMzJ1QV6SXC7WrQAApwQCVuW1nVpX0aiXK5pUUd8lSSpOj9PtSyfp8plZunBSijxM2ztvGEEBAABwDllrVVHfrfX7G7V+f5N2VnfIWik9Pkqrp2fo0hmZWj41XQnRDBUGgHOtZ3BEmypb9NqBJq2raFJT96BcRiqdlKrLZmbq8llZKslgS+lziSkeAAAA40Rrz6A2HGzW+v1N2nCwWd0DI/K6jRYXp46OrihOj2NXEAA4C6y1OtDYrQ0HmvXagWaVHW/TsN8qPsqjFdPSdfnMLK2enqmUOJ/TpUYMAgoAAIBxaNgf0Pbj7Xp1f5PW72/SoaYeSVJBaoxWTM3QimkZWlqSxugKADgD3QPDoVESzdpwsFn1nQOSpBnZCVo5PUOrpmXqwkkp8nmYuuEEAgoAAIAwUN3Wp1cPNGnjwWa9ebhVfUN+uV1GCwuTRwOLOXlJcrN2BQCMCgSs9tV36fVDwakb24+3ayRglRDl0bIp6Vo1PUMrp2coJynG6VIhAgoAAICwMzQSHF3x+qFmbTzUrD21wcXbUmK9WjYlXSumZWjF1AxlJ7EzCIDIU9XapzcqW7SpskVvHm5Re9+wJGlmTqJWTc/QqmkZWjgpRV4WuBx3CCgAAADCXEvPoDZVtmjDwWa9fqhFzd2DkqSSjDgtLUnX0pI0LZmcxjxqABNSe++Q3jzcOhpKVLX1SZKyE6O1fGq6lk8Jvg9mJhLajncEFAAAABOItVb7G7r1+qHgVJBtR9vUN+SXMdLM7EQtLUnT0ilpWlSUyvoVAMJS98Cwyo63a8uRVm2qbNHeui5ZKyVEebSkJE3Lp6Rr2ZR0lWSwqHC4IaAAAACYwIb9Ae2u6dCbla1683Crtle1a2gkILfLaF5+kpaWpOvikjQtKExWrM/jdLkA8D5dA8MqO9amrUfatOVIq8prOxWwktdttKAwZTSQmJefJA/TNsIaAQUAAEAEGRj2a0dVuzYfDgYWb1d3yB+w8riM5uQlaXFxqhYXpaq0KEXJsUwJAXD+dfYHA4ktR1q19Wib9oQCCZ/bpfkFybpocqqWTE7TwsIUxfjcTpeLs4iAAgAAIIL1DI6o7Fib3jrWpm1H27SrulND/oAkaXpWghYVp2hxcZoWF6Wy6CaAc6Kuo19lx9u1/Vibyo63q6K+691AojBZSyanacnkVC0sTFG0l0BiIiOgAAAAwKiBYb92VXforWNt2nq0TTuOt6t3yC9JKkiN0aKi4IeEBYXJmp6VwHBqAGdkxB/Q/oZubT/ePhpK1HUOSJJifW7NL0jWoqLgCIkFhckEEhGGgAIAAACnNOIPaF99l7YdDY6w2H68Xa29Q5KCHybm5Sdr4aTkUGiRolR2CgEwRkffkHbVdGr78XZtP96mt6s6RkPPnKRoXTgpRaWTUlRalKoZ2YSekY6AAgAAAB+atVZVbX3aWdWhHVXt2lHVror6bvkDwd8bi9Jig2HFpBQtKEjWtKwE+Tx84AAiwcCwX/vqu7SrukNvV3doV3WHjrUGt/x0GWlGdqJKi1KCoURRqvKSYxyuGOMNAQUAAAA+lr6hEZXXdGpHKLTYWdWulp7gKAufx6WZOYmam5ekC/KTNDc/SVMy4vmWFAhzgYDVkZbe0SBiV02HKuq7NOwPfobMTozWvIIkzStI1vyCZF2Ql8TWxvhABBQAAAA4q6y1qm7r166aDpXXdmp3TYf21HapZ3BEkhTjdWt2buJoYHFBXrImp8fJ5TIOVw7gZIb9AVU29WhPbaf21nVpb12n9tV1jU7ViI/yaG5+MIyYlx8MJFhUFx8FAQUAAADOuUDA6mhrr3bXdGh3TafKazq1p65TA8PBHUNifW5Nz07QrJxEzcxJ1KzcRM3ITlCsz+Nw5UBkGRj260BDt/bUdWpPbZf21XWqoqFbQyPv/ludlZOo2bmJmp2XpAUFyZqcES83ASPOAgIKAAAAOGLEH9Dh5l7tqunQvrouVdR3aV99l7oHgiMtjJGK0+I0MydRM3MSNCs3GF5kJ0bLGD4MAR9HIGBV096v/Q1dOtDQrf2N3TrQ0K2jLb2ja8okxXg1OzdRc/KSgoFEbpKK0+MII3DOnC6gIK4GAADAOeNxuzQ9O0HTsxNG26y1qu3oDwUW3aqo71J5baeeLa8ffU5itEdTsxI0LSteUzMTNC10nJEQRXABnERb79BoEHGgoVv7G7p1qLF7dIqGFNxGeHpWoq6ekz0aRuSnxPBvCuMGIygAAAAwLnQPDGt/QzCwONjYrYMNPTrY1K2OvuHR5yTFeIOhRVaCpmXGa1pWgiZnxCsrkeACE18gEAz3Kpt7dLipR4ebe3S4qVeVzT1qC20NLEkpsV5Nz07QjOzE0YBwWlaC4qP4fhrOY4oHAAAAwpK1Vi09QzrU2B0MLZp6Qsc96ux/N7iI9blVnB6n4vQ4TU6PU3FGnCanx6soPU5JMewqgPDSNTCs4y19OtbaGwwhmntV2dSjoy09o2u6SMEgYkpmvEoy4jUlFNjNyE5gpBHGNaZ4AAAAICwZY5SREKWMhCgtnZI+2m6tVXP3oA419ehIS6+ONPfoaEuvyms79Vx5vQJjvoNLj/epOD1ORWlxmpQWq4LU4K0wNVZpcT4+yOG8s9aqrXdIx1r7VNXWq2MtfTre2ht63Pee0RDGSPkpMSrJiNeykjSVZMaPhhKpcT4Hfwrg7COgAAAAQNgxxigzMVqZidFaNia4kKShkYCq2np1pLlXR1uCtyMtvXrtYLOauwff89wYr1uFqbEqSI0ZDS0KUmKVnxqjnKQYJUZ7CDBwxqy1au8bVm17v2o7+lTT3q+a9n7VdvSrtr1f1W196g5tySsFQ4jcpBgVpcdqzexsFaXFalIoUCtKi1OMz+3gTwOcPwQUAAAAmFB8HpemZCZoSmbC+871D/lV096n6vY+VbX2qaqtX9Xtfapu69Obh1vVN2ZBQUmK87mVkxyjnKRo5SbFKCf53fucpBjlJkezTWqEsdaqa2BETV0DaugaUGPXoBq7BlTX8W4AUdvRf9K+lJcSo7zkGC0qStGktDgVpceqMDVOBakxivIQQgBh8W5qjLlK0o8kuSX9wlr7PYdLAgAAQBiK8bk1NStBU7PeH168M+y+qi34jXd9Z7/qOgZU39mv+s4BVdR3q6Vn8H3XJUZ7lJEQpcyE6NHpKJmj9++2pcR6GY0xjo34A2rvG1Zr76Bae4bU0jOolp4hNXYNqLFrQA2dA6HjQfUP+993fUqsV3kpMZqcEadLpmaMhhH5KcFbUgx//8AHGfcBhTHGLelnkq6QVCPpLWPMWmvtPmcrAwAAwERijFFafJTS4qO0oDDlpM8ZHPGrsXNQdZ3vBhhNXQNq7hlUU9egdtV0qOkUH2C9bqOUWJ9S43yj98mx3vc8TonzKSXWq5RYnxJjvEqI8sjl4kPtmRr2B9TVP6yO/mF1vnPrC953vBNC9A6ptScYRrT2Dqm9b0gn2z/A53EpKzFK2YnRmpOXpMtmRis7MVpZSdHKSohSdlK0MhOimYYBnAXjPqCQtFhSpbX2iCQZYx6XdIMkAgoAAACcV1EetwrTYlWYFnvK51hr1TvkDwYX3YOj4UVT96Dae4fU1jek9t4h7W/oUnvf8Ck/GL8jPsqjhGiPEqO9Soj2hG7e99zH+tyK9bkV7XUr1udRjNetGJ9LMV6PYsaci/G65XWbcfVN/og/oMGRd25+DQ4HjweG/eob8qt3cES9QyPqHQwe9wyOqG9oRD2hx8HjEXX2j6izb0id/cPqHXp/QDRWUoxXafE+pcdFqSQjXouLfUqLj1J6vE9pcVFKi/cpLc6n9PgoJTPyBThvwiGgyJNUPeZxjaSLHKoFAAAAOC1jjOKjPIrPiNfkjPgPfL4/YNXVHwwq2vuG1NYbPO7qH1b3wIi6BoL33aH7lp4hHW3pDbWNaMgf+MA/40Ret5HX7Rq9+dxGXk/w2OMy8nlccruMXMbIKLiIo5GRjOQKHRszpl3SSCCgQCB477eSPxCQPyAFAjZ4zgbPDY/YYBARCiX8gdOkM6cQ43UrLsqtuCiP4nwexUW5lZccrZk5CUqO8SkpxqukGI+SY4PHiTFeJcd6Q+1eed2uM/4zAZx74RBQnCyufM+7mDHmLkl3SVJhYeH5qAkAAAA4K9wuE5za8RG3jBwY9qt/yK/+0IiDd0Ye9A/71T80MtreHzo35Lca8Qc07A9o2G815A9oeOSEx/5gcGCtFLDBe6t3HkvWBkJtwREjkuRxBUMNnyc4LcUTCjg8LiP3mJvP7VKU16Uoj0tRHnfw3vv+Y5/bpdgot+KjPIr1eYL3UW7F+TxyM+0FmJDCIaCokVQw5nG+pLqxT7DWPiTpIUkqLS098wgWAAAACFPR3uD0jZOvmgEA4SMcxja9JWmqMabYGOOTdIuktQ7XBAAAAAAAzqJxP4LCWjtijPmqpBcV3Gb0YWvtXofLAgAAAAAAZ9G4DygkyVr7nKTnnK4DAAAAAACcG+EwxQMAAAAAAExwBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxBBQAAAAAAMBxxlrrdA1nlTGmWdJxp+sIY+mSWpwuAuMSfQOnQ//AqdA3cDr0D5wKfQOnQ/8Ib5OstRknOzHhAgp8PMaYMmttqdN1YPyhb+B06B84FfoGTof+gVOhb+B06B8TF1M8AAAAAACA4wgoAAAAAACA4wgocKKHnC4A4xZ9A6dD/8Cp0DdwOvQPnAp9A6dD/5igWIMCAAAAAAA4jhEUAAAAAADAcQQUEcoY831jzH5jzG5jzB+NMcljzn3LGFNpjDlgjFkzpv1CY0x56NyPjTHGmepxrhljbjbG7DXGBIwxpSeco39glDHmqlBfqDTG3Od0PTj/jDEPG2OajDF7xrSlGmNeNsYcCt2njDl30vcQTDzGmAJjzKvGmIrQ/yn3hNrpH5AxJtoYs80YsyvUP/4x1E7/gCTJGOM2xuw0xjwTekzfiAAEFJHrZUlzrLVzJR2U9C1JMsbMknSLpNmSrpL0oDHGHbrm55LukjQ1dLvqfBeN82aPpE9K2ji2kf6BsUJ/9z+TdLWkWZI+G+ojiCyP6P3/3u+TtM5aO1XSutDjD3oPwcQzIulea+1MSUsk3R3qA/QPSNKgpEuttfMkzZd0lTFmiegfeNc9kirGPKZvRAACighlrX3JWjsSerhFUn7o+AZJj1trB621RyVVSlpsjMmRlGit3WyDC5c8JunG8144zgtrbYW19sBJTtE/MNZiSZXW2iPW2iFJjyvYRxBBrLUbJbWd0HyDpEdDx4/q3feDk76HnJdCcd5Za+uttTtCx90KftDIE/0DkmxQT+ihN3Szon9AkjEmX9InJP1iTDN9IwIQUECS7pT0fOg4T1L1mHM1oba80PGJ7Ygs9A+Mdar+AGRZa+ul4IdUSZmhdvpMhDLGFElaIGmr6B8ICQ3hf1tSk6SXrbX0D7zjAUl/Jykwpo2+EQE8TheAc8cY84qk7JOc+ra19unQc76t4BDM375z2Umeb0/TjjD1YfrHyS47SRv9I3Lx944zRZ+JQMaYeElPSvq6tbbrNEsU0T8ijLXWL2l+aC20Pxpj5pzm6fSPCGGMuVZSk7V2uzFm1Ye55CRt9I0wRUAxgVlrLz/deWPM7ZKulXSZfXe/2RpJBWOeli+pLtSef5J2hKkP6h+nQP/AWKfqD0CjMSbHWlsfmgLWFGqnz0QYY4xXwXDit9bap0LN9A+8h7W2wxjzmoLrB9A/sEzS9caYayRFS0o0xvxG9I2IwBSPCGWMuUrS/5B0vbW2b8yptZJuMcZEGWOKFVzscFtoGFW3MWZJaHeG2ySd6lt2TFz0D4z1lqSpxphiY4xPwQWq1jpcE8aHtZJuDx3frnffD076HuJAfTgPQv8f/FJShbX238acon9AxpiM0MgJGWNiJF0uab/oHxHPWvsta22+tbZIwd8t1ltrbxV9IyIwgiJy/VRSlKSXQ0Mtt1hrv2yt3WuMeULSPgWnftwdGn4nSV9RcLX2GAXXrHj+fa+KCcEYc5Okn0jKkPSsMeZta+0a+gfGstaOGGO+KulFSW5JD1tr9zpcFs4zY8zvJa2SlG6MqZH0D5K+J+kJY8xfSaqSdLMkfcB7CCaeZZL+UlJ5aJ0BSfp70T8QlCPp0dBuCy5JT1hrnzHGbBb9AyfHe0cEMO+O7AcAAAAAAHAGUzwAAAAAAIDjCCgAAAAAAIDjCCgAAAAAAIDjCCgAAAAAAIDjCCgAAAAAAIDjCCgAAAAAAIDjCCgAAAAAAIDjCCgAAAAAAIDj/j/96LE+P5NoqAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1296x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Setting up the figure\n",
    "fig, ax1 = plt.subplots(figsize=(18, 8))\n",
    "\n",
    "### Creating the plot of the MSE Cost Function\n",
    "x_range = np.arange(-225, 450, 2)\n",
    "true_value = np.mean(Y)\n",
    "y_range = []\n",
    "\n",
    "for i in x_range:\n",
    "    y_range.append(squared_error(i, true_value))\n",
    "\n",
    "### Plotting the values\n",
    "\n",
    "ax1.plot(x_range, y_range)\n",
    "ax1.scatter(x, y, color=\"r\")\n",
    "for i, txt in enumerate(label):\n",
    "    ax1.annotate(txt, (x[i], y[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simple Linear Regression Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "## Fitting the linear regression model on X and Y\n",
    "linear_regressor = LinearRegression()  # create object for the class\n",
    "params = linear_regressor.fit(X.reshape(-1,1), Y)  # perform linear regression\n",
    "\n",
    "## Predictions using the trained regression model\n",
    "def get_lin_output(model, value):\n",
    "    return model.predict(value.reshape(-1, 1))[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Example 0 : \t Regression =  1238.8244116427359 \t Neural Network = 846.8688196288764\n",
      "Example 1 : \t Regression =  2885.5924012422174 \t Neural Network = 112.88550432709715\n",
      "Example 2 : \t Regression =  3545.591844787564 \t Neural Network = 22.889176677241423\n",
      "Example 3 : \t Regression =  6894.339640064001 \t Neural Network = 666.0935329072182\n",
      "Example 4 : \t Regression =  21.893729095736152 \t Neural Network = 3494.461360084089\n",
      "Example 5 : \t Regression =  3.747461597917166 \t Neural Network = 3842.7700801330443\n",
      "Example 6 : \t Regression =  25.47221963988411 \t Neural Network = 3489.919946349301\n",
      "Example 7 : \t Regression =  3741.951330800977 \t Neural Network = 10.000747077362368\n",
      "Example 8 : \t Regression =  686.5000504178113 \t Neural Network = 1450.4264737670605\n",
      "Example 9 : \t Regression =  3613.7884737815607 \t Neural Network = 17.880365465079276\n",
      "Example 10 : \t Regression =  3793.380276885948 \t Neural Network = 7.60575978153474\n",
      "Example 11 : \t Regression =  3714.6660016744063 \t Neural Network = 11.47628200140497\n",
      "Example 12 : \t Regression =  3644.094649532878 \t Neural Network = 15.7463169139024\n",
      "Example 13 : \t Regression =  4128.608361289174 \t Neural Network = 0.009120486871740977\n",
      "Example 14 : \t Regression =  4055.6166850578916 \t Neural Network = 0.4424674532412586\n",
      "Example 15 : \t Regression =  3493.404442893695 \t Neural Network = 26.948108831077395\n",
      "Example 16 : \t Regression =  3251.1942477523335 \t Neural Network = 53.288973995781305\n",
      "Example 17 : \t Regression =  3727.940200078796 \t Neural Network = 10.672711341842401\n",
      "Example 18 : \t Regression =  2101925.2326650782 \t Neural Network = 2175970.291297479\n",
      "Example 19 : \t Regression =  4115.660298546328 \t Neural Network = 0.038439904741282574\n",
      "Example 20 : \t Regression =  10828.641079841636 \t Neural Network = 27404.5321129813\n",
      "Example 21 : \t Regression =  1827.161472912936 \t Neural Network = 9822.879754082898\n",
      "Example 22 : \t Regression =  3895.1870450453557 \t Neural Network = 3.7108681991003194\n",
      "Example 23 : \t Regression =  102.68362306525354 \t Neural Network = 5524.928298787945\n",
      "Example 24 : \t Regression =  2352.5570465502456 \t Neural Network = 249.5163177900614\n",
      "Example 25 : \t Regression =  3715.021353112552 \t Neural Network = 11.531359417463673\n",
      "Example 26 : \t Regression =  2802.545530189657 \t Neural Network = 129.712570851292\n",
      "Example 27 : \t Regression =  3020.1234463091955 \t Neural Network = 12365.03456855659\n",
      "Example 28 : \t Regression =  6604.550794170734 \t Neural Network = 20290.803644582786\n",
      "Example 29 : \t Regression =  11528.210965960721 \t Neural Network = 29041.83369396937\n",
      "Example 30 : \t Regression =  3154.5043458850614 \t Neural Network = 66.80376509729429\n",
      "Example 31 : \t Regression =  683365.6397493997 \t Neural Network = 792204.9469131744\n",
      "Example 32 : \t Regression =  328107.69616421073 \t Neural Network = 372630.0100160191\n",
      "Example 33 : \t Regression =  4093.9606640040943 \t Neural Network = 0.09741889504760654\n",
      "Example 34 : \t Regression =  3314.8873537300856 \t Neural Network = 14840.275471022165\n",
      "Example 35 : \t Regression =  2368.9062170622483 \t Neural Network = 229.28062705911756\n",
      "Example 36 : \t Regression =  3035.6624110423604 \t Neural Network = 84.4728417324929\n",
      "Example 37 : \t Regression =  4060.8410646006673 \t Neural Network = 0.38845023564314707\n",
      "Example 38 : \t Regression =  4106.592635983941 \t Neural Network = 0.07116711330661876\n",
      "Example 39 : \t Regression =  4119.057400134103 \t Neural Network = 0.028828398543659175\n",
      "Example 40 : \t Regression =  3188.609444162011 \t Neural Network = 61.78897240872447\n",
      "Example 41 : \t Regression =  12382.896374824943 \t Neural Network = 29514.161924339416\n",
      "Example 42 : \t Regression =  3306.5230113371263 \t Neural Network = 46.364352790717014\n",
      "Example 43 : \t Regression =  460.7146937667907 \t Neural Network = 7218.79879602225\n",
      "Example 44 : \t Regression =  469.834868805908 \t Neural Network = 1692.6296449329768\n",
      "Example 45 : \t Regression =  44575.94970378718 \t Neural Network = 75449.43710584684\n",
      "Example 46 : \t Regression =  3065.371955433734 \t Neural Network = 14293.144759407563\n",
      "Example 47 : \t Regression =  3972.6105034697666 \t Neural Network = 1.7233079993033371\n",
      "Example 48 : \t Regression =  2311.6041391540593 \t Neural Network = 235.65746185141646\n",
      "Example 49 : \t Regression =  2699.9331916239885 \t Neural Network = 152.12192620192283\n",
      "Example 50 : \t Regression =  1565.1303415627763 \t Neural Network = 611.1961127042189\n",
      "Example 51 : \t Regression =  3993.94667013811 \t Neural Network = 1.3175714880983367\n",
      "Example 52 : \t Regression =  4038.927087992698 \t Neural Network = 0.6338110338981574\n",
      "Example 53 : \t Regression =  3882.770650972045 \t Neural Network = 4.145596677391035\n",
      "Example 54 : \t Regression =  4115.130690669262 \t Neural Network = 0.0399222895378907\n",
      "Example 55 : \t Regression =  4663.046504611246 \t Neural Network = 47.32357112897218\n",
      "Example 56 : \t Regression =  2373.8727054360734 \t Neural Network = 242.78139727696694\n",
      "Example 57 : \t Regression =  2937.137191561105 \t Neural Network = 59.326216549593255\n",
      "Example 58 : \t Regression =  3984.9452261043925 \t Neural Network = 1.4631562883476714\n",
      "Example 59 : \t Regression =  3294.294216628107 \t Neural Network = 48.01181042888805\n",
      "Example 60 : \t Regression =  3925.415200169566 \t Neural Network = 2.8733565564476127\n",
      "Example 61 : \t Regression =  998.9336252093835 \t Neural Network = 1067.917699327574\n",
      "\n",
      " ========================================== \n",
      "\n",
      "Linear Regression Success Count:  20 \t Percentage:  32.25806451612903%\n",
      "Neural Network Success Count:  42 \t Percentage:  67.74193548387096%\n"
     ]
    }
   ],
   "source": [
    "n_count = 0 # Samples for which Neural Net produced better result\n",
    "l_count = 0 # Samples for which Linear Regrssion produced better result\n",
    "\n",
    "for index in range(len(X)):\n",
    "    err1 = squared_error(get_lin_output(linear_regressor, X[index]), Y[index])\n",
    "    err2 = squared_error(get_nn_output(weights, X[index]), Y[index])\n",
    "    print(\"Example \"+str(index)+\" : \\t Regression = \", err1, \"\\t Neural Network =\", err2)\n",
    "    if err1 < err2:\n",
    "        l_count += 1\n",
    "    else:\n",
    "        n_count += 1\n",
    "print(\"\\n ========================================== \\n\")\n",
    "print(\"Linear Regression Success Count: \", l_count, \"\\t Percentage: \", str((l_count*100)/(len(Y)))+\"%\")\n",
    "print(\"Neural Network Success Count: \", n_count, \"\\t Percentage: \", str((n_count*100)/(len(Y)))+\"%\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}