Skip to content

Instantly share code, notes, and snippets.

@CalvinTChi
Created July 6, 2016 05:52
Show Gist options
  • Save CalvinTChi/f2550dc10c86a3637b59fec9890d6f65 to your computer and use it in GitHub Desktop.
Save CalvinTChi/f2550dc10c86a3637b59fec9890d6f65 to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Neural Networks\n",
"### Author: _Calvin Chi_"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Introduction\n",
"Neural network is a powerful supervised machine learning algorithm suited for both regression and classification problems. A unique feature about this algorithm is that it allows for the implicit learning of features without having the user explicitly design them. The neural network is a constructed by several neurons with the following architecture: \n",
"\n",
"<img src=\"http://i.imgur.com/yZfR8RH.png\", width=300, height=300>\n",
"\n",
"In this representation, $x_{1},...,x_{3}$ are the input values for features $1$ to $3$, and $+1$ is the bias term to increase the model space. The neuron then accepts these input to output $h_{W, b}(x) = f(W^{T}x) = f(\\sum_{i=1}^{3}W_{i}x_{i} + W_{4}b)$, where $W \\in \\mathbb{R}^{4 x 1}$ is a vector of weights for each of the input values and $b$ represents the bias of $+1$. $f()$ is an activation function, typically chosen to be the signmoid function or tanh function. \n",
"\n",
"A neural network is built from these individual neurons. An example of a neural network with $3$ layers looks like this:\n",
"\n",
"<img src=\"http://i.imgur.com/W8jxjyy.png\", width=400, height=150>\n",
"\n",
"With this architure, our weight vector becomes a matrix instead. To go from layer $L_{1}$ to layer $L_{2}$, we would need a $W^{(1)} \\in \\mathbb{R}^{4 x 3}$ matrix, where $4$ is the number of input values $n_{in}$ and $3$ is the number of output values $n_{out}$. The neurons in the second layer are denoted as $a^{(2)}_{j}$, where $j$ denotes the $j\\text{th}$ neuron. According to above diagram, $a^{(2)}_{j}$ is calculated as $f((\\sum_{i=1}^{3} W^{(1)}_{ij}x_{i}) + W^{(1)}_{4j} b^{(1)}_{j})$. To go from layer $2$ to layer $3$, one would use a $W^{(2)} \\in \\mathbb{R}^{4 x 1}$ vector, and the value of $h_{W, b}(x)$ is computed as $f((\\sum_{i=1}^{3} W^{(2)}_{i1}a^{(2)}_{i}) + W^{(2)}_{41} b^{(2)}_{1})$. The logic of these computational steps can be extrapolated to neural networks in general with a number of layers and a given number of neurons per layer. Note that a neural network with more than $3$ layers is often called a deep neural network. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Backpropagation\n",
"Training a neural network involves iterately finding the values of the weight matrices via gradient descent to minimize error, which could be set as the mean square error or cross entropy error. Either way, the cost function is non-convex such that the result from training may represent a local solution instead of a global solution. One way to combat this is to randomly intialize the weight values prior to training. \n",
"\n",
"In this tutorial's implementation, we are going to implement neural network with stochastic gradient descent. In stochastic gradient descent, each sample will first be used to compute all $a$ using the current weight values in a procedure called feedforward propagation. Then in backpropagation, these computed $a$ are used to compute the gradients for each individual weight value, and used to update the weight value. \n",
"\n",
"Let us illustrate the procedure for deriving the gradients for each layer in matrix notation. Assume that we have $3$ layers with $2$ weight matrices. The middle layer will use the tanh activation function whereas the third layer will use the sigmoid function. We shall also derive the gradients separately for mean-square error and cross entropy error respectively.\n",
"\n",
"In our notation, we denote $n_{out}$ to refer to the last layer and $\\alpha$ as our learning rate. The sigmoid function is defiend as:\n",
"\n",
"$$g(z) = \\frac{1}{1+e^{-z}}$$\n",
"\n",
"Where as the tanh activation function is defined as:\n",
"\n",
"$$f(z) = tanh(z)$$\n",
"\n",
"In our derivation, we will make use of two mathematical properties:\n",
"\n",
"1. $\\frac{dg(z)}{dz} = \\frac{e^{-z}}{(1 + e^{-z})^{2}} = \\frac{1 + e^{-z}}{(1 + e^{-z})^{2}} - \\frac{1}{(1 + e^{-z})^{2}} = \\frac{1}{1 + e^{-z}} - \\frac{1}{(1 + e^{-z})^{2}} = g(z)(1-g(z))\\\\$\n",
"\n",
"2. $\\frac{df(z)}{dz} = 1 - tanh^{2}(z)$\n",
"\n",
"## Mean Square Error\n",
"According to our neural network design, mean square error is defined as: \n",
"\n",
"$$J = -\\sum_{j=1}^{n_{out}} (y_{j} - g_{j}(a^{(3)}))^{2}$$\n",
"\n",
"Where $y_{j}$ in our case represents the $j$th class we are trying to predict. Then our gradient for $W^{(2)}$ would be:\n",
"\n",
"$$\\frac{\\partial J}{\\partial W^{(2)}} = \\frac{\\partial J}{\\partial g(a^{(3)})} \\: \\frac{\\partial g(a^{(3)})}{\\partial a^{(3)}} \\: \\frac{\\partial a^{(3)}}{\\partial W^{(2)}} = -a^{(2)}(y - g(a^{(3)}))^{T} diag[g(a^{(3)})(1 - g(a^{(3)}))]$$\n",
"\n",
"The gradient for $W^{(1)}$ would be:\n",
"\n",
"$$\\frac{\\partial J}{\\partial W^{(1)}} = \\frac{\\partial J}{\\partial g(a^{(3)})} \\: \\frac{\\partial g(a^{(3)})}{\\partial a^{(3)}} \\: \\frac{\\partial a^{(3)}}{\\partial f(a^{(2)})} \\: \\frac{\\partial f(a^{(2)})}{\\partial a^{(2)}} \\: \\frac{\\partial a^{(2)}}{\\partial W^{(1)}} = -x[ diag(1-tanh^{2}(a^{(2)}))(W^{(2)})^{T}diag[g(a^{(3)})(1 - g(a^{(3)}))](y - g(a^{(3)})) ]^{T}$$ \n",
"\n",
"Taking care of dimensions in matrix multiplication. The update equations then would be:\n",
"\n",
"$$W^{(2)} = W^{(2)} - \\alpha \\frac{\\partial J}{\\partial W^{(2)}}\\\\$$\n",
"\n",
"$$W^{(1)} = W^{(1)} - \\alpha \\frac{\\partial J}{\\partial W^{(1)}}$$\n",
"\n",
"## Cross Entropy Error\n",
"According to our neural network design, cross entropy error is defined as: \n",
"\n",
"$$J = -\\sum_{j=1}^{n_{out}} [y_{j} ln(g_{j}(a^{(3)}) + (1-y_{j})ln(1-g_{j}(a^{(3)}))]$$\n",
"\n",
"The gradient for $W^{(2)}$ would be:\n",
"\n",
"$$ \\frac{\\partial J}{\\partial W^{(2)}} = \\frac{\\partial J}{\\partial g(a^{(3)})} \\: \\frac{\\partial g(a^{(3)})}{\\partial a^{(3)}} \\: \\frac{\\partial a^{(3)}}{\\partial W^{(2)}} = a^{(2)}(\\frac{1-y}{1-g(a^{(3)})} - \\frac{y}{g(a^{(3)})})^{T} diag[g(a^{(3)})(1 - g(a^{(3)})]$$\n",
"\n",
"And the gradient for $W^{(1)}$ would be:\n",
"\n",
"$$ \\frac{\\partial J}{\\partial W^{(1)}} = \\frac{\\partial J}{\\partial g(a^{(3)})} \\: \\frac{\\partial g(a^{(3)})}{\\partial a^{(3)}} \\: \\frac{\\partial a^{(3)}}{\\partial f(a^{(2)})} \\: \\frac{\\partial f(a^{(2)})}{\\partial a^{(2)}} \\: \\frac{\\partial a^{(2)}}{\\partial W^{(1)}} = x [ diag(1 - tanh^{2}(a^{(2)})) (W^{(2)})^{T} diag[g(a^{(3)})(1 - g(a^{(3)}))] (\\frac{1-y}{1 + e^{-a^{(3)}}} - y \\frac{e^{-a^{(3)}}}{1 + e^{-a^{(3)}}}) ]^{T}$$\n",
"\n",
"## Implementation\n",
"Our implementation shall have the following features:\n",
"\n",
"+ Learning rate of $\\alpha = \\frac{1}{100 + 10k}$, where $k$ = number of epochs.\n",
"+ Weight initialization: each weight value is drawn randomly from a $N(0, \\sigma=0.01)$ distribution. \n",
"+ The neural network could optionally accept pre-trained weights as the starting point for training. \n",
"\n",
"The dataset we will be applying our implementation on will be the MNIST digits dataset. Thus, there will be 10 classes for each digit from $0$ to $9$. To train our classifier in recognizing these digits, we will have an input layer of $784$ neurons, a hidden layer of $200$ neurons, and an output layer of $10$ neurons. The number of neurons for the hidden layer can be user-defined, but the number of neurons for the input and output layers are determined by the problem we are working on. In the MNIST dataset, each sample is a $24 x 24$ pixel image, so that translates to $784$ feature values. There are a total of $10$ classes, leading to an output layer with $10$ neurons. The neural network is trained to output a probability for each class, and in our implementation we shall pick the class with the highest probability as our prediction. The usage of our `NeuralNetwork` class will be as follows:\n",
"\n",
"```\n",
"classifier = NeuralNetwork(layers, loss=\"mean_square\", monitor=False, plot=False, weights=[], iterations=[], errors=[], accuracies=[])\n",
"```\n",
"\n",
"+ layers: a list of integers, each representing number of neurons for the layer it corresponds to by index. The integer at index $0$ indicates number of neurons for the input layer and the integer at index $len(layers)-1$ indicates the number of neurons for the output layer. \n",
"+ loss: string of either \"mean_square\" for mean square loss or \"cross_entropy\" for cross entropy loss.\n",
"+ monitor: boolean indicating whether to print to screen the iteration number, current loss based on entire training dataset, accuracy in predicting on the training set, and accuracy in predicting on the validation set if appropriate.\n",
"+ plot: boolean indicating whether to plot a error vs iterations and accuracy vs iterations on the training set at the conclusion of the training process. \n",
"+ weights, iterations, errors, accuracies: list of weights, list containing iteration counts, list of errors, and list of accuracies from prior training.\n",
"\n",
"The usage for training the neural network is as follows:\n",
"\n",
"```\n",
"classifier.train(x, Y, epochs, validation=None, validation_label=None)\n",
"```\n",
"\n",
"+ x: training data in a numpy array \n",
"+ Y: training labels in a numpy array\n",
"+ epochs: number of complete passes through the training data\n",
"+ validation: validation data in a numpy array\n",
"+ validation_label: validation label in a numpy array\n",
"\n",
"Now we implement our `NeuralNetwork` class:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import numpy as np\n",
"import sys\n",
"import pickle\n",
"import matplotlib.pyplot as plt\n",
"import collections \n",
" \n",
"class NeuralNetwork: \n",
" \n",
" def __init__(self, layers, loss=\"mean_square\", monitor=False, \n",
" plot=False, weights=[], iterations=[], errors=[], accuracies=[]): \n",
"\n",
" def tanh(x): \n",
" return np.tanh(x)\n",
"\n",
" def tanh_deriv(x): \n",
" return 1.0 - (x)**2\n",
"\n",
" def logistic(x):\n",
" return 1 / (1 + np.exp(-x))\n",
" \n",
" def logistic_deriv(x): \n",
" return x * (1 - x)\n",
" \n",
" def mean_square(x, y): \n",
" total = 0\n",
" for predict, label in zip(x, y):\n",
" total += 0.5*np.sum((label - predict)**2)\n",
" return total\n",
" \n",
" def mean_square_deriv(x, y):\n",
" return -(y - x)\n",
" \n",
" def cross_entropy(x, y):\n",
" total = 0\n",
" for predict, label in zip(x, y):\n",
" total += -(np.sum(label*np.log(predict) + (1-label)*np.log(1-predict)))\n",
" return total\n",
"\n",
" def cross_entropy_deriv(x, y):\n",
" return ((1-y)/(1-x)-(y/x))\n",
"\n",
" self.output_activation = logistic\n",
" self.output_activation_deriv = logistic_deriv\n",
" self.hidden_activation = tanh\n",
" self.hidden_activation_deriv = tanh_deriv\n",
" self.layers = layers\n",
" self.weights = weights\n",
" self.iterations = iterations\n",
" self.errors = errors\n",
" self.accuracies = accuracies\n",
" self.monitor = monitor\n",
" self.plot = plot\n",
" self.lossName = loss\n",
" if len(self.weights) == 0:\n",
" for i in range(1, len(layers)-1):\n",
" self.weights.append(np.random.normal(0, 0.01, (layers[i-1]+1, layers[i]+1)))\n",
" self.weights.append(np.random.normal(0, 0.01, (layers[i]+1, layers[i+1])))\n",
" if loss == \"mean_square\": \n",
" self.loss = mean_square\n",
" self.loss_deriv = mean_square_deriv\n",
" elif loss == \"cross_entropy\": \n",
" self.loss = cross_entropy\n",
" self.loss_deriv = cross_entropy_deriv\n",
"\n",
" def train(self, x, Y, epochs, validation=None, validation_label=None):\n",
" def accuracy(x, y):\n",
" correct = 0\n",
" for a, b in zip(x, y):\n",
" if np.argmax(a) == np.argmax(b): \n",
" correct += 1\n",
" return correct/len(x)\n",
" if len(self.iterations) != 0 and len(self.errors) != 0 and len(self.accuracies) != 0: \n",
" count = self.iterations[-1] + 1\n",
" else:\n",
" count = 0\n",
" bias = np.ones((x.shape[0], 1))\n",
" x = np.hstack((x, bias))\n",
" for k in range(epochs):\n",
" for i in range(len(x)):\n",
" a = [x[[i], :]]\n",
" y = Y[i]\n",
" gradients = collections.deque()\n",
" # Feedforward propagation step\n",
" for j in range(len(self.weights)):\n",
" if j == len(self.weights) - 1: \n",
" a.append(self.output_activation((np.dot(a[j], self.weights[j]))))\n",
" else: \n",
" a.append(self.hidden_activation((np.dot(a[j], self.weights[j]))))\n",
" # Backpropagation step\n",
" deltas = collections.deque()\n",
" deltas.append(np.dot((self.loss_deriv(a[-1], y)), \n",
" np.diag(self.output_activation_deriv(a[-1])[0])))\n",
" for j in range(len(self.weights) - 1, 0, -1):\n",
" delta = np.dot(np.dot(np.diag(self.hidden_activation_deriv(a[j])[0]), \n",
" self.weights[j]), deltas[-1].T).T\n",
" deltas.appendleft(delta)\n",
" # Final gradient calculation and weight update\n",
" for j in range(len(deltas)-1, -1, -1):\n",
" gradient = np.dot(a[j].T, deltas[j])\n",
" gradients.appendleft(gradient)\n",
" for j in range(len(self.weights)): \n",
" self.weights[j] -= 1/float(100+10*k) * gradients[j]\n",
" if self.monitor and count % 20000 == 0 or count in [1000, 5000, 10000, 15000]: \n",
" self.iterations.append(count)\n",
" predictions1 = self.predict(x)\n",
" correct1 = accuracy(predictions1, Y)\n",
" self.accuracies.append(correct1)\n",
" error = self.loss(predictions1, Y)\n",
" self.errors.append(error)\n",
" if validation is not None and validation_label is not None:\n",
" predictions2 = self.predict(validation)\n",
" correct2 = accuracy(predictions2, validation_label)\n",
" print(\"# \" + str(count), \"error: \", error, \"|\", \"training: \", \n",
" correct1, \"|\", \"validation: \", correct2)\n",
" else:\n",
" print(\"# \" + str(count), \"error: \", error, \"|\", \"training: \",\n",
" correct1)\n",
" if count % 54000 == 0: \n",
" pickle.dump(self.weights, open(\"result/weights.p\", 'wb'))\n",
" pickle.dump(self.iterations, open(\"result/iterations.p\", 'wb'))\n",
" pickle.dump(self.accuracies, open(\"result/accuracies.p\", 'wb'))\n",
" pickle.dump(self.errors, open(\"result/errors.p\", 'wb'))\n",
" count += 1\n",
" pickle.dump(self.weights, open(\"result/weights.p\", 'wb'))\n",
" pickle.dump(self.iterations, open(\"result/iterations.p\", 'wb'))\n",
" pickle.dump(self.accuracies, open(\"result/accuracies.p\", 'wb'))\n",
" pickle.dump(self.errors, open(\"result/errors.p\", 'wb'))\n",
" if self.plot:\n",
" plt.plot(self.iterations, self.errors)\n",
" plt.xlabel(\"Iterations\")\n",
" plt.ylabel(\"Error\")\n",
" plt.title(\"Training Error vs Iterations (\" + self.lossName + \")\")\n",
" plt.show()\n",
" plt.plot(self.iterations, self.accuracies)\n",
" plt.xlabel(\"Iterations\")\n",
" plt.ylabel(\"Accuracy\")\n",
" plt.title(\"Training Accuracy vs Iterations (\" + self.lossName + \")\")\n",
" plt.show()\n",
"\n",
" def predict(self, x):\n",
" predictions = []\n",
" if x.shape[1] != self.layers[0]+1: \n",
" bias = np.ones((x.shape[0], 1))\n",
" x = np.hstack((x, bias))\n",
" for i in range(len(x)): \n",
" for l in range(0, len(self.weights)):\n",
" if l == len(self.weights)-1: \n",
" a = self.output_activation(np.dot(a, self.weights[l]))\n",
" else: \n",
" a = self.hidden_activation(np.dot(x[[i], :], self.weights[l]))\n",
" predictions.append(a)\n",
" return np.array(predictions)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this tutorial, we are going to train a neural network with mean square error and cross entropy as the loss function respectively, and compare the effectiveness of each loss function at the end. Now let us load our MNIST dataset, visualize it,and pre-process it:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD8CAYAAABXXhlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvbuPZVu25vWbjzXXYz8iM87JqOqku7o8PIR1HQyuBEIY\nSO21AAeEjd9gta4HuHigFgIJBMJogYUajGqpLfgD+tIY3Ka6z+2MuhWP/ViP+RoYc+2IHXHiPKoy\nzzmZJ9cnDc25dkbGXrlzf2uMOZ5KRFiwYMHnBf1T38CCBQt+fCzEX7DgM8RC/AULPkMsxF+w4DPE\nQvwFCz5DLMRfsOAzxHsRXyn1byul/lwp9U+UUn/nQ93UggULflioPzaOr5TSwD8B/g3gK+D/Av5d\nEfnzZz+3JAosWPATQUTUS6/b9/idfwL8PyLyTwGUUv8j8LeAP//6j/7ds/1vgD99j7f9ofEblvt7\nH/yGj/f+fsPHe2/w4e/vz77xT97H1P+XgN+eXf+z+bUFCxZ85FicewsWfIZ4H1P/nwO/Orv+6/Nr\nL+A3Z/vmPd7yx8Cvf+ob+A78+qe+ge/Ar3/qG/gW/PqnvoHvwK/f8+//xSzfjfdx7hng/6Y49/4S\n+D+Bf09E/vGzn5OnZ/wFCxb8OPizD+/cE5GklPqPgX9AOTL8veekX7BgwceJ9zH1EZH/DfiXP9C9\nLFiw4EfC4txbsOAzxEL8BQs+QyzEX7DgM8RC/AULPkMsxF+w4DPEQvwFCz5DLMRfsOAzxEL8BQs+\nQyzEX7DgM8RC/AULPkMsxF+w4DPEQvwFCz5DLMRfsOAzxEL8BQs+QyzEX7DgM8RC/AULPkMsxF+w\n4DPEQvwFCz5DLMRfsOAzxHv13Fuw4GUoUOpxfbIXtMpoJSidv7ZHBAQkFyGDZHnYA6BBaVBane2Z\n1ZgiZ00WjUjZi2jyvEcU5Q3k5fUzwUL8BR8YCrR5FGWeXGuTqWzAWU9lA5XN81pESyJ7IXtIoayn\n6+wFpUBXCu1Au6+vGUOIFSE6fKgI0RCim1+ryElBziAJ8izn+8+E/AvxF3xYqJn4pnpRdJVw9UDj\nFG2daWtPU0faeqStB3QOxB5SL6ReiMO8IqQIKLC1wrRgOoXt1NkKCccwtYyTZpgqhskyTjVMLWlq\nyUFBCpBjWU8Cs5mxEH/Bgj8c58S39VOpanQdqVpN22XWnWfTwrqLrLuRdXfAJE/YCXEnhJ0QdpmA\nEKIQRwEFlVNUncJuFdVWU20V1bZcR1Vz6BWHvuLQC7Y30NfEvmMyG/Aa4jSLn88IzOeK+JN+dD8m\nFuIv+MB4RnzXQvUougtU60y78WzWPRcb4dU68mozcrE+YuOIv834RvA24xF8zEyj4HVGKXBO41YK\nt9W4S4W71NTz6lXkfl9RH1rMHtgboquZzAqttuXewgDaPiV9juWh9Xko/IX4Cz4wlCrn+hPxqxbc\nCuoVuBV65XFbT3vRs74wvHolfHER+PJi5ItXB6rQMzaZyWZGhClmxjEzHTKjLhq/cYqm09RbTXOp\nqa80zZWivtJMOlPftZi7CC0kZ5lMTa9WqHwBYl4mffLAi9OmfpZYiL/gw+Lc1K9m4tcraDZQb9Cr\niWo70L7es740vLqELy8jV5cTv7g84sKBwWYGEkPMDGNmOGRGlxl0BgWt07Sdpr3QtJea9srQvtU0\nbzWTBrtaQxtJTpiM4UhNlTu030K2oJ+RPvrysFIL8Rcs+CPxgqlfr6DeQHuBWo24zZ72Vc3mC8ur\nN8IXbyK/eDPy9s2B2u/oyfQxcRwz/SHR32V6lzmqhNLQOUPXaVZbQ3ep6a40q7eG7leaQWtoR2Id\nmQwclWGXaqqwQo9biFW5zXym6c04PwwW4i9Y8C14Fps/X22NNhXKGLTWKA1aZZSOaOVZ41mpaZax\nrHpkpUc6PVLrEXRCdEZ0BpUQlZF5BejQdBhaNJ0ydErTzquaf89Kjax0eZ/1SfCIEjIBUZFMQshk\nMoKQP5cDPgvxF/zBUE/j889WbSuqqsZqRYWnykdsjFRTT8UdGzvwRfMv2Fbv6PTvqeQO0pE4jYx9\nJIXM+FXGX2fCTSbuhNwL4ikJPIoS4++L59/fZEwzH9uByQTS70fU7w+4mztWv2/Y3lVMe0XuE6vB\nEaee4HtC6AmxJ+SBmD2B/JAj9HPHQvwFfxjOnXe6+tqqjaGqFLVRNCrQ5EQTexoUTVasVc/r6q/Y\nqr+ild9TxXuUPxD7kfEQiSExXmem64y/KSG91AvZC+Ti3BMvD8QPjaCtoMhIhKAj8X5E3x1wd3d0\ndxUXdwrZJ0w/sR5rRj8xes8YPEP0jMkzSiDK50L7hfgL/mCcMvPmM7ypn6xKg9WRRgfWBFY5sIpx\nXgNrObLWd6zlli7eUk33qOFA3I+M9wEdM/5mJv2s8dO3aXxbyCpRkUeIOpD2I2p/wO0rur0i7xNm\n76mPPYex5ugzh5A4hoyJGVIm5RI6TD/pZ/vj4b2Ir5T6C+Ce8l8SRORPPsRNLfiIca7xzey1t+3D\nqnWmoqclspLANh/Z5p4LObLlyCoeaGRPE/c00x43HOBwJHYjQxvROZXEnXt5SOR51Pg8aPzcQ7Dl\nTC4xk0dFOghZR2I/ovoD1VGx6hOm99R9z7rfcZhqdlHjosZGBVGTksZnjZLPx8H3vho/A38qIrcf\n4mYWfAp4pvHtHKev5ji9ilQp0uSeVQ5s05HLfMvrdMdlvqXze0wcsNOAGQbsoUfVA6keGesAORNP\n6bq9kI6F5F/X+JQXoppJrwh3ClQgTQNqUrgpY0ZPPfWspz1pajmGmjo5THKQHDFV+OzoxaFxgPlJ\nP90fC+9LfMVS2vt54SWNX83hOrdBKU8VepqgWOXART5yGW/5MlzzJryjZYf4gAwesQGxHqk80QaC\njWRJXyvMyb5o+XONnwCJkEfQByGcCnUU6KhQIePihAo9Olao6NDBcUwNRlrIHSl3+NwxSEuVQYll\nIf73gwD/u1IqAf+ViPzXH+CeFnzU+AaN7zbQXKAZqbijybCOxdR/HW5549/xy+m3NOmeoDNeybxm\ngs6EeU0ic0munJXlPpbonjS+RCGPgJaHEl2lwSjB5YwTT5U1tWhc1rh57aUB2ZLYMMmWnshewIpF\n0/y0H+2PiPcl/r8mIn+plHpDeQD8YxH5R1//sd+c7X89y4KPFy/V0Z/F6W2FtgZtdHkG6IzWEa09\nG5nYMrKRoTjy8oFV2rOK93ThjjrtmCviSeU3PuzDvM53cH43D/Lww+n5aVzmKLygyRigmsUBDVAD\nQkOjFDUWpxqsShgtaAxKufJT5zX6wrOa/afv+XHhL2b5brwX8UXkL+f1d0qpvw/8CfAC8f/0fd5m\nwY+JU8rtN4gxJU7vKoWzHqePOCIu91Thjk3uuQxfcRmv2aYb2rzDSk8Wj59TZCbAU2rhIoXH520w\nTi429Q17zl7j2XpO+NMZNFMeKgCjUkza4HVFMDVJtyS9RvQGMRdAAynNmX356T6fTA+e3fHH0sTj\n1zxVqv/wG3/yjya+UqoDtIgclFIr4N8C/uyP/X0LPhY8q6e3T1dtDLVRtEbRmUCnEp30tEnRoVjn\nI5t4zSZds4k3tGmHzT0inkkyiUL6wKOGTzw21zknuZlXfXatvkHOHxDm7O/CoxWRgFFpvDZ4WxFt\nTbQt2a7IdgP2AqSBGCHFsj5IKim+Tx5Tzwn/MZD/++F9NP4vgL+vlJL59/z3IvIPPsxtLfjJcNL4\n9lRddya2RhtwKtKpwJbAVgW2RDYpsM2BVT7QhBuaeEOTbmjyI/E9+UHLBx41fuYphc7Jey6WR4J/\n0/o8GCfz7z89WEYUkynED64mVi2pWpHdBqlmjR8CeF9WNT+msn88lzzcbT5bT69/GvijiS8i/y/w\nr37Ae1nwMeB5dZ1r50KbsmqdcdLT5chWApdy5DL3vJYjl/lIlw6YsMPEe0zaFTkjPjxq+cjLGv9c\nw9szqXhqAbwk50TPz35/5szUtxWhqkl1S6pXSL2Feib+NIGZQE/lLkTPN3wi9vmjZPY4fkKkhyVz\nb8HXcKbxq7oQvl5BU2rqtY64GOlSzzYGLtORK255k+64Sre0cU+OfZHUk/ORLD05e7zkJ6Q8l5MO\nfW7qnwh/EsvXjwHne+HxGHF6sOSz10almIwlnDR+3ZLbFbnZIO1s6psR1FDeLc+kDwLqm3rynd/5\np/EAWIi/4Cle0vjNCtoNNBu09jjf0wXFlqLxr+Itfy1d8za8o4k7puiZkn9cs2c6M/Wfn5Cfm/kv\nkd7NUp39+flZ/rRPPHXqwWPEYOKpxo9VIX5qVshqA90FSAv6yBNNHwT06bc8z+c/J/2ng4X4C57i\nJeLXM/G7C7QacfqOFbDNgct45I3c8ja/41fht9ThnkPOHNIsueTBj5LxZPz3uQWekv9E+ppC/NPr\nL/kBTo68c0/+ibIjz0x9N5v67YrcbZDVK0rgzzzV9DaBOZ33n2fzn0z9TwsL8X/WeB4NP21Pcfmz\nvZr3VY12FdoZtNVoC9pmtClx+gs1sVUjGzWwkSNrObDOe1bpni7e4eKOmMFnsBn0bMdnyhE5nt3N\nN4XmlCoZeAawCioFTkGtZlNfadTcTF/QJKXJSpPQBIQpC5NkfM54yYScCZKJea7AVxkho0glxVgF\nHJ5aTQi6kNwVT76kVOL3CtAK8QoRdZZYpJD8eP2pYCH+zxLfFPRSpdec1mA0GPW4n1dTVVR1jasV\nrprj9BJxscf5Oy7o+YX/itf+mnW4wYUdxJ6QPMecmTIcMvQCo4CXQvgk3/P0O9+SNuW2rCniNNQG\ntDYkXRGNI+kicV6TccQs+OAJ0eNDIAT/cJ2DR0nGJI+LPXXY0U03BNOQlC05BqZF4gRqQuoJ0RM0\nE7KdkBDJIZN9JnuZ+/4L6SG9+NMh/0L8nyXOfePn+3nkjJnZVNlHZlUGrEVbTV0pWqvoqkCnS5y+\nS4rOq5KCO11z6a9ZhRtc3EHq8dFzSBkzk77PMOZC/CCPDrzvvPNTKYCd0wdsuU1nwVWA1SRTE2zL\nZDtG2zGZebUdIQl57M9kmFfIMaIkY7Onij2N3z2QHkBJxNsWUYmsI1InpElz959I1okcSvHQgwxC\n6Oew5Ck2+QlgIf7PDud+8Rf836cCm2pmknu6ais4Pcfp9SxEtrHE6dfpwNrfFDnT+H7W+CrDIIX0\no8A0m/fp+zq7Z+LrCowrUs3iHEilmZwjVh1DteVQbTm4Lcd5H5KgDjv0sYiyu/L4SxHtR1R+qvGT\nLhTQEjFpJNQt2SlyDVKX9fw6xoyfe/77naB2AIJEIY2fik9/If7PFM9z2M5EVaBdCdc5B81JKqgd\n2iac9KzkLE4vPZf5yGUs9fQu7HDhHhd2VPGM+CkjuZD9JH+oxn8w9W0hvW1mH2MDdQOp1lA7Yr1i\ndFv29SV3s9zWl4SQcfe3VHVDZWyJCsRI5UcqpYup/6DxZ9LnQvoqHAipJeuK3FpybclrS9pU5HXZ\nh6iYbjNTIxRDQZAIceShY/engIX4Pzu8pPFPKTAGlJvLaWtwNTQ1tDV0DXR1CdelSBfP4vTxlqtY\n4vRd2EPoUaGH0EM4PpzxQ87kM/M+UNYoj4mu33n3zzS+rcF2c3v+DkKroXHEpmNot+ybS26aK/6q\nveJ3zRXBZ1rX0FhLC7Qp0kwjDAeM1ihCIX7qEQ8qR0wspG/sHZGW1LRk1ZDqhrRpyJcN6bIhv1b4\npLFzu69HTS+Yw0L8BT85zgNi58SvZuI3RZW6FpqmkH7VwrpB6xE39XRecaHmOL3c8jZd89a/o/U7\nfPTFaRaL0yxEj0+ekDIhF7P+lOh2ypz73s49ZuKfa/zucSYHnYbOEbqOobtg311y215x3b3lL7u3\neJ/ZGMsaWMdInEakP2Cto1YaLRmbSlBR54hNIy4eiNrNDsOOtFmR9JpYr0mbNekykX6hSL+0TKlC\nzaTPUYgjhEMxotCfiqG/EP9niuelLeepMPVjA426g6aFroN1B5sWrSqcuaNTc5w+lMy8t+kdv/K/\npZnuOcTM8UxCzPiUOeaMn+dOnmfkPc/F/zaol0z9s5L/tNaolSOuO8bVlv3qkpv1Fdert/zz1a8I\nU+YVMKZI9EXT2/0ddeXIWlFJhuxREjGMuDkU+CC2I8YLonpFqj1xnYiXhfTxbzRU2XDS9HEUwkHw\nd4J28knN41iI/zNEqY/PaJ1QWqN1QGtBawGnkLWBziCNQWqDVBaMQbShZaJlpJWBNh/p8oEu7unC\nPZ2/ow47QgIf52S2uWr1FLufZna/T9GqUMbYi4I8S5rXqBXBGryrGBvH0LX0qxWH7Zr9ZkuYMm5Y\n0/QrwrElHxysK8zK4DqFS4JIIksq5fYyV9vOTXyzn0gZklJkq0nOkFpLWlekC4fLGTZCXgmxhVAL\nUyWYT2wQz0L8nxm0ybgqUrmMc4HKTTinqWbRVUV2DalqyK4lu4ZMQw4t6dCwyT2r8Svq4Roz3iDj\njjD1DMGzj5kpwTG9EK77A0z5b4MAKUMM4CcYqxJtPJGqB45GGFxxsIWUSSTQCV1FLJmqSbgu0awz\n7VZYDZn1KGx88WOeqm5TelxjLK28FIIlYSSAjIj0iJScfcnQ5BqVNVkUUTReFINoLAr1pGPAx42F\n+D8zaC24OtK20HYyy+PeWEvEEalJ1ESpidREX5N8TRcH1tM1zXSNnW5g3BGnntF7DjFTxUL6PsE0\na/k/KEHnOyACOUGIMJ1Pt5r/rFdwcDA0whRKVl4mITqhbMKojG0T9Yn4Q6Ybhc0E2wh1BcF/XYTy\nvgpBS0Tj0TKhpEeLRguonGlyg4glisGLZRCLw2CkUH8h/oKfBFoLlUu0XWSzTbNE1vPeGEXwjuCr\nh9X7ihDKvvYTK39D7W8w/gbxO4LvGUMhvk1zjH6Wadb4iRc6U/0REDnT+OaR9DmXXhiDgmMjDCth\n8pmYMon8oPGVPWn8TLPJtFNm5YV1FDa5EN8PMI0wDcUTLxStrzToWeNbCVgZsaKxGWzO2Bzpc03M\njik7BnEcxVGJw6BQn1CjzoX4PzNonXEu0q48m63n1aXn9SyvLgNWC/5gmY6W6WDLPlgmX66rKbAO\nO5pwj407CDtinE39kDGpmPc+P64xf0BTXwoJQwQ9V/TkXEzyEGDQwnElDKPggxBSKhrfJHSV0CpT\ntQm3StRTppuEVRDWSdiKUFsYjqUA70T6PD9o1NzUzxCp8KVRp4CTjJNALRO1NHhpGKXhKA2NZBwK\ni0F9Muk7C/F/dtBacC7SdROb7cjry4E3VyNfXo18eTVQqcx4oxlvDSOGwWvGo2GcV9WXWvs69ZjY\nl7l2qWeMHp0yejbtT/H5cGbqfwiciB/nap6Tpg8BrIdRw3FTiH8y9ZMUja+qiDFF49ddpvGJNmS6\nLKwFNgrqORVYzcr5ZF2YkZI1SMZKwkmgARrJtBJo8kSbB+rcMuaOo0RWkmlQVHIy9RfiL/iJUEz9\nSNt5NhcDry+PfHl15JdviziJ9JViQNF7RX9Q9CiGoOmPCjkkXPa47LHZI9kTsmfInpQzeiZ5mkN2\n5/sPdsbPheinXpcmwDQX7YwGjj0MUyF+nJ17ohPaJqybNb5PNDHTJmElwloJW1Mykx80fZqPFOM8\ndPNB4yeceBpJdBJZZcNKLKtsqPPIUSI7yXSiaMTgpMKQ0AvxF/xUOJn6ReMPvL488OXVnl++3fHX\nf7WnzhNH4BDgeITjrXAEjh4OB4j7jJYiZq41jZLJcz29kmcNNJ5dvy9OGv9E+oeq4blyeLRw7IVh\nzEyhePXzmVff1DPxY6bJmVYyKyWsjbC14GZNn88iB1VfioLOTX1HohHNSjwbUWxykTq37HPmPkMn\nhkYsjhpLWjT+gh8aZ33un/W/V5Wg7YA1GmeExiQ641npkY0+UjOhlaBEULmoa4mCzCWmfnr6TsJj\n9t38zo9vpx9r50+3UWLvioxGlCIrhSiFoMizqlWSUVkepLSunvc8trF/DgHyBDJR2umcZDxblSIH\nQ862RC1MQ3Advlkz5S0SwQ+CbzKhzsQqk2wm64yoYtFoEbQIVjKVgBNoBNo57t9ITS2eSgJWIoaE\n+mA2z4+DhfifHNTTXvfKPL12DnREkicPI2nnSL+3pEoTAZOF+JWQroV0I+SdIP0fUEuumJtzlOpe\nM6+nazGaoA3RWIK2RG1JxpK0IWiLZNA+on1EhfSwf3gty2M7LXWWeDzvKwUhKSavcL3C7jX6zsDK\nkGtLaBXj1HKYNtxNr2j8RBUSSiBpS212DMbTG8+gA4P2DMozKU/Cl3Hb8z/1JRq/2NpEQH06nAcW\n4n96OG+N9ZLUDlGBHCdkGMj7muQqEoYYwKaZ9NdCvoG8gzyPoZb83TpLzem0lQPrnq6Vg2wVk7WM\ntgbryPOarMObmpwE03tM79H9hBk80hczQ8UEWdCUzjtWzQ02n61jgn7SuF7PxC9dOrI1xM4wxpZD\n2nAXJ6qUUAlStky6pjY7gunxuifoHq8GgurnHv8RMxP/20j/aUTqvx0L8T81nBPf1k+lqpHaIXpC\n0kAeD+SdI2GJXhOPEJMQb4R0w4PGL9No5fsR/9SE182lsm1ZT/tcK0xVngzZtYSqRVxLrFq8a8kB\nzG7A7gbMbkB2AxZQMaPHgCKjZ9I7HltuuVks0EfF0SvqXlPtNaY2KGsQZQldxUjLgTUVEaUgYfDU\n9HpFbe8RsyObHVnvEL0jK0qjDTV+YxPQ08VJs7/4EPiEtP5C/E8Oz4jv5tn0J6kdogYkHshDS6Im\ne0s6auJ9IX7aCemeQvodf5Spb91ZH86uNOJtVpBbDbUl1TWhbtH1Guo1qV4R6jVxEuztgdxUVLZk\n56iYUGNAdKGRZm6yqaB5JhbFMSmaSeGG2dS35ciTsyF2jtG0HGxCGUjW4m1Db1bs7AW1ucOYW7Ru\n0NqW3n46otWIfskvf8Z+9cJrnyoW4n9qOB9TbevH0rV6Nc+pd6AP5LQiDw05ONLRkqwmmlnj95B6\nIZUw/YOpz/fQ+Ohyrj9p/FPn7W4Dqw3ElSK1ltA4prZDN2uk3ZKaLb7dEkYhNxVii3v9pOnlMD4l\n/txks1bQaugUtHMDzi5Be9L41mCUhmzIoWj8oW5RNeTa4puGvl6x11taPVCbLbVpcMbiNDgdcWrE\nccDNxH+J1y+d7T+1c/05FuJ/anje/ro6DbzYQL1BjENkh8QV2bdkcaRcEUUTM8QoJM8sZ00iv+8Z\n/9zUb6HpCunXr2B9AXGjCJ1l7GrsqsV0a+i2pO41vntNGOSM9Ak9BvRhxDiLaHVqZvtE47cKVgpW\nury+SorGq3LGVxqdDcobZDCEzqFWiryyTF1DLx2VDrgq4LSnsVs6Y0tZv450eqRTB1AOi37im3+S\ngvwNpv7z/aeChfifHF4w9esV1BtoL0A5ZLpDYof4hjTVpMmSJk3yEIMQc8lYez4E9g829WeN320L\n8TeXELeacW3p1zXVqkWvV8j6grR6jV9/ge8Lc06kN4eJdHckO4vMLWxOGv90vm9n0m/0SeMr2klT\nK0WVNdprGAz5YIkrR95afKjROaNNRrs5N0FnGrNla2BjIls9EtQBUXcY5ajnSPx3OvfO/vBT1foL\n8T9KPIvTn6+2RpsKZQxa6+JlVxmlI1p5VuJpU8CFiB0TekwwZNIghAF0fDqe+qUmGaf4/Gk9j9m7\nDmyn0W1hZXaaZBXBKLzWBL1iMmtGu2Z0G8Z6y9hsGVYXjOtXeC3IJsLGo7YevZnQ2wF7MZAujiQl\nJNQ8WFPNAhMKB0yyxqcWH2oCFTFrUlSl5bUL5BDIp3/M+cjvuR9JEyeYGvRUUwVHHS0hGVLWCAoE\nJIHMczLTCHk+EsX9nOLbK9Kgyd6QU4XkClFznzBTl0/zlIzwfP1IsBD/o4N6Gp9/tmpb5tNbrajw\nVPmIjZFq6qm4Y517vpi+4sJf04Ub6rBDp8fZdcLjfPrTfLlz8/ah+01VzvK6emx1rS1UjcatLXSW\nUFmOWIKvOBwsNZYprrmJl9ykL7jJl9zIK265YKc2HPSaOGVyGstQysajtgH1RUCHhCbhdyN91BAU\nMWqmqOmDZhc1bVQMectfpi/5ndpyqxp2ytBLwstIzntQ+XHMjuIxA+k0Qyvew+/3cHeE/QD9BFMo\nDsYsqHkETx4g7SHegq9LyvAIDKKY3hn8bUXY18SxJaU1ojfgLkr/wpzL0yPPcr7/SMi/EP9jw7nz\nTldfW7UxVJWiNopGBZqcaGJPg6LJirUcuZiuuZge+96b1CO5zK7LPCX9udaHotlt9djd1jazD3Fe\nda1QzoKr8a7GS43yNVCjQs3oN9zH19yl19zLa+55xb26YKe3HO2anDLkCWX8TPyIDiX7zVYZv59g\nMKTRMA2GfjDsR4MbDC4ahrziOl3yO7XlRrXsxDDkjM8jknagwiPp8/yP9BTW9kDawd0edXdE7QdU\nP6GmACGh5p5h4ouWj3sINXhbnhlDghHNdDMT/3Ai/oqsN0h9AamGFEqT/RQeBebz1EL8BS/iZJ5W\nj6bj2ao0WB1pdGBNYJUDqxjntcynX/kbVqFIHZ9q/JPy+ybia3U2Nu8ULJibXboVUGlCKVrFq45A\nh/cdIXT4Y8cwbjikC/byigOv2KtXHPSWg9lwrNYIpecdJqDagNmkB9LbTlB7TzpYpn2F2Vu0tRgs\nOlaY0TLmlpu04YY1t9Kwz0XjhzyS037O5+WR9CdNPwBHIO1R+z3sisZX/YQaZ40vReOLhzRr/GDn\n50YEO8GgFNPe4PdnGj8X4lNfgNQQp1n8Y+tdOd3Qx4HvJL5S6u8B/w7wTkT+lfm118D/BPxN4C+A\nvy0i9z/gfX4+ONf4p6aYtn1Ytc5U9LREVhLY5iPb3HMhR7aUHnl13FGH+7KeiC+F+IrH2fQvEn8O\n11XNHCzYloBBsy2SjeIYLCE0+NBxDJsifsMhrOn7Lb1s6dkyqAt6vaU3Wwa7oXdrlE2o5FEmYJqE\nIWNm0levgH1AbiukKb3/BYfEChkdQsWUHDsadlKzk4ZdnjW+HskGyObr5n1PmbhZA/kA/R6OR1Q/\nwLFo/JOjxYNXAAAgAElEQVSpD/P5foBkZ4NhJr05wqgV42jwY0UYa+JUNL7oLeIuypuEYS73OyN9\njnMxw4/yLfpOfB+N/98A/yXw35299p8A/4eI/BdKqb8D/KfzawveG880vp3j9FVRuVpFqhRpcs8q\nB7bpyGW+5XW64zLf0qY9JvWY1KNTj0lHzEnjz/OdTmRPvGDqq3nWRnM2JPd1kfY1RK0JR8uxrwnH\njoPfcOtfcXu84LZ/xcFu8WyY1JpJb5jMhqmapV6h64zKEW0SpjkjfYKQIB8isamJtibgiLEmjjVx\nX9qFjdnSoxlE04umV5o+pUJ87SGqp+b9+ZztCshHmPao8QjTgBpn4r9k6lMagvixkF7fw2gUU7JM\nsSKkmpjakjOhZo2v65dJnzwfU+DvO4kvIv9IKfU3n738t4B/fd7/t8BvWIj/YfCSxq/mcJ3boJSn\nCj1NUKxy4CIfuYy3fBmueRPe0aYdea6jzzKv8/XJufdtra+1nk39pnTfbreF9OsvYfUleBTHqrTp\n9r7jyIZbf8G74xe8u7tkby6IakXUK4JZE6sV0a2Ibk1sVlgSWkeMKeZ9ZYRKC05DNIq0T4y2YaJh\njA3T2DDuG8aqZqJhyhovCZ8jXiW8Oq2erBKYXDT9SzO0DSVNMe4hHFFxQIUJ4pmpD+DnB2MqE3L8\nsfTNVxUMVjEpg9cVQddE1ZL0fMa3F7Mj5Bnpoy//px9RG94/9ox/JSLvAETkXyilrj7gPX3m+AaN\n7zbQXKAZqbijybCOxdR/HW5549/xy+m3NPEeTxkL7eca+vP9qbz2eQXac+deVT+a+quZ+JtfwCAa\nh4VQ448rDmy48a94d7jkt7dv2KlXZN0htiVXHeI6suuQuiO3HW7W9LbK2EZKDlIDodaExsAhM9By\njC3HseW4bzm2LUfbcqTFZ8gMRdRp9WRGMgOoyLcNC4YBZF80fx5AJlQOKJlNfTWb+rGE8sI8Z1Rp\nEA1jpZicwdcVwdVE15LqFeI2SP0KsisfZD7T9A8dQz994j/Hd5xcfnO2//UsnzNe+EaeAuamRtt5\nPn2t0S2oLqPbiG49G5nY2JGNHljLkXU6sAp7ulzm09dx96TJ80mrn8ZryPNb4PGtoczPtE6jG41q\nNdJp0koT1xq/0Ux5xXDY0rsNR7PmwIp9WnEfOu6Gjp000LoiTVVmWztdztdOUAg+gE8aj2HSlso6\nKldjVYPoTK9bjqrlcBJO0hHkdEgJpfj/awf68B2f/IhiQuHnLIFSS186BvAQxz+9S4Qnn6evhNhl\nUBltM5VONFWk6wKbzmMEsg1kHUsvwJzJMZODkH/wbJ+/mOW78ccS/51S6hci8k4p9Uvg+tt//E//\nyLf5OULxMLX2tJ7tta1wdY1rFdU64NZHqlXErXvc+p61HLk8fMXl/pqNuaFRO2zuEe+ZVDnDBx6n\nNWvK0XZ+18KVMy3Gs9WsDFVXQePwdRlR6yvHwTicdhxZ8ZW64Fq94kZdsKOjVxUl1b/0oidS2u/2\nGQ4RXATjAY+MidQeic0B3/ZMzYhpJ1QToE1IL/RfRYbrwHSjCTtF7CF7QfLpYNJTDvDflI3wfjgd\nfdL8+UUen5NJZbT1uLpn1e0Imxtk02A2lmYDR6kJ+yO+mst+6fFpIHiP1/kHnqL9a54q1X/4jT/5\nfYl/phMA+F+B/xD4z4H/APhf/oC7+8wxn+FV9XWhwliDqxVtp+jWgXYb6bY97VbTXijW+cjGXbM1\n12zUDW3eYUKPjOWLdfrSnr5gp5C2oTwAYCa9AWUfnzsP+06XJJSmxdcdvqTqFdEdB+m4Viuu1Yob\n1bFTHT3V3MIilBY5UWBKJfB9CDPpa0gj0mdy3RObgVD3jPWIqj3UocyjH4XxOjJee6YbhZ+Jn3w+\nS4AZKfG5k4Z/7qJ8P5z7QU7kP335s87oyuNcT9fukHWDubA0r2B9Eemlpq98afAhE330DJOnt4Go\nfmjif398n3De/0BR2V8opf4/4O8C/xnwPyul/iPgnwJ/+4e8yZ8XThq/AlUX0fXDXlcK5wJdG9is\nA5ttYPs6sHkd2bwKrORAY29o1Q1tvqUJe+zYk63Hz1+s81rxUweb0/UpFVfbcgsP6/zsySuNbx2+\n7Qj1Fl/NYrYEvWWfW26Ue5CdcmcaP84aP8/EP2n6qhyYvUOOmexGohvxbkS5ESqPuEhyCZmE6SYy\n3SimG0XYCbHPZB+RHOEh9/B5/mHiQ+D8wXki/pNj0zONbzeW+gLWl5HpcuQoDXud2EtmHzPGl5Tp\naDPTR5TY/328+v/+N/zRv/mB7+UzwbnGr0G3RVRZtc24uqdrI9t14PXFkdevj7z+ouf1FyVOb7jH\n5h027LDjDnMckJn4ia87ss/n5hpV/EynibS6Lh7rk4ROQ+vwzQpfbzm4Sw72kqO55KAvOeiWnVLc\nK8VOKXaULr1enUz9uR+2D9DPU3qTLelvo0VqIVlPrCZU5cF6pPIkGwhVgiD4XcTfQ9gJfpeJfSL7\ngOS50f4D4c9TkT68qX96iMaza3RGzcS3XUWzLpo+XY6kN0d6qbkVTZMU1msYNKlWTFaj1Mfj4Fsy\n935sqDPi6xPxVw+ibaSqE23Xs1kFXm+PvHl9y5sv7nhzdUub90juyaFHxiNy7JG6R2zA6/xwpodH\n0p+Hsa16mo+v56nZugbdwLjS+M5B0zHNxL+1V9yaK270FXvV0OtIryK9ShxV2Xti0fg5QDQluR0D\nyZSROKOB3kAlZBOIJoINiIkkEwkm4G2CCLGPxL5o+nBMxN6QvEHyyXaJPE1B+mFM/fPrByfprPFt\n3aNahV5H1MWIvjyg39xzzA11rLDewVARD46xdhxshdaOR/vrp8VC/B8dz039E/E3YDbl/FgPdK0q\nGn8m/l/78ppfXr2jTXt8mPCjZzp6fDfha4+visafI8gPX68T8U+Ja445Vn9KFXAlXcDMhofpNIfW\nIU2Hdxccqktuqive2be802/Z6RqvRrwa8Axlz4AnkVUoLvE4a7akwesyBcMqsBoxkHRph511JulE\n0AmtM0YXd3rpE5BIXpPPhHyKS2SeGuOn/fvjvCz39BvPoyJWZ2zlcTW4LuE2I+7VAfe6xn3p6KXB\n+g6GlnjoGNuWQ93hbItWp+qhnx4L8X90vGTqr8BswFyg7Yir7+haxWbtubw48ubVHb/84h1/4+qf\nUad7+jFzPCaOu8yxy+Q6MdmMn8+QmkJweHTq1UBHqW83+rHiztZF45sWzArUSnPbFo3vzzT+O/OW\n3+pfca8cWe1n2ZHRZEq/uuLcm7PnkipW+Knvti770n5bEFXopdQs8x4ByXnuD6CQrEo23Wn/JPPg\nefX8h9X4p3dTZ9dKnUz9MrtgtdZ0F5rVpaF7o+lzA8OWeNgy7TYcui33Dipr0epjce0txP8R8OxM\npzRaqzlHJ6NtQtuIrgLaerariW07sml6Nu7I2h5Ymx1r7unyLXXalUkz8+w6K6Ur7clvVBxQmqh0\nSa9VGq9KpxqjNDKb9taBcUKc22JbIxgFIxsG2dCnLce45eAv2E2vuB9ecVe9Yje54lAfE3gPYSzm\nfGIuREmPqvIb/G3fXZn+0zrBnjfeOQ9pZSUolTA64XSgMYqVgU1VxOWWewtrq2m1pdGOSjcYlcqD\n7SPBQvwfBOd+dfXk2ihFZROuHnG1xtUZV0+4+ohr7thuBq7WX3FZXbPOt7hxh7rvCcbTp0yIMPwl\n+N9BvC3NMlUPNoDLkJVB2YpkHYNxROsYjeNgHZVxmEpjnGAqKasWjAjaCwbhwJavuOI6X3ITL9j5\nNf3Y4PuKfNTFr/Y74Ba4p1S8jTz6137uEFBpThD0oEZBDQp1ALWn1PQfy/+JmqR8XhHUh5oj/oGw\nEP8Hw8mDe5JyrbWmtom2nui6TNdNdN2Rrq3oOstmNfB6fc2r6pq13FCfiJ88/ZCxHqZrmH4nxJuS\nfar7EjVzAklrsCWVNLqO0XUo1821tR3aGrQSjC6tqLQSTM7oIJiYOcY11/kN1/GSG3/BblrRDw3+\naMn7uQDmZpYdj8QPfFRf7B8K6jxR0IOaQPWCOir03AdEH0ANoMb54XDKqPqIPp+F+D8InpL90TWk\n0UrhqkRXJ7bdxHYN2416WNdNz6q6ZeVuWMkNbtxB6omDp7/L6AnijRBuS3eYvCtfMjs3y4xKE40j\nuo7YbAntlthsifMqpiq96HIqa5r3MaNzZvAdN+lLbvwlN9OW3bCmbxt8U5EbXYi/o2j7c41/ni74\nc8bsAHjU+DPJj4LaK3Rm1vjymFx4Cj4sxP8543ke/imoNhNfC85GVnVk20YuN5HLi1leRVZuoMo7\nnOzKOu5QfU/InqNk9FjM+7SDvC8a/2TqqwxKaaJ1RLdiaLeMq0uGWcbVJUk5lE9on+cRVgmVEjok\nlE94anbhNffja3bDBbt6Re8afF2RnSqjcY/Agcf1M9L456UBai79Vf2s5felkYc6Auca/8OmGXwQ\nLMT/QXCu8U/pM2XVKuFsoqsntt3A5Xrk6mLg6nLk6nKgswOMPYw9Mq8MPWH0hDGXL1RPSVfvZ20z\nFFPfZsBoRls0/thu2a8u2W+u2G+v2G2uCKpB9RHVp7LGVOrjfXkt5Ip+2tJXG3q75Vit6asGbyty\npSALjDI7+M7kMzvjl3qEc43Pg6mvjjKf8eeHQ5z/zkeEhfg/GPQzKTFcrfNM/JGL7sDlZs/VxZ63\nl3vevtnT6gF/7/HJE3qPHzx+5wn3Hn+fkfk8r31Zn0iGbDXMxB+aC/arS262V9xevOXm1VtGWpSN\npY9tjKUJRY4QImqIpGDwpsObVVl1hzcN3lRko8t5IgBBnibPfU4a/7mp38/m/Q60SHH0zRofDyrI\novE/D7yk8Ut/Z60irsqzxj9yub7j6tUtby9v+NWbWxrpOaTMsc8cJXMYM+E+E36X6a8z6Vi899Us\nulSIYufrpDTKOGL9qPFvN1dcv3rL9Re/opcVEMpw+DGgdCgF6D7CEJBRkVX9gljyqXXUQw7N2Rf6\nw+XQfNx47tw7Eb/M5EAJj179EZSX5Yz/eWCuguFUAneqvCvVMKrKaGvKGCoTccZTq55GHWi5o5GB\nMKe666loDjlC3IG/h3gEUZo8y8Nea7LRjPUFgyum+tGuOdg1e7NiZzruVUtPW+5FLEhVOsDGACGU\nfHoPc3IvjyksJ9tWUX7gvDjmvHPfZwKZTfrTxzKf99U0E//kyX+pqeFHgoX4HxrqeTL8WRWMcaWx\nhTsSVYOPNeNQ0e8tx0qzR+EzHP8K+lsYd6XtUxhLCrxkEGUIpiIZRzCOUTu0KWKMo28u+Kv2F9za\n1+xY0ceKaRTifkLYQ4pwF2Af4RhL+1gfyuty6gJrz6R6dh2APcWzd14a+xF+uxd8Ixbif2iciG/n\nOVPWzc3pi0hjyNWBwFPiH9DsgqLJ0N8U4k8z8eM481JKVl6ae/GJ7ZDqUag6erfhrn3NXfWKPSuO\noWIchMBEDntIUyH9LhXiD6mY+TGWPHvgYezMw3q+Dzx4Fj/meNWCb8VC/B8CxhTiu9Ngy6Yk0FQt\nUmtS1RFVyzQTf8BwCJr9ULq6jrtHmfo5K3bW+IX4jlh1RLcl1kXSvB/cmp1dFVEdx2iZBiHGCRn2\npXLumGbJc+ptKp0lnxD/awW9sySKpj/JeTOMhfifChbif2ica/yqgrqGup1lhThNsi1BNfjoGEdH\nHy3HwbC3Ch+Llj+XOJW+jSLlfB+MY7IrpnrL1F7im0umtshgVxypOFLRU9HHijFkwjCWYRZRzSNh\nMgyz+FSaZzxMzTyPRphn15nHRhgnWYj/qWEh/oeGmsvfrAHnCvHbpsyTbleI1STpiDKb+qGil9nU\nF0UTi2kfxqdrmo/RWRfij1VHX28Z2kv67op+dUXfXTGYljHAFIUxCmMQpijEMJHjWMJw01zh4/Ms\n8jg+9xuzDk/7ebjc12Qh/qeEhfgfGi9p/KaBVQfdGtGa7DtCaJhCzegr+mA5BE3ri3M9zb62fDZ6\n7VzjR+OYqo6+vmDfXLJfXbFfv2W/ecugGuIwEYaJED0xzvthQoapnOeTlL54JzldP8x1+7b+1Ccv\n/0uyEP9TwUL8D43TaObTGb920LbQdbBaIUqT6IhxNvWHir43HHtN3SvCyXsvPMysP99n9ajxj/WW\nXXvJ3eqKu81b7ra/YqQmsyfHPTLuyTGRhxHZT+T9HiY//0JmR/yzfflHnP+Dnu2f18G/VBe/4GPH\nQvwPDilVbyahTURXEe082k3oZqSVkZWfqFWgyhGdEuIzcRD8AXTgazPpTz3ylILYKEytoTbkyhIr\nx2RqBt1w0A1jrkGmefDbXEbrE4yhjIT200/9Af2k+K7H1qn+nm/Zn64/ZSzE/8AwZCrlcarHaXA6\n4vSIMwecuaOVgZX+irW+ZqVvWKsdneqplEepDGpugjk3yDDzerq2tTC1iaEJ9HbCyYANR/SwQ3EP\nuYb9HvojDANM03x+SHwsI5o/Nnwfm0W98OfftP/WN/pIsBD/A0OrTI2nVfz/7L3Lj2TZvt/1+a21\n9tqPeGRV9uk655Svj4/4A9CdM8BIDBggITHwAIR4CTExsmQGYE+uhJjA4EiA5IkBC1tCAnlyYYIA\noWsJJJAwXMmApxds9z1d3ZWVGRH7tZ4M1o7MyOys7qo+1d1Z3fGVflo7dkZG7MyI7/7+1lq/B50E\nOjXRqZ5OWzptafJIo19Rq1c0qjTEqGXALpXpWWJ/qiUE4HSsLJgqM1aRwXgaM2MZqcIBPe6RcFP6\ns/d9sXEsCn8k/lP65j0BPCza9U2TlVv1P3lyfhcX4gnOgs7E/8BQJKw4Ogls1cRWKbZKl1Ermjyi\n1RVGX2HUFVrtMDKgxaFId0sEdulY25bxeKxNpifSiadhxuahKL7fI9MN+LoQflrsrPiP4jHCP+Sn\n8FW+Hv+FH/t/8kz8DwxFwhJYSWIriUtJXOrMpU5c6kSTJ7Legbohyw5kR5aBLKUlxdHVNxZsu7Sq\n7qBZFTM6cwiRXfA0YcKGkcr36LCH0BXiz/N98750oTkT/x7eRv5TPCT/Qy/hdHyq6v4YzsT/wFBE\nrHg6cWzFcakcL5Tjhfa80I46TwRd+qoFNeBVj5eBgMMvrr42d4rfHHvUb2C1AaMy6ynSjY5mmrGh\nzPH1tEemBua6EP1oYbGz4t/i64j82H/osRYY+eGDr7gG3/bqvh+cif+BoeTo6g9cqIFLNfBCDbxU\nIy/1QJ0nRuWY1NJfTRyTlDHKA1e/LXE/3QbWz2B9AVoyq32gw9OEmTqPmNCjxgbZ20L8GB+3M/Fv\n8W2n4l9Z4HvbXQSeNPnPxP/AKK7+TCcDW9lxqXa8UDte6j2/0jtqJvY6sVeJg0rsJSGSiCTmh67+\novjdthB/cwk6J9ZEuuBpxmVxz/fo0SJ7A2O97M2/xc54FO/rpWceIf03HT8hnIn/bSACamkU8eBY\nmoRqNKbKVCZSi6NhpIt7VuGaOs1ET8m5j6V4hkmgTjaRsxZypUiNkDohrhRxI6QLIeWG6GvSWJEq\nRVKQcyQHv8zpf+h/zo8bWRZTQtKQDEQjRAvBCiFrotVEo0hGSFqRb78fP/TV3+FM/PeFkhKH/xbL\n1pLaQGgdvppxMjCHinHUDAgxwXiAuQc3lgScuKy9ASWyTxtmYzCVQWpNagyhM0xrw5BaroYV1+2a\nfb1mtC2zqQlqqZBzxrfC1wUp3+uOIJC1IlVCrAXfCn6lmNfCdCFMuWb2FX6u8KMh1ppoNEkv5H8i\nOBP/fSEL8etqCcl9YJUlGUc0E96MOEoizoimD0IMMPaF+H4EvxA/L9vsWYSgDLOxiK1JdY1vLVNX\n069qhtzwum+5blsOdctQdYX42pCfkqQ8cXyF0I+ce5z4QtZCMopYK0KjcJ1m3iiqrWLOlnmyuLEi\ntIZgNalSxYv7vv/Ir8GZ+O8LdUL8toauvjdmY0lMhDzgOeByXRQ/aIYsJL9sr48nih+WxLgT4oup\nSVWHr1umtmXoOsy6ZcwNV3vLTVOzry2DrZmNJShDfkKK8pTxMPvgG8l++vxlKpYqRaw1odX4lcGt\nNfOFZko1brS4ocLvF8WvSlm0j0rxReQ/A/5Z4POc8z++nPsD4N8AXi1P+6s55//uO7vKp4RTxe9q\nWLfFNmXM2pL8QHQHgm9LBp6vGL1m8EKcYZpLroybIbiHiq8I2pBMjbctc71GmjVqtUat14yp4XVn\nuG4N+9owVJrZmMXVV994+Wfc4V2U/uHPkTJvj5UmWINvDG5lMBuN3hqmVDP3Fr83+NYQ6kXxVWkY\n+lTwLor/N4D/BPibD87/Juf8mw9/SU8cIlCdEH/TwsXq1rJY0nAgDDv80OJ8fTfHHwrxZ1/M+bs0\n3HQkPkXxs6lLSa16TW4vyN0FeX3BkGquO+GmUexrYaiEWQtey1nx3wPfhvR3c/xTxTf4zjCvK9S2\nYsqWeV/huorQmLuFPv2RLe7lnP9nEflzj/zoCf0Z3yMeuvrrhfiXG3i+AbGkmx2RFT60uNEyLcTv\n90Ic71b0XYRwWvXqRPHDsf9dvSG0F4TukrB+zpga9l1i1yYOdWawidkkgsqlws4Z74xvmt8/fE45\nIfeI71uDXlWojUVdWOZU4zYWt6pOFF+TtXpSN+bfZY7/F0XkXwL+d+DfzjnffKBreto4uvr2geJf\nbuBnF+RsSVwT/Ao/tjhOFH8npOF+DYxjHYx0TPoQISqDMzVz1THXa+bmgrl7zrz6lDHV9J1naAJ9\nHRgqz2wCQQXSba+mM94H76L4x+O7VX1FsBrdGHxXIWsL27q4+muLXxQ/1GaZ439kiv8W/DXg38s5\nZxH594HfAP/625/+RyfHv17sI8WxtFalwZrF5bewqmHTQM6ksSbVlmgqgjL4pHBBmOdSK/9Yav20\nwp1ZxozgsyanihBr5tAy+BWj2zDMW8ZUMzrH6B1TmJmjwich5kypdvtTw1fZJApE5XujWo7bnKlz\nxqa8xFBkdMq3jUmEJSxD3YVnHB+LgG4FbUuFpaQrnFhSbvChYXYNQ2rY+4Y+NIzR4qLBJ03M6nvY\ndfmTxb4Z34r4OecvTh7+deC//frf+PPf5m2eJk5v//eb5JQ+Fbk8zpoS4KEgKkhyV6Dq+OuGErNT\nLScyYLKQnSIMCr3TcFWRGoszNRMNY6qZPxPcq4y/SoRdIg6R5IScnpCkfKd4dNkNEESDqRLaZrTN\nmNuxnGtzYOMiG5dY+UTjEtZltMulEQZ3BZT0iR0fy0bIa02uKmKqCWPLdN2RbUemY4gNrz+ruH5l\n2V9Z+p1lHiqC09/D5/Nr7ovq33nrM9+V+PemOiLyi5zzb5eH/zzwf73X9X3sOJL+YYesCkiUJjUa\nsirETwJR7pT+9FdFTkpZSoniC14xjxq9N8iVIRqLp2YKpcLO/Arcq7QQPxIHT3bqrkjuTwLH/9rR\nymOlMroWbJuwXaTqwHYZ2yWqLtHmyGqIrIdINySaMWOHjCEjISO59EExdqmM/mBkJbiVxpkKn2rc\n2OJuVnjWOLei9w1vXhmuXxl2V4ZhZ5gHg3ea9IRuzO+ynfdfUiT7ExH5/4A/AP4pEfl9yvf4T4B/\n8zu8xqeH0+/aA8UnLaQ/UfykFuJL+YfJ8ddkaVVxHAGdhdkL46DROwOmImHxoWaaiuL7q4y/ivir\nSNh54qBJTsod5ieB+2Q/vZWKyhgbsF2m2QrNFpptWizS5Ei7i7S7RLtLNLtERUYHSq+7VJS9Wqqi\n22Nl9K6MsRWk1cSqYoo189jQ0zHMa/r9loNr2F+pWxt2inlQBKeelEf2Lqv6/8Ijp//Gd3AtHwdO\nv3OPET9z15ruAemP/WY0yxohUAnYk1FnYXSKw6gK8alIocJPNdOhYco1YRcJN4Gw84SdIQ6K9JNR\n/IdLcPfr/osqrn21yjTbRHeZ6S4zq8tEdxlpUqB+k6ibhDWJmowNGTMtawIsrn29ZEeuoV2sWUOo\nhag0k6pIyTKNLQe34ma/4UZdsHc1ww6GG+h3MOxKp/PgliCtJ4Jz5N63wdeQP+cTV18XEU6L0p/2\nT1QUpbdALdAso0pw8Ao7aDQaQkWcLH5fM183jLkmDmExR+oNadDkn6zinzb90IviZ2wXabbQXcLm\nRWbzIrF+EWlSpGoixkQqEiYkqimjDwvx813pM9sVsncXJUNydQHOCFPQKF8RQ83sWg5+xXXY8KW/\nYDfXzENiGlIZ+zJ6l8hPiPln4n8bvM3VP87xj6RXj7v6yN1Kvl1IX2r0FcXfOcGi0cEgU0U6WHxV\nM9mi+Mn5xSqy0ySnf0KKf4R6YGWyJCqhbcR2iuZCWC3Ev3iZ2L4sxNcmokiokFBTRh0yynJL/FPF\nbxfir58Xm5Ww7zWqr0iuZh5bDv2KN/2GV4ctu6khuIB34d4YXCAf92yfAM7E/zZ4bHHvZI6PuU/+\nI+nvrerL3b2iWUi/kpKe23hFFRRaGRBDFItTNZMUxc/JQZrJqYJkyEmRj6uIPwm83eVSKmGOxN8K\n3SWsF+I//1UhfmaJnJoSHBJcZ7AZJJctu+Mc/6j4W1g9h83PwCDUaPRsiKkQv79e8eZqzRdvLrgZ\nyueTkiMnT0qOlIScMukYnvkEcCb+++LhTtJD4aHM7W/zto/G/UbSt6v4i1mBWpVgniplTAa1FM/I\nOZNyIuREuI0AePiKT+ML9eHx8GamEKUQJcs+/XEs+/TNNtNsEs0q0raBtvG01tGamVZNNMxk5UoD\nAxXIEkFFssqgCv+NViijoFKkWhEbRWgVrlPMacVUbRjVmjGtGHxLPzX0fc3h2tKPi9t3O7l7uPvw\nNHAm/g+BJTDkGBRy2jhD5YzkhGSPJIfkCUkDcIC8p1Ta6LnfpvrH1p/+6/bpFbpSaCtoC9omjOX2\nuN0ENp/MrFYTrRmpw0h1mFBfLt19w4j+0wH5YkJdz8jBo6aAxIiSjNEaqypQFi+WHovHcsiWOln6\nvMFVG1cAACAASURBVOKzfMGr/IyrfMEudwy5wmVIeMpncNpP8HRl5+l8Pmfi/0CQE/KrUyOjUkRS\nQJiRNIEMSO6BHVBz159+5sfbn/7xfXpRsuzTC1UHVZepurSMQrfybNYT6/VEq0dqP1IdBjQjMg3g\nR9QXE/rLEXM9ow8OM0V0SBiheBO6Bmlx0uGkQ+ggd0juOKSOV2nFq7ziKneF+FRLV4TjDfhI/OPn\nchqn+TRwJv73jCPhj99lOTUNQkaICB5hUXwZQA4ge8iOovRH+zG2qf66ffoluKaDepupt5lmC/W2\nPG4bz6qaWZmJ1kzYMGD2A2oa4LpH5gn1Zqa6nqnezFS9x86BKiasymSl8criVIeTLZ4tLm/xeYtL\nWw655SrbW9thi+ID6Zbo4cTOin/GEXJf8UWVMFGlF8UnogiAgzxBXlz9W8V3D+yYnPN0vljfHt+0\nTy9om5Z9+kx7mWkvE91lKsd2pg0zTZho4+LqTwM69hAOME2ovcccHPbgqQ+OZgrUMdFIJmpFryxe\nrXCypZdLeood8iWH1LLLwk0SdrnYgDxw9R92EX56n82Z+D8UTomvi6lbxV/m+HlGpCi+SA/suSO6\nfzDGt7/XR4e379MrVebzVRept9Belv351YvI+kWiNQ57mLGHifowYqeB6tCjDgfoDzDM6ClgpoCd\nAu28WIy0kgm6dCPupcPLlgOXvOEFb3KxfWoYUmDIgSFH+lyOHeFE8U+jNk4XY58OzsT/ASCPLe4t\nNwBFRuWI6IDk4uqTFlf/Vu1PleTpqsrvhrft05fIPNsl6otMd5lYvYhsXwa2LwONOMzrmYoJM42Y\nMGIOA+rLHnl9QA4zKiZMjNiQaGKiC5FVTKwE3KL4qA4nF/QL8T/PL/k8vWSXa1yecHlcbDkmLop/\nVP3TXZeHOzA/PM7E/77xYMH6dnHvxNWXHIviK4ekCZGR4upXlHCfb9sO4mPB2/fpC/ETVVfi8NvL\nzPpFYvsycPErT5tnFDNqmlDXE8oPqIX4/MMDHFxZvZdMLZlGMp3AmsxGMqNW2FvibznIJVe5EP8f\n5F9xkywp7Ul5T8o7EopEIOVpIb7j8cL6T+uzORP/+4YswT2VkC0kK8QKol1qs2dD9IrkILlMcpEs\nAbKHOPOkCrd9a5yscD4yKlEopVAiSy58QolHSaTdJrZrz6ZzrFvPunGsrKczjk55mjQiMqKYUHlE\n8oSKEyrOSHRUwWGUoBa3KylFFCEowSHMuWFMDUMsOfUH17J3LTdTy/XQsksWJgfzBN6AXyK0UqaE\nTn4cU64z8b9nZIFsFbGVEhTSCq5TTK1QtYop10xjzTxa/GgIgyGOipQFgjy1XaFvAVkWM/Sjo1Jg\nq4g1icpErPFUVcKacq7ZBNafeFYrz0oH1t7T7T32S4/Co/KE/vKA3g0YN6PFoduAeRbRPmPWQpVK\n6qSLy5g0h6ixSdGHLZ+5jlej5apS7HRiEIdLAynuIFZwvYd9D/1YKqe6j6834Zn43zeUlGYMrcav\nFX6jmTcas1aYjWbMDfPB4vYWv68IWhPRpKDI0w998R8AIoXkugJVfWXUVcLWM20909WRtg50TXnc\n1jNt56nXgWYdaLSnDoHmELAE9BSQPKN3A9V+oPITlXJUrad6nqh0Rg8CToOrcIvhKnAGQsXBb3jl\nWl5NFVdasSMxZIeLA8nvSuucfQ+7hfjjQvxwJv4ZX4OsKIUaG03YaNwzQ/VMY54Z9LNC/Om6ZrYV\nXlcEFtd/khLj+9FjUXxVlUwYXd8bVR2x3YGuTWw6x6YLbLuJTXdg0/U09YypAqaKGB0wPmIOoQTh\n3ERUdph5onITtZuolcO2gVpH6i7DJLhBl7r3Q40bahw1Llh8rtmHDVdzx5W2XImwy5EhzjjXk+Zd\nKbLQj3d2Vvwz3gVZCckqYqsIa4O/MMyfVKifVcgnhjHXTLbGaYvLhhA0cdLkXpF/DGXzTxVf11C1\nYNrbUbUBu050G8d2A8/XgeebkeebPc831zRmhJggRiTGUrJ4SrcdgVUOaJmx4krfQnG0baBZlX36\nNCvYadyuwpmaAy2HUOLtD6nlkNbsXMeNWHZJsQuJwTvcPJDGJftqXJojjPNZ8c94RyhuXf2w1vhn\nFeoTi7yo4Oe2uPraMmPxoSLMhtiXXm1PqTzzt8cDxTct2BVUK7ArVOeo1jPtxYHNM3h+Efj02cin\nz/Z8+uwNtfTEPhH6VMZpGQ9lVERM46naQN142jbQNZ6ujXQteCe4RoOpmKk5hJY304o3uuMqr9j7\nFYN0DNkyREXvE8PscNVAqgDUXVOEU4txKZX8ceBM/O8ZWZVOuEfFd88q5JMKfl6T/0zNmErOvQsV\nfjL43hBvNMmq2+y/jxqPKX61gnoDdoNqZ+ymp7uo2F7C808Cn15O/PKTPb+4vKLJB+bXmZnENGVm\nn5n3qZx7XYKf9LNI9SxidaRZJbo2sn4WWT3LzEE4GE2mwoWaw9xydVjxuVrzed6wCy0uF9ffOYUz\nCacdTkPSoazOxiWt92jHx2fFP+NtyHK3uBc2Bnlm4GeW/POa9LLU1JtCzTxbXF8RdprYLor/Y5zj\nHxXfbqC5QHUjdn1N96xicylcfur59NORX3y658++eEMddgxk+inTX0PvM/0hk78E/48yioz2UKlM\n3WVayXRtZvU8s/15ZkzCm6WykZtqDn3LG9vxudrwD9IFN6EhRUWSoyWSOJIEkiyrq0u6dNnCe3D8\nkeBM/A8NeXzMJz/PJ+cycnsuC9z2Uf8Kxz8i0t8mHsntMcfcea0QW2qOSZURm5AqINoj4ljhWMvM\nSiZWMtLJxEqNrNRAp3pq6YvonhQ3KY1JMj6BIKWWwfJPTQhRFF6EWQlTbhilYZCGnoZDbtjRcpNb\nrnPDLjUP/pjT2PsfD87E/xA4JXnmK2TPy88yQMool1FjwuwD9tpjV4raClYDCZrPBfulpro2mENE\njwnlE/IxzCE1qEpQ9tRAlmOtNYqElhktB/TxmB7trtmMI5eHz9jYVzT6DUZ2pDQwO0c/JXyG6TX4\nvnQZpgKzFuqfyRLiIOhnmrhRzFaTs2YeFfs3mhpNHxo+e7Xh1esVVzctu75mGCucV0+q/PV3jTPx\nfxfII8cPzn0lYDOB+IQeI+YQqa49dS3UGprSQ4v6c0392mCvDWZfoceIKulfTx6iQNVSOs50Ct3d\nH41SVCFR+bmMwVH5AyZUVKFiPQxc2lds9CtauUKnHdkPuMnRDwkn4A/geyFFEANmA7UIqoWUNbSG\n1BomWzEng4wVgoG5oncNr153vLrquLpp2B0sw2Rw/mnVvf+ucSb+74qH5F8YfhpZe4/8KSMuo8aI\n2Qequih9Q6YNiZyF5kuNfW2orqui+FNCfEI+gjmkqEXZO0W1VZgTq7aKSoR6TNSjox4cdiypR7WH\n2kEXB7bmDRu5oklXGL8jTQOudxx2CWsghdJdOEWQSjBrUK1QPYeQVWlbJpaganyy+NESZou/qTnM\nNVfXLVc3LVfXLbtDzTBVhfg/inDod8OZ+N8GD78fD8n/yM/vFD+jfEKPCX0IpQVfzjQh0U6RnKG5\nNtTXluo6UB1CUXz/cSg+ikL8lRSyX2rspcZeluOaTLvztLtAU3laAq0vY+M8bR5p2dHGHY3foacd\nuR+Ydw7VJVxdVF5MIb1UgmlBjCBG8FkxuIrZ1Uy+YXANw7iMruEw1uwONTeHMu4OR1f/rPhnfBsc\nCX8iyplHlD+BuMXV11DlTB0T9axp+0DOUB8sdikUYfbho5rjy0J80xWVt5ea+kUx+8LQElh9mVhV\nMx0jKz+yGka6PLLyA7UfMWnA+AEzDeh+INUDrnakJmHaMqev1kXpTQP69rHgsmbeG9LOMu1r9lPL\n9dhxs++43nccljn9MFUMY0W/HJ8V/4x3w9cVTX1QI/Je0mzKiF9c/ZypgsLOkaZXtDelDEczztRj\nTTUFzBhv5/jy9Hl/4uoL1YXCXirqF5rmpaF5aehyZlMl1sxsfM9m2LGudmzyjo3bU40D2Tny5MjG\nkasyusoxm4TeQPNJuaFKK1SVYDZlca/+RKGz4vCFIWGZ54Z97rgaV3zxZs0XX67Z72uc1/fN6bPi\nn/GeeFgp6pu+OwmUS2ikdHGZBdsLTQVttRDf11jvqbzHuID2ZY7/UUSGKVCWspC31cXVf1FI3/6q\nYpUiGxJb77gYDmxvrrkwV1zwmq27wowDThJOpUdH/SyTEaSF6nlx78uqvqL7PUGSRmNIs2W6adin\njqtxzW+vN3z22y27XU1KUizL3fFiPxWcif99I0OOkF0uW8M+37bVzgYgIDmgs8fkgFWBugq0JtA1\nkZBjadCQhZQUOQs5q9tz94IEHhvhqzUsT+taSi5dZm4to/LdY8iPF5ZZrFKKRlW0kmgk0qqKRsVy\nTiVaJjoZWdGzpmfDnm3ecZFuuEjX6DgyKZgy6KWzTV76DYiCnISEJokmKI03Gl8ZnNXoWjPFhlFv\n6NnQpzV7v+Jm6rgZOt7sWw57+919th8RzsT/XfEtiuAkyhfZZZgzjKk01VABJp1xJpN1QutArQNr\n7XhmHKOeqLMQQoX3hhAUwVeEYG7P5ai+SsrTUVEK+ZyauTvWKlLhsDhsXkb87WOV4v0isg9Mi6IK\nBjsbqoOhemOwraEyhgpDnWbqP31D/eWO+vpAfRip5xmbArVKaFuuQ6rSw05XYCqoKrAG8oXC/KyC\nbY1vagZpcL7m0NdUb2p63/Db65Yv9w1v+pb92DK6Fh9suTGeAZyJ/+3xdSR/cBN4OAvIuUSb+QSz\nlA9BLb/nNDidSE1E15Gm8axrz/NmJjQTddbMszDPhnnSzHPFPDUw18S5IXp9v85jXN74OB6J3yzW\nnhw3oIynzgMt0OVAR6LLjo6BLg+Y6EtF72NJ//nEBEQUOmrUZFAHjbo26Eqj0KhgqLOj/uKG+osd\n9rqn7kfs7KhjxKqMsSANqAZ0UxbvqgZsA3UDYa3Izy152xGaFU6tyGFF7lfkNx29a/jiuuLLXcV1\nX3GYKkZX4UJF+lGkN34YnIn/ofCWG8Hpgpwsz0u5KL7PMKe73Ju8hJ46k8hNwqwD9TqwXjn82sF6\nphHD0BuGITMMGt1bGBpiv8KprsTAH0u7H8ke+Krit8AKWN8fVTVjM3QEtnlimxNbHNs8sGWHdTOM\nlH4ex1Ev75MAUeSgyZMiHzTZaDKa7BWMmjp76uu+qP3NUfEdNgWsylQWVAt6BWYF1fo2cQ+/ArdS\nuM4yrzp8s2FWF8z+AtdvmbngMNVcXyve7IXrg7AfS9txH39ac/hvwjcSX0R+D/ibwM8pH+1fzzn/\nxyLyHPivgD8H/AnwF3LON9/htT4dPCT4w9qKJ0p/PCcnP04sip9LW2w55nnIckPQmVxH9DrSPPOs\nnnvysxnzzNBIxX5fU+1A7xU0lli1OLVC8hbElkKvijKevunR7Tgq/hq4ALbLeAHKjlgCqzyxzYrL\nnLjMjss8cMmOZp5K3c89ZTwlvYckQoiKMCnCXhFQZUoyasJeYQmF7IeRuh+w/YidZ+oYqFWm0qDb\nEo1XbcFvwV5A2EK4gKlVHIzFmRZvtgzqOYfwCfvhkoP7hMNYs7+O7HeJfR/ZT5HRJVyI5Py0Kt3+\nkHgXxQ/AX845/7GIrIG/KyL/PfCvAv9jzvk/FJF/B/grwL/7HV7r08RjBVV5ROlPxrSs67mFjHkh\nfBBIi+KnJqHXgfp5IH/qqH5maD81tGKx1x16laHRxKrCqZYxr1FhC7m5W6g7XtNR6eG+q7+iEP7y\nzlRjsHmiy4db4r/IMy/ywIu8oxuH8ruW8u05IT0ThCjMQZgnxYwwB8U8CvNeMdeCIVHP82KOeprv\nKb6tSsJeWEN8BvY5xMvFnoOuFS5ZSB0+bejTc679p1zNL3iTXrDva4Zrx7ifGXvHODpGN+ODI+WZ\nM/ELvpH4OeffAr9djg8i8veB3wP+OeCfXJ72XwB/xE+N+N9URTl/lfRQiB+OT1mUPkiZMpMzUWdy\nUxS/fuYxn3raXzriLzWd1OhVgAZipfCqFO+owhqZLyC2XyX90eU/neO33Cn+JfAp8AJUq7C5p8uW\ni6y4TIkX2fEyD7zMO9bD4Y70cI/0VOVmNkRhmIQhCMMoDEbo9bL1RirqHgN1CnfHsSzuWVv+hGoN\n6QLSJ5A+vTMqRT9VyNjipy3DeMl1+JQvxl/y+fSS3aHGX/e43YDvB9w44JzGx0zKx952Z7zXHF9E\nfg38PvC/Aj/POX8O5eYgIi8++NV9LHjL1tk90ue7wL7jqn5edsaCnDSKyiA6IU1CbwLmeUA+dcgv\nFepXik45aCKxyjilmXLFIbSYeYUaLsB3d9dwJL0+uZhTxT8l/gvgl6BWYNM1XbZs01HxHS/TwK/y\nju1h9xX3noky1zcwO9gF2HthD+wzmCy30xkFheAqF5OTY52pLaQW8hryM8ifQH4B+ReQfgHJKN7c\nWLjp8Gzo56L4Xwy/5B/d/F7Zp9/vSLsdqd+RJk2aMyl40o+iWumHwTsTf3Hz/zbwlxbl/7qZ7gP8\n0cnxrxf7keIbbgLHU8cWl4lC9sDSOiJldM7oHFE5ovAYNFoUWoSsJjo1sVIzoyymZiaZmWXGKHN/\nj/6uF8XtTSAvpePycrfJuuyVZwWtOFrlaPNMy0yXZ7o006mJLpX3ftC5+t6xptzUYjqxCCkVQ0FT\nCbWRMp9f4u2zUcTK4LeQN5BWkFsh1ZT+A8s1TzRMx7r3rqGfGvZDw27fcHPTst9Z6GfoLYwVzAa8\nhqj4cfQk+Dr8yWLfjHcivogYCun/Vs75D5fTn4vIz3POn4vIL4BXb3+FP/9OF/NTwzF1/3iPSIAs\nsfwyJOIuwJUmN45kFgFXA+r1gfp1Q/facPFGyDcJ3XuaaWJ23d12HksRDE1ReQqJopTMtjhB7CFd\nQ1xef93uWOXPqNMrdL4i5x0+D4zZsU+pKPsb4Aq4oSzyHTt2+7JFOVK4lnUpDGwpM4vSA1Noraaq\nFWI10Sqc1QxWk61CrxThGcQOghKiL70uo4YQhBu2/OlNxxc3ljc7xe4mMdw43M1A2u+gr2DYw9TD\nPIKbIfryB38E2Y2/G37NfVH9O2995rsq/n8O/D855//o5Nx/A/wrwH8A/MvAHz7ye2c8gresBwKQ\n0hLZM0TyLpIbjzJSvIMAURnkeo+9NqyuhXyd0DtP3U9sph7nGuSU+AIsTkAp9AlelnJxc8ltD2Zp\nvRmgqw+s0yua/AqTryDvCHlgyo5DTqSJQvijHbgjfigK7/Qd8bUGu9x8jAasYGtF1VSo2pCaClcb\ncmMIdUVuFL4RXANeCc6DPwgulOIbu7jm1b7li0PF1V6xOyTGg8PtB/JhB70ppJ96cCP4GcJPhfjv\njnfZzvsngH8R+Hsi8n9Svqt/lUL4/1pE/jXg/wX+wnd5oT9WfOUmkFhWyBJ5F8hGUEAKGTUlohLU\n3mD3im6f0HtPs59Y9z3ztCd6iyzBO5IpJa+gBNeY0gjGKZhjEUN3WJpth8ITWw2s8hV1vkJzp/hH\n4oeZQvbTLb2BMs/3kFMpPR91uckoC3bpl1FboBZUq5HWIG1Nai2utfjWMrWWaAyTwCzCJMIcYN4L\n00GYBfa+42rouBosb3phP0SGYcYPPWnYwagXpR/LeCR+jHztbPQnhndZ1f9fKM7iY/inP+zl/HTw\nkPDHqfJR8fMQyaaUk1IL6eUQSxLMoLBDQo+eZpiIy5c+jityKD79bbSgAHrJX6eo+yQwRZimwtcp\nwDTCtAejZ9Z5R5NvMOxuFX/Esc8J5ylEPw3gGbl19W/LBupCetWUmprSFKMV0kqTuorUWeKqwXcN\nqWtIqwYnhtEJwyyMTpZjbs/1c8Nu6thNlt2k2E2JcVrq3k8GZl3I7uYy+p+Sq//uOEfu/QB4OLe/\ndz5Bdgk1CJmAhEyaEnLQyHVAqYjMCTt7lJuQuUe5HWpukLlBkr5ru/3AUBAUDAJDpBAqwDDCoIuJ\neDoG6jygGSD3BIriKxJz4H6Y7mnYbiidfyuBSpf4+qqBqltsBawEt9K4tcGtLWnd4FYdbl1sShWH\nndDvhT4IBy/0B7k7N1gG3zE6y+AUg08MzuHcQPKUJpbB31n0Z1f/EZyJ/wPhreRfSnNlIoSMTAk5\nKMQGsAojARsDNk7YUGGjwYaKOlbYYNCikKPCH0l/fGzK/P6QoI9wCHDI0KdlzJByvE3IMTgyDp8d\nxw7w+riF91iSji9kb6HM6W2Jt7cdtJtieSMMG0XeVIRNTdo0uG3HsFkxbtYcvGWvhV1Q7A/C3gu7\ng7B/Ley/UAy9xoUaFy0uKFxMuOBwEVIIS/hjvKvNdXp8dvVvcSb+D4hHF/mWVX2ClNZQSha1ljJf\nF4ViwqJYZaFDsUJYZUWHwuoStSvL3F7JsUxVOT9n2HvYuzLuXDlul5p3Pt2l4eol0yeQSCQcqawb\nvC3zL0O97PFrDU1VXH27gm4D6wtIF0K+0PitYbqwxG2Du+gYt2t2F1t2U811EG564UYrrr1wc1Dc\nvBauPxOmnSJlRUrLmBMpO1IOpDQt/8glaODUjufOAM7Ef3/cBttTlG+mzHF7ykJXBjWCcqADmLy4\nvqYsbln74Du5vOa972ekbIYvb3j6dU3EUn1GgZby2lZBLdAK1PlE6WNJbZV8N+c3LG3mAgQHYYI4\nlzHM4Jby8Q+36uEk5m25EeWTG1I+3pgawa3AtTA3grXgKmHWUCkhSc0opbZ9T8NBGvY03NCwo+Em\n11xH4cYrrmfhZlTc9ML1Xri5Ucy7h3vxx6iIH1fd++8aZ+K/L46u7kgh+g33QlhFMuoKqj3YCZoI\nrYJ1A5sNNKpseZ16oTHenfsmUTpN8nGUhTp9Eho8pyUWIIIsc+6lvgaSyu/0DgYPky9E90vbt+P0\n42GdjlMTDblSZFuaf2Yr5JNRWSE1CtcIfaOIRhiTsJ8V9U6IsWbv1xzmNfuxZd83HPaW/Y1hf63Y\nT8L+M0X/ShivhHkHfoDohHyOtv1gOBP/fXEapnqg1IY+7nmERYl3GXNC/E7Dql6Ib+7WnXxYRs/t\nVti7EP+YdetYQny5uyGYhfRKuGvKs5BeUskIHDwMDuZQtvHCEmF3fO+HAX/6xEQJqRZSq8mdInV6\nsXIslSIpxSyaoBST0uikUJNCO02YLf3cMYwdfd/R7xuGnaXvKvqVpp8L6Q+3xBfcIETHmfgfEGfi\nvy8ShXFH9/74H4zlvCjQE1Qj1BM0CbpF8be5zHvnGZwDPYM7VspKJajmm3Ak/lKx654bHnKp5CNx\nIb0spM93XkCgKP0UyjjHEm0XHyj+MdjvQYGeEgRkFbHTpK0mbg1xq0nH0Rii14SgScGQvSYFTZoM\nOWicsoxjw9Q2jPuGsa0ZW8vUGMZWlW27q0L6I/GL4p+J/yFxJv77InNf8aGwaQaGXBrBRjAxY+Od\n4q9r2Bhoaqimsm+ullTZlEvU3LtEkufj85dsvpzv+sc5AZ0Wwsf7Sk8sbnrMxb13cQnaWRq9Hns+\nHol/JLt9YKKEYBVxpYhbQ7g0xMuKeFkRLg1eVYTB4HqDGwxuqJidwc3l3JwtbrDMtmaubTFrmWuD\nqxVTUMw7mG8K6YurL2dX/wPjTPz3xamrD/dIz76snisFlSqLbq0qir9qYKtKxxhjTkifykKb10uw\nzTfgqPjHjL+j2+9ZMvzSiXuv7pT+2LwyUVx7H++P8YRUR8U3FLLXizWU1whWCJ0mbDXhsiK8sIQX\nFfLCkqQiXVvmNxU9FYOr6FNFP1n6XcXsK3xl8NUymuNjg680PhaF90MZXX9W/O8CZ+K/L47EhzvS\nj/nWFxZb2r6bGmyTaWrozKL4TfEATkkfAnhXOke/C/GPlxCWlF6RMs8/pvQqOUkDXtSfZRdAZLlx\nLK59SneZdEfFPy7inSr+aXm+o+L7TuEvDP6yQr2okJc18rLGY0nWMmPpXc3NwXKdLTeT5WZvGceK\nqBRR6xNTd2Mq6h5dIXs4OT4T/8PhTPz3xZH4y5weyfeWvaUGvSmFJGrKYl6nT1b1ly/vKenneSkl\n/R6Kf1wOeKym3/HccXGP4yLfyWsctxIfjg/n+EfFb4EOUEpwVjCdRm8N6tKgXthC/F81qFQTqZld\nTb+vua5qvkw1X04NX+5qhr0hiyxtweWrlotLn9Ky2PnAzvgwOBP/vbFIZTo62UdHu8SuZudIEksa\naaXwtcGFmjl1zGwQlZl1xtuMryG0mRgyKWUSmTwv0puOxslxPr2Keyn/+cElHt19eGRP/nb/XW77\n1t/2r0cgCTkJMSlCFnwSXFLoJIgyeKlxi3llcarGqRqvaiZqRmqGXHNINfvQsPM1167mzVQzjuev\n3FPA+VN4b9yuq3MXvXNbIJuUHS7NDD6xmyuuxhWNfoZRpd5bo/eMLjHmyGQj4yoyVolpFYnPIul2\n5S0hvhzL8rjEpf6O0WcaqBRii3E7SjmXNdEbZm9IzuC9YXSGyhuMN4DBB4OfDGFv8G8MvtV4Y/AY\nDqni6jPL9auK/VXFsNPMgyK4cjM542ngTPz3xkPi35EeEikHXJwYfGbnDM3YYdQzIBOSoTE9noDH\n46uArzw+e3wOBDzJeRgCMviy4T4GGDyCL6twvyvxlSC1QlqNdI9YrohjTRpr3FgjY42Mtoy5JmEI\nAcIkhIMQ3siSzy+EIAxJc/Oq4uaV5XBlToivzq76E8KZ+O+Nh8S/Iz0EUk64OBfizxVGdUAiJMMU\nOxo7ku1MrhzpOFZlzLakkcrOkXdllN1cXj4kZPoAYalqUfZOI9sKtTXIYmpbkXNNPLTEfUfcL6Pu\niLkl+o6IJoZEGCNxn4imxPLHEAljYkrC/qricGXYXxmGnWEeNMFJKb11xpPAmfjvjVPi3yc9OFIG\nFx1DSJjZAF0hfeg4OEfTzujViDITqprQ6xG1msq51YTyE/JmRBoDpry+hFQibj5EIxhFce9XSXjV\nuwAACWNJREFUupD+skJd2sWqQvDrNfObDbNdM+s1Lm+Y/Zp5XOOzJgVPnDxx70h4YvDE0ZH2njln\nhp1hvNEMO814T/HPrv5TwZn4741T4sMp6WEiZY2LmcEDVCekh+s500RPXQ3YVY+1A3Y1YJ8P2Oc9\n9vmACQPSaNRCekIiTwEOvizC/Y6QW8VfVP7Sol7UqBc1+oUl5RWxu2C2Fwz6gj5f0PsL+umCvrrA\neUUKE3mcSIykMJGmibQfSW8mfI7MQyH7PGjmXt0q/tnVfzo4E/+9cSQ+3Cd9iWZPucLFshEWUsUU\nKg7eYnWFVZY2B9rVgY4DnT3QrfZ0zw+kn9eon1tUNMhCehUSMgXywZcFuA9A/FL9cnH1L6pb4uuX\nDfplQ8hrot0y62f0+ZIbf8nNeMnucMmNec7kNTkcyFNPDj15PJD3PckYshFiDgQnBFcW9PzJ8Vnx\nnw7OxH9vHIl/JL3cs5RrXFwtSm9QsrpnrUpsnu/Y5h2baodft6TLBvVzi/2zBpMUCsghkadIPjjk\n2lAS7T8E8bmd46utQV1W6BcW/bJB/apF8oqot7j8nN7/jJvxZ1wdPuX19c94bT5lREHYQbgh5x1g\nybctdSI5uWUPXh4dz3gaOBP/W+ErO+f3fhKzJebbEBvugl9bpimRZk/2JUVPRY9KHoPHiCMrhzo1\nmVHibk2QR0v3P7yax/LpS3BOhaZerEFLMbWMt7nyqqWXhoO07KVlJx07WsasILmSxJ9mSDVEWypr\npuocZfOR4Ez8D46HibMTd03mSmmt6HrCMDLvJsYrh24CyhTCzFkhnxnUqwp1VaN2HTJElMuoJMtr\n8ij5T/EY6QFUMmjXoIYGvWtQVw26aVCm3AwmKt78VrH7PHN4HRlvHPNhJEwHUrCltFXaQ+4hj5CX\nKpv5XNPuY8KZ+B8cx8z443bfsTlW+dmR+H4YmHczpnEoE4BECmCzQl5p1CuLXNWoXUSGjHJSCmni\n39am7x7kkWMBJGmUq1FDjd7VqKZGGYuiRgWLw3DzpWL3RaZ/HRhvZtxhIE6WHHTpxpF7SD2ksah+\nPsYwn4n/seBM/A+Or5TKWM6XNYGcILoRP0y43YQypXJlCok4gckKdWWQqwq5apBdRgZBOYWkimO7\nza8j/RGPkj9plKtQg0XtKsRYFBYVKtRkcVT0b4TDdebwJjDeOFw/EiZDDhTFz2Mh/VHxz8T/6HAm\n/gfH2/b5y7mi+A4/zChT6lOnEAhTwh3AIMjOIDcW2WXYCTJoxBkk1QjhnUgPX83vF4CkUc4gg0GZ\nCsEgoUJNBjkYAoZxrxj3mWEfGfcz80ETJiH7uBB/LpbmO+LnM/E/JpyJ/8FxqvhwvzKnWRTf44dy\nLgVPmAL+kJiuQaOQQcNQIYMgvYbBIM4iKXBaVPJdaXZP+ZNCnC43EzQSNDKZpW6/JlIxj4p5zLgh\nMI8ONwphSuTgl7f3C9lP7Kz4HxXOxP/gOJ3jn/apLvP9QvySWJtCJEwRd4hom9AWBIU4A04Qp8EZ\ncLYk6qSlN9bvAEkCTpX3CQomhRyOyTpSYvG9IrhMcKHUB3SJ4D0pTMvbx2Mp4AfjmfgfC87E/+A4\n3ec/1sq5s0L8TAqZMGVEnVohflk517epuHIvLfd3JFei1OgKgkzcS88tAUJCSoqcMjkHUkrk5Mlp\nCbnNy994rAX+FTvjY8CZ+N8JvoYEedn5emu+zekO/HeAh5U83unJZ/zY8CHSPs4444yPDGfin3HG\nTxDfSHwR+T0R+Z9E5P8Wkb8nIv/Wcv4PROQfisj/sdg/891f7hlnnPEh8C5z/AD85ZzzH4vIGvi7\nIvI/LD/7Tc75N9/d5Z1xxhnfBb6R+Dnn3wK/XY4PIvL3gT+z/PicZ3nGGR8h3muOLyK/Bn4f+N+W\nU39RRP5YRP5TEbn4wNd2xhlnfEd4Z+Ivbv7fBv5SzvkA/DXgH8s5/z7FIzi7/Gec8ZHgnfbxRcRQ\nSP+3cs5/CP9/e/cPWlcZh3H8+3TooFMptAFrrdBZRKlLHCpCEZcUBy11qA7iUOiquGRVt86lQgVL\nqYNaF/8sKhZKM6Ra0dpCOcH6JwZRMJvDr8M5SU/Se++5luZ9X3ifD1xy7sm9eX95k+fc+957kh9E\nxErvJqeAT8d/ha962/u6i5ndX013GTbtCTzvAT9GxMm1HZJmuvU/wAvAD+PvfnDKYczs3u1j44Pq\n12NvORh8SbPAy8BVSYu0p3O9BRyV9DjtSaAN8Pq9lmtmaU3zqv5F2r8y2eyz+1+OmaXgM/fMKuTg\nm1XIwTerkINvViEH36xCDr5ZhRx8swo5+GYVcvDNKuTgm1XIwTerkINvVqEMwW/SD/m/NLkLGNDk\nLmBAk7uACZrcBQxoko3k4N+lyV3AgCZ3AQOa3AVM0OQuYECTbCQ/1TerkINvViFFbG2jQ0nupGiW\nSUSM/Bf4Wx58MyuPn+qbVcjBN6tQsuBLek7SNUnXJb2RatxpSWokfSdpUdLlAuo5LWlZ0ve9fTsk\nfSHpZ0mf5+xeNKa+Yhqpjmj2eqLbX8Qc5m5Gm2SNL2kbcB14FvgNWACORMS1LR98SpJuAk9GxN+5\nawGQ9DSwCrwfEY91+94B/oqId7uD546IeLOg+uaBf0topCppBpjpN3sF5oBXKWAOJ9T3EgnmMNUj\n/lPAjYhYioj/gHO032RJREFLn4j4Fth8EJoDznTbZ4DDSYvqGVMfFNJINSL+iIgr3fYq8BOwh0Lm\ncEx9yZrRpvpFfwj4pXf9Fne+yVIE8KWkBUmv5S5mjF0RsQzrXYx3Za5nlOIaqfaavV4Cdpc2hzma\n0RbzCFeA2Yh4AngeON49lS1dae/FFtdIdUSz181zlnUOczWjTRX8X4G9vet7un3FiIjfu48rwEe0\ny5PSLEvaDetrxD8z17NBRKzEnReNTgEHctYzqtkrBc3huGa0KeYwVfAXgP2SHpG0HTgCXEg09iBJ\nD3RHXiQ9CBxiYhPQZMTG9d4F4JVu+xjwyeY7JLahvi5IawYaqSZxV7NXyprDkc1oe5/fsjlMduZe\n97bESdqDzemIeDvJwFOQ9Cjto3zQ9hP8IHd9ks7SthneCSwD88DHwIfAw8AS8GJE/FNQfc/QrlXX\nG6muracz1DcLfANcpf25rjV7vQycJ/McTqjvKAnm0KfsmlXIL+6ZVcjBN6uQg29WIQffrEIOvlmF\nHHyzCjn4ZhVy8M0qdBtlANUbvZQbIAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x103402f98>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"from scipy import io\n",
"from sklearn.preprocessing import scale\n",
"import matplotlib.pyplot as plt\n",
"import time\n",
"\n",
"datasetDir = \"datasets/MNIST/\"\n",
"datamat = io.loadmat(datasetDir + 'train.mat')\n",
"test_images = io.loadmat(datasetDir + 'test.mat')['test_images']\n",
"train_images = datamat['train_images']\n",
"labels = datamat['train_labels']\n",
"train_images = np.transpose(train_images, (2, 0, 1))\n",
"plt.imshow(train_images[0])\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"train_images = np.reshape(train_images, (train_images.shape[0], 784))\n",
"indices = np.random.permutation(labels.shape[0])\n",
"train_images = train_images[indices].astype(float)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In data pre-processing it is common to scale the dataset so that each feature(column) are standardized."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"train_images = scale(train_images, axis = 0, copy=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Conver the label vector to a matrix. The label for each sample is now represented by a $10$-dimensional vector with a $1$ at the index position corresponding to its class and $0$ elsewhere. "
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"labels = labels[indices]\n",
"train_labels = [np.zeros((1, 10)) for i in range(labels.shape[0])]\n",
"for array, label in zip(train_labels, labels): \n",
" array[0][label] = 1\n",
"train_labels = np.array(train_labels)\n",
"test_images = np.transpose(test_images, (2, 0, 1))\n",
"test_images = np.reshape(test_images, (test_images.shape[0], 784))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And now we are ready to train our neural network! Let us first split our training dataset so that $9/10$th of it is used for training and $1/10$th of it is used for validation error. Let us train a model using mean square error as our loss function."
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAaMAAAEZCAYAAAA6xErnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8VOV97/HPFxDQCAiJYARRjKBgYqJpyEU92Ympl6ZV\n01bEmIqRJk20rW2apJCeFEybi572hPSk+jpJjCJJJMQ0SqxHidGdi1caQUhEpWkhgLC9IGi8cdm/\n88d6RtYeZvbewFzWMN/36zWv/cyzbr+1Zu/9m+cyaxQRmJmZNdOAZgdgZmbmZGRmZk3nZGRmZk3n\nZGRmZk3nZGRmZk3nZGRmZk3nZLQfkjRA0vOSxtVyXWsuSV+XNKsJx71P0vGNPu7+TNLNkk5rdhxF\n4mRUACkZPJceOyW9mKu7YE/3FxHdETEsItbXct09JekfJG3Lndvzkp6s9XEaIZ3Ldak8UFK3pPF1\nPN5MSXfn6yLiIxHxpXods0oc5wJPRcSvGnncNnAl8PlmB1EkTkYFkJLB8IgYDqwF3p+ru7F8fUkD\nGx/lXvtW6dzSOY2utFKlc9rT81Syt4H2Q+kT4sqV91g/z2ufjlFDHwMWNDuIVtKf1zci7gNeJ+mE\nBoTUEpyMikfpsasie1e+UNJ3JG0FLpT0jtR98qykDZK+UvojKH/nLmlBWn5baqHcI+nIPV03LT9L\n0mPpuP8i6eeSLtrjk9x13I9LWg2sqlSX1j1F0tJ0zPslTc3t52eSPifpXuC3wBFlx/mMpBvL6v5V\n0j+l8kxJ/53O9T8lTetH+D9JPx9J230g7etsSctTnD/Nd21JWifpk5JWpDiR9HeSfp32sVLSH6T6\nNwL/Bzg135pMr83f5/b5MUmrJT0l6d8kHVZ2bT+alj8j6Su57SZK+omkLZKelPStKq/REKAjd76l\n38Ub0+/i85KWSTo6ncuTktZIem9u/RGSvinpCUm/kXRFbtkxku5K8T0p6QZJw8qu2V9LWpGu6bcl\nHdDbCyPpUEn/ntZ/RlJnbtlbU7xbU/yLStdTZS3RCn8Xf5Dbdo2k/5lb9w1p3YslrQXuSPUna9ff\n6EOSTi0L96fA+3s7n7YSEX4U6AH8N/Desrp/AF4Gfi89HwK8FXgbWeI6CngUuDQtHwjsBMan5wuA\nJ4ET07KFwA17se5o4Dng99OyvwZeAS6qci7/AHyzyrKBQDdwGzAinVOlutcCW4BpZG+ePgQ8DYxI\n+/kZ8F/ApLT9gLLjTEgxH5g7blc6v2Fp30enZWOA4/o6l1ycR+SWvw3YCJyUXpOLgf8EBqXl64Cl\nwOuBIanuj4HRqTwdeB44ND2fCdxVFsMC4O9T+XRgE/AmYDDwr8CPy+L7AXAwcCTwDOn3ClgEfCqV\nBwPvrHLOJwDPVrgOLwDvSa/Ht9P1/3R6/jHg8dz6i8kS6xDg0HQNPpyWTUz7GQi8Lr2WV+W2XQfc\nm7YbCTwGXNLH389VwL+kWAYBp+TOcx1wWTre+cC23PXscb3Z/e+iA5icym8i+xsp/T2+IV3va4Gh\n6VzHkf2evi/3ej0FjMwd41PAwmb/zynKwy2j1vHziLgNICJeiYhfRMTSyKwBvg68O7d+eXfVTRGx\nLCJ2kv0DecterPt+YFlE3BoROyPiy2T/5HpzoaTNuccdZcs/HxFbI+KVKnV/APwyIhZFNr71LbJ/\nfvl3lN+MiMdTTN35nUfEfwO/BM5JVacDmyNiWXreDbxJ0pCI6IqIR/s4n7z8dfsIcHVEPJRek+tT\n/dty68yLiI2lc42ImyLiyVReCKwBfqefx/4g8I2IWBkR24BZwLslHZ5b5wsR8duIWAt0sut13A4c\nJenwiNgWWZdRJYeQJchynRFxd7rW3yP7B3tVer4QeIOkgySNBd4HfCL9zj4FfAW4IJ3z6rSfnRHx\nNDCPnr/DAF+OiKci4lngVnr+3layHTgcOCoidkTEz1P9yUB3RPxrOt53gWVV95J59fWNiM6IWJXK\nK4HvlsUaZInt5fT6XgTcEhF3pm2WAA8DZ+a2eZ7sGhvupmsl6/JPJB0r6VZJG5V13V1B9u6ymk25\n8otk75j3dN3Dy+MA+pr48O2IGJV7nNGP7fN1h5ONo+WtBcbmnpfHVO5G0j/A9PM7ABHxfHr+58Am\nSYslTexjX9UcCfxtLuk+CxxWFmePc03dOstz6x9L769hXo/rks7l2bLjdeXK+dfxE2Qthf+Q9LCq\nd7M+S9Z6LJff70tk7/jzz0nHGk/WSujKneNXyVo6SBoj6buS1kvaAlzP7udf7Ryq+SLwG+DHqYvy\nk6n+cHb/XSv/vapK0jsl3Z26E7eQtaR6xBoRG3JPjwQ+WPb78PYUR0mpZW44GbWS8sHs/wusJOti\nGgHMYfcWTq1tpGxMhp7//PZGpUH6fN0TZN2QeeOB/B9+XwP9i4D3pVbDOaRkBBARd0TE75Iljl+T\nXde9iXkdcEUu6Y6MiIMj4qZK20maAFwN/FlpfbJuKJWvW8UTZP/wSvsbRtaV1Z8ZlF2Rzcw7nCwR\nf025ccGcx4Ahkg7ta59VrANeKLsmh0TEiWn5lWTdz8dHxCFkXZv79DucWoKfiIgJwLlkbxBOpfLv\nbn425AvAQbnnr6fna3AjWStwbIr12j5iXUfWYs+f+7CI+OfcOpPJWkuGk1ErGwZsjYiXJE0G/qwB\nx7wVOFHS+9MA71/R/3fy+3LMKZLOS8f8IFkf/b/3dwcR0QXcQ/bO+9GI+DWApMMk/b6kA4EdZP+Q\ndvZjf91k4wFH56q/Dlwm6XfSvg/O7buSg8m6CJ9O5/UR4Ljc8i5gnKRBVba/EZgp6Y1posEXgZ9G\nxMa+4k/XsvQOfWuKY7fzTt1/d7F711mfh0jbrwd+IumfJQ1T5g25gfxhZNf8eUlHAJ+ssr/+Hzi7\n5qXX5Xmy17Ub+Hm2WJem6z2NbHyv5GHgBEnHp9fs7+npYLLxs+2S3kE2xrfbOecsAD4g6X3KPss3\nVFKH0iST5H8A/2+vT3Y/42RUPP2dzvs3wMWSngOuIeurr7afvvbZr3XT+Mb5wJfJ/hlPIOt3f6Xa\nNmRjRvnPGT0naWQvx+pRl8YSziYbE3kauJxs6vvWvuIt8x3gNLIxsJKBZIPIT5B1Nb2TbIC7P+YA\nN6YumHMj4gHg48A1kjaTTSi5sJfzWkk2sL80HX8icH9ulR8Bq8m6uJ4oP3hE3AF8DriZrJU4rrfj\nlT1/O7BU0vPATWQTX6q1qL5GNv6xJ/LH+hDwGuARYDNZK3VMWjYnxbIlnUe+FVnpHPrjWOCudG4/\nIxunuycl1g8AH01xnEM2wSM7UDYe9AWymYOryM0gTD4OfCl1ic8iGzOqGmsap/sA8Fmy3601ZN2j\nAyDr9gOejojle3GO+yVF1O+jDJImkb1oQfbO4WiyF2dBqj+S7EWaVvrnImk2cAnZO5rL08Afkk4i\ne2c7FLgtIv4q1Q8GbiCbXfY0cH5E/KZuJ2WvkjSA7B/pH0XEPc2Ox+pD0j3AR2M/++CrpAXA6oj4\nXBOOfTPw1dIEB6tzyyjNcDoxIk4iSxYvkL0bmQXcGRHHknUDzAaQNIVsCu9k4CzgaunVDzFeA8yM\niEnAJEmlgfCZZLOjJpLNxrmqnufU7iSdoeyzI0PIujK2AQ82OSyro4g4eX9LRM0WEec6EfXUyG66\n9wG/joh1ZE3k+al+PtlAI2TdMQvTlMw1ZN0UU1M/67CIWJrWuyG3TX5fN5F1xVj9nEI2tboL+F3g\n3IjY3tyQrJ1I+p/qeQut0uOWPdxVEe5wYUm1wdF6OJ9ds5jGpEFlImKTpNItYsYC+c88bEh1O+g5\nS2g9u2ZxjSVN7Y2Inco+VT4qIjbX5zTaW0R8lqyr1awpIuIfgX+swX72+M4hVj8NaRkpu4XH2WRT\nI6H3wdV9PlwN92VmZg3QqJbRWcAv0swoyGYIjYmIrtQFV7qT8wZ6fhZgXKqrVp/f5gll92YbXqlV\nJMlNcjOzvRARdX+T36gxowvIPhdRspjsA24AM4BbcvXTJQ1OHwo8BngwIjYBWyVNTRMaLirbZkYq\nn0c2IaKiKMD9l/b2MWfOnKbH4PibH0e7xe74m/9olLq3jCQdRDZ54aO56iuBRZIuIbslxzSAiHhE\n0iKyzyRsJ/v8Q+lqXEbPqd23p/prgQXK7vL8DLt/GM3MzAqu7skoIl4k3YsqV7eZLEFVWv+LZJ8m\nL6//BdndcsvrXyElMzMza02+A0OL6OjoaHYI+8TxN08rxw6Ov13U9Q4MRSIp2uVczcxqRRKxH01g\nMDMzq8rJyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzM\nms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7JyMzMms7J\nyMzMms7JyMzMmq7uyUjSCEnfk7RK0q8kvV3SSElLJD0m6Q5JI3Lrz5a0Oq1/eq7+JEkrJD0uaV6u\nfrCkhWmb+ySNr/c5mZlZbTWiZfQV4LaImAy8GXgUmAXcGRHHAncBswEkTQGmAZOBs4CrJSnt5xpg\nZkRMAiZJOiPVzwQ2R8REYB5wVQPOyczMaqiuyUjScODUiLgOICJ2RMRW4BxgflptPnBuKp8NLEzr\nrQFWA1MlHQYMi4ilab0bctvk93UTcFodT8nMzOqg3i2jCcDTkq6T9JCkr0k6CBgTEV0AEbEJGJ3W\nHwusy22/IdWNBdbn6tenuh7bRMROYIukUfU6ITMzq71BDdj/ScBlEfEfkr5M1kUXZeuVP98XqrZg\n7ty5r5Y7Ojro6Oio4WHNzFpfZ2cnnZ2dDT+uImqZB8p2Lo0B7ouIo9PzU8iS0RuAjojoSl1wd0fE\nZEmzgIiIK9P6twNzgLWldVL9dODdEfHx0joR8YCkgcDGiBhdIZao57mame2PJBERVd/k10pdu+lS\nV9w6SZNS1WnAr4DFwMWpbgZwSyovBqanGXITgGOAB1NX3lZJU9OEhovKtpmRyueRTYgwM7MWUteW\nEYCkNwPfAA4A/gv4MDAQWAQcQdbqmRYRW9L6s8lmyG0HLo+IJan+rcD1wFCy2XmXp/ohwALgROAZ\nYHqa/FAeh1tGZmZ7qFEto7ono6JwMjIz23P7RTedmZlZfzgZmZlZ0zkZmZlZ0zkZmZlZ07VVMuru\nbnYEZmZWSVslo507mx2BmZlV4mRkZmZN52RkZmZN11bJaMeOZkdgZmaVtFUycsvIzKyYnIzMzKzp\n2ioZuZvOzKyY2ioZuWVkZlZMTkZmZtZ0bZWM3E1nZlZMbZWM3DIyMysmJyMzM2u6tkpG7qYzMyum\ntkpGbhmZmRWTk5GZmTVdWyUjd9OZmRVTWyUjt4zMzIrJycjMzJqu7slI0hpJD0taJunBVDdS0hJJ\nj0m6Q9KI3PqzJa2WtErS6bn6kyStkPS4pHm5+sGSFqZt7pM0vlos7qYzMyumRrSMuoGOiDgxIqam\nulnAnRFxLHAXMBtA0hRgGjAZOAu4WpLSNtcAMyNiEjBJ0hmpfiawOSImAvOAq6oF4paRmVkxNSIZ\nqcJxzgHmp/J84NxUPhtYGBE7ImINsBqYKukwYFhELE3r3ZDbJr+vm4DTqgXiZGRmVkyNSEYB/EjS\nUkl/murGREQXQERsAkan+rHAuty2G1LdWGB9rn59quuxTUTsBLZIGlUpEHfTmZkV06AGHOPkiNgo\n6VBgiaTHyBJUXvnzfaFqC771rbksTW2rjo4OOjo6anhYM7PW19nZSWdnZ8OPW/dkFBEb08+nJN0M\nTAW6JI2JiK7UBfdkWn0DcERu83Gprlp9fpsnJA0EhkfE5kqxTJs2lw98oEYnZma2Hyp/o37FFVc0\n5Lh17aaTdJCkg1P5NcDpwEpgMXBxWm0GcEsqLwampxlyE4BjgAdTV95WSVPThIaLyraZkcrnkU2I\nqMjddGZmxVTvltEY4AeSIh3r2xGxRNJ/AIskXQKsJZtBR0Q8ImkR8AiwHbg0IkpdeJcB1wNDgdsi\n4vZUfy2wQNJq4BlgerVgPIHBzKyYtOt//f5NUnzrW8GFFzY7EjOz1iGJiKg6Fl8rbXUHBnfTmZkV\nU1slI3fTmZkVk5ORmZk1XVslI3fTmZkVU1slI7eMzMyKycnIzMyarq2SkbvpzMyKqa2SkVtGZmbF\n5GRkZmZN11bJyN10ZmbF1FbJyC0jM7NicjIyM7Oma6tk5G46M7NiaqtktG1bsyMwM7NK2ioZvfRS\nsyMwM7NK2ioZvfxysyMwM7NK2ioZuWVkZlZMTkZmZtZ0TkZmZtZ0bZWMPGZkZlZMbZWM3DIyMysm\nJyMzM2u6tkpG7qYzMyumhiQjSQMkPSRpcXo+UtISSY9JukPSiNy6syWtlrRK0um5+pMkrZD0uKR5\nufrBkhambe6TNL5aHG4ZmZkVU6NaRpcDj+SezwLujIhjgbuA2QCSpgDTgMnAWcDVkpS2uQaYGRGT\ngEmSzkj1M4HNETERmAdcVS0IJyMzs2KqezKSNA74PeAbuepzgPmpPB84N5XPBhZGxI6IWAOsBqZK\nOgwYFhFL03o35LbJ7+sm4LRqsTgZmZkVUyNaRl8GPgVErm5MRHQBRMQmYHSqHwusy623IdWNBdbn\n6tenuh7bRMROYIukUZUCeflliKi0xMzMmmlQPXcu6f1AV0Qsl9TRy6q1TBGqukBz+exnYdAg6Ojo\noKOjt5DMzNpPZ2cnnZ2dDT+uoo5NBUlfAD4E7AAOBIYBPwB+B+iIiK7UBXd3REyWNAuIiLgybX87\nMAdYW1on1U8H3h0RHy+tExEPSBoIbIyI0WWhICmGDw/WroVDDqnbKZuZ7VckERFV3+TXSl276SLi\nMxExPiKOBqYDd0XEnwA/BC5Oq80AbknlxcD0NENuAnAM8GDqytsqaWqa0HBR2TYzUvk8sgkRFR14\noKd3m5kVUV276XrxJWCRpEvIWj3TACLiEUmLyGbebQcujV1Nt8uA64GhwG0RcXuqvxZYIGk18AxZ\n0qto6FBPYjAzK6K6dtMViaQ47rjg+9+HKVOaHY2ZWWvYL7rpiubAA90yMjMroj6TkaSBkv6pEcHU\n29ChHjMyMyuiPpNR+uzOKQ2Ipe7cMjIzK6b+TmBYlu4r9z3ghVJlRPxbXaKqEycjM7Ni6m8yGko2\nU+29uboAWi4ZuZvOzKx4+pWMIuLD9Q6kETy128ysmPo1m07SOEk/kPRkenw/3QC1pbibzsysmPo7\ntfs6sjsdHJ4eP0x1LcXJyMysmPqbjA6NiOvSVzvsiIjrgUPrGFddeGq3mVkx9TcZPSPpQ+kzRwMl\nfYhsQkNLccvIzKyY+puMLiG7f9wmYCPwx0DLTWpwMjIzK6Y+Z9Olr2X4w4g4uwHx1JWndpuZFVN/\n78BwQQNiqTtP7TYzK6b+fuj1HklfBb5LzzswPFSXqOrE3XRmZsXU32T0lvTzc7m6oOcdGQrP3XRm\nZsXUnzGjAcA1EbGoAfHUlbvpzMyKqT9jRt3ApxsQS925m87MrJj6O7X7TkmflHSEpFGlR10jqwMn\nIzOzYurvmNH56edluboAjq5tOPXlMSMzs2Lq7127J9Q7kEbwmJGZWTH12k0n6dO58nlly75Qr6Dq\nxd10ZmbF1NeY0fRceXbZsjNrHEvduZvOzKyY+kpGqlKu9Lzw3E1nZlZMfSWjqFKu9Hw3koZIekDS\nMkkrJc1J9SMlLZH0mKQ7JI3IbTNb0mpJqySdnqs/SdIKSY9LmperHyxpYdrmPknjq8Xjbjozs2Lq\nKxm9WdJzkp4HTkjl0vM39bXziHgFeE9EnEh2F4ezJE0FZgF3RsSxwF2kLkBJU8juDj4ZOAu4WlKp\nBXYNMDMiJgGTJJ2R6mcCmyNiIjAPuKpaPAccAN3dsGNHX5GbmVkj9ZqMImJgRAyPiGERMSiVS88P\n6M8BIuLFVBxCNnsvgHOA+al+PnBuKp8NLExf4LcGWA1MlXQYMCwilqb1bshtk9/XTcBp1WKRPG5k\nZlZE/f3Q616TNEDSMrLvQvpRSihjIqILICI2AaPT6mOBdbnNN6S6scD6XP36VNdjm3SH8S29fSDX\n40ZmZsXT3w+97rV0O6ETJQ0HfiDpePZi/GkPVJ1YMXfuXLZtgy98Ac45p4OOjo4aHtbMrPV1dnbS\n2dnZ8OMqopZ5oI+DSZ8FXgT+FOiIiK7UBXd3REyWNAuIiLgyrX87MAdYW1on1U8H3h0RHy+tExEP\npC8C3BgRoyscOyKCSZPg1lth0qSGnLKZWUuTRETUffZ0XbvpJL2uNFNO0oHA7wKrgMXAxWm1GcAt\nqbwYmJ5myE0AjgEeTF15WyVNTRMaLirbZkYqn0c2IaIqz6gzMyueenfTvR6Yn76GYgDw3Yi4TdL9\nwCJJl5C1eqYBRMQjkhYBjwDbgUtjV9PtMuB6YChwW0TcnuqvBRZIWg08Q88P6u7GY0ZmZsXT0G66\nZip103V0wJw58J73NDsiM7Pi2y+66YrIU7vNzIqn7ZKRu+nMzIqn7ZKRJzCYmRVPWyYjd9OZmRVL\nWyYjt4zMzIql7ZKRx4zMzIqn7ZKRW0ZmZsXTlsnIY0ZmZsXSdsnI3XRmZsXTdsnI3XRmZsXTlsnI\n3XRmZsXSlsnILSMzs2Jpu2TkMSMzs+Jpu2TklpGZWfG0ZTLymJGZWbG0XTJyN52ZWfG0XTJyN52Z\nWfG0ZTJyN52ZWbG0ZTJyy8jMrFjaLhl5zMjMrHjaLhm5ZWRmVjxtl4yGDoVt2yCi2ZGYmVlJ2yUj\nCQYP9iQGM7MiqWsykjRO0l2SfiVppaS/TPUjJS2R9JikOySNyG0zW9JqSasknZ6rP0nSCkmPS5qX\nqx8saWHa5j5J4/uKy111ZmbFUu+W0Q7gExFxPPBO4DJJxwGzgDsj4ljgLmA2gKQpwDRgMnAWcLUk\npX1dA8yMiEnAJElnpPqZwOaImAjMA67qKyhP7zYzK5a6JqOI2BQRy1P5t8AqYBxwDjA/rTYfODeV\nzwYWRsSOiFgDrAamSjoMGBYRS9N6N+S2ye/rJuC0vuJyy8jMrFgaNmYk6SjgLcD9wJiI6IIsYQGj\n02pjgXW5zTakurHA+lz9+lTXY5uI2AlskTSqt1g8vdvMrFgGNeIgkg4ma7VcHhG/lVQ+l62Wc9tU\nbcHcuXMBePZZuOeeDt74xo4aHtbMrPV1dnbS2dnZ8OPWPRlJGkSWiBZExC2pukvSmIjoSl1wT6b6\nDcARuc3Hpbpq9fltnpA0EBgeEZsrxVJKRj/+MUyZsq9nZma2/+no6KCjo+PV51dccUVDjtuIbrpv\nAo9ExFdydYuBi1N5BnBLrn56miE3ATgGeDB15W2VNDVNaLiobJsZqXwe2YSIXrmbzsysWOraMpJ0\nMnAhsFLSMrLuuM8AVwKLJF0CrCWbQUdEPCJpEfAIsB24NOLVj6deBlwPDAVui4jbU/21wAJJq4Fn\ngOl9xeUJDGZmxaJok1sRSHo1r51/PvzhH2Y/zcysOklERNWx+FppuzswgFtGZmZF05bJyGNGZmbF\n0pbJyHdgMDMrlrZNRm4ZmZkVR1smI3fTmZkVS1smI7eMzMyKpW2TkceMzMyKo22TkVtGZmbF0ZbJ\nyGNGZmbF0pbJyN10ZmbF0rbJyC0jM7PiaMtk5G46M7Niactk5JaRmVmxtG0y8piRmVlxtG0ycsvI\nzKw42jIZeczIzKxY2jIZuZvOzKxY2jYZuWVkZlYcbZmMSt10bfKN62ZmhdeWyWjQIBgwALZvb3Yk\nZmYGbZqMwONGZmZF0tbJyONGZmbF0LbJyNO7zcyKo67JSNK1krokrcjVjZS0RNJjku6QNCK3bLak\n1ZJWSTo9V3+SpBWSHpc0L1c/WNLCtM19ksb3NzZ305mZFUe9W0bXAWeU1c0C7oyIY4G7gNkAkqYA\n04DJwFnA1ZKUtrkGmBkRk4BJkkr7nAlsjoiJwDzgqv4G5m46M7PiqGsyioifA8+WVZ8DzE/l+cC5\nqXw2sDAidkTEGmA1MFXSYcCwiFia1rsht01+XzcBp/U3NicjM7PiaMaY0eiI6AKIiE3A6FQ/FliX\nW29DqhsLrM/Vr091PbaJiJ3AFkmj+hOEx4zMzIpjULMDAGr50VP1tnDu3Lmvll98sYOXX+6o4aHN\nzFpfZ2cnnZ2dDT9uM5JRl6QxEdGVuuCeTPUbgCNy641LddXq89s8IWkgMDwiNlc7cD4Z/fKXbhmZ\nmZXr6Oigo6Pj1edXXHFFQ47biG460bPFshi4OJVnALfk6qenGXITgGOAB1NX3lZJU9OEhovKtpmR\nyueRTYjol/Hj4c479+JszMys5hR1vEGbpO8AHcBrgS5gDnAz8D2yFs1aYFpEbEnrzyabIbcduDwi\nlqT6twLXA0OB2yLi8lQ/BFgAnAg8A0xPkx8qxRL5c92yBd78Zrj6anj/+2t62mZm+w1JRESvQyA1\nOU49k1GRlCcjgJ/+FKZPh+XLYfToKhuambUxJ6Maq5SMAD7zGVixAn74Q1DdL7eZWWtpVDJq29sB\nlcydC11dcM01zY7EzKx9tX3LCODxx+Hkk7Nuu8mTGxyYmVmBuWXUQJMmwec/Dx/8IDz3XLOjMTNr\nP05GyUc+AqecAiecAD/+cbOjMTNrL+6mK3P77VliOvtsuPJKOPjgBgRnZlZQ7qZrkjPPhJUr4YUX\nss8h3X13syMyM9v/uWXUi8WL4fLLYcQI+PCH4cIL4XWvq1OAZmYF5M8Z1djeJCOA7u6sdXTddXDr\nrXDaaXDBBXDqqTBmTB0CNTMrECejGtvbZJS3dSt897tw881w//0wahS8613Z4+STYcoUGDiwRgGb\nmRWAk1GN1SIZ5XV3w6OPwr33wj33ZD+7uuDtb88S07velZWHDavZIc3MGs7JqMZqnYwqeeopuO++\nLDHdey889BBMnLir9fSud8FRR/m2Q2bWOpyMaqwRyajctm2wbNmu1tM992SJKN+1d+KJMHhwQ8My\nM+s3J6Maa0YyKhcBa9fu6ta791545BF4/evhyCMrP8aPz74i3cysGZyMaqwIyaiSbdtg/fosSZU/\n1qzJlo3txNUSAAAJPElEQVQcWT1ZHXlkNvXczKwenIxqrKjJqC/d3bBxY+VkVXoMGrSrFXXoodln\noV772so/R43yjD8z6z8noxpr1WTUlwjYvDlLSr/5DTz9NDzzTPWfW7bA8OG9J6zXvjZ7jBoFhxyS\ntcxe8xpPvDBrR05GNba/JqM9tXNnlpD6SlpPP52t9+yz2WPbtl2JqfSzWrm87pBD3Boza1VORjXm\nZLRvtm3blZzySao/5eeegyFDstbVQQdlP/Pl/taVygcemD2GDt31c+jQrLvSzGrLyajGnIyap7sb\nXnopu/nsiy9mP/PlSnXVlr/wArz8cra/l1/eVX7ppaz1VUpM+SRVKXGVl8vXyf/sa9mgQe7CtP2X\nk1GNORnt3yJgx47dk1SlxFVteSmp7emy7u5dCWrw4OwxZMju5f7W5R8HHJA9alUeONCJ0/aMk1GN\nORlZveST4LZtux6vvLJ7uVJdpeWvvALbt+96bNtWudzbslI5X9fd3TM5VUpYlZ7n6wYNyh4DB+4q\nlz+vVq60rPQ8/3Nf6vpa5mS8Z5yM9oCkM4F5ZN/PdG1EXFlhHScja3vd3dWTWm/1+bqdO7MEXHrs\n6fNSXWlfpeW9lfe0rrd1pP4lrgEDdi0rlftbV2n5gAG7l/d02fHHZ98a0EiNSkYtP+QraQDwVeA0\n4AlgqaRbIuLR5kZWW52dnXR0dDQ7jL3m+JsnH/uAAVl34JAhzY1pT9Ty2kdkCbm3hLVjx651du7c\nVe5vXfnyhx/uZMqUjh51pUf+eX+W7c+TdPaHU5sKrI6ItQCSFgLnAE5GBeL4m6eVY4faxi/tark0\n6p6Qy5d3ct55HY05WAvbH752fCywLvd8faozM7MWsT8kIzMza3EtP4FB0juAuRFxZno+C4jySQyS\nWvtEzcyaxLPp+kHSQOAxsgkMG4EHgQsiYlVTAzMzs35r+QkMEbFT0p8DS9g1tduJyMyshbR8y8jM\nzFpfW0xgkHSmpEclPS7pb5scyxpJD0taJunBVDdS0hJJj0m6Q9KI3PqzJa2WtErS6bn6kyStSOc0\nL1c/WNLCtM19ksbvY7zXSuqStCJX15B4Jc1I6z8m6aIaxj9H0npJD6XHmUWMX9I4SXdJ+pWklZL+\nMtW3xPWvEP9fpPpWuf5DJD2Q/lZXSpqT6gt//XuJvbjXPiL26wdZwv1P4EjgAGA5cFwT4/kvYGRZ\n3ZXAp1P5b4EvpfIUYBlZd+pR6TxKrdkHgLel8m3AGan8ceDqVD4fWLiP8Z4CvAVY0ch4gZHAr4ER\nwCGlco3inwN8osK6k4sUP3AY8JZUPphsbPS4Vrn+vcTfEtc/7eeg9HMgcD/Z5xpb5fpXir2w174d\nWkavfig2IrYDpQ/FNovYvUV6DjA/lecD56by2WQv8I6IWAOsBqZKOgwYFhFL03o35LbJ7+smsokd\ney0ifg4828B435vKZwBLImJrRGwhGxN89V3cPsYP2etQ7pwixR8RmyJieSr/FlgFjKNFrn+V+Euf\nASz89U9xv5iKQ8j+UQetc/0rxQ4FvfbtkIyK9qHYAH4kaamkP011YyKiC7I/YGB0qi+PfUOqG0t2\nHiX5c3p1m4jYCWyRNKrG5zC6jvFuTfFW21et/Lmk5ZK+ketmKWz8ko4ia+HdT31/X+od/wOpqiWu\nv6QBkpYBm4AfpX/KLXH9q8QOBb327ZCMiubkiDgJ+D3gMkmnsusdS0ktZ5U04h7FrRbv1cDREfEW\nsj/Uf67hvmsev6SDyd55Xp5aGC31+1Ih/pa5/hHRHREnkrVIp0o6nha5/hVin0KBr307JKMNQH4Q\nf1yqa4qI2Jh+PgXcTNaN2CVpDEBqFj+ZVt8AHJHbvBR7tfoe2yj7DNbwiNhc49NoRLx1e90i4qlI\nndvA18leg0LGL2kQ2T/yBRFxS6pumetfKf5Wuv4lEfEc0EnW3dQy17889kJf+z0ZEGvFB9ngXWkC\nw2CyCQyTmxTLQcDBqfwa4B7gdLIB0b+N6gOig4EJ9BxULA1IimxQ8cxUfym7BhWns48TGNJ+jgJW\n5p7XPV56DoKWyofUKP7DcuW/Br5T1PjJ+uj/d1ldy1z/KvG3xPUHXkcaeAcOBH5K1qNR+OvfS+yF\nvfYN/4fcjAfZu5nHyAblZjUxjglkyXAZsLIUCzAKuDPFuCT/wgGz0y/GKuD0XP1b0z5WA1/J1Q8B\nFqX6+4Gj9jHm75B9NccrwG+AD6dfsLrHC1yc6h8HLqph/DcAK9JrcTPZGEDh4gdOBnbmfmceSr/L\nDfl9qWP8rXL935RiXp7i/btG/r3uS/y9xF7Ya+8PvZqZWdO1w5iRmZkVnJORmZk1nZORmZk1nZOR\nmZk1nZORmZk1nZORmZk1nZORWT9Jej79PFLSBTXe9+yy5z+v5f7Nis7JyKz/Sh/KmwB8cE82TLdL\n6c1nehwo4pQ92b9Zq3MyMttzXwROSV9Odnm6O/JV6cvMlkv6CICkd0v6qaRbgF+luh+kO7avLN21\nXdIXgQPT/hakuudLB5P0v9L6D0ualtv33ZK+l74MbUFu/S9J+mWK5aqGXRWzfTCo2QGYtaBZwN9E\nxNkAKflsiYi3SxoM3CNpSVr3ROD4iPhNev7hiNgiaSiwVNL3I2K2pMsiu5t7SaR9/xFwQkS8SdLo\ntM1P0jpvIbun2KZ0zHcBjwLnRsRxafvh9boIZrXklpHZvjsduCh9d8wDZPcum5iWPZhLRAB/JWk5\n2b28xuXWq+Zk4EaAiHiS7O7Lb8vte2Nk9/RaTnZD2K3AS+m7aj4AvLSP52bWEE5GZvtOwF9ExInp\n8YaIuDMte+HVlaR3k30b5tsj+z6Z5cDQ3D76e6ySV3LlncCgyL7kbCrZ1zb8PnD7Hp+NWRM4GZn1\nXykRPA8My9XfAVyavrsHSRMlHVRh+xHAsxHxiqTjgHfklm0rbV92rJ8B56dxqUOBU4EHqwaYHfeQ\niLgd+ARwQv9Pz6x5PGZk1n+l2XQrgO7ULXd9RHwlfa32Q5JE9mVr51bY/nbgY5J+Rfb1A/flln0N\nWCHpFxHxJ6VjRcQPJL0DeBjoBj4VEU9KmlwltuHALWlMCrLvrDErPH+FhJmZNZ276czMrOmcjMzM\nrOmcjMzMrOmcjMzMrOmcjMzMrOmcjMzMrOmcjMzMrOmcjMzMrOn+P2y4fUDInX1QAAAAAElFTkSu\nQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x12ae29278>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEZCAYAAABSN8jfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8HFWd9/HPNwkJa1hEQAOEfYdBfIgZxOEqKhEXEGY0\nOMqAG44DjysCM/qQmXHYZh4fdVBHNCoyQmRxABUlolwQNBBlFRMSBWMSIKwJCUsIye/545xOKp3u\nvh361u3qm+/79apX13Kq6td1+9avzznVVYoIzMzMOjGi2wGYmVnvczIxM7OOOZmYmVnHnEzMzKxj\nTiZmZtYxJxMzM+uYk8kQkDRC0lJJOw5mWRv+JJ0o6cdd2O8Fkj461PsdziR9XNLnux1HWeTfmaxL\n0lKgdmA2A5YDK/O8UyLism7FNhgkfRC4CDg+Iv6n2/FUlaQjgW9GxK55+pfANyLiuyXtb3dgbkR0\n9UuepO2BmcAeEfFCN2MZTiRtAswFDoyIp7odz2BzzaSBiNgiIsZGxFhgHvDWwrx1EomkkUMfZUdO\nBJ7Ir0NKUq995gbt21Yb712Dub8OnAz80Imkfe2cAyLiOeB64H3lRzT0eu0fuxuUhzUzpH+VNE3S\npZKWAH8raaKkX0t6StJCSV+qfcAkjZS0StLOefqSvPw6SU9LulXS+PUtm5e/RdL9eb9flnSLpKZJ\nIn/7PQz4MHC0pJfVLT9O0p2SlkiaI+mNef42kr4t6SFJT0i6Is//gKQbC+s3iv9CST/JNb7DJb29\nsI8/SfpsXQx/lY/lYknzJNWO78K6cu+S9JsG7/EwSQvq5v2NpN/m8YmSfpv3/7Ck85sdr8L65wF/\nCfxX/jt8Ic/fT9LP8jH5vaTjCuus73u/Ka+3NO/j1Q2O7+GSZua/9wxJEwrLfilpSv6MPJ0/M1vl\nZZtI+p6kxwvrbtPk7b6lFkte90hJD0o6U9KjkhZIepukt+bPyOOSTi+Ul6R/lPSHXP5SSVsWll2R\nj/uTkn4haZ+6Y9b0897kb6P82V+UPzN31bYpaVtJP8rH+1eSPl87npJ2l7Sqblu/VP7/kbRHju+J\n/D6+K2mLQtn5kj4t6R5gWZ63o6Qf5PJ/1LpNhTcBb231fnpWRHhoMQAPAm+om/evwPPA0Xl6DPBq\n4FBS4tkFmA18NC8fSWom2zlPXwI8CrwqL5sGfPcllN0OeBp4W172CVKT3Ikt3s8/A7fk8d8DpxWW\nHQY8BfTl6XHAnnn8euC/gbF5X4fn+R8AflHYRqP4nwAm5OnRQB+wb54+ML+/2rHcFVgKHE/6srMN\ncFBeNgs4srCva4FTm7zPB4AjCtM/AD6Rx28H3p3HNwMObbKNI4EHCtO/BN5XmN4MWAD8bf67vwp4\nvHDM1ve97w6srIth9fEFtgUWA+/Kx+a9eX9bFuK7H9gN2Bi4GfiXvOyj+RiMzrEeAmza5H0/CfxF\n3XFYAZyZ/74fARYB3wU2ye/jOWDHXP5TOZYd8v4uYs1nVqQa8aZ52ZeBmYV9Nf28t/hMHw3MADbP\n0/sA2+XxK4Hvkf5HDwQeKhzPRsf7l+T/H2BP4PU5jm3zsgsKZeeTmgNfkbcv4E7gjLzObqTP4esL\n6xwKPNLt81oZQ9cDqPpA82RywwDrfQr4fh4fCaxi7RPsVwtl3w7c8xLKngzcVLffh2idTB4A/j6P\nf7buH/mbwPkN1tkReKH2z1q3rFEyqY//mwMcq/+s7TfH9P0m5f4R+E4e35b0bXDbJmXPBb6ex7cC\nngFekadvyfvZZoC4GiWTEwvT7wF+XrfON4GzXuJ7HyiZnET+IlBYfjvwnkJ8nyksOw24No9/iJRc\nDmjjM78S2K3uODzNmj7WrfLf+OBCmbtYkxTnAK8rLNsJeK7JvrbN29pkoM97i3jfRPpiNKEWY54/\nipQEdy3MO582k0mD/RwP3FaYng/8bWH6MOAPdet8tvY5zNP7AM8P9DfoxcHNXC/d/OKEpL1zdfph\npaavfyb9ozTzSGH8WWDzl1D2lfVxkL4pNyTpCFJt4/I86zLg1ZL2y9M7AX9ssOpOwOMRsaxFjK3U\nH6u/lHRjbgpYTDph1o5VsxggnWjeIWkMMBm4MSIeb1L2UuA4pabG44EZEfFwXnYysD9wf27uectL\nfF/jSU1XT+bhKVKtYYdCmfV57wN5JakPr2ge6W9a0+yz8h3gBuDy3Dxzjpr34SwGtqib93jksyGp\nFgKpBkFhXm1fOwM/rB0X4B5glaTtlK5WvCA3AS0mdUgHax+D9fnfICJ+BvwX8DXgEUlflbQZsD2p\nBlf8n6g/fk1J2l7S93Oz3mLSMaz/WxW3PR4YX/d5OD3HUbMF6fgOO04mL119R+nXgXtJ3+i2BM6m\nrq+lBA+TTr5F4xoVzP6O9De/V9LDpG/oq/J8SCe+3RusNx/YVlKjf+pnSE0WNa9g3WNTP30ZcAUw\nLiK2Aqay5ljNB/ZoFHxEzAd+C7yT1MRzSaNyuey9pJPSJOAEUnKpLZsbESdExMuBLwBXSRrdbFst\n3sd8Ug11mzxsHekijY+1WKfVe68vW+8hUhNq0c7AwnWL1gUesSIi/iUi9gMOB44jNc81cg+w10Db\nbGE+8Ka647JZRDxKauKaRGpK3Yr0t16nX3J9RcSXI+LVwAGkLwqfJDXFBWv/j+xcGH8GQNLGhXnF\nLwLnk5qz98+xntQgzuLfbD4wp+59bxkRxxbK7Avcvb7vrxc4mQyeLYAlEfGcpH2BU4Zgnz8CXpU7\nQkdK+jhNvuUqXZZ4PPB+4GDgL/LwSeC9kkQ6sX1Q0hG5U3OcpL0iYgHpW+1XJG0paZSk1+VN3w0c\nJGn/vI//00bcmwNPRcQKSRNJtYya/waOkvTO/J5eJumgwvJLgLOAvYFrBtjPpaR+pImktvPasXiv\n1lx48DQpoa5ad/V1LCK1g9dcC+wv6YR8TDaSdKikPVtso9V7fxQISbs2WfdHwH5KFxOMlPQeUvIf\n8Hcokl6f/0YiNQ+uoPl7vo7Ut/NSfR04V9JOed/bSXp7XrYFqV/vqVx7OIcOr2DLx/zQXAt9jtQk\nuzIiXgSuBv5Z0saSDqBwJVVEPEL6wvHeXGP6MKl2UbMFKeEsze/l0wOE8mvgBUmflDQm/40OkHRI\nocwRwE86eb9V5WQysHY/6J8CTpL0NKm6Pa3FdgbaZltl8ze9dwP/j9QRuyupA3B5g+LHkU6c34uI\nR2sD8A1SZ+2bIuLXpLb1/wSWAL8g9ZdAqgmI1B7+CHBqjmEW6YRwE6mDfPVVQC3i/3vgvNwceCbw\n/cJ7+hOpnfxMUkfwb0nfNmuuIp3Qr4iIRu+z6DJSB+r0iFhSmH80MCvv/wLgXfnEM5AvAu/JTRj/\nERFPA0eRjs3DpJrDOaTOWFj/976M1NdzW95H8SREbtJ7R17vceBjpMvWa++t1efqlaQO+CWkGvR0\nCrW1OhcDb5O0UYvttap9foF0wvx5fp+3AP8rL/s2a47VvXlZq+22o1bDe4rUJ7iQ9D8B6XhvQ/rM\nfgP4Vt26HwL+CXiM9LmaUVh2NvAaUrPU1RS+kDSKNSJWkj5bE4A/kb4c/Be5yTB/2TqKdOHCsFPq\njxYlTSVdabQoIg5qUubLpEsRnwFOioi7SgtomMtt4A+Rfox4a7fjKYukB4G/i4ibux3LcKV0KfSf\nI+Kr3Y5lMEn6AKnT/A1d2PfHSReMfHbAwj1oVMnb/zbpW27DTJw7PnePiD0lvYaUxSeWHNOwIuko\n0rep50nNPy+QrvAZliS9m3Q1jBNJiSLizG7HMNxExBe7HUOZSk0mEXGLWv/g6BhyoomI23J7/PYR\nsajMuIaZw0nNFSOB+4BjI2JFd0Mqh9LtTPYgXZJrG5B8JeKPWLtpScCKiGj240sbQqXfmysnkx82\nauaS9EPg3Ij4VZ6+gXSd/B2lBmVmZoPKHfBmZtaxsvtMBrKQta8B35Em18xLKrcKZWY2TEVE2b95\nG5KaSasfJF1LvnNtvuZ+cav+km7fLqCT4eyzz+56DI6/+3FsiPH3cuzDIf6hUmrNRNKlpB8/vUzS\nn0nXbY8GIiIuiojrJB0t6Q+kS4NPLjMeMzMrR9lXcw141U1EnFpmDGZmVr5u95lsMPr6+rodQkcc\nf3f1cvy9HDu0jn/VKnjxRVi5Mr3WhuJ0cXz0aNiz1c12eljPPLZXUvRKrGbWWkQ6uT7/PCxfnl5f\neKH5sGJF6+WNyr74YhpfsWLN+ECvzZY1SxARMGrUmmHkyObTI0fCgQfC978/8PEZTJKIIeiAd83E\nbJiLWHOirJ1wly9PQ218fV+L48WEUP/aatnIkTBmDGy8cXodMyZ9c68NG2209nSroVZ2iy3WTG+0\nUTqJF18bzWv2WhuvH4oJYoR/XLGak4lZSVatSifN555bMzz77PpPF7+1FxNCbXygeS++mE58tZNs\n7aTd6nWgMrWTdi0RrO/rmDEpJhs+3MxlG7SI9E356afT8Mwzaw/Llq07r51lzzyTtjtmDGyyyZph\n003Xf7p2Ai5+U6+N1782W6bSGzmsqoaqmcvJxHpSRPrGvnQpLFmyJhnUhvp5rcqMGAFjx6Zv25tv\nDptt1t7QqmwtKbgZxLrNyaSOk0nvi0jNNk89lZLA0qXpZF4bLw7tzB81KiWALbdMyaA2FKdbLasN\nY8YMHLtZr3IyqeNkUj0rVsATT8Djj8Njj6XX+vH6aYCtt15TE6gfGs1vVnZ0Ow/aNdvAOZnUcTIZ\nGhHppD9/fhoWLEivixatmyCWLYOXvQy23TYNL3/5mvFm05tuOnAMZjZ4nEzqOJl0LgKefHLdRFEc\nX7gwnfB32gl23HHN6w47rJscttrKfQJmVedkUsfJZGArVqSE8MADafjzn9dNGGPGrJ0o6sfHjUsd\nyGY2PDiZ1HEyWVOzePDBNQmjOCxcCK94Bey2G+y6K4wfv27S2Hzzbr8LMxtKTiZ1NpRk8sILMG9e\n42TxwAOpzO67p4RRP+y8szulzWxtTiZ1hlsyWb4cZs+Ge++F3/1uzfDww6kGUatd1CeMrbf2D9DM\nrH1OJnV6NZmsXJmapWpJo/b64IMpORxwQLr52wEHpGGXXXybCTMbPE4mdaqeTCJSraJWw6gljd//\nPl0FVUsYtde99/aP5cysfMMmmUiaBHyR9IjgqRFxft3yrYBvAbsDzwHvj4jfN9hOpZLJypXQ3w/X\nXAN3350Sx4gRKVkUaxr7759+dGdm1g3DIplIGgHMAY4EHgJmApMjYnahzAXA0oj4V0l7A1+JiDc2\n2FbXk0kEzJgBl10GV1yRLqP967+GQw9NiWP77bsanpnZOobL80wmAHMjYh6ApGnAMcDsQpn9gHMB\nIuJ+SbtIenlEPFZybG2JgHvuSQlk2rR0874TToCbboK99up2dGZm1VB2MhkHzC9MLyAlmKK7geOA\nWyVNAHYGdgS6mkzmzl2TQJ59FiZPTk1aBx3kq6nMzOpV4bqh84AvSboDuBe4E1jZqOCUKVNWj/f1\n9Q36s6Xnz4fLL09JZMECeNe7YOpUmDjRCcTMekN/fz/9/f1Dvt+y+0wmAlMiYlKePhOI+k74unUe\nBA6MiGV180vpM3nsMbjyypRA7rsP3vnO1Ix1xBG+RNfMet9w6TOZCewhaTzwMDAZOKFYQNKWwLMR\nsULSh4Cb6hPJYFuyBK6+OiWQGTPg6KPh9NPhzW/25bpmZi9FqckkIlZKOhWYzppLg2dJOiUtjouA\nfYGLJa0C7gM+UFY8zz4L55wDF14IfX1w8slw1VW+saGZWac2mB8t/vjHcNpp6TLeL3whXdZrZjbc\nDZdmrq6bPx8+/vH0w8KvfQ2OOqrbEZmZDT/D+tFGP/kJHHJI+kX6737nRGJmVpZhWzO58EL4t39L\nHe2vfW23ozEzG96GXTJZuRI+8Qm44Qa49dZ0Z14zMyvXsEomS5em34gsXw6/+lV6RrmZmZVv2PSZ\nLF8Ob3oTvPKVcN11TiRmZkNp2FwafOqp8NBD6XcjvvWJmVniS4PXw2WXwfXXw29+40RiZtYNPV8z\nmTUL/uqv4Gc/g4MP7kJgZmYVNlQ1k57uM1m2DI4/Hs47z4nEzKyberZmEgHvex9stBF861tu3jIz\na8R9JgO49NL0BMQZM5xIzMy6rWebuW69FT70Idh0025HYmZmPZtMHn0Utt++21GYmRn0cDJZtMjJ\nxMysKpxMzMysY6UnE0mTJM2WNEfSGQ2Wj5V0raS7JN0r6aR2tutkYmZWHaVeGixpBDAHOBJ4iPRM\n+MkRMbtQ5ixgbEScJWlb4H5g+4h4sW5bqy8Nfv55GDs23Y/LV3KZmTU3XH60OAGYGxHzImIFMA04\npq5MAFvk8S2AJ+oTSb1Fi2C77ZxIzMyqouxkMg6YX5hekOcVXQjsJ+kh4G7gYwNt1FdymZlVSxV+\ntHgUcGdEvEHS7sDPJB0UEcvqC06ZMgWAOXNg1Kg+oG8IwzQzq77+/n76+/uHfL9l95lMBKZExKQ8\nfSYQEXF+ocyPgHMj4tY8/XPgjIj4Td22VveZTJ0Kt9wC3/52aaGbmQ0Lw6XPZCawh6TxkkYDk4Fr\n68rMA94IIGl7YC/ggVYb9ZVcZmbVUmozV0SslHQqMJ2UuKZGxCxJp6TFcRHweeA7ku7Jq30mIp5s\ntd1Fi2CXXcqM3MzM1kfpfSYR8VNg77p5Xy+MP0zqN2nbokUwYcLgxGdmZp3ryV/A+2ouM7Nq6clk\n4j4TM7NqcTIxM7OO9dyTFlesSM8wef55GDmy21GZmVXbcLk0eNA99hi87GVOJGZmVdJzyaR2Xy4z\nM6uOnksmvpLLzKx6ei6ZuPPdzKx6nEzMzKxjTiZmZtYxJxMzM+uYk4mZmXWs55LJo4/60mAzs6rp\nuWTimomZWfX01O1UVq4MxoyBZ5+FjTbqdkRmZtXn26k08MQTMHasE4mZWdWUnkwkTZI0W9IcSWc0\nWP5pSXdKukPSvZJelLRVo225icvMrJpKTSaSRgAXkp6kuD9wgqR9imUi4j8i4lURcQhwFtAfEYsb\nbc/JxMysmsqumUwA5kbEvIhYAUwDjmlR/gTgsmYLfSWXmVk1lZ1MxgHzC9ML8rx1SNoEmARc1Wxj\nrpmYmVXTqG4HUPB24JZmTVwAV145hdGjYcoU6Ovro6+vb8iCMzPrBf39/fT39w/5fku9NFjSRGBK\nREzK02cCERHnNyj7A+DyiJjWZFtx8snBYYfBBz9YWshmZsPKcLk0eCawh6TxkkYDk4Fr6wtJ2hI4\nArim1cbczGVmVk2lNnNFxEpJpwLTSYlrakTMknRKWhwX5aLHAtdHxHOttudkYmZWTT31C/iddgpu\nvhl22aXb0ZiZ9YahaubqqWQyZkzw1FOwySbdjsbMrDcMlz6TQTV6tBOJmVkV9VQycX+JmVk1OZmY\nmVnHeiqZ+FYqZmbV1FPJZOONux2BmZk10lPJZERPRWtmtuHoqdOzSr+4zczMXoqeSiaumZiZVVNP\nnZ6dTMzMqqmnTs9OJmZm1dRTp2f3mZiZVdOAyUTSaZK2HopgBuKaiZlZNbVzet4emCnpckmTpO7V\nD5xMzMyqacDTc0R8FtgTmAqcBMyVdI6k3UuObR1OJmZm1dTW6TnSfeofycOLwNbAlZIuGGjdXJuZ\nLWmOpDOalOmTdKek30m6sfm22onWzMyG2oBPWpT0MeBE4HHgm8DpEbFC0ghgLvCZFuuOAC4EjgQe\nIjWXXRMRswtltgS+Arw5IhZK2rbZ9lwzMTOrpnYe27sNcFxEzCvOjIhVkt42wLoTgLm1dSVNA44B\nZhfKvAe4KiIW5u0+3mxjTiZmZtXUzun5J8CTtQlJYyW9BiAiZg2w7jhgfmF6QZ5XtBewjaQbJc2U\n9L6mwTqZmJlVUjun568BywrTy/K8wTIKOAR4CzAJ+JykPRoVdDIxM6umdpq5FIUHxefmrXbWA1gI\n7FyY3jHPK1oAPB4RzwPPS7oZ+AvgD/Ub+/WvpzBlShrv6+ujr6+vzTDMzDYM/f399Pf3D/l+VcgT\njQtIPwD6WVMb+Sjw+og4dsCNSyOB+0kd8A8DtwMnFJvHJO0D/CepVjIGuA14d0T8vm5bcfrpwQUD\nXj9mZmY1koiI0q+Fbafh6CPAYaQaxQLgNcCH29l4RKwETgWmA/cB0yJilqRTJH04l5kNXA/cA8wA\nLqpPJKuDdTOXmVklDVgzqQpJcdZZwTnndDsSM7PeMVQ1k3Z+Z7Ix8AFgf2D1g3Mj4v0lxtUklqHe\no5mZtaOdhqNLgB2Ao4CbSJ3oS8sMqhk3c5mZVVM7p+c9IuJzwDMRcTHwVlK/yZBzMjEzq6Z2Ts8r\n8utiSQcAWwLblRdSc04mZmbV1M7vRS7KzzP5LHAtsDnwuVKjasLJxMysmlomk3yjxqcj4ingZmC3\nIYmqaTzd3LuZmTXT8rt+RKyixV2Bh5prJmZm1dTO6fkGSZ+WtJOkbWpD6ZE14GRiZlZN7fSZvDu/\n/kNhXtCFJi8nEzOzahowmUTErkMRSDvcZ2JmVk3t/AL+xEbzI+K7gx9Oa66ZmJlVUzvNXIcWxjcm\n3QH4DsDJxMzMgPaauU4rTkvaCphWWkQtOJmYmVXTSzk9PwN0pR/FycTMrJra6TP5IenqLUjJZz/g\n8jKDah5LN/ZqZmYDaafP5D8K4y8C8yJiQUnxtOSaiZlZNbVzev4zcFtE3BQRtwJPSNql3R1ImiRp\ntqQ5ks5osPwISYsl3ZGHzzYN1snEzKyS2qmZXEF6bG/Nyjzv0MbF18j39rqQdAXYQ8BMSdfkR/UW\n3RwR7xhoe04mZmbV1M7peVREvFCbyOOj29z+BGBuRMyLiBWkq8COaVCurd4Q95mYmVVTO8nkMUmr\naw2SjgEeb3P744D5hekFeV69v5R0l6QfS9qvabCumZiZVVI7zVwfAb4n6cI8vQBo+Kv4l+i3wM4R\n8ayktwBXA3s1KuhkYmZWTe38aPGPwERJm+fpZeux/YXAzoXpHfO84vaXFcZ/IumrkraJiCfrN3bN\nNVOYNy+N9/X10dfXtx6hmJkNf/39/fT39w/5fhURrQtI5wAXRMTiPL018KmIaHrVVWHdkcD9pA74\nh4HbgRMiYlahzPYRsSiPTwAuj4hdGmwrLr44OHEw60RmZsOcJCKi9B7ndhqO3lJLJAD5qYtHt7Px\niFgJnApMB+4DpkXELEmnSPpwLvbXkn4n6U7gi6y55f26wbqZy8ysktrpMxkpaUxELAeQtAkwpt0d\nRMRPgb3r5n29MP4V4CvtbMvJxMysmtpJJt8Dfi7p26RLeE8CLi4zqGacTMzMqqmdDvjzJd0NvJF0\nj67rgfFlB9aIk4mZWTW1e3peREokfwO8AZjVung5/KNFM7NqalozkbQXcEIeHge+T7r66/VDFNs6\nXDMxM6umVs1cs4FfAm+LiD8ASPrEkETVhJOJmVk1tTo9H0f6bciNkr4h6UjavIdWWZxMzMyqqenp\nOSKujojJwD7AjcDHge0kfU3Sm4cqwCL3mZiZVdOA3/Uj4pmIuDQi3k66HcqdwDrPJRkKrpmYmVXT\nep2eI+KpiLgoIo4sK6BWnEzMzKqpp07PTiZmZtXUU6dnJxMzs2rqqdOzO+DNzKqpp5KJayZmZtXU\nU6dnJxMzs2rqqdOzk4mZWTX11OnZfSZmZtVUejKRNEnSbElzJDX9saOkQyWtkHRcszKumZiZVVOp\np2dJI4ALgaOA/YETJO3TpNx5pGelNOVkYmZWTWWfnicAcyNiXkSsAKYBxzQodxpwJfBoq405mZiZ\nVVPZp+dxwPzC9II8bzVJrwSOjYivMcBdiZ1MzMyqqZ1nwJfti6x948imCeUb35jCddel8b6+Pvr6\n+koNzMys1/T399Pf3z/k+1VElLdxaSIwJSIm5ekzgYiI8wtlHqiNAtsCzwAfjohr67YVv/lN8OpX\nlxaumdmwI4mIKP1a2LJrJjOBPSSNJz1oazLpMcCrRcRutXFJ3wZ+WJ9IatzMZWZWTaUmk4hYKelU\nYDqpf2ZqRMySdEpaHBfVr9Jqe04mZmbVVGoz12CSFHffHRx0ULcjMTPrHUPVzNVT3/VdMzEzq6ae\nOj07mZiZVVNPnZ6dTMzMqqmnTs++0aOZWTX1VDJxzcTMrJp66vTsZGJmVk09dXp2MjEzq6aeOj07\nmZiZVVNPnZ7dAW9mVk09lUxcMzEzq6aeOj07mZiZVVNPnZ6dTMzMqqmnTs/uMzEzq6aeSiaumZiZ\nVVNPnZ6dTMzMqqmnTs9OJmZm1VT66VnSJEmzJc2RdEaD5e+QdLekOyXdLum1zbblZGJmVk2lPmlR\n0ghgDnAk8BDpmfCTI2J2ocymEfFsHj8QuDwi9m2wrViyJBg7trRwzcyGneHypMUJwNyImBcRK4Bp\nwDHFArVEkm0OrGq2MddMzMyqqezT8zhgfmF6QZ63FknHSpoF/BB4f7ONOZmYmVXTqG4HABARVwNX\nSzoc+DzwpkblzjlnCqNyxH19ffT19Q1ViGZmPaG/v5/+/v4h32/ZfSYTgSkRMSlPnwlERJzfYp0/\nAodGxJN18+P554MxY0oL18xs2BkufSYzgT0kjZc0GpgMXFssIGn3wvghwOj6RFLjZi4zs2oqtZkr\nIlZKOhWYTkpcUyNilqRT0uK4CDhe0onAC8BzwLuabc/JxMysmkpt5hpMkmLVqvD9uczM1sNwaeYa\nVE4kZmbV1FPJxMzMqsnJxMzMOuZkYmZmHXMyMTOzjjmZmJlZx5xMzMysY04mZmbWMScTMzPrmJOJ\nmZl1zMnEzMw65mRiZmYdczIxM7OOOZmYmVnHnEzMzKxjTiZmZtax0pOJpEmSZkuaI+mMBsvfI+nu\nPNwi6cCyYzIzs8FV6pMWJY0A5gBHAg+Rngk/OSJmF8pMBGZFxBJJk4ApETGxwbaiV54KaWZWFcPl\nSYsTgLkRMS8iVgDTgGOKBSJiRkQsyZMzgHElx2RmZoOs7GQyDphfmF5A62TxQeAnpUZkZmaDblS3\nA6iR9HrgZODwZmWmTJmyeryvr4++vr7S4zIz6yX9/f309/cP+X7L7jOZSOoDmZSnzwQiIs6vK3cQ\ncBUwKSKvdzDdAAAID0lEQVT+2GRb7jMxM1tPw6XPZCawh6TxkkYDk4FriwUk7UxKJO9rlkjMzKza\nSm3mioiVkk4FppMS19SImCXplLQ4LgI+B2wDfFWSgBURMaHMuMzMbHCV2sw1mNzMZWa2/oZLM5eZ\nmW0AnEzMzKxjTiZmZtYxJxMzM+uYk4mZmXXMycTMzDrmZGJmZh1zMjEzs445mZiZWcecTMzMrGNO\nJmZm1jEnEzMz65iTiZmZdczJxMzMOuZkYmZmHSs9mUiaJGm2pDmSzmiwfG9Jv5L0vKRPlh2PmZkN\nvlKTiaQRwIXAUcD+wAmS9qkr9gRwGvDvZcbSbf39/d0OoSOOv7t6Of5ejh16P/6hUnbNZAIwNyLm\nRcQKYBpwTLFARDweEb8FXiw5lq7q9Q+k4++uXo6/l2OH3o9/qJSdTMYB8wvTC/I8MzMbRtwBb2Zm\nHVNElLdxaSIwJSIm5ekzgYiI8xuUPRtYGhFfaLKt8gI1MxvGIkJl72NUydufCewhaTzwMDAZOKFF\n+aZveCgOhpmZvTSl1kwgXRoMfInUpDY1Is6TdAqphnKRpO2B3wBbAKuAZcB+EbGs1MDMzGzQlJ5M\nzMxs+OuJDviBfvg4xLH8SdLdku6UdHuet7Wk6ZLul3S9pC0L5c+SNFfSLElvLsw/RNI9+T19sTB/\ntKRpeZ1fS9q5w3inSlok6Z7CvCGJV9Lf5fL3SzpxEOM/W9ICSXfkYVIV45e0o6RfSLpP0r2S/nee\n3xPHv0H8p+X5vXL8x0i6Lf+v3qvUL9tLx79Z/NU8/hFR6YGU8P4AjAc2Au4C9uliPA8AW9fNOx/4\nTB4/Azgvj+8H3Enqm9olv49abfA24NA8fh1wVB7/e+CrefzdwLQO4z0cOBi4ZyjjBbYG/ghsCWxV\nGx+k+M8GPtmg7L5Vih/YATg4j28O3A/s0yvHv0X8PXH883Y2za8jgRmk3771xPFvEX8lj38v1EwG\n/OHjEBPr1uiOAS7O4xcDx+bxd5D+OC9GxJ+AucAESTsAW0TEzFzuu4V1itu6Ejiyk2Aj4hbgqSGM\n9w15/ChgekQsiYjFwHRg9TeoDuOHxhdrHFOl+CPikYi4K48vA2YBO9Ijx79J/LXfiVX++Oe4n82j\nY0gn2aBHjn+L+KGCx78XkknVfvgYwM8kzZT0wTxv+4hYBOkfENguz6+PfWGeN470PmqK72n1OhGx\nElgsaZtBfg/blRjvkhxvs20NllMl3SXpm4VmisrGL2kXUg1rBuV+XsqO/7Y8qyeOv6QRku4EHgF+\nlk+oPXP8m8QPFTz+vZBMqua1EXEIcDTwD5Jex5pvCzWDeVXDUFwS3WvxfhXYLSIOJv2T/d9B3Pag\nxy9pc9K3vo/lb/g99XlpEH/PHP+IWBURryLVCCdI2p8eOv4N4t+Pih7/XkgmC4FiJ/SOeV5XRMTD\n+fUx4GpSM9wipUucyVXKR3PxhcBOhdVrsTebv9Y6kkYCYyPiyUF+G0MRb2l/t4h4LHLDLvAN0t+g\nkvFLGkU6EV8SEdfk2T1z/BvF30vHvyYingb6SU01PXP8G8Vf2eO/vh1CQz2QOp5qHfCjSR3w+3Yp\nlk2BzfP4ZsCtwJtJHXpnRPMOvdHArqzdIVbrTBOpQ2xSnv9R1nSITabDDvi8nV2AewvTpcfL2h14\ntfGtBin+HQrjnwAurWr8pPbpL9TN65nj3yT+njj+wLbkTmNgE+BmUotCTxz/FvFX8vgP+Qn5pQyk\nbxP3kzqUzuxiHLuSktmdwL21WIBtgBtyjNOLBx04K/9RZwFvLsx/dd7GXOBLhfljgMvz/BnALh3G\nfCnwELAc+DNwcv5wlB4vcFKePwc4cRDj/y5wT/5bXE1qA69c/MBrgZWFz8wd+bM8JJ+XEuPvleN/\nYI75rhzvPw3l/2uJ8Vfy+PtHi2Zm1rFe6DMxM7OKczIxM7OOOZmYmVnHnEzMzKxjTiZmZtYxJxMz\nM+uYk4ltMCQtza/jJbV64udL2fZZddO3DOb2zarOycQ2JLUfVe0KvGd9Vsy3mmjlH9faUcTh67N9\ns17nZGIbonOBw/ODhT6W78x6QX4Q0V2SPgQg6QhJN0u6Brgvz/uffMfoe2t3jZZ0LrBJ3t4led7S\n2s4k/Xsuf7ekdxW2faOkK/KDjC4plD9P0u9yLBcM2VEx68Cobgdg1gVnAp+KiHcA5OSxOCJeI2k0\ncKuk6bnsq4D9I+LPefrkiFgsaWNgpqSrIuIsSf8Q6W7SNZG3fTxwUEQcKGm7vM5NuczBpPspPZL3\neRgwGzg2IvbJ648t6yCYDSbXTMzSzTpPzM+NuI1076Y987LbC4kE4OOS7iLdx2jHQrlmXgtcBhAR\nj5Lu/HpoYdsPR7qn0V2kG1ouAZ7Lz6l4J/Bch+/NbEg4mZilO6meFhGvysPuEXFDXvbM6kLSEaQn\n0b0m0rMk7gI2Lmyj3X3VLC+MrwRGRXpA0QTSbd/fBvx0vd+NWRc4mdiGpHYiXwpsUZh/PfDR/OwO\nJO0padMG628JPBURyyXtA0wsLHuhtn7dvn4JvDv3y7wceB1we9MA0363ioifAp8EDmr/7Zl1j/tM\nbENSu5rrHmBVbtb6TkR8KT+W9g5JIj0s6dgG6/8U+Iik+0i3L/91YdlFwD2SfhsR76vtKyL+R9JE\n4G5gFXB6RDwqad8msY0Frsl9MpCeV2FWeb4FvZmZdczNXGZm1jEnEzMz65iTiZmZdczJxMzMOuZk\nYmZmHXMyMTOzjjmZmJlZx5xMzMysY/8fC3TBvGinwNIAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1083d8d68>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"With 6 epochs, the training time is: 385.72842597961426\n",
"The validation accuracy is: 0.9631666666666666\n"
]
}
],
"source": [
"def accuracy(x, y):\n",
" correct = 0\n",
" for a, b in zip(x, y):\n",
" if np.argmax(a) == np.argmax(b): \n",
" correct += 1\n",
" return correct/len(x)\n",
"\n",
"train = train_images[:(9*len(train_images)//10)]\n",
"train_label = train_labels[:(9*len(train_images)//10)]\n",
"validation = train_images[(9*len(train_images)//10):len(train_images)]\n",
"validation_label = train_labels[(9*len(train_images)//10):len(train_images)]\n",
"NN = NeuralNetwork([784, 200, 10], loss=\"mean_square\", \n",
" plot=True)\n",
"start = time.time()\n",
"epochs = 6\n",
"NN.train(train, train_label, epochs, validation=validation, \n",
" validation_label=validation_label)\n",
"end = time.time()\n",
"print(\"With \" + str(epochs) + \" epochs, the training time is: \", end-start)\n",
"predictions = NN.predict(validation)\n",
"p_correct = accuracy(predictions, validation_label)\n",
"print(\"The validation accuracy is: \", p_correct) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let us train the model using cross entropy error as our loss function, with the same number of epochs. "
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAaMAAAEZCAYAAAA6xErnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4XHV97/H3JwlJQJJAlARNCGBNIMELwRqt4GG3VC61\nDWhLCKIETW0r2NLaqgl9bIIeL9CeY+xp4VRFCEGNEQtEDw9ExK2VayqJiQZCqk1MYrK5hATkltv3\n/LF+w14ZZvaencxlTebzep559prvun3Xmr33d36/9Zs1igjMzMxaaVCrEzAzM3MxMjOzlnMxMjOz\nlnMxMjOzlnMxMjOzlnMxMjOzlnMxOshIGiTpGUnj67mstZakL0ua04L93ifppGbvtxNImirpR63O\noyhcjFosFYOn02OPpOdysQsHur2I2BsRIyJiUz2XHShJn5a0M3dsz0h6rN77aYZ0LNen6cGS9kqa\n0MD9zZb0g3wsIj4UEZ9v1D6r5HEe8HhE/LyZ+22WZryWfYmIFcBzks5qxf6LxsWoxVIxGBkRI4EN\nwLtysW+ULy9pcPOz3G83lY4tHdOYSgtVOqaBHqeS/U20BqVPhys3PWA1HtcB7aOO/gJYNNCV2uh3\ntN/z3IRj+TrZee54LkbFovToDWTvyhdL+rqkHcBFkt6Wuk+ekrRZ0hdLfzTl7/YkLUrzb08tlHsk\nHTvQZdP8cyStTfv9Z0k/lnTxgA+yd78flrQOeLhSLC17mqTlaZ/3S5qW285/SPqUpHuB3wDHlO3n\nCknfKIv9q6R/StOzJf13Otb/kjSjhvR/mH6uSeu9O21ruqSVKc8f5bu2JG2U9HeSVqU8kfT3kn6R\ntrFa0h+l+OuB/wO8I9+aTK/NP+S2+ReS1kl6XNK/Szq67Nz+WZr/pKQv5tabKOmHkrZLekzSTVVe\no2FAV+54S9v+ZDpXOyQ9KOno/XztKp77WvMry3WKpO+lY10j6T25eX39Tr/stZR0RsprrqQtwJdq\nPN8fkfTLlPPnSucwHfsJuXyOlvSspCNTqBt4p9qngDdORPhRkAfw38DvlcU+DbwA/EF6Pgx4M/AW\nssJ1HPAIcGmaPxjYA0xIzxcBjwFT07zFwI37sewY4GngD9O8vwFeBC6uciyfBr5aZd5gYC9wOzAq\nHVOl2CuB7cAMsjdO7wOeAEal7fwH8EtgUlp/UNl+jk85H5rbb086vhFp269N88YCJ/Z3LLk8j8nN\nfwuwBTglvSaXAP8FDEnzNwLLgVcDw1LsT4AxaXom8AxwVHo+G7i7LIdFwD+k6TOBrcAbgKHAvwLf\nL8vvFuBw4FjgSdLvFbAE+FiaHgr8TpVjfiPwVFlsLrAid87eCBwx0Neur3Nfa365nF4BbAIuSud+\natrPxBp//8tfyzOAXek1H5KOpZbzvQwYSfaGaB3p7wL4v8Cnc9v/KPDtsmN4liq/e530cMuoPfw4\nIm4HiIgXI+InEbE8MuuBLwOn55Yv7666OSJWRMQe4GvAyfux7LuAFRHx3YjYExFfIPsn15eLJG3L\nPe4sm/+ZiNgRES9Wif0R8LOIWBLZ9a2byIrPu3LLfzUiHk057c1vPCL+G/gZcG4KnQlsi6yvHrJ/\nIm+QNCwieiLikX6OJy9/3j4EXBMRD6XX5IYUf0tumQURsaV0rBFxc0Q8lqYXA+uB365x3+8FvhIR\nqyNiJzAHOF3Sa3LLfDYifhMRG8jefZdex13AcZJeExE7I+K+Kvs4gqxA5s0G5kbEL1PeqyJie27+\nQF67aue+1vxKzgXWRsTX0rlfAdxKVuxL+vr9h5f/DewCroyI3elYajnfn4uIpyNiI/DPQOl6741k\nhbLk/by86/MZsvPd0VyM2sPG/BNJJ0j6rqQtyrrurgRe1cf6W3PTz5G9Yx7osq8pz4PsHWlfvhYR\no3OP8gu1ldbPx15Ddh0tbwMwLve8PKdy36D3H8OFZH30RMQz6flHgK2Slkqa2M+2qjkW+ESu6D4F\nHF2W5z7HKumS1K1XWv4E+n4N8/Y5L+lYnirbX09uOv86fpTs3f1/SvqpqnezPkXWgsk7hqygVFPT\na9fPua81v5JjgdPKzv0MsvNfMpDff4CeiNhd7ViqnO/8sW9I6xAR9wK7JJ2aum6PAf5f2f5KLcWO\n5mLUHsovsv4bsJqsm2MUMI+Xv7urty2UXZNh3z/G/VHp4nE+9muybsi8CcDmfraRtwT4/fQu9lxS\nMQKIiDsj4p1k/7h+QXZe9yfnjWTvpEtF98iIODwibq60nqTjgWuAPy8tD6yl9zXs75h+TfZPuLS9\nEcCR9P/mgNQK+VBEvIasGHwpdw0lby0wTNJRudivgN/qa/NlOR5XNv+l167auR9AfiUbgbvKzv3I\niLi8j3Uq5dtXvJbznf/bmJDWKbmRrEX0fmBJROzKbWtC2t+6GvI9qLkYtacRwI6IeF7SZODPm7DP\n7wJTJb0rXbT9a2p/J38g+5wi6fy0z/eS/TMsf2dZVUT0APcANwCPRMQv4KULyX8o6VBgN1m//Z4a\ntreX7JrEa3PhLwOXSfrttO3Dc9uu5HCybqon0nF9CDgxN78HGC9pSJX1vwHMlvT6NNDgc8CPImJL\nf/mnc1nqXtqR8njZcafuqLvZt/v3OuB/Snpt2tabJFXrXqr62vV17mvNL2cpcJKkCyUNkXSIpLfU\n0sqt8lpWUsv5/rikUam4/BXZtamSm8i6DS8kK0x5p5MV035/9w52LkbFUutw3r8FLpH0NHAt+/7i\nl2+nv23WtGy6vnEB8AWyP+DjyS5mv1htHbJrRvnPGT2dG0XUX6uIiHgCmE7WR/8EcDnZ0Pcd/eVb\n5utkF6a/losNBj5G9g72ceB3gMtq3N484BupW+i8iHgA+DBwraRtZANK8tcJyo9rNdmIueVp/xOB\n+3OLfI/snXKPpPw77NL6dwKfIrs2shkY39f+yp6/FVgu6RngZrKBL9VaVF8C8t1k/5j2+f3UPfxv\nwPAqx9jXa9fXuR9IfkTE08BZZAMktqRtfpZs4EGlc1Fun9eyyj76O98A3wFWAj8hG6CwMLf+BrKe\njBcj4v6y9S4iG+TQ8RTRuI8zSJoEfJPsF0Jk70A+SXYB75tkTd/1wIzSPxhJc4EPkr1jujwilqX4\nKWTvbocDt0fEX6f4ULJ3G28m+6W/ICJ+1bCDMiC7ewPZH/4fR8Q9rc7HGkPSPcCfxUH6wdcDlYZk\n7wKO6+v/jqSFwC8i4lO52MnAP0fE/2h8psXX0JZRGuU0NSJOISsWz5INOZ1D1jQ9gawrYC5knxcg\nu/g4GTgHuEZ66YOM1wKzI2ISMEm9n1qeTTZCaiKwALi6kcfUySSdlboihgH/AOwEHmxxWtZAEXGq\nC9GBSd2a04Gv5uMRsdKFqFczu+l+n+ydwUayC8mlZuxCoNQ8ng4sTkMq15N1VUxT9gGzERGxPC13\nY26d/LZuJuuOscY4jWw0VQ/wTuC8/MVYs0aQdLr2vW1Wqdt3W6tzS6p2L0n6LFl39mf66m607ENd\nzXIBvSOZxqYLy0TEVkml28SMA/KfK9icYrvZd+TKJnpHco0jDe+NiD3KPrk9OiKK8ot60IiIT5J1\ns5o1TUT8kJcPMy+ENPCg6t0TIuIK4IrmZdS+mtIyknQIWavnWynU1wXWA95dHbdlZmZN0KyW0TnA\nT9IIG8hGCY2NiJ7UBVe6m/Nm9h2vPz7FqsXz6/w6XUwcWalVJKkIN540M2s7EdHwN/nNumZ0IdlY\n/ZKlZPfvApgF3JaLz5Q0NH0w8HXAgxGxFdghaVoa0HBx2Tqz0vT5ZAMiKooC3H9pfx/z5s1reQ7O\nv/V5dFruzr/1j2ZpeMtI0mFkgxf+LBe+Clgi6YNkt86YARARayQtAdaQDZe8NHrPxmXsO7T7jhS/\nDlik7G7BT5LddNLMzNpIw4tRRDwHHFUW20ZWoCot/zmyTziXx39Cdtfc8viLpGJmZmbtyXdgaBNd\nXV2tTuGAOP/Waefcwfl3iobegaFIJEWnHKuZWb1IIg6iAQxmZmZVuRiZmVnLuRiZmVnLuRiZmVnL\nuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZ\nmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLuRiZmVnLNbwYSRol6VuSHpb0\nc0lvlXSkpGWS1kq6U9Ko3PJzJa1Ly5+Zi58iaZWkRyUtyMWHSlqc1rlP0oRGH5OZmdVXM1pGXwRu\nj4jJwJuAR4A5wF0RcQJwNzAXQNIUYAYwGTgHuEaS0nauBWZHxCRgkqSzUnw2sC0iJgILgKubcExm\nZlZHDS1GkkYC74iI6wEiYndE7ADOBRamxRYC56Xp6cDitNx6YB0wTdLRwIiIWJ6WuzG3Tn5bNwNn\nNPCQzMysARrdMjoeeELS9ZIekvQlSYcBYyOiByAitgJj0vLjgI259Ten2DhgUy6+KcX2WSci9gDb\nJY1u1AGZmVn9DWnC9k8BLouI/5T0BbIuuihbrvz5gVC1GfPnz39puquri66urjru1sys/XV3d9Pd\n3d30/SqinnWgbOPSWOC+iHhten4aWTH6LaArInpSF9wPImKypDlARMRVafk7gHnAhtIyKT4TOD0i\nPlxaJiIekDQY2BIRYyrkEo08VjOzg5EkIqLqm/x6aWg3XeqK2yhpUgqdAfwcWApckmKzgNvS9FJg\nZhohdzzwOuDB1JW3Q9K0NKDh4rJ1ZqXp88kGRJiZWRtpaMsIQNKbgK8AhwC/BD4ADAaWAMeQtXpm\nRMT2tPxcshFyu4DLI2JZir8ZuAEYTjY67/IUHwYsAqYCTwIz0+CH8jzcMjIzG6BmtYwaXoyKwsXI\nzGzgDopuOjMzs1q4GJmZWcu5GJmZWcu5GJmZWct1VDHau7fVGZiZWSUdVYz27Gl1BmZmVomLkZmZ\ntZyLkZmZtVxHFaPdu1udgZmZVdJRxcgtIzOzYnIxMjOzluuoYuRuOjOzYuqoYuSWkZlZMbkYmZlZ\ny3VUMXI3nZlZMXVUMXLLyMysmFyMzMys5TqqGLmbzsysmDqqGLllZGZWTC5GZmbWch1VjNxNZ2ZW\nTB1VjNwyMjMrJhcjMzNruYYXI0nrJf1U0gpJD6bYkZKWSVor6U5Jo3LLz5W0TtLDks7MxU+RtErS\no5IW5OJDJS1O69wnaUK1XNxNZ2ZWTM1oGe0FuiJiakRMS7E5wF0RcQJwNzAXQNIUYAYwGTgHuEaS\n0jrXArMjYhIwSdJZKT4b2BYRE4EFwNXVEnHLyMysmJpRjFRhP+cCC9P0QuC8ND0dWBwRuyNiPbAO\nmCbpaGBERCxPy92YWye/rZuBM6ol4mJkZlZMzShGAXxP0nJJf5piYyOiByAitgJjUnwcsDG37uYU\nGwdsysU3pdg+60TEHmC7pNGVEnE3nZlZMQ1pwj5OjYgtko4ClklaS1ag8sqfHwhVm3HTTfNZntpW\nXV1ddHV11XG3Zmbtr7u7m+7u7qbvt+HFKCK2pJ+PS7oVmAb0SBobET2pC+6xtPhm4Jjc6uNTrFo8\nv86vJQ0GRkbEtkq5zJgxn3e/u04HZmZ2ECp/o37llVc2Zb8N7aaTdJikw9P0K4AzgdXAUuCStNgs\n4LY0vRSYmUbIHQ+8DngwdeXtkDQtDWi4uGydWWn6fLIBERW5m87MrJga3TIaC9wiKdK+vhYRyyT9\nJ7BE0geBDWQj6IiINZKWAGuAXcClEVHqwrsMuAEYDtweEXek+HXAIknrgCeBmdWS8QAGM7NiUu//\n+oObpLjppuCii1qdiZlZ+5BERFS9Fl8vHXUHBnfTmZkVU0cVI3fTmZkVk4uRmZm1XEcVI3fTmZkV\nU0cVI7eMzMyKycXIzMxarqOKkbvpzMyKqaOKkVtGZmbF5GJkZmYt11HFyN10ZmbF1FHFyC0jM7Ni\ncjEyM7OW66hi5G46M7Ni6qhitHNnqzMwM7NKOqoYPf98qzMwM7NKOqoYvfBCqzMwM7NKOqoYuWVk\nZlZMLkZmZtZyLkZmZtZyHVWMfM3IzKyYOqoYuWVkZlZMLkZmZtZyHVWM3E1nZlZMTSlGkgZJekjS\n0vT8SEnLJK2VdKekUbll50paJ+lhSWfm4qdIWiXpUUkLcvGhkhande6TNKFaHm4ZmZkVU7NaRpcD\na3LP5wB3RcQJwN3AXABJU4AZwGTgHOAaSUrrXAvMjohJwCRJZ6X4bGBbREwEFgBXV0vCxcjMrJga\nXowkjQf+APhKLnwusDBNLwTOS9PTgcURsTsi1gPrgGmSjgZGRMTytNyNuXXy27oZOKNaLi5GZmbF\n1IyW0ReAjwGRi42NiB6AiNgKjEnxccDG3HKbU2wcsCkX35Ri+6wTEXuA7ZJGV0rkhRcgotIcMzNr\npSGN3LikdwE9EbFSUlcfi9azRKjqDM3nk5+EIUOgq6uLrq6+UjIz6zzd3d10d3c3fb+KBjYVJH0W\neB+wGzgUGAHcAvw20BURPakL7gcRMVnSHCAi4qq0/h3APGBDaZkUnwmcHhEfLi0TEQ9IGgxsiYgx\nZakgKUaODDZsgCOOaNghm5kdVCQREVXf5NdLQ7vpIuKKiJgQEa8FZgJ3R8T7ge8Al6TFZgG3peml\nwMw0Qu544HXAg6krb4ekaWlAw8Vl68xK0+eTDYio6NBDPbzbzKyIGtpN14fPA0skfZCs1TMDICLW\nSFpCNvJuF3Bp9DbdLgNuAIYDt0fEHSl+HbBI0jrgSbKiV9Hw4R7EYGZWRA3tpisSSXHiicG3vw1T\nprQ6GzOz9nBQdNMVzaGHumVkZlZE/RYjSYMl/VMzkmm04cN9zcjMrIj6LUbpszunNSGXhnPLyMys\nmGodwLAi3VfuW8CzpWBE/HtDsmoQFyMzs2KqtRgNJxup9nu5WABtV4zcTWdmVjw1FaOI+ECjE2kG\nD+02MyummkbTSRov6RZJj6XHt9MNUNuKu+nMzIqp1qHd15Pd6eA16fGdFGsrLkZmZsVUazE6KiKu\nT1/tsDsibgCOamBeDeGh3WZmxVRrMXpS0vvSZ44GS3of2YCGtuKWkZlZMdVajD5Idv+4rcAW4E+A\nthvU4GJkZlZM/Y6mS1/L8J6ImN6EfBrKQ7vNzIqp1jswXNiEXBrOQ7vNzIqp1g+93iPpX4Bvsu8d\nGB5qSFYN4m46M7NiqrUYnZx+fioXC/a9I0PhuZvOzKyYarlmNAi4NiKWNCGfhnI3nZlZMdVyzWgv\n8PEm5NJw7qYzMyumWod23yXp7yQdI2l06dHQzBrAxcjMrJhqvWZ0Qfp5WS4WwGvrm05j+ZqRmVkx\n1XrX7uMbnUgz+JqRmVkx9dlNJ+njuenzy+Z9tlFJNYq76czMiqm/a0Yzc9Nzy+adXedcGs7ddGZm\nxdRfMVKV6UrPC8/ddGZmxdRfMYoq05Wev4ykYZIekLRC0mpJ81L8SEnLJK2VdKekUbl15kpaJ+lh\nSWfm4qdIWiXpUUkLcvGhkhande6TNKFaPu6mMzMrpv6K0ZskPS3pGeCNabr0/A39bTwiXgR+NyKm\nkt3F4RxJ04A5wF0RcQJwN6kLUNIUsruDTwbOAa6RVGqBXQvMjohJwCRJZ6X4bGBbREwEFgBXV8vn\nkENg717Yvbu/zM3MrJn6LEYRMTgiRkbEiIgYkqZLzw+pZQcR8VyaHEY2ei+Ac4GFKb4QOC9NTwcW\npy/wWw+sA6ZJOhoYERHL03I35tbJb+tm4IxquUi+bmRmVkS1fuh1v0kaJGkF2XchfS8VlLER0QMQ\nEVuBMWnxccDG3OqbU2wcsCkX35Ri+6yT7jC+va8P5Pq6kZlZ8dT6odf9lm4nNFXSSOAWSSexH9ef\nBqDqwIr58+ezcyd89rNw7rlddHV11XG3Zmbtr7u7m+7u7qbvVxH1rAP97Ez6JPAc8KdAV0T0pC64\nH0TEZElzgIiIq9LydwDzgA2lZVJ8JnB6RHy4tExEPJC+CHBLRIypsO+ICCZNgu9+FyZNasohm5m1\nNUlERMNHTze0m07Sq0oj5SQdCrwTeBhYClySFpsF3JamlwIz0wi544HXAQ+mrrwdkqalAQ0Xl60z\nK02fTzYgoiqPqDMzK55Gd9O9GliYvoZiEPDNiLhd0v3AEkkfJGv1zACIiDWSlgBrgF3ApdHbdLsM\nuAEYDtweEXek+HXAIknrgCfZ94O6L+NrRmZmxdPUbrpWKnXTdXXBvHnwu7/b6ozMzIrvoOimKyIP\n7TYzK56OK0bupjMzK56OK0YewGBmVjwdWYzcTWdmViwdWYzcMjIzK5aOK0a+ZmRmVjwdV4zcMjIz\nK56OLEa+ZmRmViwdV4zcTWdmVjwdV4zcTWdmVjwdWYzcTWdmViwdWYzcMjIzK5aOK0a+ZmRmVjwd\nV4zcMjIzK56OLEa+ZmRmViwdV4zcTWdmVjwdV4zcTWdmVjwdWYzcTWdmViwdWYzcMjIzK5aOK0a+\nZmRmVjwdV4zcMjIzK56OK0bDh8POnRDR6kzMzKyk44qRBEOHehCDmVmRNLQYSRov6W5JP5e0WtJf\npfiRkpZJWivpTkmjcuvMlbRO0sOSzszFT5G0StKjkhbk4kMlLU7r3CdpQn95uavOzKxYGt0y2g18\nNCJOAn4HuEzSicAc4K6IOAG4G5gLIGkKMAOYDJwDXCNJaVvXArMjYhIwSdJZKT4b2BYRE4EFwNX9\nJeXh3WZmxdLQYhQRWyNiZZr+DfAwMB44F1iYFlsInJempwOLI2J3RKwH1gHTJB0NjIiI5Wm5G3Pr\n5Ld1M3BGf3m5ZWRmVixNu2Yk6TjgZOB+YGxE9EBWsIAxabFxwMbcaptTbBywKRfflGL7rBMRe4Dt\nkkb3lYuHd5uZFcuQZuxE0uFkrZbLI+I3ksrHstVzbJuqzZg/fz4ATz0F99zTxetf31XH3ZqZtb/u\n7m66u7ubvt+GFyNJQ8gK0aKIuC2FeySNjYie1AX3WIpvBo7JrT4+xarF8+v8WtJgYGREbKuUS6kY\nff/7MGXKgR6ZmdnBp6uri66urpeeX3nllU3ZbzO66b4KrImIL+ZiS4FL0vQs4LZcfGYaIXc88Drg\nwdSVt0PStDSg4eKydWal6fPJBkT0yd10ZmbF0tCWkaRTgYuA1ZJWkHXHXQFcBSyR9EFgA9kIOiJi\njaQlwBpgF3BpxEsfT70MuAEYDtweEXek+HXAIknrgCeBmf3l5QEMZmbFouiQWxFIeqmuXXABvOc9\n2U8zM6tOEhFR9Vp8vXTcHRjALSMzs6LpyGLka0ZmZsXSkcXId2AwMyuWji1GbhmZmRVHRxYjd9OZ\nmRVLRxYjt4zMzIqlY4uRrxmZmRVHxxYjt4zMzIqjI4uRrxmZmRVLRxYjd9OZmRVLxxYjt4zMzIqj\nI4uRu+nMzIqlI4uRW0ZmZsXSscXI14zMzIqjY4uRW0ZmZsXRkcXI14zMzIqlI4uRu+nMzIqlY4uR\nW0ZmZsXRkcWo1E3XId+4bmZWeB1ZjIYMgUGDYNeuVmdiZmbQocUIfN3IzKxIOroY+bqRmVkxdGwx\n8vBuM7PiaGgxknSdpB5Jq3KxIyUtk7RW0p2SRuXmzZW0TtLDks7MxU+RtErSo5IW5OJDJS1O69wn\naUKtubmbzsysOBrdMroeOKssNge4KyJOAO4G5gJImgLMACYD5wDXSFJa51pgdkRMAiZJKm1zNrAt\nIiYCC4Cra03M3XRmZsXR0GIUET8GnioLnwssTNMLgfPS9HRgcUTsjoj1wDpgmqSjgRERsTwtd2Nu\nnfy2bgbOqDU3FyMzs+JoxTWjMRHRAxARW4ExKT4O2JhbbnOKjQM25eKbUmyfdSJiD7Bd0uhakvA1\nIzOz4hjS6gSAen70VH3NnD9//kvTzz3XxQsvdNVx12Zm7a+7u5vu7u6m77cVxahH0tiI6EldcI+l\n+GbgmNxy41OsWjy/zq8lDQZGRsS2ajvOF6Of/cwtIzOzcl1dXXR1db30/Morr2zKfpvRTSf2bbEs\nBS5J07OA23LxmWmE3PHA64AHU1feDknT0oCGi8vWmZWmzycbEFGTCRPgrrv242jMzKzuFA28QZuk\nrwNdwCuBHmAecCvwLbIWzQZgRkRsT8vPJRshtwu4PCKWpfibgRuA4cDtEXF5ig8DFgFTgSeBmWnw\nQ6VcIn+s27fDm94E11wD73pXXQ/bzOygIYmI6PMSSF3208hiVCTlxQjgRz+CmTNh5UoYM6bKimZm\nHczFqM4qFSOAK66AVavgO98BNfx0m5m1l2YVo469HVDJ/PnQ0wPXXtvqTMzMOlfHt4wAHn0UTj01\n67abPLnJiZmZFZhbRk00aRJ85jPw3vfC00+3Ohszs87jYpR86ENw2mnwxjfC97/f6mzMzDqLu+nK\n3HFHVpimT4erroLDD29CcmZmBeVuuhY5+2xYvRqefTb7HNIPftDqjMzMDn5uGfVh6VK4/HIYNQo+\n8AG46CJ41asalKCZWQH5c0Z1tj/FCGDv3qx1dP318N3vwhlnwIUXwjveAWPHNiBRM7MCcTGqs/0t\nRnk7dsA3vwm33gr33w+jR8Pb3549Tj0VpkyBwYPrlLCZWQG4GNVZPYpR3t698MgjcO+9cM892c+e\nHnjrW7PC9Pa3Z9MjRtRtl2ZmTediVGf1LkaVPP443HdfVpjuvRceeggmTuxtPb397XDccb7tkJm1\nDxejOmtGMSq3cyesWNHberrnnqwQ5bv2pk6FoUObmpaZWc1cjOqsFcWoXARs2NDbrXfvvbBmDbz6\n1XDssZUfEyZkX5FuZtYKLkZ1VoRiVMnOnbBpU1akyh/r12fzjjyyerE69ths6LmZWSO4GNVZUYtR\nf/buhS1bKher0mPIkN5W1FFHZZ+FeuUrK/8cPdoj/sysdi5Gddauxag/EbBtW1aUfvUreOIJePLJ\n6j+3b4eRI/suWK98ZfYYPRqOOCJrmb3iFR54YdaJXIzq7GAtRgO1Z09WkPorWk88kS331FPZY+fO\n3sJU+lltujx2xBFujZm1KxejOnMxOjA7d/YWp3yRqmX66adh2LCsdXXYYdnP/HStsdL0oYdmj+HD\ne38OH54XsQWWAAAIeElEQVR1V5pZfbkY1ZmLUevs3QvPP5/dfPa557Kf+elKsWrzn30WXngh294L\nL/ROP/981voqFaZ8kapUuMqny5fJ/+xv3pAh7sK0g5eLUZ25GB3cImD37pcXqUqFq9r8UlEb6Ly9\ne3sL1NCh2WPYsJdP1xrLPw45JHvUa3rwYBdOGxgXozpzMbJGyRfBnTt7Hy+++PLpSrFK8198EXbt\n6n3s3Fl5uq95pel8bO/efYtTpYJV6Xk+NmRI9hg8uHe6/Hm16UrzSs/zPw8k1t88F+OBcTEaAEln\nAwvIvp/puoi4qsIyLkbW8fburV7U+ornY3v2ZAW49Bjo81KstK3S/L6mBxrraxmptsI1aFDvvNJ0\nrbFK8wcNevn0QOeddFL2rQHN1Kxi1PaXfCUNAv4FOAP4NbBc0m0R8UhrM6uv7u5uurq6Wp3GfnP+\nrZPPfdCgrDtw2LDW5jQQ9Tz3EVlB7qtg7d7du8yePb3TtcbK5//0p91MmdK1T6z0yD+vZd7BPEjn\nYDi0acC6iNgAIGkxcC7gYlQgzr912jl3qG/+Um/LpVn3hFy5spvzz+9qzs7a2MHwtePjgI2555tS\nzMzM2sTBUIzMzKzNtf0ABklvA+ZHxNnp+RwgygcxSGrvAzUzaxGPpquBpMHAWrIBDFuAB4ELI+Lh\nliZmZmY1a/sBDBGxR9JHgGX0Du12ITIzayNt3zIyM7P21xEDGCSdLekRSY9K+kSLc1kv6aeSVkh6\nMMWOlLRM0lpJd0oalVt+rqR1kh6WdGYufoqkVemYFuTiQyUtTuvcJ2nCAeZ7naQeSatysabkK2lW\nWn6tpIvrmP88SZskPZQeZxcxf0njJd0t6eeSVkv6qxRvi/NfIf+/TPF2Of/DJD2Q/lZXS5qX4oU/\n/33kXtxzHxEH9YOs4P4XcCxwCLASOLGF+fwSOLIsdhXw8TT9CeDzaXoKsIKsO/W4dByl1uwDwFvS\n9O3AWWn6w8A1afoCYPEB5nsacDKwqpn5AkcCvwBGAUeUpuuU/zzgoxWWnVyk/IGjgZPT9OFk10ZP\nbJfz30f+bXH+03YOSz8HA/eTfa6xXc5/pdwLe+47oWX00odiI2IXUPpQbKuIl7dIzwUWpumFwHlp\nejrZC7w7ItYD64Bpko4GRkTE8rTcjbl18tu6mWxgx36LiB8DTzUx399L02cByyJiR0RsJ7sm+NK7\nuAPMH7LXody5Rco/IrZGxMo0/RvgYWA8bXL+q+Rf+gxg4c9/yvu5NDmM7B910D7nv1LuUNBz3wnF\nqGgfig3ge5KWS/rTFBsbET2Q/QEDY1K8PPfNKTaO7DhK8sf00joRsQfYLml0nY9hTAPz3ZHyrbat\nevmIpJWSvpLrZils/pKOI2vh3U9jf18anf8DKdQW51/SIEkrgK3A99I/5bY4/1Vyh4Ke+04oRkVz\nakScAvwBcJmkd9D7jqWknqNKmnGP4nbL9xrgtRFxMtkf6v+q47brnr+kw8neeV6eWhht9ftSIf+2\nOf8RsTcippK1SKdJOok2Of8Vcp9Cgc99JxSjzUD+Iv74FGuJiNiSfj4O3ErWjdgjaSxAahY/lhbf\nDByTW72Ue7X4Puso+wzWyIjYVufDaEa+DXvdIuLxSJ3bwJfJXoNC5i9pCNk/8kURcVsKt835r5R/\nO53/koh4Gugm625qm/Nfnnuhz/1ALoi144Ps4l1pAMNQsgEMk1uUy2HA4Wn6FcA9wJlkF0Q/EdUv\niA4Fjmffi4qlC5Iiu6h4dopfSu9FxZkc4ACGtJ3jgNW55w3Pl30vgpamj6hT/kfnpv8G+HpR8yfr\no//fZbG2Of9V8m+L8w+8inThHTgU+BFZj0bhz38fuRf23Df9H3IrHmTvZtaSXZSb08I8jicrhiuA\n1aVcgNHAXSnHZfkXDpibfjEeBs7Mxd+ctrEO+GIuPgxYkuL3A8cdYM5fJ/tqjheBXwEfSL9gDc8X\nuCTFHwUurmP+NwKr0mtxK9k1gMLlD5wK7Mn9zjyUfpeb8vvSwPzb5fy/IeW8MuX79838ez2Q/PvI\nvbDn3h96NTOzluuEa0ZmZlZwLkZmZtZyLkZmZtZyLkZmZtZyLkZmZtZyLkZmZtZyLkZmNZL0TPp5\nrKQL67ztuWXPf1zP7ZsVnYuRWe1KH8o7HnjvQFZMt0vpyxX77CjitIFs36zduRiZDdzngNPSl5Nd\nnu6OfHX6MrOVkj4EIOl0ST+SdBvw8xS7Jd2xfXXpru2SPgccmra3KMWeKe1M0j+m5X8qaUZu2z+Q\n9K30ZWiLcst/XtLPUi5XN+2smB2AIa1OwKwNzQH+NiKmA6Tisz0i3ippKHCPpGVp2anASRHxq/T8\nAxGxXdJwYLmkb0fEXEmXRXY395JI2/5j4I0R8QZJY9I6P0zLnEx2T7GtaZ9vBx4BzouIE9P6Ixt1\nEszqyS0jswN3JnBx+u6YB8juXTYxzXswV4gA/lrSSrJ7eY3PLVfNqcA3ACLiMbK7L78lt+0tkd3T\nayXZDWF3AM+n76p5N/D8AR6bWVO4GJkdOAF/GRFT0+O3IuKuNO/ZlxaSTif7Nsy3RvZ9MiuB4blt\n1Lqvkhdz03uAIZF9ydk0sq9t+EPgjgEfjVkLuBiZ1a5UCJ4BRuTidwKXpu/uQdJESYdVWH8U8FRE\nvCjpROBtuXk7S+uX7es/gAvSdamjgHcAD1ZNMNvvERFxB/BR4I21H55Z6/iakVntSqPpVgF7U7fc\nDRHxxfS12g9JEtmXrZ1XYf07gL+Q9HOyrx+4LzfvS8AqST+JiPeX9hURt0h6G/BTYC/wsYh4TNLk\nKrmNBG5L16Qg+84as8LzV0iYmVnLuZvOzMxazsXIzMxazsXIzMxazsXIzMxazsXIzMxazsXIzMxa\nzsXIzMxazsXIzMxa7v8DmoiiPo8WAIMAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x12ae0d9e8>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEZCAYAAABSN8jfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4HFWZ7/HvLwkJICTcBIcACXIHYRAfYkZRtqIQ0REE\nR4KDKN5wHDheZhzgHD1mbgjMjKMzqGdwIgIKEWEEVBRkhs1NgXC/mJAoGpKA4R4IISEk7/ljrU4q\nne7eveldu6t3fp/nqafrsrrqrerueqvWqq5SRGBmZtaJUd0OwMzMep+TiZmZdczJxMzMOuZkYmZm\nHXMyMTOzjjmZmJlZx5xMSiRplKTnJe00lGVt5JN0oqSfdmG550j69HAvd2MgaVNJcyVt3e1YyuBk\nUpB35s/lbrWk5YVxxw92fhGxJiK2jIhFQ1n2lZL0cUlrJL2vrGWMBJIOk/S7wvBNkk4scXm7SVpT\nHBcRF0bEu8taZpM4dgCmA/85nMsdTmV/lq1ExArgu8Bp3Vh+2ZxMCvLOfHxEjAcWAO8ujLukvryk\n0cMfZUdOBJ7Kr8NKUq9914bs37xtrLuGcnkdOAn4cUS8NJg3KSsppmE1DL/pi4GTenDfMbCIcNeg\nA34HvL1u3N8Ds0hfiKWknfJU4FfAM8Bi4OvA6Fx+NLAG2CUPX5SnXw08B9wCTBps2Tz9XcBDebn/\nBtwMnNhifXYDXgbeB6wEtq2bfgxwd16vecA78vhtgPOBR0mJ6Id5/MeA6wvvbxT/ucDPgOeBtwJ/\nWljG74Ev1sXw1rwtnyUl8z/P23dxXbkPAHc0WMc3AYvqxv0ZcGfunwrcmZf/GHB2k211GPBw7j8r\nb7fl+XP4ah6/L/CLvE1+DRxTeP+g1j1/b1bnss8Bb2iwfQ8BZufP+1ZgSmHaTcCM/B15Ln9ntsrT\nNgO+DzxZeO82Tdb7BuADbX4vbgL+Dvgl8AKwCzAR+HHeJg8BJxXm88ZG234w8RXmNQH4Duk7+Qjw\nt4VpHwP6ga/m+f2mEPMGnyXrvrd/AcwH5rW5vf+hMP1yYEKe9nPg5Lp4HyQdmNaGfwv8Sbf3cUPd\ndT2AqnY0TyYrgCPz8Lj8wz+YdHQ5GZgLfDpPH513EsUd7OPA6/O0WcCFr6Ds9vnH8J487XOkBNEq\nmfwtcHPu/zVwamHam/KPoi8PTwT2yP3XAN8DxudlHZLHfwz4n8I8GsX/VO1HCIwF+oB98vD+ef1q\n23JX0s70WNIZ8zbAAXnaHOCwwrKuAk5psp4PA4cWhv8L+Fzuvx04Lve/Cji4yTzWJpM8fBPwocLw\nq4BFpGSn/Bk9Wdhmg1333YDVdTGs3b7AdqQE+4G8bU7Iy5tQiO8h4LXApsCNwN/laZ/O22BsjvUg\nYPMm6/008Mdtfi9uytt6z/zZjyYd0HwN2CRvkyeAt7Ta9oOJr+7z/3fS7+/VpJ36SYXtthL4cJ7f\nKcAjLT7LWjK5mpSkxgHbtrG9FwB7kZLhj4Dv5GnHk39nefgNwB+AUYVxPwU+1e193FB3vVb1UAU3\nR8TVABGxMiLujIjZkfwe+DZwaKF8/en/ZRFxd0SsJh2RHfgKyr4buDsifhIRqyPiX0k7r1Y+lOcB\n6cyqWNX1UeC8iOjP67U4IubniwHeRvriP5eXdXOLZdTH/6OIuD3P86WI6I+IOXn4fuAHrNtWfw5c\nHRGXR2o/ejoi7svTLsrxI2k74O2k5NrID4AP5rJbAUcUyr4E7CFpm4h4ISJmt1iXVut2FPBQRHw/\nf+53A1cA73+F6z6Q9wAPRMSledt8j7QjL7apzIyIhyPVy/+Qdd+VVaRktGeO9a6IWN5kORNICb2m\n4feiMP07ETEvfz93Jh1UnR4Rq/I2OZ/8udF82w8mPiTtCLwD+Hz+/T1BOoMvtmn+NiIuiLTnvgDY\nSdI2xdk0mPU/RsTSiFhJOoscaHtfEBEPRcSLwP8tLP9HwL6SJuXhE4BZEVFsE3se2KrZOvYqJ5PB\nW1gckLSXpJ9IekzSUtIZwHYt3v+HQv9yYItXUHbH+jhIR8oNSTqUdFR5aR51CfAGSfvm4Z1Jp971\ndgaejIhlLWJspX5b/Ymk6yU9LulZ0lFkbVs1iwFSMnmvpHGkBuLrI+LJJmUvBo7JddLHArdGxGN5\n2knAfsBDkm6V9K5XuF6TgEMkPZ27Z0hHsa8plBnMug9kR9KRcNEC0mda0+y78l3gOuBSSQslndmi\nDedZYMvCcKvPBNZfxx1J35UVTWJstu0HEx+kbT8OWFLY9ueSzlBq6rcFtP6dwfq/n3a298K6aeMk\nbZ3X/zLghLwe00nf36ItSdt6RHEyGbz6htL/AO4HXhsRE4Av0/jIZyg9RvqhF01sVDD7MOmzvl/S\nY6TqiDV5PKQfxm4N3rcQ2E5Sox/iC8DmheE/YsNtUz98CemoeWJEbAXMZN22Wgjs3ij4iFhIqm9/\nH+lIr/7HWSx7P2lnMo10tHhxYdr8iDg+Il5Nqi+/XNLYZvNqsR4LgesiYpvcbR3pIo3PtHhPq3Uf\nqPH9UVIVatEupLaW1oGns4S/i4h9Se0Ax5DOAhu5j1RtVdPse7F29nUxbidps0YxNtv2g4yvFtML\nddt+q4h4fYv3NIu51bpMrptev72Lv79JwIqIeCYPX0j6nh4OPB0Rd9bNax/g3jbj7RlOJp3bElga\nES9K2gc4eRiW+RPg9ZLeLWm0pM/S5Cg3/7iPJVVZHAj8ce4+Tzp6EmnH9nFJh+YLcyZK2jPSZcrX\nAd+QNEHSGElvybO+FzhA0n55Gf+3jbi3AJ6JiFWSppKO2mq+Bxwh6X15nbaVdEBh+kXAGaR66isH\nWM7FpHakqaSjxNq2OEHStnnwOVJCXbPh2zewhNQeUXMVsJ+k4/M22UTSwZL2aDGPVuv+OBCSdm3y\n3p+Qqk7+LG+bD5J28gP+D0XS2/JnJGAZqVqp2TpfTWrbqWn0vWi4jrmK9w7gTEljJR1IOhu5KMfR\ncNsPMj7yd/IGSf8iacsc126F7+VA6j/LRtrZ3ifmWolXkS5++EEhxptJbUBnU3fgI2lnUpvRYKpY\ne4KTSXPtXqr5V8BHJD0HfIsN6/KjSf9Ay2xaNiIeB44D/pXUMLgr6YqblQ2KH0P68X4/Ih6vdaS2\nnU2Bd0bEr4BPkBo1lwL/A9T+PHkC6Qh6HumI/5QcwxzgTNIVQHPya7N1qfkL4KxcHXg66/8Af0+q\nqz6d1BB8J/C6wnsvJ+0EfpjrtVu5hNTWc21ELC2MPxKYk5d/DunKpZcHmBekRuUP5mqVf46I50ht\nMSeQzhIfJW2LcbXVaTCPVuu+DPgKcFtexkHFN+Yqvffm9z0JfIZ0dVBt3Vp9r3YkNXAvJZ1BX0vh\nbK3OBcB7JG2Sl9voe1E7Im+0zONIZzZ/IFWpnh4RN+Vpzbb9YOKrOYG0Q/416btyKbBDi/LFWNf7\nLButSxvbG1KS+B7pbEWkg5eiC0nVet+vG38CcH6b37ueotRGVdLMpZmkxsMlEXFAkzL/RrrM9QXg\nIxFxT2kBjVC5bvZR4NiIuKXb8ZRF6Y+EH46IG7sdy0gl6SzS1U/f7HYsVSXpJuDbEXFhizInka4a\ne3th3Kakg743R8TT5Uc6vMo+MzmfdATXUG6E2y0i9iBVD/2/kuMZMSQdkauexpGqmF4iXX45Ikk6\njlQv7URSoog43YmkM7nq69Ok9tS1ImJFROwzEhMJlJxMct3hMy2KHEU6HSQibgMmKN3SwQZ2COly\nxSXAO4GjI2JVd0MqRz4S/BrpB2ojnNbdp+65QlcbfmO346NFtaKkI0m/yd+z7urJjcKYLi9/Iutf\nYrc4j1vSnXB6R0R8CfhSt+MYDhHRbuOqjQD5PxlbDliwSyLirS2mXc3AlyGPSG6ANzOzjnX7zGQx\n61+vvRNNrp2XVN6VAmZmI1hElH4jzuE4MxHN/8R3Ffm2Hvna+2cjomkVV1Tg/jOvtPvyl7/c9Rgc\nf/fj2Bjj7+XYR0L8w6XUMxNJF5P+BLWtpEdI/w4fC0REnBcRV0s6UtJvSJcGn1RmPGZmVo5Sk0lE\nfLCNMqeUGYOZmZWv220mG42+vr5uh9ARx99dvRx/L8cOreNfswZefhlWr06vta44XOwfOxb2aHXT\nnR5W6j/gh5Kk6JVYzay1iLRzXbECVq5Mry+91Lxbtar19EZlX3459a9ata5/oNdm05oliAgYM2Zd\nN3p08+HRo2H//eEHPxh4+wwlScQwNMD7zMRshItYt6Os7XBXrkxdrX+wr8X+YkKof201bfRoGDcO\nNt00vY4bl47ca90mm6w/3Kqrld1yy3XDm2ySduLF10bjmr3W+uu7YoIY5T9XrOVkYlaSNWvSTvPF\nF9d1y5cPfrh41F5MCLX+gca9/HLa8dV2srWddqvXgcrUdtq1RDDY13HjUkw2criayzZqEelI+bnn\nUvfCC+t3y5ZtOK6daS+8kOY7bhxsttm6bvPNBz9c2wEXj9Rr/fWvzaap9EoOq6rhquZyMrGeFJGO\n2J9/HpYuXZcMal39uFZlRo2C8ePT0fYWW8CrXtVe16psLSm4GsS6zcmkjpNJ74tI1TbPPJOSwPPP\np515rb/YtTN+zJiUACZMSMmg1hWHW02rdePGDRy7Wa9yMqnjZFI9q1bBU0/Bk0/CE0+k1/r++mGA\nrbdedyZQ3zUa36zs2HYeuGu2kXMyqeNkMjwi0k5/4cLULVqUXpcs2TBBLFsG224L222Xule/el1/\ns+HNNx84BjMbOk4mdZxMOhcBTz+9YaIo9i9enHb4O+8MO+207vU1r9kwOWy1ldsEzKrOyaSOk8nA\nVq1KCeHhh1P3yCMbJoxx49ZPFPX9EyemBmQzGxmcTOo4maw7s/jd79YljGK3eDH80R/Ba18Lu+4K\nkyZtmDS22Cgf22O28XIyqbOxJJOXXoIFCxoni4cfTmV22y0ljPpul13cKG1m63MyqTPSksnKlTB3\nLtx/PzzwwLruscfSGUTt7KI+YWy9tf+AZmbtczKp06vJZPXqVC1VSxq119/9LiWH170u3fztda9L\n3eTJvs2EmQ0dJ5M6VU8mEemsonaGUUsav/51ugqqljBqr3vt5T/LmVn5RkwykTQN+BrpEcEzI+Ls\nuulbAd8BdgNeBD4aEb9uMJ9KJZPVq6G/H668Eu69NyWOUaNSsiieaey3X/rTnZlZN4yIZCJpFDAP\nOAx4FJgNTI+IuYUy5wDPR8TfS9oL+EZEvKPBvLqeTCLg1lvhkkvghz9Ml9G+//1w8MEpceywQ1fD\nMzPbwEh5nskUYH5ELACQNAs4CphbKLMv8BWAiHhI0mRJr46IJ0qOrS0RcN99KYHMmpVu3nf88XDD\nDbDnnt2OzsysGspOJhOBhYXhRaQEU3QvcAxwi6QpwC7ATkBXk8n8+esSyPLlMH16qtI64ABfTWVm\nVq8K1w2dBXxd0l3A/cDdwOpGBWfMmLG2v6+vb8ifLb1wIVx6aUoiixbBBz4AM2fC1KlOIGbWG/r7\n++nv7x/25ZbdZjIVmBER0/Lw6UDUN8LXved3wP4RsaxufCltJk88AZddlhLIgw/C+96XqrEOPdSX\n6JpZ7xspbSazgd0lTQIeA6YDxxcLSJoALI+IVZI+AdxQn0iG2tKlcMUVKYHceisceSR84Qtw+OG+\nXNfM7JUoNZlExGpJpwDXsu7S4DmSTk6T4zxgH+ACSWuAB4GPlRXP8uVw5plw7rnQ1wcnnQSXX+4b\nG5qZdWqj+dPiT38Kp56aLuP96lfTZb1mZiPdSKnm6rqFC+Gzn01/LPzWt+CII7odkZnZyDOiH230\ns5/BQQelf6Q/8IATiZlZWUbsmcm558I//mNqaH/zm7sdjZnZyDbiksnq1fC5z8F118Ett6Q785qZ\nWblGVDJ5/vn0H5GVK+GXv0zPKDczs/KNmDaTlSvhne+EHXeEq692IjEzG04j5tLgU06BRx9N/xvx\nrU/MzBJfGjwIl1wC11wDd9zhRGJm1g09f2YyZw689a3wi1/AgQd2ITAzswobrjOTnm4zWbYMjj0W\nzjrLicTMrJt69swkAj70IdhkE/jOd1y9ZWbWiNtMBnDxxekJiLfe6kRiZtZtPVvNdcst8IlPwOab\ndzsSMzPr2WTy+OOwww7djsLMzKCHk8mSJU4mZmZV4WRiZmYdKz2ZSJomaa6keZJOazB9vKSrJN0j\n6X5JH2lnvk4mZmbVUeqlwZJGAfOAw4BHSc+Enx4RcwtlzgDGR8QZkrYDHgJ2iIiX6+a19tLgFStg\n/Ph0Py5fyWVm1txI+dPiFGB+RCyIiFXALOCoujIBbJn7twSeqk8k9ZYsge23dyIxM6uKspPJRGBh\nYXhRHld0LrCvpEeBe4HPDDRTX8llZlYtVfjT4hHA3RHxdkm7Ab+QdEBELKsvOGPGDADmzYMxY/qA\nvmEM08ys+vr7++nv7x/25ZbdZjIVmBER0/Lw6UBExNmFMj8BvhIRt+Th/wZOi4g76ua1ts1k5ky4\n+WY4//zSQjczGxFGSpvJbGB3SZMkjQWmA1fVlVkAvANA0g7AnsDDrWbqK7nMzKql1GquiFgt6RTg\nWlLimhkRcySdnCbHecA/AN+VdF9+299ExNOt5rtkCUyeXGbkZmY2GKW3mUTEz4G96sb9R6H/MVK7\nSduWLIEpU4YmPjMz61xP/gPeV3OZmVVLTyYTt5mYmVWLk4mZmXWs5560uGpVeobJihUwenS3ozIz\nq7aRcmnwkHviCdh2WycSM7Mq6blkUrsvl5mZVUfPJRNfyWVmVj09l0zc+G5mVj1OJmZm1jEnEzMz\n65iTiZmZdczJxMzMOtZzyeTxx31psJlZ1fRcMvGZiZlZ9fTU7VRWrw7GjYPly2GTTbodkZlZ9fl2\nKg089RSMH+9EYmZWNaUnE0nTJM2VNE/SaQ2m/7WkuyXdJel+SS9L2qrRvFzFZWZWTaUmE0mjgHNJ\nT1LcDzhe0t7FMhHxzxHx+og4CDgD6I+IZxvNz8nEzKyayj4zmQLMj4gFEbEKmAUc1aL88cAlzSb6\nSi4zs2oqO5lMBBYWhhflcRuQtBkwDbi82cx8ZmJmVk1juh1AwZ8CNzer4gK47LIZjB0LM2ZAX18f\nfX19wxacmVkv6O/vp7+/f9iXW+qlwZKmAjMiYloePh2IiDi7Qdn/Ai6NiFlN5hUnnRS86U3w8Y+X\nFrKZ2YgyUi4Nng3sLmmSpLHAdOCq+kKSJgCHAle2mpmruczMqqnUaq6IWC3pFOBaUuKaGRFzJJ2c\nJsd5uejRwDUR8WKr+TmZmJlVU0/9A37nnYMbb4TJk7sdjZlZbxiuaq6eSibjxgXPPAObbdbtaMzM\nesNIaTMZUmPHOpGYmVVRTyUTt5eYmVWTk4mZmXWsp5KJb6ViZlZNPZVMNt202xGYmVkjPZVMRvVU\ntGZmG4+e2j2r9IvbzMzsleipZOIzEzOzauqp3bOTiZlZNfXU7tnJxMysmnpq9+w2EzOzahowmUg6\nVdLWwxHMQHxmYmZWTe3snncAZku6VNI0qXvnB04mZmbVNODuOSK+COwBzAQ+AsyXdKak3UqObQNO\nJmZm1dTW7jnSfer/kLuXga2ByySdM9B789nMXEnzJJ3WpEyfpLslPSDp+ubzaidaMzMbbgM+aVHS\nZ4ATgSeB/wS+EBGrJI0C5gN/0+K9o4BzgcOAR0nVZVdGxNxCmQnAN4DDI2KxpO2azc9nJmZm1dTO\nY3u3AY6JiAXFkRGxRtJ7BnjvFGB+7b2SZgFHAXMLZT4IXB4Ri/N8n2w2MycTM7Nqamf3/DPg6dqA\npPGS3ggQEXMGeO9EYGFheFEeV7QnsI2k6yXNlvShpsE6mZiZVVI7u+dvAcsKw8vyuKEyBjgIeBcw\nDfiSpN0bFXQyMTOrpnaquRSFB8Xn6q123gewGNilMLxTHle0CHgyIlYAKyTdCPwx8Jv6mf3qVzOY\nMSP19/X10dfX12YYZmYbh/7+fvr7+4d9uSrkicYFpP8C+ll3NvJp4G0RcfSAM5dGAw+RGuAfA24H\nji9Wj0naG/h30lnJOOA24LiI+HXdvOILXwjOGfD6MTMzq5FERJR+LWw7FUefAt5EOqNYBLwR+GQ7\nM4+I1cApwLXAg8CsiJgj6WRJn8xl5gLXAPcBtwLn1SeStcG6msvMrJIGPDOpCklxxhnBmWd2OxIz\ns94xXGcm7fzPZFPgY8B+wNoH50bER0uMq0ksw71EMzNrRzsVRxcBrwGOAG4gNaI/X2ZQzbiay8ys\nmtrZPe8eEV8CXoiIC4B3k9pNhp2TiZlZNbWze16VX5+V9DpgArB9eSE152RiZlZN7fxf5Lz8PJMv\nAlcBWwBfKjWqJpxMzMyqqWUyyTdqfC4ingFuBF47LFE1jaebSzczs2ZaHutHxBpa3BV4uPnMxMys\nmtrZPV8n6a8l7Sxpm1pXemQNOJmYmVVTO20mx+XXvyyMC7pQ5eVkYmZWTQMmk4jYdTgCaYfbTMzM\nqqmdf8Cf2Gh8RFw49OG05jMTM7Nqaqea6+BC/6akOwDfBTiZmJkZ0F4116nFYUlbAbNKi6gFJxMz\ns2p6JbvnF4CutKM4mZiZVVM7bSY/Jl29BSn57AtcWmZQzWPpxlLNzGwg7bSZ/HOh/2VgQUQsKime\nlnxmYmZWTe3snh8BbouIGyLiFuApSZPbXYCkaZLmSpon6bQG0w+V9Kyku3L3xabBOpmYmVVSO2cm\nPyQ9trdmdR53cOPi6+R7e51LugLsUWC2pCvzo3qLboyI9w40PycTM7Nqamf3PCYiXqoN5P6xbc5/\nCjA/IhZExCrSVWBHNSjXVmuI20zMzKqpnWTyhKS1Zw2SjgKebHP+E4GFheFFeVy9P5F0j6SfStq3\nabA+MzEzq6R2qrk+BXxf0rl5eBHQ8F/xr9CdwC4RsVzSu4ArgD0bFXQyMTOrpnb+tPhbYKqkLfLw\nskHMfzGwS2F4pzyuOP9lhf6fSfqmpG0i4un6mV155QwWLEj9fX199PX1DSIUM7ORr7+/n/7+/mFf\nriKidQHpTOCciHg2D28N/FVENL3qqvDe0cBDpAb4x4DbgeMjYk6hzA4RsST3TwEujYjJDeYVF1wQ\nnDiU50RmZiOcJCKi9BbndiqO3lVLJAD5qYtHtjPziFgNnAJcCzwIzIqIOZJOlvTJXOz9kh6QdDfw\nNdbd8n7DYF3NZWZWSe20mYyWNC4iVgJI2gwY1+4CIuLnwF514/6j0P8N4BvtzMvJxMysmtpJJt8H\n/lvS+aRLeD8CXFBmUM04mZiZVVM7DfBnS7oXeAfpHl3XAJPKDqwRJxMzs2pqd/e8hJRI/gx4OzCn\ndfFy+E+LZmbV1PTMRNKewPG5exL4Aenqr7cNU2wb8JmJmVk1tarmmgvcBLwnIn4DIOlzwxJVE04m\nZmbV1Gr3fAzpvyHXS/q2pMNo8x5aZXEyMTOrpqa754i4IiKmA3sD1wOfBbaX9C1Jhw9XgEVuMzEz\nq6YBj/Uj4oWIuDgi/pR0O5S7gQ2eSzIcfGZiZlZNg9o9R8QzEXFeRBxWVkCtOJmYmVVTT+2enUzM\nzKqpp3bPTiZmZtXUU7tnN8CbmVVTTyUTn5mYmVVTT+2enUzMzKqpp3bPTiZmZtXUU7tnt5mYmVVT\n6clE0jRJcyXNk9T0z46SDpa0StIxzcr4zMTMrJpK3T1LGgWcCxwB7AccL2nvJuXOIj0rpSknEzOz\naip79zwFmB8RCyJiFTALOKpBuVOBy4DHW83MycTMrJrK3j1PBBYWhhflcWtJ2hE4OiK+xQB3JXYy\nMTOrpnaeAV+2r7H+jSObJpRvf3sGV1+d+vv6+ujr6ys1MDOzXtPf309/f/+wL1cRUd7MpanAjIiY\nlodPByIizi6UebjWC2wHvAB8MiKuqptX3HFH8IY3lBaumdmII4mIKP1a2LLPTGYDu0uaRHrQ1nTS\nY4DXiojX1volnQ/8uD6R1Liay8ysmkpNJhGxWtIpwLWk9pmZETFH0slpcpxX/5ZW83MyMTOrplKr\nuYaSpLj33uCAA7odiZlZ7xiuaq6eOtb3mYmZWTX11O7ZycTMrJp6avfsZGJmVk09tXv2jR7NzKqp\np5KJz0zMzKqpp3bPTiZmZtXUU7tnJxMzs2rqqd2zk4mZWTX11O7ZDfBmZtXUU8nEZyZmZtXUU7tn\nJxMzs2rqqd2zk4mZWTX11O7ZbSZmZtXUU8nEZyZmZtXUU7tnJxMzs2rqqd2zk4mZWTWVvnuWNE3S\nXEnzJJ3WYPp7Jd0r6W5Jt0t6c7N5OZmYmVVTqU9alDQKmAccBjxKeib89IiYWyizeUQsz/37A5dG\nxD4N5hVLlwbjx5cWrpnZiDNSnrQ4BZgfEQsiYhUwCziqWKCWSLItgDXNZuYzEzOzaip79zwRWFgY\nXpTHrUfS0ZLmAD8GPtpsZk4mZmbVNKbbAQBExBXAFZIOAf4BeGejcmeeOYMxOeK+vj76+vqGK0Qz\ns57Q399Pf3//sC+37DaTqcCMiJiWh08HIiLObvGe3wIHR8TTdeNjxYpg3LjSwjUzG3FGSpvJbGB3\nSZMkjQWmA1cVC0jardB/EDC2PpHUuJrLzKyaSq3miojVkk4BriUlrpkRMUfSyWlynAccK+lE4CXg\nReADzebnZGJmVk2lVnMNJUmxZk34/lxmZoMwUqq5hpQTiZlZNfVUMjEzs2pyMjEzs445mZiZWcec\nTMzMrGNOJmZm1jEnEzMz65iTiZmZdczJxMzMOuZkYmZmHXMyMTOzjjmZmJlZx5xMzMysY04mZmbW\nMScTMzPrmJOJmZl1rPRkImmapLmS5kk6rcH0D0q6N3c3S9q/7JjMzGxolfqkRUmjgHnAYcCjpGfC\nT4+IuYUyU4E5EbFU0jRgRkRMbTCv6JWnQpqZVcVIedLiFGB+RCyIiFXALOCoYoGIuDUilubBW4GJ\nJcdkZmZDrOxkMhFYWBheROtk8XHgZ6VGZGZmQ25MtwOokfQ24CTgkGZlZsyYsba/r6+Pvr6+0uMy\nM+sl/f399Pf3D/tyy24zmUpqA5mWh08HIiLOrit3AHA5MC0ifttkXm4zMTMbpJHSZjIb2F3SJElj\ngenAVcWWFRnbAAAH9klEQVQCknYhJZIPNUskZmZWbaVWc0XEakmnANeSEtfMiJgj6eQ0Oc4DvgRs\nA3xTkoBVETGlzLjMzGxolVrNNZRczWVmNngjpZrLzMw2Ak4mZmbWMScTMzPrmJOJmZl1zMnEzMw6\n5mRiZmYdczIxM7OOOZmYmVnHnEzMzKxjTiZmZtYxJxMzM+uYk4mZmXXMycTMzDrmZGJmZh1zMjEz\ns46VnkwkTZM0V9I8Sac1mL6XpF9KWiHp82XHY2ZmQ6/UZCJpFHAucASwH3C8pL3rij0FnAr8U5mx\ndFt/f3+3Q+iI4++uXo6/l2OH3o9/uJR9ZjIFmB8RCyJiFTALOKpYICKejIg7gZdLjqWrev0L6fi7\nq5fj7+XYoffjHy5lJ5OJwMLC8KI8zszMRhA3wJuZWccUEeXNXJoKzIiIaXn4dCAi4uwGZb8MPB8R\nX20yr/ICNTMbwSJCZS9jTMnznw3sLmkS8BgwHTi+RfmmKzwcG8PMzF6ZUs9MIF0aDHydVKU2MyLO\nknQy6QzlPEk7AHcAWwJrgGXAvhGxrNTAzMxsyJSeTMzMbOTriQb4gf74OMyx/F7SvZLulnR7Hre1\npGslPSTpGkkTCuXPkDRf0hxJhxfGHyTpvrxOXyuMHytpVn7PryTt0mG8MyUtkXRfYdywxCvpw7n8\nQ5JOHML4vyxpkaS7cjetivFL2knS/0h6UNL9kv5XHt8T279B/Kfm8b2y/cdJui3/Vu9Xapftpe3f\nLP5qbv+IqHRHSni/ASYBmwD3AHt3MZ6Hga3rxp0N/E3uPw04K/fvC9xNapuanNejdjZ4G3Bw7r8a\nOCL3/wXwzdx/HDCrw3gPAQ4E7hvOeIGtgd8CE4Ctav1DFP+Xgc83KLtPleIHXgMcmPu3AB4C9u6V\n7d8i/p7Y/nk+m+fX0cCtpP++9cT2bxF/Jbd/L5yZDPjHx2EmNjyjOwq4IPdfAByd+99L+nBejojf\nA/OBKZJeA2wZEbNzuQsL7ynO6zLgsE6CjYibgWeGMd635/4jgGsjYmlEPAtcC6w9guowfmh8scZR\nVYo/Iv4QEffk/mXAHGAnemT7N4m/9j+xym//HPfy3DuOtJMNemT7t4gfKrj9eyGZVO2PjwH8QtJs\nSR/P43aIiCWQfoDA9nl8feyL87iJpPWoKa7T2vdExGrgWUnbDPE6bF9ivEtzvM3mNVROkXSPpP8s\nVFNUNn5Jk0lnWLdS7vel7Phvy6N6YvtLGiXpbuAPwC/yDrVntn+T+KGC278XkknVvDkiDgKOBP5S\n0ltYd7RQM5RXNQzHJdG9Fu83gddGxIGkH9m/DOG8hzx+SVuQjvo+k4/we+r70iD+ntn+EbEmIl5P\nOiOcImk/emj7N4h/Xyq6/XshmSwGio3QO+VxXRERj+XXJ4ArSNVwS5QucSafUj6eiy8Gdi68vRZ7\ns/HrvUfSaGB8RDw9xKsxHPGW9rlFxBORK3aBb5M+g0rGL2kMaUd8UURcmUf3zPZvFH8vbf+aiHgO\n6CdV1fTM9m8Uf2W3/2AbhIa7IzU81Rrgx5Ia4PfpUiybA1vk/lcBtwCHkxr0TovmDXpjgV1Zv0Gs\n1pgmUoPYtDz+06xrEJtOhw3weT6TgfsLw6XHy/oNeLX+rYYo/tcU+j8HXFzV+En101+tG9cz279J\n/D2x/YHtyI3GwGbAjaQahZ7Y/i3ir+T2H/Yd8ivpSEcTD5EalE7vYhy7kpLZ3cD9tViAbYDrcozX\nFjc6cEb+UOcAhxfGvyHPYz7w9cL4ccClefytwOQOY74YeBRYCTwCnJS/HKXHC3wkj58HnDiE8V8I\n3Jc/iytIdeCVix94M7C68J25K3+Xh+X7UmL8vbL9988x35Pj/T/D+XstMf5Kbn//adHMzDrWC20m\nZmZWcU4mZmbWMScTMzPrmJOJmZl1zMnEzMw65mRiZmYdczKxjYak5/PrJEmtnvj5SuZ9Rt3wzUM5\nf7OqczKxjUntT1W7Ah8czBvzrSZa+d/rLSjikMHM36zXOZnYxugrwCH5wUKfyXdmPSc/iOgeSZ8A\nkHSopBslXQk8mMf9KN8x+v7aXaMlfQXYLM/vojzu+drCJP1TLn+vpA8U5n29pB/mBxldVCh/lqQH\nciznDNtWMevAmG4HYNYFpwN/FRHvBcjJ49mIeKOkscAtkq7NZV8P7BcRj+ThkyLiWUmbArMlXR4R\nZ0j6y0h3k66JPO9jgQMiYn9J2+f33JDLHEi6n9If8jLfBMwFjo6IvfP7x5e1EcyGks9MzNLNOk/M\nz424jXTvpj3ytNsLiQTgs5LuId3HaKdCuWbeDFwCEBGPk+78enBh3o9FuqfRPaQbWi4FXszPqXgf\n8GKH62Y2LJxMzNKdVE+NiNfnbreIuC5Pe2FtIelQ0pPo3hjpWRL3AJsW5tHusmpWFvpXA2MiPaBo\nCum27+8Bfj7otTHrAicT25jUduTPA1sWxl8DfDo/uwNJe0javMH7JwDPRMRKSXsDUwvTXqq9v25Z\nNwHH5XaZVwNvAW5vGmBa7lYR8XPg88AB7a+eWfe4zcQ2JrWrue4D1uRqre9GxNfzY2nvkiTSw5KO\nbvD+nwOfkvQg6fblvypMOw+4T9KdEfGh2rIi4keSpgL3AmuAL0TE45L2aRLbeODK3CYD6XkVZpXn\nW9CbmVnHXM1lZmYdczIxM7OOOZmYmVnHnEzMzKxjTiZmZtYxJxMzM+uYk4mZmXXMycTMzDr2/wHy\n7B6hOyuFGQAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x106f6e0b8>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"With 6 epochs, the training time is: 425.2575750350952\n",
"The validation accuracy is: 0.9636666666666667\n"
]
}
],
"source": [
"NN = NeuralNetwork([784, 200, 10], loss=\"cross_entropy\", \n",
" plot=True)\n",
"start = time.time()\n",
"epochs = 6\n",
"NN.train(train, train_label, epochs, validation=validation, \n",
" validation_label=validation_label)\n",
"end = time.time()\n",
"print(\"With \" + str(epochs) + \" epochs, the training time is: \", end-start)\n",
"predictions = NN.predict(validation)\n",
"p_correct = accuracy(predictions, validation_label)\n",
"print(\"The validation accuracy is: \", p_correct) "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment