-
-
Save alierkan/68da5af5bc18b4dad5f8e471ef9cd9c6 to your computer and use it in GitHub Desktop.
PyTorch implementation of an autoencoder.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "Implementing an Autoencoder in PyTorch\n", | |
| "===\n", | |
| "\n", | |
| "This is the PyTorch equivalent of my previous article on implementing an autoencoder in TensorFlow 2.0, which you may read [here](https://towardsdatascience.com/implementing-an-autoencoder-in-tensorflow-2-0-5e86126e9f7)\n", | |
| "\n", | |
| "First, to install PyTorch, you may use the following pip command,\n", | |
| "\n", | |
| "```\n", | |
| "$ pip install torch torchvision\n", | |
| "```\n", | |
| "\n", | |
| "The `torchvision` package contains the image data sets that are ready for use in PyTorch.\n", | |
| "\n", | |
| "More details on its installation through [this guide](https://pytorch.org/get-started/locally/) from [pytorch.org](pytorch.org)." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "## Setup\n", | |
| "\n", | |
| "We begin by importing our dependencies." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 1, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": [ | |
| "import matplotlib.pyplot as plt\n", | |
| "import numpy as np\n", | |
| "\n", | |
| "import torch\n", | |
| "import torch.nn as nn\n", | |
| "import torch.optim as optim\n", | |
| "import torchvision" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "Set our seed and other configurations for reproducibility." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 2, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": [ | |
| "seed = 42\n", | |
| "torch.manual_seed(seed)\n", | |
| "torch.backends.cudnn.benchmark = False\n", | |
| "torch.backends.cudnn.deterministic = True" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "We set the batch size, the number of training epochs, and the learning rate." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 3, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": [ | |
| "batch_size = 512\n", | |
| "epochs = 20\n", | |
| "learning_rate = 1e-3" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "## Dataset\n", | |
| "\n", | |
| "We load our MNIST dataset using the `torchvision` package. " | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 4, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": [ | |
| "transform = torchvision.transforms.Compose([torchvision.transforms.ToTensor()])\n", | |
| "\n", | |
| "train_dataset = torchvision.datasets.MNIST(\n", | |
| " root=\"~/torch_datasets\", train=True, transform=transform, download=True\n", | |
| ")\n", | |
| "\n", | |
| "train_loader = torch.utils.data.DataLoader(\n", | |
| " train_dataset, batch_size=batch_size, shuffle=True\n", | |
| ")" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "## Autoencoder\n", | |
| "\n", | |
| "An autoencoder is a type of neural network that finds the function mapping the features x to itself. This objective is known as reconstruction, and an autoencoder accomplishes this through the following process: (1) an encoder learns the data representation in lower-dimension space, i.e. extracting the most salient features of the data, and (2) a decoder learns to reconstruct the original data based on the learned representation by the encoder.\n", | |
| "\n", | |
| "We define our autoencoder class with fully connected layers for both its encoder and decoder components." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 5, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": [ | |
| "class AE(nn.Module):\n", | |
| " def __init__(self, **kwargs):\n", | |
| " super().__init__()\n", | |
| " self.encoder_hidden_layer = nn.Linear(\n", | |
| " in_features=kwargs[\"input_shape\"], out_features=128\n", | |
| " )\n", | |
| " self.encoder_output_layer = nn.Linear(\n", | |
| " in_features=128, out_features=128\n", | |
| " )\n", | |
| " self.decoder_hidden_layer = nn.Linear(\n", | |
| " in_features=128, out_features=128\n", | |
| " )\n", | |
| " self.decoder_output_layer = nn.Linear(\n", | |
| " in_features=128, out_features=kwargs[\"input_shape\"]\n", | |
| " )\n", | |
| "\n", | |
| " def forward(self, features):\n", | |
| " activation = self.encoder_hidden_layer(features)\n", | |
| " activation = torch.relu(activation)\n", | |
| " code = self.encoder_output_layer(activation)\n", | |
| " code = torch.sigmoid(code)\n", | |
| " activation = self.decoder_hidden_layer(code)\n", | |
| " activation = torch.relu(activation)\n", | |
| " activation = self.decoder_output_layer(activation)\n", | |
| " reconstructed = torch.sigmoid(activation)\n", | |
| " return reconstructed" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "Before using our defined autoencoder class, we have the following things to do:\n", | |
| " 1. We configure which device we want to run on.\n", | |
| " 2. We instantiate an `AE` object.\n", | |
| " 3. We define our optimizer.\n", | |
| " 4. We define our reconstruction loss." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 6, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": [ | |
| "# use gpu if available\n", | |
| "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", | |
| "\n", | |
| "# create a model from `AE` autoencoder class\n", | |
| "# load it to the specified device, either gpu or cpu\n", | |
| "model = AE(input_shape=784).to(device)\n", | |
| "\n", | |
| "# create an optimizer object\n", | |
| "# Adam optimizer with learning rate 1e-3\n", | |
| "optimizer = optim.Adam(model.parameters(), lr=learning_rate)\n", | |
| "\n", | |
| "# mean-squared error loss\n", | |
| "criterion = nn.MSELoss()" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "We train our autoencoder for our specified number of epochs." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 7, | |
| "metadata": {}, | |
| "outputs": [ | |
| { | |
| "name": "stdout", | |
| "output_type": "stream", | |
| "text": [ | |
| "epoch : 1/20, recon loss = 0.08389361\n", | |
| "epoch : 2/20, recon loss = 0.06247949\n", | |
| "epoch : 3/20, recon loss = 0.05559724\n", | |
| "epoch : 4/20, recon loss = 0.04590550\n", | |
| "epoch : 5/20, recon loss = 0.04079516\n", | |
| "epoch : 6/20, recon loss = 0.03859508\n", | |
| "epoch : 7/20, recon loss = 0.03622651\n", | |
| "epoch : 8/20, recon loss = 0.03278850\n", | |
| "epoch : 9/20, recon loss = 0.03084057\n", | |
| "epoch : 10/20, recon loss = 0.02930079\n", | |
| "epoch : 11/20, recon loss = 0.02798331\n", | |
| "epoch : 12/20, recon loss = 0.02642137\n", | |
| "epoch : 13/20, recon loss = 0.02503968\n", | |
| "epoch : 14/20, recon loss = 0.02353912\n", | |
| "epoch : 15/20, recon loss = 0.02222095\n", | |
| "epoch : 16/20, recon loss = 0.02105061\n", | |
| "epoch : 17/20, recon loss = 0.02026297\n", | |
| "epoch : 18/20, recon loss = 0.01937818\n", | |
| "epoch : 19/20, recon loss = 0.01851325\n", | |
| "epoch : 20/20, recon loss = 0.01769270\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "for epoch in range(epochs):\n", | |
| " loss = 0\n", | |
| " for batch_features, _ in train_loader:\n", | |
| " # reshape mini-batch data to [N, 784] matrix\n", | |
| " # load it to the active device\n", | |
| " batch_features = batch_features.view(-1, 784).to(device)\n", | |
| " \n", | |
| " # reset the gradients back to zero\n", | |
| " # PyTorch accumulates gradients on subsequent backward passes\n", | |
| " optimizer.zero_grad()\n", | |
| " \n", | |
| " # compute reconstructions\n", | |
| " outputs = model(batch_features)\n", | |
| " \n", | |
| " # compute training reconstruction loss\n", | |
| " train_loss = criterion(outputs, batch_features)\n", | |
| " \n", | |
| " # compute accumulated gradients\n", | |
| " train_loss.backward()\n", | |
| " \n", | |
| " # perform parameter update based on current gradients\n", | |
| " optimizer.step()\n", | |
| " \n", | |
| " # add the mini-batch training loss to epoch loss\n", | |
| " loss += train_loss.item()\n", | |
| " \n", | |
| " # compute the epoch training loss\n", | |
| " loss = loss / len(train_loader)\n", | |
| " \n", | |
| " # display the epoch training loss\n", | |
| " print(\"epoch : {}/{}, recon loss = {:.8f}\".format(epoch + 1, epochs, loss))" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "Let's extract some test examples to reconstruct using our trained autoencoder." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 8, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": [ | |
| "test_dataset = torchvision.datasets.MNIST(\n", | |
| " root=\"~/torch_datasets\", train=False, transform=transform, download=True\n", | |
| ")\n", | |
| "\n", | |
| "test_loader = torch.utils.data.DataLoader(\n", | |
| " test_dataset, batch_size=10, shuffle=False\n", | |
| ")\n", | |
| "\n", | |
| "test_examples = None\n", | |
| "\n", | |
| "with torch.no_grad():\n", | |
| " for batch_features in test_loader:\n", | |
| " batch_features = batch_features[0]\n", | |
| " test_examples = batch_features.view(-1, 784)\n", | |
| " reconstruction = model(test_examples)\n", | |
| " break" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": [ | |
| "## Visualize Results\n", | |
| "\n", | |
| "Let's try to reconstruct some test images using our trained autoencoder." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 9, | |
| "metadata": {}, | |
| "outputs": [ | |
| { | |
| "data": { | |
| "image/png": "iVBORw0KGgoAAAANSUhEUgAABG0AAADnCAYAAACkCqtqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd7wU9fX/8YMtigUEQWwUwY6ggogGjSixo8EWI0ajKSYhiWnGtF+iKebx1WiKiS2PFDVqNIpdiZ3YUFEEQUBBAVFpgsSC/f7+yMOT9+dwZ9i77O6d3X09/zrjfO7uMDOfmdnxcz6nQ0tLiwEAAAAAAKBY1mjvDQAAAAAAAMDKeGkDAAAAAABQQLy0AQAAAAAAKCBe2gAAAAAAABQQL20AAAAAAAAKiJc2AAAAAAAABbRWWxp36NCB+uDtpKWlpUMlPodj2K6WtLS0dKvEB3Ec2w99sSHQFxsAfbEh0BcbAH2xIdAXGwB9sSG02hcZaQPUztz23gAAZkZfBIqCvggUA30RKIZW+yIvbQAAAAAAAAqIlzYAAAAAAAAFxEsbAAAAAACAAuKlDQAAAAAAQAHx0gYAAAAAAKCAeGkDAAAAAABQQLy0AQAAAAAAKCBe2gAAAAAAABTQWu29AWhO3/3udz1eb731knUDBgzw+Oijj878jIsuusjjRx55JFl3xRVXrO4mAgAAAADQrhhpAwAAAAAAUEC8tAEAAAAAACggXtoAAAAAAAAUEHPaoGauueYaj/PmqlEffvhh5rpTTz3V4xEjRiTrxo8f7/G8efNK3US0s2233TZZnjFjhsennXaaxxdccEHNtqmZrb/++h6fe+65HmvfMzN74oknPD7mmGOSdXPnzq3S1gEAALSPjTfe2OOePXuW9Dfxmehb3/qWx1OnTvX42WefTdpNnjy5nE1EA2GkDQAAAAAAQAHx0gYAAAAAAKCASI9C1Wg6lFnpKVGaEvOvf/3L46233jppN3LkSI/79u2brBs9erTHv/rVr0r6XrS/XXfdNVnW9Lj58+fXenOa3mabbebxF7/4RY9j2uKgQYM8Puyww5J1f/zjH6u0dVC77babx2PHjk3W9e7du2rfe8ABByTL06dP9/jFF1+s2vdi1fQeaWZ28803e/y1r33N44svvjhp98EHH1R3wxpQ9+7dPb722ms9fvjhh5N2l156qcdz5syp+nZ9pFOnTsnyPvvs4/G4ceM8fu+992q2TUA9OPTQQz0+/PDDk3X77ruvx/369Svp82LaU69evTz+2Mc+lvl3a665Zkmfj8bFSBsAAAAAAIAC4qUNAAAAAABAAZEehYoaPHiwx6NGjcpsN23aNI/jcMMlS5Z4/MYbb3i8zjrrJO0mTJjg8cCBA5N1Xbt2LXGLUSS77LJLsvzmm296fMMNN9R6c5pOt27dkuXLLrusnbYEbXXggQd6nDfEutJiCs4pp5zi8XHHHVez7cB/6b3vwgsvzGz3hz/8weO//OUvyboVK1ZUfsMajFaNMUufaTQVaeHChUm79kqJ0gp/Zum1XtNbZ82aVf0NqzMbbbRRsqwp9/379/c4VjEl1azYdFqFMWPGeKyp4GZm6623nscdOnRY7e+NVVKBUjHSBgAAAAAAoIB4aQMAAAAAAFBAvLQBAAAAAAAooHad0yaWgNY8wpdffjlZ9/bbb3t85ZVXerxgwYKkHfm47UtLBMfcT8351vkXXnnllZI++zvf+U6yvOOOO2a2ve2220r6TLQ/zQnXMrRmZldccUWtN6fpfOMb3/D4U5/6VLJuyJAhbf48LSVrZrbGGv/7fwOTJ0/2+N///nebPxuptdb63y38kEMOaZdtiHNlfPvb3/Z4/fXXT9bpHFWoDu1/W265ZWa7q6++2mN9vkK2TTbZxONrrrkmWdelSxePdS6hr3/969XfsAw//vGPPe7Tp0+y7tRTT/WY5+aVjR492uNf/vKXybqtttqq1b+Jc9+8+uqrld8wVIxeH0877bSqfteMGTM81t9CqBwtua7XarN0jlUt025m9uGHH3p88cUXe/zQQw8l7YpwnWSkDQAAAAAAQAHx0gYAAAAAAKCA2jU96pxzzkmWe/fuXdLf6bDO119/PVlXy2Fn8+fP9zj+WyZOnFiz7SiSW265xWMdqmaWHqulS5e2+bNj+di11167zZ+B4tl+++09jukUcQg6Ku83v/mNxzpMtFxHHnlk5vLcuXM9/vSnP520i2k2WLXhw4d7vOeee3oc70fVFEsfa9pqx44dk3WkR1VeLO/+ox/9qKS/09TTlpaWim5To9ptt908jkPs1c9+9rMabM3Kdtppp2RZU8pvuOGGZB331pVpusxvf/tbj7t27Zq0y+ovF1xwQbKs6d7lPPOiNDEVRlOdNMVl3LhxSbt33nnH4+XLl3sc71P6XHrnnXcm66ZOnerxo48+6vGkSZOSditWrMj8fJROp1MwS/uYPmvGc6JUe+yxh8fvv/9+sm7mzJkeP/jgg8k6Pefefffdsr67FIy0AQAAAAAAKCBe2gAAAAAAABQQL20AAAAAAAAKqF3ntNES32ZmAwYM8Hj69OnJuh122MHjvLzioUOHevziiy96nFWirzWax7Z48WKPtZx1NG/evGS5Wee0UTp/RblOP/10j7fddtvMdppL2toyiut73/uex/GcoR9Vx+233+6xluQul5Y2feONN5J1vXr18ljLzj722GNJuzXXXHO1t6PRxXxuLds8e/Zsj88+++yabdMRRxxRs+/CynbeeedkedCgQZlt9dnmjjvuqNo2NYru3bsny0cddVRm289//vMe63Njtek8NnfffXdmuzinTZwPEmbf/e53PdYS7qWK87QddNBBHsey4Tr/TTXnwGhUefPMDBw40GMt9RxNmDDBY/1dOWfOnKRdz549Pda5TM0qMw8gVqbvA8aMGeNx7GMbbbRRq3//0ksvJcsPPPCAxy+88EKyTn+D6NyKQ4YMSdrpNeGQQw5J1k2ePNljLRteaYy0AQAAAAAAKCBe2gAAAAAAABRQu6ZH3XPPPbnLKpZq+0gsN7rLLrt4rMOcdt9995K36+233/b42Wef9TimbOlQKR2ajtVz2GGHeaylM9dZZ52k3aJFizz+wQ9+kKx76623qrR1WF29e/dOlgcPHuyx9jczSiNWyic+8YlkebvttvNYh/eWOtQ3Dv/U4claOtPMbL/99vM4rxzxV77yFY8vuuiikraj2fz4xz9OlnWIuA7Fjylqlab3vnhuMVy8tvJSdqKYRoB85513XrJ8wgkneKzPl2Zm//znP2uyTdHee+/t8aabbpqs+9vf/ubx3//+91ptUt3Q1F0zs5NPPrnVdlOmTEmWFy5c6PGIESMyP79Tp04ea+qVmdmVV17p8YIFC1a9sU0uPv9fddVVHms6lFmaHpyXMqhiSpSK01+g8i655JJkWdPa8sp363uDp59+2uMf/vCHSTv9XR/ttddeHutz6F/+8peknb5f0GuAmdkf//hHj6+//nqPK50qy0gbAAAAAACAAuKlDQAAAAAAQAG1a3pUJSxbtixZvu+++1ptl5d6lUeHHsdULB2Kdc0115T1+ViZpsvEIZFK9/n48eOruk2onJhOoWpZdaPRaRraP/7xj2Rd3nBTpdW8dMjnWWedlbTLS0fUz/jSl77kcbdu3ZJ255xzjsfrrrtusu4Pf/iDx++9996qNruhHH300R7HigWzZs3yuJaV1jTNLaZD3X///R6/9tprtdqkprXPPvtkrotVafLSE7GylpaWZFnP9ZdffjlZV80KQOutt16yrEP/v/rVr3oct/eUU06p2jY1Ak13MDPbcMMNPdZqM/GZRe9Pn/nMZzyOKRl9+/b1uEePHsm6m266yeODDz7Y46VLl5a07c1ggw028DhOgaDTKCxZsiRZ9+tf/9pjpkoojvhcp1WbvvCFLyTrOnTo4LH+Loip8+eee67H5U6n0LVrV4+1iumZZ56ZtNNpWmJqZa0w0gYAAAAAAKCAeGkDAAAAAABQQLy0AQAAAAAAKKC6n9OmGrp37+7xhRde6PEaa6TvuLQcNXmo5bvxxhuT5QMOOKDVdpdffnmyHMvfoj7svPPOmet0XhOsnrXW+t/lvdQ5bOLcUMcdd5zHMW+8VDqnza9+9SuPzz///KRdx44dPY7nwc033+zx7Nmzy9qOenXMMcd4rPvILL0/VZvOkTR69GiPP/jgg6TdL37xC4+bbf6hWtESpRpHMcf/qaeeqto2NZtDDz00WdZy6jqXU5yDoVQ6j8q+++6brBs6dGirf3PdddeV9V3N6mMf+1iyrHMC/eY3v8n8Oy0f/Ne//tVjvVabmW299daZn6FzrVRzPqR69qlPfcrj73//+8k6LcOtZe/NzJYvX17dDUNZ4nXs9NNP91jnsDEze+mllzzWuWUfe+yxsr5b56rZaqutknX62/L222/3OM5jq+L2XnHFFR5Xcy4/RtoAAAAAAAAUEC9tAAAAAAAACoj0qFaMGTPGYy1LG8uLz5w5s2bb1Gg222wzj+Pwbh2yqikZOuzezOyNN96o0tah0nQ498knn5ysmzRpksd33XVXzbYJ/6WlomOJ2HJTorJompOm2JiZ7b777hX9rnrVqVOnZDkrFcKs/NSLcmi5dk23mz59etLuvvvuq9k2NatS+0otz49G9Lvf/S5ZHj58uMebb755sk5Lr+vQ+cMPP7ys79bPiKW81fPPP+9xLDmNfFquO9L0t5jCn2Xw4MElf/eECRM85lm2dXmpn/rcOH/+/FpsDlaTpiiZrZxard5//32P99hjD4+PPvropN3222/f6t+vWLEiWd5hhx1ajc3S59xNN900c5vUwoULk+VapYUz0gYAAAAAAKCAeGkDAAAAAABQQKRHmdnHP/7xZDnOUv4RncnczGzq1KlV26ZGd/3113vctWvXzHZ///vfPW62qjGNZMSIER536dIlWTdu3DiPtSoDKidWvlM69LTadMh/3Ka8bTzzzDM9/uxnP1vx7SqSWNFkiy228Pjqq6+u9ea4vn37tvrfuQ/WXl4aRiUqF+G/nnjiiWR5wIABHu+yyy7JuoMOOshjrYqyePHipN1ll11W0ndrNZLJkydntnv44Yc95hmpbeL1VFPZNAUxpmBoBcxRo0Z5HKvNaF+M6774xS96rMf6mWeeKWnbm0FMhVHa3376058m62666SaPqZhXHPfee2+yrKnU+hvBzKxnz54e//73v/c4L1VU061iKlaerJSoDz/8MFm+4YYbPP7GN76RrHvllVdK/r7VwUgbAAAAAACAAuKlDQAAAAAAQAHx0gYAAAAAAKCAmNPGzA455JBkee211/b4nnvu8fiRRx6p2TY1Is0X3m233TLb3X///R7HXFXUp4EDB3occ1Kvu+66Wm9OU/jyl7/scczNbS8jR470eNddd03W6TbG7dU5bRrd66+/nixrTr7OqWGWzg+1dOnSim5H9+7dk+Ws+QUefPDBin4vWjds2DCPjz/++Mx2y5cv95hSuJW1bNkyj2Npe10+44wzVvu7tt56a491LjCz9Jrw3e9+d7W/q1ndfffdybL2HZ23Js4zkzWvRvy8MWPGeHzrrbcm67bZZhuPdX4MvW83u27dunkcnwl07ref/OQnybof//jHHl988cUea5l1s3TelFmzZnk8bdq0zG3aaaedkmX9Xcj1Nl8sw63zQXXu3DlZp3PL6ryzr776atJu3rx5Hus5ob85zMyGDBnS5u299NJLk+Uf/vCHHut8VbXESBsAAAAAAIAC4qUNAAAAAABAATVtetR6663nsZaOMzN79913Pdb0nPfee6/6G9ZAYilvHVqmKWiRDv194403Kr9hqIkePXp4vPfee3s8c+bMpJ2W0UPlaCpSLemQZjOzHXfc0WO9BuSJZXKb6dobhxBrGd+jjjoqWXfbbbd5fP7557f5u/r3758sa0pG7969k3VZKQFFSb1rdHo/XWON7P/fdtddd9Vic1BlmvIR+56mX8VrJUoXU0qPPfZYjzVtu1OnTpmfccEFF3gc0+Lefvttj8eOHZus0/SPAw880OO+ffsm7Zq5jPuvf/1rj7/97W+X/Hd6ffzqV7/aalwp2v90aofjjjuu4t/VyGK6kfaPclx++eXJcl56lKak63n2t7/9LWmnJcXbCyNtAAAAAAAACoiXNgAAAAAAAAXESxsAAAAAAIACato5bU4//XSPY+nZcePGefzwww/XbJsazXe+851keffdd2+13Y033pgsU+a7MXzuc5/zWMsH33HHHe2wNaiVH/3oR8mylj3NM2fOHI9POumkZJ2WdWw2ej2MpX8PPfRQj6+++uo2f/aSJUuSZZ07Y5NNNinpM2LeN6ojq+R6nAvgkksuqcXmoMKOOeaYZPnEE0/0WOdcMFu57C0qQ0t2a387/vjjk3ba53TuIZ3DJvr5z3+eLO+www4eH3744a1+ntnK98JmovOaXHPNNcm6q666yuO11kp/ym611VYe583/VQk6h5+eM1p23MzsF7/4RVW3A2bf+973PG7LnEJf/vKXPS7nOaqWGGkDAAAAAABQQLy0AQAAAAAAKKCmSY/SYeRmZv/v//0/j//zn/8k6372s5/VZJsaXakl+r72ta8ly5T5bgy9evVq9b8vW7asxluCarv99ts93m677cr6jGeeecbjBx98cLW3qVHMmDHDYy1Ja2a2yy67eNyvX782f7aWtY0uu+yyZHn06NGttoslylEZW265ZbIcUzQ+Mn/+/GR54sSJVdsmVM/BBx+cue7WW29Nlp988slqb07T01QpjcsVr5Oa7qPpUcOHD0/adenSxeNYorzRaYnleF3bdtttM/9u//3393jttdf2+Mwzz0zaZU3ZUC5NXx40aFBFPxut+8IXvuCxpqTFlDk1bdq0ZHns2LGV37AqYaQNAAAAAABAAfHSBgAAAAAAoIAaOj2qa9euHv/+979P1q255poe69B+M7MJEyZUd8OQ0OGfZmbvvfdemz9j+fLlmZ+hwyM7deqU+RmdO3dOlktN79IhnGeccUay7q233irpMxrRYYcd1up/v+WWW2q8Jc1Jh+rmVVDIG5Z/6aWXerz55ptnttPP//DDD0vdxMTIkSPL+rtm9tRTT7UaV8Lzzz9fUrv+/fsny1OnTq3odjSrvfbaK1nO6sOx+iLqU7wOv/nmmx6fd955td4cVNm1117rsaZHffrTn07a6fQBTN1QmnvuuafV/67pxGZpetT777/v8V//+tek3Z/+9CePv/nNbybrstJWUR1DhgxJlvXauMEGG2T+nU67odWizMzeeeedCm1d9THSBgAAAAAAoIB4aQMAAAAAAFBAvLQBAAAAAAAooIab00bnqhk3bpzHffr0SdrNnj3bYy3/jdqbMmXKan/GP//5z2T5lVde8XjTTTf1OOYLV9qCBQuS5V/+8pdV/b4iGTZsWLLco0ePdtoSmJlddNFFHp9zzjmZ7bScbN58NKXOVVNqu4svvrikdmgfOidSa8sfYQ6b6tA5+aIlS5Z4/Lvf/a4Wm4Mq0LkV9DnFzGzRokUeU+K78eh9Uu/PRxxxRNLupz/9qcf/+Mc/knXPPvtslbauMd15553Jsj6fa4noL37xi0m7fv36ebzvvvuW9F3z588vYwuxKnHuww033LDVdjonmFk6b9RDDz1U+Q2rEUbaAAAAAAAAFBAvbQAAAAAAAAqo4dKj+vbt6/GgQYMy22k5Z02VQuXEUupx2GclHXPMMWX9nZb5y0vruPnmmz2eOHFiZrsHHnigrO1oBKNGjUqWNVVx0qRJHv/73/+u2TY1s7Fjx3p8+umnJ+u6detWte9dvHhxsjx9+nSPv/SlL3msKYwonpaWltxlVNeBBx6YuW7evHkeL1++vBabgyrQ9KjYv2677bbMv9OUgI033thjPS9QP5566imPf/KTnyTrzj33XI/PPvvsZN1nP/tZj1esWFGlrWsc+ixilpZdP/bYYzP/bvjw4ZnrPvjgA4+1z37/+98vZxPRCr3efe973yvpb6688spk+f7776/kJrUbRtoAAAAAAAAUEC9tAAAAAAAACoiXNgAAAAAAAAVU93Pa9OrVK1mOJd0+Eud00DK3qI4jjzwyWdZcxLXXXrukz9hpp508bku57r/85S8ez5kzJ7Pd9ddf7/GMGTNK/nz8V8eOHT0+5JBDMttdd911HmsOMKpn7ty5Hh933HHJuk996lMen3baaRX93ljm/o9//GNFPx+1se6662auY/6E6tD7os7PF7399tsev/fee1XdJrQPvU+OHj06Wfetb33L42nTpnl80kknVX/DUFWXX355snzqqad6HJ+pf/azn3k8ZcqU6m5YA4j3rW9+85seb7DBBh4PHjw4ade9e3eP4++JK664wuMzzzyzAlsJs/R4PPPMMx7n/XbUPqDHtpEw0gYAAAAAAKCAeGkDAAAAAABQQHWfHqUlZM3Mevbs2Wq78ePHJ8uUL629c845Z7X+/vjjj6/QlqBSdGj+smXLknVaJv13v/tdzbYJK4tl1nVZU0rj9XTkyJEe6/G89NJLk3YdOnTwWIeyon6dfPLJyfJrr73m8c9//vNab05T+PDDDz2eOHFisq5///4ez5o1q2bbhPbxhS98wePPf/7zybo///nPHtMXG8vixYuT5REjRngcU3POOOMMj2MKHVZt4cKFHuuzjpZSNzMbOnSox2eddVaybtGiRVXauua23377ebzlllt6nPfbXdNGNYW4kTDSBgAAAAAAoIB4aQMAAAAAAFBAHdqSJtShQ4dC5BQNGzbM49tvvz1ZpzNOqyFDhiTLcehx0bW0tHRYdatVK8oxbFJPtLS0DF51s1XjOLYf+mJDoC+uwi233JIsn3/++R7fd999td6cVjVyX9x8882T5V/84hceP/HEEx43QHW2pu2L+iyrlYDM0hTWiy66KFmnqcjvvvtulbaubRq5LxZFrI675557erzHHnt4vBopyk3bFxtJI/TFyZMne7zzzjtntjv33HM91nTBBtBqX2SkDQAAAAAAQAHx0gYAAAAAAKCAeGkDAAAAAABQQHVZ8nvvvff2OGsOGzOz2bNne/zGG29UdZsAAGgUWgIVtffyyy8ny6eccko7bQmq5cEHH/RYS9wCrTn66KOTZZ33o1+/fh6vxpw2QCF06dLF4w4d/jdFTyyx/tvf/rZm21QEjLQBAAAAAAAoIF7aAAAAAAAAFFBdpkfl0eGC+++/v8dLly5tj80BAAAAgLL95z//SZb79OnTTlsCVNf555/favzzn/88affKK6/UbJuKgJE2AAAAAAAABcRLGwAAAAAAgALipQ0AAAAAAEABdWhpaSm9cYcOpTdGRbW0tHRYdatV4xi2qydaWloGV+KDOI7th77YEOiLDYC+2BDoiw2AvtgQ6IsNgL7YEFrti4y0AQAAAAAAKCBe2gAAAAAAABRQW0t+LzGzudXYEOTqVcHP4hi2H45j/eMYNgaOY/3jGDYGjmP94xg2Bo5j/eMYNoZWj2Ob5rQBAAAAAABAbZAeBQAAAAAAUEC8tAEAAAAAACggXtoAAAAAAAAUEC9tAAAAAAAACoiXNgAAAAAAAAXESxsAAAAAAIAC4qUNAAAAAABAAfHSBgAAAAAAoIB4aQMAAAAAAFBAvLQBAAAAAAAoIF7aAAAAAAAAFBAvbQAAAAAAAAqIlzYAAAAAAAAFxEsbAAAAAACAAuKlDQAAAAAAQAHx0gYAAAAAAKCAeGkDAAAAAABQQLy0AQAAAAAAKCBe2gAAAAAAABQQL20AAAAAAAAKiJc2AAAAAAAABcRLGwAAAAAAgAJaqy2NO3To0FKtDUG+lpaWDpX4HI5hu1rS0tLSrRIfxHFsP/TFhkBfbAD0xYZAX2wA9MWGQF9sAPTFhtBqX2SkDVA7c9t7AwCYGX0RKAr6IlAM9EWgGFrti7y0AQAAAAAAKCBe2gAAAAAAABQQL20AAAAAAAAKiJc2AAAAAAAABdSm6lHA6ujQobQJzbPatbSs/kTmlfgMtA89LziOAACgUkp9xuBZBEB7YKQNAAAAAABAAfHSBgAAAAAAoIBIj8JqWWON9L1fx44dPe7Xr1+ybuTIka1+xoABA5Ll7t27e7xw4UKPFyxYkLSbO/d/ZeznzZuXrHvjjTc8fv755z2eP39+0m7FihUex2GuDHutHB1OnJcmt9Zaa2Uuf/jhhx6/9957SbsPPvhgdTcRrYj9+yNt6Rv0o2JhaD/MOA/QnPKeP0rtB/SX5hXPH31GWmeddTLbvf/++5mfqev0OReIGGkDAAAAAABQQLy0AQAAAAAAKCBe2gAAAAAAABQQc9qgzTp37uxx//79k3Unnniix3vssUeyrnfv3h6vv/76Hq+55polfW/M9Xz33Xc9XrRoUbLu5ptv9vgPf/iDxzGvVHOTyVOuHs377dSpU7Ju1113bTU2M9too408vueeezyeNGlS0u7NN9/0mPlt2kZzr2Nf1H7aq1cvj7fYYouknfbN5557LlmnffOtt95q9W9QHj122sfy8u71Opd3PSxnG8zScygvr5/rbWXEPrveeut5vOGGG2aue/XVVz3WOeDM0r7JcWq7rH4Zted+btY5jZrp34p8WfMb5d0/9RpqZta1a1ePe/To4XGcd1GfZV955ZVknc7P+c4773jMMxIiRtoAAAAAAAAUEC9tAAAAAAAACoj0KJREh2BreosODTQz69Onj8ebbrppsk6HGOow+TiMMG+Yv9JhrnEY4dtvv+2xpmdoSlX8DFSP7ud4THVI6aBBg5J1G2ywgccvvPCCx88880zSTtNuShWHrTdrqpwej5i6NmTIEI9HjRrlcd++fZN2mpJ21113Jet0eebMmR6vWLGizC1uLnmpFmut9b9buKaydenSJWnXsWNHj/V6+NprryXt8sqS6j1AvzcOF9fv0rRFszQNR6/7DANvGz0n9FiYpWnIBx10ULJOUxzvvfdej++7776k3X/+8x+PS70Wxut6I15D81JJ9bzXVIh11103aafHa8mSJR7rPjdL+0e5x0CvF/E80XV6/Y7XgHrvm81wXiKbHv+8/qHr4j2tX1RHIPkAACAASURBVL9+Hp988snJuiOPPNJjvQbE3xrz58/3eNy4ccm62267zWN9RorXBNKLi6tWKbGMtAEAAAAAACggXtoAAAAAAAAUULumR8WhanlDT3VoZ95wNx3mqUOPYkWZvCFKDDvLrwiis5vrMGCzNIUlpqxoytL999/v8YIFC5J2OgRw6NChHu+5555JO03RWLx4cbJO02fiMEVUR14qm4rHQ1Mo4jmjw0P1GMch2+UM4W7mfq7HSof0Dh8+PGl3wgkneKypa/o3UbwmaNtrrrnGY71WmKXXlWaWN4R77bXXTtZtvvnmHm+33XYed+vWLWmnfafU1Je4HXoc9dqr3xu/a968eck6PcYxLRalpwOrmPaiVRuPPfbYZJ2m0K2zzjoeT58+PWn3+uuvr3pjV7F9jVCdKO5brcY1ePDgZN3BBx/ssab1xhTEyZMne/z44497rM9HZvnPqLpvNf0qprfquvgMrN+3fPnyzHb1chyztrMt25x1rY1V2DbZZBOP9boYn2X12SamA+t+rnTFy2ZPCcs6jrE/6zVQj/HAgQOTdl//+tc9jr9D9Jqq+z0eU/3uYcOGJetefvlljzV9OZ4zeZUfUV4qnFl6bPKqX+ozS0yByqrQGc+DvFTUUvopI20AAAAAAAAKiJc2AAAAAAAABcRLGwAAAAAAgAKq2Jw2eXmvWeVBYy68lqPUPEGzNDdU88riPAiaI6b5vHEuAM3ZfvXVV5N1muubVaI0fkbcjlLLehU1Xzhui+5XnRPh0UcfTdpNmTLF45jLrftZczXjfCSaG/jQQw95fNpppyXtNH87lujT0uOat1pOaWiksvJGS50bKuZ46jHR/mZmNmfOHI8nTZrkcSyFyJw2baPXZJ2bIfaxnXfe2eNYulbptTGWA//0pz/t8VZbbeXx9ddfn7R7+OGHPY7Ht5mPlfaxjTfeOFmn8wztuuuuHufN8aU583F+Kd3PMZ87a26wbbfdNlnecccdPb7jjjuSdS+99FKr39VMSs2FL/Wapvc3M7MhQ4Z4HPuifqZeh+PcCZUo81xPxzdrPsWtt946aadzfH3yk59M1nXp0sXj559/3uMnn3wyaafPTDoHSlvmtNFt1OvyXnvtlbTTOa9mz56drNNrQt4cRvVyHMspix5/F+j1dd999/X4sMMOS9ptv/32Huu8XbpPzcwmTJjg8bRp05J1euwrPadNVNTfGZUS50DVOYc0zvs7nYdq7733Ttrpc0uk90V9Dop9Ss+TOIeYPu/odSCeF4147LJkldCOz6HaZ/V5Nc49pHMtxvNFf4Pod8X3EPr7RJ9lzMxmzZrl8cKFCz2eO3du5neVc3wZaQMAAAAAAFBAvLQBAAAAAAAooKqkR8VUFR2WpEO4dSiTWZouFYct6jAiHT4Wh3D36NHDY02R+djHPpa00yFtMT3qxRdf9FiHu8Uh+zNmzPBYS1ibpUOi9DPqtbx41rDtWNJ1dcuZxc/Q4xTPCR0mV2qp1HrZ30W2ukNtY9nFHXbYweNNN900WXfnnXd6vGTJEo8pF9w2sX/osPmzzz7bY70+m62cevGRuP91uGm81m622WYeH3LIIR7H1INLLrnE43HjxiXrSi1T3Yi0v8R0l4MOOshjTRfV4blmaUlRTeVty77U67KmgvTs2TNpp0OI4z2g3O9GSvtz165dk3X77LOPx/rsZZb2Iy03HYd6N/Ox0eHxMR1RUwG7deuWrNNh75rqeffddyft5s+f77FeR8tNSdPnoljyW5+B43Oufp/203o99qU+l2i7eK/6xCc+4fGYMWM81pTPvM974YUXknWavtq5c+dk3V133eVxpa+L9XoM20L3e0yBOvLIIz3W/hH7gKYR67kQf2ton41pjJrqpKmPOlWEWXpPjp+h98zly5d7nJWS3CiyUqDMzDp27OixvhsYPnx40k7T7/X5KPZtvdbGqTt0n+ddH3R52bJlybpHHnnEY53WI7bTqV7iv7mUewAjbQAAAAAAAAqIlzYAAAAAAAAFxEsbAAAAAACAAqrYnDYqzmmjuZyaCx/nS9B865hLpjm3mn+mnxfbad6aznVjlpa4jOVRNR9Uy+FqSTgzs49//OMex9znP//5zx5rTlu95prqdus+rkZJOs0p1PkSYjlL3Y44t47mkWsOYSWUOn8O/kf3WeyL+++/v8exxJ7mEjd6fm81xRztz3/+8x4PHDjQ43jd1eOmOcGxRLDmaMfjFPN2P6LzLZiZnXzyyZl/c8stt3hcan+u174Yry86L0nM59b7k+bra1lvs/SeWYm5MwYMGOCxzkllZvbKK694HOe0qUQp6XqXd16WOheH9o/dd989abflllu22s4s7Ts6p0Y8Ts1M91mcf037Ylyn8wKNHz/eY51Pyiy9PurxznuuiMdR+6LODRbnidxwww09jtdNnUej1PMuqtdr7Efi84ZeX3X+onhf1Llr9Fl/6dKlSTudiyMeGz0vHnjgAY8r0Rcb+Zh9RI+dzmFjZvaZz3zG42effdbjO+64I2k3depUj/X3qM5PZZbOURLvrfr5ep/Nm3cxlpyOy1nyrhf1cFzjNuu/O875NGzYMI9POukkj3fZZZeknc7ppp8X978eN70+m6XXRv38OO+innPx+q/vB+KcgqrUa34WRtoAAAAAAAAUEC9tAAAAAAAACqgqJb/z0hh0CGkcSqhDT2NpNv18TaOKw4t0GKOmacVhkLNnz/Y4DvXfbbfdPNYhjTEFSoesxs/XtCEdEt4Iw8OrPZRah5mdddZZHutwVTOzmTNnenzTTTcl66ZNm+Zxpbe3HoYhVlOp//6s4fyaVmiWDu+fO3dusk6vA82+31eHlt02MzvxxBM91utkvJ7qENPnn3/eYy1pa2b2zDPPeLzuuusm6/r16+exlk7t3r170k7XHX/88cm6SZMmefzcc895HK+n9XSOZA2NjcNutWxsTIXRfb1gwQKPn3766aRd1jUwb4h1XKepFv379/c4Hke9r5MetWrl7BMdBn7YYYcl62J6upo1a5bHmsbW7LLuVfGZY6eddmr1b8zSdPwlS5Z4HFPI9e/0OMb0exXTo7baaiuPDzzwQI9j6WO9buozkZnZ66+/7nFeye96uaaW81wS97mmFup1WNO0zcwuuugij6+77jqP4+8APV9ier+eF4899pjHpCq2LqYQjRgxwmNNnzFL+4vu23vvvTdppymC+jexJLSK5br1927etVzPu3jt0HNNf8PGfh9/qxZV1r81/nv0N7UeTzOzL3/5yx5vs802Hsd+rs+lWn594sSJSTtNsdfjbpamMW6//fYex3uppqXG9xePP/64xy+++KLH8VzSZ+p4bygFI20AAAAAAAAKiJc2AAAAAAAABVSV6lFx+JgOI9LUpjgMXIcFxmFmWdVJSh16GmeS1r+L26FDtnRdrMCi2zhlypTM7WVIeL447PHYY4/1eI899vA4nlePPvqoxw8++GCyTof+lju8V8+lehki3J7iPtL9p8MMDz744KSdzv4e025imiRKp/1q1KhRyTrd5ypeJ5988kmPzzvvPI91yLGZ2TvvvONxTI/S4aZa5eGAAw5I2ul1V6sTmZntueeeHmuKR6P0S+0rcf9pyoOmKJmlQ4NvvPFGj+fMmZO0yxpy35b9p324V69eHseUAD2H4hBilL7P866nnTp18njXXXdN2mm/135pZnbJJZd4XM7Q7Ealw/b1OvSJT3wiaad9Mw711/Nej1V8vtT+opXXNt9886SdVsTUdCiztMqRpg5o2r+Z2X333dfq58Xt1XOtUa6ppYgVa/QZX1MoYtWhsWPHttouXguHDBnicTyGmnrXrVs3j+Nzbjn9tFGOofajeO/TFGq9H5mlqTG33nqrx1rdySw7HTjeL3Vd/D1Xid93eh7quRDvn5qmV+TfldqPdB/nVWD+6le/mqzLqt721FNPJe3+7//+r9V18beD9qNYuVqv6zqVQKwap2J68RNPPOGxTgMTU9r0uJXTTxlpAwAAAAAAUEC8tAEAAAAAACggXtoAAAAAAAAUUMXmtMkqcW2W5hHmlT3L+rz4meXk38ZtyvtuzT3VPMq4TTqvg5b7Mls5lxwpzQfXnGwzs3333dfjhQsXeqzlK83Mxo8f77HOlWSWHu+8uWlKPR9Vo+QLV5vuT51DRecpMktz/u+///5kHf2ofB07dvRYy8Kapftc87fjvAdnnXWWxw888IDHMU9X+3PMA37hhRc81uMZ8/81zzjONTB06FCPr7rqqla3vZ7p/tP5SszSktpa7twsLbU+YcIEjyuRRx2vh5rrrf05rwTnW2+9lazj2lm6uP/1HNl66609jnOhqJh3P27cuAptXWPR/qHXlLwy3IsWLUqW9XgNGjTI43id23HHHT0ePXq0x3HuG+07PXr0SNbpPFc6p8o999yTtNPrhc4FaZbdF+N518h9Ns5tof9WLdsey6Xrsdp00009/vrXv56023333T2Oc2L26dPHYy05HOfi0POg2eah0mve4MGDk3X6TBDvM3fddZfHOi9M3jwwWfPbxO2oxByZ8fP1mUn7euyz9UL3s+47fZYxMzvqqKM8jr8D9fn1zTff9Hjy5MlJO73GLVu2zOPYV3SuN71/mqXz6Wi/jNcHnSs1/lbRZ+d4PqrVnYuIkTYAAAAAAAAFxEsbAAAAAACAAqpKye84fEyHA+UN0y41jaWc7cgb7haHwOrQVi0FpsO+zcwuv/xyjxcsWJCsa7ZhjKXQY6BD0E466aSknaZGPPTQQx7PnTs3aacpALFUcdb3xvMgq0S8WXr+6PFs5OHCbZU3bFSHAw8cONBjTbMwS8vj6bBWs2KXNSyaeG5rGcwtt9wyWaf7VYeeXnDBBUk7LcGu7eKx1uU4pFfTYzX1IK8vxqHkmqaqQ9PrOX1O/73ajzbaaKOknQ7RffbZZ5N1mjKqQ4NjvyknJSoODd5ll1081rLIsYyqphLENC3ky7ueasn1ffbZp9X/bpb2ib/+9a/JOk2lQet0v8dnPk2FiWksum/1fhdTU/faay+PNRU/Pn9oyncsL65ttUT0zJkzk3Z6LpSbFtnIzzvxPtOlSxePNa0tplNo/9Oy8Mcee2zSTq+T8b6on69pVLFs+4svvuhx3m+rRqS/v4455phkne6/+PtLU2aypkqIdF1MVdTleByzjkHed8X+rMvan+O1vV5k/eaKqbyaKpr3b9Vnkfj7Yc899/RYpyyJ+1+fX8aMGZOsGzBggMeaqhafL/U3aJwS5bXXXvO4mv2SkTYAAAAAAAAFxEsbAAAAAACAAqpKelSUVeEjb9hl3tAyHRoa22WlYsV2OuxOh0aZpUNbdXjUrbfemrTT4VH1Ost3LWmKgw511P1tlqY66fB6PWZmpVcR06GH8TzQYY/x83U7dMhiow9JbYu89DKtMKNDxOP14N577/U4VjtB6eKQ2y222MLjOPRUh/I/+eSTHsdrnM6Wn9fHslIJzdK+k1dlMO8zdDb+Rul/2l90+K9eJ83yrz16fcxKt4p/l9dOt2PbbbdN1h1yyCEeawrXnDlzknYzZszwOPb1Rk61qIS8FMHevXt7PGLECI/jOaHpFHfeeWeyrlH6TjXpPorpiC+//LLHeamZeu/baaedknX6nKH9IfYV7Zvx+q3PmzosP26vXkdLTXtq9D6at891naY2jRw5Mml3wgkneKyVvGJKqd5n4/mi67RvayWp+HcxJU+v/41y3PS813TE+DtNr4/x2V1TwzWNKqbr6j1Y+2xMh9PnoPiMqn1R+1s83np8YoVNXaefH+8B5UwT0h6y/q3xmUL3XdxfWRVOY5WpQw891ON+/fp5HI+hplHF6QJ0P+sx1GurWTpdgN4LzNLfi9Xsi4y0AQAAAAAAKCBe2gAAAAAAABQQL20AAAAAAAAKqCZz2pST31Xq/Al5ZZo1NzK207Jho0aNStb17dvXYy2/F/PDtdQpueIri2XzNKcwzmOjtMzmvHnzPF68eHHSTud6yMsf1Thuk+bCak5r/MwlS5a0+nlxudnOA/23x/xbLe/Xv39/j2Ne8cSJEz3WY4q2iee25u3G81LLQ1977bUexzzdcs7nmHetef6am5w370rsz3odzpojrZ5p3ne8V6nOnTsny5ojriXZly5dmrTTfdutWzePN9hgg6Sd3hcPOuigZJ2WKtbtzTu3mu16WEk6F4OZ2aBBgzzWZ5S4j2fNmuWx3j+RLeu5cerUqUk7ndtCnwnMsucjuv3225N22md1vps4z8KwYcMy1+lcC/r5cb4NPTfic4teZ8qZ+6Ze6b8nlorWMus6j1CcZyZr38X9r6Wnda6v+He6TUOHDk3a6f1z0qRJybr58+d7XOp9sejHV7dH70edOnXKbBfnfDriiCM81t9pOjeemdkBBxzg8b777utxfJbV+UsnT56crNO5TKZPn+5xnOtN74txThs9JnoO1is9t7WvxPvR+PHjPV60aFGyTuf20+uuzgVllv5e1OPWs2fPpJ3OPRWfsfQY6rVVS4ibmd12220ea98zq92zDiNtAAAAAAAACoiXNgAAAAAAAAVUk/SoSshKd8krG6tiSTgdehqHgetwRB2+9fTTTyftGnGY/urSYX5afs3MbPTo0R536dLF4zjk84477vB44cKFHsdyijrsMabV5A33VR07dmx1m8zSoXZ6XsVh0Tq0rhlk9bG4n3Xo9/bbb++xpnGYpcNNY6ln5NN9HtOjdHhpTEXSIbg6RDxeT/Xz9bjHY63DTWPJaj0PhgwZ4nEcIqziMHO9RjRiyk3e9eWFF17wWFNkzNKU0913393jOLxbU6K0XHcckqx0uHj8O733zZ07N2lHelTblPrMst9++3nco0cPj+NzyDPPPONxvNZi1bQMbRwCryWXtZ1Z9jPqSy+9lLTT+92NN97ocUwZ1xS4mMY4YcIEjy+77DKPY2qF9r+8tJis63wj0n9fTAfW9AdNr+jevXvmZ+j1729/+1vS7u677/Y43hd32GEHj7Vva5lrs/RaHrdX75NZ6VZm6f2/6Ndk3Va958TUJn3mz0sNP/HEEz3WdCuztI/pb4G88tOa6m+WpkHpNTtO56DX4vhbRr9P4/i7pujH7iN6/ul0CPE3tPadeAz1+poVm6Xni/apeD3VfRenaNBnYO2z//znP5N2Tz31lMfx3lqr9wGMtAEAAAAAACggXtoAAAAAAAAUEC9tAAAAAAAACqhu5rRReWUMs8qBaw64mdkJJ5zgccwh1RJiN998s8dadqy170Zalm/MmDHJugEDBnisZdVmzpyZtNOSpZonGPO6de6HUvMJ41wPmuOqecVmad5j3rw4sWxkM8krp67zbeh8QVoW0Sw93sxpU744R0zv3r0z12288cYeazlTnSvBLO1X2gdiTrb2zd122y1Zd+qpp7a6Ln6G5m8/++yzyTotC18ved1RvIbov0Pz9WO5br2+xDkNevXq5XGfPn08jrn72q+0jHEsX6p/F4+Pbr/m3cd5P/S6Wa/Hqr3klbHVHH3tzzE//5577vE45t3zzNK6rDkY4nx1Op9CqfeqvH2u6+L8FfqsEssH63OpXhPacv/MmoOnmc6ROHfJTTfd5LHe0+L8jLrPtZT3uHHjknY6x1Dnzp2TdfpMpHO3bL755kk7/d0R58UpValzG7WHvO3ROaTiHDHrr7++x3nz3eicJ127dk3aaR/Tvh3nldN+pfPDRTqfmB5Ts7RUtf7+MUuv0/E6UO/y9qse3/iskDdHk9LfHfp8qc+18fP0N4eZ2eWXX+6xXgPinH96P8h791BNjLQBAAAAAAAoIF7aAAAAAAAAFFBdpkflDeXMKoF7+OGHJ+20PGr8DB0epakcDPVemaagmZnts88+Hu+///7JOh0OqkPV4hBk/Uw9NnEoq4oljfVY6RBILfFnlp4Xw4cPT9ZpqV0d4hfPFx3OqOVuzRo/3Uf3RUxf0/LE2i9jioemg7T3UN16FksEb7HFFh7HVBctb6nnfUxde/zxxz3W4cixH2lfHzlyZLJO2+o2xuup9reLL744WafDiRvlHNF/v15fYtle3S/xOOoQbO1jOuzYzOyuu+7y+I477vB4+fLlSTstZ3rAAQck63R4v17nXnzxxaSdXg8b5Vi1B71fmqXD8vXciUPOH3vsMY8b/f5Trvi8oOepPo/E/Vfp81mfUXfeeedknfb1J554IlmnaTixBG6W9hrOX2RxH2gJ7SuvvNJjLSVslp4XOp1CvJ5qu5jCo6lO2tfjNV7vB7GvZ5Vtz1O04563PZomfcMNNyTrhgwZ4nHsz5pupGmm8TeEphfnpShpelxMUdP9rqXGY7tS0y5L/X1btOOYJes5xyz9rZd3r9J/azzWmur/gx/8wGM9FmbpM8vVV1+drLviiis81nTEIv7mZ6QNAAAAAABAAfHSBgAAAAAAoIDqMj2qVFo95Rvf+EayTlM5HnrooWTdn/70J4/zUnKwckqGzt4dZ1nXtlplao899kja6TBFHTKnf2OWDp3UFAKzdBihDvk/7rjjknY6xDJWGNNhlVohJVZy0FSTOHy10elQRd3PZmbbbLNNq+0eeeSRpF0cKorSZVU9MUurDsWhtFnD8k8//fSknZ73WtVCj62Z2SabbOJxvCbosdfhprHq2llnneXxgw8+mKwrtTpckeUNdc4bQqzpg1qpxCwdcq+VoGIK4nPPPedx3j1Nh/prbJZW3siraEVKTvn0nNhxxx2TdZqeqPtYq0WZpSka9TKEvtbisPestIO8c7ncKjz6d5reeOihhybtdJh+PMax+gzKE4+ZXns1VSpOA6B9UftbvE/psc5LX9JrfKySNH78eI9nz56drNPva5S+rv8Orbg0duzYpN2UKVM8jhWA9dldK+3FylyaDq7PLR//+MeTdvpbJlagVfqbMz4H6XNuTJXLOo6lprzVi3iOlvqsoM+Qmg5lZnbZZZd5rL8/YxrVwoULPb7qqquSdZqCWPR+xEgbAAAAAACAAuKlDQAAAAAAQAHx0gYAAAAAAKCA6n5Om5i3pjnCv/rVrzyO5b80h+3Xv/51sk5zWZEv5iTGfFylecE6B0YsEazlgzUfNZZC1HkVdH4bszR/tE+fPh537949aZc1T4BZ+m/ReRs019xs5TLfjU7zbPX4jBo1KmmncxBprvijjz6atKt0Wb28uQbqsWRiqWKetO7nWL5Zz3vN89Yy4Wbp9VXjuI+z5mcxS/uizj31wx/+MGmnpahjyc1GlHX+xf+u+0/nPjBL95POd6BzAZiVnjOv/TRe13S+Li3zrbniZsUsk1lkegx0LoYDDzwwaaf3T5337ZZbbknaVXpOoXLnVain62s1tzXuP51PbODAgR7rM5FZerzz5vRq5Htaren+02uhzu1mlt4/tS+2ZZ4j/Tst4R5/f0yYMMHjOB9ZXnnoRqDXsng/0jltYnltffbcbLPNPNb5J83SZ5/tt9/eYy3xbZb+9ojHWLdR75Fxzj6dczDvmpo3D1K931vz5vWL/1a9/ul8euedd17Sbvjw4R7rtTXODXj++ed7vGjRotztKjJG2gAAAAAAABQQL20AAAAAAAAKqC7To3TYVBwW97nPfc7jESNGZH7Gv/71L4/vvvvuZF29D0GrpTgE7eabb/ZY05zMzIYOHeqxptXEUt5avk9TMmLZRR1OHNM6NLVJhy/qkFSzdNhrLNetQ1ZnzpzZ6t/Ez28Gekx06GnPnj2TdjqUV0sOx1LC5Sh1eGm5n1lPwyU/EvvinXfe6XG8Fh5xxBEea0pGTDctdV/qNTOWo9Xy3ZqyOmnSpKRdXilq/FdMk9A0qLwUtazzOR5fvZ/G79Ljqumicdh6XhpdPfaratP9pfcxvbaapfeZuXPnehxLrpcjL92x1GOYN/S9CMe9ltuj3xWfWzQl4zOf+YzHmp5hlva3mNat/VTPi9jvS32WLcLxKZqsVKnWlrPoeRD/5umnn/ZY+7OWEDdL0/Gb7VlTj0H8t2tqcNy3Ov2Fpo3r7w6ztC/G6ReU9qOYuq3PszfccIPHs2bNStrpNsbtLTWFv9H6qf574rNn586dPT7llFM8/uQnP5m00+urniOPPfZY0u6aa65p9XvrDSNtAAAAAAAACoiXNgAAAAAAAAXESxsAAAAAAIACqps5bbLKDO+9995Juy996Use61wNsYyelv9qhvKy1RJzpnW+F51fyMxsxx139FjzEvfcc8/MdjrfTcwN19xfLUFrZvbII494/PDDD3sc80x1Ho1YMlmXNdYSvK0tNzrti5pbH+cE0lzf22+/3eNYtrjS21TuPBr1nOdqtvL26/E444wzknV6DD772c96HOeX0hKKKs53Mm3aNI9/85vfJOtuuukmjzU/n7nDVl85+zDrXmpmttNOO3kc51TRa6XO8RXLi5f63are+97q0Fx+nc8tzvGk9zu9tsZzQD8vbw6MvGtm3rxE+n31VHK4vbYvlhnWUu577bWXx/Haq3NsDBs2LFmnzzETJ070OPbFZrn3VVsl9k/sizpnkf4Gic+hedf4Zj5uul/iPtJ9raW39fnDLH0O0jhe83r06OGx3vvMzC699FKPx48f73E8juXMR9Toz0h596ANNtjA42233TaznfYdnd9tzJgxSbtGmTORkTYAAAAAAAAFxEsbAAAAAACAAipselQcAqXDuHfYYQePv/KVryTtevXq5bEOh3rggQeSdrNnz/a4mYcYVpruSy27Z2Y2YcIEjx999FGPY6k3TWvTst5xKL+W747ftWLFCo91WGLesc4rWZrXrtnoftGhibGUtx6T++67z+M4bLQcbTmOzXq8dGjtwoULk3Xf//73Pb7wwgs9Hj58eNJOUxV1mLEOAzYzmzx5sscx3bRZ93+RZJUg1rKaZmYbb7yxx/Gc0fvpnDlzPI4pGTF1LkuznhfxvqIpiJpKE4fGanN8EgAABD9JREFU6/HQYeDlpkeVmtpU7jr8V3y+6dKlS6vrYr/R5Y022ijz87VdPN556RVZzzcoX94+jcdCyz6vu+66Hq+99tpJO33ujek9Wce32ful/vu1f8T9p88tmvY0duzYpJ0enzgNgD7n6ne15RjoedNoKVGlpuHqPjZLy7Frn1i2bFnS7rnnnvP4zDPP9Hj69OnlbXDBMdIGAAAAAACggHhpAwAAAAAAUEA1T48qdahUrFqy5ZZbenzUUUd5rNUu4t9ptYU4VKrZKv4UjQ4djEN6dbh9WyqTVFqzDzHNokNANU1CZ9GPtC9WI32GY9U2OjRbK5HE6mp6jWYfN5547dWhxpH226lTp3rclmonWLkf6bPIU0895fHFF1+ctNNqQpqKqmmLZuVVKYk4hqtHr5vxWXPcuHEeayr4dtttl7R79dVXPdbzwix9ntU08bYcN67nlaHHOqbClZqCpukf8TM0LTV+XiOn1VRD3Ee6rP0073dHNfpNs/TFeG5rOnD37t2Tddtss43HemyefvrppJ1WCp4yZYrHlbgPFhEjbQAAAAAAAAqIlzYAAAAAAAAFxEsbAAAAAACAAqr5nDZ5ZSY1vzDmHmpOm+bwak5wtHjxYo+ffPLJZJ3miQIoj85vE+dW0D7cLDm7jYbj1hiySqDG8qVaEnXGjBnJOr1n6jw2zKWwejT3XsuZ3nnnnUk7fQbSv6GPFo8ekzinzezZsz2+8MILPY7zOGp/0znIzMqfxwbVldcX8+ZTWbp0qcda4tssnUssztNB368O9mt1xP2q17Wtt946WXfQQQe1uk6vfWbp/EOvvfaax416XWSkDQAAAAAAQAHx0gYAAAAAAKCAap4eVao111wzWdaSeDqEe/78+Uk7HRL10EMPefz4448n7XTIKkPhgPJoqgWA+hKHEOtQ/BUrVtR6c5qSPn/o/kf9ykuP0numDvWPz6E8l9aHvCkfSv077ffxGkBZbzSK2D803U/TRs3Mrr32Wo832WSTzM/U3/nNMO0JI20AAAAAAAAKiJc2AAAAAAAABcRLGwAAAAAAgALq0JYczA4dOtQsyXaNNdL3STqnjVpnnXWSZS0hpvmfeaXy6iF3uKWlpcOqW61aLY8hVvJES0vL4Ep8EMex/dAXGwJ9sQHQFxsCfbEB0BcbAn2xAdRLX9T5muJvfl0Xf7+revj9XqZW+yIjbQAAAAAAAAqIlzYAAAAAAAAF1NaS30vMbG41NiSKpe2ySnk1Q4kvM+tVwc+q2THESjiO9Y9j2Bg4jvWPY9gYOI71j2PYGDiO9a9ujqGmNuWlQDWpVo9jm+a0AQAAAAAAQG2QHgUAAAAAAFBAvLQBAAAAAAAoIF7aAAAAAAAAFBAvbQAAAAAAAAqIlzYAAAAAAAAFxEsbAAAAAACAAuKlDQAAAAAAQAHx0gYAAAAAAKCAeGkDAAAAAABQQP8fxLRr58ZFJREAAAAASUVORK5CYII=\n", | |
| "text/plain": [ | |
| "<Figure size 1440x288 with 20 Axes>" | |
| ] | |
| }, | |
| "metadata": {}, | |
| "output_type": "display_data" | |
| } | |
| ], | |
| "source": [ | |
| "with torch.no_grad():\n", | |
| " number = 10\n", | |
| " plt.figure(figsize=(20, 4))\n", | |
| " for index in range(number):\n", | |
| " # display original\n", | |
| " ax = plt.subplot(2, number, index + 1)\n", | |
| " plt.imshow(test_examples[index].numpy().reshape(28, 28))\n", | |
| " plt.gray()\n", | |
| " ax.get_xaxis().set_visible(False)\n", | |
| " ax.get_yaxis().set_visible(False)\n", | |
| "\n", | |
| " # display reconstruction\n", | |
| " ax = plt.subplot(2, number, index + 1 + number)\n", | |
| " plt.imshow(reconstruction[index].numpy().reshape(28, 28))\n", | |
| " plt.gray()\n", | |
| " ax.get_xaxis().set_visible(False)\n", | |
| " ax.get_yaxis().set_visible(False)\n", | |
| " plt.show()" | |
| ] | |
| } | |
| ], | |
| "metadata": { | |
| "kernelspec": { | |
| "display_name": "Python 3", | |
| "language": "python", | |
| "name": "python3" | |
| }, | |
| "language_info": { | |
| "codemirror_mode": { | |
| "name": "ipython", | |
| "version": 3 | |
| }, | |
| "file_extension": ".py", | |
| "mimetype": "text/x-python", | |
| "name": "python", | |
| "nbconvert_exporter": "python", | |
| "pygments_lexer": "ipython3", | |
| "version": "3.7.5" | |
| } | |
| }, | |
| "nbformat": 4, | |
| "nbformat_minor": 2 | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment