Created
January 30, 2023 12:53
-
-
Save kplr-io/1c40fb56a436762b4821a4f714f0be6d to your computer and use it in GitHub Desktop.
MNIST in Keras.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "view-in-github", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<a href=\"https://colab.research.google.com/gist/kplr-io/1c40fb56a436762b4821a4f714f0be6d/mnist-in-keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "L8d0CykwSZWi" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "%matplotlib inline" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "1ajvOFM9SZWm" | |
| }, | |
| "source": [ | |
| "# Introduction to Deep Learning with Keras and TensorFlow\n", | |
| "\n", | |
| "**Daniel Moser (UT Southwestern Medical Center)**\n", | |
| "\n", | |
| "**Resources: [Xavier Snelgrove](https://github.com/wxs/keras-mnist-tutorial), [Yash Katariya](https://github.com/yashk2810/MNIST-Keras)**" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "pejv4cP_SZWp" | |
| }, | |
| "source": [ | |
| "To help you understand the fundamentals of deep learning, this demo will walk through the basic steps of building two toy models for classifying handwritten numbers with accuracies surpassing 95%. The first model will be a basic fully-connected neural network, and the second model will be a deeper network that introduces the concepts of convolution and pooling." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "mOw7yH5oSZWp" | |
| }, | |
| "source": [ | |
| "## The Task for the AI\n", | |
| "\n", | |
| "Our goal is to construct and train an artificial neural network on thousands of images of handwritten digits so that it may successfully identify others when presented. The data that will be incorporated is the MNIST database which contains 60,000 images for training and 10,000 test images. We will use the Keras Python API with TensorFlow as the backend." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "ydcjyK1vSZWq" | |
| }, | |
| "source": [ | |
| "<img src=\"https://github.com/AviatorMoser/keras-mnist-tutorial/blob/master/mnist.png?raw=1\" >" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "TqCc4tJ3SZWq" | |
| }, | |
| "source": [ | |
| "## Prerequisite Python Modules\n", | |
| "\n", | |
| "First, some software needs to be loaded into the Python environment." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "QZCGECl9SZWr" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "import numpy as np # advanced math library\n", | |
| "import matplotlib.pyplot as plt # MATLAB like plotting routines\n", | |
| "import random # for generating random numbers\n", | |
| "\n", | |
| "from keras.datasets import mnist # MNIST dataset is included in Keras\n", | |
| "from keras.models import Sequential # Model type to be used\n", | |
| "\n", | |
| "from keras.layers.core import Dense, Dropout, Activation # Types of layers to be used in our model\n", | |
| "from keras.utils import np_utils # NumPy related tools" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "H8zkcBSsSZWs" | |
| }, | |
| "source": [ | |
| "## Loading Training Data\n", | |
| "\n", | |
| "The MNIST dataset is conveniently bundled within Keras, and we can easily analyze some of its features in Python." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "1zfY9eQoSZWt", | |
| "outputId": "86212ae6-0335-4774-8e1d-ad4e6ebc44a9", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "X_train shape (60000, 28, 28)\n", | |
| "y_train shape (60000,)\n", | |
| "X_test shape (10000, 28, 28)\n", | |
| "y_test shape (10000,)\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "# The MNIST data is split between 60,000 28 x 28 pixel training images and 10,000 28 x 28 pixel images\n", | |
| "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n", | |
| "\n", | |
| "print(\"X_train shape\", X_train.shape)\n", | |
| "print(\"y_train shape\", y_train.shape)\n", | |
| "print(\"X_test shape\", X_test.shape)\n", | |
| "print(\"y_test shape\", y_test.shape)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "YAulEmd_SZWu" | |
| }, | |
| "source": [ | |
| "Using matplotlib, we can plot some sample images from the training set directly into this Jupyter Notebook." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "9bFgLEtRSZWv", | |
| "outputId": "a638d753-dad8-49ef-ad4f-a13680c56be1", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 657 | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/plain": [ | |
| "<Figure size 648x648 with 9 Axes>" | |
| ], | |
| "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnIAAAKACAYAAAAYdJWHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde7hVVb3/8c9XBQ1BklBC5KKGdfCGHY5i2s87Cqng/VJmHQstKRH0PKYdr12oRPB0zA4KgnlXvJspXhI14RGMUtTMCwjETVEEMxH4/v5Y099vxxmTvebac13GXO/X8/DsvT9rrTHHhPVlf/fca6xh7i4AAADEZ5N6TwAAAACVoZEDAACIFI0cAABApGjkAAAAIkUjBwAAECkaOQAAgEjRyDUIM7vEzG6s9zyAIqCegPxQT42NRq6GzOwUM5tlZqvNbLGZPWRm+9VhHr2SObT842Y2utZzASrVKPWUzOVyM3vBzNaa2SX1mAPQFg1WT33M7Akz+7uZvWJmh9RjHrGgkasRMxslabykn0jqJqmXpF9JGlrrubj7W+7e8ZM/knaTtF7S1FrPBahEI9VT4jVJ/yHpwTodH6hYA9bTLZL+KOkzki6UdKeZbVOnuTQ8GrkaMLPOki6TdJa73+XuH7j7x+5+v7ufl/KYO8xsiZmtNLPpZrZLi9uGmNlLZrbKzBaZ2blJ3tXMHjCz98xshZk9ZWbl/Bt/XdJ0d5+Xw+kCVdWI9eTuU9z9IUmrqnDKQNU0Wj2Z2c6SvijpYnf/0N2nSnpB0rHVOP8ioJGrjX0kbSHp7gyPeUhSX0nbSnpe0k0tbpso6Qx37yRpV0mPJ/loSQslbaPST1UXSNroHmxmZio1clMyzA2op4atJyBCjVZPu0h6w91b/lD0pyRHwGb1nkCT+Iykt919bbkPcPdJn3yevObmXTPr7O4rJX0sqZ+Z/cnd35X0bnLXjyV1l9Tb3V+T9FQZh9pPpaK6s9y5AXXWyPUExKbR6qmjpJUbZCsl9Sh3fs2GK3K18Y6krmZWVuNsZpua2Rgze93M3pc0L7mpa/LxWElDJM03syfNbJ8k/4VKr9V5xMzeMLPzyzjcaZKmuvvqck8GqLNGricgNo1WT6slbbVBtpV42UIqGrnaeFbSR5KGlXn/U1R6kekhkjpL6pPkJknu/py7D1XpsvY9km5P8lXuPtrdd5R0lKRRZnZw2kHM7FOSjhe/VkVcGrKegEg1Wj3NlbSjmXVqke2R5AigkauB5HLzRZKuNrNhZtbBzNqZ2WAz+3ngIZ1UKqx3JHVQaSWRJMnM2pvZV5PL2B9Lel+lFacysyPM7HPJ695WSlr3yW0pjlbpsvcTOZwmUBONWE/J8bdQ6f/UzcxsCzPbNL+zBqqj0erJ3V+VNEfSxUkdHS1pd/GuCqlo5GrE3cdKGiXph5KWS1ogaYRKP7Fs6AZJ8yUtkvSSpBkb3H6qpHnJZe0zJX01yftKelSlS9PPSvqVu2+sSTtN0m/cnRdwIyoNWE/XSvpQ0skqvV3Ch8m4QMNrwHo6SdIAlS40jJF0nLsvr+jkmoDxPRwAACBOXJEDAACIFI0cAABApGjkAAAAIkUjBwAAEKk27exgZodLukrSppKuc/cxrdyflRVodG+7e102Z6aeUDTubvU6dpZ6opYQgdTvTRVfkUveI+lqSYMl9ZN0spn1q3Q8oEHMr8dBqScgP9QTCij1e1NbfrW6l6TX3P0Nd18j6VaV3u0ZQHbUE5Af6glNoy2NXA+V3jTwEwsV2NTWzIab2Swzm9WGYwFFRz0B+Wm1nqglFEWbXiNXDnefIGmCxOsQgLainoB8UEsoirZckVskqWeLr7dPMgDZUU9AfqgnNI22NHLPSeprZjuYWXuV9ka7L59pAU2HegLyQz2haVT8q1V3X2tmIyQ9rNLy7knuPje3mQFNhHoC8kM9oZmYe+1eGsDrEBCB2e4+oN6TKAf1hEZXz/eRy4JaQgRSvzexswMAAECkaOQAAAAiRSMHAAAQKRo5AACASNHIAQAARIpGDgAAIFI0cgAAAJGikQMAAIgUjRwAAECkaOQAAAAiRSMHAAAQKRo5AACASNHIAQAARIpGDgAAIFI0cgAAAJHarN4TAAAA8TnrrLOC+X/913+lPmaTTcLXj/76178G83HjxgXz559/PpjPnDkz9dhFxRU5AACASNHIAQAARIpGDgAAIFI0cgAAAJGikQMAAIiUuXvlDzabJ2mVpHWS1rr7gFbuX/nBgNqY3drzuFqoJxSNu1u9jp2lnqilkj59+gTzW2+9NZj369cvmHfo0CH1GGbhp0TWXmT58uXBfOnSpcF8yZIlwfzwww/PdNw6Sv3elMfbjxzo7m/nMA4A6gnIE/WEwuNXqwAAAJFqayPnkh4xs9lmNjyPCQFNjHoC8kM9oSm09Ver+7n7IjPbVtI0M3vF3ae3vENSQBQR0DrqCcjPRuuJWkJRtOmKnLsvSj4uk3S3pL0C95ng7gPq9QJyIBbUE5Cf1uqJWkJRVHxFzsy2lLSJu69KPh8k6bLcZgY0Eeqp/n7+85+n3nbeeecF8wceeCCYn3zyycF89erV2SeGzKinjevZs2cwv++++4J52urUetpmm20y5Z07dw7mm2++eTD/6KOPKptYHbTlV6vdJN2dLCXeTNLN7v67XGYFNB/qCcgP9YSmUXEj5+5vSNojx7kATYt6AvJDPaGZ8PYjAAAAkaKRAwAAiBSNHAAAQKTatNdq5oOxn11F2rdvH8w7deqU+pjRo0fncuy01U1f+9rXchm/V69ewXzBggW5jF+Buu21mhX1lK/Fixen3rbttttmGmuHHXYI5m+99VamcWJXz71Ws2i2WkpbnTpkyJCqHzuvvVbz8vDDDwfzY445JpjXcTVr6vcmrsgBAABEikYOAAAgUjRyAAAAkaKRAwAAiBSNHAAAQKTaskVX0+nSpUum+69YsSKYp+0FN2rUqGC+//77B/N99tkn9djVXgGU1/i77757MK/jqlU0qVtuuSX1trPPPruGMwHykfa8PfDAA3MZ/8477wzmaXsTS+n7E++yyy65zCmrww47LJj/67/+azD/wx/+UM3pVIQrcgAAAJGikQMAAIgUjRwAAECkaOQAAAAiRSMHAAAQKVatBvTu3TuYP/3005nGmTZtWjA/9thjg/nG9k7Ny7Jly4L5+++/H8w/9alPBfMePXrkMp/Zs2fnMg6woY4dOwbzyZMnB/Mjjzwy8zFeeOGFYP7BBx9kHguo1PHHHx/Mf/SjHwXztP/X09x1113B/Dvf+U4wf/fdd1PHuuKKK4L5qaeeGsyvueaaYD527Nhgvt122wXzzTbL1u6ceeaZwZxVqwAAAMgNjRwAAECkaOQAAAAiRSMHAAAQKRo5AACASLW6jMPMJkk6QtIyd981ybpIuk1SH0nzJJ3g7unLVBpU2urU0aNHB/OsKzW/8Y1vZLr/iy++GMyffPLJYD59+vTUsdL2Qp01a1YwnzdvXjA/7rjjgvntt9+eeuyQJ554Ipi/9957mcaJXZHrqdGsXbs2mK9evTqYb2xVW9pY119/fTB/5513Wpkd8kA9ley0007BPOvq1DTf+ta3gnnaux1szG9+85tMeZq77747mL/55pvBvGfPnpnGj0k5V+QmSzp8g+x8SY+5e19JjyVfA2jdZFFPQF4mi3pCk2u1kXP36ZJWbBAPlTQl+XyKpGE5zwsoJOoJyA/1BFT+Grlu7r44+XyJpG45zQdoRtQTkB/qCU2lzTs7uLubWfgFWZLMbLik4W09DtAMqCcgPxurJ2oJRVHpFbmlZtZdkpKP4X2fJLn7BHcf4O4DKjwWUHTUE5CfsuqJWkJRVHpF7j5Jp0kak3y8N7cZVUH79u2D+a9+9atgPnjw4Ezjp+0rN2rUqGC+dOnSYJ62svOjjz7KNJ9KpK16Svs7SvP2228H80svvTSY/+Mf/8g0fkFFVU+xSFuxt/POO2ceK20V6lVXXZV5LFRdYeupc+fOwfzCCy/MZfw5c+YE848//jiX8WvhueeeC+ZZV62m7V+b9j1xxowZmcbPU6tX5MzsFknPSvq8mS00s9NVKpBDzeyvkg5JvgbQCuoJyA/1BJRxRc7dT0656eCc5wIUHvUE5Id6AtjZAQAAIFo0cgAAAJGikQMAAIhUm99HrpGk7ZU4bty4YJ7X6tT//u//DuZTpkwJ5o2oe/fuwTxtL72uXbsG8yVLlgTzF154obKJARXaaqutgvnee++deaxOnToF82HDwpsG3HPPPZmPAXxi9913D+bXXHNNMM+6p2raOyekPZ8//PDDTOPXU9r340MPPTSYp9V2u3btgvm5554bzNP2Ja8FrsgBAABEikYOAAAgUjRyAAAAkaKRAwAAiBSNHAAAQKQKtWo1bZVJ2mqVNO+9914wP+WUU4L5ww8/nGn8ejrssMOC+a233hrM0/b2S1ud+u1vfzuYp634Bapl2bLgXum6+eabg3lafUvSqlWrgjmrU1ENea64Dpk8eXIwX7hwYS7j19P06dOD+VNPPRXMhwwZkmn8tH3J64krcgAAAJGikQMAAIgUjRwAAECkaOQAAAAiRSMHAAAQqUKtWk3bD27fffcN5mmr1F566aVgPm3atMom1kBOPPHEYJ62OjXNGWecEcxnzpyZeU5ANWyxxRbBvHfv3pnHYnUqaunqq68O5maWaZz58+cH81tuuSXznGKX9neX9e806/1rgStyAAAAkaKRAwAAiBSNHAAAQKRo5AAAACJFIwcAABApGjkAAIBItfr2I2Y2SdIRkpa5+65Jdomkb0tantztAnf/bbUm2VbLly8P5ldddVWNZ1I7W265ZTDv169fpnEWLVoUzOfOnZt5TihGPcUibePxtLcj2pjf/pZ/jkYUez117949mH/qU58K5u6eafy0t8158cUXM41TBD/+8Y+D+eDBgzONk/XfoBbKuSI3WdLhgXycu/dP/jRkkQANaLKoJyAvk0U9ocm12si5+3RJK2owF6DwqCcgP9QT0LbXyI0wsz+b2SQz2zrtTmY23MxmmdmsNhwLKDrqCchPq/VELaEoKm3krpG0k6T+khZLGpt2R3ef4O4D3H1AhccCio56AvJTVj1RSyiKiho5d1/q7uvcfb2kayXtle+0gOZBPQH5oZ7QbFpdtRpiZt3dfXHy5dGSmm8JTANo37596m1pq+z22ivb/2k33nhjMH/jjTcyjYN01FN1dO7cud5TQB3EVE/Dhg0L5jvuuGMu448bNy6XcYrg3XffrfcUqqactx+5RdIBkrqa2UJJF0s6wMz6S3JJ8ySdUcU5AoVBPQH5oZ6AMho5dz85EE+swlyAwqOegPxQTwA7OwAAAESLRg4AACBSNHIAAACRqmjVKhqDmaXelrbXatpjli5dGsyvv/767BMDGsAvfvGLek8BQI197nOfC+Z33XVXLuM//PDDuYyTJ67IAQAARIpGDgAAIFI0cgAAAJGikQMAAIgUjRwAAECkWLUagY4dOwbzK664IvUxX/ziF4P5e++9F8ynTp0azD/66KNWZgcUw/Llyyu6DajUvHnzgvmqVauCeadOnTKNf9BBBwXzhx56KJgvW7Ys0/j1lLY69f777w/mffv2zTT+PffcE8wvvvjiTOPUAlfkAAAAIkUjBwAAECkaOQAAgEjRyAEAAESKRg4AACBSrFqNQNoqmTPOOCP1Me4ezM8555xgPnny5MzzAork8ccfT71t5syZNZwJmkXa6tH58+cH81133TXT+BMnTgzms2bNCubXXXddMJ80aVIwX79+fab5SOkrb7feeutg/s1vfjOYn3DCCcE86+rUtHdySHtXiEZ8JweuyAEAAESKRg4AACBSNHIAAACRopEDAACIFI0cAABApFpdtWpmPSXdIKmbJJc0wd2vMrMukm6T1EfSPEknuPu71Ztq8zrzzDMzPyZtJc7vf//7Ns4GbUE9Na6RI0fWewrIqKj1dO+99wbzrKtW0wwYMCBT/oUvfCGYr1mzJvOx99xzz2B+6KGHZh4ri7TviUcccUQwnzFjRjWnk6tyrsitlTTa3ftJGijpLDPrJ+l8SY+5e19JjyVfA9g46gnID/WEptdqI+fui939+eTzVZJeltRD0lBJU5K7TZE0rFqTBIqCegLyQz0BGd8Q2Mz6SNpT0kxJ3dx9cXLTEpUubYceM1zS8MqnCBQT9QTkJ2s9UUsoirIXO5hZR0lTJY109/db3ualbQSCWwm4+wR3H+Du4V++A02IegLyU0k9UUsoirIaOTNrp1KR3OTudyXxUjPrntzeXdKy6kwRKBbqCcgP9YRmV86qVZM0UdLL7n5li5vuk3SapDHJx/AyG5RtxIgRwXyLLbYI5hvb5+6nP/1pMJ83b17meSE/1FPtvPrqq8H8kEMOCeZf//rXU8dK23cR9VXUerr88suD+YoVK4L5qaeeGsz79++fy3zyXNFd+if739L2B89q0aJFwfzEE08M5jGtTk1Tzmvk9pV0qqQXzGxOkl2gUoHcbmanS5ovKbyDLYCWqCcgP9QTml6rjZy7Py0p3EJLB+c7HaDYqCcgP9QTwM4OAAAA0aKRAwAAiBSNHAAAQKQyvSEwquvCCy8M5ptuumkw/+CDD1LH+sUvfpHLnIBYpa3wAxrd2rVrg/lVV10VzKdOnRrMb7jhhmCett9pp06dyphdba1bty6Yv/TSS8H8rLPOCuZFWJ2ahityAAAAkaKRAwAAiBSNHAAAQKRo5AAAACJFIwcAABApVq3WQZ8+fYJ52p6qaV555ZUcZgMU07Jl4X3SZ82aFcwXLlxYzekAVZP23D3ooIOC+eDBg4P59773vWA+aNCgyiYWkLaqNG2v1TVr1gTz66+/Prc5xY4rcgAAAJGikQMAAIgUjRwAAECkaOQAAAAiRSMHAAAQKUtbKVKVg5nV7mANIG116u9+97tgvvPOO2caf9GiRam39ezZM9NY+H9mu/uAek+iHM1WT3m5/PLLg/luu+2W+phhw4ZVazqF5u5W7zmUg1pCBFK/N3FFDgAAIFI0cgAAAJGikQMAAIgUjRwAAECkaOQAAAAi1epeq2bWU9INkrpJckkT3P0qM7tE0rclLU/ueoG7/7ZaE43RkiVLgnn79u1zGf/73/9+LuOgdqinxjVlypR6TwEZUEtASauNnKS1kka7+/Nm1knSbDObltw2zt2vqN70gMKhnoB8UEuAymjk3H2xpMXJ56vM7GVJPao9MaCIqCcgH9QSUJLpNXJm1kfSnpJmJtEIM/uzmU0ys61THjPczGaZ2aw2zRQoGOoJyAe1hGZWdiNnZh0lTZU00t3fl3SNpJ0k9Vfpp6Kxoce5+wR3HxDLu+UDtUA9AfmgltDsymrkzKydSoVyk7vfJUnuvtTd17n7eknXStqretMEioN6AvJBLQHlrVo1SRMlvezuV7bIuyevUZCkoyW9WJ0pxusf//hHMH/qqaeCedrerMccc0wwv+eeeyqaF+qHeqq///zP/6z3FJADagkoKWfV6r6STpX0gpnNSbILJJ1sZv1VWvY9T9IZVZkhUCzUE5APaglQeatWn5ZkgZt4Xx4gI+oJyAe1BJSwswMAAECkaOQAAAAiRSMHAAAQKXP32h3MrHYHAyozO5b3laKe0OjcPfQatoZDLSECqd+buCIHAAAQKRo5AACASNHIAQAARIpGDgAAIFI0cgAAAJEqZ4uuPL0taX7yedfk62bB+cahd70nkAH11DxiPF9qKR7Nds4xnm9qPdX07Uf+6cBms2J5m4c8cL6opmb7++Z8US3N+HfdbOdctPPlV6sAAACRopEDAACIVD0buQl1PHY9cL6opmb7++Z8US3N+HfdbOdcqPOt22vkAAAA0Db8ahUAACBSNHIAAACRqnkjZ2aHm9lfzOw1Mzu/1sevBTObZGbLzOzFFlkXM5tmZn9NPm5dzznmycx6mtkTZvaSmc01s7OTvLDn3Ciop+I9t6in+qGeivXcapZaqmkjZ2abSrpa0mBJ/SSdbGb9ajmHGpks6fANsvMlPebufSU9lnxdFGsljXb3fpIGSjor+Xct8jnXHfVU2OcW9VQH1FMhn1tNUUu1viK3l6TX3P0Nd18j6VZJQ2s8h6pz9+mSVmwQD5U0Jfl8iqRhNZ1UFbn7Ynd/Pvl8laSXJfVQgc+5QVBPJYV6blFPdUM9lRTmudUstVTrRq6HpAUtvl6YZM2gm7svTj5fIqlbPSdTLWbWR9KekmaqSc65jqinksI+t6inmqKeSgr53CpyLbHYoQ689J4vhXvfFzPrKGmqpJHu/n7L24p6zqi/oj63qCfUQxGfW0WvpVo3cosk9Wzx9fZJ1gyWmll3SUo+LqvzfHJlZu1UKpSb3P2uJC70OTcA6knFfG5RT3VBPal4z61mqKVaN3LPSeprZjuYWXtJJ0m6r8ZzqJf7JJ2WfH6apHvrOJdcmZlJmijpZXe/ssVNhT3nBkE9lRTquUU91Q31VFKY51az1FLNd3YwsyGSxkvaVNIkd/9xTSdQA2Z2i6QDJHWVtFTSxZLukXS7pF6S5ks6wd03fMFplMxsP0lPSXpB0vokvkCl1yIU8pwbBfVUvOcW9VQ/1FOxnlvNUkts0QUAABApFjsAAABEikYOAAAgUjRyAAAAkaKRAwAAiBSNHAAAQKRo5AAAACJFIwcAABApGjkAAIBI0cgBAABEikYOAAAgUjRyAAAAkaKRAwAAiBSNXIMws0vM7MZ6zwMoAuoJyA/11Nho5GrIzE4xs1lmttrMFpvZQ2a2X53m8oSZLTez983sT2Y2tB7zACrVYPXU38yeMrOVZrbQzP6zHvMAKtVg9TTPzD5M5rLazB6pxzxiQSNXI2Y2StJ4ST+R1E1SL0m/klSvBupsSd3dfStJwyXdaGbd6zQXIJMGrKebJU2X1EXS/pK+a2ZH1WkuQCYNWE+SdKS7d0z+DKrjPBoejVwNmFlnSZdJOsvd73L3D9z9Y3e/393PS3nMHWa2JPkJf7qZ7dLitiFm9pKZrTKzRWZ2bpJ3NbMHzOw9M1uRXCEI/hu7+5/dfe0nX0pqJ6lnricOVEEj1pOkPpJucvd17v66pKcl7ZJyX6BhNGg9IQP+EmtjH0lbSLo7w2MektRX0raSnpd0U4vbJko6w907SdpV0uNJPlrSQknbqPRT1QUqNWlBSVH9Q9JMSb+XNCvD/IB6acR6Gi/p62bWzsw+n8zx0QzzA+qlEetJkm5KXv7ziJntkWFuTYdGrjY+I+ntFlfAWuXuk9x9lbt/JOkSSXskPzlJ0seS+pnZVu7+rrs/3yLvLql38hPVU+6eWijufoSkTpKGSHrE3ddnPzWg5hqxnh6QdJykDyW9Immiuz+X/dSAmmvEevqqSle5e0t6QtLDZvbpzGfWJGjkauMdSV3NbLNy7mxmm5rZGDN73czelzQvualr8vFYlZqv+Wb2pJntk+S/kPSapEfM7A0zO7+1YyUF9ZCkQbymB5FoqHoysy6SfqfSr6e2UOklCoeZ2XcrODeg1hqqniTJ3Z9x9w/d/e/u/lNJ70n6cvZTaw40crXxrKSPJA0r8/6nqPQi00MkdVbpJxNJMkly9+fcfahKl7XvkXR7kq9y99HuvqOkoySNMrODyzzmZpJ2KvO+QD01Wj3tKGmdu9/g7mvdfaGkW1X6ZgY0ukarpxD/ZHz8bzRyNeDuKyVdJOlqMxtmZh2S19IMNrOfBx7SSaXCekdSB5VWEkmSzKy9mX3VzDq7+8eS3pe0PrntCDP7nJmZpJWS1n1yW0tm9oXk2J9K5vE1Sf9H0pP5njmQv0arJ0mvlu5up5jZJmb2WUknSvpzfmcNVEej1ZOZ9TKzfZOxtjCz81S62vdMvmdeHDRyNeLuYyWNkvRDScslLZA0QqWfWDZ0g6T5khZJeknSjA1uP1XSvOSy9pkqvZ5AKr349FFJq1X6KetX7v5EYHxT6XUNy5K5nC3pxBavZQAaWiPVk7u/L+kYSedIelfSHEkvSvpR5WcI1E4j1ZNKjeI1KtXSIkmHSxrs7u9Uen5FZxt5LTwAAAAaGFfkAAAAIkUjBwAAECkaOQAAgEjRyAEAAESqrDcATGNmh0u6StKmkq5z9zGt3J+VFWh0b7v7NvU4MPWEonH3ur33V5Z6opYQgdTvTRVfkTOzTSVdLWmwpH6STjazfpWOBzSI+fU4KPUE5Id6QgGlfm9qy69W95L0mru/4e5rVHon86FtGA9oZtQTkB/qCU2jLY1cD5XeNPATC5Psn5jZcDObZWaz2nAsoOioJyA/rdYTtYSiaNNr5Mrh7hMkTZB4HQLQVtQTkA9qCUXRlityiyT1bPH19kkGIDvqCcgP9YSm0ZZG7jlJfc1sBzNrL+kkSfflMy2g6VBPQH6oJzSNin+16u5rzWyEpIdVWt49yd3n5jYzoIlQT0B+qCc0E3Ov3UsDeB0CIjDb3QfUexLloJ7Q6Or5PnJZUEuIQOr3JnZ2AAAAiBSNHAAAQKRo5AAAACJFIwcAABApGjkAAIBI0cgBAABEikYOAAAgUjRyAAAAkaKRAwAAiBSNHAAAQKRo5AAAACJFIwcAABApGjkAAIBI0cgBAABEikYOAAAgUjRyAAAAkaKRAwAAiBSNHAAAQKRo5AAAACJFIwcAABCpzeo9gXoys2Devn37XMY/5phjgnmXLl2C+QcffBDMDzzwwMzH7t27dzC/7rrrgvnNN98czNevX5/52EAj69mzZzAfOXJk6mNGjRoVzJ999tlgPn78+ExzShtnwYIFmcYBYjVgwIBgfuihhwbzc845J5h37do103HT+oCVK1cG85/97GfB/IknngjmM2bMyDSfSrSpkTOzeZJWSVonaa27h/8lALSKegLyQz2hWeRxRe5Ad387h3EAUE9AnqgnFB6vkQMAAIhUWxs5l/SImc02s+GhO5jZcDObZWaz2ngsoOioJyA/G60naglF0dZfre7n7ovMbFtJ08zsFXef3vIO7j5B0gRJMjNv4/GAIqOegPxstJ6oJRSFuefz/DWzSyStdvcrNnKfhiqWz3zmM8F8+fLlNZ5J/e22227BfO7cuTWeSd3NboQXRVV2EGYAACAASURBVMdYT40mbXXqM888k+n+9ZS2WnbcuHE1nkll3D28JLDGWqsnaqlk8ODBwXynnXbKNM6gQYNSbzvggAOC+Wabha8rbb755pmOXS8XXnhhMB8zZkxeh0j93lTxr1bNbEsz6/TJ55IGSXqx0vGAZkY9AfmhntBM2vKr1W6S7k7eg2UzSTe7++9ymRXQfKgnID/UE5pGxY2cu78haY8c5wI0LeoJyA/1hGbC248AAABEikYOAAAgUk2912rait2PPvoomOe1emb16tXBfM2aNbmML0lbbbVVME9bGQTEqharU9P2PE1bPXrnnXdmGucPf/hDML/yyiuD+aJFi4L57bffHsyBlm677bZgfsQRRwTzLbbYIrdjp+1tmtc7aOQl7f+P119/PZi/+GL91tJwRQ4AACBSNHIAAACRopEDAACIFI0cAABApGjkAAAAItXUSxhXrFgRzA877LBgvssuu+Ry3IceeiiYz5s3L5fxJenZZ58N5nvvvXduxwAaQdoKvKyrU9P2NZWyr0LNKu3YaXXco0ePXI6LYvvWt74VzIcMGRLM81ydWi9pe6WnrUK98cYbg/njjz8ezFeuXFnZxKqIK3IAAACRopEDAACIFI0cAABApGjkAAAAIkUjBwAAEKmmXrWaZvr06ZnyevnSl76UetuOO+5Yw5kA1Ze2j+g+++yTaZy0/UvT9k2thV69emW6f9o51/McUD9pq1PTnusdOnSo5nQq8vvf/z6Yr1u3LpjPnTs3mP/6178O5n/5y18qmlcMuCIHAAAQKRo5AACASNHIAQAARIpGDgAAIFI0cgAAAJFi1WoE9t1332A+derU1Mdss802mY6RtmrunXfeCeZLlizJND7QVgMHDsxlnDvuuCOXcfJ03HHHZbp/2t6vKLZBgwYF87TVynmtTl2zZk0wv+yyy4L5jBkzUsfaZJPw9aMnn3wymK9du7aV2aHVK3JmNsnMlpnZiy2yLmY2zcz+mnzcurrTBIqBegLyQz0B5f1qdbKkwzfIzpf0mLv3lfRY8jWA1k0W9QTkZbKoJzS5Vhs5d58uacUG8VBJU5LPp0galvO8gEKinoD8UE9A5a+R6+bui5PPl0jqlnZHMxsuaXiFxwGaAfUE5KeseqKWUBRtXuzg7m5mvpHbJ0iaIEkbux8A6gnI08bqiVpCUVTayC01s+7uvtjMuktaluekmlXW1anbbrttbsd+8MEHg/n9998fzL/73e8G80WLFuU2pyZCPZWhZ8+eme6fts/kxlbUVdvYsWODedqK3LRzSNt3FpIKXE+f/exng3m1V6deeumlwXzMmDG5HBdtU+n7yN0n6bTk89Mk3ZvPdICmRD0B+aGe0FTKefuRWyQ9K+nzZrbQzE6XNEbSoWb2V0mHJF8DaAX1BOSHegLK+NWqu5+cctPBOc8FKDzqCcgP9QSwRRcAAEC0aOQAAAAixV6rdZC2OjVt/8Q8V6dmdeSRRwbz3XffPZh/8YtfDObvvvtubnNCseW1p+rMmTNzGWdj0lbSpq1OPf7444N52urU0aNHVzYxFNIzzzwTzP/2t78F8+222y7T+HPmzAnmrE5tbFyRAwAAiBSNHAAAQKRo5AAAACJFIwcAABApGjkAAIBIsWq1ivr06RPM01anduvWLdP4K1euTL3tP/7jP4L5woULg/mZZ54ZzAcNGhTMe/fuHcw333zz1DkB5chrL9Tjjjsul3E2NlbaKtQ0d9xxRzAfP3585jmh+bz++uvB/Kabbgrm5513XqbxeR7GiStyAAAAkaKRAwAAiBSNHAAAQKRo5AAAACJFIwcAABApc/faHcysdgdrADvuuGMwT9sDsn379sH8nnvuCeZXXXVV6rGff/75VmZXnkcffTSYH3TQQcE8bbXsFVdckct8amC2uw+o9yTK0Wz1dMIJJwTz2267rcYzaV2vXr2C+YIFC2o8k/pyd6v3HMoRey3tvPPOwXzatGnBfPvttw/m77zzTjAfN25cMP/pT39axuyQk9TvTVyRAwAAiBSNHAAAQKRo5AAAACJFIwcAABApGjkAAIBI0cgBAABEqtW3HzGzSZKOkLTM3XdNskskfVvS8uRuF7j7b1s9WORLvPOyyy67BPN27doF8zlz5lRzOhv19a9/PZhPnjw5mN9///3BPG3T8Y8//riieVVRVd9+hHpqXc+ePYP5M888k+n+jejZZ58N5ieeeGIwj/3tSqr99iN51VNRa+lzn/tcME97W5LevXtnGv++++4L5mPGjEl9zIwZMzIdA/9Pm95+ZLKkwwP5OHfvn/xp9ZsOAEnUE5CnyaKe0ORabeTcfbqkFTWYC1B41BOQH+oJaNtr5EaY2Z/NbJKZbZ12JzMbbmazzGxWG44FFB31BOSn1XqillAUlTZy10jaSVJ/SYsljU27o7tPcPcBsWx7BNQB9QTkp6x6opZQFBU1cu6+1N3Xuft6SddK2ivfaQHNg3oC8kM9odlsVsmDzKy7uy9Ovjxa0ov5Tan45s6dW+8pVM2RRx4ZzDt16hTMV6zg5S3NWk8DBw4M5rfffnswz2t1atrK0TvuuCP1MWmbhqfJuvL2tttuC+Zf+tKXMh0XzVtPIa+99low79evXzAfOXJkMP/Rj34UzNP+vz/ssMNS53TeeecF8wkTJgTzNWvWpI6FklYbOTO7RdIBkrqa2UJJF0s6wMz6S3JJ8ySdUcU5AoVBPQH5oZ6AMho5dz85EE+swlyAwqOegPxQTwA7OwAAAESLRg4AACBSNHIAAACRanWv1VwPVtD97Ios616rb775ZjDfY489gvnq1asrmlcVVXWv1TzFUk/12js1bf/StFWxtZB2bm+99VYwb8RzyKLae63mJZZaqra0/b532mmnYH7XXXcF889//vOpxzALPyXSxho/fnwwf/rpp1OPUVBt2msVAAAADYhGDgAAIFI0cgAAAJGikQMAAIgUjRwAAECkKtprFc0jbU++NL/97W+DeQOuTkWNjB07NphnXZ26YMGCYH7CCScE8xkzZmQavxbSziEt33vvvYN5LKtWEZePP/44mL/yyivB/MADDwzmaautpfRVqEcffXQwP/jgg4P5nnvuGcznzZuXeuyi4oocAABApGjkAAAAIkUjBwAAECkaOQAAgEjRyAEAAESKvVYhKX21adqKobQ9+Xbddddg/tJLL1U2sdpjr9Wcpe0jmrZqNW0F57777pvp/jFJW4V6/PHHB/O0/SobDXutYkPr168P5ll7kaOOOiqYP/jgg5nnFAn2WgUAACgaGjkAAIBI0cgBAABEikYOAAAgUjRyAAAAkWp1r1Uz6ynpBkndJLmkCe5+lZl1kXSbpD6S5kk6wd3frd5Ua6dDhw7B/Oabbw7m/fv3zzT+r3/962A+ZsyYTONU4vDDDw/mWVen/uUvfwnmixYtqmxiTaLI9XTOOecE86x7qp577rnBvAirU5Gv2Otpu+22C+bbb799MJ8zZ04wX7NmTW5zyqJPnz7B/NRTT636sQ899NBgXuBVq6nKuSK3VtJod+8naaCks8ysn6TzJT3m7n0lPZZ8DWDjqCcgP9QTml6rjZy7L3b355PPV0l6WVIPSUMlTUnuNkXSsGpNEigK6gnID/UElPGr1ZbMrI+kPSXNlNTN3RcnNy1R6dJ26DHDJQ2vfIpAMVFPQH6y1hO1hKIoe7GDmXWUNFXSSHd/v+VtXnpL5uDbMrv7BHcfEMu75QO1QD0B+amknqglFEVZjZyZtVOpSG5y97uSeKmZdU9u7y5pWXWmCBQL9QTkh3pCsytn1apJmijpZXe/ssVN90k6TdKY5OO9VZlhHXTu3DmYDx06NJhn3SNu4cKFmeeU1Q9+8INgft555wXzrKtTDzvssGC+cuXKMmbXvIpcT/vss08u4zz77LO5jNOIBg4cmCm/8sorgzlKYq+njh07BvPTTz89mKetAJ89e3Ywv/baa4N52v7Haatle/ToEcxvuummYL7DDjsEcyl9n+Cs30evueaaTPcvsnJeI7evpFMlvWBmn6x9vkClArndzE6XNF/SCdWZIlAo1BOQH+oJTa/VRs7dn5YUbqGl8JuPAQiinoD8UE8AOzsAAABEi0YOAAAgUjRyAAAAkbKsK0XadDCz2h2sDTbZJNzfTpkyJZh/9atfzTT+6tWrg/mbb74ZzCdPnpxpfEk6++yzg3mvXr2C+auvvhrML7/88mCetlqpAGbH8r5SjVZPY8eODeajRo3KNE7aqraYpO07m7YKNW2l7oknnhjMY9l31t2j+MdstFr6whe+EMynTp0azLfddttgnrYH66OPPhrM999//2Cedb/kjUn7/rp+/fpg/ve//z2Yp+1x/vrrr1c2scaX+r2JK3IAAACRopEDAACIFI0cAABApGjkAAAAIkUjBwAAEClWrWbQrVu3YH7ZZZcF829/+9vVnM5GvfHGG8H8zjvvDOZXX311MI9ldVyOWLWasz/84Q/BPG1v1rRVruPGjcttTlml7YWaNte0PSvT9lkePXp0MI+9/li1WhtHHnlkML/55puDeYcOHao5nY3Kumo1bb/YM888M7c5RYJVqwAAAEVDIwcAABApGjkAAIBI0cgBAABEikYOAAAgUqxazUHa3pDbbbddMD/jjDOCeY8ePYL5wQcfHMxvuOGG1DlNnDgxmM+fPz/1MZDEqtWaybo3a9oKzhkzZuQyn+OPPz71trRj33HHHcF8/PjxmcYpKlat1lefPn2C+UknnRTM0/YN79evX15T0vTp04P5ww8/HMzT3lFh1apVuc0pEqxaBQAAKBoaOQAAgEjRyAEAAESKRg4AACBSNHIAAACRanXVqpn1lHSDpG6SXNIEd7/KzC6R9G1Jy5O7XuDuv21lrEKuDEKhVHXVKvWEZlLNVavUEppM6vemzcp48FpJo939eTPrJGm2mU1Lbhvn7lfkNUugCVBPQD6oJUBlNHLuvljS4uTzVWb2sqTwG54B2CjqCcgHtQSUZHqNnJn1kbSnpJlJNMLM/mxmk8xs65THDDezWWY2q00zBQqGegLyQS2hmZW9s4OZdZT0pKQfu/tdZtZN0tsqvTbhcknd3f3fWxmD1yGg0dVkZwfqCc2gFjs7UEtoEm3b2cHM2kmaKukmd79Lktx9qbuvc/f1kq6VtFdeswWKjHoC8kEtAWU0clbaSHSipJfd/coWefcWdzta0ov5Tw8oFuoJyAe1BJSUs2p1X0mnSnrBzOYk2QWSTjaz/ipdvp4nKbwTPICWqCcgH9QSoAyvkcvlYLwOAY2vJq+RywP1hEZXi9fI5YFaQgTa9ho5AAAANB4aOQAAgEjRyAEAAESKRg4AACBSNHIAAACRopEDAACIFI0cAABApGjkAAAAIkUjBwAAEKlytujK09uS5iefd02+bhacbxx613sCGVBPzSPG86WW4tFs5xzj+abWU0236PqnA5vNimUrpDxwvqimZvv75nxRLc34d91s51y08+VXqwAAAJGikQMAAIhUPRu5CXU8dj1wvqimZvv75nxRLc34d91s51yo863ba+QAAADQNvxqFQAAIFI0cgAAAJGqeSNnZoeb2V/M7DUzO7/Wx68FM5tkZsvM7MUWWRczm2Zmf00+bl3POebJzHqa2RNm9pKZzTWzs5O8sOfcKKin4j23qKf6oZ6K9dxqllqqaSNnZptKulrSYEn9JJ1sZv1qOYcamSzp8A2y8yU95u59JT2WfF0UayWNdvd+kgZKOiv5dy3yOdcd9VTY5xb1VAfUUyGfW01RS7W+IreXpNfc/Q13XyPpVklDazyHqnP36ZJWbBAPlTQl+XyKpGE1nVQVuftid38++XyVpJcl9VCBz7lBUE8lhXpuUU91Qz2VFOa51Sy1VOtGroekBS2+XphkzaCbuy9OPl8iqVs9J1MtZtZH0p6SZqpJzrmOqKeSwj63qKeaop5KCvncKnItsdihDrz0ni+Fe98XM+soaaqkke7+fsvbinrOqL+iPreoJ9RDEZ9bRa+lWjdyiyT1bPH19knWDJaaWXdJSj4uq/N8cmVm7VQqlJvc/a4kLvQ5NwDqScV8blFPdUE9qXjPrWaopVo3cs9J6mtmO5hZe0knSbqvxnOol/sknZZ8fpqke+s4l1yZmUmaKOlld7+yxU2FPecGQT2VFOq5RT3VDfVUUpjnVrPUUs13djCzIZLGS9pU0iR3/3FNJ1ADZnaLpAMkdZW0VNLFku6RdLukXpLmSzrB3Td8wWmUzGw/SU9JekHS+iS+QKXXIhTynBsF9VS85xb1VD/UU7GeW81SS2zRBQAAECkWOwAAAESKRg4AACBSNHIAAACRopEDAACIFI0cAABApGjkAAAAIkUjBwAAECkaOQAAgEjRyAEAAESKRg4AACBSNHIAAACRopEDAACIFI1cgzCzS8zsxnrPAygC6gnID/XU2GjkasjMTjGzWWa22swWm9lDZrZfHebRK5lDyz9uZqNrPRegUo1STy3mc7aZvWlmH5jZy2a2c73mAmTVSPVkZn3M7Akz+7uZvWJmh9RjHrGgkasRMxslabykn0jqJqmXpF9JGlrrubj7W+7e8ZM/knaTtF7S1FrPBahEI9VTMp9vSTpd0lckdZR0hKS36zEXIKtGqydJt0j6o6TPSLpQ0p1mtk2d5tLwaORqwMw6S7pM0lnufpe7f+DuH7v7/e5+Xspj7jCzJWa20symm9kuLW4bYmYvmdkqM1tkZucmeVcze8DM3jOzFWb2lJmV82/8dUnT3X1eDqcLVFWj1VOSXSzpHHd/yUted/cV1fkbAPLTgPW0s6QvSrrY3T9096mSXpB0bDXOvwho5GpjH0lbSLo7w2MektRX0raSnpd0U4vbJko6w907SdpV0uNJPlrSQknbqPRT1QWSfGMHMTNTqZGbkmFuQD01Wj1tn/zZ1cwWJL9evbTMH6KAemu0etpF0hvuvqpF9qckR8Bm9Z5Ak/iMpLfdfW25D3D3SZ98bmaXSHrXzDq7+0pJH0vqZ2Z/cvd3Jb2b3PVjSd0l9Xb31yQ9Vcah9lOpqO4sd25AnTVaPW2ffByk0ssUPi3pEZW+aV1b9lkB9dFo9dRR0soNspWSepQ7v2bDT4y18Y6krmZWVuNsZpua2Rgze93M3pc0L7mpa/LxWElDJM03syfNbJ8k/4Wk1yQ9YmZvmNn5ZRzuNElT3X11uScD1Fmj1dOHycefu/t7yUsU/icZE2h0jVZPqyVttUG2laRVgftCNHK18qykjyQNK/P+p6j0ItNDJHWW1CfJTZLc/Tl3H6rSZe17JN2e5KvcfbS77yjpKEmjzOzgtIOY2ackHS9+rYq4NFo9/UXSGv3zr4k2+pIGoIE0Wj3NlbSjmXVqke2R5AigkauB5HLzRZKuNrNhZtbBzNqZ2WAz+3ngIZ1UKqx3JHVQaSWRJMnM2pvZV5PL2B9Lel+lFacysyPM7HPJ695WSlr3yW0pjlbpsvcTOZwmUBONVk/u/ndJt0n6DzPrZGbbSxou6YE8zxuohgasp1clzZF0sZltYWZHS9pdvKtCKhq5GnH3sZJGSfqhpOWSFkgaodJPLBu6QdJ8SYskvSRpxga3nyppXnJZ+0xJX03yvpIeVenS9LOSfuXuG2vSTpP0G3fn6gGi0oD1NCK539+S+94saVLKfYGG0oD1dJKkASpdaBgj6Th3X17RyTUB43s4AABAnLgiBwAAECkaOQAAgEjRyAEAAESKRg4AACBSbdrZwcwOl3SVpE0lXefuY1q5Pysr0Ojedve6bM5MPaFo3N3qdews9UQtIQKp35sqviJnZptKulrSYEn9JJ1sZv0qHQ9oEPPrcVDqCcgP9YQCSv3e1JZfre4l6TV3f8Pd10i6VaV3ewaQHfUE5Id6QtNoSyPXQ6U3DfzEQgU2tTWz4WY2y8xmteFYQNFRT0B+Wq0naglF0abXyJXD3SdImiDxOgSgragnIB/UEoqiLVfkFknq2eLr7ZMMQHbUE5Af6glNoy2N3HOS+prZDmbWXqW90e7LZ1pA06GegPxQT2gaFf9q1d3XmtkISQ+rtLx7krvPzW1mQBOhnoD8UE9oJuZeu5cG8DoERGC2uw+o9yTKQT2h0dXzfeSyoJYQgdTvTezsAAAAECkaOQAAgEjRyAEAAESKRg4AACBSNHIAAACRopEDAACIFI0cAABApGjkAAAAIkUjBwAAEKmKt+gCgBj16dMnmD/++OOpj9lhhx2C+Q9+8INgPmbMmMzzAoBKcEUOAAAgUjRyAAAAkaKRAwAAiBSNHAAAQKRo5AAAACLFqlUAhZS2OnXatGnBvHfv3qljrV+/Ppjvu+++mecF1Mr06dOD+X777ZfL+GYWzOfMmZP6mLFjxwbzG2+8MZc5NSOuyAEAAESKRg4AACBSNHIAAACRopEDAACIFI0cAABApNq0atXM5klaJWmdpLXuPiCPSQHNiHqqzGabhf8b++UvfxnMd9xxx9yOnbYCFvVHPUnz588P5nmttnb3YL777runPua6664L5iNGjAjmafsZ//GPfwzm7733XuqxiyqPtx850N3fzmEcANQTkCfqCYXHr1YBAAAi1dZGziU9YmazzWx46A5mNtzMZpnZrDYeCyg66gnIz0briVpCUbT1V6v7ufsiM9tW0jQze8Xd/+mtpN19gqQJkmRm4V+oA5CoJyBPG60naglF0aYrcu6+KPm4TNLdkvbKY1JAM6KegPxQT2gWFV+RM7MtJW3i7quSzwdJuiy3mQFNhHpqXdrq1C9/+cvBfMiQIbkde9as8G/fbrvtttyOgfxQTyXf//73g/lnP/vZYH7QQQdVczqSpHbt2gXzf/u3fwvmjz76aDA/8cQTg/mdd95Z2cQi1pZfrXaTdHeyae5mkm5299/lMiug+VBPQH6oJzSNihs5d39D0h45zgVoWtQTkB/qCc2Etx8BAACIFI0cAABApGjkAAAAIpXHFl3R6tOnTzD/3ve+l2mc8ePH5zCb+ho5cmQw7969ezBPWzGUZujQocH8gQceyDQOmlfaHqlpq9qyevXVV1NvS3v+Ll26NJdjA9Wwfv36YP6Pf/wj0zhvvfVWMP/CF74QzPfYI/3lid/97neD+W677RbM+/fvH8yvvfbaYP7mm28G89mzZ6fOKXZckQMAAIgUjRwAAECkaOQAAAAiRSMHAAAQKRo5AACASJm71+5gZrU7WBkGDhwYzJ9++ulM4yTbwPwvef3dVnv8Whxj0aJFwbx37965jJ+j2e4+oN6TKEej1VO1nXfeecF8zJgxmcZ5/fXXg/mIESNSH/PII49kOgZK3D38H0uDKWot7b///sH88ccfzzTOzJkzg/mXvvSlzHNK06tXr2B+7733BvPdd989mC9cuDCYH3vsscE8bR/lBpT6vYkrcgAAAJGikQMAAIgUjRwAAECkaOQAAAAiRSMHAAAQqabea3Xu3LnB/O677w7mRx99dDWnA0BSv379gvl3vvOdXMZfvnx5MGdlKormsssuy2Wciy66KJdxNiZtP9e077tp36fTVrNOnTo1mB9zzDHBPKa9WbkiBwAAECkaOQAAgEjRyAEAAESKRg4AACBSNHIAAACRanXVqplNknSEpGXuvmuSdZF0m6Q+kuZJOsHd363eNKtj1apVwfy0004L5ptvvnkw/8pXvpLbnIrquuuuq/cUGkKR6ykv55xzTjDPui/v3//+92A+adKkzHNCY6Ke8rVgwYJgvmzZshrP5P+bN29eMH/11VeDedqq1e233z6Y/8u//EswL9qq1cmSDt8gO1/SY+7eV9JjydcAWjdZ1BOQl8mintDkWm3k3H26pBUbxEMlTUk+nyJpWM7zAgqJegLyQz0Blb8hcDd3X5x8vkRSt7Q7mtlwScMrPA7QDKgnID9l1RO1hKJo884O7u5m5hu5fYKkCZK0sfsBoJ6APG2snqglFEWlq1aXmll3SUo+1u+VkED8qCcgP9QTmkqlV+Tuk3SapDHJx3tzm1EDSFvtdtRRR+Uy/s477xzMFy9eHMzTVtduzDe/+c1gnrZ6dJNNwj39+vXrMx33wQcfDOaXX355pnGaTKHrKU2XLl2C+aBBgzKNk1avI0eODOYTJ07MNH6eTj755GB+yimn5DJ+2jiV/B8Ssaarp0MOOSSY77rrrpnGmTFjRjD/85//nHlO1bZixYYvjaxM2v8TN954Yy7j10KrV+TM7BZJz0r6vJktNLPTVSqQQ83sr5IOSb4G0ArqCcgP9QSUcUXO3cM/QkoH5zwXoPCoJyA/1BPAzg4AAADRopEDAACIFI0cAABApNr8PnLILm2PuKzSVr9K0s9+9rNg7h5+u6S01alp93/nnXeC+be+9a3UOaE5tWvXLphfeOGFwTxtT8Q0v//974N5LVan7r333sE8bZX2wIEDg/mWW26Zy3y+8Y1vBPNf/vKXuYyPxtS9e/dg/ulPf7rGM6mdH/zgB8F8+PBs7/GcttdqTLgiBwAAECkaOQAAgEjRyAEAAESKRg4AACBSNHIAAACRYtVqBDp06BDMf/KTn6Q+Jm0fy7w88cQTwXzZMvanxj/baqutgnnaHodpFi5cGMz//d//PfOcsjrhhBOC+Q9/+MNgvssuu1RzOqn69OlTl+OiGM4999x6TwEV4IocAABApGjkAAAAIkUjBwAAECkaOQAAgEjRyAEAAESKVas5uOiii4L56aefnmmc6667LpjvtddewXzw4MGZxs/TqlWrgvk555wTzMeNG1fN6aCB9e3bN5dxPvzww2C+fPnyTONsvfXWwfykk05KfcxPf/rTYN6pU6dMxwYa2fvvv1/vKaACXJEDAACIFI0cAABApGjkAAAAIkUjBwAAECkaOQAAgEi12siZ2SQzW2ZmL7bILjGzRWY2J/kzpLrTBIqBegLyQz0B5b39yGRJ/y3phg3yce5+Re4zilDnzp2D+fbbuMTLlgAADHNJREFUb59pnEsuuSSYm1kwd/dM429M2jHSZN2o/NJLLw3mQ4aE/499+umnM40fkclqsno68cQTcxkn7e150gwcODCY77///sH8Jz/5SeY5ZfXYY48F8xdeeCGYjxw5sprTKYLJarJ6CvnKV75S7ymgjlq9Iufu0yWtqMFcgMKjnoD8UE9A214jN8LM/pxc2g6/wyaAclFPQH6oJzSNShu5ayTtJKm/pMWSxqbd0cyGm9ksM5tV4bGAoqOegPyUVU/UEoqiokbO3Ze6+zp3Xy/pWknhPaRK953g7gPcfUClkwSKjHoC8lNuPVFLKIqKGjkz697iy6MlvZh2XwAbRz0B+aGe0GxaXbVqZrdIOkBSVzNbKOliSQeYWX9JLmmepDOqOMeGd8MNGy6YKunSpUswP/roo4N5x44dMx03z1Wr1T7GlltuGczTNiP/8pe/nMtxG02R6ynt3/hrX/tapnFWrlwZzKdNmxbMhw4dGsynTJkSzPPc6P5vf/tbMP/GN74RzOfPnx/Mp06dmst8Xn755VzGiUWR6ymLtBpAc2i1kXP3kwPxxCrMBSg86gnID/UEsLMDAABAtGjkAAAAIkUjBwAAECkaOQAAgEiVs9cqWvGnP/0pmH/zm98M5jvvvHMw33vvvXObE1Brm2wS/rkwbfV2mrS9i4866qhgfu655wbzrKvAN+bhhx8O5hdddFEwX7ZsWTC///77g/muu+6aaT4333xzML/++uszjQPE6swzz8xlnA8//DCXceqJK3IAAACRopEDAACIFI0cAABApGjkAAAAIkUjBwAAEClWrVbRdtttF8y7d+8ezPP06quvBvMHH3wwmJtZME/ba/W1117LNH6aNWvWZLo/GtcHH3wQzNNWUqat6k4zatSoYJ7X6tT/+Z//yXzsww47LJhPnBjeJSrr6tQ0P/rRj4L5unXrchkfaBRnnBHeKvekk07KZfyRI0fmMk49cUUOAAAgUjRyAID/2979hVaRnnEc/z27pleuEN0lSnS1lPUiiijIKlZB8GarF64gS/dC7E1VbFHBCyUiLQShF42tYClY/9R/tKwoVVAQWdSuXkiNlLq7EnfRBqtZbRU04EW1Pr0445KVd5Jz4sycvDPfz01OnuTMvO9xfpwnx3lnAESKRg4AACBSNHIAAACRopEDAACIFKtWMzBmTPhlXLNmTbA+derUhraftqK0r68v9TlLliwJ1vv7+xvaN1CvFy9eBOsDAwOZbH/cuHGZbCfNUCuoL1y4EKzPnj07WG9paWlo3729vcH6zp07g/W0VePA6+jq6grWN27cmNk+0lZur1q1KlhPW1Wa9r6b5ujRo8H6qVOnGtrOaMQncgAAAJGikQMAAIgUjRwAAECkaOQAAAAiNWwjZ2ZTzOy8mX1pZl+Y2cakPt7MzpnZV8nX1vyHC8SNPAHZIU+AZGn30vz2F8wmSZrk7tfM7C1JPZI+lPQTSY/c/VdmtlVSq7tvGWZbQ+8sUuvWrQvWd+/encn2nz17FqwPdY+4oe4biSH1uPvcvDZexTxNnjw5WB9q1XVZHT58OFjfvn17sH7nzp08h5M7dw8vuc9IVnmKJUtpdu3aFayvX78+WH/jjfBnOGmZPHnyZLB+5cqVYH3p0qXBuiQtXrw4WG9vb099TiOeP38erM+YMSNYj2gFeOp707CfyLl7v7tfSx4PSLohqV3SckkHk187qFp4AAyBPAHZIU9Ag+fImdk0SXMkXZHU5u4vL0r2jaS2TEcGlBx5ArJDnlBVdV9Rz8zGSjouaZO7Pxl8kVp397SPps1sjaTwlXGBiiJPQHZGkieyhLKo6xM5M2tRLSRH3f1EUr6fnJ/w8jyFB6Hnuvsed5+b53lHQEzIE5CdkeaJLKEs6lm1apL2Sbrh7oPvF3NK0urk8WpJ4bMhAXyLPAHZIU9AfatWF0r6TNJ1SS9vptip2nkIn0h6V1KfpI/c/dEw24p6ZdD06dOD9fPnzwfrbW3ZnJbR3d0drG/ZMuSiRoxM3qtWK5entHsFr1y5Mljv7OwM1mfNmpXZmLJy69atYH3Hjh3B+qFDh4L1tPvUxq6AVauZ5CmWLDXq4sWLwfrChQsLHkn2zpw5E6zv3bs3WE9beRuR1PemYc+Rc/dLktLCGL4zO4Ag8gRkhzwB3NkBAAAgWjRyAAAAkaKRAwAAiBSNHAAAQKSGXbWa6c4iWRmUtjr13LlzwXravSSzem1bW8P3ex4YGMhk+/iOXFetZimWPDVqwoQJwfqGDRuC9XfeeSdYX7t2bbB+9+7dYP3AgQOpY7p9+3awfuTIkWA97X6PVZP3qtWslDVLCxYsCNaPHTsWrE+cODHP4Qzp3r17wfrZs2eD9c2bNwfrjx8/zmxMo8zI77UKAACA0YlGDgAAIFI0cgAAAJGikQMAAIgUjRwAAECkKr1qddq0acH65cuXg/W0FT1p95JMe23Ttr9s2bJgndWphWLVKpARVq2OTvPmzQvWZ86cGaxv2rQpWO/o6Gh437t37w7W0+5D3NPT0/A+SopVqwAAAGVDIwcAABApGjkAAIBI0cgBAABEikYOAAAgUpVetTp//vxg/dKlSw1tJ23Vam9vb7C+aNGiYP3hw4cN7Re5YNUqkBFWrQKZYdUqAABA2dDIAQAARIpGDgAAIFI0cgAAAJGikQMAAIjUmOF+wcymSDokqU2SS9rj7rvM7JeSfirp38mvdrr7mbwGmodHjx41VB8/fnyw/vTp02B927ZtwTqrU6urzHkCikSWgJphGzlJzyVtdvdrZvaWpB4zO5f87Dfu/uv8hgeUDnkCskGWANXRyLl7v6T+5PGAmd2Q1J73wIAyIk9ANsgSUNPQOXJmNk3SHElXktLPzewfZrbfzFpTnrPGzK6a2dXXGilQMuQJyAZZQpXV3ciZ2VhJxyVtcvcnkn4v6QeSZqv2V1F36Hnuvsfd58ZytXygCOQJyAZZQtXV1ciZWYtqQTnq7ickyd3vu/v/3P2FpD9Iej+/YQLlQZ6AbJAloL5VqyZpn6Qb7r5zUH1Sco6CJK2Q9Hk+Q8zPzZs3g/UVK1YE611dXcF6d3fwDz6dPn16ZANDaZU5T0CRyBJQU8+q1R9KWiXpupn9Pal1SvrYzGartuz7n5LW5jJCoFzIE5ANsgSovlWrlyRZ4EdclwdoEHkCskGWgBru7AAAABApGjkAAIBI0cgBAABEyty9uJ2ZFbczYGR6YrmuFHnCaOfuoXPYRh2yhAikvjfxiRwAAECkaOQAAAAiRSMHAAAQKRo5AACASNHIAQAARKqeW3Rl6T+S+pLHbyffVwXzjcPUZg+gAeSpOmKcL1mKR9XmHON8U/NU6OVHvrNjs6uxXOYhC8wXeara6818kZcqvtZVm3PZ5st/rQIAAESKRg4AACBSzWzk9jRx383AfJGnqr3ezBd5qeJrXbU5l2q+TTtHDgAAAK+H/1oFAACIFI0cAABApApv5MzsAzPrNbOvzWxr0fsvgpntN7MHZvb5oNp4MztnZl8lX1ubOcYsmdkUMztvZl+a2RdmtjGpl3bOowV5Kt+xRZ6ahzyV69iqSpYKbeTM7E1Jv5P0I0kdkj42s44ix1CQP0r64JXaVkmfuvt7kj5Nvi+L55I2u3uHpPmSfpb8u5Z5zk1Hnkp7bJGnJiBPpTy2KpGloj+Re1/S1+5+y93/K+nPkpYXPIbcuftfJT16pbxc0sHk8UFJHxY6qBy5e7+7X0seD0i6IaldJZ7zKEGeakp1bJGnpiFPNaU5tqqSpaIbuXZJdwZ9/6+kVgVt7t6fPP5GUlszB5MXM5smaY6kK6rInJuIPNWU9tgiT4UiTzWlPLbKnCUWOzSB1675UrrrvpjZWEnHJW1y9yeDf1bWOaP5ynpskSc0QxmPrbJnqehG7q6kKYO+n5zUquC+mU2SpOTrgyaPJ1Nm1qJaUI66+4mkXOo5jwLkSeU8tshTU5Anle/YqkKWim7k/ibpPTP7vpl9T9KPJZ0qeAzNckrS6uTxakknmziWTJmZSdon6Ya77xz0o9LOeZQgTzWlOrbIU9OQp5rSHFtVyVLhd3Yws6WSfivpTUn73X1HoQMogJn9SdJiSW9Lui/pF5L+IukTSe9K6pP0kbu/esJplMxsoaTPJF2X9CIpd6p2LkIp5zxakKfyHVvkqXnIU7mOrapkiVt0AQAARIrFDgAAAJGikQMAAIgUjRwAAECkaOQAAAAiRSMHAAAQKRo5AACASNHIAQAAROr/fsRNOgEm6v8AAAAASUVORK5CYII=\n" | |
| }, | |
| "metadata": { | |
| "needs_background": "light" | |
| } | |
| } | |
| ], | |
| "source": [ | |
| "plt.rcParams['figure.figsize'] = (9,9) # Make the figures a bit bigger\n", | |
| "\n", | |
| "for i in range(9):\n", | |
| " plt.subplot(3,3,i+1)\n", | |
| " num = random.randint(0, len(X_train))\n", | |
| " plt.imshow(X_train[num], cmap='gray', interpolation='none')\n", | |
| " plt.title(\"Class {}\".format(y_train[num]))\n", | |
| " \n", | |
| "plt.tight_layout()" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "035GHESzSZWv" | |
| }, | |
| "source": [ | |
| "Let's examine a single digit a little closer, and print out the array representing the last digit." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "XAbodwlxSZWw", | |
| "outputId": "cd90f2da-0401-4efc-daba-2ff4e7edc354", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 194 254 254 254 255 207 92 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 164 253 253 253 253 253 248 200 65 16 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 17 187 253 253 253 253 253 253 253 84 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 15 180 253 223 7 54 160 205 253 239 51 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 63 253 253 198 0 0 0 30 207 253 225 52 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 193 253 253 68 0 0 0 0 32 207 253 225 51 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 193 253 224 46 0 0 0 0 0 169 253 253 92 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 68 237 253 168 0 0 0 0 0 0 151 253 253 204 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 93 253 253 168 0 0 0 0 0 0 16 253 253 245 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 93 253 253 28 0 0 0 0 0 0 16 253 253 245 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 93 253 253 14 0 0 0 0 0 0 16 253 253 245 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 168 253 253 14 0 0 0 0 0 0 16 253 253 245 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 247 253 253 14 0 0 0 0 0 0 16 253 253 245 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 247 253 253 14 0 0 0 0 0 0 120 253 253 236 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 247 253 253 14 0 0 0 0 0 103 208 253 253 92 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 247 253 253 145 0 0 0 44 137 249 253 253 162 14 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 145 251 253 222 143 88 162 226 253 253 253 249 36 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 193 253 253 253 253 253 253 253 253 180 79 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 58 229 253 253 253 253 253 227 156 15 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 55 99 211 253 253 145 51 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n", | |
| "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 \n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "# just a little function for pretty printing a matrix\n", | |
| "def matprint(mat, fmt=\"g\"):\n", | |
| " col_maxes = [max([len((\"{:\"+fmt+\"}\").format(x)) for x in col]) for col in mat.T]\n", | |
| " for x in mat:\n", | |
| " for i, y in enumerate(x):\n", | |
| " print((\"{:\"+str(col_maxes[i])+fmt+\"}\").format(y), end=\" \")\n", | |
| " print(\"\")\n", | |
| "\n", | |
| "# now print! \n", | |
| "matprint(X_train[num])" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "7dSo-0-VSZWw" | |
| }, | |
| "source": [ | |
| "Each pixel is an 8-bit integer from 0-255. 0 is full black, while 255 is full white. This what we call a single-channel pixel. It's called monochrome.\n", | |
| "\n", | |
| "*Fun-fact! Your computer screen has three channels for each pixel: red, green, blue. Each of these channels also likely takes an 8-bit integer. 3 channels -- 24 bits total -- 16,777,216 possible colors!*" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "64CrKKSTSZWx" | |
| }, | |
| "source": [ | |
| "## Formatting the input data layer\n", | |
| "\n", | |
| "Instead of a 28 x 28 matrix, we build our network to accept a 784-length vector.\n", | |
| "\n", | |
| "Each image needs to be then reshaped (or flattened) into a vector. We'll also normalize the inputs to be in the range [0-1] rather than [0-255]. Normalizing inputs is generally recommended, so that any additional dimensions (for other network architectures) are of the same scale." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "vC8EDgfBSZWx" | |
| }, | |
| "source": [ | |
| "<img src='https://github.com/AviatorMoser/keras-mnist-tutorial/blob/master/flatten.png?raw=1' >" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "PwbuhtvvSZWx", | |
| "outputId": "57f2049f-31ed-4e14-ba22-389aa532a98b", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "Training matrix shape (60000, 784)\n", | |
| "Testing matrix shape (10000, 784)\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "X_train = X_train.reshape(60000, 784) # reshape 60,000 28 x 28 matrices into 60,000 784-length vectors.\n", | |
| "X_test = #fill here # reshape 10,000 28 x 28 matrices into 10,000 784-length vectors.\n", | |
| "\n", | |
| "X_train = X_train.astype('float32') # change integers to 32-bit floating point numbers\n", | |
| "X_test = X_test.astype('float32')\n", | |
| "\n", | |
| "X_train /= 255 # normalize each value for each pixel for the entire vector for each input\n", | |
| "X_test /= 255\n", | |
| "\n", | |
| "print(\"Training matrix shape\", X_train.shape)\n", | |
| "print(\"Testing matrix shape\", X_test.shape)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "wV3YTokPSZWy" | |
| }, | |
| "source": [ | |
| "We then modify our classes (unique digits) to be in the one-hot format, i.e.\n", | |
| "\n", | |
| "```\n", | |
| "0 -> [1, 0, 0, 0, 0, 0, 0, 0, 0]\n", | |
| "1 -> [0, 1, 0, 0, 0, 0, 0, 0, 0]\n", | |
| "2 -> [0, 0, 1, 0, 0, 0, 0, 0, 0]\n", | |
| "etc.\n", | |
| "```\n", | |
| "\n", | |
| "If the final output of our network is very close to one of these classes, then it is most likely that class. For example, if the final output is:\n", | |
| "\n", | |
| "```\n", | |
| "[0, 0.94, 0, 0, 0, 0, 0.06, 0, 0]\n", | |
| "```\n", | |
| "then it is most probable that the image is that of the digit `1`." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "3WcJN1SjSZWy" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "nb_classes = 10 # number of unique digits\n", | |
| "\n", | |
| "Y_train = np_utils.to_categorical(y_train, nb_classes)\n", | |
| "Y_test = np_utils.to_categorical(y_test, nb_classes)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "V0meJYjdSZWy" | |
| }, | |
| "source": [ | |
| "# Building a 3-layer fully connected network (FCN)\n", | |
| "\n", | |
| "<img src=\"https://github.com/AviatorMoser/keras-mnist-tutorial/blob/master/figure.png?raw=1\" />" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "pMRztIAHSZWz" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# The Sequential model is a linear stack of layers and is very common.\n", | |
| "\n", | |
| "model = Sequential()" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "85H3ItNISZWz" | |
| }, | |
| "source": [ | |
| "## The first hidden layer" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "xtbUnhyvSZWz" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# The first hidden layer is a set of 512 nodes (artificial neurons).\n", | |
| "# Each node will receive an element from each input vector and apply some weight and bias to it.\n", | |
| "\n", | |
| "model.add(Dense(512, input_shape=(784,))) #(784,) is not a typo -- that represents a 784 length vector!" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "muQXYwRkSZWz" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# An \"activation\" is a non-linear function applied to the output of the layer above.\n", | |
| "# It checks the new value of the node, and decides whether that artifical neuron has fired.\n", | |
| "# The Rectified Linear Unit (ReLU) converts all negative inputs to nodes in the next layer to be zero.\n", | |
| "# Those inputs are then not considered to be fired.\n", | |
| "# Positive values of a node are unchanged.\n", | |
| "\n", | |
| "#fill here" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "mbr9yOvQSZW0" | |
| }, | |
| "source": [ | |
| "$$f(x) = max (0,x)$$\n", | |
| "<img src = 'relu.jpg' >" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "a_UOk65ZSZW0" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Dropout zeroes a selection of random outputs (i.e., disables their activation)\n", | |
| "# Dropout helps protect the model from memorizing or \"overfitting\" the training data.\n", | |
| "model.add(Dropout(0.2))" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "MpY_KMlSSZW0" | |
| }, | |
| "source": [ | |
| "## Adding the second hidden layer" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "tsbpjukcSZW0" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# The second hidden layer appears identical to our first layer.\n", | |
| "# However, instead of each of the 512-node receiving 784-inputs from the input image data,\n", | |
| "# they receive 512 inputs from the output of the first 512-node layer.\n", | |
| "\n", | |
| "model.add(Dense(512))\n", | |
| "model.add(Activation('relu'))\n", | |
| "#fill here" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "XNilP6HwSZW1" | |
| }, | |
| "source": [ | |
| "## The Final Output Layer" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "QiSeDRaDSZW1" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# The final layer of 10 neurons in fully-connected to the previous 512-node layer.\n", | |
| "# The final layer of a FCN should be equal to the number of desired classes (10 in this case).\n", | |
| "model.add(Dense(10))" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "-mr-z2KgSZW1" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# The \"softmax\" activation represents a probability distribution over K different possible outcomes.\n", | |
| "# Its values are all non-negative and sum to 1.\n", | |
| "\n", | |
| "model.add(Activation('softmax'))" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "HEOQz_a5SZW1", | |
| "outputId": "327070f1-f153-453a-f519-67e4adbb7e43", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "Model: \"sequential_1\"\n", | |
| "_________________________________________________________________\n", | |
| " Layer (type) Output Shape Param # \n", | |
| "=================================================================\n", | |
| " dense_3 (Dense) (None, 512) 401920 \n", | |
| " \n", | |
| " activation_3 (Activation) (None, 512) 0 \n", | |
| " \n", | |
| " dropout_2 (Dropout) (None, 512) 0 \n", | |
| " \n", | |
| " dense_4 (Dense) (None, 512) 262656 \n", | |
| " \n", | |
| " activation_4 (Activation) (None, 512) 0 \n", | |
| " \n", | |
| " dropout_3 (Dropout) (None, 512) 0 \n", | |
| " \n", | |
| " dense_5 (Dense) (None, 10) 5130 \n", | |
| " \n", | |
| " activation_5 (Activation) (None, 10) 0 \n", | |
| " \n", | |
| "=================================================================\n", | |
| "Total params: 669,706\n", | |
| "Trainable params: 669,706\n", | |
| "Non-trainable params: 0\n", | |
| "_________________________________________________________________\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "# Summarize the built model\n", | |
| "\n", | |
| "# fill here" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "sWFZFpyjSZW1" | |
| }, | |
| "source": [ | |
| "## Compiling the model\n", | |
| "\n", | |
| "Keras is built on top of Theano and TensorFlow. Both packages allow you to define a *computation graph* in Python, which then compiles and runs efficiently on the CPU or GPU without the overhead of the Python interpreter.\n", | |
| "\n", | |
| "When compiing a model, Keras asks you to specify your **loss function** and your **optimizer**. The loss function we'll use here is called *categorical cross-entropy*, and is a loss function well-suited to comparing two probability distributions.\n", | |
| "\n", | |
| "Our predictions are probability distributions across the ten different digits (e.g. \"we're 80% confident this image is a 3, 10% sure it's an 8, 5% it's a 2, etc.\"), and the target is a probability distribution with 100% for the correct category, and 0 for everything else. The cross-entropy is a measure of how different your predicted distribution is from the target distribution. [More detail at Wikipedia](https://en.wikipedia.org/wiki/Cross_entropy)\n", | |
| "\n", | |
| "The optimizer helps determine how quickly the model learns through **gradient descent**. The rate at which descends a gradient is called the **learning rate**." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "i416OMQ9SZW1" | |
| }, | |
| "source": [ | |
| "<img src = \"gradient_descent.png\" >" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "CWMED60kSZW2" | |
| }, | |
| "source": [ | |
| "<img src = \"learning_rate.png\" >" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "2RYA0SvnSZW2" | |
| }, | |
| "source": [ | |
| "So are smaller learning rates better? Not quite! It's important for an optimizer not to get stuck in local minima while neglecting the global minimum of the loss function. Sometimes that means trying a larger learning rate to jump out of a local minimum." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "Az2yjptpSZW2" | |
| }, | |
| "source": [ | |
| "<img src = 'complicated_loss_function.png' >" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "t0HI0iLzSZW2" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "# Let's use the Adam optimizer for learning\n", | |
| "model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "s8ZL4XflSZW2" | |
| }, | |
| "source": [ | |
| "## Train the model!\n", | |
| "This is the fun part! " | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "0HclES3qSZW2" | |
| }, | |
| "source": [ | |
| "The batch size determines over how much data per step is used to compute the loss function, gradients, and back propagation. Large batch sizes allow the network to complete it's training faster; however, there are other factors beyond training speed to consider.\n", | |
| "\n", | |
| "Too large of a batch size smoothes the local minima of the loss function, causing the optimizer to settle in one because it thinks it found the global minimum.\n", | |
| "\n", | |
| "Too small of a batch size creates a very noisy loss function, and the optimizer may never find the global minimum.\n", | |
| "\n", | |
| "So a good batch size may take some trial and error to find!" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "BIupkxW8SZW3", | |
| "outputId": "42ee7d64-4599-490e-b8a0-ce37308d2792", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "Epoch 1/5\n", | |
| "469/469 [==============================] - 10s 20ms/step - loss: 0.2475 - accuracy: 0.9254\n", | |
| "Epoch 2/5\n", | |
| "469/469 [==============================] - 8s 17ms/step - loss: 0.1014 - accuracy: 0.9681\n", | |
| "Epoch 3/5\n", | |
| "469/469 [==============================] - 8s 17ms/step - loss: 0.0709 - accuracy: 0.9771\n", | |
| "Epoch 4/5\n", | |
| "469/469 [==============================] - 8s 16ms/step - loss: 0.0544 - accuracy: 0.9827\n", | |
| "Epoch 5/5\n", | |
| "469/469 [==============================] - 8s 17ms/step - loss: 0.0475 - accuracy: 0.9848\n" | |
| ] | |
| }, | |
| { | |
| "output_type": "execute_result", | |
| "data": { | |
| "text/plain": [ | |
| "<keras.callbacks.History at 0x7f4b50afb8d0>" | |
| ] | |
| }, | |
| "metadata": {}, | |
| "execution_count": 40 | |
| } | |
| ], | |
| "source": [ | |
| "model.fit(#fill here, #fill here,\n", | |
| " batch_size=128, epochs=5,\n", | |
| " verbose=1)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "PDPsFYzgSZW3" | |
| }, | |
| "source": [ | |
| "The two numbers, in order, represent the value of the loss function of the network on the training set, and the overall accuracy of the network on the training data. But how does it do on data it did not train on?" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "e0ZXqmFFSZW3" | |
| }, | |
| "source": [ | |
| "## Evaluate Model's Accuracy on Test Data" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "lucBi0IUSZW3", | |
| "outputId": "748bfa3e-f2a2-4965-f075-73e4657ffbb7", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "313/313 [==============================] - 1s 3ms/step - loss: 0.0792 - accuracy: 0.9772\n", | |
| "Test score: 0.07915858179330826\n", | |
| "Test accuracy: 0.9771999716758728\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "score = #fill here\n", | |
| "print('Test score:', score[0])\n", | |
| "print('Test accuracy:', score[1])" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "S4ZS7ZDsSZW3" | |
| }, | |
| "source": [ | |
| "### Inspecting the output\n", | |
| "\n", | |
| "It's always a good idea to inspect the output and make sure everything looks sane. Here we'll look at some examples it gets right, and some examples it gets wrong." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "collapsed": true, | |
| "id": "S5JPQ0aASZW3", | |
| "outputId": "e767e41d-8778-4899-e7b3-885d72bd216a", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stderr", | |
| "text": [ | |
| "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:6: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.\n", | |
| " \n", | |
| "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:8: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.\n", | |
| " \n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "# The predict_classes function outputs the highest probability class\n", | |
| "# according to the trained classifier for each input example.\n", | |
| "predicted_classes = model.predict(X_test)\n", | |
| "\n", | |
| "# Check which items we got right / wrong\n", | |
| "correct_indices = np.nonzero(predicted_classes == y_test)[0]\n", | |
| "\n", | |
| "incorrect_indices = np.nonzero(predicted_classes != y_test)[0]" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "Qc3mT5aqSZW4", | |
| "outputId": "09da71e2-e00f-40b9-ace3-37d4fe069a96", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 102 | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/plain": [ | |
| "<Figure size 648x648 with 0 Axes>" | |
| ] | |
| }, | |
| "metadata": {} | |
| }, | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/plain": [ | |
| "<Figure size 648x648 with 1 Axes>" | |
| ], | |
| "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAABECAYAAACrpM6NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2debgdRbW3318SSAJhCkSGAIkiEIaLARUURJBJIKKiiAxickERkQuKV69wHzAKOCDqFbxMCkQEBAecQEC4gkzKjMEwRIRAgAQhEDIAMtX3x1qbU6dPd+/e+xzI8ct6n+c8Z3dXdVX16qrqVauqeimlRBAEQRAEwWBiyJIuQBAEQRAEQZFQUIIgCIIgGHSEghIEQRAEwaAjFJQgCIIgCAYdoaAEQRAEQTDoCAUlCIIgCIJBRygoQRAEQRAMOpaogiJpmqTj/fe2ku57nfJNkt5cETZL0nOSfvx6lCUIgiAIgr60VVCyF/YiSY+7UjFqoAuSUroupbRhg/JMkXT9QOdfYI+U0gGFfI+Q9KCkxZLukbRBTRm3kHRtJrMjsrDjJN0l6SVJUwvXHe3XtP6ek/SKpNWyODtJut3L8YikvbOwoZKOl/SYpIWS7pC0sodNlnSbpAV+3YmShpWUfX1Jz0s6ryvJWRqS9E1J8/zvm5KUhe8h6a9+jzdK2rgmrc9JesDL/Zik75aVO4u/o6R7JT0r6WpJ47KwaZJeKMh4aBa+tz/bhZLulvTBLGxTSVdIelJSn68bFtJcJOllSad42MaSbpX0tP9dld+zpPd4WZ+RNKuhmCtpI4PRki7y5/KkpPMlrViTVifPam+P86yka0rC6+rncH+2j7mMTpW0TKHcv/R6/5Ck/bKwSZKulzRf0lxJP5S0QhY+o/BsXpL02yz8TEn3ydralGZSrpTBcElne32dK+nILGz/QjmelQ2W3tomzdGSnlBNv9dAfhtJ+oPXsfsl7Vm4vrLOeHhdv7ODhy2QtdWDC9f+h6zvXODt4F2Fcp8u6yefkvRbSWPr5NFGVhNl/dyz/n9iFjZV0ouFZ/CminTG+7PJ4x5TEXdd9W3/SdLnszhjJF3g8n9a0vmFNLrt16fI+po87+2za/v1vumENnmtKek3fg9J0vhGiaaUav+AWcBO/nss8FfgGyXxhrVLq+SaacDxHV4zBbi+07wKaSTgze3uNzv3CWA6sDEgYD1gdMX1qwH/APYHhgMrABtl4ZOB3YBfA1PblHMq8IfseGNPezdgGLAqsF4WfjzwB2Ccl3NTYISHfRrYFljWn+NtwJdK8vw9cB1wXj/k+yngPmBtz+tu4BAPWx9YALzL7+Eo4P6q+uOyXtl/j/b7O7JG9s8AHwFGAN8C/tykvnk5X3DZCpgEPAu8wcM3BA4CPmDNpvb+RwGLgHf78crAeE93KHA4MD2LvyVwAHAwMKufdbudDE71Z7wisBJwFfCdirQ6fVY7AXsDxwLXlITX1c8ve70bDYwB/gx8Jbv2J8BFLtt3+T1u4mH7AbsCywGrAJcBp1eUUcCDwMezc58BdgRuBab0U/5f9/tYBdgImAvsWhF3CvB3QG3S/AFwLTX9Xp38/NnNBI70+rcDsBjYoGGdqex3gGX82k+5bN+O1f23ePhWntdbPfzTwBPAUA//IvAXYHXP+1zg4i5lvyzwEPA5rO893I+X9fCpNOzXsPaaqup6m2vfCLwMjM/OXQd8B2tzywCbN5Fvg3YzpU296Pp908V9V+blz/dQ4J0u1/GN0myQ6SyyF7ZX3kv8d8Ia99+AB/3c+4A7gfnAjcBm2bWbA7cDC7HO5kL8hQFsDzySxV0HuNgr8zzg+1iDf94f/iJgvscdDpwEPAw8DpwOjMzS+gIwB3gMOJAOFBTMyjQb2LHhQ/oa8OMG8c6rqzBeER8AJmfnLgCOq4i/istkvYblPBL4beHcPsBPKWnILrd7gKeBK4BxNWnfCBycHR+Ed3jAYcClBfk+10S+WMO9Cji1Ivxg4MbseHlPe4IfT6NaQdkK+Efh3BPAOwvn3kx7BWWyP7s+Lx6sA/oM8GxJ2E6UKCjABOBK4ClM8du7Ju92MrgMODQL/wxwRUVaXT0rTKG/ppP6iSkHH8mO9wNmZ/fwAv5C9XM/pmSg5GEfAu6qCNsO63+WLwm7noKC4vf8JUyRmOfto3Rw4vEfA3bJjo8DLqyIezXw5Tay3Br4E/Dv1L+I6uS3qcteWfjv8b6kQZ2p63dWx/rT5bJztwD7+u+PAjcX0k7Amn58GnBiFj4JuC87Xgv4BdYWHwQOr5HBLsCjhft8GFcQef0UlC8DVxfKNQtXykrid92v03DATnfvm5WAs7B356OYolR6D03zwvq/xgpKR2tQJK0D7A7ckZ3+INa5byxpc+BsTJteFTgD+I2b8ZYFfoV1LKOBnwEfrshnKHAJpv2Ox0a3F6aU7gEOAf6UUhqVUlrZL/kGsAEwEXuBjMVGcUjaFfhPYGdsRLhTJ/eMWQHWBjaVNNtNlV+RVCW7dwBPyUzd/3CT5bod5glm7XgD1jjztHEz2hxJ50ka7WH/BrwE7CUzLc+U9Jma9N8NzGgdyMz8X8UUl15I+gBwNNbxj8FGAz+pSXsTbFTU4i9+7tUkC79bo4JSJO0naQHwJPAWrF61zTeltBh7seR5H+qm5Nsk5fXvVuAeSe93k+oHgX9ilrNOmQycm7xFZvcxH1OwT8EU2bZIWh5TTi7A6sM+wKmqnmppJ4P/Bd4naRVJq2Bt8LK6IhR+1z6rGprUz2Jea0taCWvbL6WUZmbhxTqV06tuF5gM/MLl0oT/wPq47bCX5dOYDPvg8lyT+rrfijvOy3luVcbeD34fUxRTVbz8ksLvlvyq4raeY7s6U9nvpJQex/qCf/d2805spN+ajroMGCppK7+fA7EB7FwPPwvYRtJakpbDLM+XeX5DgN962cZiVq7PSnpvxT1tglkmc1lNp7f89/D2P0PSpyvSyXnIp1zOaTL1IUnAx4EfZaffgQ0sfiSbWr1F0naF8P7065vLpmtnSjpGNVPgNZS9b6Z53m/GjAu7YIOP148G2tAs3FqBKQyn4tYJrNHskMU9jYImiD2Y7bDG+Bi9tdsbKbGgYGagJyjRXilojFhDW0xvk9g76bHonE020sI6u04sKFt7/EvpMdXPBD5Zcf1Ml9XbMZPlycANXWi0ZwHTCude8PJtgJm6fwGc72H7eTnPAkYCm7kMdy5J+0DgEWC17Nz3gP/y31PJRhpYh3FQdjwEm/4YV1H2l/HRlx+v72UTZg1Y7M97WeAY4BXgqAZ1cX1sRLpGjcy+UTh3Az4qBrbAFOdhmKK9ENgmi3sQVtdf8vubVJJHrQUF65xfBt5YEb48ZuosS7uPBQUbgV5XOHcGFSPvBjJYC7NCveJ/V+Im8JK0unpWlFtQausnNjq7AVOA1wBu8vhrYp3n3EJ6nyzm4ed3xpSIDUrClsOmrLavKHeZBeUeMouRl+dFyvumdbzMIwrlmVUS95iy8hfifA44zX9Pod6CUie/ZbDR8Rf99y5YX3JFwzpT2e94+B6Y5fol//tkFiZscPOihz0JvD0LXwmzpCcPvwO3UGED34cL5ToKOKdCBsdQsFYB5+P9LDaVshY2zbU1ZhnYtyKtUcDbsL5ideDnVFgaC9dti/Uho7JzZ/r9HeTy3wd7R6zWTr60bzdvwqaUhmDKzN2UtE86fN/4Pf+T3jMR+5JZhmrSet0tKB9MKa2cUhqXUjo0pfRcFjY7+z0O+Lxssdp8HzGu45ViLeDR5KV0HqrIbx3goZTSSw3KNgbreG7L8rzcz+P55mWsyrOK1r2emFKan1Kahb0gdq+J/8uU0i0ppeeBrwBb14xk+uAjiY/QWwtvpX1OSmlmSmkRNgrfPQsD+GpK6bmU0nSs4fcqp1sGvg7sllJ60s9NxF6O360o0jjge5l8n8I6nrGFhVane/xF2BqHFisCi5JxLzaK/T7WQayGNapH2sklpfQ3bGR8akWUYr6tvBf69benlOallF5KKf0O67w+5DLYCTiRnpfxdsAPlS2ya8gB2IvkwYp7WIxNQZ4r6Q0N0hsHbFVoU/sDa6iwOM/j18oAm6KYia2NWhEbKZcuiO7PsyqhXf08AXs53YkNXH6FvdQeb3BPAEh6B2Zp2iv1tra0+BBWd//YQbnHAb/MZH8PpoCuLlvc2ZL/0V7OVtkqy+kUR9m9kLQWtobivxuWs1J+KaUXMSvQJMxy8XmsHrSeYzv5VvY7kiZgz/HjWLvZBPiipEl+7UHY9NQmHv4x4BK/PzBr1HBs4LA8Nq3fsuiNA9Yq1P2jsZdncWH6uu3uI6V0d0rpsZTSyymlG7FB2V5lwkwpLUop3ep9xeOYFWsXZYuvK2hZ6BZl557DlNSzUkovppQuxN5J27STL23aTUrpgZTSgymlV1JKd2FW8NJ7qqLifTMOU6bmZLI/A7OyFBeeb9tJfp3QjSmoSK5wzAZOSCmdUIzkJq2xkpQpKetiHWSR2cC6koaVKClFU+eT2EPcJKX0aElaczCFp0Wn0y33YRpunm+duXV6B3Gr2BPrSK/pIO3pJeeKUwy7YgvuJnllbrE9Zhl62CyUjMLMshunlLag57n2Wnnu3Ejf6YoZ2FTMzX78FjKTe0rp59iIBNlq9IOweesmDMMWzpYxA+sg8LSX97hV5v6WVQdsevDalNKtfnyLpJswxe3OhmUD66i/0SbOEEypHostjqtjNvDHlNLOFeHFHXXtZDAR+IwrSrhSWbk7pJ/PKqe2fvqg5zD/Q7YT5LaU0iuSZgLDJK3vSioU6pRsevk3wIEppf+rKMNkSqbe2jDb07yhJOwQ/3sVSXO8bFeWldPjbIMNnH5ek++WmPXjbm+TI4GRkuYCY1NKL+eR6+Tn4dMxpbtVhhvpeSG1qzN1/c6mwMyU0hV+fJ+kS7HFkpdi9e2STGG83GW0td//ROC/U0pPed6nAF/16ZTZmCV8/TIBpZR61X1JM7ABcv6O2YyKKTl6t/92tNKrHNRLGom96PcsBE3HrExl6bXCu+7XS9Jtek8tyt43szELymplhoKUUtX06sDSwFwzi8Kuliys11QJZhKbjZnmhGnEk7DR2rLYgqUjMM3sQ5iGXzbFMxSbdzzJ0xiBm+Kx1fqzyMzSmCb8U3p2XIwF3uu/d8NGDRtjL4XziuVud7/YPPElfh9rA/eSTXkU4u6AmZgn+n1+l8xE7+dGYCO94/330EIav8c05mLaB2ILxd7k9/JTsgW52Er/M7ARyUbYy2/HrFzz8J0lhXSXw8zCrb+TsM5jjIfvie3eau2aWIlsQV5JeodgI82xWEc8A9/F4+Fv9Wc8xu/hgpq0PpE91409rapdJ2OwHQUfdrl+k967EfbCXuhDMDP3Qtzcj3XeTwIT/Xhzl9cufixPc2OvPyOA4YX8t8amRFYonN/Z0xuKjehOxqY7Wyvxh3h6u2EWvhH07DxYwc8d4HVnGWz6cKMuZXA1tgZmpP+dSrZAsiS9Tp7VUM/zEK+LI4BlGtbPVl0RNic/m96LTS/E1josj4088108m2KWlo/WlG1tbAqhz2JDrG8agU1rfNJ/D/Gwz2Ed97hMvh+oyecbmIVmFWyKbA6FXTyYyf/cqjQ8znB6t8kjsGmbqunNdvLbzO9rOWxN3oN4/W1QZyr7HUyRWYT1L60djvfji+QxxWemXyusLTxLzwLcc7ApjdbulqMxS3urPt0O/BdWV4f6s357hQxau3iOcPkdRu9dPB/w5yJMAXyUbFFoIa2tsJ17QzDrzkW0md7ApmNmUVgcj625fNplMRTrh56iZ4qnP/36bsDq/nsC1k9/Obu2P++bX2Pv1hVdDusB29Xcf21eftxaJL0h2VRoZZptI3SgoPi5XbER1nyscf4M77AxBeYOenbxXET1Lp51MTPlPOzFcXJWCS/1B/xkduNfw+ZZF2Avx8OztL6EKSkd7+LxcytiHeRCrOEf26qE+JxjIf6nscr/NLbIa50sbJrnn/9NKXQ0L9WU7yvYHOQT2ILjVQrXXo51GA8An8rCrvZ0F2V/l1XkMZW+u3gOAO5y+c4Gzq6pM8KmS57yvxOh19qj612WT2ENr8+OiizuOdjLZ7E/m2/Re45/BrB/drwTpkA+h71Yxmdh12Ed8QJMAd6nkNdhWOe60OX3+SxsfMlzm1W4/gxKdnBho6p7XeZPYPU33922fUna12ThG/o1rR1tf8AVqQqZ1cngjV4n57n8LwfWr0mr8llhU00zsuMpJfcxrWH9fLc/32cxq+X+hXKMxvqDxdhAZ79CHXmF3nV7RuH6oyis5cnCrikp9/YeNgRbOH6fy+HvwNdq5DUcW/e2AKu3RxbCR2B9Y6NdgQXZ5mvvWlMa6zaU37ew/mgRNoVS7Lcr60yDfmdv7MW4EJs2+iY9Cp6waYeHPfwe4IDs2lWxqdZ/uFyuB7bMwtfCFNO5Xv4/U/E+8vibY59QeA5TbvLtvD/B6v0iv9e6HUH7YkrDYuw9di6ZcohN055euOYKqnfjbIv1n4uwBfnbdiDfunZzEj394wMu63xQMI0u3zeY0niaP9NnsHf3PjUya5dXMSy1q/etl2zgyL5muya2jmTyki5PEARBECyNhIISBEEQBMGgI5wFBkEQBEEw6AgFJQiCIAiCQUcoKEEQBEEwyJA5EFwkc+DX6RfQq9KcJun4gUjr9aBfCopqvBeWxP2CzCvqQtnn4r+QhdV6g1R7b6WVXmolLSvp5zKvzEmZp0cPf49qPMl62BMyT5x/kX32vVt5SfVefmu9ghbSqvWuW4i7gaRf+3085ddtmIV37elYDb1UqsIjqwbQ824bGfSrHpSk14mX37Eu/6dcvsVvZ1SmpTZearN4Zc+ma+/YkrZVeZssdU/RQF51Xn5rn02n8izEbeSxtax+SnqHpCs9nyck/UzSmll4ozojaTu/r+Ozc5VtzmV1lsxr80JJd0rarU4m7ZB5BZ/r+Z0taXgWtrWkmz2v6cq8DZek07jfKVz3cZfBJ7Jztd6FPf7iLOyHWdhlhetekHRXFl77bpK5z3jI0/+Vej4tj+xT83NcVjPzMndDnew9/AjZO3GxzJv6BgAppauSfevl4Q7ykqTDZf1Jq03/TNK/9eceumFA+pBOtrqVbCuaTHNPiV/EPjU+DNs2+RAVW5YoeIOkjbdS6p3ALQt8FvOAOofCZ65p40kW+37AMP+9FbZVbs0u5VXn5bfWK2hJWp14193S4472fI4D7s3Cu/Z0TEMvlZR4ZGUAPe82kH2/6kEhbqdefq8G/sdl/xZsy+57mqRFGy+/dc+mJM5UOvCOXbh2eyqc7DWUf6WX33bPphN5NihHLxm0qZ+7ed1cEet7zgYu76TOeBnv9Od2fHa+ss1h34qYim1tH4I5YF1Iw8+Dl5ThvdhW1E1c/tfgn7b3ejXP73Mo9qXXp8m2uRbSatzvZNesgm3r/SvwicKzqKuviYrPLZTEvQY4NjuufDe5HBZiW7NHYd/tuLAQ3vpGzASvq28daNl7+Cewj7FtTM93ZEYX0phFzdbqQtyTsa3wO2Bb3lv+jVp1axoV78rX+o8u+pCByrj2O/81gjylIuzL1HwUh4K30qZCx/Zzb18RVupJthBnS8zZW75Pf6C8/NZ6Ba1Js6133ZJrRnteq1aEd+Tp2MMrfSxQ4ZGVfnrepQMvvwNcDxp7+cU6wIR/9M7PnUnPh65q06LGS23TZ+NxOvKOXXL9OWQ+UHiNvPzWPZsm8mxzD31kUFc/S67fAljYSZ1xGZ1Imz6KkjZXCJ8OfDg7rvQaX3LtBWTfbsGc7s3N0il+M2YmFR+izOI07newb4Ycir2cB1xBwRS5Vwe0hbA+7ybsm1kXZMfrYV8LX6Hk+g0xpXnv7NxAyX4I9k2pdt7BZ9FAQcEGOy+TvZ9K4rxaDzGF6RLs2ytP+++1s7hTvL0sxL4Js3/27P+IDS6fBC5qWA969SFN/pbIGhRJwkYPfT5B7mG1fioo91Za5aW230i6RNLz2Jccr8FeGgPq5Te19wo6kLwbayTzasIbeTpuh+o9snbteVede/kdaJp6+VXhPyVx26VV6aW2g2fTqXfsngxN1nvRu02+Jl5+G9BEnlX0kUGb+lmkzkty34LadOWB2PNpR2XaklbHHMnN8ONKr/EVaZf1O6tLWrWVRTFLuvNYXVb2LbEPdJ5eEaWdd+FrfXrkYlVMH2Pvi+uS+UlrQrHf+TumoGyQlftUSc9ilp85wO/8/EDKfm3/21TSbJ/m+YrMi3M37Ih97PTmtjGNIZjSMA778N9zWFtotfmTMZ9tK2BKfMvlx3GYxXYVL/8p7TKq6EMaFXBJMJUe4RR5Fz3eI/sgaWfMfHdsdvpkTHt8A+bRcprM38WAkFJ6H/bJ8d2B3yf3b4F90vvrKaV7kvkr+BowUdk6igKjMK2zxTPAKFfKwBSUYzEfCNdhPipmM4BIWht7kZS+0CQdiHUoJ2WnjwPOSil14yTucOCmlNJtJWFFeeDHrfVFt2Mm8Hn+9zI9jgLfh41az0nm0OsO7MXzkS7K2ClXAdtJ2l7SspiSuixmTu1FSmkh9gn1YySNkLQFpmgt1zCty4EjJI2RtAYmT7Lwps9mMvDz1NuJ2drYFMWHsfYzkvLO5kPYSCl3sncIVj8fSSn9E2vTe6nc1XvLZ0qx7rdzvNaHBvKso0wGdfXzVSRthrXNL9TFK3AycEwhv7K0y9pcK2wZ7EurP0rmvBHM8nhGSummZI7vfoT1Ge+oyKKs3wGT/58wh3z7SlpG0mTMotBEnrW48ncqcFjWZ+b8FJvuG4O5GDhW0r5Z+HaYdWQCZoG7pKJ+fRyzDDSlXb9DSulQP94Wc2D4Tw8aSNmv7b93wTwRvwf7gu1BHdxLzqqYMtWIZI5Tf5FSetbb1Qlk/pqwrzNvKmlkSmlOSqmlQL+IO3JMKT2fUmoyiC7rQ9ryuisokg7DKtQk79iKTKavN8jWtaXeSlONl9qBIpkXysswj5bv99PjGCAvv2rvFbTfSBqDab6nppT6WHrUnafjuvzaeWTtj+fdcXTm5XfASJ17+d0fW1c1G/t09HmtuA3SqvRS2/TZqDvv2DmT6etkbxyvjZffJlTKs4oyGTSon614b8Ysd0eklK5rUkBJe2BTBhe1idenzWVhQ7DPnr+AOwF0xlHhNV7S/pnsW16By/odsOmqedh6kiOxtRK7YkpzN4ORIocC01NKfy4LTG28C6eUrk0pvZBSmo/513kjptC8imxB7xrUO14s0sg7tpfrekyRaFl3Bkz29HgqPjGlNN8tQGdQ3gabMA+zVDZC0nKSzpAtFl6ArcFaWdJQt2Z/FBuIzJF0qb+jwNaTCrjZLV8HNsiurA9pTyfzQTVzS43WoGDmzkeAN1WEj8Q0zB1KwjbHFvTt0SCf0yhZTEk/1x54vKuAz/nvKyj4vGhz7Y3AJwvyaK1B2Qu4oxD/f4Dvt0mz0VwwZo67g2yBViF8V2wucsvC+c9ifh7m+t8i3M9FIV6fNSjYFMDz2bXPYJ3tXGxB3sHADVn85entSKzXImHMAeMi/70vcGWX9bXf9aBwzcpe1gkN41+AWd46Tstl9qcOn83+lDsxu47eCwu3AJ4uxFmHEid72JqfbTqQ0WPAztnxV+liDUqn8qyTQbv66XHG+XWH1KTdp85gbXdBlvZz/nx+3a7NeZgwC/PVwMhC2BmY9aoT+ZyQHe+Ar4MoiTsM2zXy3jZptu13MGX66UwGL7iMS/s0zCngxRVhQ11+mxXO/4Aax4tUr0E5Pzt+ExVrUDz8h8D3Blr2mJXqn2QOXDFF8ZeFNGbR2RqUt9XEmUbPGpRjsCULa/jxRKwPH1a4ZiTwbUp8WWEzHs9Ts1aIij6kkfw6vaCQcVtPiVnc/b2Slnph9ThV3iBrvZVS46XWw4d72R7x8BGtPKj3JDvBz4/0e/2YV+QtPHzAvPzSxitoSVptvetmcVcEbqa6Y+ja07HHKfVSSRuPrPTD8y4devntTz2oSKsTL78b0ePR+2OYqXNMk7So8VLb5Nl4vK68Y3uco4FrS64dUC+/dc+mU3lWXNNHBg3q51jMavefFWm280Kdp30RZuka3a7Nefjp2M6fUSVhlV7jK9LalR6P7itjzibznSSbY+1nRUyxuqEsnS76nZULMrgRewGv5OGV3oUxC/JErF2M8nLdR29HeHUD2sp3k6e9AJu+WR5TYi70sNZ6tlGe93uxQcD7XyPZn4stTm1N+dxLYYEymYKCLVydVfN8TgH+hu2YaXnp3oeSXTzY4u3LPM5o4Jf+TIdhyyw+4Pc3BHNm+Ee/7iP4YlqX5XNUGB3q+pAmfx1fUMh4GhXeCyl4+cU6whfp7XG0kTdI2ngrpb2X2lkl5RzvYduXhF2TejrCmzCFZz62q2bPQtoD6eW30itoSVrjS8o9Kwu/DDjaf0/28MUFGbY8oV5N/zwdF8uRKq6dQmGXBP3wvEvnXn67qgcVaXXi5fezXsbFft3bOkir1kttg2fTtXdsD+/TYfr5gfbyW/lsStJqJ89FZN5i28mgqn5iuwkTvdtF3qc1rjMUdvFQ0+Ywq03CRqZ5eO61u9JrfEX+rSmcBVh/OjwL+wnWfz6DKVJvqElnfMk9l/Y7JddeQ+9dPJXehTEF7j5/xv/ArDHrF9LbF1MM+yiytPesux9mKVqMbUVuKY5jMEV6vsvqLjKr92sg+xWxqf2F2Pvj2OL90FtBOYbM+lOSlzBFewbWZzzqz3STTC4tBWUtfyaLsKn0T9GjoKxJz06d+R5vY7/uRE93EdbuSwfR7fqQJn/hLDAIgiAIBhmSdsQW/g8Hdk8pXS3p99haqHuWbOleH0JBCYIgCIJg0BG+eIIg6IWkXSXdJ+l+SV9a0uUJgmDpJCwoQRC8in+7YiawM7YOqvU147uXaMGCIFjqCAtKEAQ5WwL3p5QeSCm9gC3g69pBZhAEQbeEghIEQc5YbDdBi0f8XCUyD6Wlf69pSYMg+P+ass8GB0EQ1CLpYOyjcUEQBK8JYUEJgiDnUezLjy3W9nO9SCmdmVJ6W0rpbfn5HXfckTFjxrzGRQyCYGkgFiZhIqYAAALbSURBVMkGwVKIpHWwr1iujn2c6cyU0vckfRX78uNM7MOKK2DuJSq9+Bancl555RWGDLGxT0qp6Ck3CIKgETHFEwRLJy8Bn08p3S5pBeA2SVdiX2yehn3BdjjmKblSOSkyadKkV5WTIAiC/hA9SRAshSRzn367/15Ij48ogHtTShuklNZLKZ3QSboTJkxoHykIgqABoaAEwVKOpPGYw7ib/NRhkqZLOlvSKhXXHCzpVkm3ts4dddRRfPvb337NyxsEwdJBrEEJgqUYSaMwp2AnpJQulrQ65h04AccBa6aUDmyTxkLMsVuRDVNKKwx0mYMgWDqINShBsJQiaRnMGdn5KaWLAVJKj2fhP8BcwbfjvuJuHr/+1rLIQRAETYgpniBYCpEk4CzgnpTSd7Lza2bR9gT++nqXLQiCAMKCEgRLK9sABwB3SbrTzx0N7CtpIjbFMwv41JIpXhAESzuhoATBUkhK6Xqg7Bslv+siuTM7PB8EQdCWWCQbBEEQBMGgI9agBEEQBEEw6AgFJQiCIAiCQUcoKEEQdI2kXSX9XdKzkh6XNEPSER42VdKjku70v92XdHmDIPjXIdagBEHQFZKGYk4F98N2/fwAOAi4APggsDewKKV00hIrZBAE/7LELp4gCLplS+D+lNJNAJIuBHamt1+fIAiCrogpniAIumUsMDs7fgSYQId+fYIgCMoIBSUIgoFiOLAb8NmU0gLgNGA9YCIwBwhPgkEQNCYUlCAIuuVRYB141a/PF4Fbcr8+KaWXU0qvYOtTtlxiJQ2C4F+OUFCCIOiWW4D1Jb0ROAdYBVNSgPDrEwRB/4hdPEEQdI1vHT4ds6Q8Dsz1oKOBfbHpnVf9+qSU5iyBYgZB8C9IKChBEARBEAw6YoonCIIgCIJBRygoQRAEQRAMOkJBCYIgCIJg0BEKShAEQRAEg45QUIIgCIIgGHSEghIEQRAEwaAjFJQgCIIgCAYd/w/WXwOGPv0YIgAAAABJRU5ErkJggg==\n" | |
| }, | |
| "metadata": { | |
| "needs_background": "light" | |
| } | |
| } | |
| ], | |
| "source": [ | |
| "plt.figure()\n", | |
| "for i, correct in enumerate(correct_indices[:9]):\n", | |
| " plt.subplot(3,3,i+1)\n", | |
| " plt.imshow(X_test[correct].reshape(28,28), cmap='gray', interpolation='none')\n", | |
| " plt.title(\"Predicted {}, Class {}\".format(predicted_classes[correct], y_test[correct]))\n", | |
| " \n", | |
| "plt.tight_layout()\n", | |
| " \n", | |
| "plt.figure()\n", | |
| "for i, incorrect in enumerate(incorrect_indices[:9]):\n", | |
| " plt.subplot(3,3,i+1)\n", | |
| " plt.imshow(X_test[incorrect].reshape(28,28), cmap='gray', interpolation='none')\n", | |
| " plt.title(\"Predicted {}, Class {}\".format(predicted_classes[incorrect], y_test[incorrect]))\n", | |
| " \n", | |
| "plt.tight_layout()" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "W2Sl18ZMSZW4" | |
| }, | |
| "source": [ | |
| "# Trying experimenting with the batch size!\n", | |
| "\n", | |
| "#### How does increasing the batch size to 10,000 affect the training time and test accuracy?\n", | |
| "\n", | |
| "#### How about a batch size of 32?" | |
| ] | |
| } | |
| ], | |
| "metadata": { | |
| "kernelspec": { | |
| "display_name": "Tensorflow (GPU)", | |
| "language": "python", | |
| "name": "py3.6-tfgpu" | |
| }, | |
| "language_info": { | |
| "codemirror_mode": { | |
| "name": "ipython", | |
| "version": 3 | |
| }, | |
| "file_extension": ".py", | |
| "mimetype": "text/x-python", | |
| "name": "python", | |
| "nbconvert_exporter": "python", | |
| "pygments_lexer": "ipython3", | |
| "version": "3.6.6" | |
| }, | |
| "colab": { | |
| "name": "MNIST in Keras.ipynb", | |
| "provenance": [], | |
| "include_colab_link": true | |
| } | |
| }, | |
| "nbformat": 4, | |
| "nbformat_minor": 0 | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment