1-mnist-with-keras.ipynb

1-mnist-with-keras.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "1-mnist-with-keras.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "[View in Colaboratory](https://colab.research.google.com/gist/simecek/f0f2f38f3c283f430fb088c85ce29d6b/1-mnist-with-keras.ipynb)"
      ]
    },
    {
      "metadata": {
        "id": "IZrAitlFLdEZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# MNIST with tf.keras\n",
        "\n",
        "Welcome! In this lab, you'll learn how to train an image classifier train on the [MNIST dataset](http://yann.lecun.com/exdb/mnist/) - the \"hello world\" of computer vision. You'll go through all the steps, including loading the data, building and training a model, calculating the accuracy, and making predictions. Our focus here is on the code. For more on any of the concepts below, see [https://ai.google/education](https://ai.google/education)."
      ]
    },
    {
      "metadata": {
        "id": "jSmUsjJfMEqC",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "!pip install -q -U tensorflow==1.8.0\n",
        "import tensorflow as tf\n",
        "\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "B8Lhscw0NDln",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 1: Download the dataset\n",
        "\n",
        "The MNIST dataset contains thousands of grayscale images of handwritten digits."
      ]
    },
    {
      "metadata": {
        "id": "FKiwTuT-NE6f",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.mnist.load_data()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "eEFU58MaNPpk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 2) Visualize the data\n",
        "Let's see how the images look. This function shows a random example along with it's corresponding label."
      ]
    },
    {
      "metadata": {
        "id": "AwxNOsCMNNGd",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import random\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "i = random.randint(0, 100)\n",
        "\n",
        "print(\"Label: %s\" % train_labels[i])\n",
        "plt.imshow(train_images[i])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "e2n2NVdKNk5i",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 3) Understand the data format\n",
        "\n",
        "We are given the images as a 3-D array of integer values that is of shape (*N*, 28, 28), where *N* is the number of images in the training or test set. The labels are 1-D array of the integer values of each image."
      ]
    },
    {
      "metadata": {
        "id": "TTj2ZWMBN24i",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "print(train_images.shape)\n",
        "print(train_labels.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Eo_cZXaqODnZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 4) Reformat the images\n",
        "Here, we'll flatten (or unstack) the images. There are deep learning techniques that work with 2d images directly (rather than their flattened representation), but we'll start with this format. Instead of working with a 28 by 28 *image*, we'll unstack it into a 28 \\* 28 = 784 length *array*.\n",
        "\n",
        "* We want to convert the 3-D array of shape (*N*, 28, 28) to a 2-D array of shape (*N*, 784) where the second dimension is just an array of all the pixels in an image. This is called flattening, or unstacking, the images. \n",
        "* We also want to convert the pixel values from a number between 0 and 255 to a number between 0 and 1."
      ]
    },
    {
      "metadata": {
        "id": "OgnV5FJjP5Vz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "TRAINING_SIZE = len(train_images)\n",
        "TEST_SIZE = len(test_images)\n",
        "\n",
        "# Reshape from (N, 28, 28) to (N, 784)\n",
        "train_images = np.reshape(train_images, (TRAINING_SIZE, 784))\n",
        "test_images = np.reshape(test_images, (TEST_SIZE, 784))\n",
        "\n",
        "# Convert the array to float32 as opposed to uint8\n",
        "train_images = train_images.astype(np.float32)\n",
        "test_images = test_images.astype(np.float32)\n",
        "\n",
        "# Convert the pixel values from integers between 0 and 255 to floats between 0 and 1\n",
        "train_images /= 255\n",
        "test_images /=  255"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "GI25z0StQH-P",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 5) Reformat the labels\n",
        "\n",
        "Next, we want to convert the labels from an integer format (e.g., \"2\"), to a [one hot encoding](https://en.wikipedia.org/wiki/One-hot) (e.g., \"0, 0, 1, 0, 0, 0, 0, 0, 0, 0\"). To do so, we'll use the `tf.keras.utils.to_categorical` [function](https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical) function."
      ]
    },
    {
      "metadata": {
        "id": "E9yrkEENQ9Vz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "NUM_DIGITS = 10\n",
        "\n",
        "print(\"Before\", train_labels[0]) # The format of the labels before conversion\n",
        "\n",
        "train_labels  = tf.keras.utils.to_categorical(train_labels, NUM_DIGITS)\n",
        "\n",
        "print(\"After\", train_labels[0]) # The format of the labels after conversion\n",
        "\n",
        "test_labels = tf.keras.utils.to_categorical(test_labels, NUM_DIGITS)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "pjdbemHURkpv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 6) Build the model\n",
        "\n",
        "Now, we'll create our neural network using the [Keras Sequential API](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential). \n",
        "* Architecture wise, we'll single layer network. \n",
        "* The hidden layer will have 512 units using the [ReLU](https://www.tensorflow.org/api_docs/python/tf/keras/activations/relu) activation function. \n",
        "* The output layer will have 10 units and use [softmax](https://www.tensorflow.org/api_docs/python/tf/keras/activations/softmax) function. \n",
        "* Notice, we specify the input shape on the first layer. If you add subsequent layers, this is not necessary. \n",
        "* We will use the [categorical crossentropy](https://www.tensorflow.org/api_docs/python/tf/keras/losses/categorical_crossentropy) loss function, and the [RMSProp](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/RMSprop) optimizer."
      ]
    },
    {
      "metadata": {
        "id": "mNscbvHkUrMc",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model = tf.keras.Sequential()\n",
        "model.add(tf.keras.layers.Dense(512, activation=tf.nn.relu, input_shape=(784,)))\n",
        "model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax))\n",
        "\n",
        "# We will now compile and print out a summary of our model\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer='rmsprop',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "k3br9Yi6VuBT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 7) Training\n",
        "\n",
        "Next, we will train the model by using the [fit method](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential#fit) for 5 [epochs](https://www.quora.com/What-is-epochs-in-machine-learning). We will keep track of the training loss and accuracy as we go. Please be patient as this step may take a while depending on your hardware."
      ]
    },
    {
      "metadata": {
        "id": "gBs0LwqcVXx6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model.fit(train_images, train_labels, epochs=5)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "rcYMPkwkWIPq",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Step 8) Testing\n",
        "Now that we have trained our model, we want to evaluate it. Sure, our model is >97% accurate on the training set, but what about on data it hasn't seen before? The test accuracy is a good metric for that."
      ]
    },
    {
      "metadata": {
        "id": "iuqDe4NiWBpU",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "loss, accuracy = model.evaluate(test_images, test_labels)\n",
        "print('Test accuracy: %.2f' % (accuracy))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "jo-yoMwvXkw6",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Congratulations\n",
        "You have successfully used TensorFlow Keras to train a model on the MNIST dataset."
      ]
    }
  ]
}