Simple PyTorch Example

simple-pytorch-example.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Simple PyTorch Example",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/JonasSchroeder/ca68d9828dde186b00bb38924771745f/simple-pytorch-example.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V6tV1JwkukOO",
        "colab_type": "text"
      },
      "source": [
        "# Simple PyTorch Example for a multi-layered neural network\n",
        "\n",
        "PyTorch is an open source python deep learning framework primarily developed by Facebook. It can be used to build neural networks. Alternatives are *TensorFlow* and *Keras*.\n",
        "\n",
        "Simple Neural Network example for classifying flowers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Nlz0CiRrLlg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "\n",
        "dataset = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data',\n",
        "                      names=['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species'])\n",
        "dataset['species'] = pd.Categorical(dataset['species']).codes\n",
        "dataset = dataset.sample(frac=1, random_state=1234)\n",
        "\n",
        "train_input = dataset.values[:120, :4] \n",
        "train_target = dataset.values[:120, 4]\n",
        "test_input = dataset.values[120:, :4] \n",
        "test_target = dataset.values[120:, 4]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fUUmRGOavmSO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch\n",
        "\n",
        "torch.manual_seed(1234)\n",
        "\n",
        "# feedforward network with one hidden layer with five units.\n",
        "# the output layer as three units (one-hot encoding for target data i.e. flower type)\n",
        "hidden_units = 5\n",
        "\n",
        "\n",
        "net = torch.nn.Sequential(\n",
        "    torch.nn.Linear(4, hidden_units),\n",
        "    torch.nn.ReLU(),\n",
        "    torch.nn.Linear(hidden_units, 3)\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4iNpajg6xvTl",
        "colab_type": "text"
      },
      "source": [
        "The **loss function** will measure how different the networks output is compared to the target data. In this example, we define cross entropy as the loss function. \n",
        "\n",
        "As **optimizer** we use stochastic gradient descent (SGD) in this example, with a learning rate of 0.1 and a momentum of 0.9."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nInG3pOFxvvN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# choose optimizer and loss function\n",
        "criterion = torch.nn.CrossEntropyLoss()\n",
        "optimizer = torch.optim.SGD(net.parameters(), lr=0.1, momentum=0.9)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2RJ1Q_oTyeni",
        "colab_type": "text"
      },
      "source": [
        "We will run training for 50 epochs (iterate 50 times over the training dataset). For each run we create the torch variable that are input and target from our training set. \n",
        "\n",
        "A **Torch** is a flexible N-dimensional array, which supports basic routines for indexing, slicing, transposing, type-casting,resizing, sharing storage and cloning.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3poYo8vZyezP",
        "colab_type": "code",
        "outputId": "a9424595-667e-4f63-f6a3-19de79ece336",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "# train\n",
        "epochs = 50\n",
        "\n",
        "for epoch in range(epochs):\n",
        "  inputs = torch.autograd.Variable(torch.Tensor(train_input).float())\n",
        "  targets = torch.autograd.Variable(torch.Tensor(train_target).long())\n",
        "\n",
        "  # Zero the gradients of the optimizer to prevent accumulation from previous iterations\n",
        "  optimizer.zero_grad()\n",
        "\n",
        "  # Feed training data to the neural network (input) and \n",
        "  # compute loss function criterion (out, target)\n",
        "  out = net(inputs)\n",
        "  loss = criterion(out, targets)\n",
        "\n",
        "  # Backpropagation of loss value to \n",
        "  # calculate how each network weight affects the loss function\n",
        "  loss.backward()\n",
        "\n",
        "  # Optimizer updates the wights of the network to reduce\n",
        "  # future loss function values (improvements)\n",
        "  optimizer.step()\n",
        "\n",
        "  if epoch == 0 or (epoch + 1) % 10 == 0:\n",
        "    print('Epoch %d Loss: %.4f' % (epoch + 1, loss.item()))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1 Loss: 1.1030\n",
            "Epoch 10 Loss: 0.5945\n",
            "Epoch 20 Loss: 0.2723\n",
            "Epoch 30 Loss: 0.1730\n",
            "Epoch 40 Loss: 0.1016\n",
            "Epoch 50 Loss: 0.0882\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0btP2KkKBoUC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "3957cb6f-ad0e-46d9-824d-8e274c99ebfd"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "inputs = torch.autograd.Variable(torch.Tensor(test_input).float()) \n",
        "targets = torch.autograd.Variable(torch.Tensor(test_target).long())\n",
        "\n",
        "optimizer.zero_grad()\n",
        "out = net(inputs)\n",
        "_, predicted = torch.max(out.data, 1)\n",
        "\n",
        "error_count = test_target.size - np.count_nonzero((targets == predicted).numpy())\n",
        "print('Errors: %d; Accuracy: %d%%' % (error_count, 100 * torch.sum(targets == predicted) / test_target.size))"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Errors: 0; Accuracy: 100%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bTjiuOjOB9DY",
        "colab_type": "text"
      },
      "source": [
        "We were able to classify all 30 test samples correctly."
      ]
    }
  ]
}