perceptron.ipynb

perceptron.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "perceptron.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/denismazzu96/45e15b53cdb93420dfadb2bc31ea09db/perceptron.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "_dRvJGBtXkS7",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import random\n",
        "import numpy\n",
        "import math"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ZUhOtXgkYlzN",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Perceptron**, classe per definire l'unità della rete neurale *perceptron* definita da un'array di pesi (di input) e dal learning rate associato, definito per tutti i *perceptrons*"
      ]
    },
    {
      "metadata": {
        "id": "2TOXvPpLZFgC",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Perceptron:\n",
        "\n",
        "  def __init__(self, weights, l_rate):\n",
        "    self.weights = weights\n",
        "    self.weights[0] = 1\n",
        "    self.l_rate = l_rate\n",
        "    \n",
        "  def reloadWeights(self, deltaw):\n",
        "    for i in range(1, len(self.weights)):\n",
        "      self.weights[i] = self.weights[i] + deltaw[i]\n",
        "\n",
        "  def sum_output(self, inputs):\n",
        "    sum = 0\n",
        "    for i in range(1, len(inputs)):\n",
        "      # l'indice i serve per l'i-esimo elemento dei pesi\n",
        "      # altrimenti usavo il costrutto foreach 'for i in inputs'\n",
        "      sum = sum + (inputs[i] * self.weights[i])\n",
        "    return sum"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "f1cv9YsG_gJP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Implementazione del percettrone mediante funzione **Segno**, definita in maniera classica con *threshold* impostato a 0.\n",
        "\n",
        "La condizione di terminazione è che tutto il training set sia correttamente mappato dal percettrone"
      ]
    },
    {
      "metadata": {
        "id": "9Zg8-0v0aad7",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Sign(Perceptron):\n",
        "  \n",
        "  def __init__(self, weights, l_rate):\n",
        "    Perceptron.__init__(self, weights, l_rate)\n",
        "    \n",
        "  def output(self, inputs):\n",
        "    return sign(Perceptron.sum_output(self, inputs))\n",
        "  \n",
        "  def terminationCond(self, tr):\n",
        "    for data in tr:\n",
        "      i_nth_input = data[0]\n",
        "      i_nth_target = data[1]\n",
        "      if self.output(i_nth_input) != i_nth_target:\n",
        "        return True\n",
        "    return False\n",
        "\n",
        "THRESHOLD = 0\n",
        "def sign(x):\n",
        "  if x > THRESHOLD: return 1\n",
        "  else: return 0"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "W_n9z370gSfe",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "funzione che aggiorna lo stato del delta del cambiamento dei pesi"
      ]
    },
    {
      "metadata": {
        "id": "ptBT04oKL83f",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def updateDelta(old_delta, p, tr):\n",
        "  delta = old_delta.copy()\n",
        "  lr = p.l_rate\n",
        "  for data in tr:\n",
        "    inputs = data[0]\n",
        "    expected = data[1]\n",
        "    actual = p.output(inputs)\n",
        "    for i in range(1, len(delta)):\n",
        "      delta[i] = delta[i] + lr * (expected - actual) * inputs[i]\n",
        "  return delta"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "rWuW6PD4NYAT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "log ogni iterazione dello stato di essa"
      ]
    },
    {
      "metadata": {
        "id": "dBXGhySvM10h",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def logIteration (it, p, delta, tr):\n",
        "  print(\"new iteration: \", it)\n",
        "  print(\"weights: \", p.weights)\n",
        "  print(\"deltas: \", delta)\n",
        "  print(\"Predictions:\\n\")\n",
        "  for data in tr:\n",
        "    print(\"expected: \", data[1])\n",
        "    print(\"actual: \", p.output(data[0]))\n",
        "  print(\"\\n\\n\")\n",
        "  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "EI6WwdmiNW1u",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Funzione che implementa l'algoritmo di apprendimento, utilizza un singolo percettrone con funzione segno, per ogni iterazione stampo il log dei pesi, il loro ultimo aggiornamento e dove devono ancora arrivare a predirre il risultato aspettato"
      ]
    },
    {
      "metadata": {
        "id": "rHXqx2pk6YUT",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def learn(tr):\n",
        "  ws = [random.uniform(-0.1,+0.1) for _ in range (3)] # weights\n",
        "  print(\"init ws: \", ws)\n",
        "  print(\"\\n\")\n",
        "  lr = 0.02 # learning rate\n",
        "  p = Sign(ws, lr)\n",
        "  delta = [0 for i in range(len(p.weights))]\n",
        "  it = 0\n",
        "  \n",
        "  while p.terminationCond(tr):\n",
        "    delta = updateDelta(delta, p, tr)\n",
        "    p.reloadWeights(delta)\n",
        "    it += 1\n",
        "    logIteration(it, p, delta, tr)\n",
        "    \n",
        "  return p"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "7CbXu0mVNhyl",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "main"
      ]
    },
    {
      "metadata": {
        "id": "v_MytY2ji97g",
        "colab_type": "code",
        "outputId": "098a980c-2035-497b-b514-671e4ecac241",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 660
        }
      },
      "cell_type": "code",
      "source": [
        "x0 = 1 # lo aggiungo per avere un vettore che parte da 0\n",
        "tr = [ # training set for OR logic\n",
        "    [(x0,0,0),0],\n",
        "    [(x0,0,1),1],\n",
        "    [(x0,1,0),1],\n",
        "    [(x0,1,1),1]\n",
        "]\n",
        "p = learn(tr)\n"
      ],
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "init ws:  [0.04413551598929194, 0.06471836364183037, -0.03302630889450009]\n",
            "\n",
            "\n",
            "new iteration:  1\n",
            "weights:  [1, 0.06471836364183037, -0.013026308894500089]\n",
            "deltas:  [0, 0.0, 0.02]\n",
            "Predictions:\n",
            "\n",
            "expected:  0\n",
            "actual:  0\n",
            "expected:  1\n",
            "actual:  0\n",
            "expected:  1\n",
            "actual:  1\n",
            "expected:  1\n",
            "actual:  1\n",
            "\n",
            "\n",
            "\n",
            "new iteration:  2\n",
            "weights:  [1, 0.06471836364183037, 0.026973691105499912]\n",
            "deltas:  [0, 0.0, 0.04]\n",
            "Predictions:\n",
            "\n",
            "expected:  0\n",
            "actual:  0\n",
            "expected:  1\n",
            "actual:  1\n",
            "expected:  1\n",
            "actual:  1\n",
            "expected:  1\n",
            "actual:  1\n",
            "\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}