Created
October 1, 2020 04:00
-
-
Save JingliSHI0206/2eb561f358060017cac2628273e65421 to your computer and use it in GitHub Desktop.
ReserveEngineerofAI_UTAS.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "nbformat": 4, | |
| "nbformat_minor": 0, | |
| "metadata": { | |
| "colab": { | |
| "name": "ReserveEngineerofAI_UTAS.ipynb", | |
| "provenance": [], | |
| "collapsed_sections": [], | |
| "mount_file_id": "1biWFDiO4LVR6cSeRTrJxVv6i_4QgnpCD", | |
| "authorship_tag": "ABX9TyOIJ6HSmyNkiVS3IXZCgW6I", | |
| "include_colab_link": true | |
| }, | |
| "kernelspec": { | |
| "name": "python3", | |
| "display_name": "Python 3" | |
| } | |
| }, | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "view-in-github", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<a href=\"https://colab.research.google.com/gist/JingliSHI0206/2eb561f358060017cac2628273e65421/reserveengineerofai_utas.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "hZ40ug1qlcSZ" | |
| }, | |
| "source": [ | |
| "<p><img alt=\"Colaboratory logo\" height=\"45px\" src=\"https://www.utas.edu.au/__data/assets/image/0004/1243606/utas-logo-int.png\" align=\"left\" hspace=\"10px\" vspace=\"0px\"></p>\n", | |
| "\n", | |
| "<h1>KIT719: Artificial Intelligence and Natural Language</h1>\n", | |
| "\n", | |
| "\n", | |
| "# Semester Two: Reverse Engineering of AI Models\n", | |
| "A simple demo to help understand low-level theory of AI models, including 4 steps of training routine:\n", | |
| "- Initialization\n", | |
| "- Forward Propagation\n", | |
| "- Backward Propagation\n", | |
| "- Optimization" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "SCwp7jvhmC8H" | |
| }, | |
| "source": [ | |
| "## XOR Logic Gates Implementation using AI Models\n", | |
| "A simple demo to help understand low-level theory of AI models\n", | |
| "\n", | |
| "\n", | |
| "## XOR Logic Gate\n", | |
| "\n", | |
| "\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "eQrv_C6roGOd" | |
| }, | |
| "source": [ | |
| "## AI Model\n", | |
| "AI Models:\n", | |
| "- Input Layer: 1\n", | |
| "- Hidden Layer: 1\n", | |
| "- Output Layer: 1\n", | |
| "\n", | |
| "A bias value allows you to shift the activation function to the left or right, which may be critical for successful learning.\n", | |
| "\n", | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "yyfbfNO_q8mU" | |
| }, | |
| "source": [ | |
| "# XOR Model\n", | |
| "\n", | |
| "There are 4 steps for XOR AI models.\n", | |
| "\n", | |
| "\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "KtAdfG_MqNHR" | |
| }, | |
| "source": [ | |
| "# Import Libs\n", | |
| "\n", | |
| "numpy is a Python library for working with arrays." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "NLpYESn8kwdV" | |
| }, | |
| "source": [ | |
| "import numpy as np \n", | |
| "#np.random.seed(0)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "FI9R9sJSqlnV" | |
| }, | |
| "source": [ | |
| "# Step 1: Initialization\n", | |
| "Initilize input, weight and bias values.\n", | |
| "\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "EW1AJizT10Zk" | |
| }, | |
| "source": [ | |
| "## 1.1 Initialize X (input data)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "5hJsScXurx2M", | |
| "outputId": "395f1025-24ca-48ae-c543-9d945278249f", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 119 | |
| } | |
| }, | |
| "source": [ | |
| "#Input Data\n", | |
| "X = np.array(\n", | |
| " [\n", | |
| " [0,0],\n", | |
| " [0,1],\n", | |
| " [1,0],\n", | |
| " [1,1]\n", | |
| " ]\n", | |
| " )\n", | |
| "print('X size: ', X.shape)\n", | |
| "print('X data: \\n', X)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "X size: (4, 2)\n", | |
| "X data: \n", | |
| " [[0 0]\n", | |
| " [0 1]\n", | |
| " [1 0]\n", | |
| " [1 1]]\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "ColVcZZI18ha" | |
| }, | |
| "source": [ | |
| "## 1.2 Initialize Bias (input data)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "B2PeDew1uWPQ", | |
| "outputId": "10314e93-5a0f-4f4e-d567-eded15a6fcb3", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 68 | |
| } | |
| }, | |
| "source": [ | |
| "# Bias Data\n", | |
| "b0 = np.array(\n", | |
| " [[1,1]]\n", | |
| ")\n", | |
| "\n", | |
| "b1 = -1\n", | |
| "\n", | |
| "print('b0 size: ', b0.shape)\n", | |
| "print('b0 data: \\n', b0)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "b0 size: (1, 2)\n", | |
| "b0 data: \n", | |
| " [[1 1]]\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "HmTYbF772CZr" | |
| }, | |
| "source": [ | |
| "## 1.3 Initialize Weights" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "IEKW57REsRcC", | |
| "outputId": "e0026f78-3373-4a22-87ce-91fda80722d3", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 102 | |
| } | |
| }, | |
| "source": [ | |
| "# Weights Data\n", | |
| "W1 = np.array(\n", | |
| " [\n", | |
| " [-1, 1],\n", | |
| " [1, -1]\n", | |
| " ]\n", | |
| ")\n", | |
| "W2 = np.array(\n", | |
| " [\n", | |
| " [1], \n", | |
| " [1]\n", | |
| " ]\n", | |
| ")\n", | |
| "\n", | |
| "print('W1 = \\n', W1)\n", | |
| "print('W2= \\n', W2.shape)\n" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "W1 = \n", | |
| " [[-1 1]\n", | |
| " [ 1 -1]]\n", | |
| "W2= \n", | |
| " (2, 1)\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "m1EPKUsA2Hei" | |
| }, | |
| "source": [ | |
| "## 1.4 Initialize y (output data)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "8sb40UGrr5e7", | |
| "outputId": "b53a325d-5c1f-4dd1-f4a8-de3f29e9c071", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 85 | |
| } | |
| }, | |
| "source": [ | |
| "y = np.array(\n", | |
| " [\n", | |
| " [0],\n", | |
| " [1],\n", | |
| " [1],\n", | |
| " [0]\n", | |
| " ]\n", | |
| " )\n", | |
| "y" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "execute_result", | |
| "data": { | |
| "text/plain": [ | |
| "array([[0],\n", | |
| " [1],\n", | |
| " [1],\n", | |
| " [0]])" | |
| ] | |
| }, | |
| "metadata": { | |
| "tags": [] | |
| }, | |
| "execution_count": 6 | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "NNPt8xelxKiL" | |
| }, | |
| "source": [ | |
| "# Step 2: Forward Propagation\n", | |
| "\n", | |
| "Compute S and f(x)=sigmoid(x)\n", | |
| "\n", | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "IxEZWlkZxG1U" | |
| }, | |
| "source": [ | |
| "## 2.1 Activation Function (Sigmoid)\n", | |
| "\n", | |
| "$$\n", | |
| "f(x)={\\frac {1}{1+e^{-x}}}\n", | |
| "$$" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "oUpt4dCuw6_e" | |
| }, | |
| "source": [ | |
| "import matplotlib.pyplot as plt\n", | |
| "\n", | |
| "def plot(func,x, yaxis=(-1.4, 1.4)):\n", | |
| " plt.ylim(yaxis)\n", | |
| " plt.locator_params(nbins=5)\n", | |
| " plt.xticks(fontsize = 14)\n", | |
| " plt.yticks(fontsize = 14)\n", | |
| " plt.axhline(lw=1, c='black')\n", | |
| " plt.axvline(lw=1, c='black')\n", | |
| " plt.grid(alpha=0.4, ls='-.')\n", | |
| " plt.box(on=None)\n", | |
| " plt.plot(x, func(x), c='r', lw=3)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "JyV-NnAlqjA1" | |
| }, | |
| "source": [ | |
| "def sigmoid (x):\n", | |
| " return 1/(1 + np.exp(-x))" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "MqeD6OSww-Z0", | |
| "outputId": "ed371adb-c7ca-4848-ce75-9c5cfd3a05ab", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 269 | |
| } | |
| }, | |
| "source": [ | |
| "data = np.arange(-5, 5, 0.01)\n", | |
| "plot(sigmoid,data, yaxis=(-0.4, 1.4))" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7QcZZX38e8mCQkhBAQNF0kCAZVblAjCBAMEmKATxEZxiRdGGEQGmcELoqMjLyOvvuIFo7wCssClDCMIgzhpkCBBJFxDgIhIICACQSCQCOEWQkIu+/3jqfOmTqfPOV0n5+nqes7vs1atc05VdffunV071VXVT5m7IyIi6dqk7ABERCQuNXoRkcSp0YuIJE6NXkQkcWr0IiKJU6MXKcDMdJmaVE5yjb5er+9TdgxVonxJTKqvYmLlK7lGD4wvO4CKUb4kJtVXMVHylWKjFxGRHDV6EZHEqdGLiCQuxUa/sOwAKkb5kphUX8VEyVeKjX5J2QFUjPIlMam+iomSrxQb/diyA6gY5UtiUn0VEyVfKTb6x8oOoGKUL4lJ9VVMlHyl2OgPLzuAilG+JCbVVzFR8pVioxcRkRw1ehGRxKnRi4gkLsVGv67sACpG+ZKYVF/FRMmX6ebgIq0zM3d3KzsOkSKS26Ov1+tTyo6hSpQviUn1VUysfCXX6IF5ZQdQMcqXxKT6KiZKvlJs9JPKDqBilC+JSfVVTJR8pdjodyw7gIpRviQm1VcxUfKVYqMXEZEcNXoRkcSp0YuIJC7FRr+g7AAqRvmSmFRfxUTJV4qN/qWyA6gY5UtiUn0VEyVfKTb6MWUHUDHKl8Sk+iomSr5SbPRPlR1AxShfEpPqq5go+Uqx0R9WdgAVo3xJTKqvYqLkK8VGLyIiOWr0IiKJU6MXEUlcio1+ddkBVIzyJTGpvoqJki/deESkAN14RKoouT36er0+tewYqkT5kphUX8XEyldyjR64tewAKkb5kphUX8VEyVeKjX7/sgOoGOVLYlJ9FRMlXyk2+u3LDqBilC+JSfVVTJR8pdjoRUQkR41eRCRxavQiIolLsdHfX3YAFaN8SUyqr2Ki5CvFRr+i7AAqRvmSmFRfxUTJV4qNfquyA6gY5UtiUn0VEyVfKTb6JWUHUDHKl8Sk+iomSr5SbPRTyw6gYqaWHYAkbWrZAVTM1BhP2lKjN7ODzOwaM3vGzNzMjm/hMRPN7BYzez173JlmpsGgRETarNU9+lHAAuDzwOt9rWxmo4EbCR9D3pM97svAaf0LU0RE+qulRu/us9z93939V8C6Fh7ySWAkcJy7L8ge913gNO3Vi4g0YWabvvzyUMyGDfRTDx3oJ8xMBm5z9/ze/w3AN4GdgCcivS7AyojPnSLlS2JKp77MhgBbZtOo3LR5H3+PBEZk02a53zeY/uG44wz4J+CSgQw9VqPfDni6Yd6S3LJujd7MTgJOApgwYcI1M2bMWNDw2CeBB4FJwDzgqCavOQvYE7ijXq/vBby9Yflj2bQHcB9wZJPnuDZ7jYeAXbIp78/AU8CuWTzTmzzHTMIIdPdl8YxvWL6QkIuxWTyHNyxfV6vVZtbr9SnZe50E7NiwzgLgJWBMFk/jneNX12q1a7OxrW/N4mkcLOl+wjW7T9Tr9a3Z8CTQylqtNqterx9aq9V+n8UzpmGd+cBaQiG/BExpWL68VqvNzj3HVGDrhnXuBoYQanEFYSch7+VarXZTvV4/pFar3Vyv1w8jbGh5c7MY1mTx7NewfFmtVpuTi+NwwgaYdzvh0rYVWTz7NCxfWqvVbu/6o16vTydsnHlzgG0JuRgJvKth+bOEf9ODsniOBBr33m4i1MbSLJ69GpY/Tait/Wu12u31ev0oNvxkPptQu09l8ezesLzI9vSXLJ7+bk/P1+v1UXTS9rR27Y6jFi8eueWiRVtuvnjxlmPuv//lkUuWjFwzfPi4zZYtY8Vb3rLHkFWrNh+yevXmm6xePWqT1atHDl21agQwukl8A+7l8ePHzanXx1Bwe6rVar/u6TkL32HKzJYD/+rul/SyzmzgaXc/ITdvHKHADnD3uYVetICuJMR6/tQoX8XoDlPFtL2+zEYSmv5Ywg5S1+9vJfynNyabhrctpgLcbI25fwH38wfyeWM1+kuBbdz9iNy89xD23ia4e8xDNyLRqNGXLJzj24HwKeBt2c9dCZ8WxrHhJ8aB9jLhE9urwGvA8mx6reFn/vcVhItYVmZT/vfuk/vaGEHHOnQzF/iumY1w965jdNOAxcCiSK8JQL1en5L/iC29U74kpn7Xl9kmhPN5e2XTRMJhol0Jh8U21muEw2NLCYd+lgLPA8uAFwnN/MWG31+J1Yi7xNoeW2r0ZjaKkGAIxwPHmdnewDJ3/6uZnQ3s5+5dx4ovB/4DuMTMvkU4vvdV4CyPfzfyxuPI0jvlS2Lqu77MhhKa+H6Ey7H3JhyP37wfr7caeIZw7D8/PQ08R1dzd3+tH8/dDlG2x1b36PcFbs79fVY2/SdwPOFE3/8/0eLuL5vZNOB84F7C/4Y/AGZsfMgiUmlm2wIHA39HaOzvpthe+jLCSeJHs59d05PAEtxbuQR8UGmp0bv7HKDH45LufnyTeQ8AB/U3MBFJhNn2hMY+Nfu5W4uPfB54IJsWZNMjuC+LEGXSYh2jF5HBKnzh573A9OkjR36McNVLX54G7iFcsDEf+BPuGhBtgKTY6OeXHUDFKF+y8cy2AT4IHEG48GI0wLAVTYdXf4NwTftt2c97cH+2PYF2vCjbY4qNPupZ8QQpX9I/ZlsTvmz1UcKX9nrqJ6uAuwhfKLsFuIvu35qX9Sp1eWWZBuLSq8FE+ZLWmW0GfAg4lrDn3lMPeRK47m/vfOcf3/KnP12Gu+401Zoo22OKjf6lsgOoGOVLehe+pLQPcALwCTYcgqLLXcCvgeuAhbj7nfX6mFqtpibfuijbY4o3HmkcH0J6p3xJc2abY3YyYVyke4DPsmGTnwd8CRiP+2Tcv4/7Q6z/vozqq5go+Upxj15ENobZeOBfgM/Q/B6mjwM/Ay7DfVEbI5N+UqMXkcBsX8I32D/Ehp/2XweuIjT42/SlpGpRoxcZ7MymAGcA72uy9DHgx8AluL/c1rhkwKTY6JeXHUDFKF+DldkhhDGpDm6y9CbgXGDWRg7kpfoqJkq+Cg9TLDKYJTFMsdkk4Ds0uUkHcCVwNmEIE0lEclfd1Ov1Q8uOoUqUr0HEbBfMLgf+QPcmv4Zw7H033D8xkE1e9VVMrHxpj16kgEru0ZuNBs4EPkf3WxeuA34OfEtXz6QtxT36qWXHUCXKV8LMDLNjgUcI17rnm/xMYC/cT4zZ5FVfxcTKV4onY2PfSiw1yleKzN4FnMeGX8C5A/gK7ne2KRLVVzFR8pXcHr3IoGa2GWbfIYyCmG/yi4FPAge2sclLh0hxj15kcDI7EPgp4dadXdYQ7uz2LdxfLSUuKZ0avUjVmW1BuFzylIYlc4DP4v5w22OSjpJio7+77AAqRvmqMrMDgF8AO+fmvgJ8GfhpBwxVoPoqJkq+UjxGP6TsACpG+aois6GYnUW4S1O+yf8G2BP3izqgyYPqq6go+Upxjz7F9xST8lU1ZrsS9uL3z819mTDi5OV01pdjVF/FRMlXiv8IuslBMcpXlZh9HLgIGJWbewvwKdz/Wk5QvVJ9FRMlXykeuplcdgAVo3xVgdlwzM4HLmd9k19DGFb4sA5t8qD6KipKvlLcoxdJi9lOhLHg983NfRT4OO7zywhJqiXFPXqRdJgdQRiELN/kfwXsqyYvrVKjF+lEYZyaMwhX0bwpm7sG+DzwUdxfKS02qZwUD93oLjjFKF+dxmwkYdjgY3JznyY0+LnlBNVvqq9iouRLwxSLFBB9mGKzsYSRJd+dm3szock/H+11JWnJHbqp1+uHlB1DlShfHcRsMnAP3Zv8BcD7qtrkVV/FxMqX9uhFCoi2Rx+uj78E2DSbswY4FfcLB/y1ZNBJcY/+sLJjqBLlq2ThpOu/Ea6P72ryLwDTUmjyqq9iYuUrxZOxW5YdQMUoX2UxGwL8GPhsbu5C4AO4P15OUANO9VVMlHyl2OhFOl+4suaXwAdzc28BPoT7i+UEJalK7tCNSMczG0O4kibf5K8gnHRVk5cBp0Yv0k5hOIM7gP1yc78LfBL3VWWEJOlL8dBN1b5QUjblq13M9gBuBHbI5qwjXFlzQXlBRaf6KiZKvlJs9CPLDqBilK92MHsP8Ftg62zOKuBjuM8sL6i2UH0VEyVfKTb6NWUHUDHKV2xmhwDXsH544eXAB3G/ubyg2kb1VUyUfKV4jH5t2QFUjPIVk1kNuJ71Tf4F4NBB0uRB9VVUlHyl2Oj363sVyVG+YjE7DrgaGJ7NeQY4CPd7yguq7VRfxUTJV4qNXqR8Zv9KGNKg62bPfwGm4P5QaTHJoKVGLzLQzE4jfOO1y5+AA3FfVE5AMtip0YsMJLOvAj/IzbkLmIr7cyVFJJJko19WdgAVo3wNFLP/BZydm3M7cPgg/7ar6quYKPnSMMUiBTQdptjMgP8NnJGbezNwJO6vtTE8kaaS26Ov1+uHlh1DlShfGyk0+bPp3uRvJIxAOeibvOqrmFj50h69SAHd9uhDkz8HOC23yvXAh3FfWUJ4Ik2luEd/eNkxVIny1U+hyZ9L9yZ/LWGYYTX5jOqrmFj5Sq7Rs/4biNIa5asos00I93I9NTf318BHNALlBlRfxUTJV8uN3sxOMbMnzGylmc03swN7WXeqmXmTabeBCVukHNkGczFwcm72fxMGKHujhJBE+tRSozezYwgfU78NTALuBK43s3F9PHRPYPvc9Gj/QxUpmdmQn4ffTsjNvYwwlvzqMkISaUWre/SnAZe4+8XuvtDdTwWepfu9LptZ6u7P5SYNcCTVZDYU+K9PdZ97CXAc7hqhUTpan43ezDYF9gFmNyyaDRzQx8PvNbNnzewmC0O1tsPtbXqdVChffTEbBlwOfDw392Lg02jnpS+qr2Ki5KvPyyvNbAfCqHsHu/utuflnAp9093c0ecw7gEOAe4BNgX8kHNM82N1va7L+ScBJABMmTLhmxowZCxpWeRJ4kHDYaB5wVJNQZxEOFTnwBvD2huWPZdMewH3AkU2e49rsNR4CdsmmvD8DTwG7ZvFMb/IcM4H9s9fYExjfsHwhsAQYm8XTeJZ9Xa1Wm1mv16cQ3uskYMeGdRYALwFjsngOa1i+ularXVuv16cCt2bxbN+wzv3ACuBdwN3A1IblK2u12qx6vX5orVb7fRbPmIZ15hOGVR2ZxTOlYfnyWq02O/ccU1l/440udxMG/hqaxTO5YfnLtVrtpnq9fkitVru5Xq8fBmzZsM7cLIY1WTyNIwAuq9Vqc3JxHM6GJ71uB7bKYhhC2Llh2KuvDj349NO/uPmSJX+XW/cn11511XXrhg0b3vAcc4Bts1yMJOQ271nCv+lBWTxHAsMa1rmJUBtLs3j2alj+NKG29q/VarfX6/Wj2HCHbTahdp/K4tm9YXmR7ekvWTz93Z4+CNzA4NmetsrimdqwvNXtaVz2+MLbU61W+zU9cfdeJ8Jtzxw4qGH+mcAjfT0+t/4s4JpW1+/vNHPmzB1jv0ZKk/LVywTDHeoO3jX9KPy00mOryKT66ox8tXKM/nnCXtK2DfO3BYoM1DQPeFuB9ftrSN+rSI7y1YzZCMJY8h/MzZ3xBQh7R9Iq1VcxUfLVZ6P3cMnYfGBaw6JphKtvWrU34aNrbPu04TVSonw1Ck3+f4AjcnO/B5xeTkCVpvoqJkq+Wr1n7Azgv8zsbuAOwvH2HYALAczsUgB3/1T29xeARYTjbpsCxxKOAx49gLGLDDyzkYTjwvkdm28DZ+DumDV/nEgHa6nRu/uVZrYNYeCm7QknL6a7+5PZKo3X028KfJ9w0uN1QsM/wt1nDUjUIjGYbU64iXd+YKmzgLN0uEaqrNU9etz9AsLXvpstm9rw9/cIH3VFqsFsFPAb4ODc3DNx/2ZJEYkMmJYbfYUsLTuAilG+zLYgXBWWv5zt33E/u4dHSOtUX8VEyZeGKZbBzWwrQpPPX7//Fdy/33z1JjceEelwyY1eqRsdFDOo82U2hnAnqHyTP62nJi/FDer66gfdeERkIJntSLgTVH5E1VNxP6/3h2mPXqonxT36Zl+jlh4MynyZTQBuY32TXwec0FeTl+IGZX1thFj5SvFk7IiyA6iYwZUvsz2A37F+rJLVhGGGryovqKQNrvraeFHylWKjF2nO7N2EAbbenM1ZCRyNvt8hiVOjl8HB7L2Eq2tGZ3OWA0fiPqe0mETaJLlj9CIbMKsRDtd0NfkXgb9Xk5fBIsVGP6fsACpmTtkBRGX2GcKNu7uOfS4FpuI+r7ygBpU5ZQdQMXNiPGmKjb5xOGXpXZr5MjPCzXEuYn2dPwYcgPufygts0EmzvuKJkq8Uj9G/VHYAFZNevsyGAOcD/5ybOx+Yjru+kt9e6dVXXFHylWKjH1l2ABWTVr7MNiPc3zV/e7zZwEdwf7WcoAa1tOorvij5SvHQTeN9OqV36eTLbDvCMc58k7+McHWNmnw50qmv9oiSrxQbvQxGZu8k3Gg8f2Pwc4BPEe6SJjJoqdFL9ZkdQbjz2dhszjrCuDVfxn1deYGJdIYUj9HLYGFmwOcIt7rs2ml5FTgG9+tLi0ukw6TY6NtxA/KUVDNfZsOB84ATc3OfBD6A+4JygpImqllf5YmSrxQbvb4IU0z18mU2FvgV3Y/H3wUchfuScoKSHlSvvsoVJV8pHqM/qOwAKqZa+TKbSrgmPt/kLwMOVZPvSNWqr/JFyZduPCLVEI7Hf5Fw0/kh2dy1wGnAj2lTIevGI1JFye3R1+v1I8uOoUoqkS+zrYGrgR+wvskvJezF/992NXkprhL11UFi5SvFY/TDyg6gYjo7X2F44V+y/tJJCMfjP4L7M+UEJQV0dn11nij5Sm6PXhJhNgSzrwO30L3Jn08YfVJNXqRFKe7RS9WZjQd+DhySm/si4b6uM8sJSqS6tEcvnSMMLXwi8ADdm/ztwLvU5EX6J8VGf1PZAVRMZ+TL7K3AdcDFwBbZ3HXAN4FDcH+qrNBko3RGfVVHlHyl2OjH9r2K5JSbL7NNMPs0sAD4h9ySRwg3CTkT9zXlBCcDQNtjMVHylWKj140liikvX2Z7Ek62/hTYKpvrwA+BSbrdXxK0PRYTJV8pNvqt+l5FctqfL7ORmJ0N/BGYklvyBOEwzWm4v972uCQGbY/FRMlXio1+r7IDqJj25SucbP0o8CDwVdZf9bUG+A6wF+63tC0eaQdtj8VEyZcur5T2MNufcEhmcsOSO4CTNeKkSDwp7tFLJzHbGbPLCN9mzTf5FwhDDB+kJi8Sl/boJQ6zccDXgRPoXmergXOB/4N7lDvei0h3KTb6p8sOoGIGNl/hevivAZ8BNm1YejXwb7g/NqCvKZ1M22MxUfKVYqO/r+wAKmZg8mW2G3A68I9s2OBvBc7A/bYBeS2pEm2PxUTJV4rH6PcvO4CK6X++wlU0B2J2DbAQ+DTdm/xc4O8Jg5CpyQ9O2h6LiZIv3XhEijPbHPg4cDKwT5M15gHfAG5Ibax43XhEqii5Pfp6vX5U2TFUSaF8me2F2XnAYsKYNI1N/lrCrdAm4/7b1Jq8FKftsZhY+UrxGH1y/3lF1nu+zMYAHwOOBd7TZI1VhHu2noP7wgGPTqpO22MxUfKVYqOXjWU2GvgAobkfzvrb9+U9ClwI/CfuL7QxOhEpSI1eArNtgRpwFHAYG145A+Ea+GuAnwA3476ufQGKSH+p0Q9WZkOAfQ6YOPFojjrqS4RvrfZ0kvF24BfAVbgva1eIIjIwUmz0s8sOoCOZGbAzYW/98Oznm97ywAM9PeI+whecLsf9ibbEKCnS9lhMlHyl2Oh3Ae4vO4jSmQ0DJgHvzU3b9fKIdcBtwP8AddwXxQ5RBgVtj8VEyVeKjX7w3XIuNPXdgXdn0yTCpY+b9fHI59YMHz5n6KpV1wG/xf35uIHKIDT4tseNEyVfKTb6bYE0jyObbQKMB3YjNPbdgb2BicDwFp7hFeBO4EbCR8QHr7vyyt1qtZoui5RY0t0e44iSrxQb/e6Er+NXUzhJ+lbC8fSdsp/vILyvdwAjCjzbIsJ4713Tg7iv7bZGvV7tfEmnU30VEyVfKTb6zmU2FBgD7ABsn5vGsr6pj6N//y5PAn8gnET9A/AH3J/d+KBFpOrU6PsrXMUyEtgmm7bO/Z7/+y2sb+hj6PkSxlYtBR4m/K+/kHBbvvv0pSUR6UnLjd7MTgG+TGhYDwJf8F5GJDSzg4EZwJ6EsVG+5+4Xbly4/RAa8ohs2iybevp9C2B09rOv30fT2nHx/lhKuFH2ouznY4Sm/rAauogU1VKjN7NjCHcFOoXw5ZlTgOvNbA93/2uT9XcGZgE/I3yNfgpwgZn9zd2vHqjg8y8I3A2M/MCwYVuyevV5rG/iRY5pt8PfgGcJ//k9m/u9q7Evwn1FG+N5so2vJYOP6quYKPlqdY/+NOASd784+/tUM3s/8FnC3YQanQwsdvdTs78XWrg59OmEL+EMLHfHbCIwfMjq1QP+9L1YRbj36QuEM+Uv9DA9R2jmS3Bva4AteLDsACRpqq9iouSrz0ZvZpsSrsk+p2HRbOCAHh42mQ2/4XUDcJyZDfOGZmdmJwEnZX82G9+8T8uAN/WwbGU2vZ5NPf3+am56pZe/u35fEQ7d7JBNrTENZV51Zqbhl6Xj9HafhFb26N9MGL1wScP8JYS7BzWzHfC7JusPzZ6v29Ug7n4RcFELsfTMbBKwZs2IESuHrly5gvU9fOUI93UjgK026gXSVK/XN6nVahqcrEW68Ugxqq9iYuUrnbGi3e/D/YHrrrhiIu7P4P4C7is0wmKfdGMIiUn1VUyUfLXS6J8H1hK+sZW3LeHYczPP9bD+muz5RESkTfps9O7+BjAfmNawaBrh6/TNzO1h/Xsbj8+LiEhcrR66mQEcb2YnmtnuZnYu4QTkhQBmdqmZXZpb/0LgrWb2o2z9E4Hj2fCEroiIRNbS5ZXufqWZbQOcQfjC1AJgurt3XfM5rmH9J8xsOvBDwiWYi4HPRbmGXkREetXyN2Pd/QLggh6WTW0y7xbCkLntNquE16wy5UtiUn0VEyVf6Vx1s96eZQdQMcqXxKT6KiZKvlJs9H8pO4CKUb4kJtVXMVHylWKjH1t2ABWjfElMqq9iouQrxUb/9rIDqBjlS2JSfRUTJV8pNnoREclRoxcRSZwavYhI4lJs9I+VHUDFKF8Sk+qrmCj5UqMX5UtiUn0Vo0bfoj3KDqBilC+JSfVVTJR8pdjo7ys7gIpRviQm1VcxUfKVYqM/suwAKkb5kphUX8VEyVeKjV5ERHLU6EVEEqdGLyKSODV6EZHEpdjory07gIpRviQm1VcxUfKVYqOfVHYAFaN8SUyqr2Ki5CvFRv9Q2QFUjPIlMam+iomSrxQb/S5lB1AxypfEpPoqJkq+1OhF+ZKYVF/FqNGLiEhxavQiIolToxcRSVyKjf7PZQdQMcqXxKT6KiZKvlJs9E+VHUDFKF8Sk+qrmCj5SrHR71p2ABWjfElMqq9iouQrxUb/YNkBVIzyJTGpvoqJkq8UG/30sgOoGOVLYlJ9FRMlXyk2ehERyVGjFxFJnBq9iEji1OhFRBKXYqOfWXYAFaN8SUyqr2Ki5CvFRr9/2QFUjPIlMam+iomSrxQb/X1lB1AxypfEpPoqJkq+Umz0e5YdQMUoXxKT6quYKPlKsdGPLzuAilG+JCbVVzFR8pVioxcRkRw1ehGRxKnRi4gkzty97BgGlJmd5O4XlR1HVShfxShfxShfxcTKV4p79CeVHUDFKF/FKF/FKF/FRMlXio1eRERy1OhFRBKXYqPX8cBilK9ilK9ilK9iouQruZOxIiLSXYp79CIikqNGLyKSODV6EZHEVbrRm9kcM/OG6YoWHne0mT1kZquynx9qR7xlMrOtzezHZvawmb1uZk+Z2U/MbJs+Hnd8kxy7mY1oV+ztZGanmNkTZrbSzOab2YF9rH9wtt5KM3vczE5uV6xlMrOvmdk9ZvaKmf3NzK41s736eMxOPdTS+9sVd1nM7BtN3vdzfTxmopndkm2vz5jZmWZm/Xn9Sjf6zM+B7XPTP/e2splNBq4ELgP2zn5eZWap3yBhB+CtwFeAicCxwEHAL1t47Aq653h7d18ZKc7SmNkxwLnAt4FJwJ3A9WY2rof1dwZmZetNAs4GfmxmR7cn4lJNBS4ADgAOBdYAvzOzrVt47PvpXk+/jxRjp3mE7u97Yk8rmtlo4EZgCfAe4PPAl4HT+vXK7l7ZCZgDnFfwMVcCNzbM+x3wy7LfTwn5mw6sA0b3ss7xwPKyY21TPuYBFzfMexQ4u4f1vws82jDvp8Dcst9LCbkbBawFjuxlnZ0AB/YtO94S8vMNYEGB9T8LvAJslpt3BvAM2dWSRaYU9ug/ZmbPm9mDZnaOmW3Rx/qTgdkN824g7JkMNqOBVYQ99t5sZmZPmtnTZvYbM5vUhtjaysw2BfZhw9qYTc+10VMt7WtmwwY2wo63BeEIwYstrPtrM1tqZneY2Ucix9VJJpjZ4uzQ4BVmNqGXdScDt7n767l5NxA+me9U9IWr3ugvBz4JHAJ8EzgauLqPx2xH+DiUtySbP2iY2VaEnF3s7mt6WfUR4ASgBnwcWAncYWZvix9lW70ZGEKx2uiploZmzzeYnAv8EZjbyzrLgdOBjxI+Td4EXGlmx8YPr3TzCJ+O3w98hlA7d/Zyjqyn2upaVsjQog+Izcy+BXy9j9UOcfc53n2UtwfM7HFgnpm9293/EC/KzlEkX7nHjAKuJXwM/EpvD3T3ueQ2XjO7k7BBnwp8rn9RS0rMbAYwBZji7mt7Ws/dnwd+kJt1r5m9mVCDv4gbZbnc/fr832Z2F/A4cBwwI/brd1yjB35E3//of+1h/r2E44RvA3pq9M8B2zbM2zabX0WF8pU1+VnZnx/wgidV3X2tmd1LyHFKnifUTpHa6KmW1mTPlzwz+yHwMcLOxOP9eIp5wD8NbFSdz92Xm9mD9Lwd9VRbXcsK6bhGn/2v3xpiaKMAAAIMSURBVN+NZCLh4/ezvawzF5gGfD83bxrhyonKKZKv7PzF9YAB73f35UVfL7u8653A/UUf28nc/Q0zm0+ohatyi6bR8+HAuUDjpbnTgHvdffXAR9lZzOxc4BhCk3+4n0+zN71vr0nKLk/eDbi5h1XmAt81sxG5nbFpwGJgUeEXLPts9Eacxd4FOBPYl3ByYjqwkLAnPyS33k3krpognFhbA3w1S/TXgNXA/mW/p8j52iIrnq69iO1y06a95Os/gPcBEwgb5c+yfO1X9nuKkKNjgDeAE4HdCcedlwPjs+WXApfm1t8ZeI3wqWr37HFvAEeX/V7akKvzCVeFHNpQS6Ny65wN3JT7+zjgE1mu3kE4Xv8G8MWy308b8nUOcHBWM/sDv8ny11VbjbnakrDnfgWwF/DhbP0v9ev1y07ARiRuLHAL8ALhypG/ZBvm1g3rLQIuaZj3EeDhrMgWAh8u+/20IV9TCZe2NZum9pQv4IfAk1mOlxLO/E8u+/1EzNMpWQ5WAfOBg3LL5gBzGtY/mLBzsQp4Aji57PfQpjz1VEvfyK1zCbAo9/dxwEPZf46vEA61Hlv2e2lTvq4g7I2/QTg3djWwR0+5yuZNBG4lXADxLGGnq/Clle6u0StFRFJX9csrRUSkD2r0IiKJU6MXEUmcGr2ISOLU6EVEEqdGLyKSODV6EZHEqdGLiCTu/wHwg8hvQLI2/AAAAABJRU5ErkJggg==\n", | |
| "text/plain": [ | |
| "<Figure size 432x288 with 1 Axes>" | |
| ] | |
| }, | |
| "metadata": { | |
| "tags": [], | |
| "needs_background": "light" | |
| } | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "p3F9bxxOx0P-" | |
| }, | |
| "source": [ | |
| "# Step 3: Backward Propagation\n", | |
| "\n", | |
| "Compute gradient descent of loss/cost function\n", | |
| "\n", | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "UZcyY_cDzCPl" | |
| }, | |
| "source": [ | |
| "##3.1 Derivative of Preactivation Function\n", | |
| "\n", | |
| "(1) Preactivation Function\n", | |
| "$$\n", | |
| "s = \\sum{w_{i} * x_{i} + b} = W*X + b\n", | |
| "$$\n", | |
| "\n", | |
| "(2) Derivative of Preactivation\n", | |
| "$$\n", | |
| "\\frac{\\partial s}{\\partial W} = s'= W\n", | |
| "$$" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "SxJJk8-cyk7t" | |
| }, | |
| "source": [ | |
| "##3.2 Derivative of Activation Function\n", | |
| "\n", | |
| "(1) Activation Function\n", | |
| "$$\n", | |
| "f(x) = {\\frac {1}{1+e^{-x}}}\n", | |
| "$$\n", | |
| "\n", | |
| "(2) Derivative of Activation\n", | |
| "$$\n", | |
| "f'(x)=f(x)(1-f(x))\n", | |
| " = {\\frac {1}{1+e^{-x}}} (1 - {\\frac {1}{1+e^{-x}}})\n", | |
| "$$" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "A686VugBw8b8" | |
| }, | |
| "source": [ | |
| "def sigmoid_derivative(f_x):\n", | |
| " return f_x * (1 - f_x)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "pA0NZMz01CJE" | |
| }, | |
| "source": [ | |
| "# Step 4: Optimization\n", | |
| "\n", | |
| "Updating weights and bias\n", | |
| "\n", | |
| "$$\n", | |
| "W_{t+1} = W_{t} - \\alpha \\frac{\\partial L}{\\partial w}\n", | |
| "$$" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "a-KUEsaj8D2s" | |
| }, | |
| "source": [ | |
| "## 4.1 Update Layer 2 Weights\n", | |
| "\n", | |
| "$$\n", | |
| "W^{(2)}_{t+1} = W^{(2)}_{t} - \\alpha \\frac{\\partial L}{\\partial W^{(2)}} \\space\n", | |
| "$$\n", | |
| "\n", | |
| "\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "GSk5wkl2TiEE" | |
| }, | |
| "source": [ | |
| "$$\n", | |
| "\\frac{\\partial L}{\\partial W^{(2)}} = \\frac{\\partial L}{\\partial \\hat{y}} * \\frac{\\partial \\hat{y}}{\\partial W^{(2)}}\n", | |
| "$$\n", | |
| "\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "dplPzwPzTk9C" | |
| }, | |
| "source": [ | |
| "$$\n", | |
| "L(\\hat{y},y) = \\frac{1}{n} (y - \\hat{y})^2 \\space\\space\\space\\space, \\space\\space\\space\\space \\hat {y} = f(s^2) = f(W^{(2)}*h + b_1)\n", | |
| "$$" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "qSMz2p1fTnti" | |
| }, | |
| "source": [ | |
| "$$\n", | |
| "\\frac{\\partial L}{\\partial \\hat{y}} = \\frac{2}{n} *(y - \\hat{y}) \\space\\space\\space\\space, \\space\\space\\space\\space \\frac{\\partial \\hat{y}}{\\partial W^{(2)}} = f'(W^{(2)}*h + b_1)\n", | |
| "$$" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "a-fbX3dKTpk6" | |
| }, | |
| "source": [ | |
| "$$\n", | |
| "\\frac{\\partial L}{\\partial W^{(2)}} = \\frac{2}{n} *(y - \\hat{y}) * f'\n", | |
| "$$" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "yPN61ROLCCZm" | |
| }, | |
| "source": [ | |
| "## 4.2 Update Layer 1 Weights\n", | |
| "\n", | |
| "### how to update W1?" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "h5b2xMCplLnq", | |
| "outputId": "7b977725-a0a3-47f8-fe7d-4d430edfbb66", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 34 | |
| } | |
| }, | |
| "source": [ | |
| "#Training Model\n", | |
| "\n", | |
| "epochs = 10000\n", | |
| "lr = 0.01\n", | |
| "\n", | |
| "for _ in range(epochs):\n", | |
| "\t#Forward Propagation\n", | |
| "\ts1 = np.dot(X,W1) + b0\n", | |
| "\th = sigmoid(s1)\n", | |
| "\n", | |
| "\ts2 = np.dot(h,W2) + b1\n", | |
| "\ty_pred = sigmoid(s2)\n", | |
| "\n", | |
| "\t#Backpropagation\n", | |
| "\t#L = 1/2 * (y - y_pred)**2\n", | |
| "\tL_derivative = (y-y_pred) * sigmoid_derivative(y_pred)\n", | |
| " \n", | |
| "\tL_w_derivative = L_derivative.dot(W2.T) * sigmoid_derivative(h)\n", | |
| "\t\n", | |
| "\n", | |
| "\t#Updating Weights and Biases\n", | |
| "\tW2 = W2 - lr * h.T.dot(L_derivative)\n", | |
| "\tb1 = b1 - lr * np.sum(L_derivative,axis=0,keepdims=True) \n", | |
| "\tW1 = W1 - lr * X.T.dot(L_w_derivative) \n", | |
| "\tb0 = b0 - lr * np.sum(L_w_derivative,axis=0,keepdims=True) \n", | |
| " \n", | |
| "print('Finish model training...')" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "Finish model training...\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "WSfPd271oU8r", | |
| "outputId": "a3aba56f-7eec-4f66-eaff-43083252606d", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 544 | |
| } | |
| }, | |
| "source": [ | |
| "print('\\n' + '-'*50 +'\\n')\n", | |
| "print(\"Final layer 1 weights W1:\\n \",end='')\n", | |
| "print(W1)\n", | |
| "print('\\n' + '-'*50 +'\\n')\n", | |
| "print(\"Final layer 2 weights W2:\\n \",end='')\n", | |
| "print(W2)\n", | |
| "print('\\n' + '-'*50 +'\\n')\n", | |
| "print(\"Final layer 1 bias b0:\\n \",end='')\n", | |
| "print(b0)\n", | |
| "print('\\n' + '-'*50 +'\\n')\n", | |
| "print(\"Final layer 1 bias b11:\\n \",end='')\n", | |
| "print(b1)\n", | |
| "print('\\n' + '-'*50 +'\\n')\n", | |
| "\n", | |
| "print(\"\\nOutput from neural network after 10,000 epochs:\\n \",end='')\n", | |
| "print(y_pred)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "\n", | |
| "--------------------------------------------------\n", | |
| "\n", | |
| "Final layer 1 weights W1:\n", | |
| " [[-0.75547202 1.29550082]\n", | |
| " [ 1.29550082 -0.75547202]]\n", | |
| "\n", | |
| "--------------------------------------------------\n", | |
| "\n", | |
| "Final layer 2 weights W2:\n", | |
| " [[2.84998061]\n", | |
| " [2.84998061]]\n", | |
| "\n", | |
| "--------------------------------------------------\n", | |
| "\n", | |
| "Final layer 1 bias b0:\n", | |
| " [[1.65862956 1.65862956]]\n", | |
| "\n", | |
| "--------------------------------------------------\n", | |
| "\n", | |
| "Final layer 1 bias b11:\n", | |
| " [[1.27711062]]\n", | |
| "\n", | |
| "--------------------------------------------------\n", | |
| "\n", | |
| "\n", | |
| "Output from neural network after 10,000 epochs:\n", | |
| " [[0.99768327]\n", | |
| " [0.99756126]\n", | |
| " [0.99756126]\n", | |
| " [0.9983539 ]]\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "yyPDhAuUvGbK" | |
| }, | |
| "source": [ | |
| "# Exercises\n", | |
| "\n", | |
| "## 1. Write formula of layer 1 weight updating.\n", | |
| "\n", | |
| "## 2. Compute prediction by using different loss/cost function.\n", | |
| "\n", | |
| "\n", | |
| "## 3. Compare different bias (plot different equations)." | |
| ] | |
| } | |
| ] | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment