gbcanuck71/7.6_ActivationFunction_v2.ipynb

## 7.6_ActivationFunction_v2.ipynb
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"http://cocl.us/pytorch_link_top\">\n",
    "    <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/Pytochtop.png\" width=\"750\" alt=\"IBM Product \" />\n",
    "</a> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/cc-logo-square.png\" width=\"200\" alt=\"cognitiveclass.ai logo\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Activation Functions</h1> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Table of Contents</h2>\n",
    "<p>In this lab, you will cover logistic regression by using PyTorch.</p>\n",
    "\n",
    "<ul>\n",
    "    <li><a href=\"#Log\">Logistic Function</a></li>\n",
    "    <li><a href=\"#Tanh\">Tanh</a></li>\n",
    "    <li><a href=\"#Relu\">Relu</a></li>\n",
    "    <li><a href=\"#Compare\">Compare Activation Functions</a></li>\n",
    "</ul>\n",
    "<p>Estimated Time Needed: <strong>15 min</strong></p>\n",
    "\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll need the following libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7f354647b2d0>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import the libraries we need for this lab\n",
    "\n",
    "import torch.nn as nn\n",
    "import torch\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "torch.manual_seed(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Log\">Logistic Function</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a tensor ranging from -10 to 10: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a tensor\n",
    "\n",
    "z = torch.arange(-10, 10, 0.1,).view(-1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you use sequential, you can create a sigmoid object: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a sigmoid object\n",
    "\n",
    "sig = nn.Sigmoid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apply the element-wise function Sigmoid with the object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a prediction of sigmoid function\n",
    "\n",
    "yhat = sig(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the results: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'yhat')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhp0lEQVR4nO3deXhc9X3v8fd3tHqRV8kL8r5gY1YbYZawBQLYJoUASQspJet1aUOT9klvQ5M2bUOf3pClT5KGxOVSmpCQUAg0OI4dAWFrLhAsG2/yAvKGZWuzZVvyom3me/+YMQxiZMu2zpxZPq/nkWfOOb/RfHxGM9852+9n7o6IiOSvSNgBREQkXCoEIiJ5ToVARCTPqRCIiOQ5FQIRkTxXGHaAk1VeXu5TpkwJO4aISFZZtWrVXnevSLUs6wrBlClTqKmpCTuGiEhWMbOdfS3TriERkTynQiAikudUCERE8pwKgYhInlMhEBHJc4EVAjN72MyazWxDH8vNzL5nZnVmts7M5gWVRURE+hbkFsGPgAXHWb4QmJn4WQz8MMAsIiLSh8CuI3D3l81synGa3Aw84vF+sF8zsxFmNt7dG4LKJCK5w93pisbo6I7R2R2lsydGNOb0xGL0xJyeqNMTc6KxGN1RTyxzeqKxxHynOxoj5o47uBO/n/jd8WlwnFh8Jg7EYolbf7ed8972nrQMwN+TO+l+0pL3zk/9gKopo7jyzJTXhJ2WMC8oqwR2JU3XJ+a9rxCY2WLiWw1MmjQpLeFEJDjuzuGuKM1tHbS0d7LvcBdtR7tp6+im7WhP4rab9o74/UOdUTq7o3R0R+noicVvu6PxD+g8YBa/vfuq6TlXCCzFvJQvq7s/CDwIUFVVlScvvUj2cnea2zvZsfcwb7ceYVfrEXa2HmH3/qO0HOqkua2To93RlI+NGJSVFjFsUCHDSosYVlpE5YgiSooKGFRUQGlRhNLCAkqP3S+K3y8ujFBUYBREIhRFjIKIUVhgFEYiFL4zHb9/bH5BYr4BETPMSPwYEQMjaR6JecnLIvEPsuO1P8bs3YnkDz/ro006hVkI6oGJSdMTgD0hZRGRUxSLOVua2llff5BNjW1sbmhnc2Mb+490v9MmYnDGiEFUjhjE+RNGUFFWwpiyksRtKaOHFjN8UBHDBhUxpLggtA/EfBVmIVgK3GNmjwEXAwd1fEAk80Vjzrr6A7y+vZXXt7eyckcrbR09AAwqKmDWuDIWnDOe2ePKmFo+hEmjBlM5chBFBTpbPVMFVgjM7OfA1UC5mdUD/wAUAbj7EmA5sAioA44Anwoqi4icno7uKC9uaeG5TU08v7mZ1sNdAEwrH8Kic8czf+oo5k4ayeRRg4lE9G0+2wR51tAdJ1juwOeCen4ROT3uTs3O/Ty1up5l6xpo7+hhWGkh18wewzVnjeXSaaOpKCsJO6YMgKzrhlpEgtXZE+XpNXt4+Hfb2dzYzuDiAhacM45b5lZyybTR2sWTg1QIRASI7/75yas7+feXt7H3UCezx5XxjdvO48bzxjOkRB8VuUyvrkiei8acJ1fX851n32TPwQ4un1HO3VddwAdmjNbZO3lChUAkj23c08a9T61jXf1Bzp8wnG997Hwum1EedixJMxUCkTzU0R3le799i39/eRsjBxfx3dsv4Kbzz9AWQJ5SIRDJM9taDvHnj65mc2M7f1g1gS8vOosRg4vDjiUhUiEQySO/XtfAl55cR1GB8Z+fuogPzhoTdiTJACoEInkgFnO+Ub2FJS9tZe6kETzw8XmcMWJQ2LEkQ6gQiOS47miML/1iHU+9sZs/vngS//AHZ1NcqGsB5F0qBCI57EhXD3/+6Gpe3NLCF687k3uumaEDwvI+KgQiOepoV5RP/udKana08n9uPZc75mssD0lNhUAkB3X1xPjzR1exckcr3719Ljedf0bYkSSDaUehSI6JxpwvPrGWF7a08C+3nKsiICekQiCSY+5btpFfrd3DvQtna3eQ9IsKgUgOeXzlLn70yg4+c/lU7r5qethxJEuoEIjkiDW7DvB3v9zA5TPK+duFs8OOI1lEhUAkB+w91Mmf/XQVY4aV8G93zKVQYwbISdBZQyJZLhZzPv/zN2g93MWTf3YZI4eo3yA5OSoEIlnuR6/s4JWt+/j6redyTuXwsONIFtL2o0gWq2s+xP2/2cw1s8fwRxdNDDuOZCkVApEs1RON8cUn1jK4uICv33auuo6QU6ZdQyJZ6ocvbmXtrgM88PF5jCkrDTuOZDFtEYhkoR17D/Nvz9fx4fPGc+N548OOI1lOhUAkC31t2UaKCyN89cNzwo4iOUCFQCTL/HZTE89vbuYL185kzDDtEpLTp0IgkkU6uqN8bdlGZowZyic/MCXsOJIjdLBYJIs89D/b2LnvCD/9zMUU6ephGSD6SxLJEi3tnTzwwlYWnD2Oy2eWhx1HcogKgUiW+MGLdXRFY/zNgllhR5Eco0IgkgX2HDjKo6+9zW3zKplWMTTsOJJjVAhEssC/PV+H43z+2plhR5EcFGghMLMFZrbFzOrM7N4Uy4eb2a/MbK2Z1ZrZp4LMI5KNdu47zBM1u7hj/iQmjBwcdhzJQYEVAjMrAB4AFgJzgDvMrPfVL58DNrr7+cDVwLfNTH3oiiT5znNvUVhg3PPBGWFHkRwV5BbBfKDO3be5exfwGHBzrzYOlFm8t6yhQCvQE2AmkayyY+9hnl6zm7sunaKLxyQwQRaCSmBX0nR9Yl6y7wNnAXuA9cAX3D3W+xeZ2WIzqzGzmpaWlqDyimSch363jcJIhM9ePjXsKJLDgiwEqfrE9V7TNwBrgDOAC4Dvm9mw9z3I/UF3r3L3qoqKioHOKZKR9h3q5Imaem6ZW6mtAQlUkIWgHkgeKWMC8W/+yT4FPOVxdcB2QKNuiwA/fnUnnT0x/teV08KOIjkuyEKwEphpZlMTB4BvB5b2avM2cC2AmY0FZgHbAswkkhWOdPXwyKs7uG7OWGaM0XUDEqzA+hpy9x4zuweoBgqAh9291szuTixfAtwH/MjM1hPflfQld98bVCaRbPFETT0HjnTzp9oakDQItNM5d18OLO81b0nS/T3A9UFmEMk20Zjz0O+2ceHkkVRNGRV2HMkDurJYJMM8v7mZXa1HdaaQpI0KgUiG+clrOxk7rITr5owNO4rkCRUCkQyyc99hXn6zhY/Pn0yhxhuQNNFfmkgGefT3b1MYMW6fP/HEjUUGiAqBSIbo6I7yeM0urj97LGN1AZmkkQqBSIZYtq6BA0e6ufOSyWFHkTyjQiCSIX762k6mVwzh0mmjw44ieUaFQCQDbNzTxppdB7jzksnEO+MVSR8VApEM8ItV9RQXRPjIBb076BUJngqBSMi6emL8cs1uPjRnDCOHaFwmST8VApGQvbClmdbDXXzsQp0yKuFQIRAJ2RM19VSUlXDFzPKwo0ieUiEQCVFLeycvbGnm1rmVupJYQqO/PJEQPb1mN9GY89ELJ4QdRfKYCoFISNydX6yq5/yJI5g5tizsOJLHVAhEQrKxoY3Nje3aGpDQqRCIhGTpmj0URowPnzs+7CiS51QIREIQizlL1+7hyjMrdO2AhE6FQCQENTv303Cwg5vOPyPsKCIqBCJhWLp2N6VFEY1CJhlBhUAkzbqjMX69roEPnTWWISWFYccRUSEQSbff1e1l/5Fu7RaSjKFCIJJmS9fsYVhpIVfNqgg7igigQiCSVke7ojxT28jCc8ZTUlgQdhwRQIVAJK1eerOZw11R/kC7hSSDqBCIpFF1bRPDBxVx8bRRYUcReYcKgUiadEdj/HZTE9eeNYYi9TQqGUR/jSJp8tq2fbR19HDD2ePCjiLyHioEImlSXdtIaVGEK2fqbCHJLCoEImkQiznP1DZx1ZkVDCrW2UKSWQItBGa2wMy2mFmdmd3bR5urzWyNmdWa2UtB5hEJy5r6AzS3d2q3kGSkwK5vN7MC4AHgOqAeWGlmS919Y1KbEcAPgAXu/raZjQkqj0iYqmsbKYwY185W30KSeYLcIpgP1Ln7NnfvAh4Dbu7V5uPAU+7+NoC7NweYRyQU7vHdQpdMG83wwUVhxxF5nyALQSWwK2m6PjEv2ZnASDN70cxWmdldqX6RmS02sxozq2lpaQkorkgw6poPsX3vYW44W1sDkpmCLASWYp73mi4ELgRuBG4A/t7Mznzfg9wfdPcqd6+qqNAZF5JdqmsbAbhujo4PSGYKsg/cemBi0vQEYE+KNnvd/TBw2MxeBs4H3gwwl0haVdc2ccHEEYwbXhp2FJGUgtwiWAnMNLOpZlYM3A4s7dXmaeAKMys0s8HAxcCmADOJpNXuA0dZv/ugzhaSjBbYFoG795jZPUA1UAA87O61ZnZ3YvkSd99kZr8B1gEx4CF33xBUJpF0eyaxW0jHBySTBTo8krsvB5b3mrek1/Q3gW8GmUMkLNW1jcwcM5RpFUPDjiLSJ11ZLBKQ1sNdvL69VbuFJOOpEIgE5LlNTcQcFQLJeCoEIgF5praRyhGDOKdyWNhRRI5LhUAkAIc7e3j5rb1cN2csZqkuqRHJHCoEIgF46c0Wunpi2i0kWeGEhcDMSvozT0TeVV3byMjBRVw0ZWTYUUROqD9bBK/2c56IAF09MZ7f3MyHzhpLoYaklCzQ53UEZjaOeCdxg8xsLu/2HTQMGJyGbCJZ6dVt+2jXkJSSRY53QdkNwCeJ9xH0r0nz24EvB5hJJKtV1zYyuLiAy2eWhx1FpF/6LATu/mPgx2Z2m7s/mcZMIlkrFnOe3djE1bMqKC3SkJSSHU7YxYS7P2lmNwJnA6VJ878WZDCRbPTGrv20aEhKyTL9OWtoCfBHwF8QP07wMWBywLlEslJ1bRNFBcYHZ2vUVcke/Tml4TJ3vwvY7+7/BFzKe8cZEBHiQ1JW1zZy6fRyhpVqSErJHv0pBEcTt0fM7AygG5gaXCSR7LSlqZ2d+46oy2nJOv3phnqZmY0g3lX0auLDTT4UZCiRbFS9oQkzuG6OCoFkl/4cLL4vcfdJM1sGlLr7wWBjiWSf6tpG5k0ayZgyDUkp2aVfA9OY2WXAlGPtzQx3fyTAXCJZZVfrETY2tPHlRbPDjiJy0k5YCMzsJ8B0YA0QTcx2QIVAJKH6nSEpddqoZJ/+bBFUAXPc3YMOI5KtnqltYva4MiaPHhJ2FJGT1p+zhjYA+poj0oe9hzpZubOV67U1IFnqeJ3O/Yr4LqAyYKOZvQ50Hlvu7jcFH08k8z23sQl3dNqoZK3j7Rr6VuL2POB7QGvwcUSyT3ViSMo54zUkpWSn43U69xKAmV0HfIH4NQQPA9U6XiAS197Rzf+r28edl0zWkJSStU54jMDd/w6YCfwH8W6p3zKzfzGz6QFnE8l4L25poSsa024hyWr9Gj4psQXQmPjpAUYCvzCzbwSYTSTjVdc2MnpIMVVTRoUdReSU9ec6gs8DnwD2Eu9a4n+7e7eZRYC3gL8JNqJIZursifLilhZuPHc8BRHtFpLs1Z/rCMqBW919Z/JMd4+Z2YeDiSWS+V6p28ehzh5uOEe7hSS79aevoa8eZ9mmgY0jkj2qaxsZUlzAZdM1JKVkt34dIxCR94rGnOc2NXH17DEaklKyngqByClYuaOVvYe6WKCriSUHqBCInILfbGikpDDCNRqSUnJAoIXAzBaY2RYzqzOze4/T7iIzi5rZR4PMIzIQYjFnxYYGrjqzgiEl/erJXSSjBVYIzKwAeABYCMwB7jCzOX20ux+oDiqLyEB6Y9cBmto6WXiudgtJbghyi2A+UOfu29y9C3gMuDlFu78AngSaA8wiMmBWrG+gqMC49iydNiq5IchCUAnsSpquT8x7h5lVArcAS473i8xssZnVmFlNS0vLgAcV6S93Z8WGRi6fUc6w0qKw44gMiCALQapLLXt3Vvcd4EvuHk3R9t0HuT/o7lXuXlVRUTFQ+URO2obdbew+cJSF544PO4rIgAnySFc9MDFpegKwp1ebKuCxRK+N5cAiM+tx918GmEvklC3f0EBBxLhOu4UkhwRZCFYCM81sKrAbuB34eHIDd5967L6Z/QhYpiIgmcrdWbG+gcumj2bkkOKw44gMmMB2Dbl7D3AP8bOBNgGPu3utmd1tZncH9bwiQdnc2M6OfUdYcI7OFpLcEuhJ0O6+HFjea17KA8Pu/skgs4icrhUbGokYXD9HhUByi64sFumnFesbuGjKKCrKSsKOIjKgVAhE+qGuuZ23mg+xSGcLSQ5SIRDphxXrGwG4QZ3MSQ5SIRA5AXdn6do9VE0eybjhpWHHERlwKgQiJ7C5Mb5b6OYLzgg7ikggVAhETmDp2j0UREzHByRnqRCIHIe786u1e7h8Rjmjh+psIclNKgQix7H67QPU7z/KTedrt5DkLhUCkeNYumY3JYURrj9bfQtJ7lIhEOlDTzTGr9c3cO1ZYyhTl9OSw1QIRPrw6rZ97D3Upd1CkvNUCET68Ms39lBWUsjVszRAveQ2FQKRFA539rBiQwM3njee0qKCsOOIBEqFQCSF5esbONIV5aMXTgg7ikjgVAhEUnhiVT1Ty4dw4eSRYUcRCZwKgUgvO/cd5vXtrXz0wgkkhlEVyWkqBCK9PLmqHjO4dV5l2FFE0kKFQCRJLOY8uXo3l88oZ/zwQWHHEUkLFQKRJK9u28fuA0d1kFjyigqBSJLHa3ZRVlqoAWgkr6gQiCTsO9TJivWN3DK3UtcOSF5RIRBJeLymnq5ojDsvmRx2FJG0UiEQAaIx52ev7+TiqaM4c2xZ2HFE0kqFQAR4+c0WdrUe5U8u1daA5B8VAhHgJ6/tpKKshOvn6CCx5B8VAsl7u1qP8MKWZu64aCLFhXpLSP7RX73kvZ+9/jYG3D5/UthRREKhQiB57XBnDz/7/dtcN2csZ4zQlcSSn1QIJK89XrOLg0e7WXzl9LCjiIRGhUDyVnc0xkP/s52LpoxUd9OS1wItBGa2wMy2mFmdmd2bYvkfm9m6xM8rZnZ+kHlEki1f38DuA0f5U20NSJ4LrBCYWQHwALAQmAPcYWZzejXbDlzl7ucB9wEPBpVHJJm7s+SlbcwYM5RrZmtMYslvQW4RzAfq3H2bu3cBjwE3Jzdw91fcfX9i8jVAXT5KWvyubi+bGtpYfMU0IhENPiP5LchCUAnsSpquT8zry2eAFakWmNliM6sxs5qWlpYBjCj56ocvbmVMWQk3zz0j7CgioQuyEKT6muUpG5p9kHgh+FKq5e7+oLtXuXtVRUXFAEaUfPTq1n28snUfi6+cRkmhehkVKQzwd9cDE5OmJwB7ejcys/OAh4CF7r4vwDwiuDvfemYLY4eVqJdRkYQgtwhWAjPNbKqZFQO3A0uTG5jZJOAp4E/c/c0As4gA8OKbLazauZ+/uGamxhwQSQhsi8Dde8zsHqAaKAAedvdaM7s7sXwJ8FVgNPADMwPocfeqoDJJfnN3vv3MFiaOGsQfVk088QNE8kSQu4Zw9+XA8l7zliTd/yzw2SAziBzzmw2NbNjdxrc+dr46lxNJoneD5IXuaIxvPbOF6RVDuGXu8U5eE8k/KgSSF378yg62thzm3oVnUaDrBkTeQ4VAcl5zewffee4trp5VwYfO0lXEIr2pEEjO+/qKzXT1xPiHPzibxEkJIpJEhUBy2qqdrTy1ejefvWIqU8uHhB1HJCOpEEjO6o7G+OrTtYwbVsrnPjgj7DgiGSvQ00dFwvTAC3XU7mljyZ3zGFKiP3WRvmiLQHLS+vqDfP/5Om6ZW8mCc8aHHUcko6kQSM7p6I7yV4+voXxoCf9409lhxxHJeNpelpzzreot1DUf4pFPz2f4oKKw44hkPG0RSE55dmMTD/1uO3deMokrz1SX5SL9oUIgOWNryyH+6r/WcN6E4fzdjb1HRRWRvqgQSE5o7+hm8SM1FBdG+OGdF6qLaZGToGMEkvViMeeLj69lx74j/OQz86kcMSjsSCJZRVsEktXcnb9/egPPbGziK4vO4rLp5WFHEsk6KgSS1b5ZvYVHf/82f3rVND59+dSw44hkJRUCyVr//tJWfvDiVu6YP4l7F8wOO45I1tIxAsk67s73n6/j28++yYfPG88/f+Qc9SoqchpUCCSrRGPOP/2qlkde3cmtcyu5/6PnaaAZkdOkQiBZ42hXlL9+Yi2/Xt/A4iunce+C2URUBEROmwqBZIXtew/zZz9dxebGdr68aDaLr5wediSRnKFCIBlv+foG/uYX6ygsMP7zUxfxwVkablJkIKkQSMZqPdzFfcs28t9v7OaCiSN44I/n6WIxkQCoEEjGcXeeXrOHry3bSHtHN5+/Zgb3XDOT4kKd7SwSBBUCySivbdvH/b/ZzBtvH+CCiSO4/7bzmDWuLOxYIjlNhUBC5+7U7NzP95+v46U3Wxg3rJSv33ouH6uaqFNDRdJAhUBC09UT4ze1jfzH/2xjbf1BRgwu4suLZnPXpVPUe6hIGqkQSFq5O+vqD/LU6nqWrt3D/iPdTCsfwn0fOYfb5lUyuFh/kiLppnedBK6jO0rNjv08t6mJZzc2sfvAUYoLI1w/Zyy3XTiBq2ZW6MIwkRCpEMiAO9TZw+qd+3l9eyuv72hlza4DdPXEKCmMcMXMcj5/7QwWnDNe4wmLZAgVAjllHd1R6vcfpa75EJsb29jc0M6WpnZ27DuMOxREjHPOGMZdl0zmkmmj+cCMcgYVa9+/SKYJtBCY2QLgu0AB8JC7f73XckssXwQcAT7p7quDzCQn5u4c7Y7S0t5JS3snze/cdtDU1snbrUfY1XqExrYO3OOPMYMpo4cwe1wZH7mgknmTRzBv0kiGlOi7hkimC+xdamYFwAPAdUA9sNLMlrr7xqRmC4GZiZ+LgR8mboX4B3I05vTEet/GiMWgJxZ77/yo0x2N0dEdpaMnRmfitqM7Suex6e4oHd3xeYc6e2jv6KGto5u2o920dfTQ3tFN29EeuqKx9+UpiBjlQ4uZOHIwl04fzaRRg5k8ejBTRg9h1rgyHegVyVJBvnPnA3Xuvg3AzB4DbgaSC8HNwCPu7sBrZjbCzMa7e8NAh3lxSzP3LYs/tSf+SXyZxd1xeOfbreO4vzud3IZEu3faJM0jMe/Yc7zvMUnTx57fEw/wpN8bi0E0UQSCEDEoLSpgaEkhwwYVUVZayIjBxUwaPYSy0kKGlRYxfFARFWUl8Z+hJYwZVsLIwcU6r18kBwVZCCqBXUnT9bz/236qNpXAewqBmS0GFgNMmjTplMKUlRYxe9wwSHyOWfz3HpvE7N15x5ZjcKzFu8uPPd7i8975XLS+27z7/3jnd6VeHm8TMaMwYhREErcFx6YjFBgUFETeuzxiFEYiFESgqCBCaVEBpUURSgqTb9+9X1RgGshFRN4RZCFI9UnT+ytuf9rg7g8CDwJUVVWd0tfkCyeP5MLJI0/loSIiOS3IXrzqgYlJ0xOAPafQRkREAhRkIVgJzDSzqWZWDNwOLO3VZilwl8VdAhwM4viAiIj0LbBdQ+7eY2b3ANXETx992N1rzezuxPIlwHLip47WET999FNB5RERkdQCPd/P3ZcT/7BPnrck6b4Dnwsyg4iIHJ9G+hARyXMqBCIieU6FQEQkz6kQiIjkOXMPphuDoJhZC7DzFB9eDuwdwDgDKVOzKdfJydRckLnZlOvknGquye5ekWpB1hWC02FmNe5eFXaOVDI1m3KdnEzNBZmbTblOThC5tGtIRCTPqRCIiOS5fCsED4Yd4DgyNZtynZxMzQWZm025Ts6A58qrYwQiIvJ++bZFICIivagQiIjkuZwrBGb2MTOrNbOYmVX1Wva3ZlZnZlvM7IY+Hj/KzJ41s7cSt4GMZmNm/2VmaxI/O8xsTR/tdpjZ+kS7miCy9Hq+fzSz3UnZFvXRbkFiPdaZ2b1pyPVNM9tsZuvM7L/NbEQf7dKyvk70/090rf69xPJ1ZjYvqCxJzznRzF4ws02J98AXUrS52swOJr2+Xw06V9JzH/e1CWmdzUpaF2vMrM3M/rJXm7SsMzN72MyazWxD0rx+fR6d9vvR3XPqBzgLmAW8CFQlzZ8DrAVKgKnAVqAgxeO/AdybuH8vcH8aMn8b+Gofy3YA5Wlcf/8I/PUJ2hQk1t80oDixXucEnOt6oDBx//6+Xpd0rK/+/P+Jd6++gvgofJcAv0/DazcemJe4Xwa8mSLX1cCydP09ncxrE8Y6S/G6NhK/8Crt6wy4EpgHbEiad8LPo4F4P+bcFoG7b3L3LSkW3Qw85u6d7r6d+BgI8/to9+PE/R8DHwkkaILFBw/+Q+DnQT7PAJsP1Ln7NnfvAh4jvt4C4+7PuHtPYvI14qPZhaU///+bgUc87jVghJmNDzKUuze4++rE/XZgE/ExwLNF2tdZL9cCW939VHsuOC3u/jLQ2mt2fz6PTvv9mHOF4DgqgV1J0/WkfpOM9cQoaYnbMQHnugJocve3+ljuwDNmtsrMFgec5Zh7EpvmD/exKdrfdRmUTxP/5phKOtZXf/7/oa4jM5sCzAV+n2LxpWa21sxWmNnZ6crEiV+bsP+ubqfvL2RhrbP+fB6d9noLdGCaoJjZc8C4FIu+4u5P9/WwFPMCPXe2nznv4PhbAx9w9z1mNgZ41sw2J745BJIL+CFwH/F1cx/x3Vaf7v0rUjz2tNdlf9aXmX0F6AEe7ePXDPj6ShU1xbze//+0/72988RmQ4Engb9097Zei1cT3/VxKHH855fAzHTk4sSvTZjrrBi4CfjbFIvDXGf9cdrrLSsLgbt/6BQeVg9MTJqeAOxJ0a7JzMa7e0Nis7T5VDLCiXOaWSFwK3DhcX7HnsRts5n9N/HNwNP6YOvv+jOz/wssS7Gov+tyQHOZ2SeADwPXemLnaIrfMeDrK4X+/P8DWUcnYmZFxIvAo+7+VO/lyYXB3Zeb2Q/MrNzdA+9crR+vTSjrLGEhsNrdm3ovCHOd0b/Po9Neb/m0a2gpcLuZlZjZVOIV/fU+2n0icf8TQF9bGAPhQ8Bmd69PtdDMhphZ2bH7xA+YbkjVdqD02id7Sx/PtxKYaWZTE9+kbie+3oLMtQD4EnCTux/po0261ld//v9LgbsSZ8JcAhw8tokflMTxpv8ANrn7v/bRZlyiHWY2n/hnwL4gcyWeqz+vTdrXWZI+t8zDWmcJ/fk8Ov33Y9BHwtP9Q/zDqx7oBJqA6qRlXyF+dH0LsDBp/kMkzjACRgO/Bd5K3I4KMOuPgLt7zTsDWJ64P434GQBrgVriu0iCXn8/AdYD6xJ/TON750pMLyJ+VsrWNOWqI74fdE3iZ0mY6yvV/x+4+9jrSXxz/YHE8vUkncEWYKbLie8SWJe0nhb1ynVPYt2sJX7Q/bKgcx3vtQl7nSWedzDxD/bhSfPSvs6IF6IGoDvxGfaZvj6PBvr9qC4mRETyXD7tGhIRkRRUCERE8pwKgYhInlMhEBHJcyoEIiJ5ToVARCTPqRCIiOQ5FQKR02Rmdyf1Vb/dzF4IO5PIydAFZSIDJNHPz/PAN9z9V2HnEekvbRGIDJzvAs+rCEi2ycreR0UyjZl9EphMvF8akayiXUMip8nMLiQ+etQV7r4/7DwiJ0u7hkRO3z3AKOCFxAHjh8IOJHIytEUgIpLntEUgIpLnVAhERPKcCoGISJ5TIRARyXMqBCIieU6FQEQkz6kQiIjkuf8Pbb4/Ok/XRHgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the result\n",
    "\n",
    "plt.plot(z.detach().numpy(),yhat.detach().numpy())\n",
    "plt.xlabel('z')\n",
    "plt.ylabel('yhat')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For custom modules, call the sigmoid from the torch (<code>nn.functional</code> for the old version), which applies the element-wise sigmoid from the function module and plots the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfZElEQVR4nO3deXRb5bnv8e8jecrgjHYGnJmEDIwJJkBLmYeEtlDocKHt6XxT2nLac1bPOqXtOe255a57S4e7OtGmXA4daTlQaElp0gBlai9Q4oRMTmJwJuJ4TJxgZ/AkPfcPKUEYOVZiyVuSf5+1HO3hlfR4S/pl+9Xe+zV3R0REcl8o6AJERCQ9FOgiInlCgS4ikicU6CIieUKBLiKSJwqCeuKysjKfMWNGUE8vIpKT1q5du8/dy5OtCyzQZ8yYQVVVVVBPLyKSk8xsd1/r1OUiIpInFOgiInlCgS4ikicU6CIieUKBLiKSJ/oNdDO7z8yazWxzH+vNzH5gZrVmttHMFqW/TBER6U8qe+g/B5acYP1SYE78Zxnwk4GXJSIiJ6vf49Dd/Tkzm3GCJjcCv/TYdXhfNLMxZjbZ3RvSVaSI5C93pysSpaM7Smd3hM6eKJGo0xON0hN1eiJOT9SJRKN0Rzy+zumJROPLne5IlKg77uBObDr+2LF5cJxobCEORKPxW3+jnfPm9p6wDsDfVHfCdMKaNy9PfofKGeO49Iyk5wYNSDpOLKoA9iTM18WXvSXQzWwZsb14pk2bloanFpEguTuHuyI0t3XQ0t7J/sNdtB3tpq2jm7ajPfHbbto7YtOHOiN0dkfo6I7Q0RON3XZHYkE7BJjFbm+77PSsDXRLsizpy+Pu9wD3AFRWVg6Rl1Akd7k7ze2d7Np3mNdaj7Cn9Qi7W4+w98BRWg510tzWydHuSNL7hgxKSwoZNayAUSWFjCoppGJMIcWFYYYVhikpDFFSEKbk2HRhbLqoIERh2AiHQhSGjHDIKAgbBaEQBcfnY9PHlofjyw0ImWFG/McIGRgJy4gvS1wXigXZidofY/bGTGL4WR9tBlM6Ar0OmJowPwWoT8PjisggikadmqZ2NtW9ztbGNrY1tLOtsY0DR7qPtwkZnDZmGBVjhnHulDGUlxYzobQ4flvC+JFFjB5WyKhhhYwoCgcWbENVOgJ9BXC7mT0AXAi8rv5zkewXiTob6w7y0s5WXtrZyppdrbR19AAwrDDM3EmlLDlrMvMmlTKzbATTxg2nYuwwCsM62jlb9RvoZvZb4HKgzMzqgK8DhQDuvhxYCVwP1AJHgI9nqlgRGZiO7gjP1LTw5NYmntrWTOvhLgBmlY3g+rMns3jmOBZOG8v0ccMJhbR3nWtSOcrl1n7WO/C5tFUkImnl7lTtPsAj6+p4bGMD7R09jCop4Mp5E7hy/kQunjWe8tLioMuUNAjs8rkiklmdPREeXV/PfX/bybbGdoYXhVly1iRuWljBRbPGq+skDynQRfJMR3eEX72wm58+t4N9hzqZN6mUb733HN55zmRGFOsjn8/06orkiUjUeXhdHd974hXqX+/gktll3HbZebx99ngdbTJEKNBF8sCW+jbueGQjG+te59wpo/nO+8/lbbPLgi5LBpkCXSSHdXRH+MFfXuWnz+1g7PBCvn/Ledxw7mnaIx+iFOgiOWpHyyE+e/86tjW284HKKXzl+vmMGV4UdFkSIAW6SA7608YGvvTwRgrDxs8+fgFXzJ0QdEmSBRToIjkkGnW+tbqG5c9uZ+G0Mdz9wUWcNmZY0GVJllCgi+SI7kiUL/1uI4+8vJcPXTiNr7/7TIoKdCy5vEGBLpIDjnT18Nn71/FMTQtfvOYMbr9ytr74lLdQoItkuaNdET72szVU7Wrlf998Nrcu1lgCkpwCXSSLdfVE+ez9a1mzq5Xv37KQG849LeiSJIupA04kS0Wizhcf2sDTNS38r5vOVphLvxToIlnqzse28McN9dyxdJ66WSQlCnSRLPTgmj38/PldfPKSmdx22elBlyM5QoEukmXW7znIv/1hM5fMLuPLS+cFXY7kEAW6SBbZd6iTz/x6LRNGFfPDWxdSoGuWy0nQUS4iWSIadT7/25dpPdzFw595G2NH6LoscnIU6CJZ4ufP7+L57fv55s1nc1bF6KDLkRykv+dEskBt8yHu+vM2rpw3gf92wdSgy5EcpUAXCVhPJMoXH9rA8KIw33zv2TqlX06ZulxEAvaTZ7azYc9B7v7gIiaUlgRdjuQw7aGLBGjXvsP88Kla3nXOZN55zuSgy5Ecp0AXCdA3HttCUUGIr71rQdClSB5QoIsE5C9bm3hqWzNfuGoOE0apq0UGToEuEoCO7gjfeGwLsyeM5GNvnxF0OZIn9KWoSADu/esOdu8/wq8/eSGFOhtU0kTvJJFB1tLeyd1Pb2fJmZO4ZE5Z0OVIHlGgiwyyHz9TS1ckyr8umRt0KZJnFOgig6j+4FHuf/E13ruoglnlI4MuR/KMAl1kEP3wqVoc5/NXzQm6FMlDKQW6mS0xsxozqzWzO5KsH21mfzSzDWZWbWYfT3+pIrlt9/7DPFS1h1sXT2PK2OFBlyN5qN9AN7MwcDewFFgA3Gpmvc+C+Bywxd3PBS4HvmtmuvanSILvPfkqBWHj9itmB12K5KlU9tAXA7XuvsPdu4AHgBt7tXGg1GJXFRoJtAI9aa1UJIft2neYR9fv5SMXz9BJRJIxqQR6BbAnYb4uvizRj4D5QD2wCfiCu0d7P5CZLTOzKjOramlpOcWSRXLPvX/bQUEoxKcumRl0KZLHUgn0ZNfy9F7z1wHrgdOA84Afmdmot9zJ/R53r3T3yvLy8pMsVSQ37T/UyUNVddy0sEJ755JRqQR6HZB4xf0pxPbEE30ceMRjaoGdgEa3FQF+8cJuOnui/PdLZwVdiuS5VAJ9DTDHzGbGv+i8BVjRq81rwFUAZjYRmAvsSGehIrnoSFcPv3xhF9csmMjsCTruXDKr32u5uHuPmd0OrAbCwH3uXm1mt8XXLwfuBH5uZpuIddF8yd33ZbBukZzwUFUdB49082ntncsgSOniXO6+EljZa9nyhOl64Nr0liaS2yJR596/7eD86WOpnDEu6HJkCNCZoiIZ8tS2Zva0HtWRLTJoFOgiGfKrF3czcVQx1yyYGHQpMkQo0EUyYPf+wzz3SgsfXDydAl3vXAaJ3mkiGXD/31+jIGTcsnhq/41F0kSBLpJmHd0RHqzaw7VnTmSiTiSSQaRAF0mzxzY2cPBINx++aHrQpcgQo0AXSbNfv7ib08tHcPGs8UGXIkOMAl0kjbbUt7F+z0E+fNF0YhcfFRk8CnSRNPrd2jqKwiHec17vC5KKZJ4CXSRNunqi/GH9Xq5eMIGxIzS+iww+BbpImjxd00zr4S7ef74OVZRgKNBF0uShqjrKS4t5x5yyoEuRIUqBLpIGLe2dPF3TzM0LK3RmqARG7zyRNHh0/V4iUed9508JuhQZwhToIgPk7vxubR3nTh3DnImlQZcjQ5gCXWSAtjS0sa2xXXvnEjgFusgArVhfT0HIeNfZk4MuRYY4BbrIAESjzooN9Vx6RrmOPZfAKdBFBqBq9wEaXu/ghnNPC7oUEQW6yECs2LCXksKQRiWSrKBAFzlF3ZEof9rYwNXzJzKiOKXx1kUySoEucor+VruPA0e61d0iWUOBLnKKVqyvZ1RJAZfNLQ+6FBFAgS5ySo52RXi8upGlZ02muCAcdDkigAJd5JQ8+0ozh7sivFvdLZJFFOgip2B1dROjhxVy4axxQZcicpwCXeQkdUei/GVrE1fNn0ChrqwoWUTvRpGT9OKO/bR19HDdmZOCLkXkTRToIidpdXUjJYUhLp2jo1skuyjQRU5CNOo8Xt3EZWeUM6xIR7dIdkkp0M1siZnVmFmtmd3RR5vLzWy9mVWb2bPpLVMkO6yvO0hze6e6WyQr9Xu+spmFgbuBa4A6YI2ZrXD3LQltxgA/Bpa4+2tmNiFD9YoEanV1IwUh46p5unaLZJ9U9tAXA7XuvsPdu4AHgBt7tfkg8Ii7vwbg7s3pLVMkeO6x7paLZo1n9PDCoMsReYtUAr0C2JMwXxdflugMYKyZPWNma83sI8keyMyWmVmVmVW1tLScWsUiAaltPsTOfYe57kztnUt2SiXQLcky7zVfAJwPvBO4Dvh3MzvjLXdyv8fdK929srxcRwhIblld3QjANQvUfy7ZKZVrftYBUxPmpwD1Sdrsc/fDwGEzew44F3glLVWKZIHV1U2cN3UMk0aXBF2KSFKp7KGvAeaY2UwzKwJuAVb0avMo8A4zKzCz4cCFwNb0lioSnL0Hj7Jp7+s6ukWyWr976O7eY2a3A6uBMHCfu1eb2W3x9cvdfauZ/RnYCESBe919cyYLFxlMj8e7W9R/LtkspWFW3H0lsLLXsuW95r8NfDt9pYlkj9XVjcyZMJJZ5SODLkWkTzpTVKQfrYe7eGlnq7pbJOsp0EX68eTWJqKOAl2yngJdpB+PVzdSMWYYZ1WMCroUkRNSoIucwOHOHp57dR/XLJiIWbJTMkSyhwJd5ASefaWFrp6oulskJyjQRU5gdXUjY4cXcsGMsUGXItIvBbpIH7p6ojy1rZmr50+kQEPNSQ7Qu1SkDy/s2E+7hpqTHKJAF+nD6upGhheFuWROWdCliKREgS6SRDTqPLGlicvnllNSqKHmJDco0EWSeHnPAVo01JzkGAW6SBKrq5soDBtXzNNoipI7FOgivbg7q6sbufj0MkaVaKg5yR0KdJFeapra2b3/iC6VKzlHgS7Sy+rNTZjBNQsU6JJbFOgivayubmTRtLFMKNVQc5JbFOgiCfa0HmFLQ5u6WyQnKdBFEqw+PtScDleU3KNAF0nweHUT8yaVMn38iKBLETlpCnSRuH2HOlmzu5VrtXcuOUqBLhL35JYm3FH/ueQsBbpI3Or4UHMLJmuoOclNCnQRoL2jm/9Xu5/rzpykoeYkZynQRYBnalroikTV3SI5TYEuQqy7ZfyIIipnjAu6FJFTpkCXIa+zJ8IzNS1cPX8i4ZC6WyR3KdBlyHu+dj+HOnu47ix1t0huU6DLkLe6upERRWHedrqGmpPcpkCXIS0SdZ7c2sTl8yZoqDnJeQp0GdLW7Gpl36EulujsUMkDCnQZ0v68uZHighBXaqg5yQMpBbqZLTGzGjOrNbM7TtDuAjOLmNn70leiSGZEo86qzQ1cdkY5I4oLgi5HZMD6DXQzCwN3A0uBBcCtZragj3Z3AavTXaRIJry85yBNbZ0sPVvdLZIfUtlDXwzUuvsOd+8CHgBuTNLuH4GHgeY01ieSMas2NVAYNq6ar8MVJT+kEugVwJ6E+br4suPMrAK4CVh+ogcys2VmVmVmVS0tLSdbq0jauDurNjdyyewyRpUUBl2OSFqkEujJTp3zXvPfA77k7pETPZC73+Pule5eWV5enmKJIum3eW8bew8eZenZk4MuRSRtUvkmqA6YmjA/Bajv1aYSeCB+lboy4Hoz63H3P6SjSJF0W7m5gXDIuEbdLZJHUgn0NcAcM5sJ7AVuAT6Y2MDdZx6bNrOfA48pzCVbuTurNjXwttPHM3ZEUdDliKRNv10u7t4D3E7s6JWtwIPuXm1mt5nZbZkuUCTdtjW2s2v/EZacpaNbJL+kdPCtu68EVvZalvQLUHf/2MDLEsmcVZsbCRlcu0CBLvlFZ4rKkLNqUwMXzBhHeWlx0KWIpJUCXYaU2uZ2Xm0+xPU6ukXykAJdhpRVmxoBuE4X45I8pECXIcPdWbGhnsrpY5k0uiTockTSToEuQ8a2xlh3y43nnRZ0KSIZoUCXIWPFhnrCIVP/ueQtBboMCe7OHzfUc8nsMsaP1NEtkp8U6DIkrHvtIHUHjnLDuepukfylQJchYcX6vRQXhLj2TF27RfKXAl3yXk8kyp82NXDV/AmU6lK5kscU6JL3Xtixn32HutTdInlPgS557w8v11NaXMDlczUQtOQ3BbrktcOdPaza3MA7z5lMSWE46HJEMkqBLnlt5aYGjnRFeN/5U4IuRSTjFOiS1x5aW8fMshGcP31s0KWIZJwCXfLW7v2HeWlnK+87fwrx4RFF8poCXfLWw2vrMIObF1UEXYrIoFCgS16KRp2H1+3lktllTB49LOhyRAaFAl3y0gs79rP34FF9GSpDigJd8tKDVXsoLSnQQBYypCjQJe/sP9TJqk2N3LSwQseey5CiQJe882BVHV2RKB++aHrQpYgMKgW65JVI1PnNS7u5cOY4zphYGnQ5IoNKgS555blXWtjTepR/uFh75zL0KNAlr/zqxd2UlxZz7QJ9GSpDjwJd8sae1iM8XdPMrRdMpahAb20ZevSul7zxm5dew4BbFk8LuhSRQCjQJS8c7uzhN39/jWsWTOS0MTozVIYmBbrkhQer9vD60W6WXXp60KWIBEaBLjmvOxLl3r/u5IIZY3WZXBnSUgp0M1tiZjVmVmtmdyRZ/yEz2xj/ed7Mzk1/qSLJrdzUwN6DR/m09s5liOs30M0sDNwNLAUWALea2YJezXYCl7n7OcCdwD3pLlQkGXdn+bM7mD1hJFfO05ihMrSlsoe+GKh19x3u3gU8ANyY2MDdn3f3A/HZFwFd4k4Gxd9q97G1oY1l75hFKKRBLGRoSyXQK4A9CfN18WV9+SSwKtkKM1tmZlVmVtXS0pJ6lSJ9+Mkz25lQWsyNC08LuhSRwKUS6Ml2ezxpQ7MriAX6l5Ktd/d73L3S3SvLy8tTr1IkiRe27+f57ftZduksigt0VUWRghTa1AFTE+anAPW9G5nZOcC9wFJ335+e8kSSc3e+83gNE0cV66qKInGp7KGvAeaY2UwzKwJuAVYkNjCzacAjwD+4+yvpL1PkzZ55pYW1uw/wj1fO0TXPReL63UN39x4zux1YDYSB+9y92sxui69fDnwNGA/8OD66eo+7V2aubBnK3J3vPl7D1HHD+EDl1P7vIDJEpNLlgruvBFb2WrY8YfpTwKfSW5pIcn/e3MjmvW185/3n6iJcIgn0aZCc0h2J8p3Hazi9fAQ3LTzRwVYiQ48CXXLKL57fxfaWw9yxdD5hHXcu8iYKdMkZze0dfO/JV7l8bjlXz9dZoSK9KdAlZ3xz1Ta6eqJ8/d1nEv/yXUQSKNAlJ6zd3coj6/byqXfMZGbZiKDLEclKCnTJet2RKF97tJpJo0r43BWzgy5HJGuldNiiSJDufrqW6vo2ln94ESOK9ZYV6Yv20CWrbap7nR89VctNCytYctbkoMsRyWoKdMlaHd0R/vnB9ZSNLOY/bjgz6HJEsp7+fpWs9Z3VNdQ2H+KXn1jM6GGFQZcjkvW0hy5Z6YktTdz7t518+KJpXHqGLrUskgoFumSd7S2H+Of/Ws85U0bzb+/sPdqhiPRFgS5Zpb2jm2W/rKKoIMRPPny+Lo0rchLUhy5ZIxp1vvjgBnbtP8KvPrmYijHDgi5JJKdoD12ygrvz749u5vEtTXz1+vm87fSyoEsSyTkKdMkK315dw/1/f41PXzaLT1wyM+hyRHKSAl0C99Nnt/PjZ7Zz6+Jp3LFkXtDliOQs9aFLYNydHz1Vy3efeIV3nTOZ//mes3QVRZEBUKBLICJR53/8sZpfvrCbmxdWcNf7ztGAFSIDpECXQXe0K8K/PLSBP21qYNmls7hjyTxCCnORAVOgy6Daue8wn/n1WrY1tvOV6+ex7NLTgy5JJG8o0GXQrNzUwL/+biMFYeNnH7+AK+ZqGDmRdFKgS8a1Hu7izse28PuX93Le1DHc/aFFOmlIJAMU6JIx7s6j6+v5xmNbaO/o5vNXzub2K+dQVKCjZUUyQYEuGfHijv3c9edtvPzaQc6bOoa73nsOcyeVBl2WSF5ToEvauDtVuw/wo6dqefaVFiaNKuGbN5/N+yun6pBEkUGgQJcB6+qJ8ufqRv7zrzvYUPc6Y4YX8pXr5/GRi2foaokig0iBLqfE3dlY9zqPrKtjxYZ6DhzpZlbZCO58z1m8d1EFw4v01hIZbPrUSco6uiNU7TrAk1ubeGJLE3sPHqWoIMS1Cyby3vOncNmccp0gJBIgBbr06VBnD+t2H+Clna28tKuV9XsO0tUTpbggxDvmlPH5q2az5KzJGu9TJEso0IWO7gh1B45S23yIbY1tbGtop6apnV37D+MO4ZBx1mmj+MhF07lo1njePruMYUXqGxfJNikFupktAb4PhIF73f2bvdZbfP31wBHgY+6+Ls21yklyd452R2hp76SlvZPm47cdNLV18lrrEfa0HqGxrQP32H3MYMb4EcybVMp7zqtg0fQxLJo2lhHF+r9fJNv1+yk1szBwN3ANUAesMbMV7r4lodlSYE7850LgJ/FbIRaskajTE+19GyUahZ5o9M3LI053JEpHd4SOniid8duO7gidx+a7I3R0x5Yd6uyhvaOHto5u2o5209bRQ3tHN21He+iKRN9STzhklI0sYurY4Vx8+nimjRvO9PHDmTF+BHMnleoLTZEclcondzFQ6+47AMzsAeBGIDHQbwR+6e4OvGhmY8xssrs3pLvgZ2qaufOx2FN7/J/4ziXujsPxvU3HcX9jPrEN8XbH2yQsI77s2HO85T4J88ee3+N38ITHjUYhEg/zTAgZlBSGGVlcwKhhhZSWFDBmeBHTxo+gtKSAUSWFjB5WSHlpcexnZDETRhUzdniRjgsXyUOpBHoFsCdhvo637n0na1MBvCnQzWwZsAxg2rRpJ1srAKUlhcybNArieWSxxz02i9kby46tx+BYizfWH7u/xZYdzzfru80bv8fxx0q+PtYmZEZByAiH4rfhY/MhwgbhcOjN60NGQShEOASF4RAlhWFKCkMUFyTevjFdGDYNCCEix6US6MkSo/cuZyptcPd7gHsAKisrT2m39fzpYzl/+thTuauISF5L5SpJdcDUhPkpQP0ptBERkQxKJdDXAHPMbKaZFQG3ACt6tVkBfMRiLgJez0T/uYiI9K3fLhd37zGz24HVxA5bvM/dq83stvj65cBKYocs1hI7bPHjmStZRESSSen4NHdfSSy0E5ctT5h24HPpLU1ERE6GRhoQEckTCnQRkTyhQBcRyRMKdBGRPGHumTktvd8nNmsBdp/i3cuAfWksJ52ytTbVdXKytS7I3tpU18k51bqmu3t5shWBBfpAmFmVu1cGXUcy2Vqb6jo52VoXZG9tquvkZKIudbmIiOQJBbqISJ7I1UC/J+gCTiBba1NdJydb64LsrU11nZy015WTfegiIvJWubqHLiIivSjQRUTyRNYGupm938yqzSxqZpW91n3ZzGrNrMbMruvj/uPM7AkzezV+m5FRMczsv8xsffxnl5mt76PdLjPbFG9XlYlaej3ff5jZ3oTaru+j3ZL4dqw1szsGoa5vm9k2M9toZr83szF9tBuU7dXf7x+/JPQP4us3mtmiTNWS8JxTzexpM9sa/wx8IUmby83s9YTX92uZrivhuU/42gS0zeYmbIv1ZtZmZv/Uq82gbDMzu8/Mms1sc8KylPJowJ9Hd8/KH2A+MBd4BqhMWL4A2AAUAzOB7UA4yf2/BdwRn74DuGsQav4u8LU+1u0CygZx+/0H8C/9tAnHt98soCi+XRdkuK5rgYL49F19vS6Dsb1S+f2JXRZ6FbFRuS4C/j4Ir91kYFF8uhR4JUldlwOPDdb76WRemyC2WZLXtZHYCTiDvs2AS4FFwOaEZf3mUTo+j1m7h+7uW929JsmqG4EH3L3T3XcSuwb74j7a/SI+/QvgPRkpNM5ig3t+APhtJp8nzY4PAO7uXcCxAcAzxt0fd/ee+OyLxEa3Ckoqv//xAdDd/UVgjJlNzmRR7t7g7uvi0+3AVmJj9OaKQd9mvVwFbHf3Uz0TfUDc/TmgtdfiVPJowJ/HrA30E+hrQOreJnp81KT47YQM1/UOoMndX+1jvQOPm9laiw2WPRhuj//Je18ff+Klui0z5RPE9uSSGYztlcrvH+g2MrMZwELg70lWX2xmG8xslZmdOVg10f9rE/T76hb63rEKapulkkcD3m4pDXCRKWb2JDApyaqvuvujfd0tybKMHnuZYp23cuK987e7e72ZTQCeMLNt8f/JM1IX8BPgTmLb5k5i3UGf6P0QSe474G2ZyvYys68CPcD9fTxM2rdXslKTLDulAdAzwcxGAg8D/+Tubb1WryPWpXAo/v3IH4A5g1EX/b82QW6zIuAG4MtJVge5zVIx4O0WaKC7+9WncLdUB6RuMrPJ7t4Q/3Ov+VRqhP7rNLMC4Gbg/BM8Rn38ttnMfk/sz6sBBVSq28/M/i/wWJJVGRncO4Xt9VHgXcBVHu88TPIYad9eSWTtAOhmVkgszO9390d6r08MeHdfaWY/NrMyd8/4RahSeG2CHDR+KbDO3Zt6rwhym5FaHg14u+Vil8sK4BYzKzazmcT+h32pj3YfjU9/FOhrjz8drga2uXtdspVmNsLMSo9NE/ticHOytunSq8/ypj6eL5UBwNNd1xLgS8AN7n6kjzaDtb2ycgD0+Pcx/wlsdff/00ebSfF2mNliYp/l/ZmsK/5cqbw2QQ4a3+dfykFts7hU8mjgn8dMf+N7qj/EQqgO6ASagNUJ675K7NvgGmBpwvJ7iR8RA4wH/gK8Gr8dl8Fafw7c1mvZacDK+PQsYt9YbwCqiXU9ZHr7/QrYBGyMvykm964rPn89saMotg9SXbXE+gnXx3+WB7m9kv3+wG3HXk9ifwbfHV+/iYQjrjJY0yXE/tTemLCdru9V1+3xbbOB2JfLb8t0XSd6bYLeZvHnHU4soEcnLBv0bUbsP5QGoDueYZ/sK4/S/XnUqf8iInkiF7tcREQkCQW6iEieUKCLiOQJBbqISJ5QoIuI5AkFuohInlCgi4jkif8PlHdUgccBKDwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Use the build in function to predict the result\n",
    "\n",
    "yhat = torch.sigmoid(z)\n",
    "plt.plot(z.numpy(), yhat.numpy())\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Tanh\">Tanh</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you use sequential, you can create a tanh object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a tanh object\n",
    "\n",
    "TANH = nn.Tanh()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Call the object and plot it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgjElEQVR4nO3deZBc9Xnu8e8zm/Z9RzsgC4RZLMbCQGIbGwgQJzKpJBfi65AYX4WUqcS5yS2T8i1fV7luyrErScUJgcgJZZzYJtexiRVbNmDZjuNgMCMihIQkNCMEGjTSjPYZbaOZee8ffUSapmdTL6d7+vlUdfU55/c73e+cXp45S5+jiMDMzGpXXdoFmJlZuhwEZmY1zkFgZlbjHARmZjXOQWBmVuMa0i7gQsyePTuWLVuWdhlmZlVl8+bNhyJiTu70qgyCZcuW0dLSknYZZmZVRdKr+aZ705CZWY1zEJiZ1TgHgZlZjXMQmJnVOAeBmVmNK0oQSHpEUqekbYO0S9IXJLVK2ippdVbbbZJ2JW0PFKMeMzMbuWKtEXwJuG2I9tuBFcltHfAQgKR64MGkfRVwt6RVRarJzMxGoCi/I4iIH0taNkSXtcCXI3PO62ckTZe0AFgGtEbEHgBJjyV9XypGXWY2cr19A5w820fP2T66z/RxsreP3r4BzvUP0Ncf9A0M0Nsf9CXj5wYy9/0DQQDnT2kfAUEk928eJ+mXr60iT4hfgafpv3P1IpbPnlTUxyzXD8oWAvuyxtuTafmmX5fvASStI7M2wZIlS0pTpdkYdeZcP62dPew60M2+o6c4eOIMB46f4cCJs3R1n+HEmcyXvr2VlHYFb7Z66YyqDYJ8izKGmP7WiRHrgfUAzc3NlRfTZhXk2Kleftp2mKfbDvPMnsO0dfUwkPWpmT25iXlTx3PRtPFcs3gaUyc0MmVcA5PGNTB5XANTxmeGm+rraKivo7FeNNTV0dSQuW+oF431dTTUifo6IQTKfGkKkJTcg9AbX6bZ42/pV2nfuDWkXEHQDizOGl8E7AeaBpluZqN0rn+AH+7s5Oub2/nhzk76BoKJTfW8c9lM7rhyAZfNn8KKeVNYMnMiTQ0+YND+S7mCYANwf7IP4DrgeER0SOoCVkhaDrwO3AX8RplqMhsTzpzr5yvPvsbf/lsbnd1nmT15HPf+3HJuvWI+Vy2aRmO9v/RtaEUJAklfA94LzJbUDvwfoBEgIh4GNgJ3AK3AKeC3k7Y+SfcDTwD1wCMRsb0YNZnVgh/u7ORTG7ax78hprr94Fn9y55W8Z+Ucf/nbqBTrqKG7h2kP4GODtG0kExRmNkJnzvXzmW+/xFeefY1L507mKx+9jhsvnZ12WValqvI01Ga17FDPWe790nO80H6c33n3xfzhrSu9zd8K4iAwqyL7j53m7i8+w8ETZ1j/4Wu59Yr5aZdkY4CDwKxKHOo5y3//+2c50tPLV//Hu1i9ZEbaJdkY4SAwqwJn+/r56KMt7D92mn+49zqHgBWVg8CsCnzm2y+xZd8xHvrQat65bGba5dgY4z1MZhXue9s6+MdnXuN33nMxt1+5IO1ybAxyEJhVsKMne/nf/7Kdty+cyh/dujLtcmyM8qYhswr2fzfu4NipXr78kTX+kZiVjN9ZZhVq2+vH+efN7dz788tZddHUtMuxMcxBYFaBIoI/2biDmZOa+NhNl6Zdjo1xDgKzCvST1kM83XaY33vfpUwd35h2OTbGOQjMKtDf/LCN+VPH8xvXLU27FKsBDgKzCrNl3zF+uucw9/7ccp9DyMrC7zKzCvO3/9bG1PEN3H2dL8lq5eEgMKsgB0+c4cmXDnL3dUuYPM5Hd1t5OAjMKsjXW/bRPxDc/U6vDVj5FCUIJN0maZekVkkP5Gn/X5K2JLdtkvolzUza9kp6MWlrKUY9ZtVoYCD42s/2ceOls1g2e1La5VgNKTgIJNUDDwK3A6uAuyWtyu4TEZ+PiGsi4hrgj4F/i4gjWV1uStqbC63HrFr9pPUQrx87zd1rvDZg5VWMNYI1QGtE7ImIXuAxYO0Q/e8GvlaE5zUbUza8sJ8p4xu4ZdW8tEuxGlOMIFgI7Msab0+mvYWkicBtwDeyJgfwpKTNktYN9iSS1klqkdTS1dVVhLLNKsfZvn6e2H6AX7hiPuMa6tMux2pMMYJAeabFIH1/CfiPnM1CN0bEajKblj4m6d35ZoyI9RHRHBHNc+bMKaxiswrz45cP0X2mjw9c5dNMW/kVIwjagcVZ44uA/YP0vYuczUIRsT+57wQeJ7Opyaym/OsL+5kxsZEbL52ddilWg4oRBM8BKyQtl9RE5st+Q24nSdOA9wDfypo2SdKU88PArcC2ItRkVjV6+wbYtOMgv3DFfJ9q2lJR8C9WIqJP0v3AE0A98EhEbJd0X9L+cNL1TuDJiDiZNfs84HFJ52v5akR8r9CazKrJz145wsnefm6+3DuJLR1F+eliRGwENuZMezhn/EvAl3Km7QGuLkYNZtVq086DjGuo82YhS43XQ81SFBH8YGcnN1wyiwlNPlrI0uEgMEvRnkMnefXwKd532dy0S7Ea5iAwS9GPdmV+E3OTg8BS5CAwS9HTrYdYPnsSi2ZMTLsUq2EOArOU9PUP8OwrR7j+kllpl2I1zkFglpKtrx+n52wfN17io4UsXQ4Cs5T8tO0wAO+6eGbKlVitcxCYpeTptkNcNn8KsyaPS7sUq3EOArMU9PYN0LL3KDd4s5BVAAeBWQq27z/O2b4B3rlsRtqlmDkIzNKw+dWjAFy71EFg6XMQmKWgZe9RFs+cwNyp49MuxcxBYFZuEUHLq0dpXuqjhawyOAjMyuy1I6c41HPWm4WsYjgIzMqsZW9m/0CzdxRbhXAQmJXZln3HmDyugRVzp6RdihlQpCCQdJukXZJaJT2Qp/29ko5L2pLcPjXSec3Gmq3tx3j7wqnU1yntUsyAIlyhTFI98CBwC5kL2T8naUNEvJTT9d8j4gMXOK/ZmNDbN8COjm5++8ZlaZdi9oZirBGsAVojYk9E9AKPAWvLMK9Z1dl1oJve/gGuXDQt7VLM3lCMIFgI7Msab0+m5bpe0guSvivpilHOi6R1kloktXR1dRWhbLPye6H9GABXL5qeah1m2YoRBPk2dEbO+PPA0oi4Gvgr4F9GMW9mYsT6iGiOiOY5c+ZcaK1mqdrafowZExtZNGNC2qWYvaEYQdAOLM4aXwTsz+4QEScioicZ3gg0Spo9knnNxpKt7ce5atF0JO8otspRjCB4DlghabmkJuAuYEN2B0nzlbzzJa1JnvfwSOY1GyvOnOtnd2cPV3n/gFWYgo8aiog+SfcDTwD1wCMRsV3SfUn7w8CvAr8rqQ84DdwVEQHknbfQmswq0e6DPfQPBKsWTE27FLM3KTgI4I3NPRtzpj2cNfzXwF+PdF6zsWhHxwkALnMQWIXxL4vNymTHgRNMaKxnycyJaZdi9iYOArMy2dnRzcr5U/yLYqs4DgKzMogIdhw4weULfH4hqzwOArMyOHjiLMdOneNy7x+wCuQgMCuDHQeSHcXzHQRWeRwEZmVw/oihlfO9acgqj4PArAx2dnSzcPoEpk1oTLsUs7dwEJiVwY4O7yi2yuUgMCuxM+f62XPopHcUW8VyEJiVWGtn5tQS3lFslcpBYFZi53cUe9OQVSoHgVmJ7TzQzfjGOpbOmpR2KWZ5OQjMSmxHxwlWzvOpJaxyOQjMSigikiOGvH/AKpeDwKyEDvX0cvTUOf+QzCpaUYJA0m2SdklqlfRAnvYPSdqa3J6WdHVW215JL0raIqmlGPWYVYrWzh4ALp07OeVKzAZX8IVpJNUDDwK3kLkG8XOSNkTES1ndXgHeExFHJd0OrAeuy2q/KSIOFVqLWaVp63IQWOUrxhrBGqA1IvZERC/wGLA2u0NEPB0RR5PRZ8hcpN5szGvt7GFSUz3zp45PuxSzQRUjCBYC+7LG25Npg7kX+G7WeABPStosad1gM0laJ6lFUktXV1dBBZuVS1tXD5fMnYzkI4aschUjCPK9wyNvR+kmMkHwiazJN0bEauB24GOS3p1v3ohYHxHNEdE8Z86cQms2K4u2zh4umePNQlbZihEE7cDirPFFwP7cTpKuAv4OWBsRh89Pj4j9yX0n8DiZTU1mVe/k2T72Hz/j/QNW8YoRBM8BKyQtl9QE3AVsyO4gaQnwTeDDEfFy1vRJkqacHwZuBbYVoSaz1O3pOgnAJXP8i2KrbAUfNRQRfZLuB54A6oFHImK7pPuS9oeBTwGzgL9JtpX2RUQzMA94PJnWAHw1Ir5XaE1mlaC1qxvwEUNW+QoOAoCI2AhszJn2cNbwR4GP5plvD3B17nSzsaCt8yT1dWLJTK8RWGXzL4vNSqS1s4elsybS1OCPmVU2v0PNSqSty0cMWXVwEJiVQF//AHsPn/T+AasKDgKzEnjtyCnO9YfXCKwqOAjMSsAnm7Nq4iAwK4E2/4bAqoiDwKwEWjt7mDd1HFPGN6ZditmwHARmJdDa1ePNQlY1HARmRRYR7PHJ5qyKOAjMiqyz+yzdZ/u8RmBVw0FgVmRtyRFDXiOwauEgMCuyVl+e0qqMg8CsyNo6e5g8roG5U8alXYrZiDgIzIqs1ZentCrjIDArsrbOk/4hmVUVB4FZEXWfOceBE748pVWXogSBpNsk7ZLUKumBPO2S9IWkfauk1SOd16ya/NflKR0EVj0KDgJJ9cCDwO3AKuBuSatyut0OrEhu64CHRjGvWdXwyeasGhVjjWAN0BoReyKiF3gMWJvTZy3w5ch4BpguacEI5zWrGm1dPTTUiSUzJ6ZditmIFSMIFgL7ssbbk2kj6TOSeQGQtE5Si6SWrq6ugos2K4XWzh6WzZ5EY713v1n1KMa7Nd8xcjHCPiOZNzMxYn1ENEdE85w5c0ZZoll5tHb1+IghqzrFCIJ2YHHW+CJg/wj7jGRes6pwrn+A1w6f8v4BqzrFCILngBWSlktqAu4CNuT02QD8ZnL00LuA4xHRMcJ5zarCq4dP0jfgy1Na9Wko9AEiok/S/cATQD3wSERsl3Rf0v4wsBG4A2gFTgG/PdS8hdZklobWzsyho14jsGpTcBAARMRGMl/22dMezhoO4GMjndesGrUlJ5u72GsEVmV8aINZkbR19rBg2ngmjyvK/1dmZeMgMCuSzBFDXhuw6uMgMCuCiKCt09cpturkIDArggMnznCyt9+/IbCq5CAwK4K25IihS7xGYFXIQWBWBK2d3YAPHbXq5CAwK4K2rpNMGd/AnMm+PKVVHweBWRG0JjuKfXlKq0YOArMi8KGjVs0cBGYFOn76HF3dZ71/wKqWg8CsQOdPLeE1AqtWDgKzArX58pRW5RwEZgVq7eqhqb6OxTMmpF2K2QVxEJgVqK2zh6WzJtLgy1NalfI716xALx/s4W3zpqRdhtkFcxCYFeBUbx/7jp5yEFhVKygIJM2U9JSk3cn9jDx9Fkv6oaQdkrZL+v2stk9Lel3SluR2RyH1mJVba2cPEfC2ed5RbNWr0DWCB4BNEbEC2JSM5+oD/jAiLgfeBXxM0qqs9r+IiGuSm69UZlXl5YOZI4beNt9rBFa9Cg2CtcCjyfCjwAdzO0RER0Q8nwx3AzuAhQU+r1lF2H2wm6b6OpbOnJh2KWYXrNAgmBcRHZD5wgfmDtVZ0jLgHcCzWZPvl7RV0iP5Ni1lzbtOUouklq6urgLLNiuOXQe7uWTuZB8xZFVt2HevpO9L2pbntnY0TyRpMvAN4OMRcSKZ/BBwCXAN0AH82WDzR8T6iGiOiOY5c+aM5qnNSmb3wR7vH7CqN+xVtiPi5sHaJB2UtCAiOiQtADoH6ddIJgS+EhHfzHrsg1l9vgh8ezTFm6Wp+8w5Xj92mt+YtyTtUswKUuj67AbgnmT4HuBbuR2UOS/v3wM7IuLPc9oWZI3eCWwrsB6zstmdnFrCh45atSs0CD4L3CJpN3BLMo6kiySdPwLoRuDDwPvyHCb6OUkvStoK3AT8QYH1mJXN7oOZq5J505BVu2E3DQ0lIg4D788zfT9wRzL8EyDv1Toi4sOFPL9ZmnYd6GF8Yx2LZ/iIIatuPtTB7ALt7uxmxdwp1NX5qmRW3RwEZhdo14Fu7x+wMcFBYHYBjp3qpbP7rPcP2JjgIDC7AG+cWsJrBDYGOAjMLsDL548Y8jmGbAxwEJhdgN0Hu5k8roGLpo1PuxSzgjkIzC7AroPdXDp3MpnfS5pVNweB2ShFBC/tP8Gqi6amXYpZUTgIzEbp9WOnOXGmj1ULHAQ2NjgIzEbppf2Zk+d6jcDGCgeB2Si91HECCS7zEUM2RjgIzEZp+/4TXDx7EhObCjpVl1nFcBCYjVJmR/G0tMswKxoHgdkoHD+VuRiNdxTbWOIgMBuF7R3HAbh8gfcP2NhRUBBIminpKUm7k/u8F5+XtDe5AM0WSS2jnd+sUrzYngmCqxZNT7cQsyIqdI3gAWBTRKwANiXjg7kpIq6JiOYLnN8sdVvbj7NoxgRmTmpKuxSzoik0CNYCjybDjwIfLPP8ZmW19fVjXLXIO4ptbCk0COZFRAdAcj93kH4BPClps6R1FzA/ktZJapHU0tXVVWDZZqN35GQv+46c9mYhG3OGPRBa0veB+XmaPjmK57kxIvZLmgs8JWlnRPx4FPMTEeuB9QDNzc0xmnnNimFr+zEArxHYmDNsEETEzYO1STooaUFEdEhaAHQO8hj7k/tOSY8Da4AfAyOa36wSbG0/jgRXLnQQ2NhS6KahDcA9yfA9wLdyO0iaJGnK+WHgVmDbSOc3qxRb249z8exJTBnfmHYpZkVVaBB8FrhF0m7glmQcSRdJ2pj0mQf8RNILwM+A70TE94aa36zSRARb9h3l6sXT0y7FrOgKOllKRBwG3p9n+n7gjmR4D3D1aOY3qzSvHj7FoZ5empfOTLsUs6LzL4vNRqDl1aMAXLvUv3m0scdBYDYCm189wtTxDayYOzntUsyKzkFgNgIte4+yeukM6up8jWIbexwEZsM4dqqX3Z09NHuzkI1RDgKzYTz/2vn9A95RbGOTg8BsGD9tO0xTfR3vWDI97VLMSsJBYDaMp9sOs3rpdMY31qddillJOAjMhnD0ZC8vdZzgxktmp12KWck4CMyG8Myew0TADZfOSrsUs5JxEJgN4T/aDjGpqd6nnrYxzUFgNoSnWw+zZvlMGuv9UbGxy+9us0HsPXSSPYdO8p63zUm7FLOSchCYDeIHOzOXx3jfZfNSrsSstBwEZoP4wc5OVsydzJJZE9MuxaykHARmefSc7ePZVw7zvssHvYy22ZjhIDDL499f7uJcf/B+bxayGlBQEEiaKekpSbuT+7eclUvSSklbsm4nJH08afu0pNez2u4opB6zYvnOix3MmtTEap9WwmpAoWsEDwCbImIFsCkZf5OI2BUR10TENcC1wCng8awuf3G+PSI25s5vVm6nevvYtKOT26+cT4MPG7UaUOi7fC3waDL8KPDBYfq/H2iLiFcLfF6zktm0o5PT5/r5wFUXpV2KWVkUGgTzIqIDILkfbs/aXcDXcqbdL2mrpEfybVo6T9I6SS2SWrq6ugqr2mwI//rCfuZNHcc7l/m001Ybhg0CSd+XtC3Pbe1onkhSE/DLwNezJj8EXAJcA3QAfzbY/BGxPiKaI6J5zhz/wMdK48jJXn60q4tfvPIi6n01MqsRDcN1iIibB2uTdFDSgojokLQA6BzioW4Hno+Ig1mP/cawpC8C3x5Z2Wal8c3n2+ntH+C/vXNx2qWYlU2hm4Y2APckw/cA3xqi793kbBZKwuO8O4FtBdZjdsEigq/+7DVWL5nOyvlT0i7HrGwKDYLPArdI2g3ckowj6SJJbxwBJGli0v7NnPk/J+lFSVuBm4A/KLAeswv2s1eOsKfrJHevWZJ2KWZlNeymoaFExGEyRwLlTt8P3JE1fgp4ywndI+LDhTy/WTE98h+vMG1CI7941YLhO5uNIT5I2gxo7ezhyZcO8pvXL2ViU0H/H5lVHQeBGbD+x22Ma6jjt25YlnYpZmXnILCa9+rhkzz+n6/z682LmTV5XNrlmJWdg8Bq3uee2EVDXR3333Rp2qWYpcJBYDXt+deO8p2tHax798XMnTo+7XLMUuEgsJp1rn+ATz6+jblTxrHu3RenXY5Zanx4hNWsh3/Uxo6OE6z/8LVMGuePgtUurxFYTdqy7xhf+MFufunqi7j1ivlpl2OWKgeB1ZxDPWf53X/czLyp4/nM2ivSLscsdV4ftppy8mwf677cwpGTvXzjd29g+sSmtEsyS53XCKxmnO7t53f+YTNb9h3jL+96B29fOC3tkswqgtcIrCYcOdnLvY8+x5Z9x/j8r17NbW/3fgGz8xwENua17D3C733tPzl8speHPnStQ8Ash4PAxqzuM+f486de5tGn97J45kS+ft/1XLVoetplmVUcB4GNOYd7zvLYc/v44r/v4fjpc3zouiV84rbLmDK+Me3SzCqSg8DGhNO9/fyk9RD/vHkfm3Z00jcQ3LRyDv/zlpVcucg7hc2GUlAQSPo14NPA5cCaiGgZpN9twF8C9cDfRcT5K5nNBP4JWAbsBX49Io4WUpONfQMDwevHTrPrQDcvvn6cn+45zH++dpRz/cHsyU381g3L+LXmxb7cpNkIFbpGsA34FeBvB+sgqR54kMylKtuB5yRtiIiXgAeATRHxWUkPJOOfKLAmq3ARwbn+4Fz/AL19A5n7ZPj0uX6Onz7HidN9nDh9juPJ7cipXvYfO03HsTPsO3qKU739AEjw9oum8ZEbl3PDpbO54ZJZNNb7qGiz0Sj0UpU7ACQN1W0N0BoRe5K+jwFrgZeS+/cm/R4FfkQJg+ALm3az4YX9b4xHxH8N53aOwUez58s3b3Zz5LRG7uO+5YlH9jxveZwhnmeo+nJ7vPVxB69puMfN7TswkAmA3v6B3AKGVCeYMbGJBdPHs3TWRG64dBYr503hbfOn8LZ5U5js8wSZFaQcn6CFwL6s8XbgumR4XkR0AEREh6S5gz2IpHXAOoAlSy7s4uJzp4xj5byczQXKO3j+OQfrSm72DTXvW2LyLfNm9R32cfPPl3feN40P03fItsHnHW6ZZauvE00NdTTW1zGuoY7GetFUX0dj1rRxDfVMm9DItAmNTJ3QwLQJjUwe1zDcPxtmVoBhg0DS94F8B15/MiK+NYLnyPcJHuL/4PwiYj2wHqC5uXnU8wPctWYJd625sBAxMxurhg2CiLi5wOdoBxZnjS8Czm+fOShpQbI2sADoLPC5zMxslMqxV+05YIWk5ZKagLuADUnbBuCeZPgeYCRrGGZmVkQFBYGkOyW1A9cD35H0RDL9IkkbASKiD7gfeALYAfy/iNiePMRngVsk7SZzVNFnC6nHzMxGT7lHplSD5ubmaGnJ+5MFMzMbhKTNEdGcO90HXJuZ1TgHgZlZjXMQmJnVOAeBmVmNq8qdxZK6gFcvcPbZwKEillMslVoXVG5trmt0KrUuqNzaxlpdSyNiTu7EqgyCQkhqybfXPG2VWhdUbm2ua3QqtS6o3NpqpS5vGjIzq3EOAjOzGleLQbA+7QIGUal1QeXW5rpGp1LrgsqtrSbqqrl9BGZm9ma1uEZgZmZZHARmZjVuTAaBpF+TtF3SgKTmnLY/ltQqaZekXxhk/pmSnpK0O7mfUYIa/0nSluS2V9KWQfrtlfRi0q8sZ9qT9GlJr2fVd8cg/W5LlmNrcs3pUtf1eUk7JW2V9Lik6YP0K8syG+7vV8YXkvatklaXqpas51ws6YeSdiSfgd/P0+e9ko5nvb6fKnVdyfMO+bqksbyS512ZtSy2SDoh6eM5fcqyzCQ9IqlT0rasaSP6Piro8xgRY+4GXA6sJHMN5Oas6auAF4BxwHKgDajPM//ngAeS4QeAPy1xvX8GfGqQtr3A7DIvv08DfzRMn/pk+V0MNCXLdVWJ67oVaEiG/3Sw16Ucy2wkfz9wB/BdMlfpexfwbBleuwXA6mR4CvBynrreC3y7nO+pkbwuaSyvQV7XA2R+eFX2ZQa8G1gNbMuaNuz3UaGfxzG5RhAROyJiV56mtcBjEXE2Il4BWoE1g/R7NBl+FPhgSQol818Q8OvA10r1HCWyBmiNiD0R0Qs8Rma5lUxEPBmZ61sAPEPmandpGcnfvxb4cmQ8A0xPrsRXMhHRERHPJ8PdZK4BsrCUz1lEZV9eebwfaIuICz1zQUEi4sfAkZzJI/k+KujzOCaDYAgLgX1Z4+3k/5DMi4gOyHywgLklrOnngYMRsXuQ9gCelLRZ0roS1pHr/mT1/JFBVkVHuixL5SNk/nvMpxzLbCR/f6rLSNIy4B3As3mar5f0gqTvSrqiTCUN97qk/Z6CzBUUB/unLI1lBiP7Pipo2Q17zeJKJen7wPw8TZ+MiMEueak800p2/OwIa7ybodcGboyI/ZLmAk9J2pn811Cy2oCHgM+QWTafIbPp6iO5D5Fn3oKX5UiWmaRPAn3AVwZ5mJIss9xS80zL/fvL+n570xNLk4FvAB+PiBM5zc+T2fTRk+z/+RdgRRnKGu51SW15AShzKd1fBv44T3Nay2ykClp2VRsEEXHzBczWDizOGl8E7M/T76CkBRHRkayadpaiRkkNwK8A1w7xGPuT+05Jj5NZBSz4S22ky0/SF4Fv52ka6bIsal2S7gE+ALw/ko2jeR6jJMssx0j+/pIso+FIaiQTAl+JiG/mtmcHQ0RslPQ3kmZHRElPrjaC1yWV5ZXlduD5iDiY25DWMkuM5PuooGVXa5uGNgB3SRonaTmZRP/ZIP3uSYbvAQZbwyjUzcDOiGjP1yhpkqQp54fJ7Czdlq9vMeVsl71zkOd8DlghaXnyn9RdZJZbKeu6DfgE8MsRcWqQPuVaZiP5+zcAv5kcDfMu4Pj5VfxSSfY5/T2wIyL+fJA+85N+SFpD5nvgcInrGsnrUvbllWPQtfM0llmWkXwfFfZ5LPVe8DRuZL682oGzwEHgiay2T5LZu74LuD1r+t+RHGEEzAI2AbuT+5klqvNLwH050y4CNibDF5PZ+/8CsJ3M5pFyLL9/AF4EtiZvpgW5tSXjd5A5KqWtHLWR2bm/D9iS3B5Oc5nl+/uB+86/pmRW1x9M2l8k6wi2Etb0c2Q2CWzNWk535NR1f7JsXiCz0/2GMtSV93VJe3ll1TeRzBf7tKxpZV9mZIKoAziXfIfdO9j3UTE/jz7FhJlZjau1TUNmZpbDQWBmVuMcBGZmNc5BYGZW4xwEZmY1zkFgZlbjHARmZjXu/wMRFDZkm71EKwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Make the prediction using tanh object\n",
    "\n",
    "yhat = TANH(z)\n",
    "plt.plot(z.numpy(), yhat.numpy())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "For custom modules, call the Tanh object from the torch (nn.functional for the old version), which applies the element-wise sigmoid from the function module and plots the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgjElEQVR4nO3deZBc9Xnu8e8zm/Z9RzsgC4RZLMbCQGIbGwgQJzKpJBfi65AYX4WUqcS5yS2T8i1fV7luyrErScUJgcgJZZzYJtexiRVbNmDZjuNgMCMihIQkNCMEGjTSjPYZbaOZee8ffUSapmdTL6d7+vlUdfU55/c73e+cXp45S5+jiMDMzGpXXdoFmJlZuhwEZmY1zkFgZlbjHARmZjXOQWBmVuMa0i7gQsyePTuWLVuWdhlmZlVl8+bNhyJiTu70qgyCZcuW0dLSknYZZmZVRdKr+aZ705CZWY1zEJiZ1TgHgZlZjXMQmJnVOAeBmVmNK0oQSHpEUqekbYO0S9IXJLVK2ippdVbbbZJ2JW0PFKMeMzMbuWKtEXwJuG2I9tuBFcltHfAQgKR64MGkfRVwt6RVRarJzMxGoCi/I4iIH0taNkSXtcCXI3PO62ckTZe0AFgGtEbEHgBJjyV9XypGXWY2cr19A5w820fP2T66z/RxsreP3r4BzvUP0Ncf9A0M0Nsf9CXj5wYy9/0DQQDnT2kfAUEk928eJ+mXr60iT4hfgafpv3P1IpbPnlTUxyzXD8oWAvuyxtuTafmmX5fvASStI7M2wZIlS0pTpdkYdeZcP62dPew60M2+o6c4eOIMB46f4cCJs3R1n+HEmcyXvr2VlHYFb7Z66YyqDYJ8izKGmP7WiRHrgfUAzc3NlRfTZhXk2Kleftp2mKfbDvPMnsO0dfUwkPWpmT25iXlTx3PRtPFcs3gaUyc0MmVcA5PGNTB5XANTxmeGm+rraKivo7FeNNTV0dSQuW+oF431dTTUifo6IQTKfGkKkJTcg9AbX6bZ42/pV2nfuDWkXEHQDizOGl8E7AeaBpluZqN0rn+AH+7s5Oub2/nhzk76BoKJTfW8c9lM7rhyAZfNn8KKeVNYMnMiTQ0+YND+S7mCYANwf7IP4DrgeER0SOoCVkhaDrwO3AX8RplqMhsTzpzr5yvPvsbf/lsbnd1nmT15HPf+3HJuvWI+Vy2aRmO9v/RtaEUJAklfA94LzJbUDvwfoBEgIh4GNgJ3AK3AKeC3k7Y+SfcDTwD1wCMRsb0YNZnVgh/u7ORTG7ax78hprr94Fn9y55W8Z+Ucf/nbqBTrqKG7h2kP4GODtG0kExRmNkJnzvXzmW+/xFeefY1L507mKx+9jhsvnZ12WValqvI01Ga17FDPWe790nO80H6c33n3xfzhrSu9zd8K4iAwqyL7j53m7i8+w8ETZ1j/4Wu59Yr5aZdkY4CDwKxKHOo5y3//+2c50tPLV//Hu1i9ZEbaJdkY4SAwqwJn+/r56KMt7D92mn+49zqHgBWVg8CsCnzm2y+xZd8xHvrQat65bGba5dgY4z1MZhXue9s6+MdnXuN33nMxt1+5IO1ybAxyEJhVsKMne/nf/7Kdty+cyh/dujLtcmyM8qYhswr2fzfu4NipXr78kTX+kZiVjN9ZZhVq2+vH+efN7dz788tZddHUtMuxMcxBYFaBIoI/2biDmZOa+NhNl6Zdjo1xDgKzCvST1kM83XaY33vfpUwd35h2OTbGOQjMKtDf/LCN+VPH8xvXLU27FKsBDgKzCrNl3zF+uucw9/7ccp9DyMrC7zKzCvO3/9bG1PEN3H2dL8lq5eEgMKsgB0+c4cmXDnL3dUuYPM5Hd1t5OAjMKsjXW/bRPxDc/U6vDVj5FCUIJN0maZekVkkP5Gn/X5K2JLdtkvolzUza9kp6MWlrKUY9ZtVoYCD42s/2ceOls1g2e1La5VgNKTgIJNUDDwK3A6uAuyWtyu4TEZ+PiGsi4hrgj4F/i4gjWV1uStqbC63HrFr9pPUQrx87zd1rvDZg5VWMNYI1QGtE7ImIXuAxYO0Q/e8GvlaE5zUbUza8sJ8p4xu4ZdW8tEuxGlOMIFgI7Msab0+mvYWkicBtwDeyJgfwpKTNktYN9iSS1klqkdTS1dVVhLLNKsfZvn6e2H6AX7hiPuMa6tMux2pMMYJAeabFIH1/CfiPnM1CN0bEajKblj4m6d35ZoyI9RHRHBHNc+bMKaxiswrz45cP0X2mjw9c5dNMW/kVIwjagcVZ44uA/YP0vYuczUIRsT+57wQeJ7Opyaym/OsL+5kxsZEbL52ddilWg4oRBM8BKyQtl9RE5st+Q24nSdOA9wDfypo2SdKU88PArcC2ItRkVjV6+wbYtOMgv3DFfJ9q2lJR8C9WIqJP0v3AE0A98EhEbJd0X9L+cNL1TuDJiDiZNfs84HFJ52v5akR8r9CazKrJz145wsnefm6+3DuJLR1F+eliRGwENuZMezhn/EvAl3Km7QGuLkYNZtVq086DjGuo82YhS43XQ81SFBH8YGcnN1wyiwlNPlrI0uEgMEvRnkMnefXwKd532dy0S7Ea5iAwS9GPdmV+E3OTg8BS5CAwS9HTrYdYPnsSi2ZMTLsUq2EOArOU9PUP8OwrR7j+kllpl2I1zkFglpKtrx+n52wfN17io4UsXQ4Cs5T8tO0wAO+6eGbKlVitcxCYpeTptkNcNn8KsyaPS7sUq3EOArMU9PYN0LL3KDd4s5BVAAeBWQq27z/O2b4B3rlsRtqlmDkIzNKw+dWjAFy71EFg6XMQmKWgZe9RFs+cwNyp49MuxcxBYFZuEUHLq0dpXuqjhawyOAjMyuy1I6c41HPWm4WsYjgIzMqsZW9m/0CzdxRbhXAQmJXZln3HmDyugRVzp6RdihlQpCCQdJukXZJaJT2Qp/29ko5L2pLcPjXSec3Gmq3tx3j7wqnU1yntUsyAIlyhTFI98CBwC5kL2T8naUNEvJTT9d8j4gMXOK/ZmNDbN8COjm5++8ZlaZdi9oZirBGsAVojYk9E9AKPAWvLMK9Z1dl1oJve/gGuXDQt7VLM3lCMIFgI7Msab0+m5bpe0guSvivpilHOi6R1kloktXR1dRWhbLPye6H9GABXL5qeah1m2YoRBPk2dEbO+PPA0oi4Gvgr4F9GMW9mYsT6iGiOiOY5c+ZcaK1mqdrafowZExtZNGNC2qWYvaEYQdAOLM4aXwTsz+4QEScioicZ3gg0Spo9knnNxpKt7ce5atF0JO8otspRjCB4DlghabmkJuAuYEN2B0nzlbzzJa1JnvfwSOY1GyvOnOtnd2cPV3n/gFWYgo8aiog+SfcDTwD1wCMRsV3SfUn7w8CvAr8rqQ84DdwVEQHknbfQmswq0e6DPfQPBKsWTE27FLM3KTgI4I3NPRtzpj2cNfzXwF+PdF6zsWhHxwkALnMQWIXxL4vNymTHgRNMaKxnycyJaZdi9iYOArMy2dnRzcr5U/yLYqs4DgKzMogIdhw4weULfH4hqzwOArMyOHjiLMdOneNy7x+wCuQgMCuDHQeSHcXzHQRWeRwEZmVw/oihlfO9acgqj4PArAx2dnSzcPoEpk1oTLsUs7dwEJiVwY4O7yi2yuUgMCuxM+f62XPopHcUW8VyEJiVWGtn5tQS3lFslcpBYFZi53cUe9OQVSoHgVmJ7TzQzfjGOpbOmpR2KWZ5OQjMSmxHxwlWzvOpJaxyOQjMSigikiOGvH/AKpeDwKyEDvX0cvTUOf+QzCpaUYJA0m2SdklqlfRAnvYPSdqa3J6WdHVW215JL0raIqmlGPWYVYrWzh4ALp07OeVKzAZX8IVpJNUDDwK3kLkG8XOSNkTES1ndXgHeExFHJd0OrAeuy2q/KSIOFVqLWaVp63IQWOUrxhrBGqA1IvZERC/wGLA2u0NEPB0RR5PRZ8hcpN5szGvt7GFSUz3zp45PuxSzQRUjCBYC+7LG25Npg7kX+G7WeABPStosad1gM0laJ6lFUktXV1dBBZuVS1tXD5fMnYzkI4aschUjCPK9wyNvR+kmMkHwiazJN0bEauB24GOS3p1v3ohYHxHNEdE8Z86cQms2K4u2zh4umePNQlbZihEE7cDirPFFwP7cTpKuAv4OWBsRh89Pj4j9yX0n8DiZTU1mVe/k2T72Hz/j/QNW8YoRBM8BKyQtl9QE3AVsyO4gaQnwTeDDEfFy1vRJkqacHwZuBbYVoSaz1O3pOgnAJXP8i2KrbAUfNRQRfZLuB54A6oFHImK7pPuS9oeBTwGzgL9JtpX2RUQzMA94PJnWAHw1Ir5XaE1mlaC1qxvwEUNW+QoOAoCI2AhszJn2cNbwR4GP5plvD3B17nSzsaCt8yT1dWLJTK8RWGXzL4vNSqS1s4elsybS1OCPmVU2v0PNSqSty0cMWXVwEJiVQF//AHsPn/T+AasKDgKzEnjtyCnO9YfXCKwqOAjMSsAnm7Nq4iAwK4E2/4bAqoiDwKwEWjt7mDd1HFPGN6ZditmwHARmJdDa1ePNQlY1HARmRRYR7PHJ5qyKOAjMiqyz+yzdZ/u8RmBVw0FgVmRtyRFDXiOwauEgMCuyVl+e0qqMg8CsyNo6e5g8roG5U8alXYrZiDgIzIqs1ZentCrjIDArsrbOk/4hmVUVB4FZEXWfOceBE748pVWXogSBpNsk7ZLUKumBPO2S9IWkfauk1SOd16ya/NflKR0EVj0KDgJJ9cCDwO3AKuBuSatyut0OrEhu64CHRjGvWdXwyeasGhVjjWAN0BoReyKiF3gMWJvTZy3w5ch4BpguacEI5zWrGm1dPTTUiSUzJ6ZditmIFSMIFgL7ssbbk2kj6TOSeQGQtE5Si6SWrq6ugos2K4XWzh6WzZ5EY713v1n1KMa7Nd8xcjHCPiOZNzMxYn1ENEdE85w5c0ZZoll5tHb1+IghqzrFCIJ2YHHW+CJg/wj7jGRes6pwrn+A1w6f8v4BqzrFCILngBWSlktqAu4CNuT02QD8ZnL00LuA4xHRMcJ5zarCq4dP0jfgy1Na9Wko9AEiok/S/cATQD3wSERsl3Rf0v4wsBG4A2gFTgG/PdS8hdZklobWzsyho14jsGpTcBAARMRGMl/22dMezhoO4GMjndesGrUlJ5u72GsEVmV8aINZkbR19rBg2ngmjyvK/1dmZeMgMCuSzBFDXhuw6uMgMCuCiKCt09cpturkIDArggMnznCyt9+/IbCq5CAwK4K25IihS7xGYFXIQWBWBK2d3YAPHbXq5CAwK4K2rpNMGd/AnMm+PKVVHweBWRG0JjuKfXlKq0YOArMi8KGjVs0cBGYFOn76HF3dZ71/wKqWg8CsQOdPLeE1AqtWDgKzArX58pRW5RwEZgVq7eqhqb6OxTMmpF2K2QVxEJgVqK2zh6WzJtLgy1NalfI716xALx/s4W3zpqRdhtkFcxCYFeBUbx/7jp5yEFhVKygIJM2U9JSk3cn9jDx9Fkv6oaQdkrZL+v2stk9Lel3SluR2RyH1mJVba2cPEfC2ed5RbNWr0DWCB4BNEbEC2JSM5+oD/jAiLgfeBXxM0qqs9r+IiGuSm69UZlXl5YOZI4beNt9rBFa9Cg2CtcCjyfCjwAdzO0RER0Q8nwx3AzuAhQU+r1lF2H2wm6b6OpbOnJh2KWYXrNAgmBcRHZD5wgfmDtVZ0jLgHcCzWZPvl7RV0iP5Ni1lzbtOUouklq6urgLLNiuOXQe7uWTuZB8xZFVt2HevpO9L2pbntnY0TyRpMvAN4OMRcSKZ/BBwCXAN0AH82WDzR8T6iGiOiOY5c+aM5qnNSmb3wR7vH7CqN+xVtiPi5sHaJB2UtCAiOiQtADoH6ddIJgS+EhHfzHrsg1l9vgh8ezTFm6Wp+8w5Xj92mt+YtyTtUswKUuj67AbgnmT4HuBbuR2UOS/v3wM7IuLPc9oWZI3eCWwrsB6zstmdnFrCh45atSs0CD4L3CJpN3BLMo6kiySdPwLoRuDDwPvyHCb6OUkvStoK3AT8QYH1mJXN7oOZq5J505BVu2E3DQ0lIg4D788zfT9wRzL8EyDv1Toi4sOFPL9ZmnYd6GF8Yx2LZ/iIIatuPtTB7ALt7uxmxdwp1NX5qmRW3RwEZhdo14Fu7x+wMcFBYHYBjp3qpbP7rPcP2JjgIDC7AG+cWsJrBDYGOAjMLsDL548Y8jmGbAxwEJhdgN0Hu5k8roGLpo1PuxSzgjkIzC7AroPdXDp3MpnfS5pVNweB2ShFBC/tP8Gqi6amXYpZUTgIzEbp9WOnOXGmj1ULHAQ2NjgIzEbppf2Zk+d6jcDGCgeB2Si91HECCS7zEUM2RjgIzEZp+/4TXDx7EhObCjpVl1nFcBCYjVJmR/G0tMswKxoHgdkoHD+VuRiNdxTbWOIgMBuF7R3HAbh8gfcP2NhRUBBIminpKUm7k/u8F5+XtDe5AM0WSS2jnd+sUrzYngmCqxZNT7cQsyIqdI3gAWBTRKwANiXjg7kpIq6JiOYLnN8sdVvbj7NoxgRmTmpKuxSzoik0CNYCjybDjwIfLPP8ZmW19fVjXLXIO4ptbCk0COZFRAdAcj93kH4BPClps6R1FzA/ktZJapHU0tXVVWDZZqN35GQv+46c9mYhG3OGPRBa0veB+XmaPjmK57kxIvZLmgs8JWlnRPx4FPMTEeuB9QDNzc0xmnnNimFr+zEArxHYmDNsEETEzYO1STooaUFEdEhaAHQO8hj7k/tOSY8Da4AfAyOa36wSbG0/jgRXLnQQ2NhS6KahDcA9yfA9wLdyO0iaJGnK+WHgVmDbSOc3qxRb249z8exJTBnfmHYpZkVVaBB8FrhF0m7glmQcSRdJ2pj0mQf8RNILwM+A70TE94aa36zSRARb9h3l6sXT0y7FrOgKOllKRBwG3p9n+n7gjmR4D3D1aOY3qzSvHj7FoZ5empfOTLsUs6LzL4vNRqDl1aMAXLvUv3m0scdBYDYCm189wtTxDayYOzntUsyKzkFgNgIte4+yeukM6up8jWIbexwEZsM4dqqX3Z09NHuzkI1RDgKzYTz/2vn9A95RbGOTg8BsGD9tO0xTfR3vWDI97VLMSsJBYDaMp9sOs3rpdMY31qddillJOAjMhnD0ZC8vdZzgxktmp12KWck4CMyG8Myew0TADZfOSrsUs5JxEJgN4T/aDjGpqd6nnrYxzUFgNoSnWw+zZvlMGuv9UbGxy+9us0HsPXSSPYdO8p63zUm7FLOSchCYDeIHOzOXx3jfZfNSrsSstBwEZoP4wc5OVsydzJJZE9MuxaykHARmefSc7ePZVw7zvssHvYy22ZjhIDDL499f7uJcf/B+bxayGlBQEEiaKekpSbuT+7eclUvSSklbsm4nJH08afu0pNez2u4opB6zYvnOix3MmtTEap9WwmpAoWsEDwCbImIFsCkZf5OI2BUR10TENcC1wCng8awuf3G+PSI25s5vVm6nevvYtKOT26+cT4MPG7UaUOi7fC3waDL8KPDBYfq/H2iLiFcLfF6zktm0o5PT5/r5wFUXpV2KWVkUGgTzIqIDILkfbs/aXcDXcqbdL2mrpEfybVo6T9I6SS2SWrq6ugqr2mwI//rCfuZNHcc7l/m001Ybhg0CSd+XtC3Pbe1onkhSE/DLwNezJj8EXAJcA3QAfzbY/BGxPiKaI6J5zhz/wMdK48jJXn60q4tfvPIi6n01MqsRDcN1iIibB2uTdFDSgojokLQA6BzioW4Hno+Ig1mP/cawpC8C3x5Z2Wal8c3n2+ntH+C/vXNx2qWYlU2hm4Y2APckw/cA3xqi793kbBZKwuO8O4FtBdZjdsEigq/+7DVWL5nOyvlT0i7HrGwKDYLPArdI2g3ckowj6SJJbxwBJGli0v7NnPk/J+lFSVuBm4A/KLAeswv2s1eOsKfrJHevWZJ2KWZlNeymoaFExGEyRwLlTt8P3JE1fgp4ywndI+LDhTy/WTE98h+vMG1CI7941YLhO5uNIT5I2gxo7ezhyZcO8pvXL2ViU0H/H5lVHQeBGbD+x22Ma6jjt25YlnYpZmXnILCa9+rhkzz+n6/z682LmTV5XNrlmJWdg8Bq3uee2EVDXR3333Rp2qWYpcJBYDXt+deO8p2tHax798XMnTo+7XLMUuEgsJp1rn+ATz6+jblTxrHu3RenXY5Zanx4hNWsh3/Uxo6OE6z/8LVMGuePgtUurxFYTdqy7xhf+MFufunqi7j1ivlpl2OWKgeB1ZxDPWf53X/czLyp4/nM2ivSLscsdV4ftppy8mwf677cwpGTvXzjd29g+sSmtEsyS53XCKxmnO7t53f+YTNb9h3jL+96B29fOC3tkswqgtcIrCYcOdnLvY8+x5Z9x/j8r17NbW/3fgGz8xwENua17D3C733tPzl8speHPnStQ8Ash4PAxqzuM+f486de5tGn97J45kS+ft/1XLVoetplmVUcB4GNOYd7zvLYc/v44r/v4fjpc3zouiV84rbLmDK+Me3SzCqSg8DGhNO9/fyk9RD/vHkfm3Z00jcQ3LRyDv/zlpVcucg7hc2GUlAQSPo14NPA5cCaiGgZpN9twF8C9cDfRcT5K5nNBP4JWAbsBX49Io4WUpONfQMDwevHTrPrQDcvvn6cn+45zH++dpRz/cHsyU381g3L+LXmxb7cpNkIFbpGsA34FeBvB+sgqR54kMylKtuB5yRtiIiXgAeATRHxWUkPJOOfKLAmq3ARwbn+4Fz/AL19A5n7ZPj0uX6Onz7HidN9nDh9juPJ7cipXvYfO03HsTPsO3qKU739AEjw9oum8ZEbl3PDpbO54ZJZNNb7qGiz0Sj0UpU7ACQN1W0N0BoRe5K+jwFrgZeS+/cm/R4FfkQJg+ALm3az4YX9b4xHxH8N53aOwUez58s3b3Zz5LRG7uO+5YlH9jxveZwhnmeo+nJ7vPVxB69puMfN7TswkAmA3v6B3AKGVCeYMbGJBdPHs3TWRG64dBYr503hbfOn8LZ5U5js8wSZFaQcn6CFwL6s8XbgumR4XkR0AEREh6S5gz2IpHXAOoAlSy7s4uJzp4xj5byczQXKO3j+OQfrSm72DTXvW2LyLfNm9R32cfPPl3feN40P03fItsHnHW6ZZauvE00NdTTW1zGuoY7GetFUX0dj1rRxDfVMm9DItAmNTJ3QwLQJjUwe1zDcPxtmVoBhg0DS94F8B15/MiK+NYLnyPcJHuL/4PwiYj2wHqC5uXnU8wPctWYJd625sBAxMxurhg2CiLi5wOdoBxZnjS8Czm+fOShpQbI2sADoLPC5zMxslMqxV+05YIWk5ZKagLuADUnbBuCeZPgeYCRrGGZmVkQFBYGkOyW1A9cD35H0RDL9IkkbASKiD7gfeALYAfy/iNiePMRngVsk7SZzVNFnC6nHzMxGT7lHplSD5ubmaGnJ+5MFMzMbhKTNEdGcO90HXJuZ1TgHgZlZjXMQmJnVOAeBmVmNq8qdxZK6gFcvcPbZwKEillMslVoXVG5trmt0KrUuqNzaxlpdSyNiTu7EqgyCQkhqybfXPG2VWhdUbm2ua3QqtS6o3NpqpS5vGjIzq3EOAjOzGleLQbA+7QIGUal1QeXW5rpGp1LrgsqtrSbqqrl9BGZm9ma1uEZgZmZZHARmZjVuTAaBpF+TtF3SgKTmnLY/ltQqaZekXxhk/pmSnpK0O7mfUYIa/0nSluS2V9KWQfrtlfRi0q8sZ9qT9GlJr2fVd8cg/W5LlmNrcs3pUtf1eUk7JW2V9Lik6YP0K8syG+7vV8YXkvatklaXqpas51ws6YeSdiSfgd/P0+e9ko5nvb6fKnVdyfMO+bqksbyS512ZtSy2SDoh6eM5fcqyzCQ9IqlT0rasaSP6Piro8xgRY+4GXA6sJHMN5Oas6auAF4BxwHKgDajPM//ngAeS4QeAPy1xvX8GfGqQtr3A7DIvv08DfzRMn/pk+V0MNCXLdVWJ67oVaEiG/3Sw16Ucy2wkfz9wB/BdMlfpexfwbBleuwXA6mR4CvBynrreC3y7nO+pkbwuaSyvQV7XA2R+eFX2ZQa8G1gNbMuaNuz3UaGfxzG5RhAROyJiV56mtcBjEXE2Il4BWoE1g/R7NBl+FPhgSQol818Q8OvA10r1HCWyBmiNiD0R0Qs8Rma5lUxEPBmZ61sAPEPmandpGcnfvxb4cmQ8A0xPrsRXMhHRERHPJ8PdZK4BsrCUz1lEZV9eebwfaIuICz1zQUEi4sfAkZzJI/k+KujzOCaDYAgLgX1Z4+3k/5DMi4gOyHywgLklrOnngYMRsXuQ9gCelLRZ0roS1pHr/mT1/JFBVkVHuixL5SNk/nvMpxzLbCR/f6rLSNIy4B3As3mar5f0gqTvSrqiTCUN97qk/Z6CzBUUB/unLI1lBiP7Pipo2Q17zeJKJen7wPw8TZ+MiMEueak800p2/OwIa7ybodcGboyI/ZLmAk9J2pn811Cy2oCHgM+QWTafIbPp6iO5D5Fn3oKX5UiWmaRPAn3AVwZ5mJIss9xS80zL/fvL+n570xNLk4FvAB+PiBM5zc+T2fTRk+z/+RdgRRnKGu51SW15AShzKd1fBv44T3Nay2ykClp2VRsEEXHzBczWDizOGl8E7M/T76CkBRHRkayadpaiRkkNwK8A1w7xGPuT+05Jj5NZBSz4S22ky0/SF4Fv52ka6bIsal2S7gE+ALw/ko2jeR6jJMssx0j+/pIso+FIaiQTAl+JiG/mtmcHQ0RslPQ3kmZHRElPrjaC1yWV5ZXlduD5iDiY25DWMkuM5PuooGVXa5uGNgB3SRonaTmZRP/ZIP3uSYbvAQZbwyjUzcDOiGjP1yhpkqQp54fJ7Czdlq9vMeVsl71zkOd8DlghaXnyn9RdZJZbKeu6DfgE8MsRcWqQPuVaZiP5+zcAv5kcDfMu4Pj5VfxSSfY5/T2wIyL+fJA+85N+SFpD5nvgcInrGsnrUvbllWPQtfM0llmWkXwfFfZ5LPVe8DRuZL682oGzwEHgiay2T5LZu74LuD1r+t+RHGEEzAI2AbuT+5klqvNLwH050y4CNibDF5PZ+/8CsJ3M5pFyLL9/AF4EtiZvpgW5tSXjd5A5KqWtHLWR2bm/D9iS3B5Oc5nl+/uB+86/pmRW1x9M2l8k6wi2Etb0c2Q2CWzNWk535NR1f7JsXiCz0/2GMtSV93VJe3ll1TeRzBf7tKxpZV9mZIKoAziXfIfdO9j3UTE/jz7FhJlZjau1TUNmZpbDQWBmVuMcBGZmNc5BYGZW4xwEZmY1zkFgZlbjHARmZjXu/wMRFDZkm71EKwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Make the prediction using the build-in tanh object\n",
    "\n",
    "yhat = torch.tanh(z)\n",
    "plt.plot(z.numpy(), yhat.numpy())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Relu\">Relu</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you use sequential, you can create a Relu object: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f3538e66780>]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAc0klEQVR4nO3deXhU5d3/8ffXkLCFJZiwg4myi2AWQZH6uGAF61K1UFl+6qMtLRSKS61Uf9rWp611q3XFWm1ra8JiQeGnUrfaelF9FLIQAgQIZQsgSdjCFshy//7ItFcaEwiznZnJ53VduTI552TOZ+6Z+c4995k5tznnEBGR6HOG1wFERMQ/KuAiIlFKBVxEJEqpgIuIRCkVcBGRKNUmnDtLTk52qamp4dyliEjUy83NrXDOpTReHtYCnpqayqpVq8K5SxGRqGdm25pariEUEZEopQIuIhKlVMBFRKLUKQu4mf3OzMrMrKjBsm5m9r6ZbfL9TgptTBERaawlPfA/AOMbLZsLfOicGwh86PtbRETC6JQF3Dn3MbCv0eLrgVd9l18Fvh7cWCIicir+joH3cM7tBvD97t7chmY23cxWmdmq8vJyP3cnIiKNhfwgpnPuJedclnMuKyXlS59DFxGJaVXVtfxk2Vr2HTkR9Ov2t4DvMbNeAL7fZcGLJCISOx5aWsQfPtlK0c6DQb9ufwv4MuBW3+VbgaXBiSMiEjsWrdzBolWlzL58AJcMCv4IREs+Rjgf+BQYbGalZnYH8EvgSjPbBFzp+1tERHzW7arkwaVFjDnnTO4cNygk+zjluVCcc5ObWXVFkLOIiMSEyqpqZmbn0rVDPM9MTifuDAvJfsJ6MisRkVjnnOOHrxeyY/8xFky/kOTEtiHbl75KLyISRK+s2MJf1n7B3PFDuCC1W0j3pQIuIhIkK7fu45HlxXx1WA++9ZW0kO9PBVxEJAgqDh9nVk4efZPa8/jEkZiFZty7IY2Bi4gEqLbOMWdBPgeOVrNk5gV0aR8flv2qgIuIBOjpDzbyj5K9PHbTCM7t3SVs+9UQiohIAD7aUMYzfy1hYmZfJl3QL6z7VgEXEfHTzgPHuGthAUN6duLh64eHff8q4CIifjhRU8fM7Dxqah3zpmXSPiEu7Bk0Bi4i4odfvLOe1TsOMG9qBmnJHT3JoB64iMhp+n+rd/GHT7Zyx9g0JpzXy7McKuAiIqehpOwwcxcXknlWEnMnDPE0iwq4iEgLHT1Rw8zsXNrGx/HclHTi47wtoRoDFxFpAecc9y9Zw6ayw/zx9lH06tLe60jqgYuItETO59t5s2AXd40bxFcGRsb0kCrgIiKnsKb0ID9dto5LBqUw67IBXsf5NxVwEZGTOHi0mhnZuSQnJvDrb57PGSGanMEfGgMXEWlGXZ3j7kUF7KmsYuF3LqJbxwSvI/0H9cBFRJrxm4//yYfFZTxw9VAy+id5HedLVMBFRJrw6ea9PP5uMV8b0Ytbx6R6HadJKuAiIo2UVVYxe34+qckdefSmEWGZnMEfGgMXEWmgpraO2fPzOXy8muxvjSaxbeSWychNJiLigSff38hnW/bxq0kjGdyzk9dxTkpDKCIiPu+v28O8v21m8qj+3JjR1+s4p6QCLiICbN97lHsWFTC8T2d+fO0wr+O0iAq4iLR6VdW1zMzJBWDe1EzaxYd/cgZ/aAxcRFq9h99aR9HOSn57Sxb9unXwOk6LqQcuIq3akrxScj7bznf/6xyuHNbD6zinRQVcRFqtDV8c4oE3ihiV1o0ffHWQ13FOmwq4iLRKh4/XMCM7l45t2/Dc5HTaeDw5gz+iL7GISICcc9y3uJCtFUd4dnI63Tu38zqSXwIq4GZ2l5mtNbMiM5tvZtHZCiLSqvzx0228XbibH1w1mIvOOdPrOH7zu4CbWR/g+0CWc244EAfcHKxgIiKhkL99Pz97ex1XDOnOdy85x+s4AQl0CKUN0N7M2gAdgF2BRxIRCY39R07wvew8enRux5OTRkbU5Az+8LuAO+d2Ak8A24HdwEHn3HuNtzOz6Wa2ysxWlZeX+59URCQAdXWOOxcWUHH4BC9MzaBrh8ianMEfgQyhJAHXA2lAb6CjmU1rvJ1z7iXnXJZzLislJTImAhWR1ue5j0r4+8ZyfnzdMEb07ep1nKAIZAhlHLDFOVfunKsGlgBjghNLRCR4Vmyq4KkPNnJDeh+mjOrvdZygCaSAbwcuNLMOVn+28yuA9cGJJSISHLsPHuP7C/IZ2D2Rn98wPGInZ/BHIGPgnwF/BvKANb7reilIuUREAlZdW8esnHyqqmt5YWomHRJi6/RPAd0a59yPgR8HKYuISFA9uryY3G37eWZyOgO6J3odJ+j0TUwRiUl/KdrNyyu2cOtFZ3HdyN5exwkJFXARiTlbKo5w7+uFjOzXlfu/NtTrOCGjAi4iMaWqupYZr+USF2c8PyWdtm2iY3IGf8TWiL6ItHoPLS2i+ItD/P6/L6BvUvRMzuAP9cBFJGYsWrmDRatKmX35AC4b3N3rOCGnAi4iMWHdrkoeXFrExQPO5M5x0Tc5gz9UwEUk6lVWVTMzO5euHeJ5+uZ04qL8JFUtpTFwEYlqzjnufX01O/YfY+H0C0lObOt1pLBRD1xEotorK7bw7to9/GjCELJSu3kdJ6xUwEUkaq3cuo9Hlhdz1bk9uGNsmtdxwk4FXESiUsXh48zKyaNvUnsenzgypk5S1VIaAxeRqFNb55izIJ8DR6tZMvMCOreL9zqSJ1TARSTqPP3BRv5RspfHbhrBub27eB3HMxpCEZGo8tGGMp75awkTM/sy6YJ+XsfxlAq4iESNnQeOcdfCAob07MTD1w/3Oo7nVMBFJCqcqKljZnYetbWOedMyaZ8QuyepaimNgYtIVPjFO+tZveMAL07LIC25o9dxIoJ64CIS8Zat3sUfPtnKt8amMX54L6/jRAwVcBGJaCVlh5i7uJDMs5K4b8IQr+NEFBVwEYlYR0/UMOO1PNrHx/H8lAzi41SyGtIYuIhEJOcc9y9ZQ0n5Yf50+2h6dmnndaSIo5czEYlIOZ9v582CXdw1bhBjByZ7HSciqYCLSMRZU3qQny5bxyWDUph12QCv40QsFXARiSgHj1YzIzuX5MQEfv3N8zmjlUzO4A+NgYtIxKirc9y9qIA9lVUs/M5FdOuY4HWkiKYeuIhEjN98/E8+LC7jgauHktE/yes4EU8FXEQiwqeb9/L4u8VcM6IXt45J9TpOVFABFxHPlVVWMXt+PqnJHfnlTSNa5eQM/tAYuIh4qqa2jlnz8zlyvIacb48msa3KUkuppUTEU0+8t5HPt+zjV5NGMqhHJ6/jRBUNoYiIZ95ft4cX/76ZKaP7c2NGX6/jRJ2ACriZdTWzP5tZsZmtN7OLghVMRGLb9r1HuWdRAcP7dOaha4Z5HScqBTqE8jTwF+fcN8wsAegQhEwiEuOqqmuZmZMLwLypmbSL1+QM/vC7gJtZZ+AS4DYA59wJ4ERwYolILHv4rXUU7azkt7dk0a+b+n3+CmQI5WygHPi9meWb2ctm9qVpMsxsupmtMrNV5eXlAexORGLBkrxScj7bznf/6xyuHNbD6zhRLZAC3gbIAOY559KBI8Dcxhs5515yzmU557JSUlIC2J2IRLsNXxzigTeKGJXWjR98dZDXcaJeIAW8FCh1zn3m+/vP1Bd0EZEvOXy8hhnZuXRs24bnJqfTRpMzBMzvFnTOfQHsMLPBvkVXAOuCkkpEYopzjvsWF7K14gjPTUmne2dNzhAMgX4KZTaQ7fsEyj+B/w48kojEmj9+uo23C3dz3/ghXHj2mV7HiRkBFXDnXAGQFZwoIhKL8rbv52dvr2Pc0O5855KzvY4TUzQIJSIhs+/ICWZl59GjczuenKjJGYJN50IRkZCoq3PcubCAisMnWDxjDF06xHsdKeaoBy4iIfHcRyV8vLGcH183jPP6dvE6TkxSAReRoFuxqYKnPtjIDel9mDKqv9dxYpYKuIgE1e6Dx/j+gnwGdk/k5zcM1+QMIaQCLiJBU11bx6ycfKqqa3lhaiYdEnSYLZTUuiISNI8uLyZ3236enZzOgO6JXseJeeqBi0hQ/KVoNy+v2MJtY1K5dmRvr+O0CirgIhKwLRVHuPf1Qs7v15X7rx7qdZxWQwVcRAJSVV3LjNdyiYsznp+aQUIblZVw0Ri4iATkoaVFbNhziN/fdgF9urb3Ok6ropdKEfHbopU7WLSqlNmXDeDSwd29jtPqqICLiF/W7jrIg0uLuHjAmcwZp8kZvKACLiKnrbKqmpnZeXTtEM/TN6cTp5NUeUJj4CJyWpxz3Pv6akr3H2Ph9AtJTmzrdaRWSz1wETktr6zYwrtr9/CjCUPISu3mdZxWTQVcRFps5dZ9PLK8mKvO7cEdY9O8jtPqqYCLSItUHD7OrJw8+iW15/GJI3WSqgigAi4ip1Rb55izIJ8DR6t5YWomndtpcoZIoIOYInJKT3+wkX+U7OWxb4xgWO/OXscRH/XAReSkPtpQxjN/LWFSVl8mZfXzOo40oAIuIs3aeeAYdy0sYEjPTjx8/XCv40gjKuAi0qTjNbXMzM6jttYxb1om7eLjvI4kjWgMXESa9Iu317N6xwFenJZBWnJHr+NIE9QDF5EvWbZ6F69+uo1vjU1j/PBeXseRZqiAi8h/KCk7xNzFhWSelcR9E4Z4HUdOQgVcRP7t6IkaZryWR/v4OJ6fkkF8nEpEJNMYuIgA9Sepun/JGkrKD/On20fTs0s7ryPJKejlVUQAyPl8O28W7OLucYMYOzDZ6zjSAirgIsKa0oP8dNk6Lh2cwvcuG+B1HGkhFXCRVu7g0WpmZOeSnJjAU5PO5wxNzhA1Ai7gZhZnZvlm9lYwAolI+NTVOe5eVMCeyiqen5pBUscEryPJaQhGD3wOsD4I1yMiYfabj//Jh8Vl/N+vDSO9f5LXceQ0BVTAzawv8DXg5eDEEZFw+XTzXh5/t5hrRvTilovO8jqO+CHQHvivgR8Cdc1tYGbTzWyVma0qLy8PcHciEgxllVXMnp9PanJHfnnTCE3OEKX8LuBmdg1Q5pzLPdl2zrmXnHNZzrmslJQUf3cnIkFSU1vHrPn5HDlew4vTMklsq6+DRKtAeuAXA9eZ2VZgAXC5mb0WlFQiEjJPvLeRz7fs4+c3DGdQj05ex5EA+F3AnXM/cs71dc6lAjcDf3XOTQtaMhEJuvfX7eHFv29myuj+3JjR1+s4EiB9Dlykldi+9yj3LCpgeJ/OPHTNMK/jSBAEZfDLOfc34G/BuC4RCb6q6lpm5tQfrpo3VZMzxAodvRBpBR5+ax1FOyt5+ZYs+nXr4HUcCRINoYjEuCV5peR8tp0Zl57DuGE9vI4jQaQCLhLDNnxxiAfeKGJ0WjfuuXKQ13EkyFTARWLU4eM1zMjOJbFdG56dkk4bTc4Qc3SPisQg5xz3LS5ka8URnp2cTvdOmpwhFqmAi8SgVz/ZytuFu7n3qiFcePaZXseREFEBF4kxedv38/N31jNuaHe+c8nZXseREFIBF4kh+46cYFZ2Hj06t+PJiZqcIdbpc+AiMaKuznHnwgIqDp9g8YwxdOkQ73UkCTEVcJEY8dxHJXy8sZxf3HAe5/Xt4nUcCQMNoYjEgBWbKnjqg43cmN6HyaP6eR1HwkQFXCTK7T54jO8vyGdg90R+dsNwTc7QiqiAi0Sx6to6ZuXkc7y6lnnTMumQoFHR1kT3tkgUe3R5Mbnb9vPs5HTOSUn0Oo6EmXrgIlHqL0W7eXnFFm4bk8q1I3t7HUc8oAIuEoW2VBzh3tcLOb9fV+6/eqjXccQjKuAiUebYiVpmvJZLXJzx/NQMEtroadxaaQxcJMo8tLSIDXsO8fvbLqBP1/ZexxEP6aVbJIosWrmD13NLmX3ZAC4d3N3rOOIxFXCRKLF210EeXFrE2AHJzBmnyRlEBVwkKlRWVTMzO4+kDgk8ffP5xOkkVYLGwEUinnOOe19fzc79x1gw/ULOTGzrdSSJEOqBi0S4V1Zs4d21e5g7YQhZqd28jiMRRAVcJIKt3LqPR5YXM/7cntwxNs3rOBJhVMBFIlTF4ePMysmjX1J7Hps4Qiepki9RAReJQLV1jjkL8jlwtJoXpmbSuZ0mZ5Av00FMkQj09Acb+UfJXh77xgiG9e7sdRyJUOqBi0SYjzaU8cxfS5iU1ZdJWZqcQZqnAi4SQUr3H+WuhQUM6dmJh68f7nUciXAq4CIR4nhNLd/Lyae21vHitEzaxcd5HUkinMbARSLEL95ez+odB3hxWiapyR29jiNRwO8euJn1M7OPzGy9ma01sznBDCbSmixbvYtXP93Gt7+SxvjhPb2OI1EikB54DXCPcy7PzDoBuWb2vnNuXZCyibQKJWWHmLu4kKyzkvjh+CFex5Eo4ncP3Dm32zmX57t8CFgP9AlWMJHW4OiJGma8lkf7+Diem5JBfJwOS0nLBeXRYmapQDrwWRPrppvZKjNbVV5eHozdicQE5xz3L1lDSflhnpmcTs8u7byOJFEm4AJuZonAYuBO51xl4/XOuZecc1nOuayUlJRAdycSM3I+386bBbu4e9wgLh6Q7HUciUIBFXAzi6e+eGc755YEJ5JI7FtTepCfLlvHpYNT+N5lA7yOI1EqkE+hGPAKsN4596vgRRKJbQePVjMjO5fkxASemnQ+Z2hyBvFTID3wi4H/A1xuZgW+n6uDlEskJtXVOe5eVMCeyiqen5pBUscEryNJFPP7Y4TOuRWAug4ip+HFjzfzYXEZP73uXNL7J3kdR6KcPrMkEiafbt7LE+9u4NqRvbnlorO8jiMxQAVcJAzKKquYPT+ftOSOPHLjeZqcQYJC50IRCbGa2jpmzc/nyPEacr49msS2etpJcOiRJBJiT7y3kc+37OPX3zyfQT06eR1HYoiGUERC6P11e3jx75uZOro/X0/XmSYkuFTARUJk+96j3LOogPP6dOHBa4Z5HUdikAq4SAhUVdcyMycXgBemZmhyBgkJjYGLhMDDb62jaGclL9+SRb9uHbyOIzFKPXCRIFuSV0rOZ9uZcek5jBvWw+s4EsNUwEWCaMMXh7j/jTWMTuvGPVcO8jqOxDgVcJEgOXy8hhmv5dKpXTzPTkmnjSZnkBDTGLhIEDjnuG9xIdv2HSXnW6Pp3kmTM0joqYsgEgSvfrKVtwt3c+9Vgxl99plex5FWQgVcJEB52/fz83fWM25oD6Z/5Wyv40grogIuEoB9R04wKzuPnl3a8eTEkZqcQcJKY+Aifqqrc9y5sICKIydYMmMMXTrEex1JWhn1wEX89NxHJXy8sZyfXHsuw/t08TqOtEIq4CJ+WLGpgqc+2MiN6X2YPKqf13GklVIBFzlNuw8e4/sL8hnYPZGf3TBckzOIZ1TARU5DdW0ds3LyOV5dy7xpmXRI0GEk8Y4efSKn4dHlxeRu28+zk9M5JyXR6zjSyqkHLtJCy9fs5uUVW7htTCrXjuztdRwRFXCRlthScYR7/1zI+f26cv/VQ72OIwKogIuc0rETtcx4LZf4OOP5qRkktNHTRiKDxsBFTuGhpUVs2HOI3992AX26tvc6jsi/qSshchKLVu7g9dxSZl8+kEsHd/c6jsh/UAEXacbaXQd5cGkRYwckM+eKgV7HEfkSFXCRJlRWVTMzO4+kDgk8ffP5xOkkVRKBNAYu0ohzjntfX83O/cdYMP1Czkxs63UkkSapBy7SyCsrtvDu2j3MnTCErNRuXscRaZYKuEgDK7fu45HlxYw/tyd3jE3zOo7ISQVUwM1svJltMLMSM5sbrFAiXqg4fJxZOXn0S2rPYxNH6CRVEvH8LuBmFgc8D0wAhgGTzWxYsIKJhFNtnWPOgnwOHK1m3rRMOrfT5AwS+QI5iDkKKHHO/RPAzBYA1wPrghGsoWc/3MSy1buCfbUi/1ZVU8uOfcd4/BsjGNqrs9dxRFokkALeB9jR4O9SYHTjjcxsOjAdoH///n7tKKVTWwb20JnfJLRuvSiViVmanEGiRyAFvKkBQvelBc69BLwEkJWV9aX1LXHzqP7cPMq/4i8iEqsCOYhZCjTsrvQFNM4hIhImgRTwlcBAM0szswTgZmBZcGKJiMip+D2E4pyrMbNZwLtAHPA759zaoCUTEZGTCuir9M65d4B3gpRFREROg76JKSISpVTARUSilAq4iEiUUgEXEYlS5pxf363xb2dm5cA2P/89GagIYpxgidRcELnZlOv0RGouiNxssZbrLOdcSuOFYS3ggTCzVc65LK9zNBapuSBysynX6YnUXBC52VpLLg2hiIhEKRVwEZEoFU0F/CWvAzQjUnNB5GZTrtMTqbkgcrO1ilxRMwYuIiL/KZp64CIi0oAKuIhIlIqoAm5mE81srZnVmVlWo3U/8k2evMHMrmrm/7uZ2ftmtsn3OykEGReaWYHvZ6uZFTSz3VYzW+PbblWwczSzz5+Y2c4G+a5uZruwTkZtZo+bWbGZFZrZG2bWtZntwtJmp7r9Vu8Z3/pCM8sIVZYG++xnZh+Z2Xrfc2BOE9tcamYHG9y/D4U6l2+/J71fvGgv334HN2iLAjOrNLM7G20TljYzs9+ZWZmZFTVY1qJ6FNDz0TkXMT/AUGAw8Dcgq8HyYcBqoC2QBmwG4pr4/8eAub7Lc4FHQ5z3SeChZtZtBZLD3H4/AX5wim3ifO13NpDga9dhIc71VaCN7/Kjzd0v4Wizltx+4GpgOfWzTl0IfBaG+64XkOG73AnY2ESuS4G3wvmYasn94kV7NXO/fkH9F17C3mbAJUAGUNRg2SnrUaDPx4jqgTvn1jvnNjSx6npggXPuuHNuC1BC/aTKTW33qu/yq8DXQxKU+l4HMAmYH6p9hMi/J6N2zp0A/jUZdcg4595zztX4/vxf6mdv8kpLbv/1wB9dvf8FuppZr1CGcs7tds7l+S4fAtZTP+9sNAh7ezXhCmCzc87fb3oHxDn3MbCv0eKW1KOAno8RVcBPoqkJlJt6cPdwzu2G+icE0D2Emb4C7HHObWpmvQPeM7Nc38TO4TLL9zb2d828ZWtpW4bK7dT31poSjjZrye33tI3MLBVIBz5rYvVFZrbazJab2blhinSq+8XrxxTUzwjWXGfKizaDltWjgNouoAkd/GFmHwA9m1j1gHNuaXP/1sSykH3+sYUZJ3Py3vfFzrldZtYdeN/Min2v0iHLBswD/of6tvkf6od4bm98FU38b8Bt2ZI2M7MHgBogu5mrCUmbNY7axLLGtz+sj7f/2LFZIrAYuNM5V9lodR71QwSHfcc33gQGhiHWqe4Xz9oLwOqndLwO+FETq71qs5YKqO3CXsCdc+P8+LeWTqC8x8x6Oed2+97ClYUio5m1AW4EMk9yHbt8v8vM7A3q3yoFXIxa2n5m9lvgrSZWhWQy6ha02a3ANcAVzjf418R1hKTNGmnJ7fdkwm4zi6e+eGc755Y0Xt+woDvn3jGzF8ws2TkX0pM2teB+8XqC8wlAnnNuT+MVXrWZT0vqUUBtFy1DKMuAm82srZmlUf8K+nkz293qu3wr0FyPPlDjgGLnXGlTK82so5l1+tdl6g/iFTW1bTA1Gne8oZl9hn0yajMbD9wHXOecO9rMNuFqs5bc/mXALb5PV1wIHPzXW+FQ8R1TeQVY75z7VTPb9PRth5mNov75uzfEuVpyv4S9vRpp9t2wF23WQEvqUWDPx1AfnT3NI7k3UP+KdBzYA7zbYN0D1B+t3QBMaLD8ZXyfWAHOBD4ENvl+dwtRzj8A3220rDfwju/y2dQfTV4NrKV+GCEc7fcnYA1Q6HsQ9Gqczff31dR/ymFzOLJRf9B5B1Dg+3nRyzZr6vYD3/3XfUr929rnfevX0OATUSHMNJb6t86FDdrp6ka5ZvnaZjX1B4PHhCFXk/eL1+3VIF8H6gtylwbLwt5m1L+A7AaqfTXsjubqUTCfj/oqvYhIlIqWIRQREWlEBVxEJEqpgIuIRCkVcBGRKKUCLiISpVTARUSilAq4iEiU+v/S1kOK/g8o4QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create a relu object and make the prediction\n",
    "\n",
    "RELU = nn.ReLU()\n",
    "yhat = RELU(z)\n",
    "plt.plot(z.numpy(), yhat.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For custom modules, call the relu object from the nn.functional, which applies the element-wise sigmoid from the function module and plots the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAc0klEQVR4nO3deXhU5d3/8ffXkLCFJZiwg4myi2AWQZH6uGAF61K1UFl+6qMtLRSKS61Uf9rWp611q3XFWm1ra8JiQeGnUrfaelF9FLIQAgQIZQsgSdjCFshy//7ItFcaEwiznZnJ53VduTI552TOZ+6Z+c4995k5tznnEBGR6HOG1wFERMQ/KuAiIlFKBVxEJEqpgIuIRCkVcBGRKNUmnDtLTk52qamp4dyliEjUy83NrXDOpTReHtYCnpqayqpVq8K5SxGRqGdm25pariEUEZEopQIuIhKlVMBFRKLUKQu4mf3OzMrMrKjBsm5m9r6ZbfL9TgptTBERaawlPfA/AOMbLZsLfOicGwh86PtbRETC6JQF3Dn3MbCv0eLrgVd9l18Fvh7cWCIicir+joH3cM7tBvD97t7chmY23cxWmdmq8vJyP3cnIiKNhfwgpnPuJedclnMuKyXlS59DFxGJaVXVtfxk2Vr2HTkR9Ov2t4DvMbNeAL7fZcGLJCISOx5aWsQfPtlK0c6DQb9ufwv4MuBW3+VbgaXBiSMiEjsWrdzBolWlzL58AJcMCv4IREs+Rjgf+BQYbGalZnYH8EvgSjPbBFzp+1tERHzW7arkwaVFjDnnTO4cNygk+zjluVCcc5ObWXVFkLOIiMSEyqpqZmbn0rVDPM9MTifuDAvJfsJ6MisRkVjnnOOHrxeyY/8xFky/kOTEtiHbl75KLyISRK+s2MJf1n7B3PFDuCC1W0j3pQIuIhIkK7fu45HlxXx1WA++9ZW0kO9PBVxEJAgqDh9nVk4efZPa8/jEkZiFZty7IY2Bi4gEqLbOMWdBPgeOVrNk5gV0aR8flv2qgIuIBOjpDzbyj5K9PHbTCM7t3SVs+9UQiohIAD7aUMYzfy1hYmZfJl3QL6z7VgEXEfHTzgPHuGthAUN6duLh64eHff8q4CIifjhRU8fM7Dxqah3zpmXSPiEu7Bk0Bi4i4odfvLOe1TsOMG9qBmnJHT3JoB64iMhp+n+rd/GHT7Zyx9g0JpzXy7McKuAiIqehpOwwcxcXknlWEnMnDPE0iwq4iEgLHT1Rw8zsXNrGx/HclHTi47wtoRoDFxFpAecc9y9Zw6ayw/zx9lH06tLe60jqgYuItETO59t5s2AXd40bxFcGRsb0kCrgIiKnsKb0ID9dto5LBqUw67IBXsf5NxVwEZGTOHi0mhnZuSQnJvDrb57PGSGanMEfGgMXEWlGXZ3j7kUF7KmsYuF3LqJbxwSvI/0H9cBFRJrxm4//yYfFZTxw9VAy+id5HedLVMBFRJrw6ea9PP5uMV8b0Ytbx6R6HadJKuAiIo2UVVYxe34+qckdefSmEWGZnMEfGgMXEWmgpraO2fPzOXy8muxvjSaxbeSWychNJiLigSff38hnW/bxq0kjGdyzk9dxTkpDKCIiPu+v28O8v21m8qj+3JjR1+s4p6QCLiICbN97lHsWFTC8T2d+fO0wr+O0iAq4iLR6VdW1zMzJBWDe1EzaxYd/cgZ/aAxcRFq9h99aR9HOSn57Sxb9unXwOk6LqQcuIq3akrxScj7bznf/6xyuHNbD6zinRQVcRFqtDV8c4oE3ihiV1o0ffHWQ13FOmwq4iLRKh4/XMCM7l45t2/Dc5HTaeDw5gz+iL7GISICcc9y3uJCtFUd4dnI63Tu38zqSXwIq4GZ2l5mtNbMiM5tvZtHZCiLSqvzx0228XbibH1w1mIvOOdPrOH7zu4CbWR/g+0CWc244EAfcHKxgIiKhkL99Pz97ex1XDOnOdy85x+s4AQl0CKUN0N7M2gAdgF2BRxIRCY39R07wvew8enRux5OTRkbU5Az+8LuAO+d2Ak8A24HdwEHn3HuNtzOz6Wa2ysxWlZeX+59URCQAdXWOOxcWUHH4BC9MzaBrh8ianMEfgQyhJAHXA2lAb6CjmU1rvJ1z7iXnXJZzLislJTImAhWR1ue5j0r4+8ZyfnzdMEb07ep1nKAIZAhlHLDFOVfunKsGlgBjghNLRCR4Vmyq4KkPNnJDeh+mjOrvdZygCaSAbwcuNLMOVn+28yuA9cGJJSISHLsPHuP7C/IZ2D2Rn98wPGInZ/BHIGPgnwF/BvKANb7reilIuUREAlZdW8esnHyqqmt5YWomHRJi6/RPAd0a59yPgR8HKYuISFA9uryY3G37eWZyOgO6J3odJ+j0TUwRiUl/KdrNyyu2cOtFZ3HdyN5exwkJFXARiTlbKo5w7+uFjOzXlfu/NtTrOCGjAi4iMaWqupYZr+USF2c8PyWdtm2iY3IGf8TWiL6ItHoPLS2i+ItD/P6/L6BvUvRMzuAP9cBFJGYsWrmDRatKmX35AC4b3N3rOCGnAi4iMWHdrkoeXFrExQPO5M5x0Tc5gz9UwEUk6lVWVTMzO5euHeJ5+uZ04qL8JFUtpTFwEYlqzjnufX01O/YfY+H0C0lObOt1pLBRD1xEotorK7bw7to9/GjCELJSu3kdJ6xUwEUkaq3cuo9Hlhdz1bk9uGNsmtdxwk4FXESiUsXh48zKyaNvUnsenzgypk5S1VIaAxeRqFNb55izIJ8DR6tZMvMCOreL9zqSJ1TARSTqPP3BRv5RspfHbhrBub27eB3HMxpCEZGo8tGGMp75awkTM/sy6YJ+XsfxlAq4iESNnQeOcdfCAob07MTD1w/3Oo7nVMBFJCqcqKljZnYetbWOedMyaZ8QuyepaimNgYtIVPjFO+tZveMAL07LIC25o9dxIoJ64CIS8Zat3sUfPtnKt8amMX54L6/jRAwVcBGJaCVlh5i7uJDMs5K4b8IQr+NEFBVwEYlYR0/UMOO1PNrHx/H8lAzi41SyGtIYuIhEJOcc9y9ZQ0n5Yf50+2h6dmnndaSIo5czEYlIOZ9v582CXdw1bhBjByZ7HSciqYCLSMRZU3qQny5bxyWDUph12QCv40QsFXARiSgHj1YzIzuX5MQEfv3N8zmjlUzO4A+NgYtIxKirc9y9qIA9lVUs/M5FdOuY4HWkiKYeuIhEjN98/E8+LC7jgauHktE/yes4EU8FXEQiwqeb9/L4u8VcM6IXt45J9TpOVFABFxHPlVVWMXt+PqnJHfnlTSNa5eQM/tAYuIh4qqa2jlnz8zlyvIacb48msa3KUkuppUTEU0+8t5HPt+zjV5NGMqhHJ6/jRBUNoYiIZ95ft4cX/76ZKaP7c2NGX6/jRJ2ACriZdTWzP5tZsZmtN7OLghVMRGLb9r1HuWdRAcP7dOaha4Z5HScqBTqE8jTwF+fcN8wsAegQhEwiEuOqqmuZmZMLwLypmbSL1+QM/vC7gJtZZ+AS4DYA59wJ4ERwYolILHv4rXUU7azkt7dk0a+b+n3+CmQI5WygHPi9meWb2ctm9qVpMsxsupmtMrNV5eXlAexORGLBkrxScj7bznf/6xyuHNbD6zhRLZAC3gbIAOY559KBI8Dcxhs5515yzmU557JSUlIC2J2IRLsNXxzigTeKGJXWjR98dZDXcaJeIAW8FCh1zn3m+/vP1Bd0EZEvOXy8hhnZuXRs24bnJqfTRpMzBMzvFnTOfQHsMLPBvkVXAOuCkkpEYopzjvsWF7K14gjPTUmne2dNzhAMgX4KZTaQ7fsEyj+B/w48kojEmj9+uo23C3dz3/ghXHj2mV7HiRkBFXDnXAGQFZwoIhKL8rbv52dvr2Pc0O5855KzvY4TUzQIJSIhs+/ICWZl59GjczuenKjJGYJN50IRkZCoq3PcubCAisMnWDxjDF06xHsdKeaoBy4iIfHcRyV8vLGcH183jPP6dvE6TkxSAReRoFuxqYKnPtjIDel9mDKqv9dxYpYKuIgE1e6Dx/j+gnwGdk/k5zcM1+QMIaQCLiJBU11bx6ycfKqqa3lhaiYdEnSYLZTUuiISNI8uLyZ3236enZzOgO6JXseJeeqBi0hQ/KVoNy+v2MJtY1K5dmRvr+O0CirgIhKwLRVHuPf1Qs7v15X7rx7qdZxWQwVcRAJSVV3LjNdyiYsznp+aQUIblZVw0Ri4iATkoaVFbNhziN/fdgF9urb3Ok6ropdKEfHbopU7WLSqlNmXDeDSwd29jtPqqICLiF/W7jrIg0uLuHjAmcwZp8kZvKACLiKnrbKqmpnZeXTtEM/TN6cTp5NUeUJj4CJyWpxz3Pv6akr3H2Ph9AtJTmzrdaRWSz1wETktr6zYwrtr9/CjCUPISu3mdZxWTQVcRFps5dZ9PLK8mKvO7cEdY9O8jtPqqYCLSItUHD7OrJw8+iW15/GJI3WSqgigAi4ip1Rb55izIJ8DR6t5YWomndtpcoZIoIOYInJKT3+wkX+U7OWxb4xgWO/OXscRH/XAReSkPtpQxjN/LWFSVl8mZfXzOo40oAIuIs3aeeAYdy0sYEjPTjx8/XCv40gjKuAi0qTjNbXMzM6jttYxb1om7eLjvI4kjWgMXESa9Iu317N6xwFenJZBWnJHr+NIE9QDF5EvWbZ6F69+uo1vjU1j/PBeXseRZqiAi8h/KCk7xNzFhWSelcR9E4Z4HUdOQgVcRP7t6IkaZryWR/v4OJ6fkkF8nEpEJNMYuIgA9Sepun/JGkrKD/On20fTs0s7ryPJKejlVUQAyPl8O28W7OLucYMYOzDZ6zjSAirgIsKa0oP8dNk6Lh2cwvcuG+B1HGkhFXCRVu7g0WpmZOeSnJjAU5PO5wxNzhA1Ai7gZhZnZvlm9lYwAolI+NTVOe5eVMCeyiqen5pBUscEryPJaQhGD3wOsD4I1yMiYfabj//Jh8Vl/N+vDSO9f5LXceQ0BVTAzawv8DXg5eDEEZFw+XTzXh5/t5hrRvTilovO8jqO+CHQHvivgR8Cdc1tYGbTzWyVma0qLy8PcHciEgxllVXMnp9PanJHfnnTCE3OEKX8LuBmdg1Q5pzLPdl2zrmXnHNZzrmslJQUf3cnIkFSU1vHrPn5HDlew4vTMklsq6+DRKtAeuAXA9eZ2VZgAXC5mb0WlFQiEjJPvLeRz7fs4+c3DGdQj05ex5EA+F3AnXM/cs71dc6lAjcDf3XOTQtaMhEJuvfX7eHFv29myuj+3JjR1+s4EiB9Dlykldi+9yj3LCpgeJ/OPHTNMK/jSBAEZfDLOfc34G/BuC4RCb6q6lpm5tQfrpo3VZMzxAodvRBpBR5+ax1FOyt5+ZYs+nXr4HUcCRINoYjEuCV5peR8tp0Zl57DuGE9vI4jQaQCLhLDNnxxiAfeKGJ0WjfuuXKQ13EkyFTARWLU4eM1zMjOJbFdG56dkk4bTc4Qc3SPisQg5xz3LS5ka8URnp2cTvdOmpwhFqmAi8SgVz/ZytuFu7n3qiFcePaZXseREFEBF4kxedv38/N31jNuaHe+c8nZXseREFIBF4kh+46cYFZ2Hj06t+PJiZqcIdbpc+AiMaKuznHnwgIqDp9g8YwxdOkQ73UkCTEVcJEY8dxHJXy8sZxf3HAe5/Xt4nUcCQMNoYjEgBWbKnjqg43cmN6HyaP6eR1HwkQFXCTK7T54jO8vyGdg90R+dsNwTc7QiqiAi0Sx6to6ZuXkc7y6lnnTMumQoFHR1kT3tkgUe3R5Mbnb9vPs5HTOSUn0Oo6EmXrgIlHqL0W7eXnFFm4bk8q1I3t7HUc8oAIuEoW2VBzh3tcLOb9fV+6/eqjXccQjKuAiUebYiVpmvJZLXJzx/NQMEtroadxaaQxcJMo8tLSIDXsO8fvbLqBP1/ZexxEP6aVbJIosWrmD13NLmX3ZAC4d3N3rOOIxFXCRKLF210EeXFrE2AHJzBmnyRlEBVwkKlRWVTMzO4+kDgk8ffP5xOkkVYLGwEUinnOOe19fzc79x1gw/ULOTGzrdSSJEOqBi0S4V1Zs4d21e5g7YQhZqd28jiMRRAVcJIKt3LqPR5YXM/7cntwxNs3rOBJhVMBFIlTF4ePMysmjX1J7Hps4Qiepki9RAReJQLV1jjkL8jlwtJoXpmbSuZ0mZ5Av00FMkQj09Acb+UfJXh77xgiG9e7sdRyJUOqBi0SYjzaU8cxfS5iU1ZdJWZqcQZqnAi4SQUr3H+WuhQUM6dmJh68f7nUciXAq4CIR4nhNLd/Lyae21vHitEzaxcd5HUkinMbARSLEL95ez+odB3hxWiapyR29jiNRwO8euJn1M7OPzGy9ma01sznBDCbSmixbvYtXP93Gt7+SxvjhPb2OI1EikB54DXCPcy7PzDoBuWb2vnNuXZCyibQKJWWHmLu4kKyzkvjh+CFex5Eo4ncP3Dm32zmX57t8CFgP9AlWMJHW4OiJGma8lkf7+Diem5JBfJwOS0nLBeXRYmapQDrwWRPrppvZKjNbVV5eHozdicQE5xz3L1lDSflhnpmcTs8u7byOJFEm4AJuZonAYuBO51xl4/XOuZecc1nOuayUlJRAdycSM3I+386bBbu4e9wgLh6Q7HUciUIBFXAzi6e+eGc755YEJ5JI7FtTepCfLlvHpYNT+N5lA7yOI1EqkE+hGPAKsN4596vgRRKJbQePVjMjO5fkxASemnQ+Z2hyBvFTID3wi4H/A1xuZgW+n6uDlEskJtXVOe5eVMCeyiqen5pBUscEryNJFPP7Y4TOuRWAug4ip+HFjzfzYXEZP73uXNL7J3kdR6KcPrMkEiafbt7LE+9u4NqRvbnlorO8jiMxQAVcJAzKKquYPT+ftOSOPHLjeZqcQYJC50IRCbGa2jpmzc/nyPEacr49msS2etpJcOiRJBJiT7y3kc+37OPX3zyfQT06eR1HYoiGUERC6P11e3jx75uZOro/X0/XmSYkuFTARUJk+96j3LOogPP6dOHBa4Z5HUdikAq4SAhUVdcyMycXgBemZmhyBgkJjYGLhMDDb62jaGclL9+SRb9uHbyOIzFKPXCRIFuSV0rOZ9uZcek5jBvWw+s4EsNUwEWCaMMXh7j/jTWMTuvGPVcO8jqOxDgVcJEgOXy8hhmv5dKpXTzPTkmnjSZnkBDTGLhIEDjnuG9xIdv2HSXnW6Pp3kmTM0joqYsgEgSvfrKVtwt3c+9Vgxl99plex5FWQgVcJEB52/fz83fWM25oD6Z/5Wyv40grogIuEoB9R04wKzuPnl3a8eTEkZqcQcJKY+Aifqqrc9y5sICKIydYMmMMXTrEex1JWhn1wEX89NxHJXy8sZyfXHsuw/t08TqOtEIq4CJ+WLGpgqc+2MiN6X2YPKqf13GklVIBFzlNuw8e4/sL8hnYPZGf3TBckzOIZ1TARU5DdW0ds3LyOV5dy7xpmXRI0GEk8Y4efSKn4dHlxeRu28+zk9M5JyXR6zjSyqkHLtJCy9fs5uUVW7htTCrXjuztdRwRFXCRlthScYR7/1zI+f26cv/VQ72OIwKogIuc0rETtcx4LZf4OOP5qRkktNHTRiKDxsBFTuGhpUVs2HOI3992AX26tvc6jsi/qSshchKLVu7g9dxSZl8+kEsHd/c6jsh/UAEXacbaXQd5cGkRYwckM+eKgV7HEfkSFXCRJlRWVTMzO4+kDgk8ffP5xOkkVRKBNAYu0ohzjntfX83O/cdYMP1Czkxs63UkkSapBy7SyCsrtvDu2j3MnTCErNRuXscRaZYKuEgDK7fu45HlxYw/tyd3jE3zOo7ISQVUwM1svJltMLMSM5sbrFAiXqg4fJxZOXn0S2rPYxNH6CRVEvH8LuBmFgc8D0wAhgGTzWxYsIKJhFNtnWPOgnwOHK1m3rRMOrfT5AwS+QI5iDkKKHHO/RPAzBYA1wPrghGsoWc/3MSy1buCfbUi/1ZVU8uOfcd4/BsjGNqrs9dxRFokkALeB9jR4O9SYHTjjcxsOjAdoH///n7tKKVTWwb20JnfJLRuvSiViVmanEGiRyAFvKkBQvelBc69BLwEkJWV9aX1LXHzqP7cPMq/4i8iEqsCOYhZCjTsrvQFNM4hIhImgRTwlcBAM0szswTgZmBZcGKJiMip+D2E4pyrMbNZwLtAHPA759zaoCUTEZGTCuir9M65d4B3gpRFREROg76JKSISpVTARUSilAq4iEiUUgEXEYlS5pxf363xb2dm5cA2P/89GagIYpxgidRcELnZlOv0RGouiNxssZbrLOdcSuOFYS3ggTCzVc65LK9zNBapuSBysynX6YnUXBC52VpLLg2hiIhEKRVwEZEoFU0F/CWvAzQjUnNB5GZTrtMTqbkgcrO1ilxRMwYuIiL/KZp64CIi0oAKuIhIlIqoAm5mE81srZnVmVlWo3U/8k2evMHMrmrm/7uZ2ftmtsn3OykEGReaWYHvZ6uZFTSz3VYzW+PbblWwczSzz5+Y2c4G+a5uZruwTkZtZo+bWbGZFZrZG2bWtZntwtJmp7r9Vu8Z3/pCM8sIVZYG++xnZh+Z2Xrfc2BOE9tcamYHG9y/D4U6l2+/J71fvGgv334HN2iLAjOrNLM7G20TljYzs9+ZWZmZFTVY1qJ6FNDz0TkXMT/AUGAw8Dcgq8HyYcBqoC2QBmwG4pr4/8eAub7Lc4FHQ5z3SeChZtZtBZLD3H4/AX5wim3ifO13NpDga9dhIc71VaCN7/Kjzd0v4Wizltx+4GpgOfWzTl0IfBaG+64XkOG73AnY2ESuS4G3wvmYasn94kV7NXO/fkH9F17C3mbAJUAGUNRg2SnrUaDPx4jqgTvn1jvnNjSx6npggXPuuHNuC1BC/aTKTW33qu/yq8DXQxKU+l4HMAmYH6p9hMi/J6N2zp0A/jUZdcg4595zztX4/vxf6mdv8kpLbv/1wB9dvf8FuppZr1CGcs7tds7l+S4fAtZTP+9sNAh7ezXhCmCzc87fb3oHxDn3MbCv0eKW1KOAno8RVcBPoqkJlJt6cPdwzu2G+icE0D2Emb4C7HHObWpmvQPeM7Nc38TO4TLL9zb2d828ZWtpW4bK7dT31poSjjZrye33tI3MLBVIBz5rYvVFZrbazJab2blhinSq+8XrxxTUzwjWXGfKizaDltWjgNouoAkd/GFmHwA9m1j1gHNuaXP/1sSykH3+sYUZJ3Py3vfFzrldZtYdeN/Min2v0iHLBswD/of6tvkf6od4bm98FU38b8Bt2ZI2M7MHgBogu5mrCUmbNY7axLLGtz+sj7f/2LFZIrAYuNM5V9lodR71QwSHfcc33gQGhiHWqe4Xz9oLwOqndLwO+FETq71qs5YKqO3CXsCdc+P8+LeWTqC8x8x6Oed2+97ClYUio5m1AW4EMk9yHbt8v8vM7A3q3yoFXIxa2n5m9lvgrSZWhWQy6ha02a3ANcAVzjf418R1hKTNGmnJ7fdkwm4zi6e+eGc755Y0Xt+woDvn3jGzF8ws2TkX0pM2teB+8XqC8wlAnnNuT+MVXrWZT0vqUUBtFy1DKMuAm82srZmlUf8K+nkz293qu3wr0FyPPlDjgGLnXGlTK82so5l1+tdl6g/iFTW1bTA1Gne8oZl9hn0yajMbD9wHXOecO9rMNuFqs5bc/mXALb5PV1wIHPzXW+FQ8R1TeQVY75z7VTPb9PRth5mNov75uzfEuVpyv4S9vRpp9t2wF23WQEvqUWDPx1AfnT3NI7k3UP+KdBzYA7zbYN0D1B+t3QBMaLD8ZXyfWAHOBD4ENvl+dwtRzj8A3220rDfwju/y2dQfTV4NrKV+GCEc7fcnYA1Q6HsQ9Gqczff31dR/ymFzOLJRf9B5B1Dg+3nRyzZr6vYD3/3XfUr929rnfevX0OATUSHMNJb6t86FDdrp6ka5ZvnaZjX1B4PHhCFXk/eL1+3VIF8H6gtylwbLwt5m1L+A7AaqfTXsjubqUTCfj/oqvYhIlIqWIRQREWlEBVxEJEqpgIuIRCkVcBGRKKUCLiISpVTARUSilAq4iEiU+v/S1kOK/g8o4QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Use the build-in function to make the prediction\n",
    "\n",
    "yhat = torch.relu(z)\n",
    "plt.plot(z.numpy(), yhat.numpy())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref3\"></a>\n",
    "<h2> Compare Activation Functions </h2>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f3538e8db70>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD5CAYAAADLL+UrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0m0lEQVR4nO3dd3xUVd7H8c9JIwlJSO8hoYSaUEIIRWlWYFcQsa4FsSC4rri7NnRVdB/X7rPsrqIsi6DygBVFRVTUBawkofciAUJ673XO88cdIGBCIJnkziS/9+s1r3sn987cXy7km5M7556jtNYIIYToHJzMLkAIIUT7kdAXQohOREJfCCE6EQl9IYToRCT0hRCiE5HQF0KITsSltW+glIoC3gRCAQuwSGu94Ix9FLAAmAxUALdqrTc3996BgYE6JiamtSUKIUSnkZqamqe1Dmpqe6tDH6gD/qy13qyU8gZSlVJfaa13N9hnEhBrfYwAFlqXZxUTE0NKSooNShRCiM5BKXXkbNtbfXlHa515otWutS4F9gARZ+w2FXhTG34CfJVSYa09thBCiPNj02v6SqkYYCjw8xmbIoBjDZ6n8+tfDCfeY5ZSKkUplZKbm2vL8oQQotOzWegrpbyAD4D7tNYlZ25u5CWNjv+gtV6ktU7UWicGBTV5WUoIIUQL2OKaPkopV4zAX661/rCRXdKBqAbPI4GMlhyrtraW9PR0qqqqWvLyTsPd3Z3IyEhcXV3NLkUIYUds0XtHAf8B9mitX25it9XAPUqplRgf4BZrrTNbcrz09HS8vb2JiYnBOLQ4k9aa/Px80tPT6dGjh9nlCCHsiC1a+hcANwM7lFJbrV97BOgOoLV+DViD0V3zIEaXzZktPVhVVZUEfjOUUgQEBCCfiQghztTq0Ndaf0fj1+wb7qOB37f2WCdI4DdPzpEQojFyR64QQtiRzUcLeX39oTZ7fwn9NuLl5WV2CUIIB/PlrixuWPQTKzYdpby6rk2OIaHfClprLBaL2WUIITqAt35MY/bbqfQL8+GDOaPp2sUmnSt/RUL/PKWlpdG/f3/uvvtuEhIS+Otf/8rw4cMZNGgQTzzxxK/2/+9//8tvf/vbk8/vueceli5d2o4VCyHsmcWiefbzvTz28S4u6hfMijtHEODVpc2O1za/StrJk5/sYnfGmfeBtc6AcB+euGLgWffZt28fb7zxBldeeSXvv/8+mzZtQmvNlClT2LBhA2PHjrVpTUKIjqmmzsKD72/jo60Z/G5Ed56aMhAX57Zti0tLvwWio6MZOXIkX375JV9++SVDhw4lISGBvXv3cuDAAbPLE0I4gJKqWm59YxMfbc3ggcv78vSVcW0e+ODgLf3mWuRtpWvXroBxTX/evHncddddTe7r4uJy2nV/uZNYCJFZXMnMN5I5mFPGy9cO5qqEyHY7trT0W+Hyyy9nyZIllJWVAXD8+HFycnJO2yc6Oprdu3dTXV1NcXExX3/9tRmlCiHsxL6sUq569QfSCyt5Y+bwdg18cPCWvtkuu+wy9uzZw6hRowCjm+bbb79NcHDwyX2ioqK49tprGTRoELGxsQwdOtSscoUQJvvxUD6z3krBw9WZd+4aycDwbu1egzJulrVPiYmJ+sxJVPbs2UP//v1NqsixyLkSwn6s3pbB/e9uIzrAk6W3JRHh69Emx1FKpWqtE5vaLi19IYRoQ1pr/r3xF/62Zi9JPfz5982JdPM0b/RbCX0hhGgj9RbNXz/dzdIf0vjNoDBeumYw7q7OptYkoS+EEG2gqrae+1ZuZe2uLO64sAePTO6Pk5P5AyFK6AshhI0Vltdwx5spbD5ayGO/HcDtF9rPvBYS+kIIYUPHCiqY8cYm0gsr+dcNCfxmUJjZJZ1GQl8IIWxkR3oxM5cmU1NXz1u3JTGiZ4DZJf2K3JxlA3fccQe7d+9u02NMnjyZoqKiX319/vz5vPjii216bCFE877dl8N1i36ki4sTH9492i4DH6SlbxOLFy9u82OsWbOmzY8hhGiZd5OPMW/VDvqGePPGzOGE+LibXVKTpKV/nsrLy/nNb37D4MGDiYuL45133mH8+PGcuInsP//5D3369GH8+PHceeed3HPPPQDceuutzJkzhwkTJtCzZ0/Wr1/PbbfdRv/+/bn11ltPvv+KFSuIj48nLi6Ohx566OTXY2JiyMvLA+Dpp5+mb9++XHLJJezbt6/9vnkhxGm01vx93X4e/GA7o3sF8M5dI+068MHRW/qfPwxZO2z7nqHxMOnZJjevXbuW8PBwPvvsMwCKi4tZuHAhABkZGfz1r39l8+bNeHt7c9FFFzF48OCTry0sLOSbb75h9erVXHHFFXz//fcsXryY4cOHs3XrVoKDg3nooYdITU3Fz8+Pyy67jI8++ogrr7zy5HukpqaycuVKtmzZQl1dHQkJCQwbNsy250AI0azaegt/WbWTd1KOcVVCBM9NH4RrO4yS2Vr2X6GdiY+PZ926dTz00ENs3LiRbt1OjZ2xadMmxo0bh7+/P66urlxzzTWnvfaKK65AKUV8fDwhISHEx8fj5OTEwIEDSUtLIzk5mfHjxxMUFISLiws33ngjGzZsOO09Nm7cyLRp0/D09MTHx4cpU6a0y/cthDilvLqOO99M4Z2UY/zhot68dM1ghwh8cPSW/lla5G2lT58+pKamsmbNGubNm8dll112cltz4xh16WLMhuPk5HRy/cTzuro6XFzO7Z9DKfNv8BCis8otrea2pcnsyijm6Wlx3Dgi2uySzotj/GqyIxkZGXh6enLTTTdx//33s3nz5pPbkpKSWL9+PYWFhdTV1fHBBx+c13uPGDGC9evXk5eXR319PStWrGDcuHGn7TN27FhWrVpFZWUlpaWlfPLJJzb5voQQzTuUW8ZVC7/nYE4Zi25OdLjAB0dv6Ztgx44dPPDAAzg5OeHq6srChQu5//77AYiIiOCRRx5hxIgRhIeHM2DAgNMu/zQnLCyMZ555hgkTJqC1ZvLkyUydOvW0fRISErjuuusYMmQI0dHRjBkzxqbfnxCicalHCrhjWQpOSrFi1kiGRPmaXVKLyNDKNlZWVoaXlxd1dXVMmzaN2267jWnTpplSi72fKyEcxdqdWcxduYWwbu4snZlETGBXs0tqUnNDK8vlHRubP38+Q4YMIS4ujh49epzW80YI4Xje/DGNOctT6R/mwwdzRtt14J8Lm1zeUUotAX4L5Git4xrZPh74GDhs/dKHWuunbHFseyN3xwrRMVgsmue+2Mvr63/hkv7B/POGBDzczB0W2RZsdU1/KfAv4M2z7LNRa/1bGx1PCCHaTHVdPQ++v52Pt2Zw44juPDllIC4O0iWzOTYJfa31BqVUjC3eSwghzFRcWcvst1L58Zd8HpzYlznjenWobtLt+atrlFJqm1Lqc6XUwKZ2UkrNUkqlKKVScnNz27E8IURnl1FUybWv/UhyWgEvXzuYu8f37lCBD+3XZXMzEK21LlNKTQY+AmIb21FrvQhYBEbvnXaqTwjRye3NKuHWJcmUVdexdGYSF8YGml1Sm2iXlr7WukRrXWZdXwO4KqUc8owWFRXx6quvtvj1DQdnE0LYhx8O5XHNwh/RaN69a1SHDXxop9BXSoUq699ISqkk63Hz2+PYttba0BdC2JePtx5nxpJNhHZz58O7L2BAuI/ZJbUpm4S+UmoF8CPQVymVrpS6XSk1Wyk127rL1cBOpdQ24B/A9dqe7wo7i4cffphDhw4xZMgQ/vjHP3LxxReTkJBAfHw8H3/8MQBpaWn079+fO++8k4EDB3LZZZdRWVl58j3ee+89kpKS6NOnDxs3bjTrWxGiU9Na8/r6Q8xduZWE7n68P3s0Eb4eZpfV5mzVe+eGZrb/C6NLp009t+k59hbstel79vPvx0NJDzW5/dlnn2Xnzp1s3bqVuro6Kioq8PHxIS8vj5EjR54c9fLAgQOsWLGCf//731x77bV88MEH3HTTTQDU1dWxadMm1qxZw5NPPsm6dets+j0IIc6u3qJ56pNdLPvxCL8ZFMbL1w6mi4vj98E/FzL2TitorXnkkUfYsGEDTk5OHD9+nOzsbAB69OjBkCFDABg2bBhpaWknX3fVVVc1+nUhRNurqq1n7sotfLErmzvH9GDepP44OXWsHjpn49Chf7YWeXtYvnw5ubm5pKam4urqSkxMDFVVVQCnDZ3s7Ox82uWdE9ucnZ2pq6tr36KF6MQKy2u4fVkyW44V8fhvB3DbhT3MLqndOXTom8Hb25vS0lLAmDUrODgYV1dXvv32W44cOWJydUKIphwrqGDGkk2kF1Xyyu8SmBwfZnZJppDQP08BAQFccMEFxMXFMXz4cPbu3UtiYiJDhgyhX79+ZpcnhGjEjvRiZi5NprbewvI7RjA8xt/skkwjQyt3YHKuhIBv9+Xw++Wb8fN0Y9ltw+kd7G12SW2quaGVpaUvhOiw3kk+yiOrdtIv1Js3bh1OsI+72SWZTkJfCNHhaK35+7oDLPj6AGP7BPHqjQl4dZG4AwcNfa11hxsEydbs+bKdEG2ptt7Co6t28G5KOlcPi+SZq+Jx7SDDItuCw4W+u7s7+fn5BAQESPA3QWtNfn4+7u7yp6zoXMqr67h7+WbW78/l3ot688dL+0hOnMHhQj8yMpL09HRk2OWzc3d3JzIy0uwyhGg3OaVV3LY0mT2ZpTxzVTw3JHU3uyS75HCh7+rqSo8ene+GCiFE0w7lljFjySbyy2r49y3DuKhfiNkl2S2HC30hhGgo9UgBty9LwcVJ8c5dIxkU6Wt2SXZNQl8I4bDW7sxi7sothPt6sGxmEt0DPM0uye5J6AshHNKyH9KY/8kuhkT58p8Zw/Hv6mZ2SQ5BQl8I4VAsFs1za/fy+oZfuGxACAuuH4qHW+cYFtkWJPSFEA6juq6eB97bzuptGdw8Mpr5Uwbi3ImGRbYFCX0hhEMorqzlrrdS+OmXAh6e1I+7xvaUPvgtIKEvhLB7GUWVzHwjmV/yylhw/RCmDokwuySHJaEvhLBre7NKuHVJMuXVdSybmcTo3oFml+TQJPSFEHbrh4N53PVWKl27uPDu7FH0D/MxuySHJ6EvhLBLH289zv3vbaNHYFeWzkwi3NfD7JI6BAl9IYRd0Vrz2vpfeG7tXkb29Of1mxPp5uFqdlkdhoS+EMJu1Fs0T36yizd/PMIVg8N58ZpBdHGRPvi2JKEvhLALVbX13LtiC1/uzuaucT156PJ+OEkffJuT0BdCmK6gvIY7liWz5VgRT04ZyIzRMWaX1GHZZDoZpdQSpVSOUmpnE9uVUuofSqmDSqntSqkEWxxXCOH4juZXMH3hD+zKKGHhjcMk8NuYreYQWwpMPMv2SUCs9TELWGij4wohHNj29CKuWvg9hRU1/N+dI5gYF2p2SR2eTUJfa70BKDjLLlOBN7XhJ8BXKRVmi2MLIRzTt3tzuO71n3B3deaDOaMZFu1vdkmdQnvNFhwBHGvwPN36tV9RSs1SSqUopVJkSkQhOqaVm45yx5sp9Aruyod3j6ZXkJfZJXUa7RX6jX0ErxvbUWu9SGudqLVODAoKauOyhBDtSWvNy1/t5+EPd3Bh70DemTWKYG93s8vqVNqr9046ENXgeSSQ0U7HFkLYgdp6C498uIP3UtO5NjGSp6fF4+rcXu1O+2XRFkprSimoKqCgqoDCqkLqLHVM7HG2j0lbrr1CfzVwj1JqJTACKNZaZ7bTsYUQJiurruPu5ZvZsD+X+y6JZe7FsR1+WOSquipyK3PJrcglpzKH3Irck89zK3NPBnxhVSH1uv601/p18bPv0FdKrQDGA4FKqXTgCcAVQGv9GrAGmAwcBCqAmbY4rhDC/uWUVDFzaTJ7s0p5bno81w3vbnZJraa1pqCqgMzyTDLKMk4uM8ozyCzLJLM8k5Kakl+9zs3JjSDPIAI9Aon0imRQ4CD83f3xc/fDz90Pf3f/k4+2YpPQ11rf0Mx2DfzeFscSQjiOgzmlzFiSTGFFDYtnJDKhb7DZJZ0zrTWF1YUcLTlKWknaacujpUeprKs8bf+url0J9wonvGs4Q4KHENo1lCCPIOPhGUSwZzA+bj6m/4Ujd+QKIdpEcloBdyxLwdVZsXLWSAZF+ppdUpMKqwo5UHiAA0UHTi4PFx+mtKb05D7OyplI70i6e3dneOhwIr0jCesaRrhXOGFdw+wi0M+FhL4QwuY+35HJ3He2EuHrwbKZSXQP8DS7JMD40PRIyRF25u1kT8EeDhYe5EDRAfIq807u49vFl1i/WCb3mEy0T/TJR7hXOK5Ojj/ap4S+EMKmlnx3mL9+tpuhUb4snjEc/65uptShtSa7IpudeTuNR/5OduftprTWaL13ce5CL99eXBB+AbF+scbDN5ZAj8D2b7HXVEBppvEoyYT6ahh6U5scSkJfCGETFovmmc/38O+Nh7lsQAj/uGEo7q7tNyyy1ppDRYdIzk4mOSuZrTlbya00bvB0US7E+sUyqcck4gLjiAuMo2e3njg7tXF9WkNVEZRkQPFxKDnxyDAepVlQmgFVxae/zsNPQl8IYb+q6+r587vb+HR7JreMiuaJKwbi3MbDIlu0hYNFB0nOSiY1O5WUrBQKqwsBCPEMISksifjAeOIC4+jn348uzl1sX0RdjRHixcegOB2KjlnXj50K+try01+jnMArBHzCIaAXxFwIPmHgbX34hIN3241BJKEvhGiV4opaZr2Vws+HC3h4Uj/uGtuzzS6PFFcX893x79iQvoEfMn6gqLoIgLCuYYyJHENiSCKJoYlEekXapob6WiPAi45C4REoOnJqvfiY0VI/c3ABr1DoFgnB/aH3JeATYQR5t0hj6RUKzuZFr4S+EKLFjhdVcuuSTaTll7Pg+iFMHdLokFotprVmf+F+Nh7fyIb0DWzL3YZFW/Dr4seFERcyMmwkiaGJRHi14rgVBVB4GAoOQ2GadT3NCPiS46Atp/ZVztAtAnyjodfFRpD7RkG3KGO9WyS4tMFfFDYkoS+EaJHdGSXMXLqJiup6lt2WxOhegTZ5X4u2kJqdyhdpX7A+fT1Z5VkA9Pfvzx3xdzA2cixxAXHnfj1eayPYCw5B/qEGy1+MgD/zenrXYPDvAdGjjXD3iwbf7sa6T4SprXRbcOzqhRCm+O5AHrPfTsWriwvvzRlFv1CfVr2f1podeTv4/PDnfJn2JTmVOXi4eDAqbBSzB81mTOQYgj2bubGrpsII9LwDxiP/oDXgD54e7MrJCHH/nhAxzAh4vx7WZQy4dW3V92LvJPSFEOflw83pPPj+dnoFebH0tuGEdfNo0fucuHSzNm0tnx/+nONlx3F1cmVMxBgm9ZjE2MixeLp6nvkiKMuG3H2Qt9/6sAZ8ccPR25VxySWgJ8RdbXxg6t/LWPpGg4s53UjtgYS+EOKcaK159b+HeOGLfYzqGcBrNw+jm8f536xUXF3Mp798yvv73+dg0UGclTMjw0Yye/BsLup+ET5uPmCxGCGeuw/y9kHuXsjdbzyvbtBqd/OGwN7QfRQE3gKBsRAQa4S7a8t+GXV0EvpCiGbV1Vt4YvUulv98lCmDw3nhmkF0cTn3Pu5aa7blbuO9/e/xRdoXVNdXExcQx19GPMql/vH4l2RCzh7YOQ9ydhvh3rCrY9dgCOoL8VdDUD8I6gOBfY2ujQ4w9IE9kdAXQpxVZU09f1ixmXV7cpg9rhcPXt4Xp3Psg19SU8Inhz452arv6uzOVJ8+XG3xoH/2Mdjx4Oktd68QI9QTboHgfsZ6YB/wlKkUbUVCXwjRpPyyam5flsK29CKemjqQW0bFNP8iSz2Hj2xg2e6lfJa/jSpdT1ydZn5hAZPKK/DU+8G9GwQPNFruwf0heICxlHBvcxL6QohGHckvZ8aSTWQWV7HwxmFMjGvkLtHqMsjeBdk7IGsH27K38EZtFt94uOGmNVeUV3KNSyADggZBnwEQEgchA4yuj3JZxhQS+kKIX9l6rIjblyZTrzX/d+cIhkX7Q3keZG6DrO2Qud1YL/gFC5rvPNxZ4udPahcXfNy8uDNoOL8bcDMBEUmduqeMPZLQF0Kc5uvdWTy14hsu9TjKg4Or8f9+CXyw3bg79QTf7tSGxLOm13CWlh/iYGU2oZ6hPDjwFqbHTv91V0thNyT0hejMtDYGBsvcChlbOb7nRwblbGe9czHUAClORhfI6NEQNhhCB2EJGcjn2T/zytZXOJa3g96+vfnbsLlM7DGxQ4w339FJ6AvRmZTlQMYWOL7ZWGZshnJj+GELTpRaIsjzTsJn9EV0iUqA0PiTd6hqrdl4fCP/+Ho2+wr30cevD/+Y8A/GR413iBmjhEFCX4iOqrLoVLAf3wwZW6Ek3dimnIx+7r0vpT5sMK/s9ebVvR5MTezN/0yLw9XZ6bS3Ss1OZcHmBWzJ2UKkVyTPjnmWST0m4aScfnVYYd8k9IXoCGqrIHsnHE899cg/eGq7fy/oPhIiEiB8KIQOgi5elFXXMeftVDYeyOO+S2KZe3Hsaa32vQV7WbB5Ad8d/44gjyAeG/kY02KnyWUcByahL4SjsViMQD+eAukpRsBn7wJLrbHdKwQiEmHwDcaAYuFDjJmYzpBTUsXMpcnszSrl+emDuHZ41MltBVUFLNi8gA8PfIiPmw9/HPZHbuh3Ax4uMrSBo5PQF8LeledZw/1EyG8+dRerm7cR6qPvsQZ8gjFRRzPX2A/mlDJjSTKFFTUsnpHIhL7GCJZ1ljre2/8e/9zyTyprK5kxYAazBs8yxsMRHYKEvhD2pK4asnZCerLxOJ5iTOwBxgQeIQMgbprRko9MNIYoOM95XpPTCrhjWQquzk68M2sU8ZHdANiSs4Wnf3qafYX7GBE2gkeSHqGnb08bf4PCbBL6QphFa2MkyfRkowWfnmzc8FRfY2z3iTBa74m3QeRwo8tkK8d6X7Mjk/ve2UqknwfLZiYR5e9JXmUeL6e8zCe/fEJo11BeGvcSl0ZfKj1yOigJfSHaS0250YMmfdOpkC/LNra5eBgfsI6YbbTgIxKNafls6D/fHeZ/PttNQnc/Ft+SiI+HM2/tfotXtr5CTX0Nd8bfyR3xd8iNVR2cTUJfKTURWAA4A4u11s+esX088DFw2PqlD7XWT9ni2ELYJa2N6fjSk+HYJmOZvQt0vbHdvxf0nGAEfGSiMSaNc9v0iLFYNH9bs4fF3x3m8oEhLLh+KBnlR7jnv4+xPW87F0RcwLykeUT7RLfJ8YV9aXXoK6WcgVeAS4F0IFkptVprvfuMXTdqrX/b2uMJYZeqS41eNOnJcMx6Pb6ywNjm5m10lRzzJ+MyTUQidA1ol7Kqauv583vb+Gx7JreOjmHe5D68tecNFm5diKerJ8+PfZ6JMRPlUk4nYouWfhJwUGv9C4BSaiUwFTgz9IXoGE50mUxPNi7VHEs2Jv5AG9sD+0LfyRA13Aj5oH7n/WGrLRRX1HLnWylsOlzAI5P7MT6unhlrb2ZX/i4ujb6UR0Y8QqCHbSYzF47DFqEfATScnDIdGNHIfqOUUtuADOB+rfWuxt5MKTULmAXQvXt3G5QnRCtVFRut+BMt+PRkqCoytnXpZlye6X+FEfIRwxrtE9/ejhdVMmPJJo7kl/PydXHkOX/BdZ8txNvVmxfHvcjlMZebXaIwiS1Cv7G/C/UZzzcD0VrrMqXUZOAjILaxN9NaLwIWASQmJp75PkK0LYvFmI/1RLinpxjP0YAyJvoYMAUikyAqyRiMzMm+hiLYlVHMzDeSqayt5+nrAlh55AH2FOzh8pjLeWTEI/i7y0QlnZktQj8diGrwPBKjNX+S1rqkwfoapdSrSqlArXWeDY4vRMuV51tvejrRL34zVFv/u3r4GZdn4q6y9qgZZsz4ZMc2Hshlztub8XJ3ZuakdJ7dNg9vN29eHv8yl0ZfanZ5wg7YIvSTgVilVA/gOHA98LuGOyilQoFsrbVWSiUBTkC+DY4txLmrqzFmeEpPPRXyhdYOZcoZQgZC/DVG0EcOh4BeDjW704eb03nw/e30CNFE913FG3u/Z1zkOJ664Clp3YuTWh36Wus6pdQ9wBcYXTaXaK13KaVmW7e/BlwNzFFK1QGVwPVaa7l0I9qO1lB05NTYNOkp1hufqo3tXqFG633YrUbAhw9p9Y1PZtFa8+p/D/HCF/sY1DuHEu832ZxbzMNJD/O7fr+TnjniNMqeszcxMVGnpKSYXYZwBCc+bE1PNS7XHE89OU48Lu7G3ayRw6394od3mDla6+otPLF6F8t/Pkxc3E8crf+UmG4xvDD2Bfr69zW7PGECpVSq1jqxqe1yR65wPCfGpzltGOEDp7YH9oHel0LkMKNPfMjANrvxyUwVNXXcu2ILXx/cS49BqzhSe4DpsdN5cPiDcletaJKEvrBvlnrI22+d7SnV+KA1a0cjwwhfZywjEuz+w1ZbyC+r5rZlKewu3oB/n4+oxpkXxr3AxJiJZpcm7JyEvrAfJ4YuyNhyakq/zG1QW25sd/MyxqcZ9XujJ03EsHMaRrijScsrZ8YbP5Lr+iHuERvpFzCI58c+T4SXbcfqER2ThL4wh9ZGz5mMrScn5SZzq3FtHozr8KHxMPQmI+gjEiCgtyl3ttqTrceKuO3Nb6gNfBNn90Pc0O8GHkh8ANcOePlKtA0JfdH2LBYj4BuGe+a2UwHv5GqMEz9wmjEJSPhQ4yYoCbLTrNudzR9WrcI17G3cXKuYP/pvXNHrCrPLEg5GQl/YVn0t5O4zQj1rO2RuN67B15Qa253djA9WB15l9KgJHwLBA8Cli6ll27u3f0rjqfWL6RLxKaFeofzzov9I7xzRIhL6ouWqio3hgrN2GgGftQNy9pzqC+/qaVyiGXw9hA0yQj6oP7i4mVu3A9Fa89wXO1i67yW6hG7mgvAxPDf2Gbp16fgfVou2IaEvmmexGDc6Ze+C7J1GuGftML52gmeAMSb8iFkQNgRCBxl3tHbya/CtUVNnYe7761hf/CKuvlnMHjSHOUNm46Tsa6wf4Vgk9MXpKgshe/epgM/ZbbTea8qsOygjzCMSIOEWI9xD48A7rNP1omlLpVW13LJ8Jfv5Fx4eiv+d8C/GRo01uyzRAUjod1bVZca199w9RqifeJQ2GCvP3ddovQ+50figNXigsXTQ4QocRXZJFdcs/zsFHisJ6hLGst+8JrNaCZuR0O/oqkshdz/k7TOGCM7dZ7Tei46e2se5CwT1hR5jjA9VQwYaD2m9t7u9mYXc+OHj1Hj9lwHdElk8eQE+bj5mlyU6EAn9jkBrKMsxhiLI22+EfO5eY73k+Kn9nFwhMNYYe2boLRDczwh5vxi59m4Hvt1/hHu//jN47WNi1NU8M/5RXJzkR1TYlvyPciS1lcYdq3kHjEf+ieXBU2PAA7h2haA+EDPGWAb1M6bw84sBZ/knt0fLklN4fstDOHnk8YdBDzNr6I1mlyQ6KEkAe1NXDYVpRpDnH4KCQ9blL6e32sEYKTIwFgZdZywDehtLn0i7m81JNO2JLz/ig/S/4eLmxN/HL2RCzGizSxIdmIS+GSqLjDtUCw43WKYZy5LjnDbbpIe/0VsmZoyx9O95KuDlA1WHZrFobv/wFZLL/k1X51CWT1lEbz/5wFa0LQn9tlBTYXxQWnTU6MtemNZg/cipSbVP6BoM/j0g5kLjEkxAL/DvBQE97WKSbWF7FTW1XL3ycY7pTwl1G8yHVy3Ex93b7LJEJyChf760NvqyFx2F4nQoPmYsGz4/MXnHCS7u4NsdfKON4X/9YoyQ9+thrHfxMuM7ESbJLi3lqnfvpcQlhTjvy3nzymdwdZJxhkT7kNBv6ESgl2QYl1lKjkPxcevzdGNZfBzqKk9/nYsH+EZBt0hj2AHfKPCNAb9oI+i9gqXrowBgV1YGN386h1rXX5gYfjvPXzJXpjMU7apzhL7WRu+W0mwoyzK6N5ZkQGmWcTNSadap5yfGjTlBORn91X0ijBuV+kw0xnDvFmUN+ihjCAL5wRXNWHdgJ39a/we0cxF39ZvPPSOmm12S6IQ6XuhrDZ8/aAR4WbZ1mfPr1jkYA4J5hxkhHpVkrHuHgY815H0ijJmZpJujaKUlKd/w8vZ5ODkpnh71ClP6SQ8dYY6Ol2ZKwf4vjKF6vUKMMPcKAe9QY3li3TsUuvhIC120uflfv837R1/ElQCWTHydoeG9zS5JdGIdL/QB7ttudgVCYLFYmLX6RX4ufgsvYnl/+utEdgsyuyzRyXXM0BfCZNW1tUx/7yGO1H5FiNMIVl3/T7y7eJhdlhAS+kLYWkFFKVPf/T1FagsDPKfwf9OfwlnGNhJ2QkJfCBs6mJ/F9R/fSZXTES4Nvov/nXSP2SUJcRqbDNCilJqolNqnlDqolHq4ke1KKfUP6/btSqkEWxxXCHvy/ZG9TP/od1SpdGbGPiGBL+xSq1v6Siln4BXgUiAdSFZKrdZa726w2yQg1voYASy0LoXoEN7f8T1PJv8ZFDye+A+ujR9jdklCNMoWl3eSgINa618AlFIrgalAw9CfCryptdbAT0opX6VUmNY60wbH/5WPthynzqKb31EIG9icu5HVmS/gjA+vXLSQC2P6m12SEE2yRehHAMcaPE/n1634xvaJAH4V+kqpWcAsgO7du7eooHkf7qCytr5FrxXifLj6/kSX0I9x191ZMeXfxAaGmV2SEGdli9Bv7O6mM5vZ57KP8UWtFwGLABITE1vUXP/yjzKBtGhbFm1h2d6FvHfwI0aEjOHvE17Aq4sMdS3sny1CPx2IavA8EshowT42E+Xv2VZvLQQ19TU89v1jrDm8hmv7XMu8EfNkWkPhMGzReycZiFVK9VBKuQHXA6vP2Gc1cIu1F89IoLitrucL0ZZKakqYs24Oaw6vYW7CXP4y8i8S+MKhtPp/q9a6Til1D/AF4Aws0VrvUkrNtm5/DVgDTAYOAhXAzNYeV4j2llWexZx1c0grSeNvF/6NK3pdYXZJQpw3mzRRtNZrMIK94ddea7Cugd/b4lhCmGFfwT7u/vpuKmorWHjJQkaGjTS7JCFaRGbPFqIZP2b8yIy1MwBYOnGpBL5waBL6QpzFqgOruHvd3YR1DWP55OX09e9rdklCtIp8AiVEI7TW/Gvrv1i0fRGjwkbx0viX8HaTicuF45PQF+IMDbtkTo+dzqMjH5WJy0WHIaEvRANFVUXM/XYum3M2MzdhLrfH3S4Tl4sORUJfCKujJUe5++u7ySzL5PmxzzOpxySzSxLC5iT0hQC25mzl3m/uRaNZfPlihgYPNbskIdqE9N4Rnd7aw2u5/Yvb8Xbz5u3Jb0vgiw5NWvqi07JoC69ufZXXt79OQnACf5/wd/zc/cwuS4g2JaEvOqWK2gr+8v1f+OrIV0zrPY3HRj6Gq7P00BEdn4S+6HSyyrO495t72Ve4j/sT7+eWAbdIDx3RaUjoi05le+527v3mXqrqq/jnRf9kbKTMvSA6F/kgV3Qan/7yKTPXzsTDxYPlk5dL4ItOSVr6osOzaAv/3PJPFu9YTGJIIi+Pf1k+sBWdloS+6NCKq4uZt3EeG49vNIZUGPGofGArOjUJfdFh7S3Yy33f3kd2RTaPjniU6/peJx/Yik5PQl90SKsPreapH5+im1s33rj8DYYEDzG7JCHsgoS+6FBq62t5Lvk53tn3Dokhibww7gUCPQLNLksIuyGhLzqM7PJs/rT+T2zP3c6MATO4b9h9Mmm5EGeQnwjRISRnJXP/+vuprKvkxXEvcnnM5WaXJIRdktAXDq3eUs/iHYtZuG0hUd5RLLl8Cb18e5ldlhB2S0JfOKys8izmbZxHSnYKk3pM4vGRj+Pl5mV2WULYNQl94ZC+OfoNj//wODX1NfzPBf/DlF5TpDumEOdAQl84lOr6al5KeYkVe1fQ378/z499nphuMWaXJYTDkNAXDuNQ0SEe2PAABwoPcMuAW5ibMBc3ZzezyxLCoUjoC7untea9/e/xQvILeLp68urFrzImcozZZQnhkFoV+kopf+AdIAZIA67VWhc2sl8aUArUA3Va68TWHFd0HpllmTzxwxP8mPkjo8JG8bcxf5ObrYRohda29B8GvtZaP6uUetj6/KEm9p2gtc5r5fFEJ3Gidf9SyksAPDbyMa7uczVOSkYDF6I1Whv6U4Hx1vVlwH9pOvSFOCfppenM/2E+P2f9zMiwkcwfPZ8IrwizyxKiQ2ht6IdorTMBtNaZSqngJvbTwJdKKQ28rrVe1NQbKqVmAbMAunfv3sryhCOxaAvv7HuH/039X5yUE0+MeoLpsdOlK6YQNtRs6Cul1gGhjWx69DyOc4HWOsP6S+ErpdRerfWGxna0/kJYBJCYmKjP4xjCgR0rOcbjPzxOSnYKo8NHM3/UfMK8wswuS4gOp9nQ11pf0tQ2pVS2UirM2soPA3KaeI8M6zJHKbUKSAIaDX3RuVTWVbJk5xKW7FiCm7MbT41+iit7XymteyHaSGsv76wGZgDPWpcfn7mDUqor4KS1LrWuXwY81crjCgenteabo9/wfPLzZJRnMLnHZP6c+GeCPZu6QiiEsIXWhv6zwLtKqduBo8A1AEqpcGCx1noyEAKssrbcXID/01qvbeVxhQM7XHyYZzc9yw8ZP9DbtzdLLl/C8NDhZpclRKfQqtDXWucDFzfy9QxgsnX9F2Bwa44jOoaK2gpe3/46b+5+E3dndx5Oepjr+l4nY94L0Y7kp020uXpLPZ8d/owFmxeQU5HD1F5TuW/YfXKTlRAmkNAXbUZrzTfHvuFfW/7FwaKDDAgYwEvjXpL5aoUwkYS+aBObMjexYPMCtudtJ8YnhhfHvcil0ZfKHbVCmExCX9jUrrxdLNi8gB8zfyTEM4QnRz/JlF5T5Lq9EHZCfhKFTewt2Mui7Yv46shX+Hbx5f7E+7m+3/V0ce5idmlCiAYk9EWLaa3ZlLWJJTuX8EPGD3i6eDJn8BxuGXCLTFsohJ2S0Bfnrd5Sz7qj61iycwm783cT4B7A3IS5XNv3WnzcfMwuTwhxFhL64pxV1VXx8cGPWbZ7GcdKjxHtE80To57gil5XyGUcIRyEhL5o1tGSo7x/4H0+OvARhdWFxAfG86dhf2JC1AScnZzNLk8IcR4k9EWjautr+frY17y//31+zvwZZ+XMuMhx3DTgJhJDEmVANCEclIS+OM2JVv3HBz+moKqAsK5h3DPkHqbFTpPB0IToACT0BcXVxaw7so41h9ewKWvTyVb91X2uZnT4aLmEI0QHIqHfSZXXlvPN0W9Ym7aWH47/QJ2uo7t3d34/5PdM6z2NkK4hZpcohGgDEvqdSGVdJRvTN7I2bS0b0jdQXV9NaNdQbh5wMxN7TKS/f3+5Vi9EByeh38FllGWwIX0DG9I3sClrE9X11QS4BzA9djqTekxiUNAgGQ9HiE5EQr+DqbPUsS13G+vT17MxfSMHiw4CEOUdxdV9rmZC1AQSQxLlOr0QnZSEvoOzaAsHCg+Qkp1CSlYKm7I2UVJTgotyYVjIMK5MvJKxkWOJ8YmRSzdCCAl9R2PRFvYX7ic5K5mUrBRSc1Ipri4GIMIrgou7X8yYyDGMChsl498IIX5FQt/O5VXmsSN3Bzvzd7Irbxfb87ZTWlMKGJdsLoq6iOGhw0kMSSTMK8zkaoUQ9k5C344UVBWwv3A/O/OMgN+Rt4PsimwAnJUzvX17c1n0ZQwLGcbw0OGEdg01uWIhhKOR0DdBZV0lh4oOcaDwAPsL93Og6AAHCw+SX5V/cp8o7ygSQhKIC4gjPiiefv798HDxMLFqIURHIKHfRuot9WSUZ3C05ChHSo4Yj9IjHCk+wvGy42g0AO7O7vTy7cWYyDHE+sbS2683A/wH4Ovua+43IITokCT0W8iiLeRX5pNRnkFmWSYZ5RlklGWQWZ7JsdJjpJemU2upPbm/p4sn0T7RxAXGMaXXFGL9Yon1iyXSK1K6Twoh2o2EfiMq6yrJrcglpyKHvMo8cipyyK3MPbnMLs8mszzztFAH8HHzIdwrnJ7dejIhagLRPtEnHwHuAdJlUghhug4f+jX1NZTUlFBcXUxxdfHJ9cKqQgqqCiioKqCwupCCSuuyqoDKuspfvY+bkxtBnkEEewbTP6A/F3e/mDCvMMK7hp9cShdJIYS9a1XoK6WuAeYD/YEkrXVKE/tNBBYAzsBirfWzrTnu2Witue7T6yioKqCkpqTRAD/B1ckVf3d//N398XP3I7pb9MnnQR5BRsh7BBPkGYSPm4+01IUQDq+1Lf2dwFXA603toJRyBl4BLgXSgWSl1Gqt9e5WHrup49HLtxd9nfrSza0bPl186ObWjW5duuHj5nNy6efuR1fXrhLkQohOpVWhr7XeAzQXnEnAQa31L9Z9VwJTgTYJfYBnxjzTVm8thBAOrT2GV4wAjjV4nm79WqOUUrOUUilKqZTc3Nw2L04IITqTZlv6Sql1QGO3fj6qtf74HI7R2J8BuqmdtdaLgEUAiYmJTe4nhBDi/DUb+lrrS1p5jHQgqsHzSCCjle8phBCiBdrj8k4yEKuU6qGUcgOuB1a3w3GFEEKcoVWhr5SappRKB0YBnymlvrB+PVwptQZAa10H3AN8AewB3tVa72pd2UIIIVqitb13VgGrGvl6BjC5wfM1wJrWHEsIIUTryeSoQgjRiUjoCyFEJ6K0tt9ekUqpXOBIC18eCOTZsBxbktpaRmprGamtZRy1tmitdVBTL7Tr0G8NpVSK1jrR7DoaI7W1jNTWMlJby3TU2uTyjhBCdCIS+kII0Yl05NBfZHYBZyG1tYzU1jJSW8t0yNo67DV9IYQQv9aRW/pCCCHOIKEvhBCdSIcJfaXUC0qpvUqp7UqpVUop3yb2m6iU2qeUOqiUeridartGKbVLKWVRSjXZzUoplaaU2qGU2qqUanTqSRNrM+O8+SulvlJKHbAu/ZrYr93OW3PnQRn+Yd2+XSmV0Jb1nGdt45VSxdbztFUp9Xg71bVEKZWjlNrZxHYzz1lztZlyzqzHjlJKfauU2mP9GZ3byD7nf+601h3iAVwGuFjXnwOea2QfZ+AQ0BNwA7YBA9qhtv5AX+C/QOJZ9ksDAtv5vDVbm4nn7XngYev6w439m7bneTuX84Ax5tTnGPNIjAR+bqd/x3OpbTzwaXv+/7IedyyQAOxsYrsp5+wcazPlnFmPHQYkWNe9gf22+P/WYVr6WusvtTGiJ8BPGOP2n+nk1I1a6xrgxNSNbV3bHq31vrY+TkucY22mnDfrMZZZ15cBV7bDMc/mXM7DVOBNbfgJ8FVKhdlJbabQWm8ACs6yi1nn7FxqM43WOlNrvdm6XooxSvGZsw6e97nrMKF/htswfvud6bymbjSBBr5USqUqpWaZXUwDZp23EK11Jhg/AEBwE/u113k7l/Ng1rk61+OOUkptU0p9rpQa2A51nQt7/7k0/ZwppWKAocDPZ2w673PXqqGV29u5TN2olHoUqAOWN/YWjXzNJn1WbTCtJMAFWusMpVQw8JVSaq+1JWJ2baact/N4mzY5b404l/PQZueqGedy3M0Y47KUKaUmAx8BsW1d2Dkw65ydC9PPmVLKC/gAuE9rXXLm5kZectZz51Chr5uZulEpNQP4LXCxtl7wOkObTd3YXG3n+B4Z1mWOUmoVxp/srQ4vG9RmynlTSmUrpcK01pnWP1lzmniPNjlvjTiX82DW9KDNHrdhYGit1yilXlVKBWqtzR5UzG6nVDX7nCmlXDECf7nW+sNGdjnvc9dhLu8opSYCDwFTtNYVTexmt1M3KqW6KqW8T6xjfDDdaI8CE5h13lYDM6zrM4Bf/VXSzuftXM7DauAWa6+KkUDxiUtUbazZ2pRSoUopZV1Pwvj5z2+H2ppj1jlrlpnnzHrc/wB7tNYvN7Hb+Z87Mz6VbosHcBDj2tZW6+M169fDgTUN9puM8Sn4IYzLG+1R2zSM38jVQDbwxZm1YfS62GZ97LKn2kw8bwHA18AB69Lf7PPW2HkAZgOzresKeMW6fQdn6a1lQm33WM/RNozODqPbqa4VQCZQa/2/drsdnbPmajPlnFmPfSHGpZrtDXJtcmvPnQzDIIQQnUiHubwjhBCieRL6QgjRiUjoCyFEJyKhL4QQnYiEvhBCdCIS+kII0YlI6AshRCfy/z6T6b6C/mbwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the results to compare the activation functions\n",
    "\n",
    "x = torch.arange(-2, 2, 0.1).view(-1, 1)\n",
    "plt.plot(x.numpy(), torch.relu(x).numpy(), label='relu')\n",
    "plt.plot(x.numpy(), torch.sigmoid(x).numpy(), label='sigmoid')\n",
    "plt.plot(x.numpy(), torch.tanh(x).numpy(), label='tanh')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref4\"></a>\n",
    "<h2> Practice </h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare the activation functions with a tensor in the range <i>(-1, 1)</i>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f3538d83550>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2iUlEQVR4nO3dd3gU1frA8e9JgYQQWhqBEEIJEHoJRZEmCoIUwYtYr1y9YkO9VpoFxQLKvYiKSLOhBpUuImCj+qN3kgABgoRAQkISAqmbPb8/ZgNLSCBhN9kk+36eZ5/dnTln9mWznHfmzJkzSmuNEEII5+Xi6ACEEEI4liQCIYRwcpIIhBDCyUkiEEIIJyeJQAghnJybowO4Eb6+vjokJMTRYQghRIWyc+fOJK21X8HlFTIRhISEsGPHDkeHIYQQFYpS6kRhy6VrSAghnJwkAiGEcHKSCIQQwslVyHMEhcnNzSUuLo6srCxHh1KueXh4EBQUhLu7u6NDEUKUE5UmEcTFxeHt7U1ISAhKKUeHUy5prUlOTiYuLo5GjRo5OhwhRDlRabqGsrKy8PHxkSRwDUopfHx85KhJCHGFSpMIAEkCxSDfkRCioEqVCIQQorJKy8xl0oqDnM/Ktfu2JRE4QPXq1R0dghCiAtl7MpVBH2/kmy0n2HbsnN23X2lOFpc3Wmu01ri4SK4VQtwYrTXzNx1n6upo/L09+P7xm+jUsLbdP0daKTuKjY0lLCyMp556io4dOzJ58mQ6d+5M27ZteeONN64qv27dOgYNGnTp/ZgxY/jyyy/LMGIhRHmVcjGHx77ewds/R9GnuT+rnu1RKkkAKukRwZs/HSQy/rxdt9myXg3eGNzquuUOHTrEF198wV133cWiRYvYtm0bWmuGDBnChg0b6Nmzp13jEkJUPjtiz/FMxG6SL+QwaXBLHr65dIfFyxGBnTVs2JBu3bqxdu1a1q5dS4cOHejYsSPR0dEcOXLE0eEJIcoxs1kz888YRs7ZQhU3FxY/eTOjujcq9dF+lfKIoDh77qXFy8sLMPr2xo8fz+OPP15kWTc3N8xm86X3Mr5fCOeVdCGb57/fw8YjSQxqG8h7w9vg7VE2MwDIEUEp6d+/P59//jkXLlwA4NSpUyQmJl5RpmHDhkRGRpKdnU1aWhq///67I0IVQjjYX0eTGDBjI9uOn+PdYW34+L4OZZYEoJIeEZQH/fr1Iyoqiptuugkwhox+8803+Pv7XyrToEED7rnnHtq2bUtoaCgdOnRwVLhCCAfIM2s++v0IH/1xhMa+Xnz9SBfCAmuUeRxKa13mH2qr8PBwXfDGNFFRUYSFhTkooopFvishHC/hfBbPLdzNlmPnuLtjEG8NbYVX1dLdN1dK7dRahxdcLkcEQghRxtYfPssL3+8hIyePaSPa8Y9OQQ6NRxKBEEKUkdw8M/9de5jP1h+lRV1vPrm/A039vR0dliQCIYQoC6dSM3k2Yjc7T6Rwf9dgXh/UEg93V0eHBUgiEEKIUvdrZAIv/bjXODl8XweGtKvn6JCuIIlACCFKSY7JzJRfovl883Fa16/BJ/d1JMTXy9FhXUUSgRBClIITyRd5JmI3++LSGHVzCOMHtqCqW/noCirILheUKaXuUEodUkrFKKXGFbL+ZaXUHsvjgFIqTylVx7IuVim137Jux9Vbr9j+/e9/ExkZWaqfMXDgQFJTU69aPmnSJKZNm1aqny2EuNrKffEM+mgTsUkXmf1QJyYNaVVukwDY4YhAKeUKzARuB+KA7UqpFVrrS62f1voD4ANL+cHA81pr60m1+2itk2yNpTyaN29eqX/GqlWrSv0zhBDXl5Wbx+SVkXy79W/aN6jFx/d1oEGdao4O67rscUTQBYjRWh/TWucAC4Gh1yh/HxBhh88tdy5evMidd95Ju3btaN26Nd9//z29e/cm/+K3+fPn06xZM3r37s1jjz3GmDFjABg1ahRPPvkkffr0oXHjxqxfv55HHnmEsLAwRo0adWn7ERERtGnThtatWzN27NhLy0NCQkhKMvLoO++8Q/Pmzbnttts4dOhQ2f3jhXByR89e4K6Zm/l269883rMxPz5xU4VIAmCfcwT1gZNW7+OAroUVVEpVA+4Axlgt1sBapZQGZmut5xRRdzQwGiA4OPjaEf0yDs7sL2b4xVS3DQyYcs0iq1evpl69evz8888ApKWlMWvWLADi4+OZPHkyu3btwtvbm1tvvZV27dpdqpuSksIff/zBihUrGDx4MJs3b2bevHl07tyZPXv24O/vz9ixY9m5cye1a9emX79+LFu2jLvuuuvSNnbu3MnChQvZvXs3JpOJjh070qlTJ/t+D0KIqyzZFceryw5Q1c2FL0Z1pk8L/+tXKkfscURQ2PyoRc1bMRjYXKBbqLvWuiMwAHhaKVXohP1a6zla63Ctdbifn59tEZeSNm3a8NtvvzF27Fg2btxIzZo1L63btm0bvXr1ok6dOri7uzNixIgr6g4ePBilFG3atCEgIIA2bdrg4uJCq1atiI2NZfv27fTu3Rs/Pz/c3Nx44IEH2LBhwxXb2LhxI8OGDaNatWrUqFGDIUOGlMm/WwhnlZFj4qUf9/LCD3tpXa8mq57rUeGSANjniCAOaGD1PgiIL6LsvRToFtJax1ueE5VSSzG6mjYUUrf4rrPnXlqaNWvGzp07WbVqFePHj6dfv36X1l1vTqeqVasC4OLicul1/nuTyYSbW/H+VKU9b7kQwnDoTDpPf7eLo2cv8OytTXm2byhurhVzQmd7RL0dCFVKNVJKVcFo7FcULKSUqgn0ApZbLfNSSnnnvwb6AQfsEJNDxMfHU61aNR588EFeeukldu3adWldly5dWL9+PSkpKZhMJhYvXlyibXft2pX169eTlJREXl4eERER9OrV64oyPXv2ZOnSpWRmZpKens5PP/1kl3+XEOIyrTULt/3NkE82kZqRy4JHuvJCv+almgRyzbn88fcfPPPb05w+/7fdt2/zEYHW2qSUGgOsAVyBz7XWB5VST1jWf2YpOgxYq7W+aFU9AFhq2Yt1A77TWq+2NSZH2b9/Py+//DIuLi64u7sza9YsXnrpJQDq16/PhAkT6Nq1K/Xq1aNly5ZXdB1dT2BgIO+99x59+vRBa83AgQMZOvTKc/IdO3Zk5MiRtG/fnoYNG9KjRw+7/vuEcHbpWblMXHqAFXvj6d7Uh+kj2+Pv7WHfD9EaLiZB0mFiT21lafxGVmQcJ4k8fE15nKjTnsCOj9n1I2Ua6jJ04cIFqlevjslkYtiwYTzyyCMMGzaszOOoCN+VEOXNgVNpjPluF3+fy+CF25vxZO+muLrY0BWblwspsZB02PI4AkmHyUw6zK9uJhZ7e7HLwwNXrelhdmd4tRB6BHTGre1I8GlyQx8p01CXA5MmTeK3334jKyuLfv36XTHiRwhRPmmt+fr/TvDOz1HU8arCwtE30aVRneJvIDvdaOjPHjIelgaflONgNhmfAUTWCmRxrdr8Us+HC9pEsIcvzzUaxNCWD+JXPaB0/nEWkgjKkFzlK0TFkpaZy9hF+1h98Ax9mvvx33vaU8erSuGFM87B2WirBv8QnD0M5+Mul3FxN/bm/VtAyyGk1WrAypwEliRs5XBaDB6uJm5vOIDhocPpFNCpzAZ/SCIQQohC7P47hWcidnMmLYuJA8N49JZGuCjgfPzVjf3ZaMiwmhzBvRr4NoOQ7uDXHHybg18LqB2C2cWFrae3svTIUn6L/Jhccy6tfFrxWrfXGNBoAN5Vyv7+BJIIhBDCitmsmb/xGF+u+T+6eCXy4i15BKWsgs8te/vZ5y8X9qhpNPDNBxjPfs2NR40gcLlyFFFSZhKLD8xnacxSTl04RY0qNRjRbATDQ4fTvE7zMv5XXkkSgRDCeWkN6aeNPfrEaLJOHyTu0C5GZsXyWJUMyAW2AdV8wC8M2t5jNPi+zYzn6v5wne6bwymHWRC5gJ+P/UyuOZeugV15tsOz9G3Yl6quVa9Zt6xIIhBCVH5aw4VESIy0NPpRlv78aMhKu1QsC2/OmYMwBQ/Cu01nlH8Y+IeBl2+JPs6szWw6tYkFkQvYcnoLnm6eDA8dzoNhDxJSM8TO/zjbSSKwk9TUVL777jueeuqpG6rfu3dvpk2bRnj4VSO7hBAlkZlqNPSJkZZny+tMq5ltPGsbe/it78bs24Ll8TV4d7vGq3ZdPnmgEy3qF/8anys+2pTJT0d/4puobziedhx/T3+e6/gcI5qNoGbVG9tmWZBEYCepqal8+umnN5wIhBAllJNhnKwt2OifP3W5TBVvY48+bDD4tzRG6/i3BC8/UIqz6dm88MMeNh5JYnC7erw7rDXeHu4lDuVsxlkioiP48fCPpGanElYnjPd6vEf/hv1xdy359sqaJAI7GTduHEePHqV9+/b06dOHffv2kZKSQm5uLm+//TZDhw4lNjaWAQMGcMstt/DXX39Rv359li9fjqenJwA//vgjTz31FKmpqcyfP1+uDBYCIM8E545BwoErG/1zx7g0v6VrVfBrBiG3WBr8lkYCqBlUZB/+5pgknlu4h/SsXKYMb8PIzg1KPFwz+lw0CyIXsOr4KvLMefRp0IeHWj5UpkM/7aFSJoKp26YSfS7arttsUacFY7uMLXL9lClTOHDgAHv27MFkMpGRkUGNGjVISkqiW7dul2YCPXLkCBEREcydO5d77rmHxYsX8+CDDwJgMpnYtm0bq1at4s033+S3336z679BiHJNa7iQAAkHjUdipPF89hDkZRtllAv4NIW6rY0Tt/5hRqNfuxG4Fq85M+WZ+ej3I3z8ZwxN/Krz7b+70rxu8YdsmrWZDXEbWBC5gG1ntuHp5sk9ze7hgbAHCK5xnSnyy6lKmQgcTWvNhAkT2LBhAy4uLpw6dYqEhAQAGjVqRPv27QHo1KkTsbGxl+oNHz680OVCVDrZF4wTtQkHICHycqNv3Y9fvS4EtILGvcC/FQS0NMbju9/43D5n0rJ4duFuth0/x4hOQbw5tBXVqhSvGcwyZbEsZhnfRH3DifMnCKgWwAudXmB46PBy3f9fHJUyEVxrz70sfPvtt5w9e5adO3fi7u5OSEgIWVlZAFdMMe3q6kpmZual9/nrXF1dMZlMZRu0EKXBnGfMp5Nw4PKefsIBY1k+d6/L/fgBrYw9/IBWUK0E0zgUw5+HEnnxh71k5ebxv3vaMbxjULHqZZmyWHxkMfP3z+ds5lla+7Tm/Z7vc1vD23B3Kf/9/8VRKROBI3h7e5Oeng4Ydybz9/fH3d2dP//8kxMnTjg4OiHKQMa5y3v2+Q1/YhTkZhjrlQvUaQKB7aH9g8Yevn9LqNXwqouv7Ck3z8y0NYeYveEYLep688n9HWnqX/269QomgM51OzO151TCA8IrVP9/cUgisBMfHx+6d+9O69at6dy5M9HR0YSHh9O+fXtatGjh6PCEsJ+8XEiOubLBTzh45WgdzzpGP36nUcbefUAr4wIsd88yDTUuJYNnInaz++9UHugazGuDWuLh7nrNOkUlgM51O5dR1GVPpqF2QvJdiWK7mAwJ++FMfoO/33LyNsdY7+JuTKmQ39gHtIKA1lA94LpX3Ja2NQfP8PKPe9Ea3ru7DYPa1rtm+cISwJPtnqxUCUCmoRZCFC3PZNnLP2A8zlie009fLlM9wHLyto/R2Ae0MqZacCtiNk4HyTbl8d6qaL78K5Y29Wvyyf0daOjjVWR5ZzwCKEgSgRDOJjPlckOf3+ifjQaTMaABFzejG6dRL6Oxr9saAtpAdT/Hxl0MsUkXGROxiwOnzvNI90aMHdCcqm6FdwVJArisUiUCrXWlO4ljbxWxK1DcILPZuPnJmf3GI7/Rt54fv5qv0dB3/rexl1+3tTFEs5zt5RfHT3vjGb9kP64uijkPdaJfq7qFlpMEcLVKkwg8PDxITk7Gx8dHkkERtNYkJyfj4WHne6wKx8u+YIzYudTg7zfG5+dabhGuXME3FIK7Xd7Dr1s++vJtlZWbx5s/RRKx7W86Btfi4/s7Ur/W1Sels/OyWXR4kSSAQlSaRBAUFERcXBxnz551dCjlmoeHB0FBxRs/LcohrY3ROWfyT+Banq2nW6ha02jkOzwIdS0Nvl+YTRdilVcxiemM+W430WfSeaJXE17s1wx31yuHomqtWR27mhm7ZnDqwilJAIWoNInA3d2dRo0aOToMIezHlGP03Vvv5Z/ZD1mpl8vUbmSZbmGk8Vy3DdRsUOH38otj0c44Xlt2AM8qrnz5r870bu5/VZndibuZtn0a+5L20ax2M2bfPpub693sgGjLN7skAqXUHcAMwBWYp7WeUmB9b2A5cNyyaInW+q3i1BXCKVgP08xv+M9GX7q5OW6exgVYLYda9vLbGBdjedRwbNwOcDHbxGvLD7Bk1ym6NqrDR/d1IKDGlUc7f5//mw93fcivJ37F39Oft25+iyFNhuDqcu1rCJyVzYlAKeUKzARuB+KA7UqpFVrryAJFN2qtB91gXSEqh0sncPdd7t45sx/S4y+X8Q40TtyG3m40+AFtjBueSyNG1OnzjPluF8eSLvJs31Ce6xuKq8vlo5+07DQ+2/sZCw8txN3FnafaP8XDLR+mmns1B0Zd/tnjiKALEKO1PgaglFoIDAWK05jbUleI8i3nonHC9sw+q7H5B688gevXHBr1sIzYsezpl/BuWM5Aa03EtpO8+dNBani68+2jXbm56eXvKScvh4joCGbvm83F3IsMazqMp9s/jV+18j/ktTywRyKoD5y0eh8HdC2k3E1Kqb1APPCS1vpgCeqilBoNjAYIDq6YU72KSkprOB9/ZT9+wgFIPsqlE7geNY09+44PWfbyW1umXKh8J3DtLT0rl/FL9rNy32l6hPryv3va4+dtTNCotWbtibV8uPND4i7E0b1+d17o9ALNajdzcNQViz0SQWFnpQoOVt8FNNRaX1BKDQSWAaHFrGss1HoOMAeMKSZuOFohbGHKgaTDVidw9xl7+tbTJ9cOMRr7NvdcHrXjJCdw7W1/XBpjInYRl5LJK3c054meTXCxdAXtSdzDtB3T2Ht2L6G1Q5l922xuri8ngm+EPRJBHNDA6n0Qxl7/JVrr81avVymlPlVK+RanrhAOk3Huyguxzuy3nMDNNda7eVimTx5kGZffxrgS1wlP4Nqb1pov/4rl3VVR+FavysLR3egcYkxLfTL9JB/u/JC1J9bi5+knJ4LtwB6JYDsQqpRqBJwC7gXuty6glKoLJGittVKqC+ACJAOp16srRKkz58G54wX68g9cOZtm9QCjO6fprVC3rfHap2mx74olii81I4dXFu1jbWQCfVv4M21EO2p7VSE9J53P9n7Gd9HfGSeC2z3Fw63kRLA92Pwr1lqblFJjgDUYQ0A/11ofVEo9YVn/GfAP4EmllAnIBO7VxlwHhda1NSYhipSdbpzAtR6qmRhpNWe+5QRuw+6WK3AtJ3GrXz1GXdjfrr9TeOa73SSmZ/HqnWE8eotxbdDKYyuZtn0a57LOcVfTuxjTYQz+1eRvYi+VZhpqIa6gNaSeMEbpWF+Bm3L8cpn8E7j5/fhyAtdhzGbN3I3H+GDNIQJrefDxfR1p36AWx1KP8c7Wd9h2ZhutfVrz6k2v0sqnlaPDrbBkGmpReeVkGHfCujRvvmWYZnb+qSkFdRobDX77+y/v5dcMkhO45UDyhWxe/HEv6w6dZUDruky5uy1V3E3M2DWDLw9+iaebJ691e427Q++W8wClRBKBqDguzbNz4MpG33qYZhVv44RtmxGXJ1fzD4Oq1781oSh7W48l8+zC3aRczGXy0FY82K0h6+PW897W94i/GM+QJkN4odML+Hj6ODrUSk0SgSifci5a9vIL3PQ8K+1ymVoNjT371v+43LVTyve/FfaRZ9Z8+mcM0387TEMfLz4f1ZlaNS7w7J/Psu7kOprWasoX/b8gvO5VvRiiFEgiEI5lNl/uy7e+B671bJpVqhvz6rQabrlRivPOs1MZJKZn8fz3e9gck8xd7evxxpAWLD76LbP/mI1Sihc6vcCDLR/E3cXd0aE6DUkEouxkpVlG7Fjt5SdGQs4FSwFLX35AK2M2zfx74MpefqWx6UgS//l+NxeyTbz/j7Y0rB/Pw2tHcjztOLcF38bYLmOp61X4DWVE6ZFEIOzPlAPJR4xGP/Gg5TkS0qxmE8kfsdP+gcs3PPdvAVWKvresqLhMeWY+/O0IM9fF0NSvOjP/GcrSEzOZvPZngqoHMbPvTHoG9XR0mE5LEoG4cfknb/P38hMjjddJhy9ffeviZtzgPLgb+D9yeS+/Rn0ZseMkTqdl8mzEbrbHpjCiUz1ah0Xyn03jyM7L5vG2j/PvNv/Gw02G7DqSJAJRPJmpxvQK1l06iZFXnrytEWTMmd+sH/i3Ml77hFbI+98K+/gjOoEXf9hLjsnMK4O9WHfuPVbvjKJbYDcmdp1ISM0QR4cokEQgCsrJgKRDxoidxEjLc9SV0y1UrWGcrG19t7F379/KGKLpWcthYYvyJcdk5oM10czdeJwWgZ7c3Gk3c44uoLZHbT7o+QH9Q/rLvcXLEUkEziovF5JjrmzsEyONOXfyR+u4VjWmWwjpYTT0/i2NZ7kQS1zDyXMZjInYzd6TqdwZnstJtzn8ePQYQ5oM4ZXOr1Czak1HhygKkERQ2eVPqHY2yujaSYw2Gn3rfnzlYkygVreNMVonv9Gv3UgmVRMlsvrAaV5etA/I5c5eO9l0dgl+nn582vdTegT1cHR4ogjyv7yyMOdBSqzRyJ+NvtzoJx2GvOzL5WoGGw196O2Wbp0wox9f5tcRNsjKzeO9VVF89X8naN7wLPj9wIbEk/yj2T94odMLeFfxdnSI4hokEVQ0+Q2+dWN/NgqSjoAp63K5mg2MCdQa9zIae78w8GsGVeU/pLCv40kXGfPdLg6eTqJjh/8jJms19VQ95vabS7fAbo4OTxSDJILyKi/XuLr27CHjkXTIaPiTYsCUeblcjSBj/H2jXkbD7x9m9OtLgy/KwPI9p5iwZD9uXsdo2G4pR7LOcH+L+3mu43Nyn4AKRBKBo+VmGnvz1o392cNw7iiYTZfL1Qw29uhDehoNv5+lwZdpFoQDZObk8eZPB1m48whBjf8gzX0D1asE878+X9IpoJOjwxMlJImgrGSmGHvzSYeu3MtPOcGlUTrKFeo0MvbsW9xpNPR+zY0+fJk9U5QTRxLSefq7XRy9sIuAsOWk6xQeDnuYpzs8jaebp6PDEzdAEoE9mfOMaRSSjhgnaZMOW14fgYuJl8u5VjVG6dTrCO3uMxp73+bg0wTcqjoufiGuQWvNjzvjeP2nbVT1X0W14G3U9W7MW91n0M6vnaPDEzaQRHAjsi8YY/ALNvjnjl55wtazttHAN+tvTLPg2wx8Q6F2CMgNNkQFcjHbxGvLDrD8yK/UaLQcs8sFHmv9GI+3e5yqrrLzUtFJIiiKOc+YHjn5qNHo5zf8yTFXXmWrXIyG3beZcWNzn9DLjb6X3ExDVHyR8ed5KmIzZ9y+p1qDnTSu3Yy3us+WW0ZWIs6dCLSGi2cLNPSWhj/lOOTlXC5btSb4NoWQW4y9+vzGvk5j6c4RlZLWmm+3/s3k336mauD3VHFL5bG2o3mi7RO4u8q9AioTuyQCpdQdwAzAFZintZ5SYP0DwFjL2wvAk1rrvZZ1sUA6kAeYCruxst3EboLYzZaG39LoX7qvLeBaxWjYfUOh+QCjHz//4eUr0yoIp3E+K5exi3bze8ICqgatJ7B6fab2nEF7//aODk2UApsTgVLKFZgJ3A7EAduVUiu01pFWxY4DvbTWKUqpAcAcoKvV+j5a6yRbY7muqJWwdZZxsZVPU2M6hfyG3repsVz67oWT23sylad+WEVq9a+o6hvP3U3v5pUur8h1AZWYPY4IugAxWutjAEqphcBQ4FIi0Fr/ZVV+CxBkh88tud5j4bY3wF2GuAlRkNaa+ZuOMW3LPNz9VlOzSnXeueUj+gT3cXRoopTZIxHUB6xuPUUcV+7tF/Qo8IvVew2sVUppYLbWeo4dYiqcZ+1S27QQFVlqRg7P/LiOHRdn4e4fQ/fAnrzT4y18PGXAgzOwRyIorONcF1pQqT4YieAWq8XdtdbxSil/4FelVLTWekMhdUcDowGCg4Ntj1oIAcDOE+d4Ysl8Mmv+iGd1zYRub3B36N1yvwAnYo9EEAc0sHofBMQXLKSUagvMAwZorZPzl2ut4y3PiUqppRhdTVclAsuRwhyA8PDwQhONEKL4zGbNR+v2MfvgNNzq7CG0Ritm9H2f4Bqyo+Vs7JEItgOhSqlGwCngXuB+6wJKqWBgCfCQ1vqw1XIvwEVrnW553Q94yw4xCSGuIelCNo/9sJDD5rm417jAv1s/xVMdHsPNxblHlDsrm//qWmuTUmoMsAZj+OjnWuuDSqknLOs/A14HfIBPLYeb+cNEA4CllmVuwHda69W2xiSEKNqGI/E8t+Y9TN7r8PUMYuZtn9Har7WjwxIOpLSueL0s4eHheseOHY4OQ4gKJc+smbTmV5b8/T4uHgn0bzCcyT3HyURxTkQptbOwa7XkOFAIJ3A6LYN/LprGaZcleHh4M7XXJ9wW0svRYYlyQhKBEJXcygOHmLDpVbRnNC28b2LewPep5VHL0WGJckQSgRCVlCnPzEsrf+TXpA9x8cjm8ZYv83T4QzIsVFxFEoEQldCJc+d5aMkkUtx/pWaVIOb0/5DW/s0dHZYopyQRCFHJROzaybs7XoWqcXT1GcQnd7yBh5uHo8MS5ZgkAiEqiRyTmSeXz2Fr2lxc3d0Y12kq97Ue6OiwRAUgiUCISiD6TCL/WjmeC+7b8KsaxpeDptOwZn1HhyUqCEkEQlRws7f8yScH3gS3FPrXe5ipfZ/HVaZTFyUgiUCICiojJ5d/LX2fg5k/4O5ai/du+Yw7mt7k6LBEBSSJQIgKaPvJWJ5c8zLZ7tEEe3RjwZAP8KlWy9FhiQpKEoEQFcyU9Uv45uj7KNcc7m30AhN6jJJrA4RNJBEIUUGkZWbywJLXOGFag4cKYubt/6VrUEtHhyUqAUkEQlQA645F8vyfL2JyiyOs2kC+GDoJryoyWZywD0kEQpRjWmte/30BS0/OQCk3nmgxmae73uXosEQlI4lAiHIq6WI69y8Zz2nzerxoyvw7p9M6IMTRYYlKSBKBEOXQ6sO7GbfxFUyuCXSs8Q/mDplAVTd3R4clKilJBEKUI2azmVfWzmX16dko5cGLbT7gX536OzosUclJIhCinIg/n8IDy14mSW+lhmrJl4P/RzNfmSZClD5JBEKUA8sit/LG/40jz/UcN9V+gFmDXsbNVaaJEGVDEoEQDmQ2m3l21cesO/sFLsqbiR0+4r52cgtJUbYkEQjhICdSzvLQ8hdJUbup49KOb4ZMJ7i2n6PDEk7IxR4bUUrdoZQ6pJSKUUqNK2S9Ukp9ZFm/TynVsbh1haiMIvZuYPCSuznHPnr7Psq6h76WJCAcxuYjAqWUKzATuB2IA7YrpVZorSOtig0AQi2PrsAsoGsx6wpRaZjy8njip2lsSf0OV1Wbd7rMZmjLro4OSzg5e3QNdQFitNbHAJRSC4GhgHVjPhT4WmutgS1KqVpKqUAgpBh17eavmCSizqSXxqaFuK4MUyoRx6eS7nIQf5cufDv8AwJr1HF0WELYJRHUB05avY/D2Ou/Xpn6xawLgFJqNDAaIDg4+IYC/eXAGRZsOXFDdYWwhWu1GDzqf49yyWRA3TFMvf0xXFzs0jMrhM3skQgKm/9WF7NMceoaC7WeA8wBCA8PL7TM9UwYGMZL/ZvfSFUhbojJbOLLyLl8FTmfYO+GTOnxAa39Wjg6LCGuYI9EEAc0sHofBMQXs0yVYtS1G88qrngiY7NF2Ui4mMDYjWPZmbCTu5rexfgu46nmXs3RYQlxFXskgu1AqFKqEXAKuBe4v0CZFcAYyzmArkCa1vq0UupsMeoKUeFsiNvAxE0Tyc7L5t1b3mVwk8GODkmIItmcCLTWJqXUGGAN4Ap8rrU+qJR6wrL+M2AVMBCIATKAf12rrq0xCeEouXm5zNg1g68iv6J57eZM6zWNkJohjg5LiGtSxkCeiiU8PFzv2LHD0WEIcYW49Dhe2fAK+5P2M7L5SF7u/DJVXas6OiwhLlFK7dRahxdcLlcWC2EHv574lTc2vwHA/3r/j9sb3u7giIQoPkkEQtggOy+bD7Z/wPeHvqetb1um9pxKkHeQo8MSokQkEQhxg46nHefl9S9zKOUQ/2r1L57p+AzuLnLzGFHxSCIQ4gb8dPQnJm+ZjIerB5/2/ZQeQT0cHZIQN0wSgRAlkJGbwTtb32HF0RV0CujE1B5TCfAKcHRYQthEEoEQxXTo3CFe3vAysWmxPNHuCR5v+zhuLvJfSFR88isW4jq01kRER/DfHf+lRtUazO03l66BMmOoqDwkEQhxDSlZKby++XXWxa2jZ1BPJnefTB0PmTFUVC6SCIQowtbTW5mwcQIp2SmM6zKO+1vcj1KFzZMoRMUmiUCIAnLNuczaM4t5++cRUjOEmbfNpEUdmTFUVF6SCISwEpcex9gNY9mXtI+7Q+/mlc6vyIyhotKTRCCExapjq5i8ZTIKxQe9PuCOkDscHZIQZUISgXB6GbkZvLftPZbFLKOdXzum9pxK/er1HR2WEGVGEoFwapHJkYzdMJYT508wuu1onmz3pFwbIJyO/OKFUzJrM99EfsP0XdOp41GH+f3n07luZ0eHJYRDSCIQTic5M5lXN7/KplOb6NOgD2/d/Ba1PGo5OiwhHEYSgXAqf536iwmbJpCek87ErhMZ2XykXBsgnJ4kAuEUsvOy+WjXR3wd+TVNajZhTr85NKvdzNFhCVEuSCIQld6hc4cYt3EcMakxjGw+khfDX8TTzdPRYQlRbkgiEJVWnjmPryO/5uPdH1Ozak25b4AQRZBEICql+AvxTNg0gZ0JO+kb3Jc3bnqD2h61HR2WEOWSiy2VlVJ1lFK/KqWOWJ6v+p+mlGqglPpTKRWllDqolHrOat0kpdQppdQey2OgLfEIobVmxdEV3L3ibqLPRfN297eZ3nu6JAEhrsGmRACMA37XWocCv1veF2QCXtRahwHdgKeVUi2t1k/XWre3PFbZGI9wYqlZqby4/kUmbppIs9rNWDR4EUObDpVRQUJch61dQ0OB3pbXXwHrgLHWBbTWp4HTltfpSqkooD4QaeNnC3HJ5lObeW3za6Rkp/Cfjv9hVKtRuLq4OjosISoEWxNBgKWhR2t9Winlf63CSqkQoAOw1WrxGKXUP4EdGEcOKUXUHQ2MBggODrYxbFFZZJoymb5zOhHRETSp2YSZfWcS5hPm6LCEqFCu2zWklPpNKXWgkMfQknyQUqo6sBj4j9b6vGXxLKAJ0B7jqOG/RdXXWs/RWodrrcP9/PxK8tGikjqYfJCRK0cSER3Bg2EP8v3g7yUJCHEDrntEoLW+rah1SqkEpVSg5WggEEgsopw7RhL4Vmu9xGrbCVZl5gIrSxK8cE4ms4n5++fz2d7PqONZh7n95tItsJujwxKiwrL1ZPEK4GHL64eB5QULKONM3XwgSmv9vwLrAq3eDgMO2BiPqOROnj/JqNWj+GTPJ9ze8HaWDFkiSUAIG9l6jmAK8INS6lHgb2AEgFKqHjBPaz0Q6A48BOxXSu2x1JtgGSH0vlKqPaCBWOBxG+MRlZRZm1l0eBHTdkzDTbkxtcdUBjaW0cZC2INNiUBrnQz0LWR5PDDQ8noTUOj4Pa31Q7Z8vnAOJ9NPMumvSWw7s42ugV15u/vb1PWq6+iwhKg05MpiUW6ZtZmI6Ahm7JqBq3Jl0k2TGB46XK4LEMLOJBGIcik2LZbX/3qd3Ym7uaX+Lbxx0xtyFCBEKZFEIMqV/IniZu6ZSRXXKrxzyzsMbjxYjgKEKEWSCES5EZMSw+t/vc7+pP3c2uBWXu32Kn7V5JoRIUqbJALhcLnmXL448AWf7f0ML3cv3u/5PneE3CFHAUKUEUkEwqGiz0Xz+ubXiToXRf+Q/ozvMh4fTx9HhyWEU5FEIBwiNy+X2ftmM3//fGpWrcn03tO5rWGRF7ELIUqRJAJR5g4kHeC1za8RkxrD4MaDeaXzK9TyqOXosIRwWpIIRJnJzsvm0z2f8uXBL/H18OWTWz+hV4Nejg5LCKcniUCUiU2nNvHu1nc5mX6S4aHDeTH8RWpUqeHosIQQSCIQpezMxTO8v/19fj3xKyE1QmSmUCHKIUkEolTkmnP5Luo7Zu6ZiVmbeabDM4xqNYoqrlUcHZoQogBJBMLudiXsYvKWycSkxtAzqCfju4wnyDvI0WEJIYogiUDYzbmsc0zfOZ1lMcsI9ApkRp8Z9GnQRy4ME6Kck0QgbGbWZhYfWcyHOz8kIzeDR1s/yui2o6nmXs3RoQkhikESgbBJVHIUb295m31J++hctzMTu06kSa0mjg5LCFECkgjEDUnPSeeT3Z+w8NBCalWtxbu3vMugxoOkG0iICkgSgSgRrTW/HP+FD3Z8QHJmMiObj+SZjs/INQFCVGCSCESxHU09ynvb3mPr6a208mnFJ7d+QivfVo4OSwhhI0kE4rqSMpP4dM+nLD6yGC83L17t+ir/aPYPXF1cHR2aEMIOJBGIImWaMlkQuYD5++eTk5fDvc3v5Yl2T1Dbo7ajQxNC2JFNiUApVQf4HggBYoF7tNYphZSLBdKBPMCktQ4vSX1RtszazMpjK/lo10ckZCRwa4Nbeb7T84TUDHF0aEKIUuBiY/1xwO9a61Dgd8v7ovTRWrfPTwI3UF+UgW2nt3HvynuZuGkivp6+fN7/c2bcOkOSgBCVmK1dQ0OB3pbXXwHrgLFlWF/YybG0Y0zfMZ11cesI9ApkSo8pDGg0ABdl676CEKK8szURBGitTwNorU8rpfyLKKeBtUopDczWWs8pYX2UUqOB0QDBwcE2hi3yJWcmM2vvLBYdXoSnmyf/6fgfHgh7AA83D0eHJoQoI9dNBEqp34C6hayaWILP6a61jrc09L8qpaK11htKUB9L8pgDEB4erktSV1wty5TFN1HfMG//PLJMWYxoNoIn2z9JHY86jg5NCFHGrpsItNZF3khWKZWglAq07M0HAolFbCPe8pyolFoKdAE2AMWqL+zHrM2sOr6Kj3Z9xOmLp+ndoDfPd3qexjUbOzo0IYSD2NoBvAJ42PL6YWB5wQJKKS+llHf+a6AfcKC49YV9aK35v/j/476f72P8xvHUqlqLz/t/zse3fixJQAgnZ+s5ginAD0qpR4G/gREASql6wDyt9UAgAFhqmYPGDfhOa736WvWF/Wit2XJ6C7P2zmJ34m7qetXl3Vve5c7Gd8qJYCEEYGMi0FonA30LWR4PDLS8Pga0K0l9YbuCCcC/mj8Tu05keOhwuUuYEOIKcmVxJSMJQAhRUpIIKglJAEKIGyWJoIKTBCCEsJUkggpKEoAQwl4kEVQwkgCEEPYmiaCCMGszm05tYt7+eZIAhBB2JYmgnMs0ZfLT0Z9YELmA2POxBFQLkAQghLArSQTlVGJGIgujF/LD4R9Iy06jpU9LpvSYQr+Qfri7uDs6PCFEJSKJoJyJSo5iQeQCfon9hTxzHn0a9OGfrf5JR/+OWK7OFkIIu5JEUA6YtZn1J9ezIGoB289sx9PNk3ua3cODYQ/SoEYDR4cnhKjkJBE4UEZuBsuPLufbqG85cf4Edb3q8mKnFxnebDg1qtRwdHhCCCchicABzlw8Q0R0BIsOL+J8znna+Lbh/Z7vc1vD26T/XwhR5iQRlBGtNXvP7iUiOoK1sWsxY6ZvcF/+2fKftPNrJ/3/QgiHkURQypIzk1l5bCVLjizhWNoxvNy9uLfFvTwQ9gBB3kGODk8IISQRlIY8cx5/xf/F0pil/Pn3n5i0ibZ+bZl00yTuaHQHXu5ejg5RCCEukURgR3HpcSyLWcaymGUkZCRQu2pt7gu7j+FNh9O0dlNHhyeEEIWSRGCj7Lxs/vj7D5YcWcKW01tQKG6udzOvdH6FPg364O4qJ3+FEOWbJIIbdOjcIZbGLOWnoz9xPuc89bzq8VT7p7iryV0EVg90dHhCCFFskghK4HzOeVYfX83SI0s5kHwAdxd3+gb3ZVjoMLoFdpN7AAshKiRJBNeRnpPOupPrWBO7hs3xmzGZTTSt1ZSxnccyqPEgannUcnSIQghhE5sSgVKqDvA9EALEAvdorVMKlGluKZOvMfC61vpDpdQk4DHgrGXdBK31KltisocLORdYF2dp/E9tJtecS12vutzf4n4GNBpAK59WMu5fCFFp2HpEMA74XWs9RSk1zvJ+rHUBrfUhoD2AUsoVOAUstSoyXWs9zcY4bJaRm8H6uPWsiV3DxriN5Jhz8K/mz8jmI+kf0p+2fm2l60cIUSnZmgiGAr0tr78C1lEgERTQFziqtT5h4+faRUZuBhtObWBt7Fo2xG0gOy8bP08/RjQfQf+Q/rTzayeNvxCi0rM1EQRorU8DaK1PK6X8r1P+XiCiwLIxSql/AjuAFwt2LdlbpimTTac2sSZ2DRviNpBpysTHw4dhTYdxR6M76ODfQRp/IYRTuW4iUEr9BtQtZNXEknyQUqoKMAQYb7V4FjAZ0Jbn/wKPFFF/NDAaIDg4uCQffclnez/j8wOfk2nKpI5HHYY0GUL/kP509O+Iq4vrDW1TCCEquusmAq31bUWtU0olKKUCLUcDgUDiNTY1ANiltU6w2val10qpucDKa8QxB5gDEB4erq8Xd2ECqgUwqPEg+of0p1NAJ9xcZNCUEELY2hKuAB4Gpliel1+j7H0U6BbKTyKWt8OAAzbGc03DQocxLHRYaX6EEEJUOLZ2hk8BbldKHQFut7xHKVVPKXVpGKhSqppl/ZIC9d9XSu1XSu0D+gDP2xiPEEKIErLpiEBrnYwxEqjg8nhgoNX7DMCnkHIP2fL5QgghbCfDY4QQwslJIhBCCCcniUAIIZycJAIhhHBykgiEEMLJSSIQQggnp7S+oYt0HUopdRa40YnrfIEkO4ZjbxKfbSQ+20h8tivPMTbUWvsVXFghE4EtlFI7tNbhjo6jKBKfbSQ+20h8tqsIMRYkXUNCCOHkJBEIIYSTc8ZEMMfRAVyHxGcbic82Ep/tKkKMV3C6cwRCCCGu5IxHBEIIIaxIIhBCCCdXKROBUmqEUuqgUsqslCpyGJdS6g6l1CGlVIxSapzV8jpKqV+VUkcsz7XtHN91t6+Uaq6U2mP1OK+U+o9l3SSl1CmrdQOv+pBSjs9SLtZyP4k9SqkdJa1fmvEppRoopf5USkVZfgvPWa0rle+vqN+T1XqllPrIsn6fUqpjceuWUXwPWOLap5T6SynVzmpdoX/rMo6vt1Iqzerv9npx65ZRfC9bxXZAKZWnlKpjWVfq359NtNaV7gGEAc2BdUB4EWVcgaNAY6AKsBdoaVn3PjDO8nocMNXO8ZVo+5ZYz2BcDAIwCXipFL+/YsUHxAK+tv77SiM+IBDoaHntDRy2+vva/fu71u/JqsxA4BdAAd2ArcWtW0bx3QzUtrwekB/ftf7WZRxfb2DljdQti/gKlB8M/FFW35+tj0p5RKC1jtJaH7pOsS5AjNb6mNY6B1gIDLWsGwp8ZXn9FXCXnUMs6fb7Ake11jd6NXVJ2frvd/j3p7U+rbXeZXmdDkQB9e0ch7Vr/Z7yDQW+1oYtQC1l3Ou7OHVLPT6t9V9a6xTL2y1AkJ1jsCm+UqpbWvFddWve8qxSJoJiqg+ctHofx+WGIkBb7qVsefa382eXdPv3cvWPaozlEP5ze3e9lCA+DaxVSu1USo2+gfqlHR8ASqkQoAOw1Wqxvb+/a/2erlemOHXLIj5rj2IcveQr6m9d1vHdpJTaq5T6RSnVqoR1yyK+/Fvz3gEstlpc2t+fTWy9eb3DKKV+A+oWsmqi1np5cTZRyDK7jaW9Vnwl3E4VYAgw3mrxLGAyRryTgf8Cjzggvu5a63illD/wq1IqWmu9oSRxlHJ8KKWqY/yH/I/W+rxlsc3fX2EfVciygr+nosqU6m/xOp99dUGl+mAkglusFpfa37oE8e3C6B69YDmvswwILWZdW5XkMwYDm7XW56yWlfb3Z5MKmwi01rfZuIk4oIHV+yAg3vI6QSkVqLU+bTl0T7RnfEqpkmx/ALBLa51gte1Lr5VSc4GVjohPG/emRmudqJRainH4vIFy8v0ppdwxksC3WuslVtu2+fsrxLV+T9crU6UYdcsiPpRSbYF5wABt3JMcuObfuszis0rkaK1XKaU+VUr5FqduWcRn5aoj+DL4/mzizF1D24FQpVQjy173vcAKy7oVwMOW1w8DxTnCKImSbP+qvkZL45dvGHDArtEVIz6llJdSyjv/NdDPKg6Hf39KKQXMB6K01v8rsK40vr9r/Z6s4/6nZfRQNyDN0rVVnLqlHp9SKhhYAjyktT5stfxaf+uyjK+u5e+KUqoLRvuVXJy6ZRGfJa6aQC+sfpNl9P3ZxtFnq0vjgfGfOw7IBhKANZbl9YBVVuUGYowmOYrRpZS/3Af4HThiea5j5/gK3X4h8VXD+KHXLFB/AbAf2IfxYwws6/gwRk/stTwOlrfvD6NbQ1u+oz2Wx8DS/P4K+z0BTwBPWF4rYKZl/X6sRrQV9Vu08/d2vfjmASlW39eO6/2tyzi+MZbP34txMvvm8vT9Wd6PAhYWqFcm358tD5liQgghnJwzdw0JIYRAEoEQQjg9SQRCCOHkJBEIIYSTk0QghBBOThKBEEI4OUkEQgjh5P4fatwz2QdQ9RQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Practice: Compare the activation functions again using a tensor in the range (-1, 1)\n",
    "\n",
    "# Type your code here\n",
    "x = torch.arange(-1, 1, 0.1).view(-1, 1)\n",
    "plt.plot(x.numpy(), torch.relu(x).numpy(), label='relu')\n",
    "plt.plot(x.numpy(), torch.sigmoid(x).numpy(), label='sigmoid')\n",
    "plt.plot(x.numpy(), torch.tanh(x).numpy(), label='tanh')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- \n",
    "x = torch.arange(-1, 1, 0.1).view(-1, 1)\n",
    "plt.plot(x.numpy(), F.relu(x).numpy(), label = 'relu')\n",
    "plt.plot(x.numpy(), torch.sigmoid(x).numpy(), label = 'sigmoid')\n",
    "plt.plot(x.numpy(), torch.tanh(x).numpy(), label = 'tanh')\n",
    "plt.legend()\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"http://cocl.us/pytorch_link_bottom\">\n",
    "    <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/notebook_bottom%20.png\" width=\"750\" alt=\"PyTorch Bottom\" />\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>About the Authors:</h2> \n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other contributors: <a href=\"https://www.linkedin.com/in/michelleccarey/\">Michelle Carey</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright &copy; 2018 <a href=\"cognitiveclass.ai?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu\">cognitiveclass.ai</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}