Skip to content

Instantly share code, notes, and snippets.

@kdneal
Forked from thearn/gradient.ipynb
Last active June 23, 2020 08:34
Show Gist options
  • Save kdneal/c86ce4c00e547efa362854ed729f5e0f to your computer and use it in GitHub Desktop.
Save kdneal/c86ce4c00e547efa362854ed729f5e0f to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"import keras\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from keras.layers import Dense, Dropout, Activation\n",
"from keras.models import Sequential\n",
"import keras.backend as K\n",
"import tensorflow as tf\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define model & data"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x1ee96adf888>"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deVyVZf7/8dd1WEUQFNBU4BzccgcVt9TMFtMWbTHT0LKxyJZplrJyKC1nmG/LTIuV42Bji0OaWU1WlrZoWbkrue+yu+ACLuyc6/cHyA/hIEc4cJ9z+DwfDx8PuM997vt9e/TDzXVfi9JaI4QQwvWZjA4ghBDCMaSgCyGEm5CCLoQQbkIKuhBCuAkp6EII4SY8jTpxSEiItlgsRp1eCCFc0ubNm09orUNtvWZYQbdYLGzatMmo0wshhEtSSqXW9Jo0uQghhJuQgi6EEG5CCroQQrgJw9rQhesrLi4mIyODgoICo6MIO/j6+hIWFoaXl5fRUUQDkYIu6iwjI4OAgAAsFgtKKaPjiEvQWnPy5EkyMjKIjIw0Oo5oILU2uSilFiiljiuldtTwulJKzVFKHVBKbVNK9XV8TOGMCgoKCA4OlmLuApRSBAcHy29Tbs6eNvT3gFGXeH000Ln8Txzwr/rHEq5CirnrkM/K/dVa0LXWPwGnLrHLWOADXWYdEKSUauuogFUlp+fw4td7kGl/hRCuxmrVJHy1i51ZuQ1yfEf0cmkPpFf6PqN8WzVKqTil1Cal1Kbs7Ow6nWx7Rg7zfjzIjswzdXq/cB85OTnMnTu3Tu+96aabyMnJueQ+M2fO5LvvvqvT8evLnnOvXr2aX3/9tZESCUdYf/gU89ccZt+xsw1yfEcUdFu/x9m8fdZaJ2qtY7TWMaGhNkeu1mpMdHt8vUws2phWp/cL93Gpgl5aWnrJ9y5fvpygoKBL7jN79myuv/76OuerD3vOLQXd9SzZlE6AryejezZMI4YjCnoGEF7p+zAgywHHtSmwmRc39WrLsuQs8opKGuo0wgU888wzHDx4kOjoaKZPn87q1asZMWIE99xzD7169QLgtttuo1+/fvTo0YPExMSK91osFk6cOEFKSgrdunXjwQcfpEePHowcOZL8/HwApkyZwtKlSyv2nzVrFn379qVXr17s2bMHgOzsbG644Qb69u3LQw89hNls5sSJE9Wy+vv788QTT9C3b1+uu+46LvyGmpyczKBBg+jduze33347p0+ftuvcKSkpzJs3j9dee43o6GjWrFnDxx9/TM+ePYmKiuLqq69uoL91UVe5+cUs336EMVHt8PXyaJBzOKLb4jLgMaXUYmAgkKu1PuKA49ZoQv8IPt2SyZfbjjA+Jrz2N4gG98IXO9mV5dhmsO7tWjDr1h41vv7iiy+yY8cOkpOTgbI71g0bNrBjx46KrnkLFiygVatW5Ofn079/f+68806Cg4MvOs7+/ftZtGgR8+fPZ/z48XzyySdMmjSp2vlCQkLYsmULc+fO5R//+AfvvPMOL7zwAtdeey0zZszgm2++ueiHRmXnz5+nb9++/POf/2T27Nm88MILvPXWW9x77728+eabDB8+nJkzZ/LCCy/w+uuv23XuadOm4e/vz5NPPglAr169WLFiBe3bt6+1OUk0vs+TMykssTKhf0SDncOebouLgLXAlUqpDKXUVKXUNKXUtPJdlgOHgAPAfOCRBktbrr+lJR1Dm/PRxvTadxZNyoABAy7qZz1nzhyioqIYNGgQ6enp7N+/v9p7IiMjiY6OBqBfv36kpKTYPPYdd9xRbZ+ff/6ZCRMmADBq1Chatmxp870mk4m7774bgEmTJvHzzz+Tm5tLTk4Ow4cPB+C+++7jp59+svvcVQ0ZMoQpU6Ywf/78WpucROPSWrNoQzo927egV1hgg52n1jt0rfXEWl7XwKMOS2QHpRQT+keQsHw3+46dpUubgMY8vbDhUnfSjal58+YVX69evZrvvvuOtWvX4ufnxzXXXGOzH7aPj0/F1x4eHhVNLjXt5+HhQUlJWXNfXXtbXW4XQlvnrmrevHmsX7+er776iujoaJKTk6v9NiKMsS0jl91HzvC323o26Hlcdi6XO/q2x8tDsXiD3KU3VQEBAZw9W3NvgdzcXFq2bImfnx979uxh3bp1Ds8wdOhQlixZAsDKlSsr2sCrslqtFW3iH374IUOHDiUwMJCWLVuyZs0aABYuXFhxt26Pqtd/8OBBBg4cyOzZswkJCSE9Xf5vOIvFG9No5uXB2Oh2DXoely3owf4+jOx+BZ9uzaCwRH69bIqCg4MZMmQIPXv2ZPr06dVeHzVqFCUlJfTu3ZvnnnuOQYMGOTzDrFmzWLlyJX379uXrr7+mbdu2BARU/42xefPm7Ny5k379+vHDDz8wc+ZMAN5//32mT59O7969SU5Orthuj1tvvZXPPvus4qHo9OnT6dWrFz179uTqq68mKirKYdcp6u5cYQmfJ2dxS++2BPg27Dw6yqgBOjExMbq+C1z8tC+bexdsYM7EPoyJatiffKK63bt3061bN6NjGKqwsBAPDw88PT1Zu3YtDz/8cMVD2sr8/f05d+6cAQkvJp9Z41u0IY0Zn27nk4evop/Z9jOWy6GU2qy1jrH1mktPzjW0Uwjtg5rx0cY0KejCEGlpaYwfPx6r1Yq3tzfz5883OpJwMos3pNGljT99Iy497sERXLqgm0yKu/uH8+q3+0g9eR5zcPPa3ySEA3Xu3JmtW7fWup8z3J2Lxrcr6wy/ZeQy85bujTKXjsu2oV9wV0wYJlU2AksIIZzJ4o1peHuauKOvzdlQHM7lC3rbwGaMuLI1H2/KoKTUanQcIYQAIL+olM+2ZjK65xUE+Xk3yjldvqAD3N0/nONnC1m1t24TfgkhhKMt336EswUlDToytCq3KOjXdm1N6wAfFm+QCbuEEM5h8cY0IkOaM6hDq0Y7p1sUdE8PE+P6hbFq73GO5Noe5SfcT32mzwV4/fXXycvLq3W/1atXc8stt1xyn+TkZJYvX17nLMK9HDh+lo0pp7m7f3ijLiziFgUdYOKACDTIyFEnlpSUhMViwWQyYbFYSEpKqtfxGqug20MKuqhs8YZ0PE2KO/uGNep53aagh7fyY3iXUBZvTKNYHo46naSkJOLi4khNTUVrTWpqKnFxcfUq6lWnzwV45ZVX6N+/P71792bWrFlA2UyHN998M1FRUfTs2ZOPPvqIOXPmkJWVxYgRIxgxYkS1Y3/zzTd07dqVoUOH8umnn1Zs37BhA1dddRV9+vThqquuYu/evRQVFTFz5kw++ugjoqOj+eijj2zuJ5qGguJSPtmSwQ3d2xAa4FP7GxxJa23In379+mlHW7nzqDY//aX+enuWw48tqtu1a5fd+5rNZk3ZwicX/TGbzXU+/+HDh3WPHj0qvl+xYoV+8MEHtdVq1aWlpfrmm2/WP/74o166dKl+4IEHKvbLycmpyJSdnV3tuPn5+TosLEzv27dPW61Wfdddd+mbb75Za611bm6uLi4u1lpr/e233+o77rhDa631u+++qx999NGKY9S0n9Eu5zMTdfPJ5nRtfvpL/cv+6v+2HAHYpGuoqy49sKiqa7u2pl2gL0nr0xjVQCuCiLpJS7P9wLqm7XWxcuVKVq5cSZ8+fYCywTz79+9n2LBhPPnkkzz99NPccsstDBs27JLH2bNnD5GRkXTu3Bkom+72wjznubm53Hfffezfvx+lFMXFxTaPYe9+wv0sXJdKh9DmDO7Y+DNduk2TC4CHSTFxQARr9p/g8InzRscRlURE2O66VdP2utBaM2PGDJKTk0lOTubAgQNMnTqVLl26sHnzZnr16sWMGTOYPXt2rceq6UHWc889x4gRI9ixYwdffPGFzel4L2c/4V52ZuWyNS2H2IHmRn0YeoFbFXQo65PuaVJ8uD7V6CiikoSEBPz8/C7a5ufnR0JCQp2PWXX62BtvvJEFCxZUDLPPzMzk+PHjZGVl4efnx6RJk3jyySfZsmWLzfdf0LVrVw4fPszBgwcBWLRoUcVrubm5tG9fNurvvffeqzFLTfsJ9/bfdWn4epkY18gPQy9wu4LeuoUvI3u04ePNGRQUy7S6ziI2NpbExETM5rI7F7PZTGJiIrGxsXU+ZtXpc0eOHMk999zD4MGD6dWrF+PGjePs2bNs376dAQMGEB0dTUJCAs8++ywAcXFxjB49utpDUV9fXxITE7n55psZOnQoZrO54rWnnnqKGTNmMGTIkItWBRoxYgS7du2qeCha037CPSUlJRERYebFO6M4Mm8qX362xJAcLj19bk1+PXCCe95Zz6vjo7jDoJ+UTYFMxep65DNzvAs9uCp3gfXz86v3DUtNLjV9rtvdoQMM7hhMh5Dm/HedNLsIIRpWfHx8tfEMeXl5xMfHN3oWtyzoSinuGRjBlrQch69EL4QQlTVGDy57uWVBBxjXLwwfTxNJ8nC0QRnVZCcun3xWDaMxenDZy20LepCfN7dGteN/WzM5V2h7lXRRP76+vpw8eVIKhQvQWnPy5El8fX2NjuJ2nnnuBZTnxSNC69uDq67camBRVZMGmVm6OYPPtmYyeZC59jeIyxIWFkZGRgbZ2TJtsSvw9fUlLEw6CThaaYchtBr1GD7JSziSmUFERAQJCQkN8kC0Nm5d0KPCAunRrgVJ61KZNDDCkI7+7szLy4vIyEijYwhhmFKr5sP1aYwcM44Pv/iH0XHct8kFyh6OTh5kZs/Rs2w4fMroOEIIN7N673Eyc/KZ5CQtAG5d0AHGRrcnsJkX769NMTqKEMLNLFyXSusAH27o3sboKEATKOjNvD2YMCCcFTuPkZUji18IIRzjUPY5Vu/NJnagGS8P5yilzpGigU0eZEZrLQONhBAO88HaVLw8ysa8OIsmUdDDWvpxfbc2LN6YLvO7CCHq7WxBMUs3Z3Br73aNv4jFJTSJgg4w5SoLp84X8cVvWUZHEUK4uE82Z3CusIT7rrIYHeUiTaagD+4YTJc2/ry/NkUGwggh6sxq1XywNpU+EUFEhQcZHecidhV0pdQopdRepdQBpdQzNl6PUEqtUkptVUptU0rd5Pio9aOU4t7BFnZknmFL2mmj4wghXNRP+7M5dOI8U5zs7hzsKOhKKQ/gbWA00B2YqJTqXmW3Z4ElWus+wASg7kuxN6Db+7QnwNeTd39JMTqKEMJFvfdrCqEBPox2wmUu7blDHwAc0Fof0loXAYuBsVX20UCL8q8DAadsqG7u48n4mHC+2XGUY2dkSTAhxOU5fOI8q/dmM2mgGW9P52uxtidReyC90vcZ5dsqex6YpJTKAJYDv7d1IKVUnFJqk1Jqk1Hzf9w72Eyp1iRJF0YhxGV6/9cUvDwUEweGGx3FJnsKuq0JUKo+VZwIvKe1DgNuAhYqpaodW2udqLWO0VrHhIaGXn5aBzAHN+faK1vz4YY0CkukC6MQwj7nCktYujmDW3q3o3WAc85aaU9BzwAq/zgKo3qTylRgCYDWei3gC4Q4ImBDuO8qCyfOFbF8+xGjowghXMSFrorO+DD0AnsK+kags1IqUinlTdlDz2VV9kkDrgNQSnWjrKA77ZyqQzuF0CG0Oe/+Il0YhRCXlpSUhNlsZsqQDhxPnMqOn74yOlKNai3oWusS4DFgBbCbst4sO5VSs5VSY8p3ewJ4UCn1G7AImKKduFKaTIrfDYlkW0Yum1KlC6MQwrYLC0CXLSenyT99jLi4OJKSkoyOZpMyqu7GxMToTZs2GXJugPyiUga/+D0DI1vx78k2F9AWQjRxFouF1NTqHSjMZjMpKSmNHwhQSm3WWtssWs7X76aRNPP2YNJAMyt3HSP15Hmj4wghnJAzLQBtjyZb0KGsC6OnSclAIyGETc60ALQ9mnRBb93ClzFR7VmyKZ3cvGKj4wghnMzTzz3vNAtA26NJF3SAqUMjySsqZdFG5/wVSghhnPzwwQSPfoz2YeEopTCbzSQmJhqyALQ9mnxB796uBUM6BfPeLykUl1qNjiOEcBL5RaUsXJfK7eMmkJGehtVqJSUlxWmLOUhBB8ru0o+eKZCBRkKICp9uzeB0XjEPDOtgdBS7SUEHrunSmg6hzXlnzWEZaCSEwGrV/GfNYXqHBdLf0tLoOHaTgk7ZQKOpQyPZnpnLxhQZaCREU7dq73EOnTjPA8M6oJSt6ayckxT0cnf0CaOlnxfvrDlkdBQhhMHeWXOYdoG+jO55hdFRLosU9HLNvD2YNMjMt7uPkXJCBhoJ0VTtyMxl7aGTTBliwcvDtUqka6VtYJMHm/EymfjPz4eNjiKEMMh/fj5Mc28PJgxwzsFDlyIFvZLWAb7c3qdsoNGJc4VGxxFCNLLMnHy++C2Lu/tH0MLXy+g4l00KehVxwztQVGrl/V9TjI4ihGhkF56hPTAs0uAkdSMFvYqOof6M7N6GD9amcr6wxOg4QohGcvp8EYs3pDMmuh3tgpoZHadOpKDbMG14R3Lzi1m0QaYDEKKpeH9tCvnFpUwb3tHoKHUmBd2GPhEtueL4Rh4dMxiTyYTFYnHaCe2FEPWXV1TC+7+mcH231nRpE2B0nDrzNDqAM0pKSiJ50csUFeQDkJqaSlxcHIBTz+MghKibjzamczqv2KXvzkHu0G2Kj4+nsLyYX5CXl0d8fLxBiYQQDaW41Mo7aw7T39KSGEsro+PUixR0G1xtlRIhRN19uS2LzJx8l787BynoNrnaKiVCiLrRWjNv9SG6tPFnxJWtjY5Tb1LQbUhISMDPz++ibb7NmjntKiVCiLpZtfc4e4+dZdrwjphMrjMJV02koNsQGxtLYmIiZrMZpRRega0ZOHmGPBAVws3MW32I9kHNuDWqndFRHEIKeg1iY2NJSUnBarXyytKfSWnZl91HzhgdSwjhIJtTT7Eh5RRTh0a63CRcNXGPq2hgU66y4O/jyVurDhgdRQjhIG/+cIBWzb2ZMCDc6CgOIwXdDoF+Xtw72Mzy7Uc4cPys0XGEEPW0LSOH1XuzmTo0Ej9v9xmOIwXdTlOHRuLr6cHcVQeNjiKEqKe3fjhAC19P7h1sNjqKQ0lBt1Owvw+xAyP4/LcsUk/KAhhCuKrdR86wctcx7h8SSYALTpF7KVLQL0Pc1R3wMCn+tVru0oVwVW+tOoC/jyf3D7EYHcXhpKBfhtYtfJnQP5xPtmSQmZNf+xuEEE7lwPGzLN9+hMmDzQT5eRsdx+GkoF+mh8qHB//7R7lLF8LVzF11EF9PDx4Y6poLWNTGroKulBqllNqrlDqglHqmhn3GK6V2KaV2KqU+dGxM59E+qBl39g1j8cZ0jp8pMDqOEMJOqSfP8/lvWcQOjCDY38foOA2i1oKulPIA3gZGA92BiUqp7lX26QzMAIZorXsAf2yArE7jkWs6UWrVzC9frkoI4fzmrjqIh0kRd3UHo6M0GHvu0AcAB7TWh7TWRcBiYGyVfR4E3tZanwbQWh93bEznEhHsx9iodvx3XRonZTFpIZxexuk8PtmSwYT+4bRu4Wt0nAZjT0FvD6RX+j6jfFtlXYAuSqlflFLrlFKjbB1IKRWnlNqklNqUnZ1dt8RO4pERnSgoKWX+msNGRxFC1GLejwdRCreYIvdS7CnotqYg01W+9wQ6A9cAE4F3lFJB1d6kdaLWOkZrHRMaGnq5WZ1Kp9b+jIlqxwdrUzghd+lCOK3MnHyWbMxgXL8wl1382V72FPQMoPJkB2FAlo19PtdaF2utDwN7KSvwbu3x6zpTUFwqPV6EcGJvrzqARvPoiE5GR2lw9hT0jUBnpVSkUsobmAAsq7LP/4ARAEqpEMqaYNz+iWHHUH9u69OeD9amSo8XIZxQ+qk8lmxMZ0L/CMJa+tX+BhdXa0HXWpcAjwErgN3AEq31TqXUbKXUmPLdVgAnlVK7gFXAdK31yYYK7Uz+cF1nSqyauTJ6VAin8+YP+zGZVJO4O4eytu9aaa2XA8urbJtZ6WsN/Ln8T5NiDm7OuL5hfLghjYeGd6BtoHu30QnhKlJOnOeTLZncO9jMFYHu27OlMhkp6gCPXdsJrTVvy3zpQjiNOd/vx8tD8fA17t2zpTIp6A4Q3sqP8THhfLQxnYzTeUbHEaLJO3D8HP9LzuTewRZaBzSNu3OQgu4wj47ohELx1g9yly6E0V7/bh++Xh485MajQm2Rgu4g7YKaMXFAOB9vzpD50oUw0J6jZ/hq+xHuH2Jx2zlbaiIF3YEeGdEJT5Nizvdyly6EUV7/dj/+3p48OKxp3Z2DFHSHatPCl0mDzHy2NUPWHhXCANsycvhm51F+NzTSLec7r40UdAd75JqO+Hl78sqKvUZHEaLJefmbvbRq7s0Dw9xzvvPaSEF3sGB/Hx4c1oEVO4+xJe200XGEaDJ+3n+Cnw+c4NERndxurVB7SUFvAA8MiyTE35uXvt5D2ZgrIURDslo1L32zh/ZBzZg0KMLoOIaRgt4Amvt48vtrO/P9l5/QNiwCk8mExWIhKSnJ6GhCuKXlO46wPTOXP9/QBR9PD6PjGMauof+iDg7+zOkVb2EtLptaNzU1lbi4OABiY2ONTCaEWykutfKPFXu5sk0At/WpulRD0yJ36A1k1nPPVhTzC/Ly8oiPjzcokRDuacmmdFJO5jH9xivxMNlavqHpkILeQNLS0i5ruxDi8uUXlfLGd/uJMbfkum6tjY5jOCnoDSQiwvaDmZq2CyEu34JfDnP8bCFPj+6KUk377hykoDeYhIQE/PwunlC/mZ8fCQkJBiUSwr2cPl/EvB8Pcl3X1vS3tDI6jlOQgt5AYmNjSUxMxGw2o5TCo0Uot//+BXkgKkQ9JSUlYbFYaBXgy+7XJtMtb5vRkZyGFPQGFBsbS0pKClarlWn/+obNXj04kptvdCwhXFZSUhJxcXGkpqaC1pSeyWbW9MelS3A5KeiN5OlRXbFqZEoAIeohPj6evLyL1xyQ3mP/nxT0RhLeyo/7h1j4dEsm2zNyjY4jhEuS3mOXJgW9ET06ohOtmnvzt692yZQAQtRBeHi4ze3Se6yMFPRG1MLXiz9d35n1h0+xctcxo+MI4XJue/BJlOfFi1b4Se+xClLQG9nEARF0au3Pi1/voajEanQcIVxGflEp603dib7nKSIiIlBKYTabSUxMlN5j5aSgNzJPDxN/uakr23/8knZh4TJxlxB2mr/mEEfPFDDv+T+SmpqK1WolJSVFinklMjmXAY5s/o6clW9TWlQAyMRdQtTm+JkC5v14kNE9r2BApAwiqoncoRsgPj6+ophfIF2vhKjZS9/spbjUytOjuhodxalJQTeAdL0Swn6bU0/zyZYMHhjWAUtIc6PjODUp6AaoqYuVyWSStnQhKrFaNc8v20mbFj48NqKT0XGcnhR0A9iauAugtLSUyZMn88gjjxiQSgjns2RTOtszc/nLTd1o7iOP/GojBd0AFybu8vCovlSW1pp58+bJnbpo8nLzinl5xV4GWFoxJqqd0XFcghR0g8TGxmK12u6HrrWWB6SiyXvtu33k5BUxa0x3mevcTlLQDXSp4crygFQ0ZXuPnmXhulTuGRhBj3aBRsdxGXYVdKXUKKXUXqXUAaXUM5fYb5xSSiulYhwX0X0lJCTUeOchc1OIpkrrsgehAb6ePHHDlUbHcSm1FnSllAfwNjAa6A5MVEp1t7FfAPA4sN7RId1VbGws06ZNq1bUfXybydwUosn6YtsR1h46yZMjr6Rlc2+j47gUe+7QBwAHtNaHtNZFwGJgrI39/gq8DBTYeE3UYO7cuSxcuLBiZSOfoDZEjP0jd9w1wehoQjSaC6sQmUwm7hreh1ZH1jNxgPyWernsKejtgfRK32eUb6uglOoDhGutv7zUgZRScUqpTUqpTdnZ2Zcd1l1VXtlo9eadFFmGMOeH/UbHEqJRVF6FSGtNUe5xdn/8DxYv+tDoaC7HnoJuq5G3YjJvpZQJeA14orYDaa0TtdYxWuuY0NBQ+1M2IYM6BDOuXxjzfzrE3qNnjY4jRIOztQpRQX6+9PSqA3sKegZQeVb5MCCr0vcBQE9gtVIqBRgELJMHo3X3l5u6EeDrSfxn27FaZSEM4d5kKgzHsaegbwQ6K6UilVLewARg2YUXtda5WusQrbVFa20B1gFjtNabGiRxE9CquTczburGptTTLNmUXvsbhHBhNfXokp5el6/Wgq61LgEeA1YAu4ElWuudSqnZSqkxDR2wqbqrXxgDIlvxf1/v4cS5QqPjCNFgnoyfhfKSVYgcwa5+6Frr5VrrLlrrjlrrhPJtM7XWy2zse43cndefUoq/396TvKIS/v7VbqPjCNFgtvv2os3Nj9M+LFxWIaonGSnqxDq1DuChqzvy6dZMfjlwwug4QjjcNzuOsmLnMZ77QxwZ6WmyClE9SUF3co9d24nIkOY88+k28opKjI4jhMPk5hXz3Oc76N62BQ8O62B0HLcgBd3J+Xp58NKdvUk/lc8rK/YaHUcIh/nrV7s4db6Il8f1xstDSpEjyN+iCxgQ2Yp7B5t579cUNqWcMjqOEPX2475slm7OYNrwDvRsL5NvOYoUdBfx1KiutAtsxlOfbKOguNToOELU2bnCEv7y6XY6hjbn99d2NjqOW5GC7iL8fTx58c5eHMo+zxvfy7QAwnW99PUesnLzeXlcFL5e1Rd5EXUnBd2FDOscyt0x4ST+dIhtGTlGxxHisq0/dJKF61KZcpWFfuaWRsdxO1LQXcxfbu5GiL83Ty3dRlGJ7RWPhHBG+UWlPPPpdsJbNWP6jTLPeUOQgu5iApt5kXBbL/YcPcscaXoRLuT/vt7N4RPneemO3vh5y4LPDUEKugu6vnsb7uoXxtzVB9icetroOELU6qd92XywNpX7h1i4qlOI0XHclhR0FzXz1u60C2rGn5ckc75QBhwJ55WbV8xTS7fRqbU/T4/qanQctyYF3UUF+Hrxz7uiSDuVR8JymetFOK/nPt/BiXOFvDY+Wnq1NDAp6C5sYIdg4oZ14MP1aazac9zoOEJU88VvWSz7LYvHr+tMrzAZQNTQpKC7uD+P7ELXKwKYvnQbp84XGR1HiArHzhTw7P92EB0exCPXdDQ6TpMgBd3F+Xh68Nrd0ZzJLyb+s+1oLSscCSiJeTUAABIRSURBVONZrZrpS7dRWFLKq+Oj8JS5WhqF/C27gW5tW/DEyC58veMoH22UFY6E8f7z82F+2pdN/M3d6RDqb3ScJkMKupt4cFgHhnYK4fkvdrLvmCwuLYyzLSOHl1fs4cYebZg0UJaRa0xS0N2EyaR49e4o/H08eezDLTKBlzDE2YJifr9oK6H+Prx0Z2+UUkZHalKkoLuR1gG+vDo+mn3HzjH7y11GxxFNjNaaZ/+3g/RTebwxsQ9Bft5GR2pypKC7mau7hDJteEc+XJ/GV9uOGB1HNCFLN2fweXIWf7y+C/0trYyO0yRJQXdDT4zsQnR4ENOef52w8AhMJhMWi4WkpCSjowk3dTD7HLOW7WRQh1Y8OqKT0XGaLJkhxw15eZi41msfy758A11cCEBqaipxcXEAsgCvcKi8ohIe/u9mfDxNvH53HzxM0m5uFLlDd1Ov/v2FimJ+QV5eHvHx8QYlEu5Ia80zn2xn//FzzJnYhysCfY2O1KRJQXdTaWlpl7VdiLr4YG0qy37L4okbujCsc6jRcZo8KehuKiLCdv/fmrYLcbk2p57mb1/t4rqurXnkGmk3dwZS0N1UQkICfn5+F20zefnw7KzZBiUS7uTEuUIeTdpC28BmvDo+GpO0mzsFKehuKjY2lsTERMxmM0oprmgfRsjo3/Or6obVKvO9iLorKbXy+KKtnM4r4l+T+hLo52V0JFFOCrobi42NJSUlBavVypGMdF555lF+2HOcN2TpOnGZkpKSsFgsmEwmQtqGsXLZUv52W096tJMpcZ2JFPQmZPIgM3f2DeON7/fLoCNht6SkJOLi4khNTUVrTW72Ec58+zZFe38yOpqoQgp6E6KU4u939CTG3JInPk5me0au0ZGEC4iPjycvL++ibcWFBdIF1gnZVdCVUqOUUnuVUgeUUs/YeP3PSqldSqltSqnvlVJmx0cVjuDj6cG8yf0Ibu7DAx9s5NiZAqMjCScnXWBdR60FXSnlAbwNjAa6AxOVUt2r7LYViNFa9waWAi87OqhwnBB/H965L4azBSXEfbBJZmYUlxQWHm5zu3SBdT723KEPAA5orQ9prYuAxcDYyjtorVdprS/8TrYOCHNsTOFo3dq24I0JfdiWmcuTH/8mKx0Jm0pKrXQY9QDK0+ei7X5+fiQkJBiUStTEnoLeHqi8DE5G+baaTAW+tvWCUipOKbVJKbUpOzvb/pSiQdzQvQ1Pj+rKl9uO8MqKvUbHEU5Ga82sZTtJadmXh+JfrOgCazabSUxMlDmBnJA9k3PZGjFg83ZOKTUJiAGG23pda50IJALExMTILaETeOjqDqSdymPu6oNcEejLvYMtRkcSTuJfPx4kaX0a04Z35JnRN8PzfzQ6kqiFPQU9A6jciBYGZFXdSSl1PRAPDNdaF1Z9XTgnpRSzx/Tg+JlCZi3bSesAX0b1vMLoWMJg/9uaycvf7GVMVDueuvFKo+MIO9nT5LIR6KyUilRKeQMTgGWVd1BK9QH+DYzRWh93fEzRkDw9TLw5sQ/R4UE8vngrf309sWIQicyj3vT8evAE05f+xqAOrXjlrt4yrN+F1FrQtdYlwGPACmA3sERrvVMpNVspNaZ8t1cAf+BjpVSyUmpZDYcTTqqZtwf/ua8/nod+YdZTj1cMIklNTWXy5Mk88sgjRkcUjWB7Ri5xH2wmMqQ5/54cg4+nh9GRxGVQRvVuiImJ0Zs2bTLk3KJmYeERZGakV9uulGLhwoXyIMyN7T92lvH/XouftydLHx5M28BmRkcSNiilNmutY2y9JiNFxUWyMjNsbtday8hAN5Z+Ko9J/1mPp4eJpAcGSjF3UVLQxUUuNVhERga6p2NnCoh9Zz2FJVb+O3UglpDmRkcSdSQFXVwkISEBpWw/BJORge7nxLlCJr2znpPnCnn//gFceUWA0ZFEPUhBFxeJjY1l2rRp1Yq6h5cP1984Snq/uJHss4VMTFxH+uk83rmvP1HhQUZHEvUkBV1UM3fuXBYuXFgxMrB12zD8el3He+++d1Hvl7i4OCnqLqTynObhERFcM+2vZJzO590pAxjcMdjoeMIBpJeLsEubduEcP1L9ganZbCYlJaXxA4nLcmFO88rT4CovH2a99Aaz/vSQgcnE5bpULxcp6MIuJpPJ5gReSimsVqsBicTlsFgspKamVtsuP5Bdj3RbFPVW0wNReVDqGmRO86ZBCrqwS0JCAn5+fhdtM3n58PjTzxmUSFyOK9rZntFafiC7Fynowi6xsbEkJiZWPCht2z6ciLF/4vXv9tM2LFx6vjixXw+ewDRgIiYvmdPc3UlBF3aLjY0lJSUFq9VKVkYaf7q+C5lfvM7RzAzp+eKkvt5+hCkLNtJt6E3MmfsvmdPczclDUVFn8qDNeWmtmffjIV76Zg99IoJYcF9/Wjb3NjqWcIBLPRS1Zz50IWySB23OqajEyrP/286STRnc0rst/7grCl8vmTWxKZAmF1FnNT1Q82vZhty84kZOIwBy8oq4d8F6lmzK4PFrOzFnQh8p5k2IFHRRZ7Z6vnj7NqP50EncPvcXDmafMyhZ07QjM5db3/qZLak5vHZ3FH8eeaUsTtHESEEXdVa154vZbGbBO/P54vUZ5OQXM/atX/h6+xGjYzYJSzdncOe/fqW4RLP4oUHc3sd2N0Xh3uShqGgQmTn5PJq0heT0HO4fYmHG6G54e8r9g6MVlpQy+4tdJK1PY3CHYN68pw8h/j61v1G4LHkoKhpd+6BmLHloMC9+vYcFvxxmS1oOb9/Th7CWfrW/WdjlwPFz/GHxVnZmnWHa8I48ObILnh7yQ7Mpk09fNBhvTxMzb+3Ov2L7cuj4OUa/sYY/JbyJ2WyWgUj1oLUmaX0qt7y5hqycfObfG8Mzo7tKMRdyhy4a3uhebenRLpBxT77MG/99EV1SCFAxEAmQAS52OnmukGc+3c63u44xrHMI/7writYtfI2OJZyEtKGLRmM2m232UZeBSLXTWvN5chazv9zFuYISnh7dlfuvskgvliZI2tCFU0hPT7e5XQYiXVpmTj7PfradVXuziQ4P4uVxvenSRpaKE9VJo5toNDUNRPJsEcp7vxym1GrMb4vOqqjEyjtrDjHy1R9Zd+gUM2/pzicPXyXFXNRICrpoNLYGIjVr1owBdz3C81/s4tY3f+bXAycMSudcVu05zqjXf+JvX+2mf2QrVv7pan43NBIPaWIRlyAFXTQaWwOR5s+fz5rEmbw5sQ+5+cXc8856pr63kQPHzxod1xB7j55lyrsbuP+9jaDg3Sn9ee/+AYS3ku6eonbyUFQ4jYLiUt77NYW3fzhAXnEp4/qG8ciIjpiDmxsdzaGSkpKIj48nLS2NiIgIEhISGHjDWF7/bh9fbT+Cv48nf7iuM/cOtshgLFGNrCkqXMqp80XM+X4/H25Io9SqGRvdjkdHdKJjqL/R0erN1mLNnt6+BN34KKHR13P/EAsPDutAkJ9MdStsk4IuXNLxMwX8+6dDJK1PpbDEysjubbjvKguDOwSjlGu2Jdc0h3xgaFsOHjpMsAzbF7WQgi5c2olzhfzn58Ms2pBGTl4xV7YJYPJgM7dGtSOwmZfR8ex28lwhIS2agY3/c0oprFarAamEq5GCLtxCQXEpy37L4v1fU9iZdQZvTxM3dGvDbX3aM7xLqFO2N+cXlfLjvmyW/ZbJt7uOcfitKZSeya62nwyuEvaSgUXCLfh6eTA+Jpy7+oWxLSOXz7Zm8sVvWXy1/QgBvp4M7xLK9d3aMLxLqKHLrZ08V8jPB06wYudRVu3JJr+4lFbNvbl3sAW/9v/HzOmPX9SGLos1C0exq6ArpUYBbwAewDta6xervO4DfAD0A04Cd2utUxwbVYgySimiwoOICg8i/uZurNmfzTc7jvLDnuN8ue0IJgW92gcSY2lFf0srYiwt7Z5SNikpiT/84Q+cPHkSgODgYN54440a55rRWnMkt4DtmblsOHyKXw6cYM/Rsi6XoQE+3NmvPaN7tmVgZKvyybO6c0Wgb7VeLjKXjXCEWptclFIewD7gBiAD2AhM1FrvqrTPI0BvrfU0pdQE4Hat9d2XOq40uQhHs1o12zJz+WH3MdYdPkVyeg5FJWXt0q0DfLjyigCubBNAh1B/2gb50i6wGVcE+tLC1xOlFElJSfzud7+jqKjoouN6eXnx5rz5XHfLHaSfziftZB4pJ89zKPs8OzJzOXm+bH9vTxP9LS25qmMIQzqF0Kt9oAwEEg5X3yaXAcABrfWh8oMtBsYCuyrtMxZ4vvzrpcBbSimljWqgF02SyaSIDg8iOjwIKFv8YUdmLltSc9hz9Cz7jp1l4bqyHjMXvU9Bc29P9s35c7ViDlBcXMyjf5pO2L6Qim3NvDywhDRnRNfW9A4LpGf7QLq3bSHrdwpD2VPQ2wOVZ1XKAAbWtI/WukQplQsEAxeN41ZKxQFxUPO8HkI4io+nB/3MrehnblWxrdSqOXamgCO5+WTlFHA0t4AzBcWcKyzh+dzqDysr3nf2BG9MiKZ9UDMigv0I9fdx2a6Twn3ZU9Bt/auteudtzz5orROBRChrcrHj3EI4lIdJ0S6oGe2CmtHPfPFr70ZE2OwjDmCOiGBsdPtGSChE3dnTzysDCK/0fRiQVdM+SilPIBA45YiAQjSWhIQEvL2r947x8vKSXijCJdhT0DcCnZVSkUopb2ACsKzKPsuA+8q/Hgf8IO3nwtXExsayYMECgoODK7YFBwfz7rvvSi8U4RLsGliklLoJeJ2ybosLtNYJSqnZwCat9TKllC+wEOhD2Z35hAsPUWsivVyEEOLy1XtgkdZ6ObC8yraZlb4uAO6qT0ghhBD143xjpYUQQtSJFHQhhHATUtCFEMJNSEEXQgg3Ydj0uUqpbMD2KI7ahVBlFGoTINfcNMg1Nw31uWaz1jrU1guGFfT6UEptqqnbjruSa24a5Jqbhoa6ZmlyEUIINyEFXQgh3ISrFvREowMYQK65aZBrbhoa5Jpdsg1dCCFEda56hy6EEKIKKehCCOEmnLqgK6VGKaX2KqUOKKWesfG6j1Lqo/LX1yulLI2f0rHsuOY/K6V2KaW2KaW+V0qZbR3HldR2zZX2G6eU0kopl+/iZs81K6XGl3/WO5VSHzZ2Rkez4992hFJqlVJqa/m/75uMyOkoSqkFSqnjSqkdNbyulFJzyv8+timl+tb7pFprp/xD2VS9B4EOgDfwG9C9yj6PAPPKv54AfGR07ka45hGAX/nXDzeFay7fLwD4CVgHxBiduxE+587AVqBl+fetjc7dCNecCDxc/nV3IMXo3PW85quBvsCOGl6/CfiashXfBgHr63tOZ75Dr1icWmtdBFxYnLqyscD75V8vBa5Trr3QY63XrLVepbXOK/92HWUrSLkyez5ngL8CLwMFjRmugdhzzQ8Cb2utTwNorY83ckZHs+eaNdCi/OtAqq+M5lK01j9x6ZXbxgIf6DLrgCClVNv6nNOZC7qtxamrLup40eLUwIXFqV2VPddc2VTKfsK7slqvWSnVBwjXWn/ZmMEakD2fcxegi1LqF6XUOqXUqEZL1zDsuebngUlKqQzK1l/4feNEM8zl/n+vlV0LXBjEYYtTuxC7r0cpNQmIAYY3aKKGd8lrVkqZgNeAKY0VqBHY8zl7Utbscg1lv4WtUUr11FrnNHC2hmLPNU8E3tNa/1MpNRhYWH7N1oaPZwiH1y9nvkNviotT23PNKKWuB+KBMVrrwkbK1lBqu+YAoCewWimVQllb4zIXfzBq77/tz7XWxVrrw8Beygq8q7LnmqcCSwC01msBX8omsXJXdv1/vxzOXNCb4uLUtV5zefPDvykr5q7ergq1XLPWOldrHaK1tmitLZQ9NxijtXblBWnt+bf9P8oegKOUCqGsCeaS6/Q6OXuuOQ24DkAp1Y2ygp7dqCkb1zLg3vLeLoOAXK31kXod0egnwbU8Jb4J2EfZ0/H48m2zKfsPDWUf+MfAAWAD0MHozI1wzd8Bx4Dk8j/LjM7c0NdcZd/VuHgvFzs/ZwW8CuwCtlO28LrhuRv4mrsDv1DWAyYZGGl05npe7yLgCFBM2d34VGAaMK3SZ/x2+d/Hdkf8u5ah/0II4SacuclFCCHEZZCCLoQQbkIKuhBCuAkp6EII4SakoAshhJuQgi6EEG5CCroQQriJ/wfZiC9quTHD/QAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def f(x):\n",
" # quadratic\n",
" return ((x - .5)**2)/.25\n",
"\n",
"def df(x):\n",
" # derivative of quadratic is linear\n",
" return 2*(x - .5) / .25\n",
"\n",
"# sample size\n",
"n = 1000\n",
"# these will be my training data sets, with regular sampling\n",
"x = np.linspace(0,1,n)\n",
"y = f(x)\n",
"\n",
"# get test data too, but randomly sampled\n",
"x_test = np.random.ranf(20)\n",
"y_test = f(x_test)\n",
"\n",
"plt.plot(x,y, label='training points')\n",
"plt.plot(x_test,y_test, 'ko', label='test data')\n",
"plt.legend()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define and run tiny nonlinear regression NN"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<keras.callbacks.callbacks.History at 0x1ee9574e988>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model = Sequential()\n",
"# 1d input\n",
"model.add(Dense(64, input_dim=1, activation='relu'))\n",
"model.add(Activation(\"linear\"))\n",
"model.add(Dense(32, activation='relu'))\n",
"model.add(Activation(\"linear\"))\n",
"model.add(Dense(32, activation='relu'))\n",
"# 1d output\n",
"model.add(Dense(1))\n",
"\n",
"# minimize mse\n",
"model.compile(loss='mse', optimizer='adam', metrics=[\"accuracy\"])\n",
"\n",
"\n",
"model.fit(x, y,\n",
" batch_size=10,\n",
" epochs=25,\n",
" verbose=0,\n",
" validation_data=(x_test, y_test))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plot performance (not bad, not perfect but really just an example model)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x1ee96815f08>"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhU5fn/8fednZCQEEgQyMqmRAiLYRMQEKVgFa2igkFEEYpLtbW2Lmhd+VVtvxaxWIrgHsVdcW9FEES2IMi+SVbCEgIEQvbM8/sjgYYwIQPMzJmZ3K/r4rqSMycznzNJbk7OeZ77EWMMSimlvJ+f1QGUUko5hxZ0pZTyEVrQlVLKR2hBV0opH6EFXSmlfESAVS/cunVrk5iYaNXLK6WUV1qzZs0BY0y0vccsK+iJiYlkZGRY9fJKKeWVRCS7ocf0kotSSvkILehKKeUjtKArpZSP0IKulFI+Qgu6Ukr5iEYLuoi8IiL7RWRjA4+LiMwUkZ0isl5Eejs/plJKqcY4cob+GjDyNI+PAjrX/psC/OvcYymllDpTjRZ0Y8wS4OBpdrkaeMPUWAFEikhbZwWsb13uYZ75aiva9lcp5W1sNsP0LzazKb/IJc/vjGvo7YHcOp/n1W47hYhMEZEMEckoKCg4qxfbkHeY2d//wsbdR87q65VSyiorMw/y8tJMtu876pLnd0ZBFzvb7J4+G2PmGGNSjTGp0dF2Z642anTP9oQE+vHO6pyz+nqllLLKexm5hIcEMKqbay5iOKOg5wFxdT6PBfKd8Lx2RTQL5IrubVmwLp+SiipXvYxSSjlVUWklX27Yw+ge7QgJ9HfJazijoC8AJtSOdukPFBlj9jjheRs0tk88xeVVfL7epS+jlFJO8+m63ZRX2RjbJ95lr+HIsMV3gOXA+SKSJyKTRGSqiEyt3eVLYBewE3gZuNNlaWv1SWxJx+jmvLs6t/GdlVLKYsYY3lmVS7f2LegeG+Gy12m026IxZlwjjxvgLqclcoCIMLZPPNO/3ML2fUfp0ibcnS+vlFJnZH1eEVv2HOHpa7q59HW8dqbotb3bE+gvzF+lZ+lKKc82f3UOzQL9ubpnO5e+jtcW9FZhwYxIPo+P1uZRXlVtdRyllLKruLyKT9flc2VKW8JDAl36Wl5b0AFu7BPH4ZJKvtm0z+ooSill12c/51NSUc3Yvq67GXqcVxf0QZ1a0z6yGe/qmHSllIeavyqHLm3C6B0f6fLX8uqC7ucn3NgnjmU7C8kuPGZ1HKWUOsnm/CP8nFfE2D7xiNibg+lcXl3QAa5PjcVPamZgKaWUJ5m/OoegAD+u7W23G4rTeX1BbxvRjGHnx/B+Rh5V1Tar4yilFAClFdV8vHY3o7qdR2RokFte0+sLOtTcHN1/tJxF286u4ZdSSjnblxv2cLSsyqUzQ+vziYJ+6QUxxIQHM3+V3hxVSnmG+atzSGrdnP4dotz2mj5R0AP8/RhzUSyLtu1nT1Gp1XGUUk3czv1HWZ11iBv7xLnlZuhxPlHQAcb1jceAzhxVSllu/qpcAvyE63rHuvV1faagx0WFMqRLNPNX51CpN0eVUhYpq6zmw5/yuDy5DdHhwW59bZ8p6ABp/RLYd6SchVt05qhSyhpfbtjDoZJKbu6f4PbX9qmCfukFMbSLCCF9pd4cVUpZ480V2XSIbs6Ajq3c/to+VdD9/YRxfeNZuuMAmQd05qhSyr025RexNucwaf0S3Hoz9DifKuhQMyY9wE94e2W21VGUUk3MWytyCAn0Y4ybb4Ye53MFPaZFCCMubMP7a/Ioq9S2ukop9zhaVsmn63ZzVUo7IkJd2ya3IT5X0AHG90vgcEnNgqxKKeUOH6/dTUlFNTcPcP/N0ON8sqAP6NiKDq2b89YKveyilHI9YwxvLs8mJTaClFjXt8ltiE8WdBHhpn7x/JRzmM35R6yOo5TycasyD7JjfzHj+1l3dg4+WtABxlwUS3CAH+l6c1Qp5WJvrcyhRUgAV/Vw7ZqhjfHZgh4ZGkSnhHU89/Nl+D3hR+KMRNI3pFsdSynlYwqOlvP1xj1cd1EszYL8Lc0SYOmru1D6hnS+2/cUVVLTrCu7KJspn00BIK17mpXRlFI+5L2MXCqrDWkWX24BHz5Dn7ZwGmXVJ3deLKksYdrCaRYlUkr5mmqb4e2VOVzcsRWdYsKsjuO7BT2nyP70/4a2K6XUmVq8bT+7D5cy3oK+Lfb4bEGPj7C/SkhD25VS6ky9uSKbmPBgLk9uY3UUwIcL+vTh0wkNDD1pW7OAZkwfPt2iREopX7KroJjF2wpI65dAoL9nlFLPSOECad3TmHPVHBIiEhAEf1s0oxOe0BuiSimneGN5NoH+NXNePIXPjnKBmqJ+vIBPeSODjMxDlFVWExJo7dAipZR3O1pWyQdr8rgqpZ3bF7E4HZ89Q69v4sWJHDxWwWc/51sdRSnl5T5ck0dxeRW3XJxodZSTNJmCPqBjK7q0CeP15VkYY6yOo5TyUjab4Y3l2fSKj6RHnHV9W+xxqKCLyEgR2SYiO0XkQTuPx4vIIhFZKyLrReQK50c9NyLChAGJbNx9hJ9yDlkdRynlpZbsKGDXgWNM9LCzc3CgoIuIPzALGAUkA+NEJLnebo8A7xljegFjgZecHdQZftOrPeEhAby6LMvqKEopL/Xaj1lEhwczqltbq6OcwpEz9L7ATmPMLmNMBTAfuLrePgZoUftxBOCRF6qbBwdwQ2ocX2/cy74jZVbHUUp5mcwDx1i8rYDx/RIICvC8K9aOJGoP5Nb5PK92W12PA+NFJA/4EvidvScSkSkikiEiGQUFBWcR99xNGJBAtTGka690pdQZev3HLAL9hXH94qyOYpcjBd3eSqf17yqOA14zxsQCVwBvisgpz22MmWOMSTXGpEZHR595WidIaNWcS8+P4e1VOZRX6RJ1SinHFJdX8cGaPK5MaUdMeIjVcexypKDnAXX/O4rl1Esqk4D3AIwxy4EQoLUzArrCLRcncqC4QpeoU0o57PhQRU+8GXqcIwV9NdBZRJJEJIiam54L6u2TAwwHEJGu1BR0a66pOGBQp9Z0iG7Oq8t0CKNSqnE2m+H1H7PoGed5QxXrarSgG2OqgLuBb4At1Ixm2SQiT4rI6Nrd/ghMFpGfgXeAicaDK6Wfn3DbwCTW5xWRka1DGJVSp3d8qOKtAxOtjnJaDk39N8Z8Sc3Nzrrb/lLn483AQOdGc63resfy9/9sY+7SXfRJjLI6jlLKg837IZMYDx2qWJfnjbtxk2ZB/ozvl8B/Nu8ju/CY1XGUUh5q296jLN1xgFsuTvTIoYp1eXY6F5swIIEAP9GJRkqpBs1duotmgf6keVBXxYY06YIe0yKE0T3a815GLkUllVbHUUp5mP1Hy/h0XT7Xp8YSGRpkdZxGNemCDjBpUBIlFdW8s1qXplNKneyt5dlU2mzcNjDJ6igOafIFPbldCwZ2asVry7KorLZZHUcp5SFKK6p5c0U2l3dtQ2Lr5lbHcUiTL+hQc5a+90iZTjRSSp3w0do8DpVUcvvgDlZHcZgWdGBolxg6RDdn7tJMnWiklMJmM8xbmklKbAR9EltaHcdhWtCpmWg0aVASG3YXsTpLJxop1dQt2rafXQeOcfvgDojYa2flmbSg17q2VywtQwOZu3SX1VGUUhabuzSTdhEhjOp2ntVRzogW9FrNgvwZ3z+B/27ZR9YBnWikVFO1cXcRy3cVMnFgIoH+3lUivSuti908IIFAPz/m/ZBpdRSllEXm/ZBJ8yB/xvb1/IlE9WlBryMmPITf9KqZaHSguNzqOEopN9t9uJTPfs7nxj7xtAgJtDrOGdOCXs+UIR2oqLbx+o9ZVkdRSrnZ8Xtotw/2jolE9WlBr6djdBgjktvwxvJsjpVXWR1HKeUmh45VMH9VLqN7tqNdZDOr45wVLeh2TB3SkaLSSt5Zpe0AlGoqXl+eRWllNVOHdLQ6ylnTgm5Hr/iW9EuKYu7STCqqtB2AUr6upKKK13/M4rKuMXRpE251nLOmBb0BU4d2ZO+RMj5dt9vqKEopF3t3dS6HSiq9+uwctKA3aGiXaC44L5x/L9mFzabtAJTyVZXVNuYuzaRPYktSvXz1Mi3oDRARpg7pyM79xSzcut/qOEopF/l8fT67D5d6/dk5aEE/rStT2tI+shmzv//F6ihKKRcwxjB78S66tAlj2PkxVsc5Z1rQTyPA34/Jg5NYk32I1VkHrY6jlHKyRdv2s23fUaYO6Yifn/c04WqIFvRG3NAnjpahgcxerGfpSvma2Yt30T6yGVf1aGd1FKfQgt6I0KAAJl6cxMKt+9my54jVcZRSTrIm+yCrsg4yaVCS1zXhaohvHIWLTbw4kbDgAP65aKfVUZRSTvLidzuJah7E2L5xVkdxGi3oDogIDWTCgAS+3LCHnfuPWh1HKXWO1ucdZvG2AiYNSiI0KMDqOE6jBd1BkwYlERLgz0uL9Fq6Ut7un9/tpEVIABMGJFgdxam0oDuoVVgwaf3i+fTnfLILdQEMpbzVlj1H+M/mfdw6MIlwL2yRezpa0M/AlEs64O8n/EtHvCjltf65aCdhwQHcOjDR6ihOpwX9DMS0CGFsnzg+/CmP3YdLrY6jlDpDO/cf5csNe7h5QAKRoUFWx3E6Lehn6Le104P/rbNHlfI6Ly36hZAAf24f5J0LWDTGoYIuIiNFZJuI7BSRBxvY5wYR2Swim0TkbefG9BztI5txXe9Y5q/OZf+RMqvjKKUclF14jE9/zietXzytwoKtjuMSjRZ0EfEHZgGjgGRgnIgk19unM/AQMNAYcyHwexdk9Rh3Du1Etc3wcu1yVUopz/fSol/w9xOmXNLB6igu48gZel9gpzFmlzGmApgPXF1vn8nALGPMIQBjjE+3J4xvFcrVPdrx1oocCnUxaaU8Xt6hEj78KY+xfeKIaRFidRyXcaSgtwdy63yeV7utri5AFxFZJiIrRGSkvScSkSkikiEiGQUFBWeX2EPcOawTZVXVvLw00+ooSqkGpG9IJ3FGInEzw8gOmkhMm9VWR3IpRwq6vRZk9Vd8CAA6A0OBccBcEYk85YuMmWOMSTXGpEZHR59pVo/SKSaM0T3a8cbyLA7oWbpSHid9QzpTPptCdlE2YKiSAh747m7SN6RbHc1lHCnoeUDdZgexQL6dfT41xlQaYzKBbdQUeJ92z/DOlFVW64gXpTzQtIXTKKksOWlbSWUJ0xZOsyiR6zlS0FcDnUUkSUSCgLHAgnr7fAIMAxCR1tRcgvH5O4Ydo8O4pld73lierSNelPIwOUU5Z7TdFzRa0I0xVcDdwDfAFuA9Y8wmEXlSREbX7vYNUCgim4FFwJ+MMYWuCu1J7h3emSqb4SWdPaqUR4mPiD+j7b7AoXHoxpgvjTFdjDEdjTHTa7f9xRizoPZjY4y5zxiTbIzpboyZ78rQniShVXPG9I7l7VU57CnS2aNKeYrf9/0LYk4ebx4aGMr04dMtSuR6OlPUCe6+tBPGGGZpv3SlPEZObm/a2O4hNjwOQUiISGDOVXNI655mdTSX8Z1GwBaKiwrlhtQ43l2dy9QhHYltGWp1JKWatJ37i/lk3W5+N/hWHr7iOavjuI2eoTvJXcM6IQj//E7P0pWy2oxvtxMS6M9vfXhWqD1a0J2kXWQzxvWN4/01edovXSkLbd17hC827OHWgYk+27OlIVrQnejOYZ0I8BNmLtSzdKWsMuO/OwgLCmDy4KZ1dg5a0J2qTYsQxvdP4OO1ebr2qFIWWJ93mK837eW2QUk+2e+8MVrQnezOoR0JDQrgb99sszqKUk3Oc19vI6p5ELcP9s1+543Rgu5krcKCmTy4A99s2sdPOYesjqNUk/HDjgP8sPMAdw3r5HNrhTpKC7oL3D44idZhQTz71VaMqd/HTCnlbDab4dmvt9I+shnj+/vuTNDGaEF3gebBAfzu0s6szDzI99u9u02wUp7seHvcgKf8+erAtaR23URwgL/VsSyjBd1FxvWNJy6qGc9+vQ2bTc/SlXK2uu1xDYZqvwLmbnrQp9vjNkYLuosEBfjxx8vPZ8ueI3y2vn63YaXUuWqK7XEbowXdhUb3aEfXti34v/9sp6LKZnUcpXxKU2yP2xgt6C7k5yf8eeT55BwsYf7qpvtDppQrNMX2uI3Rgu5iQ7tE0zcpipkLd3CsvMrqOEr5jIcHPtHk2uM2Rgu6i4kID426gAPFFbpUnVJOtGdvKq0q76ZdWNNpj9sYbZ/rBr3iW3JlSlvmLN3FuH7xtI1oZnUkpbzaroJi3lyeze2pt/DXa/9udRyPoWfobvLAyAuwGbQlgFJO8MxXWwkO8OO+y7tYHcWjaEF3k7ioUG4dmMhHP+1mQ16R1XGU8lordhXyn837uHNYJ6LDm1Z73MZoQXeju4Z1Iqp5EE9/sVlbAih1Fmw2w9NfbKZdRAiTBjXNBlynowXdjVqEBPKHy2paAvxn8z6r4yjldT5eu5uNu4/w55EXEBLYdKf4N0QLupuN6xtPp5gwnvlqq042UuoMlFZU87dvtpESG8HoHu2sjuORtKC7WYC/Hw9fcQGZB47x1opsq+Mo5TVeXrqLvUfKeOTXyfj5idVxPJIWdAsMOz+GQZ1a88LCHRwuqbA6jlIeb/+RMmZ//wujup1H36Qoq+N4LC3oFhARpv26K0fLKpnx7Q6r4yjl8Z79ehuV1TYeGHmB1VE8mhZ0i3Rt24Kb+sXz5opstu49YnUcpTzWmuxDfPhTHrcP7kBi6+ZWx/FoWtAt9MfLzyc8JIDHPt2kwxiVssNmMzy+YBNtWgRz97BOVsfxeFrQLdSyeRD3jziflZkH+Xz9HqvjKOVx3svIZcPuIh6+oivNg7VTSWO0oFtsXN94LmzXgv/35RZKKrQbo1LHFZVU8tw32+ibGKXDFB2kBd1i/n7CE6MvZE9RGbMW7bQ6jlIe4x/fbudwSQWPjU5GRIcpOkILugdITYziN73a8/KSTLIOHLM6jlKW27b3KG+uyOamfvFc2C7C6jhew6GCLiIjRWSbiOwUkQdPs98YETEikuq8iE3DQ6MuINBfeOrzzVZHUcpSxtTcCA0PCeCPl59vdRyv0mhBFxF/YBYwCkgGxolIsp39woF7gJXODtkUxLQI4Z7hnVm4dT/fbdU+L6rp+mz9HpbvKuT+EefTsnmQ1XG8iiNn6H2BncaYXcaYCmA+cLWd/Z4CngPKnJivSbl1YBIdo5vzl083UVpRbXUcpdyuqLSSpz7fTEpsBOP6Nt21Qc+WIwW9PZBb5/O82m0niEgvIM4Y8/npnkhEpohIhohkFBQUnHFYXxcU4Mf033Qn71ApM7/TGaSq6UjfkE7ijEQinwtmbcVNDOy+FX/t13LGHCno9t7VE7NgRMQP+Afwx8aeyBgzxxiTaoxJjY6OdjxlE9K/QyvGXBTLy0t2sW3vUavjKOVy6RvSmfLZFLKLsgFDtV8BTy67l/QN6VZH8zqOFPQ8IK7O57FAfp3Pw4FuwGIRyQL6Awv0xujZe/iKroSHBDDt4w3YbDqDVPm2aQunUVJZctK2ksoSpi2cZlEi7+VIQV8NdBaRJBEJAsYCC44/aIwpMsa0NsYkGmMSgRXAaGNMhksSNwFRzYN46IquZGQf4r2M3Ma/QCkvllOUc0bbVcMaLejGmCrgbuAbYAvwnjFmk4g8KSKjXR2wqbr+oljatFnNhK/64PeEH4kzEvVPUOWT2oXH2t0eH6E3Rc+UQ80RjDFfAl/W2/aXBvYdeu6x1Nsb32Z9yXNUUgpAdlE2Uz6bAkBa9zQroynlVF1CJpN/ZDpGyk9sCw0MZfrw6Ram8k46U9RDTVs4jbKq0pO26XVF5Wu+3riXXTm9mdD1/5EQkYAgJEQkMOeqOXricha0fZmH0uuKytcVlVTy6KcbSW7bgpfH/J5A//usjuT19AzdQzV0/VCvKypf8dQXmzl4rILnxqQQ6K+lyBn0XfRQ04dPJzQw9KRtYoKZlPKwRYmUcp7vtxfwwZo8pg7pQLf22nzLWbSge6i07mnMuWrOieuKcS3i6RRwH4vWdqGsUtsCKO9VXF7Fwx9toGN0c353aWer4/gUvYbuwdK6p510Y2jpjgJunreKFxbu0MVyldd69qut5BeV8sHUiwkJ9Lc6jk/RM3QvMrhzNDemxjFnyS7W5x22Oo5SZ2zlrkLeXJHNxIsTuSihpdVxfI4WdC/z8K+70josiD9/sJ6KKpvVcZRyWGlFNQ9+tIG4qGb86Vfa59wVtKB7mYhmgUy/pjsZBQto87c4nUWqvMZfv9pC5oFjPHttCqFBerXXFfRd9UL7qr+lKHgWVRU1red1FqnydEu2F/DG8mxuHZjIxZ1aWx3HZ+kZuheatnAaVebkdUR0FqnyVEUllfz5g/V0ignTm/kupgXdC+ksUuVNHv10IweKy/nHDT11VIuLaUH3QjqLVHmLz37OZ8HP+dwzvDPdY3UCkatpQfdCdmeREszDA5+wKJFSp9p3pIxHPtlIz7hI7hza0eo4TYIWdC9UfxZp27A4YqruYe22bhijKxwp69lshj99sJ7yqmqev6EHAdqrxS10lIuXqj+L9N/f/8Jfv9rKu6tzGaurpSuLzfshkyXbC3jqmm50iA6zOk6Tof9t+ojJgzswqFNrHv9sE9v36eLSyjrr8w7z3Ddb+dWFbRjfT08u3EkLuo/w8xOev7EHYcEB3P32T9rAS1niaFklv3tnLdFhwTx7XQoiYnWkJkULug+JCQ/h+Rt6sn1fMU9+vtnqOKqJMcbwyCcbyT1YwgvjehEZGmR1pCZHC7qPuaRLNFOHdGROxuvEPBerrQGU23ywJo9P1+Xz+8u60Ccxyuo4TZLeFPVB7dplcDjon1SX1iy6q60BlKv9UlDMYws20b9DFHcN62R1nCZLz9B90F8WPUI15SdtK6ks4d6v7rUokfJF6RvSSZyRiN8TfnT7VyeO+S9mxo298PfT6+ZW0YLugxpqAVBYWqiXXpRTpG9IZ8pnU8guysZgKDP72Oc/k4U5H1odrUnTgu6DTtcCQBt4KWeYtnAaJZUlJ20rry7Vny+LaUH3QdOHT2/wMW3gpZxBG8R5Ji3oPiitexqtmrWy+1hsizg3p1G+qH24/Z8jbRBnLS3oPuqFUS+c2sDLBJMUeDs2m/Z7UWevqtpGnP9tiAk+aXtoYOhp/zpUrqcF3UfVb+CVEJHAlO7PkJ3Xmwnznz8xOkHHqKsz9fQXW8jfk8rdvZ476edrzlVzdFisxcSq7nypqakmIyPDktduqowx/HruM3y9+wmM/G9YY2hgqP4yKoe8syqHhz7awKRBSTx6ZbLVcZokEVljjEm195ieoTchIsKm4tknFXPQ5euUY1buKuTRTzZySZdoHhqlS8l5IocKuoiMFJFtIrJTRB608/h9IrJZRNaLyEIRSXB+VOUMuUdy7W7X0QnqdHIPlnBH+k/EtwrlxXG9tL+5h2r0uyIi/sAsYBSQDIwTkfp/a60FUo0xKcAHwHPODqqcQ5evU2equLyKyW9kUFVtY+6EVCKaBVodSTXAkf9m+wI7jTG7jDEVwHzg6ro7GGMWGWOOzzJYAcQ6N6ZyFnvL1wVICNMv1dEJ6lRV1TbufWct2/cd5Z839dbFKjycIwW9PVD37/S82m0NmQR8Ze8BEZkiIhkiklFQUOB4SuU09Ue/RAW3I6L8LnbnX2R1NOVhjDE8tmATC7fu54nRF3JJl2irI6lGONJt0V6nHbtDY0RkPJAKDLH3uDFmDjAHaka5OJhROVnd5euMMUz7ZCMvLf6F8yJCmDAg0dpwylLpG9KZtnAaOUU5RAa3xe/oOO4fcjs368+FV3CkoOcBdaeFxQL59XcSkcuAacAQY0x5/ceVZxIRnhx9IfuPlPPYgk3EhIcwstt5VsdSFjjecOt4j5ZD5fkEBM8itl0vQEe1eANHLrmsBjqLSJKIBAFjgQV1dxCRXsC/gdHGmP3Oj6lcKcDfjxfH9aJnXCT3zF/L6qyDVkdSFrDXcKvKlPHIIh3S6i0aLejGmCrgbuAbYAvwnjFmk4g8KSKja3f7GxAGvC8i60RkQQNPpzxUsyB/5t3Sh/aRzbj99Qx26ELTTUr6hnSyi7LtPqZDWr2HzhRVJ8k9WMJvXvqRAD/h/akDiIsKbfyLlFerf6mlvoSIBLJ+n+XeUKpBOlNUOSwuKpQ3butLaWU1aXNXsreozOpIysXsXWo5ThtueRct6OoUye1a8PptfSksLidt7goKi/Uet6+pu3xcQ5daAO3x42W0oCu7esZF8srEPuw+XMrN81ZRVFJpdSTlJOkb0rn1k1tPLB/XkISIBC3mXkYLumpQvw6t+PfNqezcX8wtr65i7po3tO2uD7j3q3uptJ3+P2i91OKdtKCr0xrSJZp/3tSL5Xs/Yern/1sUOLsomymfTdGi7oUKSwsbfEx7m3s3RyYWqSZuxIXnQYv5VJfYb7urv/i+w/aYzeoI6hx4VEGvrKwkLy+PsjIdWeFqISEhxMbGEhjoWOe8gpLddrfrGGXvsreojABaUMWRUx5raB1a5T08qqDn5eURHh5OYmIiIvZayChnMMZQWFhIXl4eSUlJDn1NfES83dEQ2nbXe2QeOMb4uSuJsf2WgoAXqLRVnHgsyD+IF0a9YGE65QwedQ29rKyMVq1aaTF3MRGhVatWZ/SXkL22u34Ec0/qo86Op1xg4+4ixvzrR0orq/lq8sO8es0rJ60H+srVr+ilMx/gUWfogBZzNznT9/n4L/vxTnznhcUSfCyNN76LY0jcIS5KaOmKmMoJfvzlAFPeWENEs0DenNSXDtFhpMSmaQH3QR5X0JXnqtt2FyCnsIQJr6wkbe4Krh2YydtbnyOnKIf4iHimD5+uBcMCddvfxkfEc0PnP/HJjx1IaBXKG5P60jaimdURlQt51CUXX7d48WJ+/PFHq2M4TXyrUD6442JCI5bzzIr7dEijxY73ZKn7ffj76j/SotUK3vvtAC3mTYAWdDfytYIO0DosmAP+r2PE/pBG5T72erIYKWe/vEbL5kEWpVLu5LGXXJ74bBOb808dWnUuktu14LGrLmx0v7feeouZM2dSUS8CGqcAABB6SURBVFFBv379ePjhh7nssstYvnw5UVFRDBkyhEcffZQRI0ZwzTXXkJubS1lZGffeey9TpkwB4Ouvv+bhhx+murqa1q1bM2/ePGbPno2/vz9vvfUWL774IoMHD3bq8Vkl70iu3e06pNG9Gnq/G/r+KN/jsQXdKlu2bOHdd99l2bJlBAYGcuedd/L999/zwAMPMHXqVPr160dycjIjRowA4JVXXiEqKorS0lL69OnDddddh81mY/LkySxZsoSkpCQOHjxIVFQUU6dOJSwsjPvvv9/io3SuhoY0tg3TtcLd6bywWPYUn1q8dWhp0+GxBd2RM2lXWLhwIWvWrKFPnz4AlJaWEhMTw+OPP87777/P7NmzWbdu3Yn9Z86cyccffwxAbm4uO3bsoKCggEsuueTEGO+oqCj3H4gbTR8+/ZR+2kIwcmQcX23Yw6jubS1M1zR8sCYPc3gsfgEzsfG/y1/ak6Vp8diCbhVjDLfccgt//etfT9peUlJCXl4eAMXFxYSHh7N48WK+/fZbli9fTmhoKEOHDqWsrAxjTJMafll/SGN8RDz393+Mb9d04Y70n7h1YCIPjepKUIDesnG28qpqnvxsM+krcxja4VqG9LqQZ5Y9pqONmigt6PUMHz6cq6++mj/84Q/ExMRw8OBBjh49yt///nfS0tJISEhg8uTJfP755xQVFdGyZUtCQ0PZunUrK1asAGDAgAHcddddZGZmnnTJJTw8nCNHnHtfwFPUH9IIMCXVxjNfbeWVZZn8lHOYWTf1IralroDkLDv3F3Pv/LVsyj/C1CEduX9EFwL8+zO1zy1WR1MW0VOmepKTk3n66acZMWIEKSkpXH755WRlZbF69WoeeOAB0tLSCAoK4tVXX2XkyJFUVVWRkpLCo48+Sv/+/QGIjo5mzpw5XHvttfTo0YMbb7wRgKuuuoqPP/6Ynj17snTpUisP0y2CAvz4y1XJ/CutN7v2FzPqhaV8vDYPq5Y99BXGGNJXZnPli0vJP1zKyxNSeXDUBQT4669zU+dRa4pu2bKFrl27WpKnKXLn+51TWMJ9761jcd5HlIemc6xqn14SOAuFxeU8+NEG/rt5H4M7t+b/ru9BTIsQq2MpNzrdmqJ6yUW5RXyrUK6+OJMFC2ZRWVXTQ+b4BCRAi3ojjDF8ui6fJz/fTHFZFY9emcytFyfi59d07tWoxunfaMptHl00jUrbyQ3BSipLePC/D1mUyDvsPlzKba+t5vfvriM+KpTP7xnEpEFJWszVKfQMXblNwxNf8nhtWSY3D0jEX4vUCRVVNt5YnsU//rsdm4G/XJnMLRfre6QapgVduU1DE5CaB7Th8c82815GHo/8uisXd2ptQTrPsmjrfp76fDO7Dhxj6PnRPHV1N+KidISQOj295KLcxl5P9dDAUP49+u+8OK4XRaWV3DR3JZNeW83O/UeBmoZTTWlh6m17jzLx1VXc+tpqEHh1Yh9eu7WvFnPlED1DV25jbwJS3VEulye34bUfs5j13U5+NWMpyR1+ZuG+pyitKgV8+ybqzv3FzPh2O19s2ENYcACP/LorEwYk6mQsdUa8+qfFG87ewsLCAMjPz2fMmDGn3XfGjBmUlPxv+vwVV1zB4cOHXZrP3dK6p5H1+yxsj9nI+n3WSYU5JNCfqUM68v2fh3Fz/wS+zptxopgf52tdHLftPcp9765jxD++57ut+7lzaEeW/nkYtw/uoMVcnTGvHYd+vPdz3f4hoYGhzLlqjsvP3qqrq/H393do37CwMIqLix3aNzExkYyMDFq3ds81ZE8f9+/3hB+GU38+BfHq1emNMfyw8wAvL81kyfYCQgL9uLl/AlOHdKRVWLDV8ZSHO904dK89BbDX+9kZZ29ZWVlccMEF3HLLLaSkpDBmzBhKSkpITEzkySefZNCgQbz//vv88ssvjBw5kosuuojBgwezdetWADIzMxkwYAB9+vTh0UcfPel5u3XrBtT8h3D//ffTvXt3UlJSePHFF5k5cyb5+fkMGzaMYcOGATUF/sCBAwA8//zzdOvWjW7dujFjxowTz9m1a1cmT57MhRdeyIgRIygtrTmjnTlzJsnJyaSkpDB27Nhzek+s0lCXwGCJ4a0V2RSVVro50bkpLC5n7tJdjPjHEm6et4ote47wp1+dz4qHhjPt18lazNU589pr6A0NgXNGD+5t27Yxb948Bg4cyG233cZLL70EQEhICD/88ANQ0/Nl9uzZdO7cmZUrV3LnnXfy3Xffce+993LHHXcwYcIEZs2aZff558yZQ2ZmJmvXriUgIOBEr5fnn3+eRYsWnXKGvmbNGl599VVWrlyJMYZ+/foxZMgQWrZsyY4dO3jnnXd4+eWXueGGG/jwww8ZP348zzzzDJmZmQQHB3vtZRt7XRyD/JtxfvAUHvlkI09+vpnLu7bhml7tGdIl2iMvUZRWVPP99gIW/Lyb/27eR2W1oVd8JH8bk8Lonu0IDnDsLz2lHOG1Bb2hIXDO6P0cFxfHwIEDARg/fjwzZ84EONGTpbi4mB9//JHrr7/+xNeUl9e0LF22bBkffvghADfffDMPPPDAKc//7bffMnXqVAICat7+xtrr/vDDD/zmN7+hefPmAFx77bUsXbqU0aNHk5SURM+ePQG46KKLyMrKAiAlJYW0tDSuueYarrnmmrN6H6zW0E3Um7rdxPq8Ij5eu5vPfs7niw17CA8JYEiXaC7r2oYhXaItXaGnsLicH3Ye4JtNe1m0tYDSymqimgcxYUAiN/aJo0ubcMuyKd/mUEEXkZHAC4A/MNcY80y9x4OBN4CLgELgRmNMlnOjnsze2Zuzej/Xb317/PPjBdVmsxEZGXlSX/TTfX19Z9pe93T3OYKD//dnur+//4lLLl988QVLlixhwYIFPPXUU2zatOnEfyDexF4XR4AecZH0iItk2q+7snRHAV9v3Mt3W/fz+fo9+Al0bx9BamIUfRKjSE1sSet6lzPqL6Z8tj1ljDHsKSpjw+4iVmUeZNnOA2zdWzPkMjo8mOsuas+obm3plxSlzbOUyzX6Gy4i/sAs4HIgD1gtIguMMZvr7DYJOGSM6SQiY4FngRtdEfi4xobAnYucnByWL1/OgAEDeOeddxg0aBBr16498XiLFi1ISkri/fff5/rrr8cYw/r16+nRowcDBw5k/vz5jB8/nvR0+6NuRowYwezZsxk6dOhJl1zCw8M5evToKZdcLrnkEiZOnMiDDz6IMYaPP/6YN998s8H8NpuN3Nxchg0bxqBBg3j77bcpLi4mMjLynN8bTxPo78elF7Th0gvaYLMZ1u8u4rst+1iReZA3V2Qz74dMAGLCgzn/vHDObxPOnspvmbfpQcqrHRsOWW0zHC6p4FBJBbmHSskpLCGr8Bi7Co6xcXcRhccqgJrukn0SW/KnX53PwE6t6d4+Qmd1Krdy5JStL7DTGLMLQETmA1cDdQv61cDjtR9/APxTRMS4eAhNQ2dv56pr1668/vrr/Pa3v6Vz587ccccdvPjiiyftk56ezh133MHTTz9NZWUlY8eOpUePHrzwwgvcdNNNvPDCC1x33XV2n//2229n+/btpKSkEBgYyOTJk7n77ruZMmUKo0aNom3btixatOjE/r1792bixIn07dv3xNf36tXrxOWV+qqrqxk/fjxFRUUYY/jDH/7gk8W8Pj8/oWdcJD3jao61vKqajbuL+Cn7MFv3HmX7vqO8uSKbX/yfptrv1OGQt310H3O/aY/NGKqqDTZjKKmo5khZJfV/kpsF+pPYujnDLoghJTaCbu0jSG7bgpBAvSaurNPosEURGQOMNMbcXvv5zUA/Y8zddfbZWLtPXu3nv9Tuc6Dec00BpgDEx8dflJ198jVwTxhGl5WVxZVXXsnGjRstzeEOnvB+u1u1zRD4lL/d4ZAg3N1lDf5S85+DvwjNgvyJDA0iKjSQls2DaB/ZjPhWoUSHBTepVamU5zjX9rn2fmrr/zY4sg/GmDnAHKgZh+7AayvlVP5+0uAN9YSIeF4c18uCVEo5hyN3afKAuDqfxwL5De0jIgFABHDQGQHdLTExsUmcnTdlDfWU0cWUlbdzpKCvBjqLSJKIBAFjgQX19lkAHF/IcAzw3dleP9flydyjKb/Pad3TmHPVHBIiEhCEhIgEt8wwVsrVGr3kYoypEpG7gW+oGbb4ijFmk4g8CWQYYxYA84A3RWQnNWfmZzU1MSQkhMLCQlq1aqXXJ13IGENhYSEhIU136TJX3VBXykoe1culsrKSvLw8ysrKGvgq5SwhISHExsYSGBhodRSl1BnwmjVFAwMDSUpKsjqGUkp5JZ26ppRSPkILulJK+Qgt6Eop5SMsuykqIgXAqbM7HNMaONDoXr5Fj7lp0GNuGs7lmBOMMdH2HrCsoJ8LEclo6C6vr9Jjbhr0mJsGVx2zXnJRSikfoQVdKaV8hLcW9DlWB7CAHnPToMfcNLjkmL3yGrpSSqlTeesZulJKqXq0oCullI/w6IIuIiNFZJuI7BSRB+08Hiwi79Y+vlJEEt2f0rkcOOb7RGSziKwXkYUikmBFTmdq7Jjr7DdGRIyIeP0QN0eOWURuqP1ebxKRt92d0dkc+NmOF5FFIrK29uf7CityOouIvCIi+2tXdLP3uIjIzNr3Y72I9D7nFzXGeOQ/alr1/gJ0AIKAn4HkevvcCcyu/Xgs8K7Vud1wzMOA0NqP72gKx1y7XziwBFgBpFqd2w3f587AWqBl7ecxVud2wzHPAe6o/TgZyLI69zke8yVAb2BjA49fAXxFzYpv/YGV5/qannyGfmJxamNMBXB8ceq6rgZer/34A2C4eHcj9UaP2RizyBhTUvvpCmpWkPJmjnyfAZ4CngN8obeyI8c8GZhljDkEYIzZ7+aMzubIMRugRe3HEZy6MppXMcYs4fQrt10NvGFqrAAiRaTtubymJxf09kBunc/zarfZ3ccYUwUUAa3cks41HDnmuiZR8z+8N2v0mEWkFxBnjPncncFcyJHvcxegi4gsE5EVIjLSbelcw5FjfhwYLyJ5wJfA79wTzTJn+vveKI/qh16P0xan9iIOH4+IjAdSgSEuTeR6pz1mEfED/gFMdFcgN3Dk+xxAzWWXodT8FbZURLoZYw67OJurOHLM44DXjDH/JyIDqFkFrZsxxub6eJZwev3y5DP0JrU4dS1HjhkRuQyYBow2xpS7KZurNHbM4UA3YLGIZFFzrXGBl98YdfRn+1NjTKUxJhPYRk2B91aOHPMk4D0AY8xyIISaJla+yqHf9zPhyQXdrYtTe4hGj7n28sO/qSnm3n5dFRo5ZmNMkTGmtTEm0RiTSM19g9HGmAz7T+cVHPnZ/oSaG+CISGtqLsHscmtK53LkmHOA4QAi0pWagl7g1pTutQCYUDvapT9QZIzZc07PaPWd4EbuEl8BbKfm7vi02m1PUvMLDTXf8PeBncAqoIPVmd1wzN8C+4B1tf8WWJ3Z1cdcb9/FePkoFwe/zwI8D2wGNgBjrc7shmNOBpZRMwJmHTDC6szneLzvAHuASmrOxicBU4Gpdb7Hs2rfjw3O+LnWqf9KKeUjPPmSi1JKqTOgBV0ppXyEFnSllPIRWtCVUspHaEFXSikfoQVdKaV8hBZ0pZTyEf8fDs1OK9raMScAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"y_result = model.predict(x_test)\n",
"plt.plot(x,y, label='exact')\n",
"plt.plot(x_test, y_result, 'go', label='predictions')\n",
"plt.legend()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Derivatives\n",
"Extracting what the model thinks dy/dx is at a given point"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# compute the gradient of the output wrt the input\n",
"get_grads = K.gradients(model.output, model.input)[-1]\n",
"\n",
"# this function returns the grads given the input \n",
"function_grad_x = K.function([model.input], get_grads)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x1ee96a5dd48>"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEICAYAAABCnX+uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deViVdfr48fcHOIpwEFkFF8JwSwUNl9LMpnJcWmyxmaZst2gb69s0ozU67TWjTZvl71t8rWZKKys1d3Mps9xyQzBFBHNBWWQRPOxwPr8/zoGQ9QAHzjlwv66L6+I86/0c4OY59/NZlNYaIYQQrsvN0QEIIYRoGUnkQgjh4iSRCyGEi5NELoQQLk4SuRBCuDhJ5EII4eIkkQuXpZRyV0qZlFJh9txWCFcjiVy0GWsirfwyK6WKqr2e1tTjaa0rtNZGrfVJe27bVEqpV5RSZUqp89avI0qp+UqpkCYc4yel1H32jk10DJLIRZuxJlKj1toInARurLZscc3tlVIebR9lsy3WWvsAAcBUoDewRynV3bFhiY5AErlwGtY72yVKqc+VUueBu5RSo5VSO5VS55RSadY7XYN1ew+llFZKhVtfL7KuX2e9M96hlOrT1G2t6ycrpZKUUnlKqXeVUttsuWPWWpdqrQ8CfwDOAU9ZjxeglFqrlDqrlMpVSq1SSvW0rpsLjAbet346edu6/D2lVKpSKl8ptVspNcYub7RodySRC2dzC/AZ4AssAcqBJ4FA4ApgEvBwA/vfCfwD8Mdy1/9yU7dVSgUDXwJ/s573V2BUUy5Ca10OrASutC5yA/4PCAMuAsqAd6zbzgJ2AI9YP538j3WfXUCUNb6vga+UUp2bEofoGCSRC2fzk9Z6ldbarLUu0lrv1lrv0lqXa62PAbHAVQ3s/7XWeo/WugxYDAxrxrY3AHFa6xXWdW8BWc24ljNYkjBa67Na6+XWa8oHXmvkOtBaf6q1zrH+U5gHdAX6NiMO0c65Ug1SdAynqr9QSg0E3gCGA15Yfmd3NbB/erXvCwFjM7btUT0OrbVWSqU2GnltPYEcAKWUN5Y78AlAN+t6n4Z2VkrNBB4AQgENeGP5hCDEBeSOXDibmsNxfgAcBPpqrbsCzwGqlWNIA3pVvlBKKSxJ2WZKKXfgRuBH66KZQB9glPU6rqmxywXXrZS6GvgLlgen3QA/wETrX7twQZLIhbPzAfKAAqXUJTRcH7eX1UC0UupGa8uZJ4EgW3ZUShmUUoOAL7CUVd62rvLBctefq5QKwPIPqboM4OJqr32wPB/IAgzAC1juyIWoRRK5cHZPA/cC57HcnS9p7RNqrTOA24E3gWwgAtgPlDSw2zRrS5tcYAWWxDxCa11ZvnkTywPcbGA7sK7G/m8Dd1hb57wJrAU2AUeB40A+lk8KQtSiZGIJIRpmLZOcAW7TWv/Y2PZCtDW5IxeiDkqpSUopX2tzv39gKXP87OCwhKiTJHIh6jYWOIalRj0JuFlr3VBpRQiHkdKKEEK4OLkjF0IIF+eQDkGBgYE6PDzcEacWQgiXtXfv3iytda2msA5J5OHh4ezZs8cRpxZCCJellDpR13IprQghhIuTRC6EEC5OErkQQrg4pxn9sKysjNTUVIqLix0dSofi6elJr169MBgMjg5FCNFMTpPIU1NT8fHxITw8HMtgc6K1aa3Jzs4mNTWVPn36NL6DEMIp2a20Yp2lfL9SanVz9i8uLiYgIECSeBtSShEQECCfgoRwcfaskT8JHG7JASSJtz15z4VwfXZJ5EqpXsD1wEJ7HE8IIdqb3IJSXlz1C/nFZXY/tr3uyN/GMgOKub4NlFIxSqk9Sqk9Z8+etdNpW88LL7zAv//970a3+/zzz3n11VdrLQ8PDycrqznTPNrfli1b2L59u6PDEKJD0lqzJj6N37/1A5/uOMHPx3Lsfo4WJ3Kl1A1AptZ6b0Pbaa1jtdYjtNYjgoJsmmzFJaxfv55JkyY5OowGSSIXwjEy84t5+NO9PP7ZPkJ9u7BqxljGD+pu9/PY4478CmCKUuo4lumtrlFKLbLDcdvcq6++yoABAxg/fjxHjhwBICUlhejo6Kptjh49yvDhwwHLf9q4uDiio6PJzs5mwoQJXHrppTz88MNUjiq5e/duoqKiKC4upqCggMGDB3Pw4MFa5160aBGjRo1i2LBhPPzww1RUVHDixAn69etHVlYWZrOZK6+8kg0bNgBw8803M3z4cAYPHkxsbGzVcdavX090dDRDhw7l2muv5fjx47z//vu89dZbDBs2jB9/lHkRhGhtWmu+3H2Ka9/8gR+SzvLs5IEsf2wMl4R2bZXztbj5odb6WeBZAKXU74C/aq3vaskxX1z1C4fO5Lc0tAsM6tGV528cXO/6vXv38sUXX7B//37Ky8uJjo5m+PDhRERE4OvrS1xcHMOGDePjjz/mvvvuA2D//v0MHToUpRQvvvgiY8eO5bnnnmPNmjVVyXXkyJFMmTKFOXPmUFRUxF133cWQIUMuOPfhw4dZsmQJ27Ztw2Aw8Nhjj7F48WLuueceZs2axSOPPMJll13GoEGDmDBhAgAfffQR/v7+FBUVMXLkSKZOnYrZbOahhx5i69at9OnTh5ycHPz9/XnkkUcwGo389a9/tet7KoSo7WR2Ic8uj2dbcjaj+vgzd2oUfQJbd7pVp2lH7mg//vgjt9xyC15eXgBMmTKlat2DDz7Ixx9/zJtvvsmSJUv4+WfLRDHr169n8uTJAGzdupVly5YBcP311+Pn51e1/3PPPcfIkSPx9PRk/vz5tc69efNm9u7dy8iRIwEoKioiODi46txfffUV77//PnFxcVX7zJ8/n+XLlwNw6tQpjh49ytmzZxk3blxVm3B/f3/7vDlCiEZVmDX/2X6cf397BHc3xSs3D+HOUWG4ubV+yzC7JnKt9RZgS0uP09Cdc2uqryne1KlTefHFF7nmmmsYPnw4AQEBAGzYsIGlS5c2un9OTg4mk4mysjKKi4vx9r7wv7PWmnvvvZd//vOftfYtLCwkNTUVAJPJhI+PD1u2bGHTpk3s2LEDLy8vfve731FcXIzWWpoTCuEARzPOM3NpPPtPnuPqAUG8ekskPbp1abPzy1grVuPGjWP58uUUFRVx/vx5Vq1aVbXO09OTiRMn8uijj3L//fcDkJeXR3l5eVVSHzduHIsXLwZg3bp15ObmVu0fExPDyy+/zLRp05g1a1atc1977bV8/fXXZGZmApbEf+KEZbTKWbNmMW3aNF566SUeeuihqnP7+fnh5eVFYmIiO3fuBGD06NH88MMP/Prrr1XHAfDx8eH8+fP2e7OEEACUlpuZv/ko18//ieNZBbx9+zA+um9kmyZxkNJKlejoaG6//XaGDRvGRRddxJVXXnnB+mnTprFs2bKqGvXGjRsZP3581frnn3+eO+64g+joaK666irCwsIA+OSTT/Dw8ODOO++koqKCMWPG8N1333HNNddU7Tto0CBeeeUVJkyYgNlsxmAwsGDBAo4fP87u3bvZtm0b7u7uLF26lI8//pg777yT999/n6ioKAYMGMDll18OQFBQELGxsdx6662YzWaCg4PZuHEjN954I7fddhsrVqzg3XffrXVtQoimi089x8yv40lMP8+NQ3vw/I2DCDR2dkgsDpmzc8SIEbrmxBKHDx/mkksuafNYbPXvf/+bvLw8Xn75ZcBSu37wwQerkqgrc/b3XghnUlRawdubkvi/H48R5NOZV26O5Pet0KSwLkqpvVrrETWXyx25DW655RZSUlL47rvvqpYtXCidWIXoaHYey+aZpfEczy7kjlG9efa6S+jq6fiRQyWR26CydYgQomM6X1zGv9YlsnjXScL8vfjswcsY0zfQ0WFVkUQuhOiw0k3pLNy7kMSsRIrKisgrycO3sy8GNwNl5jLySvIoKfHkYFoeBeX5RIQEExTixxt7Knhum2VboM796lrn1cmLyf0mM7nfZIydjHa7DknkQogOKd2UTugboY1vqAEFeEBinuWrJT4+8DGhxlCSZiTZLZlL80MhRIe08shK2zZsha4ZWYVZJGQk2O14ksiFEB3SlAFT6l/Zyo35Ar0CieweabfjSWmlHi+88IJN45N8/vnnHDt2jNmzZ1+wPDw8nD179hAYaNsDEaPRiMlkAmDt2rU8+eSTbN68uao9uhDCvkKMIaQ9nWapkZ9NJCkjm4OZaSizF4NC/Ont35n80qbXwaVG7oLWr1/PE088Ybfjbd68mRkzZrBhwwabk3h5eTkeHvKjFKKpQowh3D3kKZ5ZmkDmqWxuujiAf02N5KKA1h3kyt5curRiKjWx49QOTKUmuxzPkcPYgmXgroceeog1a9YQEREBwNmzZ5k6dSojR45k5MiRbNu2DbB8YoiJiWHChAncc889HD9+nCuvvJLo6Giio6Orxh9PS0tj3LhxDBs2jCFDhsgwtkJYVZg1C388xsS3t3LwdB7/vDWSzx66zOWSOLjwHbmp1MSgBYPILc7Fz9OPQ48fatFHFUcOYwtQUlLCTTfdxJYtWxg4cGDV8ieffJKnnnqKsWPHcvLkSSZOnMjhw4erYv7pp5/o0qULhYWFbNy4EU9PT44ePcodd9zBnj17+Oyzz5g4cSKzZ8+moqKCwsLCZr9HQriilJwU5m6bS35RflXZww0vElJNnCsyMyjwEj7+4yyierhuGdNlE3lCRgK5xblVd+MJGQmM7j262cdz5DC2AAaDgTFjxvDhhx/yzjvvVC3ftGkThw4dqnqdn59fNQDWlClT6NLFMjhPWVkZf/7zn4mLi8Pd3Z2kpCTA8o/kgQceoKysjJtvvplhw4Y1+z0SwtWk5KTQ992+tVdUNik0wL68H7h0YSyn/3KaEGNIW4doFy5bWonsHomfpx/GTkb8PP3s8gS4oWFs161bx+rVq2sNY1s5iFZD+1cOY3v+/HmKi4vr3MbNzY0vv/yS3bt389prr1UtN5vN7Nixg7i4OOLi4jh9+jQ+Pj4AFwyH+9Zbb9G9e3cOHDjAnj17KC0tBSyjMm7dupWePXty991388knnzThHRHCtcXui617RY0/VbM2294c0Qm5bCI3djJy6PFDbLhrQ4vLKuDYYWwreXl5sXr1ahYvXsyHH34IwIQJE3jvvfeqtqk+uUR1eXl5hIaG4ubmxqeffkpFRQUAJ06cIDg4mIceeojp06ezb9++5rw9Qji9lJwUYlbFMPWLqYz/73imfjGVuDPV/l4aaFLoptwabo7o5FpcWlFKeQJbgc7W432ttX6+pce1hbGTsUXllOocOYxtdf7+/qxfv55x48YRGBjI/Pnzefzxx4mKiqK8vJxx48bx/vvv19rvscceY+rUqXz11VdcffXVVXfrW7Zs4fXXX8dgMGA0GuWOXLRL9ZZQKmkI9x5NVI/uFJSfr2oOaHA3MDBoINOjp7tsWQXsMIytstQTvLXWJqWUAfgJeFJrvbO+fWQYW+fi7O+9EI2ZtWkW87bNa3CbaZHTWHSrS84LX6XVhrHVlv8Ele3/DNavth/kvBXJMLZCOLeY6JjaibzygabVzDEz2zSmtmSXVitKKXdgL9AXWKC13lXHNjFADOByvRVlGFshnFuEfwQ/P/AL93w1h2M56XgbSogO74mvp4EKXcFLV79EVEiUo8NsNXZJ5FrrCmCYUqobsFwpNURrfbDGNrFALFhKK/UcRyYPbmOOmCFKCHvSWrMi7gwvrkqlouQh/n1NXx6+KoJOHi7blqPJ7NqOXGt9Tim1BZgE1N19sR6enp5kZ2cTEBAgybyNaK3Jzs7G09PT0aEI0SxnzhUx55uDfJeYyaVh3Zg3NYp+3X0cHVabs0erlSCgzJrEuwDjgblNPU6vXr1ITU3l7NmzLQ1JNIGnpye9evVydBhC1FJXj8zK1ialFWUcPptOajZos6Z3D/ANHYibYTbQ8RK5PVqtRAH/BdyxtEv/Umv9UkP71NVqRQghKjXanBBqPcyslDwjmQj/iFaJy9Hqa7XS4iKS1jpea32p1jpKaz2ksSQuhBCNqbdHZnX1VGBt2red6ThPA4QQLiMmOqbuFTYUEOrdtx1z2UGzhBDtV4R/BMkzknlt67/Yc+o0RzPPoTyKGBrSgzA/H8p07ckbwnzDmD1udrstqzREErkQwimdO+/PiWN3kZdp4pHonvzj+kH4eXdydFhOSRK5EMKpFJaW8/q3R/jP9uOEdvXk4/tHcvWAYEeH5dQkkQshHCbdlG6ZMzMrkaKyIo5nZ/JrFhSWwYDQQB4ddzsj+3g5OkynJ4lcCOEQ6aZ0Qt8IvXBhtQkf4nPhnhXfELo5lKQZSXadrLi9kVYrQgiHqHMihzqaFGYVZpGQkdD6AbkwuSMXQrSpyh6bJ8+lWRbU07GnUqBXoF1mAGvPJJELIewqPj2eOd/PwaAMGNwMF3SvN5Wa2PDrht82rpbEr+t7HSXlJVXd8L06eTG532Qm95ssZZVGSCIXQthNfHo8Qz8YavsO1e7Ebxp4EzHDO15nHnuQGrkQwm7mbW94lp76uPqcmY4md+RCCLuZOWYmixMW172yRi184sUTCfIKahdzZjqaJHIhhN1EhURx4OEDzPl+Du54cDK7hMMZuSiPQoYE96CXXxcqaP8z9rQ1SeRCCLuKConitXGfMmtpPNln8rl3SAgv3jSYYB+ZwKS1SCIXQthNcVkF7353lPd/OIafVyf+d1o0kyNDG99RtIgkciGEXew5nsPMpfEcO1vAbcN7Mef6S+jmJYNctQV7TPXWG/gECAHMQKzW+p2WHlcI4RpMJeW8vj6RT3aeoIdvFz55YBTj+gc5OqwOxR535OXA01rrfUopH2CvUmqj1vqQHY4thHBiPySd5e/LEjiTV8S9o8P528QBeHeWD/ptrcXvuNY6DUizfn9eKXUY6AlIIheinTpXWMrLqw+zdF8qEUHefPXwaEaE+zs6rA7Lrv86lVLhwKXArjrWxQAxAGFhYfY8rRCiFVTvag9UdbPPzC8jPi2X4vJ8+vXojk+QkWd+yMd3p6VrvW8XX2ZeMbNDztTjKEprGybBs+VAShmBH4BXtdbLGtp2xIgRes+ePXY5rxDC/hrsat/IIFeV2vNs9o6ilNqrtR5Rc7lduugrpQzAUmBxY0lcCOH8Guxqb0MSh445m72jtDiRK6UU8CFwWGv9ZstDEkI42swxMy9c0IwP7h1xNntHsUeN/ArgbiBBKRVnXfZ3rfVaOxxbCOEAg4MjeWHUWubteAk33BgQ3BVfnzK6WYeYrT40LfxWP5cauWPYo9XKT9j8YUsI4eySM88za2kCe0+Yua3/O7x2ayQ9u3VxdFiiAdLgU4gOIt2UzoJdC9idthtvD+9ad9ZmrTmUkUZqtsLDw4NR/Xvzj5telCTuAiSRC9EB1DnRcV00VVlh0yno995n0vrEBcjEEkJ0AHVOdFyXOoqk0vrE+UkiF6IDaHD2nUZapEjrE+cnpRUhOoAQYwhpT6exYNcCdqTu5FQ2nMgpxtsA4UEQHhAMSOsTVyWJXIgOIsQYwjW9ZrD5599Rml/M38f04a8T++PVSdKAq5OfoBAdQG5BKS+vPsSy/afpF2xk6aNjiA7zc3RYwk4kkQvRjmmtWZOQxvMrfiGvqIwnrunL49f0pbOHu6NDE3YkiVwIF2cqNZGQkUBk90iMnYxVyzPyi5nzzUE2Hsogqpcvix68jEtCuzowUtFaJJEL4cJMpSYGLRhEbnEufp5+HHr8EN4Gb77cc4pX1hymtNzM368byANX9MHDXRqptVeSyIVwYQkZCWQXZlNYXoip1MT4/0zm1Fl3MotzCfL0Y3jfAL49A18uyiPUGMqUgVOY3G/yBXfuwvVJIhfChQV7B1NYXlj1eteZnyzfeEBaOaxOuXD7RQcXEWoMJWlGkiTzdkQ+awnhwjb/uvnCBYpGh7DLKswiISOh1WISbU8SuRAuqrTcTEbGkCaPFR7oFUhk98jWCUo4hJRWhHAhplIT65LW8fH+r9h5LIXCss7073YdY/v25rTpON4e3sCFPTQrRziUGnn7ZZdErpT6CLgByNRaD7HHMYUQFzKVmoh4J4LMwszf5s00QFIBJB2QOTI7MnuVVv4DTLLTsYQQdfhs309kFmZbXsgohaIau9yRa623KqXC7XEsIYRFuimdhXsXkpB+iP2nMzmWmw4e9T/JlFEKO642q5ErpWKAGICwsLC2Oq0QLifdlM5b299i3o5qM9lXm/Ch0theYwn2DibQO1BGKezg2iyRa61jgViAESNGNGNObiHav3pn8qnjRvzuYXcTM1zuwoU0PxTCqaxMtG0mHzfl1vBkEaJDkeaHQrSBymaDKxJXkF6QXqtpoG9nX4rKy9n964nfWqRUc13f6ygpLyGwSyCRIZFMj55OiDHEIdcinI+9mh9+DvwOCFRKpQLPa60/tMexhXB1plITfef3JaMgo+ENKwuO1iTeWXVm5tiZPDbqMUnaokH2arVyhz2OI0R7lJCRQHZRduMb1rgLNyszk/tNliQuGiU1ciFaWWT3SAK6BNS9UlNvF3vpSi9sJTVyIVqZsZOR5CeSq2rkKTmnOJapySvWdDd6EBYEwd7+ANKVXjSLJHIh2oCxk5EpA2/lVNpQdu5LplcXAwv/MJgbokJRqpHhCoVohCRyIdrA/pO5zFoaT1KGiVsu7ck/bhiEv3cnR4cl2glJ5EK0osLSct7YkMRH234lpKsnH903gmsGdnd0WKKdkUQuRCvZnpzFM8sSOJlTyF2XhzFr0kB8PA2ODku0Q5LIhWimykGtErMSKSorqurYU1ZhJu70GTLz3TEaOjGkn5E958t5ffsoaRMuWoXSuu2HPRkxYoTes2dPm59XCHupd0yUSnX0zgRL1/rTfzktyVw0i1Jqr9Z6RM3l0o5ciGZYeaSRMVHqaYhi1ubG9xWiiSSRC9EMdQ5YZcOHWxnsSrQGqZEL0QwhxhD2P3iM+7+cS2J2Ir6eZvoEu9Gjq6UHZ805MwvKChjVU2rkonVIIheiicxmzeKfTzJ3XRIV5ptZMGkA944Jx91NOvYIx5BELkQ94tPjmfP9HAzKUHVnnZafzfGzkF1Uir+XYuZVD/DHkWMliQuHklYrQtQhPj2eoR8Mrb2ixlCzAN29u5P8RLKMiyJanbRaEaIJ5m2fV/cKRa0WKdlF2SRkJLR6TELURxK5EHWYOWbmby8a+dAa0CVAhpsVDmWvGYImAe8A7sBCrfW/7HFcIRwlz+THwE7P8mvhRnr5diWypz9uylzVGgWgpKKE2wffzi2DbpGyinCoFidypZQ7sAD4PZAK7FZKrdRaH2rpsYVoawUl5by6bj/z4q5DqwICuvoR98T3kqiFU7PHHfkoIFlrfQxAKfUFcBMgiVw4pZScFOZum0t+Uf4Fkx/nFFRwIDWb/PI0zB65aCooLHMjISOB0b1HOzpsIeplj0TeEzhV7XUqcFnNjZRSMUAMQFhYmB1OK0TTpeSk0PfdvnWvrBwfpdpfRUFZAcHewW0RmhDNZo+HnXU1oK31eEhrHau1HqG1HhEUFGSH0wrRdLH7YutfWU9T8M2/bm6dYISwE3sk8lSgd7XXvYAzdjiuEHYXEx1z4YJGWqTI2CjCFdijtLIb6KeU6gOcBv4E3GmH4wphF9V7aKJhsP8YTuSAudyNfsFe+PmU4+/Zrar3Zl5JHoFdAokMiWR69HQZG0U4vRYncq11uVLqz8C3WJoffqS1/qXFkQlhB3X20LTWwlfdsYMbBl3ukLiEsCe7dAjSWq/VWvfXWkdorV+1xzGFsIc6e2haa+FfJL7XtsEI0Upk0CzRbphKTaxLWseKxBWkF6Tj29mXDNM5y8o6Zuy5oPemEC5MErloF0ylJvrO70tGQcaFKyoTuIKJF0/E2+BNha7gpatfIiokyhGhCmF3ksiFQ1R2yikqLWJg0MAWP1RMyEgguyi79opqd+FDQ4cyd/zcZp9DCGcliVy0ubo65Ty35bkWTUoc2T0S/y7+ZBZk1jvxca2mh0K0E5LIRZurq1NO5aTEMcObl2wPnyllgP4PFSU/EBQUT1DXIoK8/TG4GfDt4svMK2YS4R/R0tCFcEqSyEWbi4mOYd62C1uTNLfjjamknHnrE/lkxwl6+XVh2X1PM66/9BwWHYskctHmIvwjSJ6R3OIa+Q9JZ/n7sgTO5BVx35hw/jZxAN6d5VdadDwy1ZtwCummdBbuXVj10LKhnpWp53KY/sUCfji9EqOHJ5deFEBXT3eZqV60e/VN9SaJXDhUuimdt7a/xbwddU+t5qbcqh6Caq1ZFpfCH1aOQuvcege5qr6PEO2JzNkpnE66KZ3QN0LrTeLw20PQzPxiHlm0lz9/9Q2avHqTePV9hOgopKAoHMbWZPu/Oz7nb6sWUVZRTs+gCgz57pRhrnd7GbFQdDRSWhEOU3lHXp0BA5f1uoxdqbsoo8yysJ524QDX9b0OH4MPZeYyqZGLdq++0orckQuHCTGGkPZ0Ggv3LuRk/kl+f/HvmdxvMnHp8Yz7eNxvGzZQRrlp4E3NbnsuRHshiVw4VIgxhDlXzal6nZx5nnmrCtBmH3A71+C+UkIRwkISuXAKZRVm3t+SwrvfJePV2Z3FN+yk3GMXSw4twdPdUyZ9EKIBksiFwyWk5vG3rw+QmH6e66NCeXHKYAKNnYEB3HPpPY4OTwin16JErpT6A/ACcAkwSmstTzCFzYrLKnhrUxL/t/UYgcbOfHD3cCYOljtsIZqqpXfkB4FbgQ/sEIvoQHYdy+aZZQn8mlXAn0b25tnrLsG3i8HRYQnhklqUyLXWhwGUaqBZgRDVnC8uY+76RBbtPElv/y4sfvAyrugb6OiwhHBpbVYjV0rFADEAYWFhbXVa0UyVY5+kF6Tz1OVP2WUI2O8TM/n78gTS84uZPrYPT0/oj1cneUwjREs1+leklNoE1FW4nK21XmHribTWsUAsWDoE2RyhaHM1O+os2L2A5BnJzU7mOQWlvLTqF76JO0O/YCNLHx1DdJifvcIVosNrNJFrrce3RSDCedTVdT52X2yTp0nTWrM6Po0XVv5CXlEZT1zbj8evjqCzh7u9QhVCIINmiTrU1cmmqdOkZcBbdMgAABEuSURBVOQX89Ane5nx+X56+nVh1Yyx/OX3/SWJC9EKWtr88BbgXSAIWKOUitNaT7RLZMJhqnedb2qNXGvNkt2neHXtYUrLzcy+7hLuvyIcD3e5ZxCitcigWaLJUnJSeG/3e4zpNYbJ/SZj7GQE4GR2Ic8si2d7SjaX9fFn7tQowgO9HRytEO2HDJol7CIlJ4W+7/YF4G3exoCBG/rfyPHsAg5lpGFwMzKwd1eyPMt4epPvBV3rfTv/9rpCV/DS1S8RFRLl4CsSwvVJIhdNErsv9oLXZZSxPGmZZahZDygB9mQBWY0fa0XSCg48fECSuRAtJIVL0ST1PvRsZp+wedvrnx1ICGEbSeSiSSL8I4id+I3lhR0er8wcM7PlBxGig5PSirBZUWkFb248woc/eRDl9Qk9L/qWrp7lABfUwYFaNXGpkQvReiSRi3qZSk2sS1rH2qNrOZWby/5T2eSX53NJ9z7894//YliPux0dohACSeSiHqZSE33n9yWjIMOyoHLeTA9IyIvn0v9b0aJu+0II+5EaeTuWkpPCrE2zSMlJafK+CRkJZBVm/7agjoeZNVuwCCEcQxJ5O1XZ3nvetnn0fbdvk5J5tqmE/241U1Hh0+B2Te22L4RoHS5VWjGVmkjISCCye2RVb0JRt5p3y7YMeqW1ZuWBM7yw8hdMJeXMHfcdvUKPsDFlPUVlRVUPLcN8w5g9braUVYRwEi7TRd9UamLQgkHkFufi5+nHoccPSTJvQPUemECj9ey0vCLmLD/I5sRMhvXuxrzboujfveE7ciFE23L5LvoJGQnkFudiKjVVvR7de7SDo3JeEf4RJM9IJnZfLDHRMfUmcbNZ8/nuk/xzbSLlZjNzrr+E+6/og7ubzPokhKtwmUQe2T0SP0/LZAR+nn5Edo90cETOL8I/gpjoGF7Z+gqn8k7VatdtUEYOnDzH2ZJcgj39iA4PYN1pWL1E2ngL4UpcprQCUiNvqprllVoqf/T13HzLOChCOJf6Sisu1WrF2MnI6N6jJYnbqNHmgYoGx0iRcVCEcA0tSuRKqdeVUolKqXil1HKlVDd7BSZars7mgU34ACbjoAjhGlpaI98IPKu1LldKzQWeBWa1PCxhD5UPPJ9a9w9+Sj5CYVlnwvy6MiCkC0Xl5+sdF0XGQRHCtbQokWutN1R7uRO4rWXhCHsqLC3ns+0lJBycxoCunrx2eyRXDwx2dFhCCDuzZ6uVB4Al9a1USsUAMQBhYWF2PK2oy7bkLJ5ZFs+pnCLuujyMWZMG4uNpsGnf+qZyE0I4p0ZbrSilNgEhdayarbVeYd1mNjACuFXb0AxG5uxsPXlFZby25jBL9pyiT6A3/7o1kssuDrB5/5otXUKNoSTNSJJkLoQTaHaHIK31+EYOfC9wA3CtLUlcNE31oWTzS/LrHPM70DuQmVfMJCXNmznfHCS7oJRHrorgf8b3w9Pg3qTz1WzpklWYJZ2vhHByLSqtKKUmYXm4eZXWutA+IYlKtYaSbUDsvlhCi2IZGjqQD+8dSWQv32adMyY6hnnbfmt2GOgVKJ2vhHByLa2Rvwd0BjYqpQB2aq0faXFUArAMQ5BdlN34hgAa/Ht8QXCvMN7b34mZXjObNahVZUsXqZEL4TpcqmdnR2PTHXnlhA91kIkfhGhfXH7QLOh4XfSP5RxjVM9RGJShqo33uZI88k0GDmfkUaELGNS9Bz0DOrP26GoqdMUF+9sydK0QwvW5TCLvaMPYxqfHM/SDoVWv3ZQb2+9N4o11maScyuHmvoH889ZIevt7EZ8ez8bkb2sl8r2n9xKfHi8de4Ro51xmrJXqw9jmFueSkJHg6JBaVc1xTszazI0L3yExPZ95t0Xx6fRR9Pb3IiUnhaEfDKXYXFzrGJuPb2boB0OJT49vq7CFEA7gMom8chhbYydjhxjGttY4J1ox8eLr2PSXq/jjiN5YHy7bNG+mDH4lRPvmMonc2MnIoccPseGuDe2+rALQP2AwTw35hi7lowlhGouu38+n908iuKvnBdvZMm+mDH4lRPsmrVac0N4TOcz8Op6UswVMje7FP264hG5enerdPiUnhbnb5qJQzLzCkrRf2foKuUW5MviVEO1Ifa1WJJE7kYKScl7/9gj/3XGcHr5deO3WSK7qH+TosIQQTqJdND9sz7YmneXZZQmcySvinssv4m+TBmLsLD8eIUTjJFM42OlzuTzy1UK+O/EtXQwVRPbzZtu5Yg4u/20MFenUI4RoiCRyB1oel8xtKy7DrHPAAIXAllMXbhO7L1Z6aAohGuQyrVbak4Npxxk1/1H+tPQPmMlpcN5MsK2JoRCi45I7cjtKN6WzcO9CErMSKSorqnPI2RNZ59mbvdEyRoqN774tTQyFEB2XJPJmSMlJ4c2db9LD2IPp0dMJMYaQbkon9I3QxneuHOSqjrvw/v79GRI0pNY441JWEUI0RBJ5E9WcQee5Lc9x+i+nWXlkpW0HaKCMsnbaWknaQogmkxp5E9WsV5u1mZVHVjJlwJT6d2qgqf6U/lOIiY6RB5pCiGaTO/ImqjmDjptyY8qAKYQYQ0h7Oo2FexdyKDORQ+lnScxMx+BhZFCwLz7epfh29sXgZsDgbmBg0MCqsowQQrRES6d6exm4CTADmcB9Wusz9gjMWVXOoFOzRg4QYgzh5r4z2B4Xz7m0fB6IDOHFKUMI8uns4KiFEO1Zi7roK6W6aq3zrd8/AQyyZaq39thFv7isgnc2HyV26zH8vTvx8k2DmTTEhoefQghho1bpol+ZxK28abAa3H7tPp7DrK/jOZZVwB+G92LO9YPw9TI4OiwhRAfR4hq5UupV4B4gD7i6ge1igBiAsLCwlp7WKZhKypm3PpFPdpygl18XPp0+iiv7ySBXQoi21WhpRSm1CajridxsrfWKats9C3hqrZ9v7KTtobSy5Ugms5cf5ExeEfeNCeevEwbgLYNcCSFaUbNLK1rr8Tae4zNgDdBoIndluQWlvLzmEF/vS8bbdx89+uwmvqAr9yyHkooSbh98O7cMuqXdT3whhHAeLW210k9rfdT6cgqQ2PKQnJPWmrUJ6Ty/8iA5hfnkej/MqdIcSMPyZbUmeQ3dN3Un+YlkSeZCiDbR0g5B/1JKHVRKxQMTgCftEFOrSTelM3/XfFYnrcZUarJ5v8z8Yh7+dC+Pf7aPUN8uvPIHX4p0Xr3bZxdlt/vJoYUQzqOlrVam2iuQ1pZuSqfnGz0xYwagh08Pjvz5SIN3zVprvtqTystrDlFabuaZyQN5cGwfiisK8d/kz9nCs3XuF9AloN1PDi2EcB4d5uncyiMrq5I4wJnzZ7hh8Q1c7Hcxk/tNZnK/yWSYMnhl6yucyjtFJzcjCakmMgsK6dq5jOEX92BjmgdfL8ojzDeMTXdv4kD6Ab745Qs83S0TIkuNXAjhCB1mzs7GRicM7BJIVlHWbwsq35YGBrmS8VGEEG2pvlYrHWbQrBBjCK///vV612cXZV+4oJ6hZquTCR+EEM6gwyRygLui7qp3nZebX5P7pcqED0IIZ9BhauRA1QiFC3YtYHfabrw9vMkrKmPfydMUlnQmwtfIwB7eFJefrxqpsMxcVmumnzDfMGaPmy1lFSGEU+gwNfKaissqmLNyI28kTK5VQpHatxDCGXX4Gnl1O49lM+ntrcTuX1hnHVxq30IIV9KhEvn54jJmL0/gT7E7MWtYeOusOreT2rcQwpV0mBr5d4kZzF5+kIz8Yh4c24e/TOiPVycPhocnM3fbXPKL8vHt4iuTHQshXE67T+Q5BaW8tOoXvok7Q79gI//v0TFcGuZXtT7CP4LYG6WUIoRwXe02kWutWRWfxgsrfyG/qIwnr+3HY1dH0NnD3dGhCSGEXbWbRB6fHs+c7+dgUAbKzYq4k9mcKcjBv7MvQwd0Y82ZQvatCGXKwClM7jdZutALIdqNdpHI49PjGfrB0N8WVLao9IDMCth4/LdViw4uItQYStKMJEnmQoh2oV20Wpm3fd6FCxrpXp9VmCXDzAoh2g2XuyNPyUmpamVSZi4jtySP07mFNg1yVSnQK1CGmRVCtBsulchTclLo+27f2is0VQl84sUT6da5W51d60ONUiMXQrQ/dknkSqm/Aq8DQVrrrMa2b656e1xWuwsfGjqUuePntlYIQgjhdFpcI1dK9QZ+D5xseTgNu6DHZT1DxEivTCFER2OPO/K3gJnACjscq0Fr95vpWRxLsec3DOllIMDoXlU+CfQOlF6ZQogOqUWJXCk1BTittT6gVMNPGZVSMUAMQFhYWLPOFxbgxT2jLueZyffR1dPQrGMIIUR70+gwtkqpTUBIHatmA38HJmit85RSx4ERttTInWEYWyGEcDX1DWPb6B251np8PQeMBPoAlXfjvYB9SqlRWuv0FsYrhBDCRs0urWitE4DgytdNuSMXQghhP+2iZ6cQQnRkdusQpLUOt9exhBBC2E7uyIUQwsVJIhdCCBcniVwIIVycJHIhhHBxjXYIapWTKnUWONHM3QOBjtbEUa65Y5Br7hhacs0Xaa2Dai50SCJvCaXUnrp6NrVncs0dg1xzx9Aa1yylFSGEcHGSyIUQwsW5YiKvZ3aJdk2uuWOQa+4Y7H7NLlcjF0IIcSFXvCMXQghRjSRyIYRwcU6byJVSk5RSR5RSyUqpZ+pY31kptcS6fpdSKrzto7QvG675L0qpQ0qpeKXUZqXURY6I054au+Zq292mlNJKKZduqmbL9Sql/mj9Of+ilPqsrWO0Nxt+r8OUUt8rpfZbf7evc0Sc9qSU+kgplamUOljPeqWUmm99T+KVUtEtOqHW2um+AHcgBbgY6AQcAAbV2OYx4H3r938Cljg67ja45qsBL+v3j3aEa7Zu5wNsBXZiGfPe4bG34s+4H7Af8LO+DnZ03G1wzbHAo9bvBwHHHR23Ha57HBANHKxn/XXAOkABlwO7WnI+Z70jHwUka62Paa1LgS+Am2pscxPwX+v3XwPXqsYmDnVujV6z1vp7rXWh9eVOLLMyuTJbfs4ALwPzgOK2DK4V2HK9DwELtNa5AFrrzDaO0d5suWYNdLV+7wucacP4WoXWeiuQ08AmNwGfaIudQDelVGhzz+esibwncKra61Trsjq30VqXA3lAQJtE1zpsuebqpmP5j+7KGr1mpdSlQG+t9eq2DKyV2PIz7g/0V0ptU0rtVEpNarPoWoct1/wCcJdSKhVYC8xom9Acqql/7w2y28QSdlbXnXXNdpK2bONKbL4epdRdwAjgqlaNqPU1eM1KKTfgLeC+tgqoldnyM/bAUl75HZZPXD8qpYZorc+1cmytxZZrvgP4j9b6DaXUaOBT6zWbWz88h7Fr/nLWO/JUoHe1172o/XGrahullAeWj2QNfZRxdrZcM0qp8cBsYIrWuqSNYmstjV2zDzAE2GKdE/ZyYKULP/C09fd6hda6TGv9K3AES2J3VbZc83TgSwCt9Q7AE8vAUu2ZTX/vtnLWRL4b6KeU6qOU6oTlYebKGtusBO61fn8b8J22PkVwUY1es7XM8AGWJO7qtVNo5Jq11nla60Ctdbi2TCW4E8u173FMuC1my+/1N1geaqOUCsRSajnWplHaly3XfBK4FkApdQmWRH62TaNseyuBe6ytVy4H8rTWac0+mqOf7jbw1Pc6IAnLE+/Z1mUvYflDBssP+ysgGfgZuNjRMbfBNW8CMoA469dKR8fc2tdcY9stuHCrFRt/xgp4EzgEJAB/cnTMbXDNg4BtWFq0xAETHB2zHa75cyANKMNy9z0deAR4pNrPeYH1PUlo6e+1dNEXQggX56ylFSGEEDaSRC6EEC5OErkQQrg4SeRCCOHiJJELIYSLk0QuhBAuThK5EEK4uP8P+tqygpGziv4AAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# gradients at training points\n",
"grads_x = function_grad_x([x.reshape(-1,1)])\n",
"plt.plot(x, df(x), label='dy/dx exact')\n",
"plt.plot(x, grads_x, 'g.', markersize=5, label='dy/dx Keras')\n",
"plt.title('Training Data')\n",
"plt.legend()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x1ee96ad7f08>"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEICAYAAABCnX+uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhUVZrH8e+bEAmbIFu7YAg62C7IEsAdFEGgtcEFul1Q1Faioo5jqy2adkE7QiutQDsjkxaZpicqbowLSrOJuKACgqKgAkIQwTYBiWAIZjnzRxVlKmslVcmt5fd5njxP7lL3vtwHXk6de95zzDmHiIjEriSvAxARkfAokYuIxDglchGRGKdELiIS45TIRURinBK5iEiMUyKXuGZmzc1sr5kd7nUsIo1FiVw84U+uB37KzWxfhe0xYVz3fTO7/MC2c26/c661c257ZCIPutdkMysxsz3+n8/NbJqZdW5ovCINoUQunvAn19bOudbAVmBEhX25XsdXD393zrUBOgC/AdKBlWbWydOoJKEokUtUMrNkM7vHzL4yswIzyzWzdv5jrczsWTPbZWa7zewDMzvEzP4C9Aee9Lfs/2JmqWbmzKyL/7PPmtlUM/unvxX9rpl1rXDf88xsg/+6U0NtMTvnfnLOrQVGAz8Ct/iv18nM3jCzfH+8L5vZYf5jVeL173/CzLaZ2Q9m9qGZnRLRhytxR4lcotUdwFDgDKALUAI85j92LdAMOALoCNwE/OScuw1YAVzrb9nfVsO1LwPuAtoDO4CJAGZ2KDAHuBXoBGwH+tYnaOdcCfAqMMC/KwmYAaQB3fz7HvOfW1O8y4ET8bXyXwaeN7OU+sQhiUWJXKLVdcAE59x251wxvmR7sZkZvqTeCTjaOVfqnFvhnPuxHtd+zjn3kT/pPg309u8fCaxwzr3mPzYF+L4BsW/H958Ezrl/Oededs7tc84VApOAM2v7sHNutnPue38MD+FL6Ec1IA5JEErkEnX8yfpI4HV/F8duYDW+v68dgJnAW8AL/i6Ih8wsuR63+LbC70VAa//vhwNfHzjgnCsHvmnAH+EIYJf/z9LGzJ4ys61m9gOwAN+3iBqZ2V1m9oWZFeL7jyS1rs9IYlMil6jjfFNyfgOc7ZxrV+En1TlX4B+Jcq9z7lhgIL6XjJcc+HgYt96BrxsHADNLwpeUQ2ZmzYBfA2/7d03wX7O/c+5gfN1FVuEjrtLnzwFuBi4E2uFr2e+r9BmRIErkEq1mAJPN7EgAM+tsZiP8vw8xs+P9ifYHoBQo83/uXzS8G+IV4GQzO9efkH8PHBLKB80sxcxOAJ4D2gDT/Yfa4Gv17zazjsAfK320crxt8HUd5QMHAQ/ga5GL1EiJXKLVw8AiYImZ7QHeAzL8x47A9xJwD/Ap8Dq+BAq+F4ljzex7M3u4Pjd0zu0ALsWXhAvwtaTXAvtr+diV/vh2A3PxfZPo75z7zn98Cr5ukZ3AO/5YK6oc76vAMmAT8JU/jvz6/Dkk8ZgWlhCpnr9V/i2+Me7LvY5HpCZqkYtUYGa/MrO2ZpYK3IevW2SVx2GJ1EqJXCTYQGAz8B0wGLjQOfeTtyGJ1E5dKyIiMU4tchGRGNfMi5t27NjRpaene3FrEZGYtWrVqgLnXJUJ2TxJ5Onp6axcudKLW4uIxCwzy6tuv7pWRERinBK5iEiMUyIXEYlxnvSRV6ekpIRt27ZRXFzsdSgJJTU1lS5dupCSoumuRWJV1CTybdu20aZNG9LT0/HNYiqNzTnHzp072bZtG926dav7AyISlaKma6W4uJgOHTooiTchM6NDhw76FiTSBHLX5pI+NZ2kiUmkT00nd23klqaNmhY5oCTuAT1zkcaXuzaXzFczKSopAiCvMI/MVzMBGHPimLCvH3aL3L+47Ydm9rGZfWZmE8OOSkQkjmQtzgok8QOKSorIWpwVketHomtlP76VXHrhW/tweDys+n3//fczZcqUOs975plnyM7OrrI/PT2dgoKCxgit3pYuXcp7773ndRgiCWtr4dZ67a+vsBO589nr30zx/yTMTFzz589n+PDhXodRKyVyEW+UlTtmvrOZ5KpV9QCktU2LyH0i8rLTzJLNbA2+qT8XOuc+qOacTDNbaWYr8/Ojc8GT7OxsfvnLXzJkyBC++OILADZt2kRGRkbgnA0bNtC3b1/AN+pjzZo1ZGRksHPnToYOHUqfPn247rrrODCr5IoVK+jZsyfFxcX8+OOPnHDCCXz66adV7v2///u/nHTSSfTu3ZvrrruOsrIy8vLy6N69OwUFBZSXlzNgwAAWLFgAwAUXXEDfvn054YQTyMnJCVxn/vz5ZGRk0KtXLwYPHsyWLVuYMWMGjz32GL179+btt9+ucm8RibzPv/2Bi554jwdfW8dpHW+iRbMWQcdbprQke3DVb/MNEZGXnc65MqC3mbUD5ppZD+fcp5XOyQFyAPr161dri33iq5+xbvsPkQgt4PjDD+a+ESfUeHzVqlU8++yzrF69mtLSUjIyMujbty9HH300bdu2Zc2aNfTu3ZtZs2Zx1VVXAbB69Wp69eqFmTFx4kTOOOMM7r33XubNmxdIrv3792fkyJH88Y9/ZN++fVx++eX06NEj6N7r169nzpw5vPvuu6SkpDB+/Hhyc3MZO3Ysd955J9dffz0nn3wyxx9/PEOHDgXgqaeeon379uzbt4/+/fszatQoysvLGTduHMuWLaNbt27s2rWL9u3bc/3119O6dWtuv/32iD5TEalqf2kZ/7lkI/+1dBMHt0hh2iW9GdnrXJ7+9CiyFmextXAraW3TyB6cHZEXnRDhUSvOud1mthQYjm8txZjx9ttvc+GFF9KyZUsARo4cGTh27bXXMmvWLB599FHmzJnDhx9+CPhav7/61a8AWLZsGS+99BIA5513Hocc8vOavffeey/9+/cnNTWV6dOnU9nixYtZtWoV/fv3B2Dfvn107tw5cO/nn3+eGTNmsGbNmsBnpk+fzty5cwH4+uuv2bBhA/n5+QwcODAwJrx9+/aReTgiEpKVW3Zx54ufsCn/Ry7qcwR//PXxtG91EOAbnRKpxF1Z2InczDoBJf4k3gIYAvw5nGvW1nJuTDUNxRs1ahQTJ07k7LPPpm/fvnTo0AGABQsW8OKLL9b5+V27drF3715KSkooLi6mVatWQcedc1x55ZVMmjSpymeLiorYtm0bAHv37qVNmzYsXbqURYsWsXz5clq2bMlZZ51FcXExzjkNJxRpYrlrc/nDgrvYvncbyeUdOSrlWv7+u1s585jq+8UbQyT6yA8D3jSzT4AV+PrIX4vAdZvUwIEDmTt3Lvv27WPPnj28+uqrgWOpqakMGzaMG264gauvvhqAwsJCSktLA0l94MCB5Ob6Bvi/8cYbfP/994HPZ2Zm8uCDDzJmzBjuvPPOKvcePHgwL7zwAt9951t4fdeuXeTl+WarvPPOOxkzZgwPPPAA48aNC9z7kEMOoWXLlnz++ee8//77AJx66qm89dZbbN68OXAdgDZt2rBnz57IPSwRAXxJfOxL17B979eAoywpn202jW37FzRpHJEYtfKJc66Pc66nc66Hc+6BSATW1DIyMrj44ovp3bs3o0aNYsCAAUHHx4wZg5kF+qgXLlzIkCFDAsfvu+8+li1bRkZGBgsWLCAtzfc2evbs2TRr1ozLLruMCRMmsGLFCpYsWRJ07eOPP54//elPDB06lJ49e3LOOeewY8cO3nrrLVasWBFI5gcddBCzZs1i+PDhlJaW0rNnT+655x5OOcU32rNTp07k5ORw0UUX0atXLy6++GIARowYwdy5c/WyUySC3ttYwJUv3Eo5+4P27yuN3PjwUHmyZme/fv1c5YUl1q9fz3HHHdfksYRqypQpFBYW8uCDDwK+vutrr702kERjWbQ/e5FoUl7uOOru1wHISx0BVjWHGkb5feURv7eZrXLO9au8P6pK9KPVhRdeyKZNm4Ja0k8++aSHEYmIF55b+TV/eOGTwPYhqYfx/f7tVc6L1PjwUCmRh+DA6BARSUzFJWUce8/8oH1f/ulXPL/+4aA5VCCy48NDpUQuIlKLP8//nCeWbgps//XSPozodTjw84RXjTU+PFRK5CIi1SjYu59+f1oUtG/zpHOrDPFtzPHhoVIiF5GElrs2t0qL+s2PjmHR+u8C57x4w6n07Rq9BXZK5CKSsKqbJ/yKF6+hfclNtGYQx/yiNQtuPdPjKOsWNSsERZumnsa2devWgd9ff/11unfvztatkZniUkSqV9084c72s7vZbJbdMSgmkjjEcCJvzGWT6iPS09guXryYm2++mfnz5weKiupSWloasfuLJJKa5gMvTyogrUPLJo6m4WIykR/4OpRXmIfDBZZNCjeZezmNLfgm7ho3bhzz5s3j6KOPBiA/P59Ro0bRv39/+vfvz7vvvgv4vjFkZmYydOhQxo4dy5YtWxgwYAAZGRlkZGQE5h/fsWMHAwcOpHfv3vTo0UOVnSJ+ZeWOpPKO1R5r6nHg4YrJPvLalk1q6NtjL6exBdi/fz/nn38+S5cu5dhjjw3sv+WWW7j11ls544wz2Lp1K8OGDWP9+vWBmN955x1atGhBUVERCxcuJDU1lQ0bNnDppZeycuVKnn76aYYNG0ZWVhZlZWUUFRVVubdIovn9nDW8tPob2iWPZVfK4zj7uczei3Hg4YrJRN4YyyZ5OY0tQEpKCqeddhozZ85k2rRpgf2LFi1i3bp1ge0ffvghMAHWyJEjadHCN1l9SUkJN910E2vWrCE5OZkvv/wS8P1H8rvf/Y6SkhIuuOACevfu3eBnJBLr9hSXcOL9P09o1bpsEI+M7sXEt+7xdBx4uGIykae1TSOvMK/a/eHwahpbgKSkJJ577jmGDBnCQw89xN133w1AeXk5y5cvDyTsiipe57HHHuMXv/gFH3/8MeXl5aSmpgK+WRmXLVvGvHnzuOKKK7jjjjsYO3ZsiE9EJH6cPnkJ3+zeF9i++vT0wJTZV/e5wquwIiIm+8izB2fTMiX4RUS4X4e8nMY28Gdo2ZLXXnuN3NxcZs6cCcDQoUN5/PHHA+dUXFyiosLCQg477DCSkpL4xz/+QVlZGQB5eXl07tyZcePGcc011/DRRx815PGIxKwtBT+SPmFeUBLfPOlcz9Y9aAwx2SJvjLLYitPYdu3atdppbF966aVap7G99NJLycjI4Mwzz6x2GtuysjJOO+00lixZwtlnn11tHO3bt2f+/PkMHDiQjh07Mn36dG688UZ69uxJaWkpAwcOZMaMGVU+N378eEaNGsXzzz/PoEGDAq31pUuX8sgjj5CSkkLr1q2ZPXt2g5+RSKxJnzAvaPuR0T35Tb8jPYqm8Wga2xBpGluR2PHOhgIunxm8BvyWyed5FE3kaBrbMGgaW5HYUbkV/uINp9G36yE1nB0flMhDoGlsRaLfU+9s5oHX1gXti4dWeCiiKpFr8eCm50XXmkgklZU7jvav2HPA8rvO5rC2VUd6xauoSeSpqans3LmTDh06KJk3EeccO3fuDAxVFIk1Bwp7DujZpS2v3HSGhxF5I+xEbmZHArOBQ4FyIMc5N632T1XVpUsXtm3bRn5+frghST2kpqbSpUsXr8MQqZcfikvoeX/wSvWfPzic1JRkjyLyViRa5KXAbc65j8ysDbDKzBY659bV9cGKUlJS6NatWwTCEZF4duqkxewoLA5sVyzsSVRhJ3Ln3A5gh//3PWa2HjgCqFciFxGpzcbv9jDk0WVB+6pbsScRRbSP3MzSgT7AB9UcywQygZCnZxURAeiYdTu7m82mLLWAZNeRGzPuYdr5N3sdVtSIWIm+mbUGXgT+wzn3Q+Xjzrkc51w/51y/Tp06Req2IhLHnl/5NR2zbmdXyuOUJeWDOcqS8nnyswmerUEQjSLSIjezFHxJPNc591Ikrikiie1AYc/u5rODppmF8KetjjeRGLViwExgvXPu0fBDEpFEducLnzBn5deB7TKrfsnEcKatjjeR6Fo5HbgCONvM1vh/zo3AdUUkgZSWlZM+YV5QEl9y25l0bVf9O7VYW8WnMUVi1Mo7gF4bi0iDnZS9iO/2BHefHCivzx6cHbTSPcTmKj6NKWoqO0Uk8ezcu5++f1oUtO+zicNo1fzn1NQY01bHm6iZxlZE4sf4eePJWZVDmSsj2ZI5K/0sNu7ayNbCrbRv0R6AnUW7SHYdaVc6ltZlg+h9ZDv+78bTPY48umkaWxFpEuPnjeeJlU8EtstcGYs3Lw5s79y30/eLQZnlsyvlcaZd3IcxPRNjpsLGEJNLvYlI9MpZlVOv853tJ2tJViNFkxiUyEUkospcWb0/o6GE4VEiF5GISrb6z0CooYThUSIXkYjK7JtZdWctYyo0lDB8SuQiElHThj9Oq9JzwSX5ErhL4rQuZ9G1bVcMo0OLDnRo0QHD6Nq2KzkjcjSUMEwatSIiEXNgfpSOjKdjyXggcdbN9JISuYiE7dvCYk6ZtDho3yf3D+Xg1BSPIkosSuQiEpYDrfCK1ApvWkrkItIg724sYMyTwWvIaMUebyiRi0hIctfmBuY7SSr3l9YzCIBf9zyMxy/L8DjCxKVELiJ1yl2bGzQDYVmSr7QeoCB7ipehCRp+KCIhyFqcFTSNLPhK61t3esGjiKQiJXIRqdXIx98hb3f1JfQqrY8O6loRkWqVlJXTPesNAJKbd6TM8quco9L66KBELiJVVB5S2K50LPtaPaFVeqKUulZEJODbwuIqSfyT+4dSkD2FnBE5gTJ7ldZHF60QJCJA1Va4GWyepMKeaNKoKwSZ2VPAr4HvnHM9InFNEWka72wo4PKZKuyJZZHqI/8f4HFgdoSuJyJNoHIrfESvw/nrpX08ikYaKiKJ3Dm3zMzSI3EtEWl8Uxd9ydRFG4L2aX6U2NVko1bMLBPIBEhL05AlEa9UboU/PKonv+1/pEfRSCQ0WSJ3zuUAOeB72dlU9xURnxF/fYe13xQG7VMrPD5oHLlInKtY2HPA6/8+gOMPP9ijiCTSlMhF4pjmCk8MkRp++AxwFtDRzLYB9znnZkbi2iJSfzsK93HqpCVB+7RiT/yK1KiVSyNxHREJX+VWeJLBVyrsiWvqWhGJE29vyOeKmR8G7VNhT2JQIheJA5Vb4ef3Ppxpl6iwJ1EokYvEsMcWfsm0xSrsSXRK5CIxqkphz+ie/LafCnsSkRK5SIw5b/rbfLb9h6B9aoUnNiVykRihwh6piRK5SAxQYY/URolcJIqpsEdCoUQuEqUqt8KbJRkbHzrXo2gkmimRi0QZFfZIfSmRi0SRyq3wC3ofzlQV9kgdlMhFosCjC79kugp7pIGUyEU8VrkV/sjonvxGhT1SD0rkIh45d9rbrNuhwh4JnxK5SBP7qbScY/4YXNjzxi0DOO4wFfZIwyiRizQhFfZIY1AiF2kC23fv47TJwYU9a+8fShsV9kgEKJGLNLLKrfCUZGNDtgp7JHKUyEUaybIv8xn7lAp7pPEpkYs0AhX2SFOKSCI3s+HANCAZeNI5NzkS1xWJNY8u+ILpSzYG7dPLTGlsYSdyM0sG/hM4B9gGrDCzV5xz68K9tkgsUWGPeCUpAtc4CdjonPvKOfcT8CxwfgSuKxJVctfmkj41naSJSaRPTSd3bS4Av5r2dpUkvmXyeUri0mQi0bVyBPB1he1twMmVTzKzTCATIC0tLQK3FWk6uWtzyXw1k6KSIgDyCvPIfCWTW55dTeuyQYHz5v/HAI49VIU90rQi0SKv7hW8q7LDuRznXD/nXL9OnTpF4LYiTSdrcVYgiR9QVFrE7mazA9tbJp+nJC6eiESLfBtQ8TtkF2B7BK4rEjW2Fm6tdn+ZFaiwRzwXiRb5CqC7mXUzs4OAS4BXInBdkaiR1rb67sCu7dKUxMVzYSdy51wpcBPwT2A98Jxz7rNwrysSLVZv/Z69+aMx1zxof8uUlmQPzvYoKpGfRWQcuXPudeD1SFxLJJocGI3SGt8LzfI2z7B7/w7S2qaRPTibMSeO8TI8EUCVnSLVevqDrdw9d23QvoLsKcAUbwISqYUSuUgllceEz7q6P4N+2dmjaETqpkQu4nfrnDXMXf1N0D6V10ssUCKXhFdSVk73rOAVe9664yy6dmjlUUQi9aNELgmt5/3/5Ifi0qB9aoVLrFEil4SUv2c//bMXBe1b/8BwWhyU7FFEIg2nRC4Jp/LLzJO7tWfOdad6FI1I+JTIJWF8tPV7Lvqv94L2acUeiQdK5JIQKrfCbzvnGG4e3N2jaEQiS4lc4lruB3lkzf00aJ9eZkq8USKXuFW5Ff4/V/fnLBX2SBxSIpe4c8uzq3l5TfBMymqFSzxTIpe4UV1hz7I7BpHWoaVHEYk0jUjMRy7imQPraNr9SbR44FD2Jr8ZOLZl8nlK4pIQ1CKXmJW7Npdxr2Syr7QIDMosn10pj/PI6F5c3ecKr8MTaTJqkUvMuvKFW31JvAJn+5n41j0eRSTiDSVyiTnLN+0kfcI8yqyg2uM1ra8pEq/UtSIxpeKQwmTXkTLLr3JOTetrisQrtcglJsx4a1OVceF/H/0YLVOCX2ZqHU1JRGqRS9SrnMBnXN6X4T0ODWxnLc5ia+FWraMpCcuccw3/sNlvgPuB44CTnHMrQ/lcv3793MqVIZ0qCeyyv73Pe5t2Bu1TYY8kMjNb5ZzrV3l/uC3yT4GLgP8O8zoiAT+VlnPMH4MLe5bcdiZHdWrtUUQi0S2sRO6cWw9oGlCJmMrdKKBWuEhdmqyP3MwygUyAtDSNKpBg3xYWc8qkxUH71j0wjJYH6TWOSF3q/FdiZouAQ6s5lOWceznUGznncoAc8PWRhxyhxL3KrfCjOrViyW1neROMSAyqM5E754Y0RSCSeN7bVMBlf/sgaJ9W7BGpP31vFU9UboWPG9CNrPOO9ygakdgWViI3swuBvwKdgHlmtsY5NywikUlc+q+lG3l4/hdB+/QyUyQ84Y5amQvMjVAsEufqKuwRkYZR14o0uktz3mf5VyrsEWksmmtFGs3f1/yDZvd15tntp7Gt+dXsTX6TJbedWW0SP7BARNLEJNKnppO7NteDiEVik1rk0ig6Zt3OrpTHcUn7Ad+iD/taPcHyb/twVKfguVBy1+aS+WomRSW+ucXzCvPIfDUTQPOmiIRALXKJqLydP5I+YR67m83G2f6gY0UlRWQtzqrymazFWYEkXte5IlKVWuQSMRVfZtZn0YeaFoLQAhEioVGLXMI2/9MdVUakpLU7stpzq1v0oaaFILRAhEholMil3iq+mGx2X2cuf+YvgWPDTziULZPP46HBD4W86EP24GwtECESBnWtSL1UfjFZluRbuR6gIHtK4LwDLylDWfShPueKSFVhLSzRUFpYInalT00nrzCvyv6ubbuy5T+2NH1AIgmkpoUl1LUSxyI9NvuSnOXk7daLSZFoo66VOBXJsdkVV+xJbq6V60WijVrkcSpSY7PTJ8wLWnatXelYvZgUiTJK5HEq3LHZOwr3VRlSuO6BYRRkTyFnRA5d23bFMLq27UrOiBy9mBTxkLpW4lRa27RqX0qG0gVSOYF379yahb8/M7A95sQxStwiUUQt8jjVkLHZ720sqJLEN086NyiJi0j0UYs8TtU2Njt3bW6V/Vm57YI+f93Ao7jr3OO8CF1E6knjyBNM5dEsAOaa077kJlqXDQI0V7hItKppHLla5AmmutEszvazu9lsnr7sdoaeoBV7RGKN+sgTTE2jVsqTCpTERWKUEnkCKSkrJ6m8Y7XHVNAjErvCSuRm9oiZfW5mn5jZXDNrV/enxAvDpy6je9YbtCsdi7nmQcdU0CMS28JtkS8EejjnegJfAneFH5JEUsHe/aRPmMfn3+4BoHXZIJ46/28q6BGJI2G97HTOLaiw+T4wOrxwJJJ+O2M5H27ZFdge3bcLU37TC4Cr+lzhVVgiEmGRHLXyO2BOTQfNLBPIBEhLU39sYyosKqHXAwuC9m2edC5m5lFEItKY6uxaMbNFZvZpNT/nVzgnCygFapwn1TmX45zr55zr16lTp8hEn4Dqmpp22qINQUn8H9ecxJbJ5ymJi8SxOlvkzrkhtR03syuBXwODnRfVRQmktqlpz+xyIWf8+c3AuePPOpo/DD/WkzhFpGmF1bViZsOBO4EznXNFdZ0v4alpatobXrmD9nt+HjD00T3n0L7VQU0dnoh4JNw+8seB5sBC/1f3951z14cdlVSrpmKePSXf0h7IvrAHY07u2rRBiYjnwh218m+RCkTqVtPUtCl04vMHh5OakuxBVCLiNVV2RpnaXmZmD86meXKLoPObJ7dg1qhHlcRFEpgSeRQ58DIzrzAPhwu8zMxdm0txSRmP/l9nWu8bT3J5J8BIa5vGzPP/VqWYJ9KLLotIdNM0tlEkfWp6tV0nHVscQatd/x3YfvnG0+l1ZPWzIVQ3TW3LlJaq3hSJAzVNY6sWeRSp6WVmQdF2AEb0OpzNk86tMYlD5BZdFpHYofnIo0hNLzOTXUfeuuMsunZoVec1wl10WURij1rkUSR7cDYtmgW/zExJSuXvox8LKYlDzdPRappakfilRB5FVq07gZZF/peZzuhycBqzLniyXn3bDVl0WURim7pWosCn3xTy67++A0BrBjF95L9z2ckNa0HXtuiyiMQnjVrxUHm5Y/SM9/ho624A2qQ2Y0XWEI0JF5FqafHlKPPWl/lc+dSHge2ZV/Zj8HG/8DAiEYlVSuRNrLikjNMnL2Hnjz8B0LNLW+aOP53kJE0zKyINo0TehJ5b8TV/ePGTwHZthT0iIqFSIm8ClVfsGdHrcKZf0luLPYhIRCiRN7Kpi75k6qINge1ldwwirUPLWj4hIlI/SuSN5OtdRQx4+OcVe24cdDR3DNOKPSISeUrkjeD3c9bw0upvAttasUdEGpMSeQRVLOwBeOjCExtc2CMiEiol8gioXNhzcGozPlRhj4g0ESXyMC394juumrUisK3CHhFpakrkDVRcUsZpk5ewS4U9IuKxsBK5mT0InA+UA98BVznntkcisGhWubDnlZtOp2cXFfaIiDfCbZE/4py7B8DM/h24F7g+7Kii1O6in+j9wMLA9sheh7zH/OUAAAacSURBVDNNhT0i4rGwErlz7ocKm62App9KsYk8tvBLpi1WYY+IRJ+w+8jNLBsYCxQCg2o5LxPIBEhLi50heSrsEZFoV+d85Ga2CDi0mkNZzrmXK5x3F5DqnLuvrpvGynzklQt7Vt9zDoeosEdEPNLg+cidc0NCvMfTwDygzkQe7VTYIyKxJNxRK92dcwc6jkcCn4cfknfKyx2jZrzHahX2iEgMCbePfLKZ/RLf8MM8YnjESuXCnllX9WfQsZ09jEhEJDThjloZFalAvFJcUsapkxbzfVEJAL26tOUlFfaISAxJ6MpOFfaISDxIyESuwh4RiScJl8grF/a8/YdBHNlehT0iErsSJpFXLuy5adC/cfuwX3oYkYhIZCREIr91zhrmqrBHROJUXCfyyoU9ky46kUtPUmGPiMSXuEzklQt72rZI4YO7B6uwR0TiUtwlchX2iEiiiZtEXlxSximTFrNbhT0ikmDiIpFXLux59aYzOLFLWw8jEhFpOjGdyCsX9pzf+3CmXdLHw4hERJpezCbyRxd+yXQV9oiIxF4ir1zYc/PZ/8ZtQ1XYIyKJK2YSee7aXG6bP4F//fgNyc070q50LBvufkiFPSKS8JK8DiAUuWtzyXw1k38VbQNzlCXls6/VE7z+1fNehyYi4rmYSORZi7MoKikK2ldUUkTW4iyPIhIRiR4xkci3Fm6t134RkUQSE4k8rW3186PUtF9EJJHERCLPHpxNy5TgoYUtU1qSPTjbo4hERKJHTCTyMSeOIWdEDl3bdsUwurbtSs6IHMacOMbr0EREPGfOufAvYnY78AjQyTlXUNf5/fr1cytXrgz7viIiicTMVjnn+lXeH3aL3MyOBM4B9OZRRMQDkehaeQz4AxB+015EROotrERuZiOBb5xzH4dwbqaZrTSzlfn5+eHcVkREKqizRN/MFgGHVnMoC7gbGBrKjZxzOUAO+PrI6xGjiIjUos5E7pwbUt1+MzsR6AZ8bGYAXYCPzOwk59y3EY1SRERqFJFRKwBmtgXoF8qoFTPLB34E6jw3gXVEz6cueka10/OpXSw+n67OuU6Vd3oy+6FzrpOZraxuGI346PnUTc+odno+tYun5xOxRO6cS4/UtUREJHQxUdkpIiI18zKR53h471ig51M3PaPa6fnULm6eT8RedoqIiDfUtSIiEuOUyEVEYlyjJ3IzG25mX5jZRjObUM3x5mY2x3/8AzNLb+yYokkIz+f3ZrbOzD4xs8Vm1tWLOL1S1/OpcN5oM3NmFhfDyUIVyvMxs9/6/w59ZmZPN3WMXgvh31iamb1pZqv9/87O9SLOsDjnGu0HSAY2AUcBBwEfA8dXOmc8MMP/+yXAnMaMKZp+Qnw+g4CW/t9v0PMJfj7+89oAy4D38RWleR57tDwfoDuwGjjEv93Z67ij8BnlADf4fz8e2OJ13PX9aewW+UnARufcV865n4BngfMrnXM+8Hf/7y8Ag81f858A6nw+zrk3nXMHVp5+H99UCIkilL8/AA8CDwPFTRlcFAjl+YwD/tM59z2Ac+67Jo7Ra6E8Iwcc7P+9LbC9CeOLiMZO5EcAX1fY3ubfV+05zrlSoBDo0MhxRYtQnk9F1wBvNGpE0aXO52NmfYAjnXOvNWVgUSKUvz/HAMeY2btm9r6ZDW+y6KJDKM/ofuByM9sGvA7c3DShRU5jl+hX17KuPN4xlHPiVch/djO7HOgHnNmoEUWXWp+PmSXhmw//qqYKKMqE8venGb7ulbPwfZt728x6OOd2N3Js0SKUZ3Qp8D/Oub+Y2anAP/zPqLzxw4uMxm6RbwOOrLDdhapfWwLnmFkzfF9tdjVyXNEilOeDmQ3BN23wSOfc/iaKLRrU9XzaAD2Apf5J204BXkmgF56h/vt62TlX4pzbDHyBL7EnilCe0TXAcwDOueVAKr4JtWJGYyfyFUB3M+tmZgfhe5n5SqVzXgGu9P8+Glji/G8dEkCdz8ffdfDf+JJ4ovVv1vp8nHOFzrmOzrl055vr5318zylRFoQN5d/X/+F7YY6ZdcTX1fJVk0bprVCe0VZgMICZHYcvkcfU6jeNmsj9fd43Af8E1gPPOec+M7MH/KsLAcwEOpjZRuD3QI1DzOJNiM/nEaA18LyZrTGzyn8J41aIzydhhfh8/gnsNLN1wJvAHc65nd5E3PRCfEa3AePM7GPgGeCqWGtMqkRfRCTGqbJTRCTGKZGLiMQ4JXIRkRinRC4iEuOUyEVEYpwSuYhIjFMiFxGJcf8PvKifSyyTkBMAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# gradients at testing points\n",
"grads_xtest = function_grad_x([x_test.reshape(-1,1)])\n",
"plt.plot(x_test, df(x_test), label='dy/dx exact')\n",
"plt.plot(x_test, grads_xtest, 'go', label='dy/dx Keras')\n",
"plt.title('Testing Data')\n",
"plt.legend()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Comments\n",
"Keras has estimated the gradient. Can the estimated gradient be improved? Why is gradient not smooth? I briefly investigated this.. seems Keras is using a \"computation graph\" within the *K.function* command to map *model.inputs* to *grads*. Alternatively, one may be able to train their own model to map *model.inputs* to *grads* with better accuracy. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
@kdneal
Copy link
Author

kdneal commented Mar 23, 2020

My solution to the example proposed by thearn. Note that I modified Define model & data section so that x need not be scaled.
Original example: https://gist.github.com/thearn/be9f98c5c2a6f87068490808f374a07c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment