0. FrozenLake-v0(Q-Net)

0. FrozenLake-v0(Q-Net).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q-Net Learning with Neural Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While it is easy to have a 16 X 4 table for this simple grid problem, the number of possible states in any real-world env. is nearly infinite larger. For most interesting problems, table simply don't work. Instead, We need some way to take a discription of out state, and produce Q-values for actions. On this way, we no longer need a table : that is where neural networks come in.\n",
    "\n",
    "In the case of the FrozenLake example, we could used  a one-layer network which take the state encoded in a one-hot vector(1X16) and produces a vector of 4 Q-values(The action is the one element of {U,D,L,R}). Such a simple network act kind of prior table, with the network weights serving as the old cells. We can easily expand the Tensorflow network with 1) added layers, 2) activation functions, and 3) preprocessing different input types, whereas all that is impossible with a regular table.\n",
    "\n",
    "The method of updating is a little different as well.\n",
    "Instead of directly updating our table, with a network we will be using 1) backpropagation and 2) a loss function.\n",
    "Our loss function will be sum-of-squares loss, where the difference between the current predicted Q-values, and the“target”value is computed and the gradients passed through the network. In this case, our Q-target for the chosen action is the equivalent to the Q-value computed in the equation \"Q(s,a) = r + γ(max(Q(s’,a’))\" and Loss = ∑(Q-target - Q)²"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import gym\n",
    "import numpy as np\n",
    "import random\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "env = gym.make('FrozenLake-v0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Q-Network Approch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementing the network itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tf.reset_default_graph()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#These lines establish the feed-forward part of the network used to choose actions\n",
    "inputs1 = tf.placeholder(shape=[1,16],dtype=tf.float32)\n",
    "W = tf.Variable(tf.random_uniform([16,4], 0, 0.01))\n",
    "\n",
    "Q_out = tf.matmul(inputs1, W)\n",
    "predict = tf.argmax(Q_out, 1) # return : index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Below we obtain the loss by taking the sum of squares difference between the target and prediction Q values.\n",
    "next_Q = tf.placeholder(shape=[1,4],dtype=tf.float32)\n",
    "loss = tf.reduce_sum(tf.square(next_Q - Q_out))\n",
    "trainer = tf.train.GradientDescentOptimizer(learning_rate=0.1)     # learing by SGD\n",
    "updateModel = trainer.minimize(loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Percent of succesful episodes: 0.393%\n"
     ]
    }
   ],
   "source": [
    "init = tf.global_variables_initializer()\n",
    "\n",
    "# Set learning parameters\n",
    "y = .99\n",
    "e = 0.1\n",
    "num_episodes = 2000\n",
    "\n",
    "# Create lists to contain total rewards and steps per episode\n",
    "\n",
    "jList = []\n",
    "rList = []\n",
    "with tf.Session() as sess:\n",
    "    sess.run(init)\n",
    "    \n",
    "    for i in range(num_episodes):\n",
    "        # Reset environment and get first new observation\n",
    "        s = env.reset()\n",
    "        rAll = 0\n",
    "        d = False\n",
    "        j = 0\n",
    "        \n",
    "        # The Q-Network\n",
    "        while j < 99:\n",
    "            j+=1\n",
    "            \n",
    "            # Choose an action by greedily (with e chance of random action) from the Q-network\n",
    "            a, all_Q = sess.run([predict,Q_out], feed_dict={inputs1:np.identity(16)[s:s+1]})\n",
    "            if np.random.rand(1) < e:\n",
    "                a[0] = env.action_space.sample()\n",
    "                \n",
    "            # Get new state and reward from environment\n",
    "            s1,r,d,_ = env.step(a[0])\n",
    "            \n",
    "            # Obtain the Q' values by feeding the new state through our network\n",
    "            Q1 = sess.run(Q_out, feed_dict={inputs1:np.identity(16)[s1:s1+1]})\n",
    "            \n",
    "            # Obtain maxQ' and set our target value for chosen action.\n",
    "            max_Q1 = np.max(Q1)\n",
    "            target_Q = all_Q\n",
    "            target_Q[0,a[0]] = r + y*max_Q1\n",
    "            \n",
    "            # Train our network using target and predicted Q values\n",
    "            _, W1 = sess.run([updateModel,W], feed_dict={inputs1:np.identity(16)[s:s+1], next_Q:target_Q})\n",
    "            rAll += r\n",
    "            s = s1\n",
    "            \n",
    "            if d == True:\n",
    "                # Reduce chance of random action as we train the model.\n",
    "                e = 1./((i/50) + 10)\n",
    "                break\n",
    "                \n",
    "        jList.append(j)     # step\n",
    "        rList.append(rAll)  # total reward\n",
    "        \n",
    "print(\"Percent of succesful episodes: \" + str(sum(rList)/num_episodes) + \"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Some statistics on network performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGjRJREFUeJzt3XtwW+d55/HvQ/AmiqQoidSNkkxJpi+0nVg2I8uXOOnK\ndSS3lZtskrG229y8UXc27i3dnTqbjuv17mSbZNvMdNZt1tmmaTytHTfZbjSpEtl13TabiW1Rjm+y\nIouWJYuWJdISrbtEXZ79A4c0SALgAQUCxHt+nxkOgYP3AM95cfDDwXlxcMzdERGRsFSVuwARESk+\nhbuISIAU7iIiAVK4i4gESOEuIhIghbuISIAU7iIiAVK4i4gESOEuIhKg6nI9cGtrq3d0dJTr4UVE\nKtK2bdvedve2idqVLdw7Ojro6ekp18OLiFQkM9sbp512y4iIBEjhLiISIIW7iEiAFO4iIgFSuIuI\nBGjCcDezb5pZv5m9nON2M7M/NbNeM3vRzK4rfpkiIlKIOFvu3wLW5rl9HdAZ/W0E/vziyxIRkYsx\n4ffc3f1fzKwjT5M7gW97+nx9T5tZi5ktdPe3ilRjVq8NHKf/6BluXDF33G3uzveee5Nffs9C6mtS\nABw6foZnXz/MumsW8pc/eZ3Gumo+1r1k1Hy7B45z4OhpblrROjLtpb4jOM57FrcA8Nwbg9RVV3HV\nolnsPXSCfYdP0VRfTarKuLp9Vs56f/bGIP3HzvD07kPcfcsyFs9uAODNd07x6oFjnD57nid/3s/B\no6dZuaSF9tkzODl0nr2HTvLR6xdz1aJmvrutj/XXLqKuOsXWPYd5/e0TvDl4itXL53L09FmOnjpL\nqsr48Mp2Dp0Y4uGf7uXWy9qYM7OWp3cfoq2xjh++fIAjp87SMbeBmztb+R9bdnLtkhYOHj0NGP+w\n4yBL5syga2EzJ4fOs3h2A0dODbHn7ZN0zm+MHqOK9pZ6buuazz+8cpDLFzTz410D7Dx4jLVXLaC1\nsY7vPdfH9v1HAbimfRZ9gydprK9mVcdczl+4wDOvH+aGZXNYMqeBWTNq+NoTr3Lm3AXOXXB+e00n\nN1/aymM9+zh49DQ/3vU2AA21Ke5631J69h5m1owaevYMcs3idJ/XpIz+o2fY1X+cG5fP5er2Zrbv\nP8rhE0NcvqCJH718gDPnLow8H1e3N1NfneKWzlZ2vHWULdsPAnDJ3AbOnL3Ax7sX85trOqlJVdHb\nf5yBY2c4fe48z75+mMETQ/zzqwP81ppO3tcxm6/8aCd7Dp3g1YPHWXPFPPYfOc2+wydJVRnNM6pZ\nNGsGz7x+eOSxZzfUMHjyLAAbVi1l0ax6frT9AB2tM2mur+GRZ98Yte7Mb66jviZF/9Ez1NdUjcy7\nom0m/+qKecyaUcOzewY5cuos72mfxSPPvsFV7bPAnWNnznH01Dnqa6roGzzFjcvncvjEEDsPHgPg\nuqUtvH18iDcOn2TVsjnMqEnR1pR+/tyhvWUGb75zaqSWxrp0ZFy+oIltewdprKtmRm2KlUtamFGb\n4vvP7wfg9q75zG2sY/fA8ZFlb6qvpqmumupUFZ+5uYM/eeJV7ry2nRffPMLi2TPYeeAYvf3HaWmo\n4Z2TZ7luaQtnzl3gwJHT1NekRtVxw7I5tDbW8dKbR9K1d8xh8OQQu/qP01CbYumcBroWNvPUzn4u\nndfI4RNDOLB74AQAM2pSOM7psxdY0TaTI1Ef1VZXUZuq4vau+fzjzn72vn2ST9x0CS0zavnGj3cz\nZ2YtPz+Q7rtUlXH+grPu6gVs3TNIa2P6tobaFCeHzlNfU8XpsxdYOqeBwyeG6JzfyE0r5vKjlw/w\n+O9+gFSV5cyLYrA451CNwv0H7n51ltt+APyRu/+/6PqTwO+7+7gjlMxsI+mte5YuXXr93r2xvouf\nVce9fw/Anj/6pXG3/dPOfj71l1v51E0d3L/+KgA+8mc/4bk33uHZ/7yGVV96Muu82e5z7LTM68OX\nh2WrZez9ADTVVfPSf/kQANc+8DjvRC/WfB769evZ+PA2fuPW5XzhjivHPXam72xczX/7+x289OYR\nAN67pIUX9r0z4WPIaI98djU3rpibt69FJuMPf6WLT9+8bFLzmtk2d++eqF0xBlSzvf1kfcdw94fc\nvdvdu9vaJjx6dtKOnT4HwMDxMyPT9g2m3/Uzt97K5diZcyOX4wQ7wNEsy5TLiaFz7Bs8OXL95Sjk\npTBnz5d/XZEwDcZ83V+MYoR7H5C5f2MxsL8I9ysiIpNUjHDfBHwi+tbMauDIVO9vFxGR/CYcUDWz\nR4APAq1m1gf8IVAD4O5fBzYDdwC9wEng01NVrIiIxBPn2zIbJrjdgc8VrSIREbloOkJVRCRACncR\nkQAp3EVEAqRwFxEJkMJdRCRACncRkQAp3EVEAqRwFxEJkMJdRCRACncRkQAp3EVEAqRwFxEJkMJd\nRCRACncRkQAp3EVEAqRwFxEJkMJdRCRACncRkQAp3EVEApS4cHcvdwUiIlMvceEuIpIECncRkQAp\n3EVEAqRwFxEJUOLC3dGIqoiEL3HhLiKSBAp3EZEAKdxFRAKkcBcRCVDiwl1HqIpIEsQKdzNba2Y7\nzazXzO7NcvtSM3vKzH5mZi+a2R3FL1VEROKaMNzNLAU8CKwDuoANZtY1ptkfAI+5+0rgLuDPil2o\niIjEF2fLfRXQ6+673X0IeBS4c0wbB5qjy7OA/cUrUUREClUdo007sC/jeh9ww5g29wOPm9lvAjOB\n24pSnYiITEqcLXfLMm3ssOQG4Fvuvhi4A3jYzMbdt5ltNLMeM+sZGBgovNoi0HiqiCRBnHDvA5Zk\nXF/M+N0udwOPAbj7T4F6oHXsHbn7Q+7e7e7dbW1tk6tYREQmFCfctwKdZrbMzGpJD5huGtPmDWAN\ngJldSTrcy7NpLiIiE4e7u58D7gG2ADtIfytmu5k9YGbro2a/B3zWzF4AHgE+5a5vlIuIlEucAVXc\nfTOwecy0+zIuvwLcXNzSRERkshJ4hKo+UIhI+BIX7iIiSaBwFxEJkMJdRCRACncRkQAlLtw1nCoi\nSZC4cE+KbL8ZISLJoXAPlD6hiCSbwl1EJEAKdxGRACUu3HWAqogkQeLCPSk0oCqSbAr3QOkDikiy\nKdxFRAKkcBcRCVCQ4e7jLmQOpFbmDouRnyqOUf7YQWP9zPHkqNekkgUZ7vko50QkCYIMdxt3AcxG\n/680NrIAcdrmmFcKol6TShZkuIuIJJ3CXUQkQEGGe74B1Urd564B1dJTr0klCzLc89ELVkSSIMhw\nzzegWqk0oFp66jWpZEGGu4hI0incRUQCFGS4a0A1x7xSEPWaVLIgwz0f10tWRMqtBBtcQYa7BlRz\nzCsFUa9JJQsy3EVEkk7hLiISoCDDXQOqOeaVgqjXpJLFCnczW2tmO82s18zuzdHm42b2ipltN7O/\nKW6ZxaOcE5FyK0UMVU/UwMxSwIPALwJ9wFYz2+Tur2S06QS+ANzs7oNmNm+qCo4j7oCqu1fMYONF\nD6jqXa1glbFmiGQXZ8t9FdDr7rvdfQh4FLhzTJvPAg+6+yCAu/cXt0wRESlEnHBvB/ZlXO+LpmW6\nDLjMzH5iZk+b2dpsd2RmG82sx8x6BgYGJlexiIhMKE64Z/t0OvYzfjXQCXwQ2AD8bzNrGTeT+0Pu\n3u3u3W1tbYXWKiIiMcUJ9z5gScb1xcD+LG2+7+5n3f11YCfpsJ92dISqxKU1RaZKKYbA4oT7VqDT\nzJaZWS1wF7BpTJv/C/wCgJm1kt5Ns7uYhRZLUsYVNRgokmwThru7nwPuAbYAO4DH3H27mT1gZuuj\nZluAQ2b2CvAU8J/c/dBUFV0sIQd9wIsmIjFM+FVIAHffDGweM+2+jMsOfD76ExGRMgvyCFURkaRT\nuIvkoJ9tkKlSii92JC7ck/J61YCqSLIlLtwzhZzzIS+biEws0eEuIhIqhbuISIASF+46QlXi0poi\nU2W6HKEaFA2oikgSJC7cM4X8Vbdwl0xE4kh0uIuIhErhLiISoMSFu3ZXSGxaWWSKlGLVSl64B7yf\nPZMGVEWSLXHhninkmA952URkYokOdxGRUCncRUQClLhw1+4KiUtHM8tU0RGqUyAh46kaUBVJuMSF\ne6aQgz7gRRORGBId7iIioVK4i4gEKIHhrh0WEk/Iu+0kfIkL96S8YDWgKjJ96QTZIiIyKYkO95C/\nxxzukolIHIkOdxGRUCUu3LVFK3ElZXxGwpS8cE/IC1YDqiLTmH5+QEREJiPR4R7yVnzAiyYiMcQK\ndzNba2Y7zazXzO7N0+6jZuZm1l28EkVEpFAThruZpYAHgXVAF7DBzLqytGsCfgt4pthFFlNSTrMn\nF09rilSyOFvuq4Bed9/t7kPAo8CdWdr9V+ArwOki1ld0SXnBakBVZPqaLifIbgf2ZVzvi6aNMLOV\nwBJ3/0ERaxMRkUmKE+7ZNgJH3njMrAr4GvB7E96R2UYz6zGznoGBgfhVSsGS8glFRLKLE+59wJKM\n64uB/RnXm4CrgX8ysz3AamBTtkFVd3/I3bvdvbutrW3yVYuISF5xwn0r0Glmy8ysFrgL2DR8o7sf\ncfdWd+9w9w7gaWC9u/dMScUXSeOpEpcG36WSTRju7n4OuAfYAuwAHnP37Wb2gJmtn+oCiy3kHwvL\npAFVkemrFBsO1XEauftmYPOYafflaPvBiy9LREQuho5QDVTAiyYiMSQ63EVEQpW8cNcmrcSkVUUq\nWeLCPSkvWA2oikxfpdglnLhwFxFJgkSHe8hfiwx3yUQkjkSHu4hIqIIMdx934d19XJX69ceRgx5i\n1D92GXWk5eSo26SSBRnu+YS8K0ZEKsN0+cnfimPjLoBV+NdHbHgBYiyH2ehmVukLLyIFCzLc4wr5\nY3fAiyYiMSQ63EVEQhVkuGtANce8UiD1m1SuIMM9H71cRaTcdITqJGlAVQOqIkkXZLiLiCRdosM9\n5F00IS+biEwsyHDPP6BambGnAdXSU7dJJQsy3PPR61VEyq0UR8oHGe4aUNWAqkjSBRnuIiJJl+hw\nD3lfdLhLJiJxBBnu+QZUKzX1NKBaeuo1qWRBhns++slfESk3HaE6SRpQ1YCqSNIFGe4iIkmX6HAP\neQdNyMsmIhMLMtz1k7855pWCqNukkgUZ7vnoBSsiSRBkuGtAVQOqIkkXZLiLiCRdrHA3s7VmttPM\nes3s3iy3f97MXjGzF83sSTO7pPilFl/Iu2gCXjQRiWHCcDezFPAgsA7oAjaYWdeYZj8Dut39PcB3\nga8Uu9BC5B1QjTP/NEx9DaiWng54k0oWZ8t9FdDr7rvdfQh4FLgzs4G7P+XuJ6OrTwOLi1tm8Sjo\nRKTcSpFDccK9HdiXcb0vmpbL3cAPs91gZhvNrMfMegYGBuJXWSANqGpAVSTp4oR7tmTI+rZjZv8W\n6Aa+mu12d3/I3bvdvbutrS1+lSIiUpDqGG36gCUZ1xcD+8c2MrPbgC8CH3D3M8Upb4oFvIcm4EUT\nkRjibLlvBTrNbJmZ1QJ3AZsyG5jZSuB/Aevdvb/4ZRbm4gdUi1xQEWhAtfTUbVLJJgx3dz8H3ANs\nAXYAj7n7djN7wMzWR82+CjQCf2tmz5vZphx3V3Z6wYpIEsTZLYO7bwY2j5l2X8bl24pc10XRgGqW\nAVW9q4lMG6V4NeoIVRGRACncRUQCFGS45z2HasbEXEcgTscdGIUOqPqo69NxiaY/9ZpUsiDDPR/l\nnIgkQZDhrgFVHaEqMp3pBNkiIjIpCncRkQBVfLhnGyyMe4Rqro9G03EAUgOqpad+k0pW8eFeKL1e\nRSQJggx3DahqQFVkOivFiWCCDHcRkaRTuIuIBKjiwz3bPvT8A6o+vl2u+acRDaiKSCEqPtwLpZwT\nkSQIMtw1oKoBVZHpTEeoiojIpCjcRUQClOhwD3mgMdwlE5E4Kj7cCw2xSj1BtpSe1gOpZBUf7oUK\neWs9k4ZQRaYvnUNVREQmReEuIhKgRId7yDtoQl42EZlYxYf7VOxDL8Uvtsn0p/VAKlnFh3uhEjKe\nqgFVkWlMR6iKiMikKNxFRAKU6HAPeRdNwIsmIjFUfLgXfoTqxHNUeuhXev3ThfpRKlnFh3uhkvKC\n1YCqyHQ2Tc6hamZrzWynmfWa2b1Zbq8zs+9Etz9jZh3FLlREROKbMNzNLAU8CKwDuoANZtY1ptnd\nwKC7Xwp8DfhysQsVEZH44my5rwJ63X23uw8BjwJ3jmlzJ/BX0eXvAmtMp/8RESkbm+gITzP7KLDW\n3f9ddP3XgRvc/Z6MNi9Hbfqi669Fbd7Odb/d3d3e09NTcMGPbd3HN368m139xwG4dF7juP3L75w6\ny8CxMwB0zmsEGGk/v7mOg0fTty1vnUmq6t25h9usaJtJVfTeNDxt7P10zmscuTxsuE02udqOnZ7L\n7IYaBk+ezfnYmRbNqmf/kdOx7ldym99cR3N9TeznSCSuf3PDUr704WsmNa+ZbXP37onaxdlyz7YF\nPvYdIU4bzGyjmfWYWc/AwECMhx6vpaGGzvmNrGibSW11FZfNb6RzzN/7OmYDcNuV80amrV4+B4Dr\nL5k9cl9XLGwaPe+8RmpTVVy+4N3pbU11tDbWjVxvaahh0ax6Ouc3csWCJsygtbGWeU114+rI/Jvd\nUDPyuNe0zxqZ/t7FswCorc79VCxorufGFXOjZZpP5/xGZtamRm5vqqse1f7apS2sWpZe3ub6am65\ntJWalPH+ztZR7TrmNkzY37dc2prztqVz0vO3t8wYmfbHH3svt3fNH9XuyoXNWeevra5izsxaujOe\nE4DaVBXN9dXjlmu45sYs0wsV53Pl9ZfMHrWuDZtR827fr2ibOWqerhzLmkt1lY27j2K4YkETv5jx\nPMydWQvAp27q4NbL2kamr2ibOfJ6GfaRle0577etqS7n9N+4dfmk6/3cL6wYN+0Dl7WN2vjK9Lu3\nXcZ/+OC78/zaDUtZc8U8IP3ctjXVjaxXv3rtIgBqUuPvq76malT/37h8Llctaubbn1k1Mu0PfunK\nUev4WF/68DXUpCzn62lVxxxmzagZNW3tVQty3l+xxNlyvxG4390/FF3/AoC7//eMNluiNj81s2rg\nANDmee58slvuIiJJVswt961Ap5ktM7Na4C5g05g2m4BPRpc/CvxjvmAXEZGpNeHnW3c/Z2b3AFuA\nFPBNd99uZg8APe6+CfgL4GEz6wUOk34DEBGRMom189LdNwObx0y7L+PyaeBjxS1NREQmK3FHqIqI\nJIHCXUQkQAp3EZEAKdxFRAKkcBcRCdCEBzFN2QObDQB7Jzl7K5Dzpw3KSHUVZrrWBdO3NtVVmBDr\nusTd2yZqVLZwvxhm1hPnCK1SU12Fma51wfStTXUVJsl1abeMiEiAFO4iIgGq1HB/qNwF5KC6CjNd\n64LpW5vqKkxi66rIfe4iIpJfpW65i4hIHhUX7hOdrHuKH3uJmT1lZjvMbLuZ/XY0/X4ze9PMno/+\n7siY5wtRrTvN7ENTWNseM3spevyeaNocM3vCzHZF/2dH083M/jSq60Uzu26Karo8o0+eN7OjZvY7\n5egvM/ummfVHZw0bnlZw/5jZJ6P2u8zsk9keqwh1fdXMfh499t+ZWUs0vcPMTmX029cz5rk+ev57\no9ov6jSXOeoq+Hkr9us1R13fyahpj5k9H00vZX/lyobyrWPuXjF/pH9y+DVgOVALvAB0lfDxFwLX\nRZebgFdJnzT8fuA/ZmnfFdVYByyLak9NUW17gNYx074C3Btdvhf4cnT5DuCHpM+gtRp4pkTP3QHg\nknL0F3ArcB3w8mT7B5gD7I7+z44uz56Cum4HqqPLX86oqyOz3Zj7eRa4Mar5h8C6KairoOdtKl6v\n2eoac/sfA/eVob9yZUPZ1rFK23KPc7LuKePub7n7c9HlY8AOIPc5ydK1PeruZ9z9daCX9DKUSuaJ\ny/8K+NWM6d/2tKeBFjNbOMW1rAFec/d8B65NWX+5+7+QPtfA2McrpH8+BDzh7ofdfRB4Alhb7Lrc\n/XF3PxddfRpYnO8+otqa3f2nnk6Ib2csS9HqyiPX81b012u+uqKt748Dj+S7jynqr1zZULZ1rNLC\nvR3Yl3G9j/zhOmXMrANYCTwTTbon+nj1zeGPXpS2XgceN7NtZrYxmjbf3d+C9MoHzCtDXcPuYvSL\nrtz9BYX3Tzn67TOkt/CGLTOzn5nZP5vZ+6Np7VEtpairkOet1P31fuCgu+/KmFby/hqTDWVbxyot\n3GOdiHvKizBrBL4H/I67HwX+HFgBXAu8RfqjIZS23pvd/TpgHfA5M7s1T9uS9qOlT8+4HvjbaNJ0\n6K98ctVR6n77InAO+Oto0lvAUndfCXwe+Bszay5hXYU+b6V+PjcwegOi5P2VJRtyNs1RQ9Fqq7Rw\n7wOWZFxfDOwvZQFmVkP6yftrd/8/AO5+0N3Pu/sF4Bu8uyuhZPW6+/7ofz/wd1ENB4d3t0T/+0td\nV2Qd8Jy7H4xqLHt/RQrtn5LVFw2k/TLwa9GuA6LdHoeiy9tI78++LKorc9fNlNQ1ieetlP1VDXwE\n+E5GvSXtr2zZQBnXsUoL9zgn654y0T69vwB2uPufZEzP3F/9YWB4JH8TcJeZ1ZnZMqCT9EBOseua\naWZNw5dJD8i9zOgTl38S+H5GXZ+IRuxXA0eGPzpOkVFbVOXurwyF9s8W4HYzmx3tkrg9mlZUZrYW\n+H1gvbufzJjeZmap6PJy0v2zO6rtmJmtjtbRT2QsSzHrKvR5K+Xr9Tbg5+4+srullP2VKxso5zp2\nMSPE5fgjPcr8Kul34S+W+LFvIf0R6UXg+ejvDuBh4KVo+iZgYcY8X4xq3clFjsjnqWs56W8ivABs\nH+4XYC7wJLAr+j8nmm7Ag1FdLwHdU9hnDcAhYFbGtJL3F+k3l7eAs6S3ju6eTP+Q3gfeG/19eorq\n6iW933V4Hft61PZfR8/vC8BzwK9k3E836bB9DfifRAcoFrmugp+3Yr9es9UVTf8W8O/HtC1lf+XK\nhrKtYzpCVUQkQJW2W0ZERGJQuIuIBEjhLiISIIW7iEiAFO4iIgFSuIuIBEjhLiISIIW7iEiA/j8r\n6i2IhgQRIwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x27ee77352b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(rList)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl8FdX5/z9PEgj7HnYhgCjigkKKKIIIimu1rVrXiktL\nW7Xubem+fP22tLZa+60/rTu17itWpC6AirLvO4SdhJCEhISQkP38/rgzN3PvnZl7Zp977/N+vfLK\nvXNnzjlz5pznPPOc5zyHhBBgGIZh0pesoAvAMAzDeAsLeoZhmDSHBT3DMEyaw4KeYRgmzWFBzzAM\nk+awoGcYhklzWNAzDMOkOSzoGYZh0hwW9AzDMGlOTtAFAIA+ffqI/Pz8oIvBMAyTUqxevfqwECIv\n2XmhEPT5+flYtWpV0MVgGIZJKYhon8x5bLphGIZJc1jQMwzDpDks6BmGYdKcpIKeiJ4nojIi2qQ5\n1ouIPiGiQuV/T+U4EdHfiWgnEW0gorFeFp5hGIZJjoxG/yKAS+KOzQKwQAgxEsAC5TsAXApgpPI3\nE8CT7hSTYRiGsUtSQS+E+AJAZdzhqwDMUT7PAfANzfF/iQjLAPQgogFuFZZhGIaxjl0bfT8hRAkA\nKP/7KscHATigOa9IOZYAEc0kolVEtKq8vNxmMRiGYZhkuD0ZSzrHdPcqFEI8LYQoEEIU5OUl9fdn\nQkRdYzPeXVsEABBC4O3VRahvagEArN5XiW2HjkbP/XjzIZQdrTdMSwiBtzTXJ2P9gSpsLKqWLmtV\nXSPmbShBXWMznvtyD6qPNwEA3l9/EEt3VWCdkt6Goip8saMc+yvqotcWltbgzVUH8GXh4YR0V++r\nxNaStvusqW/C3HXFCedVHGvA/I0lAICG5hY8/+UeVNY2xpwTX4eHquvx6ZZS3fuZu64YNfVNCcff\nWHUA97y6Nlqm9QeqsKm4rZ5Kj9bj+y+tQkOzeT1/uLEElbWN2F1+DEt2HcbOshos211hek1dYzPe\nWVOEBVtLcai6PnpPv//PFixPcq0ZQgi8tGwfDlS2PZOWVoE3Vh1AS2uiWNlfUYe7XlmDllaBQ9X1\n+O+mEryx6gBadc41Y19FLRYXxiqfb646gBe+2hNzzOiZa6k+3oT31x8EEPtc311bhGcX74ZfW7na\nXTBVSkQDhBAlimmmTDleBOAEzXmDARx0UkAmfPzqvc14e00RhvTqjNqGZjz45npsKKrC7646DVc/\nuRQAsHf25WhqacXMl1ZjRF5nLHhwim5aC7eV4aE312NbyVH88orRSfO+6omvounLcNcra/DVzgrc\nd+FI/O3TQtQ1NOOyMwbgnlfXGl6jpn3RY18kHFPR3icAzHp7I+ZtLMHIvl0xemC36Hnf+9cqrNlf\nhTW/uggr9lTi9x9swaGj9fj5ZadEz1lceDiuDpeguOp4Qp6bD1bj3tfW4fIzBuCJG2P9HH7y1gYA\nkQFs7+zLE+rp7D8sAAD87dNC/PSSUbr3XVZTjztfXoPx+b2wYm+stdasvn/3/ha8viryIj+gewcs\n/dk0LNxWhue/2oPnv9oj/aziOVhdj1+9twnnndgH//7u2QCAl5buxW//swXHG1sw49z8mPMnP7II\nAFAwtCeeXbwHxVXHAQC5OVm46kxdw4Iu5z/yGYDYe/6xUr+3TRwWPWb0zLU8+MZ6fLq1FKMHdMOM\n51eguOo4vvzpBbj/9fUAgBN6dcLFp/aXLptd7Gr07wOYoXyeAWCu5vgtivfNBADVqomHSR9KFQ29\nrrEZNfXNAIDyYw0J56laV9GR44ZpmV3vBl/tjGiUFcciWnRlXSOON8q9PVhBFSrH495MDij33tzS\niuNNkXuNf8OJrwM1rXjUcqtasx0O1xjXc2Nzq2n+RhzS3E+JUjb1npygauI7SmuixyrrIm8zR+oa\nda+J/NYUcw/qW5zbHKzWf+Yx5yjlqG9qiZapQalnwJ16kiGpRk9ErwKYAqAPERUB+A2A2QDeIKI7\nAOwHcK1y+ocALgOwE0AdgNs8KDOTIjQrHTUnS8+i5y9C34LoYvoRyMNbdeMOZNJww5zgRj34ZNVw\nAeOChuUWkgp6IcQNBj9N0zlXALjLaaGYcKMVmmYCtKUl8lt2CAS9X5jdqVuCy6vaJEU6h0U4qYSt\nPCp2n0MQAxivjGU8o7k18oqaEYLehd6bLInU0XAzg1R6HCzoGcuQRpchE72mRagafeY0M/LSdqNh\nZ1lN1P5rBS8GC69u2WtzW1D41ERiCEWYYia1kO2A6mRsdgbIeSsiyWgwiD8shDA898JHIx5Bdj1a\ndPN3LaXMIJXqKwO6IGOFHaU10l4KBDK30UcnY8PTzMzeQNxJ33ucaIQyg7QbWr8bbzZezGkIIbB6\nX6WlCedCjdePHjJJaatDe75ffvTh6YFMKJj+2Be45skllq/TE6CqoDeT8+nyei7TX53a4P0SClYJ\nu2arrbU3Vh3A1U8uxYcbD0lfr11PYbsMAT86FvRMAoVlxyxfoyewg9botStD/cJIkW2x0dNlLtlZ\nZq5tBoUbwj/Z2oq6xmbsLrfWVncfrgUA7NestnWK1ZeXIGz0LOgZz4hq9CYN2ytTyn/WH8QV//dl\n9LvXGpXRm0m5skDp7wsKk3ZwOwLgwke/QG2DhUU3JvUQhAAy49qnlpr+ftsLKzH1r58nTcfr27Jq\nuok97k+ls6BnXMFMYPvVmLXYeStxA6N62FBUbXmwMTo9Po9GzUpLN5A1p9U3tcTEofEKvXoTAli+\nJz6orr/Ybddso2dSgqDtjTL4PbRYqROjsiW10ctn4WkaKve+thaT/rwougJaS9jeDjIdFvSM54Rh\nEtHrEAXqLcqkH18bRtcY1ltIhOiibZEIj3qmI8+9mxwm77YTgBBCN0ZQGNo+wIKesYFsJwtHE08k\nJH0vitPyWLncTPDYFc5r9lfZus4JpnXmwT0m44Wv9mLi7IUxoatlYBs94xuHjzWgtqHZk6iOQDA2\n+qCQWYNgVBsJC6acF8cWsgNPKrrGelFmAURj9u+rqJW8QvkU8nj0TJqws6wmusqyY7tsqWvCphFb\ngeCN+UatkpueXY4dD1+K9jmJOlSyags61k14x2OLcwAB3EiWGhDO0NoWbOWyRp/h7Cpv00DM4mo7\nwUxr8UorNOvrXghM7T2qwdwMiSubZT9sa6fH4Oatm3tauZiRDmEz3aj3a3EzK99gQc9Yxmg5dzyp\nrPl7iqxpJO48v0wlbuQS1pcDN+tQe49WBjYOU8ykBFYbqpmN3q9X2jAPOk7L1twi70dvlpfVJxGk\njd6S6cYHU47ajsM6b8GCPsNxImSIwmvXjR9AQlVOw1WSsd9lhcZ4ZT9YtwjzoKhiyXSj+e6ZYsGm\nGybcOGuZcsG8/LfR+03sakfnaZgRlkErSBu9U9wezNTbDYvffDws6DMc19qlTscOkxDXLmjyxsXO\nPM3NB4/ioTfX6/5mvGAqIRMp8mfNM/zN7p3nz5qHX7y7MS6t8JhuYu7ZJdONWT1qEcKaC3EQtcaC\nnnEH02BZ/tvog9Qovc7ba/ODES8v3+9eYg6x63Wj4sYz0qYR8hcYFvSMM8L+iq6H1xPAloOX+aji\nyZkWwvMm5hVu1Lk2jTb3ynDWHQv6DMdOs7Rqjw6b3dIT0024btEXglwEFLYFU202et+zloIFPeMO\nejZ6mUHAN+3RWj4biqzFb7GSuqxHUKIffbgwe3ZhXTClEl++xYXlyJ81TzcwmUwaqnnygTfWY/Kf\nF5lex370jO/YaXTWd9RJPfvOvI0lQRfBEGd7xpqkG3pLsz523hjjL3lt5QEAwOp9R2zlr605vd2r\ngu4CHOuGsYxMv7ri/xZH43+Y4dlkrMQRN3FinpIOIuaTJujK5uDOk4iiV57HFxSaZJ7c7fORj7bj\nrgtObDsezcvmzYd8jGRBz9jGrG1vKrYWrjV0hMBO4olZKwT35TlCRNxorWwG40DlJqLQvw2xoM9w\nnAgTd3Y8ck/yXPr4Ylx6Wn/cM22kbk6Ad2Yku3dx4zPLsLtcJrQtA3inODsKFKcMLPq/6RwLYLRl\nG32GE1YvATtsLTmKRz/ZASB1/OiX7KrAoaP1ur8Zhrz1eqLThTT8HFAT6ikub5mSqJekqeWGBT1j\nH4JzoRD2V15pXAiBMH/TIeyvMN5s2w1NMOjVyn/8cCvufW2tt5nEPQCZO47a6G3Wj9G4FvQkrAoL\neoYJEe+sLYp+NhI5Xg2OXoXw1fLPL3Zj7rqDrqfrFtY2eSfdz3bT8xIW9BmOc43caf6REmw5eBQX\nPfo5auqTb8Ung98uncLgsx5+b4piNf3K2kZvC+EAqeqhWLErZ7ox3yFKIktpUs6PnojuJ6LNRLSJ\niF4log5ENIyIlhNRIRG9TkTt3SosEz7carOFZcdQWHYMK/dWupRiLN4LUC9W24ZEHQwYy7Vgo96c\nqgV29Qq/nrBtQU9EgwDcA6BACHEagGwA1wP4E4DHhBAjARwBcIcbBWW8wS1hEhJTZFIIsit2veWl\nZfvwv/O2WLomleS+Vy9UesnGV8vfF+60nbD9Kg53D3DqXpkDoCMRNQHoBKAEwFQANyq/zwHwWwBP\nOsyHCTl2O4hfk7F+eqrIDJ6/em+TpTS1OFoZaxo6wH668XhV316NddEdouyssoX9+/VreLAt6IUQ\nxUT0FwD7ARwH8DGA1QCqhBDNymlFAAY5LiUTKrSTduHWY9pIFW3YbMVnitwCAHuhBNyEDFZMJWuv\nMqEvnli0E/26dYhJLFm6Rvss+/VMbQt6IuoJ4CoAwwBUAXgTwKU6p+reCxHNBDATAIYMGWK3GEzA\nOG2ovm14HYJJzjDgZn2b3fMTi3a5lo8W55P/BukqCX+2vTxpGo98tB0AMD6/V8L1YcXJZOyFAPYI\nIcqFEE0A3gFwLoAeRKQOIIMB6PpSCSGeFkIUCCEK8vLyHBSDcYKtoGa+R5KxjpOOV6qzgOln72yQ\n3oTbLVGaKoOHn+gvmHJeUY4nY0PXA2JxIuj3A5hARJ0o4ps0DcAWAIsAXKOcMwPAXGdFZLzEjobn\nrr91+Gz0v9Sxn7+64gBWmHgEBb0QSZZMHzyMmoEtxSAmTLH+KcnahV/Dg21BL4RYDuAtAGsAbFTS\nehrATwE8QEQ7AfQG8JwL5WRCwtH6JizbrQg8j1rp3HXFmLfB3TDBUQHnoMzZJtLAymYs0kWIj0ev\nJOzVLkZWU/XLXKHV2GWzNDrPzUVoK/ZE+oEQ1pqVVviH3kYPAEKI3wD4Tdzh3QDGO0mXCS9P2HFd\nM0FP47n3tXUAgMvPuNx2uk7eFITQNwdkZYXj9VwVMEETxNuBnSxlBiSng5bRAr2wmHR4ZWyGY7Wz\ntrRm7rv/I//djopjDb7lZ/Ta72gQs31leNG122s/C2DR9jI89skO/PNz9yeJTVc665Qu5VbGMkwq\nkryf6Z+xYm8lfjVX3/89pvN63JFzssOhJQaB7J3HKyS3vbASjy8ohJGeYhxmWKK1CCA7JG97RrCg\nz3CcahdWPB5eWb4f2w/VxBwLy6utFrNbamz2Tx0z2jPWiVAxu7cwT9QKIfDU57twqFo/pLMZcmYZ\nuTqta2zWPZ6T5JnMWbJPKn2v4I1HMhw/+/bP390IANg7u8327pW3Snzn9nqRl3bA89oDJ+zaoxfs\nrajD7PnbPEtf1kb/D4M5qmRvWa+u2G+1SK7CGj3jiFTb+JtAScWw14Of0ypLpj2ase7AEfxn/UE8\n+dmu0M23rNpbiUXbynR/M5sbcfImcrDqOF5evk968K9v0l9LkZ0VblHKGj3jiHSMsOj1PUlvBm5w\nXGbTdSMOH2vEj16NbPzRr1suvjV2sO203Oaap5YCiH3jU/nJWxs8yfM7zy3HrvJaXH76AN3fZZ9V\nu5C/ZYV7GGI8x7XolTaFT7yNXlucrSVHsXBbqc10zfNJhlG1HDp6HO9qNgeJni9xrWOUdJ1o9Fri\ntdMwL/qqPu7OPgXxuBF7X0Ag28B0ExY9iDX6DMdJO9QKTy+04EsfXwxAX8OzihUhZnbmpuKjuP/1\n9YnXWLh92THRqE5zsjNPP/PcQugw/XZsumHSFQFhS5NvaG5Bbk52NA0tRsntOVyLfRW1lvOySxin\nHtS6cstKEGYNXkskDHAIH4jC0eNN+EAi6qVKEFo+C3rGEXY0+X8s3IkHp59s6ZoL/vKZ5XyMkPWN\ntp1+kt9lzUhG6WRi0DSzGnNjwDKa95BJ+b7X1xlO0oYFFvSZjkFL3ldRi6G9O0snY6ZxCSEwd11x\n9HuY9yQF7AlAa6ahcErYsAj+t1YXeWaTN8IwNo5EpZgJeb1uEcTLCQt6RpfzH/ksqW1cVjPdVV4b\njV9jNw2rmHWmZGYAO/IuNqiZOxLTKBm3aiwkcj2Bh95MnANx4mkU5KI8vWfIIRAY33FLu7Qr3Jzk\nX9vQjEXb9f2uE/JRsolsPORPT6tvatE9vnR3ha303C620+BofsorR9snhmxIC6I8LOgznLC8rtvh\nx2+tx20vrMTew+5O0grhTlf8/Qf6m38fqDwuVw6PBcJ/1uvuCRRKTG30Hs6npHD3iIEFPeMKQXhF\n7C6PCPi6xkTNOf513dfNwZX/+yvqvM3UZSzHo/ekFAZ5BWS68UIRCmJBMgt6xne0fdYN+2m8DKiq\nS5zsTac9Y9NFy5Qlld869Xhm8W7f82RBn+EE3YecmCeMJkAbmxO9IKxsMOVUsCzfXenOPEDCDlOR\n//tcflPYV1GLkmo5c1JMeVwtRZK8ApL2XpjPgnjTY0HPpDyRCdbEY3ax17nbrrnrlTV4P4Xs3+c/\n8hnO+ePCoIsRShyH8XanGI5hQZ/hpPJrcRB7b8pyoLLO8bxA/D3ZvUdZn3QZrVkbD94vG31DcwsO\n2ohDL4PV9u+6hu9Tw2VBz4QKOwNPGDYvCYu/tB7f/H9fuZbWhD8uiH726/b8WHXqpy9BEO6VvGAq\nw3HS6Ox2DgJBCIHCsmOueETI+MZbEbphEdBulUP1TvILvTmSSDmO+VqOZARVHu1zLSyrMT7RRVij\nZ1zBqrh+adk+TH/sCyyPW7RjZfDQTrDquTfqIR85Ur4cRnmGOA6Xp/xaZ1/dytpGTP3r5wGUxpip\nf/08atYyet5eD/rPLN6DeRvkA6LZhQV9huNWQ7aazIaiagBIiEhpTfPW2OiTXGdpALFRJ17E74l/\n20qVTV70Vv7W1MfOE9jx8jHCy2rxw8yyvdR7rZ4FfYaTGqJDn6hGT7EdUk+mx7hiJk3XWq3Eb3ge\nn1+q4GWZ40105/xxoaGJx/W8Q/ZmleAh5kOeLOgZV7DbWOM7ob1OGbdLlc4ZXmpmBypTawWsGzQ0\ntzjec7apxR9Bf1TC68gwJLQHzeZYQ3PMdz8GIhb0jO9oG3Z8R7LUsWIWTMXnERcCwcJQFBb7fJjf\nCr47Z5Xhb2Er9x/nb7N9rRe3sj8AxYAFfabjVkjdALNPeCtwmJ4bNeKFsAuT/FxceNjS+V5qrW68\nrRnNf/gxL+KHezALeiZU2PG6AeRt8GHwuZclTILdC9L9/mRh0w3jOZYjFho0SrtttdUFjSniXikS\njmmxpPUJiztG6dyDF53XC+XSrwnRsGNoo3eabkjsWCzoGUu43W7j/eitYNSJnJtu3DAFOE7CFyFx\n0i/nu55mEDZorwiJnHYMC3rGEWHoCESU1HSTSiabdMNr+7cryQTYjtm9kvGcIAS1WcPeWHRUOh2z\n1bD+xi7xKF2fn41X+f3krQ26x0//7cfeZGiDILcbDL2Nnoh6ENFbRLSNiLYS0TlE1IuIPiGiQuV/\nT7cKy7iPU63KfrwbfVbstb6fKkEi1o020mWSWw7DW4o+4SlYloXn/ubqotAtWpImPFXuCKca/eMA\n/iuEGAVgDICtAGYBWCCEGAlggfKdCSlO2rGTvmuU71c7EwW90SbbVla7pkuHtcqnW0o9STcnK72M\nAYaxbnxoOH5sw2n7aRFRNwCTATwHAEKIRiFEFYCrAMxRTpsD4BtOC8mEFz+035r6ZtPf9fqJWddJ\n1q+s3lJ43wCA99YVe5JumDT0sK57CBNOhuXhAMoBvEBEa4noWSLqDKCfEKIEAJT/fV0oZ1rz300l\nyJ81D2U17m+usKO0Bvmz5mH1viOup33d08vw9wWFAID31x/EpuJqqevmLN2HjzYfcpT301/sivHu\nkO2oertRxSOECE3Hv+zxxbj2qSW2r7e20Fj+bKuCPojN490gLO3AKU4EfQ6AsQCeFEKcBaAWFsw0\nRDSTiFYR0ary8nIHxUh9/r1sPwD94FhO+Xx7pG7nb9QPheq0Ie8+3BZ98tOt8maCZFp6Mv72aWHs\ngWTC21FuwSAEsKXkKFbuPRL9HhbSzYvJaJCLP6r3DIbndXa/QC7jRNAXASgSQixXvr+FiOAvJaIB\nAKD8L9O7WAjxtBCiQAhRkJeX56AY4aeythH5s+YZ2kuDnPF3kyAFUcKCKQca5Jr9VfjAUozw9Hh+\nstQ3tYTKdOMlMs4KWSaVIdMyQu11I4Q4BOAAEZ2sHJoGYAuA9wHMUI7NADDXUQnTgK0lEZfB57/a\nY3peumlJfiJjjrHC4WMNDkrjDq4oAB6MQVV1TZZbathbdpBKih/93ulWgj8C8DIRtQewG8BtiAwe\nbxDRHQD2A7jWYR4pT5heueN55KPtls4PiyYX43EjMk2n9gYr7TRMNnc3Fl4ZpfC9f8VG6XxxyV6p\n9OauK8ZVZw5yVigXcSTohRDrABTo/DTNSbrpSoj6RpTjBq6LRoR50JLH3Zvwqk4SQjjbScOjITBM\nTdnLJrlmf5Wt62bP3yYt6ENtumHkSdbZ0kN4+kvMAigkanXxnUf91U8BFQat10rb2nxQflWy1Yq8\n9qml1i7wGSd9MCyBy8xgQe8jRrY4tZ2EQC6kLNa6mj8V7VQAyHh8uMmLS8znkLRYrcHiKvf2iE03\nONYN4wmbiqsx+c+LUC2xxRoAtLYKXPmPL/HR5kOhHIys+727bLpxNTV3mb9Jfr2C6sYpQ5aVGAge\nE7RCvau8Vvf4s4t3Y7fBb1rYdJMmBN0Q43nskx3YX1mHlZIhguuaWrChqBr3v74uNPeSaL9OEo/e\n7wBhCIfpxitCdWchDAlNAB6et9XVNJ3g1OuGsYBRv1eFlN+dR7ZpNymbU7TLDo9eoN2wxEoX/fvC\nneicmxrNPsy233QexNzAypPjrQTThPB2VzmaWtoEfVj6d1OLNZcU7c9ONovWTVsnby+qKUwL60LS\nDACEq17swKYbJimHquvxneeWS9vbrfDqikhohkZF0LfPptCYbrSE0Y8+xffCSEpYBnzApd28nCcR\nQ4iqBwAL+nBho3U8sWgnFhcexlwPohT+7J2NAIBmRXvOCZHpJp4wDkDpTdhEmTPCbCZzg/D23DQi\nDI3ISQnaTDdh7dwi8fU9TuX08hnomQ7cqKkQNBtDQqXRB12AFIAFvY8YTWC5s5G0UZ5yx8xoDKGN\n3ip+CwMv8guT4E/RZmBIkFXrx8R2argfhJyF20pxsKoeN08Y6igdO7PvSTfR0GnBVgWGarppn5Nl\nem1QneXn725C8ZF0XJATIskeR6oO+JkKC3oXuP3FSOAju4I+vN05glajDyMrdNYD+CmH/NK0w9RO\nwhRp1ZWgZgFWrh8W0XD23DRDtg3JaknPLt6N9Qfkgi25oXmpfvQ5IVoNmQo4ra1NxRZiz/hMmDT6\nRz/Z4TgN171uLFRQuxzvxTBr9D7iVt9QV9ztnX150nPd0FQaVPfKnGBs9K7kGSZ1WJL7Xl8XdBEM\nCZGcx8vL9wddBEe082GjdRb0YcDnJdxWBaeq0S8uPIyendpbu9gFrA5WQvircabgGOIIovRbGfvF\njuC2M22Xwytj04Mgd69xoQ21tLbdwPvrDzpPkLFFGNx0Gfcx24rQtTw8z4GJEoQSFMZVg16jV8+p\nvkw+bKSZQu86YQvLzILeB5JuPOIgqFmo+ltINE4hEr1CvCxapmnafpvGGOewoPcRL/uGkagx65CH\njtbLpR2wHFuy63CwBWASCJN7ZarjR/9iQZ/m6DWiqrpIALRfvbdJLo2AzR73vhZe75NMJDIZG3Qp\n0gc/+hcLeh+QHbGdeDIYXfnfzYk7DK3aJ7+TkBVeWREONzcBwYKISRlYo08z3HZJU4ONAcamm9Vx\nQn3zwWrL+cg2xMPHGi2n7RdBmJ/cHmyCNqFp4XE0tWBBr0Nrq8Bbq4vQrBGkTkjWQZ10YKuDR31T\ni+U8QiRfbFNSHS4viFQmMhnLot4tWKMPiLdWF+GhN9fj2S/3+Jov9x130Os464usv8k4yY9hZPGj\n+bCg1+FIXcQEUVnrjili9X5zm7jRgz5a34SF20oNr1MnVa1gRyhlmvugVRZtL0s4JoS7A8CnW0px\nrKHZvQQd4KZCsrPsmHuJMYawoPeBJz/bBSC5XTP+93tfXYvbX1xlaHa4/cWVpullsoD269bLauox\nd533q4W/+69V+KWkl1QqceGjnwddhMDxo5+yoNfBq2q3qgntOVwLADjeqG9X31Xepg3Jxp3PBNHv\n5z3WN+rP4xABa5K8yaUqJdX10bbJOIdNNxmC3RE96SSvrVSt58MkIgRQUx8OU4vbfOOJr4IuAmMR\nFvQ+srWkBoePNVi+zq6Hg94AYlVoHz7WEPiCKTv4VWaeQHeOEAJLdmbw6mf2ukkviquO+2qTdKP9\npKINNQxzEzwAyPPR5lLc+OzyoIsRGLwyNg3R85RRH7NV4aBtIHpNRd9uH3vwrCE9TPOoqmtKOdPN\nrvJa7DhUE2gZUq3OgiRskR7TEd54JAWwq6G2uiRtUk1ozXh+hW95pVrdhJEwvIEFSUosmCKibCJa\nS0QfKN+HEdFyIiokoteJyP8tiTIEbQPRvgxsOXjUsPPEH91yMLz7kqYy20u5Xhk5UsXr5l4AWzXf\n/wTgMSHESABHANzhQh4Zgr7tRmYyVm0sK/dW4rK/L8ZzX+6R0hQampOHechsfcseH25MDCbHMHqE\nXqMnosEALgfwrPKdAEwF8JZyyhwA33CSRyZg90ELJNr191fUAQA2HzyKUsl480nzyfBXazP2VbI/\nuVPS1Q3OqaNmAAAgAElEQVQ1TDjV6P8G4CcAVLWwN4AqIYT65IoADNK7kIhmEtEqIlpVXh7cxrxh\nwm1PjSl/+SzhGMtsd/nOc/7NB6Qrjy8oDLoIgRJqrxsiugJAmRBitfawzqm6dyGEeFoIUSCEKMjL\ny7NbjIzASKNulDC7xNPQnJnRKxkmrPihfDnxupkI4EoiugxABwDdENHwexBRjqLVDwbgfSAQj/DL\nFdrJc/6yUH+hiVHZb33BPD6OLizpGSalsa3RCyF+JoQYLITIB3A9gIVCiJsALAJwjXLaDABzHZfS\nIhU2Vp+6QVNLK4qrjtvStIFE4Vx9PHl0ykIl+l+81s+ymWFSg2mn9PU8Dy8WTP0UwANEtBMRm/1z\nHuRhyPyNJRj38KdYsafSz2wBAPe8uhYTZy/EzS6s8qtvasERG2GIjXDyepiKIRAYJlUY0L2j53m4\nsmBKCPEZgM+Uz7sBjHcjXTssVwT8puJqjB/Wy9e852+KuNSt2GtxkNGRwtqIlUHv5sMTuAyT2nAI\nBB3SUbA5GSs2Fnu3OxPDMN7Dgj6kWBXMrUKguaUVLcooFX+5k8FrEwt6hklpONZNCGgLatYmnkkj\nqmUWLP3hw234w4fb3C5aJH9PUmUYxi9Yo88QOGwuw2QuLOhDxr6KWuTPmof/bi6JHtNq+l7tRmXG\nhiI23TBMKsOCPkQIIbBZiSbpx4bTDMNkBizoQ4BW285StPeWVnct42y6YZjMhQW9GQEIxywlT+2m\nISLms2RCcWVPR5dRhmHkYEEfIgSAnOyIhG52qtGzYGcYRoEFfQjQhhhIZrqxK7/ZdMMwmQsLeg+p\nPt6E/FnzLF2TrdhumlscquRsumEYRoEFvYfsLKuxdL4QGkHf2hYBM+hYNwzDpDYs6D1EVovWnpdN\niTZ6YTAxyzAMIwMLeg+xPp8qkKVo9K0uu1cyDJO5pIWg/3BjCaY/9rnvwvHHb67HHz7cavi7He27\nzXTj7mQswzCZS1oENXvgjXWob2pFvY39UJ3w5uoi099lxx29BVOs0TMM4xZpodGrgrKqrgkvLtnr\nPD2X9Gar6WgFvvbKq59cEt0bVvolgccJhmEU0kPQK/8/2nwo0HIkYEPYqv41Wj/6I3VNeOjN9V5n\nzTBMmpIWgj6sSJtuHFxrxLtri50lwDBM2pAegj6k6qtl0w20t2I0GRvSm2UYJrSkhaBXhZ/bLub/\n/Hy3o+udaOXx1/KaKYZh7JIegl4RimHTda26VwrRdk1r3LXRewzbTTIME3rSQtCrhG3VqLSDjEg0\n2FTVNbleHoZhMpO0EPThEu9thG3gYRgmM0kLQR9v5vCanWU1ePTj7UnPs1qsViEwe/42/bQcDmfP\nf7XH0fUMw6QuabEy1m+uf3o5Dh9rSHqe1cnY3eW1WLGn0mapGIZh9EkLjV7VnLWafbyXytJdFXhn\njXnIAlkaJEItfLDhoGXTTYsmNLERbA3yhstO748rzhgQdDEYxhPSQtCrxIQQiBOINzyzDA+8Ibe6\nNKkwlRC2d7+y1rKxhcPbBAeBOO4/k7aklaC3wtJdFbZDJsjOCZhp9KVH6/HU57sghNB9I0lMS/kf\n2qnnFIcC2QueYXwhYwX9Dc8sw/dfWm3rWnm3SePf7nx5DWbP34adZceix1ijD447zhuGLJb0TJqS\nVoJeT07WNjTjn5/vSn6tEHjuyz04Utsoca5ceRqajW3ux+qbAUTizm8vrYmWwa289fhse5n9i9Oc\nEX26sOkmRbj0tP5BFyHlSCuvGz3Tx+z52/DSsn1Jr12zvwr/88EWLNtdgTNP6GE5Hz2e/iJ5CIW1\n+6ssp2uXW19Y6Wn6KQ2bblKGLB6QLWNboyeiE4hoERFtJaLNRHSvcrwXEX1CRIXK/57uFdc6NfVy\nK0xVT5qjx5OfLyuOjzU0Jz2nqaVN6zeT82U1Dfhq52HJnBmrEPEm7KkCPybrODHdNAN4UAhxCoAJ\nAO4iotEAZgFYIIQYCWCB8t13LDcGIX+drNuk2cSp+luWxjCczEZ/07PLeSrWIwjALecMDboYjATZ\nPJliGduCXghRIoRYo3yuAbAVwCAAVwGYo5w2B8A3nBZSvkxtn+dvOoQPN5bIX6v8J4kXeFkLi8x5\n2aQV9CzGg4KIMCaJyY4JB36bbm6bmO9rfl7gio2eiPIBnAVgOYB+QogSIDIYEFFfg2tmApgJAEOG\nDHGjGDGa9oo9lVixpxJTTs6TvFYtV/Jz5d0rk5+jVU5k9onl+DlMpuO36SYdupxjrxsi6gLgbQD3\nCSGOyl4nhHhaCFEghCjIy5MTxnaolbCTA22mlPhG9N7a4pht/QB5N0iZAUGrnXywQf4NhHEXNgak\nDn5r9Onwpu1I0BNRO0SE/MtCiHeUw6VENED5fQCAQH36Vu49InVeVKOP6/L3vb4OLy9P7rVjlqYp\nmuxUN0vTNG2VhEkGT/ClDn6b6DNa0FPEReE5AFuFEI9qfnofwAzl8wwAc+0XzxofbS61fW3URq/T\niA4fa/OtX7hNPo9DR+uTnsPyJRzIzM0w4cBvjb4leQiq0ONEo58I4DsAphLROuXvMgCzAVxERIUA\nLlK++8LG4mrb18rYvtcdqMLtL66ynYce7EEQDlijTx3M3GBPG9TN9fzSYV7MidfNl0IIEkKcIYQ4\nU/n7UAhRIYSYJoQYqfwPddzdrSVH8cbKA1Ezi5m2cKQu+apZWXaURkIfWBX0adDmAuUfN54VdBE8\no3fn9kEXwRXunDLC9HezLpPfu3PS9CeN7GOpPOlgukmrlbF2uPTxxQAiYWqBJJpd6j/vUDO4Z0cU\nHTkeSN7poNFnpcnbYbJu1qWDsdiS6aLxzhXJkDm9a4cc1NTLOX4EQVrFunFCsSJg9LrK0l2Hpb13\nZDhQWRf9bLXReT3Y3DzBHVdXLYN6dJQ6b/69k1zPW5Z0sNHnWBD0L3/37JR9A+iQky197k1nJ7Zn\n64I++fk/v+wUS2n6DQt6hbbJ2MTOsnLvEdz1yhrXQgRP+vOi6OfmNA1ZOfkk6y6zfkyyGQn0dNDo\nc7LlbyK/T+eUfQNw2k6smmJkTg97TWacoC+uOq67DWCbe6U+a/dXQWIDKMuUVif3zNHidTx6s0b9\n8DdOk07nhVu/Fv0sK4CCFLZh76gy5GTJdeftD1+CQT06xqzKVunTJfxavtN2YqTRn9i3i+5x7cDw\n0PSTpMt04Sn9rBfOIzJO0E+cvRAFD39q+LtRI8oib6wmf/1khwepekNujnxz0U4y6wmUeIiCNZ+k\nQ0Czbh3bSZ2Xq5g+9BwBUiEypGkJ4zrpxBMTJ16NBL2R6WvyyLa30xF5+oOBXtu10l+8JjwlCRih\niXajx5G6prRws0qGlTss/N9LTX8/oVfENi8jO9pnZ3mm0XvhcifL5af7tw9tl9xsdGovb7/2Q6b/\n+orRjtO4IC6MiUy5H7tuDFb8fBou06n/Fov9+OpxgzGkVycAwOCenTBmcPfEkyTrsnfn9lj7q4ui\n39f9+iKTs92DBb2CTKybMIh5r8ca2fTzuuaiXbZ581F/l3EhjYQJlsvbKlNP1g23FJu/N1lLa9lu\nkEWELrnyjnR+rOHoaGHgUTkrLrhcp7h7Mnr7ys6iqMLWLjsLfbt10D3PzgKoroqnDxFwxRkDpa5R\nixlfzT01k+Ad2lmvHzuwoI/DrOlXS8SqzxRkZIQapE3GHJBF5Jnp5ppxJ+geX/+b6dHPRkXc/LuL\n8dTNYx2X4XdXnuo4jWRYNT/ZNdOoXlT9DQRpTJl0jo3q31X33AHdO2DFL6Zh+qnWdpD6fzeNxZc/\nvQAbtM/TpC3JBA8047uThmHlLy7Elt9frMkvEdUUNPvqM9rOC8gylvKCvqzG2mSmEeqzN2v8P3lr\ngyt5OcH7twq5HGSEhPqKLKM5Th/dz7MYJl01ftfaYnfXaNtGQrJzbg46tne+3MSKR4xdCNYEiV59\ny1yvTtjKmIn02kmugRZLAPp2TT54JKSXk4XBPTuhc26O1BupkenGzFNMewkRIa9rLjq1z4k5ZkQ7\nH559MlJe0FdK7PEqg2p/D/tc1JaD0gFCbaPVdI0wqyY1NLTqpZTMv3vpz6biz9eM8WxCtFvHdvjo\nvslY/csLPUk/LGSZTGjH27kB/QG4e5yp6dMHJuOXl8f5iKvPyeRxjc/vZZiHkUZt9/lbdRPVm4xd\nMmsqfnrJKMP+n2z8MF1nGXNx7Jl+TdimvKB3+3U/7IL+5ueWe5q+EImdXQ+zTtmzU0TjUzvU2KHm\nu0kO6N4R7XOyPI1KeHL/rujdJdfWHEeqTMKbPZN4Ozegr21PHRXrEji4ZycMz4sNK6A+J7O3urxu\nuQD032SMvF7O0/GQ0WL0BjGwe9uCPJn+q5f/wB4dkZ1FSd9UjX7WO67uPT20dyeMUOqwe0e5+Qa3\nSX1B71I9qcuXjzW0oMrFmDaphqxMM3PZVvfBVV+Rvz/ZPHaJSnyjn3hib3z6wOSE88YlGTjcYuUv\nIm8Aboh5r8YKrQdHlsUJbT2h9pOLT4753qFddsJ5qrus6cBsYgo1WrD0P0nWaahpad8IPn3gfJys\nsfmb1fPrMycAMF8Za1ec6NX7jHPz8ekDkzFuaC/8Vpmj6d0l12YOzkh5Qe8WxVWREAhf7CjHM4v3\nBFya4JBdkGWm+TS3RNIQFmz0ehAIJ/ZNnLg70cCXWSpNC0XJ6xpMp7SC1oODyNr77dfyYwfMi0/t\np2sGiV+INahnRIM+fVDyrRf16ntAd307fHsjM4bSJNV2pLWxGy1y0sv3BMVF0lTQG5lukozUg3t2\n0kmrrf3Ge6j18jn8RMoHNVOFChMemhXjvNqhbAt6o8VrFtUTK7l/u2Aw3lhVFHvQhSbmxxt6RKOX\nz+inl47CnKX7AAAf3z856iuekG5cfZ95Qg98b9JwdGqfjbfXFOleo6InH888oScWbS+XLqeKOtdj\n5jVjJo/1Bop4IvVnpvHr1+/XlDkJWRY+eD6OHvcvCFrKa/TNXsQlYBzRqAy+qo3S7oSTcUwS76Sm\n3krKVAlTq33L6tEp+TyL1mvkpH5dDX268+LMDQTgtEHdzUMumDwiqwuW1LSyJAS1asbp1y3xbUxr\n+jEK6JasZTkesJWi9+jUHkN66w+sXpDygv6Yi1ElM5Xnby2Ifjbrg7Lds1mx0f/jxrH44EfnobOF\nRTzz752EZ26JlMco7KuTzpbs0ivHJC6G8Wv9xA/OH4GnvzMu5tic28fj7R+em9R+DcQK+pe/e7Zr\n5RrZryveufNcXP+1yHoEVeDaDelt149d1ejNFjzdM20k3v7hORg3NFHDztEMFJ88cL7u9Ubt3+lY\nH7SPR0oLeiEEbnzGWy+UTKCfxMKXeEbG2UbHDmmz16peGp1zc3DaIJ3l4iacMqBb1IRQVacvYJ14\n5yR1k1Okl9aObGW1aTxDFa2tn4R/+En9uqBHp1hNc8zg7hg3tCe65EqsoCTg1IGRcA/x/uhOBc3Y\nIT2jcV7UhVJ2B1zLGr2Canoxe8PKziJdIQ8Aue0i4m70gG6GNnKjso0aEHlTcNIWgiSlBX1DM5tt\n3MaoC00fHet296u4GCYv3DY++vmXlxvHN3n3znOTlkGdlGs2UN2sutRa9bP+z93n4YMfnRf9ftHo\nfphz+3iTK4z53qTh+Nft43HhaHuRDFW5YyYbf37ZKAARjf6x687Emz84R3oiefFPLsCCB/W123ju\nOG8YXrpjPC5S7kU7H/C/34y8cYzq3xWf/3hKzMjy4T2T8L1Jw6LfW1sFvvjxBdHvH98/GV/NmmqY\nb2vcXI/lPRwUOrXPwds/PAdPKW9NX82aimduKYiUV8Fo0nX2t87A6zMnRCd0U43UHJ4Ujje2BF2E\ntMNIoJw7onfM95w4LwKt771Z/I6zhrR5evTrlovSo4kho1X3PSPtyuuJzdPjglYREc63EV8fiAgn\n2dj8elqmul+BmZfGKQMiWvyQXh3ROTfH0sSgFcGVlUWYpInkqI6f/bt1iNq8h/buhKFx2/mNHtgN\nhWVtddrSKjCkdyeM6t8V2w7VYGTfLqaTyE0t7gh6ADHa/qAeHRM2xTFKumP7bJw9vLf+jwqd2mej\nLolM8jrMuBEprdEfb2JB7wZaDdmoId5yTn6MHu00XggA/OdH5+GtH5yTcFyd4zPKQk8kaOPfx+Rx\n93kx39Vrh/WJCKMP75mEzx6aIlHaRLSrTWWE+X/uPg8f3Ze4LkBFbzBRzRTnn5QXnbuIZ9LIyG/3\nXagfK90ropObmgHZSFG4csxAjB8WEbLqAP7K9ybgtZkTknoKtSgOFzkuCHovWfjgFLz9Q/031qDD\nYKe0oH80BWK5j8hLvllxKpCVRY50ET3Pm75dO6BAR/tUNTf1NTp+ZaZeVMILRiVGqByf3ytBO1ff\nPFShOnpgN+T3sfeMTtfMP1x+evJAXKcP7o7BPY23VSQidI6zxWdHJz4pajLR46LR/ZJGE3UbVXRF\n5K7yzJRjA5U5DrW+iQiXnhapo1bNW8qEJFoy0FYHWZo3PSvbJvpF/+4dDBfzqc9Vz9/eD1LadBO+\nR52I353PMTrSXK9PWXU5XPjQFOyrqJU6N2q6UQTC6zPPwXX/XIrdh2tx5gk98P3Jw1F05DheXbHf\nPCGdcp8zojf+74azTIWmEW//8Fz07NQO97++DuuLqmO0tGvHnYAO7bLx8LytKK9JNEfJcurA7njq\n5rE4Y3APbC05ij4uraT0QqNsS1NE24jaLB66+GScMbgHJo1sc1dtm0yVz+Opm8fh8x3lsde3Cnz2\n4ymBbSRvh1MHdseTN421tcWmG6SYFIrFjreI33gl6O0IKrvo+ZbH2+iTMahHR5w7wjyWiUq8LTav\nay6+edYgAMDNE4YiJzsL009Nfv99dSYkiQhfHzPQVhzwcUN7Ynhel2hnzSKKCuKsLMJVZw5COxc0\nzUtOG4CBPTpimotb0XmxRaAq51tFm9BX38Jyc7Lx9TEDYwaYrCRzL1pU2/n00f2iZVfnKVpaBQb3\n7CT1NmCX9hbat+wjv/T0AZZcjd0kpTX6H007Ef9YtDPoYphiNTztvHvOwzNf7MZ76w6anveXa8dg\nzO8+dlK0KHrK3kf3Tcb764vxxKJdutpg/CQWEPHd7m+wvN0KeprfD6aMQN9uufiWIvCnnJSHP19z\nhmno6J9fdorhb05Q32ayCHj/7okxEUWTac5OrcvP31qA219cJXXuh/dMQl1jM3YfrsWVYwbiha/2\nSl332swJuP7pZUnPy9IId5lWHh3AJVazv3PnuVi7vwpZWYQfTR2JwT07goiwuPCwbfdMK7TPycJ/\n75uEg1XmYdBfvO1rGN7HfkgOv0hpjV7d+zLMWNXoTx3YHX+7/qyk58lEmLSD2oVO7t9V135uxsQT\n+xjuqRlPT5OVm1k6/tLtsrNw3deGaBbrEL5doL+hiEpHj3bv6dohUvbundphYI+OMa6TySwkTiNh\nxkeXNGP0wG4oyO+FbxecYOkNRlZT1tro1fs2u7tk3lRa+nXrgEsUm377nMizV+O6u+EIYIQaIXNw\nz44YntcF5400fwudcnJfX1e42iWlNfowc2LfLvjW2EHYX1GHFXsqpa7ReqC8cOvX0NDcgh/8ew2A\nSDiBdQeqYs7v3bk9KlyKx68SI4jU7RXjzjlDb89MC/zjxrMwZrBxQKxsHW8OGT65fzL2V9bhjjly\nGq9dbp84DB1ysnDj+CEJv6la7m++PlrX5OWmiLK6+vX5WwukJwP/+Z1xhgHDVGI0+qgZx/gOs7Kc\nCWo1v2YPBf0nD5yPX767EX/S7AqVDqS0Ru8HJ/Wz91p2Qs+OuHPKiWi0sEGlVoO+YFRfXHJa28bG\nZw3pkeC5ctPZiYLGKSLms/5mLCf1098KTpYrzhho6r/dNulmLd2R/bpi2in90E2zv6cXtM/Jwq0T\nh+nOU6j22ikn99WtJyM5GO9ZJIPeQGLG1FH9pJ/dxaf2T/p2lq1o2D07t4+66JoHFYv8txs7SI3N\n4+Xq1EE9OuKF28Yb7jebqrBGnwQ908vztxbg2cV7sGRXRczxC07Oi0blU221sqt3X7pDf+XlpJF9\nsLjwMISIlKWhuTWqyd1ybj5aBWLmKTq3z8b1iqY5bmhP3Ply5I3gm2cNwrtri/HvO862vHmJ395N\nMlEGVebeNRFlcV4u6lVe7UFrBiV5G3FjE5MnbxqbsCgpCLrk5uCP3zodk0/Kw47SGgDmbyxtk7H2\n8ps2qi9+ftko3Hj2UEvX/fuOs33ZyjHMsKBPQs9Oid4Ko/p3w91TT4wK+q4dclBT34xpp/SLCnp1\ndj1X0kZv5JFy+ekDsLjwMHJzstC7S3sca2iOxjPp0yUXD118coygv/jU/jHhCS48pS8+3VqGG8YP\nwWPXnambh3ZPyxjLTUDrUlTTjYzmNuaERBNQtw7tUFPfDArgfTVqqzZc7BU5ITuLYhb+dLWgpV56\n+oDkJ/nEDYpSse9wxHW2k8lcQLZT000WYabkJjZaktnZMwEW9HFcM24w3lrdFmP791ediql//Tzh\nPK0Z5b27JmLJrgp8u+AElB2tR3OriO6q9Ouvj8awPp2xZFcFlu6uSEhHxShm+9XjBqOkuh7fP384\nbjx7CD7ZUpoQ+Orhb5yGs4f1wvxNhzDjnPyY3/509Rl4beWBhE0mAOCN75+DytoGjMjrgtEDumFL\nif5+tGaeJM/NKDDeMMImWVmE/7nqVMumCZVXvzcBC7eVolsHbyaszdBbLaqle6d2+M3XR2P0gG64\n7ull6N+tA24/Lx9XjhnkZzFdZ8Lw3nho+km4yUTbjmr0IV3Z6haPfnsMRupsmBMkGSHoc7LIcALn\n+5OH459f7I5+/8u1Y2IEvTbGyPC8zthdXguitngu4/N7YURel6g984HpsVux9ejUHj+aNhJr4yZS\nZWmXnYX7L4osbR/aOwffnTQ84ZybJ0Q610gd+2vvLrm464ITddNWl6QDwF0XnIi7XlkTY6pSJ8/U\nQU3V/LXnuOnrreU7cQOWFYb07oRbJw5LfqIHdDbY11TLbROH4VB1xG2vY/tsW1pq2MjKItw9daTp\nOapverqbUb41dnDQRUjAE0FPRJcAeBxANoBnhRCzvcgHiMTrPnq8CUt2VeDVFfsxbmhP5GQR1uw/\ngqYWgXumnojrxg/BxNkL8ddrx+DBN9fjjvOGQQigvrkFP5wyArk5WWhoacVoJTjUS3eMx5p9VWiX\nQ+jRqT3m3jURG4urMXlkHt5fX4z+3Tqgf7cOuHfaSFz3NXMXP5U/XX0Gfv/BFiwuLMcztxTg062l\n+GjTIZx7Yp+oKcYv/u+GsxLcMy8+tR9+cP4I/OD8toFk8sg83DllBO44LyI0rzhjILYdqsGdU/QH\nDgZ48uZxeGPVgYQwzvH065aLh6aflPKavBUuGNUXP5wyAt/TUVYYbyG3d7gnomwAOwBcBKAIwEoA\nNwghthhdU1BQIFat8tYljmEYJt0gotVCCP1odxq8mK4aD2CnEGK3EKIRwGsArvIgH4ZhGEYCLwT9\nIAAHNN+LlGMMwzBMAHgh6PVmWhLsQ0Q0k4hWEdGq8nLrO8IzDMMwcngh6IsAaGcoBwNIiNAlhHha\nCFEghCjIywsmdCfDMEwm4IWgXwlgJBENI6L2AK4H8L4H+TAMwzASuO5eKYRoJqK7AXyEiHvl80KI\nzW7nwzAMw8jhiR+9EOJDAB96kTbDMAxjDY5eyTAMk+a4vmDKViGIygHss3l5HwCHXSyOW3C5rBHW\ncgHhLRuXyxrpWK6hQoik3iyhEPROIKJVMivD/IbLZY2wlgsIb9m4XNbI5HKx6YZhGCbNYUHPMAyT\n5qSDoH866AIYwOWyRljLBYS3bFwua2RsuVLeRs8wDMOYkw4aPcMwDGNCSgt6IrqEiLYT0U4imuVz\n3icQ0SIi2kpEm4noXuX4b4momIjWKX+Xaa75mVLW7UR0sYdl20tEG5X8VynHehHRJ0RUqPzvqRwn\nIvq7Uq4NRDTWozKdrKmTdUR0lIjuC6K+iOh5Iiojok2aY5brh4hmKOcXEtEMj8r1CBFtU/J+l4h6\nKMfziei4pt6e0lwzTnn+O5WyO9rSyaBclp+b2/3VoFyva8q0l4jWKcf9rC8j2RBcGxNCpOQfIuEV\ndgEYDqA9gPUARvuY/wAAY5XPXRHZbGU0gN8CeEjn/NFKGXMBDFPKnu1R2fYC6BN37M8AZimfZwH4\nk/L5MgDzEYk6OgHAcp+e3SEAQ4OoLwCTAYwFsMlu/QDoBWC38r+n8rmnB+WaDiBH+fwnTbnytefF\npbMCwDlKmecDuNSDcll6bl70V71yxf3+VwC/DqC+jGRDYG0slTX6QDc4EUKUCCHWKJ9rAGyFedz9\nqwC8JoRoEELsAbATkXvwi6sAzFE+zwHwDc3xf4kIywD0IKIBHpdlGoBdQgizRXKe1ZcQ4gsAlTr5\nWamfiwF8IoSoFEIcAfAJgEvcLpcQ4mMhRLPydRki0WANUcrWTQixVESkxb809+JauUwwem6u91ez\ncila+bcBvGqWhkf1ZSQbAmtjqSzoQ7PBCRHlAzgLwHLl0N3KK9jz6usZ/C2vAPAxEa0mopnKsX5C\niBIg0hAB9A2gXCrXI7YDBl1fgPX6CaLebkdE81MZRkRriehzIpqkHBuklMWPcll5bn7X1yQApUKI\nQs0x3+srTjYE1sZSWdBLbXDieSGIugB4G8B9QoijAJ4EMALAmQBKEHl9BPwt70QhxFgAlwK4i4gm\nm5zraz1SJHT1lQDeVA6Fob7MMCqH3/X2CwDNAF5WDpUAGCKEOAvAAwBeIaJuPpbL6nPz+3negFhl\nwvf60pENhqcalMG1sqWyoJfa4MRLiKgdIg/yZSHEOwAghCgVQrQIIVoBPIM2c4Nv5RVCHFT+lwF4\nVylDqWqSUf6X+V0uhUsBrBFClCplDLy+FKzWj2/lUybhrgBwk2JegGIaqVA+r0bE/n2SUi6teceT\ncnSwkTcAAAGoSURBVNl4bn7WVw6AbwF4XVNeX+tLTzYgwDaWyoI+0A1OFBvgcwC2CiEe1RzX2re/\nCUD1CHgfwPVElEtEwwCMRGQSyO1ydSairupnRCbzNin5q7P2MwDM1ZTrFmXmfwKAavX10iNiNK2g\n60uD1fr5CMB0IuqpmC2mK8dchYguAfBTAFcKIeo0x/OIKFv5PByR+tmtlK2GiCYobfQWzb24WS6r\nz83P/nohgG1CiKhJxs/6MpINCLKNOZldDvoPkdnqHYiMzr/wOe/zEHmN2gBgnfJ3GYCXAGxUjr8P\nYIDmml8oZd0OhzP7JuUajohHw3oAm9V6AdAbwAIAhcr/XspxAvCEUq6NAAo8rLNOACoAdNcc872+\nEBloSgA0IaI13WGnfhCxme9U/m7zqFw7EbHTqm3sKeXcq5Xnux7AGgBf16RTgIjg3QXgH1AWRrpc\nLsvPze3+qlcu5fiLAH4Qd66f9WUkGwJrY7wylmEYJs1JZdMNwzAMIwELeoZhmDSHBT3DMEyaw4Ke\nYRgmzWFBzzAMk+awoGcYhklzWNAzDMOkOSzoGYZh0pz/D3G0JOu6rXHhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x27ee78b2630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(jList)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the network learns to solve the FrozenLake problem, it turns out it doesn’t do so quite as efficiently as the Q-Table. While neural networks allow for greater flexibility, they do so at the cost of stability when it comes to Q-Learning."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}