Temperature and it's effects.

temperature.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn.functional as F\n",
    "import torch\n",
    "from torch import nn, optim\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Activations\n",
    "\n",
    "Our activations typically include positive and negative values, we're generating these using `torch.randn`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([-1.0198, -0.0233, -0.6869,  1.4886,  0.6051, -0.2313,  0.7285,\n",
      "         0.7995, -0.7978,  0.5780])\n"
     ]
    }
   ],
   "source": [
    "n = 10\n",
    "acts = torch.randn(n)\n",
    "print(acts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probabilites\n",
    "\n",
    "I generate probabilities using softmax on the activations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "probs = F.softmax(acts, dim=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helpers\n",
    "\n",
    "To see how the distribution changes, matplotlib and corresponding helpers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADNRJREFUeJzt3X+s3Xddx/Hni5aJDGQmuybaVtbEwmyIZvNmTpcocZh0YNo/JGZNhkom/YciKtEUNdPMf0AM/kgqWgFRROacxDRSrYnMmBi39I4h0taZa5nrLTO7DEQj0dL49o97ao6Xtud713Pvad/3+UiWnO/3fHK+77PdPfvt9/y4qSokSb28aNYDSJKmz7hLUkPGXZIaMu6S1JBxl6SGjLskNTQx7kk+mOS5JJ+5zP1J8htJFpN8Osnt0x9TkrQWQ87cPwTsucL99wC7Rv8cAN539WNJkq7GxLhX1d8AX7jCkn3A79eKx4CbknzjtAaUJK3d1ik8xjbg7Nj20mjfs6sXJjnAytk9N95443fceuutUzi8JG0eTzzxxOeram7SumnEfbCqOgIcAZifn6+FhYWNPLwkXfeS/MuQddN4t8w5YMfY9vbRPknSjEwj7keBHx69a+ZO4EtV9VWXZCRJG2fiZZkkHwVeC9ycZAn4BeDFAFX1W8Ax4PXAIvBl4M3rNawkaZiJca+q/RPuL+CtU5tIknTV/ISqJDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDQ2Ke5I9SZ5Kspjk0CXu/+YkjyZ5Msmnk7x++qNKkoaaGPckW4DDwD3AbmB/kt2rlv088HBV3QbcC/zmtAeVJA035Mz9DmCxqs5U1XngIWDfqjUFfN3o9iuAz01vREnSWg2J+zbg7Nj20mjfuF8E7kuyBBwD3napB0pyIMlCkoXl5eUXMK4kaYhpvaC6H/hQVW0HXg98OMlXPXZVHamq+aqan5ubm9KhJUmrDYn7OWDH2Pb20b5x9wMPA1TV3wEvAW6exoCSpLUbEvcTwK4kO5PcwMoLpkdXrXkGuBsgybeyEnevu0jSjEyMe1VdAA4Cx4HTrLwr5mSSB5PsHS17B/CWJH8PfBT40aqq9RpaknRlW4csqqpjrLxQOr7vgbHbp4C7pjuaJOmF8hOqktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGBsU9yZ4kTyVZTHLoMmt+KMmpJCeT/OF0x5QkrcXWSQuSbAEOA98PLAEnkhytqlNja3YB7wTuqqovJvmG9RpYkjTZkDP3O4DFqjpTVeeBh4B9q9a8BThcVV8EqKrnpjumJGkthsR9G3B2bHtptG/cq4BXJfnbJI8l2XOpB0pyIMlCkoXl5eUXNrEkaaJpvaC6FdgFvBbYD/xOkptWL6qqI1U1X1Xzc3NzUzq0JGm1IXE/B+wY294+2jduCThaVV+pqs8C/8RK7CVJMzAk7ieAXUl2JrkBuBc4umrNn7Jy1k6Sm1m5THNminNKktZgYtyr6gJwEDgOnAYerqqTSR5Msne07DjwfJJTwKPAT1fV8+s1tCTpylJVMznw/Px8LSwszOTYknS9SvJEVc1PWucnVCWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWpoUNyT7EnyVJLFJIeusO4Hk1SS+emNKElaq4lxT7IFOAzcA+wG9ifZfYl1LwfeDjw+7SElSWsz5Mz9DmCxqs5U1XngIWDfJdb9EvBu4L+mOJ8k6QUYEvdtwNmx7aXRvv+T5HZgR1V9/EoPlORAkoUkC8vLy2seVpI0zFW/oJrkRcB7gXdMWltVR6pqvqrm5+bmrvbQkqTLGBL3c8COse3to30XvRx4DfDXSZ4G7gSO+qKqJM3OkLifAHYl2ZnkBuBe4OjFO6vqS1V1c1XdUlW3AI8Be6tqYV0mliRNNDHuVXUBOAgcB04DD1fVySQPJtm73gNKktZu65BFVXUMOLZq3wOXWfvaqx9LknQ1/ISqJDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqaFBv2ZPuuXQx9f9GE+/6w3rfgxps/DMXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJamhQ3JPsSfJUksUkhy5x/08lOZXk00n+Kskrpz+qJGmoiXFPsgU4DNwD7Ab2J9m9atmTwHxVfRvwCPDL0x5UkjTckDP3O4DFqjpTVeeBh4B94wuq6tGq+vJo8zFg+3THlCStxZC4bwPOjm0vjfZdzv3An1/qjiQHkiwkWVheXh4+pSRpTab6gmqS+4B54D2Xur+qjlTVfFXNz83NTfPQkqQxQ35B9jlgx9j29tG+/yfJ64CfA763qv57OuNJkl6IIWfuJ4BdSXYmuQG4Fzg6viDJbcBvA3ur6rnpjylJWouJca+qC8BB4DhwGni4qk4meTDJ3tGy9wAvA/44yaeSHL3Mw0mSNsCQyzJU1THg2Kp9D4zdft2U55IkXQU/oSpJDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIYGvRVS0uZzy6GPr/sxnn7XG9b9GJuVZ+6S1JBxl6SGvCwjXcPW+9KIl0X68sxdkhoy7pLUkJdldF3w8oS0NsZdksZ0eQuol2UkqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ359QNr1OWjyZJ688xdkhoy7pLUkJdlpAn8umFdjzxzl6SGjLskNWTcJakhr7lfR3wbpjYLf9avnmfuktTQdXnm7p/qknRlnrlLUkOD4p5kT5KnkiwmOXSJ+78myR+N7n88yS3THlSSNNzEuCfZAhwG7gF2A/uT7F617H7gi1X1LcCvAu+e9qCSpOGGnLnfASxW1ZmqOg88BOxbtWYf8Huj248AdyfJ9MaUJK1FqurKC5I3Anuq6sdG228CvrOqDo6t+cxozdJo+59Haz6/6rEOAAdGm68GnprWExngZuDzE1f14/PeXHze/b2yquYmLdrQd8tU1RHgyEYe86IkC1U1P4tjz5LPe3PxeeuiIZdlzgE7xra3j/Zdck2SrcArgOenMaAkae2GxP0EsCvJziQ3APcCR1etOQr8yOj2G4FP1KTrPZKkdTPxskxVXUhyEDgObAE+WFUnkzwILFTVUeADwIeTLAJfYOUPgGvNTC4HXQN83puLz1vAgBdUJUnXHz+hKkkNGXdJaqh93Cd9dUJHSXYkeTTJqSQnk7x91jNtpCRbkjyZ5M9mPctGSnJTkkeS/GOS00m+a9YzbYQkPzn6Of9Mko8mecmsZ7oWtI77wK9O6OgC8I6q2g3cCbx1kzzvi94OnJ71EDPw68BfVNWtwLezCf4dJNkG/DgwX1WvYeVNH9fiGzo2XOu4M+yrE9qpqmer6pOj2//Byv/k22Y71cZIsh14A/D+Wc+ykZK8AvgeVt65RlWdr6p/m+1UG2Yr8LWjz9i8FPjcjOe5JnSP+zbg7Nj2EpskcheNvqHzNuDx2U6yYX4N+Bngf2Y9yAbbCSwDvzu6JPX+JDfOeqj1VlXngF8BngGeBb5UVX8526muDd3jvqkleRnwJ8BPVNW/z3qe9ZbkB4DnquqJWc8yA1uB24H3VdVtwH8C7V9jSvL1rPxtfCfwTcCNSe6b7VTXhu5xH/LVCS0leTErYf9IVX1s1vNskLuAvUmeZuUS3Pcl+YPZjrRhloClqrr4N7RHWIl9d68DPltVy1X1FeBjwHfPeKZrQve4D/nqhHZGX7f8AeB0Vb131vNslKp6Z1Vtr6pbWPlv/Ymq2hRncVX1r8DZJK8e7bobODXDkTbKM8CdSV46+rm/m03wQvIQ1+XvUB3qcl+dMOOxNsJdwJuAf0jyqdG+n62qYzOcSevvbcBHRicyZ4A3z3iedVdVjyd5BPgkK+8SexK/igDw6wckqaXul2UkaVMy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJauh/AbXHBCrzkQlPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_probs_bar(vals):\n",
    "    H, = vals.size()\n",
    "    xs = list(range(H))\n",
    "    vals = vals/vals.sum()\n",
    "    ys = vals.tolist()\n",
    "    plt.ylim(0, 1)\n",
    "    plt.bar(xs, ys)\n",
    "\n",
    "plot_probs_bar(probs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Draw\n",
    "\n",
    "We're drawing index `i` with probability `probs[i]`. The following code does the draw.\n",
    "\n",
    "Sample a value from a uniform distribution. Use CDF to sample based on the probability.\n",
    "\n",
    "Instead of argmax, we're using drawing with `probs` to get a particular index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "def draw(probs):\n",
    "    val = random.random()\n",
    "    csum = 0\n",
    "    for i, p in enumerate(probs):\n",
    "        csum  += p\n",
    "        if csum > val:\n",
    "            return i  \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verifying \n",
    "\n",
    "I make 10000 random draws from `probs[i]` and see how the samples are distributed. This verifies the sampling is woking, if the graph looks similar to the one above.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADM9JREFUeJzt3X+s3Xddx/Hni5aJDGQmuybaVtbEwmyIZvNmTpcocZh0YNo/JGZNhkom/YciKtEUNdPMf0AM/kgqWgFRROacxDRSrYnMmBi39I4h0taZa5nrLTO7DEQj0dL49o97ao6Xtud713Pvad/3+UiW3O/3fHK+77N1z377Ped8m6pCktTLi2Y9gCRp+oy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkMT457kg0meS/KZyzyeJL+RZDHJp5PcPv0xJUlrMeTM/UPAnis8fg+wa/TPAeB9Vz+WJOlqTIx7Vf0N8IUrLNkH/H6teAy4Kck3TmtASdLabZ3Cc2wDzo5tL432Pbt6YZIDrJzdc+ONN37HrbfeOoXDS9Lm8cQTT3y+quYmrZtG3AerqiPAEYD5+flaWFjYyMNL0nUvyb8MWTeNT8ucA3aMbW8f7ZMkzcg04n4U+OHRp2buBL5UVV91SUaStHEmXpZJ8lHgtcDNSZaAXwBeDFBVvwUcA14PLAJfBt68XsNKkoaZGPeq2j/h8QLeOrWJJElXzW+oSlJDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1NCguCfZk+SpJItJDl3i8W9O8miSJ5N8Osnrpz+qJGmoiXFPsgU4DNwD7Ab2J9m9atnPAw9X1W3AvcBvTntQSdJwQ87c7wAWq+pMVZ0HHgL2rVpTwNeNfn4F8LnpjShJWqshcd8GnB3bXhrtG/eLwH1JloBjwNsu9URJDiRZSLKwvLz8AsaVJA0xrTdU9wMfqqrtwOuBDyf5queuqiNVNV9V83Nzc1M6tCRptSFxPwfsGNvePto37n7gYYCq+jvgJcDN0xhQkrR2Q+J+AtiVZGeSG1h5w/ToqjXPAHcDJPlWVuLudRdJmpGJca+qC8BB4DhwmpVPxZxM8mCSvaNl7wDekuTvgY8CP1pVtV5DS5KubOuQRVV1jJU3Ssf3PTD28yngrumOJkl6ofyGqiQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpoUFxT7InyVNJFpMcusyaH0pyKsnJJH843TElSWuxddKCJFuAw8D3A0vAiSRHq+rU2JpdwDuBu6rqi0m+Yb0GliRNNuTM/Q5gsarOVNV54CFg36o1bwEOV9UXAarquemOKUlaiyFx3wacHdteGu0b9yrgVUn+NsljSfZc6omSHEiykGRheXn5hU0sSZpoWm+obgV2Aa8F9gO/k+Sm1Yuq6khVzVfV/Nzc3JQOLUlabUjczwE7xra3j/aNWwKOVtVXquqzwD+xEntJ0gwMifsJYFeSnUluAO4Fjq5a86esnLWT5GZWLtOcmeKckqQ1mBj3qroAHASOA6eBh6vqZJIHk+wdLTsOPJ/kFPAo8NNV9fx6DS1JurJU1UwOPD8/XwsLCzM5tiRdr5I8UVXzk9b5DVVJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaGhT3JHuSPJVkMcmhK6z7wSSVZH56I0qS1mpi3JNsAQ4D9wC7gf1Jdl9i3cuBtwOPT3tISdLaDDlzvwNYrKozVXUeeAjYd4l1vwS8G/ivKc4nSXoBhsR9G3B2bHtptO//JLkd2FFVH7/SEyU5kGQhycLy8vKah5UkDXPVb6gmeRHwXuAdk9ZW1ZGqmq+q+bm5uas9tCTpMobE/RywY2x7+2jfRS8HXgP8dZKngTuBo76pKkmzMyTuJ4BdSXYmuQG4Fzh68cGq+lJV3VxVt1TVLcBjwN6qWliXiSVJE02Me1VdAA4Cx4HTwMNVdTLJg0n2rveAkqS12zpkUVUdA46t2vfAZda+9urHkiRdDb+hKkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDQ36m5ikWw59fN2P8fS73rDux5A2C8/cJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDU0KO5J9iR5KslikkOXePynkpxK8ukkf5XkldMfVZI01MS4J9kCHAbuAXYD+5PsXrXsSWC+qr4NeAT45WkPKkkabsiZ+x3AYlWdqarzwEPAvvEFVfVoVX15tPkYsH26Y0qS1mJI3LcBZ8e2l0b7Lud+4M8v9UCSA0kWkiwsLy8Pn1KStCZTfUM1yX3APPCeSz1eVUeqar6q5ufm5qZ5aEnSmK0D1pwDdoxtbx/t+3+SvA74OeB7q+q/pzOeJOmFGHLmfgLYlWRnkhuAe4Gj4wuS3Ab8NrC3qp6b/piSpLWYGPequgAcBI4Dp4GHq+pkkgeT7B0tew/wMuCPk3wqydHLPJ0kaQMMuSxDVR0Djq3a98DYz6+b8lySpKvgN1QlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDU06EtMkmbjlkMfX9fnf/pdb5jZsScdX1fHM3dJasi4S1JDxl2SGvKau64Ls7z2LF2PjLskjenyRrKXZSSpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIe8usUZf7TkjqzTN3SWrIM3dpAm83rOuRZ+6S1JBxl6SGjLskNeQ19+uIn9TRZuGv9avnmbskNXRdnrn7u7okXZln7pLU0KC4J9mT5Kkki0kOXeLxr0nyR6PHH09yy7QHlSQNNzHuSbYAh4F7gN3A/iS7Vy27H/hiVX0L8KvAu6c9qCRpuCFn7ncAi1V1pqrOAw8B+1at2Qf83ujnR4C7k2R6Y0qS1iJVdeUFyRuBPVX1Y6PtNwHfWVUHx9Z8ZrRmabT9z6M1n1/1XAeAA6PNVwNPTeuFDHAz8PmJq/rxdW8uvu7+XllVc5MWbeinZarqCHBkI495UZKFqpqfxbFnyde9ufi6ddGQyzLngB1j29tH+y65JslW4BXA89MYUJK0dkPifgLYlWRnkhuAe4Gjq9YcBX5k9PMbgU/UpOs9kqR1M/GyTFVdSHIQOA5sAT5YVSeTPAgsVNVR4APAh5MsAl9g5TeAa81MLgddA3zdm4uvW8CAN1QlSdcfv6EqSQ0Zd0lqqH3cJ906oaMkO5I8muRUkpNJ3j7rmTZSki1JnkzyZ7OeZSMluSnJI0n+McnpJN8165k2QpKfHP06/0ySjyZ5yaxnuha0jvvAWyd0dAF4R1XtBu4E3rpJXvdFbwdOz3qIGfh14C+q6lbg29kE/w6SbAN+HJivqtew8qGPa/EDHRuuddwZduuEdqrq2ar65Ojn/2Dlf/Jts51qYyTZDrwBeP+sZ9lISV4BfA8rn1yjqs5X1b/NdqoNsxX42tF3bF4KfG7G81wTusd9G3B2bHuJTRK5i0Z36LwNeHy2k2yYXwN+BvifWQ+ywXYCy8Dvji5JvT/JjbMear1V1TngV4BngGeBL1XVX852qmtD97hvakleBvwJ8BNV9e+znme9JfkB4LmqemLWs8zAVuB24H1VdRvwn0D795iSfD0rfxrfCXwTcGOS+2Y71bWhe9yH3DqhpSQvZiXsH6mqj816ng1yF7A3ydOsXIL7viR/MNuRNswSsFRVF/+E9ggrse/udcBnq2q5qr4CfAz47hnPdE3oHvcht05oZ3S75Q8Ap6vqvbOeZ6NU1TurantV3cLKf+tPVNWmOIurqn8FziZ59WjX3cCpGY60UZ4B7kzy0tGv+7vZBG8kD3Fd/h2qQ13u1gkzHmsj3AW8CfiHJJ8a7fvZqjo2w5m0/t4GfGR0InMGePOM51l3VfV4kkeAT7LyKbEn8VYEgLcfkKSWul+WkaRNybhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJamh/wWYpQQoR6/iZQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def check_sampling(probs):\n",
    "    from collections import defaultdict\n",
    "\n",
    "    max_samples = 10000\n",
    "    counter = defaultdict(int)\n",
    "    for i in range(max_samples):\n",
    "        choice = draw(probs)\n",
    "        counter[choice] += 1\n",
    "\n",
    "        \n",
    "    vals = torch.Tensor(list(counter.values()))\n",
    "    return plot_probs_bar(vals)\n",
    "\n",
    "check_sampling(probs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Temperature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TSoftmax(nn.Module):\n",
    "    def __init__(self, temperature):\n",
    "        super().__init__()\n",
    "        self.T = temperature\n",
    "    \n",
    "    def forward(self, tensor):\n",
    "        # Take all samples, divide them by T, pass through exp(x)\n",
    "        entries = tensor.data.view(-1).div(self.T).exp()\n",
    "        return entries/entries.sum()\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What temperature does to probabilities\n",
    "We try to use temperature and see how the probabilties end up different."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAaOCAYAAACQh3NOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3X3QnXdd7/vP14aCtEDVRkfzYMoxUAPKLt6nVDlij60zLWi79/FhtzwoHCTjSBEF9RTE6tR9Ng86bHWmIDmIKCK1VoYdJVjmCB5mGNrdlOe0lomltCloQ4FKYUOpfM8fa1Xu3k1yr6a/dK07eb1m7pl1XeuXtb6Z1SbvXNe11qruDgAAD943zXsAAICjhbACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgo44qrq5qo6+wg99jOr6lNV9aWqekdVfesh1u6oqhur6utV9dwV9/1cVV1XVf9aVfuq6jVVtW7Z/f9QVV+pqrumPzceid8PsLYJK2DNqqonJHlDkuck+Y4kX07yukP8ko8k+cUkHzzAfY9M8stJTk7ylCRnJfnVFWsu6u4Tpz+Pf5DjA0ehdasvATh8VfWWJJuT/E1V/VuSS7v7NYMe/llJ/qa73zd9rt9MckNVPaq7v7hycXdfNl33lQPc9/plm7dV1VuT/O+D5gSOEY5YAUdUdz8nyS1JfmJ6pOd+UVVVm6vqC4f4eeZBHv4JmRyFuve5/inJ3UkeN2D0pyXZs2LfK6vqs1X1/qo6c8BzAEcZR6yAuevuW5KcdBi/9MQkd67Yd2eSRz2Yearq/0yylOTnl+3+v5Jcn0m4XZDJEbj/MI05gCSOWAFr211JHr1i36OT3O804Kyq6j8meWWSc7v7s/fu7+5ruvuL3f3V7v7TJO9P8vTDfR7g6CSsgIdCH+rO6anAuw7x86yD/NI9SZ607HEem+ThST5xOENW1TlJ/p9MTlt+bJXlnaQO53mAo5dTgcBD4V+SPPZgd05PBZ54GI/71iQfqKofzuSdfpcmefuBLlxPkqo6PpN/UFaSh1XVI5Lc3d1fr6ofnT7ef+ru/7Hi152UyTsF/78k9yT5z5lcg/Xiw5gZOIo5YgU8FF6Z5BXTC9FXfoTBYevuPUl+IZMguj2Ta6t+8d77q+pdVfXyZb/k3Un+Z5IfSrJjevtp0/t+M8ljkuxadqTsXdP7HpbkvyTZn+SzSV6U5D9292EdGQOOXtV9yCP0AADMyBErAIBBVg2rqnpTVd1eVR8/yP1VVX9YVXur6qNV9eTxYwIALL5Zjli9Ock5h7j/3CRbpz/bk7z+EGsBAI5aq4bV9KsiPneIJecn+bOeuDrJSVX1naMGBABYK0ZcY7Uhya3LtvdN9wEAHFMe0s+xqqrtmZwuzAknnPADp5566kP59AAAh+W66677bHevX23diLC6LcmmZdsbp/vup7t3ZPLZMVlaWurdu3cPeHoAgCOrqj41y7oRYbUzyUVVdXkmn0x8Z3d/ZsDjwpqy5eJ3znuEVd38qmfMewSAo9qqYVVVb0tyZpKTq2pfkt/K5FOI091/lGRXJl9EujfJl5M870gNCwCwyFYNq+6+cJX7O8kLh00EALBG+eR1AIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAaZKayq6pyqurGq9lbVxQe4f3NVvbeqPlRVH62qp48fFQBgsa0aVlV1XJLLkpybZFuSC6tq24plr0hyRXefluSCJK8bPSgAwKKb5YjV6Un2dvdN3X13ksuTnL9iTSd59PT2Y5J8etyIAABrw7oZ1mxIcuuy7X1JnrJizW8neXdVvSjJCUnOHjIdAMAaMuri9QuTvLm7NyZ5epK3VNX9HruqtlfV7qravX///kFPDQCwGGYJq9uSbFq2vXG6b7nnJ7kiSbr7A0kekeTklQ/U3Tu6e6m7l9avX394EwMALKhZwuraJFur6pSqOj6Ti9N3rlhzS5KzkqSqvjeTsHJICgA4pqwaVt19T5KLklyV5IZM3v23p6ourarzpstemuQFVfWRJG9L8tzu7iM1NADAIprl4vV0964ku1bsu2TZ7euTPHXsaAAAa4tPXgcAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgkJnCqqrOqaobq2pvVV18kDU/U1XXV9WeqvqLsWMCACy+dastqKrjklyW5MeS7EtybVXt7O7rl63ZmuRlSZ7a3Z+vqm8/UgMDACyqWY5YnZ5kb3ff1N13J7k8yfkr1rwgyWXd/fkk6e7bx44JALD4ZgmrDUluXba9b7pvuccleVxVvb+qrq6qc0YNCACwVqx6KvABPM7WJGcm2ZjkfVX1fd39heWLqmp7ku1Jsnnz5kFPDQCwGGY5YnVbkk3LtjdO9y23L8nO7v5ad38yyScyCa376O4d3b3U3Uvr168/3JkBABbSLGF1bZKtVXVKVR2f5IIkO1eseUcmR6tSVSdncmrwpoFzAgAsvFXDqrvvSXJRkquS3JDkiu7eU1WXVtV502VXJbmjqq5P8t4kv9bddxypoQEAFtFM11h1964ku1bsu2TZ7U7ykukPAMAxySevAwAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADDITGFVVedU1Y1VtbeqLj7Eup+sqq6qpXEjAgCsDauGVVUdl+SyJOcm2ZbkwqradoB1j0ry4iTXjB4SAGAtmOWI1elJ9nb3Td19d5LLk5x/gHW/k+TVSb4ycD4AgDVjlrDakOTWZdv7pvv+XVU9Ocmm7n7noR6oqrZX1e6q2r1///4HPCwAwCJ70BevV9U3JXltkpeutra7d3T3UncvrV+//sE+NQDAQpklrG5LsmnZ9sbpvns9KskTk/xDVd2c5IwkO13ADgAca2YJq2uTbK2qU6rq+CQXJNl5753dfWd3n9zdW7p7S5Krk5zX3buPyMQAAAtq1bDq7nuSXJTkqiQ3JLmiu/dU1aVVdd6RHhAAYK1YN8ui7t6VZNeKfZccZO2ZD34sAIC1xyevAwAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgkJnCqqrOqaobq2pvVV18gPtfUlXXV9VHq+rvq+q7x48KALDYVg2rqjouyWVJzk2yLcmFVbVtxbIPJVnq7u9PcmWS14weFABg0c1yxOr0JHu7+6buvjvJ5UnOX76gu9/b3V+ebl6dZOPYMQEAFt8sYbUhya3LtvdN9x3M85O868EMBQCwFq0b+WBV9ewkS0l+5CD3b0+yPUk2b9488qkBAOZuliNWtyXZtGx743TffVTV2Ul+I8l53f3VAz1Qd+/o7qXuXlq/fv3hzAsAsLBmCatrk2ytqlOq6vgkFyTZuXxBVZ2W5A2ZRNXt48cEAFh8q4ZVd9+T5KIkVyW5IckV3b2nqi6tqvOmy343yYlJ/qqqPlxVOw/ycAAAR62ZrrHq7l1Jdq3Yd8my22cPngsAYM3xyesAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADDJTWFXVOVV1Y1XtraqLD3D/w6vqL6f3X1NVW0YPCgCw6FYNq6o6LsllSc5Nsi3JhVW1bcWy5yf5fHd/T5L/luTVowcFAFh0sxyxOj3J3u6+qbvvTnJ5kvNXrDk/yZ9Ob1+Z5KyqqnFjAgAsvnUzrNmQ5NZl2/uSPOVga7r7nqq6M8m3JfnsiCEB+IYtF79z3iOs6uZXPWPmtUfb74dj2yxhNUxVbU+yfbp5V1Xd+FA+/yAnRzAuMq/PIdR8T9J7bRbb0Ndnzv+tDef/HZJ89yyLZgmr25JsWra9cbrvQGv2VdW6JI9JcsfKB+ruHUl2zDLYoqqq3d29NO85ODCvz+Ly2iw2r8/i8tqsLbNcY3Vtkq1VdUpVHZ/kgiQ7V6zZmeTnprd/Ksl7urvHjQkAsPhWPWI1vWbqoiRXJTkuyZu6e09VXZpkd3fvTPLHSd5SVXuTfC6T+AIAOKbMdI1Vd+9KsmvFvkuW3f5Kkp8eO9rCWtOnMo8BXp/F5bVZbF6fxeW1WUPKGTsAgDF8pQ0AwCDC6gFY7at9mI+q2lRV762q66tqT1W9eN4zcX9VdVxVfaiq/nbes/ANVXVSVV1ZVf9YVTdU1Q/Oeya+oap+Zfrn2ser6m1V9Yh5z8ShCasZzfjVPszHPUle2t3bkpyR5IVem4X04iQ3zHsI7ucPkvxdd5+a5EnxGi2MqtqQ5JeSLHX3EzN5A5k3hy04YTW7Wb7ahzno7s909went7+YyV8MG+Y7FctV1cYkz0jyxnnPwjdU1WOSPC2Td3anu+/u7i/MdypWWJfkm6efEfnIJJ+e8zysQljN7kBf7eMv7wVTVVuSnJbkmvlOwgq/n+TXk3x93oNwH6ck2Z/kT6anad9YVSfMeygmuvu2JL+X5JYkn0lyZ3e/e75TsRphxVGjqk5M8tdJfrm7/3Xe8zBRVT+e5Pbuvm7es3A/65I8Ocnru/u0JF9K4vrRBVFV35LJmZFTknxXkhOq6tnznYrVCKvZzfLVPsxJVT0sk6h6a3e/fd7zcB9PTXJeVd2cySn0H62qP5/vSEztS7Kvu+89wntlJqHFYjg7ySe7e393fy3J25P80JxnYhXCanazfLUPc1BVlck1Ijd092vnPQ/31d0v6+6N3b0lk/9v3tPd/tW9ALr7n5PcWlWPn+46K8n1cxyJ+7olyRlV9cjpn3NnxZsLFt5Mn7zOwb/aZ85jMfHUJM9J8rGq+vB038un3xgAHNqLkrx1+g/Gm5I8b87zMNXd11TVlUk+mMm7nz8Un8K+8HzyOgDAIE4FAgAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqyAh1xV3VxVZx+hx35mVX2qqr5UVe+oqm89yLrHVdV/r6r9VfW5qrpq2XfmpaqeW1X/VlV3Lfs580jMDBw9hBVw1KiqJyR5QybfHfkdSb6c5HUHWX5SJl+k/vjp2v+R5L+vWPOB7j5x2c8/HJHBgaOG7woEHlJV9ZYkz0ry1ST/luTS7n7NoMf+r0m2dPczp9v/S5Ibknxbd39xlV/7rUnuSHJyd99RVc9N8vPd/b+NmA04NjhiBTykuvs5SW5J8hPTo0D3i6qq2lxVXzjEzzMP8vBPSPKRZc/1T0nuTvK4GUZ7WpJ/7u47lu07rao+W1WfqKrfrKp1M/9GgWOSPySAhdPdt2Ryqu6BOjHJnSv23ZnkUYf6RVW1McllSV6ybPf7kjwxyacyCba/THJPklcexlzAMcIRK+BocleSR6/Y9+gkBz0NWFXrk7w7yeu6+2337u/um7r7k9399e7+WJJLk/zUEZgZOIoIK2AeDnlx5/RU4F2H+HnWQX7pniRPWvY4j03y8CSfOMjzfEsmUbWzu//vGWauVdYAxzinAoF5+Jckjz3YndNTgScexuO+NckHquqHk3wwk6NMbz/QhetV9egkVyV5f3dffID7z03ywe7+l6o6NclvJvmrw5gJOIY4YgXMwyuTvGJ6IfqvjnrQ7t6T5BcyCazbM7m26hfvvb+q3lVVL59u/qck/2uS5604GrZ5ev9ZST5aVV9KsivJ25P811GzAkcnH7cAADCII1YAAIOsGlZV9aaqur2qPn6Q+6uq/rCq9lbVR6vqyePHBABYfLMcsXpzknMOcf+5SbZOf7Ynef2DHwsAYO1ZNay6+31JPneIJecn+bOeuDrJSVX1naMGBABYK0ZcY7Uhya3LtvdN9wEAHFMe0s+xqqrtmZwuzAknnPADp5566kP59AAAh+W66677bHevX23diLC6LcmmZdsbp/vup7t3JNmRJEtLS7179+4BTw8AcGRV1admWTfiVODOJD87fXfgGUnu7O7PDHhcAIA1ZdUjVlX1tiRnJjm5qvYl+a0kD0uS7v6jTD6R+OlJ9ib5cpLnHalhAQAW2aph1d0XrnJ/J3nhsIkAANYon7wOADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgkHXzHgCOFlsufue8R1jVza96xrxHADiqOWIFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBZgqrqjqnqm6sqr1VdfEB7t9cVe+tqg9V1Uer6unjRwUAWGyrhlVVHZfksiTnJtmW5MKq2rZi2SuSXNHdpyW5IMnrRg8KALDoZjlidXqSvd19U3ffneTyJOevWNNJHj29/Zgknx43IgDA2jBLWG1Icuuy7X3Tfcv9dpJnV9W+JLuSvOhAD1RV26tqd1Xt3r9//2GMCwCwuEZdvH5hkjd398YkT0/ylqq632N3947uXurupfXr1w96agCAxTBLWN2WZNOy7Y3Tfcs9P8kVSdLdH0jyiCQnjxgQAGCtmCWsrk2ytapOqarjM7k4feeKNbckOStJqup7Mwkr5/oAgGPKqmHV3fckuSjJVUluyOTdf3uq6tKqOm+67KVJXlBVH0nytiTP7e4+UkMDACyidbMs6u5dmVyUvnzfJctuX5/kqWNHAwBYW3zyOgDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABpkprKrqnKq6sar2VtXFB1nzM1V1fVXtqaq/GDsmAMDiW7fagqo6LsllSX4syb4k11bVzu6+ftmarUleluSp3f35qvr2IzUwAMCimuWI1elJ9nb3Td19d5LLk5y/Ys0LklzW3Z9Pku6+feyYAACLb5aw2pDk1mXb+6b7lntcksdV1fur6uqqOmfUgAAAa8WqpwIfwONsTXJmko1J3ldV39fdX1i+qKq2J9meJJs3bx701AAAi2GWI1a3Jdm0bHvjdN9y+5Ls7O6vdfcnk3wik9C6j+7e0d1L3b20fv36w50ZAGAhzRJW1ybZWlWnVNXxSS5IsnPFmndkcrQqVXVyJqcGbxo4JwDAwls1rLr7niQXJbkqyQ1JrujuPVV1aVWdN112VZI7qur6JO9N8mvdfceRGhoAYBHNdI1Vd+9KsmvFvkuW3e4kL5n+AAAck3zyOgDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDzBRWVXVOVd1YVXur6uJDrPvJquqqWho3IgDA2rBqWFXVcUkuS3Jukm1JLqyqbQdY96gkL05yzeghAQDWglmOWJ2eZG9339Tddye5PMn5B1j3O0leneQrA+cDAFgzZgmrDUluXba9b7rv31XVk5Ns6u53DpwNAGBNedAXr1fVNyV5bZKXzrB2e1Xtrqrd+/fvf7BPDQCwUGYJq9uSbFq2vXG6716PSvLEJP9QVTcnOSPJzgNdwN7dO7p7qbuX1q9ff/hTAwAsoFnC6tokW6vqlKo6PskFSXbee2d339ndJ3f3lu7ekuTqJOd19+4jMjEAwIJaNay6+54kFyW5KskNSa7o7j1VdWlVnXekBwQAWCvWzbKou3cl2bVi3yUHWXvmgx8LAGDt8cnrAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwyU1hV1TlVdWNV7a2qiw9w/0uq6vqq+mhV/X1Vfff4UQEAFtuqYVVVxyW5LMm5SbYlubCqtq1Y9qEkS939/UmuTPKa0YMCACy6WY5YnZ5kb3ff1N13J7k8yfnLF3T3e7v7y9PNq5NsHDsmAMDimyWsNiS5ddn2vum+g3l+knc9mKEAANaidSMfrKqenWQpyY8c5P7tSbYnyebNm0c+NQDA3M1yxOq2JJuWbW+c7ruPqjo7yW8kOa+7v3qgB+ruHd291N1L69evP5x5AQAW1ixhdW2SrVV1SlUdn+SCJDuXL6iq05K8IZOoun38mAAAi2/VsOrue5JclOSqJDckuaK791TVpVV13nTZ7yY5MclfVdWHq2rnQR4OAOCoNdM1Vt29K8muFfsuWXb77MFzAQCsOT55HQBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDr5j0AAMe2LRe/c94jrOrmVz1j3iOwRjhiBQAwiCNWAGuMIzywuByxAgAYRFgBAAwirAAABnGNFXA/a+EansR1PHCk+bPggZvpiFVVnVNVN1bV3qq6+AD3P7yq/nJ6/zVVtWX0oAAAi27VsKqq45JcluTcJNuSXFhV21Yse36Sz3f39yT5b0lePXpQAIBFN8sRq9OT7O3um7r77iSXJzl/xZrzk/zp9PaVSc6qqho3JgDA4pvlGqsNSW5dtr0vyVMOtqa776mqO5N8W5LPjhiSCee64fD4fwd4qDykF69X1fYk26ebd1XVjQ/l8w9ycgTjIdV8TwR7fQ7haHtt5vz7Ge5oen28NkP5c20VD9Hr892zLJolrG5LsmnZ9sbpvgOt2VdV65I8JskdKx+ou3ck2THLYIuqqnZ399K85+DAvD6Ly2uz2Lw+i8trs7bMco3VtUm2VtUpVXV8kguS7FyxZmeSn5ve/qkk7+nuHjcmAMDiW/WI1fSaqYuSXJXkuCRv6u49VXVpkt3dvTPJHyd5S1XtTfK5TOILAOCYMtM1Vt29K8muFfsuWXb7K0l+euxoC2tNn8o8Bnh9FpfXZrF5fRaX12YNKWfsAADG8F2BAACDCKsHYLWv9mE+qmpTVb23qq6vqj1V9eJ5z8T9VdVxVfWhqvrbec/CN1TVSVV1ZVX9Y1XdUFU/OO+Z+Iaq+pXpn2sfr6q3VdUj5j0ThyasZjTjV/swH/ckeWl3b0tyRpIXem0W0ouT3DDvIbifP0jyd919apInxWu0MKpqQ5JfSrLU3U/M5A1k3hy24ITV7Gb5ah/moLs/090fnN7+YiZ/MWyY71QsV1UbkzwjyRvnPQvfUFWPSfK0TN7Zne6+u7u/MN+pWGFdkm+efkbkI5N8es7zsAphNbsDfbWPv7wXTFVtSXJakmvmOwkr/H6SX0/y9XkPwn2ckmR/kj+ZnqZ9Y1WdMO+hmOju25L8XpJbknwmyZ3d/e75TsVqhBVHjao6MclfJ/nl7v7Xec/DRFX9eJLbu/u6ec/C/axL8uQkr+/u05J8KYnrRxdEVX1LJmdGTknyXUlOqKpnz3cqViOsZjfLV/swJ1X1sEyi6q3d/fZ5z8N9PDXJeVV1cyan0H+0qv58viMxtS/Jvu6+9wjvlZmEFovh7CSf7O793f21JG9P8kNznolVCKvZzfLVPsxBVVUm14jc0N2vnfc83Fd3v6y7N3b3lkz+v3lPd/tX9wLo7n9OcmtVPX6666wk189xJO7rliRnVNUjp3/OnRVvLlh4M33yOgf/ap85j8XEU5M8J8nHqurD030vn35jAHBoL0ry1uk/GG9K8rw5z8NUd19TVVcm+WAm737+UHwK+8LzyesAAIM4FQgAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAuaqqm6uqrOP0GM/s6o+VVVfqqp3VNW3HmJtT9fdNf1545GYCTi6CSvgqFRVT0jyhky+R/I7knw5yetW+WVP6u4Tpz8/f6RnBI4+wgqYm6p6S5LNSf5mepTo1wc+/LOS/E13v6+770rym0n+j6p61MDnALgPYQXMTXc/J8ktSX5iepToNSvXVNXmqvrCIX6eeZCHf0KSjyx7rn9KcneSxx1ipPdV1T9X1durasth/8aAY9a6eQ8AcCjdfUuSkw7DP2cTAAAgAElEQVTjl56Y5M4V++5McrAjVj+S5Ookj0zyX5L8bVX9h+6+5zCeGzhGOWIFHK3uSvLoFfseneSLB1o8PWV4d3d/IcmLk5yS5HuP7IjA0UZYAfPWh7pzeirwrkP8POsgv3RPkicte5zHJnl4kk88gLlqxrUASZwKBObvX5I89mB3Tk8FnngYj/vWJB+oqh9O8sEklyZ5e3ff74jV9B2ED0vysSTfnMmpwNuS3HAYzwscwxyxAubtlUleMb0Q/VdHPWh370nyC5kE1u2ZXFv1i/feX1XvqqqXTze/I8lfJvnXJDcl2ZLkx7v7a6PmAY4N1X3Io/AAAMzIESsAgEFWDauqelNV3V5VHz/I/VVVf1hVe6vqo1X15PFjAgAsvlmOWL05yTmHuP/cJFunP9uTvP7BjwUAsPasGlbd/b4knzvEkvOT/FlPXJ3kpKr6zlEDAgCsFSM+bmFDkluXbe+b7vvMyoVVtT2To1o54YQTfuDUU08d8PQAAEfWdddd99nuXr/auof0c6y6e0eSHUmytLTUu3fvfiifHgDgsFTVp2ZZN+Jdgbcl2bRse+N0HwDAMWVEWO1M8rPTdweekeTO7r7faUAAgKPdqqcCq+ptSc5McnJV7UvyW5l89UO6+4+S7Ery9CR7k3w5yfOO1LAAAIts1bDq7gtXub+TvHDYRAAAa5RPXgcAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCAzhVVVnVNVN1bV3qq6+AD3b66q91bVh6rqo1X19PGjAgAstlXDqqqOS3JZknOTbEtyYVVtW7HsFUmu6O7TklyQ5HWjBwUAWHSzHLE6Pcne7r6pu+9OcnmS81es6SSPnt5+TJJPjxsRAGBtWDfDmg1Jbl22vS/JU1as+e0k766qFyU5IcnZQ6YDAFhDRl28fmGSN3f3xiRPT/KWqrrfY1fV9qraXVW79+/fP+ipAQAWwyxhdVuSTcu2N073Lff8JFckSXd/IMkjkpy88oG6e0d3L3X30vr16w9vYgCABTVLWF2bZGtVnVJVx2dycfrOFWtuSXJWklTV92YSVg5JAQDHlFXDqrvvSXJRkquS3JDJu//2VNWlVXXedNlLk7ygqj6S5G1JntvdfaSGBgBYRLNcvJ7u3pVk14p9lyy7fX2Sp44dDQBgbfHJ6wAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMMlNYVdU5VXVjVe2tqosPsuZnqur6qtpTVX8xdkwAgMW3brUFVXVcksuS/FiSfUmuraqd3X39sjVbk7wsyVO7+/NV9e1HamAAgEU1yxGr05Ps7e6buvvuJJcnOX/Fmhckuay7P58k3X372DEBABbfLGG1Icmty7b3Tfct97gkj6uq91fV1VV1zqgBAQDWilVPBT6Ax9ma5MwkG5O8r6q+r7u/sHxRVW1Psj1JNm/ePOipAQAWwyxHrG5LsmnZ9sbpvuX2JdnZ3V/r7k8m+UQmoXUf3b2ju5e6e2n9+vWHOzMAwEKaJayuTbK1qk6pquOTXJBk54o178jkaFWq6uRMTg3eNHBOAICFt2pYdfc9SS5KclWSG5Jc0d17qurSqjpvuuyqJHdU1fVJ3pvk17r7jiM1NADAIqrunssTLy0t9e7du+fy3AAAD0RVXdfdS6ut88nrAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEHWzXsAOFpsufid8x5hVTe/6hnzHgHgqOaIFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYZKawqqpzqurGqtpbVRcfYt1PVlVX1dK4EQEA1oZVw6qqjktyWZJzk2xLcmFVbTvAukcleXGSa0YPCQCwFsxyxOr0JHu7+6buvjvJ5UnOP8C630ny6iRfGTgfAMCaMUtYbUhy67LtfdN9/66qnpxkU3e/c+BsAABryoO+eL2qvinJa5O8dIa126tqd1Xt3r9//4N9agCAhTJLWN2WZNOy7Y3Tffd6VJInJvmHqro5yRlJdh7oAvbu3tHdS929tH79+sOfGgBgAc0SVtcm2VpVp1TV8UkuSLLz3ju7+87uPrm7t3T3liRXJzmvu3cfkYkBABbUqmHV3fckuSjJVUluSHJFd++pqkur6rwjPSAAwFqxbpZF3b0rya4V+y45yNozH/xYAABrj09eBwAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGCQmcKqqs6pqhuram9VXXyA+19SVddX1Uer6u+r6rvHjwoAsNhWDauqOi7JZUnOTbItyYVVtW3Fsg8lWeru709yZZLXjB4UAGDRzXLE6vQke7v7pu6+O8nlSc5fvqC739vdX55uXp1k49gxAQAW3yxhtSHJrcu29033Hczzk7zrQHdU1faq2l1Vu/fv3z/7lAAAa8DQi9er6tlJlpL87oHu7+4d3b3U3Uvr168f+dQAAHO3boY1tyXZtGx743TffVTV2Ul+I8mPdPdXx4wHALB2zHLE6tokW6vqlKo6PskFSXYuX1BVpyV5Q5Lzuvv28WMCACy+VcOqu+9JclGSq5LckOSK7t5TVZdW1XnTZb+b5MQkf1VVH66qnQd5OACAo9YspwLT3buS7Fqx75Jlt88ePBcAwJrjk9cBAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMMtMHhALAkbLl4nfOe4RV3fyqZ8x7BNYIR6wAAAYRVgAAgzgVCLDGOHUGi8sRKwCAQYQVAMAgwgoAYBBhBQAwiIvXgftZCxdHJy6QBhaPsAIADsg/sh44pwIBAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBfFfgGuI7mwBgsc10xKqqzqmqG6tqb1VdfID7H15Vfzm9/5qq2jJ6UACARbfqEauqOi7JZUl+LMm+JNdW1c7uvn7Zsucn+Xx3f09VXZDk1Un+85EYGOCBcrQXeKjMcsTq9CR7u/um7r47yeVJzl+x5vwkfzq9fWWSs6qqxo0JALD4ZrnGakOSW5dt70vylIOt6e57qurOJN+W5LMjhjxc/pW62NbC63OsvjbA4fNn27GtuvvQC6p+Ksk53f3z0+3nJHlKd1+0bM3Hp2v2Tbf/abrmsysea3uS7dPNxye5cdRv5CF0cuYcjByS12dxeW0Wm9dncXltFsN3d/f61RbNcsTqtiSblm1vnO470Jp9VbUuyWOS3LHygbp7R5IdMzznwqqq3d29NO85ODCvz+Ly2iw2r8/i8tqsLbNcY3Vtkq1VdUpVHZ/kgiQ7V6zZmeTnprd/Ksl7erVDYQAAR5lVj1hNr5m6KMlVSY5L8qbu3lNVlybZ3d07k/xxkrdU1d4kn8skvgAAjikzfUBod+9KsmvFvkuW3f5Kkp8eO9rCWtOnMo8BXp/F5bVZbF6fxeW1WUNWvXgdAIDZ+K5AAIBBhNUDsNpX+zAfVbWpqt5bVddX1Z6qevG8Z+L+quq4qvpQVf3tvGfhG6rqpKq6sqr+sapuqKofnPdMfENV/cr0z7WPV9XbquoR856JQxNWM1r21T7nJtmW5MKq2jbfqZi6J8lLu3tbkjOSvNBrs5BenOSGeQ/B/fxBkr/r7lOTPCleo4VRVRuS/FKSpe5+YiZvIPPmsAUnrGY3y1f7MAfd/Znu/uD09hcz+Ythw3ynYrmq2pjkGUneOO9Z+IaqekySp2Xyzu50993d/YX5TsUK65J88/QzIh+Z5NNznodVCKvZHeirffzlvWCqakuS05JcM99JWOH3k/x6kq/PexDu45Qk+5P8yfQ07Rur6oR5D8VEd9+W5PeS3JLkM0nu7O53z3cqViOsOGpU1YlJ/jrJL3f3v857Hiaq6seT3N7d1817Fu5nXZInJ3l9d5+W5EtJXD+6IKrqWzI5M3JKku9KckJVPXu+U7EaYTW7Wb7ahzmpqodlElVv7e63z3se7uOpSc6rqpszOYX+o1X15/Mdial9SfZ1971HeK/MJLRYDGcn+WR37+/uryV5e5IfmvNMrEJYzW6Wr/ZhDqqqMrlG5Ibufu285+G+uvtl3b2xu7dk8v/Ne7rbv7oXQHf/c5Jbq+rx011nJbl+jiNxX7ckOaOqHjn9c+6seHPBwpvpk9c5+Ff7zHksJp6a5DlJPlZVH57ue/n0GwOAQ3tRkrdO/8F4U5LnzXkeprr7mqq6MskHM3n384fiU9gXnk9eBwAYxKlAAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBWwkKrq5qo6+wg87ndW1c6q+nRVdVVtGf0cwLFLWAHHmq8n+bskPznvQYCjj+8KBBZOVb0lybOSfDXJvyW5tLtfM/g51iX5WpJTuvvmkY8NHLuEFbCQqurmJD/f3f/vQe7fnOSjh3iIX+zuvzjE4wsrYLh18x4A4HB09y1JTpr3HADLucYKAGAQYQUsqkNep1BVm6vqrkP8POuhGhTgXk4FAovqX5I89mB3Tk8Fnng4D1xVj0hy3HTz4VX1iO7+yuE8FsByjlgBi+qVSV5RVV+oql8d/Nj/M8ld09v/ON0GeNC8KxAAYBBHrAAABlk1rKrqTVV1e1V9/CD3V1X9YVXtraqPVtWTx48JALD4Zjli9eYk5xzi/nOTbJ3+bE/y+gc/FgDA2rNqWHX3+5J87hBLzk/yZz1xdZKTquo7Rw0IALBWjLjGakOSW5dt75vuAwA4pjykn2NVVdszOV2YE0444QdOPfXUh/LpAQAOy3XXXffZ7l6/2roRYXVbkk3LtjdO991Pd+9IsiNJlpaWevfu3QOeHgDgyKqqT82ybsSpwJ1Jfnb67sAzktzZ3Z8Z8LgAAGvKqkesquptSc5McnJV7UvyW0keliTd/UdJdiV5epK9Sb6c5HlHalgAgEW2alh194Wr3N9JXjhsIgCANconrwMADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVwP/P3v3GWnrX9d7/fJ2hIi0Wc7pNdP7Qehyoc6PexZ1S7YkSW5MpaPtAjmkVVA4y5wFFVJQUJdXUO1HA+O+konMQUQRqrdxmlJGSSA05xjbdpYjM1JJxqJ0pYAcoVeTW0vi9H+yF7O7OzFqd/qZr7ZnXK5lkXdf6da3vZE1n3vu6rrUWwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADDITGFVVbuq6t6qOlhV1x3j/u1VdVtV3V1VH6mqF40fFQBgsU0Nq6ralOTGJFck2ZnkmqrauW7ZG5Lc3N0XJbk6yW+NHhQAYNHNcsTq4iQHu/tQdz+S5KYkV61b00m+enL73CSfGDciAMDGsHmGNVuSHF6zfSTJC9at+YUk76+qVyc5O8nlQ6YDANhARl28fk2St3f31iQvSvKOqnrcY1fV7qpaqaqVo0ePDnpqAIDFMEtYPZBk25rtrZN9a70iyc1J0t1/k+TpSc5b/0Ddvae7l7t7eWlp6eQmBgBYULOE1Z1JdlTVBVV1VlYvTt+7bs39SS5Lkqr6pqyGlUNSAMAZZWpYdfejSa5NcmuSe7L67r/9VXVDVV05WfbaJK+sqr9N8u4kP9rdfaqGBgBYRLNcvJ7u3pdk37p916+5fSDJpWNHAwDYWHzyOgDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDzBRWVbWrqu6tqoNVdd1x1vxAVR2oqv1V9a6xYwIALL7N0xZU1aYkNyb5niRHktxZVXu7+8CaNTuSvD7Jpd39UFV97akaGABgUc1yxOriJAe7+1B3P5LkpiRXrVvzyiQ3dvdDSdLdD44dEwBg8c0SVluSHF6zfWSyb63nJHlOVf11Vd1eVbtGDQgAsFFMPRX4BB5nR5IXJtma5INV9c3d/bm1i6pqd5LdSbJ9+/ZBTw0AsBhmOWL1QJJta7a3TvatdSTJ3u7+Ynd/PMnHshpaj9Hde7p7ubuXl5aWTnZmAICFNEtY3ZlkR1VdUFVnJbk6yd51a/40q0erUlXnZfXU4KGBcwIALLypYdXdjya5NsmtSe5JcnN376+qG6rqysmyW5N8pqoOJLktyc9092dO1dAAAIuounsuT7y8vNwrKytzeW4AgCeiqu7q7uVp63zyOgDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABpkprKpqV1XdW1UHq+q6E6z7/qrqqloeNyIAwMYwNayqalOSG5NckWRnkmuqaucx1j0zyWuS3DF6SACAjWCWI1YXJznY3Ye6+5EkNyW56hjrfjHJG5P828D5AAA2jFnCakuSw2u2j0z2/aeqen6Sbd393oGzAQBsKE/64vWq+ookv5rktTOs3V1VK1W1cvTo0Sf71AAAC2WWsHogybY121sn+77kmUmel+Svquq+JJck2XusC9i7e093L3f38tLS0slPDQCwgGYJqzuT7KiqC6rqrCRXJ9n7pTu7++HuPq+7z+/u85PcnuTK7l45JRMDACyoqWHV3Y8muTbJrUnuSXJzd++vqhuq6spTPSAAwEaxeZZF3b0vyb51+64/ztoXPvmxAAA2Hp+8DgAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhk87wHgNPF+de9d94jTHXfL7943iMAnNYcsQIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAg8wUVlW1q6ruraqDVXXdMe7/qao6UFUfqaq/rKpnjx8VAGCxTQ2rqtqU5MYkVyTZmeSaqtq5btndSZa7+1uS3JLkTaMHBQBYdLMcsbo4ycHuPtTdjyS5KclVaxd0923d/YXJ5u1Jto4dEwBg8c0SVluSHF6zfWSy73hekeQvnsxQAAAb0eaRD1ZVL02ynOS7jnP/7iS7k2T79u0jnxoAYO5mOWL1QJJta7a3TvY9RlVdnuTnklzZ3f9+rAfq7j3dvdzdy0tLSyczLwDAwpolrO5MsqOqLqiqs5JcnWTv2gVVdVGS38lqVD04fkwAgMU3Nay6+9Ek1ya5Nck9SW7u7v1VdUNVXTlZ9uYk5yT546r6cFXtPc7DAQCctma6xqq79yXZt27f9WtuXz54LgCADccnrwMADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYJChX2kDAE/U+de9d94jTHXfL7943iOwQThiBQAwiCNWABuMIzywuByxAgAYRFgBAAziVCDwOBvhVFPidBOweIQVAHBMfsh64pwKBAAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYxCevbyA+ARcAFpsjVgAAgwgrAIBBnAoETntOowNPFUesAAAGEVYAAIM4FcjcbITTM07NAPBECCsAGMgPjWe20zqsNsIf7sQfcAA4Xcx0jVVV7aqqe6vqYFVdd4z7v7Kq/mhy/x1Vdf7oQQEAFt3UsKqqTUluTHJFkp1JrqmqneuWvSLJQ939jUl+LckbRw8KALDoZjlidXGSg919qLsfSXJTkqvWrbkqye9Pbt+S5LKqqnFjAgAsvlnCakuSw2u2j0z2HXNNdz+a5OEk/2XEgAAAG0V194kXVL0kya7u/rHJ9suSvKC7r12z5qOTNUcm2/8wWfPpdY+1O8nuyeZzk9w76jfyFDovyaenrmJevD6Ly2uz2Lw+i8trsxie3d1L0xbN8q7AB5JsW7O9dbLvWGuOVNXmJOcm+cz6B+ruPUn2zPCcC6uqVrp7ed5zcGxen8XltVlsXp/F5bXZWGY5FXhnkh1VdUFVnZXk6iR7163Zm+RHJrdfkuQDPe1QGADAaWbqEavufrSqrk1ya5JNSd7W3fur6oYkK929N8nvJnlHVR1M8tmsxhcAwBllpg8I7e59Sfat23f9mtv/luS/jx1tYW3oU5lnAK/P4vLaLDavz+Ly2mwgUy9eBwBgNjN98joAANMJqydg2lf7MB9Vta2qbquqA1W1v6peM++ZeLyq2lRVd1fVn897Fr6sqp5VVbdU1d9X1T1V9e3znokvq6qfnPy99tGqendVPX3eM3FiwmpGM361D/PxaJLXdvfOJJckeZXXZiG9Jsk98x6Cx/mNJO/r7guTfGu8RgujqrYk+fEky939vKy+gcybwxacsJrdLF/twxx09ye7+0OT2/+S1X8Y1n87AHNUVVuTvDjJW+c9C19WVecm+c6svrM73f1Id39uvlOxzuYkXzX5jMhnJPnEnOdhCmE1u1m+2oc5q6rzk1yU5I75TsI6v57kdUn+Y96D8BgXJDma5Pcmp2nfWlVnz3soVnX3A0l+Jcn9ST6Z5OHufv98p2IaYcVpo6rOSfInSX6iu/953vOwqqq+N8mD3X3XvGfhcTYneX6St3T3RUn+NYnrRxdEVX1NVs+MXJDk65OcXVUvne9UTCOsZjfLV/swJ1X1tKxG1Tu7+z3znofHuDTJlVV1X1ZPoX93Vf3hfEdi4kiSI939pSO8t2Q1tFgMlyf5eHcf7e4vJnlPku+Y80xMIaxmN8tX+zAHVVVZvUbknu7+1XnPw2N19+u7e2t3n5/V/28+0N1+6l4A3f2pJIer6rmTXZclOTDHkXis+5NcUlXPmPw9d1m8uWDhzfTJ6xz/q33mPBarLk3ysiR/V1Ufnuz72ck3BgAn9uok75z8wHgoycvnPA8T3X1HVd2S5ENZfffz3fEp7AvPJ68DAAziVCAAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCClhIVXVfVV1+Ch73xVX1f6rqc1X1qap6a1U9c/TzAGcmYQWcac5N8v8k+fok35RkS5I3z3Ui4LQhrICFU1XvSLI9yZ9V1eer6nWjHru739Xd7+vuL3T3Q0n+d1a/yBvgSRNWwMLp7pcluT/J93X3Od39pvVrqmr75HTe8X794IxP951J9o+cHzhzbZ73AAAno7vvT/KsJ/MYVfU9SX4kyQuGDAWc8RyxAs5IVXVJkncleUl3f2ze8wCnB2EFLKo+0Z2TU4GfP8GvHzrBf3tRkr1J/kd3/+XowYEzl1OBwKL6pyTfcLw7J6cCz3miD1pVz0vyviSv7u4/O/nxAB7PEStgUf1SkjdMLkT/6YGP+9okS0l+d83RLRevA0NU9wmPtgMAMCNHrAAABpkaVlX1tqp6sKo+epz7q6p+s6oOVtVHqur548cEAFh8sxyxenuSXSe4/4okOya/did5y5MfCwBg45kaVt39wSSfPcGSq5L8Qa+6PcmzqurrRg0IALBRjLjGakuSw2u2j0z2AQCcUZ7Sz7Gqqt1ZPV2Ys88++9suvPDCp/LpAQBOyl133fXp7l6atm5EWD2QZNua7a2TfY/T3XuS7EmS5eXlXllZGfD0AACnVlX94yzrRpwK3JvkhyfvDrwkycPd/ckBjwsAsKFMPWJVVe9O8sIk51XVkSQ/n+RpSdLdv51kX5IXJTmY5AtJXn6qhgUAWGRTw6q7r5lyfyd51bCJAAA2KJ+8DgAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgM4VVVe2qqnur6mBVXXeM+7dX1W1VdXdVfaSqXjR+VACAxTY1rKpqU5Ibk1yRZGeSa6pq57plb0hyc3dflOTqJL81elAAgEU3yxGri5Mc7O5D3f1IkpuSXLVuTbY2rdIAACAASURBVCf56sntc5N8YtyIAAAbwyxhtSXJ4TXbRyb71vqFJC+tqiNJ9iV59bEeqKp2V9VKVa0cPXr0JMYFAFhcoy5evybJ27t7a5IXJXlHVT3usbt7T3cvd/fy0tLSoKcGAFgMs4TVA0m2rdneOtm31iuS3Jwk3f03SZ6e5LwRAwIAbBSzhNWdSXZU1QVVdVZWL07fu27N/UkuS5Kq+qashpVzfQDAGWVqWHX3o0muTXJrknuy+u6//VV1Q1VdOVn22iSvrKq/TfLuJD/a3X2qhgYAWESbZ1nU3fuyelH62n3Xr7l9IMmlY0cDANhYfPI6AMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGmSmsqmpXVd1bVQer6rrjrPmBqjpQVfur6l1jxwQAWHybpy2oqk1JbkzyPUmOJLmzqvZ294E1a3YkeX2SS7v7oar62lM1MADAoprliNXFSQ5296HufiTJTUmuWrfmlUlu7O6HkqS7Hxw7JgDA4pslrLYkObxm+8hk31rPSfKcqvrrqrq9qnaNGhAAYKOYeirwCTzOjiQvTLI1yQer6pu7+3NrF1XV7iS7k2T79u2DnhoAYDHMcsTqgSTb1mxvnexb60iSvd39xe7+eJKPZTW0HqO793T3cncvLy0tnezMAAALaZawujPJjqq6oKrOSnJ1kr3r1vxpVo9WparOy+qpwUMD5wQAWHhTw6q7H01ybZJbk9yT5Obu3l9VN1TVlZNltyb5TFUdSHJbkp/p7s+cqqEBABZRdfdcnnh5eblXVlbm8twAAE9EVd3V3cvT1vnkdQCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGmSmsqmpXVd1bVQer6roTrPv+quqqWh43IgDAxjA1rKpqU5Ibk1yRZGeSa6pq5zHWPTPJa5LcMXpIAICNYJYjVhcnOdjdh7r7kSQ3JbnqGOt+Mckbk/zbwPkAADaMWcJqS5LDa7aPTPb9p6p6fpJt3f3egbMBAGwoT/ri9ar6iiS/muS1M6zdXVUrVbVy9OjRJ/vUAAALZZaweiDJtjXbWyf7vuSZSZ6X5K+q6r4klyTZe6wL2Lt7T3cvd/fy0tLSyU8NALCAZgmrO5PsqKoLquqsJFcn2fulO7v74e4+r7vP7+7zk9ye5MruXjklEwMALKipYdXdjya5NsmtSe5JcnN376+qG6rqylM9IADARrF5lkXdvS/JvnX7rj/O2hc++bEAADYen7wOADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCAzhVVV7aqqe6vqYFVdd4z7f6qqDlTVR6rqL6vq2eNHBQBYbFPDqqo2JbkxyRVJdia5pqp2rlt2d5Ll7v6WJLckedPoQQEAFt0sR6wuTnKwuw919yNJbkpy1doF3X1bd39hsnl7kq1jxwQAWHyzhNWWJIfXbB+Z7DueVyT5iyczFADARrR55INV1UuTLCf5ruPcvzvJ7iTZvn37yKcGAJi7WY5YPZBk25rtrZN9j1FVlyf5uSRXdve/H+uBuntPdy939/LS0tLJzAsAsLBmCas7k+yoqguq6qwkVyfZu3ZBVV2U5HeyGlUPjh8TAGDxTT0V2N2PVtW1SW5NsinJ27p7f1XdkGSlu/cmeXOSc5L8cVUlyf3dfeUpnBsWzvnXvXfeI0x13y+/eN4jAJzWZrrGqrv3Jdm3bt/1a25fPnguAIANxyevAwAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMMtMnrwPAqeLroDidCCuADUaIwOISVsDjbIR/uBP/eAOLR1gBAMfkh6wnzsXrAACDCCsAgEGEFQDAIMIKAGAQYQUAMIh3BW4g3p0BAItNWAGnPT+UAE8VpwIBAAYRVgAAgzgVyNxshNMzTs0A8EQIKwAYyA+NZ7bTOqw2wh/uxB9wADhduMYKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDzBRWVbWrqu6tqoNVdd0x7v/Kqvqjyf13VNX5owcFAFh0U8OqqjYluTHJFUl2JrmmqnauW/aKJA919zcm+bUkbxw9KADAopvliNXFSQ5296HufiTJTUmuWrfmqiS/P7l9S5LLqqrGjQkAsPiqu0+8oOolSXZ1949Ntl+W5AXdfe2aNR+drDky2f6HyZpPr3us3Ul2Tzafm+TeUb+Rp9B5ST49dRXz4vVZXF6bxeb1WVxem8Xw7O5emrboKf1Km+7ek2TPU/mco1XVSncvz3sOjs3rs7i8NovN67O4vDYbyyynAh9Ism3N9tbJvmOuqarNSc5N8pkRAwIAbBSzhNWdSXZU1QVVdVaSq5PsXbdmb5Ifmdx+SZIP9LRzjAAAp5mppwK7+9GqujbJrUk2JXlbd++vqhuSrHT33iS/m+QdVXUwyWezGl+nqw19KvMM4PVZXF6bxeb1WVxemw1k6sXrAADMxievAwAMIqwAAAYRVk/AtK/2YT6qaltV3VZVB6pqf1W9Zt4z8XhVtamq7q6qP5/3LHxZVT2rqm6pqr+vqnuq6tvnPRNfVlU/Ofl77aNV9e6qevq8Z+LEhNWMZvxqH+bj0SSv7e6dSS5J8iqvzUJ6TZJ75j0Ej/MbSd7X3Rcm+dZ4jRZGVW1J8uNJlrv7eVl9A9np/Oaw04Kwmt0sX+3DHHT3J7v7Q5Pb/5LVfxi2zHcq1qqqrUlenOSt856FL6uqc5N8Z1bf2Z3ufqS7PzffqVhnc5KvmnxG5DOSfGLO8zCFsJrdliSH12wfiX+8F05VnZ/koiR3zHcS1vn1JK9L8h/zHoTHuCDJ0SS/NzlN+9aqOnveQ7Gqux9I8itJ7k/yySQPd/f75zsV0wgrThtVdU6SP0nyE939z/Oeh1VV9b1JHuzuu+Y9C4+zOcnzk7yluy9K8q9JXD+6IKrqa7J6ZuSCJF+f5Oyqeul8p2IaYTW7Wb7ahzmpqqdlNare2d3vmfc8PMalSa6sqvuyegr9u6vqD+c7EhNHkhzp7i8d4b0lq6HFYrg8yce7+2h3fzHJe5J8x5xnYgphNbtZvtqHOaiqyuo1Ivd096/Oex4eq7tf391bu/v8rP5/84Hu9lP3AujuTyU5XFXPney6LMmBOY7EY92f5JKqesbk77nL4s0FC2/qV9qw6nhf7TPnsVh1aZKXJfm7qvrwZN/Pdve+Oc4EG8Wrk7xz8gPjoSQvn/M8THT3HVV1S5IPZfXdz3fH19ssPF9pAwAwiFOBAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWwEKqqvuq6vJT/Bxvq6quqm88lc8DnDmEFXBGqqr/luS/znsO4PQirICFU1XvSLI9yZ9V1eer6nWDH39zkv+V1e/JAxhGWAELp7tfluT+JN/X3ed095vWr6mq7VX1uRP8+sETPMVPJvlgd3/kVP0egDPT5nkPAHAyuvv+JM96ov9dVW1L8j+TfNvwoYAzniNWwJnm15Pc0N0Pz3sQ4PQjrIBF1Se6c3Iq8PMn+PVDx/lPL0vy5qr6VFV9arLvb6acOgSYiVOBwKL6pyTfcLw7J6cCzzmJx31OHvtD5SeTfF+Svz2JxwJ4DEesgEX1S0neMLkQ/adHPWh3P9jdn/rSr8nuT3f3/zfqOYAzV3Wf8Gg7AAAzcsQKAGCQqWE1+cqHB6vqo8e5v6rqN6vqYFV9pKqeP35MAIDFN8sRq7cn2XWC+69IsmPya3eStzz5sQAANp6pYdXdH0zy2RMsuSrJH/Sq25M8q6q+btSAAAAbxYhrrLYkObxm+8hkHwDAGeUp/Ryrqtqd1dOFOfvss7/twgsvfCqfHgDgpNx1112f7u6laetGhNUDSbat2d462fc43b0nyZ4kWV5e7pWVlQFPDwBwalXVP86ybsSpwL1Jfnjy7sBLkjzc3Z8c8LgAABvK1CNWVfXuJC9Mcl5VHUny80meliTd/dtJ9iV5UZKDSb6Q5OWnalgAgEU2Nay6+5op93eSVw2bCABgg/LJ6wAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMMlNYVdWuqrq3qg5W1XXHuH97Vd1WVXdX1Ueq6kXjRwUAWGxTw6qqNiW5MckVSXYmuaaqdq5b9oYkN3f3RUmuTvJbowcFAFh0sxyxujjJwe4+1N2PJLkpyVXr1nSSr57cPjfJJ8aNCACwMWyeYc2WJIfXbB9J8oJ1a34hyfur6tVJzk5y+ZDpAAA2kFEXr1+T5O3dvTXJi5K8o6oe99hVtbuqVqpq5ejRo4OeGgBgMcwSVg8k2bZme+tk31qvSHJzknT33yR5epLz1j9Qd+/p7uXuXl5aWjq5iQEAFtQsYXVnkh1VdUFVnZXVi9P3rltzf5LLkqSqvimrYeWQFABwRpkaVt39aJJrk9ya5J6svvtvf1XdUFVXTpa9Nskrq+pvk7w7yY92d5+qoQEAFtEsF6+nu/cl2bdu3/Vrbh9IcunY0QAANhafvA4AMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIDOFVVXtqqp7q+pgVV13nDU/UFUHqmp/Vb1r7JgAAItv87QFVbUpyY1JvifJkSR3VtXe7j6wZs2OJK9Pcml3P1RVX3uqBgYAWFSzHLG6OMnB7j7U3Y8kuSnJVevWvDLJjd39UJJ094NjxwQAWHyzhNWWJIfXbB+Z7FvrOUmeU1V/XVW3V9WuUQMCAGwUU08FPoHH2ZHkhUm2JvlgVX1zd39u7aKq2p1kd5Js37590FMDACyGWY5YPZBk25rtrZN9ax1Jsre7v9jdH0/ysayG1mN0957uXu7u5aWlpZOdGQBgIc0SVncm2VFVF1TVWUmuTrJ33Zo/zerRqlTVeVk9NXho4JwAAAtvalh196NJrk1ya5J7ktzc3fur6oaqunKy7NYkn6mqA0luS/Iz3f2ZUzU0AMAiqu6eyxMvLy/3ysrKXJ4bAOCJqKq7unt52jqfvA4AMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIDOFVVXtqqp7q+pgVV13gnXfX1VdVcvjRgQA2BimhlVVbUpyY5IrkuxMck1V7TzGumcmeU2SO0YPCQCwEcxyxOriJAe7+1B3P5LkpiRXHWPdLyZ5Y5J/GzgfAMCGMUtYbUlyeM32kcm+/1RVz0+yrbvfO3A2AIAN5UlfvF5VX5HkV5O8doa1u6tqpapWjh49+mSfGgBgocwSVg8k2bZme+tk35c8M8nzkvxVVd2X5JIke491AXt37+nu5e5eXlpaOvmpAQAW0CxhdWeSHVV1QVWdleTqJHu/dGd3P9zd53X3+d19fpLbk1zZ3SunZGIAgAU1Nay6+9Ek1ya5Nck9SW7u7v1VdUNVXXmqBwQA2Cg2z7Kou/cl2bdu3/XHWfvCJz8WAMDG45PXAQAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhkprCqql1VdW9VHayq645x/09V1YGq+khV/WVVPXv8qAAAi21qWFXVpiQ3Jrkiyc4k11TVznXL7k6y3N3fkuSWJG8aPSgAwKKb5YjVxUkOdveh7n4kyU1Jrlq7oLtv6+4vTDZvT7J17JgAAItvlrDakuTwmu0jk33H84okf3GsO6pqd1WtVNXK0aNHZ58SAGADGHrxelW9NMlykjcf6/7u3tPdy929vLS0NPKpAQDmbvMMax5Ism3N9tbJvseoqsuT/FyS7+rufx8zHgDAxjHLEas7k+yoqguq6qwkVyfZu3ZBVV2U5HeSXNndD44fEwBg8U0Nq+5+NMm1SW5Nck+Sm7t7f1XdUFVXTpa9Ock5Sf64qj5cVXuP83AAAKetWU4Fprv3Jdm3bt/1a25fPnguAIANxyevAwAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADDITJ9jBUx3/nXvnfcIU933yy+e9wgApzVHrAAABhFWAACDCCsAgEFcYwXAXLk+kdOJsAIeZyP8Q5ecuf/YbYTX50x9bU43G+HPWrJYf96cCgQAGERYAQAM4lTgBuKQLAAsNmEFnPb8UAI8VZwKBAAYxBEr5mYjHEVwBAGAJ0JYAcBAfmg8s53WYbUR/nAn/oADwOnCNVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIPMFFZVtauq7q2qg1V13THu/8qq+qPJ/XdU1fmjBwUAWHRTw6qqNiW5MckVSXYmuaaqdq5b9ookD3X3Nyb5tSRvHD0oAMCim+WI1cVJDnb3oe5+JMlNSa5at+aqJL8/uX1LksuqqsaNCQCw+GYJqy1JDq/ZPjLZd8w13f1okoeT/JcRAwIAbBTV3SdeUPWSJLu6+8cm2y9L8oLuvnbNmo9O1hyZbP/DZM2n1z3W7iS7J5vPTXLvqN/IU+i8JJ+euop58fosLq/NYvP6LC6vzWJ4dncvTVu0eYYHeiDJtjXbWyf7jrXmSFVtTnJuks+sf6Du3pNkzwzPubCqaqW7l+c9B8fm9VlcXpvF5vVZXF6bjWWWU4F3JtlRVRdU1VlJrk6yd92avUl+ZHL7JUk+0NMOhQEAnGamHrHq7ker6toktybZlORt3b2/qm5IstLde5P8bpJ3VNXBJJ/NanwBAJxRZjkVmO7el2Tfun3Xr7n9b0n++9jRFtaGPpV5BvD6LC6vzWLz+iwur80GMvXidQAAZuMrbQAABhFWT8C0r/ZhPqpqW1XdVlUHqmp/Vb1m3jPxeFW1qarurqo/n/csfFlVPauqbqmqv6+qe6rq2+c9E19WVT85+Xvto1X17qp6+rxn4sSE1Yxm/Gof5uPRJK/t7p1JLknyKq/NQnpNknvmPQSP8xtJ3tfdFyb51niNFkZVbUny40mWu/t5WX0DmTeHLThhNbtZvtqHOejuT3b3hya3/yWr/zCs/3YA5qiqtiZ5cZK3znsWvqyqzk3ynVl9Z3e6+5Hu/tx8p2KdzUm+avIZkc9I8ok5z8MUwmp2s3y1D3NWVecnuSjJHfOdhHV+PcnrkvzHvAfhMS5IcjTJ701O0761qs6e91Cs6u4HkvxKkvuTfDLJw939/vlOxTTCitNGVZ2T5E+S/ER3//O852FVVX1vkge7+655z8LjbE7y/CRv6e6LkvxrEtePLoiq+pqsnhm5IMnXJzm7ql4636mYRljNbpav9mFOquppWY2qd3b3e+Y9D49xaZIrq+q+rJ5C/+6q+sP5jsTEkSRHuvtLR3hvyWposRguT/Lx7j7a3V9M8p4k3zHnmZhCWM1ulq/2YQ6qqrJ6jcg93f2r856Hx+ru13f31u4+P6v/33ygu/3UvQC6+1NJDlfVcye7LktyYI4j8Vj3J7mkqp4x+XvusnhzwcKb6ZPXOf5X+8x5LFZdmuRlSf6uqj482fezk28MAE7s1UneOfmB8VCSl895Hia6+46quiXJh7L67ue741PYF55PXgcAGMSpQACAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVsJCq6r6quvwUPfarq+rjVfXPVbVSVf/tVDwPcOYRVsAZpapekOSXk7wkyblZ/QLv/7eqNs11MOC0IKyAhVNV70iyPcmfVdXnq+p1Ax/+/CT7u/uuXv2y1D9Icl6Srx34HMAZSlgBC6e7X5bk/iTf193ndPeb1q+pqu1V9bkT/PrB4zz8XyTZVFUvmByl+h9JPpzkU6fsNwScMTbPewCAk9Hd9yd51kn8p/+S5E+S/J8kleRzSa6YHL0CeFIcsQLONK9I8vIk/1eSs5K8NMmfV9XXz3Uq4LQgrIBFdcIjSJNTgZ8/wa8fOs5/+n8n+fPu/lh3/0d3vy/JJ5N8x+jfAHDmcSoQWFT/lOQbjnfn5FTgOSfxuHcm+bmq+l9JPp7k8iTPSfLRkxkSYC1HrIBF9UtJ3jC5EP2nBz7uHyS5KclfJfnnJL+Z5H92998PfA7gDFWu1wQAGMMRKwCAQaaGVVW9raoerKpjXn9Qq36zqg5W1Ueq6vnjxwQAWHyzHLF6e5JdJ7j/iiQ7Jr92J3nLkx8LAGDjmRpW3f3BJJ89wZKrkvxBr7o9ybOq6utGDQgAsFGMuMZqS5LDa7aPTPYBAJxRntLPsaqq3Vk9XZizzz772y688MKn8ukBAE7KXXfd9enuXpq2bkRYPZBk25rtrZN9j9Pde5LsSZLl5eVeWVkZ8PQAAKdWVf3jLOtGnArcm+SHJ+8OvCTJw939yQGPCwCwoUw9YlVV707ywiTnVdWRJD+f5GlJ0t2/nWRfkhclOZjkC1n9clMAgDPO1LDq7mum3N9JXjVsIgCADconrwMADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwiLACABhEWAEADCKsAAAGEVYAAIMIKwCAQYQVAMAgwgoAYBBhBQAwyExhVVW7qureqjpYVdcd4/7tVXVbVd1dVR+pqheNHxUAYLFNDauq2pTkxiRXJNmZ5Jqq2rlu2RuS3NzdFyW5OslvjR4UAGDRzXLE6uIkB7v7UHc/kuSmJFetW9NJvnpy+9wknxg3IgDAxrB5hjVbkhxes30kyQvWrfmFJO+vqlcnOTvJ5UOmAwDYQEZdvH5Nkrd399YkL0ryjqp63GNX1e6qWqmqlaNHjw56agCAxTBLWD2QZNua7a2TfWu9IsnNSdLdf5Pk6UnOW/9A3b2nu5e7e3lpaenkJgYAWFCzhNWdSXZU1QVVdVZWL07fu27N/UkuS5Kq+qashpVDUgDAGWVqWHX3o0muTXJrknuy+u6//VV1Q1VdOVn22iSvrKq/TfLuJD/a3X2qhgYAWESzXLye7t6XZN+6fdevuX0gyaVjRwMA2Fh88joAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAgwgrAIBBhBUAwCDCCgBgEGEFADCIsAIAGERYAQAMIqwAAAYRVgAAg8wUVlW1q6ruraqDVXXdcdb8QFUdqKr9VfWusWMCACy+zdMWVNWmJDcm+Z4kR5LcWVV7u/vAmjU7krw+yaXd/VBVfe2pGhgAYFHNcsTq4iQHu/tQdz+S5KYkV61b88okN3b3Q0nS3Q+OHRMAYPHNElZbkhxes31ksm+t5yR5TlX9dVXdXlW7jvVAVbW7qlaqauXo0aMnNzEAwIIadfH65iQ7krwwyTVJ/ndVPWv9ou7e093L3b28tLQ06KkBABbDLGH1QJJta7a3TvatdSTJ3u7+Ynd/PMnHshpaAABnjFnC6s4kO6rqgqo6K8nVSfauW/OnWT1alao6L6unBg8NnBMAYOFNDavufjTJtUluTXJPkpu7e39V3VBVV06W3ZrkM1V1IMltSX6muz9zqoYGAFhE1d1zeeLl5eVeWVmZy3MDADwRVXVXdy9PW+eT1wEABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwAgAYRFgBAAwirAAABhFWAACDCCsAgEGEFQDAIMIKAGAQYQUAMIiwgv+/vfuLtewsywD+vM5YlT8BI3NjZ8pMYoOZEE3JSamSqLG9aIOZXohJm0AIwcyNhaokpkrSi3ojalAvGmODGqPEiiMXExmtieAlTUdKgHZsMpSGPH1R5gAACDFJREFUTsUw+KcaCZQJrxdn1545PdO9oB+stc/8fld7rf1l7Wfy7Tn72WutvRYADKJYAQAMolgBAAyiWAEADKJYAQAMolgBAAyiWAEADKJYAQAMolgBAAyiWAEADKJYAQAMolgBAAyiWAEADKJYAQAMolgBAAyiWAEADDKpWFXVrVX1RFWdr6p7XmLcz1dVV9XWuIgAAJthbbGqqgNJ7k9yW5LjSe6squN7jHt1kruTPDw6JADAJpiyx+rGJOe7+8nufi7Jg0lu32Pcbyb5QJKvDcwHALAxphSra5M8vWP5wmrd/6uqNyU50t0fG5gNAGCjvOyT16vqe5J8MMn7Jow9WVVnq+rsxYsXX+5LAwAsypRi9UySIzuWD6/WPe/VSd6Y5J+q6qkkNyU5vdcJ7N39QHdvdffWoUOHvv3UAAALNKVYPZLk+qo6VlXXJLkjyennn+zuZ7v7dd19tLuPJvlkkhPdffY7khgAYKHWFqvuvpTkriQPJTmX5CPd/VhV3VdVJ77TAQEANsXBKYO6+0ySM7vW3XuFsT/z8mMBAGweV14HABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhEsQIAGESxAgAYRLECABhkUrGqqlur6omqOl9V9+zx/K9W1eNV9Zmq+seqev34qAAAy7a2WFXVgST3J7ktyfEkd1bV8V3DHk2y1d0/luRUkt8eHRQAYOmm7LG6Mcn57n6yu59L8mCS23cO6O5PdPdXV4ufTHJ4bEwAgOWbUqyuTfL0juULq3VX8u4kf/dyQgEAbKKDIzdWVW9PspXkp6/w/MkkJ5PkuuuuG/nSAACzm7LH6pkkR3YsH16tu0xV3ZLk/UlOdPfX99pQdz/Q3VvdvXXo0KFvJy8AwGJNKVaPJLm+qo5V1TVJ7khyeueAqrohyR9lu1R9eXxMAIDlW1usuvtSkruSPJTkXJKPdPdjVXVfVZ1YDfudJK9K8tdV9emqOn2FzQEA7FuTzrHq7jNJzuxad++Ox7cMzgUAsHFceR0AYBDFCgBgEMUKAGAQxQoAYBDFCgBgEMUKAGAQxQoAYBDFCgBgEMUKAGAQxQoAYBDFCgBgkEn3CgTWO3rPx+aOsNZTv/XWuSMA7Gv2WAEADGKPFQCzsreX/USxAl5kEz7okqv3w24T5udqnZv9ZhPea8my3m+K1QbxBgeAZVOsgH3PlxLgu0WxYjab8GHngw74VvnbdnXb18VqE97ciTc4AOwXLrcAADCIYgUAMIhiBQAwiGIFADCIYgUAMIhiBQAwiGIFADCIYgUAMIhiBQAwiGIFADCIYgUAMIhiBQAwyKRiVVW3VtUTVXW+qu7Z4/nvq6q/Wj3/cFUdHR0UAGDp1harqjqQ5P4ktyU5nuTOqjq+a9i7k/xnd/9Ikt9L8oHRQQEAlm7KHqsbk5zv7ie7+7kkDya5fdeY25P82erxqSQ3V1WNiwkAsHxTitW1SZ7esXxhtW7PMd19KcmzSX5oREAAgE1R3f3SA6reluTW7v7F1fI7kry5u+/aMeZzqzEXVsufX435yq5tnUxycrX4hiRPjPqHfBe9LslX1o5iLuZnuczNspmf5TI3y/D67j60btDBCRt6JsmRHcuHV+v2GnOhqg4meU2Sf9+9oe5+IMkDE15zsarqbHdvzZ2DvZmf5TI3y2Z+lsvcbJYphwIfSXJ9VR2rqmuS3JHk9K4xp5O8c/X4bUk+3ut2hQEA7DNr91h196WquivJQ0kOJPmT7n6squ5Lcra7Tyf54yR/XlXnk/xHtssXAMBVZcqhwHT3mSRndq27d8fjryX5hbHRFmujD2VeBczPcpmbZTM/y2VuNsjak9cBAJjGLW0AAAZRrL4F627twzyq6khVfaKqHq+qx6rq7rkz8WJVdaCqHq2qv507Cy+oqtdW1amq+peqOldVPzF3Jl5QVb+y+rv2uar6y6r6/rkz8dIUq4km3tqHeVxK8r7uPp7kpiS/ZG4W6e4k5+YOwYv8QZK/7+4fTfLjMUeLUVXXJnlvkq3ufmO2f0Dmx2ELp1hNN+XWPsygu7/U3Z9aPf6fbH8w7L47ADOqqsNJ3prkQ3Nn4QVV9ZokP5XtX3anu5/r7v+aNxW7HEzyA6trRL4iyb/OnIc1FKvpptzah5lV1dEkNyR5eN4k7PL7SX4tyTfnDsJljiW5mORPV4dpP1RVr5w7FNu6+5kkv5vki0m+lOTZ7v6HeVOxjmLFvlFVr0ryN0l+ubv/e+48bKuqn0vy5e7+57mz8CIHk7wpyR929w1J/jeJ80cXoqp+MNtHRo4l+eEkr6yqt8+binUUq+mm3NqHmVTV92a7VH24uz86dx4u85YkJ6rqqWwfQv/ZqvqLeSOxciHJhe5+fg/vqWwXLZbhliRf6O6L3f2NJB9N8pMzZ2INxWq6Kbf2YQZVVdk+R+Rcd39w7jxcrrt/vbsPd/fRbP+/+Xh3+9a9AN39b0merqo3rFbdnOTxGSNxuS8muamqXrH6O3dz/Lhg8SZdeZ0r39pn5lhse0uSdyT5bFV9erXuN1Z3DABe2nuSfHj1hfHJJO+aOQ8r3f1wVZ1K8qls//r50bgK++K58joAwCAOBQIADKJYAQAMolgBAAyiWAEADKJYAQAMolgBAAyiWAEADKJYAQAM8n/sS49HlYD4AgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x2160 with 7 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "m = 4\n",
    "less_than_one = [2**(-b) for b in range(1, m)]\n",
    "greater_than_one = [2**(b) for b in range(1, m)]\n",
    "\n",
    "tvals = list(reversed(less_than_one)) + [1]  + greater_than_one\n",
    "n_tvals = len(tvals)\n",
    "\n",
    "plt.figure(figsize=(10, 30))\n",
    "\n",
    "for i, T in enumerate(tvals):\n",
    "    v = i+1\n",
    "    ax1 = plt.subplot(n_tvals,1,v)\n",
    "    ax1.set_title(\"t = {}\".format(T))\n",
    "    transform = TSoftmax(temperature = T)\n",
    "    tprobs = transform(acts)\n",
    "    plot_probs_bar(tprobs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To summarize, as $ T \\rightarrow \\infty $ it renders the probabilities into a uniform distribution. All samples are equally probable. This is not good, our char rnn will be predicting garbage.\n",
    "\n",
    "What we want is the opposite. We want to chose one from the more likelier ones in each go. The choosing part adds the randomness, and the temperature changes the distribution so the less likelier aren't chosen at all. Hence we go for $T < 1$. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

Great work!! You can write a blog on this :D