davidpfahler/how_to_train_your_resnet_fastaiv1.ipynb

## how_to_train_your_resnet_fastaiv1.ipynb
{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "how_to_train_your_resnet_fastaiv1.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/davidpfahler/6f2be99f1c54aaa25aa4523ecd443676/how_to_train_your_resnet_fastaiv1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OKqJD7RwMOKQ",
        "colab_type": "text"
      },
      "source": [
        "# \"How to train your ResNet: Part 1\" in fastai v1\n",
        "\n",
        "The goal of this notebook is to reproduce the results of the great article series called \"How to train your ResNet\" [Part 1](https://myrtle.ai/how-to-train-your-resnet-1-baseline/) by myrtle.ai. If all goes well, I might implement the entire article series, but we are starting with Part 1 for now.\n",
        "\n",
        "You can find the code for the experiments corresponding to the diffferent parts of the article series here: https://github.com/davidcpage/cifar10-fast/blob/master/experiments.ipynb."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HBNI1f-vCG_P",
        "colab_type": "text"
      },
      "source": [
        "# Setup\n",
        "\n",
        "Install & import fastai and download the CIFAR-10 dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "31Pw8JjsJP43",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install fastai"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_NP5L7hrCF1D",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%reload_ext autoreload\n",
        "%autoreload 2\n",
        "%matplotlib inline"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fTRD0vpIDQF-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from fastai.vision import *"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uxPKhALbChdZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "path = untar_data(URLs.CIFAR)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zjx0ruKgj4HM",
        "colab_type": "text"
      },
      "source": [
        "# Transforms\n",
        "\n",
        "The below is supposed to give us the same transforms as in the Part 1: Baseline  experiment, namely `[Crop(32, 32), FlipLR()]`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AHUznZRXiNxm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "crop = rand_resize_crop(32, ratios=(1.,1.))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sY1zckY9kZnU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "outputId": "bfd757ef-1f7c-4daf-815d-03d57d7b28ed"
      },
      "source": [
        "crop"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[RandTransform(tfm=TfmCoord (zoom_squish), kwargs={'scale': (1.0, 2.0, 8), 'squish': (1.0, 1.0, 8), 'invert': (0.5, 8), 'row_pct': (0.0, 1.0), 'col_pct': (0.0, 1.0)}, p=1.0, resolved={}, do_run=True, is_random=True, use_on_y=True),\n",
              " RandTransform(tfm=TfmPixel (crop), kwargs={'size': 32}, p=1.0, resolved={}, do_run=True, is_random=True, use_on_y=True)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PJSHFbUUu_8j",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tfms = get_transforms(max_rotate=0., max_zoom=0., max_lighting=0., max_warp=0., xtra_tfms=[*crop])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E8VGEK7BkIXO",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        },
        "outputId": "aba9095c-3165-40e6-f893-c42e62e8df9c"
      },
      "source": [
        "tfms[0] # train set transforms"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[RandTransform(tfm=TfmCrop (crop_pad), kwargs={'row_pct': (0, 1), 'col_pct': (0, 1), 'padding_mode': 'reflection'}, p=1.0, resolved={}, do_run=True, is_random=True, use_on_y=True),\n",
              " RandTransform(tfm=TfmPixel (flip_lr), kwargs={}, p=0.5, resolved={}, do_run=True, is_random=True, use_on_y=True),\n",
              " RandTransform(tfm=TfmCoord (zoom_squish), kwargs={'scale': (1.0, 2.0, 8), 'squish': (1.0, 1.0, 8), 'invert': (0.5, 8), 'row_pct': (0.0, 1.0), 'col_pct': (0.0, 1.0)}, p=1.0, resolved={}, do_run=True, is_random=True, use_on_y=True),\n",
              " RandTransform(tfm=TfmPixel (crop), kwargs={'size': 32}, p=1.0, resolved={}, do_run=True, is_random=True, use_on_y=True)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bdVDoIOYlV4Q",
        "colab_type": "text"
      },
      "source": [
        "TODO: check if the transforms are actually identical"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Em3uv7BzlYgp",
        "colab_type": "text"
      },
      "source": [
        "# create databunch"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6QIGq9O93XYo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "batch_size = 128"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9ZGZf2KEix88",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data = ImageDataBunch.from_folder(path, valid='test', bs=batch_size, ds_tfms=tfms)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v_Lbl0l2z5qn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "cifar_stats = ([0.491, 0.482, 0.447], [0.247, 0.243, 0.261])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "odwwR8Rj0ECj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 306
        },
        "outputId": "3bec9203-1680-4907-fb33-614b52ade70a"
      },
      "source": [
        "data.normalize(cifar_stats)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ImageDataBunch;\n",
              "\n",
              "Train: LabelList (50000 items)\n",
              "x: ImageList\n",
              "Image (3, 32, 32),Image (3, 32, 32),Image (3, 32, 32),Image (3, 32, 32),Image (3, 32, 32)\n",
              "y: CategoryList\n",
              "ship,ship,ship,ship,ship\n",
              "Path: /root/.fastai/data/cifar10;\n",
              "\n",
              "Valid: LabelList (10000 items)\n",
              "x: ImageList\n",
              "Image (3, 32, 32),Image (3, 32, 32),Image (3, 32, 32),Image (3, 32, 32),Image (3, 32, 32)\n",
              "y: CategoryList\n",
              "ship,ship,ship,ship,ship\n",
              "Path: /root/.fastai/data/cifar10;\n",
              "\n",
              "Test: None"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TGnWaj3JKr9p",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 369
        },
        "outputId": "7abfc16e-ba3a-407d-e636-e81dfbad3eae"
      },
      "source": [
        "data.show_batch(rows=3, figsize=(5,5))"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVIAAAFgCAYAAADpZ/FJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsvXmwZetVH7a+PZ35Tt23u1+/1/0G\nDUgCIWQExJYSRCAMAhKVDY5dNhkMxMaJgSIVsCklUcUhuFJlV5y44lDYFAFiYgU7timohAIxOWaS\nhAY0POkN/Xruvrf73nPPuM8evvyxfmt/63Zf3fek897tPq31q+o6t8/eZ0/f3vv7rbV+ay3nvSeD\nwWAwfPGIHvQBGAwGw6rDXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxF+jJw\nznnn3Osf9HF8qcI592XOuY8650bOuR980MdjWA6P6ngmD/oAXg045y4R0fd573/9QR+L4VXHjxLR\nb3rvv+pBH4jhVcEjOZ6PPCN1zj0Sk8WXMJ4kok8etcA5F5/wsRiWxyM5niv/InXO/TwRXSSiX3bO\njZ1zPwpz/Hudc5eJ6IPOuXc7567e87tLzrlvwt+xc+7HnXPPw+T4sHPuwhH7epdz7opz7t0ncW5f\n6nDOfZCIvoGI/gHG9p845/6hc+5XnXMTIvoG59y6c+7nnHM7zrmXnHPvc85F+H3snPu7zrld59yL\nzrn/AveGTa4PAI/0eHrvV/4fEV0iom/C308RkSeinyOiHhF1iOjdRHT1mN/8V0T0CSL6MiJyRPQ2\nIjqFZZ6IXk9E30pEV4joax/0+X4p/SOi3yJ22xAR/SwRDYnoncQkoI1x/pdENMDYf5aIvhfr/zUi\n+hQRPUFEm0T06xjP5EGf15fqv0d1PFeekR6D93vvJ9772StY9/uI6H3e+2c942Pe+ztq+XcT0U8R\n0bd57//wNTlawyvFv/Te/3/e+5qICiL6C0T0t7z3I+/9JSL6u0T0PVj3zxPR3/feX/Xe7xHR33kg\nR2w4Do/EeD7KL9IrX8C6F4jo+WOW/zARfcB7/yfLHZLhVYAe19NElBLRS+q7l4jocfx9/p71v5B7\nwnAyeCTG81F5kR5Vwkp/NyGirvwHTu1ttfwKEb3umO1/NxG91zn3Q8scpOFVgR7XXWIW86T67iIR\nXcPfN4jNQMF9fm/DA8cjMZ6Pyov0FhE9c8zyzxJR2zn37c65lIjeR0QttfwfEdHfds69wTG+0jl3\nSi2/TkTfSEQ/5Jz7gVf74A1fHLz3FRF9gIh+wjk3cM49SUQ/QkS/gFU+QDxmjzvnNojoxx7QoRpe\nAVZ5PB+VF+lPEtH7nHP7RPRd9y703g+J6K8TvzCvETNUHcX/e8SD9GtEdEBE/5g4SKW3cZn4Zfo3\nnXPf9xqcg+GLw98gHs8XiOhfE9E/IaKfwbKfJh7TjxPRHxPRrxJRSUTVyR+m4RViJcfTIRpmMDzy\ncM59GxH9b977J192ZcNDj4dpPB8VRmow3AfnXMc59x7nXOKce5yI/lsi+r8f9HEZvjg8zONpjNTw\nyMI51yWi3yaiNxHRjIh+hYh+yHt/8EAPzPBF4WEeT3uRGgwGw5Iw095gMBiWxInmqH7bd3ybJyJ6\n7OLjzXdnHj9HRESjyYiIiNr9AX9/5kyzzmNnHyMiolObm0RElM/mzbLf/I3fICKiT33y40REVBYL\nIiJ6+vWh8t23fPt7iYjoqTe8iYiI0g7vYzbLm3XyIe9/dMBWwp07d5tlN2/fIiKi69dvEBHRrVv8\n/93dvWaduuTA4YULfG7f9V3/frPs6adY7tbppPiG141j16wjlsEbzz4dvnzI8eQTW56IyLlwyI6c\n/MEf+NSGj6wty6IozOeu+R1/5yJH9+4jy/g6pgnfvr7i61nVZdgONhnjM1JXtdkv/oiw8PA6fMC1\n5y/Lsm6WVTiXLgR0j51lgcfp7V6zTr+fERHR3/n7f7gy4/mXv//f8UREg/5a891kws/TaMzPiq9x\nrdTvioqvTWeNFYOnT59rln35W76aiIi+/p3vJiKipx5nGWiWhldPXfPvx9MJERHtDHeJiGitv96s\ns97lv8uKx3i6CM9uG/dDv8UDsj/m7fzxZz7VrPPRT/0RHz/x73rdQbPsxtVP8zZHl4mIKElQO8Vl\n4TrM+f3wD3/iV44cT2OkBoPBsCROlJFGqJI1m06b765fZTnn9ZvXiYio3eHZ/c52SDwa7zE7nJ9n\ntpdGodrWfMwMcjHnlPoJtn3phZBldv0y7+PM2YtERNSq+bSvvBSkpJ/78CeIiOjWTZ4NbylGehv7\n3xvuExHRaDzmfc1CGn8U8zHltcf53G6WPX6ez6Xb4XnLgzkt6kDTquqBS+G+YESgmQ0LJSJJVLlv\n2j7OF+/1amCAWL/V4rG68GSwUE49zYwp2WwTEdFsyL+5c3nUrDPa4b8XUx6joizu220MutpqMfNI\nYnVfweqR3/k6LBO2W+MYb9zmsbuzP27WSZLV4yjFnNnaLA7Pp7D0VsL3bJHzp1P3brifsW47PLtr\ngz42xOM4zpnhJkWwHuQa39zlBKbPvcRMcvtUsFy3N88SEVG+4HXn6veba8xWT21s8D7mPHazeTiP\nIuf74ebOs/z/atEsq3KMW83rL8oF9hXWKYv77x+N1Rttg8FgeMhgL1KDwWBYEidq2ldw2E9Gk+a7\nYsgBm707XLWu32eHfazsPXH5pnBKb/SDo7gFx/AWAlFJyg7nsgim8rUrbDI89To20dcdO6dv37zV\nrPMH/+ZDvO51Nu3vTkJAayrBjAhmK6afKgnBhTjjSzkhPp6bwyBtG8Gc6VZ8bLXn8yirEMAojzA9\nH3a0E74eURSudYgJff4Yi4xsTRJICsskkCNfnTnH1+zL3hLGvHeRzcX9dYxHxWsnT24065w/4KBG\nOuExKsfhWtcLNgvbHXYNnDnDQZJWljbrvPjCJSIi+uSfcMGv0cF+syzCphY5/3Eg91oU9rGKHKWC\n2Tyh4KKIUwn68f8TjLlXpn2aYBwzfmYWPri1bu+xmf6p53nbvZs8HnrMy5qXjcb8fN7d4+d1NLnU\nrHP1FtccqipeN1LF9Nttvh/aGbsFFzm7KHZu31A74b+7KR/j3IVgFUnQ1/HvW57fOO0y3NcLFZg+\nCqs32gaDwfCQ4UQZ6QIzXumDo3hR8wyRJswGNteYVfTbge15zPhDyI1aFGajrQGv33oKgQc4iPf3\nAyO8fYNno91bN4mIqAfndNwKDGRY8z5GOI5qvd0sy9rtQ+tLrKuOw+VzCFxEA1731jyc4/UJBzxc\nj5el8OD7Kszq9erFmqjbEoZ+P/uUM/NC0hQFETJTeZnHtQyMv1tbZ1bw+FPMNta2ghRlOGF28FLN\n13W0xmPePx/qzDz+NpbMven01xIR0Rvo6WbZIGeWmyDItL7F+1j4EFz42Cf/mLdNHKR49sMhkBUh\nuCLBNpFPHaJZbvU4ikjMXByYddri80jB1iPc/E4H0xBEjVu8rNML12FYXyIiomKHLb32mK99lNwf\noKw8PwR1xu+J2g3DKiWODVapU9f3YIIHcsTbKRGIyqvAIn2f/+60+ZlNlfHg7/sL0jf1fOYzCzYZ\nDAbDa4oTZaR37zKjzDqKyWFm6kEm0WmxL6SYhxlgeIdnphslS6QOzgW2KfNap8cM9vQW+7sunr/Y\nrPPxT7Cfa/8O+26efAOXLr34+lA05umvewcREVXXeOYc12qOgdBXCFQN+ugVM/aYKSvi2fBzO2E2\nbX2OC3k/MWJ5xUaHt9dWfp54ZWTbAd02/GX+KLYJdoBrpLvayNpCarRnsYML8QQsjKdex77O0+ef\natbZmfJ9MHbMIIcQTk+UxGqRsk9z+zG2eN65HaQ0b0743hBB/tzzvfbCJPjM64rvp7W3bRERUfv5\nYKHQhH2IKR4fUU1FipnHKyh/2nwcsrJuuC/TDv+dwBqTWECUqnsXEjURskeJWgZLzcfwn0Z8rY+S\nzMkt4lJI4Fy4M0LyBpipPnD8vMYfJaRadSf8PmrBj+7BrJWcra7lPj7MSLUf2HWOf1Wu3mgbDAbD\nQ4YTZaR5zrNSRcHfJYxUGIsvmBEulNh9OmYm52TGqML7X9JFszZv8/wFZjBvfvObmnXmEGXv3mYf\n6XCf97F+8almnfUL54koiPVneXBaii+08eFIhF3Nar7g2a8oEDEcB0b9ieeYQV25wcqEHmb1rkos\nyMBgvuddf4ZWBb0OGIBOn6zE38XnFkPpUCrRvUTrE/jivKKkaYuv8fYTPJ5nz7IPPM7CmJf7rPrw\nI76eqfj2stD0YJbzGF8q2Rr50MUQ0S9P8xht99h6SRPe1zgOfu2yzwx04zzfF/2NfrNsseD7qQu2\nJuL0yKlIdrjFVwannmb2ra91nIlPVD6RIhqrdfB3jFRfbWHcm47r3GHWd+ivexYdbaS5Q6tqNIYR\njkOPB0lqq8ezrNhmBGtSGK0wU0ldJQrn+PlgjNRgMBiWhL1IDQaDYUmcqGlfisA1VlqfWgS/COCU\nEMxOgmk/g2keg9YfHIQc2oN9DupEqCaTobJTUQRaPpvw+tcuc3WX7Re58/JT68HcI5iFPmOTrq6C\naV5CalGKfep4X07NQx7HVsHcnysx785dPv7hPh9HAuMnVQbKKgabMkhjVIo6oTgPeZGQoNJVoays\nqhlzrFOq6lG4I2sEGp6//lkiIrq5EwKMNyGDmyDvWzw9LlMBhAFLoS5d5t8dXLjTLPv442zuP3nh\njURE9MbH3kBERFv9zWadN/VZLlV3eV+fbv1uOAHcawlM4EYGpcYzjVevzm9nE412Fb0SWV+j7JJq\nXto0v9cUVjKwNOJxSCL2dVQI7BVVeL7LpmoXXD6RuIWCPDFFYFb2oSWUkuDSeAb84eMiInL3ugu0\nHA/bFtdhjefTK9eAO9KZEGCM1GAwGJbEiTLSAkJZp4TsIpVwOBQns5CajRzkLTInzJTYfTxDNRlo\nk+ZwKmv97HjKwYWDl7giVHudAwfR2lazzmzKjLQCk9QppiWCTCWqwqRwZjsl2PUScBEirAJR4tie\nY8aT+pma9eq/VwUScNB1VctI0l/lGz5nHXspERWQWp+VYgcSlLhzly2Na7c4ULc/V+l6GOMKVNij\nLikp2U2EtNHoLkuVdiafbpY999JzRES09jiL7t/45rcREdHbL/ypZp3Xd7iGbIEkkkxZDB7/kSCZ\niMtbSmSuWfqqQIKpSnVEjkQAL0Ge+tAnURhHoXvtONQzHaQsO2slSHqoOVA4qm426xS11PXleyWJ\nmcVudUIVqfUuahEv2Kq7Ow0WxmQxPnT8zUOoAkry8mjSkxVbbeRP9wSbdMWyl+skYozUYDAYlsSJ\nMtIJfJVKLUMtiH9TMIhS0ic1I01Fccs/HM1CKt9wDClKyr7NKuXZbBpIK03hSxvu7xARUdxiv1uM\nit5ERLPT7CeTdFTSjBRMukBV7hTiZO0jJcyGHsxST2DNlnDeUttSZ4XWq+dSo0x8kopRFmIZFPId\nxteFs40adsNIlLQkg29xkUvRGF6W+EDxxFoQn3Vz6VphO6gPQxXWrdvKD4tiI4ucU4enQ2ZJV6+9\n2KzzzPk3ExHRYM73VZaq+xGJCE4cusT3Y7sd9t/KTvTRelWwKPiaR2o8pUZr08XA3SteJ/Kg5tIl\nInKhdmzLs9WXEA9IhtTvVBWIEf9piWdnEPFvnugHCeOT57jjRQ0f69W7LzTLrg+5rvAYNUfLmp9T\n7YcVWaJU2JdCKUREVZOffZiZ1uqhrLVG7wgYIzUYDIYlYS9Sg8FgWBInK3+CiRwlwcxLGktacrNh\niul3vEiTEKTJ58G0r5GVQsjRn6PW6BXVmO4AwabZlH935xY7ql/83HPNOr5i2VTu2TWgO3+ETCbU\nJRWJj2raVsFDXyNbJ1JZS5LtU98rr9DtGvzqzWndLl/7XMmXvPSbkIZy/rBJSBSkQSlkZbXK297c\n5uufbvF22nDx7B+EAbmxw8GFGiZ6HMGto5qVVbj+FaQtVRyW+RYHPuqMxzzNuBrYwX7Yx2fmHyUi\noqdQz7SXhXuu6PJxZ3BJyL2SpcpFkaxetMnDNNf3tWShST2FpuqSrnSF52JxwK6ByTTUI73TkcAN\nX4/B4DQREa2ffqxZp5dy9tgBWoNMdvh5/dzVEFDyr+Mxe90b2OXyxvOhTsb5LXbNzBb8OS9Y8rY3\nvh6O44D/nhQcxJwuQjWvOYJVRS2Zl3DPUfAPepM/GQwGw2uLE2WkjcxAVT1qBPCocu0xk3ulH3GY\nIUXHX+4FtkkIPC0wY+5J7c/bu2EfCFJJ1f18wjPXzethxnJrnKNfIv+60tVhELCQJl8lZqpItZSV\nWbyWqj+KkXrJ0ZcWv/X9jJTq453ZDyNSBFRKNR8nKZ9rH5ZFgmulCkQFVoMv41ZYePF1qFa/xQGL\nBRjRx54LwYVpwmyi7oO9ijSmG/Yx7cCKwHi005Ar31tnsX3rKQ5gnPtyFuY/3g7rjD71m0RElD3/\nB0RE1IlD4MJ3mSULA609W0xOVQPThG1lIKUsVHHc5u8m2ITqT069OnDvzpAok6nq8xUSbaoFP+fX\nbzD7e+H5K806i7k0TERl+g6PQ0LBCti9zs0p5zlX9Xr6jV/XLEszXr+b8D66KQer1tuh4teFTSRv\n4L6czlUHixlX5h/O+J0xWfD7ZZyHd8jBfIeOgzFSg8FgWBInykibLDNVGifqsX8q2jpHREQVZnmv\n5BEOflBhpE5p1+sRz3CzEhInpJOmqn/OGir5UMQzVwXpw2IR+jItpuhTE0GYrys7QT4lTLISn55T\nNSrFVwtWosmmg3TCwbnqRHyvdGB+BUvk7x+AMSgLo4fx6/QOt54ua10Fnz9TMIjOqcAE6x77LRfw\nrU4nvI/dUbg+I6SWDgaodL/GAvCyHVhjHjE7kmr2gyzcc6d7uO1xI83m7CvfeDywnDdtfxUREd1J\nWRK1yIL14nEjSh5AHKeHzgtr0apB9OzaNmrSJBtmCqtKydkIvcdqj+fJB0F+VLO1sMh5PId77Pc8\nuBNqvyaQsW1ucp1Yhyr2hbLSItQRfeEz7Lv2almScar3AdphTyejwydERJ0u3ysXn2GZY7sbknFi\nxGD68TbOg7edl+F+2puEBIKjYIzUYDAYlsSJMtIUUc5EsYN4AEa6jSie+J1aKqkQflQU1yZ3J0Tc\n6ohF1VJPRAT9aTsw2u3HOFKYLhBlhvB4GAVGuQdSVSByWKk01BodBCvpF4MeTonqAioVuF0sfmA1\nR6G2qpvjE32lKq+lAavHYDYQvdbuwA5E6W34EX3N1zxSkeASnV7vgMm+WAZf2I3PcNfJvGQ/9jrE\n2SMf7odinZ2h4x5/luvoHLuuuro6ZikOPvRcOWkPPLrY5uynG93lMX9i+2yzzjvfyL2e3gK/6XO/\n/2vNsqvwm86RYBLBn69F+Fmyek7SppPtUQ5eMFMxPnQV/DThcVjbZl9zfBDqwuYT/ltqAuc5+ybr\nKjzDVcH3yk2oaW6+wD7K2SwUJ3rqDO/j3CledzYKEf285O+GB/yc5njOyAfTtY2q+zcvc7T/TW//\n082ybg9MtBKft1RmUQkWebi3joIxUoPBYFgS9iI1GAyGJXGipn1fGmepxmASiMlzNsHEuiiVwZjG\nyNOFODtSroGsz8GJ9oCpf9MaYhBOrQsfeBcOa6mD2Y9VsKhEjvicVx4Pg+mR77HkosQxxmjUV3cC\n3Xdt3n+Mz7QVtj25zdKJcsxmjS+R96vK7KyeYU904RySF7R0C7KYFs4/2+Jgw2wtBCB+//IlIiJ6\n9vI1IiIa70yaZZKYIc3WEshWon64ntGAr3uBsY87vKzfP9esk0D4nSMAUnq1DwQ4N7fYpDuPamC9\n+HSzznSD7793nPq3+TzU/Tje/xwREU1KNj33YVK6KrgoWup4VwVS3YyOqMYVWm5Lm5hgvp9bg3xs\nk8XyEzWeO2jvUxQcrHNo5+JVy+cFAsUHB/yc7I9Q0U1VFctgppdzvr/GY3XPxXj24G5otVHP1On3\nDO/jygtcZ6OuwjE+8+av4XON+L6qkGCib+tctT46CsZIDQaDYUmcKCM93WMGU6qUwAIBlxxSpBoB\nHC0l8VGCT9S6LEIgqDNgVtLZZiF3b4MZ0Hwe2iFPkfqWggm3pJq+qjA1yHnG20Jga6wagIlahhAk\nqTzPmF6lqiaOj7+NykB9FewaoaHbFMGJKY5tNAuzol+92ARtQH600N0EMDRxxud8N2Y2v68Ca7cS\nnt1vb+HCzoMUJb3F25JAFrX4XtlvKanaBgeS/CkeKxczk/ias29r1vmax7+aiIj2WswWbyZBgL3d\n5/W3USMzJR6rtopvPobK7luO2U5XVc9vd3hZhhtDgqhVGa5D4sO9sSqQFGZda1Q6P0j7Yo+mkLlK\n2U2Q7LB5nmu4Ui8I2SddBJkgcpfKaZHqZiB54h0I+c8OUN+1FVhvCwxyUfCYr/XCsj7uh7pEFakc\niR7q7RY5sN49fvauozYxEVELluWpMzh+BEhz1QBzPgv331EwRmowGAxL4kQZ6TpY2lxJQ+Yi8AWr\naWpMLtSMDr9lHUHcXYQUtBbYQasNtguWuH/tRrNO5Hg2aw0gxcH8UbnAbPto0zqAf6WlaltmXV62\nucG+tds7LCaOEiX6h/Rm8zGenTtrQWRedvmcRj2e4XZv8vHf3g1+2OCDWh1kSP+rizBWFSp4RKCm\n9T6P651JsBA2IG1rw8c6eiYwUo9MvDs3eP2kx9d4uq0SJM5BiH+WrZCz6VuJiOirL7y9WeedTzxD\nREQ34YO/5oKPa4BCGl2SQhz8/VOozE5E9BZUeY8rYWnh9x3Ux60ySfTgc6wUg0mS1eMoTequrn6P\nayN1Xec7KOxxJ7Dv7pTrgXbjZ4mIaDoN6ZQTVLJ3MT+DbTDJvAj0fzFjayGTdtiIZcR1eD7HY/67\n3YUUUrUsyKQwCuRLVSyxEFUDF6nLmylbFpNRsAZvXWMZXLslNVORFKLGUzolfD6s3mgbDAbDQ4aT\nFeRL0Q79+kZkrk5F3M3/XyiGJh385Pda3C2FM6TE3mSHZ8OdF0K18zZmrN42/FyiDFCpje2EZ6Mo\n5RnXqeraKRjHZp9nuuke/7/fD5fvPJjo2SeZXUWtUEEj2ea/RwM+7lsp/IBpmPHiFWzyM58yo66q\nwBwyOKbaYAyPIaI6XARx9fUJ+8u6kgSxHdg7vZUjwIMZisfc4qjvgUrRrDeh/hig+v6Cr2uqSvWJ\naOIxqAiyMrDNLIEfuyFgfPyb2irAtq7NmFFdvfLxsExYFtaXAv+RsrSybPUsDC+8Sl8GKfQzYitq\ndAMJK8PwDN50XFBmNufyeVEc/NG147TNtS32Yw7WmYnWPihedsdI88Ymp0g91pX62wMpaAKVTxEK\nF03QoC2NkdzjmFFWKi25ypmBiqu+p1Qk0xFvax/pq/01xEDUi8q6iBoMBsNrDHuRGgwGw5I4UdO+\nguBX0+Q2Ag8pRO6zBS8rVK59C8v6ELsnBypPF5WcSghmF7twdO/dbdYZH/A6I5F3iGmv5E9jYnNk\nuM/m4mS83yzLkX/vCM3vUP1nTYmuN/r8u7UuWgUn4RwdAmhyShsDNjMHaagSnibhWFYFdQ3htKrm\n1UJzszYkZm2IpF+3EVrr3hnxtd69cZmIiKorIThRnEYF9Xdw4OirHAviR3eDXOUjB9xa+S4SGzYR\nIBwfBInKR569REREp0/zvXN6EORLNSRW+/t8j+RTDkzeaI+bdZ7b5lqWdy+z+P7Op367WeYO2BR0\nTuqSwr2kXEVxtHr1ZUs8H4l2cRR8HsUQlctyBF631ptV2j2Y/WNOsGh3Az+ToJBL2dxe24DkTNUl\nKAoE6yq4BCAXdKrmaX+Ttxm1eJ1ZGQLOCaqoOdQATqTddxWez7KUBn18j8RxcL2leDCn0wmWwZWo\nOh7o9uxHwRipwWAwLIkTZaSnznDV83knOHpnbcgh8P8KTv60FQ5tgGo/612e1aaDMJscjFgmEyMF\nbBtSpceffqJZZ7LDjEM6qs7RV2imWrJOh+xo9pBEzeaBnVQV5FeoGvXlr3+KiIjWuiGAIbmtB2DC\nkT7HA57phreYec33eVZtKRbq3OoFm9YgbHdq5o7BQCJJnsBFXyjH/WTE183t8Wdfpd/lnmuDHqAN\n8uRtvI93PfbOZp0v73Cb3j+5yusm+7zuZ67/cbPOC9jdm97IaYuXy2th2Ud4vbvXP0NERGmLx+fC\nV15s1nnyHVwdKEMdyk4nWDjzHjPxbM7n2obIvC5Vi2G3ekm/EujVhDQf87nWUAttbXEQ8PGnQsvl\nJIM1iBbc/bWNZlnW5eegLdYk7vmsHRilg1RsbYOtwMUM7Fc3kECK7wLMMu2EZy+BFRSLZQSG66uw\ngQgie1/z+0FbUZ2Ez7FAtakxnlfNSKuX6WBhjNRgMBiWxMkWLdlEap8qFjKFdsTHMhvyIdUUfBLT\nGbNDEXkXqn6l1FBsQ9i/tcYzzeYgiLyv58xaa0hzpDbleBH2Md7nWaiueaZclEGuE0HYW6IOaQsp\nq1EZZrwcAt8cxU+ibli2e5OZ6BDFS+bjCbarajqqGXJV0JN6oKrbYilFQpCWW0pXgSpYEa2YfcMd\njL3bDckTbVTCr3dZUnN59//l778uWA9f/SRXsv+Pvoyrnc93+X74aH6pWecOenfd/DRLcq59/BPN\nsmuf/RAREXni++LiW3k7p09/RbPOdsLs9NTj7FvdVz2brn6G65GeWrA1NTjF91rlQtriofzEFUEP\nKdCRGs/RgbA7/v/aNq+zdiqcaxusM0nZrxypYiM+ki6i/FGj0n6k5H6Ddb7G/T76tmEd3TSiLKX4\nDAoHKWtOOhQQ0ljrhdQEDhvw0DzG6DTrlA+7Dat4H++J4d5u86tmHypd9SgYIzUYDIYlYS9Sg8Fg\nWBInm9mEfPhE7TYWyUUpufYIBM2DKTdBIGgMkzzfD3myi5xN8VbM60dt/n2WKfkR5AwFAh9zZOLM\nVBWpArn9NWQVWqLVQjOujpgQ+N1ion8PyQW8DtU4HP/uDQ5YjO+y6SCVZCa5qiewgv17PcZDFT2i\nomTzKi9RNxKVePaUr74L8+p30f79AAAgAElEQVTs6bcQEdHN28HMy6+xJCq5xeNQ371ERESXRr8R\n9vsuvu5n3vrNRET05NPsKrihMqzGL3AmVHmH3SlOVe+JcB9k6+xq6m2zGT9dhPvyNsZsgIy1wXbI\n449FYYeMnDFM4qnKzZ7i3ngvrQ5S3POlaqEjNXjzHOeKFjwH43CuZc1SqEzKZ6nAaY0g8KKA68zz\nuEa6VigaRjq8C+rmU7mM8imODRLKKDyfdS21DqS5JGriJiF7qpUh+ItKVYt5OP4FMvTyGs9nwW4c\n5UGkVn28680YqcFgMCyJk/WIS2M6zU5kNhmxvGQy5plnNgmsc4bqTx6i2EhrY8EuYojk98FIo06Y\nFceQNRSNJIdP26mip5IbLvnGSRoCYmt9ZhzrqDxTLXjGmqqGdQ6ibHF4jxchaWC0zwLuEqy3RKWr\n4TisM5+tXv3K63tgYvNwHdAnkKRYjlSDmmeBEQ7nLK5voYZs3AtN56bXmZFm0nv7BhoG5kGQ/8Lk\ng0RE9P+gkvqZc6/n49kJ1/PuFa5ItHaL95tmgTb3NxAc6fFYlVPezrMf+qNmnSuwcPYvcoLAppL0\n7O+Cld1lljaacUBrPA0Byr19BKf+Kq0MClhjhar+FHXQEG7E53x3j+l4XofnM0MVrAySRU9Kyocu\nEFUtwTq+ru5QFX4EaPGdl8QZVcNWnv36CBmSlxrGSB6oYQX5IrzevOd7tZbPQHapLKW5Je41bK/X\nGTTrDNas+Z3BYDC8pjhRRipzUKr9j2B3Q/gPJ7eZvQkzJSLKG/+l1CtU28RG4Zojj7bIRTfIIzx+\n0O2iHinEvDqlr7fGfxdoG6vZahd1VDcGqJOI9E+vlMuhbiMzoKli1B6SDalxmmUsBSrUeQwpVMxZ\nFZTbXIV+eCOI3auCWdo6KujENbPEmWIXL95mecn+8JNERLQYK+YAEXUJSituR3czUIhqwn13PrnD\n98ont7lXk1PttasdZhU3r/DxuGFI+S0rZiAZxmO6B4tB+c1qVKt6YYOZyLmLoZ/T9mlUzx+gjS+O\ntaTbzTpJvXrjWUvutCKUndNodV2yb3Gyw+d1cDXUl3W4d2M8D5ozuqbnk1xbpHOqdZq+UE62JzWK\nlcXXtEjGF7qlBB4+YZkV5E+VSiONIqR545kv8vD7HH7fJOPPPtK9B7q9d3q8xWiM1GAwGJbEiTJS\nmUwiNWdJIfqNDrPFYoPZmmp5RPOcZ4gKjFQn3yUQtbfRB6oPP02/q/q9dFBRHT4PD/+nV/PiJvr3\n1IhSViqi75Cr1m6j7iV6zFRKMbyAX2WB6v2Jqk25uck+uVSSDaSvkQsn2V7BOe30GZyXqqvahsj+\n1Gleto8U3j/+nd9r1jm4yaz97ugSERF5HyiQQ+S1yuHrxrYXqpuBm/F4OFgqi+fYb9dWfZVaA+4m\nO4SFMb4ZWHOMbaW4ZxIUL3G1qsKP8XczXtbZDEkDa9t8rybrX0ZERGc2+b5aV8x6Y2P1GKlHEoyu\n99tGfIDOwVeK+MLiVji/OYrFRGCJ4hcnIqrrw5F4Xx/+1H9LSmjUMNMAKaQihxbFKuov1uM9jc90\nid9UHjVYRrMyvINmMHvgDm7eCzrxZ6iKIB2F1Xt6DQaD4SGDvUgNBoNhSZysaS+UXTHwTCq/wBSX\nEoRra8E0L2DSV8jf9sq4FzOkhdzmTsKf3VYwm1vIpaUMjnPUIdXbWYPpkEA2pR3dFUwHh2Z3JfLx\nK5U0UMIMqEkSAsKl7WC/MZzi5QyugSwcY9Q5Ppf3YcT1P/ldIiJyUTjXss9BGb/FAaA7aP/wsc+E\nep7XbvJ1qyNpIROuY9NeArKZDOu0VP52Y3kiHllDbO/LYH61e+y+KVsIXnWUKShmIg5bYo6pcjn1\nB/z32W3e9mA7BC6yNuRPBSeKxO5JIiJaVw3vBqq26coAl7jWvjNc9xSttwd4zpzyvc3vosYnnpla\nJSakSMwo4bJZiDsmD2Oe4zmSBnsx9pko0zzDfSFX+JARL/+RVkRYN1buwQj3U4ljrFTSgfyuRIWy\n6VRcDcG0z1vHJ8wYIzUYDIYlcaKMtAYD1O92yRQTMe8aFCyDKDA0YbLCSOmIRlTi6E4xN7RUZaUY\nLNU3Mx1+rwoeJrU00ML2UhV4EMkEZrV8zuwkV7mRC0gtZB/xof1jv2CkDv/vqnqH9QpOaf/6d1i+\nFKvgRNRGSuULHIzwHWak8zwwtAG6CLTaaEqotM6dTb7GrU2+tuuQpYnYmoho/4Dvg9EQQmwEn9Ke\nags95xa79ZxpSdJWWRwYGrF+MmSIbAxCQOvMKWZZp/DpNlWAVNiJY0nVXUj4kiTUyJQqWKuECg+j\nisNQhdq9wkwjdCwcPKHE6mfxN57PbBGe8IvElkk8521PRiyH08GbvTssPzuYIA0UwUAfq+fTSbAL\nAn8lT5SKUhLYqvCZz5RViXXkNprOlXTRo42zWIwQ8vsypIW2lbTuKKzg42swGAwPF06UkRYoKnG0\nrAGfSA1MlQ9GlhWYTtRkFFLNRKtfSEpamFZdLT1dJBUtxr70djALinRDF1WAMw0TFy3gO6lUvcMq\nQq1U+F4KVes0ySCbSpmxdFGPta0SAiYHx8srHkZc3ecZe9AL59pPmZ21958lIqK04HN841tDPc8B\n7rpOj6/V1uPhWifnMX59HuTtFv9/724Y9Cs7fK2HyAit7unFRUR0cIW/m43QE0z1ESLxzaI30PoW\nj+djjwcWeeYUJFKwWubKRzbYhtUBS2c84d9PU+VHXV+9IjQi4asVv6rAACVmIP5LXT836kkNYSnu\nE6zJp9tPExHR6ZJ956NdlsNdqy8161wdcvqvLzlRYxf1h4eVqjsM/7XUN62VVSr1S0XIHzVvGLUO\nzqlsJFrhvB2OO0HsZNBnK2prPdTQ3egfzzmNkRoMBsOSsBepwWAwLImTlT8hq6BSMpMaNDyDmVTD\nxC5VbnaM9710T46VLkIqx5QLcSZDKqUc5tE9WRWStxun4fQztGR1hfRqDnai1EV0sl9k2+gKU1EM\nU0dkUOrK1jjupMfr9BNUkVL5u/7m6rXvFXP53JMhSHPqLF+3rS3IllA0qW6Fc+2g0lbc58/1c+E6\nDjFGt2/BLDtA8MmHC7qFbfa20T6iA9eNuoS762gVjYDWfBJ+n2L/2QDBkdO8btRR9WWxLfRFo0jV\nt03WeFvrcD/cugo3grLm+2ur1/yuuqceKBE1WkUxm8WT5n242JU8M1gnV7Kh4YwDcn1CViEifXES\ngjct1J7oOP4ulnq9B6oCHCq3deEXaqlnL5H7KRb5kxxjGBCJU9dT3k6i0p5Entjr83GswaTfWAv3\nzPqamfYGg8HwmuKEGSkcxkrW4CEFKqTBHOoVJkq170UEDPbq1es/gfPZS5HT7H4mIJIJqZRP+CxV\nYdM55A0xgkYtpXboQjhfIr93D5KQJFGV/tEGOq55dqt0L1nMfiUCaQU+EyUST/ur1/xuDR15t54O\n17ELJtpD70EPRjhRCQ4zyWmGzmb/RmA3N+7y3ygGRrex7uu3w/2w3kPAA9c86vD/96fhOFrr/Pfm\nM+h4EAe2ubHF44nC9rQz5GO7uROOcQg2s414gy6BO77F/3sTgpeIkdA4V1XNbtHKQWRDuuanJC9I\nXFSsOt3wUIK3YumVaqyHJcvg1lM0sAQjTVTCTAdtzaU+xuaMn4XK95t1cjzDkoATHWFxhlcG6l7o\niBICUmu4V1rKGpVD6cAiWeswo+6rd4AEij8fjJEaDAbDkjjZCvnie1Gv7xq1PXPMcF6q/ChtkoPz\nS+a5WlVlSdGWNZbWvign5dWsKjNsCgG8A2stSPVVQpX0pODP7UGYjtb7PBtVIuiHtrzSBUUxKyeQ\nSsUqpZEwQ0sFoznxPro6jXTj+ArcDyM6p/n407PhXNe2+Rq/4TG+fjfQu+q5S0EalFbwaYG1T5Rw\nuoRlcQ5sVyrtf3YvSJM8CjGdBYO5cIavnRbtj++yf22ElMSkG9hJD9WmzqCyUYL76+adwLJkd3MI\ntyd3wjFuz3j90QbYGVx5ZXDp0X6+ej7SspERKV2g+BlxaRuyqU6vxvUTZlpPwjjsFiy23+hxZa6B\nY2lRmql2zh0ev0Gvf2hf62vrYR/YYSUSRBUEkSpNJZJqFuhAQUW4ZxyYdIL3Q6qSSOBipV4myTz+\nnpMlqmsT5BsMBsNripNlpNVhYTxR8CXOUbVchPE6AisBQmGWi0VgN204M9tSGASR9Ur5xKQ4iTBS\nwueiDOvcusNOuQyq7n4n1LZ06QDbxj7hm5sMg8g8R+QyafFxtLIwg0knxhKetgJ+2IXyA3c2gz9o\nVXDmMQjqzymx+iZf/90Fn+O1yyhQshd+l52CLw4/y5TP/EmI86XU66UXeNndOyqlDwbJMOJtz+ac\nWnhaRVnlrxws8c6tsI/dF2D9vBH75LKi9PTT4b586bO8/pXn+P+Fcrc9xlmPlMIfPIOPTTeadP3V\nE+R7SWFWKTPyrDpwriYBRkXtpY5pE+1XDZFGCY/NgefsiQ6x0zlOg4+025cUU95mp4vawIp1CiOt\nURhGa1wkdbyANSk93saTkJYs/bQWYs0qZcICMY+JqHvkvaDSQrtJSIk9CsZIDQaDYUnYi9RgMBiW\nxIma9iXaIre6IYe1kQaBxs9RAWam1M2pCOGdyBwCLRdTPpHud2JmxMHsFiX/HE50lBOlqaqbWCOA\nkLQlbzhghspOccS2Wxemx51hcGZLnUUR5seqbcUcAuMM7VSSFi+bLVRyuBIYrwq663xevUEYjzla\ndFxCG+QJWjUPTilTFy2zJzD3p6OwLEbloOEYZhbM5XNPhJ9L2c8D/P5ZBLI+MQ/jsb4p9wpvb3RL\nibMRJLx7nsfsHMzXTivwivmE743prlQKC/ufws3QhQzr9j6vs9ELv988vXrBJl/dbzbLeTfel8Yk\nVqa9yAtxrZN2uJclmDyp2McyL/m5FBOfiKjV5mBuDLdQD7Ilr+RLUvrCS1BZ1cKQ1iBV4/rj52oy\nCQ00x6g6NcU7KF+E98NttEu/c5e/m5SoK7Ch3lPJGh0HY6QGg8GwJJz3qzdzGgwGw8MEY6QGg8Gw\nJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQG\ng8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvC\nXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAY\nDEvCXqQGg8GwJOxFajAYDEvCXqQGg8GwJOxFajAYDEvCXqQvA+ecd869/kEfh+EwnHM/65z77x/0\ncRi+eDxKY/hIvEidc5ecc9/0oI/DYDB8aeKReJEeB+dc8qCPwWAwnAwe1PO+8i9S59zPE9FFIvpl\n59zYOfejMMe/1zl3mYg+6Jx7t3Pu6j2/a1iscy52zv24c+5559zIOfdh59yFI/b1LufcFefcu0/i\n3AwBzrm3O+c+gvH5p0TUVsu+3zn3nHPurnPuXznnzqtl3+yce9Y5N3TO/a/Oud92zn3fAzmJL3G8\nzBh+h3Puo865fefcv3HOfaVadt4598+cczvOuRedcz+olr3fOfdLzrlfcM4dENF/cqInBaz8i9R7\n/z1EdJmIvtN73yeiD2DR1xPRm4noW17BZn6EiP4iEb2HiNaI6K8Q0VSv4Jz7ViL6RSL6c97733pV\nDt7wiuCcy4joXxDRzxPRFhH9X0T057Ds3yWinySiP09EjxHRS0T0f2LZaSL6JSL6W0R0ioieJaI/\nc8KHb6CXHcO3E9HPENFfJR6nnyKif+WcaznnIiL6ZSL6GBE9TkTfSEQ/7JzTz/V/QDzOG0T0f5zI\nCd2DlX+RHoP3e+8n3vvZK1j3+4jofd77Zz3jY977O2r5dxMP7rd57//wNTlaw3H4t4goJaL/yXtf\neO9/iYj+CMv+EhH9jPf+I977nPil+aedc08RT4yf9N7/c+99SUT/MxHdPPGjNxAdP4b/GRH9lPf+\nD7z3lff+fyeiHL/5GiLa9t7/d977hff+BSL6aSL6C2rbv+e9/xfe+/oVPu+vOh5l/+GVL2DdC0T0\n/DHLf5iIfs57/yfLHZLhi8R5Irrmvffqu5fUso/Il977sXPuDjF7OU/qPvDe+3tdPIYTw3Fj+CQR\n/cfOub+hlmX4TUVE551z+2pZTES/q/7/hTzrrwkeFUbqX+a7CRF15T/OuZiIttXyK0T0umO2/91E\n9F7n3A8tc5CGLxo3iOhx55xT313E53XiB5GIiJxzPWLz8Bp+94Ra5vT/DSeK48bwChH9hPd+Q/3r\neu9/EctevGfZwHv/HrWdo57/E8Wj8iK9RUTPHLP8s0TUds59u3MuJaL3EVFLLf9HRPS3nXNvcIyv\ndM6dUsuvE/tmfsg59wOv9sEbXha/R0QlEf2gcy51zv1ZIvpaLPtFIvpPnXNf5ZxrEdH/QER/4L2/\nRES/QkRvdc69F9Hc/5yIzp384Rvo+DH8aSL6a865r8Pz18OzOiCiPySikXPux5xzHQSGv8I59zUP\n6DyOxKPyIv1JInof6P933bvQez8kor9O/MK8RsxQtYn394iDVL9GRAdE9I+JqHPPNi4Tv0z/pkV9\nTxbe+wUR/VniiOxdIvoPieifY9mvE9F/TUT/jJj1vI7gP/Pe7xJbE/8jEd0horcQ0YeI/W+GE8TL\njOGHiOj7iegfENEeET2H9ch7XxHRdxDRVxHRi0S0S/wcr5/k8b8c3GGXhcHw6AIR4KtE9Je897/5\noI/H8OjgUWGkBsORcM59i3NuA2b/jxORI6Lff8CHZXjEYC9Sw6OOP02syNglou8kovc+KImM4dGF\nmfYGg8GwJIyRGgwGw5I4UUG+P0H6e2hHNX/83gd+hYiIPvTLv05ERHcuXw+rlCX/rqyIiMgdeaT+\n0PaOmoWan6lTdfjW+erwykpSJ3+9/6O/6mhF8PX/zf/iiYiyOijJOm6NiIi6xJ+U87kX8xAor4sF\n/uJr7qholnU7vK1Omz89rlldhHWqmv8uKx6IxSLmdeqwjyjiZXHMy8aLRbOs3+OAb7/LxxhHvK/+\netasc3AwJiKindu7RES0vxcSopKM1ytqPv7pnD0F5SIc4+aps0RE9MGf/cGVGc8f/y+/2RMRTXFe\nRERxwtevE/E51zmfo6/0A4JTxFetVtwsaXVTXiPip2WBazSfzZt1Euxjc2uT18XzlcXhCcuwzbzi\n3++Mxs2yc6e2+PdrLLRJ8PAWi3Ae1OFXXbzW4/2r38d49vsp3wdRzecznoRjLHFuf/kHfvbI8XyU\nM5saeDxw49uc9Vnv3CUios4wXMwIN4O86w9dLbzwmvceLqqjI65p8wI96kVaH1rVRfe/SFcJHg9A\nc5cRka/4pVLjmlPBt5ir9YN3eJaSlycREZ43yhf8UpRnqdNJm3Vq3Oh5yQ9Vu8u5Fr4OD3BR8otz\njge/OR4KL2dP/FnhxXEwCusU2Fbc7RMR0WIeXrILmXTlkHD8i1l4cGct9RCvCGLHF7uThms9xaRV\n4VpnCcZTsYiqlOvI109ejEREMQa01ZYJkuuUtNJwPSdTLmuR47q2Yt5HTfffM2sDHg9KwjEm0eEH\nUyZPCofRTJbzBR9rmYdJT8hTXvOxOtxftbpn5Tw+H8y0NxgMhiVhL1KDwWBYEo+saa+Nggrmdpmz\nzyOBv61bBZ9lDLu9MSfc/ca28/fY9sogb9y/+NT7j/C/SJyr+N2RroEVgmuxueRI+X5hnlULPldf\nwL6q1a3mef5uDGntK4aDrKqrQ9vpdsLvxXSsY17mYr6+lbKmxR1dYifaNEvwp7vHtPdFWCeHuZi3\n+btis9csi+B2cClGOcP9NQo+tWn7Hn/4CiDCNYqUSSxWep3jfHCJslYwzeX8FzlflywLZne7zeuJ\nae9hNpdFuD4R/DcFxjyB+0C/nOTxauOeS5Kw/6Lk6x7HDsv4oKMqjKfz/F2Vs8unLsITWsuxlHz8\nEe5PUveMK48fT2OkBoPBsCQeOCN1RzC/z4cvJOiv16yFgUT8bUISbFDRSRzHkfPOPQT02FJTR4T7\nAyM9zHbdEVH7VUKEgIErQ0Scy34SVXDq14jSRi4wiAgRYAnwlUUI8kRZjXX4czbnbeelYqQpD6jH\ntfYOgaVFiNpPcz6OAtvO0sAZ2i3+O0t4/5MZr1PHYRRmuBPGKc4jC6UXIo9jwTF6h4CMD8c4j1dP\nnx2nfPwSGScKDNCDrTlco6wdWKcEoGY4/bQVroMw0Qg0dwHFhlgcREQJGKw8jXI3eM0IHVgm7plU\nPYUVhk2YrQSbvAv7KOY4J2wyVpGoEhbJHJZqTMLMFTWvjx9PY6QGg8GwJB44I/1icIiZilxJ2B0+\nvZqx5G+RSaRgMpULTEjIiEiVyB8lbTq86NAcde/qipnKojC/3e+HdUfy3IcbFfxdi4WSdeFvB3rh\nRZOorrX43dptlrJMpypjE0wua+F6xMwqRnoduXwY+26bb+ORkrNNpsIumO2stQKD2hwwS0pbbazL\nLGminKzDmP+eRPC/+cB2XasxcfgTvtVYse5yBTMG+x1m3eP9cK5JBr8jrp+wx1QxUpErUYQWTEfQ\ns7zgbS4go4rT8DR0sa0JdJ/ivkxq/VCBZUKWls9CJ6BFzePX6kJGByZbKz3saDQhIqL22oC3lgTJ\n3QyxklIc69j/XGmfu51DxeDugzFSg8FgWBKrz0jDl4f/qxiezBY9RPo6YBILlWkUBSfnEds7vO1a\nxPvquyYCLx9qoXPiIz28vcP6/NVjMFEbkWxV3dMnOT6RIVbLuQZ2UNfMLh1YRhQpITuY5GhSYl3e\nTlEEBiLXOgHz2B/yOgfjUTgOXNz1PrPeLj6JiKYzZq7D2zeIiOjS1R3+XvnU5mCkkrTV6QUGFq9v\n8HFkzMBqEtF5iOxLBHiV0ENiwVoR2NcIfsM0Q2ZQBsWEsjAKZI15MLtuL1wHGQd5vhr/qQuMNJL4\nBJ65KSLrU5Xx1ob/NgGTjLphrKoJrCBE5gs4TWfzcF+lLT6nbpuTN/JFuJ8iifbjfnQIquhExJcz\nMIyRGgwGw5KwF6nBYDAsiZUy7Wvwa50DS+XhohaSoz2dheBEhRxoN+ZlrcMp70REFHkxD+7n8DEi\nJ40sA0nWToWPGqm+iHm1qe4OS3qa89EBsRUMTrhWF5/qvCDO9pCbNPUEVPJDBbnUdHpARI2Gn79D\nDvRczEWY27NpMNtryHMkQFfCjM7zYK5lUN2nKKIyPAhmoux3Z/c2ERHdxufChwMpMdpi9uWDpnci\ntU+f4WU93rbL2JTN1jaadToqF3xVIHUNTqnkg+kuN++skUefi0RIBUqzDME7x68THadtN8J9XEeI\n3/NCSQ+x40EXRUeQ618oEbyPDvvMTp850yyLdkWIjzoAKKgSqffExiaPTYRg2TwPgUkJVGeSYALX\nQDttN+uIDO/zwRipwWAwLImVYqQSQJoMD5rvdi5xD7udy/y5N9ojIqLZJLCTeoKZ6bMv8icoUKaq\n3MRwbLsmKBJms8SDzbjDspdYR4tkGhYnuqrsFIGRxiI5xiLNT6uXEfw+jPAQ1jtVyYdSnKPEFHJc\nDxXIKSGBmSHoUyq2KkylgHC7RPrfdLLbrDPPD7AOj7ETJlkF1lAiOBGVHGSaT0Lr8+GQWdbwgO+V\nouTtlIolSUUjEDGaqIpGvTu3+FQHzHKSPtjOuQthnbPnadXgcbLtdngtrA0giTpgC8/jri2SsE6F\nxAZJDZ3qkom4r9soPRjhuagqHcnhbbawzbZIrZRkLRWGD0ujreRIXZS/qxGkisBat/qBUW6d5jG6\nO+f7SRUso6pJI+Zja9ilYtZRevyr0hipwWAwLImVYqQl/BTXnn2h+e7Tv/uHRER0/XPPExHRJGfh\nrdf+FWhYesMhERGdw+xy6tRms04GRlrD36brVyYVz2wRLldILdWpkZKuiAQBlV6WwXeU1iiOixlY\nS7TK1SOk5GeQOil2ISmEMdh+miD9UsmfCML1UgrwlsFfNZ8xS1zgkwpmn2mt5CqQT5WN7or3n7UC\nhUglXbBAIRFVfDrDsXRTXn/q76+ZKuck/thKpcEuhlzP1uNeK4fMlvPboVB49cxX0KqhQFpvnITr\nuNbhe386BCNtavOEa9UUG8HzsVDPXi1VYySJQlJ1k8DhCnl2cM0lxVNbjAnkV+LHnk7D/dAUypmi\nhi0kWuun1ppVOj3UjJ3zvZYra7JoilSjnioszrkac8qU1XUEjJEaDAbDkrAXqcFgMCyJh860P04G\nNB+zeXHj2UvNd1c//CkiIhrf5J46VSy598G0rpENcZDz7zfgTM9awWG9hpzsElWGFodKbKJeopcW\nCDBpVUpPqEQkecdh/wsHuRRkFVL5Rpz72MLnPe+HFnMJQARpUYlKSBFqhcYZskR8mLMlN380YfN9\nNAz9kOqS3S8J8bazGOaVU2aWXGLkdosp6HQ6mVxbf0/7CSLK4NrJEECS7JtcBRcka4oQSKuURqtG\nTn6NflROsq9UPYD9Fz5Fq4Zc+jGpZzCCu6SD3HiRIWn5k29qx0IapcY6gZuglGpP0u+sHUxlkQHO\nUPMgg0Sqowuj4phGcwQGVUArnbL7pkLFr47j57ofh9fbaMFjcwCJ3FQFJuV5jCO44CDZGuVhHRU7\nPhLGSA0Gg2FJPDSM9JXo0XPM+CM0sSMiyneY1bSk3iByeSPVgbBEEGGM5Nmx9GVT1WWSe0o7TVy4\nNDcinuHyFKJsyJnqI4JFXvJ1laO8lFqM9+SIx2rGrKch4LIyENreCte6RjChgAwqgrC+qkJwYDzl\n4Mze3ctERDQbBUbaStDQrg0pSiRC8FB9Xi57hgBACkF4pao3FZBYlQh8pEpsJo3XulL9XgJhU8XE\nUJchFctGBScm8wl+h6uvDSwAACAASURBVICUBFdUsGoxvE2rhgUY3WgYxqrT4fM/vcVVk2qw/lJb\nUFJZH5+lomeDAVsNBao/TeYSoFSV1yDEj2ApLhBUdmVYJ8LfEYJUe7mqPbvLyRpxxce6hqBueWcS\nDgRM9gD30VzVQ22aUyKg1RlwQkJcKBlXfDwlNUZqMBgMS+KhYaTicjlKXiEpXCXS0yrlu4jBPFKw\nCplokkiJ7bGtA/jN5mAQhapg0xwApBAT5W/7XMQzVH3uCd4X2GZVBh/OHEUUS5FSqBTBCjNtAbmN\nVCTqd0NFov2XXqRVQ42ZP1E9epyTnvMYjwP2ec4PrjXrDHdZHL+3w4w09oFtdvpI04P/c4ox39sP\nSRjSr1wY6cYG6kkqP+Ycwuu59BFSSQMb6yyL6Xd5XNd6zLaKPLCUGFZEr81sKVOC/NaYx3Mf1abG\nuchuwj6y7KF5tF4xKlh144PA5NIWXyMPyyDFWEda/iSp29LOWdUqLfHMZagVmkJIn6vKTFNYLTWs\nuRrPyVzJ6mqMp7RzdsrijGVsMWYJ6sz2uirV1TEjLZyw5sAwW3ieJZ01zSRlNUjmFibINxgMhtcW\nD3za/LxRev29VL33UtczzFQpCSNlViDphlGsGCmIZ4LZMZaGiEnYR7O2VMd2YY7ZWecCCetv/1O8\nLvywxUylPS5QSxGz+kRFFbMuz3Q9+IukSnerExjpYhoY16pA/MFFrfrfLNjXWyCNsnvrEhERVXsq\nMj/l+p91hUIkKgK8QOdHjz5KE4j+x9PAWguw1AhMMIdVUlfKZw1GXMEXl6aBAUm6nwPL7KKuqE4J\nFP/nBH6yQhWoSaRupyQf4NgWKsU0ix/4o/UFQ1QPWpAvfZPE1yz+5ZZi3+ITnS7u9xXXVfOwERFR\niuuix6pdS780dPJFnKJQqpaFbAdFiTY7qgcYnuNd+HYhHjhUQzbZgB8X91qiYhgpvhO2LdX7E2VV\nFC8TwzFGajAYDEvCXqQGg8GwJB6I/XFs7c1jFslbP1MrdZH33oKJLxIjPUVIW9UOWhG0YO7FuuIS\nhPxiTBRqA6M2ByeyjVO8D9QnnSmR+Bz+gynkMnPVSrYlBaGw8f0J/242V83alPN9VRAh77lYBEH+\nfJ/N9vwa10NI9tnEn4/vNuvM5mzSl8hjT1VgroaZJe07pCpWqcy8UqQz+K7WjfGApvYBfq/rzNYw\nwSOpdpRIe+bwOEzgNphLXdU8/L6HgIfEK1K4CArVDpqUvGZVIKZsz4Xaq3LZJMFF2rykynWRiVgf\nz+B0qhoF4okqcY94WOSxao/dd9IGWtrK8Ph4FVCS45CqUXGhAlFItNnfg1sJgaXOWvj9esbnFMMd\n01IBUjmTGgMqnzq/Ph8qKdURMEZqMBgMS+JEGWnDEnQKWhNoONw8TpPWBeRO47tcR5ImYXbYQOBg\nY50DOBGkRa31c806NQTbdJmr88Sotp5oliLqJ2FEShm1u8v7v/JHLNeZHnBCQFXqIAlaE0OZ76Jw\naRtyCpaSwZmeqFS61kFgbCsDtMSt5qr26y5fY3+H5U4VKpHP1TpS/V6qrh9qXQ3mE8UIaqD6fpIE\n1ikyJ7mPJMDoDmmmD6cydhW76OPvLhhoek+iBFGo0DUDy9TVq2SbbQReEgk6hQ6Kh5r9rQpiPEv9\ndpD9DMf8rIlVJ2ytVIJ6YfLrfZY2aQtFqj8VYPYVrMlOJ9QK7aGeqASSFggmOjWgcq2d1KudBmtw\nNOJ7Ywo5Wg0LZ5oPwnHso+LXgN8PSSsEDyXIViBhZ477y7XCdciLIR0HY6QGg8GwJE6WkYpPUvsm\nMbFVpYiBWQa0e/VGs8rVF1nAffvTnyEiosnll5plBfwyQ1Rg3/I8U52ehTTSOueZpS+sqMaMp5rL\neMwpEb6Lq3Bphns8Q13/DPv/EqQ7lkr2s/BSnAMSLeXfkRqdCzBSJ/VIVd3GrfJ+P9/DjsUO+vnk\nIR0ym/DfPXQVqKV2q2JoUneySRPUyRe4/imq77dRR7KYB/mTFA3JF4f76BxyeWNoO5C5XDh7tlm2\nBetFCpmMJrztXNfRlOIaTettRXchjUvBxBKw56oKTGyh0htXBaJI0tKmuuLncQGWFoHJLZQ/uAWW\n+fjpLf5C9TEeIx4wlZ5HskixzVp81HgduRzPjrqEbfhRJRt5pDpg5HNI3PAO6AwgT9sIrPcAPdyi\niMe6FYeCRZRJYRXEWTCM2fpWs0qcHd+DyxipwWAwLIkTZaSzMc8iY1V05OAm/71zgwXbt28w+xze\nCOtMEI2L9sCA9vebZbfBVO6C8ZxHNNCpqGAKP09WyiwKH4gu04UZEsF7ihUDKTAzjhGNlLnMq8tX\nCSPFNKor7DvpV1TDN+iEkYWZO/KBzawK9na5YEwyC2L7rGDWHlfMAGaSmKASHKSM3RzR9mqh/I8d\nvg4D+EY3N5g9+jIkL5SoXO7BfNriA1eUVEavBz/X+XOnmmVnz/AIVmAg+VWkk+4H1ps3RTF4m1pZ\n0FhR4ut2Uign+NRqXWRlRZC17mddGfyXBxC7SzJDrCrcC5OTOMOZ06HzRNLiMXbjw/fDeBKuzwiW\n2iYSVk7FPOblWHUl2OdneHbAftB8Gp4X8cm2kJraXePxda3wfMbokCA9pEpVJ1OsjaYtFP6oi7D/\n2h9vYRgjNRgMhiVhL1KDwWBYEidq2n/8Dz5MREQ7nwjVw2fX2RQcozHdBO2Uq5mSjyD/ui/VffKw\nbAIL7DICEHfTXP+EiIguYv0BKHzUVH1SAaHmL6l0H34vzuszZ7eJiCiZspnRSoIzO2tLtXaRT6k8\nfAlCRNIYD/IZZS5md0JwbVUwG7OLJVOBPUKFex+JaYxWvbo+a3lYrF4rF4dUW4+aRmQI3qlqPRLk\n6aDKz0avjz2FdZprDpNsmodg3mwuAnJIcyqRU6n6l9ifuH/08YvbppHpQDZVq/2vYHftJhmmUm4p\nkaiJjGkIEzvNglus2+Pg1HjEy9bPheciTSUIy+MqjfF0mHCKVs+SdLEGCVytRPCTHb6vpIp/oXL1\nJ3geux0eo40e7z/TFaLwrIk7rVTuJAk0V5AsNjHxIrgPDnezuB/GSA0Gg2FJnCgj/ejv/D4REeUf\n+3jzXRezmFTX7ks/nkOBHDA4zIqVmo366IeUSQUatPrdiYKEYxMSqRZyNFtgIocE3CLEx3e1Eolv\nbfAMef7tT/GyfWZgqWIwrdZhJ7yWy0zg2F4gKDGAKHi9FVLxRp8+LOVZCVQcBKzLwBwKyLhqpNH2\nOnKOumcTGKBsRs32BayOOZIwDhCkyPP7r08HgaQ1MNJE9WWa5RzMGE44OHF9ZzfsvzicmjqBREcL\nwGOkfUpd0k4WAkkTtAJeiE6mYaQB/nB2wEpgigCblj8VSMUUkiqBJR08rBFonWKMFrrXEa6NpJEm\nYIlaHlYjmWU6g2h/htTpvZBCXSA4VaIKl5aqSRrxqTWuP/r0Nge74k6QOEn6qlgfpbIYxcIQJl7h\n2LzuARYdzzmNkRoMBsOSOFFGmu+yn2NrHmasLQh7K6k1AuemSISIiAqRmYgvTU0Uj6P6fJ3ybFSh\n+vnZ02vNOt3bnK7okWLqm9lIpfTJp/jkVM3TrYxnwwuPse9lscbbFr8uEdEcqZCy7URV1O5AxFzM\nkR4HttMifY7HF0V4GLGWwUdYBfZVkdSv5HMVdlMcYhAoZCEFRdR0fjDjay3+xkV5OPWWgR/gq0gq\noyufs1gL6YK/myuf2HDE+8hQmHYOKc1CSWKkgIa4RttqPAv4aJu1JYlDMeLo5dpOPoTYhzyx3wnX\nWqxBScNNIyleoqSDGI8FLEW5vkShn1YHaaeSWlpNgv8xl3qkeHZEJukmwa8txp9c81JZpV3UJj2H\nZ/7CJvpLqTTtCm7bvQI92ZTaMIc/PV9Iny90aVB9muqXcXobIzUYDIYlYS9Sg8FgWBInatqfP83y\noY3LV5rvNiM2aXMv2T5MobV8aSG1EJ2YF8FRPWixQ/k8GphFqBm6uX2mWWeC1gkTtITwCC5oSUMt\nlanw/9QHU3BrzoGK7vXn+dgmaGOxH8zxEo56j6ypWh1/C+bQaWRKRDfR6E7ZtOs7q9e+16OtiFPX\nSkxaMbOl6VyuagnkTatk5G8rmYr0fJHvRGajax/ElUijUFsS9nemqvVIsv0M+5L2zEREMdZP0Y6m\n0+btjVXuvgS9xA1xKLghd4mTltF0P15Jf/GHDBLYW6gmgD1IiaRkgsM9qzN9SlRkKgu+/rvDMNan\nNyFVQ9aRxOC0Am6GjD+P6xrVh68v756/y5HJqC/v1hoHNLfW2b0nyqxcVYk7izGXyk5T3TobqYse\nrp15G1Xa2qqWxhHBTg1jpAaDwbAkTpSRDiBHyBQTTCVvGqwxQuVtrxz3Ur+zxqyUqEBUihz1aIG8\nbeTjexVIksItEfKGcwkEqGlNtiiMakOxxacXaCn8UZZvnZrzMW7OlZBcKhrJsSmWluJ8E+xFqoa3\nlMN8QMcLfh9GFLjmqar2k8B6SFOxIiRYEa6HsD2xAqJDKhP8HoOWIliVHeqcDdbb/Ab1FdSxiTg/\nlsCQYiBSW1M+3RGUUhIqmrqkqrGdHHcKqVsc3V8hSuqqrhJyJK4URbjYSZevn7SjlqpPlbq/5VTl\nWi+UhXEaLFGkZjJWtQpQjpumeSJLhEBe17tAQGg253suVs/31hlmpNLrbgZ52t5eaCjZlzbOkDZN\n8mChHCAQGWO/+7CSN1WFqFqN/1EwRmowGAxL4kQZ6U34Ac/nQR5RgaXV0i9Hqp6XyicB0b1MQl5J\nLxqfKiRVixrpZnMlKp4dbt/rqiPYXyTsgre9qaQTb2uDVZXsE2zBwRNV4Tx8JRWJxJmk20nHOFfM\ntGBwkZrVE5UmuSoQ+ZJmlClYXksSE7BOXWr2ztdfuiM4xf6bilxOMjNQhzILOxFRt/wqwgFUivVK\nGmpQut2bBExU3MM2ddV316zLay/UtjPcK1JRP3LS7yucR5KGNMlVQUvaDytrUGRnna7IyNCXSfUY\nKzHWIvlbH4REE7m2g5iTJtaQTpqkoZ7pFNd4skBb6xEnUSym4XpKXdgFqkelFPbfQ7+0CL+foYV2\nrurDZlKErLl3FOsWNR1ulv07bIGmSv6UpuF9chSMkRoMBsOSOFFGOkFvlUrX+UMd0bo6zGB0SmCc\n8GwQS98W5X+qa2F72AdmQJm5iIgSRAUz7DcBE4l01F6itI16IPhXWjP2u3alZmklvaeU36w5Aimy\nEc5b/HW1dFREulqke1et4JzW7j9GRET5ONQjpZpZu3TmjMEABp3AUhZ9Pn/p0KnF+jMUq4kivlcy\nMLtU1ZcNrS35o9djX5ZXUokSlkldQnyfBEYhRSpmGPMx7pVS1zMVP2xTYEYV6YC/Tfyn8yZVNBxi\n2lmnVcM6UpcPVDcCSZsU9i6+a93dQVJl90dSzCe8Vg6Q2tnHmAvrjeJwrdv4Tp7lrMuxjLmqOTqF\ngl5YY7sTxlPuMUn9LWHhONXpNEVCQI33SqESNKqm0yz/v8R9OR2rxIKOUoQcgdV7eg0Gg+Ehg71I\nDQaDYUmcqGm/QFOrhZI1zKUNMph2G2ZbrEXe0ohMJBM+5FRXMPMLEfpKkGEaRMGJgxBems85+V7l\nRseyTWkLrSIoEhzBvOOdVC8KrgHfBEykVoCWwsh6kHfgHA9JZKLjndkPI9oDTnooinCtK4jaczj8\n2xFMMeWs3+hz8sQcEpS7B6HKjzRZG0McLrrvzfXQWrfdhosH5vYBgogqHhRa6iJwUqhg3nTM60tb\n6ClayDglqenBfO/DpOsoU1ISCqY4ftlXplr8dqPjm6U9jNjY5Fz16W3VxgPPk4xVK5b6rOHVkUuQ\n5oBlR6lXwWAZoy6WNe08wn6lGWQmgUa0aq7bwb2XQB6ZwQ3Q0ZWdcK0XcO14CRyrGrIOtWurueTV\nh5tFagRI9TDxHGkRfqnfB0fAGKnBYDAsiZOVP+Vot0qBXexBwJ7gnb4N9thWs0kMJuelyZZSL0l8\noQJtldkkUjW4EzDCWGoqOlRxam+E7fS5hqFIWGpVL7EEA4nAOv3icFtnvd+6CSypY5Tv3OGAmiPF\nWsBOv5JWB+0ep/yWi9AadwEHfY7A3gQylY5iMAnoSL/HAaipmvkrqWCOwMHoAEGOWFWoxxgluMgz\nCTKoMWsyGIVdLMJYiahbqv1IsKGrmr/1u3xs6whk6WpOEpwSRiv69ViTULd6HGUDjQaHypq7u89B\n1xJ1SWNJRlDXKsW9O4Wwfm9v1CzrQAo1RiUnKXU66IfgjcjnErDXCvdKpKrwZy25Z1r37V+qPFUY\nAAk+ORVgLBF4Egu2UvdKIQkBcouCoS6UaL+qTZBvMBgMrylOlJF+csQzxKXybPNd37Nfph8xk3md\nY6nRBSW673r4OMVXqVMKxccqbZAlXVFVwPborVREnK52t+Jtj91Ws06R8DGVNWYu5aMtREAPv2dR\n3MWCIJFyIhDGwRWqqMMUBTeK+DSvW4t/R7eDZpbzHlodtLpcICaqA6OcYqzG+3w++6h2Pl8EKYkI\noKW1b1MYg0Jvn9kM8jWMtRScISKaSq8kMI8OfKZ1rRmpmAiwBpRcR2pZOvCINpjPoB+Oow0/nUho\nqkNFSEQahYIqsbRlDmvEK1iPdB3pnI/VoXX1dCHPHv9fKv9rqZg8FwOw+NksMFqxEuYQ8E9nvL2W\nYpRdWJ8zjH0uYQplxYj1UsAa0MsqsfRkPFCj2KnEgqoJjICtKovBw+JcwIqSdNBFEcYwbh/v8zZG\najAYDEvCXqQGg8GwJE7UtH92iAyl4pw6ADRJ82zSD2Gu5VFw9PbRhiNF9lDbhcNuQzYUo/tdjfYR\npcqcKFrsRC+gsnnugJe9OOk36wwdmzVlCTNRtXyWDJYYZt50iA3NVfO7lI83y9DoTuX53p2zg7yA\nBChyHNjSzdba7SABWhUkKbtlor5u2cLjUNRsCo33LxER0TC/G9apODjlReoWBROs1+cxkTqmB2jV\nu1C1QiuYXGJmFvPPX6dA1tHqFWl/Ig0L0xS1S1WFn+k+aq3CBNSZahXcBHHTahrbUU3j1jeC+2pV\n0IZJfGozBIPnCDLt7XP+uVRg87peBf7swR3iSdc8YFO8QPuOKVq/1JOQebjVRcYg3CHraxwEnpbh\nOZ8foEGhBCrXw7PrWhgjyM/aCEjpMZdgl7QzkepkREQpnmupqyptRXJVTyDR0eMjYIzUYDAYlsSJ\nMtLZAsLbODj1Zwj83Mkxw0h1lpaqWUrMDpIMDMAFx68Ip9c2OYDTGfApzbxyeEMgPIOE4bMTnpU+\nm4cGeXeIGWmNwJArAstpamRiNhznXDE/ysOsNgBr7TlpXxtms32Q0zmE6xlOf30j5J+vPxFYwKrA\nwTKI25vNdynE0RnypqOc2WddBwZCCEh5sBWvKjNlYHUppCv5ApWBVO0EYYSSWVHQ/dIUT58/2COB\nDpHbxIgSjaZKxgV20rBmFReU4Mj6Gt+z26eYQXXWQn59sh4CNqsCqaKVKWuui2t0E1KxCfLwDzF8\nMDoJum1thXtZBPQS25E2zHUenu9+67ClmeL6eqXaX0gyDO4Lp+SRHkx4iveCX6BzguqkIfeKsM9M\ntddOUVlMJF7SZ7FSPLPITf5kMBgMrylOlJGeiS8TEVFf1fmTdNGrU55VdhzP8p/TR5ZBLiRpYUmY\nTbqQRWyd4t+laCVbucBgpK1qBf/pdTCY/VzVTUz491Lf1Kl8Q/FlFpjNcrDn2AWfmPTxERlUpmbD\nVoS0yWqHiIiefOYZIiJ61zvf0qzz9q9YPZ+aSJWiJKTrJRBFt+Bnak2YvScU+ud4XI8Ks/xcsf/K\nQ2wPNpK04FdWgnrp3dXk+uLjcMdc+c/93RCETonfU8a3Un6/GokBGSyktV44xw38/cRZZp1PnOdU\n2awX/Hb71eql/ErlM11pq9fBeILFD1HNqZirdspOxlFSZQMzHwwOi+RFDlWr30/h44wxDrUXS0MB\n90ECiVXaCuNRwPrZ2+V7TKrnnz0dnm/pUiH+z1Qx2jRFGiySgUQ5GSufd7Wwnk0Gg8HwmuJEGelX\nr90hIqIn1sJMcYA0rN+f8TRwEHNEf+3ixWady7scMZxOES1VEbc1OByLMWoajpCiqNI30zXMLMgn\nHcIn5lWUNqnYD9v0FlJ9iEL6J+93rQufjhLtnwcDfabPM2VP+XdevMMpcy/mLOB/zzc8QURE3/zv\nvblZ5+K5kK66Kmi6AlTB5+2EkYIxtDp8XlUVfIaVzw/9vlyooie4H2I67KPUCodYHJZNR0vZeVin\n6ePk7veVRkgplHWk1KgWiWcQbp9BBPsdX/HGZlm/y+e7PmC/ehcJBaNJ8LGOD4JKYVWQF1JgJjxf\n/TUeR/F7jqc8ZncnKglDOk/MEH1XDuUeKuu3pReb1ORVFsJ0Jp1npfcW2KOyXLsbzPbbGKO+qg86\nggJgZ5efM7GUOoOwThtp4nKOkSoYlDR1S+GH9ZJgoSL7iQnyDQaD4TWFvUgNBoNhSZyoaf96vLef\nqANl3yWm5bf7bFpfgxThm7/9W5p1PvfCHhERffwT3NJiZy+YgvmU1585aUkB80Ll2rsh7yODmXF2\nmyn/dqFycUs4uCGVOqQ39lJ9ij/Pdtlc9TfC5TsPcf5bOnyOPSUyzwZsHt0hPo+nLvD+L5wLJvHW\nYPXqVy5yHrNUmUAxAoExasdmCZu/i3Yw7WsJ9sTscun3VPIDXCuLKW97NrlORESlqicqRqFUZBIp\nTqLyr0VGlUlQQeVWy3euxccawVRvtcIxZpBrybhIGw4iogmqI43GkMFh223lGmjFqxdsmqEKUqEq\nI62t8717ehuVoVBzdLgfgodiLouL5dbOsFnWQSDp9DZatkgNAyVPy+HOmUqOfHR/zdN1uFiiNb6f\nWqp2rAS5xBNQIGA81/UV4C4ooG2q1O99fTgwKe25S+36S49/VRojNRgMhiXh/KGqNgaDwWD4QmGM\n1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNh\nSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgN\nBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaE\nvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL1GAwGJaEvUgNBoNhSdiL9GXgnPPOudc/6OMwvDqw8Xz4\n4Jx7v3PuF45Z/knn3LtP8JC+YDwSL1Ln3CXn3Dc96OMwvDqw8TRoeO+/3Hv/Ww/6OI7DI/EiPQ7O\nueRBH4Ph1YONp+FhxMq/SJ1zP09EF4nol51zY+fcj8J8+17n3GUi+qBz7t3Ouav3/K5hPc652Dn3\n4865551zI+fch51zF47Y17ucc1cedjNjlWHj+WjDOfdjzrlrGJdnnXPfiEWZc+7n8P0nnXPvUL/R\nY/t+59wvOef+Kdb9iHPubQ/kZBRW/kXqvf8eIrpMRN/pve8T0Qew6OuJ6M1E9C2vYDM/QkR/kYje\nQ0RrRPRXiGiqV3DOfSsR/SL9/+19SawsWZKVXR9ievHmP+TPWUlVZhVNlZAQEgvEAlYIiaGh1RvY\nNQsQUvcSpFqUhMSwqRVCQoIVUqsFLIAFSEgskBq6qQakmrsyK7sz88//zfFi8vDhsrBj95rHi3yv\nqqLy1Y8vO4sX8cI9PNz9uvs9ZnbMjOhvvuxmxibDxvPVhXPuAyL6B0T0Z73328Rj+QkW/1Ui+h0i\n2iOi/0xE/+KaTf01Ivr3RHRARL9NRP/ROZd/Qbv9U2HjH6TX4Jve+4n3fvZTrPsbRPQN7/2PPeM7\n3vsTtfzXiOhfEdFf9t5/+wvZW8NNsPHcfNRE1CWiP+mcy733n3jvP8ay3/Xe/xfvfU1E/5aIrmOZ\n/9d7/x+89yURfYuIekT0577QPb8Br/KD9OHPsO5bRPTxNct/i4j+nff+++vtkmEN2HhuOLz3PyE+\n998kohfOud9xzr2Oxc/UqlMi6l3jDw/Xgve+IaJHRPT656x7K3hVHqT+hs8mRDSQf5xzKRHdVcsf\nEtGfuGb7v0ZEf90595vr7KThp4aN5ysK7/1ve+//PBG9Qzym//zn2EzwdzvnEiJ6k4ie/GL28OfD\nq/IgfU5E712z/EPiGe6vwJfyDWITQ/CviegfO+e+7Bhfd84dquVPiOgvEdFvOuf+3i965w1XYOP5\nCsI594Fz7i8657pENCeiGRE1P8em/oxz7lfBWH+LiAoi+v1f4K7+zHhVHqT/lIi+4Zw7J6K/tbzQ\ne39BRH+f+AZ7TMxodNT3W8RBjf9GRCMi+jdE1F/axmfEN98/dM79xhdwDIYIG89XE10i+mdEdExs\nyt8jon/0c2znPxHRrxPRGRH9HSL6VfhLf2lw3q+yogwGg+Hlg3Pum0T0Je/93/5l74vGq8JIDQaD\n4ZcGe5AaDAbDmjDT3mAwGNaEMVKDwWBYE7daAOJv/Prf9URESRKf31VVERHRxcU5ERFNxpdERLQo\nYgJL09RERNTvc+C10+3EjYJQn+P7sq5WVXCyBFFV82dpwofd7fTCOsPtHf6NHn9WzOdh2dHRCyIi\nmk4nRERU1zV+S/+Gb/2+XibvWTsc19WQzy5On7srC19SfHZ+7ImIvD4P8ubaKZoPUS6+jo8rd1zK\nX8c14pxrva76TC/7vHX0Oddjo5fVSqpaYqwqvNbqeqowxnXN6/vGYbtxmwl26b27dzdmPD86K3g8\n1eXpll6T8KoPa+kQ1b9Nw/f3bM73jtwDq8ZTrqM05Ssjy2PWZ55rdVt7PMN9tbQDXl2EVzVW6vt4\n75r2sdXqQGrH63z1oLtyPI2RGgwGw5q4VUZ6enpMRESJmo1qsMQZ2F6xYCbYVFEWFlkFpgw99WOZ\nbFNml8ZfZYROSSkiCAAAIABJREFUZrEwK8ZZKUt5Tul2eBas1e8Lo43wS69x3yL79FfWv8Kc9KxK\nm4deyuzRpfoywjjgUGU8VrHwBB/laj7PkvSn+OU221xmr6015bNrYgHBmlDXTLjm5Hh83HZCOG45\nNvz+Kia3SSj95++1jFWKI9MML54r/K/OQyNjCytQ7qXGa/6I34U1wslK6rtL75d/RMaokdtbvqOu\nh8bHtXk/1LHifXw+uCvfr264QY2RGgwGw5qwB6nBYDCsiVs17bPkquM/BpXYWOhkTO8b9YivEZAS\n6t3UytReMtcjv192PRNl2Haasvku5jxvk3+jXCxav0mkTcirQQW1I+3/fgpZ2Saa8xo5TFoxdfV7\n79rmsqfPd3XowIULgUgxwa7+bgh8wASUV23ZX/maWpimbfeBBCGv7qH+Nb2PvDTGTGGKNps9os01\npn2EU3/lk/b3WuPg22Z3VUkwVrvLeJ00aZ9rp4YpW3LbaNO8WQolNUumPn8mX2y94L1vH4eXYFNE\nfcOpMUZqMBgMa+JWGWkB9qlnMA95RBJpBhbE79USHMKHyQrmIWxVAkirZog848PtdFg+lWVRXiHB\nrcWCv1lVi7AsxiuWA0mrWFb7O7z+ip15BSBBAc0oMy80AoxyKegUl6xGIBphYFd8aemEhlVWBO9W\njVTgPU6CCvy/5jUSwAiBUX91QOMYyxslA2tuoDAvNT5/38N51Bf40nloXftybsFAFwUHk+s6Wnwh\nUJxJ0A73uwo8Nt32c6EdTBZGubSPK47Dr2SkS7gyrqtWasMYqcFgMKyJW2WkIrbPsqs/K/5HkBzK\nlYMkTViMu7O9TUREg0GsiCY+tS5E+mXJTFKEwERxhss7vJ08MNE448znBa+LmVOzm1R8qcuSnhXT\nVGSkinUvMahXJS3Xr5IWudZLZH2tL8r5a6+jP7uClpQFDGTJP+4VSxH/Vh12UfEUfJbiGquciO6V\nV2xpHNMWO1mWO63a681jpMl1h3HFVbyC44skqWUyStIC34/FnFtnNcri6+B5UFdgpmmJrarroh/q\nePP3VZykRDwjDTI4kadFntj2vEcrZBWi7159doP/2BipwWAwrIlbZaQiuk8VI5X3Mpv0uswah8Nh\nWGdvb5eIiN588w0iIrpz505Y1usyO724uCAiotElv85mk7BODZYpM1VR8Gw4mcR10kt+H9IH1aza\nq/k3ZkgbXU4xJCJqhKVKqNB9Putclba4iQj+x5YrCWxRzp9YFjqNNKRYXhVnS1qgLJNzlKt0QVF0\neDBIJymaiiUtM1JNEIWRlr6dTtzoxAvZVAjNK87hJaUR/644ETcxmJcRfinOwB8umRjLvmtSYy7n\nZaU/GyoOsEfNKB3UNC6wV1wXtarVLLEUjIceqwTvfS2+1fZ+8fr4TJap8Vm+C+W60uN5kyDDGKnB\nYDCsCXuQGgwGw5q4VdNeqrs0ZaTsCYj1ziGb61/5yq8QEdH7738lrPPG69xpdR8mvhZUL+Zspo9G\nIyIiKkoOGlVK8CtGpQSmyoVUpIkVnooZb6cKMqi4bDYdExHRs+fcMfYcbgSpBkVEVJa8/ghVqKSa\nFRHRWNwGYvXL+dhwSX4INq0wm8U1EsZcm1nyinUXKvlhWvD4iQmX5xl+4rpE9rbUSv9eHQKDEalb\nCj0E0bgyzeUwmtaavE0MZBNM+6vJA9VNydkvJVbY7cFsX1q00nUh2qQVmwwBKTHN4yolRPpyzaT4\nsVTXwsBAJMFlpNw4Ytov/ZYegWZ5WStXf2lXQ06PDhjTtTBGajAYDGvidhkpHMy1krIkkBZtQ9r0\n9rvvEhHR+x98NaxzsL9PRFGGNLkch2XTCTPR2YyZTI7qTXk3yiWE7wiT3d7J8b86fMw+FdhyBWZL\nRLRA4OruvQdERDS65N+cTKdhnbLkZIPnT7mZ5aef/HFYNpuC3dbttMn2vL95DEaIhyfN/iEdweFI\nMkWzQiIl46EvwgLnX5alGM/W2VlOF1xRYapu2nVEWxIrCSA1wlaFJUVLJxCWFcxLjiUyUxnPuG61\n0YFELRVbSgn1S//f8H15K8kbkp69WMT7azbj+8N3eVkvBKBVoocT0T3qxDbRqq2lYhzOeZqJDCvW\nLQ61Sa8R5IfPliwN9dHnwhipwWAwrIlbZaR1EzmMIAUL6PSQttnhXZqV0Uf56OkTIiIqMYudHB2F\nZc+fPuX1J8wOd/b2iIio24vV7+fwwXUhyL9z5y6/3r0X1ulCdlWLODiL7KQrMySqdO8f8vcqJfon\nz+93t1m2NZ/FCv9PHvM+BjnHijy1ZAPlMjIPazYtLM27trSpxbijWp+wclwkiRlORO9gfa2OB8vf\n5xddNKRZqpruFLtp6jZLDhXuVWWKWC39Kipcx0XF12gx42tvpsZ8BuvrS4e/smILLyeSJNDH8FkY\nqiX//lUvqmZwK67lpcSM+TwK8qdI1Nka8D1by2/p4jK4HkpYLOfnF2GRxEdEOre7x/f3cOcgrNPt\nsYW6Mh14eXevZqmvTlbQy69fbDAYDIabYA9Sg8FgWBO3atqvkqlUMIGeQlr0e9/+n0RE9Icf/zCs\nI9WadocckBIznojo5PiEt4NspeNTNqOl7iER0WjEpsMeglYffMDSqu2+ypbJOtgfNtHnRXSGj5EB\ndXzKkqZSmnSpbJsBXBOzAsEqJX+JKeG+/UELm2faNxI80+1BgqIIxpOYUiukSaFmkjodYm43S2ai\n12a7bLuRTBpZR5nv8iZksqjxwH5XWAteGfLKtG+WspZ0JaIxgowff/R9IiL60Q/+HxERnZ6chHWm\nqGv7F/7Jt2hT4JbteL1sqQ5pqwbpcjW0FZEcySrMIGfr9WO9DAkibw+3iIiogwBjSFEiogmCyRKY\nmi9UKyKMTYLgsdSXLcvoeut0rz57ruz+8rH9DPFCY6QGg8GwJm6Vkfb6HKzRjn9pnFaCCY7G7Dhu\nkugO7vWkahPPPItFdFRLRfvwCmaqHf9np2dEFMXhc5EtqfxvEeJL3cRiHr9/fMLtmH/44w95Wcnr\nbO/shnUkaUBkWDIrEilGGqoU0VVsYLCpAPtulIwsBa3JQjUvEcZHyPug7VZsswaTrxMJUgVpfFgn\nlotsM9J2YoCIsoVuqt8H05Eg5HwM+YxipCXGT5jQvIjX3NFzDn7+79/970RE9L/+x38lIqIRrjMi\nojmukX+5QYxUmGirieAyW1t57fKHhdxzehykQSIkZyksv/5WrKUh0schgk0iJysU6xQGOoaUUIvl\ne1v8fbFchf2WKtFD3ieZdHC4eoih1sAqQf6Kw9YwRmowGAxr4lYZ6cH+DhG1ZQ0e/o0mCHaREqj8\nIzJ9pIkIdVXbXJHgwD8mlZ4kDZQozmwLvDb1VXltrftAUWSoRETjMUstHj/6lIiIpmBid+7cD+sc\n7rPsSto4e8V2Y93Mtm9Q+4ASt3lz2gzns06VnE1SO8VvJamiup7oUs8mPR5lBdbuePyy8qpvy4W6\nk9LqFwu0wiqkbcqrEsvj9fERj+uz5+z7VgQmjPEL+OBfPH8elr14+DEREX30nd8jIqKf/PgjIiJa\njKPvvtnoCvkabS7mgyBf+axxz71AnENbnFvbfM/3IQuUa15qAxMR9eATlapTJe6hQt3Di5KvixHO\nsa5p3B3wtiUJJ3Ztj9fVHOngPallrOVwQQYnX2y94LjpWmze3WswGAwvGW6Vkb71xmv8RhcdgX9s\ntuBZLYfovZNEQX0Q7eOzVtdAmbTq5eIKVytphMhrqGkYV6kDK4JYW1fYr4XJlq3/nY/rdHPeZp2J\nj1D/vDAw+R8MO41R/zyLM/SmQHyMeraWqvM1TnWI9uq04MAo+f+qjhsoSkkB5OshQbV07ZxzKdJQ\n4UcNIn61b8JKaogvtF/9YsEL/+B7bGF8+uKU11EXxPk5Kz2ePGUmegRrhIjo8vFP+Pifcxpwg312\nyseabqDPW5QuriXIR9olzk3l+ZHRKgiCavfnx3yuYidYdW0gWj+XFGB9exb8u8+efEZERFNR5eQx\nxXPrgJNgCunyq3ZAYhZ+woMtbLUlJsHvJiWsWt2l44qLHeOpfffGSA0Gg+GLhT1IDQaDYU3cbrBp\nlwW3PQhviYiyHr93MG2DEFoHa/C+D1FtOVYtQjy/P9gWMS5/nijzv4Kfe3uLP+vm7HhO3GXcDmxx\nCXwkPgryE5iZYoijRxflKgE3RwAsdWK6RLM/xXoZXBrivuipClW9Xru51yZgDsGzdpHIOamW6glo\nQ1fqgUpgqmw1MvP6a1QsRNivBPWJVF2SOpTiYojrSILG6JgDSc+fxfoMR3Pe5rd/9AkREZ1IWxMV\n8DuFuX/8hKVv0+MobSpP+bqpUGfWS964tv82sPrTqjY5YtpLGw+RjjVqRKWxXTUd4YO4bIFrfNFH\nHn0tY6XaKaNh5egJy8qmkCf6bhTtz+E2mWFZbxDHaoHA4PiSl4kMqt+P7sE+EgCkulvSqkCXyoG0\njkdf2O4GzmmM1GAwGNbErTLSB/cPiYjo3oMH4bPdO+xE7vaZNkpl+tkkssUZqsPMZ1yHtJmr+pcH\nPLNs9VG1CRPd7jw6qvf3eNvDLeaUW13MeGWUtIjsKhTWr6KURSIWiCdRjddUycx9xWLkusS261i9\nShzrwsBSeMFjW2iiXjfOnpuCGdJoF6rjQQcSNUkFFBPDKYYmRD4BqylVOu8EMpVcggHSWlfJVRaQ\nx5ycMss8PmLZzdHR07DOCQIfZ5AtPXkUl5UDvg6PEpbmzGGN5CoIWpwyI23Ojvn3R7HaUI7gRpay\nZVHimBcuBrSS5Jazr38B8PVVRhoCu4GRilxRBaQQdMwWuAcWkW3WOZ+3GmJ7CWQJMyUiKse4r0fM\n8BNUeZsVcZ3jaYH94bFKFAecw5o9Q+eKgaSfNjthHZFYSaCYmjjWQaInFk6o0hZWoTSJ9+oqGCM1\nGAyGNXGr0+Y773Aa5Xtf/nL47LXX3yIiohz+kNmUfVpnp8/COi+e8cx0esLMVMtMtsDkuh1IYnBI\njVcV8huesVKHggk9ngHL6cO4c2CHJRyg5VzJn8BOu5mkeMIfStGPWsx5NlzMMANXkZEGmY7UQpA+\nRqqvVOI2Ty4zmvJ4dJSMa4GZuwGbOD1mH6OuzyqMtJzzzF+Wceof7nJiw9YW+84HBZ/HTLHFD3/E\nBW2++90/ICKihxDInxxH1imWTa/H19XlZfz9qssi+4sK6cnobkAqCaOeMrt0FdqEN5Fz1B4JHmBi\nKZi1pD8SEbkbGMzLiEJ6mK3wkcpH4iNNlIWRYmxrFAeaX0Rr0uE8Spp3gjrBXhUUKS+YiW6lPFa7\nu8wkLxRr/Qz+U7GCpqpLRnHIvy9p2QOYjqmSJ6aoqJ+AiSaqwn6K+1GMngo9oJpWruv1Pm9jpAaD\nwbAmbpWRvvflLxER0ZtvvRk+29nlggOLBc9Kl3P2e00uHoV1RudSGo9nNzXxUx9zQQb/1KDH/3c7\n8dCcA9uEcLqY82w2u4x+rylmJamToLL9qIFPbDhARB4R4YTizDsdgXmhs2hdqag/XiWqvEA/KJ0G\n2x/EIg6bgu997zv8RjFrIRFFw8d4dsQ+Sl1AQhj5JUTvZRG/f+/BG0RE9Np9Tr998Dr70w/39sM6\n40uOoP/Rx39IREQffvhdIiKqFpF1bqEQhgjBR6pHEDXo6gqFwOICEXnFSCVi2+ki/bAfx0fSkcsR\n73cH0eG7918P6+T9qEzZFEi03Lmr6gPpeVQjXuBVERc/4vO3uGRGOz05jdsEO61xjvbuc1JOqkT7\njXTuxX3mIMTvqaj9A3S1ODnn39pVyp/X7nAlfBHZD4dsjQ63olXa7/EzoML+18r/mhOzZbGCKim0\noxN2bihzaYzUYDAY1oQ9SA0Gg2FN3KppfwYJSX4UHfFlCTN7zKbx2TGb9OOzKE3yNa8jbVY7Suye\nZe1c4E7Or3mq8vHBymsENUoEIi7Oo7B/Poe5jSBB3cRTk+e8gf09dlT3C5HmqIPzMHUQiEpUom8K\nUyWBeSsVqnTzvNks7sum4P/8PnczKIq471JfNoFJncA+krbbREQFBNgzCKgTJUURt8f0koOOvuF1\nT549jutA+H3vDpv7VcnBy8vLuB8zkdDMRICtpE1oVteTZJAhtjPX7hgkUeRsXjYqeFSAf1SQTQ1Q\nW/NNFUQ9eC2a+ZuCEoEcLZaPzQchW4L5XU9jMLU6Q0t0yBIvzqPLq4CLa47avVuHbIbnA5WAAn1g\nKfJCyOm05G1vyOdYZEx7h7Gx3Wuvsdkvze+6aOvcyeM9LFX4ZwgiOxePUVo9S72MHGNdqqpkUgvk\n82CM1GAwGNbErTLSczBSYRlERNWMndDzCcudphfMTMvZKKyTobWvkIo0VYELpGYmmDCyRATg8XeD\nUkPqDCLIoOtPLkpUbcKslCi5TQ8BrL2EGYzUSayqOA+lWH8ugvQ0RsS6HT7GUoS+dDW1cqrkQZuC\nz37CHQPuv34nfLa9z0EeqVuZgMF0VMKBx3mc4zOvZ3sEIQqwzk9+8mMiInr2JDLSYsrXUSdHYkOH\n2U1/ZzusI4GPBZh+qVjzfMLnugv2vL3L7OZsHiV3HbDNJMP1qUTmXvpxLSlitvqRZR3s7NOmIaSB\n6uAh3tdyE4Etei0Vw6m5RKbK0UT1OxuzZTFDXdd3QTI7g3g9FAUiu1t8z1zCUinnccxyyKYkUH14\nL15zQ6l1mkrlNaltrLoqhM4NPEaJCnaF2qYqLZ2IqFaJIvMb7k9jpAaDwbAmbpWRyoyn0z8vGqSH\nFeirVE6wY6pLICqwL5qrdUAdGGhG8ioCYlUTkdCnRVgrhL/DrSiv6PVRaV9S4NR+N6COWQfsCgxq\noTqFNg3PppmkG2Zx/3s9FGoAs05RY7NQnUqlO+Im4cmnnxAR0bvvvBE++9qf+joRET36jJMdPvkR\nM8rt3Ziud+8NljT10D/nxeMnYdnFiJloBXPh8JDTOc9PX4R1JiNmOeILS7rsf5vWUaI0WSBleMEM\nRBItiGJfqMsziMVh4RTK75d3UDMV11GmBPk9sBlhaRVkQyePomSv5zavvqycc50bEgrtSOEdIW/K\nYqsrjGPN4/HReWRvI3RWfWcf9UQLscbivbOAH/U5xvgEBWLKMjLCIToIv/uld4mIaGsY791Mxgq1\ngHMkiOiuE1KMqAkyR90Bg59BE4zjxQX/P9eWEizMP/3+27QKxkgNBoNhTdiD1GAwGNbErZr20iI5\n7amWDAM2gepG8q6Z+k8n0Xy/HLMZ8NnjU2wnph31+7ytd95i6r8zRFuSTLUJgGkubN4l/JsDlR2R\n5PxeKs7oVrAVTPATVAI6O+NgR6XMxYN9zsRJ0Q5l0I/LdlArtdeHiwLBqkJlh8z7KvNmQzA5RSDh\nIppyhwdstj/5lM3cZzB3i1kMvty5z4GCXHKiVV3XumTzukCOvfNstt9Rcpc+7EtpB1OgFc1UmWLn\n07aLJ/HxUq+xfoG6tkXCx1EqCdoC14GHVM4rU91hHzuZ9Khgc/HZ48/COn7RDlxsAiRI0+3FQGne\nkXPN1+ocWWCjUcx1f3aGRoHIUDupo9l/CUnU4AlnLE4hjar3YmCwwHl/+JDbubx4wfeZV+6Ug0O+\nZiqkzmV5/I0OXDwNxkEaWc6U62yKAOMlKslpt5q8l/txjPbcpCRvva3Yen0VjJEaDAbDmrhVRhpm\nedWK1YMxSAWg0YRnt6MXkR08eswM9JNPmZFOJpGRIj021E18+02e6fa21W+AkUp3V6lpmCqJUoZK\n3lIxpikjyxJx9/EJz8IvXvCsXKsqVAlqW3Y7/Pt5HqUwgwHPnhlaDed51Xrl76k87w3BcIuPtSqV\ncHk8wyufK2Ebs2mc3ceocdDtolOAksLcucvBJWmdnUOALRIyIqJ+jopfYCITBDnGT+I1c3TJzEPE\n5YmqJNTUaKCGmrFJwd/vaZaT4Fqt+XjmC9VqGTUfpOLYYJvZymQSg1XPj6OUalMwQKvigRoPqVlR\nzPkemuLcjcaxTsWzUz43I2kw2ImBoC7ug+oIASQEkqq7keE1CHKNEWgcneH+UqReiu7P5qj0pKRK\nGQJXYkXOEDScqIpfl8j5FwmmboYY2zZLS3d+PnTyeM1mneuDh8ZIDQaDYU3cKiMdj5nZdZR0QqrO\njy/gZznhdR4+iRKpDz/idNGTE/jP1GzSnUCWsQX/zhZqj+ZxBpHaiZNC0t1Q4VxVuK8zfl9AGjNT\nFYnORzyznaJuovwvVYCIiIZbvGxvhylykuhZXdpBo8Uw6qKmSfyNzga2Y37tTZaCHN6J/ssTpPhe\nnPKYZbm0Y47fG4MtzuGbqivFLmAZDHf5/En/nUKxvQFSOw9QEegC18DDF7FlMiENtQGVcTr7QmQ+\n8MtnEJ0Pt6PPfAv1MyV1d7aIPkGp7NVA0N9FGumiF3lJ0tm8jgeHe2xVZcpnTSGNmc/R3i4zyZmy\nQl6MeZ1Bwuf8wTBaY120Ud55wRXc+pCuuXlk+ANs870vvc+/j2ptRy9in60xKkzNp3zvjS6i9XE2\nAiOGz3sC/+1irit+IYUc9Wn7qpqXSLEKxGdcwtecJHoQEeXd63uqGSM1GAyGNXGrjPTRE47GPXsa\nZ7weiozUFbOUC1TZfvo8pog+PeXviS9tuB0ZrQjyjy54Vnp2giIVqmhJgpTMMZQAjRSk6MR5JO/x\n+nP4cUWUS0R0hI6SUpQhQUFUYZZERBX8pVLzNM3U70MYDP05OamormsyppsX5b3/OisVeoM4Ho8+\n5bTRizMWV0uCglSqJyJKEzl/qJquxqELX5T4Tx+gv9f4PPrkarDNwZCr6S8QUe6o8+lEoiFFJZVP\nrYEvzSG66+A/LVRShHShlRq4pJJACKm+xQT+QvhsD96MiQkP3n6XNg2f/PEnRES0vxPZ2j5YehcW\nU2cPFp9iaB6MPEG8Y6EUFr0Rj9/emIu4PHiXUzyHdw7jOrDiPkj5PObwsab5x2GdoxO2cOao4n96\nGru6LmAZLmDZJEjA0Z15pWeY5Ok4p3s28WsqjyVZpq6nm3rCGiM1GAyGNWEPUoPBYFgTt2raX47Z\nJBpdqBYfkFGUixFe2VyaTKN0YQ6TehdmRX8QZUtSnUakDo8e83YuzqOcSNqrimneQJztVNM2qaMp\nDc20WL6YQS7TiNkOQb8yHYbb7KjvowJQmsbgUQPTQ1wKIsfS1Z+a5ibj4eXDwQGSGObRDXOBaj81\nqgNloS9MvNSylL+3jTqgrWpDGCtpDzOBOPrsNP7GPvL2xUXiERBJXdxOjvqwNcZTx08kWCRBldRD\nLnOuzMUx6iIgSiZuACKiHKXFnLgt0NLi4LVo2m8f3qVNww++z00F330rHkf3HW5OubvNxyhtmHv9\n6Kp5/QEHlDqowVvej4GgDEL4AdqU56gOVqqA8xzusAztdob7LL7fuxvHvBLXCtxkktRCRFQGex05\n9/KqKmaIDK4S818/+XBMFcZYKpdJk0N+b61GDAaD4QvFrTLSouDZ4PwiOvWPj5iRzmeoiC6zgmIp\nDarV12ifW5eRSYooXsoFPn/Os+FJOlffl9QxfABncpKqCtoQY0sAKM1UtR+wTJHi9BEQGSgJxe42\ns6suUk11IEriHEKKAiNVLLTeQEZ6fsmVfYoiWg8dCLj7qDU6RvqoLvXYQXCm1+Pz1ygZmrDLCoz2\n7Azpm5W+HqSiOY/Z/i6Pz/5OtAI6KdL+0IHBK0bRVGjH3fC+OVTodz5aIUJuc8jouqqieyBTeLOF\nOqgu0yxr8xIsfvgDrtTlVNe3A1S2F3H6FIkqTRbP5xbkS2+/zYGl8iBeDwt0kawQIBxN0OTyLLLN\nKWRHHimZUCnSAK25iYju4XbsDfj+yrtxrDO518BECzwMJDBFFIOGHuwzqeP9GcT+CDY6x/e50wHj\nxfXjaYzUYDAY1sStMtIqCK/jzO2xCxUKHTQyGyoG4YhnqtEFxPKTODs08H3M57z+HFQiSdRskoj8\nCL1cutLuNYqmRYw9HIr/Tvk/4QOT73XQgylXfaFz+EQdjke1vQn1L4V11lKJXLEsXXtxUyD+TGlj\nS0SUg+0fnRxhHWlrHM+VBwOdF8wMVdnI0ApYBPydDqwHF78/m0v9SNSZBdfvdeN1ce+Ax7oP1jgb\nR3biSskVrrBvvD/dTrR0tuCv64ZEiXg9elRezyC67+L6yBVL6yh2uimQ6/JCSc0ePuQEiwmKfZyi\nFXZvO/pI3//qB/x9nNezs9iO+fiIrZYLxDwKjEOunJQ7KAjSwb3X2eVxGCo/rL/LPufhDrPUjkp4\nCP2kcH8l6PUk7Zn1ZwtI3SbT6MedgrmK1TPosw++41QMxV0vTzRGajAYDGvCHqQGg8GwJm7VtG9A\nq3X9yQFymp1HJSGYi02jpQt4ddLiQTedgxmwjawIBIm0mTbo87JdyGb29tiU2N2L7S+2kc3Rx/70\nlCnayeK2eN+uBovElK9gHpUqF7lE3n6JepmxyZj6fr15waYOHP6pOj/Hz9ikn6ACz0BM5E681BYw\n6RuStru6SZmcW2lrjeCfsv+liaAUAZPgT55FSczhAZuFe0NUL5rFQNJ4xPtbVbwBn0LqpFpo58iy\nkbxvp0xRh2Cj5G13EQBJvAqalfH3NgVjmO9PVeGqCsGl/QM2qbd3+T4dqntnMedjPT3hDMTJOJrN\noUqTNIUU6aCqLbHT5/OXwdVWo15w7aNpnzjJRuRljXK1yH1YlVLTWOSOqoIb3i8qWRZdPdIeXILQ\nHVRl66q6yXknPg9WwRipwWAwrIlbZaQiifHDyL4ysIA58uhF9lLr3GipKRly1ONMkWUyizBz2EZN\nxb3dWIH7DmbTPbzuoH3rYEtVqUFtS2nhmpAOdsGZDf2UBM2q+mqwaBGqDOn6l/gE6+gtbzJEZnIC\nGRQR0RyJEZ0Bn//uNtozK9YpY+wwjokaT0/C1tFWG5/rwJ4IraVTwi5ytfeUXGaM/ZhBEJ7msRJQ\ngu4Ji5LaorOMAAAEpElEQVS3WYNJZipBIwUTlcpUOtSQSN1SBDOcBDVUfdvKb56FcX7OQSItA8uQ\ngC51Lh6gnsD2VmSkY1RiukDShD5yyXfPujhnaCDZ09XnEVSSIB6FfHgdheSXhdxnql5wqHAPlrko\n+X8tf5oiuCRV83UFOUmUEesppNyrkmW6LsYqGCM1GAyGNXGrjFRkMh3lbxhgNpKK1cLyGuVvEiep\nTFiZEst38gzb4RnzcJ9ZyX1VXeb+XU45Gw55xpEeLy0m5NvtWmvFNoWVVCEFDfxEt3sVV5AwqZbf\nT3r7hA/anxNR4jaPnZ48Z59YUUTZ0RDV4kWkncKn5VvnCicrlEBfUWVHTjHOi9dzPnxZc4iktyBb\n2VKi+b1drkBUNZDyqOSLroxNCV8cvu8UlUrheBWS6TXDFKslRerv0riq3d8onJ6xf/tgL57H1994\njYiIvvb1rxER0Z17XPGrUtK9J2inLfGKvpItdXoiU0LSA5RyXlmcpRdrjllmiXuvVOuINGsBRnlx\nfh73+5QtoinE/ims1CxXfm3c65fo3HA5idXdpMrTW/B1i1WqubXu8bQKxkgNBoNhTdwqIxUmmqte\nKFJ3Umb8yETjbCA+0Ry+qZ6KAG9t8Yx355BTNIV9HuzGnjAD+GaFga5inaIWkJRT7SMVJ2ezRCzJ\n67RFWbUdddbHFoipMFLFiF26eQLuMWq29rejP1reZxhj8S9rx5mkyIrSQftPQ5Q+lXPEn6d5FGDL\nuaqR/ikdIncUI91HVHkOV/VsEb+fV7xeMmcGU4s11Orjg8QOYZ9qrILgG+0dpHylU37x9AYB98uI\nGVJ9O/0YUd/FfVVAcfP0OdcF1V0NJOkigaUoleaJiOYL6drJ51Z8k0UVz3XwaYL1SSfhdiEf3rb0\nd5pNY4X9GfyfwbdZYqyiizT4QUOvJpXw04H/3eE3hG2XVbS0Mmc+UoPBYPhCYQ9Sg8FgWBO3atqL\nSZsoUy6DdESc+2K+56o1rojsJZd5exid2QcHbMLfQXuD3aHkSCsRfd2W1Ejwymt5A+YUydvW+nip\nHCTxhjqI7pVrACaHiOxb6pcQjBBdB9ZRqzQ3NjN4+ZAhgLO1HaUwGeoQSOBGWjworXtovSJmc6oC\nQaEiV+gQgnOsTCuRGzUL/kzywFXvORoOkXyxw2a8m+m2MP3Wb9UZm4mFcq8s0II7uH+0mYnxlKaK\nrpHWzUqEX26eq6aBy6JQbqkxTPIcDeWcBI2U60ukQVM0NWzJAmEez9GIbr64Kj8qFmLas2tB3AZa\n/hTkb6GuaHQfyH2dyfiFAKVyz2GsJCEg1+1tkISTJu3HoQ4wuhuCwcZIDQaDYU04v4HCYYPBYHiZ\nYIzUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9SA0G\ng2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9\nSA0Gg2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY1oQ9SA0Gg2FN2IPUYDAY\n1sT/BwlVjSMjcJ2UAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 360x360 with 9 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pDnf_yQxleHx",
        "colab_type": "text"
      },
      "source": [
        "# reproduce the architecture"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9NXBdCgzJyIB",
        "colab_type": "text"
      },
      "source": [
        "Custom resnet variant: ![ResNet18 variant graph](https://296830-909578-raikfcquaxqncofqfm.stackpathdns.com/wp-content/uploads/2019/06/Artboard-1-5.svg)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YS-_x9MUI-N6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def conv(ni, nf, ks=3, stride=1, padding=None, bias=False):\n",
        "    if padding is None: padding = ks//2\n",
        "    return nn.Conv2d(ni, nf, kernel_size=ks, stride=stride, padding=padding, bias=bias)\n",
        "\n",
        "def noop(x): return x\n",
        "  \n",
        "class ResBlock(Module):\n",
        "    def __init__(self, c_in, c_out, stride=1):\n",
        "        self.bn1   = nn.BatchNorm2d(c_in)\n",
        "        self.relu1 = nn.ReLU(True)\n",
        "        self.branch = nn.Sequential(\n",
        "            conv(c_in, c_out, ks=3, stride=stride, padding=1),\n",
        "            nn.BatchNorm2d(c_out),\n",
        "            nn.ReLU(True),\n",
        "            conv(c_out, c_out, ks=3, stride=1, padding=1)\n",
        "        )\n",
        "        projection = (stride != 1) or (c_in != c_out)\n",
        "        self.idconv = noop if not projection else conv(c_in, c_out, ks=1, stride=stride, padding=0)\n",
        "\n",
        "    def forward(self, x):\n",
        "      _out = self.relu1(self.bn1(x))\n",
        "      return self.branch(_out) + self.idconv(_out)\n",
        "\n",
        "class Pool(Module):\n",
        "  def __init__(self, concat=True):\n",
        "    self.concat = concat\n",
        "    self.maxpool = nn.MaxPool2d(4)\n",
        "    if concat:\n",
        "      self.avgpool = nn.AvgPool2d(4)\n",
        "    \n",
        "  def forward(self, x):\n",
        "    if self.concat:\n",
        "      return torch.cat([self.maxpool(x), self.avgpool(x)], 1)\n",
        "    return self.maxpool(x)\n",
        "\n",
        "def make_DAWN_Net(c=64, Block=ResBlock, prep_bn_relu=False, concat_pool=True):\n",
        "  if isinstance(c, int):\n",
        "    c = [c, 2*c, 4*c, 4*c]\n",
        "  layers = [conv(3, c[0], ks=3, stride=1, padding=1)]\n",
        "  if prep_bn_relu:\n",
        "    layers.append(nn.BatchNorm2d(c[0]))\n",
        "    layers.append(nn.ReLU(True))\n",
        "  layers.append(nn.Sequential(\n",
        "    Block(c[0], c[0], 1),\n",
        "    Block(c[0], c[0], 1)\n",
        "  ))\n",
        "  layers.append(nn.Sequential(\n",
        "    Block(c[0], c[1], 2),\n",
        "    Block(c[1], c[1], 1)\n",
        "  ))\n",
        "  layers.append(nn.Sequential(\n",
        "    Block(c[1], c[2], 2),\n",
        "    Block(c[2], c[2], 1)\n",
        "  ))\n",
        "  layers.append(nn.Sequential(\n",
        "    Block(c[2], c[3], 2),\n",
        "    Block(c[3], c[3], 1)\n",
        "  ))\n",
        "  layers.append(nn.Sequential(\n",
        "    Pool(concat=concat_pool),\n",
        "    Flatten(),\n",
        "    nn.Linear(2*c[3] if concat_pool else c[3], 10, bias=True),\n",
        "  ))\n",
        "  return nn.Sequential(*layers)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bDlv591gV7lG",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "f460621a-899c-4585-ff7d-7a68c52e1baf"
      },
      "source": [
        "model = make_DAWN_Net(); model"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Sequential(\n",
              "  (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "  (1): Sequential(\n",
              "    (0): ResBlock(\n",
              "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "    )\n",
              "    (1): ResBlock(\n",
              "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "    )\n",
              "  )\n",
              "  (2): Sequential(\n",
              "    (0): ResBlock(\n",
              "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "      (idconv): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "    )\n",
              "    (1): ResBlock(\n",
              "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "    )\n",
              "  )\n",
              "  (3): Sequential(\n",
              "    (0): ResBlock(\n",
              "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "      (idconv): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "    )\n",
              "    (1): ResBlock(\n",
              "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "    )\n",
              "  )\n",
              "  (4): Sequential(\n",
              "    (0): ResBlock(\n",
              "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "      (idconv): Conv2d(256, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "    )\n",
              "    (1): ResBlock(\n",
              "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu1): ReLU(inplace)\n",
              "      (branch): Sequential(\n",
              "        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "        (2): ReLU(inplace)\n",
              "        (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      )\n",
              "    )\n",
              "  )\n",
              "  (5): Sequential(\n",
              "    (0): Pool(\n",
              "      (maxpool): MaxPool2d(kernel_size=4, stride=4, padding=0, dilation=1, ceil_mode=False)\n",
              "      (avgpool): AvgPool2d(kernel_size=4, stride=4, padding=0)\n",
              "    )\n",
              "    (1): Flatten()\n",
              "    (2): Linear(in_features=512, out_features=10, bias=True)\n",
              "  )\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YDh6LqLIl4EC",
        "colab_type": "text"
      },
      "source": [
        "TODO: find a way to test if the model is actually idendical at the PyTorch level"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b4zXSIhTl9Ro",
        "colab_type": "text"
      },
      "source": [
        "# reproduce optimizer"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9D-0FWxc7VM2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "opt = partial(optim.SGD, lr=1e-3, momentum=0.9, weight_decay=5e-4*batch_size, nesterov=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ixxilqyRmGIn",
        "colab_type": "text"
      },
      "source": [
        "TODO: make sure the optimizer is actually identical"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tl2MbkD1mJ5d",
        "colab_type": "text"
      },
      "source": [
        "# create learner"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9dm1VX4lmMsQ",
        "colab_type": "text"
      },
      "source": [
        "TODO: find a way to use `nn.CrossEntropyLoss` as loss function like in Part 1"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hFPuzq329mqP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "learn = Learner(data, model, metrics=accuracy, loss_func=CrossEntropyFlat(), opt_func=opt)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u9JPIWkAICjc",
        "colab_type": "text"
      },
      "source": [
        "# learning rate schedule\n",
        "\n",
        "lr_schedule from the bag of tricks article:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-2EDNTj6Ejyv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class PiecewiseLinear(namedtuple('PiecewiseLinear', ('knots', 'vals'))):\n",
        "    def __call__(self, t):\n",
        "        return np.interp([t], self.knots, self.vals)[0]\n",
        "\n",
        "sched = PiecewiseLinear([0, 15, 30, 35], [0, 0.1, 0.005, 0])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WhuxJLSNFIZc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jzrNsWmBFZgU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 286
        },
        "outputId": "a8a9d59d-babd-4796-834a-132d4e20ecf2"
      },
      "source": [
        "plt.plot([sched(o) for o in range(35)])"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7fb0087eb898>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XtcVXW+//HXhzuCgAiiIgoqiqCA\nSdrF7GpZjpF5SX8zc5qZzqNmppuppd3M7KppVjN1znSmqU7T5D3TtLSyycrJROUiXvEKiIqKqKhc\nv78/9nYOkcpWNqx9+Twfjx5u1l5783aFb/Ze67PXEmMMSimlvIOP1QGUUkq1HC19pZTyIlr6Sinl\nRbT0lVLKi2jpK6WUF9HSV0opL6Klr5RSXkRLXymlvIiWvlJKeRE/qwM0FBUVZeLj462OoZRSbmX9\n+vWHjTHRja3ncqUfHx9PVlaW1TGUUsqtiMheR9bT3TtKKeVFtPSVUsqLaOkrpZQX0dJXSikvoqWv\nlFJexKHSF5EhIrJNRApEZPI57h8kIhtEpEZERja4724R2WH/725nBVdKKXXxGi19EfEF3gRuBZKB\nsSKS3GC1fcBvgH80eGwk8AwwAOgPPCMibZoeWyml1KVw5JV+f6DAGLPLGFMFzAEy669gjNljjMkF\n6ho89hbgC2PMUWNMGfAFMMQJuZVy2Mr8A+w+XGF1DKVcgiOlHwsU1vu6yL7MEQ49VkTuFZEsEckq\nLS118KmVatzq7aXc+8F6fv3OWo6fqbY6jlKWc4kDucaYt40xGcaYjOjoRj9FrJRDyk9XM2lhLrER\nwZSUn2HK4k1WR1LKco6UfjEQV+/rTvZljmjKY5Vqkuc+3cyhE5W8+cvLeOiGRBZn72fxRv3xU97N\nkdJfBySKSIKIBABjgCUOPv8K4GYRaWM/gHuzfZlSzeqLzQdZsL6IP17XjfS4CO6/vhsZXdrw1OJN\nFB49ZXU8pSzTaOkbY2qAB7CV9RZgnjEmX0SmicjtACJyuYgUAaOAv4hIvv2xR4HnsP3iWAdMsy9T\nqtkcraji8UV59OoQxoM3JALg5+vD7LvSEeDhORupqW04c6CUdxBjjNUZfiIjI8PoWTZVU9z/jw2s\nzD/AkgcG0qtD2E/u+yS7mIfnZPPwjYk8MriHRQmVcj4RWW+MyWhsPZc4kKuUsyzN2c+y3BLG3dTj\nZ4UPkJkey519Y/nTqh1k7dE3ncr7aOkrj3HoxBme/mQTaXER3Deo63nXezYzhU5tWvHwnGwd41Re\nR0tfeQRjDI8vzON0VS2zRqXh53v+H+3WQf68PiadA8fP8LSOcSovo6WvPMKC9UV8tfUQjw1Jonu7\n0EbX79u5DeNuTOST7P18vLGoBRIq5Rq09JXbKz52mmlLNzMgIZLfXhXv8OP+eH13+sdH8vTifPYd\n0TFO5R209JVbq6szTFqQS60xzByVho+POPxYXx9h9ph0RGDcXB3jVN5BS1+5tQ/X7uW7gsM8NTSZ\nuMhWF/342IhgXhzehw37jvHGVzuaIaFSrkVLX7mtvUcqeHH5Vgb1iGZs/7jGH3Aew9I6MuKyTvz5\n6wJ+3K1jnMqzaekrt1RbZ5g4Pwd/X2HGiFREHN+tcy7PZqYQF9mKR+ZmU35axziV59LSV27pb9/t\nZt2eMqbenkL78KAmP19ooB+v3WUb43xq8SZc7ZPqSjmLlr5yOzsOnuCVldu4OTmG4X0dvbRD4/p2\nbsMjNyWyNGc/izbo2TiVZ9LSV26luraOCfNzCA3044XhfZq8W6ehP1zXnf4JkUz5ZBN7j+jVtpTn\n0dJXbuW//rmT3KJynr+jN9GtA53+/L4+wuy70vH1ER6ek021jnEqD6Olr9zGpuJy3vhqB5npHbmt\nT4dm+z6xEcG8eGcfsgt1jFN5Hi195RYqa2qZMC+HyJAAnr09pdm/3y9SOzKyXyfe1DFO5WG09JVb\neO3LHWw7eILpI1KJaBXQIt9z6u06xqk8j5a+cnnr95bxl292MubyOK5Patdi3zc00I/Xx/Tl4PEz\nPPlxno5xKo+gpa9c2umqWibOz6FDeDBPDu3V4t8/PS6CRwb34NPcEhbqGKfyAFr6yqXNWLGV3Ycr\neGVkKq2D/C3J8PtruzEgIZJnPtnEnsM6xqncm5a+cllrdh7m3e/38Jur4rmqe5RlOX4yxjlXxziV\ne9PSVy7pZGUNj87PJSEqhElDkqyOQ8eIYF66M5WcwmO8/qWOcSr3paWvXNILyzZTUn6amaNSCQ7w\ntToOAENTOzCqXyfe/GcBP+w6YnUcpS6Jlr5yOV9vO8RHPxZy76Bu9OsSaXWcn5h6ewpdzo5xntIx\nTuV+tPSVSzl2qopJC3LpERPKI4MTrY7zMyH2Mc7SE5U8oWOcyg1p6SuXMnVJPkcrqnh1dDqBfq6x\nW6ehNPsY57K8Euav14uqK/eipa9cxmd5JSzO3s+DNyTSOzbc6jgXdHaMc+qSfHbrGKdyI1r6yiUc\nPlnJk4s30Sc2nD9e383qOI06O8bp7+vDuDkbdYxTuQ0tfWU5YwxPLMrj5JkaZo1Ow9/XPX4sO0YE\n8/KdfcgpKmf2F9utjqOUQ9zjX5fyaIuzi1m5+SATbu5Bj5jWVse5KLf26cBdGXH81zc7+ddOHeNU\nrk9LX1mqpPw0Uz7Jp1+XNvznNV2tjnNJpgxLJr5tCOPnZXPsVJXVcZS6IIdKX0SGiMg2ESkQkcnn\nuD9QROba718rIvH25f4i8r6I5InIFhF53LnxlTszxjBpYR41tYZZo9Lw9XHupQ9bim2MM13HOJVb\naLT0RcQXeBO4FUgGxopIcoPV7gHKjDHdgdnAdPvyUUCgMaYP0A+47+wvBKU++rGQ1dtLefy2JOKj\nQqyO0ySpnSKYcHNPlucdYH6WjnEq1+XIK/3+QIExZpcxpgqYA2Q2WCcTeN9+ewFwo9iuWG2AEBHx\nA4KBKuC4U5Irt7bvyCmeX7aZq7u35VcDulgdxynuG9SVK7u2ZepSHeNUrsuR0o8FCut9XWRfds51\njDE1QDnQFtsvgAqgBNgHzDTG6LXnvFxdneHRBTn4iDBjZBo+brpbpyEfH+HVu2zTRw/P2UhVjY5x\nKtfT3Ady+wO1QEcgAZggIj87Wici94pIlohklZaWNnMkZbV31+xh7e6jTPlFMrERwVbHcaoO4cFM\nH9GH3KJyZn+pY5zK9ThS+sVAXL2vO9mXnXMd+66ccOAI8P+Az40x1caYQ8D3QEbDb2CMedsYk2GM\nyYiOjr74v4VyGztLTzLj863cmNSOURmdrI7TLIb07sCYy+P47292smbnYavjKPUTjpT+OiBRRBJE\nJAAYAyxpsM4S4G777ZHAKmMbYdgH3AAgIiHAFcBWZwRX7qemto4J83IIDvDlpTv7YDvs45mmDEsm\noW0I4+fm6BincimNlr59H/0DwApgCzDPGJMvItNE5Hb7au8AbUWkABgPnB3rfBMIFZF8bL883jXG\n5Dr7L6Hcw19W7yK78BjTMnvTLizI6jjNqlWA7WycRyoqmbxQxziV6/BzZCVjzHJgeYNlU+rdPoNt\nPLPh406ea7nyPltKjvPal9sZ2qcDw1I7WB2nRfTpFM6Em3vy8mdbmbuukDH9O1sdSSn9RK5qflU1\ndYyfl0N4sD/P3dHbo3frNHTvNV25qltbnl26mZ2lJ62Oo5SWvmp+f1q1gy0lx3npzlQiQwKsjtOi\nfHzEdm0Afx/GzcnWMU5lOS191axyCo/x1j93MuKyTgxOjrE6jiXahwcxfUQqecXlzPpim9VxlJfT\n0lfN5kx1LRPm59CudSBThjU8c4d3uSWlPWP7d+bt1btYU6BjnMo6Wvqq2cxauY2CQyeZPiKV8GB/\nq+NY7ulf9CIhKoRH5mVTVqFjnMoaWvqqWfy4+yh//W43v7qiM4N66AfuwDbG+caYvhytqGLyolwd\n41SW0NJXTldRWcPE+TnEtWnF47f2sjqOS+kdG86jt/RkRf5B5qwrbPwBSjmZlr5yupc/20ph2Slm\njkojJNChj4J4lf8c2JWB3aOYpmOcygJa+sqpvt1Rygc/7OWeqxPonxBpdRyX5OMjzBqdRpC/no1T\ntTwtfeU0x89U89iCXLpFhzDxlp5Wx3FpMWG2Mc5NxceZtVLHOFXL0dJXTjNt6WYOnahk1uh0gvx9\nrY7j8m5Oac//G9CZv6zexfc6xqlaiJa+coovNh9kwfoi/nhdN9LjIqyO4zaeHppMt2jbRdV1jFO1\nBC191WRHK6p4fFEevTqE8eANiVbHcSvBAb68bh/jnLRQxzhV89PSV0329CebKD9dxauj0wjw0x+p\ni9U7NpzHbkli5eaD/OPHfVbHUR5O/4WqJlmas59luSWMu6kHvTqEWR3Hbd0zMIFrEqN47tPNFBw6\nYXUc5cG09NUlO3T8DE9/son0uAjuG/SzSx+ri+DjI8wclUawvy8PfZRNZU2t1ZGUh9LSV5fEGMPj\ni/I4XVXLrNFp+Pnqj1JTxYQFMWNkGptLjjNzhY5xquah/1LVJVmwvoivth7isSFJdIsOtTqOxxic\nHMMvB3Tmf77dzbc7Sq2OozyQlr66aMXHTjNt6WYGJETy26virY7jcZ4amkz3dqFMmJfDUR3jVE6m\npa8uSl2dYdKCXGqNYeaoNHx8vOfShy3FNsaZzrFTtk846xinciYtfXVRPly7l+8KDvPU0GTiIltZ\nHcdjpXQM57EhPflyy0E+XKtjnMp5tPSVw/YcruDF5VsZ1COasf3jrI7j8X53tW2M8/llOsapnEdL\nXzmkts4wcX4O/r7CjBGpiOhunebm4yPMGpVGqwA/HtQxTuUkWvrKIe98t4usvWU8m5lC+/Agq+N4\njXb2s3FuKTnOK5/rGKdqOi191ajtB08wc8V2bkmJ4Y70WKvjeJ3ByTH86orO/PW73azermOcqmm0\n9NUFVdfWMWFeDqFBfrwwvI/u1rHIk7fZxzjn53DkZKXVcZQb09JXF/TW1zvJKy7nhTt6ExUaaHUc\nrxUc4MsbY/pSfqpaz8apmkRLX53XpuJy/rRqB5npHbm1Twer43i95I5h9jHOQ/xdxzjVJdLSV+dU\nWVPL+HnZRIYE8OztKVbHUXa/uzqBQT2ief7Tzew4qGOc6uJp6atzmv3FDrYfPMn0EalEtAqwOo6y\ns52NM5XQQD8emqNjnOriaemrn1m/9yhvr97JmMvjuD6pndVxVAPtWgcxY6RtjHOGjnGqi+RQ6YvI\nEBHZJiIFIjL5HPcHishc+/1rRSS+3n2pIvIvEckXkTwR0SFvF3aqqoYJ83LoEB7Mk0N7WR1HnceN\nvWL4jyu78I6OcaqL1Gjpi4gv8CZwK5AMjBWR5Aar3QOUGWO6A7OB6fbH+gF/B35vjEkBrgOqnZZe\nOd2Mz7ex58gpXhmVSusgf6vjqAt44rZe9IjRMU51cRx5pd8fKDDG7DLGVAFzgMwG62QC79tvLwBu\nFNtA981ArjEmB8AYc8QYozshXdSagsO8t2YPv7kqnqu6RVkdRzUiyN92UfXy03o2TuU4R0o/Fiis\n93WRfdk51zHG1ADlQFugB2BEZIWIbBCRx871DUTkXhHJEpGs0lJ9q2qFE2eqeXRBLglRIUwakmR1\nHOWgXh3CmDwkia+2HuLvP+y1Oo5yA819INcPGAj80v7ncBG5seFKxpi3jTEZxpiM6OjoZo6kzuWF\nZVsoKT9tu05rgK/VcdRF+O3V8VzbI5rnl21hu45xqkY4UvrFQP3z6HayLzvnOvb9+OHAEWzvClYb\nYw4bY04By4HLmhpaOdfXWw8xZ10h913bjX5d2lgdR10kEdtF1UMD/Xjoo42cqdY9qOr8HCn9dUCi\niCSISAAwBljSYJ0lwN322yOBVca2g3EF0EdEWtl/GVwLbHZOdOUMx05VMWlhLj1jWjPupkSr46hL\nFN06kFdGpbL1wAkd41QX1Gjp2/fRP4CtwLcA84wx+SIyTURut6/2DtBWRAqA8cBk+2PLgFex/eLI\nBjYYY5Y5/6+hLtUzS/I5WlHFrNFpBPrpbh13dkNSDHdf2YW/fb+bf247ZHUc5aLE1Y74Z2RkmKys\nLKtjeIXP8kr4w4cbeOSmHjysr/I9wpnqWjL//D1HKqr4fNw1epI8LyIi640xGY2tp5/I9VKHT1by\n5OJN9IkN54/Xd7M6jnKSIH9fXh+bzvEz1Tw6P0fHONXPaOl7IWMMTyzK42RlDbNGp+Hvqz8GniSp\nfRhP3JrE19tK+d9/6Rin+in91+6FFmcXs3LzQSbe3IMeMa2tjqOawd1XxXN9z2heWL6FbQd0jFP9\nHy19L1NSfpopn+ST0aUN9wzsanUc1UxEhBkj0wgL0jFO9VNa+l7EGMOkhXnU1BpmjkrD10cvfejJ\nbGOcaWw7eIKXP9tqdRzlIrT0vchHPxayenspT9yWRHxUiNVxVAu4vmc7fnNVPO+t2cPXOsap0NL3\nGoVHT/HCss0M7B7FLwd0sTqOakGTb00iqX1rHp2fQ+kJPRunt9PS9wJ1dYaJ83PwEWH6yFR8dLeO\nVzl7Ns7jZ2p4bIGOcXo7LX0v8N6aPazdfZSnhyUTGxFsdRxlgZ7tW/Pkbb34elsp76/ZY3UcZSEt\nfQ+3s/Qk0z/fyo1J7RjVr5PVcZSF/uPKLtyQ1I4XP9vK1gPHrY6jLKKl78FqauuYMC+H4ABfXrqz\nD7br2ihvZRvjTCUsyJ+HP8rWMU4vpaXvwf6yehfZhcd4LrM37cL00sQKokIDmTkqVcc4vZiWvofa\nUnKc177cztDUDgxL62h1HOVCruvZjt9ebRvjXLX1oNVxVAvT0vdAVTV1jJ+XQ3hwAM9l9rY6jnJB\nk4acHePM5dCJM1bHUS1IS98D/WnVDraUHOelO/sQGRJgdRzlgoL8fXljbF9OVtbw6Pxc6up0jNNb\naOl7mOzCY7z1z52MuKwTg5NjrI6jXFiPmNY8ObQX32wv5T0d4/QaWvoe5Ex1LRPmZdOudSBThiVb\nHUe5gV9f0YUbk9rx8mdb2VKiY5zeQEvfg8xcsY2dpRVMH5FKeLC/1XGUGxD7p7TDgv31bJxeQkvf\nQ6zddYR3vt/Nr67ozKAe0VbHUW4kKjSQWaPT2HHoJC8u32J1HNXMtPQ9QEVlDRMX5BDXphWP39rL\n6jjKDV3bI5rfXZ3A//5rL19t0TFOT6al7wFeXL6ForLTzByVRkign9VxlJt6bEhP2xjnAh3j9GRa\n+m7um+2lfLh2H/dcnUD/hEir4yg3FuTvy5/G9rW9c9QxTo+lpe/Gyk9XM2lBLt3bhTLxlp5Wx1Ee\nIDGmNU8N7cXq7aW8q2OcHklL341NW7qZ0pOVzBqVRpC/r9VxlIf41RVduKlXO6Z/tpXN+3WM09No\n6buplfkHWLihiD9e1420uAir4ygPIiK2sd9W/jw8R8c4PY2Wvhs6WlHFEx/n0atDGA/ekGh1HOWB\n2oYG8qp9jPOFZTrG6Um09N2MMYanFudRfrqaV0enEeCn/wtV87gmMZr/HJjABz/s5cvNOsbpKbQx\n3MzS3BKW5x1g3E096NUhzOo4ysM9OqQnyR3CeGxhLoeO6xinJ9DSdyOHjp/h6cWbSI+L4L5BXa2O\no7xAoJ8vb4xN51RVDRPm5+gYpwfQ0ncTxhgmL8rjTHUts0an4eer/+tUy+jerjVPDU3m2x2H+dv3\nu62Oo5rIoeYQkSEisk1ECkRk8jnuDxSRufb714pIfIP7O4vISRGZ6JzY3md+VhGrth5i0pAkukWH\nWh1HeZlfDujM4OQYZny+jfz95VbHUU3QaOmLiC/wJnArkAyMFZGG5+29BygzxnQHZgPTG9z/KvBZ\n0+N6p6KyU0z7dDMDEiL5zVXxVsdRXujsGGdEK9vZOE9X6Rinu3LklX5/oMAYs8sYUwXMATIbrJMJ\nvG+/vQC4UUQEQETuAHYD+c6J7F3q6gyPLcjFGMPMUWn4+IjVkZSXigwJ4NXR6ewsreD5ZZutjqMu\nkSOlHwsU1vu6yL7snOsYY2qAcqCtiIQCk4Bnmx7VO/197V7W7DzCU79IJi6yldVxlJcbmBjFvYO6\n8uHafazMP2B1HHUJmvto4FRgtjHm5IVWEpF7RSRLRLJKS0ubOZL72H24gpeWb+XaHtGMuTzO6jhK\nATDx5p6kdAxj0sJcDuoYp9txpPSLgfqN08m+7JzriIgfEA4cAQYAM0RkDzAOeEJEHmj4DYwxbxtj\nMowxGdHRegEQgNo6w8T5Ofj72val2veWKWW5AD8fXh/Tl9PVtUyYp2Oc7saR0l8HJIpIgogEAGOA\nJQ3WWQLcbb89ElhlbK4xxsQbY+KB14AXjTF/dlJ2j/bXb3exfm8Zz2am0D48yOo4Sv1E93ahTPlF\nCt8VHOad73SM0500Wvr2ffQPACuALcA8Y0y+iEwTkdvtq72DbR9+ATAe+NlYp3Lc9oMnmLVyO7ek\nxHBHesPDJ0q5hrH947glJYYZK7ayqVjHON2FGONab80yMjJMVlaW1TEsU11bx/C3vmf/sTOsfGQQ\nUaGBVkdS6rzKKqoY8vpqQgP9+PTBawgO0FN8W0VE1htjMhpbTz/W6WLe+nonm4qP8+Lw3lr4yuW1\nsY9x7jpcwXM6xukWtPRdyKbicv60agd3pHdkSO8OVsdRyiFXd4/i3mu68o+1+1ihY5wuT0vfRVTW\n1DJ+XjZtQwN49vbeVsdR6qJMuLknvWPDmKxjnC5PS99FzP5iB9sPnuRl+xWLlHInZ8c4z1TXMX5e\nto5xujAtfRewfu9R3l69k7H947i+Zzur4yh1SbpFhzJlWDLfFxzhr9/tsjqOOg8tfYudrqpl4vxc\nOkYE8+TQhuexU8q9jLncNsb5yoptOsbporT0LTb9863sPlzBKyPTCA30szqOUk0iIrx8ZyptQwJ5\naM5GTlXVWB1JNaClb6E1Ow/z3po9/PbqeK7s1tbqOEo5hW2MM43dhyt47lO9qLqr0dK3yIkz1Tw6\nP5euUSE8dkuS1XGUcqqrukdx36BufPTjPj7fpGOcrkRL3yIvLNtCSflpZo5O008xKo80fnAP+sSG\nM3lRLgfKdYzTVWjpW+DrrYeYs66Q31/bjcs6t7E6jlLNwjbGmU5ldR0T5usYp6vQ0m9hx05VMWlh\nLkntW/PwTYlWx1GqWXWNDmXq7bYxzv/5Vsc4XYGWfgt7Zkk+RyuqmDkqjUA/3a2jPN/ojDhu7d2e\nmSt1jNMVaOm3oOV5JXySvZ+Hbkykd2y41XGUahEiwkt39iEqNJCHPtIxTqtp6beQ0hOVPLV4E6md\nwvnDdd2sjqNUi4poZTsb5+4jFTz3qZ6N00pa+i3AGMOTH+dxsrKGWaPS8PfVza68z5Xd2vL7a7vx\n0Y+FfJZXYnUcr6Xt0wI+3ljMys0HefTmniTGtLY6jlKWeeSmHqR2CmfyojxKyk9bHccraek3s5Ly\n0zyzJJ+MLm343cAEq+MoZamzZ+Osrq1j/NwcanWMs8Vp6TcjYwyPLcilptYwc1Qavj5idSSlLJcQ\nFcLUYSn8a9cR3l6tY5wtTUu/Gf3jx318u+MwT9yWRHxUiNVxlHIZozI6cVuf9sxauY3comNWx/Eq\nWvrNZN+RU7ywbAsDu0fxywFdrI6jlEsREV4ankp060AenpNNRaWOcbYULf1mUFdnmLggB18Rpo9M\nxUd36yj1M+Gt/Jl9Vzp7jlQwbamOcbYULf1m8O6aPfy4+yhThiUTGxFsdRylXNYVXdvyh2u7MTdL\nxzhbipa+kxUcOsmMz7dyU692jOzXyeo4Srm8Rwb3IM0+xrn/mI5xNjctfSeqqa1jwvwcggN8efHO\nPojobh2lGuPvW2+Mc162jnE2My19J/rL6l3kFB7juczetGsdZHUcpdxGfFQIU29P4YddR/nL6p1W\nx/FoWvpOsnn/cV77cjtDUzswLK2j1XGUcjuj+nViaGoHXl25nZxCHeNsLlr6TlBVY3tbGh4cwHOZ\nva2Oo5RbEhFevKMP7VoHMm6ujnE2Fy19J3jjqx1sPXCCl+7sQ2RIgNVxlHJb4a38edU+xvns0nyr\n43gkLf0m2rivjLf+WcDIfp0YnBxjdRyl3N4VXdvyx+u6MS+riOU6xul0WvpNcKa6lgnzc2gfFsSU\nYclWx1HKY4y7qQdpcRFMXpirY5xO5lDpi8gQEdkmIgUiMvkc9weKyFz7/WtFJN6+fLCIrBeRPPuf\nNzg3vrVeWbGNXaUVzBiZRliQv9VxlPIY/r4+vH5XOrV1hkfm6hinMzVa+iLiC7wJ3AokA2NFpOHL\n2nuAMmNMd2A2MN2+/DAwzBjTB7gb+MBZwa22dtcR/vb9bn59RRcGJkZZHUcpj3N2jHPt7qP89zc6\nxuksjrzS7w8UGGN2GWOqgDlAZoN1MoH37bcXADeKiBhjNhpj9tuX5wPBIhLojOBWqqisYeKCHDpH\ntmLyrUlWx1HKY420j3HO/kLHOJ3FkdKPBQrrfV1kX3bOdYwxNUA50LbBOiOADcaYyobfQETuFZEs\nEckqLS11NLtlXly+haKy07wyMo2QQD+r4yjlsc6OccaEBfHwnI06xukELXIgV0RSsO3yue9c9xtj\n3jbGZBhjMqKjo1si0iX7ZnspH67dx38OTKB/QqTVcZTyeGfPxrnv6CmmLtExzqZypPSLgbh6X3ey\nLzvnOiLiB4QDR+xfdwI+Bv7DGOPWO+bKT1czaUEu3duFMuHmnlbHUcpr9E+I5P7ruzN/fRHLcnWM\nsykcKf11QKKIJIhIADAGWNJgnSXYDtQCjARWGWOMiEQAy4DJxpjvnRXaKs8uzaf0ZCWzRqUR5O9r\ndRylvMpDNyaSHhfB44tyKdYxzkvWaOnb99E/AKwAtgDzjDH5IjJNRG63r/YO0FZECoDxwNmxzgeA\n7sAUEcm2/9fO6X+LFrAi/wCLNhRz/3XdSIuLsDqOUl7HdjZOHeNsKjHGtTZcRkaGycrKsjrGTxw5\nWcktr62mXesgFt9/NQF++pk2payycH0RE+bn8OgtPbn/+u5Wx3EZIrLeGJPR2HraXo0wxvD0J5so\nP13Nq3elaeErZbE7L4tlWFpHZn+xnWwd47xo2mCNWJpbwvK8AzwyuAdJ7cOsjqOU1xMRnr+j97/H\nOE/qGOdF0dK/gEPHz/D04k2pGcTWAAALmklEQVT07RzBvdd0tTqOUsouPNif18akU3j0FM98omOc\nF0NL/zyMMUxelEdlTS2zRqXh56ubSilXcnl8JA9c352FG4pYmrO/8QcoQEv/vOZnFbFq6yEmDUmi\na3So1XGUUufw0I2J9O0cwRMf51FUdsrqOG5BS/8cispOMe3TzVzRNZK7r4y3Oo5S6jz8fH14/a6+\nGAPj5+boGKcDtPQbqKszPLYgF2MMr4xMw8dHrI6klLqAzm1bMS0zhR/3HOWtrwusjuPytPQb+OCH\nvazZeYSnfpFMXGQrq+MopRwwvG8smekdee2rHWzYV2Z1HJempV/P7sMVvPTZFq7rGc2Yy+Maf4BS\nyiWICM/d0ZsO4UGMm5PNiTPVVkdyWVr6drV1hgnzsgn082X6iFREdLeOUu4kLMif1+5Kp6jsFM/o\n2TjPS0vf7q/f7mLDvmM8e3sKMWFBVsdRSl2CjPhIHrwhkUUbilmiY5znpKUPbD94glkrtzMkpT2Z\n6R2tjqOUaoIHb+hOvy5teHKRjnGei9eXfnVtHePnZdM6yI/nh/fW3TpKuTk/Xx9euysdgEfmZlNT\nW2dxItfi9aX/5tcFbCo+zgvDexMV6vaX71VKAXGRrXjujt6s21PGW/9062s3OZ1Xl35eUTl/XlXA\n8L6xDOndweo4SiknuqNvLHekd+T1r3awfq+OcZ7ltaV/prqWCfOzaRsawNRhKVbHUUo1g2lnxzjn\nbtQxTjuvLf3ZX25n+8GTTB+RSngrf6vjKKWaQViQP6+PSWf/sTNM0bNxAl5a+uv3HuXt1bsY2z+O\n63q65dUblVIO6tclkgdv6M7HG4v5JLvY6jiW87rSP1VVw4R5OcRGBPPk0GSr4yilWsAD13cno0sb\nnvp4E4VHvXuM0+tKf/pnW9lz5BSvjEwjNNDP6jhKqRbg5+vDbPsY5zgvH+P0qtJfU3CY9/+1l99e\nHc+V3dpaHUcp1YLiIlvx/PDerN9bxp+9+GycXlP6J85U8+iCXLpGhfDYLUlWx1FKWSAzPZbhfWN5\n46sdrN971Oo4lvCa0n/+0y2UlJ9m5ug0ggN8rY6jlLLItMwUYtsE8/CcbI574RinV5T+qq0HmZtV\nyH3XduOyzm2sjqOUslDrIH9eu6svJeVn+N2763j3+93kFB6jqsY79vN7/JHMsooqJi3Mo2dMa8bd\nlGh1HKWUC+jXpQ3P3p7Cn1cV8OzSzQAE+vnQJzacvp0juKxzG/p2bkP7cM87464Y41rXlMzIyDBZ\nWVlOe76HPtrI8rwSFt9/Nb1jw532vEopz1BSfpoNe4+xcV8ZG/aVsan4OFX26Z6O4UH07dzG9oug\nSxtSOoYR6Oeau4dFZL0xJqOx9Tz6lf6y3BKW5Oxn/OAeWvhKqXPqEB7M0NRghqbazr9VWVPL5v3H\n2bjvGBv2lbFx3zGW5ZUAEODrQ0psGH3j2nBZlwj6dm5Dx/Agtzo7r8e+0i89UcnNs78hLrIVC/9w\nFf6+XnH4QinVDA4eP8PGff/3biC3qJxK+zGAmLDAn/wS6BMbTpB/y78b8OpX+sYYHl+UR0VVLbNG\npWnhK6WaJCYsiCG92zOkd3vAdh2OLSU/fTfwef4BAPx8hOSOYfbjArbjA53aBLvMuwGPLP1FG4r5\ncstBnrytF4kxra2Oo5TyMP6+PqR2iiC1UwR3XxUP2PYuZBee/SVQxtx1hby3Zg8AUaGB9O0c8e9f\nAqmdwmkVYE39OvRdRWQI8DrgC/zVGPNyg/sDgf8F+gFHgLuMMXvs9z0O3APUAg8ZY1Y4Lf05lJSf\nZurSfC6Pb8PvBiY057dSSql/i24dyODkGAYnxwBQU1vHtoMn2LDvGBv3lrGx8BhfbD4IgK+PkNS+\n9U/eDXRp26pF3g00Wvoi4gu8CQwGioB1IrLEGLO53mr3AGXGmO4iMgaYDtwlIsnAGCAF6Ah8KSI9\njDG1zv6LgG23zmMLcqmpNcwclYavj2u8nVJKeR8/Xx9SOoaT0jGcX1/RBYCjFVVkF5b9e7fQxxuL\n+eCHvQC0aeXP6MvjePzWXs2by4F1+gMFxphdACIyB8gE6pd+JjDVfnsB8Gex/crKBOYYYyqB3SJS\nYH++fzkn/k/948d9fLvjMM/d0ZsubUOa41sopdQliwwJ4IakGG5Isr0bqK0z7Dh0wvZLYG8ZUSHN\nf8lWR0o/Fiis93URMOB86xhjakSkHGhrX/5Dg8fGXnLaC9h35BQvLNvCNYlR/GpA5+b4Fkop5VS2\n3TxhJLUPY2z/luktlxhrEZF7RSRLRLJKS0sv6TnqjKFflzZMH5HqMkfJlVLK1ThS+sVAXL2vO9mX\nnXMdEfEDwrEd0HXksRhj3jbGZBhjMqKjox1PX098VAgf3DOAjhHBl/R4pZTyBo6U/jogUUQSRCQA\n24HZJQ3WWQLcbb89ElhlbJ/6WgKMEZFAEUkAEoEfnRNdKaXUxWp0n759H/0DwApsI5t/M8bki8g0\nIMsYswR4B/jAfqD2KLZfDNjXm4ftoG8NcH9zTe4opZRqnMeehkEppbyJo6dhcIkDuUoppVqGlr5S\nSnkRLX2llPIiWvpKKeVFtPSVUsqLuNz0joiUAnub8BRRwGEnxWkJ7pYXNHNLcbfM7pYXPCtzF2NM\no59udbnSbyoRyXJkbMlVuFte0Mwtxd0yu1te8M7MuntHKaW8iJa+Ukp5EU8s/betDnCR3C0vaOaW\n4m6Z3S0veGFmj9unr5RS6vw88ZW+Ukqp8/CY0heRISKyTUQKRGSy1XkcISJ7RCRPRLJFxCXPMici\nfxORQyKyqd6ySBH5QkR22P9sY2XGhs6TeaqIFNu3dbaI3GZlxvpEJE5EvhaRzSKSLyIP25e77Ha+\nQGZX3s5BIvKjiOTYMz9rX54gImvt3THXfgp5y10g73sisrveNk6/qCc2xrj9f9hO+bwT6AoEADlA\nstW5HMi9B4iyOkcjGQcBlwGb6i2bAUy2354MTLc6pwOZpwITrc52nrwdgMvst1sD24FkV97OF8js\nyttZgFD7bX9gLXAFMA8YY1/+38AfrM7aSN73gJGX+rye8kr/3xdvN8ZUAWcv3q6ayBizGts1EurL\nBN63334fuKNFQzXiPJldljGmxBizwX77BLAF27WkXXY7XyCzyzI2J+1f+tv/M8ANwAL7cpfZzhfI\n2ySeUvrnuni7S/8A2hlgpYisF5F7rQ5zEWKMMSX22weAGCvDXIQHRCTXvvvHZXaV1Cci8UBfbK/q\n3GI7N8gMLrydRcRXRLKBQ8AX2PYQHDPG1NhXcanuaJjXGHN2G79g38azRSTwYp7TU0rfXQ00xlwG\n3ArcLyKDrA50sYztvac7jID9F9ANSAdKgFnWxvk5EQkFFgLjjDHH69/nqtv5HJldejsbY2qNMenY\nrtfdH0iyONIFNcwrIr2Bx7HlvhyIBCZdzHN6Suk7dAF2V2OMKbb/eQj4GNsPoTs4KCIdAOx/HrI4\nT6OMMQft/4DqgP/Bxba1iPhjK88PjTGL7ItdejufK7Orb+ezjDHHgK+BK4EIETl76ViX7I56eYfY\nd60ZY0wl8C4XuY09pfQduXi7SxGREBFpffY2cDOw6cKPchlLgLvtt+8GPrEwi0POlqfdcFxoW4uI\nYLvO9BZjzKv17nLZ7Xy+zC6+naNFJMJ+OxgYjO1YxNfASPtqLrOdz5N3a70XAoLt+MNFbWOP+XCW\nfTTsNf7v4u0vWBzpgkSkK7ZX92C7QP0/XDGziHwEXIftzH4HgWeAxdgmHjpjOyPqaGOMyxw4PU/m\n67DtcjDYpqbuq7e/3FIiMhD4FsgD6uyLn8C2j9wlt/MFMo/FdbdzKrYDtb7YXvDOM8ZMs/9bnINt\nV8lG4Ff2V9GWukDeVUA0tumebOD39Q74Nv68nlL6SimlGucpu3eUUko5QEtfKaW8iJa+Ukp5ES19\npZTyIlr6SinlRbT0lVLKi2jpK6WUF9HSV0opL/L/AS+8x3TLH6X4AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zbLvQmb5H97t",
        "colab_type": "text"
      },
      "source": [
        "So this is basically the one cycle scheduler.\n",
        "\n",
        "TODO: make sure it is actually the learning rate scheduler"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_6sOwvl1mdkV",
        "colab_type": "text"
      },
      "source": [
        "# half precision training"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dhy7_bcu7JX6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "learn.to_fp16();"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cvzaCwzmmgNy",
        "colab_type": "text"
      },
      "source": [
        "# learning rate"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JrQ2ctUqmbeq",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "f86d8ab5-50fd-4c9d-875d-78c7ac0ce53e"
      },
      "source": [
        "learn.lr_find()"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tl8zAhGkmfUD",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "outputId": "673e95f2-7615-4785-d3f3-99bb53f085de"
      },
      "source": [
        "learn.recorder.plot()"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8XGd97/HPT/u+WZJ3W17iJI6z\n2Fb2FUIgZSmEraQ0NEkhF0ppUlJKC7fQW0op5JJSSEtIWQI0QIGkBXJpFogdJ2SVHcdr5C1e5E2y\nte8aze/+MccTRZFkOdKZRfq+X695eeacZ+b8ZizpO895znmOuTsiIiIAGckuQEREUodCQURE4hQK\nIiISp1AQEZE4hYKIiMQpFEREJE6hICIicQoFERGJUyiIiEhcVrILOFWVlZVeU1OT7DJERNLK+vXr\nj7l71cnapV0o1NTUUFdXl+wyRETSipntG0877T4SEZE4hYKIiMQpFEREJE6hICIicQoFERGJUyiI\niEicQkFEROIUCiIiaeBffrOTp3YdC307CgURkRR3rLOPr/12B3X7WkLflkJBRCTFrdvRhDu88Yzq\n0LelUBARSXFr6puoKs5l+eyS0LelUBARSWGRwSjrdjRx1bIqMjIs9O0pFEREUtjGA6209QzwhgTs\nOoIQQ8HM5pvZGjPbZmZbzezWEdqUmtmvzOzFoM1NYdUjIpKO1tQ3kplhXHZaZUK2F+bU2RHgdnff\nYGbFwHoze9Tdtw1p83Fgm7u/w8yqgHozu8/d+0OsS0QkbTz2UhO1C8spyctOyPZC6ym4+2F33xDc\n7wC2A3OHNwOKzcyAIqCZWJiIiEx7R9p62X64PWG7jiBBF9kxsxpgJfDssFV3Ab8EDgHFwB+4ezQR\nNYmIpLq19Y0AvOH0xIVC6APNZlYE3A/c5u7tw1a/BdgIzAHOA+4ys9ccc2Vmt5hZnZnVNTU1hV2y\niEhKWFPfyJzSPJbNLErYNkMNBTPLJhYI97n7AyM0uQl4wGN2AS8DZwxv5O73uHutu9dWVZ30EqMi\nImmvPxLlyZ3HuOqMamJ72BMjzKOPDPgOsN3d7xyl2X7g6qD9TOB0YE9YNYmIpIu6vc109Q8mdNcR\nhDumcClwA7DZzDYGyz4DLABw97uBLwD3mtlmwIBPu3v4Mz6JiKS4NfWN5GRmcMmSGQndbmih4O5P\nEvtDP1abQ8Cbw6pBRCRdralv4oJFFRTmJuR4oDid0SwikmIaWrrZ1djJVacnfgxVoSAikmLW1seO\nsrwqweMJoFAQEUk5a+ubmFeez5KqwoRvW6EgIpJC+iNRntp9jCuXVSX0UNQTFAoiIimkbm8z3f2D\nSdl1BAoFEZGUsnZHU1IORT1BoSAikkLW1jdy/qLyhB+KeoJCQUQkRRxq7WHH0U6uWpacXUegUBAR\nSRmP7zhxKGry5nhTKIiIpIi1wayoS6sTNyvqcAoFEZEU0B+J8rtdx7ny9MTOijqcQkFEJAWs39dC\nZ18kqbuOQKEgIpIS1u5oJDvTuHRpZVLrUCiIiKSAZ/Y0s3J+OUVJOhT1BIWCiEiS9fQPsvVgG6tr\nypNdikJBRCTZXmxoJRJ1ahcqFEREpr31+1oAWK1QEBGR9ftaWFpdRFlBTrJLUSiIiCRTNOqs39fC\n6gXJ7yWAQkFEJKl2N3XS1jOQEoPMoFAQEUmqumA8IRUGmUGhICKSVOv3tVBRmMOiysRfenMkCgUR\nkSRav6+FVQvKkzrf0VAKBRGRJDnW2cfLx7qoTZHxBFAoiIgkzfoUG08AhYKISNJs2NdCTmYGK+aW\nJruUOIWCiEiS1O1rYcXcEvKyM5NdSpxCQUQkCXoHBtnc0EZtTUWyS3mV0ELBzOab2Roz22ZmW83s\n1hHafMrMNga3LWY2aGap9QmJiIRgy8E2+gejrEqRM5lPCLOnEAFud/flwEXAx81s+dAG7n6Hu5/n\n7ucBfwM87u7NIdYkIpISNuxPnUnwhgotFNz9sLtvCO53ANuBuWM85Xrgx2HVIyKSSg4091Can01V\ncW6yS3mVhIwpmFkNsBJ4dpT1BcC1wP2jrL/FzOrMrK6pqSmsMkVEEqa5u58ZhcmfFXW40EPBzIqI\n/bG/zd3bR2n2DuB3o+06cvd73L3W3WurqpJ7UWsRkcnQ3NlPxXQLBTPLJhYI97n7A2M0/QDadSQi\n00hz1zQLBYtN5PEdYLu73zlGu1LgSuAXYdUiIpJqjqdoKGSF+NqXAjcAm81sY7DsM8ACAHe/O1h2\nHfCIu3eFWIuISMqIRp2W7mkWCu7+JHDSaf/c/V7g3rDqEBFJNR29EQajnpKhoDOaRUQS7HhXHwAz\nihQKIiLTXnNXPwDlBQoFEZFp73gQCjMKU+vENVAoiIgkXEsQChXafSQiIq/0FBQKIiLTXnNXPwU5\nmSl1HYUTFAoiIgnW3NWfkoPMoFAQEUm44139KXk4KigUREQSriVFp7gAhYKISMKl6mR4oFAQEUm4\n4119KXnkESgUREQSqrs/Qu9AlHKFgoiINKfwOQqgUBARSagToVCRglNcgEJBRCShjsdDQT0FEZFp\nr7lToSAiIoFm9RREROSE5u5+sjONkrwwr4b8+ikUREQSqLkzNu+R2UmvVpwUCgURkQQ6nsJnM4NC\nQUQkoZq7+hQKIiIS09I9oFAQEZGY452pO+8RKBRERBJmYDBKe28kZc9mBoWCiEjCtMTPUchOciWj\nUyiIiCRIc3dqz3sECgURkYRJ9SkuIMRQMLP5ZrbGzLaZ2VYzu3WUdleZ2cagzeNh1SMikmwnJsNL\n1eszA4R5nnUEuN3dN5hZMbDezB51920nGphZGfBvwLXuvt/MqkOsR0QkqVJ93iMIsafg7ofdfUNw\nvwPYDswd1uwPgQfcfX/QrjGsekREku1ET6Esf5oPNJtZDbASeHbYqmVAuZmtNbP1ZvahRNQjIpIM\nLV39lBVkk5WZusO5oU/TZ2ZFwP3Abe7ePsL2VwNXA/nA02b2jLvvGPYatwC3ACxYsCDskkVEQtGc\n4vMeQcg9BTPLJhYI97n7AyM0aQAedvcudz8GrAPOHd7I3e9x91p3r62qqgqzZBGR0BzvSu2zmSHc\no48M+A6w3d3vHKXZL4DLzCzLzAqAC4mNPYiITDnNXbFps1NZmLuPLgVuADab2cZg2WeABQDufre7\nbzezh4BNQBT4trtvCbEmEZGkae4aYPXCaRoK7v4kcNKrSLj7HcAdYdUhIpIKolGnpXuajymIiEhM\ne+8Ag1FP6SkuQKEgIpIQ8bOZ1VMQEZETZzOXT4VQMLMlZpYb3L/KzP48mKJCRETGoXmK9RTuBwbN\nbClwDzAf+FFoVYmITDFH2noBqCqeGmMKUXePANcB33D3TwGzwytLRGRq2XqojRmFOVRPkVAYMLPr\ngT8GHgyWpe6MTiIiKWbzwXbOmltK7Lze1DXeULgJuBj4oru/bGaLgB+GV5aIyNTROzDIzqMdrJhT\nkuxSTmpcJ68F10D4cwAzKweK3f3LYRYmIjJV1B/pIBJ1zp5bmuxSTmq8Rx+tNbMSM6sANgD/bmaj\nzWckIiJDbDnUBsCKqRIKQGkw7fW7gR+4+4XAm8IrS0Rk6thysJ3S/Gzmlecnu5STGm8oZJnZbOD9\nvDLQLCIi47DlYBsr5pak/CAzjD8U/h54GNjt7s+b2WJgZ3hliYhMDf2RKPVHOlgxJ/V3HcH4B5p/\nBvxsyOM9wHvCKkpEZKrYcbSD/sFoWownwPgHmueZ2X+ZWWNwu9/M5oVdnIhIutuaRoPMMP7dR98D\nfgnMCW6/CpaJiMgYthxspzg3i4UVBckuZVzGGwpV7v49d48Et3sBXSxZROQkNh9sY/mcEjIyUn+Q\nGcYfCsfN7I/MLDO4/RFwPMzCRETSXWQwyvbD7Wmz6wjGHwo3Ezsc9QhwGHgvcGNINYmITAm7mjrp\ni0TT4kzmE8YVCu6+z91/392r3L3a3d+Fjj4SERnTloPtAKyYm/pzHp0wkSuvfXLSqhARmYK2HGyj\nICeTRZVFyS5l3CYSCukxaiIikiRbDraxfHYJmWkyyAwTCwWftCpERKaYwaizLc0GmeEkZzSbWQcj\n//E3IPVndhIRSZKXj3XR3T84tULB3YsTVYiIyFRysLUHgJoZ6XHS2gkT2X0kIiKjaOnqB6C8MCfJ\nlZwahYKISAhauoNQKFAoAGBm881sjZltM7OtZnbrCG2uMrM2M9sY3D4XVj0iIonU0j2AGZTmZye7\nlFMyrqmzX6cIcLu7bzCzYmC9mT0aXO95qCfc/e0h1iEiknCt3f2U5GWn1eGoEGJPwd0Pu/uG4H4H\nsB2YG9b2RERSSXNXPxVpNp4ACRpTMLMaYCXw7AirLzazF83sf8zsrETUIyISttbuAcoK0mvXESQg\nFMysCLgfuM3d24et3gAsdPdzgW8A/z3Ka9xiZnVmVtfU1BRuwSIik6Cluz/tBpkh5FAws2xigXCf\nuz8wfL27t7t7Z3D/10C2mVWO0O4ed69199qqKl3GQURSn3oKw5iZAd8Btrv7naO0mRW0w8wuCOrR\ndRpEJO01d/VTkYY9hTCPProUuAHYbGYbg2WfARYAuPvdxK7L8DEziwA9wAfcXXMqiUha6x0YpGdg\nMO1OXIMQQ8Hdn+QkM6m6+13AXWHVICKSDK3dAwDafSQiIul7NjMoFEREJp1CQURE4lq6YruPygu1\n+0hEZNpTT0FEROJag1DQQLOIiNDSPUBBTia5WZnJLuWUKRRERCZZS1d6TnEBCgURkUnX0t2floPM\noFAQEZl0Ld0D6imIiEhMa3c/ZQoFERGBE5PhafeRiMi0FxmM0t4bUU9BRESgrSc4m1k9BRERaek+\nMcWFegoiItNeOk9xAQoFEZFJ1dKlUBARkUA6X2AHFAoiIpMqvvtIYwoiItLSPUB2plGYk36T4YFC\nQURkUp2YDM9szEvUpyyFgojIJGrpTt8ZUkGhICIyqVq7B9J2kBkUCiIik0o9BRERiYtdS0GhICIy\n7bk7rd0DaTvvESgUREQmTUdfhEjUtftIRESgtSu9z2YGhYKIyKRpDs5mrtCYwmuZ2XwzW2Nm28xs\nq5ndOkbb880sYmbvDaseEZGwnZjiIl0vsAOQFeJrR4Db3X2DmRUD683sUXffNrSRmWUCXwYeCbEW\nEZHQtcanzdbuo9dw98PuviG43wFsB+aO0PQTwP1AY1i1iIgkQkvXiauupW9PISFjCmZWA6wEnh22\nfC5wHfDNkzz/FjOrM7O6pqamsMoUEZmQlu5+zKAkXz2FUZlZEbGewG3u3j5s9deAT7t7dKzXcPd7\n3L3W3WurqqrCKlVEZEJauvspy88mMyM9J8ODcMcUMLNsYoFwn7s/MEKTWuAnwWyClcBbzSzi7v8d\nZl2pbN/xLtbWNzG/Ip8rTqsiK1MHiImki5bugbTedQQhhoLF/tJ/B9ju7neO1MbdFw1pfy/w4HQL\nhL7IIFsPtbP2pUYe3nqU+qMd8XUzS3J57+p5vG/1fGoqC5NYpYiMR2t3f1qfowDh9hQuBW4ANpvZ\nxmDZZ4AFAO5+d4jbTprDbT08svUoLx5opaIwh9ll+cwuzaM0P5uuvghd/RE6+wbZd6yLDftb2HKo\nnf5IlAyD82sq+Nu3L+fqM6p56UgHP607wDfX7uZf1+ymsiiXxZWF1FQWsKiyiGUzi1g2s5i5Zflk\nBF3VaNRp6xmgtWeA7v4IvQODdPcPEnXIy8ogNzuTvOwMolHoGRikb2CQvkiUBTMKWFxZmLbzv4uk\niuauAeaU5iW7jAkJLRTc/Ulg3H9l3P3GsGoZTXd/hMNtvRxu7eVwWw+9A4OUFuRQlp9NWUE2BTlZ\n5GZlkJOVQXZmBl19EVq6+2nu6qelu5/+SJSBQWcwGpvv5LH6Rl480ApAdXEu7b0D9A6MPFySk5XB\nOXNLufGSGlYtKOP8mgpmFOXG19dUFnLtilkcaevlwU2H2HG0g73HunnspSaOdTbE2xXkZDKnLJ+2\nngGau/oZjPrr+iwqi3KoXVjBufPLaO3uZ3dTF3uaOjnS3svps4pZvaCc1QvLWVJdxNH2Xhpaemho\n6aale4CsDCMrI4PsTKOqOJfz5pdx1pxS8sd55anBqNPZG6EvMkhlUW485ETSTWt3P2fNKUl2GRMS\n6phCKmnrGWDjgVa2HGxjc0Mbmw+2cbC1Z1K3cc68Uj71ltN5y1mzWFpdhHvsm/vhtl7aewYozM0K\nbpmUF+SQPY7xglmleXz48sWvWtbeO8DOo53sONpB/ZEODrf1UF6Qw4yiHCqLcikryCY/O4v8nEwK\ncjIxoC8SpS8ySO9AlAwz8rIzyMvOJDszgx1HO3j+5Wae29vMQ1uPkJOZQU1lAafPKuby0yrZdrid\nHzyzj28/+fKr6sjKMMoKsolEnYFIlIGo0x+JhWBmhnH6zGLKC7PpG4jSGxmkbyBKJBoL0cGoE4lG\n6eobpLMvEn/NvOwMFlcWsaS6iIUVBZTmZ1OUl0VxXhYVBTnMKy9gdlneuD47kUSLTZut3UdpYW19\nI7f+JLYXa+GMAs5bUMb1F8xnbnk+s0rymVOWR35OJu09A7R2D9DSHdsF0x+J0j8YpT8SpTAni/LC\nHCoKcygvyCYvO5OsDCMzw8jNzqQo99Ufp5lRVpAz6Wc3luRls3ph7Jv7ZFi9sJzrL1gAxL7pFOVm\nvWaAuz8SZeuhNvY3dzOrJI95FQXMLM59TbvG9l5ebGjjxQOtvNjQSldfhLzsTErys8nNyiArM4NM\ng8yMDDIzoCg3m+Lgj352Zgb7m7vZ3dTJxgMtPLjpED5Cxyczw5hVkseS6iJWzClhxdxSVswpZV55\nvnoZkjS9A7EvXel8NjNMo1C4bGklP/rwhZw1t5TSMY4hri5O7/2BEzXaD3ROVgYrF5SzcsHYQVRd\nksc1y/O4ZvnMCdcSjXowBhOhszdCU2cfDc09HGjp5kBzNzuOdnLPuj1Egl1mmRlGZVEO1cV5VBfn\nMr+igCVVhSyuKmJxVSGzSvI0biKh2bCvBYh96Uxn0yYUZhTlcsnS3JM3lJSRkWEU52VTnJcNpXDa\nzGJY8uo2fZFBdhzpZMuhNhpaumls76Ops49Dbb08vec43f2D8bb52ZksnFHAospCFlcVcs68MlYu\nKJv2XwRkctz37H7KCrJ505kT/0KUTNMmFGRqys3K5Ox5pZw9r/Q169ydo+197GnqZPexLvYGt/qj\nHTy67Wi8hzGvPJ/z5pdx3vxYSJw1p5S87PENkosANHb08vDWI9x4SU3a/+woFGTKMjNmleYxqzSP\nS5ZWvmpd70Ds/JAX9rfwwv5WNuxr4cFNh4HYAPpZc0q4dGkll51WyeqF5eRmpfcvuoTrZ3UNRKLO\n9RcuSHYpE6ZQkGkpLzvzNYP1JwbJNx5o4dk9zXxr3R7+be1u8oO2qxaUsXJhOSvnl6X9YKJMnsGo\n8+Pn9nPJkhksqSpKdjkTplAQCQwfJO/oHeCZPc08ubOJ5/e2cNeaXZw4DWThjALOnFXC8jklnDm7\nhHnl+VQX51JekKMjoKaZdTubaGjp4W9+78xklzIpFAoioyjOy+aa5TPjIdHVF2FTQxsb9rew7VA7\n2w6389DWI696TlaGMbMkjwsXV/DGM6q5/LSqMY92k/R33zP7qSzKnZQj7lKBQkFknApzs7h4yQwu\nXjIjvqyzL0L9kQ6OtPXS1NFLY0cf+5q7eeylRh7YcJDMDKN2YTlXLKviymVVLJ9dop7EFHKotYfH\nXjrKR69cQk7W1DihUqEgMgFFuVkjnkQ4GHVe2N/CYy81sra+iTserueOh+upLMrhwkUzWFodO3di\nSXAORUGOfhXT0U+eP4BD/OTPqUA/iSIhyMwwamsqqK2p4K+uPYPGjl6e3HmMdTuaWL+/hV9vORw/\nW9sMFlQUsGxmMWfMKmZeeT5lBTmUF+TEZ9zs6R+MTWIYibKgooCFFQXqcSRZR+8AP35uP1cuq2J+\nRXqfsDaUQkEkAaqL83j3qnm8e9U8IHZI7L7jsSk9Tsxj9dKRdh57qXFckxoW5mRy5uzYFB9vP2c2\nqxeW62ztBPvKQ/Uc6+zj1qtPS3Ypk0qhIJIEedmZnD6rmNNnFcPZryzviwzS1NEXzL/VT2v3ABlm\n5OdkBHNtZfDysU62HWpn66F2fvL8fu59ai+LKwt5b+083rNqHjNLdIZ22Or2NvPDZ/Zx06U1J536\nJd2YjzTjWAqrra31urq6ZJchkhK6+iL8evNhflbXwHN7mzGDVQvK40dNTYXj5lNN78Agb/v6E/QO\nRHnkL66gMDc9vlub2Xp3rz1pO4WCyNTw8rEufrnxEI9uP8KWg7HLoS+bWcSHLq7h3avmajB7knz1\nkXq+8dguvn/zBVy5LH2uGa9QEJnGDrX28JvtR/lZXQObD7ZRmp/N9Rcs4IMXLphSg6KJ9tKRdt7+\n9Sf5/XPncOcfnJfsck6JQkFEcHfW72vhu797mYe2HCHqsd7DG86o5o2nV7N6YflrrokhI3N33nf3\n0+w51sVvPnklFYXpNdXJeENB/UmRKczslUNjG1q6eWjLER57qZHvPPEy33p8D9XFubx39TzeXzuf\nmsrCZJeb0tbUN1K3r4UvXrci7QLhVKinIDINdfQOsG7HMR7Y0MCa+kaiDhctruDDly3m6jOrdXjr\nMNGo8467nqSjN8Jvb78yLS8Hq56CiIyqOC+bt50zm7edM5sjbb3cv6GBHz+3nw//oI5z55fxyWuW\nccVplQqHwENbj7D1UDt3vv/ctAyEU6GegogAMDAY5YENDXz9t7s42NrD+TXlfPDChVx9ZnXs6nfT\n1GDUecvX1uHuPPIXV5KZpmeSq6cgIqckOzODPzh/Ae9aOZefPn+Aux/fw23/uZGcrAyuXFbF7587\nh7edPXvaTa/xyxcPsquxk3/9w1VpGwinQqEgIq+Sm5XJDRfX8MELF/LCgdgV6X69+TCPbjvKg5sO\ncef7z0ubE7YmamAwyj8/upPls0v4vRWzkl1OQkztnWMi8rplZBirF1bw+XecxdN/fTV/+/blPLrt\nKO/55lM0tHQnu7yE+FldA/ubu7n9zcumTQ9JoSAiJ5WRYfzJZYv43k0XcLC1h3fe9Tvq9jYnu6xQ\ndfQOcOejO1i1oIw3nlGd7HISRqEgIuN25bIq/utPL6U4L4v3f+tpPvnTjbx8rCvZZYXiG4/t4lhn\nH59/x1nT6iis0ELBzOab2Roz22ZmW83s1hHavNPMNpnZRjOrM7PLwqpHRCbH0uoifvHxy7j50kX8\nevNhrv7qWj75n1MrHHY3dfK9373M+2vnce78smSXk1ChHZJqZrOB2e6+wcyKgfXAu9x925A2RUCX\nu7uZnQP81N3PGOt1dUiqSOpo6ujjnnW7+eEz+xiMOv/riiX82RuXkpedmezSJuTG7z3H+r0tPPaX\nV1FVnJvscibFeA9JDa2n4O6H3X1DcL8D2A7MHdam019JpUIgvU6aEJnmqopz+ezblrPur97AO86Z\nw11rdvGWr61j3Y6mhGzf3XlgQwP3rNvNkbbeSXnNx146ytr6Jm5902lTJhBORUJOXjOzGmAdsMLd\n24etuw74ElANvM3dnx7h+bcAtwAsWLBg9b59+8IuWUReh6d2HeN///cW9hzr4j2r5vEP71pBfk44\nvYbOvgifvn8T/2/TYQAyDC4/rYr3rp7HNctnvq7eSl9kkGu/9gRm8NCtV5CTNXWGXVNmltRgF9Hj\nwBfd/YEx2l0BfM7d3zTW62n3kUhq64sMctdju7hrzS7OmFXCPTesnvTpuncc7eCj/7Gevce6+NRb\nzuDaFbN4YEMD969v4FBbL3nZGVy2tIprllfzxjNmnvQbfzTqrNvZxN2P7+aZPc3ce9P5XHX61Dri\nKCVCwcyygQeBh939znG03wNc4O7HRmujUBBJD2vqG7n1xy+QkWF8/QMruWKCF6SJDEZ5saGNtfWN\nfPuJlynMzeIb16/k4iUz4m0Go84ze47zyNYj/GZ7IwdbewAozsuiND+bsoJsyvJzmFmSx5yyPGaX\n5tMfGeSHz+xjd1MX1cW5fOyqJdx06aIJ1ZqKkh4KFjuG6/tAs7vfNkqbpcDuYKB5FfArYJ6PUZRC\nQSR97D3WxUf/Yz07jnbwkcsX87GrllBWcGrTTtcf6eDrv93JEzubaO+NkGFwxbIqvvKec6ge43rU\n7s72wx2s29nEkbZe2nsGaO2JXfv6SFsvR9t7iQZ/ac6eW8qfXLaIt549e0rtMhoqFULhMuAJYDMQ\nDRZ/BlgA4O53m9mngQ8BA0AP8Cl3f3Ks11UoiKSX7v4In//FVn6+oYGinCw+fPlibr6sZlyT7G1q\naOWG7zyHGVxz5kyuPL2Ky5ZWnnKwjCQyGKWxo4/u/kGWVBVO+XMRkh4KYVEoiKSnl460c+cjO3hk\n21HKC2JTd1+ypJKLF8+gfISL1qzf18KN332O0oJsfvyRi3QZ0QlSKIhIStrU0Mq/rtnFkzuP0dU/\niBksn13CBYsquHBR7Cpxuxs7ufne56kqzuW+j1zE3LL8ZJed9hQKIpLSBgajbGpo5aldx3lq93Fe\nONBC70BsT3NmhlEzo4AffeQiZo4xbiDjp+spiEhKy87MYPXCClYvrOATV59GfyTK5oNtPL+3maPt\nvfzpVUun5cljyaZQEJGUkJOVweqF5axeWJ7sUqa1qXnslYiIvC4KBRERiVMoiIhInEJBRETiFAoi\nIhKnUBARkTiFgoiIxCkUREQkLu2muTCzNmDnCKtKgbYxlg1ff+LxSG0qgVGv6XASI9UxnvUnq3/4\n45Huq/7UqB9e/3s4Wf1jtRmr3uGPp2L9Q++nQv1j1Tn0caL+Bi1095Nf1MLd0+oG3DPe5UOXDV9/\n4vFIbYC6ya5vovWP9X6GvxfVn9z6J/IeTlb/qbyH6Vb/ZPwMTWb9Y9U5xuce+u/AyW7puPvoV6ew\n/FdjrP/VONq8Hid7jddb//DHI91X/VO//rHajFXv8MdTsf7xbn8sk1n/8GWp8jdoTGm3+ygRzKzO\nxzGbYKpS/cmX7u9B9SdXMutPx55CItyT7AImSPUnX7q/B9WfXEmrXz0FERGJU09BRETipnwomNl3\nzazRzLa8jueuNrPNZrbLzL7PXJYLAAAG90lEQVRuQ67sbWafMLOXzGyrmX1lcqt+VQ2TXr+Z/Z2Z\nHTSzjcHtrZNfebyGUD7/YP3tZuZmVjl5Fb+mhjA+/y+Y2abgs3/EzOZMfuXxGsKo/47gZ3+Tmf2X\nmZVNfuWvqiOM9/C+4Hc3amaTvu9+IjWP8np/bGY7g9sfD1k+5u/I6zKRQ5fS4QZcAawCtryO5z4H\nXAQY8D/A7wXL3wD8BsgNHlenWf1/B/xlun7+wbr5wMPAPqAyneoHSoa0+XPg7jSr/81AVnD/y8CX\n0+1nCDgTOB1YC9SmSs1BPTXDllUAe4J/y4P75WO9v4ncpnxPwd3XAc1Dl5nZEjN7yMzWm9kTZnbG\n8OeZ2Wxiv7zPeOzT/wHwrmD1x4B/cve+YBuNaVZ/woRY/z8DfwWEOigWRv3u3j6kaSEhvoeQ6n/E\n3SNB02eAeWHVH+J72O7u9alW8yjeAjzq7s3u3gI8Clwb1u/4lA+FUdwDfMLdVwN/CfzbCG3mAg1D\nHjcEywCWAZeb2bNm9riZnR9qta810foB/izo/n/XzBJ9/cMJ1W9m7wQOuvuLYRc6igl//mb2RTM7\nAHwQ+FyItY5kMn5+TriZ2DfURJvM95Ao46l5JHOBA0Men3gfoby/aXeNZjMrAi4BfjZk99upXh08\ni1hX7iLgfOCnZrY4SOtQTVL93wS+QOwb6heArxL75Q7dROs3swLgM8R2YSTcJH3+uPtngc+a2d8A\nfwZ8ftKKHMNk1R+81meBCHDf5FQ37u1O2ntIlLFqNrObgFuDZUuBX5tZP/Cyu1+X6FqnXSgQ6x21\nuvt5QxeaWSawPnj4S2J/OId2i+cBB4P7DcADQQg8Z2ZRYnOVNIVZeGDC9bv70SHP+3fgwTALHmai\n9S8BFgEvBr9c84ANZnaBux8JuXaYnJ+foe4Dfk2CQoFJqt/MbgTeDlydiC9Dw0z2/0EijFgzgLt/\nD/gegJmtBW50971DmhwErhryeB6xsYeDhPH+JnuAJRVvQA1DBnyAp4D3BfcNOHeU5w0fxHlrsPyj\nwN8H95cR69pZGtU/e0ibvwB+kk6f/7A2ewlxoDmkz/+0IW0+Afw8zeq/FtgGVIVZdyJ+hghpoPn1\n1szoA80vExtkLg/uV4zn/b2uuhP1n5qsG/Bj4DAwQOwb/p8Q+6b5EPBi8MP9uVGeWwtsAXYDd/HK\nyX45wH8E6zYAb0yz+n8IbAY2EftGNTud6h/WZi/hHn0Uxud/f7B8E7F5auamWf27iH0R2hjcQjt6\nKsT3cF3wWn3AUeDhVKiZEUIhWH5z8LnvAm46ld+RU73pjGYREYmbrkcfiYjICBQKIiISp1AQEZE4\nhYKIiMQpFEREJE6hIFOCmXUmeHvfNrPlk/RagxabMXWLmf3qZLOOmlmZmf3pZGxbZDgdkipTgpl1\nunvRJL5elr8y6VuohtZuZt8Hdrj7F8doXwM86O4rElGfTC/qKciUZWZVZna/mT0f3C4Nll9gZk+b\n2Qtm9pSZnR4sv9HMfmlmjwG/NbOrzGytmf3cYtcPuO/EfPXB8trgfmcwwd2LZvaMmc0Mli8JHm82\ns38YZ2/maV6Z+K/IzH5rZhuC13hn0OafgCVB7+KOoO2ngve4ycz+zyR+jDLNKBRkKvsX4J/d/Xzg\nPcC3g+UvAZe7+0piM5T+45DnrALe6+5XBo9XArcBy4HFwKUjbKcQeMbdzwXWAR8Zsv1/cfezefVs\nliMK5u65mthZ5gC9wHXuvorYNTy+GoTSXwO73f08d/+Umb0ZOA24ADgPWG1mV5xseyIjmY4T4sn0\n8SZg+ZBZKUuC2SpLge+b2WnEZorNHvKcR9196Dz4z7l7A4CZbSQ2n82Tw7bTzyuTCq4HrgnuX8wr\n89v/CPi/o9SZH7z2XGA7sfnyITafzT8Gf+CjwfqZIzz/zcHtheBxEbGQWDfK9kRGpVCQqSwDuMjd\ne4cuNLO7gDXufl2wf37tkNVdw16jb8j9QUb+nRnwVwbnRmszlh53Py+YFvxh4OPA14lda6EKWO3u\nA2a2F8gb4fkGfMndv3WK2xV5De0+kqnsEWKzkAJgZiemLS7llSmGbwxx+88Q220F8IGTNXb3bmKX\n57zdzLKI1dkYBMIbgIVB0w6geMhTHwZuDnpBmNlcM6uepPcg04xCQaaKAjNrGHL7JLE/sLXB4Os2\nYlOeA3wF+JKZvUC4veXbgE+a2SZiF09pO9kT3P0FYrOnXk/sWgu1ZrYZ+BCxsRDc/Tjwu+AQ1jvc\n/RFiu6eeDtr+nFeHhsi46ZBUkZAEu4N63N3N7APA9e7+zpM9TySZNKYgEp7VwF3BEUOtJOiSpyIT\noZ6CiIjEaUxBRETiFAoiIhKnUBARkTiFgoiIxCkUREQkTqEgIiJx/x8+oG6jt+1DOAAAAABJRU5E\nrkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lh6GxXeBmkzn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lr = 1e-2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yzu9P7NNmj-G",
        "colab_type": "text"
      },
      "source": [
        "# training"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7xiLFBFm91KX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 715
        },
        "outputId": "c1f0fa19-4593-4687-9b67-222b4e77506b"
      },
      "source": [
        "learn.fit_one_cycle(35, max_lr=1e-2)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "        <style>\n",
              "            /* Turns off some styling */\n",
              "            progress {\n",
              "                /* gets rid of default border in Firefox and Opera. */\n",
              "                border: none;\n",
              "                /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "                background-size: auto;\n",
              "            }\n",
              "            .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "                background: #F44336;\n",
              "            }\n",
              "        </style>\n",
              "      <progress value='20' class='' max='35', style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      57.14% [20/35 43:51<32:53]\n",
              "    </div>\n",
              "    \n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>1.570242</td>\n",
              "      <td>1.554749</td>\n",
              "      <td>0.416400</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>1.233263</td>\n",
              "      <td>1.200655</td>\n",
              "      <td>0.564800</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>1.000742</td>\n",
              "      <td>1.038036</td>\n",
              "      <td>0.635000</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.837638</td>\n",
              "      <td>0.897606</td>\n",
              "      <td>0.684200</td>\n",
              "      <td>02:10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.700422</td>\n",
              "      <td>0.754672</td>\n",
              "      <td>0.738100</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.621898</td>\n",
              "      <td>0.741733</td>\n",
              "      <td>0.746100</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>0.596334</td>\n",
              "      <td>0.748664</td>\n",
              "      <td>0.744200</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0.520615</td>\n",
              "      <td>0.684422</td>\n",
              "      <td>0.766800</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>0.472359</td>\n",
              "      <td>0.548142</td>\n",
              "      <td>0.815100</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>0.450922</td>\n",
              "      <td>0.547788</td>\n",
              "      <td>0.814800</td>\n",
              "      <td>02:12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.426367</td>\n",
              "      <td>0.536002</td>\n",
              "      <td>0.817400</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>0.394311</td>\n",
              "      <td>0.495053</td>\n",
              "      <td>0.831300</td>\n",
              "      <td>02:10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>0.390423</td>\n",
              "      <td>0.457856</td>\n",
              "      <td>0.842900</td>\n",
              "      <td>02:09</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>0.352253</td>\n",
              "      <td>0.465816</td>\n",
              "      <td>0.843700</td>\n",
              "      <td>02:11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>14</td>\n",
              "      <td>0.344112</td>\n",
              "      <td>0.468566</td>\n",
              "      <td>0.840900</td>\n",
              "      <td>02:12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>15</td>\n",
              "      <td>0.333694</td>\n",
              "      <td>0.520868</td>\n",
              "      <td>0.826000</td>\n",
              "      <td>02:12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>16</td>\n",
              "      <td>0.323888</td>\n",
              "      <td>0.411523</td>\n",
              "      <td>0.864300</td>\n",
              "      <td>02:12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>17</td>\n",
              "      <td>0.311073</td>\n",
              "      <td>0.415248</td>\n",
              "      <td>0.860700</td>\n",
              "      <td>02:12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>18</td>\n",
              "      <td>0.271027</td>\n",
              "      <td>0.392089</td>\n",
              "      <td>0.867500</td>\n",
              "      <td>02:12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>19</td>\n",
              "      <td>0.278712</td>\n",
              "      <td>0.378842</td>\n",
              "      <td>0.874800</td>\n",
              "      <td>02:10</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>\n",
              "\n",
              "    <div>\n",
              "        <style>\n",
              "            /* Turns off some styling */\n",
              "            progress {\n",
              "                /* gets rid of default border in Firefox and Opera. */\n",
              "                border: none;\n",
              "                /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "                background-size: auto;\n",
              "            }\n",
              "            .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "                background: #F44336;\n",
              "            }\n",
              "        </style>\n",
              "      <progress value='121' class='' max='390', style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      31.03% [121/390 00:38<01:26 0.2508]\n",
              "    </div>\n",
              "    "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LKATetdEy86y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}