geyang/Learning Punctuations by reading Engadget.ipynb

## Learning Punctuations by reading Engadget.ipynb
{
  "cells": [
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "# Learning Auto-Punctuation by Reading Engadget Articles  [[Repo]](https://github.com/episodeyang/deep-auto-punctuation#learning-auto-punctuation-by-reading-engadget-articles)\n\n[[DOI]](https://zenodo.org/badge/latestdoi/86387963)\n\n\n## Overview\n\nThis project trains a bi-directional GRU to learn how to automatically punctuate a sentence by reading it character by character. The set of operation it learns include:\n```\ncapitalization: <cap>\n         comma:  ,\n        period:  .\n   dollar sign:  $\n     semicolon:  ;\n         colon:  :\n  single quote:  '\n  double quote:  \"\n  no operation: <nop>\n```\n\n### Performance\n\n24 epochs takes about 8 hours on a i7 PC. Performance at 24 and 48 epochs are similar with slight improvements. Here is the performance at 48 epoch, for detailed record refer to the cell output at the end of this notebook.\n\n```\n    Test P/R  Epoch 48 Batch 380\n    =================================\n    Key: <nop>\tPrec:  97.1%\tRecall:  98.0%\tF-Score:  97.6%\n    Key: <cap>\tPrec:  73.2%\tRecall:  58.9%\tF-Score:  65.3%\n    Key:   ,\tPrec:  35.7%\tRecall:  32.2%\tF-Score:  33.9%\n    Key:   .\tPrec:  45.0%\tRecall:  39.7%\tF-Score:  42.2%\n    Key:   '\tPrec:  81.7%\tRecall:  83.4%\tF-Score:  82.5%\n    Key:   :\tPrec:  12.1%\tRecall:  10.8%\tF-Score:  11.4%\n    Key:   \"\tPrec:  25.2%\tRecall:  44.8%\tF-Score:  32.3%\n    Key:   $\tPrec:  51.4%\tRecall:  87.8%\tF-Score:  64.9%\n    Key:   ;\tPrec:   0.0%\tRecall:   0.0%\tF-Score:   N/A\n    Key:   ?\tPrec:   5.6%\tRecall:   4.8%\tF-Score:   5.1%\n    Key:   !\tPrec:   0.0%\tRecall:   0.0%\tF-Score:   N/A\n```\n\n\n### Requirements\n\n```\npytorch numpy matplotlib tqdm bs4\n```\n\n## Model Setup and Considerations\n\nThe initial setup I began with was a single uni-direction GRU, with input domain [A-z0-9] and output domain of the ops listed above. My hope at that time was to simply train the RNN to learn correcponding operations. A few things jumped out during the experiment:\n\n1. **Use bi-directional GRU.** with the uni-direction GRU, the network quickly learned capitalization of terms, but it had difficulties with single quote. In words like \"I'm\", \"won't\", there are simply too much ambiguity from reading only the forward part of the word. The network didn't have enough information to properly infer such punctuations.\n    \n    So I decided to change the uni-direction GRU to bi-direction GRU. The result is much better prediction for single quotes in concatenations.\n\n    the network is still training, but the precision and recall of single quote is nowt close to 80%.\n    \n    This use of bi-directional GRU is standard in NLP processes. But it is nice to experience first-hand the difference in performance and training.\n    \n    A side effect of this switch is that the network now runs almost 2x slower. This leads to the next item in this list:\n2. **Use the smallest model possible.** At the very begining, my input embeding was borrowed from the Shakespeare model, so the input space include both capital alphabet as well as lower-case ones. What I didn't realize was that I didn't need the capital cases because all inputs were lower-case. \n    \n    So when the training became painfully slow after I switch to bi-directional GRU, I looked for ways to make the training faster. A look at the input embeding made it obvious that half of the embedding space wasn't needed. \n\n    Removing the lower case bases made the traing around 3x faster. This is a rough estimate since I also decided to redownload the data set at the same time on the same machine.\n    \n3. **Text formatting**. Proper formating of input text crawed from Engadget.com was crucial, especially because the occurrence of a lot of the puncuation was low and this is a character-level model. You can take a look at the crawed text inside [./engadget_data_tar.gz](./engadget_data_tar.gz). \n\n4. **Async and Multi-process crawing is much much faster**. I initially wrote the engadget crawer as a single threaded class. Because the python `requests` library is synchronous, the crawler spent virtually all time waiting for the `GET` requests.\n    \n    This could be made a *lot* faster by parallelizing the crawling, or use proper async pattern. \n\n    This thought came to me pretty late during the second crawl so I did not implement it. But for future work, parallel and async crawler is going to be on the todo list.\n\n5. **Using Precision/Recall in a multi-class scenario**. The setup makes the reasonable assumption that each operation can only be applied mutually exclusively. The accuracy metric used here are **precision/recall** and the **F-score**, both commonly used in the literature<sup>1,</sup> <sup>2</sup>. The P/R and F-score are implemented according to wikipedia <sup>3,</sup> <sup>4</sup>.\n    \n    example accuracy output:\n    \n    ```\n    Epoch 24 Batch 760 Test P/R\n    =================================\n    Key: <nop>\tPrec:  97.1%\tRecall:  97.8%\tF-Score:  97.4%\n    Key: <cap>\tPrec:  68.6%\tRecall:  57.8%\tF-Score:  62.7%\n    Key:   ,\tPrec:  30.8%\tRecall:  30.9%\tF-Score:  30.9%\n    Key:   .\tPrec:  43.7%\tRecall:  38.3%\tF-Score:  40.8%\n    Key:   '\tPrec:  76.9%\tRecall:  80.2%\tF-Score:  78.5%\n    Key:   :\tPrec:  10.3%\tRecall:   6.1%\tF-Score:   7.7%\n    Key:   \"\tPrec:  26.9%\tRecall:  45.1%\tF-Score:  33.7%\n    Key:   $\tPrec:  64.3%\tRecall:  61.6%\tF-Score:  62.9%\n    Key:   ;\tPrec:   0.0%\tRecall:   0.0%\tF-Score:   N/A\n    Key:   ?\tPrec:   0.0%\tRecall:   0.0%\tF-Score:   N/A\n    Key:   !\tPrec:   0.0%\tRecall:   0.0%\tF-Score:   N/A\n    \n    400it [06:07,  1.33s/it]\n    ```\n    \n6. **Hidden Layer initialization**: In the past I've found it was easier for the neural network to generate good results when both the training and the generation starts with a zero initial state. In this case because we are computing time limited, I zero the hidden layer at the begining of each file.\n\n7. **Mini-batches and Padding**: During training, I first sort the entire training set by the length of each file (there are 45k of them) and arrange them in batches, so that files inside each batch are roughly similar size, and only minimal padding is needed. Sometimes the file becomes too long. In that case I use `data.fuzzy_chunk_length()` to calculate a good chunk length with heuristics. The result is mostly no padding during most of the trainings.\n    \n    Going from having no mini-batch to having a minibatch of 128, the time per batch hasn't changed much. The accuracy report above shows the training result after 24 epochs.\n\n\n## Data and Cross-Validation\n\nThe entire dataset is composed of around 50k blog posts from engadget. I randomly selected 49k of these as my training set, 50 as my validation set, and around 0.5k as my test set. The training is a bit slow on an Intel i7 desktop, averaging 1.5s/file depending on the length of the file. As a result, it takes about a day to go through the entire training set.\n\n## Todo:\nAll done.\n\n## Done:\n- [x] execute demo test after training\n- [x] add final performance metric\n- [x] implement minibatch\n- [x] a generative demo\n- [x] add validation (once an hour or so)\n- [x] add accuracy metric, use precision/recall.\n- [x] change to bi-directional GRU\n- [x] get data\n- [x] Add temperature to generator\n- [x] add self-feeding generator\n- [x] get training to work\n- [x] use optim and Adam\n\n## References\n1: https://www.aclweb.org/anthology/D/D16/D16-1111.pdf  \n2: https://phon.ioc.ee/dokuwiki/lib/exe/fetch.php?media=people:tanel:interspeech2015-paper-punct.pdf  \n3: https://en.wikipedia.org/wiki/precision_and_recall  \n4: https://en.wikipedia.org/wiki/F1_score  "
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:06:47.894982Z",
          "start_time": "2017-03-29T05:06:47.887736-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "%load_ext autoreload\n%autoreload 2",
      "execution_count": 1,
      "outputs": []
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:06:48.054793Z",
          "start_time": "2017-03-29T05:06:47.895765-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "import matplotlib.pyplot as plt\n%matplotlib inline\n%config InlineBackend.figure_format = 'retina'",
      "execution_count": 2,
      "outputs": []
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "## Data Preparation\n\nHere we build up generators for batches and take a quick look at the data. I collect all of the different characters that occurs in the data, and plot then ordered by the appearance frequency. It looks like some Japanese and Korean article got mixed in."
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:06:48.105458Z",
          "start_time": "2017-03-29T05:06:48.055909-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": true
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "import os\n_, _, files = list(os.walk(\"./engadget_data\"))[0]\n\"number of files: \", len(files)",
      "execution_count": 3,
      "outputs": [
        {
          "data": {
            "text/plain": "('number of files: ', 48717)"
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:06:52.203346Z",
          "start_time": "2017-03-29T05:06:48.106548-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": true
        },
        "scrolled": true,
        "trusted": false
      },
      "cell_type": "code",
      "source": "import data\n\nfig = plt.figure(figsize=(12, 2))\nplt.subplot(131)\nplt.plot([len(src) for fn, src in data.source_gen()], linewidth=3, alpha=.7)\nplt.title(\"Distribution of Document Length\")\nplt.xlabel('Document Index')\nplt.ylabel('Document Length')\nplt.subplot(132)\nplt.plot([len(src) for fn, src in data.validation_gen()], linewidth=3, alpha=.7)\nplt.title(\"Validation Set\")\nplt.xlabel('Document Index')\nplt.ylabel('Document Length')\nplt.subplot(133)\nplt.plot([len(src) for fn, src in data.test_gen()], linewidth=3, alpha=.7)\nplt.title(\"Test Set\")\nplt.xlabel('Document Index')\nplt.ylabel('Document Length')\n\nplt.tight_layout()\n\nplt.show()",
      "execution_count": 4,
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABrAAAAEOCAYAAADBtVesAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAIABJREFUeJzs3XmYHVWd//H3JwlZSVizIKggiqLiAioIAgGRATdcQHEE\nxfXnhoDMqKMIAddRBsVtHJmBKDqAgwKKu2JExcEB3EVBJCJIEiBk35Pv749ziq7cVPVaffve7s/r\neeq5favOqTpV93bXt+tsigjMzMzMzMzMzMzMzMzMOsW4kS6AmZmZmZmZmZmZmZmZWZkrsMzMzMzM\nzMzMzMzMzKyjuALLzMzMzMzMzMzMzMzMOoorsMzMzMzMzMzMzMzMzKyjuALLzMzMzMzMzMzMzMzM\nOoorsMzMzMzMzMzMzMzMzKyjuALLzMzMzMzMzMzMzMzMOoorsMzMzMzMzMzMzMzMzKyjuALLzMzM\nzMzMzMzMzMzMOoorsMzMzMzMzMzMzMzMzKyjuALLzMzMzMzMzMzMzMzMOoorsMzMzMzMzMzMzMzM\nzKyjuALLzMzMzMzMzMzMzMzMOoorsKwtJM2VFJIWjnRZWklakMt2Ssv6ji0zdH75hoOkV0j6uaSV\n+dxD0tyRLpdZN5M0P/8uzRvpspiZdStJX8p/S89qWT+hFLPs0dR+20HS3fnYz2r3sc3MzMzMzMAV\nWNaH0oPN8rJR0gOS/izpaknvkbRXG8u0o6R5o/lhq6RT8jk+ZaTL0ikkvRL4b+AgYBKwOC8b+pF3\nbsX3eIukFfnhzE8kfULS0ZI0vGdiQyHpRfl3Y+4Q9nFK8T1osGgdqfhbKWnHkS6LmdlwKcWrfxhA\nnrfmPOvG2t9ISfvne8OrRrosTcsVhq+V9F1JiyRtkLRU0h8kfVPSuyQ9veFjviRfz8Oa3K+ZmVkn\nq3jG0t9lwQiV9035fv24Ie5nXG5cfa2keyStl7RM0h9z/HFW041/JB2Ty35Mk/s16xYTRroA1jU2\nAkvzzwJmADsDewPHAR+QdCXwloi4vyL/GuBPwD0NlGVH4Jz887wG9ncXqWzLG9hXU04BDgcWAr+q\nSdPkNe0Gp+fXjwPvjIhNg9zP/cDm/PNk4GHA7sCzgNOAOyS9MSKuG0phbdi8CHh1/nnBCJajWxR/\nK+cDy0awHGZmw+kLpHvDvpKeFhE39SNPUXlzTUQM59/HIMVrkOLpTrA/6f7wQ+CLvaT7M7CKFHN2\nPEmzgG+Tzq+wltRo83HAvsBzgQeAXRs89EuAVwKbgOsb3K+ZmVknW1yzfmdgO2Ad1c/Zllasa4c3\nAU8mPWP742B2IGkH4OtAudHKelK8tw/wWODoIvmgS7qtY0jPqy4EvtPgfs26giuwrL9uiIi55RW5\ntepBpMqWE/JysKSDIuLuctqI+AXpH8eOExFd2fq0k6/pMHlCfr14CJVXAE+PiIXFG0kTgCcBzwfe\nRqqU/YGk10XEJUM4jpmZmbXHAuCvwCNJFVO9VmBJeizwjPz2C8NZsIjYTJfGa62xfxe4jFR5tQI4\nD/hSRCwGkDQDOJBU2XR07R7MzMysXyJiTtX63MPqcOCKiDilnWVqg8+TKq/WAR8CLgHuiYiQNA14\nOqmRf1c+ZzTrVB5C0AYtIpZFxHci4kTgeaQ/4LsDV45syWyUmpJfVzW504jYFBG3RMR5wBOBG0kt\nZf5D0pObPJaZmZk1LyICuDS/PTE3TulN8VBhEfDdYSuYtY2kJwJH5revjoh/KyqvACJiRUR8PyLe\nTE+jKDMzM7N+kbQbqeE+wKkR8f6IuDvHoUTE6ohYEBFnkJ6NmllDXIFljYiI7wD/lN8eKOkF5e2l\nOYgWtubN48eeIulHeW6tjZLuk/R7SReXx3jNLTnuLL1vHUt3XjltXndKnjfrX/OYtGskLatK19s5\nSnpBLuODklZJ+rmkf6xJu2dfc+xUXZNibh5SaxWAS1rOb2Fv+SuOcYSkr5XmAFgk6SpJR/aSpzjW\nnpIeIekipXmi1ku6U9L5uRXroEiakcfu/XW+jqsk/UbSubk7djlt1XW8s1TG+YMtR5WIWEJqmbuc\n1OX97JpzGCfpdZJ+rDSvwrp8bT4v6dG9HUPJy5XmYViUr+s9kq6XdIakXUppB/U9Km1bmLfNlbSb\npM9J+puktZJuzccbV0p/gtJ8YMuU5gf7Zn4g1Nv5zJT0YUm/zZ/lakm/k/RBSTvX5CmXa2dJF+Tr\nV1yLi5SCw23Ok57hA89p+d1oy3xW+bM/WdL3lf5ObZD0d0lXSDqwJs+88vdV0qsl3ShpZb7OP5L0\nnD6O+/h8jCX58/tj/p2Z3Lr/nH5+L783tb87ksZLOj3/fq7J3+9rJT1toNfKzKzNiqHwZgLH1iWS\nJOCk/PbLuYdUsW28pOfl+/ktkhbne9PfleKpuQMtlNKcTMXf3j1q0jwz/61dmu+lv5R0ai5rb/s+\nQCm+/Wm+v69XiqV/pDQP1LiW9BPyveGivOrZrfdSleZsUIr/tlrXsr/dJH1c0p/yvWl5vr+dIWlS\nTZ4v5X2ela/3O5TiwOKe83VJ+1fl7cN+pZ+/2VvCiFhXt03S9Fy2m/L5rJV0m6QLJe3ekvaofD1f\nmVe9v+VaDmXEADMzszFD0vGSvpX/3y2eXX1N0uG95DlQ0mWS7sp5Vki6I8dUb5U0Mac7Pd+viwbK\nV7Xcr+um7Wj1RHqGBby2t4R9xBpTJJ2p9EzxQaXnSX9Rel7zqJa0T8llPy2vOq0idhtTc7na2OQh\nBK1JFwHvA2YD/wh8o5/5Ls3pC8tJc2ztCjw+L8UYr0tJcxgV49a3jrlb1TtnJnAz8CjS2LQb+lmu\nh0g6nTT3UuTyTSENn3iQpIMj4m0D3WeNtaRzKsYMXpHXFe4bQJk/ALw3vy3KPYs0h9CLJH0kIv6l\nl108Gbg4l2UlqcJ7T+BM4PB83gOay0GpcucHpCF+oGdOhf3ycoqkoyLi9rx+Mz2f8ez8Wp7DqvF5\nyyLi75IuIc259UJJ0yJidekcpgJX0TP8zMZ8HnsCbwBOlnRiRFzTum+lCrorgaOKw5HmJdqZNBfX\nocCDpPmKmrQXaVidOaTv1Hak4YwuIP1enCrpI8C7SNd2DTCdNE/EwZKeUfpMyufzLOCaXH5Iv1tb\nSC2bn0C6Fs+JiD+15s32IJ3rI/Mxg3QdXg8cJWn/iHiwtO/FwA6kuctW03BvvL5Img58ja0/v5XA\nbsDLgOMlnRYRn+5lH/8JvI50nVeT/tbNBQ6T9LKI+GpFnqNIf08n51UrSJ/p2aTv4YKKQy0nXa+q\n35tie6sJpId+/0D6Xq8HdiL1sH22pCMj4ud152ZmNpIi4nZJNwAHk3pY1cWhc4FH5J9bhw/cj60f\nSKwg/T3cDXgx8GJJ74qIjzZVbkmvJFW+FZVNy0gPSD4JHEKaV6nOD0n3RUj3lDWke/LcvBwn6SWl\nSrog3RumkO4/G0hxR1m/4mRJBwHfIt0nIN0PJ5GGZnwGcJKkYyKiLnbdjhTfH5WPuSHv6wWkGGBu\nHi57MB5GGlJyQCQ9gTSH1sPzqk2ke+Fj8nKSpOdFxP/m7evZOjZZRfocKOU3MzOzGpKmkJ5VHFda\nvYL0f2wRe703Ij7Uku/lwH/TEz8Vz80elZfnAV8l9bZfTbpf70L6n3cZ6R5euH8QRd8973tAJO1J\nin8em1dtJpV9L+D/Aa+U9NKI+F7evjGXfTowlRTrrWzZ7ZaBlsOs60SEFy+1C+nhcgAL+pn+v3P6\nu1vWz83rF7asPyyv30SqMJie14v0sODVwPktefbMeaKPsiyg5wHzXaRJD8flbY+uSHdKTZlXk/6p\n/gIwO2/bCTi/KAfwjwMtY9016a1MA8h/YqlsnwJ2zet3IT0QKbadVJG32PYg6cHIE/P6ScBrSUNF\nBvCWAX6XJgK/znnvAp6TP2cBzyY9aAjgd8CkXsq15yC+x3MHkp9UKVCkP6pl2+fy+nWkAGNSXr8P\n8KPSd2afiv1em7evAd4O7Fj6vu8LnAsc1+D3aCE9lWQ3AE/K66cCZ+VtW4D35O/4acC0nOaJpIlN\nA/hKxb4fmb8jAXwWeDQpeByX8343b/s9ML6mXA8CvwSemddPAF5Y2u9HK447P2+bN9DvQWkfp/R1\nXWvyXZXz3Zy/I5NLfw/em6/hZuCQlnzzSue7ljR57NS8bS/gx3n734EJLXl3JQXUQRresvh93I5U\n8b+ydL3mD+b3pnRNHyRNbP8yYGLe9iTgt3n7LwZ7zb148eKlHQvwRnru0TvWpLkkp7mlYtu+wH+S\nYpTppfWzSY0GNuW/8wdU5P1S3u9ZLesnlP4W79GybR964qpvA3vl9dOAf87HWla135zucuDlwJzS\nummkCrzFOd8ZFflen7f9oI/reXdO96yW9buQHtoEaSL0A/L68fkeUpT5271cpwdJDbOOz/c0kRpP\n/T5vv2GAn/1jStf5m+TYdwD5dyLFp0F6kLYfOX4hzY96GT336hn9+ey9ePHixYuXsbjQ8zxrfj/S\nFnHZ70iNWIr/k3cA3kFPQ9djS3kmAEvy+suL+Clv25E0pPB8YJeWY/0q53nRIM9rZo4FA/gJ8LAB\n5p9SinO+ATyN/P8/qfHM50sx0m4teT+Rt31ipD9fL15GYhnxAnjp7IWBV2D9S+mfx+1K6+dSXYH1\nzrp/cHs5xp7FMfpIV9w0N5Af+vaR7pSW9XNL5/I9QL1cn9vL2/tTxrpr0luZ+pOf9ADg9rztspq8\nRUXjneRKvdK24pzrKpI+lbdfN8Dv0sm9fR6kHjsbcprXVmzv80F8P65Vv/KTKk+L9G9o+Vw35/X/\nryLfVODPefsXW7Y9l54Ko2Oa+q738T1amLctpeJBHqmCsjjPsyu2H0rPg8CJLduKhzUfrilXucLy\n+JpyLaIlqMzbz8zb/1KxbT4jUIFFaiEepEq9HWrSvDunubZl/bzSdX5lRb6HkVqABXBYy7Zz8/rF\nNZ/hy0r7nj+Y35vSNQ1aHlLm7QeUtj9isNfdixcvXoZ7IT20WJv/Xr2xYvtUUqveAE4bxP6Lv8kX\nVWwbTAXWF+hp7FEVc5XvHwOqHAGOyPlur9g21Aqs4jo8AMyqyPfcUrlb72vFddoCHFSR98BS3t0H\neM5fLuVdD3wfeD+pcczMPvJ+JOe7tGa78v4COL0/n70XL168ePEyFhf6WYFV+j/zrrr7NKnxZwA/\nLa3bJ6/bRG6A289yDakCK+/jwlKssZHUGPXDpKkodusj7z/R09BmXE2ay3OaD7SsdwWWlzG9eA4s\na9rS0s+V89+0WJFfZ7WO09+gb0fE74a4jw9HRFSs/2B+fTQ94+mOtKeQygPwgZo05+bXPUnDvFS5\nICLWV6y/Or/2OjdShePz6zVVn0dE/J40vB6kh/Ijqe57/GJSD6NFpBbaW4mINUAxrNBLJI0vbS4m\njP9upDnj2ulzEbGsYv0P8usG0nCCrX5GqryaRM93qhhG8QTSw6eqfETEBno+z7r5nT4fEQ9UrC++\nY3tJmlaTt91enV8vioi6oSu/nF+PaPnsC3eRKo+3EhF/B4phklp/r16SXz9f9RlGxFeAv/RW8AH4\nSUT8tOIYN5MeYlaVz8ysY+S/k8UQvq+qSPJi0hAsm6j4e9wPxbCEhwwi71Zy3Pvi/LYu5rqAdB8e\njAWkXrqPljRrkPuoU8R0n480f+hWIuJbwP/lt3Ux3YLoGYqvnPdGeobkecIAy/Va0oOljaSGNEeR\nepxfAyzJ83NVzl9Lz33+/KqN+f+A4jvT67yVZmZm1i+n5NcvRP2Qw0Xj64MkbZ/XFc8Rx5OmyWin\nM0nP2daSGikdRmrI+lXg75J+Jen1Nc83i1jjgoioG/aveKbgWMOsxBVY1rReJ5uu8EPSw/P9gQWS\nTpL0sIbLNNQ5WzaSHuRvI9K8QPfmt4OZcHo4FOW4L1cKbSPSnET3tKRv9X8164t8O9Vs76tcP+ol\nzXV9lKld6r7HRbl+EqVJ31sU5zCNnnGNIc2ZBmm+iHb7bc364qHTwojYZj6pHFQV40GXP+8DSA+G\nBPw2T7C6zUJqYQQ9c0m06us7Bqk1fSc4OL+e1cv5FuczlTS8UqubairCoeL3StIk0hyAANtULJX0\ntm0g6j4PGPzvvZlZu83Pr4e0ToRNT6XWt+selEiaKukdkn6cJxLfWEySTc/fySZi1ceQKtMgtd7d\nRkSsIA21W0nJyyRdI+lveRLwoqxbSvtvLLbOc1Xsm98OJaZr/J4TEesj4nRS3PFmUivmomc8pEZb\nX84Tvj/0f7CkvUjzhAJ8t5f7/L/lNHVxjZmZmfVf8T/2qb3ce2/LacaT45mIWERPfHS9pHdKeqKk\ngT6PHLCI2BQR7yPN5/064FLSKC1FhdSTgYuAb0qaWOSTNIOehjmX9XK+l+Q0jjXMSiaMdAFs1Cn/\no7m0NlUWacLtNwOfJg1XdiiApIWkiQ0/HxG1/7j3U11Ljv66P/cmqXMPaci5mUM8TlOKctzTa6rU\no2J36svdOjFkoWgJPNC/H/0pV9HLYxdJ6uVh/3Cr+x4P5BzK6SHNnwGpF0673VuzfnMf28tptiut\n2y2/ip7z6s3UmvWV37GIWFeKPberSjMCinPub4Va1TnX/U5Bz+9V+Xx3oqehSW+f0d/7Waa+DLR8\nZmad6Pukv5m7kYYvPhdA0m6kOTchDd23DUm7k3ouPbq0ejVpLoItpIcnu5IaqQxVOUbo7e94Zcwh\naTtST+cXllavJzU8Ke7dM0n3kSZ7M+9CT0Of/sRDA40zYYj3nIhYTJqz9HMAkuaQrlPxwOlEUuOP\nz+Qsu5WyDyWuMTMzs/4r7r875KUv5fvvyaSe8XsB/5qX5ZIWkHoxfbWXXk5DFhFLgYvzgqRdgOeR\nYo1HA8eQplgpRj+aQ0/81J9nh441zErcA8ua9qT8endEbOxPhoi4mHTTOZ00xMcDpKHt3gTcLOk9\nQyxTXU+Z0W7ySBegRqeWq+xJpZ+rhmfrhnMYTsW9Y3lEqB/L3JEsbEOKc35xP8954UgW1sxsrMo9\npL+U355c2nQSqQJqKT1DAbb6JOmhw59Jw/vtFBHbR8SsiJgDPCunG/YWvv3wJlKlzGrgVODhETE5\nImZGxJxc3qKn9XCVtyvioYhYFBGfJ/UgLxq2vbaUpPw/8fR+3OPLFZxmZmY2OMX99zX9/B/7V0XG\nPNrQ40hDFV9Mmgd+B+A44CukEZ4mtetEIuKBiPgiqef5nXl1XayxVz/OtVNGojHrCK7Assbk7rFH\n5rc/GUjeiFgcERdGxItIrRGeAVxF+of7/ZKe1OsOhteu5a6/FYphWco9vTYVP0iq++e+Py1MBqMo\nR19djvdoST/ciuM8opc0RZkeGMHeVwDH5teNQHl+hoGcQzk9wOL8+sgBlGMkv0e9Kc5lhqSROP5I\nKM65t8++aUWLf9i6dXir3raZmY1FRQ+rvSUVw9MUlVmXV/Wsz/fZ5+e3r4iIqyvmHuxP75z+KscI\nvQ3xV7fthPw6LyI+HRHlHuBFD63+zEc7UA/QMyRff+KhdsWZfcrzdRWVl/uUNi0u/dzO+7yZmdlY\nNqT/sSNiQ0T8T0S8LiL2IT0Dex/pOcqh9Exp0DYRsRL4n/z2EaVKtPKcoY41zAbIFVjWpDfQM4Hi\nl3tL2JtI/o/0j/ndpO/ps0pJHuoG3I4xbknDlzyzaoOkR9PzYOGW0qbyA49yhUbZ03s5ZnGOgzm/\nohzTJD2jKoGkfUjDB5bTD7fiOEf0kqaoAG1XmbaR52A7Jb+9JiJWlzYX5TpQUl2X7uIcVgN/Kq0v\nKsKeO4DiDPV7NFxuIgWFInWNb6eh/G4MRTGX3rG9pmpQRKwH/pDfPquXpIf2tpv82gm9BczM2iK3\nyr05v32VpKcC++X3lcMHkmLYosHSr2rSHNVMCYHUUrgYRu+wqgSSplM/h1QRF9QNtX0oPefTatD3\n0ohYC9ya33Z0TFejiOvKlZh/pmfOz8Hc50cqNjEzM+tmjf6PHRF3R8QHgP/Mqw5vSdKu+3URa2wh\nN0rOQw4Wz4cca5gNkCuwrBGS/gH4WH7784j4Zj/z1fZsykPAFMMQlrv+rij93K5utf9SU1n2L/n1\n9pbuzKuAhfntca2Z8vi4r+/leMU5Dub8fkX6RxygbvjFefl1IfCLQRxjMK7Mr8fmB0lbkfQE4Pj8\n9ittKlNrGWYBXyP1atoAnNeS5GukwGEX4I0V+acC/1ykzd/hwhfz69GS+lXp08D3aFjkVkVfzW/P\nyw/YKkmaIGn7Bg8/lN+NoZifX/+hr89P0oAmne/DVfn1DVW93SS9FHhUL/lH6nqZmY20oqLqZaRG\nVgB/jIi6uKccXz6xdaOkPYC3NlW4PC/D1/LbM2pi4tOpH6ZveX7dr3WDpAnA+3s5/FDvDUVM91pJ\n2/RKk/RcehrYtCWmk7S3pL36SDONnniqHLcHPd+Xd+b50ur2oYr7se+1ZmZmAzc/vx4k6eW9JSz/\nj93HCEkAa/Nr6xCCQ7pfS9pd0r59pNmOnudav2t5JjQ/v741N4bvbT+tzxQca9iY5gosGzRJO0j6\nB0mXAd8CpgB/o+ePdX98SNKVkl4k6aFhTiTNlvRJ0txYQZqQG4A8nEsx2fVrhnoe/bCGNOn3f+UK\nDiTtKOlf6RnTdl5FvuIf9rMkvTA/TEDSQcAPqG8VC/D7/PqSgQ7Rlv8JPyu/PU7Sp3JFB5J2ydf1\nFUXZhnNiyxZXAL/JP18t6aiiUlDSs0nfoe1I5z7oHnwDJWm8pKdIeh/wW+BAUiXVGyLit+W0EfFX\n4PP57UckvbHoEp57tX2TNHfGGuADLYf6dl4EfFXSqZJ2zHkl6fGS/k3Si1ryDeV7NJzeTZpHZB/g\nBknH5GCtOJ/HSfpnUiujpzV43OJ345jeHjD1l6Rd+1imAkTEd0gPGgVcJemfJc1s2c/xkr4JXDDU\ncpV8ijSU4Gzg27mit6gYPBG4hK176rUqrterJI1vsFxmZp3uMlJDqJ2A/5fX1fW+KuLL/8tv50t6\nMoCkcZKeAyygp1drUz4ErCdVmF0lac98zKmSziTFl8tr8hax8TmSXlD8jZf0eFI8sj8pHqlS3Bv2\nkzSYe/QnScP+TAO+I2n/fOzxkk4A/jun+05EXD+I/Q/GfsBtkr4q6QRJc4oNkqZJOo40xHkxbM+F\nLfk/RGo0NAv4eb6nTynt41GS3kqKZV/Qkre4ns8tH9fMzMzqRcT/kuavArhU0tkt9++dJB0n6WvA\nf5WyHizpFklvkfSoUvpJkk6iJ+77bsshi/v1CYNsZLsX8DtJ10p6ZW7cVBx7iqSjSfFi0bioNdb4\nJOmZ03Tgp5JOLpdD0sMlvV7SL4BX15T92ZI8BKGNPRHhxUvtQmohEKTeKIvyspj0D3GUli2kCopd\na/YzN6db2LL+Ey37WU5qWVBe956K/Z1b2l70UlkInF5KsyBvP6WPc6xMVy4zqQVscZ5Lgc2l43+6\nZr87AXeU0q3LZQ3gr6TJxLe5Jjnv40gPNIL08OWeXI6f9nVNS9s/UDr25opyf7gmX7F9z5rtexZp\nBvF9enQ+j+IYq/NSvP8rsM9gytXHceeW8t9X+i4vy59p+ft2OzC3l31NBb5XSr+BVMFQ/pyPq8m7\nY+n7VnwuD5BaCBXrWr+HQ/keFde68nxIQyUGsKCX863dB6l19T0t1+L+0ne3WA4fSLl6+7yBXemZ\nf2MzcG/eX+XvQR/n3Z9lXinfNFKPqPLfvQdJQ0CV81zScrx5ef38Xso0v/V4pW3/kD/3Yv/LSu9v\nAD6cf/6PiryvKeVbm78zC4Hz+3PsUpoFVd9PL168eOnkpeVv9mZgjz7SH8zW9+RVpff3Ay/KP2+q\nyPulvO2slvUTSvvb5vjAK9k6PnuQFPsFKbau2+8uwF9K+TaQ4ugidjyZNBR3AM+qOO7PSnnvpyeW\nflopTW/5n8nW8c+KlnvVLcDM/l6nljQ/zWlOGsBn/bzSsctx5rKWdRuBd9bs4zHAH0tpN+Vrs65l\nH69syTerdC3KscmfR/p3wIsXL168eGn3UvrfcX4/0k4CLm25zy5j2+eCV5XyzG3Ztpb0jKD8bGcB\nMLnlWE8rxVwbcpyzEPhWP8/rwIpYY21LPFQ8J6h73rZHjpHK8ekDbPuM9bSWfNNIDfmL/S+iJ3ab\nMdKfuRcvw724B5b113akHgCzSf8wryP90/x14L3AoyLi5RFxf/0uKn0ceDtwDXAbqXfDJFJPriuA\nwyLiQxX5zgPeRWoFKeCReRmW7rQR8QnghcCPST0X15HmNDopIt5Wk+dB0oOQz5NuNONIN6ZPkVrF\n3l2VL+f9I/Ac4DukhxFzSOdXNw9S1T7OIvUcu4b0z/f2+fhfB46KiH/pJfuwiIg/A08mfX6/K236\nHWmomydFxG3DXIxd6fkuTyA9ZPgZqTXM0aQKtAV1mSNiDWnM4teTWvKuIVVq/ZU01vJ+EXFNTd5l\npDkhXk3qPbWU1PrmAdJ363TS51POM+jv0XCLNFfd40i/izeQHvTtSLomN5Gu6eER8eMGj3k/ac6N\nr5EqI2fS8/s/rCJidUS8GHh+Pv7fSZ/9BNKwnV8hVRid2vBxv0sKtq8kffaTSIHqeaTvU9FCfJue\nWBFxCWnorF+QHsQ9nHStdm2yjGZmHarc4+q6iOj1nhkRN5DuudeQHkZMID0g+HfgKWwduzQiIr5M\nmufwW6S/4xNJrWzfTk+P+ap8DwAHAZ8jNSaB9BDla6T4+dI+Dn1cznsnKRYp7qV1Qxa2Hv/nwBNI\nrYtvz+XeSLr/nwk8MyLu68++mhBp+PJ9SUM5X0Nq/CNS/LuMNCfax4EnR8RHa/ZxO+lzfhvpwdcy\neoaV/jXwH6QY8LKWfEtIsclVpJi7iE32bO4MzczMRp+IWB8RJ5Oef11OehY4hRRX3EmKa95Izzzl\nkHrMv4LUK+vXpAalM0ix24Kc/tkRsa7lWDeR5iT/IenZxW6k+/XD6IeIuJE0fP9ppCkVbiNVQE0n\nVbj9GvgMqTFQ5fO2HIs+A3gdqWH0A7nsm0nx38XAi4HPtuRbTaq4u5wUm+5CT+zmZ/s26ikimttZ\nGrrisaQysbtJAAAgAElEQVQeA5VDFUX7hpEwMzMb9ST9hPTw8zURMX+EizOmOO4xMzOzscAxj5mZ\nmY2UCU3tKM8fcwaplVpvPAeHmZlZAyQ9k1R5tYXUkszaxHGPmZmZjQWOeczMzGwkNVKBJemdpDmJ\nlpPGLv0baagiMzMzGwJJbyQN+XcFaa6vzXmy15eQhkMC+EpE/G2kyjjWOO4xMzOzscAxj5mZmY20\nRoYQlHQ7abz0/ds51rmZmdloJ+kDpLkGIY2NvZw011gx1vWvgOcMYg5CGyTHPWZmZjYWOOYxMzOz\nkdbUEIIPBy5yQGNmZta4y0kT2R4O7AHsTJok9g/AlcDnImLtyBVvTHLcY2ZmZmOBYx4zMzMbUU1V\nYC1ucF9mZmaWRcTvgDNHuhy2Fcc9ZmZmNhY45jEzM7MRNa7vJP3yFeA5kiY1tD8zMzOzTuW4x8zM\nzMYCxzxmZmY2opqaA2sq8F1gGfD2iLhzyDs1MzMz60COe8zMzGwscMxjZmZmI21QFViS/lKxejvg\nYfnn5aQAp1VExN4DPqCZmZnZCHHcY2ZmZmOBYx4zMzPrNIMdy3gc0FrztQm4q/ReFfmq1pmZmZl1\nMsc9ZmZmNhY45jEzM7OO0sgQgmZmZmZmZmZmZmZmZmZNGWwPLOtwku4EZgALR7goZmZmhT2BFRGx\n10gXxEYXxz1mZtZh9sQxjw0DxzxmZtZh9mSYY55GKrAkXQfMj4gv9pLmJOC1EXFkE8e0Ps2YMmXK\nzvvuu+/OI10QMzMzgFtvvZW1a9eOdDGGzHFPR3LcY2ZmHcMxjw0jxzxmZtYx2hHzNNUDay6woI80\njwQOb+h41reF++67784333zzSJfDzMwMgAMOOIBbbrll4UiXowFzcdzTaRz3mJlZx3DMY8PIMY+Z\nmXWMdsQ844Zz5y2mkCb/NDMzMxvtHPeYmZnZWOCYx8zMzIZNk3NgRdVKSQIeATwX+FuDxzMzM7Nh\ncM+ytdy9dM026x+24xQevvPUEShRR3LcY2ZmNgpc/ct7WLNhM7OmT2LWjEnMmj6ZXbefyITx7Wzv\n29Ec85iZmY0Cv//7cn5+xwPMmjGZOXmZNWMSk7cbP9JF69WgK7AkbWHrQGaepHm9ZQE+NNjjmZmZ\nWXv88q4HueqWe7ZZ/9z9dhuzFViOe8zMzEanG+98gCUr1m+1ToJzXvCEMRn3OOYxMzMbne64bzU/\nv+OBbdYfu99uHH/AHiNQov4ZSg+s6+kJag4D7gIWVqTbDDwA/BD4zyEcz8zMzGykOO4xMzMbZTZt\n3sL9qzZssz4Cdp42cQRK1BE6IuaRdBJwaX77hojY5hiSDgbOAg4iDWV4O3Ax8KmI2Fyz31cDbwUe\nn8/hl8D5EXFtTfopwLuBE0nzfa0gzQt2TkTcWpNnD+A84BhgF+Be4Grg3Ih4sK9zNzMzGw5LVqyr\nXL/jlO3aXJKBGXQFVkTMLX7OLXQuiYjzmiiUmZmZWSdx3GNmZjb6LF29gS1bth0hb9qkCUyb1OSM\nC92jE2IeSQ8HPg2sAravSXMc8FVgHXAFsBR4AfBx4BDghIo85wNnAncDFwETSRVT35B0akR8uiX9\nJOD7eX83ARcCD8/7fp6kIyPixpY8ewM3ALOAa4A/As8ATgOOkXRIRGzb/N3MzGyYLVpeXYE1Z4fJ\nbS7JwDQVke0FLGtoX2ZmZmadzHGPmZnZKLBk5frK9TOnT2pzSTpW22OePLfWJaTeXV8D/qkizQxS\nBdRmYG5E3JTXvw+4Djhe0okRcXkpz8Gkyqs7gKcXPaEkfQy4GThf0rURsbB0qHeQKq+uBF4eEVty\nnitIPaoulrRfsT77LKny6u0R8anS8S8AzgA+CLxpkJfHzMxsUCKCRTU9sGbN6Oy4p5FZSSPirxGx\nvIl9mZmZmXUyxz1mZmajw301FVizXIEFjFjM83bgSOA1wOqaNMcDM4HLi8orgIhYRxpSEODNLXmK\nSqMPlofxyxVWnwEm5WMCD1WkFXneWa6kiohrgJ+QhiE8vJRnb+Bo0pCLn2k5/jn5fE6WNK3mvMzM\nzIbFyvWbWLth29F1x48Tu07r7LinkR5Yks7uR7ItpLGCbwV+HBHbDjRtZmZm1uEc95iZmY0OS1ZW\nt0SePaOzh9Jpl3bHPJL2BT4CXBgR10s6siZpsf47FduuB9YAB0uaFBHr+5Hn28D7cppz8rq9gUcA\nt0XEnTV5Ds15fpTXHZFfv9fSK4uIWCnpZ6QKroNIc4eZmZm1Rd38V7NmTGLcOLW5NAPT1BCC8+iZ\n5BOgfNat6wN4II8vfEVDxzczMzNrl3k47jEzM+t6S1Z4CME+zKNNMY+kCcClwF3Ae/pI/tj8elvr\nhojYJOlO4AnAo4Bbc4+n3YFVEXFvxf5uz6/79OcYQ8xzdM7TawWWpJtrNj2ut3xmZmZVFi2vjnlm\nT+/8RjuNDCFIamVyDbAR+C/gFODY/HpxXn81aaLLjwCTgS9JOrSh45uZmZm1i+MeMzOzUWBxbQ8s\nV2Bl7Yx5zgaeCpwSEWv7SLtDfq0b3rBYv+Mg07czj5mZ2bCrm/9q9g6dX4HVVA+sRwLPIU2E+duW\nbV+U9GngZ8BVEfFeSZeTJsn8J9K4wWZmZmbdwnGPmZlZl4sI7l9ZPdrdzO07/2FOm7Ql5pF0IKnX\n1b9FxM+bKXp3i4gDqtbnnln7t7k4ZmbW5RbXVGDN6YJhk5vqgXUG8JWKgAaAiPg18D/AO/L73wLf\nBJ7Z0PHNzMxsmKmzh0VuJ8c9ZmZmXe7BNRvZuHnLNusnbTeOGVOaauvb9YY95slDB36RNOze+/qZ\nrejJtEPN9mL9skGmb2ceMzOzYVc3B1Y3zPvZVAXWY4FFfaT5Oz3jAUMa+9fdps3MzKzbOO4xMzPr\ncvetrJ4LYtb0ycitdgrtiHm2J80JtS+wTlIUC3BOTnNRXveJ/P5P+XWfln0VFWJ7AZuAvwBExGrg\nHmB7SbtVlOEx+bU8d1XtMRrOY2ZmNqwigsU18352Qw+sppoVraTvFjYHA6tK76flfGZmZmbdxHGP\nmZlZl1tSM//VzOme/6qkHTHPetL8WlX2J82L9VNS5VAxvOB1wCuBY4DLWvIcBkwFro+I8tO664CT\nc55LWvIcW0pTuAO4C9hH0l4RcWc/8vwovx4taVxEPNTFT9J04BBgDfC/lWdrZmY2DJau3lDZ63zy\nduO7otd5Uz2wvgUcLulDkqaVN0iaJunDpCDiW6VNTwQWNnR8MzMzs3Zx3GNmZtbl6loiz3IFVtmw\nxzwRsTYiXl+1AF/Pyb6Q112R318J3A+cKOlppTJNBj6Q3/57y6E+l1/fK2mnUp49gbeSKtIeqtiK\niCjl+aikcaU8xwGHAn8AflzKcwfwPaDYZ9m5pMq9S3OPMDMzs7ZY1Mvwgd3Q67ypKrZ/AY4A3gW8\nSdJvgMXAbOBJpO7jd5Em5SR32X40PcGAmZmZWbdw3GNmZtbl6oYQdA+srXRkzBMRKyS9gVSRtUDS\n5cBS4IWk4QyvBK5oyXODpAtI83X9RtKVwETg5cDOwKkRsbDlUBcAzweOB26U9EPgEcAJpJ5Ury33\nssreAtwAfFLSs4FbgQNJ1/E24L1DvwJmZmb9t6Sm0c7sGd0R8zRSgRURiyQ9HfgIcCKpBU5hLTAf\neHdELMnp7wV2b+LYZmZmZu3kuMfMzKz7Le7iyczbpZNjnoi4WtLhpAqhlwKTgT+TKqg+mXtQteY5\nU9JvSb2j3ghsAW4BPhYR11akXy/pOcC7gVcAZwArgKuBcyLiDxV57si9ws4jDVf4XOBe4ELg3Ih4\ncMgnb2ZmNgB1PbDm7NAdMU9jgxxGxH3A6yS9idTiZQfSjf2PEbGxqeOYmZmZjTTHPWZmZt0rIrhv\nlYcQ7I+RjHkiYh4wr5ftPyNVEA1kn/NJFW/9Tb8GODsv/c3zN+A1AymXmZnZcFm0vLoCa9b0MVaB\nVcgBzO+a3q+ZmZm1x7btVa2O4x4zM7Pus3L9JtZt2LzN+vHjxE5TJ45AiTqfYx4zM7PutGRld/fA\nGtd3EjMzMzMzMzOz0aFuLoiZ0ycxblznT2ZuZmZm1h+bNm+pnfdzTM2BBSDpMcBpwDOAnYDxFcki\nIvZu6phmZmZmI8Fxj5mZWfeqa4ncLUPptJNjHjMzs+5136r1laPsTJ88gakTGx+cb1g0UkpJzwR+\nAEwBNgGL8+s2SZs4npmZmdlIcdxjZmbW3epaIs/qkpbI7eKYx8zMrLvVzX81u0uGD4TmemB9GJgE\nvAm4OCKqAhozMzOz0cBxj5mZWRerrcCa7gqsFo55zMzMutjimmGT58wYexVYTweujIjPN7Q/MzMz\ns07luMfMzKyLLV7hIQT7yTGPmZlZF6sbNnl2F1VgjWtoPxuAuxral5mZmVknc9xjZmbWxZZ4CMH+\ncsxjZmbWxWqHEByDFVg3AE9taF9mZmZmncxxj5mZWZdas2ETq9ZtOxKeBLtMmzgCJepojnnMzMy6\n2KKaXudzumgOrKYqsN4DHCzp5Ib2Z2ZmZtapHPeYmZl1qbr5r3aZNokJ45t6RDJqOOYxMzPrUus2\nbmb5mo3brJdg5vbd0+u8qTmwjgOuA+ZLej1wM7CsIl1ExPsbOqaZmZkNgxjpAnQ+xz1mZmZdysMH\nDohjHjMzsy61ZEV1zLPztIlMnNA9jXaaqsCaV/r50LxUCcBBjZmZWQeLqK7CGie1uSQda17pZ8c9\nZmZmXaTuYc6s6a7AqjCv9LNjHjMzsy5SO3xgF81/Bc1VYB3R0H7MzMzMOp3jHjMzsy61ZGX1w5yZ\n07vrYU6bOOYxMzPrUnUVWLO7aP4raKgCKyJ+3MR++kPSQuCRNZsXR8ScijwHA2cBBwFTgNuBi4FP\nRcTmmuO8Gngr8HhgM/BL4PyIuLYm/RTg3cCJuXwrgAXAORFxa02ePYDzgGOAXYB7gauBcyPiwZpz\nNDMzG1Y1HbBwB6yknXGPmZmZNctDCPafYx4zM7PutaSuAqvLGu001QOr3ZYDn6hYv6p1haTjgK8C\n64ArgKXAC4CPA4cAJ1TkOR84E7gbuAiYSKqY+oakUyPi0y3pJwHfz/u7CbgQeHje9/MkHRkRN7bk\n2Ru4AZgFXAP8EXgGcBpwjKRDIuKB/lwMMzMzMzMzM+ubhxA0MzOzsWDR8poKrDE6hCAAkp4E/COw\nLzAtIo7K6/ckVc58v6GeRcsiYl4/yjODVAG1GZgbETfl9e8jTUR6vKQTI+LyUp6DSZVXdwBPL8or\n6WOkCUvPl3RtRCwsHeodpMqrK4GXR8SWnOcKUo+qiyXtV6zPPkuqvHp7RHyqdPwLgDOADwJv6v8l\nMTMza0ZQ3QVL7oK1lTbGPWZmZtaADZu2sGzNhsptM12BVcsxj5mZWXeJiF6GEOyumGdcUzuSdB5w\nC/BOUg+n8ljJ44DLgJOaOl4/HQ/MBC4vKq8AImIdaUhBgDe35CkqjT5YDsByhdVngEnAa4r1Sk/z\nijzvLFdSRcQ1wE9IwxAeXsqzN3A0UOyz7BxgNXCypGn9P1UzM7Nm1A0haD06NO4xMzOzXty3qrr3\n1Q5Tt2PShPFtLk13cMxjZmbWfVau38TaDdvOnDR+nNh12hiswJJ0IqlC6PvAU4APl7dHxF9IQ+u9\nsInjAZMknSTpPZJOk3SEpKpo88j8+p2KbdcDa4CD8xCA/cnz7ZY0AHsDjwBui4g7+5mnCPi+19Ir\ni4hYCfwMmEqas8vMzKwjuP9VMgJxj5mZmTWgdi6ILhtKp10c85iZmXWnuphn1oxJjBvXXU93muqB\n9Xbgz8BxEfEboKpP/q3AYxo63hzgUtIwe58gDQd4u6TDW9I9Nr/e1rqDiNgE3EkaRvFRALnH0+7A\nqoi4t+K4t+fXffpzjIbzVJJ0c9UCPK6vvGZmZlXqemB5BMGHtDvueUhuwBN5eX1NmoMlfUvSUklr\nJf1G0uk1jX2KPK+W9AtJqyQtl7RA0vN7ST9F0rmS/iRpnaQlkr4iad9e8uwh6WJJf5e0XtJCSZ+Q\ntNPAroKZmdngLFlZ3QNr5vbd1RK5jUYs5jEzM7PBW7S8OuaZPb37Gu00VYG1H/DdiKgeTDr5OzC7\ngWNdAjybVIk1LR/7P4A9gW9LenIp7Q75dXnNvor1Ow4yfTvzmJmZtYVHEOxTO+Oeh0h6OPBpYFUv\naY4j9TI/DLgqp58IfBy4vCbP+cB8YDfS3KFfIp3jNyS9rSL9JFJL7LOBFcCFwA+AFwM3STqwIs/e\npLlEXwP8IpfnL8BpwM8l7dLX+ZuZmQ1VXQXWrBmuwKoxIjGPmZmZDU39/FfdV4E1oaH9CNjSR5rZ\nQPWVG4CIOLdl1e+AN0laBZwJzCM9QBkTIuKAqvW5F9b+bS6OmZmNYvIggoW2xT0PHTDNuXkJ8ADw\nNeCfKtLMIFVAbQbmFvN/Snofqbf68ZJOjIjLS3kOJsVPdwBPL+b/lPQxUoXT+ZKuzXOBFt4BHAJc\nCby8GA5Z0hXA1cDFkvZrGSb5s8As4O0R8anS8S8AziD1qn8TZmZmw+g+DyE4UG2PeczMzGzoFtfE\nPHO6MOZpqgfW7cDBdRsljQOeBfy+oeNV+Vx+Pay0rujJtAPVivXLBpm+nXnMzMzaImrGEPQQgg8Z\nibjn7aT5NF8DrK5JczwwE7i8qLwCiIh1pPkrAN7ckqeoNPpgUXmV8ywEPgNMyscEHqpIK/K8s1xJ\nFRHXAD8BHg8cXsqzN3A0UOyz7Jx8PifnoZzNzMyGjYcQHLBOeNZjZmZmAzSa5v1sqgLrK8D+ks6s\n2f4e4NHAfzd0vCr35dfyw48/5ddt5pKSNAHYC9hEGsKGiFgN3ANsL2m3imMU4zqX566qPUbDeczM\nzNqidg6s9hajk7U17snzSn0EuDAiru8l6ZH59TsV264H1gAH5yEA+5Pn2y1pAPYGHgHcFhF39jPP\nEfn1ey29soiIlcDPgKnAQRX7MzMza8SmzVu4f1X1SHgeQrBWJzzrMTMzswGICBavqG60M5Z7YH0C\n+DXwUUk3AsdCmlMhvz8X+F/g8w0dr0rx0OMvpXXX5ddjKtIfRnpYckNElD/R3vIc25IG0pA7dwH7\nSNqrn3l+lF+Pzi2WHiJpOmlYnjWka2ZmZtYR3APrIW2Le3KDm0tJscZ7+kj+2Py6TQOYiNgE3Eka\nPvpRed/TgN2BVRFxb8X+bs+v5QY3tcdoOE8lSTdXLcDj+sprZmZj29LVGyp7mU+bNIGpE5uaXWHU\n6YRnPWZmZjYAS1dvYOPmbUcAnrzdeGZM6b6Yp5EKrIhYS2pdeylp3qVnkBpqvwM4gDQZ+DH54cmg\nSdq3angZSXuSJiknH6twJXA/cKKkp5XSTwY+kN/+e8vuiqEI3ytpp5ZjvBVYT5qDAoBIEXCR56Pl\nCqk8kfqhwB+AH5fy3AF8Dyj2WXYuqRfZpblHmJmZWVsFNV2w3AcLaF/ck50NPBU4JR+3N8UQxMtr\nthfrdxxk+nbmMTMza1RdS+TZ7n1Vq80xj5mZmTVgUS/DB6oLWyY3VuUWEcuBUyS9A3g6sAvpgcQv\nIuK+XjP338uBMyVdD/wVWEkayuZ5wGTgW8D5pTKtkPQGUkXWAkmXA0uBF5JaA18JXNFyHjfkCcXf\nAfxG0pXAxHzsnYFTWyYyB7gAeD5p7okbJf2QNLzOCaSeVK9tHTIHeAtwA/BJSc8GbgUOJAWHtwHv\nHcwFMjMzG6q6IQStRzviHkkHknpd/VtE/LyJfXa7iDigan3uhbV/m4tjZmZd5L5V1Q9zZk3vvqF0\n2qlNz3rMzMysIUtGWaOdxvuMRcRS4Lut6yUdBTw+Ij45hN3/iFTx9FTSMHvTgGXAT0ktgi6NljEB\nIuJqSYeTKoReSqro+jOpguqTrelznjMl/ZbUO+qNwBbgFuBjEXFtRfr1kp4DvBt4BXAGsAK4Gjgn\nIv5QkeeO3CvsPNJwhc8F7gUuBM4tT6RuZmbWCbqwoc6wG664Jw8d+EVSo5b39TNb0ZNph5rtxfpl\ng0zfzjxmZmaNqnuYM3N6dz7MabdhftZjZmZmDemtB1Y3auegh68EXgUMOqiJiB9TGopvAPl+Rqog\nGkie+cD8AaRfQxrm5+wB5Pkb8JqBlMvMzGy4eQDBRgw17tmenjmh1tV0879I0kXAhRFxOvAn4Gk5\n383lhLlCbC9gE3m+0IhYLekeYHdJu1XMg/WY/Fqeu+pP+bVuvqqm8piZmTWqbgjBWV3aGrmDDPlZ\nj5mZmTVn0XJXYJmZmdko5iEEO8J64L9qtu1P6o3+U1LlUDG84HWkh0jHAJe15DkMmApcHxHlJ3jX\nASfnPJe05Dm2lKZwB3AXsI+kvSLizn7k+VF+PVrSuPKwypKmk3rVryFNAm9mZjYs6ocQdAWWmZmZ\njR5LVlbHPHN26M4KrHEjXQAzMzPrDt042We3ioi1EfH6qgX4ek72hbyumM/zSuB+4MQ8TDEAkiYD\nH8hv/73lUJ/Lr++VtFMpz56koZTXU6rYykMvF3k+KmlcKc9xwKHAHyj1mI+IO4DvAcU+y84lDQl9\naUSs7v2qmJmZDU5EcN/Kuh5Y3fkwx8zMzKzVps1bamMez4FlZmZmo0LUDCLo6qvOFhErJL2BVJG1\nQNLlwFLghaQ5RK8ErmjJc4OkC0hzg/5G0pXARODlwM7AqRGxsOVQFwDPB44HbpT0Q+ARwAmknlSv\nLfeyyt4C3AB8UtKzgVuBA4EjSEMHvnfoV8DMzKzag2s2smnztvHN5O3GM32SH4uYmZnZ6HDfqvWV\no+pMnzyBqRO7M+ZxDywzMzPbiocQ7F4RcTVwOHA98FLgVGAjqYLqxNyDqjXPmaQ5ORcBbyTNY/F7\n4AUR8emK9OuB5wDvB3YEzsjvrwaeHhE3VuS5gzQ/13xSxdWZwN7AhcBBEfHAUM7bzMysN4trJjOf\nOX2Se5ibmZnZqFE7/1WXDh8I7oFlZmZmLerqr/x8pzNExDxgXi/bfwY8d4D7nE+qXOpv+jXA2Xnp\nb56/kSrKzMzM2mpJ7fCB3TmUjpmZmVmVxSuqY545XTxk8qArsCS9aoBZHj3YY5mZmVkb1XTBGjeG\na7Ac95iZmXWv2vmvpnfvw5zh4pjHzMysey1ZWdMDayxWYJFa6Q5kkCENML2ZmZmNAN+sK83HcY+Z\nmVlXqhtCcNZ098CqMB/HPGZmZl2pdgjBMVqB9UUcpJiZmY0dY7cDFjjuMTMz61q1PbA8hGAVxzxm\nZmZdalFNo505Y3EOrIg4pcFymJmZWYeoGUFwTNdfOe4xMzPrThFRO5yOhxDclmMeMzOz7rRu42aW\nr9m4zXoJZm7fvY12xo10AczMzKyzRF0NlpmZmVmXWbFuE+s3btlm/YTxYqep241AiczMzMyat2RF\ndY/znadNZOKE7q0G6t6Sm5mZ2bCoq76SxnIfLDMzM+tG99X0vpo5fZJjGzMzMxs16oYP7Ob5r8AV\nWGZmZtZPfsRjZmZm3aauNbKHDzQzM7PRxBVYZmZmNibUzoHlGiwzMzPrMktW1lVgde9cEGZmZmat\nltRUYM1xBZaZmZmNJp4By8zMzEaLJTVDCM6a4QosMzMzGz0WLXcPLDMzMxvD5EEEzczMrMvcV9sD\nq7sf5piZmZkVIqJ+CMEdurvRjiuwzMzMbCtRM4aghxA0MzOzbrO4dg6s7n6YY2ZmZlZYuX4Tazds\n3mb9+HFi12ndHfM0UoEl6WxJh/WR5lBJZzdxPDMzM7OR4rjHzMysO6zZsInV6zdts14SO0+bOAIl\n6i6OeczMzLpD3fxXs2ZMYty47m6N3FQPrHnA3D7SHAac09DxzMzMbJjUdMByD6we83DcY2Zm1vGW\n1PS+mjl9IhPGe0CafpiHYx4zM7OOt2h5dcwzexQMmdzOiG07YEsbj2dmZmaDENTUYNlAOO4xMzMb\nYUtq5r+auX13D6XTYRzzmJmZjbDFtfNfuQJrIPYH7m/j8czMzGwQantg4S5YA+C4x8zMbIQtWVn9\nMGfmjO5/mNNBHPOYmZmNsEU1FVhzRkHMM2GwGSVd17LqFElzK5KOBx4OPBK4bLDHMzMzs5E1locQ\ndNxjZmbWfeqGEJw13T2w6jjmMTMz6z51c2DNHssVWGw9DnIAe+al1RbgAeAK4IwhHM/MzMzaoG4A\nwTFcfwWOe8zMzLpO3RCCrsDq1dzSz455zMzMOlxEsLim0c6Y7oEVEQ8NPyhpCzAvIs5rpFRmZmY2\nYuqGEBzLHPeYmZl1n7ohBEdDa+Th4pjHzMysuyxdvYGNm7edjnLSduOYMWUo/Zc6Q1Nn8Brglw3t\ny8zMzDqQxvIYgltz3GNmZtbh1m/azPI1Gyu37bq9e2D1k2MeMzOzDlc3/9XsGZNHxXOcRiqwIuIL\nTezHzMzMRl7UDCI4CuKeRjjuMTMz63z31QwfuNO0iUycMK5ym23NMY+ZmVnnq5vzczQMHwjN9cAC\nQNIs4GnATqQJPbcREV9s8phmZmbWLA8h2D+Oe8zMzDqX579qznDHPJJ2AV4MPA/YD9gd2AD8FrgE\nuCQithkbSdLBwFnAQcAU4HbgYuBTEbG55livBt4KPB7YTOphdn5EXFuTfgrwbuBE4JHACmABcE5E\n3FqTZw/gPOAYYBfgXuBq4NyIeLD3q2FmZjYwvfXAGg0aqcCStB3wOeBVQF1TJpEmAPWDHDMzsy7k\nDliJ4x4zM7POV9caeaYrsPqtjTHPCcC/kyp6fgTcBcwGXgL8J3CspBMieppZSToO+CqwDrgCWAq8\nAPg4cEjeZ+v5nA+cCdwNXARMJFVMfUPSqRHx6Zb0k4Dv5/3dBFwIPDzv+3mSjoyIG1vy7A3cAMwC\nrgH+CDwDOA04RtIhEfHA4C6TmZnZthYtdwVWf7yfNDbyHcCXgb8Bmxrat5mZmXUADyH4EMc9ZmZm\nHe6+laP7YU6btCvmuQ14IfDNck8rSe8BfgG8lFSZ9dW8fgapAmozMDcibsrr3wdcBxwv6cSIuLy0\nr4CojuoAACAASURBVINJlVd3AE8vekJJ+hhwM3C+pGsjYmGpXO8gVV5dCby8KJukK0g9qi6WtF9L\n77DPkiqv3h4Rnyod/wLgDOCDwJuGcK3MzMy2sqQm5pmzw+iIeZqqwPpHUsDx1IhY29A+zczMbARs\n3lI9huA412AVHPeYmZl1uLohBN0Da0DaEvNExHU16xdJ+hyp0mcuuQILOB6YCXyxqLzK6ddJOgv4\nIfBm4PLS7opKow+Wh/GLiIWSPgO8j1RZdw6A0qz3RZ53liupIuIaST8BDgUOJ/UaK3pfHQ0sBD7T\ncjrnAG8ETpZ0ZkSs7uOymJmZ9WnT5i21837OnjE6Yp6mZi6dBXzLD3HMzMy636bN20wxAMCEcZ7w\nPHPcY2Zm1uHqhhCcPX10tEZuk06IeTbm13LPryPz63cq0l8PrAEOzkMA9ifPt1vSAOwNPAK4LSLu\n7GeeI/Lr91rn7IqIlcDPgKmkObvMzMyG7L5V6yvnMZ8+eQJTJzbVd2lkNXUWdwEzGtqXmZmZjaCN\nNT2wJox3D6zMcY+ZmVkH27R5Cw+sdg+sBoxozCNpAmn+Ldi64umx+fW21jwRsUnSncATgEcBt0qa\nBuwOrIqIeysOdXt+3ac/xxhinqNznh/WpAFA0s01mx7XWz4zMxtbaue/GiXDB0JzPbDmkybV3KGh\n/ZmZmdkIqeuBtZ0rsArzcdxjZmbWse5ftaG2NfKUiePbX6DuNZ+RjXk+AjyR1Avsu6X1RXmW1+Qr\n1u84yPTtzGNmZjZoi2t6nM8ZRXN+NlWB9RHgp8APJB2RJ9Q0MzOzLrRxc3UPrPEeQrDguMfMzKyD\n1c0FMWsUPcxpkxGLeSS9HTgT+CNwcruO2yki4oCqhXQ9zMzMAFiysqYH1iiKeZoaQrAYk1jADwBU\nPdF7RMToGHzRzMxslNpcN4TgOPfAyhz3mJmZdbDFK6of5szy8IEDNSIxj6S3ARcCfwCeHRFLW5IU\nPZnqeoYV65cNMn0785iZmQ1a7RCCrsDaxk+A6qddZmZm1lU2bakbQtA9sDLHPWZmZh3svlWe/6oh\nbY95JJ0OfBz4HanyaklFsj8BTyPNJbXVXFF53qy9gE3AXwAiYrWke4DdJe1WMQ/WY/Jree6qP+XX\nfajWVB4zM7NBW1TTaGfOKJoDq5EKrIiY28R+zMzMbOTVDSE4wXNgAY57zMzMOt2SmvkgZk0fPQ9z\n2qHdMY+kd5GGLfwV8JyIuL8m6XXAK4FjgMtath0GTAWuj4j1LXlOznkuaclzbClN4Q7gLmAfSXtF\nxJ39yPOj/Hq0pHER8VCrMEnTgUPg/7d353GSVfX9/1+f6mVmevZ9kG0A2dySKIrBDVCIa0xcIsav\nW6KJJl+3aL5JSBQwZjVBE/0akpiEuHyDBn+KMRqNIKKgEMANZIeBAWafYfbprc7vj3ur+3ZNVU8v\n1dXVVa/n43Ef1X3XU6dr6r7nnnvO5QDw/TrvS5KkCTs0OMzuA4OHzY+A1Yva56Ydb6WWJEljDA3X\n7oHlEIKSJGku2FLneRBrlrTPxZx2ExHvJ2u8uoWs51W9xiuAK4HtwAURcUZhH/OBD+W//l3VNpfl\nr38YEcsL26wHfhvop9CwlVJKhW3+MiJKhW1eDjyHbIjDbxe2uQ/4BlDZZ9ElwELg0yml/eO8N0mS\nJqTeDTsrFvbS290+zT4Nfy5DRCwk6y69KKX0nUbvX5Ikzayhes/AcgjBw5h7JElqLeVyYvveej2w\nbMCaqpnMPBHxRuCDwDDZsIXvrPGsrQ0ppcsBUkp7IuKtZA1Z10bEFcBO4BeBU/P5nytunFK6ISIu\nBX4H+HFEXAn0Aq8BVgDvSCltqDrmpcBLgVcBN0bE1cBxwKvJelL9WrGXVe63gBuAv42I5wN3AGcC\n55ANHfiHk6ocSZLqqDd8YDs9/woa2AMrIo6JiC8Au4CbGe06TUQ8OyJ+GhFnT/MYKyPiLRHxxYi4\nNyIORsTuiPhuRPx68Y6YfP31EZHGma4Y51hvjIibImJffoxrI+Kl46y/ICIuiYi7IuJQRGyNiM9H\nxOnjbHNMRPxzRDwaEf0RsSEiPlq8G0iSpGYbqjeEoD2wRjQj90iSpMnbeWCA4Ro348zv7WLRvIbf\nw9v2mpR5Tshfu4B3AxfVmN5U3CCl9CXgecB1wCuBdwCDZA1UF+Q9qKja5r3Am4HNwG8AbwBuB16W\nUvp4jfX7gfOAPwaWAe/Jf/8S8PSU0o01trmP7Plcl5M1XL0XOAn4G+CZKaUdE6oRSZKOoFMasBqS\n3iLiKOBGYC3wZWAN8POFVW7M570GuHYah3o1WTfwTWSh6aH8mK8APgm8KCJeXSOo/IgsYFS7rdZB\nIuKvyELGw8A/kt2VcwHwHxHxjupgExHzgP8mG8/4ZrJgcmxe3pdExLnVwSYiTiK7K2cNcBVwJ/AM\n4F3ACyPiWQYbSVKzpZQYKtceQrDHHlhAU3OPJEmapG3j9L6q0atH42hW5kkpXQxcPIXtrgdePMlt\nLidrXJro+geAD+TTRLfZSNZQJknSjNlapwFrnQ1YNV1EFlrOSyl9KyIuohBqUkqDEfEdsgae6bib\nrEv4f1Y9DPNC4Cayu25eAXyharsf5oHoiCLiLLLGq/vI7qjZlc//MNlYzH8VEV+p6lr+O2Tv7Urg\nNZWyRcTnyBrO/jkinlzVtfwTZHX2zpTSxwrHv5Tsrp4/Ad42kTJLktQo/UNlDr9fNWu86rIHVkWz\nco8kSZqkLXUu5qxZ3F4Xc5rEzCNJUovavLszemA16lbqFwNfTil9a5x1HgIeN52DpJSuSSn9R/UY\nwymlzYw+XPPs6RyD0UajP6k0XuXH2AD8X2AehTtpIruFq7LN/ymWLaV0Fdn4zU8g69pe2eYk4Hyg\nss+ii4D9wOvzMaYlSWqagwPDNecv6O1qckla2oznnskOm1zY7qyI+GpE7My3+XFEvDsi6v4BHTZZ\nktROtvr8q0ZqyrUeSZI0OSml+kMILm2vzNOoBqy1wD1HWGcQmMkGmcH8dajGssdFxG9GxIX561PG\n2c+5+et/1Vj2tap1IBvL+Djg7pTSAxPc5pz89Rs1GuP2AtcDfcAzxymnJEkNd3CwdgPW/B4bsAqa\nkXteTTaM8Zlkw/N8lKyH+ZPIhk3+fFSNgxQRLyd7DsRzgS8CHycbBvkjQM3nfubDJl8OHJUf7zPA\nk8mGTf7fNdavDJv8AWAP2bDJ3wR+Gbg5Is6ssc1JZL3Y30zWY/4jwP1kwyZ/LyJWTqxKJEk6srpD\nCC5pr4s5TdIK13okSVKVvf1DNW9A7ioFqxa2V+Zp1BCCO8me+TSeU8gelNlwEdFN9vBNqN3wdF4+\nFbe5FnhjSumhwryFwNHAvpTSphr7qQS3UwrzTs1f765TvKluc36+zdV11qmU+ZY6i04bbztJkmo5\nVLcBy+dfFTQj90xq2OSIWELWADUMnJ1Sujmf/37gGuBVEXFBSumKwr4cNlmS1HbqPQ/CIQSnZFav\n9UiSpNrq5p0l8yi12eMfGnU16nrgFyNiXa2FEXEy8EJgvG7n0/HnZHckfzWl9PXC/APAHwNPA5bn\n0/PycpwNXF01TN/S/HV3neNU5i+bhW0kSZpx9XpgLbAHVtGM554pDJv8KmA1cEWl8Spf/xDwR/mv\nb686jMMmS5LaSkqJbfscQrCBZvtajyRJqmHLntp5Z20b3rDTqAasDwPzgW9HxIvIhr8jIhbmv/8H\nUAb+ukHHGxER7yS7e/hO4PXFZSmlrSmlD6SUbk0pPZZP15FdSLkReDzwlkaXqZlSSk+rNZHVhyRJ\nk1KvB5YNWGPMWu7J1Ro2ebwhkK8ju6nnrHwIwIls47DJkqQ5Z8/BIfoHy4fN7+kqsayvZxZKNOfN\nduaRJEk1bN5d7/lX7deA1ZAhBFNKN0bEbwJ/B3ylsGhP/joE/FpK6fZGHK8ifzbD3wA/BZ6fUto5\nwfIORcQnyZ4r8dx8HzDa82lpzQ1H5z9WmNesbSRJmnG7Dw7WnN83r1GjDs99s5V7YNxhk+sOT5zn\nngeAJwInAnfMxWGTwaGTJUnj27q39sWc1YvnUfXoSE3AbGYeSZJU3+Y6QwiuW2IDVl0ppX+OiO8A\nv0V2F+1Ksoaa7wMfTynd1ahjAUTEu8keAn4bWePV1knuYlv+OjJkTUppf0Q8AhwdEUfVuKBzcv5a\nvAhTeV+nUFujtpEkacbt2DdQc/7Khb1NLklra3buKag3bPJkhyd22GRJUtvZutfhAxttFjOPJEmq\no94zsNbagDW+lNI9ZA/jnlER8XtkF3B+CJyXUto+hd1Uhqq5v2r+NWRDEb4Q+JeqZS8qrFNxH/AQ\ncEpEnFBjSJ1a21TGhz4/IkpVD2dfTPZw9ANkgVCSpKbZsb92A9YKG7AO06zcUzHesMmdIh8m+TB5\nz6ynNrk4kqQWU68H1polNmBNR7MzjyRJqi+lVP8ZWG2YeRr1DKymiYj3kzVe3ULW86pu41VEPDUi\nDnuPEfF8RsPXZ6oWVx6M/ocRsbywzXrgt4F+Cg1bKaVU2OYvi8eLiJcDzyEb4vDbhW3uA74BVPZZ\ndAlZr7BPp5T213tvkiTNhJ11GrBWLrIBazZVDZt8To1hkyc7PLHDJkuS2s7WOhdz1rThA80lSVJn\n2rl/gMHhw5/5Oa+nxNIF7ffMz4b2wMobb44GjgFq1lZK6bpp7P+NwAeBYeA7wDtrjGO9IaV0ef7z\npcDJEXED8HA+7ymMPlz8/SmlG6rKd0NEXAr8DvDjiLgS6AVeA6wA3pFS2lB1zEuBlwKvAm6MiKvJ\nHnL+arKeVL9W/eBysu73NwB/mzeo3UH2TK5zyIYO/MOJ1IkkSY20fV/tCz8rF7bfXTzTNdO5p3Cc\niQybfBdwBtnwxGOeE5U/N+sEsudU3J+Xy2GTJUltp+4Qgm14N3IzNSvzSJKkI6v3/Ku1S+a35TM/\nG9aAFRG/C7wPWHWEVbumcZgTCvt4d511vg1cnv/8aeCXgaeTDeXXA2wBPk82VvN3au0gpfTeiPgJ\nWe+o3wDKwK3Ah1NKX6mxfn9EnAf8PvBast5de4AvARellH5aY5v7IuIMsga5FwIvBjaR3V19SUpp\nV/1qkCSp8YaGy+w+MFhzmUMIjtWk3DOZYZOvAV5Hlin+rWrZc4E+4LqUUn/VNg6bLElqG/UasFb7\nDKwpa1bmkSRJE1Ovx/m6Nnz+FTSoASsiLgY+AOwA/hV4hOwu34ZKKV0MXDyJ9f8J+KcpHutyRhvC\nJrL+AbI6+MAkttkIvHmyZZMkaSbUu4sHoLd7zo06PGOalXvyYZM/SNaj6vwawwYWXQn8BXBBRHws\npXRzvo/5wIfydf6uapvLyBqw/jAivlS5eWa8YZMj4jLgT8mGTX5NpUFqvGGTI+IbwPn5Pj9WOH5l\n2OS/d9hkSdJ07esf4kD/4afjUinsST5Fzco8kiRp4sbrgdWOGtUD69fJhqR5Wkpp95FWliRJreeq\nHz4620WYK2Y890x22OSU0p6IeCtZQ9a1EXEFsBP4ReDUfP7nihs7bLIkqZ1sq9P7atWiXrpK7Tec\nTpN4rUeSpBazebcNWFOxErjMQCNJ0tx164OOXjtBzcg9kx02mZTSlyLieWQNQq8E5gP3kjVQ/W1K\nKVXvwGGTJUntYmudu5HXLG7PizlN4rUeSZJazNa9tTPPuqXtmXka1YB1L7C8QfuSJElqZTOeeyY7\nbHJhu+vJGogms83lOGyyJGmOq/f8qzVLHD5wGrzWI0lSCxkaLtftdb62TTNPox5o8QngpRGxrkH7\nkyRJTVSjc86IVz3tmCaWZE4w90iS1GLqNWCtXtSeF3OaxMwjSVIL2bavn1qXbxbP76avt1F9lVpL\nQ95VSumyiDgFuD4iPkg29EzNLuYppYcacUxJktQ492/fX3fZ809f28SStD5zjyRJrafecDrt+jyI\nZjDzSJLUWrbsqdP7qk2HD4TGDSEI8CPgTcA/j7NOavAxJUlSA/zpf95Rd1lvd6M6bLcVc48kSS1k\nW50LOqsX2wNrmsw8kiS1iM276zz/qo1v2GlIwIiItwB/DwwB1wKP5j9LkiS1FXOPJEmt5dDgMLsP\nDh42PwJWOYTglJl5JElqLZ3Y47xRd8i8D9gKnJVSeqBB+5QkSU2wv9/rEJNk7pEkqYXUe5j58r5e\ne5JPj5lHkqQWUq8H1tol7XvDTqOS3PHAlQYaSZLmng9//a66y45ZvqCJJZkzzD2SJLWQencjr2nj\nizlNYuaRJKmFbN7TeT2wGtWA9QjQ06B9SZKkJtq480DdZcevXNjEkswZ5h5JklpIvR5Yaxa378Wc\nJjHzSJLUIg4NDrP7QO0hk9s58zSqAetTwIsjYnGD9idJkppgvMYryIKQDmPukSSphWyt04C1erE9\nsKbJzCNJUovYuqd23lmxsL2HTG7UO/tT4CbgmxFxtuFGkqS54eIv3z7u8q6SLVg1mHskSWoh9S7o\ntPPzIJrEzCNJUovoxOEDAbobtJ9KWgzgaoCofct2Sik16piSJGka9h46vOt5tTrn805n7pEkqYXU\nfQZWGw+n0yRmHkmSWsSm3QdrzrcBa2K+A6QG7UuSJDXBu6/44RHXsfmqJnOPJEktYnC4zM79AzWX\nOYTgtJl5JEmaZf1Dw/zHjzbx9ds311y+zgasI0spnd2I/UiSpOb49cv/Z0LrleyBdRhzjyRJrWPH\nvgFSjSaWJQt6mN/T1fwCtREzjyRJs+tHGx/jszc+yI59tW/WAXtgSZKkNvPA9v0TXtdHYEmSpFa2\npc7zINbY+0qSJM1RO/cP8G83PcStD+4ad72uUrB+VV+TSjU7bMCSJKnDfOgrP53wuvbAkiRJrWzb\n3v6a8x0+UJIkzTXD5cQ379jCVT98hP7B8hHXf8ET1rJ4fk8TSjZ7GtKAFREfmOCqKaX0x404piRJ\nmryJDh1YcczyBTNUkrnL3CNJUuvYWqcBa02bD6fTDGYeSZKa575t+/j09x5k484DR1x3Xk+JX/65\nY3jB6WuaULLZ1ageWBePs6wyGnXkPxtqJEmaBZNtvOrpKnHG+hUzVJo57eJxlpl7JElqoq17HUJw\nBl08zjIzjyRJDbC/f4gv3Pow1929reZzPas99fjlvPYZx7FiYe/MF64FNKoB65w685cBTwfeCfwn\ncFmDjidJkiZhIo1XJ6xaOPJ8rAh4w1nH09tdmumizUXmHkmSWsChwWEefexgzWU2YDWEmUeSpBmS\nUuJ79+/g8/+zkb2Hho64/spFvbzuzOP5mWOXNaF0raMhDVgppW+Ps/iqiPgccBNwRSOOJ0mSJmZo\nuMxvfvqWI6733vNP5fFrFnH3lr1s29fP6euWsG6pQ+/UYu6RJGl27Tk0yDV3bOXqO7dyoL/2BR+H\nEJw+M48kSTNj0+6DfPp7D3LX5r1HXLerFLzwSet4yVOOYl53VxNK11oa1QNrXCmln0TEVcCFwFXN\nOKYkSZ3uz752B/du2TehdZ/wuCUAPOnopTNZpI5g7pEkaWZs29vP12/fzHfv2c7gcP0Hmy/o7WJh\nb+dd4Gk2M48kSZNzaHCYr922ia/9ZDPD5SOPF3jKusW8/pnH87hlnft88qY0YOUeAl7WxONJktSR\nUkq85V9vnvD6//Smp89gaTqWuUeSpAbZuPMAX7ttEzc9sIs0gYdDPH7NIiKiCSUTZh5JksbVPzTM\njx/ezU0P7OQnD+8e9yacikXzu/mVM47lrJNWdnymaWYD1plA7cGpJUnStJXLibd+auINV2Dj1Qwy\n90iSNA0pJe7eso+v/mQTtz2ye8LbVYbZUdOYeSRJqjI4XOa2R7JGqx89/Bj9g0dutKp47imreeXT\njmHRvGY23bSuhtRCRBw3zv6PBd4KPBv4fCOOJ0mSRu3rH+Jd//aDSW9n49XUmHskSZo5KSVufegx\n/uu2Tdy/bf+ktl2/aiG/euZxnLR60QyVrrOYeSRJmrih4TJ3bNrLjQ/s4AcbH+PQwPCktj9m+QJe\n//PH8/g1i2eohHNTo5rxNgDj9eMP4B7gfQ06niRJHW0qva2KbLyalg2YeyRJaqjB4TLfv38HX7tt\nM1t2H5rUtk88eikvetI6Tlu3uOOH2WmwDZh5JEmqa7icuGvzXm56YAe3PvQY+/uHJr2P3u4SL//Z\no3nB6Wvo7irNQCnntkY1YH2K2qGmDOwCbgKuSin1N+h4kiR1nOFy4jem0WgF8Ne/8jMs6+ttUIk6\nlrlHkqQG6B8a5t6t+7hj015uuG87uw8MTnjbCHj6+hW86ElHcdzKvhksZUcz80iSVCWlxD1b93Hj\nAzu5ZcNO9h6afKNVxc8dt4zXPuM4Vi6a18AStpeGNGCllN7UiP1IkqSxdu4f4Hf//UcN2Ze9rhrD\n3CNJ0tQMDZfZsGM/P920lzs37eHerfsYLo/XwedwPV0lnnXyKn7hiWtZs3j+DJVUYOaRJAmyBqst\ne/q5a8te7t68lzs275nUTTfV5vWU+Nljl/Gck1dz+lFLGljS9uSTwCRJaiEDQ2Xe/plbGrrPP3vF\nk1mzxAs8kiSpucrlxMZdB7hj0x7u2LSXe7bundRDzIv65nVz7mmref7pa1kyv6fBJZUkScqklHjk\nsYPcs2Ufd27eyz1b9rL74NQbrCC7Aecpxy7lGetX8ORjljKvu6tBpW1/DWnAioiTgGcB/5lS2lFj\n+SrgxcB3U0r3N+KYkiTNdSklbnpgJ/9w3cycGn/juSdy5okrZ2TfnczcI0lSbSklNu0+xB2b9nDn\n5r3cuXkvB6bwLIiiZX29/MIT1/LcU1Yzv8eLPc1k5pEkdYJyOfHwroPcuXkP92zdx12b907pWVbV\nukrBk49eytNPWMHPHrvMHDNFjeqB9fvALwH/Vmf5buCvgC8Ab2/QMSVJmlPu2ryXv/yvO2f8OG84\naz3PO2X1jB+ng5l7JEkC9vcPsXHXAR7acYAHtu/nrs3Tv0O54qhl83nRk47izBNW+EDz2WPmkSS1\nnaHhMg/tPMDdW/Zy1+Z93LN1LwcHhhuy74jgiY9bwjNOWMHPHbeMvl4HwJuuRtXg2cA3U0o1k2pK\naTAi/hs4t0HHkySpJaWUuG/bfq685WHu2bK3acd9z3mn8KSjlzbteB3ubMw9kqQOklJi275+Nu48\nwMadB9m48wAP7TzAzv0DDT/WSWsW8aInreNnj11GRDR8/5qUszHzSJLmsJQSW/f288D2/WzYvp8H\nduznoR0HGBia2pDGtUTAqesW8/T1K3ja8ctZ7FDHDdWoBqyjgSuPsM5DwC826HiSJM2qcjnxg427\n+OpPNrNh+/5ZK8c/venps3bsDmbukSS1rYGhMo8+djDrWZU3VD286yCHGnRncrW+ed2ctm4xTzhq\nCacftYR1S31uZwsx80iS5oyUErsODPLA9n1s2H6ADTv288D2/Q3rXVUUkd1084z1Kzjj+BUs7bPR\naqY0qgFrAFhyhHUWA6lBx5MkqSl2Hxjkyz9+lGvv3DrbRQHgxNULufDFp3tH8uwy90iS5ryBoTJb\n9hxiy55DbNp9iE27D7Jx50E27T5ESjN3CpvXU+KUtYs5bd0SnnDUEo5dscBc07rMPJKklrXn0CAb\ntu9nw44DPLBtPxt27GdPg4YyrhYRrF/ZxynrFnPq2sWcvHaRwwM2SaNq+TbgJRHx7lpdyyOiF3gp\n8NMGHU+SpIYolxOPHRwcuYDz6e89ONtFOsw/vuEMSiUv7LQQc48kaU5IKbH74CCbdh9i855DbNmd\nNVZt3n2IHfv7mcF2qhFdpeDktYs4bd0STj9qMetXLvSZVnOHmUeSNGtSSuztH2LHvgF27u9n+74B\nduwbYMe+fjbuOsCOfY0fyriiqxScsGohp65bzClrF/P4NYuY39M1Y8dTfY1qwPoM8Ang8xHx9pTS\n5sqCiFgHXAYcC/xlg44nSdKElMvZMxse3nWA79+/k339Q6xc2Mv37tsx20Wr66KXPZHjVvbNdjFU\nn7lHktQyKhd3dh8YHOlNVXndvPsQhwZnZui/eiKCE1b1cfpRSzht3RIev2YRvd02WM1RZh5J0owZ\nLid2Hcgbpfb3s3P/aAPV9v0D7Nw3wOBw455VNZ6erhInrh5tsDpx9ULmddtg1Qoa1YD1D8ArgJcD\n50XEj4FHyMZLfgrQB3yTLNxIkjRlg8NlDg4Oc6B/mP0DQxzoH2bH/n6uu3s7D+6YvWdRTdUrn3YM\nL3ziOntYzS3mHknSjEopcXBwmN0HB9l9cJA9B4cKPw+O/nwoWzaTQ/4dyZol8zh2RR/HLu/j+JV9\nnLxmMQt6veDTJsw8kqRxDZezzHJgYIhDA2UODA5xcGCYgwPDHBgY5sDgMIcGsuUHB8scHBjiwECW\ncXYdGJy1DDOvp8RJqxdxaj4k4PpVC+mxh3hLakgDVkqpHBEvAS4B3g48s7D4MeCjwCUppeY0mUqS\nZlRKieFyYqicvQ6nxPDw2N+HhssMlRNDw4mhcnnkdWAocWhwmO/fv4MlC3o4fmUfA0Nl+ofKDA6X\nR34+NDjMTx/dM+a4PV2lpt1902hrl87nd847hVWL5s12UTRN5h5JUi0pZVmof6hM/+Bw9jpUpn9o\nmP7BsT8fGhpmYKjMocF83lCZ/sEyew+NNk4Nl1vrsUI9XSWOWb6A41ZmjVXHrljAMcv7HE6njZl5\nJKm1pZQYHD78OszI9ZrhxGC5nC0vXJ8ZLmfbDQ6X8ykxMFxmKL8mM1hODA6VR5YP5OsWlx8aHObQ\nYJZrWl13V3Dcij7Wr1rICSsXsn7VQtYtme+NxHNEw540lo+HfGFE/BFwGrCMLNDcaZipLyKOAT4I\nvBBYCWwCvkQWAnfNZtmkyUgpkVL29N7K3RPZz5BIY8bXr8wb/bmwj3weE1gnZSsxVB49Xrk8uk02\nL1H5v385FZdlZaq1rFLe4s+V5eXCtsXjlaveZzmNbpdvkpWlPFpHlX2Va21TeK0cJ5HYe2iIMXSx\nkQAAIABJREFUWx7cxcBQmfWrFnL8yj6Gy9k+UmG78sj2o+9zOP+9sn45JcqFnyuBp7LOcLmyrzQS\ndrJtx/49p+vWByf+VTcXGq+6u4JLf+VnWTjPh3m2M3OPJE1MJfOkQj6B6rwydnmWUUZ/ToWsUr2v\n6nXLKbs4M5zPH87zTuVCzuh+qvMNDJdHL/oMDo+9CDRYLo/crFO88DP2hp00qz2hGmnpgp6sV9WK\nPo7NG63WLvZCTycy80hqhOprRsVrKGOufaREKo9ey6lcA8quiTDmOku5PPa6S/U1kGKOqLVeOTFy\nTaRYpuFy7Wsr5cK8VMggw9XLy2PXLZfTmH3Vug5TWWeonOeWfP3hSqbJ16lkmXKZtsodjRQRHLN8\nAetXZg1WJ65axOOWzff5m3NYw6+u5QHGB3hOQEScBNwArAGuAu4EngG8C3hhRDwrpdSyD2kp9sDI\nWumz/8ANV/7TmN99uOfQIIvnd1f9x/LwE8ZwsQEhVZ2s6jQKpMToBXVqr1tzPmOPQWGdsQ0khcaY\nwxoxio0e2bZD5cSBgSH6ekf/aVX2R+GYo/sebYSpPlaqWn/jzgMj+3zcsgUjjTuj5YdKM0+5fHjj\nz8ix8tVG6qXqvRYbWyr7ymaN1mF1WTU7Nmzfz4btc2/IvLlu8fxuzj51DU89bjlHL19AlxdyOpq5\nZ+K8aae1pEKmKeaBYuNCreX1tqu+waRWBqvOXsUcNbrPw28kGdm2aruR7RmbS6rLVK7KLLXeG4cd\nd+x7gsPzW+X3ynuotf+R3+tkwJF9VP0NjpRDx+SykXw8erxi3RePUynL2GOM/VtU9l99c0xxH5X6\nLY+ZN/YCUWV7tbalfT0ctXQ+65bMZ93SBRy1dD7HLu9jaV/PbBdNLcbMM3FmnsmrfTPm2AaHYmNF\ndYND5bpSMWvUukFiZHlVA0nx5tJKQ0mx4aO4T2qUrzp3VOeg6vNoMRcUz+vFc/9oOcbWUXE7GHsO\nrnUOr1f26nP64WXP62RMXRXz0uENStVZoDhPminrls7nhFULWZ/3rDpuRZ/P3mwzDW/AiohnAz9H\ndlfObuDWlNJ3G32cNvEJssard6aUPlaZGRGXAu8B/gR42yyVDchONNv29nPvtn3ct3UfG3YcYPfB\nwZFuop6EZsejjx2c7SJIbempxy/nvm372H1gkJWLennDz6/nmOULWDivm1KEjVU6jLlnYlr9pp19\n/UNs29vPcLk8OgRHeXT40+KQqYPD5TG/Zz0vxg7VUeztOnKRoVzjBp7KxYLCXZlTaUwgjb2AUusi\nwtiLHV5IkDpJT1eJtUvmsW7pAtYtncfaJfM5aukC1i2Z77OqNGFmnolp9cyTUuLhXQezocDKaWTo\nsOIwYkNV84YKQ4tVbmAeHMpzUnm050j1CB7Fm5UP75UytletuURSPT1dJVYs6mXVwl5WLprHykW9\nrFjYy+pF8zhmeZ9ZpgM0rAErIp4GfBo4tTKL/IbCiLgLeENK6eZGHW+uy0PN+cAG4P9WLb4I+A3g\n9RHx3pRS07tYPHZggE9/70Hu27aPvYeGmn14SZpRL3nKURyzvI/Vi+exbEEPy/p6iLBxShNn7pm0\nlr5p5/ZHdvMP190/W4eXpGmb39PFkgU9rFjYM9KbKnudz8qFvQ79pykz80xaS2eecoKLv3z7bB1e\nkg7TN6+blQt7sylvoFq1qJcVC7OfF8/r9npNh2tIA1ZEPB64GlgCfBe4hqyL9FHAucCzgf+OiGek\nlO5pxDHbwDn56zeqx41OKe2NiOvJGrieSVa3TbVwXjc/eWR3yz04WJIiYEFvN309XSya301fbxcH\nB4Z5aOcBzli/nIMDZZ598koWz+9hYKjMUUvns3LRvNkuttqIuWdyWv2mHcieXSdJraanq8SSBd0s\nXdDDkvk9LO3ryX6u/L6g8ns387q9+1iNZ+aZnLmQebpKQUQ4xKukhomAeT1dLOjpoq+3iwW9XfT1\ndLOgt8SC3u7R+T35spGpmxULe5nfY4bR+BrVA+v9wGLgNSmlf69adnFEvAq4Avgj4I0NOuZcV7l7\n6e46y+8hCz6nME4DVkTcUmfRaVMvWvafpfWrFnLf1n3T2Y2kNtZVCkoRdHdlQ+t1lYKuwu/dpRI9\nXUFXqUR3vry3O/v53q37WLV4Hsev6KMUQakU9PV2Mb+nxLzuLnq7S8zLxyz+ySO7OW3dYk5ctYgF\nvV3M6y55941mm7lnclr6ph2ArpJjpEtqjIgYyTPzerI8M687yy/zekZ/nt+T5Z353aXstSeb39fb\nPdIwNb/HzKNZZ+aZnJbPPAC93UH/oA1YUrvoKmXXYbpLpZFrMz2F6zLZaxReS/n6QU9XiZ7uEr1d\n+c/51Ntd/L1qWVeJnu6gt6vEgrxhyryimdSoBqwXAF+sEWgASCldGRFX5espszR/3V1neWX+siaU\npaaTVtuApckZPV8FEdnYEuXCeNbdXVmDR7YsWyFGto2RnysjnIzMi5G9jhwj8uMFQVdXjPm9cuzK\nCfTw+WP3fdiyyu8xdr+lGN02K2eMlDdi9PdK2Uv5vMq+yeeN2VdeCaXCsSoNOsXfK+9ny55D3PTA\nThbO62LFwnmceeKKbP3CsbLfKz9n23Xl80ql0eVdpUo58+Wl0UapyvOeSgGlvGFqpMGqFE0dhuYp\nx8za16BUj7lnchpy0w7M3I073Q6tJc2oGJOJssxRySABIzmmkokquaSSp7oi6zEwkk3yrDSafypZ\nJ88tpco+gq4gv5hTyvZVGs00h/+cNWj3VC7ydGU33lQuCnV3BT2lEl35RZ/u0uj84sUhL+KojZh5\nJqflMw9Ad6lEP+Ujryg1WOQXYYrXQyrn+ywPjF4/qZzfR65rVK1bvPYBlZ+LmePw36OwXuWaRuX3\nrqrfS6WqfFHIKbWySHF+lgUK80tV65RizHEPyyT5Nt15dhm5JtNVdf0mzB3qDI1qwFpF9mDK8dwJ\nvLRBx1MupfS0WvPzsPPU6ez78WsW8Y3bt4y7Tilv1e8utN53d5VGvkS7S8HgcJmHdx3k2BV9LJ7f\nfdiXdvEC/cjJauRCPyMnrOLJBsY2GlRODsWGh5F1ajRolGLsSW9sY0XtxpBKQ0flGtfoyW6kzkfK\nd3BwmAU9XSNljZF1imXMGy8K7yVfa0x5K7MPDQ2zvK+XgaHy2AaYQp2MlLVQb3nRx7yv/ChE6QjL\nq8pdPMZh6x/hhJlS8qTaIG88a/1sF0HqdOaeyWn5m3Yq/2HtNMVz+JiMVap3M0chl1Xd/FEq5o46\nWap4Y0kxz1E4dvVNHcVlY7NcdS4Z/8aV0fUPP+5o1quXH6tyUL3yFd5/9fsq5WGyVo4a2X9hWXUZ\nauXQ0TLEmDw6ZtvCz8WbW6rLUr3NmMamynuOqsxeGrt98QJSZd3KtpLmLDPP5LR85oFsxJ1WNPZa\n0OhF/+qbQ6tv4ByznMrvo+e86oaQ4nrV56viua/W9anR82+x4WRs40Z1NqqVi4rn4uptinUxZl/Z\nJmPyQBTKXrzONXrs8c/pULlJJNtfMV+MHKe6zorlq1P+wxuizAOSpq5RDVjbgCccYZ3TgO0NOl47\nqASXpXWWV+Y/1oSy1HTS6kVAdgI7fkUfJ61ZxEmrF3H8yj4WzutmfneJ7hYNPmo9hhVJbcTcM0tm\n6sadhb3drF+1cKT3RGW4jepeFZXeGGOH4RgdlmNkSNXixZZS8cJC4c7P0uEXYirrTLYxoXgRoXih\nYGwjz9gLHHm9TbXKJEmdwcwzS2byZuVjli9gWV8P3V3ZEGCVzNPblfc0zYcI6x4ZNiwfHr577FBi\nI9knYqRHSaUxZGRkj1KxZ+3YHie1GqIkSarWqAasa4BfjYgLUkpXVC+MiFcCLwc+26DjtYO78tdT\n6iw/OX+t1/V8xi3r6+XCl5zOscv76O22oUqSpJy5Z3Ja/qad41b28f6XHun6nCRJHcfMMzktn3kA\n3nNevctQkiS1nkY1YH2QPLRExG8D3wI2AeuAs4FnA3uBDzXoeO3gW/nr+RFRKj7gMyIWA88CDgDf\nn43CVVR6YUmSpBHmnslp+Zt2JElSTWaeyTHzSJLUYA1pwEop3RsRLwA+Rdbw8iwgwcjjeO4C3phS\nuqcRx2sHKaX7IuIbZA/w/G3gY4XFlwALgb9PKe2fjfJJkqTazD2TNidu2pEkSWOZeSbNzCNJUoNF\nSqmxO4w4i2w83qVk3ad/kFK6vqEHaRMRcRJwA7AGuAq4AzgTOIfsjpyzUko7prjvHQsWLFhx+umn\nN6q4kiRNyx133MHBgwd3ppRWznZZGsXcMzER8XWym3bemVL6WGH+pcB7yG7aeds09m/ukSS1DDNP\n5zLzSJI6STMyT8MbsDQ5EXEsWbf8FwIrybrjfxG4JKW0axr7fQBYAmxoQDFPy1/vbMC+Oo11N3XW\n3dRZd1Nn3U3dROpuPbAnpXTCzBdHrWQmb9rJ92/uaR3W3/RYf9Nj/U2P9Tc9xfpbj5mnI5l5Oor1\nNz3W3/RYf9Nj/U1PUzNPQxuwIuJ4YDVZl/JtKaWHGrZzzZqIuAUgpfS02S7LXGPdTZ11N3XW3dRZ\nd1PXiXVn7pmcmbppp9E68bPcSNbf9Fh/02P9TY/1Nz3tXH9mnskx83QG6296rL/psf6mx/qbnmbX\n37SfgRURq4ALgdeS3WFSXLYF+CzwZymlndM9liRJ0mwy90xdSmkj8ObZLockSToyM8/UmXkkSWqc\n0nQ2joiTgZuBdwFrgWFgK7At/3kd8DvAzRFx4vSKKkmSNHvMPZIkqROYeSRJUquYcgNWRJTI7rg5\nDvg28AJgUUrpqJTSOmAx2YMrryMbC/Ez0y6tJEnSLDD3SJKkTmDmkSRJrWQ6PbDOB84APg88P6V0\nTUppoLIwpdSfUvomcC5wJXBmRJw3rdJKkiTNDnOPJEnqBGYeSZLUMqbTgPVKoB94R0op1VspX/a/\ngUHgVdM4niRJ0mwx90iSpE5g5pEkSS0jxskj428YcQuwK6X0ggmu/01gaUrp6VM6oCRJ0iwx90iS\npE5g5pEkSa1kOj2wjgVun8T6twPHT+N4kiRJs8XcI0mSOoGZR5IktYzpNGAtAR6bxPqPkT3sU5Ik\naa4x90iSpE5g5pEkSS1jOg1YvcDwJNYv59tIkiTNNeYeSZLUCcw8kiSpZUynAQtgag/QkiRJmnvM\nPZIkqROYeSRJUkuIlKaWSyKizBRCTUqpa0oHlCRJmiXmHkmS1AnMPJIkqZV0T3P7mOT63sUjSZLm\nKnOPJEnqBGYeSZLUEqbcA0uSJEmSJEmSJEmaCdN9BpbaWEQcExH/HBGPRkR/RGyIiI9GxPLZLttU\nRcSrIuJjEfGdiNgTESkiPnOEbc6KiK9GxM6IOBgRP46Id0dE3SESIuKNEXFTROyLiN0RcW1EvHSc\n9RdExCURcVdEHIqIrRHx+Yg4fZxtmvb3iYiVEfGWiPhiRNyb18PuiPhuRPx6RNT8LrHuRo73FxFx\ndURszOthZ0T8ICIuioiVdbax7mof/3/l/25TRLylzjodX3f5vlOdaXOdbTq+3tTZ/KxN3VS+czpN\nNCmDtqvJ1F9ErB/n85gi4opml3+2RROzfDuabP35GVSrM/NMnZlnYsw902PumTozz/S0auaxB5Zq\nioiTgBuANcBVwJ3AM4BzgLuAZ6WUdsxeCacmIn4I/AywD3gYOA34bErpf9VZ/+XAF4BDwOeAncDL\ngFOBK1NKr66xzV8B7833fyXQC1wArADekVL6eNX684CrgWcBNwPXAMcCrwYGgHNTSjdWbdPUv09E\nvA34O2AT8C3gIWAt8ApgKVkdvToVvlCsuzHHHABuBX4KbAUWAs8EzgAeBZ6ZUtpYWN+6qyEijgV+\nAnQBi4C3ppQ+WbWOdZcdbwOwDPhojcX7Ukp/VbW+9aaO5mdteib7ndOJmpFB29lk6i8i1gMPAD8C\nvlRjd7ellK6cscK2oGZl+XY12frzM6hWZuaZHjPPxJh7psfcM3Vmnulp2cyTUnJyOmwCvk42jvU7\nquZfms+/bLbLOMX3dQ5wMtmY3mfn7+UzddZdQtbY0A+cUZg/nyzwJeCCqm3OyuffCywvzF8P7CD7\nMlxftc0f5Nv8O1AqzH95Pv/24vzZ+PsA55J9eVeXYx3Zl1kCXmnd1a2/+XXm/0l+zE9Yd0eswwC+\nCdwHfDg/3luq1rHuRve7AdgwwXWtN6eOn/ysTbv+Jvyd06kTM5xB232aZP2tz5dfPtvlbpWJJmT5\ndp6mUH9+Bp1adjLzTLv+zDwTqydzT/Pqz3PO2Pow8zS3/pry+Zv1inFqvQk4Kf/wPVDjA7uY7A6A\n/cDC2S7rNN/nkU4Cv5Yv/9cay87Nl327av6n8vlvrrHNB/NllxTmBfBgPv+EGttcly87p1X/PsCF\neXk+Zt1Nuu5+Ji/Pf1t3R6yrdwFl4LnAxdRuwLLuRve7gYk3YFlvTh09+VlrSB1O+DvHaWYyaCdN\nE6i/9XghZzL12ZAs36lTnfrzM+jUkpOZpyF1aOaZfJ2Ze2a2/jznTLwuzTyNr7+mfP58BpZqOSd/\n/UZKqVxckFLaC1wP9JENgdbOzs1f/6vGsuuAA8BZ+bBYE9nma1XrQBYgjwPuTik9MMFtWu3vM5i/\nDhXmWXcT87L89ceFedZdlciej/TnwN+klK4bZ1Xrbqx5kT0z7MKIeFdEnFNn/GbrTZ3Oz1pjTPQ7\nR0c2le9lHe5xEfGb+WfyNyPiKbNdoBbVqCzfqWrVX4WfQbUaM09jmHkay3NOY3jOOTIzz/TMWuax\nAUu1nJq/3l1n+T356ylNKMtsqlsPKaUhsruWuoETASJiIXA02bjHm2rsr1a9TaWuW+bvExHdwBvy\nX4tf9tZdDRHxvoi4OCI+EhHfAf6YrPHqzydSxk6su/wz9mmyrsoXHmF1626sdWR19ydkY7RfA9wT\nEc+rWs96U6fzs9YYE/3O0ZFN6ntZdZ0HXEb2mbwM+FFEfCsijpvdYrWORmX5TjVO/VX4GVSrMfM0\nhpmnsTznNIbnnHGYeaZntjOPDViqZWn+urvO8sr8ZU0oy2yabD1Mpd6atc1M+XPgScBXU0pfL8y3\n7mp7H3AR8G7g2WRf+uenlLYV1rHuxvoA8HPAm1JKB4+wrnU36l+A55P952oh8GTg78m6d38tIn5m\nGuVr53pTZ/KzNn2T+c7RkfmZnJ4DZDcJPQ1Ynk/PI3sQ9dnA1fnNGGpclu9U9erPz6Balf+2p8/M\n03h+LqfHc87EmHmmZ1Yzjw1YkqYkIt4JvBe4E3j9LBdnTkgprUspBVnYfQXZHRw/iIinzm7JWlNE\nnEnW6+qvU0rfm+3yzCUppUtSSteklLaklA6klG5LKb2N7OHMC8ieIyZJDeF3jlpJSmlrSukDKaVb\nU0qP5dN1wPnAjcDjgbfMbilnn1l+esarPz+DUvsy86jVeM45MjPP9LRC5rEBS7VUWpeX1llemf9Y\nE8oymyZbD1Opt2Zt01AR8b+BvwF+CpyTUtpZtYp1N4487H6R7At9JfCpwmLrjpHuyZ8i68b9/glu\nZt0d2WX563ML86w3dTo/azOn1neOjszP5AzIh4H5ZP5rR38mZyDLd5QJ1F9NfgbVAvy3PXPMPFPn\n53IGeM7JmHmmp1Uyjw1YquWu/LXeuMcn56/1xk1uF3XrIb+4fgLZg+vuB0gp7QceARZFxFE19ler\n3qZS17P694mIdwMfA24j+/LaXGM1624CUkoPkp0EnhgRq/LZ1l1mUX6s04FDEZEqE9kwjAD/mM/7\n6JHK2GF1N57KcJXFLtzWmzqdn7WZU+s7R0c2qe9lTUrHfyZnIst3kgnW33g6/jOoWWXmmTn+2546\nzzkzp6M/l2ae6WmlzGMDlmr5Vv56fkSM+YxExGLgWWRjXH6/2QVrsmvy1xfWWPZcoA+4IaXUP8Ft\nXlS1DsB9wEPAKRFxwgS3mbW/T0T8HvAR4IdkX15b66xq3U3c4/LX4fzVusv0A/9UZ/pBvs53898r\nwwtad0f2zPy1GL6sN3U6P2szp9Z3jo5sKt/LmpiO/kzOcJZve5Oov/F09GdQs87MM3P8tz11nnNm\nTsd+Ls0809NymSel5OR02AR8HUjAO6rmX5rPv2y2y9iA93h2/l4+U2f5ErLW4n7gjML8+cAN+bYX\nVG1zVj7/XmB5Yf56YAdwCFhftc0f5Nv8O1AqzH95Pv/24vzZ+vuQDeOWgJuBFUdY17ob3e8pwNIa\n80vAn+THvN66m1SdXpwf7y1+7mrWz+nAwhrz1wP35Me70Hpzchqd/KxNq+4m9Z3jNDMZtJOmCdTf\nU6vPJ/n85+fnpwScNdvvYxbqbUazfLtPk6w/P4NOLTuZeaZVd2aeqdWbuWdm689zzuHv3czTvPpr\nyucv8p1KY0TESWT/UNcAVwF3AGcC55B1Jz8rpbRj9ko4NRHxS8Av5b+uA36BrCX4O/m87Sml91Wt\nfyXZP7orgJ3ALwKn5vN/JVX9I4qIvwZ+B3g4X6cXeA3Zs47ekVL6eNX688ha/M8i+3K4GjgOeDUw\nAJybUrqxapum/n0i4o3A5WS9hD7G6BixRRtSSpcXtrHuGOly+2dkvYUeILu4vxZ4HnAisBl4fkrp\np4VtrLtxRMTFZMMIvjWl9MmqZR1fd3n9vBe4DngQ2AucBLyELIR9FfjllNJAYZuOrzd1Nj9rUzeV\n75xO1IwM2s4mU38RcS3ZMFg3kJ2jAJ4CnJv//P6U0oeaUOyW0aws364mW39+BtXKzDxTZ+aZOHPP\n9Jh7ps7MMz0tm3lmu1XPqXUn4FjgX4BNZBcXHwQ+SuFu+7k2Mdpzo960ocY2zyILIruAg8BPgPcA\nXeMc503A/wD7yULNt4GXjrN+H/BBsrt2+sla//8deEIr/H0mUG8JuNa6q3msJwEfJ+t2u51s7Nzd\n+Xu8mDp3M1h3E/o8vqXO8o6uO7LG0X8D7iR70OhgXr7/Bt4A2c0r1puT09jJz9qU621K3zmdNtGk\nDNqu02TqD/h14CvABmBffr55CPgc8JzZfi8tWn8Ny/LtOE22/vwMOrX6ZOaZcr2ZeSZeV+aeJtWf\n55xJ152Zp4H116zPnz2wJEmSJEmSJEmS1FJKR15FkiRJkiRJkiRJah4bsCRJkiRJkiRJktRSbMCS\nJEmSJEmSJElSS7EBS5IkSZIkSZIkSS3FBixJkiRJkiRJkiS1FBuwJEmSJEmSJEmS1FJswJIkSZIk\nSZIkSVJLsQFLkiRJkiRJkiRJLcUGLEmSJEmSJEmSJLUUG7AkSZIkSZIkSZLUUmzAkiRJkiRJkiRJ\nUkuxAUuSOlhEXBsRabbLIUmSNNPMPZIkqROYedRObMCS2lxEpKqpPyK2RcStEfHJiHhRRHTNdjnb\nVV7n105hu8vzbd/U+FJJktSezD2zy9wjSVJzmHlml5lHap7u2S6ApKa5JH/tApYBTwReD/w6cHNE\nvC6ldPdsFU6SJKmBzD2SJKkTmHkktTUbsKQOkVK6uHpeRKwFPga8GvhmRJyRUtra7LJJkiQ1krlH\nkiR1AjOPpHbnEIJSB0spbQEuAK4FjgUurF4nIk6OiE9FxCMRMRARj+a/n1xrnxHRFRFvi4jrI2J3\nRByMiHvzLuwnF9ardJteX2MfZ+fLLq6af20+vyciPhAR90XEoYi4KyLeWljvbRHxk/zYD0fEJRFR\n8/suIs6MiCsjYnP+/jZGxN9HxONqrFs5fndEXBgR9+Td9DdGxF9ERG9h3TcVxht+XlXX/our9z1R\nkylD1XYXRMQteZ1sjYhP13qPVdv8QkR8NSK258e4LyI+HBHLqtZ7T16mL9TYxwsiYjj/eyyY6vuW\nJGm6zD3mniMcy9wjSWoLZh4zzxGOZebRnGIPLKnDpZTKEfEh4GzgtRHxnpRSAoiIpwPfBBYDXwZ+\nCpwG/C/g5RHxgpTS/1T2lZ9QvwKcB2wE/h+wB1gP/DLwXeCeBhT7CuBM4KvAIPAq4B8iYhB4CvDG\nvBxXA78IfAA4APxFcScR8WvAPwD9+fvbCJwMvAV4WUQ8M6X0UI3j/z/gOcDX8vf3YuD/AGuAN+fr\n/JCsK/9FwIPA5YXtr53qG59kGYAsdACXAo8Bn8pffwG4Adhda+cRcRFwMbCTrC63ktXt+4AXR8TP\np5T2AKSUPhIR5wKviIjfSil9It/HOuAzwCHgV1JKBxvwviVJmjJzj7mn1s7NPZKkdmPmMfPU2rmZ\nR3NSSsnJyamNJyBl/9THXWceWThIwAn5vADuyOe9rmr91+Tz7wRKhfl/ms//MjCvxjFWF36/PF93\nfY3ynJ0vu7hq/rX5/P8BlhXmnwgMALuAB4CjC8uWAduBbUB3Yf4p+Tb3FtfPlz0fGAa+WOf4twAr\nCvMX5vsZBtbVqP9rp/B3q9TPm6ZTBrJAOUAWTtYX5peAL9T6fADn5PNvKNZzvuxN+bKPVM1fSRYK\nDwI/k+//m/m6b57tfwdOTk5OTp0xmXvMPeYeJycnJ6dOmMw8Zh4zj1OnTA4hKImUUj+wI/91df56\nFtkdON9LKX22av3Pkd1hcyrwbMi6kwO/RXZSe1u+zzHHSClta1CRfz+l9Fhh3/fn5VkG/HFK6ZHC\nsseA/wBWAUcX9vF2oAd4V3H9fJuryYLZyyJicY3j/15KaWdh/f3AZ8lO5GdM871N1ETL8Dqy9/mx\nlNKGwvpl4HeBco19vzN/fWuxnvPtLie74+h1VfN3AK/Nj/U54ENk4fCzKaV/mfzbkyRpZph7zD1V\nzD2SpLZk5jHzVDHzaE5yCEFJFZG/pvz1qfnrNXXWv4Ys0PwccB1ZAFoK3JhSenSmCpm7uca8yjFv\nqbGsElqOIeviDfDz+evz8u7z1dYAXWR371Tvs9bxN+avy2sVeAZMtAyVv+O3q1dOKd0fERuB46sW\n/TzZXVqvjohX1zhOL7A6IlbmYaayv+/m3dE/BPwB2RACb5vIm5EkqcnMPWOZe8w9kqQJdfofAAAE\n5klEQVT2ZOYZy8xj5tEcYwOWJCJiPrAi/7Vy58zS/HVTnc0q85dVvT5SY92GSinVGst3KH8db1lP\nYd7K/PV3j3C4RTWO/1iN9SrH6DrC/hpiEmWo/B231NnVZg4PNSvJzg8XHaEYixi9m6vi/wM+SHZ3\n0CdTSvuOsA9JkprK3DMuc0995h5J0pxi5hmXmac+M49aig1YkiC7u6Yb2FLoelwJB+vqbHNU1XqV\nk+zRNdatpdKdudb30LIa8xqtUu6lKX9AZZuqvM+1wO01ltf6++4mG+96RY1ldeXh+N/yX3cBH4iI\nq1JKd01mP5IkzTBzT/sy90iSNMrM077MPOoYPgNL6nARUQL+MP/1/xUW/SB/PbvOpufkr7fmr3eS\nBZunRMTjJnDoXfnrsTWWNWNs4e/nr8+Z4eOUadKdOnVU/j7Pq14QESdSu/6/DyyPiCdO8liXkj3U\n88+AC4A+4HMRMW+S+5EkaUaYe8w9NbYx90iS2o6Zx8xTYxszj+YkG7CkDhYRa4AryILLQ8CfFhZf\nD9wFPDsiXlW13avIwsDdZA/UJKU0DHwCWABcVn0ii4jeiFhdmHVT/vrWqvWeDLxrWm9sYj5ONvbv\nRyLilOqFeXkbEXh2UDs4NMtnyd7nOyJifWVmHmY/TO3zwEfy13+sFVAjYmFEPLNq3ivJHpZ6PXBR\nSukbwF+ShZyPVO9DkqRmM/eYezD3SJI6gJnHzIOZR23EIQSlDhERF+c/lsi6bT+RrDt5L1nAeF1K\naXtl/ZRSiog3Av9NdmfFVWR33pwK/BKwF3hDSqnMqEuAM4GXAXdHxFfy9Y4Fzicbg/jyfN2ryB78\n+NqIOAa4ETgOeHm+7Fca+PYPk1K6MyJ+Dfhn4PaI+C+ykNaTl+M5ZGNEnzbNQ10NXBAR/0F2h8wg\ncF1K6bpp7ndCUkobIuL3gb8GfhARnyPrNv4LZJ+DHwNPqdrm6nybPwPuiYivAg+QjYN8PNkdPt8F\nXgiQh6VPkt1p9at5wAX4I+C5wNsj4uqU0hdm8K1KkjTC3DOWucfcI0lqT2aescw8Zh61HxuwpM5R\neUjjAFnQeBD4FPAF4BtV4QSAlNKNEfF0spPTC8jCynaysW//uHq825TSQES8EHgb8AbgjUAAjwJf\nJL+DJ1/3UEQ8H/gr4Dzg6cBtwK8CO5nhUJOX4TMR8SPgvWTd5M8H9uflvRL4XAMO8y4gAc8HXkwW\nKi8BmhJqAFJKl0bEJrJQ+Sayv//Xgf/D2KEEitv8RURcD7yTLPy+nCwMPQL8Q2W7iOghu7NrGfDK\nlNJDhX0MRcRrgR8Cn4yIWwrjbkuSNJPMPVXMPeYeSVJbMvNUMfOYedReIqU022WQJEmSJEmSJEmS\nRvgMLEmSJEmSJEmSJLUUG7AkSZIkSZIkSZLUUmzAkiRJkiRJkiRJUkuxAUuSJEmSJEmSJEktxQYs\nSZIkSZIkSZIktRQbsCRJkiRJkiRJktRSbMCSJEmSJEmSJElSS7EBS5IkSZIkSZIkSS3FBixJkiRJ\nkiRJkiS1FBuwJEmSJEmSJEmS1FJswJIkSZIkSZIkSVJLsQFLkiRJkiRJkiRJLcUGLEmSJEmSJEmS\nJLUUG7AkSZIkSZIkSZLUUmzAkiRJkiRJkiRJUkuxAUuSJEmSJEmSJEktxQYsSZIkSZIkSZIktZT/\nH5tma4FLRXLbAAAAAElFTkSuQmCC\n",
            "text/plain": "<matplotlib.figure.Figure at 0x7fdf5238f438>"
          },
          "metadata": {
            "image/png": {
              "height": 135,
              "width": 856
            }
          },
          "output_type": "display_data"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:07:01.022613Z",
          "start_time": "2017-03-29T05:06:52.204330-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": true
        },
        "scrolled": false,
        "trusted": false
      },
      "cell_type": "code",
      "source": "from tqdm import tqdm\nfrom IPython.display import HTML\n\ndef get_chars():\n    step = 0\n    freq = {}\n    keys = []\n    for file_name, source in tqdm(data.source_gen()):\n\n        for char in source:\n            try:\n                freq[char] += 1\n            except KeyError:\n                freq[char] = 1\n                keys.append(char)\n    \n    return keys, freq\nks, freqs = get_chars()\nHTML(\"<pre style='border: 1px solid #919191;border-radius:10px;background-color:#e4e4e4;padding:10px'>\" + \"\".join(ks) + \"</pre>\")",
      "execution_count": 5,
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "48716it [00:08, 5535.12it/s]\n"
        },
        {
          "data": {
            "text/html": "<pre style='border: 1px solid #919191;border-radius:10px;background-color:#e4e4e4;padding:10px'>\n Pewp.[ViaSlshdot]W'myuHCr,nDkvbgMxKjTc?I!YUz10-fJqB:FOEQ85…AN3\"R%GL\t29746&X Z()$@—/_;#ûé+^¥£–*=ö€üñ\\àµ~âα’“”<>×±çúëè|íÏÃ¸￡°áóəê·Президнт®Éμ₩²äïموقعزارةلأتصã{}ø™`ʼ•ΩôÔÜИкаоğ♥≠Û″ōгî�å«»알려드립니다일부언론에서보도한갤럭시Ⅲ후속모델출예정이라는추측은단순루머뿐사실아닙해진게무것없데왜런소문퍼질까요ㅜāìウニカÖºæ‐★ćē‘¯ツ¾Łš§′👍©ㅡ※´½¹¢ış‑℠、Ⓞ●―㎜†＋ │‡傅珮倫─＊−Ò　ﬁ₹Ǿ ÀΦﬂⅡ¼○̈□討鬼閃乱％‖π³⁴‚意外Î≥≦℗■ū¡〈만␣̃÷◦（）¬‮‭‟„⁰˚▪Žč・？さんマジでしょうか！◎－～ÂÁ♦〈⁵⁶⁷\fⓇ∆电子书包数字公文̳ｘ１痴匠⎯：∑①②③④⑤⑥⑦⑧⑨⑩⑪⑫⑬⑭⑮⑯›‎◇年月日報道関係者各位略称はタッチパネルを搭載好きなスイ使えるハペク脱着式モバノートソコン「」本向けに発売すこと表また。土より順次開始卓越処理能力高い応答性備利用シ合わせて液晶画面キボド体分離の型取最薄部軽約対フブレテプロセサビメも十ォ揮装速大容量デ保存両方実現傷強グ社化ガラ品オィム暗場所入やミがれどあらゆリ機満製名予定／価格下旬以降想円主仕様詳細つ添付ァご参照くだ番ワ点ア示解像度記憶置無線出ェ駆動時間ズ幅奥行質統属ダュ証書専特徴頑丈ヘ加工施金手持ち運べ精採上左右広視野角輝誇並み硬汚拭圧倒的ナ揃ほ可充技術世界有共同革新内蔵ピ原音限近提供明自調節電源ケ接続簡単ャじ比率操作っ連携毎業快適お客問先通話料号負担受曜祝祭末盆休当吉ヨ資写真等第ザ市荷台め販総エそ門評得ざ送々賞達成測法基状況設変米査会株全国元英語版⅓･➢≤ </pre>",
            "text/plain": "<IPython.core.display.HTML object>"
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ]
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "### Character Frequency\n\nThere is a long tail of characters that appears only once, mostly emoji's and CJK characters. \n\nIn this project, we will just manually specify lowercase alphabets and numbers as our input domain (plus space and carriage return)."
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:07:01.320936Z",
          "start_time": "2017-03-29T05:07:01.023840-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": true
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "import math, numpy as np, matplotlib.pyplot as plt\n\norder = np.argsort([freqs[k] for k in ks])[::-1]\nchars_ordered = \"\".join(np.array([k for k in ks])[order])\nprint(chars_ordered[:140])\n\nplt.title('Frequency of Each Character', fontsize=18)\nplt.plot(np.array([math.log10(freqs[k]) for k in ks])[order])\nplt.ylabel('$\\log_{10}$ Frequency', fontsize=14)\nplt.xlabel('Character Index', fontsize=14)\nplt.show()",
      "execution_count": 6,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": " etoainsrlhdcupmgfwyb,v.k-\n'STA0P1MICxD2\"BWEGN3OHRLFV5)(4Uzj9:q\t867$X/KJY?;QZ[]&!–®™*%+=•_#·@£\\ €—é¥’{}‐ü|>~α×°à©^�―”ö　“<●…ー℠ó§ñ‟±◦・ッ、：µ‖\n"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwAAAAI7CAYAAABbS/xUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAIABJREFUeJzs3Xe8XHWZ+PHPk94bpABJ6L0TQHCRYgGsi8raGxZE3XXd\nVVdddy2rrr2uP3Wl2FdWxbaiIhaa9N4hEEoCCQmpJCHt5vv745w7d+7klplkzpx773zer9d5zZmZ\nc77z3Jm5Z85zvi1SSkiSJElqD8PKDkCSJElS65gASJIkSW3EBECSJElqIyYAkiRJUhsxAZAkSZLa\niAmAJEmS1EZMACRJkqQ2YgIgSZIktRETAEmSJKmNmABIkiRJbcQEQJIkSWojJgCSJElSGzEBkCRJ\nktqICYAktZmIGBsR74qISyNicURsjIhUtYwpO8bBKCLOqXoP7y07niJFxJia78xxZcckqX4jyg5A\n7SciPgZ8dDt3/2pK6T1NDEdqKxGxC/Bn4IACyr4QeOUOFLExpWTyUZCICOBY4HnAycAcYGdgErAG\nWAHcCdwA/F9K6Y5yIpVUNBMASWov59L95H8hMB/YXPVYR0sjUuEi4iXAx4EjetlkWr7sA5wBfCoi\nbgc+C/w4pZRaEqh2SER8BvhAfveSlNLpZcajgcsEQAPBJQ1se09hUUhDXETsBryw6qEPpJQ+V9DL\nrQOuanCfTUUE0s4iYhzwXeDvenh6GfA42ZX/ScAsYLeq5w8DfgScCbys0EAltZQJgErnFQqpZY6p\nWt8AfKHA11rk/3a5ImIq8EfgqKqHVwNfAi5KKd3Vwz6zgRcA76CrtmDXgkOV1GJ2Apak9rFT1fpj\nKaWtpUWiQuXt/X9A95P/PwB7pZT+o6eTf4CU0qKU0rfz/c4kqyGQNMSYAEhS+xhZtb6ltCjUCv9E\n9+Ze/we8MKW0op6dU+YiskTgsuaHJ6lMJgAa9CLigJrh6Gblj+8cEe+OiL9ExCNVQx2+qo+yxkfE\nWyPiooh4ICLWRMTT+f7/FxFv354hEiPiRRHxk4h4OCI2RMSSiLguIj5UFW9dw+pFxGeqtvl9na9/\netU+GxqIe05EfCAi/hQRj+bvxZqIuD8ivhcRL66znB6HR4yI3SLi3yPixohYlr83CyPiZ3mnxYZF\nxD4R8eGI+HMe8/q83MURcVlE/GdEPLNmn8j/ps4Y/6vB1/x01b6LImL49sTeS9n7RsTHI+KaqiE7\nn4yI2yLiq7V/Sw/7V74vwDerntq/5vvW7f9nIIqIXSPirIj4TkTckL8PmyJibf4/+puI+Oe86cv2\nlD8iIv4uIs6PiLvy8jfn3/m7IuLH+fGhofKL+J7383rjgA9VPbQEeHNKqeGkL6X0RErpgw289siI\neFVEXJwf7zq/r1fnx7uJDZS1d0S8IyJ+FBG3RsSKqs9jQf7+vT0ixtZZXo/Hwfw496E8xsfy1+jx\nGBwRh0TEP0XET/PvxKp8+5X5MeRHEfHaiNiuJtYRMSUi3hkRP4/sN2h1Xv7y/Dv/rYg4IyJG1ey3\nJP8f/0DVw6f18j/e5+9g1XtS5LF//4j4ZP4/sSQiOgb68WfISSm5uLR0AT4GpM6lCeUdUF0eWUe2\nFwBP1Dzeubyql3LeRPZD2dM+1cujwKl1xjYV+E0/5a0AXgKMqXn8uF7K/EzVNr+vM47Tq/bZUMf2\no4DPk7UT7+/9uBbYs5/yzqna/t78sdeSDT3YV9kXAaPr/BsnAxeQXdnuL+YEfKtm/3+uem4VMK7O\n1x1Z8735WJP+T0YCXyYbnae/v+WXwM51fF/qWWbtQMwX1n7OzVrIOqN21Pk3rAHe0mD5LwEeqLP8\np4E9yvie1/m3/H1N+e9t5meRv8Y2xytgb+D6fv7Wx4HD6yj/yga+s0vJajf6K3Ob4yDZcf+pXso9\nrmrficDdDcS0AHhGA+9nAB8kO/bUU/69NfvX89tVvfT2O1josT//Oz9A1uG/qccfl8YWOwFrKDqR\n7GSh8/t9P9mPzmRg/552iIgvkp0AVnsceIjshHIvsjGzyW8vjojXpZT+t7cg8itdl9C942Ui+xFZ\nSpaoHEiWJPwceFF9f16xImIS8CuyccI7JbKhIh8HRpMlXZ1XQZ8BXBMRp6SU6hqlKSLeSDYyCWQn\nuHeRJUKzgIOqNn0Z2VXrN/dT3h7A79h2bPvHgUeAjWTjne9PVzOYKTXbfgf4JDCW7Lvyyvyx/pwB\nzMzXO8iG2dwhkdUy/Qo4teapB4BFZO/9IUBnTcPfAgdGxHNSSotq9rmPrpG25tD1/vY2Ss/GHYu+\nMIfTVWudyBLxxcBaYAKwH9kwlpCdrJ0XEZNSSl/ur+CI+BDwKbKTk06byE5YngTGAXvS9TmPyZf+\nym3q97wBL61a30h93+MdtStZItPZYfhhsv+90WSjCY3LH98FuDQiDkopPdlHeUdWrW8lO6FeCqwn\n+/88gOxzBpgO/DoiXpGyZkt1iYjX0PXebCUbZW4pWV+Z2mPJaLLjdafNwINk34+NZN+9A8iOH5B9\nXy6PiJNTStf2E8cY4Mdkx5Jqq8mOu2vo+v2akD9Xe/z6cx7Dfvlrk8d2Uy8vu7iHOAo/9pPVTH0q\nX99CNu/ESrL/rabPTaI+lJ2BuLTfQvE1AJ1X235CzdUJsh+MmTWPvadm/58Bh/bwOsfS/erWWmD/\nPuI6t6bcXwK712yzN/B7uq5i9Xj1qWafQmsAgF9UbbsZ+E9qrsqQnXieSferTncAY3ops/oq0Bqy\nH8zNwEeAyTXb7k/2o1X9XszrI96xwG01218EHNbDtmPIEq2fAT/q4fkLqsq4ts739o/Vn3GT/ke+\nWvP3XAYcXLPNTOC8mu0uB4b1Ue42V6ibuVBsDcBNZJ1azwAm9bLNs4C/VsWwqfZ962Gf19a8h0uA\ns4EJPWy7N9lV2keBA1r5PW/gfRpFdpLcWebVzf6c89eprQFYlt9eSs3xM/8f/XzN9v/VT/mLyZKi\n04CxPTw/nKym966qMlcB0/sos/o42EF2DE/A19n2d2FnYErN/SfJRs46ARjVy3v/6vz70fk6D/e0\nbc1+tf/Ht+SxjqjZbhjZ79BXgfm9lNXw70PVvkUf+9fm7/tmsvOA2v+J2TSxJsyln8+77ABc2m+h\n+AQgUdO8o4999yX7ke7c79/62X4M3U8wLuplu3lkV5Q6t7sQiF62HU7WQa/2b2h5AgC8sXo74Ln9\nlLtn/qPYuc8/9LLdOTV/21bgjD7KnU52pbRz+//Xx7afqyn7X+t8T3o6wTu6pqw+myqQTZpU/Tk/\nvwnf58NryrwEGNnA399r0xcGdwKwzefVy3YjyUa76Yzj/H6+Z2urtp0PzK3zNbY5USnye97A+3RU\nTQxfafbnnL9ObQKQgJ8Cw/vY54dV266kjxPjBj7vScDtVeX+ex/bnt5DzB+s83WG08tJbg/b7kp2\nxbzzNV7fx7YvqInn1z19t+p9f9jOBIDWHfsTvTQ/cmntYidgla6PTkq1y3vqLPJxtm3O05v3kl21\nAbg0pfTJvjZOKW0gq6bvHD7xjIiY08Om76SrOcFK4J0pPxr2UGYH8DayE5HSRERn28xO/5FS+mNf\n+6SUHqJ7Z8N/qPPlvpNS+mUf5S6jq+kEZFd2t5F3xnxH1UO/SCn9Zz0BpJS2eb9TSjcCN1Q99PZ+\ninkbXZ/zQzQ2qV1v3l1V5lrgrJTS5j62/1eyZmWd/rEJMTRDb52M+1rO6a2wnj6vXrbbTPZ/3emM\n/Lvdk/cA4/P1LcArU0qP1vMaKaV6mko15XveoBk19xc2ocx6rCRLPvuaRbp60rkpZM3YetTA570G\n+HDVQ41MWHYz2UzH9bxOR378r2fbx4FP1xlTdeyPAK+t57tV7/tTjxYf+3+ZUrqwwRBVABMADUU/\nSCmt72+jfJSG11Y9VNekSCml+4Dr8rvDgFNqyg26t8H9Uepn6L2U0hKyq2dlegZdbVw3kFWL1+N/\nyKp0AfaNiLl17PP/6tjm8qr1A3s5iftbutrEQtbUYkd9o2r9tRExvqeN8lE4zqp66Ny0g+Pq539j\ndTvgC/OTiV6lbGSXr1Y9dGhE7LMjcQx2KaU7yPo4QNYuurf3o/r//1cppZubHEqzvueNmFZzf9UO\nllevH+Yn471KKd1eE8/BTXrta6rWD406RwUi+5/t8cJME1THdGxPG+R9l6pH8fpCSumpguLpSyuP\n/f/dYGwqiJ2ANRDUe9X04Tq3u6LO7ebRdfK4hcbGur4NOD5fPxr4ftVz+9LVSQqyzqn1uJjuJ5St\ndlLV+vX9/Zh3Simti4gH6erAdTRZG9jerANuraPo6s6sI8g+q9ofx+qY70kp3VlHuf25EPgi2YnU\nJLI2vef1sN1LyZpwQPYjeEETXru6IytkTcPq8Su6/7AeT9ZhuEy9dTLuS79X3wEi4miyv/Egsvdr\nAl0dojtVz3kwm6x5T3UZuwO7Vz30kwZj7U8zv+eNqO2c3KpO3X+tc7tFdHVgre3Iuo2IGAb8DdlJ\n9AH5PuPpfgGzen04WSfrh+qIpd7fitqYRpJd+JlHdryflMdUnbxVXzjYJSKG9XCB4KSa+2VdBGrV\nsX8rjR8TVBATAJUupXR6k4tcUOd2h1Wtd5CNIlHva1SPJjS95rn9au7fUWeZ9W5XlOr3Y9+oc46B\n3K5V67XvR61ldV4pX1dzfxzbnhhVj8pxYx1l9iultCEivkNXM5K303MCUN086JcppSea8PK1V6pv\nq2enlNITEbGUruYfA6EGYFGz/7cj4uVkHRNr/8f609OJ5oE195vy/anSzO95I1bW3J+8A2U1Ykmd\n21X/veN62yg/8X8rWa3ebg3G0m9ikav3t6IzplHA+8iamO7Uz+bddiX7HGo/m+rv4MImHUO2RyuP\n/aU2dVUXEwANRXVdvaD7AXw02WgT26P2B7Z2kqDldZZT73ZFqX4/dsmX7dHfCcf2XpHsKTurvlq+\ndDvL7ck3yX7kAzg6Io6qbh4SEfvSfai8bzXpdWu/O8sa2HcZXQnAdk2ENZBFxBfo3ra/EaN7eKy2\nqUwzvz/Q3O95I2qbG9b+nUXZnr+3x781b575Q7KheLdHT593rc31tunPYxoP/JZsmOlmxVTU8atR\nrTr21/vbrBawD4CGonrbYffYtns71P4fjaq5v6nOcsoef72o96NI1c0d6v4x709K6UGykWQ61XYG\nPpuuk5f7gb806aVrTxLq/e5A9+9Pw7NVD2QR8Uq6n/wvJBtN7Dlko5FMIBsyMToXsokA+1L7HjXt\n+1OyB2vuH9bjVgPbe+l+8n8P8C9knaTnkDf5qvqs623zX63R/jpfoPvJ//VkHfaPIzthHk82BG9n\nTLU1TD0p5Pi1HVp17N+hPlJqLmsA1M6qO6MtTSnN7HXLxtRe5ZjItlW/PZnUpNfvSW376J5Uvx8/\nSSlt79W3VlpJ1wRtzW7q8A26aoVeExHvSyk9lTcDeFPVdt9uYkfC2g6b9X53oPv3p1UdP1vl36vW\nrwZOq6MpwcR+nu+pqUzZtXA7LKW0KCIWkE1eCFkHz0Ejb1//waqHLgJe3c9IWP191jsa00yypL/T\nV4F/6uf/vp6Yqr+DrWqq1ZPBeOzXDrIGQO2sus3qtLzauRlqrzzu2eNW29qr/02A7leFR/a6VXf1\nNAmpfj+alQwVrXo2yx5ned4Bv6GrQ9sE4DX5+svJJgWC7Kr7d5v4mrXNAPauZ6f8pKl6BI4ymxM0\nVUTMpvtoMe/r7+Q/IqbRR/vyXO1MqM3+/pTpz1XrcyKiGcOLtsoz6GrDn4B/7OfkH7ouAhTleXSd\nL60BPlBH0l9PTNXfwb2a+BvUqMF47NcOMgFQO7u6an0EcEyTyr2F7lWd9V6Bq3e76hqGetv3HlrH\nNtXvx9H5SeVAVz3U3jMiomlNX/IOnNUj63Q2A6q+EvizlFIzrxrfSvfvzjN727DGPLo3PWt2h9Yy\nVY/Uk+g+T0NvTqhjm1vo3uzi5AZiGuhq+6TUO4fKQFD9eT+WUnqsjn3q+bx3RHVMt9c5B0Q9MVUf\nv8bRy3ChDao+ftTbn2QwHvu1g0wA1LbyiUzuq3rorU0q9ymyk4tOr65z19f0vwmQTRbTaf+8SUqv\nehhbvjd/IBsNCbI2oa+qM54yVY9WMYnu47o3w3l01bgcGRGvo5jOv0Dlu1M9Fv3r6tz1jVXrT1Pf\nSfJgUXsyUs9JzVv62yA/ibus6qG3lngFtqlSSjcB1RM5vSwiztze8iKi0VGXdkRDJ5/58e3NBcXS\nqdGYRlPfsehaYHXV/Xf0tmEDqkdZqrdvxGA89msHmQCo3X2+av0NEfGcJpX7nar1Z0XEC/vaOO/k\neFSdZVdf3R0LvKSf7d9CHcMm5lfaflz10GciYtfeth8IUkrXkf2IdmpqzCmlpWRtkDt9u2r9rpRS\nEWNan1u1fkxE9JlARsTBdD/h/VFKqXZoycGseiK0oJ8rqxHxAvr/n+j0lar1PYGPNxbagPZuus8u\nfkFEPLuRAiJiWER8hO7Hs6JVf96zI6K/ppHvAA4vMB7oHtOREdFf+/5PsO2MzNvIk9BvVj302og4\ndTviq1bdrKiu4YAH47FfO84EQO3ue3RdLR0B/DIi+r1yExHjI+KNEXFdL5t8j2ykkk4/iIgemxhF\nxIn0PM58j/Kai+qrxJ/LO6n1VPYZwNfqLZuss2Vnx7RdgcvzSZf6FBG7RMSH8/HzW+1f6Lp6tTNZ\nzH02eYqIQyLiDXWWXz0zcPUVtaJmtPwB3ccnP7e3E7d8xt/f0HWF8mngcwXFVYqU0v1AdTOQL0dE\nj+O85ydP/9tA2ZfQfbSnf42IT/TVBCIiRkfE2yOi0bHpWyqldA/dE8OJwO8i4uN1nMB2HjtuJUuK\n6hlEoFmupvuIVt/Mr6hvIyJeT/dZsItSPcrXeOCr+TwFtfFERHwQeH8DZX+RrgQjgF/0V1sTETMi\n4n29PH1T1fouEVFvzfJgPPZrBwyJ6k5pe6WUtkTES4HryCabmQD8MD+IX0TWlGcFWfvqqWQzjx5D\nNvzgWHoZujOltDYi3k52cjYs3/fqiPgB2Yy/y8g6W72YrKp4GNm06vUerD9N16yRewK3RsTXyJKZ\nDrIOxS8Hnp9v8z26NxPpUUrp4Yh4FV0nlfsAN0TEpWQzGt9LNknROLIrXIeRzdJ5PNmPV72zOjdN\nSunKiPgQXSe++wC3RMSv8pgfJmvGM53sSuHzyWas/F+6z+DcW/lXRcTtdB9OcX09+26PlNLTeVOj\ny8k+g/HAHyPiJ2Qz/j5G9n16DtkJXnVn139OKc1nYJjd4IRCnT6cN2Gp9kXgS/n64cAdEfENsu/7\nFmAPspmZO6/8/4Lse9nvVViyZlY30NXO+9/IrsT+mGyoxxVk/+t7kfXJeAnZiC31DPNYqpTST/Im\ngueSDTk5imxirXdHxB/ImkA9RnbiN4lsBt3jgBfQfYKnVsa8NiL+m6wGA+BU4LaI+CZwO9lxZl+y\nZion59t8CzinwJjmR8Sv6fp+nQUcEhHnkTUjHUX22/AGumpy64oppfRkfsy9lGwY4HHATyPiGrLf\noLvJjrmT89c4BXgu2ffyCz2Ud1tE3Akckj/0o4j4KNnwsNUDSHyhugZzMB77tYNSSi4uLV3Ixu9O\nnUsTyjugujxg1naUsSvZlafU4LKhn3LPIjsh76+cq8k69FY/dlw/ZZ9bR7lbyYbUO73emPOyn0l2\nVarR9+P3vZR3TtU29xbxuZI1BdjcQKwXNvD9OKdm3/Nb8H/yPLIf3Hr+lq1ko6U08nfU9Tk0GPOF\n2/GdqV1O76Hc4cD/1bn/tWRXu5dUPfaqfuLehSwJaCTOA8r4nm/n53Is2cWM7fk87gBO7aHMMTXb\n9Xm8qtrv2qp9PtjLNuMa+Dx+RXbhpt9YaPA4WLPvTLKauXpi+kajnyvZMXdpA5/Lkj7KOoYsQehr\n/x7/JxgAx36X1iw2AZKAlNLjZJPMvI7uHXh7czfZFec+q0hTSt8ha7N8cy+bPEXWD+FksqvKjTib\nrKq5t9kV7wFekFL6TIPlklK6muwH7MPAon423wz8FXgf3cfHb6mU0jfJrnr9L31PqraRbEbPRjrw\n1n4nimr+U5FSupRs+Mvv0/vfs5Wss+exKaVWNIUoRUqpg+wK/yfp3q692vL8+WelrDN1I+UvJruS\neQ7bTqRV6+H8dRb2s92AkVK6nuzK9MvJmjw93c8uG4Cfkw0ecHhK6Q/9bN9UKaX1wElk7eN7mwzv\nMbLRjc4gqwUqOqYnyBKp/6X3Ca3mA69JKb1zO8rvPOZ+lm1nc+62KVk/sH/ro6wbyI6F/0F2bF5G\nnZMKDsZjv7ZP5BmapCp5m/pnklWJTyX7gVlFdnJwZ/5j0GiZB5MN9TmD7KT9EeAv+Y8d+RCW1T/M\nx6eUrt2moG3LHUv2Y7kv2ZWwJcAdKaWmDQUZEfuTDTW5M1lTgfVkJ1z356/V36RMLRUR48gSrz3I\nprnfSle8N3a+5w2U9w26Rui4OaU0r3nR1vX648hmId2DrKboKbKrdJenlJ5sZSxly9uvn0zWRGEM\n2VXTh4ArUkpNORGMiH3JrqLOIPufWkt2wn9bSumBZrxGmfI29ceSjVW/M1mNyVqy/5G7yI5x/Y29\n3xIRsRPZ570HWbPlJ8j+j69JJZ3A5B1kTwZmk1+NJxsUoLcLPY2WP4zs4tJBZE0XR5KNFrSA7Pi1\nrBmvU2csg+rYr/qZAEgDxPYmACpWfsL5GF0ze56dUjq3j10kSRrQbAIkSX37B7pO/lcCPyoxFkmS\ndpgJgCT1Ih+i9cNVD3250eZDkiQNNA4DKkm5iDgS+CjZxZE5wBFVTz8KfLmMuCRJaiYTAEnqMhP4\n2x4eXw+82g5vkqShwCZAktSzDrJh8L4PHJUPjydJ0qDnKECSJElSG7EGQJIkSWojJgCSJElSGzEB\nkCRJktqIowDtoIh4iGx67IdLDkWSJElD2x7AmpTSnjtSiAnAjps0duzYaQceeOC0sgORJEnS0HXP\nPffw9NNP73A5JgA77uEDDzxw2k033VR2HJIkSRrC5s2bx8033/zwjpZjHwBJkiSpjZgASJIkSW3E\nBECSJElqIyYAkiRJUhsxAZAkSZLaiAmAJEmS1EZMACRJkqQ2YgIgSZIktRETAEmSJKmNmABIkiRJ\nbcQEQJIkSWojJgCSJElSGzEBkCRJktqICYAkSZLURkwAJEmSpDZiAjCIbenYyur1m0kplR2KJEmS\nBokRZQeg7XPYxy5hzYYtANz7idMZM3J4yRFJkiRpMLAGYJAaMbzro1u7cUuJkUiSJGkwMQEYpMaP\n7rriv3aDCYAkSZLqYwIwSE0YPbKybg2AJEmS6mUCMEhNqK4BMAGQJElSnUwABqkJo7v6b68zAZAk\nSVKdTAAGqfFVCYA1AJIkSaqXCcAgNXGMCYAkSZIaZwIwSI0fZRMgSZIkNW7IJwAR8aaISP0sHWXH\n2agJ1TUADgMqSZKkOrXDTMC3Ah/v5blnAc8Gfte6cJpjQrc+AIMuf5EkSVJJhnwCkFK6lSwJ2EZE\nXJOvfrt1ETVH9wRgc4mRSJIkaTAZ8k2AehMRhwLHAY8BF5ccTsPGdxsG1BoASZIk1adtEwDg7Pz2\n/JTSoDuDru4D8JSdgCVJklSnId8EqCcRMRZ4HdABnFfnPjf18tQBzYqrEU4EJkmSpO3RrjUArwCm\nAL9PKS0sO5jt0a0PgKMASZIkqU5tWQNAV/Of/653h5TSvJ4ez2sGjmpGUI2YNHZkZX3l+k2tfnlJ\nkiQNUm1XAxARBwPPBBYBvy05nO02Y+JoIrL1ZWs3srlja7kBSZIkaVBouwSAQd75t9PI4cOYMXE0\nACnBktUbSo5IkiRJg0FbJQARMQZ4PVnn3/NLDmeH7TJ5bGV9sQmAJEmS6tBWCQDwd8BU4HeDtfNv\ntV2njKmsL179dImRSJIkabBotwSgs/nPoJv5tye7VtUAPLbKBECSJEn9a5sEICIOBE5gkHf+rbbL\nlK4E4HETAEmSJNWhbYYBTSndA0TZcTTTnjuPq6zftnB1iZFIkiRpsGibGoCh6Og9pjF8WJbT3PHY\napausSOwJEmS+mYCMIhNGjOSI+ZMqdx/zXnXsWmL8wFIkiSpdyYAg9zpB8+qrD+wdC1/uHtJidFI\nkiRpoDMBGOTefMKePGvfnSv3z7vyIbY4K7AkSZJ6YQIwyA0fFnzqjEMr929duIovXnp/iRFJkiRp\nIDMBGALm7jSOVx87t3L/R9c+wsYtHSVGJEmSpIHKBGCI+OQZhzBrUjYz8JoNW/jzPUtLjkiSJEkD\nkQnAEDF8WHDmvNmV+1+89H422xdAkiRJNUwAhpA3HL8740cNB7IRga5+cHnJEUmSJGmgMQEYQmZM\nGsNpVcOCPrJ8XYnRSJIkaSAyARhidt9pfGV98WpnBpYkSVJ3JgBDzC5TxlTWF696usRIJEmSNBCZ\nAAwxu0yuSgCsAZAkSVINE4AhpjoBWLLGBECSJEndmQAMMbMmj62sP7J8PSmlEqORJEnSQGMCMMRM\nGD2CiaNHVO5f8NeHywtGkiRJA44JwBD0/EO7hgI994oFJUYiSZKkgcYEYAj6yIsPrqwvWbOBFes2\nlRiNJEmSBhITgCFowugRHLrb5Mr9e5esKTEaSZIkDSQmAEPU/rMmVtbvW/JUiZFIkiRpIDEBGKIO\nqEoA7l1sAiBJkqSMCcAQdfCuXU2AbnhkRYmRSJIkaSAxARiijpw7hVEjso93wbJ1LHVSMEmSJGEC\nMGSNGTmcI+dMqdy/7L5lJUYjSZKkgcIEYAg7cb/plfWv/+UBNndsLTEaSZIkDQQmAEPY647bnclj\nRwLw6Ir13L5oVckRSZIkqWwmAEPY5LEjOWHfnSv3FyxbV2I0kiRJGghMAIa4PXcaX1l/eLkJgCRJ\nUrszARjidt9pXGX94eXrS4xEkiRJA4EJwBC3585dNQD3PL6mxEgkSZI0EJgADHG7VzUBWvDkOn5/\n55ISo5EkSVLZTACGuJ0njGKPqmZAn/rt3aSUSoxIkiRJZTIBGOIigvPfdEzl/sIVT3OXTYEkSZLa\nlglAG9jOAZvEAAAgAElEQVR7+gTOOGLXyv2v/mm+tQCSJEltygSgTfzd0XMq65fe/QRXzH+yxGgk\nSZJUFhOANvE3++zMy4+aXbn/2d/day2AJElSGzIBaCNvfObulfW7F6/h4/93d4nRSJIkqQwmAG3k\n0N0mM3da14hAP7lxIVu3WgsgSZLUTkwA2khE8I3XHlW5v35TB4tWPl1iRJIkSWq1tkoAIuI5EfGL\niFgSERsj4vGIuCQiXlB2bK1yyG6TeebeO1Xu37vEIUElSZLaSdskABHxOeCPwNHAr4EvAhcD04GT\ny4us9fafNbGyfu+Sp0qMRJIkSa02ouwAWiEi3ga8H/gecHZKaVPN8yNLCawkB86aVFm/87HVJUYi\nSZKkVhvyNQARMRr4FPAoPZz8A6SUNrc8sBIdOXdKZf2aBcvZ0rG1xGgkSZLUSkM+AQCeR9bM5+fA\n1oh4YUR8ICL+MSKOLzm2UuwzYwIzJ40G4KkNW7jdWgBJkqS20Q5NgI7JbzcAtwCHVD8ZEVcAZ6aU\nlvVVSETc1MtTB+xwhC0WEZywz3QuunkRANctWMFRc6eWHJUkSZJaoR1qAGbkt+8HEvAsYCJwGPAH\n4ETgp+WEVp7qZkD3LHYkIEmSpHbRDjUAnUnOFuAlKaWH8/t3RMRLgfuAkyLi+JTSNb0VklKa19Pj\nec3AUT09N5AduEtXR2ATAEmSpPbRDjUAq/LbW6pO/gFIKa0HLsnvHtvKoMp2wKyJRGTrC55cx4bN\nHeUGJEmSpJZohwTgvvx2VS/Pr8xvx7YglgFj/OgR7D5tHAAdWxNXzn+y5IgkSZLUCu2QAPyJrO3/\nQRHR09/b2Sn4odaFNDCctN/0yvrnL7mXlFKJ0UiSJKkVhnwCkFJ6BPg/YC7wj9XPRcSpwGlktQO/\nb3105fqH5+zL6BHZV+D+J9aycn1bTYcgSZLUloZ8ApB7F7AQ+FJE/DEiPh8RPwN+C3QAb00ptd1g\n+DtPGM2eO4+v3H9k+boSo5EkSVIrtEUCkFJaBMwDvg7sS1YTcDJZzcDfpJQuKi+6cs3N+wEAPLpi\nfYmRSJIkqRXaYRhQAPKJvv4hX5TbfaeuBOCR5SYAkiRJQ11b1ACod3N36moCZA2AJEnS0GcC0OZ2\nn1ZdA2AfAEmSpKHOBKDNVXcCfnCZCYAkSdJQZwLQ5nabMpZxo4YDsGLdJpav3VhyRJIkSSqSCUCb\nGzYs2GfGhMr9+UvXlhiNJEmSimYCoG4JwH1LnioxEkmSJBXNBEDsP3NiZf1nNy2iY2sqMRpJkiQV\nyQRAPP+QXRg1PPsq3PHYan5/55KSI5IkSVJRTADE3J3G8ZZn7Vm5f+ENj5YYjSRJkopkAiAAXvuM\nuURk61c98CTLnnI0IEmSpKHIBEAAzJ46jnlzpwKQEtz86MqSI5IkSVIRCkkAIuIrEXFIEWWrOPN2\nn1pZv/kREwBJkqShqKgagHcDt0XE9RHx9oiYVNDrqImOnNuVAFz70IoSI5EkSVJRikoAOoAA5gHf\nABZHxPci4qSCXk9NMG/3qZV+ALctXMVf7ltabkCSJElquqISgNnAh4D7yRKBscDrgD9HxPyI+FBE\n7FrQa2s7TZ84mpcesVvl/vlXPlRiNJIkSSpCIQlASumJlNJnU0oHAicA3wHWkSUDewOfBB6JiN9E\nxEsjYkQRcahx7z1t/8r6VQ88yYp1m0qMRpIkSc1W+ChAKaWrU0pvAWYBbwGuIksEhgPPB34GPBYR\nn4+Ig4qOR33bbcpYDtqlq8vGUZ+4lA2bO0qMSJIkSc3UsmFAU0rrU0rfSSmdCOwHfBZYTJYMTAf+\nGbgjIq6JiLdExIRWxabunn/IrG73r12wvKRIJEmS1GylzAOQUnogpfQhYC7wUmAJWSIQwLHAt4HH\nI+KbEbFXGTG2s+pZgQEWrlhfUiSSJElqttImAsub+3ye7GR/JpDypxJZIjABOBu4JyI+UkqQbWrc\nqBG879T9KvcfNQGQJEkaMlqaAETExIg4OyKuBe4A3kPW/CeA24G/B3Yi6zj8XWATMBL4aES8uZWx\ntrs508ZV1heueLrESCRJktRMLUkAIuKkiPg+WZv/b5I18wlgLVkNwDEppSNTSt9IKa3OOw6/GdgX\nuCnf9h9aEasy1QmANQCSJElDR2HDb0bEbsCb8qWzHX8+zRTXAecCF6aUej27TCktiogPAX8gSwbU\nInNrEoCtWxPDhkUfe0iSJGkwKKQGICJ+BzwM/AfZuP8BrAK+BhyaUjo+pXRBXyf/VR7Ob8cWEKp6\nsdP4UUwZNxKAtRu38JFf38njq2wKJEmSNNgV1QToNLJx/gO4jGwW4F1TSu9JKd3VYFnrgSvyRS0S\nEbzm2LmV+z+89lFO/Nxf+Jef3cYDS58qMTJJkiTtiKISgCeAzwH7pZSenVL6n5TSxu0pKKX0eErp\n5JTSKc0NUf05+8S92H2nrqZAW7YmfnLjIl74tau48eEVJUYmSZKk7VVUAjAnpfTBlNIDBZWvFpgy\nbhS/fOff8P7T9me/mV3zsm3cspUvXXp/iZFJkiRpexWSAKSUthRRrlpv6vhRvOuUffj9P57IF/7u\n8Mrjtzy6is0dW0uMTJIkSdujtInANLgMGxacOW82u03J+mI/vbmDexavKTkqSZIkNaqoUYBmRsQF\n+bJbHdvvlm97fkRMKyImNce83adW1q9bYD8ASZKkwaaoGoDXk43/f2RK6bH+Ns63OSLf57UFxaQm\nOH7vnSrrv7tzcYmRSJIkaXsUlQA8D0jAzxrY5ydkw4Y+v5CI1BSnHTyL4fmEYDc/uorL719WckSS\nJElqRFEJwKH57fUN7HNjfntYk2NRE00bP4qT9pteuf/m797Aly+9n9XrN5cYlSRJkupVVALQ2U5k\naQP7dF5K3rnJsajJPvKig5gxcTQAHVsTX/3TfE78/F9YtLKeiZ0lSZJUpqISgKfz24kN7NO5rZeS\nB7g9dh7Pz9/5zG4dglc/vZlf3fp4iVFJkiSpHkUlAJ0df49pYJ9j81t7lg4Cs6eO43/PPo5j9uhK\nAq5dsLzEiCRJklSPohKAK8g69P59RIzub+OIGAO8i6zj8BUFxaQmGzF8GF9+5RGV+zc8vIIr59sp\nWJIkaSArKgE4P7/dA/hpRPTaFCh/7ifAnvlDFxQUkwowe+o45k4bB8CGzVt5/fnX8+bv3sDKdZtK\njkySJEk9KSQBSCndCHyPrBbghcB9EfHJiDgtIg7Ll9Mi4lPAffk2CfhxSunqImJScT74/AMYO3J4\n5f6f713Kmd+6mjsWrS4xKkmSJPWkqBoAgLcDvyVLAmYBH8rv35IvvwU+mD8XwCXAWwqMRwV5waG7\ncPn7T+YVR8+uPPbgsnW89Bt/5b4lT5UYmSRJkmoVlgCklDallF4E/D3wKNlJfk/Lo/k2L0wpbSwi\nloh4OCJSL8uSIl6z3cyYNIbPnXk4n37ZoZXHtmxNnPaVK5wsTJIkaQAZUfQLpJS+ERHfJJvg6yig\ncxapJ4GbgNtTSqnoOIDVwFd6eHxtC167bbz62LnMmDiat3zvxspjX/vT/G6Th0mSJKk8hScAAPkJ\n/m35UpZVKaWPlfj6beM5B87kQ88/gE//7l4AbnpkJfOfeIp9ZzYyLYQkSZKKUGQfALWxt5+0N6cf\nPKty/3lfvoLzrlzA2o1bSoxKkiRJhdcARMQw4CBgL7LZfof3vQeklL5fQCijI+J1wFxgHXA7cEVK\nqaOA1xJwzsl78/u7urpYfPLie7h3yVN84e8OLzEqSZKk9lZYAhARo4APA+cAOzewawKKSABmAT+o\neeyhiDgrpXR5fztHxE29PHXADkc2RB0xZwqvPnYuP77+0cpjv7r1Mf7xOfsyJ587QJIkSa1VSBOg\nfPbfS4F/I+v029sIQL0tzfYd4DlkScB44FDgv8kmKvtdRHhJuiCfOuMQfnrO8ZX7mzsSz/nS5bzh\nguv578sfZMNmK2AkSZJaqagagHcDz8rX7wb+C7gRWAFsLeg1e5VS+njNQ3cC50TEWuC9wMeAl/ZT\nxryeHs9rBo5qQphD0rBhwTF7TOO8NxzN235wIynBpi1bueL+ZVxx/zLWbergn5+3X9lhSpIktY2i\nOgG/Or+9Djg6pfTfKaWbUkoPpZQe6W8pKKaefCu/PbGFr9mWnnvQTC544zEcMWdKt8edI0CSJKm1\niqoB2JesLf/nUkobCnqNZug8+xxfahRt4pQDZnDy/tO5deEqXvqNqwG4+/HVbNjcwZiR/fYNlyRJ\nUhMUVQPQ2bB7QUHlN8tx+e1Aj3PIiAiOnDuVPXfOcq7NHYmbH11ZclSSJEnto6gEYH5+W/r0rxFx\nYERsc4U/IvYAvp7f/WErYxIcOberKdD7fnIbf7l3aYnRSJIktY+iEoAfk43m85KCym/EK4ElEXFx\nRHwjIj4bET8D7gH2AX4LfKHUCNvQ64/bneHDsgGfHl+9gbO+ewM/uq6V3T8kSZLaU1EJwNeBW4Gz\nI+KUgl6jXn8BfgPsDbwG+GfgJOAq4I3Ai1JKm8oLrz0dOXcqn3v5YYwa3vUV/Oof59OxNZUYlSRJ\n0tBXSAKQn1CfTjb05+8j4gsRcWREjCni9fqJ5fKU0qtTSgeklKaklEamlKanlJ6XUvp+SskzzpK8\nfN5srvzAKYwcntUELH1qI9cuWF5yVJIkSUNbUROBdQCLgeOBkcA/kSUD6yKio59lSxExaWCaOWkM\nrzl2buX+FfMdFlSSJKlIRTUBqp3Vt+yZgDWAHV41N8CiFU+XGIkkSdLQV9Q8ALUz70q9mjNtXGV9\n0cr1JUYiSZI09BWSAKSUTABUt9lTx1bWF660BkCSJKlIRTUBkuo2Y+KYSkfgFes2sW6j3UAkSZKK\nYgKg0g0fFuw2pasWYMGydSVGI0mSNLSZAGhAqO4H8Onf3YOjs0qSJBWj8AQgIoZFxHMi4t8i4usR\ncUFE7FKzzaiIGBcRo4uORwPTy4+aXVm/+sHlXON8AJIkSYUoNAGIiBcA84E/kI0M9A6y2Xen1mz6\nNuApYGlEjC8yJg1MZxy5Gy87crfK/X/7xZ2sXOcEzZIkSc1WWAIQEWcBvwb2JBvbfwW9j/F/HrAG\nmACcUVRMGtj+/tn7MCz/hix4ch1f/dP8cgOSJEkagoqaCXgv4FtkJ/yXAwenlKb3tn1KaSPw83z7\nU4uISQPfXtMn8J7n7le576zAkiRJzVdUDcB7gJHAvcDpKaV76tjnyvz2yIJi0iBw9ol7MWp49rVc\nsGwdy9duLDkiSZKkoaWoBOC5QAK+kl/dr0dne485xYSkwWDMyOEcOnty5f4Hf36HIwJJkiQ1UVEJ\nQOdJ/M0N7LM2v7UTcJt79gEzKuuX3v2EIwJJkiQ1UVEJQGdn3zEN7NM5MtCaJseiQeZtz9qLQ3fr\nqgW48PqFJUYjSZI0tBSVACzOb/dqYJ/j89tHmhyLBplRI4bx2ZcfVrl/6d1P2AxIkiSpSYpKAK4g\nqwV4dT0bR8RY4O1k/QYuKygmDSIH7TqJsSOHA/D05g7WbtxSckSSJElDQ1EJwAX57WkR8eK+NoyI\nccCFwFxgK3BuQTFpkJk6bmRlfdX6zSVGIkmSNHQUkgCklP4K/ICsFuCiiPhaRJxYtcl+EfHsiPgI\n2VChLyK7+v/1lNK9RcSkwWfq+FGV9RXOCixJktQUIwos+23AFODFwLvypbMh90VV23V2GP4p8N4C\n49EgM3VcVwKwcr0JgCRJUjMU1QSIlNKmlNLfkiUCD5Kd6Pe0LATOSSm9MqW0tah4NPhU1wCYAEiS\nJDVHkTUAAKSUzgfOj4iDgKOBGcBw4EngFuCW5BAv6kF1H4CV6+wDIEmS1AyFJwCdUkp3A3e36vU0\n+NkESJIkqfkKawIk7ahpNgGSJElqukJqAGpG/GlYSumKZsWiwWuKTYAkSZKarqgmQJfRNeJPoxIt\nbJqkgau6BuDJtRtLjESSJGnoKPJEO/rfROrdHjuNr6zf+dhqtnRsZcRwW61JkiTtiKISgFPq2GY8\ncADwGuAo4Crgo2SzAUvMnjqWXSeP4fHVG1i3qYM7H1/DEXOmlB2WJEnSoFZIApBSurzOTX8LfCki\n/h34OPCmlNIbi4hJg09EcNxeO/HzWx4D4LoFy00AJEmSdtCAaE+RUvoE8BvgdRFxZtnxaOB4xl7T\nKuvXLlheYiSSJElDw4BIAHLfJes3cHbJcWgAOW6vnSrrNz68ki0dthCTJEnaEQMpAXgovz2i1Cg0\noMydNo5Zk8YA8NTGLdy9eE3JEUmSJA1uAykBmJrfTiw1Cg0oEcGxe3Y1A7rl0VUlRiNJkjT4DaQE\n4Kz8dlGpUWjAObyq4+9ti0wAJEmSdkTpCUBE7BcR5wGvJZsE7OKSQ9IAc/jsyZX1n9/8mJ2BJUmS\ndkAhw4BGxII6NhsGTKF7k58lwKeLiEmD18G7Tmb4sKBjaza59Ku+fS0ff8nBvPGZe5QbmCRJ0iBU\nVA3AHnUsc4FJZCP/BPBX4KSU0hMFxaRBauyo4Zy03/Ruj/3PdY+WFI0kSdLgVtRMwN+rY5utwFPA\nAuDylNJtBcWiIeDLrzyCX97yGB/99V0A3PfEUzyyfB277zS+5MgkSZIGl6JmAj6r/62k+k0eO5I3\nPnMPrpy/jD/esxSA39y+mHedsk/JkUmSJA0upXcClhrxwsN2qaz/5MaFbM37BUiSJKk+bZkARMTr\nIiLly1vLjkf1e/4huzBxTFZx9cjy9Vw+f1nJEUmSJA0ubZcARMQc4OvA2rJjUePGjBzOmfNmV+5/\n87IHSclaAEmSpHoVNQzo3CLKTSnt0NAvERHAd4DlwM+B9zUjLrXWW07Yk+9f8wgdWxPXP7SCS+56\ngtMPmVV2WJIkSYNCUaMAPVRAmYkdj/fdwLOBk/NbDUKzp47jtc+Yy/eveQSAb172gAmAJElSnYpq\nAhQFLdsfUMSBwGeAr6aUrtiO/W/qaQEO2JG4tH3+6bn7MWJY9pW4bdFqHlv1dMkRSZIkDQ5F1QB0\nDgP6TuAYYDPwB+B6oHOir5n5c6cCI4EbgW8UEUxEjAB+ADwK/GsRr6HWmjp+FMfvvRNXzn8SgD/c\ntYSz/mbPkqOSJEka+IqaB+B7EXE+cDRwCfDWlNJjPW0bEbsB5wKnAc9KKRUxKs9HgCOBE1JK23Wp\nOKU0r6fH81qAo3YgNm2n0w+ZVUkALjEBkCRJqkshTYAi4kyyWoAbgBf2dvIPkD/34nzbsyLiFU2O\n5RlkV/2/mFK6ppllq1zPO2gmkTcMu/6hFSxfu7HcgCRJkgaBovoAvJ2s0+6XUkpb+9s4pdQBfIms\nnf/ZzQoib/rzfeB+4N+bVa4GhhkTxzBv7lQAtia47qEVJUckSZI08BWVAByW385vYJ/ObQ9tYhwT\ngP2AA4ENVZN/JeCj+Tbn5o99pYmvqxY5aNdJlfXFqzeUGIkkSdLgUFQn4In57cwG9plRs28zbATO\n7+W5o8j6BVwF3AfYPGgQmjlpTGV96RoTAEmSpP4UlQA8Qnbl/Y3A7+vc50357Q5N9lUt7/DbY6fi\niPgYWQLwvZTSec16TbXWjImjK+tLTAAkSZL6VVQToF+Rted/RUT8ez4Db48i82HglWT9Bn5RUEwa\ngmZN7qoBeMIEQJIkqV9F1QB8Bng9MAv4GPCqiPgB2Ug/S8lO9DvnAXgdWRt9gCXAZwuKSUNQ9yZA\njgIkSZLUn6LmAVgVEc8lmwNgNtlsuZ/qY5cAFgGnp5RWFRFTrZTSx8iSEw1iMyd2JQBL1mwgpUQf\nFU6SJEltr6gmQKSU7gEOBr4IrCI7ye9pWUU2BOghKaW7i4pHQ9OksSMYMzL7Gq/f1MHajVtKjkiS\nJGlgK6oJEAAppaeA90fEvwLzyIb4nJY/vRK4A7gppbSpyDg0dEUEMyeN4ZHl64GsH8DEMSNLjkqS\nJGngKjQB6JRS2gxcmy9SU82cWJ0AbGSfGc0cSVaSJGloKawJkNQqMx0JSJIkqW6F1wBExDDgFOB4\nslGBxgEfTiktrtpmVB5LR0rJoVzUkJnOBSBJklS3QhOAiHgB8F/AHjVPfQFYXHX/bcDXgLURsWtK\naV2RcWlocShQSZKk+hXWBCgizgJ+DexJNtrPivy2J+cBa4AJwBlFxaShySZAkiRJ9SskAYiIvYBv\nkZ3wXw4cnFKa3tv2ebOfn+fbn1pETBq6qpsA3fDwClJKJUYjSZI0sBVVA/AeYCRwL9nkXvfUsc+V\n+e2RBcWkIWpWVQ3Ak2s38a3LF5QYjSRJ0sBWVALwXCABX2mgU+/8/HZOMSFpqJozdRyH7Dapcv9L\nl97HgmVrS4xIkiRp4CoqAeg8ib+5gX06z9jGNzkWDXHDhgUXnn08B8zKxv/f3JG45K4nSo5KkiRp\nYCoqAejs7Dumz626m5rfrmlyLGoDE0aP4KVH7la5b2dgSZKknhWVAHQO8blXA/scn98+0uRY1Caq\n+wIsfcoEQJIkqSdFJQBXkNUCvLqejSNiLPB2sn4DlxUUk4a4GROrhwN1PgBJkqSeFJUAXJDfnhYR\nL+5rw4gYB1wIzAW2AucWFJOGuBmTuoYDtQZAkiSpZ4UkACmlvwI/IKsFuCgivhYRJ1Ztsl9EPDsi\nPkI2VOiLyK7+fz2ldG8RMWnoq50R2PkAJEmStjWiwLLfBkwBXgy8K186z8guqtqus8PwT4H3FhiP\nhrgJo0cwbtRw1m/qYOOWrazZsIXJY0eWHZYkSdKAUlQTIFJKm1JKf0uWCDxIdqLf07IQOCel9MqU\n0tai4lF7mFE1K/BSRwKSJEnaRpE1AACklM4Hzo+Ig4CjgRnAcOBJ4BbglmRbDTXJblPH8vDy9QDc\nsnAV+86cWHJEkiRJA0vhCUCnlNLdwN2tej21p5P3m8FfH1gOwO/uWMwrjnZiaUmSpGqFNAGKiHfn\ny0lFlC/15vRDZlXW//rAcrZ02KpMkiSpWlF9AL4CfBnYqaDypR7NmTaO6Xk/gE0dW1liPwBJkqRu\nikoAVua3DxVUvtSr3aaMraw/vsoEQJIkqVpRCUDnib81AGq56gTgsVXrS4xEkiRp4CkqAfgF2RCf\nLymofKlXu021BkCSJKk3RSUAXyOrBTg7Ik4t6DWkHu06uWtG4MdWPV1iJJIkSQNPIQlASukp4FTg\nXuDiiDgvIk6JiGkREf3sLu2Q3aaOq6wvXGETIEmSpGqFzAMQER3Vd4Gz8qXz+b52Tymlls1PoKFn\nz53HV9aveXA5C1esZ860cX3sIUmS1D6KagIUVUvt/XoWabvtPX08z9hzGgBbtib+5/pHS45IkiRp\n4CjqSvvHCypX6ldE8Ibj9+C6h1YAcMei1SVHJEmSNHAUkgCklEwAVKqDd51UWb9n8RpSSv01PZMk\nSWoLRTUBkko1d9o4xo8aDsDydZtY9tTGkiOSJEkaGEwANCQNGxbsP2ti5f5l9y0rMRpJkqSBY4cT\ngIj4SL7s3IyApGY5ds+uiag/d8l9bNqytcRoJEmSBoZm1AB8DPgoMKOnJyNi14i4ICLOb8JrSXV7\nx0l7My5vBvTk2o0seHJtyRFJkiSVrxVNgKYCb8oXqWUmjxvJ4bOnVO4vWb2hxGgkSZIGBvsAaEib\nNXlMZX3pGjsCS5IkmQBoSJs5qSsBWLLGGgBJkiQTAA1psyaNrqybAEiSJLVJAhARn42IP0XEwoh4\nOiJWRMQtEfHRiNip/xI0WFXXACw1AZAkSWqPBAD4J2A8cCnwVeBHwBayEYxuj4g55YWmIs2s6gPw\n0JPrSoxEkiRpYBhRdgAtMimltM3l34j4FPCvwIeAd7Y8KhVu18ljK+sPLlvHH+9+guceNLPEiCRJ\nksrVzATgnRGxtIfHK/MDRMRH6ikopfQfTYsqK6+3th8/IUsA9m3m62ngmDV5DM/ad2eunP8kAN+6\n/EETAEmS1NaamQC8o4/nUn770TrLamoC0IcX57e3t+j1VIJPv+xQTvjsXwC4+dGVrF6/mcnjRpYc\nlSRJUjmalQBEk8qBrmSh6SLifcAEYDJwNHAC2cn/Z+rY96ZenjqgaQGqELOnjuOIOVO4deEqtia4\nYv4yXnz4rmWHJUmSVIpmJACnNKGMVnkfUN3+4/fAm1JKy0qKRy1ywj47c+vCVQDc9fgaEwBJktS2\ndjgBSCld3oxAWiGlNAsgImYCzyS78n9LRLwopXRzP/vO6+nxvGbgqGbHqubae8b4yvojyx0NSJIk\nta92GQa0m5TSEymlXwCnAjsB3y85JBVs7rTqBGB9iZFIkiSVqy0TgE4ppUeAu4GDI2LnsuNRcfbY\naVxl/ZHl60ipsK4mkiRJA1pbJwC5zsbgHaVGoUJNGz+KCaOzFm/rNnWwfN2mkiOSJEkqx5BPACJi\nv4iY3MPjw/KJwGYAV6eUVrY+OrVKRLB7VS3A3Y+vKTEaSZKk8rTDTMAvAD4dEVcBDwHLyUYCOgnY\nC1gCvK288NQqR+8+lbvyE/9f3voYJ+43veSIJEmSWm/I1wAAfwTOB6YDLwPeD7wcWAF8HDg4pXR3\neeGpVV4+b3Zl/ec3P8Z5Vy5g61b7AkiSpPYy5GsAUkp3An9fdhwq36G7Tebw2ZO5bdFqAD558T2M\nGTmc1x23e8mRSZIktU471ABIQNYP4IuvOKLbY52Tg0mSJLULEwC1lX1mTOATZxxSub/66c0lRiNJ\nktR6JgBqO/tMn1BZX73eBECSJLUXEwC1ncljR1bWVz3tfACSJKm9lN4JOCJGAJ3j9K9OKW0pMx4N\nfVPGdSUANgGSJEntppQagIg4LiIuiIgFwAZgab5siIgFEXF+RDyjjNg09FUnAKtsAiRJktpMyxOA\niPgK8FfgTWRj898NXJMvd+ePnQVcHRFfbnV8GvrGjhzOyOEBwMYtW9mwuaPkiCRJklqnpQlARLwN\neDdwFXAyMDmldFhK6YR8OYysOdApZEnCuyPira2MUUNfRDB57KjKfZsBSZKkdtLqGoB3AHcCz0kp\nXXaL4CwAACAASURBVJFS2lq7QUppa0rpcuA5ZDUC72xxjGoDk8d2dX8xAZAkSe2k1QnAAcDF9XT0\nTSltBi4G9i88KrWdKeO6agDsByBJktpJqxOAp4FdGth+F7JOwlJTVQ8Fag2AJElqJ61OAP4KvDIi\nTu1vw4g4HXglWX8BqammVtUALH3KHFOSJLWPVs8D8FHgecDvIuLPwKXAfGB1/vxkYF/gVLKOwBvz\nfaSm2n2ncZX1h5atKzESSZKk1mppApBSuiW/+n8+WSffZ/eyaZAlBm9NKd3aqvjUPvaePqGyvuBJ\nEwBJktQ+Wj4TcErpyog4kOwK/8nAflTNBAzcD1wG/CWl5ADtKsTeM8ZX1h9ctrbESCRJklqr5QkA\nQH5i/8d8kVpuj53GEwEpwcIV69m4pYPRI4aXHZYkSVLhWj4TsDQQjBk5nDlTs34AWxNcNf/JkiOS\nJElqDRMAta3TD5lVWf/u1Q+XF4gkSVILDegEICLeEhEXlB2HhqbXH7d7Zf26h1bQsTWVGI0kSVJr\nDOgEADgBeGPZQWhomjNtHNMnjgZg05atLFq5vuSIJEmSijfQEwCpUHtPdzQgSZLUXlo6ClBEvKbB\nXfYqJBApt/f0CVy7YAUA859Yy7MPmFlyRJIkScVq9TCgPwQaaWgdDW4vNaR6QrBP/+5eXnjYLsye\nOq6PPSRJkga3VicAm4AlwHfq3P4M4LDiwlG7O3T25G73v3nZg3zqpYeWFI0kSVLxWp0A3AnMTil9\nvJ6NI2IPTABUoKN3n8qZ82bzs5sWAfCrWx/nQy84kAmjS5kjT5IkqXCt7gR8EzA9Ima3+HWlHkUE\nnz/zMPaZkTUFWrtxC9+56qGSo5IkSSpOqxOAK4BHgf3q3P4q4PvFhSNlScDZz+rqb/6tyx/k4SfX\nlRiRJElScVqaAKSUfpRS2jOl9Oc6tz8/pXRW0XFJLztqt0otwLpNHbzga1dy68JVJUclSZLUfM4D\nIAEjhg/jP6s6/67f1MHX/zy/xIgkSZKKYQIg5Y7dcxqfOOOQyv3L71/GqvWbSoxIkiSp+UwApCqv\nP253DsuHBt3ckbjkriUlRyRJktRchYx1GBFzt2O3BGwAVqeUvOyq0rzk8F25fdFqAH592+O88pjt\n+TpLkiQNTEUNdr5D4yhGxGPAtcD3UkoXNyckqT4vOmxXPvXbe0gJrnlwOUuf2sCMiWPKDkuSJKkp\nimoCFDu4zAZeDvw6Ii6NiGkFxSltY9bkMRy7R/aV25rg4tsXlxyRJElS8xRVA9A5dOc5wDPImvZc\nAtwILMufmw4cDZwGjAauB74NTAIOAf4W2Bl4NvD/27vz+Kiq84/jnycBAgQI+44iKIKIgrhrBdxa\nrWvdWqutWrRu1apt3WpLF+vWVv1Z9w2rVutSd1oRF1xwFwQEBYQIouwEAmELnN8f5ySZJDOTSTLJ\nzWS+79frvu7MXZ/7ZJLcZ+459z4PHNhAsYpUc8zw3ry/YBUAz077hjMP2CHiiERERETSo0GuADjn\nHgL2A/YGngP6O+eOd85d65y7JwzXOueOB/oDL4RlD3DO3eqcOxvYDv8QMAP2M7MfNkSsIvEcsWsv\nWuYaAJ8uKuK9+SsjjkhEREQkPRqkADCz44BzgCnAD5xzyxItG+Ydj2/zf6aZnRymbwTOAj4Ji6oA\nkEbTOb8VPxjRt/z9Nc/OZP2m0ggjEhEREUmPhuoDcD7+rj7/55xzNS0clrkF/23/OTHTtwH3hukj\nGyZUkfguPHhH8lr4X5G5y9Zx5xtfRhyRiIiISP01VAGwWxjPq8U6ZcsOqzL90zDuUq+IRGqpX+e2\njDtmaPn7h6YUsmbDlggjEhEREam/hioAOoRx91qsU7Zs+yrTN4Sxzryk0Z28Zz8GdssHoHhTKZNm\nLY04IhEREZH6aagCYGEYn1aLdX5SZd0yPcN4OXVgZl3MbKyZPWNm88xsg5mtMbO3zexnZqanIUtC\nuTnGCSMr+gLoycAiIiKS6Rrq5Pd5fLv9U83sVzUtbGa/Bn6E7zfwfJXZ+4RxYR1jOQnfj2Af4H18\nX4On8bcavQ94wsysjtuWLDBm54oLWRNnLeWJDxdFGI2IiIhI/TTUcwCux3+j3w24wcx+DDwKfEzl\n5wCMBH5MRZ+B5WHdWGWFwSt1jGUOcAzwUuhUDICZXYV/9sAJwA/wRYFINYN7tme7zm1ZuKoEgCuf\nmcH6zaXs1reAYX060qqFLiKJiIhI5miQAsA5t8rMDgP+B/TCn+DvlmQVA5YA33POrSqfaDYAeDsM\ndTpBd869lmD6EjO7C7gWGF3X7UvzZ2bcffpIznvkYwpXlrB1m+MPL8wCYJ8dOvPQWXvTumVuxFGK\niIiIpKbBvrp0zs0AdgFuBdbgT/LjDWuB24ChzrnpVbYx3zl3dhhqc0ehVJV1LNYN3iWpIb068PDP\n9qF3QetK099fsIrj75jC6vWbI4pMREREpHYaqgkQAM65NcAlZnY5sCe+3X2nMHs18BnwkXNuU0PG\nEY+ZtaCi4/H/Ulj+4wSzBqctKGnS+nVuy8RLR/HY+wsZP6WQxUX+BlWzv13LzZPm8Mdjd404QhER\nEZGaNWgBUMY5txn/VOApjbG/FF2PL0gmOOdejjoYyQzt8lpw9kEDGPudHbjhf19w12T/cLBnpi7m\niiMG07ZVo/xKiYiIiNRZVvZeNLOLgMuAz4HTU1nHOTcy3hC2IVnGzPjNd3dm+y5tASjeWMr1/9VH\nQURERJq+RikAzCzPzA4yswvM7JowXBCm5TVGDDGxXIjvlzALGBPb6VikNnJyjPNHDyx///B7X7F0\n7cYIIxIRERGpWYO2VzCzfOAPwM+oeDpwVWvN7D5gnHNufQPH80vgZmAmcIhzbllD7k+av5P37Me/\nP1zEJwuLcA5enb2MU/fZLuqwRERERBJqsCsAZtYPmAZcAhSQ+C5ABcClwFQz6xt/a2mJ53L8yf80\n/Df/OvmXejMzjtqtd/n7/3zyNVu3uQgjEhEREUmuQQqAcIed/wID8Sf5XwBX4O+3PzgMo4HL8W3o\nDdgRmGBmab+hupldg+/0+zH+m/8V6d6HZK/DdulB2bOkP/pqNcff8Q6FKxr0YpaIiIhInTVUE6Cf\n4Z8B4ICbgKudc1urLDMHeNPM/g78GV8MDA3r3pOuQMzsp8Afga3AW8BFVna2VqHQOTc+XfuU7NKv\nc1vOHTWQO9/wdwSa/vUajvnH20y6dBTdO7SuYW0RERGRxtVQBcBJYfyic+6KZAuGwuBKMxsKHBXW\nTVsBAOwQxrnALxMsMxkYn8Z9Spb59eE7M3fpOibNXgrA2o2l3P/OAq48YkjEkYmIiIhU1lB9AIbh\nv/2/rxbr3BvGu6UzEOfcOOec1TCMTuc+Jfvk5Bj/OHUEI7fvVD5t/DuFzFlaHGFUIiIiItU1VAHQ\nMYy/rsU6ZcsWpDkWkUbRumUuT/58P3bp5W94tal0G1c/MwPn1ClYREREmo6GKgBWh3G/WqxTdgeg\nojTHItJocnKMW344nJa5vp/Jh4Wref0L3XBKREREmo6GKgBmhPHZtVhnbBhPT3MsIo1qUI/2/HCv\nimcBXPbEp8z4ek2EEYmIiIhUaKgC4En8rT2PNLObkt3a08xyzOxa4Gh8v4EnGygmkUZz8aE70bFt\nSwBWl2zhtPvfp6hkc8RRiYiIiDRcAfAA8Bm+CLgUmG1mV5vZIWY2zMx2Da+vCsuV3SloZlhXJKN1\nbZfH307avfz9mg1b+NOLs9UfQERERCLXIAWAc64UOBKYhy8CBuLvxT8R/yTeT8PrPwGDwjJzgSPj\nPC9AJCMdMqQH1x6/a/n7pz/5mkfe+yrCiEREREQa7goAzrlFwHD8g8CK8Cf58YYi4EZgD+fc4oaK\nRyQKJ43sx+Ce7cvfPzilUFcBREREJFINVgAAOOdKnHOXAz2BA4CfA1eG4edhWk/n3BXOufUNGYtI\nFFq1yOHxc/Ytfz9/+XqueW4mS9ZsjDAqERERyWYN9STgSpxzW4B3wyCSVTq2bcX3h/XipRnfAvDI\newt5/fPlvPiLA+mU3yri6ERERCTbNOgVABHxLjlsJzq0rqi3Fxdt4IS7prC4aEOEUYmIiEg2qtcV\nADPbrualas85t7AhtisSlR27t+eVS0dxzbMzmThrKeCbA/384Y+488cj6de5bcQRioiISLaobxOg\nBWmJojJHIzVNEmlMPTq05p6f7MkTHy3iiqens83BzMVr+c6NrzO8X0f2GdCZ0YO6s9/ALlGHKiIi\nIs1YfU+0LS1RiGSRk/fsx/LiTdz08hfl06YtKmLaoiLunjyfru3yuOmk3Rizc/cIoxQREZHmqr4F\nwJlpiUIky5w/eiDd2uVxxxvzKFxZUmneinWbuPTf05h4ySi6tc+LKEIRERFprupVADjnHkpXICLZ\nxMw4ea9+nLxXP6YuXM37C1Yx+YvlvDt/JQCrS7aw17WT+O7QHhw7vA9tW+WyV//O5OepdZyIiIjU\nj84mRCI2YrtOjNiuE+eOGsiL07/hwn9NLZ/38mdLefkz32m4Q+sW3H/GXuzVv3NUoYqIiEgzoNuA\nijQhR+3Wm1t/OJxObVtWm7d2Yymn3/8+Ez9bwtK1Gykq2RxBhCIiIpLpdAVApIk5dngfjtm9Ny9O\n/5YXp3/D5tJtTP96DSvXb2bjlm2c8/DH5cv+aO9+/Pm4YeTmqD++iIiIpEYFgEgTZGYcvXtvjt69\nNwCFK9bz4/ver/bgsMc+WMTHX63md0cN5cCdukYRqoiIiGQYNQESyQD9u+bz1Hn7ccSuPenePo/O\n+a3K581Zuo7T7n+fk+6awtylxRFGKSIiIplABYBIhuhV0IY7TxvJB1cfykdXH8pJI/tWmv9h4Wp+\ndO/7LFpVkmALIiIiIioARDJSTo5xwwm7ccMJw+jQuqIl34p1mzhz/Ies2bAlwuhERESkKVMBIJKh\ncnKMU/bajvevOpRLDh1UPn3esnWMfehDijduYeOWrRFGKCIiIk2ROgGLZLg2rXK5+NCd6N+1LRc/\nPg3wzYGGjZsIwJ7bd+KBM/eiQ+vqtxYVERGR7KMrACLNxLHD+/CrwwdVm/7RV6u54unpOOciiEpE\nRESaGhUAIs3IBWN25Iz9+9MurwV5LSp+vSfMWMIOV07gpLumsHLdpggjFBERkaipABBpRsyMcccM\nZeYfvssXfz6C0/fdvtL8DwtXM/LPkzjmH2/z6aKiiKIUERGRKKkAEGnGfnvUEMbs3K3a9Olfr+HY\n29/hpw98wMTPlkQQmYiIiERFnYBFmrG8Frk8eObebC7dxt9e+YK7J8+vNH/ynOVMnrOco3fvzfEj\nenPw4B4RRSoiIiKNRVcARLJAqxY5XHnEED7+7aFcfeSQavNf+PQbzhr/Ede+NItpi4r4dFER6zeV\nRhCpiIiINDRdARDJIl3a5XH2QQM4eEh3Xpu9jGsnzK40/963FnDvWwsA6NY+j+cvPIBeBW2iCFVE\nREQaiAoAkSw0sFs7BnZrx74DuvDctMXc9/aCasssL97Efte9Rv8ubStN7981n5tO3J1u7fMaK1wR\nERFJIxUAIllsWN8ChvUt4MjdevH3iXMo3riF+cvXUxzT/KdwZUmldQpXlrDXtZP447FD6d6+NYfv\n0oOcHGvs0EVERKSOVACICHts14lHxu5T/v6aZ2fy8HtfJV3nd899BsDFh+zEJYdVfwCZiIiINE0q\nAESkmj8dtysXHrwjJZu3Vpp+0WNTmbF4TaVpt746lwffWUB+Xgu279KWW04ZQc+C1o0ZroiIiNSC\nCgARiatHh+on8f86ex8eeLuQZcUbefT9heXT124sZe3GUr5ds5HfPjuDnx04gB4d8hjQrV1jhiwi\nIiIpUAEgIilr37olFx+6EwDHjejDTx/4oNpVgkmzlzFp9jIAbjhhGKfstV2jxykiIiKJqQAQkTrZ\nq39nPrnmMFat3wzA5U9P5625Kyot89eJc1iyZlO1dXt0yOP4PfqQ1yK3UWIVERGRCioARKTOWrfM\npXdH/5yAG0/cjb9M+Jxlazcy65u1FG8qZXnxJm6eNCfuuvOWreO3R+3SmOGKiIgIWVIAmNmJwChg\nOLA70B541Dl3WqSBiTQjvQracNuPRgBwz5tf8pcJnydd/l8fLGR9TPOhlrnG94f1Yp8BXRo0ThER\nkWyXFQUA8Fv8if864GtgcLThiDRvYw8cQOf8PBauXF9t3jPTFrNo1QZKNm/lsQ8WVpr3xEeLePPX\nY+gepwOyiIiIpEe2FACX4E/85+GvBLwebTgizVtOjnHiyL5x523fJZ/Lnvw07ryNW7ZxxX9mMKhH\n+0rT99y+E4fu0iPtcYqIiGSjrCgAnHPlJ/xmemKpSJR+sEcfehW0ZkHM1YHPvy0uf/DYa58v47XP\nl1Vb77Gz92W/gWoeJCIiUl9ZUQCISNNhZuy/Y1f237Fr+bT1m0p5+bMlLCuufsegMve+NZ8tW7eR\nY8awvgUUtGnZGOGKiIg0OyoAUmRmHyeYpf4EIvWUn9eCJ8/dj1dmLWXz1m3l09duKOWuyV8Cla8M\ndG3XiomXjKJzfqtI4hUREclkKgBEpEnYvks+Y78zoNI05xyvzl7K3GXrKk1fsW4zL07/hp/s178R\nIxQREWkeVACkyDk3Mt70cGVgj0YORyQrmBm3/HA4d7zxJWs3bGHlus3M+nYtALdOmsuzUxdXW/7g\nwd25YMyOUYQrIiKSEVQAiEiTNrR3Abef6mvsJWs2su91rwKwcv1mVoanEMf6+KvV7LFdJ3UYFhER\nSSAn6gBERFLVs6A1xw7vXeNyr85e2gjRiIiIZCZdARCRjHLLKcM5d9RASjaXVpo+c/Fafv/8ZwDc\n9/YCxk8pTHmbO3Zvx/1n7EWfjm3SGaqIiEiTpAJARDKKmTGkV4dq03fpVcC1E2azudTfRah0m0t5\nm58vKebeN+cz7pihaYtTRESkqcqKAsDMjgOOC297hvF+ZjY+vF7hnPtVowcmImnTplUuv/nuzvxt\n4hw2bNla6/Xfm7+yAaISERFperKiAACGAz+tMm1AGAC+AlQAiGS4sd8ZwJkH7MA2l9q3/yWbt7LH\nn15h6zbH50uKOeLWt2pcJzcHjhzWi/NH605DIiKSmbKiAHDOjQPGRRyGiDSC3BwjF0tp2YI2Oeza\np4BPFxUBMDvcYrQmMxev5aCdurFrn4I6xykiIhIV3QVIRLLaWQf0Jye1eqGSjwpXpT8YERGRRpAV\nVwBERBI5dngf9h/YlWXFG2tcdsKMb7n99S8BmPLlSg7cqVut9tWmVa7uNCQiIpFTASAiWa9b+zy6\ntc+rcbnNpdvKC4CJs5YycVbtnzfwgxF9+Pspw2u9noiISLqoCZCISIp26d2B/Fa59drGf6YuZsW6\nTWmKSEREpPZ0BUBEJEV5LXK5/oTduPvNLynZXLtbjX5btLH89qRfLCmm6441X3EQERFpCCoARERq\n4ejde3P07r1rvd4VT0/n8Q8XAf5uQwfs2DXdoYmIiKREBYCISCMY3LN9+et/vD6vvBioDwMOHtyd\nK44YjFkdbmUkIiJZSQWAiEgjGNyrQ/nropItFJVsSct25y5bx5jB3dl3QJe0bE9ERJo/dQIWEWkE\ne27fid37NsyDw2Z9k9oDzEREREBXAEREGkWL3ByeveAAFqxYz9Ztrt7be+qTr7l78nwACleur/f2\nREQke6gAEBFpJGbGgG7t0rKtPbfvzN34AmDBChUAIiKSOhUAIiIZaIeubctfz1i8hr9MmJ3W7Re0\nacmJI/vSo0PrtG5XRESipwJARCQD9evclhyDbc53Kr7nzflp38e7X67kkbH7pH27IiISLXUCFhHJ\nQHktctl/YMM+S+DDwlU4V//+CiIi0rToCoCISIa67Ucj+O/MJRRvTM8tRcvc+upcSjZvZVPpNlat\n30yXdnpqsYhIc6ICQEQkQ3XKb8Wp+2yX9u3+55PFfLG0GIBv12xUASAi0syoCZCIiFTSq2NFx9/F\nRRsijERERBqCCgAREamkV0Gb8tffqgAQEWl21ARIREQq6RNzBeDBKYW89sXyCKNJzc492vHLQweR\nn6d/ayIiNdFfShERqST2CsBXK0v4amVJhNGk5s05y+nRoTVjvzMg6lBERJo8NQESEZFKRu/cjfat\nM+/7oXnL1kUdgohIRsi8v/AiItKgurTL4+3LD2baoqIm/xyATxYW8X+vzgVg5frNEUcjIpIZVACI\niEg1BW1aMmpQt6jDqFGblrnlBcAqFQAiIilREyAREclYXdq1Kn+tAkBEJDUqAEREJGN1zq94SJkK\nABGR1KgAEBGRjFXQpiU55l+v2bCFLVu3RRuQiEgGUAEgIiIZKzfH6Ni2ohnQ6hJdBRARqYk6AYuI\nSEbrnN+qvPnPvKXr2KaLAI2iVYscOue3qnlBEWlyVACIiEhGiz0JPfW+9yOMJPucsmc/bjhxt6jD\nEJFaUhMgERHJaP06tY06hKz1748WUaRmVyIZR1cAREQko507agALV61n4aqSqEPJGsuLN7EtPCOu\neGNppX4YItL0qQAQEZGMtlOP9jx57v5Rh5FVDr95MnOWrgNg/ebSiKMRkdpSEyARERGplbatKr4/\nXL9pa4SRiEhdqAAQERGRWsnPyy1/XaIrACIZRwWAiIiI1IquAIhkNhUAIiIiUiv5rXQFQCSTqQAQ\nERGRWmmbF3sFQAWASKZRASAiIiK10i62ANisJkAimUYFgIiIiNRK29gmQLoCIJJxVACIiIhIreS3\n0hUAkUyWNQWAmfU1swfM7Bsz22RmhWZ2i5l1ijo2ERGRTNJWtwEVyWhZ8SRgMxsITAG6A88BnwN7\nAxcD3zOzA5xzKyMMUUREJGPk6zagIhktW64A3IE/+b/IOXecc+4K59zBwM3AzsC1kUYnIiKSQdrq\nNqAiGa3ZFwDh2//DgULg9iqzfw+sB043s/xGDk1ERCQj5efpCoBIJsuGJkBjwniic25b7AznXLGZ\nvYMvEPYFXm3s4ERERDJN7BWA2UvWcv6jH0cYjUg0+nZqy1VHDok6jDrJhgJg5zCek2D+XHwBMIgk\nBYCZJfrrNrjuoYmIiGSe2OcAFJVsYcKMJRFGIxKNIb06RB1CnTX7JkBAQRivSTC/bHrHRohFREQk\n4w3o1o7+XdpGHYaI1FE2XAFIC+fcyHjTw5WBPRo5HBERkcjk5hgvXfQd3pm3gi1bXdThiESifevM\nPY3O3MhTV/YNf0GC+WXTixohFhERkWYhP68Fhw/tGXUYIlIH2dAE6IswHpRg/k5hnKiPgIiIiIhI\ns5ENBcDrYXy4mVU6XjNrDxwAlADvNXZgIiIiIiKNrdkXAM65L4GJQH/ggiqz/wDkAw8759Y3cmgi\nIiIiIo0uG/oAAJwPTAH+z8wOAWYD++CfETAHuDrC2EREREREGk2zvwIA5VcB9gTG40/8LwMGArcC\n+zrnVkYXnYiIiIhI48mWKwA45xYBZ0Ydh4iIiIhIlLLiCoCIiIiIiHgqAEREREREsogKABERERGR\nLKICQEREREQki6gAEBERERHJIioARERERESyiAoAEREREZEsogJARERERCSLqAAQEREREckiKgBE\nRERERLKIOeeijiGjmdnKNm3adB4yZEjUoYiIiIhIMzZ79mw2bNiwyjnXpT7bUQFQT2a2AOgAFEaw\n+8Fh/HkE+85kylvtKWd1o7zVjfJWN8pb3ShvdaO81U1989YfWOuc26E+QagAyGBm9jGAc25k1LFk\nEuWt9pSzulHe6kZ5qxvlrW6Ut7pR3uqmqeRNfQBERERERLKICgARERERkSyiAkBEREREJIuoABAR\nERERySIqAEREREREsojuAiQiIiIikkV0BUBEREREJIuoABARERERySIqAEREREREsogKABERERGR\nLKICQEREREQki6gAEBERERHJIioARERERESyiAqADGRmfc3sATP7xsw2mVmhmd1iZp2ijq0xmNmJ\nZnabmb1lZmvNzJnZIzWss7+ZTTCzVWa2wcymm9kvzSw3yTo/NbMPzGydma0xszfM7Kj0H1HDM7Mu\nZjbWzJ4xs3khB2vM7G0z+5mZxf1bkO15AzCzG8zsVTNbFHKwysymmtnvzaxLgnWyPm9Vmdlp4XfV\nmdnYBMtkfd7C33OXYFiSYJ2szxuAmR0S/sYtCf8bvzGzl83syDjLZn3OzOyMJJ+1smFrnPWUO7Pv\nm9lEM/s65GC+mT1pZvslWL7p5cw5pyGDBmAgsBRwwLPA9cBr4f3nQJeoY2yEHEwLx1sMzA6vH0my\n/LFAKbAOuB+4KeTKAU8mWOevYf4i4GbgdmBlmHZh1DmoQ87ODbF/AzwKXAc8ABSF6U8RHgyovFU7\nps3AeyFf1wO3AR+GY1oM9FPeasxhv/BZKw7HNDbOMsqbP6bCkKtxcYZfKW8J83ZjzDHdA/wFuBf4\nBLhROYt7TMMTfM7GAa+GY3tRuat2PDeE2FcA9+H/LzyF/1+xDTgtE3IWeSI11PIHBi+HD8Avqkz/\ne5h+V9QxNkIOxgA7AQaMJkkBAHQAlgGbgD1jprcGpoR1f1hlnf3D9HlAp5jp/cMv4Eagf9R5qGXO\nDgaOBnKqTO8JLAzHe4LyFjd3rRNMvzYc7x3KW9L8GTAJ+DL846tWAChvlY6rEChMcVnlzcd+djim\n8UCrOPNbKme1zum74ZiPUe4qHU9PYCuwBOheZd6YcKzzMyFnkSdTQ60+eAPDh2IB1U/k2uOry/VA\nftSxNmJORpO8ADgrzH8ozryDw7zJVab/M0w/M846fwzz/hD1sacxh1eFY7pNeatV3nYPx/SK8pY0\nTxfjvxU7CP/NYrwCQHmriL2Q1AuArM8bkIc/wfqKOCf/ylmdcjosHNPXQK5yVynmfULMzyWYvxYo\nzoScqQ9AZhkTxhOdc9tiZzjnioF3gLbAvo0dWBN2cBj/L868N4ESYH8zy0txnf9WWaY52BLGW0H3\n6wAAEzFJREFUpTHTlLeaHR3G02OmKW8xzGwI/vL4rc65N5MsqrxVlhf6TFxlZheb2ZgEbYWVNzgM\n6Ab8B9gW2mZfHvIWrz22clazc8L4fudcbB8A5Q7m4pv67G1mXWNnmNlB+C9jJ8VMbrI5a1HfDUij\n2jmM5ySYPxc4HBiEb78nSXLmnCs1swXAUGAAMNvM8oE+wDrn3Ldxtjc3jAc1RLCNzcxaAD8Jb2P/\n2ChvVZjZr4B2QAGwJ3Ag/uT/+pjFlLcgfLYexjcxu6qGxZW3ynricxdrgZmd6ZybHDNNeYO9wngj\nMBXYNXammb0JnOicWx4mKWdJmFkb4DR8M5f7qszO+tw551aZ2eX4ZtezzOxZfLOcgcAxwCvAz2NW\nabI5UwGQWQrCeE2C+WXTOzZCLJmitjnLthxfj/+HOcE593LMdOWtul8BPWLe/w84I+bEApS3WL8D\nRgAHOuc21LCs8lbhQeAt4DN8p+kBwIX4b2X/a2b7Oec+Dcsqb9A9jH8NzAK+g79RxA74jpSHA0/i\nm4uCclaTk/HH8pJzblGVecod4Jy7xcwK8TeGODtm1jxgvHNuWcy0JpszNQESyVJmdhFwGf5uBKdH\nHE6T55zr6Zwz/LezP8CfmE01sz2ijazpMbN98N/6/805927U8WQS59wfnHOvOeeWOudKnHMznXPn\n4r9xbIPvRyEVys5jSvEdVt92zq1zzs0Ajse3Yx+V6PaMUk1Z85+7I42iCTOz3+Dv+jMe/81/PjAS\nmA88amY3Rhdd6lQAZJayyq8gwfyy6UWNEEumqG3OsiLHZnYhcCv+G7MxzrlVVRZR3hIIJ2bP4L9Z\n7ILvsFUm6/MWmv78E3/J+5oUV8v6vKXgrjA+KGaa8lYR61TnXGHsDOdcCf7OeQB7h7FyloCZDcXf\ngeZrYEKcRbI+d2Y2Gn8b0Oedc5c65+aHQv0TfMG5GLjMzAaEVZpszlQAZJYvwjhR26+dwjhRH4Fs\nlDBn4URlB/w3R/MBnHPr8b/A7cysV5ztZXyOzeyX+HvZz8Sf/Md7uJDyVgPn3Ff4AmpoTGcw5c33\nkxgEDAE2xj5UCPh9WObeMO2W8F55q1lZU7P8mGnKW0UOEp0QrQ7jNlWWz+acJZKo828Z5Q7KHsT1\netUZoeD8AH9uPSJMbrI5UwGQWco+cIdblSe3mll74AB8j/L3GjuwJuy1MP5enHkH4e+aNMU5tynF\ndY6oskxGCZ2Xbsa3kR1Tpa1iLOUtNb3DuOyfpfLm73d9f4Jhaljm7fC+rHmQ8lazsru7zY+ZprxV\nPLBql6r/F4OyTsELwlg5i8PMWuObgm7F/27Go9z5286Cv/NUPGXTN4dx081ZQ90rVUPDDOhBYFXz\nMZqaHwS2nCb4EI4IcnVNOK6PgM41LKu8+dgHAQVxpudQ8SCwd5S3lPM5jsQPAsv6vOGvmlR7jks4\nprnheK9S3qrl57lwTJdUmX44/hkUq8t+j5WzhDk8PRzjC0mWyfrc4TtJO/yDwPpUmXdE+LxtALo0\n9ZxFnkwNtf7wDQSWhg/Hs8B1+ErQ4S81dYk6xkbIwXH4zjfj8XdicfinjJZN+2uc5csew30f/pHx\n5Y/hBizOPv5G9cdwryCNj+Fu5Jz9NMReGo5nXJzhDOWt2vH8MvwxfwW4J/y+PRA+bw74FthFeUs5\nn+OIUwAob5XyUwy8BNyBb2v8VPgMujC9VZV1lDfoS8UTzSfhnzj9VMjLFmKecq6cJczhW+FYjq5h\nuazOHf7Ln1dC7GuBh8Lv6fP4k38HXJwJOYs8mRrq8EODfvhbxX2Lv8z0FXALMZVicx6oOIlINBTG\nWecAfKem1eGf6QzgEmKechhnnTOAD/FPVy4GJgNHRX38DZQzB7yhvFU7ll2Bf+CbTK0If8TXhOMb\nR4IrKdmetxQ+h9UKAOXNAYwCHgsnB0X4k9fl+BOOnxDnREF5Kz+ebvi+TV/h/y+uAJ4B9lbOaszd\nECpONhMev3JXfiwt8V8OvYcvAkrxT6N+ETg8U3JmYSciIiIiIpIF1AlYRERERCSLqAAQEREREcki\nKgBERERERLKICgARERERkSyiAkBEREREJIuoABARERERySIqAEREREREsogKABERERGRLKICQERE\nREQki6gAEBERERHJIioARERERESyiAoAEZEqzKzQzJyZjY86FsluZvZG+Cy+EXUsItJ8qAAQkWbH\nvCPM7HYz+9TMlprZZjNbbWazzexhM/uRmbWJOlZpWsxsfDjhdmbWP+p4REQaggoAEWlWzGx/4GNg\nAnA+sBvQHWgJdAQGA6cB/wK+NrNLzUx/C9PAzPrHnDyfEXU8IiISX4uoAxARSRczOx24D2gVJn0E\nPA18AqwA2gP9ge8BxwCdgb8BDwBFjRyuiIhIJFQAiEizYGajgQeBXKAEGOuceyzOopOBh8ysBzAO\nOLexYhQREWkKVACISMYLbfn/hT/53wYc7Zx7Ldk6zrmlwHlm9iqwpeGjFBERaRrU7lVEmoOzgF7h\n9Z01nfzHcs495Zxbn2wZMxtkZneZ2QIz22hmy83sJTM7pIb1OpnZmWb2iJnNMrN1oTPyEjN72czO\nMbNWSdav1qbezI4zsxfMbLGZlZrZtCrr7Gtmfw53j1kS9rc27P9OM9sl1dyY2aFm9pCZzTWzYjPb\nZGaLwv7PN7OOMcs6YEHM6g/GxF42jEuwn2Ghw/asEGuJmc0zs/vNbHiS+EbHbHt06Px9hpm9Go59\nq5k9m+rxpiKmk3BheF9gZr83s5nh57vGzN4NP9vcFLa3r5k9GeLdGD5j95jZzrWMq7+Z3WRm08ys\nKGxroZk9ZmajEqzTKfw8nZktM7OeSbZ/W0yux9YmNhFpgpxzGjRo0JDRA76tv8N/+79jGrZXGLY3\nHjgOWBfexxsuTGE7yYZPgJ4J1u8fs9yZIZ6q60+LWf6MFPZXCpxfw/F3BF5KYVvjYtapadlKy4d1\nDLge2JpknW3ANQniHB2z3PeAl+Os/2wtf/axOe6fZH4hsDMwP0nsTwCWZF+XJDn2dcCRwBvh/RtJ\ntnMxsKmG3N8F5CbIYVkM/40Xb4ijbDtPR/37rkGDhvoPagIkIhnNzDoAZd8Sz3HOzUvj5ocBpwDL\ngKuBD/AnS6PC+wLg72Y2yTn3eZz1c4H3gReBqcBSfAflHfB3IvoeMAJ4HH8ilswv8Xc0mgLcAXyB\n79Q8OGaZFsBq4DngTWAusB7oDewBXAR0Bf5hZp+7OFdKzKw18GpYHmAG/uTxU2BD2Nb+wMlVVh0W\n5r0c3v82xBFrWZX3twK/CK/fB+4H5gFrgV2AC4B9gD+a2Srn3O3V01LuBnx+XsL3BSkEugA9kqxT\nH22BF/B3mLoemAisCXFfAwwCTsLn4/6qK5vZ8cDfw9u1wI3A6/iT7NHAFcCjwPJkQZjZpfiO7ACz\n8Z+NOfhO7wOAscB3gZ8DxcCvY9d3zr1hZjcBl+M/jxfhfy5l2++BzyfAYuDsZPGISIaIugLRoEGD\nhvoMwAFUfDv5aJq2WRizzalApzjLjIpZ5uYE29mphv2cGbONQ+LM70/lb3EfBXKSbK8P0DbJ/AL8\nibwD3kqwzI0x+7uXON8ah+VygD5J4j2jhmM/NGbZXyRYJhfft8PhT5I7Vpk/ukp+rkvDz358zPb6\n1zB/DbBbnGW64osdB0yNM78V/mTa4U/Kh8VZZmjYftm+3oizzBBgc5h/Y6LPBr44cvjidVCc+S2B\nD8MyG2Pjwd9Ot+xKTLXPqAYNGjJzUB8AEcl0XWJeL22A7Z/pnFtddaJzbjL+W2vwxUA1zrm5yTbs\nnHsQKGvDf1wNcawBznPObUuyvcXOuZIk89cAvwtvDzSz2NxhZgX4ZycAzMQ3FdqaYFvbnHOLa4g5\nmSvCeIJz7rYE+9iKvwqwCX+146Qk25uH/+a9Mf3OOTe96kTn3Ar8rWUBdg95jXUs/moJwPXOuRlx\ntvEZcG0N+78Mf/I+E7giyWfjt8A3+KLtp3H2tQX4Mf5qUR7wLzNrbWa/AI4Ii/3VOfdqDfGISIZQ\nASAima59zOuknXnrYKZzblqS+R+F8YCaNhQ6qPYMHYp3LRvw3wQD7F7DJl5wzq1NIebYfeaHzqFD\nY/YXe8ejqvscA+SH17eFE8O0C822Roe3TyZbNhRfM8Pb/ZIs+rhzrrT+0aWs7IpMImWfDcM3+Yp1\naMzrB0nswbCfRI4J46drKAy3AO+Gt3Fz6Jybg29mBrAr/srLjeH9VHwRISLNhPoAiEimK455nZ9w\nqbqZXcP8VWHcPtECZvZ94DzgoGTL4ZuNJPNpDfPL9tcVuBQ4AdgJfwKa6j73iHn9Vir7q6MR+OY9\n4O8WlOwkOFbCu9SQYn7SaEX4pj+RVTGvq/7ch4XxN865bxJtwDm3PNxtqGoBgZltD3QLb39vZr+v\nOWQgSQ6dc/eZ2RHAD4Djw+QS4FTn3OYUty8iGUAFgIhkupUxr9Pd4TNhc5qg7FvXaldTzczwbeh/\nluK+2tQwv1ozpDj7HInvdNqlpmUT7LNbzOtvU9xGXXSv43ptk8yrMT9plupnAyqKnTKdw7hqp+h4\nlhKnAKBhcgi+WD0a37QI4CoXv4O7iGQwFQAikulm4Ds35gIjI44l1llUnPxPA27B9xlYDJSUta03\ns38Cp5P8m3rwx5iQ+ecJPIE/+d8C3Ia/C88cYLVzblNYbgDwZdlqtTuktIk9Ib4If/ebVCRr4pU0\nP01UsuY9NYnN4fUkb44Uq6Zv8s+j4uQf4DBi7gokIs2DCgARyWjOubXhYVgjgUFmtlNNnW8bSdnt\nEucB+zvnNiRYrnOC6bV1MBV9Ec53zt1Xh/3FNmnpBRSlI7Aa9rPBOTcz4ZLNU9nVilSuWCVaJjaH\nW9ORQzPbl4q2/muBDsD3zewCl/wWrCKSYdQJWESag7I25IZ/KFJTMDSMn0908h+aCe0Rb1499gfw\n7yTL7Zlk3scxrw+qQwypfqNdditSgAPrsJ9MV3bXn95m1jvRQmbWDX9r1XgW4O8MBWnIoZm1x19F\naIEv/EZS0a/ipto8QVpEmj4VACLSHDxARZv188zs4FRXNLMTzCzdnYeh4gprsm0fi/+mPZ37S7hP\nM8sh+YOcXqeimc2FZlbbq8QbY17nJVrIObcc/0AzgJPMrE8t95PpJsW8rnZbzhhnkKCZVmhC9mJ4\ne5CZ1beQvI2KK0jnOf9AvVPxD39rAzwampmJSDOgAkBEMl74hv3H+HbgOcALZnZKsnXMrJuZ3Q48\nReU2z+lS1gzpaDOr1uzGzAYC6WxWEdvs6YwEy1xHkisO4TkBd4a3uwJ3hqKhGjPLifPt9Uoq2pgP\nrCHeP4VxW+A/4e5FcZlZrpmdZmZ9a9hmpniWioL1SjMbWnUBMxuCf9p0MtcBpfgi4YnQvyOucBva\no81stzjzTqKiEHnYOfc4gHNuFhVPDh4O/KWGeEQkQ6gAEJFmwTn3Or7j7Wb8SeXjZvaBmf3GzA4z\nsz3M7CAzO93MHgHmU/HQq4bwzzDuDbxrZmeZ2d4hhnH45jadgU/StL+XqbirzJ/N7C4z+66ZjTSz\nU8xsEvAb4J0atvM7Kpp+jAWmmtl5ZrafmY0wsyPN7I/AF8A5sSuG+/B/GN6eZWY/MrMhZrZjGDrH\nLPsy8Lfwdm9gtpn9OfyshpvZ/mb241CkfQ08DHSsS2KamnBLzV+Et+2Bd8zsqpDjfc3sciqukMxL\nsp3PgEvC24HAp2Z2S/gZjQjbOtHM/or/vD8PbBe7jVBU3R3eLgAurLKP2/FPAwa41MwOqcsxi0gT\nE/WjiDVo0KAhnQNwAP6k2qUwLMef8ORU2UZhmD++hn2NK9tWnHkt8SflifZdgn+y7fjwvjDONvrH\nLH9GCsf+XXyTjUT7fB3fVyDpNvGFyaQU8jcuzrrfx98CM9Xlr8Q/6bemfW0Cdqyy7uiY+aPT9PkZ\nH7PN/knmV/t51TY24FdJcrU+5PKN8P6NJPs6A1iXQg63AmNi1ssBXgvzSoEDEmy/O/52pA5fjHWO\n+vdcgwYN9Rt0BUBEmhXn3Dv4DozfxzdnmY4/0S/Fd5qcjf82+YdAX+fcP1ySp6jWI44tIYaL8E+F\nLcGfnM8D7gL2cM4lfQpuHfb5Mr6T7yPAN/jbgS4HJuO/rT+EFJ6W7Jxb5Zw7FH8/+MeBr/Dt+zcB\nC/HfJJ9DxTf4seu+FPbzXEwMyfZ1Hf6BZX8BPsA3IyrFn9DOwTfROhfo43y79GbDOfdXfAfe/+Cv\n3mzC5/oBYM+Qy1S2Mx5fLP4WeJOKz3sJ/pv/F/BXCvo7f6WszGX4pz8D/CX87sTb/jL81TWAPsA9\nKR2giDRZ5lx9bkMsIiIiIiKZRFcARERERESyiAoAEREREZEsogJARERERCSLqAAQEREREckiKgBE\nRERERLKICgARERERkSyiAkBEREREJIuoABARERERySIqAEREREREsogKABERERGRLKICQEREREQk\ni6gAEBERERHJIioARERERESyiAoAEREREZEsogJARERERCSLqAAQEREREckiKgBERERERLKICgAR\nERERkSyiAkBEREREJIv8PwjGUDayUW4CAAAAAElFTkSuQmCC\n",
            "text/plain": "<matplotlib.figure.Figure at 0x7fdf4e55ea20>"
          },
          "metadata": {
            "image/png": {
              "height": 285,
              "width": 384
            }
          },
          "output_type": "display_data"
        }
      ]
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "### Input Domain\n\nInput domain is [A-z0-9], plus space and new-line \"\\n\" characters. I have considered `\"-\"` but decided not to add it. \n\nIn speech recognition tasks, we might be able to distinguish between <kbd>space</kbd> and <kbd>⏎</kbd> by the length of the pause.\n\n### Output Domain\n\nThe output domain includes the following. The punctuation extraction made sure that strings like \"I'm a boy\" doesn't introduce conflict between the capitalization of the \"i\" and the single quote afterward.\n\n```\ncapitalization: <cap>\n         comma:  ,\n        period:  .\n   dollar sign:  $\n     semicolon:  ;\n         colon:  :\n  single quote:  '\n  double quote:  \"\n  no operation: <nop>\n```"
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:07:01.328128Z",
          "start_time": "2017-03-29T05:07:01.321998-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "input_chars = list(\" \\nabcdefghijklmnopqrstuvwxyz01234567890\")\noutput_chars = [\"<nop>\", \"<cap>\"] + list(\".,;:?!\\\"'$\")",
      "execution_count": 7,
      "outputs": []
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "### Data Module\n\nBesides loaders for the training data, the data module also contains helper functions such as `extract_punc` and `apply_punc`. Here is an example of what they do:"
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T08:08:30.055165Z",
          "start_time": "2017-03-30T03:08:30.031359-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "import data\n\ni, o = data.extract_punc(\"ATI'd. I'm not sure if $10 is enough. \", input_chars, output_chars)\nprint(\"Punc-less Text:\\n========================\\n\", i)\nprint(\"\\nPunctuation Operators Extracted:\\n========================\\n\", o)\nresult = data.apply_punc(\"\".join(i), o)\nprint(\"\\nVarify that it works by recovering the original string:\\n========================\\n\", result)",
      "execution_count": 33,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Punc-less Text:\n========================\n ['a', 't', 'i', 'd', ' ', 'i', 'm', ' ', 'n', 'o', 't', ' ', 's', 'u', 'r', 'e', ' ', 'i', 'f', ' ', '1', '0', ' ', 'i', 's', ' ', 'e', 'n', 'o', 'u', 'g', 'h', ' ']\n\nPunctuation Operators Extracted:\n========================\n ['<cap>', '<cap>', '<cap>', \"'\", '.', '<cap>', \"'\", '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '$', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '<nop>', '.']\n\nVarify that it works by recovering the original string:\n========================\n ATI'd. I'm not sure if $10 is enough. \n"
        }
      ]
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "### Accuracy Metric\n\nThe accuracy metric is the standard precision/recall plus F-score. The implementation can be found in script [metric.py](metric.py), referenced from wikipedia and two other papers."
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:07:01.366362Z",
          "start_time": "2017-03-29T05:07:01.350965-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "import metric",
      "execution_count": 9,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Key: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 99.1\u001b[0m%\tRecall: \u001b[32m 96.6\u001b[0m%\tF-Score: \u001b[32m 97.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m100.0\u001b[0m%\tRecall: \u001b[32m 75.0\u001b[0m%\tF-Score: \u001b[32m 85.7\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 66.7\u001b[0m%\tRecall: \u001b[32m100.0\u001b[0m%\tF-Score: \u001b[32m 80.0\u001b[0m%\u001b[0m\n\n\ntrue_p:\t{'<nop>': 114, '<cap>': 3, \"'\": 2}\np:\t{'<nop>': 118, '<cap>': 4, \"'\": 2}\nall_p:\t{'<nop>': 115, ',': 2, '<cap>': 3, '.': 1, \"'\": 3}\n"
        }
      ]
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "### Report Progress Via Visdom\n\nTo use visdom, refer to doc and run server\n```sh\npython visdom.server\n```"
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-29T10:07:01.530379Z",
          "start_time": "2017-03-29T05:07:01.367130-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "# https://github.com/facebookresearch/visdom#setup\nimport visdom\nvis = visdom.Visdom()\nvis.env = \"deep-punc-experiment\"\n\nemb = None\nopt = None\nsmx = None\nlss = None\ndef plot_progress(embeded, output, softmax, losses):\n    global emb, opt, smx, lss\n    emb = vis.heatmap(embeded, win=emb or None, opts=dict(title=\"Input Embedding\"))\n    opt = vis.heatmap(output, win=opt or None, opts=dict(title=\"GRU Output\"))\n    smx = vis.heatmap(softmax, win=smx or None, opts=dict(title=\"Softmax Activation\"))\n    lss = vis.line(Y=losses, win=lss or None, opts=dict(title=\"Loss\"))",
      "execution_count": 10,
      "outputs": []
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T09:31:20.636924Z",
          "start_time": "2017-03-30T04:31:20.583491-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "import utils, data, metric, model\nfrom tqdm import tqdm\nimport numpy as np\nfrom IPython.display import HTML, clear_output\nfrom termcolor import cprint, colored as c\n\ninput_chars = list(\" \\nabcdefghijklmnopqrstuvwxyz01234567890\")\noutput_chars = [\"<nop>\", \"<cap>\"] + list(\".,;:?!\\\"'$\")\n\n# torch.set_num_threads(8)\nbatch_size = 128\n\nchar2vec = utils.Char2Vec(chars=input_chars, add_unknown=True)\noutput_char2vec = utils.Char2Vec(chars = output_chars)\ninput_size = char2vec.size\noutput_size = output_char2vec.size\n\ncprint(\"input_size is: \" + c(input_size, 'green') + \"; ouput_size is: \" + c(output_size, 'green'))\nhidden_size = input_size\nlayers = 1\n\nrnn = model.GruRNN(input_size, hidden_size, output_size, batch_size=batch_size, layers=layers, bi=True)\negdt = model.Engadget(rnn, char2vec, output_char2vec)",
      "execution_count": 55,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "input_size is: \u001b[32m40\u001b[0m; ouput_size is: \u001b[32m11\u001b[0m\u001b[0m\n****** Engadget Model Initialize ******\n"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T08:27:51.095515Z",
          "start_time": "2017-03-30T03:27:51.066110-05:00"
        },
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "**Pre-trained model**: run the cell below (after uncomment) to load the pre-trained model"
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T09:31:22.943166Z",
          "start_time": "2017-03-30T04:31:22.817302-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "egdt.load('./data/Gru_Engadget_epch-24.tar')",
      "execution_count": 56,
      "outputs": []
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "## Training\n\nThe training below initializes the hidden layer at the beginning of each file. I believe this makes it more consistent, and easier for the network to converge."
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T19:11:23.247798Z",
          "start_time": "2017-03-30T04:31:41.324895-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "scrolled": false,
        "trusted": false
      },
      "cell_type": "code",
      "source": "learning_rate = 0.5e-4\negdt.setup_training(learning_rate)\n\nseq_length = 500\n\nfor epoch_num in range(24, 48):\n    \n    \n    for batch_ind, (max_len, sources) in enumerate(tqdm(data.batch_gen(data.train_gen(), batch_size))):\n        \n        # prepare the input and output chunks\n        input_srcs = []; punc_targs = []\n        for chunk in sources:\n            input_source, punctuation_target = data.extract_punc(chunk, egdt.char2vec.chars, egdt.output_char2vec.chars)\n            input_srcs.append(input_source)\n            punc_targs.append(punctuation_target)        \n        \n        # at the begining of the file, reset hidden to zero\n        egdt.init_hidden_(random=False)\n        seq_len = data.fuzzy_chunk_len(max_len, seq_length)\n        for input_, target_ in zip(zip(*[data.chunk_gen(seq_len, src) for src in input_srcs]), \n                                   zip(*[data.chunk_gen(seq_len, tar, [\"<nop>\"]) for tar in punc_targs])):\n            \n            try:\n                egdt.forward(input_, target_)\n                egdt.descent()\n                    \n            except KeyError:\n                print(source)\n                raise KeyError\n        \n        if batch_ind%25 == 24:\n            if batch_ind%100 == 99:\n                clear_output(wait=True)\n            \n            print('Epoch {:d} Batch {}'.format(epoch_num + 1, batch_ind + 1))\n            print(\"=================================\")\n            punctuation_output = egdt.output_chars()\n            plot_progress(egdt.embeded[0,:400].data.numpy().T, \n                          egdt.output[0,:400].data.numpy().T, \n                          egdt.softmax[0,:400].data.numpy().T,\n                          np.array(egdt.losses))\n\n            metric.print_pc(utils.flatten(punctuation_output), utils.flatten(target_) )\n            print('\\n')\n            \n        if batch_ind%100 == 99:\n            \n            validate_target = data.apply_punc(input_[0], target_[0])\n            result = data.apply_punc(input_[0], \n                                     punctuation_output[0] )\n            print(validate_target)\n            print(result)\n            \n    # print('Dev Set Performance {:d}'.format(epoch_num))\n    egdt.save('./data/engadget_train_epoch-{}_batch-{}.tar'.format(epoch_num + 1, batch_ind))",
      "execution_count": 57,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Epoch 48 Batch 300\n=================================\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "\r300it [08:40,  4.06s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Key: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 97.8\u001b[0m%\tRecall: \u001b[32m 97.7\u001b[0m%\tF-Score: \u001b[32m 97.7\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 35.7\u001b[0m%\tRecall: \u001b[32m 34.5\u001b[0m%\tF-Score: \u001b[32m 35.1\u001b[0m%\u001b[0m\nKey: \u001b[31m  \"\u001b[0m\tPrec: \u001b[32m 24.3\u001b[0m%\tRecall: \u001b[32m 22.1\u001b[0m%\tF-Score: \u001b[32m 23.1\u001b[0m%\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 62.8\u001b[0m%\tRecall: \u001b[32m 64.8\u001b[0m%\tF-Score: \u001b[32m 63.8\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 52.2\u001b[0m%\tRecall: \u001b[32m 51.8\u001b[0m%\tF-Score: \u001b[32m 52.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 79.4\u001b[0m%\tRecall: \u001b[32m 79.9\u001b[0m%\tF-Score: \u001b[32m 79.6\u001b[0m%\u001b[0m\nKey: \u001b[31m  :\u001b[0m\tPrec: \u001b[32m 26.7\u001b[0m%\tRecall: \u001b[32m 33.3\u001b[0m%\tF-Score: \u001b[32m 29.6\u001b[0m%\u001b[0m\nKey: \u001b[31m  ?\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  $\u001b[0m\tPrec: \u001b[32m 67.5\u001b[0m%\tRecall: \u001b[32m 64.3\u001b[0m%\tF-Score: \u001b[32m 65.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  !\u001b[0m\tPrec: \u001b[32m  9.1\u001b[0m%\tRecall: \u001b[32m 16.7\u001b[0m%\tF-Score: \u001b[32m 11.8\u001b[0m%\u001b[0m\nKey: \u001b[31m  ;\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\n\n\nou the ability to share the fruits of your labor your, ahem, \"Rappertoire\") all of the requisite ways  email, Twitter and Facebook. After all, the dopest beats ought not be kept to yourself.\n\n \n In the end, AutoRap doesn't amount to too much more than a fun distraction  apologies in advance to all of those hoping to jumpstart their burgeoning rap careers. Sucker MCs can download it today from Google Play and the Apple App Store.\n\n\n                                                                                                                               \nou the ability to share the fruits of your labor your aHem. \"Rappertoire,) all of the requisite ways  email Twitter, and Facebook after all, the dopest Beats ought not be \"kept to yourself.\n\n \n In the end autoraP doesn't amount To too much more than a fun distraction  apologIes in advance to all of 'those hoping to jumpstart their burgeoning RAp careers sucker. MCS can download it today from Google play and the Apple app store.\n\n\n                                                                                                                               \n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "324it [10:53,  6.68s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Epoch 48 Batch 325\n=================================\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "\r325it [11:00,  6.96s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Key: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 98.1\u001b[0m%\tRecall: \u001b[32m 98.2\u001b[0m%\tF-Score: \u001b[32m 98.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 76.2\u001b[0m%\tRecall: \u001b[32m 71.2\u001b[0m%\tF-Score: \u001b[32m 73.6\u001b[0m%\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 68.2\u001b[0m%\tRecall: \u001b[32m 66.3\u001b[0m%\tF-Score: \u001b[32m 67.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 34.5\u001b[0m%\tRecall: \u001b[32m 33.5\u001b[0m%\tF-Score: \u001b[32m 34.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 59.2\u001b[0m%\tRecall: \u001b[32m 61.2\u001b[0m%\tF-Score: \u001b[32m 60.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  :\u001b[0m\tPrec: \u001b[32m 32.1\u001b[0m%\tRecall: \u001b[32m 36.2\u001b[0m%\tF-Score: \u001b[32m 34.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  $\u001b[0m\tPrec: \u001b[32m 77.8\u001b[0m%\tRecall: \u001b[32m 77.8\u001b[0m%\tF-Score: \u001b[32m 77.8\u001b[0m%\u001b[0m\nKey: \u001b[31m  \"\u001b[0m\tPrec: \u001b[32m 55.0\u001b[0m%\tRecall: \u001b[32m 31.9\u001b[0m%\tF-Score: \u001b[32m 40.4\u001b[0m%\u001b[0m\nKey: \u001b[31m  ;\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  ?\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  !\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\n\n\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "349it [14:36, 10.71s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Epoch 48 Batch 350\n=================================\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "\r350it [14:48, 10.89s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Key: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 97.2\u001b[0m%\tRecall: \u001b[32m 97.2\u001b[0m%\tF-Score: \u001b[32m 97.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 76.8\u001b[0m%\tRecall: \u001b[32m 76.8\u001b[0m%\tF-Score: \u001b[32m 76.8\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 43.8\u001b[0m%\tRecall: \u001b[32m 41.8\u001b[0m%\tF-Score: \u001b[32m 42.8\u001b[0m%\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 66.6\u001b[0m%\tRecall: \u001b[32m 67.6\u001b[0m%\tF-Score: \u001b[32m 67.1\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 32.1\u001b[0m%\tRecall: \u001b[32m 32.3\u001b[0m%\tF-Score: \u001b[32m 32.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  ?\u001b[0m\tPrec: \u001b[32m 10.0\u001b[0m%\tRecall: \u001b[32m 12.5\u001b[0m%\tF-Score: \u001b[32m 11.1\u001b[0m%\u001b[0m\nKey: \u001b[31m  !\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  :\u001b[0m\tPrec: \u001b[32m 34.5\u001b[0m%\tRecall: \u001b[32m 40.8\u001b[0m%\tF-Score: \u001b[32m 37.4\u001b[0m%\u001b[0m\nKey: \u001b[31m  \"\u001b[0m\tPrec: \u001b[32m 43.6\u001b[0m%\tRecall: \u001b[32m 37.5\u001b[0m%\tF-Score: \u001b[32m 40.3\u001b[0m%\u001b[0m\nKey: \u001b[31m  $\u001b[0m\tPrec: \u001b[32m 67.5\u001b[0m%\tRecall: \u001b[32m 60.0\u001b[0m%\tF-Score: \u001b[32m 63.5\u001b[0m%\u001b[0m\nKey: \u001b[31m  ;\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\n\n\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "374it [21:16, 21.98s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Epoch 48 Batch 375\n=================================\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "\r375it [21:41, 22.87s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Key: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 97.3\u001b[0m%\tRecall: \u001b[32m 97.3\u001b[0m%\tF-Score: \u001b[32m 97.3\u001b[0m%\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 68.2\u001b[0m%\tRecall: \u001b[32m 67.6\u001b[0m%\tF-Score: \u001b[32m 67.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 33.1\u001b[0m%\tRecall: \u001b[32m 36.2\u001b[0m%\tF-Score: \u001b[32m 34.6\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 45.7\u001b[0m%\tRecall: \u001b[32m 42.6\u001b[0m%\tF-Score: \u001b[32m 44.1\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 76.8\u001b[0m%\tRecall: \u001b[32m 76.5\u001b[0m%\tF-Score: \u001b[32m 76.6\u001b[0m%\u001b[0m\nKey: \u001b[31m  :\u001b[0m\tPrec: \u001b[32m 40.7\u001b[0m%\tRecall: \u001b[32m 44.6\u001b[0m%\tF-Score: \u001b[32m 42.6\u001b[0m%\u001b[0m\nKey: \u001b[31m  ;\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  !\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  ?\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  $\u001b[0m\tPrec: \u001b[32m 52.9\u001b[0m%\tRecall: \u001b[32m 55.1\u001b[0m%\tF-Score: \u001b[32m 54.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  \"\u001b[0m\tPrec: \u001b[32m 37.0\u001b[0m%\tRecall: \u001b[32m 28.2\u001b[0m%\tF-Score: \u001b[32m 32.0\u001b[0m%\u001b[0m\n\n\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "380it [24:13, 31.83s/it]\n"
        }
      ]
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "Let's take a look at the performance!"
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T19:11:23.826187Z",
          "start_time": "2017-03-30T14:11:23.248846-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "egdt.save('./data/Gru_Engadget_epch-48.tar')",
      "execution_count": 58,
      "outputs": []
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T07:16:59.391920Z",
          "start_time": "2017-03-30T02:16:59.311743-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "print('Epoch {:d} Batch {} Training P/R'.format(epoch_num + 1, batch_ind + 1))\nprint(\"=================================\")\npunctuation_output = egdt.output_chars()\n_ = metric.print_pc(utils.flatten(punctuation_output), utils.flatten(target_) )",
      "execution_count": 20,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Epoch 24 Batch 760\n=================================\nKey: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 97.4\u001b[0m%\tRecall: \u001b[32m 97.5\u001b[0m%\tF-Score: \u001b[32m 97.4\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 36.5\u001b[0m%\tRecall: \u001b[32m 37.8\u001b[0m%\tF-Score: \u001b[32m 37.1\u001b[0m%\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 64.9\u001b[0m%\tRecall: \u001b[32m 62.2\u001b[0m%\tF-Score: \u001b[32m 63.5\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 29.3\u001b[0m%\tRecall: \u001b[32m 29.3\u001b[0m%\tF-Score: \u001b[32m 29.3\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 76.0\u001b[0m%\tRecall: \u001b[32m 75.5\u001b[0m%\tF-Score: \u001b[32m 75.7\u001b[0m%\u001b[0m\nKey: \u001b[31m  \"\u001b[0m\tPrec: \u001b[32m 53.1\u001b[0m%\tRecall: \u001b[32m 54.8\u001b[0m%\tF-Score: \u001b[32m 54.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  !\u001b[0m\tPrec: \u001b[32m 20.0\u001b[0m%\tRecall: \u001b[32m 33.3\u001b[0m%\tF-Score: \u001b[32m 25.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  :\u001b[0m\tPrec: \u001b[32m 48.0\u001b[0m%\tRecall: \u001b[32m 44.4\u001b[0m%\tF-Score: \u001b[32m 46.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  ?\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  $\u001b[0m\tPrec: \u001b[32m 61.5\u001b[0m%\tRecall: \u001b[32m 38.1\u001b[0m%\tF-Score: \u001b[32m 47.1\u001b[0m%\u001b[0m\nKey: \u001b[31m  ;\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\n"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T07:52:49.480465Z",
          "start_time": "2017-03-30T02:52:25.008323-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": true
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "punc_outputs = []\npunc_targets = []\nfor fn, source in tqdm(data.test_gen()):\n    input_text, punc_target = data.extract_punc(source, input_chars, output_chars)\n    egdt.model.batch_size = 1\n    egdt.init_hidden_()\n    egdt.next_([input_text])\n    punc_outputs.append(egdt.output_chars()[0])\n    punc_targets.append(punc_target)\n    \nprint('Test P/R  Epoch {:d} Batch {}'.format(epoch_num + 1, batch_ind + 1))\nprint(\"=================================\")    \n_ = metric.print_pc(utils.flatten(punc_outputs), utils.flatten(punc_targets))",
      "execution_count": 28,
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "25it [00:24,  4.14s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Epoch 24 Batch 760 Test P/R\n=================================\nKey: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 97.1\u001b[0m%\tRecall: \u001b[32m 97.8\u001b[0m%\tF-Score: \u001b[32m 97.4\u001b[0m%\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 68.6\u001b[0m%\tRecall: \u001b[32m 57.8\u001b[0m%\tF-Score: \u001b[32m 62.7\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 30.8\u001b[0m%\tRecall: \u001b[32m 30.9\u001b[0m%\tF-Score: \u001b[32m 30.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 43.7\u001b[0m%\tRecall: \u001b[32m 38.3\u001b[0m%\tF-Score: \u001b[32m 40.8\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 76.9\u001b[0m%\tRecall: \u001b[32m 80.2\u001b[0m%\tF-Score: \u001b[32m 78.5\u001b[0m%\u001b[0m\nKey: \u001b[31m  :\u001b[0m\tPrec: \u001b[32m 10.3\u001b[0m%\tRecall: \u001b[32m  6.1\u001b[0m%\tF-Score: \u001b[32m  7.7\u001b[0m%\u001b[0m\nKey: \u001b[31m  \"\u001b[0m\tPrec: \u001b[32m 26.9\u001b[0m%\tRecall: \u001b[32m 45.1\u001b[0m%\tF-Score: \u001b[32m 33.7\u001b[0m%\u001b[0m\nKey: \u001b[31m  $\u001b[0m\tPrec: \u001b[32m 64.3\u001b[0m%\tRecall: \u001b[32m 61.6\u001b[0m%\tF-Score: \u001b[32m 62.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  ;\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  ?\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  !\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "\n"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T19:18:28.367775Z",
          "start_time": "2017-03-30T14:18:03.062362-05:00"
        },
        "collapsed": false,
        "run_control": {
          "frozen": false,
          "read_only": true
        },
        "trusted": false,
        "editable": true,
        "deletable": true
      },
      "cell_type": "code",
      "source": "punc_outputs = []\npunc_targets = []\nfor fn, source in tqdm(data.test_gen()):\n    input_text, punc_target = data.extract_punc(source, input_chars, output_chars)\n    egdt.model.batch_size = 1\n    egdt.init_hidden_()\n    egdt.next_([input_text])\n    punc_outputs.append(egdt.output_chars()[0])\n    punc_targets.append(punc_target)\n    \nprint('Test P/R  Epoch {:d} Batch {}'.format(epoch_num + 1, batch_ind + 1))\nprint(\"=================================\")    \n_ = metric.print_pc(utils.flatten(punc_outputs), utils.flatten(punc_targets))",
      "execution_count": 59,
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "25it [00:25,  4.18s/it]"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Test P/R  Epoch 48 Batch 380\n=================================\nKey: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 97.1\u001b[0m%\tRecall: \u001b[32m 98.0\u001b[0m%\tF-Score: \u001b[32m 97.6\u001b[0m%\u001b[0m\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 73.2\u001b[0m%\tRecall: \u001b[32m 58.9\u001b[0m%\tF-Score: \u001b[32m 65.3\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 35.7\u001b[0m%\tRecall: \u001b[32m 32.2\u001b[0m%\tF-Score: \u001b[32m 33.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 45.0\u001b[0m%\tRecall: \u001b[32m 39.7\u001b[0m%\tF-Score: \u001b[32m 42.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m 81.7\u001b[0m%\tRecall: \u001b[32m 83.4\u001b[0m%\tF-Score: \u001b[32m 82.5\u001b[0m%\u001b[0m\nKey: \u001b[31m  :\u001b[0m\tPrec: \u001b[32m 12.1\u001b[0m%\tRecall: \u001b[32m 10.8\u001b[0m%\tF-Score: \u001b[32m 11.4\u001b[0m%\u001b[0m\nKey: \u001b[31m  \"\u001b[0m\tPrec: \u001b[32m 25.2\u001b[0m%\tRecall: \u001b[32m 44.8\u001b[0m%\tF-Score: \u001b[32m 32.3\u001b[0m%\u001b[0m\nKey: \u001b[31m  $\u001b[0m\tPrec: \u001b[32m 51.4\u001b[0m%\tRecall: \u001b[32m 87.8\u001b[0m%\tF-Score: \u001b[32m 64.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  ;\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\nKey: \u001b[31m  ?\u001b[0m\tPrec: \u001b[32m  5.6\u001b[0m%\tRecall: \u001b[32m  4.8\u001b[0m%\tF-Score: \u001b[32m  5.1\u001b[0m%\u001b[0m\nKey: \u001b[31m  !\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\n"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": "\n"
        }
      ]
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "hide_input": false,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "## Now use the network to add punctuations!"
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T08:30:57.007047Z",
          "start_time": "2017-03-30T03:30:56.978818-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "from ipywidgets import widgets\nfrom IPython.display import display, clear_output",
      "execution_count": 38,
      "outputs": []
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "**Note**: Again, using a zero initial state for the hidden state gives a much better result for the prediction."
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T08:37:18.146106Z",
          "start_time": "2017-03-30T03:37:18.066217-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "def predict_next(source, gen_length=None, temperature=0.05):\n    global epoch_num\n    global batch_ind\n    \n    input_text, punc_target = data.extract_punc(source, input_chars, output_chars)\n    egdt.model.batch_size = 1\n    egdt.init_hidden_()\n    egdt.next_([input_text])\n    punc_output = egdt.output_chars(temperature=temperature)[0]\n    result = data.apply_punc(input_text, punc_output)\n    \n    clear_output()\n    print('Demo P/R (Epoch {:d} Batch {} )'.format(epoch_num + 1, batch_ind + 1))\n    print(\"=================================\")    \n    _ = metric.print_pc(punc_output, punc_target)\n    print(\"\\n\" + result)\n\npredict_next(\"This won't be a simple sentense. It doesnt have puntuations\\n\\nI'm not sure. \", 200, 1)",
      "execution_count": 48,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Demo P/R (Epoch 24 Batch 760 )\n=================================\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 66.7\u001b[0m%\tRecall: \u001b[32m100.0\u001b[0m%\tF-Score: \u001b[32m 80.0\u001b[0m%\u001b[0m\nKey: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 96.9\u001b[0m%\tRecall: \u001b[32m 96.9\u001b[0m%\tF-Score: \u001b[32m 96.9\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m100.0\u001b[0m%\tRecall: \u001b[32m 66.7\u001b[0m%\tF-Score: \u001b[32m 80.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m  0.0\u001b[0m%\tRecall: \u001b[32m  0.0\u001b[0m%\tF-Score:   N/A\u001b[0m\n\nThis won't be a simple sentense, it doesn't have puntuations.\n\nI'm not sure \n"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2017-03-30T08:51:08.825500Z",
          "start_time": "2017-03-30T03:51:08.792664-05:00"
        },
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "scrolled": false,
        "trusted": false
      },
      "cell_type": "code",
      "source": "text_input = widgets.Text()\ndisplay(text_input)\n\ndef handle_submit(sender):\n    #print(text_input.value)\n    predict_next(text_input.value, 2000, temperature=1)\n\nprint(\"Type in a sentence with punctuations just as you would normally have!\\nThe demo will strip off those punctuations, and try to predict them.\")\n\ntext_input.on_submit(handle_submit)",
      "execution_count": 51,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Demo P/R (Epoch 24 Batch 760 )\n=================================\nKey: \u001b[31m<cap>\u001b[0m\tPrec: \u001b[32m 65.0\u001b[0m%\tRecall: \u001b[32m 61.9\u001b[0m%\tF-Score: \u001b[32m 63.4\u001b[0m%\u001b[0m\nKey: \u001b[31m<nop>\u001b[0m\tPrec: \u001b[32m 97.4\u001b[0m%\tRecall: \u001b[32m 96.9\u001b[0m%\tF-Score: \u001b[32m 97.2\u001b[0m%\u001b[0m\nKey: \u001b[31m  .\u001b[0m\tPrec: \u001b[32m 16.7\u001b[0m%\tRecall: \u001b[32m100.0\u001b[0m%\tF-Score: \u001b[32m 28.6\u001b[0m%\u001b[0m\nKey: \u001b[31m  '\u001b[0m\tPrec: \u001b[32m100.0\u001b[0m%\tRecall: \u001b[32m100.0\u001b[0m%\tF-Score: \u001b[32m100.0\u001b[0m%\u001b[0m\nKey: \u001b[31m  ,\u001b[0m\tPrec: \u001b[32m 50.0\u001b[0m%\tRecall: \u001b[32m 40.0\u001b[0m%\tF-Score: \u001b[32m 44.4\u001b[0m%\u001b[0m\n\nSamsunG: GalaXY Note II, was a complete disaster, this is what happens when you don't control the entire softwarehardware stack Unlike Apple, Samsung can't optimize the Android OS as aggressively as Apple does to the IOs and as a result the power consumption can't be optimized, and the batter capacity is pushed to the brim, it turned out that lithium batteries Don't like to be pushed to the Brim. and When they are They explode   \n"
        }
      ]
    },
    {
      "metadata": {
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        }
      },
      "cell_type": "markdown",
      "source": "The demo looks pretty good ;-) "
    },
    {
      "metadata": {
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {
        "collapsed": false,
        "deletable": true,
        "editable": true,
        "run_control": {
          "frozen": false,
          "read_only": false
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "",
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "_draft": {
      "nbviewer_url": "https://gist.github.com/e261821a98490523e543170cd7229b8f"
    },
    "anaconda-cloud": {},
    "gist": {
      "id": "e261821a98490523e543170cd7229b8f",
      "data": {
        "description": "pytorch_playground/engadget_rnn/Learning Punctuations by reading Engadget.ipynb",
        "public": true
      }
    },
    "hide_input": false,
    "kernelspec": {
      "name": "deep-learning",
      "display_name": "deep-learning-python3",
      "language": "python"
    },
    "language_info": {
      "name": "python",
      "version": "3.6.0",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "toc": {
      "threshold": 4,
      "number_sections": false,
      "toc_cell": false,
      "toc_window_display": false,
      "toc_section_display": "block",
      "sideBar": true,
      "navigate_menu": true,
      "moveMenuLeft": true,
      "widenNotebook": false,
      "colors": {
        "hover_highlight": "#DAA520",
        "selected_highlight": "#FFD700",
        "running_highlight": "#FF0000"
      },
      "nav_menu": {
        "height": "107px",
        "width": "253px"
      },
      "toc_position": {
        "height": "556px",
        "left": "0px",
        "right": "auto",
        "top": "130px",
        "width": "413px"
      }
    },
    "widgets": {
      "state": {
        "17298efebe594dc1b94fb6ba924f3c4b": {
          "views": [
            {
              "cell_index": 9
            }
          ]
        },
        "7e82dbc7be6b4d6a824f1c93e8f5826e": {
          "views": [
            {
              "cell_index": 10
            }
          ]
        }
      },
      "version": "1.2.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}