FGM_2.ipynb

{
 "metadata": {
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "General Structured Output Models with Shogun Machine Learning Toolbox"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Shell Hu (GitHub ID: [hushell](https://github.com/hushell))"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Thanks Patrick Pletscher and Fernando J. Iglesias Garc\u00eda for taking time to help me finish the project! Shoguners = awesome! Me = grateful!"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Introduction"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In this demo, we show how to train a factor graph model using structured SVM. We will go through the basic knowledge of factor graph and structured output learning. Next, the corresponding APIs in Shogun will be covered. For testing the scalability, we show an experiment on a real OCR dataset for handwritten character recognition. "
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Factor Graph"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A factor graph explicitly represents the factorization of an undirected graphical model in terms of a set of factors (potentials), each of which is defined on a clique in the original graph [1]. For example, a MRF distribution can be factorized as \n",
      "\n",
      "$$\n",
      "P(\\mathbf{y}) = \\frac{1}{Z} \\prod_{F \\in \\mathcal{F}} \\theta_F(\\mathbf{y}_F),\n",
      "$$\n",
      "\n",
      "where $F$ is the factor index, $\\theta_F(\\mathbf{y}_F)$ is the energy with respect to assignment $\\mathbf{y}_F$. In this demo, we focus only on table representation of factors. Namely, each factor holds an energy table $\\theta_F$, which can be viewed as an unnormalized CPD. According to different factorizations, there are different types of factors. Usually we assume the Markovian property is held, that is, factors have the same parameterization if they belong to the same type, no matter how location or time changes. In addition, we have parameter free factor type, but nothing to learn for such kinds of types. More detailed implementation will be explained later."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Structured Prediction"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Structured prediction typically involves an input $\\mathbf{x}$ (can be structured) and a structured output $\\mathbf{y}$. A joint feature map $\\Phi(\\mathbf{x},\\mathbf{y})$ is defined to incorporate structure information into the labels, such as chains, trees or general graphs. In general, the linear parameterization will be used to give the prediction rule. We leave the kernelized version for future work.\n",
      "\n",
      "$$\n",
      "\\hat{\\mathbf{y}} = \\underset{\\mathbf{y} \\in \\mathcal{Y}}{\\operatorname{argmax}} \\langle \\mathbf{w}, \\Phi(\\mathbf{x},\\mathbf{y}) \\rangle \n",
      "$$\n",
      "\n",
      "where $\\Phi(\\mathbf{x},\\mathbf{y})$ is the feature vector by mapping local factor features to corresponding locations in terms of $\\mathbf{y}$, and $\\mathbf{w}$ is the global parameter vector. In factor graph model, parameters are associated with a set of factor types. So $\\mathbf{w}$ is a collection of local parameters. \n",
      "\n",
      "The parameters are learned by regularized risk minimization, where the risk defined by user provided loss function $\\Delta(\\mathbf{y},\\mathbf{\\hat{y}})$ is usually non-convex and non-differentiable, e.g. the Hamming loss. So the empirical risk is defined in terms of the surrogate hinge loss $H_i(\\mathbf{w}) = \\max_{\\mathbf{y} \\in \\mathcal{Y}} \\Delta(\\mathbf{y}_i,\\mathbf{y}) - \\langle \\mathbf{w}, \\Psi_i(\\mathbf{y}) \\rangle $, which is an upper bound of the user defined loss. Here $\\Psi_i(\\mathbf{y}) = \\Phi(\\mathbf{x}_i,\\mathbf{y}_i) - \\Phi(\\mathbf{x}_i,\\mathbf{y})$. The training objective is given by\n",
      "\n",
      "$$\n",
      "\\min_{\\mathbf{w}} \\frac{\\lambda}{2} ||\\mathbf{w}||^2 + \\frac{1}{N} \\sum_{i=1}^N H_i(\\mathbf{w}). \n",
      "$$\n",
      "\n",
      "In Shogun's factor graph model, the corresponding implemented functions are:\n",
      "\n",
      "- <a href=\"http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CStructuredModel.html#a15bd99e15bbf0daa8a727d03dbbf4bcd\">FactorGraphModel::get_joint_feature_vector()</a> $\\longleftrightarrow \\Phi(\\mathbf{x}_i,\\mathbf{y})$ \n",
      "\n",
      "- <a href=\"http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CFactorGraphModel.html#a36665cfdd7ea2dfcc9b3c590947fe67f\">FactorGraphModel::argmax()</a> $\\longleftrightarrow H_i(\\mathbf{w})$\n",
      "\n",
      "- <a href=\"http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CFactorGraphModel.html#a17dac99e933f447db92482a6dce8489b\">FactorGraphModel::delta_loss()</a> $\\longleftrightarrow \\Delta(\\mathbf{y}_i,\\mathbf{y})$"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Experiment: OCR"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Show Data"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "First of all, we load the OCR data from a prepared mat file. The raw data can be downloaded from <a href=\"http://www.seas.upenn.edu/~taskar/ocr/\">http://www.seas.upenn.edu/~taskar/ocr/</a>. It has 6876 handwritten words with an average length of 8 letters from 150 different persons. Each letter is rasterized into a binary image of size 16 by 8 pixels. Thus, each $\\mathbf{y}$ is a chain, and each node has 26 possible states denoting ${a,\\cdots,z}$. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%pylab inline\n",
      "import numpy as np\n",
      "import scipy.io"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Populating the interactive namespace from numpy and matplotlib\n"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dataset = scipy.io.loadmat('../../../data/ocr/ocr_taskar.mat')\n",
      "# patterns for training\n",
      "p_tr = dataset['patterns_train']\n",
      "# patterns for testing\n",
      "p_ts = dataset['patterns_test']\n",
      "# labels for training\n",
      "l_tr = dataset['labels_train']\n",
      "# labels for testing\n",
      "l_ts = dataset['labels_test']\n",
      "\n",
      "# feature dimension\n",
      "n_dims = p_tr[0,0].shape[0]\n",
      "# number of states\n",
      "n_stats = 26\n",
      "# number of training samples\n",
      "n_tr_samples = p_tr.shape[1]\n",
      "# number of testing samples\n",
      "n_ts_samples = p_ts.shape[1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Few examples of the handwritten words are shown below. Note that the first capitalized letter has been removed. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import matplotlib.pyplot as plt\n",
      "\n",
      "def show_word(patterns, index):\n",
      "    \"\"\"show a word with padding\"\"\"\n",
      "    plt.rc('image', cmap='binary')\n",
      "    letters = patterns[0,index][:128,:]\n",
      "    n_letters = letters.shape[1]\n",
      "    for l in xrange(n_letters):\n",
      "        lett = np.transpose(np.reshape(letters[:,l], (8,16)))\n",
      "        lett = np.hstack((np.zeros((16,1)), lett, np.zeros((16,1))))\n",
      "        lett = np.vstack((np.zeros((1,10)), lett, np.zeros((1,10))))\n",
      "        subplot(1,n_letters,l)\n",
      "        imshow(lett)\n",
      "        plt.xticks(())\n",
      "        plt.yticks(())\n",
      "    plt.tight_layout()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "show_word(p_tr, 174)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAA/CAYAAAC1plyPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnelbGln2x78sCi4gm+IGohAVFJeYaPpnYqeT7sykl5ln\n5nnmmTfzJ84zPd3TT9KTdLrtTtIxJC4RFIkKIi6gbAFFdn4vpu9N0ZFEDVWQpD4vseAe61bdU3Xu\nOecrKBQKBfDw8PDw8FQZwkobwMPDw8PDcxy8g+Lh4eHhqUp4B8XDw8PDU5XwDoqHh4eHpyoRl/qD\nQCDg0g4AwEnyNarVLoB726rVLoCfy9PC23V6qtU23q7TU8q2kg7qdV9ig9OclGq1CyifbYVCAfl8\nHrlcDgAgEokgFAqpPZWy6yTwc3k6eLtefj+XyyGfzxf9llAohFAohEgkqphtzN95F+/LarULeL1t\nr3VQPJUjkUjA7XZjfX0dAoEAPT09MBqNqK+vr7RpPDysEIlE4Ha74fF4cHBwQD9vb29Hd3c3uru7\nUVNTU0EL+fuSa3gHVaUkEgksLy/j3r17EAqFuHbtGtrb2/kbgee9JRwOY2FhAdPT09jb26Ofj4yM\n4JNPPoFOp6sKB8Xfl9zBO6gqolAoIJlMIpVKYXt7Gw6HAw8ePEBtbS16enqQSqUqbSIPD2vE43Gs\nra3h/v372N3dhUQigVQqhVwux9DQEA2rVZJkMgmPx4OZmRkIhUL+vmQZ3kFVEblcDuvr61hZWYHD\n4cDCwgKi0ShaWloqbRoPD6doNBr09fWht7cXw8PDMBqNFX974uEe3kFVEdlsFmtra/jhhx/w+PFj\nhEIh3kHxfJA0NzdjbGwMN27cgF6vh1qthljML1cfGhWZ8UQigYODA8TjcdTV1UEmk1XCjKojn89j\nb2+PvkHJZDI0NzdDr9dDqVTyNyjPe0c6nUY8HsfBwQE2NzcRDoeRzWbR2NgIvV6P0dFRNDc3V9TG\nQqGAWCyGeDwOj8eDUCiEVCqFurq6itr1IVCRFS8QCMDhcGBpaQldXV0YHByshBlVTX19PSwWC6xW\nK4aGhmCxWPgbgue9IxaL0bXA4XDA6XTi8PCw0mYVkc1msb6+DrvdjsXFRSwuLuLFixf8/cgBFXFQ\nfr8fNpsNt27dwsWLF9HQ0FAJM6oa4qA+//xzWK1WyOVy/obgee+IxWKw2+24desWnE4n4vE4EolE\npc0qIpfL0dD7r7/+ing8jlgshtbW1kqb9t7DmYM6ODhAOBymqaQOhwMrKytob29HLBbjyozXEovF\nEIlEEA6HIZPJoFQqORmXjLm7uwuv14t4PI6amhqo1WoYDAYYDAZWx89kMtSGTCYDpVIJlUpVlDqb\nSCTo/KXT6WN/R6FQcHLOkskktSWZTL7xeK7sAl7OZSQSeeOxMpkMarWaA6teks/nEQqFEA6HEY/H\nK25XOp1GKBSCx+PB/v4+VCoVOjs70d/fD61WWxVh7Xw+j2g0Cp/Ph62tLahUKphMJlgsFrS2tvLJ\nG7/Bxn3J2ewHg0HMz89jfn4eTqcT6+vryGazXA1/IgKBAObn57GwsACj0YiRkRFOxvX5fJidnaXn\nZm9vj9N2I8lkEi6XC/Pz84jFYhgdHcXIyEiRg4pEIlhYWMD8/Dyi0eixv2OxWDg5Z+Spe35+Hvv7\n+288niu7gJdz6XA43nis0WjE2NgYB1a9hISrZmdn4Xa7q8YuAFAqlRgZGcHo6CgsFgt6e3shlUo5\nt+N1kMjG2NgYrFYrzGYzH9n4DTbuS04d1OzsLP79739jd3cXR0dHyGQyXA1/IgKBAGw2G7755ht8\n9NFHUCgUnIzr8/lw//593Lp1C0dHR0gmk9BqtZyMDbx0UHfu3KHO0WAwoKOjgx5DHNQ333yD7e3t\nY3/ns88+4+SckRvhu+++w/r6+huP58ou4OVc3r59+43HTk5OcvZmRyDhqnv37uHRo0dVYxfw0kH9\n+c9/Rnd3N6RSKSQSCed2vA7ioG7evAmr1Yq6ujreQf0GG/clqw4qHo8jEAggEAhgbm4Oy8vL2Nra\nglAoRFdXF7RaLYaHh6smlptKpRCLxRAIBBCJRHB0dMTJuMlkEtFoFMFgEK2trTCZTOjt7YXRaERj\nYyPr4+fzeRweHiIcDmN/fx8HBwfIZDJIJBLw+/0IBAJYXFzE0tISfD4fUqkUtFottFotDg4O6ByX\n+5yl02n62y9evKCf+3w+LC4uYnNzE4eHh9QW5tN2OBxmzS7gfyFrv98Pv99fVKj55MkTrK6uYn9/\nH62trdBqtUWLPbkn/H4/wuEwZ/ste3t78Pv92NrawtzcHNbX1xGNRqHVatHa2opCocCZXclkks7N\n8vIy3G43EokENBoNGhsb0dzcDJVKxdr4b4NIJEJjYyM0Gg00Gk2lzakqstksDg4OEAwG4ff76edN\nTU1nXi9YdVDkqdtms8HpdMLtdiOVSqG7uxvj4+O4ePEient70d3dzaYZ7wwikQhGoxGXLl3C+fPn\nYTQaOXvyPw7yRGSz2YoWEq1Wi7GxMUxMTMDn8+HJkycIh8NlH5+82dlstqJwVDQahdvtRiwWo0/d\n4+PjRYua0+lkzS7g5bU9MzNTFPLc2NiAz+crmsve3l76962tLczMzBQ5XC7w+XyYmZnBkydPsLa2\nhr29Pfo2cOnSJRQKBc7sOjg4wPLyMmw2G+x2O51LfsF/P9FoNGdeLzhxUN988w28Xi9yuRxyuRy0\nWi3Gx8fxt7/9DU1NTUVdij9kxGIxTCYTrl+/jqmpKYhEooqeG+Yru8vlovNHiij/+te/wuFwIBwO\nY25uruzjEwd19+5dPH78mH5eKBSQzWaRy+VgMBhoWEin09FjpqenWbML+J+TnJ+fx9dff43d3V36\nOTlHEomEzuWVK1fo3+12O168eIFnz56xYlcpSOjx22+/peeuvb2dZorm83nO7Do8PMTy8jJu376N\nZ8+eUXt43k/eZr14o4PK5/PY2trC1tYWQqEQOjs70dnZeaLiuXw+j2w2i2QyicbGRvrdCxcuoLe3\nF42NjaitrT2RoScllUpRe5PJJB2zqanp2ONDoRA9/unTp/B4PBVN3hCLxbQHGZtks1malbS2tobF\nxcVXnmrI/JGiRHIuR0dHYTab0dTUhNraWojFYlaSOogjSqVSEAgEdHzmW2V3dzcsFgtUKhUSiUTR\nXPp8PtYWPua1XSpMQeaSuUchkUhQU1PDSRLM/v4+fD4ffD4ffv31V2xsbCCdTkOv10On00Gv16O+\nvp7W9Xg8Hk76yhUKBWQyGaRSKdTU1KC7uxudnZ0YGhpCb29v1TRe3dnZwebmJnw+H6LRKMxmM3p7\nezEwMFByPfnQYN5zS0tLtFRAoVCUZb14o4MqFArY2NjAgwcPsLKygsnJSdTU1Jy6upu85k1OTsJk\nMrHWmZg8dT98+BCRSASTk5NoaGgoeUGR5I2HDx9ibW0NPp+v6pI32CCTyWB9fR0PHz6ki3kwGCzZ\n1YOE0iYnJ9HX1wedTsfpBnZDQwMsFgsmJyeL0u4VCgV0Oh3q6+tpBt2HNpelCAQCePz4Me7fv09D\nj2KxGEajEVeuXIHBYIDX68Wvv/4Kr9cLn8/HeQ2SXC6H1WrF5OQkLBYLnctqYHNzE/fv38fs7CwM\nBgOGhobQ1dUFvV7PO6jfIG/DDx8+hN1uh8/nQzweh0ajKct6caI3KK/Xi4cPH+LBgweoq6vDuXPn\nSh7PFB3LZDLIZrMoFApQq9UYHh7GV199xWrrkmQyidXVVfz000/Y3d1FU1MTBgYGStro9/sxNzeH\nb7/9FuFwmAqj1dTUsBpey+fzVPisEiGOTCYDt9uNX375BT/99BMVhSP7ONlsFul0ms6fQqHA4OAg\nPv/8c+j1elZtI3NDxs/n82hoaEB/fz8+/fTTotTUaphLJmRMiUQCsVgMoVDIybgE5vnY2dmBzWbD\n119/jXQ6DZFIhIaGBhiNRly+fBldXV3w+Xx4/PgxlpaWqPgel+dLJpPBYrHgD3/4A/r6+jgZ86Rs\nb2/j0aNH+O9//4t//OMfuHnzJj7++GNWx2QKIpYSDSTXGNfX1nEcHh5iZWUFP/zwAxYWFujnJpOp\nLOtF2fegYrEYFR2z2+1wOp1VVxnOFEZ79uwZXC4XzUzr6elBd3c3Lly4wGqBbCAQoMJnNpsNW1tb\nrI31JqRSKRWE0+l0qK2txczMDOLxOBwOB2KxGGfdPjKZDDweD9xuN1wuF+bm5hAMBkseXw1zyUSn\n08FoNOLcuXMYHx/nvNEvOR/r6+s0Wy+Xy1G7uru7oVKpsLi4SGv+otEolEolFd8jCTo83MMURCxV\n7G0wGNDT04Ouri6OreMeVhyU3W7H9PQ0nE4n/H5/1fXWYgqjPX/+HIFAAKlUCiaTCePj47h69Sr0\nej2rtUh7e3t4/PgxfvzxR/h8vqK0TK6RSqXo6+vD1atX0dHRAafTiZmZGXg8Hvj9fsRiMbS1tXFi\nSyaTwdraGqanp2Gz2eD3+xEMBkvW5VTDXDLR6XS4cuUKpqam0NbWxrmDikajWFhYwL1797C6ugq/\n308d1JUrVzAxMYGlpSXY7Xb6d9Ixf2RkBNevX8e5c+eqpvTjQ4MpiLixsXHsMZcvX0ZtbS3voE5K\nLpdDKpVCMpmEz+eD3W6ncW+pVIqGhgYolUrU19ez8lrKFPoLBoO0juf3NiaTSSSTSXi9Xjx79gzT\n09NFRacqlQpmsxmffPIJ6x3Ww+EwvRALhQKkUimam5shk8nKnjjyJmpra6HX63Hx4kW0tbVhdXWV\nLmJSqRR1dXVQqVRoaGhgLfRD5iYUCsHpdOLRo0d48OAB/TvTQZWay729PUgkEsjlcvT09OD8+fO4\ndu1a2WrJ8vk8HTccDuPw8BD5fB61tbU0saW3txfj4+O4du3asb8hFovR0NAAjUYDhUJRliJPpl1e\nr5c67HA4DIlEArVaDb1ej76+PlgsFiwvL9NzTCBvfVNTU2hvb39rm46zkawRoVAIh4eHyGazyGaz\ntAaPqaJbU1NDC3W5DGWl02lqZywW4yxp5DihUqfTeezxZN+uUpSaS5FIBKlUCqlUWrb1oiwO6vDw\nEM+fP4fL5aILG6lRIaJjJEOHjY11suHvcrngdDqPDQvFYjG4XC64XC7aNblaQo86nQ79/f0wm824\ncOFC1eg/yeVy9PX1oa+vD4ODgzCbzaxsYOfzeXg8Hjx//hwrKyuYm5tDIBAoeXypuWRb5O7w8BAu\nlwsrKytYXFyE0+nE0dERWlpa0N/fj/7+fpw/f74o3f33KJVKjI6OIpfLoaWlpSyhtJPYJZfL4fF4\naIYj0xlwQTKZpGvE8vIyFhYWEIlEkMlkYLfbX0m8am9vR29vL/r6+jjt1EDkbsh16PP5WB+zlFBp\ntVJqLtlYL8rmoJaXl3H37l08e/YM4XAYsVgMXV1dGBkZwY0bN2A0GqFWq1lJnyZhoR9//BGPHj2i\nzTCZhX8k9Hjnzh2srKxQz18N6HQ6TE1N4ZNPPoFGo+G8gWgpyJPajRs3YDab6VNRucnn89jY2MD0\n9DTu379P568UpeZSp9OxKnJHwi/ff/897HY7QqEQEokETCYTJiYm8Mc//hGtra2vnT+FQoGRkRHo\ndDpIpdKyzPVJ7FpeXsaTJ0+wsLCAYDCIUCj01uOeBmZN29OnT2lD3Xw+D7vdjp2dnaK1gbwhGAwG\nTh0UyXz8/vvv4ff7OTlPpYRKq5VSc9nc3Fz29aIsd28qlcLOzg4cDgdcLhdkMhm0Wi36+/sxNDSE\nsbExVsIGhFwuR598FhcXaSdyg8EAtVoNiUSCo6MjbG9vw263w+Vy0e9KpVLIZDLIZDJ0dHSgqamJ\n8+wY0h35woULnI5LyOVyiEaj2NraQiqVKhJk6+jogNVqZTXDqlAoIBgM4vnz55ifn6fzodFoEI/H\nEY/HkclkEA6Hsbm5SVsvzc7OYmdnh84dud4uXLjASqucdDpNr3NmM1i5XE6TMd70AFZfX4/6+vrX\nvmWdBKaIntvtht1ux+zsLLxeL2QyGTo7O2EwGNDW1galUol0Og2v14unT5++1bhnJZPJwO/3Y3l5\n+ZVi4O3t7Vf6O2YyGbS1taGrqwudnZ1obGyETCZj/d6MxWLweDx4+vQpxGIxPZctLS2sOcpSQqWl\nok1tbW2ctED7vY1EWNLn88HhcND2daT1E3O9L1d3oLInScjlcgwODmJgYIC+5nGp90Q2/AcGBmC1\nWjE4OAilUlnyiaS5uZnaOjg4CKPRWBUt/rmEPBHV1NSgsbERDocDkUikIm2WampqYDQaMTg4iObm\nZipkR9rjCIVCiMViOBwOxONxKJVKOneDg4Po6+urugajbFBKRE+hUFCRS5lMhmAwiH/+859wOp0V\nzRQ9LaQ+MZ1OF60nXO7P6nQ6ei6HhoY4Cb0zhUpLJfaYzWZ0dnaybgsTso1CHs4cDgeCwSCtT2Te\ng+Xcv2fFQVmtVnz++ecwm82QyWScFt4RB3Xjxg1cvHgRMpkMcrm8pIMiBcRffPEFOjs7IZfLPzh9\nl6OjI7hcLuzs7EAsFlNBtko5KJPJhGvXruHcuXOQSCS0YS1pNiwQCOjbA5FF+eKLL9DT0wOZTFZ1\nEg1sUEpEr7u7G6Ojo/jqq68QCAQwPT2Nn3/+mYbd3xWIg1pdXUUgEKD3NdcOampqCjdu3IBcLmc9\ncQooFipl9nBkQtY0LmFuozx48IBGNlQqFe3uPjQ0VPbzdGYHxRS5W11dxfb2NhKJBGpra6FWq9Hd\n3c1qGmShUKDiWDs7O/B6vYjFYhCLxVAqldDr9ejs7EQ4HMb6+jqWl5exs7PzipBWXV0dWlpaYDKZ\nWC0gBl6K/oVCITx//hzBYBD5fJ7VMU9CLpdDJBJBJBJBbW0tVCoVjEYjLBYL2tvbOV3whUIh7Q7R\n39+Pzc1N7O7uvvJWS/Zu+vr6YLVa0d/fz2lqdH19PVQqFdRqNXp7e6HRaDgNDZcS0RsaGsLg4CD6\n+/tpK6a1tTV63QuFQqjVaqhUKkgkEno9VgqS8aVSqYquMyJwurGxga6uLkSjUc7vFZlMhvb29tc2\nJig3hUIB6XQah4eHJQUl4/E4dnZ2TvR75RKfZF5vTCkNIqza1dXFStPvMzsopsjd4uIiXC4XIpEI\n1Go1J33GyMb63NwcLdD8feYXCQvNz89TBd9KPkWSGpXZ2VksLS1hfX29KhwUk8bGRlgsFoyOjtKF\njuunNQJ5agaAwcHBY49paWlBX18fp2Fk4GWyw9jYGAYGBmA0GitW2X8aET3S6mhsbAwqlQpzc3Os\nNdQ9CSTiMjo6WvSAuLGxQQUyPyRIwkuhUChLd/dKiU+Wi7d2UHfu3IHNZqNNM9VqdckWHeWEpCb/\n/PPP+OGHH6jQH7Ne5vddk0mtSKVgdsD2+Xw4Ojqqui7OJKZ88+ZNDA8P07qGSkAclF6vL9lTj9TL\ncG2jQqHA6Ogo/vKXv0Cn06Gurq5inedPI6InEolol/Wuri7k8/mSyrpcQBzUl19+WZRy//TpU7pY\nf0iQ/9nj8ZRlL7xS4pPl4sxngClyF4/HqSCVxWJBT08PJ/tOR0dHiEQiCIVC0Gq1MJlM6OzshFQq\nxerqKm1/s7GxgUQiQQXkcrkcFc3ikkwmg4ODA+zv76NQKMBgMKC1tRVWq5X18OJJIYJsarW64vo8\nQqGQZr1VilwuR/fAnj9/jvX1dRwcHEAul1eNuN5JRPRaWlrQ2tqKzs5O2spIqVSisbGRtaSgbDZL\n7zO3243V1dVX9KbEYjG1nRmiJSnK1dBvjm1EIhE6Ojpw/vz5ss+FwWA4814yEbEMBALY3NzEysoK\n3QYg6z0RVmVrf64sZ4OEhcbHx2G1WtHT08NpWIhkfo2Pj8NoNCIYDGJubg67u7twu920iMxqtWJ8\nfBypVApPnjwpGePlAq1Wi4mJCUxMTODcuXNvnXbMww6kiHJmZqZIibZSYc+zotPpcOnSJVy8eBEm\nkwktLS2sd3onmV82mw3z8/Nwu92v7av4oULeagUCwSuNrd+Wtra2MxeDkyiVzWbD7Ows3G43/H4/\njWyMj49jeHgYPT09rL2llcVB/T4sJBaLOQ13MIX+LBYL/vWvf+Hu3buw2WxUQM5kMtFQQiKRwMHB\nQVEtC9e0tLRgYmICf//731ltIcTzdpAiynv37uGXX36h19O7BunF96c//Ynen7+vPSo3zHM3PT39\nzp47tiH7ggaDoex70qQs4ywwC+i///57On+kY8tnn32GS5cusSqseirLSafpBw8ewOv1UpE7gUCA\nmpoaToT2joOI79lsNuzu7sJutyMQCEAoFNJsPlIXpdFoEAgEqGhcKBTC4uIi5HJ50VNxW1sbq7UG\nRAairq6Os9RZpvikx+PB/Pw89vf3IZFI0NnZCZ1OR2vIKiU1z7zGmIKTUqmU2l6qZECtVtPjy1kL\nRUQTK7l/eVJevHhBRQqfPHmCjY0N5HI5iEQi1NbWltybevHiBRwOB27fvo3e3t6yvNETwcl0Ol10\n7urr6+k8DQwM0HKUDxmxWFyV9ZfHzZ9AIDiVsOrbiMKe6oyQJ6JUKgW5XP5GkTuuKGUXU+TOarVC\np9NBJpMV7T2RegvSaJQwNjaG//u//6vEv8MaTPFJm80Gn89XVGMyOTmJ0dFR6HS6ijko5lxubm5i\ncnISUqkUSqWSCqOV6vLc29uLycnJ11bhv++QTNFffvkFLpcLm5ubJ3oqj0QiWFhYQDgcxoULFzA1\nNcWajaXuS573k7cRhT2RgyJP+0KhED6fD16vF4VCgYrcqdVqiEQiTtLLS9m1s7PzSm1AV1cXBgYG\n8Nlnn2F4ePjY3wiFQgiFQlhYWKD/j0gkQi6XK0sdF1OAjCkAWAmY4pO3bt2iometra0wm824fv06\nJiYmKmLbcdfYysoK6urqaAx9aWkJd+/exeLiIrWdec1dunQJ7e3tGB8fZ8VGgUBArw82pe5PC/NN\nZW9vD/Pz8/juu+9oFIE8nb8uDPPixQvY7XbY7XYkEomy74mScycUCqFUKo+9L38vdvo60T4e7iH3\nHLn2T5rAEgqF8OzZM/znP/9BKBQ6lZDoGx2UUCiEwWDAxx9/jMbGRng8Hng8HkSjUSpyZzabMTIy\nwmk6I9OuUllUarX6VHaxIXLHFCBjdpquNDU1NfR/7evrw/nz5yuWtVfqGiNduu/cuQOFQlHUNZnY\nzpTeNplMrLY6qq+vp6J+Q0NDr6034hKSmnz79m3k83l6jZHrmSQQcd0ehwlTFLPUelFKfJKn8pRr\nvTjtGnsiB9Xd3Y26ujr09PRgenoaiUQCyWSSitydP38eWq2WcwdF7CpVxFlXV3cqu1pbW8sucscU\nILPb7fD7/VXjoIxGI65evYrx8XG0trZW1EEdd40tLy/D5XIhGo1CIpEgEAjQHoFWqxVXr14tWnQV\nCgW0Wi2rDspiseD69euwWq1obW2tKgcViURQKBToNdbV1YWJiQlcu3YNOp2uoiKETFHMUutFKfFJ\nnspTrvXitGvsGx2UQCBAW1sbTVcknaR3d3epyN1HH310akPfFqZdp4WIxqnV6qLNPzZE7pLJJDwe\nD2ZmZuB0Omn3dCJWx3WISCqVoqmpCW1tbTCbzfjoo49w+fLlM/9ebW0t7b7c1NR0piQZ5lyaTCbE\nYjFsbm7S1lRer5ceW1dXR5t4Tk1NsS5NLhAIUFdXB4VCgY6ODgwODuLy5cswm82sjnsau+RyORKJ\nRFELmqamJvT09GBsbAyffvrpsU5bJBKhoaEBKpWqqBFquYQUmTBFMZnrxUnEJ9m+V0hLqGQyiWg0\niqOjIz60+BvlWC9IE2qNRnPqNbb60kY4QKVSYWRkhPaXIpw7d67sIndMFAoFFZAbGRmB0WjkNL2c\nvKmQpxfyJPs2aLVajI+P0yestw2Lkmax169fh16vP/aYlpYWWK1WTjbWSY3Kp59+ilAohJGRkYol\nkJSyq1QPNJPJ9NprjLwRHh0dYXR09JXvcUGlxCeZRKNRKlK4sLCA9fV1Ph0e5VsvmpubMTY2BoFA\ngPb29lPN5QfpoJRKJUZGRqDX64tCCHK5vOwid8eNe/PmTfT09ECtVnNaKU/2euRyOcbGxmjT0LeB\nvLKbTKayNKYkDkqpVJbUxyIS9FwUy5IaO4VCgVQqBbVaXRUOimlXqaQQcj2XusaIg9JqtUXq0uR7\nXFAp8UkmJIPx9u3bcLvdCIVCVdcjsxKUa70gihEkjH+auTzVjAuFQjQ1NaGjo4M+5byL0gblEo07\ny7g6nQ4jIyOsCjiWQiAQvLYdzlkg9WPl6mQsEonQ3NxcNa2fhEJhVdlDKIddtbW1Zw6Tvw2k5Vc8\nHqcio1yLTzJJJBLw+XyvJOFUQhiwmijXetHY2IjGxsYzZUWfykExwy9EDLDS/dp4eHjeLUgTZ6b4\nXbWIT2q1WgwNDcFqtcJqtVY085HnjA6qpaUF6XSaSnPz8PDwnBSmysDi4mJViU+SHplffvklNBrN\nO9dz8X3jTCE+Zu0Jz8lJJBLY3t7G4uIi4vE4jel+CB2beT5sMpkMAoEAVlZWEAwGaU3g7u4uVCoV\n2traMDw8zJn4ZDqdpmKNy8vL8Pv9yGQyqK+vp12638Xti/eNDzJJolIQPahkMonh4WGMjY1BoVDw\nDornvYfoxwH/a3VEBE6ZgoUDAwOciU+S2rHZ2VnY7faqKaDnKYZ3UBxCsoVIthLJ6qvGJpE8POWE\nOKjNzU0IhUIqcGowGKjKQH9/P2fik8zuG3a7HUdHR7yDqkL4lZFlpFIpDAYDJiYmioohiZBYNfRy\n4+FhA7FYjI6ODoyOjpa8znU6HYaGhmgqOVfU1NRAo9HAaDQWRTBIhi0f1agOeAfFMqTWRCKRIBwO\n089JZw5eB4rnfYUkVQmFwpKdNxQKBecCpwB/X74rCAolenoIBAJO232cdLxqtavUsaRDM+nSTCBd\nps96I7ytXWzyvs4lW7zPdmWz2Veu/d9/j9wHp4km8Pfl6ahWu9403mvfoKo1/FStdgHVaxtv1+mp\nVtt4u04mjaxcAAAAT0lEQVRPtdrG2/V6Sr5B8fDw8PDwVBJ+J5CHh4eHpyrhHRQPDw8PT1XCOyge\nHh4enqqEd1A8PDw8PFUJ76B4eHh4eKoS3kHx8PDw8FQl/w/28WFI82U1DQAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x40f1550>"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "show_word(p_tr, 471)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAA6CAYAAADla808AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFMVJREFUeJztnctTGtnbx78ExQZBEFTQDJCYeMGUUbxNKpWKv5rCSxYz\nVZlNFvn7ZpEss0mMzlSZpDKTREVjRoKWGhsvoHhBBFq88C4yp18MLYJ2N62cT5Wb7rbPQ5/T/fQ5\nz7efR5VKpVKgUCgUCkVhXCu0ARQKhUKhCEEdFIVCoVAUCXVQFAqFQlEk1EFRKBQKRZFQB0WhUCgU\nRVJy2g6VSiV547kKCOWwBcjNHiXZAhRfP1FbhKG2CENtEUZJtgCn23Oqg8r2T2exv7+PWCyG/f19\nfhvDMNDpdCgrKwOQ/w+XWg2fjz1KsgXIzZ5c+qQYbTkvxWLL0dER4vE4EokESktL6Xj5j/TrcnR0\nlLFfyc87KWw5PDzkr8fx8XHGfo1GA61WC51Ol5c9WR3UeVlaWsL4+Djm5ub4H9fU1ISOjg40NjZK\n0STlDJTUJ0qyhZKdeDyO8fFxTExMoK6ujo6X/0i/LtFoFMD3B61S7JObnZ0deL1eTE5OIh6Pn7gW\nAOBwOOB2u9He3p7XeSVzUENDQxgZGeG39fX1oaqqqmg6TGkoqU+UZAslO+RB/Mcff6CtrY2Ol/9I\nvy7r6+sZ+wttn9zs7Ozgw4cPePbsGba3tzP2d3d3Q6fTFc5BcRwHlmXBsizevn2LmZkZbG5uwuFw\nwOFwoL6+HkajUazmKHnCcRw2NzcV0SdGoxH19fVwu938tkKPj/Txy3HcqccxDAOHwyGjZYXl6OgI\n0WgUwWAQP/30U9ZrIxVKGi+hUAgsy8Ln82F8fBxLS0vY2trKOG5mZgZv374FgKIYL4eHh9jd3cXa\n2ho2Nzcz9t+8eROxWCzv84rmoMgbxdDQEHw+H1iWhU6nQ2dnJwYGBuByuYqio5SOEvrE6XRiYGDg\nxNuUzWYr6PhIH79Cb4AEs9mMgYEBGS2jKGm8sCyLV69e4d27d1haWkI8Hhc8jsz6WJal4+UC5OWg\nSLDy8PAQOp0OOp0O1659V6pzHAe/34/Xr19ja2sL5eXlcDgc6OrqwqNHj1BTUyPJD1ACyWQSiUTi\n1MHKMAy0Wi0YhpHZMmFbmpqa0N/fj9ra2oLYYLVaYbVaZW9XKNhOCIVCGBsbw8uXLxGJRFBeXn4i\nAM9xHOLxOCwWC27cuCGaTfkG24uRQo0XIYLBID5+/Ig///wTOp0OFRUVqKyszDiO4zhMTk4iEAiI\nOl6EEBpDarU64xktBaTdjY0N7O3tCQokgO/PyEgkgrW1Neh0Omi1Wmg0mjPPn5eDIsHKtbU1dHZ2\norOzE3q9PuM4p9PJ7+/o6IBWq82nmUtHMBjE5OQkvF4vAGQECJubm9He3o6mpqZCmUiBcLCdEI1G\nMTExgUQiwY/f9PiB3+/HxMQE9vb2RLWJBtsvJ2QloqOjAwaDIWO/VONFCKExZDAYsj6jxcLv98Pr\n9cLr9eLz58+CL3/A92fk6OgootEo2tvb4Xa7Ybfbzzx/3g5qaGgI09PTUKlUcLlcpzqogYEBDAwM\noLy8XFBaeJUIhUIYHR3Fs2fPBPf39/fDbDZTB1VghILthKOjI8RiMcTjcX78ejwefv/w8DDC4TBm\nZmZEtYkG2y8nxEE9ffpUcHVIqvEihNAYslqtWZ/RYpBKpeD3+/HixQv8888/iMVip8Yog8Eg3rx5\ng/HxcTx58gR1dXXiOygSaA8Gg9jd3cXR0dGJoOHs7Cz29/fBMAwsFgtsNls+p5eEZDKJQCCAQCDA\nv82oVCrY7XbY7XbB6Xm+7O/vY2dnBysrK7Db7XA4HCgtLeXbJQHTVCrFtyvlWw1FGKFguxA9PT1o\naWmBxWLhhRMLCwuIRCKi2SIUbC8tLYXD4YDRaATLslhaWsLm5mZBhAmU7KjVahgMBthsNsGlcovF\nIvmSfjbBRiqV4p/RUhKLxbCxsYFYLMY/27a2thAIBLC9vc1vI8/h5eVl7OzsIJlM5nT+c4skyMdV\nuQYNCwXHcZiamsLIyAhWVlb47R6PB319faI4KIJKpUJzczM8Hg8MBgOGh4exvr7Ov7kvLi7C4/Gg\noqKioA5Krq/DlYZQsF0IEoAXEv6o1WpRrp/QfdPa2orBwUHU19djaGgIGxsbF26HIh25jgOp7rez\nnr1y3ucmkwk///wzPB4PfD4fhoeHkUgk0NbWBo/Hg93dXYyMjGB5eTmv857bQZE18tOChlK8PRwf\nH/NBucPDw4z9JSUlfACOCBdWV1cxMTGB169fY35+HsD3jquurkZnZ6foNjocDjx48AA1NTUIh8Pw\n+/1YW1vD58+fsbCwgOvXr+PevXuit5srRDa8vr5edAH4fIPtq6urGcKf2tpaUV4uhO4bl8uFhw8f\noqWlBd++fcPo6OiF26FcnHQRQjweh06nQ01NDfR6PdRq9Zn/f96sEGdl8RAaQ1qtlndUUmWjIM/W\nWCyGSCSCZDIJvV6PlpYWDA4OQq/XY3p6GgsLC2hoaMAvv/yCjY0NzM7O5t2WaDLz9KBhZ2enJBJQ\nMhuanJxEOBw+EUxWqVSwWCxwu91wu91gWZYP3nm9XlGXZ3KBvFFoNBr+C2uh7wPkhswKAPD9ReMb\nZ/Oj8Ecs5LhvKBfjRxGCy+VCT0+P5AKwXLN4pI8h8kIupTgjXRTm9XoRCoVQXl4uSVuiO6inT5/C\n4XBIYjBxUM+fP+dnQ+ncvn0barUaLpeLX1YbGhrig99yQhxUa2srampqsL29rSgH5fP5EA6HaQA+\nR34U/oiF0H0j9OEnpXCkixAaGxvx+PFj9Pb2Si4AyzWLR/oYmpqaklyckS4KI8/W+vp6Sdo6l4Pi\nOA6zs7MYGRnB3NwcQqHQiaChxWIR204A35f4otEoQqHQiXgSwWAwIBqN4vj4GIlEAltbW9jc3BQM\n1LEsi/fv3yOZTOakJvmRdPHFx48fEQgETuznOA6rq6sIBAKiB9jPg9VqRXd3N58xgQbgT0cq4U96\ntopPnz7Jdt9cdkh/hEIhfpvVaoXT6ZTl+8ofs2mUl5fLIgDLNYtH+hhaXl6WXJzxoyjszp07aG1t\nxa1bt6DRaGCz2dDV1QW9Xo/m5mbo9fpzx1PP5aCIZw+Hw9jZ2RE1eCw2DMNkBOoCgQD8fj8SiQQW\nFhbQ19eX93nTxRefP3/OcFAkNxVx4izLivWTzgWZATgcDhqAPwOphD9Sii6uMqQ/xsbG+G3d3d0Y\nGBi4FAkA5OhfoTakbjddFNbV1QWn04mysjL+WdPd3c2rUs9LXg6KYRhUVlaioqICKysrmJub478c\nrqurk7wMxVkcHh7yM6zt7W1wHAeNRsMH6iKRCEKhEP79919EIhF8/PgR4XAYN2/ezLutZDKJubk5\n/PXXX1hcXIRWq0VtbS2MRiM0Gg329vYwMzODV69e8Ut7VVVVYv/knCECAbvdXvAA/HnLJ8iFVMIf\noWwrRHShVqv567K+vo5oNCq5RPiyQPrj5cuXvAjKZDJJLjYSqz/EfC6mCyfIM06oDbnEUEQUdv/+\nfX6bzWYTbYaZl4NyOBwYHBxEdXU1xsfHMT4+LsuX0rlCZi3JZJJfxkrHZrOht7cXBoPhROaHi2Kz\n2fivo91uN6xW67kSIxYLSiyfIISUAgahbCvkupDAeCKREK29q4DcghIl9ke6cGJ8fPzUlZmrIobK\ny0E5nU5UVVWhubkZqVQKPp9PkQ5qenoayWQSsVjshCTYZrPh4cOH6OzsxLNnz7C6uiqK/VarFb29\nvXjy5AkfOF1YWLjwea8qSiyfIISUwh+hbCuFFvYoHTmEWOkosT/ShRMsyyIWi8FsNp963GUXQ+W9\nxMcwDPb391FRUQG1Wg2r1QqHwwGXy4XGxkbZlmjMZjMvfiDs7e3xwgXypXK6g9JoNNBoNDCZTDCZ\nTDklK8yFsrIymEwmXL9+XZTzXXXkLP2RaxmNdIiA4eDgAGtra/jy5QsvyhGr3IaQ6CI9UwslE7kF\nJaQ/dnZ2+HHa3d1d0MS16cKJkpIStLa2nnj2XjUx1IVl5mTZ78GDB3A6nbLl3SOzofR8aSsrK3z2\nhlxTaVAKhxylP3Ito5FOMBgEy7InxEAk/kTLbRSWQghKLjpOpbJZ6Nl71cRQF3ZQNpsNPT096O/v\nF8OenDGbzWhvb8dvv/3Gb/v69SsWFxfx5s0bvsSF1WqFwWBASYkkxYMpF0CO0h9CwgShWT4JPsfj\ncZSWlqK8vByVlZXY3d3F9PQ0f5zVakVra6sktiodjuOwvb2NtbW1jH2kvIPUSC3EEhIhXHScimEz\nufbpgg2hZy/DMIoRQ4nBlX1qO51OXrTgdrtpNV+KYBkNQjQa5QPidXV1px5nMBhEzSRxmSBy78XF\nxYyHLinvcNnJVYQgN+TaT01NKUawIQdX2kEVU8kPytkIldEghEIhXviT7Ti1Wi15cF6pLC0tIRwO\nC76Vk/IOl51cRQhyQ659aWmpYgQbcnBlHZRWq4XZbJa9amwgEADLspiensb8/DySySQvJGloaOC/\nrKbIA8MwaGxshMfjQUNDA1paWgTFLCqVihf+EAHDVRa95Coe2dra4rNpGI1GOBwOQZGA2WxWRHmd\ni3KWCCEbQplCxILjOH65USmCDTm4sg6qEKRSKXz9+hUjIyMYGxsDy7LY399HS0sLPB4PHjx4cOEv\nq68aUr91kwB3VVUVTCYTTcb6H7mKRziOyygH0tXVlXEcwzBwOp1SmgxAXpFEvgIwOTKFyCEsUhLU\nQYkMy7J49+4d3r9/z28zmUxoaGiA2+2GTqfLyEhAgrLHx8dFtxQpddCbBLhPq2ZMsgWEw2EA37N9\nSFUuRkkcHBxgZWUFk5OTgpV8OY5DPB7H4eFhRjkQoaVPuZAzW02+ArBsmULEQg5hkZKgDkoGyAd/\n6+vrvGgj/TsOv98Pr9eLRCJxZrVXiriQbAGzs7NQqVT4/fff0djYeOXfTMmbOADBj9X9fj8mJiaw\ntrZGy4GcA6nKsxQb1EHJwLdv37CxsYEvX77g4OAAN2/e5B1UKpWC3+/HixcvsLOzk1MBNIp4kJeH\nyclJPH78GI8fP+YzVl9liINyuVyCOeaGh4cRDocRiURkzd5wVZCqPEuxUbQOyuFw4P79+9jb28Ot\nW7dEP79er+czXWxtbSEQCGBpaQkTExOwWCzw+Xz8sX///TdmZ2eRSqUQjUZFt4VyOiRbAPmgsaam\nRtayF2cF1kOhED59+iT4vxfJaqFWq1FRUYGKigrB/RaLBQzD0HIgOSBVeRYx7fv06ZNoWVDkpCgd\nlEqlQlNTE18aXorgLilY6PF44PP5MDw8jMnJSUxNTWF7e5tfl06lUggEAgiFQpeidIDYKEmaXAhb\nhALr6aTP8H5ErqwWxd5HZyFVeZbTyPcakDHEsuyly4JSlA4KAC/VFINkMolIJIK1tTVEIhEkk0no\n9Xq0tLRgcHAQVVVVWF5eRiAQQCwWOzF7IpSVlaG6urroJOiFKNGSa8kCOTgtsJ4OmWH9iFxZLQpd\nRicdJdlCkKo8y2nkcg3UajX0ej1qamqwvr7OK4rlzIJyfHzM32ebm5vnctxF66DEJBgMYnR0FLu7\nu/B6vQiFQifWncl69I0bN7Kep6qqCnfv3pXYWopSswVky3QhRDFntVAicpcDycUW4P9FMHKPF1LU\ndXJyEhMTE/D7/XmfgzooEQgGg3jz5g3Gxsb4r7zr6+v5/Tdu3EB1dTUePnyY9TwlJSU0oCoDSs0W\nkC2DhRDFnNVCichdDiQXW9JFMHKPF+Kgnj9/Dp/Pd64aeZfKQWk0Gty+fRv/+9//YDQaT5TaAL4L\nE1wuF/r7+9HS0iLbV9bJZBLJZBLb29uw2+24c+cOWltbcevWLWg0Gr5MCUUZpGcLINWOOY7D7Ows\nRkZGUFlZmfE/VqsVTqdTtDghCVynZ2oohgwWVwUhcYuUgpJ8yxqdJYKRg+PjY77C+dHRERoaGmC3\n23H37l3Be0yIS+WgGIZBW1sbKisrUVZWluGgiDChtrYWZrNZ9im2SqVCc3MzPB4Purq64HQ6FVHC\nXMkoJegtVFojne7ubgwMDIjmoEjg2mg0yhJYvwhK6SNAObbIkTUinUKVNRILUh6pr68Pdrs958nD\npXJQZAZ1+/Ztwf1EmNDS0iK5LdeuXYNer0d1dTV2d3cBfL95Wltb8csvvwimg6FkUuigd1lZGV+C\n48fSGumYTCbcu3fvQm0JBa4JlZWVis1gUeg+IpCZ7/r6OhiGgU6nK9gLYLasEWJ9y5g+XpSQxSNX\nOI5DIpFAKBRCNBrF4eEhqqur0d7ejl9//TWvc10qB6UkyGzu+PiYXyYCALfbfeUTOF4lchUmiJFd\n4qzsDcWQweIikFkLAF6MoIQy5j9mjdBqtaKcN3281NXVXZqxsbS0BK/Xy/9FIpFzp2WiDuqcMAyD\n9vZ2NDQ04PDwkN9OS3tcLnIVJjAMc+EA81nZG8Ro4ypDHJTP50M4HEZVVZViHJQUpX3SxwsponkZ\nIMvXQ0NDFy4NQh3UOSFLfMX23dJVINcSHGKjhMD1Zaa0tBQ2mw0OhwP19fWSVwXIlsVDSNwidtaI\nyzBeWJbF+/fvsbW1xW/78OEDfD4fNjc3+Ww6PT09GZqBXKAOilJQChH0piU48kMpwgS5S01ky+KR\nXoakWCF5RBOJBN6+fctvDwaDYFmWD4N4PB7cvXtXPgeVHrxTamCXIowS+o5hGFRWVqKmpkbUoHI+\n7WcrwUEpfB8J2eJwONDV1YVHjx5JlhZMSMiS7UNuJYtbpEKj0cBoNKK2thYHBweYn5/H/Pz8iWNU\nKhXsdjs6OjrQ399/7nyn53JQlzV4R1FG3xHJbFtbm6hBZYp4KKmP5LTlLCGLEMUmbrHZbOjt7T1z\n6VGn08Htdl9oKVaVOkVDqlKpTpWXHh0d8cEvErzL9w0i2/kvcux5ybUNJdlyHnvy7TspbOE4DrFY\nDAcHB3xQOZc3dCWNmatui5L6SE5b0u8PISGLEETcIvd9dF4uaksymcxJ/HDt2jW+v0pKTp8LZbMn\n6wxKKWvPALUlG0qyh9oiDLVFGGqLMNSW/9o+bQZFoVAoFEohuVZoAygUCoVCEYI6KAqFQqEoEuqg\nKBQKhaJIqIOiUCgUiiKhDopCoVAoioQ6KAqFQqEokv8Dz6rGZs39bFYAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x597ad90>"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "show_word(p_tr, 57)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAA6CAYAAADla808AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFitJREFUeJztndlTGtnbx7+AKKCiIgqa2O4LZoi4ZeaXMnFqyiVTNUnV\n3CQX8/fNReZulqoZRmamTHKRGDUkptyj0owRFHFBEXF7L/KenlYaaKGbbuP5XLKdh3MeeLrP8z3P\nozk7OzsDhUKhUCgqQ6u0ARQKhUKhCEEDFIVCoVBUCQ1QFAqFQlElNEBRKBQKRZXQAEWhUCgUVZKX\n7AmNRiP74GIFhLmwBRBnj5psAa7fOlFbhKG2CENtEUZNtgDJ7UkaoFK9SQou+8XlVsNfxh412QJc\nn3WitghDbRGG2iKMmmwBUttDt/goFAqFokpogKJQKBSKKkm5xZcpwWAQLMsiGAxyj9lsNtTW1qKy\nslKOISkUSpbEYjGwLAuWZRGLxUS9h/6uKZfF7/eDZVkcHR2hpqYm5WtlCVAsy+KPP/7A+Pg491hv\nby+Gh4epI1MoKiUajWJiYgJutxtbW1ui3kN/15TLcHZ2htnZWXg8HkQiEQwODqZ8fdYB6vDwEPv7\n+zg8POQeW1pawvv37/Hy5UtEo1FEo1EcHBygtrYWtbW1MJlM2Q5LkQChtSPodDpJ1+nk5ITzg5OT\nk6SvI+OaTCZotfLsQKezxWAwwGQyoaCgQJbx1UosFsPc3Bz+/PNPhMNhFBYWCs4Bmb9oNIqzszO4\nXC4FrKWIhfzOj4+PVfHfy7IsXrx4gXA4jPr6+pSvzTpA+Xw+TExMYGFh4Zzaw+Fw4ObNm5iYmMDE\nxAR3V7WxsYHu7u5sh6VIQLK1A4Di4mJJ14lcnU9OTiISiQD4pN5JNm53dzeKiookG/8ytrS2tqKr\nqwstLS2yjH8VqK2tRXd3t+AcRCIR7ndNUT/kd762tnbl/nslCVButxsej4d7rK+vD99//z2am5tx\ndnaGmZkZ+Hw+hEIhzM7O5kxbT0mN0NoRbDabpOtEgsKPP/6I9fX1pK8j4zocDtkDVDJbBgcHYbVa\nr32AGh4exsDAQMJzwWCQ+11T1A/5nU9NTV25/96MAhQ/mfr8+XNMT09jc3MTDMOAYRjcunULDQ0N\nqKurQ09PDxeYfD4fAoEAdnd3pf4eAIC9vT34/X74/X7E4/Gkr9NoNKipqUFNTQ3KyspksSUd4XAY\nfr8f//77b05sERKujI2NJazdzs6OLOt0cnKCSCSCQCCAtbW1pK87ODjA+Pg4rFYrNx8GgwEMw2Rt\nA5mDmZkZTExMwOfzQa/Xg2EYlJSUgGVZ+Hw+TE9P4/nz5wAAhmFQW1ur2Haf0LoR5JiX+fl5HB4e\nwmAwoLy8HDdu3Eh4vUajgdlshk6ny3psIDNxRiqknpednR1F/IA/LyUlJRmLUWKxGDY3N2X975WL\njAIUP5k6MzMDlmVhMpnQ3d2N4eFhOBwOMAzDPWa1WvH8+XO43W6srq7KFsW3t7fx6tUrLgGXioGB\nAQwODioWoAKBAJ49ewaPx5MTW4SEK4FAIGHtFhcXZV+nVBDfCoVCMBgMAACLxYLh4eGsP5vMwYsX\nL+Dz+RCNRuF0OvHgwQM0NDTA7XZjY2ODu+JkWRbDw8OoqKhQLEAJrRtBznnJJZmIM1Ih9bwsLS0p\n4gf8eWlqapJEjPJZ30GRZBvLshgfH8fvv/+OnZ0dFBYWgmEY9PT04Ntvvz03ia2trWhtbYVGo4HP\n58PR0ZEkWzfxeBwHBwfnfkx+vx+zs7OYmJhAMBjEwcEB4vE4TCYTjEYjTk9PcXBwgIODAxQWFqKu\nrg7l5eWKJA7D4TC8Xi9+/fVXSWxJl/ifmZnBs2fP8M8//3DzodPpoNVqz63d27dvJV0n4jPr6+uI\nRCI4OTlBQUFB0gQ8AOzu7mJqagqxWAzRaBTl5eWoq6vLaHz+vFycA7PZDIfDgfv378PpdCIUCnEX\nXF6vF36/H3V1dejv789iBoRJJVDhk2zdgE/boU6nM2tbAoEAxsbG8Ndff3HzUlZWxl0gpCIWi2Fr\nawvr6+sZ+65YcYYQfMHG6ekpAKC6ujpjf+FD5mVyclI2P0gFf14+fvx4TmSWTkTE9/utrS3uzjTT\nqhCnp6fc5x0fH3OPE5/Mz8/P6HPTcakARZJtJMFMlHkkqd3V1QWj0Sj4XoZh8ODBA3R0dKCrqytr\nwwOBALxeL968eQPgvyS3wWDAkydPMDs7y/3JdHR0wOVyIRqN4s2bN/B6vZibm8PPP/8MlmXR2dmZ\ntT2ZcnZ2Jokt6RL/8/Pz5+6Wurq6UFxcDOCTMIGsndTrlMpn0uV45ubmMDk5ib29vYzH588LEevw\n56C7u/vc3T4A7vXZjJuOVAKVy66bVAjNSzqkFj+J9Q0CX7Ah53opzcV5TiciEvL7bIjFYnj79i28\nXi9CoRB3J+ZyudDZ2Zn2PFOmXDpAud1uuN1u7O/vIxqNcsnU4eFhFBYWJr2Kqq2thdVqxdHREQoL\nC7M2PBgMYnR0FE+fPj03xqNHj/Dw4UO8evUKW1tb2NjYQEdHBx4/foxwOIxoNAqv14vZ2Vn4/X4s\nLi5KtpeeKVLYki7xH4vFsL+/j5KSEnR3d+OHH37g7nR1Oh23dlKvUyqfEUrA8xkZGUEoFML09HTG\n4/PnhWXZhDlgGAaFhYXQ6/Xo7u6Gw+GA1WrNetx0pBKo8BGzblJBAhR/XtIhtfhJrG8Q+IKNzzlA\nXZzndCIiIb+3WCwZj08C1E8//YQPHz4A+HQh9eTJE1RXV6sjQJFk2/b2NpdUv3fvHtrb22G321O+\n12AwiNoyEMvh4SG2t7exurqKmpoaMAwDp9OJ5uZm1NTUIBKJoK+vD1arFb29vWhsbITVasX//ve/\nc2KKYDCYNl+VDiJ48Pv93GNFRUWc+IHc/sbjce51Y2Nj3OvJFkU2tvBFCKenp2AYBjabLeF1FosF\n3d3dqK2tFXRYnU4n6Tql8hmhBDyf8vLyrG3hz0teXh6cTiccDofgHJjNZpjNZknGFSKduIgIVPgX\nGGQbL5nNUqLT6VBcXAy73Y7y8vKkrzMYDGhpacHAwIAk4if+55nNZuzv73M7I8B/1SrMZnOCmCIc\nDiMQCOD4+Bg2mw0Mw8DhcHwWCsxU85zqLCGQ3O8znZfT01NEIhEEg0Gsrq4C+BSgtre3UwrSsiUj\nkYSQIEIpNBoN2traMDAwgJ6eHk5pQ67Eent7OZWW0WhEf38/6uvr4fF4MDIyIokNfMED4caNGxgc\nHERlZSUXoMhViMfjwbt3784FNCkh23Q9PT0JzxkMBkUOS2frM1JcnZN56evrEz0Hckjtk4mLiECF\nH6AysTkbxHxfqcVP/M+bn5/H9PQ03G439zypVtHY2JggpojFYgmCFzJXUqKEuECqeb7oQ1eJjAKU\nwWBAa2srhoaGUFVVJbVNl4ZhGPT19eHu3bvcY3a7XfCuzmKxoKOjAxsbG5iYmEA4HM56fCJ4+OWX\nX7jH2traUF9fj/v373OPxeNxLCws4O+//8by8jKMRiOqqqq45KNU2O123LlzB999951kn5kt2fqM\nFCX/ybwMDQ0lPCd1UlkIITGAkECF/OFGo9GUNmdDNt+XrGVraysODw/h9Xqxurqa8VzxP0+j0WBs\nbEywCo1er08QZxExRVlZGSd4Ebs9eBnkbjkhhNA8syyLSCSC9fV10RVP5PIh4NNF1+bmJjY2NjjB\nhJQVYGSpxUdJj91uh8vlgsvlOif2oCiD1EnldAiJi8iVbkVFheyVGnL9fcUiNAdEIPD27duUQpuW\nlhZFd3NyAVk3AJzvKLWdKSTw6uzsTCqUywQaoBTCZrOhv78fjx8/xtOnT/Hx48fPOsmrdqROKqdD\nSFxEBCptbW2yV2rI9fcVi9AcEIGAXq9PKbQxGAySikbUCFm3mZkZhEIhxSueXBR4ORyO3AYofmKX\nZVkwDAOLxYKWlpacH15MJjK4ihQUFKC0tBQ3btxAaWmppOcIgsEgXr9+zZ2oV/IE/OvXrwWrIKiF\nZNUl+EllqeaOn/Rubm5OEBcRgcrh4aGklRqE4CfRNzc3ZRvnshAxFX8OYrFYQoWJnZ0dLC0tCYop\n5BC4KI3NZkNvby/320pW8WR7ezuhKojUWCwW1NTU4ObNmwliM3IW7SJ8Idm7d+9EH8hOG6D4id3i\n4mK0t7ejublZkUR7rkQGVx2lKyEIiQF0Ol1WiWa5ktSpqktILUzgJ71LS0tVuR2lxjUSgvi41+vl\nHpOz9YfSFRjIHSPDMCkrnuSiKojdbudyfWLFZkRINjIywgU0Mesk6g6KJHZ7enowPDwsS7JNDMlE\nBiUlJbKdZE4G/2T15uZmgiMcHx9zskwiwwyHw4hEIudOYstBMBhEMBjEysoKrFYrHA6H6BpnUrSa\nEBIDVFVVoaioKO1dgZxiBVL1gF8LMFV1CamT7fyktxohd1WpivleRC5BCZ9klUdYlsXy8nJOWvoo\nIZLgY7PZYLPZUFdXl1DxhP87F/JnsVVBxGKxWOByufDw4UNObEYqxQSDQWi1WhiNxnNj8ivnEDFF\nRUVF2mo1VzYHRUQGJDEndOZHTvgnqycnJzE3N3fueVIXMB6Pw2g0QqPRcOc7dnZ2cqJ+5CdUiSMI\ntbiQs9WE2EojF22WI3lPri6Xl5dTVmoQW0XhcyMTfyHzJydiW39ch5Y+QhVP1tbWuHX7+PGjIv7M\n/7/r7OyEy+VKeiHW2trK/W/fvn075ede2QBFRAZPnjxJWcFCLvgnq2dmZrC/v3/uebJgU1NTnOzy\n9PSUS/LmAn5CVWw+Q+pWE2IrjRDkTN6TZPvo6Cj3mFClBrFVFD43MvEXMn+5EJSka/1xHVr6kMDD\nr3jy4cMHbt2Ojo4U8Wf+/936+josFotggNJoNGhtbcWjR4/w1VdfpbXrygYovshACfgnq09OTrgK\nFgSh1h/5+flcdYk7d+7IVh6EoNfrYbfbwTCM6Fv8hoYGlJSUSGYDaduQrtKIGLFCtggl23NZqUGt\nkAoMmexCBINB+Hw+Gaz6j1StP4xGo6QtfdQu8NHpdFzFk/b2dty7dw8AOOEEEUVYLBZRVUGyhWEY\n3L17F/n5+dz/XTgcTnmus7CwEBUVFaL87coGKDXBTxoSVldXMTIygvX1dS5AGQwGdHR0YGBgALdv\n35Y9QPGrFIht5UECWq4RI1aQg1xXalAjqSqPpOP169eKtmeRuqqFHAIfuRASTvBVe3LbTO6GjEYj\nGhoauP87KblWAYq06Njf38fOzo5kNaRI0vDRo0fcY4uLiwiFQpibm+MklRaLBV1dXRgaGkJjYyMA\neZOvaqv4kQqhlg9SixVIsj0vL49LrJeWlqK5uRkulwsmkynnYhs1kE3lEY1Gk3UliWzQ6/Worq5G\nWVkZAoEAXr58mZUtmQh8UrVO0el0sl30EOFETU0NVlZWMDo6yvl4ZWWlKFFStpA6klarFcvLy3j2\n7BknRgoGg5wgIlOuVYDit+h48+YNgsGgbHuzpaWl+PLLL5Gfn8/d7ppMJnR2dkq6hfa5IWdylyTb\nq6qqkibW07UxoKgLOUU1YgU+qVqnFBcX51SwcVlRkhwQ+fv6+jonhsiUaxWg+C06iFihoaFBlrFI\ngHI6ndzhNa1Wq4ig4yqRScsHsZAtEafTmTSxnq6NAUVdyCmqESvwSdU6xWaz5XR78LKiJDlYWVnB\nxsYG3r9/j6OjI9TX12f8WZ9tgCJJd36Sc3p6GlNTU1yLjlu3bsHpdHLbbVKSl5eHkpISUXdLJNG4\nt7cniy1qRSghLbblQyaQZHtdXV1CYn1lZQXj4+OwWq0p83WkOgcl9/D9hd9uQy5RDfGX0tLShHH5\njI2NJbROIaIki8WSViCUKXxhEakaIVaUJDVFRUVwOBwYGhrCwsIC/H4/1tbWsLu7m9W5z882QK2s\nrMDj8WBsbIx7bHNzE36/X7BFh1LwE43xePyzaBMgFqUS0kKJ9ampKUxMTCAUCqVUPFosFgwPD8tq\n31VF7nXj+0su220IjcsnEAgktE4hFzmkvY0c5KJqhFjIjlFVVRVevHiBkZERLC0tZf25OQtQJJF4\nfHwsyW1nPB7Hzs7OuaoAfMiJao/HwyXq8vI+fd2qqio4nU588803GSmXpIZcdWWDTqdDUVERKisr\nEYvFOGdV+gS8EMQXWJZNaJ8gtuJENlxs7+Dz+fDvv/9id3cXU1NTKd9LZOmURDL1NX71ENKIj1Qm\nODk54ZLuQv4iZ7sNMeMStFrtudYpcpRbuoiQsEjqqhFiKSoqQnt7O9rb23F4eIipqSksLi5ib28P\nGxsbgtV2xJCzAEUSiWtra5IkDQOBAEZHR7nzDhdPvM/NzcHn83HSbpfLBavVyj2vRPUJORE6Ya7W\n6ujEF4id/PYJuU7uEom12FxkcXExurq6ZLbqesEXOpCO0pFIhPMNpdptiB2XQHwj18IEtVZBIcUM\ntFot9vf3E6rtiCGnAYpspUixFUCKD46Pjws+T064m0wmdHR04PHjx+fyO5+bWEHohPn09LTSZglC\nfMHtdie0T8h1cpe0d+jv7xf1ep1Ody2rTMgJX+hAztGcnJxwvqFUuw2x4xKIbyjVrVptVVBisRi8\nXi8WFha4KjqXJW2A4rcIMJvNYFkWv/32G/c8KXHPv6UVEiiQRGI2p7zz8/PR1NSEr7/+Gs3NzaLe\nYzab0dvbi8bGRkmrTvBtKSkpkf3QbTr4J8zLy8tV03KAtP7gQ3xhe3ub2968d+9eQvuJXEDaO6iJ\ni205Pqc7fSH0ej2qqqrwxRdfiG7DcOfOHbS3t0teSYY/90K2yDVuNsgpLMoEu92Onp4exGIxrroE\n2c257M1J2gDFTyjPz89jenoabrebe16oxD25Nebf3ZBEYjZJcLJdV1ZWJnr7Kj8/HwzDSH72iG9L\nQUGB4gEqGUqLJITaIggllR0Oh2q2JpTmKrTlECJTX+N/X7FV9+WqeJLOFqUqraRD6d85H3KXWV9f\nD4/Hg93d3YzTDaLuoPgJ5bGxMbx8+TKhxP3Nmze591ws+W40GqHT6aDVarkTzplA7lqampoyer+U\nqMmWixgMBpSVleXsNPlF+IKN9fV1TqLLJ1dJZb4tSiWQL4va23LwuehrmX6GWr6vmmxJh9K/82TY\n7XbY7XY0NTVhZ2cHHz584M6CajSaS7VHulQOiiSUKyoqEk7ip2thUFxcDIAmmXMBWaeOjg5Fk7YA\nUl455SKpzLelurpalVe/V5mLvkbJHUr/ztNBdplOT0/PdW2+jEDtUgGKJJTb2toSTuKna2FArpBp\nkll+yDodHR0pmrR1OBycbFiIXCSV+bbo9XrqexJz0dcouUPp33k6DAYDXC4Xmpubzx3WvYytlwpQ\nJKGs0Wi4k/ipkpoWi+XatjBQEqUT/3zBhtKoyRY1IZUQQ2lfu87kcu4zEYVptVoUFRVlVTosI5m5\n2KQmOUWttshOoVx3rqoQg6IMSonCNGdJjn8LtXqWdOBLfL7ctlxmDDXZkgt7qC3UFmoLtUVOUo2h\nlXVkCoVCoVAyhAYoCoVCoaiSlDkoNR3+orYkR032UFuEobYIQ20Rhtry/2Mny0FRKBQKhaIkdIuP\nQqFQKKqEBigKhUKhqBIaoCgUCoWiSmiAolAoFIoqoQGKQqFQKKqEBigKhUKhqJL/A33afPb0//M0\nAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x4c81fd0>"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Define Factor Types and Build Factor Graphs"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's define 4 factor types, such that a word will be able to be modeled as a chain graph.\n",
      "\n",
      "- The unary factor type will be used to define unary potentials that capture the appearance likelihoods of each letter. In our case, each letter has $16 \\times 8$ pixels, thus there are $(16 \\times 8 + 1) \\times 26$ parameters. Here the additional bits in the parameter vector are bias terms. One for each state. \n",
      "- The pairwise factor type will be used to define pairwise potentials between each pair of letters. This type in fact gives the Potts potentials. There are $26 \\times 26$ parameters. \n",
      "- The bias factor type for the first letter is a compensation factor type, since the interaction is one-sided. So there are $26$ parameters to be learned.\n",
      "- The bias factor type for the last letter, which has the same intuition as the last item. There are also $26$ parameters.\n",
      "\n",
      "Putting all parameters together, the global parameter vector $\\mathbf{w}$ has length $4082$."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from modshogun import TableFactorType\n",
      "\n",
      "# unary, type_id = 0\n",
      "cards_u = np.array([n_stats], np.int32)\n",
      "w_gt_u = np.zeros(n_stats*n_dims)\n",
      "fac_type_u = TableFactorType(0, cards_u, w_gt_u)\n",
      "\n",
      "# pairwise, type_id = 1\n",
      "cards = np.array([n_stats,n_stats], np.int32)\n",
      "w_gt = np.zeros(n_stats*n_stats)\n",
      "fac_type = TableFactorType(1, cards, w_gt)\n",
      "\n",
      "# first bias, type_id = 2\n",
      "cards_s = np.array([n_stats], np.int32)\n",
      "w_gt_s = np.zeros(n_stats)\n",
      "fac_type_s = TableFactorType(2, cards_s, w_gt_s)\n",
      "\n",
      "# last bias, type_id = 3\n",
      "cards_t = np.array([n_stats], np.int32)\n",
      "w_gt_t = np.zeros(n_stats)\n",
      "fac_type_t = TableFactorType(3, cards_t, w_gt_t)\n",
      "\n",
      "# all initial parameters\n",
      "w_all = [w_gt_u,w_gt,w_gt_s,w_gt_t]\n",
      "\n",
      "# all factor types\n",
      "ftype_all = [fac_type_u,fac_type,fac_type_s,fac_type_t]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Next, we write a function to construct the factor graphs and prepare labels for training. For each factor graph instance, the structure is a chain but the number of nodes and edges depend on the number of letters, where unary factors will be added for each letter, pairwise factors will be added for each pair of neighboring letters. Besides, the first and last letter will get an additional bias factor respectively.   "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def prepare_data(x, y, ftype, num_samples):\n",
      "    \"\"\"prepare FactorGraphFeatures and FactorGraphLabels \"\"\"\n",
      "    from modshogun import Factor, TableFactorType, FactorGraph\n",
      "    from modshogun import FactorGraphObservation, FactorGraphLabels, FactorGraphFeatures\n",
      "\n",
      "    samples = FactorGraphFeatures(num_samples)\n",
      "    labels = FactorGraphLabels(num_samples)\n",
      "\n",
      "    for i in xrange(num_samples):\n",
      "        n_vars = x[0,i].shape[1]\n",
      "        data = x[0,i].astype(np.float64)\n",
      "\n",
      "        vc = np.array([n_stats]*n_vars, np.int32)\n",
      "        fg = FactorGraph(vc)\n",
      "\n",
      "        # add unary factors\n",
      "        for v in xrange(n_vars):\n",
      "            datau = data[:,v]\n",
      "            vindu = np.array([v], np.int32)\n",
      "            facu = Factor(ftype[0], vindu, datau)\n",
      "            fg.add_factor(facu)\n",
      "\n",
      "        # add pairwise factors\n",
      "        for e in xrange(n_vars-1):\n",
      "            datap = np.array([1.0])\n",
      "            vindp = np.array([e,e+1], np.int32)\n",
      "            facp = Factor(ftype[1], vindp, datap)\n",
      "            fg.add_factor(facp)\n",
      "\n",
      "        # add bias factor to first letter\n",
      "        datas = np.array([1.0])\n",
      "        vinds = np.array([0], np.int32)\n",
      "        facs = Factor(ftype[2], vinds, datas)\n",
      "        fg.add_factor(facs)\n",
      "\n",
      "        # add bias factor to last letter\n",
      "        datat = np.array([1.0])\n",
      "        vindt = np.array([n_vars-1], np.int32)\n",
      "        fact = Factor(ftype[3], vindt, datat)\n",
      "        fg.add_factor(fact)\n",
      "\n",
      "        # add factor graph\n",
      "        samples.add_sample(fg)\n",
      "\n",
      "        # add corresponding label\n",
      "        states_gt = y[0,i].astype(np.int32)\n",
      "        states_gt = states_gt[0,:]; # mat to vector\n",
      "        loss_weights = np.array([1.0/n_vars]*n_vars)\n",
      "        fg_obs = FactorGraphObservation(states_gt, loss_weights)\n",
      "        labels.add_label(fg_obs)\n",
      "\n",
      "    return samples, labels"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# prepare training pairs (factor graph, node states)\n",
      "samples, labels = prepare_data(p_tr, l_tr, ftype_all, n_tr_samples)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "An example of graph structure is visualized as below, from which you may have a better sense how a factor graph being built. Note that different colors are used to represent different factor types."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "try:\n",
      "    import networkx as nx # pip install networkx\n",
      "except ImportError:\n",
      "    import pip\n",
      "    pip.main(['install', '--user', 'networkx'])\n",
      "    import networkx as nx\n",
      "\n",
      "import matplotlib.pyplot as plt\n",
      "\n",
      "# create a graph\n",
      "G = nx.Graph()\n",
      "node_pos = {}\n",
      "\n",
      "# add variable nodes, assuming there are 3 letters\n",
      "G.add_nodes_from(['v0','v1','v2'])\n",
      "for i in xrange(3):\n",
      "    node_pos['v%d' % i] = (2*i,1)\n",
      "\n",
      "# add factor nodes\n",
      "G.add_nodes_from(['F0','F1','F2','F01','F12','Fs','Ft'])\n",
      "for i in xrange(3):\n",
      "    node_pos['F%d' % i] = (2*i,1.006)\n",
      "    \n",
      "for i in xrange(2):\n",
      "    node_pos['F%d%d' % (i,i+1)] = (2*i+1,1)\n",
      "    \n",
      "node_pos['Fs'] = (-1,1)\n",
      "node_pos['Ft'] = (5,1)\n",
      "\n",
      "# add edges to connect variable nodes and factor nodes\n",
      "G.add_edges_from([('v%d' % i,'F%d' % i) for i in xrange(3)])\n",
      "G.add_edges_from([('v%d' % i,'F%d%d' % (i,i+1)) for i in xrange(2)])\n",
      "G.add_edges_from([('v%d' % (i+1),'F%d%d' % (i,i+1)) for i in xrange(2)])\n",
      "G.add_edges_from([('v0','Fs'),('v2','Ft')])\n",
      "\n",
      "# draw graph\n",
      "fig, ax = plt.subplots(figsize=(6,2))\n",
      "nx.draw_networkx_nodes(G,node_pos,nodelist=['v0','v1','v2'],node_color='white',node_size=700,ax=ax)\n",
      "nx.draw_networkx_nodes(G,node_pos,nodelist=['F0','F1','F2'],node_color='yellow',node_shape='s',node_size=300,ax=ax)\n",
      "nx.draw_networkx_nodes(G,node_pos,nodelist=['F01','F12'],node_color='blue',node_shape='s',node_size=300,ax=ax)\n",
      "nx.draw_networkx_nodes(G,node_pos,nodelist=['Fs'],node_color='green',node_shape='s',node_size=300,ax=ax)\n",
      "nx.draw_networkx_nodes(G,node_pos,nodelist=['Ft'],node_color='purple',node_shape='s',node_size=300,ax=ax)\n",
      "nx.draw_networkx_edges(G,node_pos,alpha=0.7)\n",
      "plt.axis('off')\n",
      "plt.tight_layout()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAagAAACKCAYAAAD/od5JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEOJJREFUeJzt3XtQleXaBvCLxVFFRTm1lUMoHvIAwhIr9YtRM3XIz8iy\ncoY+s8EEKnEKdtpJ7EsLHXQQAY9ZNDWjE+ogKqY7BRuNw8J041kJR2RHyCFJWMdn/9G0dzVK4vu+\ni4fF9fvX9d7PzS2La6336CSEECAiIpKMrqsbICIiuhMGFBERSYkBRUREUmJAERGRlBhQREQkJQYU\nERFJiQFFRERSYkAREZGUGFBERCQlBhQREUmJAUVERFJiQBERkZQYUEREJCUGFBERSYkBRUREUmJA\nERGRlFy6ugGyn6tXr2Lt2v+HEDZFdXx9/4aVK1er1JXja2xsxPvvvwWLxaSojodHH3z0UQbc3d1V\n6syxmc1m/P3vS9HW1qqojrOzK1asWA0fHx+VOqN75cQn6vYcO3fuxEcf/R/i49vvu4bVCiQn62Cx\nWFXszLGVlpbi2Wej8dZb9z93AFi2zA3//OcVBAQEqNSZY/vxxx8RGhqI9HSzojpr1vRCXt7XmDRp\nkkqd0b3iN6geJjTUFQkJ9/+H0mIBkpNVbKiH8Pd3VzR3APjwQ75dO6t3bxckJCgLqM8/d1OpG+os\nHoMiIiIpMaCIiEhKDCgiIpISA4qIiKTEgCIiIikxoIiISEoMKCIikhIDioiIpMSAIiIiKTGgiIhI\nSgwoIiKSEgOKiIikxIAiIiIpMaCIiEhKvH9/D/Ovf1nw9df3v72Vj4G6Ly0tyuYOAO3tHH5nGY1W\nxXNvbrao0wx1GgOqBxk5ciQ8PMKRnn73N1xdXR369u0LT0/Pu75m3rzBWrTnsAICAjBkSCTS09vu\n+pqffmqAi4szBgwYcNfXPPpovw7/nf6of//+eOyx/0F6estdX9Pc3Ayz2QxfX9+7viYw0B1BQUFa\ntEh/gU/UpT9YvXo1oqOjMXHixK5upUfZtm0bBg4ciNjY2K5upUcpKChAXV0dFi1a1NWt0B3wGBQR\nEUmJAUVERFJiQBERkZQYUEREJCUGFBERSYkBRUREUmJAERGRlBhQREQkJQYUERFJiQFFRERSYkAR\nEZGUGFBERCQlBhQREUmJAUVERFJiQBERkZQYUEREJCUGFBERSYkBRUREUmJAERGRlBhQREQkJQYU\nERFJiQFF/2Gz2WA2m2E2myGE6Op2egwhBOfeBTh3+TkJ/s/0WCaTCXv27MGhQ4dQUVGBqqoqODk5\nwWazwdPTE5GRkRg/fjyee+45REZGdnW7DsNqtaKoqAj79u1DRUUFTp8+DavVCgBwc3NDREQE9Ho9\nYmNj8dhjj8HJyamLO3YMQggcP34c+fn5qKioQGVlJYxGIwBAp9MhLCwMer0eMTExmDVrFpydnbu4\nY+I3qB6ora0NaWlpCA4ORm5uLiIjI5Gbm4uWlhYYjUaYzWZcuHABKSkpcHd3R2xsLB599FEUFBR0\ndevdmsViQWZmJoYNG4b33nsPoaGhWLt2Lerr62EymWAymXD9+nWkpaXB398fCQkJGDNmDHbs2MFP\n+AoIIZCXl4ewsDDEx8fD19cX77//Pq5du/afuTc0NCAjIwMjRozABx98gKFDh2LdunWwWCxd3X7P\nJqhH+fbbb8Xw4cPFvHnzRFVV1T1tY7FYxO7du0VoaKiYP3++aGho0LhLx1NVVSWioqLEtGnTxHff\nfXdP29hsNnHkyBGh1+vF9OnTRU1NjcZdOp5r166JmTNnioiICHH48GFhs9nuabuysjIxffp0odfr\nxZkzZzTuku6GAdWDbN26Vfj7+4uvvvrqvrb/5ZdfRHJysggODhYXLlxQuTvHVVhYKHx8fERubu49\n/4H8PbPZLFatWiX8/PzEiRMnNOjQMZWWlgp/f3+xcuVKYTKZOr29zWYTW7ZsET4+PmLv3r0adEh/\nhcegOqmmpgZffPGF4jqDBw/Giy++qEJH92b79u1IS0vD4cOHMWzYMEW1tm3bhhUrVqC4uBghISEq\nddixxsZGbNmyBTabTVGdvn37IikpyW7HdYqKihAXF4eCggI8/PDDimrt378fCxYswP79+zF+/HiV\nOuyY0WhEVlYWTCaTojqurq549dVX4eHhoVJnHausrMSMGTOwbds2zJ49W1Gt8vJyxMTEYPv27YiJ\niVGpw44JIZCTk4OWlhZFdXQ6HV5++WX4+Pio1Jl9MaA6afPmzXht5WuwDFGwb9oM9DrXC63Nreo1\n1oHf3qzHjx/H8OHDVam5fv165OXl4eTJk3B1dVWlZkcOHTqEOXNegcn0vMJKa9HS0gRPT09V+urI\n9evXERERgT179mDSpEmq1Ny9ezeWLFmCM2fOoH///qrU7MjFixcxduzDsFgWK6rj4rIFFRVHMWbM\nGJU6u7tbt24hLCwM6enpePbZZ1WpefLkScyePRvl5eUIDg5WpWZHjEYjevfqjUlOyn5vzruexydf\nfWK3YFWbS1c30B05BzjDNE3BJ8rbAM6p1k6HTCYTFixYgLVr16oWTgCwZMkSHDx4EB9//DHeeecd\n1ep2xN19GNrbVyuq4eq6QaVuOiaEQHx8PF577TXVwgkAYmNjcfDgQbzxxhvYunWranU74u7uC5NJ\n2dw9POx3gk1qaiqmTJmiWjgBwCOPPIKlS5ciPj4eRUVFdvkG7qxzxjTrNEU1mj2aVeqma/AsPge3\nadMmDBo0CHFxcarWdXJywpYtW5CRkYG6ujpVazuCffv2oba2FsuWLVO99po1a1BUVISKigrVa3d3\n33//Pfbu3YuMjAzVa6empqKhoQG7d+9WvTbdGQPKgQkhkJWVheXLl2vyiS8wMBDz5s2z2yf57iQr\nKwspKSma7P7s168fkpKSkJ2drXrt7i47OxsJCQnw8vJSvbaLiwtSUlKwceNG1WvTnTGgHNixY8fg\n5uaGyZMna7ZGYmIiNm/erPjkBUdy9epVVFZWqrqL6c8WLlyI/Px8xQfRHUlrayt27tyJ+Ph4zdaY\nO3cuqqqqcPHiRc3WoP9iQDmwo0ePYvbs2ZruLw8LC4NOp8OVK1c0W6O7KS4uxvTp0zU9Y83Pzw9h\nYWEoLS3VbI3upry8HKNGjcIDDzyg2Rpubm6YMWMGiouLNVuD/osB5cAqKiqg1+s1X0ev1/N4yO9w\n7l2Dc3c8DCgHdu7cObuc1jt27FicO2en0xK7gbNnz2L06NGar8O5/5G95j5mzBicPXtW83WIp5l3\nGaPJiNdff13TNW7evGmX63369OmDXbt2oampSbM1rl27hra2NsV1rFYr3nzzTbi5uanQ1Z1VV1fb\nbe4VFRWa/h41NTWhvd2ouE57eztWr14Nb29vFbq6s7KyMkydOlWz+r/x9PRETU2NpnO3Wq08rgsG\nVJdxdXVFcnKypmvk5+fb5WaXFosF48eP1/TnKSkpwT/+kQeFNzSATueMpKQk9OnTR53G7qC0tNRu\ncw8NDdV07tXV1di9+yTMZmV13NzcEBcXp+q1eH9WW1trl7mbzWb4+vpqOnej0YjNuZuBHn4bBQZU\nF9E56TBkyBBN1wgKCkJ1dbXmV75XV1dj3Lhxmv48ly9fhk6nfI+0kxMQEhKi6TecIUOGoLq6GtHR\n0ZqtAfw692HDhmk6d4vFAp1O+Uk2Op0OAQEBmvY6YsQIVFdXa1b/N9XV1QgJCdH0ZzEajXzMCngM\nyqHZ62CuvQ5Odxd6vR4Gg0HzdTj3P7LX3A0GA+duJwwoBzZhwgTNT4dtamrCpUuXEB4eruk63Yk9\n5m6xWHDixAlMmDBB03W6k6ioKJw8eRJmpfsj/0JJSQmioqI0XYN+xYByYHPmzEFxcTFqa2s1W2PH\njh2YM2cOevXqpdka3c3EiRPx888/o6ysTLM1CgoKMHToUM13E3cnQUFBGDVqFPbs2aPZGgaDAfX1\n9ZrvvqVfMaAcWL9+/fDCCy8gJydHk/oWiwU5OTlISEjQpH535ezsjMWLFyMzM1OT+kIIbNiwAYmJ\niZrU784SExORmZmp2ROIN2zYgFdeeYWPg7cTBpSDS0lJQW5uLs6fP6967fT0dAQHB2PixImq1+7u\nFi1ahG+++QZHjx5VvfaXX36J+vp6PPPMM6rX7u5iY2Nx69Yt5OXlqV67pKQERUVFWLxY2aNH6N7x\nLL77INoF0KCgQLtqrfylkJAQpKWl4aWXXkJxcbFqNy89ffo01q1bh/LycrudbWSz/QLggqIaQtjn\n2hIvLy9s2rQJCxcuxKlTp9CvXz9V6t64cQNLly5FYWGhptdy/Z7NZoLSuVutyq+luheurq7YsWMH\nnnjiCUydOhUBAQGq1G1tbcXChQuRnZ2NgQMHqlLzr9iEDQ2K/tAA7VY7/rHRAAOqk0JCQjCwZSBE\ngbJdCMOjtLse5M8SEhJQVFSEBQsW4LPPPlO8e+KHH37Ak08+ifXr19vl4W0AMGjQIHh5NcLT838V\n1RkwYBTc3d1V6qpjMTExOHDgAObMmYPCwkL07t1bUb3GxkbMmjULycnJdnui7oABA+Dv3xdtbcrm\n7uHhrulFur83btw4pKamYubMmTh69Kjip8m2tbXh6aefxtSpU/HUU0+p1GXHXFxcMHbUWBTcVPYc\nLRcXFwwePFilrrpAlz1snuzq9u3b4vHHHxexsbGiqanpvuuUlZWJwMBAsXHjRhW7c1wWi0XExcWJ\nyZMni9ra2vuuc/HiRTF69GiRmpoqbDabih06JpvNJt5++23x0EMPifPnz993nRs3bojo6Ggxf/58\nYbFYVOyQ7gUDqgdpb28XiYmJIiAgQBQWFnZq27a2NrF8+XLh5+cndu3apVGHjslqtYoVK1YIX19f\n8emnn3YqYCwWi8jIyBDe3t4iKyuL4dRJubm5wtvbW6xZs0aYzeZ73s5ms4nPP/9c+Pn5iXfffVdY\nrVYNu6S7YUD1QEeOHBGhoaEiKipKfPLJJ6KlpeWOr7PZbOLy5cti2bJlws/PT8ydO1fU1dXZuVvH\nYTAYRGRkpBg5cqTIzMwUP/30011fW1tbK1atWiWCgoLElClTxKVLl+zYqWO5cuWKmDZtmggMDBQf\nfvihuH79+l1f29DQILKyssSoUaNEeHi4KCsrs2On9GdOQmh0PiZJzWq14uDBg8jOzsaxY8cQEBCA\n8PBw9O/fH1arFbW1tTAYDNDpdHj++eeRkJCAESNGdHXb3Z4QAiUlJdi4cSMOHDgAb29vREREwNvb\nG0II1NfXw2Aw4Pbt24iNjUViYiLvWqASg8GAnJwc5Ofnw8PDA5GRkfD394eTkxMaGxtRWVmJhoYG\nzJgxA0lJSYiOjubthroYA4pgsVhw7tw5nD59Gq2trXB2doafnx/0ej0GDRrEN6lGbDYbLl26hFOn\nTqG5uRk6nQ4DBw5EZGQkHnzwQc5dI0II1NTUwGAw4ObNm7DZbPDy8kJ4eDiGDx+uyj0fSR0MKCIi\nkhI/KhARkZQYUEREJCUGFBERSYkBRUREUmJAERGRlBhQREQkJQYUERFJiQFFRERSYkAREZGUGFBE\nRCQlBhQREUmJAUVERFJiQBERkZQYUEREJCUGFBERSYkBRUREUmJAERGRlBhQREQkJQYUERFJiQFF\nRERSYkAREZGUGFBERCQlBhQREUmJAUVERFJiQBERkZQYUEREJCUGFBERSYkBRUREUmJAERGRlBhQ\nREQkJQYUERFJiQFFRERSYkAREZGUGFBERCQlBhQREUmJAUVERFJiQBERkZQYUEREJCUGFBERSYkB\nRUREUmJAERGRlBhQREQkJQYUERFJiQFFRERSYkAREZGUGFBERCQlBhQREUmJAUVERFJiQBERkZQY\nUEREJKV/A96/zvPmbfZNAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x517d310>"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Training"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we can create the factor graph model and start training. We will use the tree max-product belief propagation to do MAP inference."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from modshogun import FactorGraphModel, TREE_MAX_PROD\n",
      "\n",
      "# create model and register factor types\n",
      "model = FactorGraphModel(samples, labels, TREE_MAX_PROD)\n",
      "model.add_factor_type(ftype_all[0])\n",
      "model.add_factor_type(ftype_all[1])\n",
      "model.add_factor_type(ftype_all[2])\n",
      "model.add_factor_type(ftype_all[3])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In Shogun, we implemented several batch solvers and online solvers. Let's first try to train the model using a batch solver. We choose the dual bundle method solver (<a href=\"http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CDualLibQPBMSOSVM.html\">DualLibQPBMSOSVM</a>) [2], since in practice it is slightly faster than the primal n-slack cutting plane solver (<a a href=\"http://www.shogun-toolbox.org/doc/en/latest/PrimalMosekSOSVM_8h.html\">PrimalMosekSOSVM</a>) [3]. However, it still will take a while until convergence. Briefly, in each iteration, a gradually tighter piece-wise linear lower bound of the objective function will be constructed by adding more cutting planes (most violated constraints), then the approximate QP will be solved. Finding a cutting plane involves calling the max oracle $H_i(\\mathbf{w})$ and in average $N$ calls are required in an iteration. This is basically why the training is time consuming. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from modshogun import DualLibQPBMSOSVM\n",
      "from modshogun import BmrmStatistics\n",
      "import pickle\n",
      "import time\n",
      "\n",
      "# create bundle method SOSVM, there are few variants can be chosen\n",
      "# BMRM, Proximal Point BMRM, Proximal Point P-BMRM, NCBM\n",
      "# usually the default one i.e. BMRM is good enough\n",
      "# lambda is set to 1e-2\n",
      "bmrm = DualLibQPBMSOSVM(model, labels, 0.01)\n",
      "\n",
      "bmrm.set_TolAbs(20.0)\n",
      "bmrm.set_verbose(True)\n",
      "    \n",
      "# train\n",
      "t0 = time.time()\n",
      "bmrm.train()\n",
      "t1 = time.time()\n",
      "\n",
      "w_bmrm = bmrm.get_w()\n",
      "\n",
      "print \"BMRM took\", t1 - t0, \"seconds.\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "BMRM took 3402.20772505 seconds.\n"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's check the duality gap to see if the training has converged. We aim at minimizing the primal problem while maximizing the dual problem. By the weak duality theorem, the optimal value of the primal problem is always greater than or equal to dual problem. Thus, we could expect the duality gap will decrease during the time. A relative small and stable duality gap may indicate the convergence. In fact, the gap doesn't have to become zero, since we know it is not far away from the local minima."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import matplotlib.pyplot as plt\n",
      "fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(12,4))\n",
      "\n",
      "primal_bmrm = bmrm.get_helper().get_primal_values()\n",
      "dual_bmrm = bmrm.get_result().get_hist_Fd_vector()\n",
      "\n",
      "# plot duality gaps\n",
      "xs = range(dual_bmrm.size)\n",
      "axes[0].plot(xs, (primal_bmrm-dual_bmrm), label='duality gap')\n",
      "axes[0].set_xlabel('iteration')\n",
      "axes[0].set_ylabel('duality gap')\n",
      "axes[0].legend(loc=1)\n",
      "axes[0].set_title('duality gaps');\n",
      "axes[0].grid(True)\n",
      "\n",
      "# plot primal and dual values\n",
      "xs = range(dual_bmrm.size-1)\n",
      "axes[1].plot(xs, primal_bmrm[1:], label='primal')\n",
      "axes[1].plot(xs, dual_bmrm[1:], label='dual')\n",
      "axes[1].set_xlabel('iteration')\n",
      "axes[1].set_ylabel('objective')\n",
      "axes[1].legend(loc=1)\n",
      "axes[1].set_title('primal vs dual');\n",
      "axes[1].grid(True)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAtYAAAEVCAYAAADEhKJBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4TGf7B/DvZBOyR4hIRJTYiX1phahStPalthJ0fSle\nuv1arWh10aLqbXm1VWotJS1qqb7aKK29tkaFICK2IEFC9szvj6dnlmQmMyeZ5Uzm+7muXDPnzJlz\n7plMnrnznPs8j0qtVqtBREREREQV4mLvAIiIiIiIKgMm1kREREREFsDEmoiIiIjIAphYExERERFZ\nABNrIiIiIiILYGJNRERERGQBTKzJ7mJjY/HWW29VeD8pKSlwcXFBcXExAKBv375YtWpVhfdLRESl\n7d27F40bN7bKvi31vSBHTEwMli1bVuH9JCQkoE6dOhaIiBwRE2uyO5VKBZVKZfH9bt++HU8//TQA\nYMWKFYiOjrb4MYiInFV0dDTOnDljlX1b63tBacekyoeJNSkC5ykiInIchYWFVj8GvxfIETGxJps7\nduwY2rRpA19fX4wYMQK5ubmaxwz1LLu4uODChQsAgG3btqF169bw8/NDeHg4Zs+ebfQ40mm9M2fO\n4IUXXsD+/fvh4+ODwMBAHDlyBMHBwXoNd3x8PFq1amVwX7dv30a/fv3g5+eHDh06YObMmXpxTp06\nFeHh4fDz80O7du2wb98+zWNxcXEYOnQoRowYAV9fX7Rt2xYnT57UPD537lyEhYXB19cXjRs3xi+/\n/GLmO0lEZDkRERH48MMP0axZMwQGBmLChAnIy8sDIMobwsLC8NFHHyEkJAQTJ04sVfIQERGBefPm\noWXLlvDx8cHEiRNx48YN9OnTB35+fujZsyfu3Lmj2X7YsGEICQmBv78/unXrhtOnT5uMMS8vD/7+\n/khMTNSsu3nzJqpVq4Zbt27h1q1bePLJJxEQEIDq1auja9euRhP0n3/+GY0bN4a/vz9eeuklve3i\n4uI0ZzyB0qWGy5cvR9OmTeHr64v69evjiy++MPNdpsqOiTXZVH5+PgYOHIhx48YhMzMTw4YNw6ZN\nm8w+/ebt7Y3Vq1fj7t272LZtG5YsWYLNmzcb3FY6rde4cWMsXboUnTt3RlZWFjIyMtCuXTsEBQXh\np59+0my/atUqjBs3zuC+Jk2aBB8fH9y4cQPffPMNVq5cqRdzhw4dcOLECWRmZmLUqFEYNmwY8vPz\nNY9v2bIFw4cP1zw+cOBAFBUVISkpCZ9//jmOHDmCe/fuYdeuXYiIiDDrvSAisrS1a9di165dOH/+\nPM6ePYs5c+ZoHrtx4wYyMzORmpqKpUuXlnquSqVCfHw8du/ejaSkJPz444/o06cPPvzwQ6Snp6O4\nuBiLFi3SbP/EE08gOTkZN2/eRJs2bTB69GiT8VWpUgVDhgzBunXrNOs2bNiAmJgYBAUFYf78+ahT\npw5u3bqF9PR0fPDBBwa/X27duoUhQ4bg/fffx+3bt1G/fn38/vvveq+lLMHBwdi2bRvu3buH5cuX\n49///jeOHTtmMn6q/JhYk00dOHAAhYWFmDp1KlxdXTFkyBC0b9/e7Od369YNzZo1AwC0aNECI0aM\nwJ49e0w+z1CPxdixY7F69WoAQEZGBnbt2oVRo0aV2q6oqAjx8fGYPXs2PD090aRJE4wbN05vn6NH\nj0ZAQABcXFwwffp05OXlISkpSfN4u3btMHjwYLi6umL69OnIzc3FgQMH4Orqiry8PCQmJqKgoADh\n4eF46KGHzH4/iIgsRaVSYfLkyQgNDUVAQADefPNNvQTWxcUFs2fPhru7Ozw9PQ3u46WXXkKNGjVQ\nu3ZtREdHo3PnzoiKikKVKlUwaNAgveQzNjYWXl5ecHd3x6xZs3DixAlkZWWZjHPUqFH49ttvNctr\n167VtN0eHh64du0aUlJS4OrqikceecTgPrZv347mzZtr2uVp06ahVq1amsdNlaH07dsX9erVAwB0\n7doVvXr1wt69e03GTpUfE2uyqatXryI0NFRvXd26dc1+/sGDB9G9e3fUrFkT/v7+WLp0KW7fvl2u\nWEaPHo2tW7fiwYMH2LBhA7p27Yrg4OBS2928eROFhYV6pzzDwsL0tpk3bx6aNm0Kf39/BAQE4O7d\nu7h165bB7VUqFcLCwnD16lU0aNAACxcuRFxcHIKDgzFy5Ehcu3atXK+HiKiidNu58PBwXL16VbNc\no0YNeHh4lPl83Ta0atWqesuenp7Izs4GIDosXn/9dTRo0AB+fn6aJFW33TQmJiYGDx48wKFDh5CS\nkoITJ05g0KBBAIBXXnkFDRo0QK9evVC/fn3MnTvX4D6uXr1aqh2XM5LHjh070KlTJ1SvXh0BAQHY\nvn17ub+LqHJhYk02FRISgitXruitu3Tpkua+l5cXHjx4oFm+fv263rZSGUVaWhru3LmDF154QVPz\nVhZDp/XCwsLQqVMnxMfHY/Xq1Xr1dLpq1KgBNzc3XL58WbNO9/7evXvx8ccf47vvvsOdO3eQmZkJ\nPz8/vR4P3e2Li4uRlpaG2rVrAwBGjhyJvXv34tKlS1CpVHjttddMvh4iImtITU3Vuy+1U4Dp8ghD\njPX8rl27Flu2bMHu3btx9+5dXLx4scztdbm6umL48OFYt24d1q1bh379+sHLywuAKBecN28ezp8/\njy1btmDBggUGr1upXbu2XrusVqv1lr29vY1+F+Xl5WHIkCF49dVXkZ6ejszMTPTt25cXWxIAJtZk\nYw8//DDc3NywaNEiFBQUID4+HocPH9Y8HhUVhcTERJw4cQK5ubmIi4vTe352djYCAgLg4eGBQ4cO\nYe3atWY19sHBwUhLS0NBQYHe+rFjx2Lu3Ln466+/MHjwYIPPdXV1xeDBgxEXF4ecnBycOXMGq1at\n0hw3KysLbm5uCAoKQn5+Pt555x3cu3dPbx9Hjx7F999/j8LCQixcuBCenp7o1KkTzp49i19++QV5\neXmoUqUKPD094erqas5bSURkUWq1GosXL8aVK1eQkZGB9957DyNGjLDKsbKzs1GlShUEBgbi/v37\neOONN0rFUhapHES3DAQQF7gnJydDrVbD19cXrq6uBtvUJ554AomJiZp2edGiRXrJc6tWrfDbb7/h\n8uXLuHv3Lj744APNY/n5+cjPz0dQUBBcXFywY8cO7Nq1q7xvBVUyTKzJptzd3REfH48VK1agevXq\n2LBhA4YMGaJ5vGHDhnj77bfx2GOPoVGjRoiOjtZLnBcvXoy3334bvr6+ePfdd/HUU0/p7d9Ykt2j\nRw80a9YMtWrVQs2aNTXrBw8ejNTUVAwaNMhozSAAfPbZZ7h79y5q1aqFcePGYeTIkZpTor1790bv\n3r3RsGFDREREoGrVqggPD9eLacCAAVi/fj0CAwOxZs0axMfHa+qr/+///g81atRASEgIbt26pdeA\nExHZikqlwqhRozRlFJGRkZg5c6be44aeY2qfuvel5bFjx6Ju3boIDQ1F8+bN0blzZ6PbGtKhQwd4\ne3vj2rVr6NOnj2b9uXPn0LNnT/j4+ODhhx/GpEmT0K1bt1LPr169Or777ju8/vrrCAoKQnJyMrp0\n6aJ5/LHHHsNTTz2Fli1bon379ujXr58mHh8fHyxatAjDhw9HYGAg1q1bhwEDBsh6X6jyUql57oKc\nXGRkJJYuXYpHH33U7Oe89tprSE9Px/Lly01uO3v2bCQnJ3MWSCJStHr16mHZsmWy2kIi0me1HuvL\nly+je/fuaNasGZo3b643xI6uKVOmIDIyElFRURyqhmwuPj4eKpXK5BdJUlISTp48CbVajUOHDuHr\nr7/WXCxjCv93JUfANpuIqOLcrLVjd3d3fPLJJ2jVqhWys7PRtm1b9OzZE02aNNFss337diQnJ+Pc\nuXM4ePAgXnzxRRw4cMBaIRHpiYmJ0dRLm5KVlYWRI0fi6tWrCA4Oxssvv4z+/fubdRxOk0uOgG02\nEVHFWS2xrlWrlmZMSG9vbzRp0gRXr17Va6S3bNmimZCjY8eOuHPnDm7cuGFwyDMiS0tISDB723bt\n2uHcuXPlOs6sWbPK9TwiW2KbTdLIHERUflZLrHWlpKTg2LFj6Nixo976K1eulBobOC0tTa+RZk8f\nETkyRywFYptNRM6qom221UcFyc7OxtChQ/Hpp5/C29u71OMlX4ChRlmtVivuZ9asWXaPgXExLsal\n7LgcEdtsxsW4GJezxmUJVk2sCwoKMGTIEIwZMwYDBw4s9XhoaKjegOxpaWmlZuVTqpSUFHuHYBDj\nkodxycO4Kje22bbHuORhXPIwLtuzWmKtVqsxceJENG3aFNOmTTO4Tf/+/bFy5UoAwIEDB+Dv789a\nPSIiO2CbTURUcVarsf7999+xevVqtGzZEq1btwYAvP/++5rpUp9//nn07dsX27dvR4MGDeDl5WXW\nmMBKERsba+8QDGJc8jAueRhX5cU22z4YlzyMSx7GZXuKnyBGpVJZrO6FiMiWnLH9csbXTESVgyXa\nL05pXk5yhmqzJcYlj9LiCgwM1Ix7zR/H+QkMDLT3R4dMUNrfuoRxycO4tPh9ocw22ybD7RGReTIz\nM9nb54BUKg4xR0S2xe+L8rNmm81SECIF4efdMRn7vTnj79MZXzORPfBvrfys2WazFISIiIiIyAKY\nWJcT67zkYVxEZE9K/VtnXPIwLlI6JtZERERERBbAGmsiBXG0z3tsbCzq1KmDd999t0L7SUlJwUMP\nPYTCwkK4uLigb9++GDlyJJ5++mkLRWpdrLHWcsbXTGQPleVvbe/evXj22Wdx5swZi+/b2HcUa6yJ\nSJGkoYssbfv27ZqkesWKFYiOjrb4MYiIyP6io6OtklQD1vuOKgsT63JSaj0V45JHqXE5ksrQY0KV\nn1L/1hmXPIyrciksLLT6MWz9HcXEmojMduzYMbRp0wa+vr4YMWIEcnNzNY8Z6ll2cXHBhQsXAADb\ntm1D69at4efnh/DwcMyePdvocWJiYrBs2TKcOXMGL7zwAvbv3w8fHx8EBgbiyJEjCA4O1mss4+Pj\n0apVK4P7un37Nvr16wc/Pz906NABM2fO1Itz6tSpCA8Ph5+fH9q1a4d9+/ZpHouLi8PQoUMxYsQI\n+Pr6om3btjh58qS8N42IyMlERETgww8/RLNmzRAYGIgJEyYgLy8PCQkJCAsLw0cffYSQkBBMnDgR\nCQkJqFOnjt5z582bh5YtW8LHxwcTJ07EjRs30KdPH/j5+aFnz564c+eOZvthw4YhJCQE/v7+6Nat\nG06fPm2Pl6zBxLqcYmJi7B2CQYxLHqXGZYxKZZmf8sjPz8fAgQMxbtw4ZGZmYtiwYdi0aZPZp9m8\nvb2xevVq3L17F9u2bcOSJUuwefNmI69TnL5r3Lgxli5dis6dOyMrKwsZGRlo164dgoKC8NNPP2m2\nX7VqFcaNG2dwX5MmTYKPjw9u3LiBb775BitXrtSLuUOHDjhx4gQyMzMxatQoDBs2DPn5+ZrHt2zZ\nguHDh2seHzhwoE16WciylPq3zrjkYVzms+f3BQCsXbsWu3btwvnz53H27FnMmTMHKpUKN27cQGZm\nJlJTU7F06VIDcasQHx+P3bt3IykpCT/++CP69OmDDz/8EOnp6SguLsaiRYs02z/xxBNITk7GzZs3\n0aZNG4wePbr8QVsAE2siB6JWW+anPA4cOIDCwkJMnToVrq6uGDJkCNq3b2/287t164ZmzZoBAFq0\naIERI0Zgz549Zrzm0gGPHTsWq1evBgBkZGRg165dGDVqVKntioqKEB8fj9mzZ8PT0xNNmjTBuHHj\n9PY5evRoBAQEwMXFBdOnT0deXh6SkpI0j7dr1w6DBw+Gq6srpk+fjtzcXBw4cMDs101EZA/2/L5Q\nqVSYPHkyQkNDERAQgDfffBPr1q0DIM5kzp49G+7u7vD09DT4/Jdeegk1atRA7dq1ER0djc6dOyMq\nKgpVqlTBoEGDcOzYMc22sbGx8PLygru7O2bNmoUTJ04gKyurfIFbABPrclJqPRXjkkepcSnR1atX\nERoaqreubt26Zj//4MGD6N69O2rWrAl/f38sXboUt2/fLlcso0ePxtatW/HgwQNs2LABXbt2RXBw\ncKntbt68icLCQr3TjGFhYXrbzJs3D02bNoW/vz8CAgJw9+5d3Lp1y+D2KpUKYWFhuHbtWrniJvtR\n6t8645KHcTkO3XY3PDwcV69eBQDUqFEDHh4eZT5Xtz2vWrWq3rKnpyeys7MBiM6T119/HQ0aNICf\nnx/q1asHAHptuK0xsSYis4SEhODKlSt66y5duqS57+XlhQcPHmiWr1+/rretVEaRlpaGO3fu4IUX\nXkBxcbHJ4xoqNQkLC0OnTp0QHx+P1atXGx2Wr0aNGnBzc8Ply5c163Tv7927Fx9//DG+++473Llz\nB5mZmfDz89Pr0dbdvri4GGlpaahdu7bJuImInFlqaqrefandLM8oHcYuQFy7di22bNmC3bt34+7d\nu7h48WKZ29sCE+tyUmI9FcC45FJqXEr08MMPw83NDYsWLUJBQQHi4+Nx+PBhzeNRUVFITEzEiRMn\nkJubi7i4OL3nZ2dnIyAgAB4eHjh06BDWrl1rVgMbHByMtLQ0FBQU6K0fO3Ys5s6di7/++guDBw82\n+FxXV1cMHjwYcXFxyMnJwZkzZ7Bq1SrNcbOysuDm5oagoCDk5+fjnXfewb179/T2cfToUXz//fco\nLCzEwoUL4enpiU6dOpnzlpGCKPVvnXHJw7gcg1qtxuLFi3HlyhVkZGTgvffew4gRIyx+nOzsbFSp\nUgWBgYG4f/8+3njjjVJx2JpDJNbx8UBRkb2jIHJu7u7uiI+Px4oVK1C9enVs2LABQ4YM0TzesGFD\nvP3223jsscfQqFEjREdH6yXOixcvxttvvw1fX1+8++67eOqpp/T2byzJ7tGjB5o1a4ZatWqhZs2a\nmvWDBw9GamoqBg0aZLRODwA+++wz3L17F7Vq1cK4ceMwcuRIzWnI3r17o3fv3mjYsCEiIiJQtWpV\nhIeH68U0YMAArF+/HoGBgVizZg3i4+Ph6uoq781zMt99Z+8IiMieVCoVRo0ahV69eqF+/fqIjIzE\nzJkzoVarDbb1pjpZdB/XHZt67NixqFu3LkJDQ9G8eXN07tzZ6La24hAzLwJq/PQT0KuXvaPRSkhI\nUOR/qIxLHqXFVVlm0rKVyMhILF26FI8++qjZz3nttdeQnp6O5cuXm9x29uzZSE5OxqpVq8rcjjMv\nakltdmYm4O9v72i0lPa3LmFc8jAuLSW3L/Xq1cOyZctktc22xJkXAfz5p70jICIliY+Ph0qlMtlw\nJyUl4eTJk1Cr1Th06BC+/vprDBo0yKxjKPVLyxGwzSYiZ+Rm7wDMpbSL8JX4HzPAuORSalxUtpiY\nGE29tClZWVkYOXIkrl69iuDgYLz88svo37+/Wcexx2nEyiItzd4R6FPq3zrjkodxkdI5TCnIuHHA\nihX2jobIupR8ao+MYymIltRmf/IJMG2avaMhqrycsX2xFJaCALh7194R6FPqmJWMSx6lxkXk6DIy\n7B2BPqX+rTMueRgXKR0TayIisjilJdZERLbgMKUgbdoAR4/aOxoi6woMDERmZqa9wyCZAgICkGEg\nk3TGU7VSmz1yJLB2rb2jIaq8nLF9sRRrloI4zMWL/8xeSVSpGUrOiBwRzzISkTNymFKQEpOu2Z1S\n66kYlzyMSx7GReYqLLR3BPqU+hlhXPIwLlI6h0msldZIExGRcWyziUhXbGws3nrrrQrtIyUlBS4u\nLiguLrZQVJbnMDXWISHA1av2joaIyHzOWAMptdnR0cBvv9k7GqLKy9Hal/Hjx6NOnTp45513yr2P\nlJQUPPTQQygsLISLS/n7hjncHpRXCkJERMaxx5qISnKkfwTKy2ESa6U10kqtp2Jc8jAueRgXmYtt\ntnkYlzyMy3EcO3YMbdq0ga+vL0aMGIHc3FwAwIoVKxAdHa23rYuLCy5cuAAA2LZtG1q3bg0/Pz+E\nh4dj9uzZNo+9IhxiVBBXV+U10kREZJirK88yEtmbarbKIvtRz5Lfy5yfn4+BAwdi+vTpmDx5Mn74\n4QeMHDkSr7/++j/lYsZ5e3tj9erVaNasGU6dOoWePXuiVatWGDBgQHlfgk05RI21l5cahYXAP//s\nEBE5BEergbQEqc2uVw84dcre0RBVXkpuX3777TeMHDkSV65c0ax75JFH8Oijj6JBgwb46quvsHfv\nXs1jLi4uSE5OxkMPPVRqX9OmTYOLiwsWLFjAGmtLqVJF9FhnZADff2/vaIiIqCxSm713L/Dll/aO\nhohs7erVqwgNDdVbV7duXbOee/DgQXTv3h01a9aEv78/li5ditu3b1sjTKtwmMS6qAhYuBAYPNje\n0QhKradiXPIwLnkYF5mjShVRCjJ9OvDcc/aORlDqZ4RxycO4HENISIhebzUAXLp0CQDg5eWFBw8e\naNZfv35db7tRo0Zh4MCBSEtLw507d/DCCy8oeni9khwisXZ1ZZ01EZGjkHqsicg5Pfzww3Bzc8Oi\nRYtQUFCA+Ph4HD58GCqVClFRUUhMTMSJEyeQm5uLuLg4vedmZ2cjICAAHh4eOHToENauXWuyLltJ\nHKLGOjxcjfR0YPJkYN48QNkRExEJSq6BtBaVSoVGjdTIzgZq1QKOHmWbTWQNSm9fjh49imeffRbJ\nycno27cvVCoVIiMj8c477+D999/HJ598gmrVquH999/H2LFjce7cOTz00EPYtGkTZsyYgYyMDHTr\n1g316tXDnTt3sHLlSqSkpKB+/fooKChQbI21QyTW9eqpcfMm8MwzohxE2RETEQlK/+KzBpVKhZYt\n1bh+HahTh4k1kbU4Y/tiKU5/8aKrK+Dmpqzhm5RaT8W45GFc8jAuMoenp/JKQZT6GWFc8jAuUjqH\nSazd3ZXXUBMRUWmssSYiZ+UQpSBNm6qRmQn07QssW8bTikTkGJzxVK1KpcJjj6nx++9A06YsBSGy\nlsDAQGRmZto7DIcUEBCAjIyMUust0WY7zMyLSisFISIiw6RSECbURNZjKDEk+3OoUhAlJdZKradi\nXPIwLnkYF5lDiaUgSv2MMC55GJc8jMv2HCaxdnNTXkNNRESlubuLWwea04GIyCIcosa6QwcxJmqj\nRmJKc2VHTEQkOGuN9ejRamzYADRsCCQmss0mIsfgNMPtubgorxSEiIgMk9psnmUkImfjEIm1EktB\nlFofxLjkYVzyMC4yhxIvOFfqZ4RxycO45GFctudQiXV+vr0jISIiU5R4wTkRkS04RI119+5qqFRA\nTg6wfz/r9YjIMThrjfVzz6mxYwfw4AFw+zbbbCJyDIqusZ4wYQKCg4PRokULg48nJCTAz88PrVu3\nRuvWrTFnzhyj+3J1Bby8gOxsa0VLRESWarddXQFvb7bZROR8rJZYjx8/Hjt37ixzm27duuHYsWM4\nduwYZs6caXQ7qZHOyrJ0lOWn1PogxiUP45KHcVVulmq3pTY7L88aUZaPUj8jjEsexiUP47I9q828\nGB0djZSUlDK3Mbe7/a+/YhEcHIH0dADwR0JCK8TExADQ/nJsvSyx1/GNLR8/flxR8fD9Kt+yRCnx\n8P0yb3nhwoU4fvw4IiIi4Igs1W7v3h2LnJyIf5bYZpe1zL8pect8v+Qt8/0qe9kabbZVa6xTUlLQ\nr18/nDp1qtRje/bsweDBgxEWFobQ0FDMmzcPTZs2LR2gSoWDB9X49ltgyRIgN5f1ekTkGByxxrqi\n7bZKpcJff6nxf/8HbN0q1jnYW0BETsoSbbbVeqxNadOmDS5fvoxq1aphx44dGDhwIM6ePWtw2w4d\ngB9/FEk1ERHZh7ntdrNmohSEiMjZuNjrwD4+PqhWrRoAoE+fPigoKEBGRobR7ZXWSJc8naEUjEse\nxiUP43JuctptttnmYVzyMC55GJft2S2xvnHjhqa7/dChQ1Cr1QgMDDS6vW4jXVzM8VGJiGxNTrvt\n5aW9f+kSYKJ0m4ioUrBajfXIkSOxZ88e3Lp1C8HBwZg9ezYK/smGn3/+eXz++edYsmQJ3NzcUK1a\nNSxYsACdOnUqHeA/9S7LlwMTJoh1b7wBvP8+6/aISNkcrcbaEu229JrfeAP44AOxzs9PjBCSk2Pr\nV0REZD5LtNkOMUGMWq3GmjXAmDFi3ZNPipprZUdORM7O0RJrS5Be8+zZQFycWOfiIs40OtlbQUQO\nRtETxFhalSr2jkCfUuuDGJc8jEsexkXm8vDQ3lep7BeHRKmfEcYlD+OSh3HZnsMk1kprpImIyDil\ndYYQEdmCw5SC/PQT0Lu3WNevnxgfVdmRE5Gzc+ZSkM8+A156SaxzcwMKC9lmE5GyOVUpiG6PdXGx\n/eIgIiLT2GNNRM7IYRJr3UZaCYm1UuuDGJc8jEsexkXm0u0MUQKlfkYYlzyMSx7GZXsOk1izx5qI\nyHHodobwuhgichYOU2N96hTQsqVY17Mn8PPPrNcjImVz5hrr+HhgyBCxzt1dTOrlZG8FETkY1lgT\nEZEisceaiJyRwyTWrLE2D+OSh3HJw7jIXKyxNg/jkodxycO4bM9hEmv2WBMROQ72WBORM3KYGuvb\nt4GgILEuOhrYu5f1ekSkbM5cY33gANC5s1hXpQqQl8c2m4iUjTXWRESkSOyxJiJn5DCJNWuszcO4\n5GFc8jAuMpfSJohR6meEccnDuORhXLbnMIk1e6yJiBxHw4ba++yxJiJn4TA11gBw+bIYyzoyEjh8\nmPV6RKRszlxjDQBpaUC9emIc65wcttlEpGxOVWMNAHXqAH5+7LEmInIEtWqxvSYi5+JQiTUAVK0K\n5ObaOwrl1gcxLnkYlzyMi+RwcwNcXMSsi/am1M8I45KHccnDuGzP4RJrLy8gMdHeURARkTkKC8UP\nEZEzcKgaa0Ak1g8eiPvKjpyInJ2z11iLZe1jTvZWEJGDcboaa0BMMkBEREREpDQOl1grZdgmpdYH\nMS55GJc8jIsclVI/I4xLHsYlD+OyPYdLrImIiIiIlMjhaqzd3bUXwig7ciJydqyxZo01ETkOS7TZ\nbqY2yMn8FQt8AAAgAElEQVTJweLFi7Fv3z6oVCpER0fjxRdfhKenZ4UOTERERERUmZgsBRk7dixO\nnz6NKVOmYPLkyUhMTMTTTz9ti9gMWrvWbofWo9T6IMYlD+OSh3GRXGyzy8a45GFc8jAu2zPZY52Y\nmIjTp09rlh999FE0bdrUqkGVZdgwux2aiIhk6trV3hEQEdmOyRrrMWPGYNKkSejcuTMA4MCBA/j8\n88+xatUq2wRooN5FqtljvR4RKRlrrIH0dCA4WNx3sreCiByMJdpsk4l148aNcfbsWdSpUwcqlQqp\nqalo1KgR3NzcoFKpcPLkyQoFYDJAJtZE5KCYWAN37gABAeK+k70VRORgbDJBzM6dO3HhwgXs2bMH\nCQkJuHDhAnbs2IGtW7diy5YtFTp4RR07Zr9jK7U+iHHJw7jkYVwkl7u79v6OHfaLQ6mfEcYlD+OS\nh3HZnska64iICABAeno6cnNzNevDw8OtFpS5ZswAfvnF3lEQEZExuon19ev2i4OIyBZMloJs2bIF\nM2bMwNWrV1GzZk1cunQJTZo0QWJiom0CLKMU5F//Aj7/3CZhEBHJZo9SkKSkJPzrX//C9evXkZiY\niJMnT2LLli2YOXOmTY5f8jWr1YDLP+dGN28G+ve3SRhERLLZpBRk5syZ2L9/Pxo2bIiLFy9i9+7d\n6NixY4UOail169o7AiIiZXn22Wfx/vvvw8PDAwDQokULrFu3zm7xSB0h4eFAQYHdwiAisgmTibW7\nuzuCgoJQXFyMoqIidO/eHUeOHLFFbCbZs5FWan0Q45KHccnDuJTvwYMHep0fKpUK7rr1GHbi5qad\nNfeXX8RoIbak1M8I45KHccnDuGzPZI11QEAAsrKyEB0djdGjR6NmzZrw9va2RWwmSY30s88CoaFA\nXJxdwyEisrsaNWogOTlZs7xx40aEhITYMSJBpdJ2hvToAcTGAsuX2zUkIiKLM1ljnZ2djapVq6K4\nuBhr1qzBvXv3MHr0aFSvXt02AZZRY/3mm8CcOWK5Zk3gxg2bhEREZBZ71FifP38ezz33HP744w8E\nBASgXr16WLNmjeZCdGsz1mZHRgJvvCESapWKiTURKY8l2myTPdZS77SrqytiY2MrdDBLUqm0PdZE\nRCRERERg9+7dyM7ORnFxMXx9fe0dEgBxAeOVK/aOgojIukzWWPv4+JT6CQsLw6BBg3DhwgVbxGhQ\ntWrA3LlASopYzssDsrJsd3yl1gcxLnkYlzyMS/nq1auH5557DgcPHoSPj4+9w9EIDgZmzgQePBDL\nK1YAr75qu+Mr9TPCuORhXPIwLtszmVhPnToV8+bNw5UrV3DlyhXMnz8fo0ePxlNPPYUJEybYIsZS\n/vxTlIEAwOHD4vbuXcDPzy7hEBEpxt9//40ePXrgs88+Q0REBCZPnoy9e/faNaYHD4AWLcT9oiLt\nepaCEFFlY7LGumXLlqWmLW/VqhWOHz+OqKgonDhxwroBGql3WbwYmDQJ2LQJGDJEu55T5hKRUth7\nSvPMzExMmTIFa9euRZFuRmtFxl7ziy8C//0vkJmpneLc318sExEpgU3Gsa5WrRrWr1+P4uJiFBcX\nY8OGDfD09NQEYC9u/1SHcyYvIiJ9CQkJePHFF9GmTRvk5eVhw4YN9g4JxcXi9t//1q7jdTJEVNmY\nTKzXrFmDVatWoWbNmqhZsyZWrlyJ1atXIycnB5999pktYjRIGpZ16lT7HF+p9UGMSx7GJQ/jUr6I\niAgsXLgQXbt2xalTp7BhwwYM0T2tZydSh7lujm/LxFqpnxHGJQ/jkodx2Z7JUUHq16+PH3/80eBj\nXbp0sXhA5pJ6rNnjQUSkdfLkScWMBKJLSqylixcBtt9EVPmYrLG2N2P1LuvWAaNGld5e2a+GiJyJ\nLWus586di9deew0vvfSSwTgWLVpkkziMveaxY4FVq0pvzzabiJTCJuNYKxUbYyIiraZNmwIA2rZt\nq3f9i1qttuv1MBIbXTtJRGRXJmusbXUluVz2Dkup9UGMSx7GJQ/jUq5+/foBEBecjxs3TvMTGxuL\nqlWr2jk67cWLJR05YpvjK/UzwrjkYVzyMC7bM5lYR0ZG4pVXXsHp06dtEY/ZjCXW77xj2ziIiJTk\ngw8+MGudrRlrs48ds20cRETWZLLG+t69e/j222+xYsUKFBUVYcKECRg5cqTJi2MmTJiAbdu2oWbN\nmjh16pTBbaZMmYIdO3agWrVqWLFiBVq3bl06QCP1Ll9/DUycWHqfgwYB8fFlhkZEZBO2rLHesWMH\ntm/fjvXr12PEiBGa42ZlZeH06dM4dOhQmc+3dps9ZIjhtnn1amD0aDNeIBGRldlkHGtfX18899xz\n+OOPPzB37ly88847qFWrFsaNG4fk5GSjzxs/fjx27txp9PHt27cjOTkZ586dwxdffIEXX3xRVuDG\nej+qVZO1GyKiSqF27dpo27YtPD090bZtW7Rt2xbt2rVD//798dNPP5l8vr3a7JJVKtevA6mpsnZN\nRKQYJhPrwsJCbN68GQMHDsS0adMwY8YMXLhwAf369UPfvn2NPi86OhoB0vRaBmzZsgXjxo0DAHTs\n2BF37tzBjRs3zA7c3om1UuuDGJc8jEsexqVcUVFRiI2NxV9//YUxY8YgNjYW48aNw4ABA1ClShWT\nz7dXm10yse7SBahb1+zdmk2pnxHGJQ/jkodx2Z7JUUEaNmyImJgYvPrqq3j44Yc164cOHYo9e/aU\n+8BXrlxBnTp1NMthYWFIS0tDcHBwqW1jY2MREREBAPD390erVq3g7h7zz6MJ/9yK5du3E5CQAMTE\niOVZsxKwfj1w5oxYln6Z0uPlXZZYan+WWj5+/Lii4uH7Vb5liVLi4ftl3vLChQtx/PhxTXtlD716\n9cL//vc/eHt7AwAePHiAxx9/HH/88UeF9lvRNrtHjxiIKRES/tkqBgBw4kQCqlbVvoc3b+o/Xtk+\nI/ybqtgy3y95y3y/yl62SputNmHv3r1mrTPk4sWL6ubNmxt87Mknn1Tv27dPs9yjRw/10aNHS21n\nLMT8fLU6KkqtFgPvaX9ef11/uzFjxHoiIlszo4m1uKioKLPWGWLNNlutVqu7dy/dZv/wg/42QUFs\ns4nIPizRZruYSrynTJlSap2hCQjkCg0NxeXLlzXLaWlpCA0NNfv57u6ATge6RslSEAUM30pEZDNe\nXl44evSoZvnIkSMWGW6vom02oJ0xVxdnXySiysRoYr1//37Mnz8f6enpWLBgAebPn4/58+cjLi4O\nxcYGJJWhf//+WLlyJQDgwIED8Pf3N3hKsSyGGumS66yVWJc8naEUjEsexiUP41K+hQsXYtiwYejS\npQu6dOmCp556Cv/5z38qvF9LtNnu7qXXlUys2WYrA+OSh3HJo9S4LMFojXV+fj6ysrJQVFSErKws\nzXpfX19s3LjR5I5HjhyJPXv24NatW6hTpw5mz56NgoICAMDzzz+Pvn37Yvv27WjQoAG8vLywfPly\n+cGXiN7VtfQFMuyxJiJn0r59eyQlJSEpKQlqtRqNGzeGu6GMtgR7tNm+vkBGhv46ttlE5MhMjmN9\n6dIl1LXGJdpmKmtMwVdfBT7+WLvs5wcMGwZ8+aV2XWws8M03nAKdiGzPluNYS+7fv48FCxYgNTUV\nX375Jc6dO4ekpCQ8+eSTNjl+Wa+55FjW3t5AdrZ++1yrFnDjBttsIrI9S7TZRnusp06dik8//RST\nJ082eOAtW7ZU6MCWULL3w8MD+OorYNIkoFUrsY69H0TkTMaPH4+2bdtqRgGpXbs2hg4darPEuiyG\nyvcAUQ5i7DEiIkditMb66aefBgDMmDGj1M/06dNtFmBZSp7dlIZqvXfP+sdWan0Q45KHccnDuJTv\n/PnzeO211+Dh4QFAXMyoFCWTZ6ljKCfH+sdW6meEccnDuORhXLZntI+gXbt2ALRj/SlRyUZaSqx1\nr61kjzUROZMqVaogRydTPX/+vFkTxNiCsVLvnBzAx0fcZ5tNRI7MaI11ixYtjD9JpcLJkyetFlTJ\nYxmrd/ngA+CNN7TLTZoAf/8N7N4NPPqoWDdxIvD116zXIyLbs0eN9a5du/Dee+/h9OnT6NmzJ37/\n/XesWLEC3bt3t8nxy3rNzzwDLFumXa5WDXjwALh0CQgPF+tq1wauXWObTUS2Z9Ua661bt1Zox7bg\n5iZ6qfPyxLLUG6I7Mgh7P4jImfTq1Qtt2rTBgQMHAACLFi1CUFCQnaMS/P2BoCDg1i2xbKgUhG02\nETkyozXWERERZf4owZNPAuPHa5elBllKtK1JqfVBjEsexiUP41Kuv//+GwBw9OhRpKamIiQkBCEh\nIUhNTcWxY8dw6dIlO0cIfPQRMHZs6fXHj1u/h1qpnxHGJQ/jkodx2Z7J67D379+PKVOm4PTp08jP\nz0dRURG8vb1xzxZXCJrQpImYffG//xXLUsOckiJ6rV1d2ftBRM5hwYIF+PLLLzFjxgyoDDR8t2/f\nRsuWLbF69Wo7RCe4uIifkkaMAB56CGjfnm02ETk2k+NYt23bFt9++y2GDx+OI0eOYOXKlUhKSsKH\nH35omwBN1LusWweMGiXut2wJSKXfCxcCU6cCzz0nxrVmvR4R2Zo9aqzL0qtXL+zatcuqxzD1mnXn\nH/D0BHJzxf1ffwViYoA6dYC0NLbZRGR7lmizjZaC6IqMjERRURFcXV0xfvx47Ny5s0IHtSTdkUF0\n34srV2wfCxGRveXk5GD+/PkYNGgQBg8ejE8++QS5/2Sv1k6qzfHPZI6lSNfGsMeaiByZycTay8sL\neXl5iIqKwquvvooFCxYoqgfm7l3tfd2wpCH3rNVIK7U+iHHJw7jkYVzKN3bsWJw+fRpTpkzB5MmT\nkZiYqJmXQAmys7X3ddtsqefaWpT6GWFc8jAueRiX7ZmssV65ciWKi4vx2Wef4ZNPPkFaWho2bdpk\ni9jM0qED8NhjwP/+p7+evR9E5IwSExNx+vRpzfKjjz6Kpk2b2jEifYsWAY8/Dgwbpp9YZ2WJW7bZ\nROTITNZY25s59S4pKUC9ekCzZkBiolj30kuiAX/xRXFxo7JfJRFVRvaosR4zZgwmTZqEzp07AwAO\nHDiAzz//HKtWrbLJ8c15zQkJQPfugIcHkJ8v1n3xBfDss0DdukBqKttsIrI9q45jLalXr57BA1+4\ncKFCB7YkQ5OK6Y5lTURU2UmTehUWFuKRRx5BnTp1oFKpkJqaikaNGtk5On1Sm637/SUNNMUeayJy\nZCZrrA8fPqz52bt3L6ZOnYrRo0fbIjazlZVYs8ZaGRiXPIxLHqXGZUtbt27F1q1bsXPnTvz555+Y\nMWMGZsyYgT///FNRF5wD2jZ7+nTtOmuXgij1M8K45GFc8jAu2zOZWAcFBWl+wsLCMG3aNGzbts0W\nsZlNaqSffhqQ5q65f1/csveDiJyBNHnXDz/8gDFjxuDWrVu4efMmnn76aWzevNne4emR2uzx44HJ\nk8V9W0zsRURkbSZrrI8ePaqZbKC4uBhHjhzBkiVLcOLECdsEaEa9S2GhmM78xx/F5AN9+wJDhwLf\nfSca7c8/Z70eEdmePWqsW7RogQMHDsDLywsAcP/+fXTq1AmnTp2yyfHNec3JyUBkJHDuHLB9u5hz\n4N//BhYsEBPFXLzINpuIbM8mNda6s3i5ubkhIiICGzZsqNBBLU0ay1qtBmrUEPfZ+0FEzspFZ3pD\nF0NTHdqZp6e4dXERnR+ZmUB6uljHs4xE5MhMtrgJCQn49ddf8euvv+Lnn3/Gl19+qbgLYSQFBUC7\ndsBXX2mvNLcWpdYHMS55GJc8jEv5xo8fj44dOyIuLg6zZs1Cp06dMGHCBHuHpUcqBZGmOA8J0XaG\nGEqsdce+Li+lfkYYlzyMSx7GZXtGe6znz58PAJre6pKm6151ohDSjF716pXdSBMRVVbTp09Ht27d\nsG/fPqhUKqxYsQKtW7e2d1h6pMTa1VW7XNZZRh8fICkJaNjQ+rEREVWE0RrruLg4qFQqJCUl4fDh\nw+jfvz/UajV+/PFHdOjQAatXr7ZNgGbWu6hUwJo1wKhRwO+/A6+8AvzxBzBlCvCf/4iZGJlkE5Et\n2aPG2t7Mec15eaIcJC0NCA0Fvv0WiI8HNmwQtdfJyaK0b9o00ZaHhQEHDgAdO9roRRCRU7JqjXVc\nXBwAIDo6Gn/++Sd8fHwAALNnz0bfvn0rdFBrkXqsdXs/pKnNi4u1vSMqlWjIn3rK9jESETk7Dw9x\nK3V26LbZuh0gn34KSJNGOtn/J0TkoEzWWKenp8Pd3V2z7O7ujnTpKhOFKSuxLioSvdnffy+WdWb8\nLRel1gcxLnkYlzyMiyxBSp4LC8Wt1Gbn5ZU+syhdeym15eWl1M8I45KHccnDuGzP5KggY8eORYcO\nHTB48GCo1Wr88MMPGDdunC1ik81QYi1NFFNUBKxbB+TkiGX2fhAR2Zdum71vnygPkeYikEhnGtlm\nE5EjMDmONSDGst67dy9UKhW6du1q0wth5NRYr18PDB8OXLoEtGkD1KwpLnbZskVMl+vrCwweLGr5\nZs4E3n3XBi+AiJwWa6yNi48HBg4UPdL79gHR0WJ9cDBw44ZIpFUqYMUKIDYW2LsX6NLFqqETkZOz\nyTjWANC2bVu0bdu2Qgeytps3gerVxf0qVYCMDPEjVbFIpxwtdVqRiIjKb/Bg7X1plBBAJNWA9myj\n1GPNNpuIHIHyZg4op6Ag/QthJNJkY9I4qJY6rajU+iDGJQ/jkodxkTXottkS6XoYqTOEbbZtMS55\nGJc8So3LEipNYq3LUCOdmSluLdVIExGR9QwbJm6lNvvaNe2wqSdP2i8uIqKymFVjbU/lqXcpKtJO\ncy7x8gLu3wfGjAFWrwZefRWYO9eCgRIRlcAaa/OkpQF16hh+bPVq0W4DwH//C7zwArB5M9C/fwUD\nJSIqwRJtdqXssZbKPUJCtOvu3xe3JXusd+4EFDp6IBGRUwgLM34WMTdXe//6dXErXTNDRKQ0lTKx\nBsQwTjrDb2tISffHHwMffgj06QPMmiV//0qtD2Jc8jAueRgX2cKzz2rv6051LiXfly4BO3bI26dS\nPyOMSx7GJQ/jsj2zRgVxRG5uQH5+6fUuOv9KSBfHSFefExGR/SQnAw0aAPXra9fp9lhLifX06frL\nRERKUSlrrCWbN4txUidOBJYtE+ueew744gtxv3174PBhYMIE7eNERJbCGmv5unUTZxF79BDL770H\nvPmmuD9zJjBnjnbbpCQxVwERkSWwxtqEAQPEbe3a2nW6PdbSe3fmjJhQhoiI7GvPHuM91rpJNQA0\nalR6CnQiInuq1Ik1ANy+Dbz1lnZZN7GWJhz44w/g2DF5+1VqfRDjkodxycO4yBaqVtXeP3LEMvtU\n6meEccnDuORhXLZX6RPrwED9ixh1ezdKjgby66+2iYmIiIzz9NTeN+ciRY4SQkRKUalrrPX3I26b\nNwf++sv4dq1aiQTb37/ChyQiJ8ca6/LJzxcTfT38sDijaEpWFuDtXaFDEhGxxlquWrXKTqoB4Phx\n4OJF28RDRESlSWcZze2J1q3DNmTePMOjRBERWZpTJdbGZvYqyZyLYZRaH8S45GFc8jAusgWpDc7J\nMW97U9u98gqwalVChWKyFqV+dhmXPIxLHqXGZQlOlViby8nO3BIRKc6KFUBQkHnbmuqxBsSkYURE\n1uZUNdYxMYA5/yTt3w906lThQxKRk2ONdcV06QL8/rt2uVcvYNeu0tvt3w/cvStqsn18DMUE7NsH\nPPKIRcIiokqKNdYyXL8OPPGEedua6v24d4+92kRE1qZb4nHtmv4wfLo6dwZ69wbWrjW+r/x8MRX6\nRx9ZNkYiIl1Ok1gHB5dOmF96yfC2phJrPz/grbcSLBKXpSm1bolxycO45FFqXFQxvXtr79eqBXh4\nlL19cLDxxw4fTsBXXwGvvWaZ2CxFqZ9dxiUP45JHqXFZgtMk1oB+78ePPwLVq5vezpirVy0TExER\nGfbee8BTT2mXdeckMKSoyPhjBQXaScGIiKzFaWqsAeDnn0WNHiBKOebPB15+2fj277yjP2ujNiZg\n5kzg3XctEhYRVVKssa64sWOBVatEmz1+vLiosSxFRaVn2HV1BTZsAI4eBebOZSkfERnGGmuZevYE\nvvlGu2ysXk9y9qzxx3iFORGR9en2UpcsBXn9deCxx/TXXbwIfP65dlkaC/uZZ9hjTUTW51SJNaDf\nSFerVva2ERHGHzt/PgG3b4uZGpVEqXVLjEsexiWPUuOiiisrsXZ11Z/+HAA2bQImT9YuS+Uh9+4l\nIDVVf1u1GjhwwHKxlodSP7uMSx7GJY9S47IEp0usdRtmU4n1jRvGTxkWFgJnzgAnTlguNiIi0qfb\nZhuqsS6ZWCcm6i/rzt64fr3+Y6mpYkQRlQq4fbticRIRAVZOrHfu3InGjRsjMjISc+fOLfV4QkIC\n/Pz80Lp1a7Ru3Rpz5syxZjgA9BvmkqUgrVoBb7yhXf7yS+NToAcFxSjytGJMTIy9QzCIccnDuORR\nalyORoltdp8+QNOm4n7JHuubN0sn1ocOiVupp1p7QWNMqX3rtuHXr1c00vJR6meXccnDuORRalyW\n4GatHRcVFWHy5Mn43//+h9DQULRv3x79+/dHkyZN9Lbr1q0btmzZYq0wSinrtCIABAbqL589Czz3\nHLBnj/72d++WfQU6EZEjUWqb3aeP+AG0bfCbb4oRQwoKSneQnDkjbu/fB3x9RY91YCCQkaHdRqUS\nZyN1z0jm5VnvNRCR87Baj/WhQ4fQoEEDREREwN3dHSNGjMDmzZtLbWfrK+YjI7X3DSXWfn76y+fO\niRq8zEz99d99l6B3ilEplFq3xLjkYVzyKDUuR6LUNluX1DEyZ46YqGvxYmDkSMPb3r8vbouKADc3\nAEgotU1+vvb+nTsi4bY1pX52GZc8jEsepcZlCVbrsb5y5Qrq1KmjWQ4LC8PBgwf1tlGpVPjjjz8Q\nFRWF0NBQzJs3D02lc346YmNjEfHPlYT+/v5o1aqV5jSC9MsxdzktLQG//goAMf8k1gn/HCUGqalA\nSop2GQB27xbLubn6+wOAWbPE8xMSzD++tZePHz9u1+MbW5YoJR6+X+Vb5vtV9vLChQtx/PhxTXvl\nSJTaZusut20LlGxzVaqEf44a88+tWM7OjoFKBcyZU7ITRLu9SKzF8l9/iefv2pUADw+x/z17gMLC\nBLi68m/K3vHw/SrfMt+vspet0marrWTjxo3qZ555RrO8atUq9eTJk/W2uXfvnvr+/ftqtVqt3r59\nuzoyMrLUfqwYovrgQXEy8OJFtbpvX7W6e3e1euNG6QSh+GnRQtwmJurGpP9TXKx9rGNHtfraNf11\nROScrNl+WZojtNnGSG3xsWPa+z/9JG4nTFCra9RQq8+f12+3ExL0l7t0EbfXr6vV+/ap1fPmieUf\nfrD5yyEiO7FE++ViuRRdX2hoKC5fvqxZvnz5MsLCwvS28fHxQbV/hubo06cPCgoKkKFbCGdlHh7i\nNiJCzMT4yy9A/fr62/z9t7iVTisCpeuwdce0PngQCAkBNm60eLhERFbjCG12WTw99Yc/ffxxcfv1\n1+IiR29v/e3XrtVf3rdP3N69CyQlaZd1y0WIiEyxWmLdrl07nDt3DikpKcjPz8f69evRv39/vW1u\n3Lihqdc7dOgQ1Go1AktmrVbUsiUQHy/uS7V1rVoB/5whACDGSQX0E2txWjFBs1ylSukLXy5csHCw\nZip5mkUpGJc8jEsepcblSByhzS6Li4lvs2PHEvSWv/jC8Hb37on2PDtbLFt79CelfnYZlzyMSx6l\nxmUJVquxdnNzw2effYbHH38cRUVFmDhxIpo0aYKlS5cCAJ5//nls3LgRS5YsgZubG6pVq4Zvv/3W\nWuEY5OICDBpk/HEvLyA8XPRa378PDBggptQ1dNFidrb+xZDs5SAiR+IIbXZZpHkJVq8Gxowp/XiV\nKsBbbwHvvlv2fu7eFe13VpZYVuKwqkSkXCq12o6XeJvBEvO2y9WtG/Dbb6Lyrn9/YOtWcTpxwgRg\nyBBRNrJxI9Cvn/Y5qalArVra5HrmTDEZwb59QM2aNg2fiBTCHu2XvdnjNZ87J0YMka4/OnIEeOop\n/TOHUkjS2ck+fYAdO0rva9MmIDkZ+OYb4PRpYM0aICoKaNbMqi+BiBTAEu2X1UpBHJnueyoNzzdh\ngrjdtEmcJpTq9yTZ2cC6dfrL584BV65YN1YiImcXGalNqgGgXTtg6tSyn1OrluH1v/0m2vi7d8Vy\nQQHQvLm2Lc/NrXC4RFSJMbE2QDexbtDA8DZ79yboLe/bB4wbp12Wps7NyRGnEi9dEmOuJiVZNtaS\nlFq3xLjkYVzyKDUush+p5lrqBCn5GWnUqPRzJk4Evv9elILcuiXWxcaKW6kEsGpV0WliKUr97DIu\neRiXPEqNyxKYWBswdSowZYq4P3gwMHRo6W1KXiizYYP+8rVr4vbePXFactgwYNIk4KOPLB8vERHp\nk9po3dl2JQsXAi+/XHp9ixaiM+TChdIXpOfkaO8rZCAUIlIg1libITMTeOEF/eRZrQaOHQPatCn7\nud9+CwQHA88/L6ZHj40Fli+3arhEpBBKaL9sTSmvefdu4LHHgIEDgR9+0K+x/vFH4IknSs+0uGiR\nGIavsFB0iOg6elS09yoVsH8/0KmT6RgyM8XMj0FBlnlNRGRdrLG2kYAAYNq00utbtzb93JJDNyng\n+4aIqNLr0UOU4ZXssW7YsOy2OyysdFINiDa8qEjcN9aOFxaKs5UrVoixs7t0AZo0KVf4ROSgmFib\nyddX3L7/vrgtWR9Uo4bh59l66Cal1i0xLnkYlzxKjYvsS6XSzkUgfUaSkoDatcW6XbvEMHwStRpI\nSTG8rzFjtN8D9+4Z3mbePLHv8eOBJUuAtDRtrbYhN24A8fEJ5r4cm1Lq3xTjkodx2R4TazNJY6T6\n+GBxWtgAACAASURBVBh+vF49w+v//rv0ZAPZ2ey5JiKyBbcyZmvo2VNcR6Nr0ybtfSkpB4DLl4EH\nD8T93r21nSW6btzQ3s/LM1zfrat9ezGEK2u2iSoP1lib6dYt0Sv93/+Kemvder2uXUVjXLLXukUL\nMb71f/4DjB0r1o0cKYbl27xZjJFNRJWXUtovW1Laa54wQVzXYiykkyeBnTuB114TNdYvvaStvVap\njD/v5ZeB6dPFPAWnTgE//QS8/rr28e7dxTjYN24Y30eVKuKM5ssvAx9/XP7XSESWwRprGwoKEj0U\nhnogWrQQj5e8EKZ2bTFWqm69njTWtTRqyJEj+lebExGR5XTuXPZ05y1bAq++avgxY2coAVH20bu3\nOFvZv79+Ug0Av/6qHfP67FlxseOrr4qhWSXSd4ZUu01Ejo+JtQze3trTilJ90IsvirFPgdJJt0ol\nJhb466/S+5Ia+vbtxdBPlqLUuiXGJQ/jkkepcZH9PfusSFzN+YyU7Kjy9QWuXhXJt6FRQE6eFCUi\n0mQyJUnrT5wADh4UvdIffaTtTBGJdQKKi8WQrFeuiLOj06eb++qsR6l/U4xLHsZle0ysZSqZPC9e\nrL3CfNQo/cfUanGFeXJy6f3o1u6VHC+ViIhsr+S1Mr6+QEiIOLO4dy/Qtq3h5xm7mFGi295v3So6\naf77X20Hy6efAhs3Ar//LspSPvlErL90SfyY6/5987clIutgjbVM330HDB9uuGauqEicHpROCT7+\nuOiRnjOn9LbLlonaP5UKmDULiIuzathEZAdKa79swVFfc34+4OEh7uflAYcOiTkIGjbUbjNpkuhM\nMYe/P3Dnjrg/dapInkvy8tJPhjduFD3W0nU8gYEipuvXTR/v0iUxrbsDvvVEisEaazuoWtX4Y66u\npadAL3nFudRzcfKk9mIVNoRERPYlJdWAuKgwOlo/qQZERwkghuTz8hL3ExNFD7NaDVSvrt3277/F\nbWioSJgNKdnDfPeudgSpq1fFBDO3b2trsIuKRO94ye+MoiJtEk9E9sXEWqa+fYE//jBeHzRwoHgc\nEL3RJScikBrmTz/VXjBz+bLhYxUXi16TI0dE7d2+fcCbb5Ydn1LrlhiXPIxLHqXGRcphic9IbKxI\nauvW1Z5lbNpUO4FYaKh2WynJ7tdPtN+SVatKRaa5N3Gi9vtA2ldhoUj616wRI5f4+YkSkkOHxHJG\nBlCnjvb7xFhp4VdfieECb9/WJu9lUerfFOOSh3HZHhNrmVxcxFXmxri6lv24IcuXG77AMSEB6NhR\nJOMjRoihoKQJaoiIyH6efVZ/zGtAXFMzfLi4L12P06+f/jaGJhNbvhwYOlTcN1QyUlwsJqiRRpUC\nRFnKRx+JBP7aNTHZDQAcO1b6+R06iHj//FMk4c2aaTuAiMiyWGNtJSqVGIppxw79Yfhatzbc8E2c\nKGqt69TRrtu1S9RpS89r0gRYu5alI0SOwlHbr4pwxtcs+ftvkbyOGSPaaQ8PMdRe3bqiVCM3V0x1\nHhWl/zy1WpSFLFkCvPKKeccqa4ztTz8Vcyj07g089pj2O+irr4BnntE/rrmSk4HISMPPkSY+k2am\nJHJUrLFWMFfX0o0nAGzbJmr3Slq2DGjUSAzrNHOmaAilpBoQDXJZM4gBoiH9+eeKxU1EROXTpAnw\n6KNAly5iOT9fXFCoUgEBAWKEkZYtRXKamiq2kXq9vbyAGTNE2YjuBe87d2rvf/CB9n5Z3/3TpwPz\n5wPvvSeSXsmkSfrbJSWJEsP8fODiRXHsXbuAuXNFyYhKJSa4yc0VteSA4TG3lywRJSqWNnw4UFBg\n+f06C3NKfspDrRbXA5T1GSwuFhMnSbOVSi5dEp+3zEzgzBnx+c7KEtsXFIjbtDTg6FGx3b59wPr1\n4p/W9HTxeTx9WlynduKEWL9zp8h91q8XEzXt3QucO2e/zw57rMspISEBMTExRh8vKBDJtYuLqKl7\n4QVx4eOtW6LH4t690hc6SqKixAempHbtRL21sbdjwABgy5YEHD0agzZt5L8mazL1ftkL45KHccmj\n1PbLmpT6mpX4GVGpgOXLExAbG1PqsVmzgHfeEYmGi4u4mNHPT5uA16sHTJ5seL9Nm4rkw5ixY4GV\nK0uv79VLW1Iiar9FXAEBYmSTuDjxj0FSEnDggPinwd8fmDJFzDC8bJm4zkiqCe/cWSTsBw4AgwaJ\nBD0qCvjXv8Tz/v1vw9+DmZkiqapVSxyrVi3g8GHx/RcWloDRo2OQni6StKgo0dt/5ozonAoMNP66\ns7JEPIGBomxHrRalOp9/LmraVSrtAANqtXgvYmLEuOP+/qX3p1aLMxEBAeLz1bFjjGaAA93ZmfPz\nxe8xNVUMtZibK249PMTra9FC/L7eeksM6RgUBDz8sLiINiREJImLF4syojffFP8ABQaKBFStFsnl\n+vUijuxs8Vr+9z9xvFdeSUB4eAwaNBCvbfFiUTrk6yv20bq19kx4dDSwZ4+I1dVVfMZ+/lm8x9Wr\ni/3euSOOBYic4/p1kQR36CC2f/BAJLa3boltPDxE7nLzpngPDx/W/3wFB4vrAu7cEe9V1ari+Lm5\nYoImtVr7D0LVqqKDMSxM3N6/L/4u6tUDPD3F9jdvijKspCTx2v74o/TkfWWxRPtlog+Uykt3vOun\nnxajgxQWiuUaNcTP5s3ig1mSoaQa0M7WmJMjhoFKSBB/eM2aifXSmKhjxpTdqBIRkf3l5RmvdX7l\nFZHo6pZ8SLcvvgjUrw888YRICnfsEMnLN9+IBNfbGxg/XkzP/thjwNtva79X8vNFErNypUiGDh3S\nHlObVOvLzNRerHnypLht3VrENn26OCYgShqff177XefmJu6rVMCHHwLVqun3YC5eLJK2kBDR6+jq\nKoarfe897TaNGonE6eJF7bqdO4HVq7XLLi7anvnatUWiX7068MMPIglt0wb4/nvxOgCRgD35JLB9\nu+h53bZNu68nnhCJfM2a+mcInn1WvFcdOoieWFdX8T2bmak/tGL79iIx3rFD/z00VgYKiN+XlDzq\nnqGQNGmiHWXGUA1+UJBIZD09xT9V0rF79QI2bBDv+/Xr4izEwIFA167ifbl5U+QRAwaI9yE4WAzQ\ncOWK+D0FBorHOnYUMeblifyjZ0/xD969e+J5DRqIz2NxsSh7ki7mDQkRv//jx8UIO/fvi+c0bize\n+yFDxO/r1CmRLAMise/fX5wZSU8X+7hzR8SSmys+S56eht/HklJT5SXVlsIeazvauRPo00f+81JT\ngfBwcT8wUPyxACJ5//570RCdOWO5OImofCpz+2WMM75mW9q4UbT1ZU3TfuuWSH7+/FMkFrt3iwTp\nu++03zn794skMDNTJEu3b4vvj759tRfgz5sHfP21KCG5dk0k6W3aaGup27UTPZLduokEKypK9Kg+\n9JBI6nr2FI9J43ifPSsS8R9/1I/X01Mk5IDhxPHf/9ZOmrNxo0gc27QRyd2XXwIXLogSnL59gdGj\nxfEvXDD83kybBvzf/4kedV9f8f5MnAj06CFm1/TyEt+fu3Zpxw+fM0f8E+DpKV5jx44i+ZT+odiw\nQUz8s2uXSC4nTxa3nTqJntOaNUWi+ssvogfcw0Mk5gEBYjk9XfzD0rKluF+3riiBiIoS8TVoIJJa\nDw9xe+uW2OeFCyJJBUTC6uUl9pufL/6p0f2M5OaK7Zs2Nf65Icu0X0ys7SgjQ1xI8uWX4tTc9u3i\n6vDx48XV3+PGiT+Qko4d0x/Gb9o0cdrwmWdEvV5kJPDFF+K/3OBg270eItJXmdsvY5zxNVc2KSmi\nB7V5c8OPd+kCfPaZKGHQnVUSEL3qhnoJ8/LEGVa1WiSlrq6it/LePdGDLJHqbC9fFr2dvr5iu/h4\nUQrTo4fp+KVjBASIMpT0dDHqyuHDosSipMJC09cw6ZZ26L4mDw/79Io6q4KiAuQU5iCnIAcPCh4g\np/Cf24KcUvc93TwxqsUo0zvVwcTajixdr5eYKBqx4mJtfVO3bsBvv4nHk5JETzQght779tvS+xg6\nFNi4MQFSXdzEieI0oaen2Kc9//iVWN8IMC65GJc8Sm2/rEmpr1mpnxHGJQ/jksdecRUWFyI7PxvZ\n+dnIyssSt/lZmuU/9/+JkBYheutLbns//36pZLlYXYxq7tVQ1a2quHWvqlkueT/cLxyzY2bLips1\n1pXIQw+JCxx1L54IDRV1R2lp2v/oIyNL125JdCchAMR+dOuyXV1FL4Tu7GBERETkvNRqNXIKc4wm\nwaWSXyOP6S7nF+XD28Mb3h7e8PHwEbdVfDTL927fA7IBbw9vhHiHGNy2mns1zY+ULLu7uEOl8FME\n7LFWsDfeEKeaFiwQZSPShQSbN2u3mTFDDKtkyOTJone7Sxdx5ezs2cDSpRwHm8hWnLH9csbXTGRL\nUm+wwaS3ZLJrRhKcnZ8ND1ePUgmwsaS4zPX/LFd1q6r4BNgQ9lhXcjNnip7mBQugGcKneXNx4cLv\nv4vll14ynlgnJACPPCLub9qkf/W3OTIzRY0aERERWY5arcb9gvvIyMlAZk4mMnIyxP1cI/d1tnlQ\n8ECb+BpJbKXl2t61TW7r7eENNxemg5bCHutyslXdklotSjdu3xYXLTZsqB2a5/x5UUKiP9tVAmbM\niEF2tuid7t4d+PVX/X0OHy6uiFapxFTqLVsarr+WJgeoWbPir4P1Z/IwLnmUGpdS2y9rUuprVupn\nhHHJo7S4CosLkZmTiZ27dyKyTaSsRNnD1QMBVQMQWDUQ/9/evQc1eWZ/AP+GiyIXBSGAXBQECTcN\nKGJttaVVi9si6tqLOtWt1bZj1+3a3Tq1szs7nd9s1a7jTN11221tt1Y71bqXbtWVaq26WhWtBUcF\nEYREuQoBInfD5fz+OCUYBUuUkBdyPjMZkjfJ+543pKeHx/M+z8hhI+Hj1sP9217jNdQLTqrere+n\ntM+rk1LjkhFrB6BScRsIAItFXzw9eSoeIh6N/t//gD//mZ976y2eCL+oqPuVGHfv5tuHHwIvvcTb\nmpt5Qvm0NG4baW7m7bevmiSEEEIMNkSEupt1MDQZUNVUharGKlQ1VcHQZOhx5Li2pRaNpkZ4u3nD\nrcQNIZUhdxTBYd5hmDhq4h2Fso+bD4a6DLX3aQsbkBHrQUSlAl5/Hdi4kR/X1nIP9iefdP/6WxcH\nWLuWJ/D/y1+4vWTYMC6uz5/nKZWEENZzxPzliOcslMnUbkJFQwXK6stQVl+GysbKOwrnWwvoIc5D\noHZXQ+2hhtpdDT93P/i5+8F3mG+PI8vWjB4L5ZPp9oSFadN4Ivvb/3XlwgWe+7Oqiif7f+EF3h4U\nxKPTAM82UlBw5z4//xxYtMhy282bPNr95JPciiKE6J4j5i9HPGfRvzqoA1WNVSirL0Npfam5cL79\ncW1zLfw9/BHkFYQgryD4e/hbFM23FtFqDzXcXHq5pJ8YtKSwtiOl9gf1FNeJE1x4d07en5wMvPtu\n95Pl3+7QIV5sxsmJl3pNSeHWk61beYq/Cxd4Ra+7XQA80D4ve5O4rKPUuJSav2xJqees1O+IxNWF\niHDj5g2U1vVcLBdlF8EYaIS3m7e5YA7yCkLw8GAEeVo+Vrur4ezk/NMH7gPye7SOUuOSHmvRaw88\nwCs7AnwR5K0rZk2bBoSG8mqPnaZN4yVVAV7GFuAR7tJSLqoBXp52927g6lXA1ZVXrxJCCCG6c6Pl\nBnRGHXS1OuiNepTUl3QVzz8W067Orl3FsVcwgryCEOUbhZSwFAR7BeNa4DXMmz1P+pOFYsmItYNT\nqXjkecUKbhNZvpy3P/kk8N//8n0/P57iDwD+8x9g3jy+7+YGtLR07Ut+TUJYcsT85YjnLFhzazP0\nRr25eNYZdRaPTe0mhPuEI9w7HOE+4QgdHmox6hzkxVPDCWEv0goi7tvZszy7iKsrP66tBQ4c4CJ5\n8WLg8ceBZcvu7LPuzo4dgE4HrFwJtLfzNH01NbxqpKsrF+d+flzME/HiNZ9/3jVyLsRg44j5yxHP\n2VG0treiuK7YXDTfOvqsM+pQ21yL0SNGdxXPPxbQ4d7hCPMOg5+734BcNEQ4Dims7Uip/UG2iKuu\njgviU6eAhATuuZ49m5/LyuJ5sF26aSraswdIT+f7K1cexfvvp2DrVuD55/lCys7WkqCgPg3XKo70\ne+wLEpd1lJq/bEmp56zU74iS4iIiGJoMKKwtxL6D+zA0YiiKjEXmQrqioQKBnoEWBXPn/TDvMAR5\nBdl8hgwlfV63kriso9S4pMda9Ivhw7monjiRR5snTwbCwoAtW7jQVqm4J3vJEuDll7ve11lUA8D7\n7/PPF18E/u//gN/9jh9/+y2PbKemdr32yhWepUSB/28WQogBjYhQ0VCBC5UXUFhTCJ1Rh6LaIhTW\nFqKwphDOTs6I8ImAV7kXpo6ZiodCH8Jz458zt264Orva+xSEUDQZsRZ9igiorOSZQlavBh5+uKuo\nvpuFC7nYHjcOOHiQi/K6Ol4IR/7lUAxUjpi/HPGclYaIUNVUhaLartHmotoi6Iw6XKy8iPaOdkwI\nmIDIkZEY6zMW4d7hiBgZgQifCPgM87F3+ELYjbSCiAHho4940ZkffgDWr+ee7s8+45lKUlIs+7fT\n0oB9+7oer1zJU/3l5fEoeXo6sH07ry7Z2Ah4e3M/t5OTFOBCeRwxfzniOdvLzbabuFJzBZerLyPP\nkIfL1Zdx2cD3nZ2cMdZnrLlw7mzZiFPHIcgrSHqdheiGFNZ2pNT+oIEYV2d+X7CAi+ShQ3mZ9u6m\n7xsyBDCZuCgvLuZR7lde4ec+/pj7t516aPFTqYBz57iw701c9iRxWUepcSk1f9mSUs9Zqd+Rn4qL\niFDZWGlRPOcZ8nDZcBkldSUI8w6Dxk8Dja8G0X7R0PhqoPHTwM/dz6Zx2YvEZR2JyzrSYy0Ghexs\nXsFx+PA7nzt9GtDreVaRxEQgMpJHrffsAdas6SqqAZ4qcPlyHs1etw549lle0v377wG1ml+zbx8Q\nF8fbjEaZkUQIoQxEhOuN15FTmYOcqhxcrLyInKoc5FblwknlZC6ao/2i8fDohxHtF42xPmOl51kI\nhZERazGgEAEdHVyMJyVxcezjwxdAVlbya1QqnqVk8mTg5Mmf3ufp07wy5bZt3Lbi5wccPgzMmcP7\nFeJeOWL+csRztkZreyv0Rj3yq/MtRqFzKnNAIMSp4xDvH484dRzi/OMQp46D2kNt77CFcAjSCiLE\nLRYsADZt4l5sgFtGtm0D6uuB11/vet3q1bycO8CtI9u2AR4e3LN9u+nTgV/+klefvHiRL6hMSuIR\n9LNnuR0lKIgX1AF4X489BowebbPTFAOII+YvRzzn7rR1tCG3KhfZ5dk4X3kelw2XkV+dj2s3rplX\nE+xs4dD4ahDnH4cAjwDpfRbCjqSwtiOl9gdJXD1raQFaW3nBmq++4plLpk3riqupCdi7l+fpDgnh\niyafffan9ztyJC8RX13NxTfAxfWYMbzIzsMPc2/40qXAH/7Ac3c/8sjd96mEz6s7Epd1lJq/bEmp\n52yr70hreyuu3riKy4bLuGS4hEuGS8ipzMGFygsIHR6KxFGJ0AZoEeMXgyjfKIz1GWuxHLdSv7sS\nl3UkLusoNS7psRbCCm5ufAOAuXP5dvRo1/Pu7lxI31pMP/MMX1D5+efcXpKTA7z9NvDee9zfvXMn\n7+df/wKKioAXXgB+8xtuJQF4BJuI+8fr6nh1SoCL7WvXgCeeAJqb+djR0Tx7yvr1XHwfOwb8+tf8\n3oYGbnsZNYpH4t3dufWls3ec6M6LNon4j4lhw2zxaQrhWBpMDcgz5OFi5UVcrLyIC5UXkF+dj7L6\nMvMIdIxfDJKDkvG89nkkBCbAa6iXvcMWQvQzGbEWwgodHdxaMmIEz1rS3YqTeXncCjJ0KBe7V6/y\nFIFnznC7yWuvccF9/jwQGgpoNNwzfuYMv7+zCL+bGTN4cR1PTy66AS7wi4u7fn70EW9fsYKPkZjI\no/A1NUB+Pr/vuee4J72jw/JCzuZmy4KciP9wGDtWpjW0hiPmr4F+zk2tTbhUdYkL6KqLyKnkCwkN\nTQZE+UYh3j/efIv2i8aYEWPkAkIhBglpBRFiECko4GI9JoaL3sJCLr4fe4zbVqZMATIzeRRap+O2\nFU9P7vcuL+eiPTGRn9+9u+fjBAfziDjAfwBcu8YXbBoM3L5CxNs6hYQAJSV8/3e/41aaBx8EAgOB\nXbt45NzPj/e7fDmP7jc08Mh/YSGPqru7c8tMezvHe7vW1u63D3SOmL8Gyjl3jkDnVOYg15CL3Kpc\n5FTmoLyhHONGjrMooOPUcQjzDoOzk0wjJMRgJoW1HSm1P0jiss5gjautjYtYgEeic3N5hpO6OiAq\nigvlIUN4pLyggF+bnc1L1Ds58XSEvr48veHp07wa5uzZQHb2UXh6puDSJW5NiYjg/er1XNwDPJrf\n1MT7uXmz+/jCw4HYWC7Sd+/mEfOnnuICvaQEmDSJC/rAQL5ItKKCjxEVxaPsJhPPBpOfz3HcuHEU\nAQEpKC8HfvYzvqB0yBBu/amr4z861Go+Tnl51/O2ptT8ZUtKO+f6m/W4ZLiEf2f8G+1j2pFbxUX0\n9YbriPKNQqw6FnHqOMSqYxGrjkXEyAi4OPVfl+RgzUG2InFZR+KyjvRY29G5c+cU+aWQuKwzWONy\ncbFsU5kwgX8GBvLPkBD+mZrKt7tZtarr/rvvnsPq1Sloa+M+8pkzu3q7O3u629p4lhWTiZ/LyuLt\nRFwEBwbyCHVNDRe5jz/Oo+UmE7ebnDgB1NZyQTx8OBfhRUX8B4JeD8ybx6Pfzc283d0daGo6ByAF\njz8OrF3L+1epuHBvbOQ/HMLC+P2dYmJ4P0Qc05AhfNz6er6vUnFhPno0Pz9zJjBt2j3/SoQNtXe0\no7S+FFdqrqCwppCnr/txDuiqxipE+0XDKdMJCzQL8NLElxCrjsVYn7GKGIEerDnIViQu60hc/c+m\nhfXXX3+N1atXo729HStWrMAbb7xxx2teffVVZGRkwN3dHdu2bUNiYqItQ+ozRqPR3iF0S+KyjsRl\nnc64XFy4IL6VSmXZl915oegDD1h3jI8/7v1r6+u5Heatt4x4800+Zm0tF/ZEwJdf8nzkjY1chI8e\nDZSVcVHu5ARUVfEoeUUFcP06n8PIkfxcWxvv32jklpZf/YqLfnd3685nIFFyziYiGJoMKKwtRGFN\nIQprC3Gh8gIuVV1CYW0hfIf5ImJkBCJ8IhDlG4VXkl5BrDrW3MLxVvlbeHP6m/0SqzWU/t+60khc\n1pG4+p/NCuv29nasWrUKhw4dQnBwMCZPnoz09HTExMSYX7N//35cuXIFBQUFOH36NFauXInMzExb\nhSSEGGS8fpx0oXN0GeAWkU4LF/JPD4+uxX6Cgu7tWESD+8JNe+fsDuqAocmA4hvFKK4rRkldCYrr\nilF8oxhXaq4gvzofKpUKET4R5gJ6nmYefj/99xjnOw7uroP4Lx4hxIBhs8L6zJkziIyMRNiPq3Us\nXLgQX331lUWS3rNnD37xi18AAKZMmQKj0Yjr168jICDAVmH1Gf2t/6asIBKXdSQu6zhyXIO5qAZs\nm7OJCNXN1d0WzZ33S+tK4TnEE6EjQhEyPAShw0MROjwU8ZHxiBjJo9B+7n73fH6O/N29FxKXdSQu\n6yg1rj5BNvKPf/yDVqxYYX68Y8cOWrVqlcVr0tLS6MSJE+bHM2bMoLNnz1q8BoDc5CY3uQ3Y20Ah\nOVtucpOb3O4/Z9tsxLq3y7LSbVdf3v6+258XQgjR9yRnCyHE/XP66Zfcm+DgYBQXF5sfFxcXI6Rz\nKoIeXlNSUoLg4GBbhSSEEKIHkrOFEOL+2aywTkpKQkFBAfR6PUwmE7744gukp6dbvCY9PR3bt28H\nAGRmZsLb23tA9FcLIcRgIzlbCCHun81aQVxcXLBlyxakpqaivb0dy5cvR0xMDD744AMAwMsvv4wn\nnngC+/fvR2RkJDw8PPDJJ5/YKhwhhBB3ITlbCCH6wH13adtQRkYGaTQaioyMpA0bNvTrsZctW0b+\n/v4UHx9v3lZdXU0zZ86kcePG0axZs6i2ttb83Lp16ygyMpI0Gg0dOHDAZnFdu3aNUlJSKDY2luLi\n4mjz5s2KiK25uZmSk5NJq9VSTEwMrV27VhFxdWpra6OEhARKS0tTTFxjxoyh8ePHU0JCAk2ePFkx\ncdXW1tKCBQsoOjqaYmJiKDMz0+5x5eXlUUJCgvk2fPhw2rx5s93j6jxObGwsxcfH06JFi6ilpUUR\ncdmD5Ow7Sc6+N5Kze09ytnVsnbMVW1i3tbVRREQE6XQ6MplMpNVqKTc3t9+Of+zYMcrKyrJI0mvW\nrKF33nmHiIg2bNhAb7zxBhER5eTkkFarJZPJRDqdjiIiIqi9vd0mcZWXl1N2djYREdXX11NUVBTl\n5uYqIrbGxkYiImptbaUpU6bQ8ePHFREXEdGmTZto8eLFNGfOHCJSxu8yLCyMqqurLbYpIa6lS5fS\nxx9/TET8uzQajYqIq1N7ezsFBgbStWvX7B6XTqej8PBwamlpISKiZ555hrZt22b3uOxBcnb3JGff\nG8nZvSc5u/f6I2crtrA+efIkpaammh+vX7+e1q9f368x6HQ6iySt0WiooqKCiDhZajQaIuK/Zm4d\nnUlNTaVTp071S4xz586lb775RlGxNTY2UlJSEl28eFERcRUXF9OMGTPo8OHD5tEPJcQVFhZGBoPB\nYpu94zIajRQeHn7HdnvHdasDBw7QtGnTFBFXdXU1RUVFUU1NDbW2tlJaWhodPHjQ7nHZg+Ts3pGc\n/dMkZ/ee5Gzr9EfOttnFi/ertLQUoaGh5schISEoLS21Y0SwWAghICAA169fBwCUlZVZXD3fX7Hq\n9XpkZ2djypQpioito6MDCQkJCAgIwKOPPoq4uDhFxPXaa69h48aNcHLq+rorIS6VSoWZM2ciIgWQ\nmgAABtBJREFUKSkJW7duVURcOp0OarUay5Ytw8SJE/Hiiy+isbHR7nHdateuXVi0aBEA+39eI0eO\nxG9/+1uMHj0aQUFB8Pb2xqxZs+welz1Izv5pkrN7R3J270nOtk5/5GzFFta9nVPVXlQq1V1jtHX8\nDQ0NWLBgATZv3gyvznWd7Rybk5MTzp07h5KSEhw7dgxHjhyxe1z79u2Dv78/EhMTe5xf116f14kT\nJ5CdnY2MjAz89a9/xfHjx+0eV1tbG7KysvDKK68gKysLHh4e2LBhg93j6mQymbB37148/fTT3R63\nv+MqLCzEu+++C71ej7KyMjQ0NOCzzz6ze1z2oPTzsPfvQXJ270jOto7kbOv0R85WbGHdmzlV+1tA\nQAAqKioAAOXl5fD39wfQ/3O7tra2YsGCBViyZAnmzZunqNgAYMSIEXjyySfxww8/2D2ukydPYs+e\nPQgPD8eiRYtw+PBhLFmyxO5xAcCoUaMAAGq1GvPnz8eZM2fsHldISAhCQkIwefJkAMBTTz2FrKws\nBAYG2v3zAoCMjAxMmjQJarUagP2/92fPnsWDDz4IX19fuLi44Oc//zlOnTqlmM+rP0nO7pnk7N6T\nnG0dydnW6Y+crdjCujdzqva39PR0fPrppwCATz/91Jwg09PTsWvXLphMJuh0OhQUFCA5OdkmMRAR\nli9fjtjYWKxevVoxsRkMBhiNRgBAc3MzvvnmGyQmJto9rnXr1qG4uBg6nQ67du3CY489hh07dtg9\nrqamJtTX1wMAGhsbcfDgQYwfP97ucQUGBiI0NBT5+fkAgEOHDiEuLg5z5syx+3cfAHbu3Gn+J8XO\n49szrujoaGRmZqK5uRlEhEOHDiE2NlYxn1d/kpzdPcnZ1pGcbR3J2dbpl5zd553hfWj//v0UFRVF\nERERtG7dun499sKFC2nUqFHk6upKISEh9Pe//52qq6tpxowZ3U7H8vbbb1NERARpNBr6+uuvbRbX\n8ePHSaVSkVarNU9jk5GRYffYzp8/T4mJiaTVamn8+PH0pz/9iYjI7nHd6ujRo+YrzO0dV1FREWm1\nWtJqtRQXF2f+fts7LiKic+fOUVJSEk2YMIHmz59PRqNREXE1NDSQr68v1dXVmbcpIa533nnHPHXT\n0qVLyWQyKSIue5CcfSfJ2fdOcnbvSM62jq1ztoqohyYmIYQQQgghRK8pthVECCGEEEKIgUQKayGE\nEEIIIfqAFNZCCCGEEEL0ASmshRBCCCGE6ANSWIsB5aGHHgIAXL16FTt37uzTfa9bt67bYwkhhLh3\nkreFI5FZQcSAdPToUWzatAl79+7t9Xva2trg4uLS4/NeXl7meUqFEEL0LcnbwhHIiLUYUDw9PQEA\na9euxfHjx5GYmIjNmzejo6MDa9asQXJyMrRaLT788EMAnMinT5+OuXPnIj4+HgAwb948JCUlIT4+\nHlu3bjXvr7m5GYmJiViyZInFsYgIa9aswfjx4zFhwgTs3r3bvO+UlBQ8/fTTiImJwXPPPdevn4UQ\nQgwEkreFQ7HN9NtC2IanpycR8cIBaWlp5u0ffPAB/fGPfyQiopaWFkpKSiKdTkdHjhwhDw8P0uv1\n5tfW1NQQEVFTUxPFx8ebH3fu+/Zj/fOf/6RZs2ZRR0cHXb9+nUaPHk3l5eV05MgRGjFiBJWWllJH\nRwdNnTqVvvvuO9udvBBCDECSt4UjkRFrMSDRbR1MBw8exPbt25GYmIgHHngANTU1uHLlCgAgOTkZ\nY8aMMb928+bNSEhIwNSpU1FcXIyCgoK7Huu7777D4sWLoVKp4O/vj0ceeQTff/89VCoVkpOTERQU\nBJVKhYSEBOj1+j4/VyGEGAwkbwtH0HPjkhADzJYtWzBr1iyLbUePHoWHh4fF42+//RaZmZlwc3PD\no48+ipaWlrvuV6VS3fE/BJVKBQAYOnSoeZuzszPa2tru9zSEEMJhSN4Wg42MWIsB6fYLVlJTU/He\ne++ZE2R+fj6amprueF9dXR18fHzg5uaGvLw8ZGZmmp9zdXXtNsFOnz4dX3zxBTo6OlBVVYVjx44h\nOTn5jqQthBCiZ5K3hSOQEWsxoHSOOGi1Wjg7OyMhIQHLli3Dq6++Cr1ej4kTJ4KI4O/vjy+//BIq\nlcr8HgCYPXs2/va3vyE2NhYajQZTp041P/fSSy9hwoQJmDRpEnbs2GF+3/z583Hq1ClotVqoVCps\n3LgR/v7+uHTpksW+b41PCCEEk7wtHIlMtyeEEEIIIUQfkFYQIYQQQggh+oAU1kIIIYQQQvQBKayF\nEEIIIYToA1JYCyGEEEII0QeksBZCCCGEEKIPSGEthBBCCCFEH/h/pEE/cueE8vQAAAAASUVORK5C\nYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x5995c50>"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There are other statitics may also be helpful to check if the solution is good or not, such as the number of cutting planes, from which we may have a sense how tight the piece-wise lower bound is. In general, the number of cutting planes should be much less than the dimension of the parameter vector."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# statistics\n",
      "bmrm_stats = bmrm.get_result()\n",
      "nCP = bmrm_stats.nCP\n",
      "nzA = bmrm_stats.nzA\n",
      "\n",
      "print 'number of cutting planes: %d' % nCP\n",
      "print 'number of active cutting planes: %d' % nzA"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "number of cutting planes: 113\n",
        "number of active cutting planes: 101\n"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In our case, we have 101 active cutting planes, which is much less than 4082, i.e. the number of parameters. We could expect a good model by looking at these statistics. Now come to the online solvers. Unlike the cutting plane algorithms re-optimizes over all the previously added dual variables, an online solver will update the solution based on a single point. This difference results in a faster convergence rate, i.e. less oracle calls, please refer to Table 1 in [4] for more detail. Here, we use the stochastic subgradient descent (<a href=\"http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CStochasticSOSVM.html\">StochasticSOSVM</a>) to compare with the BMRM algorithm shown before."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from modshogun import StochasticSOSVM\n",
      "\n",
      "# the 3rd parameter is do_weighted_averaging, by turning this on, \n",
      "# a possibly faster convergence rate may be achieved.\n",
      "# the 4th parameter controls outputs of verbose training information\n",
      "sgd = StochasticSOSVM(model, labels, True, True)\n",
      "\n",
      "sgd.set_num_iter(100)\n",
      "sgd.set_lambda(0.01)\n",
      "    \n",
      "# train\n",
      "t0 = time.time()\n",
      "sgd.train()\n",
      "t1 = time.time()\n",
      "    \n",
      "w_sgd = sgd.get_w()\n",
      "    \n",
      "print \"SGD took\", t1 - t0, \"seconds.\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "SGD took 444.253559113 seconds.\n"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We compare the SGD and BMRM in terms of the primal objectives versus effective passes. We first plot the training progress (until both algorithms converge) and then zoom in to check the first 100 passes. In order to make a fair comparison, we set the regularization constant to 1e-2 for both algorithms. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(12,4))\n",
      "\n",
      "primal_sgd = sgd.get_helper().get_primal_values()\n",
      "\n",
      "xs = range(dual_bmrm.size-1)\n",
      "axes[0].plot(xs, primal_bmrm[1:], label='BMRM')\n",
      "axes[0].plot(range(99), primal_sgd[1:100], label='SGD')\n",
      "axes[0].set_xlabel('effecitve passes')\n",
      "axes[0].set_ylabel('primal objective')\n",
      "axes[0].set_title('whole training progress')\n",
      "axes[0].legend(loc=1)\n",
      "axes[0].grid(True)\n",
      "\n",
      "axes[1].plot(range(99), primal_bmrm[1:100], label='BMRM')\n",
      "axes[1].plot(range(99), primal_sgd[1:100], label='SGD')\n",
      "axes[1].set_xlabel('effecitve passes')\n",
      "axes[1].set_ylabel('primal objective')\n",
      "axes[1].set_title('first 100 effective passes')\n",
      "axes[1].legend(loc=1)\n",
      "axes[1].grid(True)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAtIAAAEXCAYAAACAp6MwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4FNX6B/DvbBrpDUhICIROgoFAQCCUhMsVa0B6ryri\nvdIUL8UCigVEAbFRBARpKkoVUEBKAEED0nsJvQkJpNfz+2N+u2QhZTdbZpL5fp4nj87u7Jz3LLNv\nTs6+c0YSQggQEREREZFZdEoHQERERERUFnEgTURERERUChxIExERERGVAgfSRERERESlwIE0ERER\nEVEpcCBNRERERFQKHEgTdDodzp8/b/brEhMTodPpkJ+fb4OoShYfH4/69etbfV8iIls7deoUIiMj\n4eXlhc8//xyvvPIK3n//faXDsrlVq1YhJCQEnp6eOHTokNH78MUXX1i1LU9PTyQmJlr1mEQP40Ca\nFDFp0iT079/fomO0adMGJ0+etPq+RES29vHHH6N9+/a4f/8+hg8fjq+//hpvvfVWqY4VGhqK33//\nvcjnc3Jy0K1bN9SoUQM6nQ47dux4ZJ+xY8eiYsWKqFixIsaNG2f0XGJiItq1awd3d3eEhYVh69at\npYoTAMaMGYOvvvoKKSkpaNSokdH78Oqrr5b6uLGxsZg/f77RYykpKQgNDS31MYlMwYE0qZIQAuXx\nXkG5ublWPV5eXp5Vj0dE9nHx4kWEh4ebtG9JeUOSpBLzZdu2bbFkyRIEBgZCkiSj5+bMmYM1a9bg\n8OHDOHz4MNatW4c5c+YYnu/duzeioqJw9+5dfPDBB+jWrRv++ecfk2IvSAiBS5cuGfXbnPehOA/3\nichuBJVLCxYsEHFxcYbt2rVri+7duxu2q1atKg4dOiSEEEKSJDF79mxRp04d4ePjI/773/8a9svP\nzxeTJ08W1atXF5UrVxYDBgwQ9+7dE0IIceHCBSFJksjLyxNCCJGcnCyGDBkiqlSpIoKDg8Vbb71l\neK6gjRs3CmdnZ+Hk5CQ8PDxEZGSkEEKImJgY8eabb4ro6Gjh6uoqzp49KxYsWCDCwsKEp6enqFmz\nppgzZ47hONu2bRNVq1Y1bFevXl188sknomHDhsLb21v07NlTZGZmmr2vEEJMnTrV0I958+YJSZLE\nuXPnCn2vY2JixLhx48Tjjz8uvLy8RKdOncTdu3eN3qP58+eLatWqiZiYmGLfUyGEWLRokahWrZrw\n9/c37Ld161YhhBATJ04UXbt2Ff369RNeXl5i/vz5xb7vZ86cEW3bthXe3t6iYsWKomfPnoZ/11Gj\nRonKlSsLLy8vERERIY4ePVpo/4jIutq1ayccHBxEhQoVhKenpzh9+rQYOHCgeOutt4QQcr4KDg4W\nU6dOFYGBgWLAgAHin3/+Ec8++6zw8fERfn5+ok2bNiI/P1/069dP6HQ64erqKjw8PMS0adOKbbtq\n1apix44dRo+1bNlSzJs3z7C9YMEC0aJFCyGEEKdOnRIuLi4iNTXV8Hzbtm3F7NmzCz1+fn6++Oij\nj0StWrWEv7+/6NGjh7h7967IzMwU7u7uQpIk4e7uLmrVqiX+9a9/Gb0PZ86cEZmZmeL1118X1apV\nEwEBAWLYsGEiIyPDcPzVq1eLRo0aCS8vL1GrVi2xadMmMWHCBMNxPDw8xPDhw4UQwpC39+7dKwID\nA0V+fr7hOD///LNo2LChEEKIvLy8QmMujP7f5sMPPxQVK1YUoaGhYunSpYbn169fLyIjI4WXl5cI\nCQkRkyZNMjyXkZEh+vbtK/z9/YWPj49o1qyZuHnzphBCiIULF4qaNWsKT09PUaNGDaNjzp8/X4SF\nhQlfX1/x5JNPiosXLxqeYx5XHgfS5dT58+eFj4+PEEKIq1eviurVq4uQkBAhhBDnzp0Tvr6+hn0l\nSRJxcXHi3r174tKlS6JSpUpi06ZNQgj5A1y7dm1x4cIFkZqaKrp06SL69+8vhHh0IP3888+LYcOG\nifT0dHHr1i3x+OOPGw18C5o0aZLhOHoxMTGievXq4vjx4yIvL0/k5OSIX375RZw/f14IIcSOHTuE\nm5ubOHDggBDi0cFxaGioaN68ubh+/bq4e/euCAsLMyR7c/bduHGjCAwMFMePHxfp6emib9++QqfT\nFTuQDg4OFseOHRNpaWmGgW7B92jgwIEiPT1dZGRkFPueHjt2THh4eIjdu3eL7OxsMWbMGOHk5GQ0\nkHZychJr1qwRQsiJubj3vVevXuLDDz8UQgiRlZUldu/eLYQQYtOmTSIqKsowgD958qS4fv16of0j\nIuuLjY0V8+fPN2wPGjRIvP3220IIOV85OjqKcePGiezsbJGRkSHGjRsnhg0bJnJzc0Vubq7YtWuX\n4bWhoaGGHFGSwgbS3t7e4s8//zRsJyQkCE9PTyGEPOAMCwsz2n/48OGGwerDZs6cKVq2bCmuXr0q\nsrOzxcsvvyx69+5teP7hSYmH34dRo0aJTp06iaSkJJGSkiLi4uLE+PHjhRBC7Nu3T3h7e4stW7YI\nIeTfbSdPniz0OA+3VatWLbF582bDc926dRNTp041KeaC9P82r7/+usjOzhY7duwQ7u7u4tSpU0II\nIbZv324YzB4+fFgEBASI1atXCyGEmD17toiLixMZGRkiPz9fHDhwQNy/f1+kpqYKLy8vcfr0aSGE\nEDdu3BDHjh0TQsh/ONSuXVucPHlS5OXliffff19ER0cLIZjH1YKlHeVUjRo14Onpib///hs7d+7E\nk08+iaCgIJw6dQo7duxA27ZtjfYfN24cvLy8EBISgnbt2uHQoUMAgKVLl+L1119HaGgo3N3d8dFH\nH2HFihWPXGB48+ZNbNy4ETNmzICrqysqVaqEUaNGYcWKFYXGJwop3ZAkCYMGDUJYWBh0Oh0cHR3x\nzDPPoEaNGgDkryY7dOiA+Pj4Ivs9YsQIBAYGwtfXF3FxcTh48KDZ+/7www8YMmQIwsLC4Orqinff\nfbfYr00lScKAAQMQHh4ONzc3TJ48GT/88IPRayZNmgRXV1dUqFChyPc0Ly8PK1euRMeOHREdHQ0n\nJye89957j3xlGR0djY4dOwIA7t27V+z77uzsjMTERFy9ehXOzs6Ijo42PJ6SkoITJ04gPz8f9erV\nQ2BgYJF9JCLrezivFNzW6XR499134eTkhAoVKsDZ2RnXr19HYmIiHBwc0KpVK6vFkZqaCm9vb8O2\nl5cXUlNTC31O/3xKSkqhx5ozZw7ef/99BAUFwcnJCRMnTsTKlSuLvShd328hBObNm4fp06fDx8cH\nHh4eGD9+vCGfzZ8/Hy+88ALat28PAAgKCkK9evUeOU5hevfujeXLlwOQa6c3btyI3r17lzrmyZMn\nw8nJCW3btsWzzz6LH374AQAQExODBg0aAAAiIiLQq1cvQ026s7Mz7ty5gzNnzkCSJDRu3Bienp4A\n5H/vI0eOICMjAwEBAYZyl9mzZ2P8+PGoV68edDodxo8fj4MHD+LSpUvM4yrBgXQ5FhMTg+3btyM+\nPh4xMTGIiYnBjh07sHPnTsTExBjtW/DD5+bmZkii169fR/Xq1Q3PVatWDbm5ubh586bR6y9evIic\nnBxUqVIFvr6+8PX1xbBhw3D79m2zYg4JCTHa3rhxI1q0aAF/f3/4+vpiw4YNuHPnTpGvL9gPV1dX\nQz9M2TctLQ2A3OeCcVStWtWsuKtVq4acnByjGsKCzxf3nl6/ft2oPVdXV/j7+xu1VfD5kt73jz/+\nGEIIPP7443jsscewcOFCAEC7du3w6quv4r///S8CAgLw8ssvF/mLkYhso7i63kqVKsHZ2dmw/cYb\nb6B27dro0KEDatWqhalTp1otDg8PD9y/f9+wfe/ePXh4eBT6HAAkJyfDy8ur0GMlJiaic+fOhnwU\nHh4OR0fHR35nFKR/H27fvo309HRERUUZXv/0008bcumVK1dQq1atEo9TmN69e+Pnn39GdnY2fv75\nZ0RFRRnysrkx+/r6wtXV1bBdvXp1XLt2DQCwb98+tGvXDpUrV4aPjw/mzJlj+J3Vv39/PPnkk+jV\nqxeCg4MxduxY5Obmwt3dHd9//z1mz56NoKAgPPfcczh16hQAOcePHDnSEJv+98G1a9eYx1WCA+ly\nLCYmBtu2bUN8fDxiY2MNA+sdO3Y8MpAuSlBQkNHyQZcuXYKjoyMCAgKM9gsJCYGLiwvu3LmDpKQk\nJCUl4d69ezhy5Eihx9XpCj/1CibCrKwsdO3aFf/73/9w69YtJCUl4ZlnnrH5RYhVqlTB5cuXDdsF\n/78oly5dMvp/JycnVKxY0fBYwX4V9Z4GBgaiSpUquHLliuG5jIyMR/5wKHiskt73gIAAzJ07F1ev\nXsWcOXPwn//8x7DU4fDhw5GQkIDjx4/j9OnTmDZtWon9JCLbKfjZfnhQ6OHhgU8++QTnzp3D2rVr\nMX36dGzbtq3Qfc3VoEEDo2/vDh06hMcee8zw3Pnz540mJQ4dOmSYdX1YtWrVsGnTJkM+SkpKQnp6\nOqpUqVJiHBUrVoSrqyuOHz9ueG1ycrJhIB8SEoKzZ88W+tqS3oPw8HBUr14dGzduxLJly9CnT59S\nx6x/Xu/ixYsIDg4GAPTp0wfPP/88rly5guTkZAwbNswws+3o6Ih33nkHx44dw549e7B+/XosXrwY\nANChQwf89ttvuHHjBurXr4+XXnrJENvcuXONYktLS0OLFi0AMI+rAQfS5Zh+IJ2ZmYmgoCC0bt0a\nmzZtwt27d9G4ceMiX1ew7KJ3796YMWMGEhMTkZqaigkTJqBXr16PDISrVKmCDh064LXXXkNKSgry\n8/Nx7tw57Ny5s9A2AgICkJiYWOxXm9nZ2cjOzkbFihWh0+mwceNG/Pbbb6V9O0qkb7tHjx5YuHAh\nTp48ifT0dEyePLnE1y1ZsgQnTpxAeno63nnnHXTv3r3IxF7ce9q1a1esW7cOf/zxB7KzszFp0qRi\n/3Ao6X3/8ccfDQNzHx8fSJIEnU6HhIQE7Nu3Dzk5OXBzc0OFChXg4OBQmreNiEqp4Ge7sHK3gn75\n5RecPXsWQgh4eXnBwcHBkIcDAgJw7ty5YtvKyspCZmbmI/8PAAMGDMD06dNx7do1XL16FdOnT8eg\nQYMAAHXr1kVkZCTeffddZGZm4ueff8bRo0fRtWvXQtsZNmwYJkyYYJhcuH37NtauXWvS+6DT6fDS\nSy9h1KhRhm/Vrl69asj7L7zwAhYuXIjff/8d+fn5uHr1qmHm1pT3oE+fPpg5cybi4+PRvXt3i2Ke\nOHEicnJyEB8fj19++cVwvNTUVPj6+sLZ2Rl//vknli1bZvhdsH37dhw5cgR5eXnw9PSEk5MTHBwc\ncOvWLaxZswZpaWlwcnKCu7u7IR8PGzYMH374IY4fPw5A/rbgxx9/BADmcZXgQLocq1OnDjw9PdGm\nTRsAcl1brVq10KpVq2JnPiRJMjw2ZMgQ9O/fH23btkXNmjXh5uaGzz//vNDXLl68GNnZ2QgPD4ef\nnx+6d++OGzduFBqbPun4+/ujadOmhR7P09MTs2bNQo8ePeDn54fly5ejU6dOj8RalIL9MGffp556\nCiNGjEC7du1Qt25dtGzZEgDg4uJS5Gv79++PQYMGoUqVKsjOzsasWbOKbLe497RBgwb4/PPP0atX\nLwQFBcHT0xOVK1c2tP1wn4Di3/eEhAS0aNECnp6e6NSpE2bNmoXQ0FDcv38fQ4cOhZ+fH0JDQ1Gx\nYkW88cYbRb4/RGR9D+en4vLVmTNn8MQTT8DT0xPR0dH473//a/hmcfz48Xj//ffh6+uL6dOnF9pW\nvXr14ObmhmvXruHJJ5+Eu7u7YeD48ssvIy4uDhEREWjYsCHi4uIwdOhQw2tXrFiBhIQE+Pn54c03\n38RPP/30SMmZ3siRI9GxY0d06NABXl5eaNmyJf78888i+/XwY1OnTkXt2rXRokULeHt744knnsDp\n06cBAM2aNcPChQsxevRo+Pj4IDY21tCHkSNHYuXKlfDz88OoUaMKja13797YuXMn2rdvDz8/P5Nj\nfpj+2pqgoCD0798fc+bMQd26dQEAX331Fd555x14eXlh8uTJ6Nmzp+F1N27cQPfu3eHt7Y3w8HDE\nxsaif//+yM/Px4wZMxAcHAx/f3/Ex8fj66+/BgA8//zzGDt2LHr16gVvb29ERETg119/BQDmcZWQ\nhK2/Jycq406cOIGIiAhkZ2cXWpLSrl079O/fH0OGDLF62/rZjbNnzxrVVRMRkf1t374d/fv3N6nk\nj7TBZjPSQ4YMQUBAACIiIgyP3b17F0888QTq1q2LDh06IDk52VbNE1lk1apVyMrKQlJSEsaOHYuO\nHTsWWdcNFH+1uLnWrVuH9PR0pKWlYcyYMWjYsCEH0WRzzNlEROaz2UB68ODB2LRpk9FjU6ZMMXxN\n0759e0yZMsVWzRNZZO7cuQgICEDt2rXh5ORk+JqtKNa8q9batWsRHByM4OBgnDt3rsglBImsiTmb\nyDS8iyIVZNPSjsTERMTFxRlWEKhfvz527NiBgIAA3LhxA7GxsTh58qStmiciIjMwZxMRmcfRno3d\nvHnTsGxaQEBAoWs08i89IirLytNlJ8zZRKQFluRtxVbtKGz1AT39MkBq+pk4caLiMTAuxsW41B1X\neVbWcrYWzz/2mX1mn83/sZRdB9L6rwcB+e5ulStXtmfzFil4Aw01YVzmYVzmYVzaVpZzti1p8fxj\nn7VBi322lF0H0h07dsSiRYsAAIsWLcLzzz9vz+aJiMgMzNlERMWz2UC6d+/eiI6OxqlTpxASEoKF\nCxdi3Lhx2Lx5M+rWrYvff/8d48aNs1XzVqe/y5PaMC7zMC7zMC7tKG8525a0eP6xz9qgxT5bSnU3\nZJEkySo1K0RE9qbF/KXFPhNR+WFpDuMtwk20fft2pUMoFOMyD+My5ufnZ7iIjD+m/xS8vTBpj1rz\niC2xz+rBvK2uvG3X5e+ISF2SkpI4m1gKksQl34hIGczbpWOrvM3SDiIN4+etdIp637T4fmqxz0RK\n4meudGyVt1naQURERERUChxIm0ittVKMyzyMi4gspcXPK/tMVDgOpImIiIiISoE10kQaxs9b6bBG\n+gEt9plISfzMlQ5rpIlIU0JDQ+Hm5gZPT0/4+fnhueeew5UrVwDINw3Q6XRYu3at0WtGjx4NnU5n\nuBvft99+CwcHB3h6esLb2xsNGzbEqlWrDPsnJiZCp9OhSZMmRsf5559/4OzsjBo1ati4l0RE5YcW\n8zYH0iZSa60U4zIP4yo7JEnC+vXrkZKSguvXryMgIADDhw83PF+3bl0sXrzYsJ2bm4sffvgBtWvX\nNlrmqFWrVkhJSUFycjJeffVV9OnTB0lJSUZtZWRk4NixY4btZcuWoWbNmlzmjgqlxc8r+0ym0GLe\n5kCaiFTPxcUFXbt2xfHjxwHIyTouLg67du1CcnIyAGDTpk1o1KgRAgICjF6r/8pOkiT069cPWVlZ\nOHfunNE+/fv3N8yGAMB3332HAQMG8OtTIqJS0kre5kDaRLGxsUqHUCjGZR7GZT5Jss5PaegTYnp6\nOr7//nu0bNnS8FyFChXQqVMnrFixAgCwePFiDBgwoMhj5eXlYeHChfDx8UG9evWMnuvbty9WrFgB\nIQSOHz+O1NRUNG/evHRBU7mn5s+rrbDPZYeSORvQXt7mnQ2JqFhKTcoKIfD888/D0dERaWlpqFy5\nMjZt2mS0z4ABA/DGG2+gd+/e2LlzJxYvXowvv/zSaJ+9e/fC19cXaWlpcHR0xObNm+Hp6Wm0T9Wq\nVVGvXj1s3rwZv//+e7GJnYhIzZT8Ik2LeZsz0iZSa60U4zIP4yo7JEnCmjVrkJSUhKysLHz++eeI\niYnBzZs3Dc+3atUKt2/fxvvvv4+4uDhUqFDhkeO0aNECSUlJSEpKQseOHTF16tRC2xowYAAWLlyI\nFStWoH///izroCJp8fPKPpMptJi3VTmQfv2315UOgYhURJIkdO7cGQ4ODti1a5fRc/369cP06dNL\nnI1wd3fH119/jR07dmDHjh2PPN+lSxds2LABtWrVQtWqVa0af3nXcn7LknciIk3RSt5W5UA6MzdT\n6RAeodZaKcZlHsZVtuhnF4QQWLNmDZKTkxEeHg4hhOG5ESNGYMuWLWjTpk2Jx/P19cXQoUMxZcqU\nR55zd3fHtm3b8M0331i3ExpwN+Ou0iHYlRY/r+wzmUpreVuVNdL8SpWIACAuLg4ODg6QJAmhoaFY\ntGgRwsLCIEmSYYkjX19ftGvXrtDXF9xPb9SoUahVqxYOHz4MLy8vo+cfXpeUy9+ZJiMnQ+kQiEgl\ntJa3VXlnw2Hrh+HrZ79WOhQj27dvV+Vfp4zLPIzLGO+QVTq8s+EDkiTBf6o//vnfP0qHYjdqzSO2\nxD6rhxbzjDVo6s6GPEGIiMqOjFzOSBORNqlyRnrouqGY89wcpUMhKvc4s1E6nJF+QJIkSJMk5L2T\nx1IYIjvQYp6xBk3NSOeLfKVDICIiEzk5OCErL0vpMIiI7E6VA2k1/qWl1vUkGZd5GBeR9bk6umrq\ngkMtfl7ZZ6LCqXMgDfUNpImIqHBuTm5Iz0lXOgwiIrtTZY304NWDsaDTAqVDISr3WGtXOqyRfkCS\nJNT8rCZ+7fcravvVVjoconJPi3nGGjRVI80ZaSKisoMz0kSkVeocSKvwLy211koxLvMwLiLrY410\n+cc+ExVOlQNprtpBRFR2cEaaiLRKlQNpNZZ2qPHuRgDjMhfjKlt27dqF6Oho+Pj4wN/fH61bt0ZC\nQgIA4Pr163jppZcQHBwMT09P1KpVC4MHD8apU6cAAImJidDpdPD09ISnpycCAwMRFxeHLVu2KNml\ncsnVyVVTN2XR4ueVfSZTaS1vq3MgrcLSDiKyr/v37+O5557DyJEjkZSUhKtXr2LixIlwcXHBnTt3\nEB0djczMTOzatQspKSk4cOAAYmJisHnzZqPj3Lt3DykpKTh8+DCeeOIJdO7cGYsWLVKoV+WT1ko7\niKhwmszbQmUAiD4/9VE6jEds27ZN6RAKxbjMw7iMqTAFGPz111/Cx8en0OfefPNNERkZWezrL1y4\nICRJEnl5eUaPf/LJJyIgIMCi2Ip639T8ftoKANH3p75i8cHFSodiN2rNI7bEPquHmvOMFvO2o3JD\n+KIJzkgTqYb0rnVu+ywmmve5rlevHhwcHDBo0CD06tULzZs3h6+vLwBgy5Yt6Ny5c6ni6Ny5M954\n4w2cOnUK9erVK9UxyJjWSjuI1EypnA1oM2+rciCtxosN1VorxbjMw7jMV5pkag2enp7YtWsXpk6d\nipdeegk3btzAM888g7lz5+LOnTsIDAw07Lt27VoMHDgQeXl5aNmyJX799dcijxsUFAQAuHv3rs37\noBVau9hQzZ9XW2Gfyw6lcjagzbytzhppFV5sSET2V79+fSxcuBCXL1/G0aNHce3aNYwePRr+/v64\ndu2aYb+OHTsiKSkJM2bMQHZ2drHHvHr1KgDAz8/PprFrCWukiUhPa3lbnQNpFZZ2qHU9ScZlHsZV\ndtWrVw8DBw7E0aNH0b59e6xevfqRXGFK7li1ahUCAgJU9/VgWebm5Ib0XO3MSGvx88o+U2loIW+r\ncyDNGWkizTt16hSmT59umIm4fPkyli9fjpYtW+K1115DUlIS+vfvj/Pnz0MIgZSUFBw8eBCSZFwf\nqE/SN2/exBdffIH33nsPH330kd37U55xRpqIAG3mbXUOpFU4I63WWinGZR7GVXZ4enpi3759aN68\nOTw8PNCyZUs0bNgQn376Kfz9/bF3715UqFABrVu3hpeXFxo3boy0tDR8/fXXRsfx8fGBh4cHGjZs\niE2bNmHlypUYNGiQMp0qp9yc3DR1saEWP6/sM5lCi3lbEgqMWj/66CMsWbIEOp0OERERWLhwIVxc\nXOSAJAldvu+Cn3r8ZO+wiDRHkiRV/uGqdkW9b+X1/SwpZ88/MB/xl+KxsNNChSMlKv/Ka56xNVvl\nbbvPSCcmJmLevHk4cOAAjhw5gry8PKxYscJoHzWu2qHWWinGZR7GRWQeU3K21ko7tPh5ZZ+JCmf3\n5e+8vLzg5OSE9PR0ODg4ID09HcHBwUb78C8tIiJ1MCVna235OyIiPbsPpP38/PD666+jWrVqcHV1\nxZNPPol///vfRvsc+PoAJp2cBECuk4mMjDTUKun/QuS2vK1/TC3xqH1b/5ha4lF628vL65GLPKhk\n+hsMzJw5EwcPHkRoaKiyAdmQKTn7yze/xDlxDpNOTdJMztZTSzzctv52bGysquLRbzNvl46vry+2\nb9+OgwcPIjk5GYD8jZul7F4jfe7cOcTFxSE+Ph7e3t7o3r07unXrhr59+8oBSRLilsVhbe+19gyL\niMhi5bF20ZScHX8xHmO3jMXuIbsVjpaIyDxlrkY6ISEB0dHR8Pf3h6OjI7p06YI9e/YY7aPG5e8e\nnoVQC8ZlHsZlHsZFpuRs1kiXf+yzNmixz5ay+0C6fv362Lt3LzIyMiCEwJYtWxAeHm60T3mb0SEi\nKqtMydmskSYirVJk+buPP/4YixYtgk6nQ5MmTfDNN9/AyclJDkiS8PSSp7Gh7wZ7h0VEZJHyWNoB\nlJyzLyRdQMy3Mbg46qLCkRIRmcfSvK3IQLo4kiThqSVPYWPfjUqHQkRklvI6kC6OJEm4kXIDEV9H\n4NYbt5QOh4jILGWuRtoUavxFpNa6IcZlHsZlHsZFptBaaYcWzz/2WRu02GdLqXMgrcKLDYmIqHCu\nTq7IyM1Q5SQIEZEtqbK044nFT+C3/r8pHQoRkVm0WtohhIDzZGekjE+Bi6OL0iEREZmsXJZ2qPEW\n4UREVDT9rDQRkZaociCtxtIOtdYNMS7zMC7zMC4ylZbqpLV4/rHP2qDFPltKnQNpjX01SkRU1mnt\npixERIBKa6Rjv43FtoHblA6FiMgsWq6Rfuyrx7C863JEBEQoHRIRkcnKZY201n4RERGVdayRJiIt\nUudAmjXSJmNc5mFc5mFcZCotlXZo8fxjn7VBi322lCoH0ly1g4iobNHSxYZERHqqrJFuNb8Vdg3Z\npXQoRERX+zdNAAAgAElEQVRm0XKNdOfvO6N/w/7oEtZF6ZCIiExWPmukVVjaQUREReOMNBFpkToH\n0iqc0VFr3RDjMg/jMg/jIlOxRrp8Y5+1QYt9tpQ6B9KckSYiKlM4I01EWqTKGunH5z2OfS/uUzoU\nIiKzaLlGeuyWsfCt4ItxrccpHRIRkcnKZ420xn4RERGVdW6OnJEmIu1R50BahaUdaq0bYlzmYVzm\nYVxkKi3dkEWL5x/7rA1a7LOl1DmQ5ow0EVGZoqWLDYmI9FRZI91kThPsH7pf6VCIiMyi5Rrp+Qfm\nY/fl3VjQaYHSIRERmYw10kREpDgtlXYQEempciCtxluEq7VuiHGZh3GZh3GRqbS0/J0Wzz/2WRu0\n2GdLqXIgrcaLDYmIqGiskSYiLVJljXTEVxE4/MphpUMhIjKLlmuk4y/GY/zW8dg1ZJfSIRERmczm\nNdKnTp1C+/bt0aBBAwDA4cOH8f7775e6QVNwRpqIqHSUyNkAa6SJSJtKHEi/9NJL+PDDD+Hs7AwA\niIiIwPLly20alBpndNRaN8S4zMO4zMO4yh4lcjYgl3awRrr8Yp+1QYt9tlSJA+n09HQ0b97csC1J\nEpycnGwalBovNiQiKguUyNmAfLEha6SJSGtKHEhXqlQJZ8+eNWyvXLkSVapUsWlQaiztiI2NVTqE\nQjEu8zAu8zCuskeJnA1oq7RDi+cf+6wNWuyzpRxL2uGLL77A0KFDcfLkSQQFBaFGjRpYunSpTYNS\nY2kHEVFZoETOBrS1/B0RkV6JM9KhoaHYunUrbt++jZMnT2L37t0IDQ21aVBqnJFWa90Q4zIP4zIP\n4yp7lMjZwIPl77QwEaLF84991gYt9tlSJQ6ka9SogaFDh2Lfvn3w9PS0R0yaSMRERLagRM4GACcH\nJ0iShJz8HLu1SUSktBLXkU5LS8P69euxYsUKHDhwAHFxcejZsyfatGljm4AkCbVn1caZ4Wdscnwi\nIltRwzrSSuRsfZ+9PvLC5dGX4V3B2yZtERFZm83XkXZ3d0fPnj2xatUqHDx4EPfu3bN5MTpX7SAi\nKh0lcrYe66SJSGtMukX49u3b8corr6BJkybIysrCDz/8YNOglJ7RKYxa64YYl3kYl3kYV9lk75yt\np5WVO7R4/rHP2qDFPluqxFU7QkNDERkZiZ49e2LatGnw8PCweVBqvNiQiKgsUCJn62nppixERIAJ\nNdL379+Hl5eXveKBJEmoPqM6Ekcl2q1NIiJrUEONtBI5W9/npnOb4utnv0az4GZ2a5+IyBKW5u0i\nZ6SnTp2KsWPH4s033yy00VmzZpW60ZJwRpqIyDxK5mw9rZR2EBHpFVkjHR4eDgCIiopC06ZNDT9R\nUVGIioqyqNHk5GR069YNYWFhCA8Px969e42eV+PFhmqtG2Jc5mFc5mFcZYeSOVtPKxcbavH8Y5+1\nQYt9tlSRM9JxcXEAADc3N/To0cPoOUsvXBk5ciSeeeYZrFy5Erm5uUhLSzN6XumvRomIyholc7ae\n/qYsRERaUWKNdOPGjfH333+X+Jip7t27h8aNG+P8+fOFByRJCPo0CFdfu1qq4xMRKUUNNdJK5Gx9\nn/v81AfP1nkWfRv2LVVbRET2ZrMa6Y0bN2LDhg24evUqRowYYWgkJSUFTk5OpW7wwoULqFSpEgYP\nHoxDhw4hKioKn332Gdzc3Az7JC1PwqT7kwAAPj4+iIyMNKyDqv/agdvc5ja3ld6eOXMmDh48aJdb\ncJdEyZw9aNAghIaG4uSpk8gPykdwXLBq/o24zW1uc7vg9sGDB5GcnAwASExMhKWKnJE+dOgQ/v77\nb7zzzjuYPHkyhBCQJAmenp5o164dfH19S9VgQkICWrZsiT179qBZs2YYNWoUvLy88N5778kBSRIC\nPwnE9devl75XNrB9+3bDP4SaMC7zMC7zMC7zKDkjrWTO1vf51Q2voq5/XYxoPsJq/VIjtZ5/tsQ+\na4MW+2yzGelGjRqhUaNG6NKlC9zd3eHg4AAAyMvLQ1ZWVqkbrFq1KqpWrYpmzeTlkbp164YpU6YY\n7aP0V6NERGWNkjlbz83JjTXSRKQpupJ26NChAzIyHiTG9PR0/Pvf/y51g4GBgQgJCcHp06cBAFu2\nbEGDBg2M9lHjqh1q/QuNcZmHcZmHcZU9SuRsPa0sf6fF84991gYt9tlSJd7ZMDMz0+jOWJ6enkhP\nt2x5o88//xx9+/ZFdnY2atWqhYULFxo9z3WkiYhKR4mcrefm6IY7GXcsaouIqCwpcUba3d0d+/fv\nN2wnJCTA1dXVokYbNWqEv/76C4cOHcLPP/8Mb29vo+fVWNqhL1hXG8ZlHsZlHsZV9iiRs/W0MiOt\nxfOPfdYGLfbZUiXOSM+cORPdu3dHUFAQAOD69ev4/vvvbRoUZ6SJiEpHiZytp5UbshAR6ZW4jjQA\n5OTk4NSpUxBCoH79+hYtpVRiQJIE3ym+uDv2rs3aICKyBTWsIw3YP2fr+7z08FL8cuYXLOu6zGbt\nERFZk6V5u8TSjrS0NEyZMgWfffYZIiIikJiYiPXr15e6QVNwRpqIqHSUyNl6nJEmIq0pcSA9ePBg\nODs7Y8+ePQCAoKAgvPnmmzYNSo2rdqi1bohxmYdxmYdxlT1K5Gw91kiXX+yzNmixz5YqcSB97tw5\njB07Fs7OzgDkC1lsTQ1fjRIRlUVK5Gw9V0dXzkgTkaaUOJB2cXExWpP03LlzcHFxsWlQaiztUOva\niozLPIzLPIyr7FEiZ+tp5YYsWjz/2Gdt0GKfLVXiqh2TJk3CU089hStXrqBPnz7YvXs3vv32W5sG\nxRlpIqLSUSJn62mltIOISM+kOxv+9NNPWLhwIfr06YP9+/ejXbt2Ng2KNdKmY1zmYVzmYVxljxI5\nW8/bxRvJmcl2aUtJWjz/2Gdt0GKfLVXkjPSJEycQFhaG/fv3Q5IkVKlSBQBw6dIlXL58GX5+fqhe\nvbpNgsoTeTY5LhFReaVkztar5F4Jt9NuQwgBSZJs2hYRkRoUuY70Sy+9hHnz5iE2NrbQhHjnzh00\nbNgQS5YssW5AkgTH9xyR83aOVY9LRGRrSq4jrWTOLthn7yneuDjqInwq+Fi1HSIiW7A0bxc5Iz1v\n3jwAxU/zd+jQodQNFycvnzPSRETmUDJnF1TJrRJupd3iQJqINKHEGumMjAx8+umn6Ny5M7p06YIZ\nM2YgMzMTAPDbb7/ZJCgBoboLDtVaN8S4zMO4zMO4yh4lcnZBld0r41baLZu3oyQtnn/sszZosc+W\nKnHVjgEDBsDLywsjRoyAEALLli1D//798eOPP9osKJ2kQ57Ig6NUYnhERFSAEjm7IC0MpImI9Iqs\nkdYLDw/H8ePHS3zMagFJEpzec0LqhFQ4OzjbpA0iIltQskZaT4mcXbDPQ9cNRVSVKLzc9GWbtEdE\nZE2W5u0SSzuaNGmCP/74w7C9d+9eREVFlbpBk4KSdKyTJiIqBSVydkGckSYiLSlyIB0REYGIiAjs\n378frVq1QvXq1REaGoro6GgkJCTYNCgHnYPqlsBTa90Q4zIP4zIP4yo7lMzZBVV2r4xb6eV7IK3F\n84991gYt9tlSRRYhr1u3zmhbv5ySPb62dJAcOCNNRGQGJXN2QZXdK2P35d12bZOISCkl1kgDwMGD\nBxEfHw9JktCmTRs0atTIdgFJEnym+OD8iPPwdfW1WTtERNamhhppwP45u2Cff7/wOybvnIxtA7fZ\nrE0iImuxeY30Z599hn79+uH27du4efMm+vXrh1mzZpW6QZOC+v9VO4iIyDxK5OyCWCNNRFpS4kD6\nm2++wb59+/Dee+9h8uTJ2Lt3r2Hhf1tRY2mHWuuGGJd5GJd5GFfZo0TOLkgLA2ktnn/sszZosc+W\nKnEgDQA6na7Q/7cVB50D8kW+zdshIiqP7J2zC/J39UdyZjJy83Pt2i4RkRJKrJGePn06vv32W3Tp\n0gVCCKxevRqDBg3C6NGjbROQJCHo0yDse3EfqnpVtUkbRES2oIYaaSVy9sN9rjytMo68cgQBHgE2\naZOIyFoszdsmXWy4f/9+7Nq1y3DhSuPGjUvdYIkBSRJCpocgfnA8qvtUt1k7RETWpoaBNGD/nP1w\nnxt81QAruq5ARECEzdolIrIGm19sCABRUVEYOXIkRowYYdOErMd1pE3HuMzDuMzDuMome+fsh1V2\nr4zb6bft3q69aPH8Y5+1QYt9tpR9i+dM5CCxRpqIqKzSwgWHRESAiaUd9iRJEurMqoN1vdehXsV6\nSodDRGQytZR22FNhfR6+cTjq+NXBiOYjFIqKiMg0dintsDc1lnYQEZFpKrtxRpqItKHIgbSHhwc8\nPT0L/fHy8rJpUFxH2nSMyzyMyzyMq+xQMmc/rLyXdmjx/GOftUGLfbaUY1FPpKam2jMOI1xHmojI\nPErm7IeV94E0EZGeyTXSt27dQmZmpmG7WrVqtglIkhA5OxLzO85HkypNbNIGEZEtqKlG2p45++E+\n7760G29sfgN7XthjkzaJiKzF5jXSa9euRZ06dVCjRg3ExMQgNDQUTz/9dKkbNIUaSzuIiMoCJXL2\nwzgjTURaUeJA+q233sIff/yBunXr4sKFC9i6dSuaN29u06DUWNqh1rohxmUexmUexlX2KJGzH1be\nB9JaPP/YZ23QYp8tVeJA2snJCRUrVkR+fj7y8vLQrl07JCQk2DYoScdVO4iISkGJnP0wLxcvZOVl\nITM3s+SdiYjKsCIvNtTz9fVFSkoK2rRpg759+6Jy5crw8PCwaVBqLO2IjY1VOoRCMS7zMC7zMK6y\nR4mc/TBJkuS7G6bdRoh3iF3btgctnn/sszZosc+WKnFGevXq1XBzc8OMGTPw1FNPoXbt2li3bp1N\ng+I60kREpaNEzi5MJbdK5bq8g4gIMGEg7eHhAQcHB2RkZCAuLg59+/aFJEk2DUqNtwhXa90Q4zIP\n4zIP4yp7lMjZhSnPddJaPP/YZ23QYp8tVWJpx5w5czBx4kS4uLhAp5PH3ZIk4fz586VuNC8vD02b\nNkXVqlULnSnRSTrVlXYQEZUFtsjZQMl5+2HleSBNRKRX4jrStWvXxt69e1GxYkWrNTp9+nTs378f\nKSkpWLt2rXFAkoQO33XA6Baj8VTtp6zWJhGRralhHWlb5Gyg6LxdVJ/H/DYGAe4BeKPVG1aNg4jI\nmmy+jnTNmjXh6upa6gYeduXKFWzYsAEvvvhikYGr8WJDIqKywNo5GzAtbz+ssntl3ErnjDQRlW8l\nlnZMmTIFLVu2RMuWLeHs7AxAHr3PmjWrVA2OHj0a06ZNw/3794vc58jcI1i6dyn+8v8LPj4+iIyM\nNFxJqq/fsfe2/jGl2i9qe+bMmap4f/h+Wbatf0wt8fD9Mm175syZOHjwIEJDQ6EW1s7ZQMl5e9Cg\nQYb3QJ+zK/tUxrHbxxT/N7LF9sGDBzFq1CjVxGOPbf1jaonHHtsP913peOyxrdacb+3Pb3JyMgAg\nMTERliqxtKNp06Zo27YtIiIioNPpIISAJEkYOHCg2Y2tX78eGzduxJdffont27fj008/faTWTpIk\ndFzeEYMjB+P5+s+b3YatbN++3fAPoSaMyzyMyzyMyzxqKO2wZs4GSs7bRfV5w5kN+PzPz7Gx78ZS\n90Wt1Hr+2RL7rA1a7LOlebvEgXTjxo3x999/l7qBgiZMmIDvvvsOjo6OyMzMxP3799G1a1csXrz4\nQUCShM4rOqNvRF90De9qlXaJiOxBDQNpa+ZsoOS8XVSfE64lYNj6YUgYat+bwRARmcPmA+kJEyag\nevXq6NixI1xcXAyP+/n5lbpRANixYwc++eSTQmeku/3QDd3Du6NHgx4WtUFEZE9qGEjbKmcDheft\novp86d4lRM+PxpXXrljcLhGRrdj8YsNly5ZhypQpiI6ORlRUlOHHGopa29RR54jc/FyrtGEtBWul\n1IRxmYdxmYdxlT22zNlA0Xn7YUGeQfgn/R9k5GRYrW210OL5xz5rgxb7bKkSLza0RiF2YWJiYhAT\nE1Poc046J9UNpImIygJb5Wyg+Lz9MEedI2r41sC5pHN4rPJjNouJiEhJRZZ2bN26Fe3bt8dPP/1U\n6AxEly5dbBOQJGHw6sFoFdIKLzR5wSZtEBHZgpKlHUrm7KL6rL9wvHNYZ5u0TURkKUvzdpEz0jt3\n7kT79u2xbt06uyZlAHBycEJOfo7Njk9EVN4ombOLUse/Ds7cPWP3domI7KXIgfS7776L/Px8PP30\n0+jZs6c9Y1JlaYdal4RhXOZhXOZhXGWHkjm7KHX96uKva38pHYbVafH8Y5+1QYt9tlSxFxvqdDp8\n/PHH9orFwFHniJw8zkgTEZlDqZxdFM5IE1F5V+Lyd+PGjUPFihXRs2dPuLu7Gx63xlJKhQYkSRjz\n2xhUcquE/7X6n03aICKyBTUsf6dEzi6qz5fvXUbzb5rj2uvXbNI2EZGlbL6OdGho6CP1dpIk4fz5\n86VutNiAJAnjt4yHh7MHJrSZYJM2iIhsQQ0DaSVydlF9zhf58PjQAzfH3ISni6dN2icisoTN15FO\nTEzEhQsXjH5slZD11Fjaoda1FRmXeRiXeRhX2aNEzi6KTtKhtl9tnL17VpH2bUWL5x/7rA1a7LOl\nSlxHOiMjA1999RV27doFSZLQpk0bvPLKK6hQoYLNgnLSOSErL8tmxyciKq+UyNnF0ddJN67SWJH2\niYhsqcTSju7du8PLywv9+vWDEALLli3DvXv38OOPP9omIEnCR/EfITkzGVP+PcUmbRAR2YIaSjuU\nyNnF9Xn81vFwd3LHW23fskn7RESWsNk60nrHjh3D8ePHDdv/+te/EB4eXuoGTQpK58h1pImISkGJ\nnF2cOn51sOPiDsXaJyKypRJrpJs0aYI//vjDsL13715ERUXZNCgnnRNrpE3EuMzDuMzDuMoeJXJ2\ncer41cGZO+VrCTwtnn/sszZosc+WKnFGOiEhAa1atUJISAgkScKlS5dQr149REREQJIkHD582PpB\n6RxVd0MWIqKyQImcXZw6/nVw+s5pu7ZJRGQvJdZIJyYmFnuA0NBQK4Yj16rMSZiDv679hXlx86x6\nbCIiW1JDjbQSObu4Pgsh4D3FGxdHXYSvq69V2yYispTNa6StnXRNocbSDiKiskCJnF0cSZIMK3c8\nHvy40uEQEVlViTXSSlBjaYda64YYl3kYl3kYF1lDHb/yVd6hxfOPfdYGLfbZUqocSDs5OHHVDiKi\ncqKuf12cuVu+LjgkIgJMqJG2N0mSsPLYSiw7ugw/9fhJ6XCIiEymhhppezOlz98d+g4bz27Esq7L\n7BQVEZFpbH6LcCWo8RbhRERUOly5g4jKK1UOpNVY2qHWuiHGZR7GZR7GRdZQz78eTt85jbz8PKVD\nsQotnn/sszZosc+WUuVA2sXBBVm5WUqHQUREVuDr6otAj0Acu31M6VCIiKxKlTXSuy7uwhub38Ce\nF/YoHQ4RkclYI120wWsG4/Ggx/FKs1fsEBURkWnKZY20i6MLsvI4I01EVF60DmmN3Zd3Kx0GEZFV\nqXIgXcGxgupKO9RaN8S4zMO4zMO4yFpaVWuFXZd2KR2GVWjx/GOftUGLfbaUKgfSLg4uyMzNVDoM\nIiKyknr+9ZCanYqr968qHQoRkdWoskb6UvIlRC+IxuXRl5UOh4jIZKyRLl7H5R3Rr2E/9GjQw8ZR\nERGZptzWSHNGmoiofGldjXXSRFS+qHMgrcLl79RaN8S4zMO4zMO4yJpahbTC7ktlfyCtxfOPfdYG\nLfbZUqocSFdwrMAZaSKiciYqKAon/jmB1OxUpUMhIrIKVdZI5+fnQ/eeDnnv5EEnqXKsT0T0CNZI\nl6zVglZ4L/Y9tK/Z3oZRERGZplzWSEuSpMryDiIisgzrpImoPFHlQBpQ301Z1Fo3xLjMw7jMw7jI\n2lqFtCrzA2ktnn/sszZosc+WUu1AmnXSRETlT3RINPZe2YvsvGylQyEispgqa6SFEKjxWQ1sHbAV\nNX1rKh0SEZFJWCNtmlYLWmFC6wl4tu6zNoqKiMg05bJGGgDcndyRlp2mdBhERGRlvR/rjeVHlysd\nBhGRxdQ7kHZ2R1qOegbSaq0bYlzmYVzmYVxkC93Du2P96fVIz0lXOpRS0eL5xz5rgxb7bCm7D6Qv\nX76Mdu3aoUGDBnjssccwa9asQvfjjDQRkfJMzdnmCPAIQLPgZvjl9C9WiJCISDl2r5G+ceMGbty4\ngcjISKSmpiIqKgqrV69GWFiYHND/16rELY/DS01eQsd6He0ZHhFRqZXHGmlTc7a5Fvy9AOtPr8fP\nPX+2dshERCYrczXSgYGBiIyMBAB4eHggLCwM165de2Q/zkgTESnP1Jxtri5hXbD1wlbcy7xn8bGI\niJTiqGTjiYmJ+Pvvv9G8eXOjxwcNGoTTWaex3Gs5bobdRGRkJGJjYwE8qN+x97b+MaXaL2p75syZ\nqnh/+H5Ztq1/TC3x8P0ybXvmzJk4ePAgQkNDoQXF5Wz9e+Dj42PyORMbGospS6bgydpPKn5OmbN9\n8OBBjBo1SjXx2GNb/5ha4rHH9sN9Vzoee2yrNedb+/ObnJwMQM5pFhMKSUlJEVFRUWLVqlVGj+tD\nGr5huJj5x0wlQivUtm3blA6hUIzLPIzLPIzLPAqmVJsrKWeXxvIjy8VTS56yNDS7U+v5Z0vsszZo\nsc+W5m1F1pHOycnBc889h6efftrwV72evlZlwtYJcHdyx5tt37R3eEREpVIea6QB03J2aaRlp6Hq\njKo4POwwQrxDrBEqEZFZylyNtBACL7zwAsLDwx9JyAV5uXjhXhZr54iIlGRqzi4Nd2d3vNjkRUzb\nM82qxyUishe7D6R3796NJUuWYNu2bWjcuDEaN26MTZs2Ge0jBOBTwUdVA+mCtVJqwrjMw7jMw7jI\nlJxdUG6uecd/rcVrWHJ4CW6m3rQwUvvR4vnHPmuDFvtsKbtfbNi6dWvk5+cXu49OB3ywxhv3stUz\nkCYi0iJTcrbevn1A27aAhwdQrRpQrx7QrRvw3HNAhQqFv6aKZxX0juiNGXtnYMq/p1gxciIi21Ok\nRro4kiQBEHjq1Y0QzT/Dpn5Fz3wQEalJea2RLk7BPnftCsTGAr16ARcvAocOAcuXAwcOAH37AjNn\nAg4Ojx7jYvJFNJnbBGeHn4Wvq699O0BEmlbmaqRNdeWct6pKO4iIqGjnzgE7dgCDBwOVKgFNmwIv\nvABs2QIcOSIPqmfMKPy11X2qo2O9jvj8z8/tGzQRkYVUOZD28QHu3/JB4vVkPP+80tHI1Fo3xLjM\nw7jMw7jIVDNnAkOHymUdDwsOBhYtAqZOBY4dK/z141uPx+d/fo6kjCSjx4WQy0POnrVB0KWkxfOP\nfdYGLfbZUqocSNeuDSRd88c/6XewZo3S0RARUXHu3gWWLAFefbXofWrUAD78EBgwAMjJefT5uv51\n0bNBT4zZPMbo8fh44KefgHnzrBw0EZEVqLJGuksXgTXrcpE33hWYnAWRr8rxPhGREa3WSH/wgcDp\n08C33xa/rxDAM88AzZsDkyY9+nxKVgoivo7A3Li56FCrAwCgXz/A3R1Yuxa4dAlwcrJ6F4hIw8pl\njbSPD1A1yBFSlhfgelfpcIiIqBhffAG89lrJ+0kS8M03wPz5QJ8+wPnzxs97unhibtxcDF03FKnZ\nqbhzB1i/HvjoI3lGu5hV94iIFKHKgXSzZvKMhUirBLjdxoULwJkzysak1rohxmUexmUexkWm6NwZ\naNjQtH2Dg4ETJ4CwMODxx4HRo4GsrAfPd6jVAe1qtMP4reOxaBEQFwf4+ckXLi5YYH5sBw8CN26Y\n/7riaPH8Y5+1QYt9tpQqB9LDhgGNGwNIqwS430Lz5kDdukpHRUREhfnyS/P29/AA3n4bOH4cuHAB\n6NIFyMx88Pz0DtOx6sQqTF+/Di+/LD/WowewfTtw04z7tgghz3y/8YZ58RERmUqVNdJCCHz/PdBr\nZU/g5PPwSOyN1FQ5KRIRqZVWa6Qt6XNODtC/v3zB4urVgJub/PhXa/dhxB/PIWHEFkRWaQRAXlrv\nsceA11837di7d8sXNyYnA/v3A6GhpQ6TiMopS3OYagfSf/4JNH/nNSAlCN7HxuDePQ6kiUjdOJAu\nndxcYNAg+SYunTrJ18n88AMQ1OF7bHP8H/a9uA+BHoGIj5e/sTx6VK63LsmgQfLA+59/gLQ04HMu\nU01EDymXFxsCQGQkEP1YMOB5FfdUcF8WtdYNMS7zMC7zMC6yB0dHeZ3pnj2B69eBvXuBqlWBmS/2\nxAuNX0CnFZ2Qlp2G1q3lCZVXX5UHx3qpqcD33xuXfSQnyzPcAwcCI0cCS5cCt29bJ14tnn/sszZo\nsc+WUu1A2tkZGDk4BPC+pHQoRERkYw4O8gD500/llT0WLJBnpt9u+zYeq/wYnlzyJO5lJWPHDkCn\nA+rXB957T74JTEgI8MknQMeOD2qtly4FOnSQ77JYpYpcY80ZaSKyNtWWdgBAwrUEPDt7KG5NPgBA\nTpAuLkpGR0RUNJZ22Ea+yMeoTaOw69Iu/NrvV1Ryr4TTp4EpU4BateTa6SpVgN695UmYRYvkC9an\nTQOeeEI+xtmzQMuW8gWOt2/LS+81bQoEBto0dCJSuXJbIw0AdzPuosZnNXB/fDIACT/+KN8qdu9e\n+e6HFSsqGysRUUEcSNuOEAJvbXsLq06swoa+GxDqE/rIPunpQOvW8lJ8O3fKg2ddge9dBw8Gli2T\nLzqsVEkuCdm7F6hQwfrxpqfLFzvqB/JEpE7ltkYaAHwr+MJJ5wS430LDhkD37sC9e/KsQqVKJd9F\ny5rUWjfEuMzDuMzDuEgtJEnCB//6AMOaDkPzb5pj/en1j+zj5ibXRW/YALz4ovEgGpDLRdLTgVOn\n5FuP16kDjB374HkhgKlTgenTi4/l99+3IyOj+H0++0y+i+OlYqoTU1PlOK1Vu21LWvzMsc9kClUP\npM0k6zwAACAASURBVCVJQlilMCzbfBwtW8qP1a794PnBg7mSBxGRloxoPgKreq7Cf375DyZsnYDc\n/Fyj56tVk2/CMmbMo6+VJLkWW///c+fKA+9164DsbGDIEHm1kKlTgQMHCm9/2zbg5Zflm4bl5BS+\nT2oqMGOGXLM9dWrRfVm1Sm6/fftHB9P37xf9OiJSD1WXdgDAK7+8grCKYdg3awSWLXt0/19+kf/q\n17t0SZ5xqF/fDsESERXA0g77uZ12GwNWD8DN1JuYGzcXTYOaluo4u3bJJYPh4fKNYpYvB376SR4I\n//kn4OQk73fhgrz6x9Gjcm32/PnAk08Wfmv0jz+WB+KzZsm/i44eBYKCHt3viSfkGekjR4C1a4Hf\nfwdOnpRvif7rr3J5SnR0qbplN7dvA5cvA02aKB0JUemU69IOAGgS2AT7r+9Hz56FP//w12ZPPCHf\nepaIiMqvSu6VsKHPBoxuMRrPLXsOo38djftZ5k/jtm4tl3dERckzxO7u8g1iAgPlATEgD6ybN5cH\ntSdOyCuAfPEF8OGHwJUrxsdLS5NLQ95+G6hcWV5+75NPHm33yhV5sN2pEzB5sjx7XauWvH9cHLBw\noTzILnj7dEC+3bma/labPFmOt+CdKYk0RajMwyEduHZA1P+ivhBCiF69hJBTiPFPWtqD/atVkx+z\ntm3btln/oFbAuMzDuMzDuMyjwpRqc2ro8+2022Lw6sEiYFqA+GzvZyIzJ9PiY168KETFikIMGCBE\njRpC/Pnng+f059877wjRvbvx66ZNM37syhUhfH2FuHXLeL8pU4QYOvTBdn6+EAcPCpGT82C7Y0ch\nJk16sM+SJUI4OQnRtasQd+9a3EWzFPaZS0sTws9PiBYthJg1y77x2INa84wtabHPluYw1c9INwxo\niGsp1/BP+j9o3dq4ZMPdXf6vh8eDWQF9/RsREWlDRbeKWNBpATb334zfzv2G+l/Wx7z985CZW/pp\n0mrV5DWt8/LkmeNmzR7dZ9w4+dbjy5fL/92yRX7N228/2Cc4GOjVy/gCRiGAxYvl25frSRLQqJF8\ncxr99ldfyTPfx47JZSLjxsmrjAQHy8v77d79aEwnTgCdOwNvvgnculXq7ptk5Up5pv7LL+VylJIu\nwCQqj1RfIw0Azy17DgMaDUCPBj3w889A167y466uDz64CxbIFx/WqSMveaSuXhGRFrBGWh3iL8Zj\n6u6pSLiWgP80+w9ejnoZAR4BNmlryxbgv/+VJ3R8fIB27YC33jLe5+JF4PHH5cF0377yoLtHD/l3\nVUm3Op89G5g0CfD2Bn77DaheXX583TrgpZfkuwD37g08/bQ86P7qK3nAfe6cfLfHfv2AiRMBf3/r\n9711a/mizuefl39iY4FRo6zfDpEtlet1pPW++usr/HHlD3zX+TsAcoKoXVu+unrxYmD9emD48AcX\ndpw6JQ+kc3LkC0X27JGXzCspYRERWUKNg0pbU3Ofj98+jhl7Z2Dl8ZVoU60NBkcOxrN1n4Wzg7Pd\nYzl6VL4wfuRI+doePz95gFuS/Hz5xjJDhsjLvhaUliYPqJcvBzZtkmuVP/tMnrEG5Nutv/uufNHi\nb7/Jt123lmPH5GuSLl6Uf88ePCgP5s+dk5chtFR2tnxzndL44w/54kfewI1MYXEOs6gwxAYKC+na\n/WvCZ4qPSMuWi6FzcuQ66LNn5ed//FGuYZs4UYg6dR7USANCbNgg/3fXLsviUmvdEOMyD+MyD+My\njwpTqs2VhT6nZKWIBQcWiDYL2gi/qX5i8OrBYuOZjSIrN6tUxyvt+XfpkhBhYXKd87lzpTpEkfS1\n1YWZNk2I6tWFOHlS3s7LE+L0aSF+/VWIRYuE+PhjId56S4gxY4R49VUhFiwQIjvb+BhbtmwzemzE\nCCHefNN4n86dhRg/XohMC0vUZ88WwsNDiGPHzH/t+fPy+ztihGUxCKHePGNLWuyzpTlM9TXSAFDF\nswpaVG2BH4/9CECuIfv2W/nuVABQr568NNG77wJnzsiP6Wefr12T//vll8DVq/KMwOXLLP0gItIK\nD2cPDG48GDsH78ShYYfQMKAh3tvxHipPq4yuP3TF/APzcfneZZvHERIiL7c3dy5Qs6Z1j62vrS7M\nmDHy7HdsLNChg1zm8e9/yzPdv/0mrwTi7CyvMlK7NrB0KVC3rhznr7/Kq4d06SLPdI8fL3/ru2SJ\n/HhBU6fK3wAHBQHDhgHffQe88458M7XevR+t2V65Uo7phx/kmXch5JVQpk4FRo+Wy2Cys817H959\nF3jlFWDNGnm2vjD5+fLM/a5d5h1b77vv5Jpwc8YReXmla4vUr0yUdgDAxjMb8b8t/8OhYYegk4zH\n//n58l2qliwBvvkGaNoUSEiQn/vf/x4sYVS/vrxGZ2Sk/DXUtGlyTdvDyYCIqDTUXOZgK2W5z7fS\nbuHXs7/ilzO/YOuFrfB28UZsaCzaVm+LViGtUNO3JqRyVBO4ezeQlCRfOBlQQsn47t3ABx8Aycny\nOttdu8qD2i+/lJfmi44GNm4s/LUXL8qD8UOH5ImuBg3k///uO2DFCqBFC/liyBUr5AszZ8+WSzEj\nI+X68V9/BapUkeuuw8LkdbsBeeB64gTg6ysvT/jwP83x4/LA/MwZuZSma1f5ePpSFwBISZGXN7x2\nTZ5Ue/FFebDv5CS/N+vXy3/ktGpVeN8uXZLLRqpWld/Hr78u/o8YQD7+jz/KA/eiatUvXpTr21u1\nAv71L7nm/mG3b8t/dAwa9GCxBbKcJmqkAUAIgZbzW+I/zf6DAY0GFPJK2T//yCeYOTVaSUnyhRxF\n5cv//Ef+C9nT0/RjEpH2lOVBZWmVlz7ni3wcv30c2y5sQ/yleOy+vBt5+XloGdISzYKa4fHgxxFV\nJQq+rr5Kh6q41FR5htXb27zXbdggLwoQHCzXiK9YAVSsKA+QV6+WZ8c/+EB+DpAHjo0aAcuWyXd6\n/OADeQCclSXfeK1ePXmirH17ef+uXeXrofR3tXz/fWDrVmDmTKBCBbmmfOBAeSD/5ZfA3bvyoPTu\nXfkPi5075YH44cPyzPzEifIFlXpCyOt9N28uX1TZtatch71iRdFjjqVL5T8WnnpKPu7mzfJCCQXd\nuiW307at/O36n3/K2yNGyK+TJPmPiyFDgBo15DHLTz9ZduO53NyS/wAoi7Ky5HXhX3lFPj9MoYka\nab0/Lv8hAj8JFLdSbxW5j96CBUIMGvRgremAACFcXApfh1r/M3u2/Nrc3IdjEuKDD7aJ+HghHnvM\nkt5Zn1rrmRiXeRiXedQalwpTqs2V1z7n5+eLC0kXxIojK8Trv74u2ixoIzw+9BDVZ1QXrd5uJd7Z\n9o74/uj34sjNI6WutS5LrPWZS0wU4osviq/pLuiXX4RwcBAiMlK+Hkr/+/nePSHWrBGiShUhJkwQ\nYs8eIYKChEhPf/Da3Fx5HfCICCHq1pWvpZo1S16jWy8vT4iFC4VYvFg+phByffg33wgRGLhNdO4s\nxLVr8uM//ihEeLgQWf//z52VJcTAgUK4ucnx9eolH1+/ZviePUJUqiTEkSNyO716CdGtm/z/evfu\nCdG4sRBvv/3gsfv35dr1Ro3k9gYMEKJqVSF+/12Ofd48eY3zpUuNj5WdLcTMmUIEBwuxcqXx+5id\nLcQHHwjx9NNChIYKodMJMWSIECkpxvvNmbNN/PmnEKmppv37FJSfb/zeKuHTT4Vo0EAIf3+59l9/\nvuTnC3HjRuHxWZrDVJcBS+rQuC3jRLtv25m04H5a2oPBtBD/1965R0Vxpmn86Qs3EfDGRcUEb4BI\n07SinomXGI0SJ+BoTDQmh1zUnBOzmY25meTsZpPJxlsmxkucOMaT2THDrErc7EZNNEbFFY3GUTRq\ncBUNKAiichFoLt1d/e4fr30BQW0Eumne3zl1quqrqq+eqi5enqp6v6+IXnmloXEeNepWM71oEY9t\nF2d9vW1Zpn2do0d5WXU1/5H89rdE5eUN/4DbC081FKLLNUSXa3iqLm81lbejMx2zYlXo3PVz9N5/\nvEf/uvdf6bHNj1HMpzHk9+9+NGDVAHok/RH6/Xe/pxWHVtDW/9tKp0tOU1V91Z0r7gC4828uN7d5\ng3blClFyMpGvL9Hata273127Mulf/oX/z69dy0b94MFb16us5A/2bNhAlJZGFBJClJrKJn/7dsd6\ndXVE48bxsg8+IPr8c6IHHyR66aWmj89qJdq9mxuClpY2XJadzQa8d29uIPrll2wgH36YTXREBH/A\nh4jo2jWihx5iE/3NN9zQtLyc/VF0NHuanTtZW3h4JiUmEgUEEA0YQDR5MtELLxB9+CHf1FRWNn2u\n9u/n/Q8axPtwh6EuK+PfKieHG/M+9BDfiCQmcuPVXr34XDTmXmNYh0ntsKFYFczaMgs15hpsfnwz\ngvxun29hNAKnTvGrnO3buXugr77iHOqpU/mVyQMPONb38+NXAy+8AKxfzw0svv761nrT0/nVyn//\nN8+HhwMlJUBpKec6/fQT505lZNz96wVBEDo23pLm4Aqd8ZgbY1JMyCvPQ25ZLnJLc/Frxa+4UHYB\nv5b/iks3LqGLTxfc3+1+9Avuh8jgSEQGR6JvUF/0CepjH4L9gr0qH7s9sVo5bSQ5mXOdW5vjxzmt\nYswY4NNP77x+dTX7g+Bg/jiOMxUVnO5RXMyeISyMP7OubmHXD2fPsqfJyuIGntOmsa/JyeGGpfPm\ncTfBTzzBDTkbf7Ru0yZOg7A1JJ01i1M+LBbuyvDCBfY0+fmccvKPfwA6HbdFGzSI88n/6784hWbF\nCs7tfu01ru/3v+cGtn36sKYLF7jv9KIi4MYNTtfp1YvTZ8PCWE95OeeU79jB/uvZZznHHmBNxcWc\n/37yJNfxzjuOtNs33uBz/+c/87zVCmRmcgrSoEHcJq4pOk2OtDNmxYx/+u6fcLDgINKnp8PQ23BP\n+3z5Zb6IP/qIWxEvWcIXoTMWCze4+J//uX1daWlAfDw3eszNZZN+7RpffCEhwLp1fGH4+9+TZEEQ\nPJDOaCo74zG7AhHhqvEqLt64iMLKQhTcKEBhVSGKqopQVFWEy5WXUVRVBAKhd9feCO8ajvDAcIR3\nDUdYYBhCu4QiLDAMvbr0sg89AnrAXyv/RNoT2yXeke51cnP5q5qvvsof5mmO2lp+iHg3Zr62lvvp\nPnmSTfH580BCAueB2wyt2cw9vmzfzsa3qIhz6gcO5KFfP/ZDISHcH/nmzfyVz+ho4IMP+Objuef4\nQWV6OuuqruaGor16scfS6dhbZWdzDy0aDTf+PH2aG6K6Qqc00gAHp/ST6Xjjhzfw6OBH8dbotxDT\nq/Ue/ebk8J1Z//7chU5m5j6MHz8eY8awEZ4+nVvt1tc7tnnjDeB//5dNc2MeeYQbR+zfzy1yP/uM\nu++rqeGLrqKCL6gvvuC71Fdf5TvMvDzuaL++ni8mrdbxh7x/P7Bu3T58+ul4e+MMT2HfPj5fnobo\ncg3R5Rqd0VR2xmNui+uvqr4KxdXFKKkuQYmxBCXVJbhWcw1XjVdxreYartdctw+lNaXQqrXoEdAD\n3QO6o5t/N3T3726f7ubfDSF+IQjxC0GwXzBC/HlsG4J8g9DVtys0as2dhbXhMXs6csztQ1ER8PHH\n3CPLokXcK4oNRWEf1L07D43N/p//zI1CBw3iNxL/9m+u7/9eY1iHbbOpUqmQpk9DakwqVhxegbH/\nMRa6cB2ein8KU2OmIjQw9M6V3Ia4OE4J2b2bja4N534ny8u5+xxbNzuxscDrr3O3PQB/ynXKFP6h\nd+50bLd3b8PWtsHB/IrDmaefdkxv2MB3ZwDfjY0bxy2L//mfuWzPHjbfhw5xa+BZs7js6FG+i544\n0dFKuLKSy3r0YHPu78+vR7RaNvG2OhujKHzHR9Sx7sgFQRA6AkF+QQjyC0J0z+g7rktEqDHXoLS2\nFBV1Faioq0B5bbljuq4cBZUFOF13Gjfqb6CyvhKV9ZW4UXcDVaYqVNVXwWg2IkAbgK6+XRHkx8ba\neQj0CUSgbyCPfQJRfLoYOYE56OLTBYE+geji04WnfR3TAdoAHvsEwEftI6kqwl3Rpw/wySdNL9No\n2CQ3x4svcurH4sXsv9xBh30i3Zg6Sx22nd2GjJwM7LqwC4N6DMLofqMxvPdw6MJ1iOkZg0Dftut4\n8dIlfkKtVrO5tlg4H6d7d87TUauBc+e47+oXX+Q8qw0b+LVFUREwfDjw5pv8mgLg/jbT0tgw/+Mf\nnOsNsDm/eLHhvrt25dceAPdRWVratMbu3dn8A/wKpKkn5/ffz7nj4eH8eiQoiPvivniRn8T//DN/\n2Gb6dODBB7m7wb592Yh368bHYjTytidO8A1Dly58s7BqFXd9FBTEy/v2ZVNeWAhs3cr5Z7Gx/HEA\ns5k/EBASwuc2Pp5ztPz9+XWO2cz9iwYE8HRZGb+a6tmTNfn5cZ0TJ/K5Ly/nc9SzJ58vs5lfE3Xv\nzvPXr/PNQng461WpuM/UykrWbntrcOMGn6ewMH7FVVLCby18fIC6Ov6j9/XlOi5e5BuWigq+NkpL\neb6sjI9Dq3X8Vlot36R07843OESOGxyTieszmfi6slp5XlH4GPz8+DydP8/HYrtBUql4XFXF+q1W\n3r9t2teX62v8oQCVipefP8+/ExH/ZgEB/FvZPpygUjl02QazmT9jbLHw+VAURzdLGo2jrFs3Llep\neEzEy1Uq1mu18rpBQY6cPiIuU6v5uCwWx6Aot34cQa1mPfX1XIftHNvqut1Yq+Ub4kcfbfpvqTk6\n49PZznjM3oCVrKgx16CqvgpVpipUm6pRbaq2m2yjyYhqUzVqzDUwmo0NxyYjjGYjas219vJacy1q\nLTxfa66FQgr8tf4I0AYgwCcAAdoA+Gv97YOf1o/HGj/4af3sY1+NL/w0Dce+Gl/7ssaDj9oHPhof\n+Kh9eP7mtI/G55bltrFWrYVWrRWjLwDoxKkdt8OkmHDk8hEcLjyM7OJsnLp6CufLzqNHQA8M6D4A\n94Xch8jgSEQERnAOWmCoPe+su393BPkF3fLRl/bAbOZ//ocOsaksLmYjY7U6zI1Wy+X19dw3Zloa\nm3CrlU3fL7+w8duzhw3JjBlsyqOj2fAlJ/OXn3x9OX1l+3ZunNClCzcUGDCAjfTKlYDBANx3H9dl\nM6FTp7IpycjgxgY281ZaymkqUVH8JB9wNNy0HcPVq8DIkdxgwbbc9lObTLzetWtsstRqNm6XLrFp\nrahgLaWlbIo1GocB1Gh4G5sxbWyqnG8gIiPZeAFsxq5eZXPm68vHV1LiMH4Wi+MYbISFsVG8etWx\nzrVrvF5gIOuuq+ObCZvZjIjg9bt2ZfPdowePb9xg7X378m9nM5nBwXy+goMdpri+nuu0NZaor+fz\nExTE25WVOcx9YCDX5+/v6Ou1tpbna2tZl6+v4y2DzcACDc9beDj/7jZzazRybpta7Tg2k4nrsg0q\nFd+UqNWsw2LhsaLwEBDAx1ZdzefPZsp9fHga4POkUvHy2lqHLpXKUW4y8dg2qNWO5bb1rVaul8ix\nL8Cxzu3GZjPrXr36rv98b27b+UxlZzxm4c5YrBbUmmtRZ6lDnaUOtRanaXMt6pV61FvqUWeps0/X\nK/UwKSb7tNlqdkwrZl7WxLTZyvO26duVWawWmBUzFFKgUWkamGytWtvAaGvVWnu5bZlGrWmwXKPi\needyW5lzuUaluXVaxcvVKrW9zHn6dmUa1c3yZrZ3XtbccvsydcN5522aKm88qFQqHkNln+9IiJG+\nSxSrgsLKQuRV5HFjj8pCXDFe4fwzI+efldWWoaKuAkazEV18uiDIN8j+yiukOAR739vb6rrulbbK\nZ2qcwuFKSoeiAHv37sOkSU3runaNzZ1G4zBAzts6l9luIiwWR2tsk4nXKSzkJ+jArQbKZqZtT0Jr\nanibw4dvPV82Q2nbHmADZ3uCWVfHeo3Gpr8m1VizrR5n82c7Dud1bG8qAMfv6GmpM56aI+ipujqj\nqeyMx+yp119b4m3HTERsqm+abOdphRSYFTMOZh3E8N8Mty+zWC0NBsWqOKZJaVBuq0Mhxb7eLdM3\nt7GS1b6N87RivTl/c9p5bFvPeXnjadt84/pt8wSylxMRrGSFMdcIv4F+9noI1GD95qZt2xM4FjQw\n2jcNtrPpbmy8XV1XBVWzy52XNS7fOnsregQ0bFTWaXOkXUWj1uD+bvfj/m7333FdxarwKy6nXLL/\nXP+f7aDSdU6cONEmwa2xmXPF3Gk0wC+/nGjWSIfeJn29cdc8NqPp3KWRry+Po6Ju3V6lcjxldcb2\n1ammzpfzurbjdP5KlZ8fj5v7JGvjfdnqcT5nTbWGdi6z6fIkEw203fV1r3iqLqFz0BmvP287ZpVK\nxU+jNT5AM13mfZv/LXTTdO0rzM2sXLkSCxYsuKc6bKba2aDbTLZ9ulE5EdnXtxn8psqdt3U28M71\n29Z1Xs9W3tW3iW+v3yNuMdI7d+7EggULoCgK5s2bh7feessdMppFo9YgxD8EIf6O759up+1uVNQ8\nFRUV7pbQJKLLNUSXa3iqLm/F02N2e9MZrz855s5BaxyzSqXiVBLcfa8wHZl2TwRWFAUvv/wydu7c\niZycHGzcuBFnzpxpbxmCIAjCXSAxWxAEoXna3UgfOXIEgwYNQlRUFHx8fPDkk0/im2++aW8ZLpOf\nn+9uCU0iulxDdLmG6BI6asxuSzrj9SfH3DnojMd8r7R7Y8MtW7bg+++/x/r16wEA6enp+Omnn/Dp\nze9udrTWnoIgCM54W8M7idmCIHg7Haqx4Z2Crrf9ExIEQejISMwWBEFonnZP7ejbty8KCgrs8wUF\nBYiMjGxvGYIgCMJdIDFbEAShedrdSCclJSE3Nxf5+fkwmUzYvHkzpk6d2t4yBEEQhLtAYrYgCELz\ntHtqh1arxZo1a5CcnAxFUTB37lwMGTKkvWUIgiAId4HEbEEQhOZp/+9gA5gyZQrOnj2L8+fP4513\n3rGX79y5E7GxsRg8eDCWLVvWrprmzJmD8PBw6HSOztfLysowadIkREdHY/LkyQ36V1yyZAkGDx6M\n2NhY7Nq1q810FRQU4KGHHsLQoUMRHx+P1Te/WexubXV1dRg1ahQSExMRFxdn/x3drcuGoigwGAxI\nTU31GF1RUVFISEiAwWDAyJEjPUZXRUUFHn/8cQwZMgRxcXH46aef3K7r7NmzMBgM9iEkJASrV692\nuy7bfoYOHQqdToennnoK9fX1HqGrLfHEmN1etCQGewuuxFFvwNVY6A24Gs86Im3u78hDsFgsNHDg\nQMrLyyOTyUR6vZ5ycnLabf/79++n7Oxsio+Pt5e9+eabtGzZMiIiWrp0Kb311ltERPTLL7+QXq8n\nk8lEeXl5NHDgQFIUpU10FRcX0/Hjx4mIqKqqiqKjoyknJ8cjtBmNRiIiMpvNNGrUKMrKyvIIXURE\ny5cvp6eeeopSU1OJyDN+y6ioKCotLW1Q5gm6nnnmGfriiy+IiH/LiooKj9BlQ1EUioiIoEuXLrld\nV15eHvXv35/q6uqIiGjmzJn017/+1e263IG7Y3Z74WoM9ibuNo56C67EQm/A1XjWUWlrf+cxRvrH\nH3+k5ORk+/ySJUtoyZIl7aohLy+vwYmOiYmhK1euEBEH05iYGCIiWrx4MS1dutS+XnJyMh06dKhd\nNP7ud7+jH374waO0GY1GSkpKotOnT3uEroKCApo4cSLt3buXUlJSiMgzfsuoqCi6fv16gzJ366qo\nqKD+/fvfUu5uXc58//33NGbMGI/QVVpaStHR0VRWVkZms5lSUlJo165dbtflDjwhZruDO8Vgb8GV\nOOoNuBoLvQFX41lHpi39nVtSO5ri8uXL6Nevn30+MjISly9fdqMioKSkBOHh4QCA8PBwlJSUAACK\niooatFpvL635+fk4fvw4Ro0a5RHarFYrEhMTER4ebn/16Qm6Xn31Vfzxj3+EWu24vD1Bl0qlwsMP\nP4ykpCR7n7zu1pWXl4fQ0FA8//zzGDZsGF544QUYjUa363Jm06ZNmD17NgD3n68ePXrg9ddfx333\n3Yc+ffqgW7dumDRpktt1uQNPjNltzd3EYG/BlTjqDbgaC70BV+OZN9GaMdtjjLSnd+qvUqluq7Gt\n9VdXV2PGjBlYtWoVgoKCPEKbWq3GiRMnUFhYiP379yMzM9PturZv346wsDAYDIZm+7d11/k6ePAg\njh8/jh07duBPf/oTsrKy3K7LYrEgOzsbL730ErKzsxEYGIilS5e6XZcNk8mEbdu24Yknnmhyv+2t\n68KFC1i5ciXy8/NRVFSE6upqpKenu12XO/CW47hb7iUGdzRaI452NFojFnY0WiOeeQP3GrM9xkh7\nYl+l4eHhuHLlCgCguLgYYWFhAG7VWlhYiL59+7aZDrPZjBkzZiAtLQ3Tpk3zKG0AEBISgkcffRTH\njh1zu64ff/wRW7duRf/+/TF79mzs3bsXaWlpbtcFAL179wYAhIaGYvr06Thy5IjbdUVGRiIyMhIj\nRowAADz++OPIzs5GRESE288XAOzYsQPDhw9HaGgoAPdf90ePHsUDDzyAnj17QqvV4rHHHsOhQ4c8\n5ny1J54Ys9sKV2KwN+BqHPUGXI2F3oCr8cybaM3/JR5jpD2xr9KpU6diw4YNAIANGzbYA+jUqVOx\nadMmmEwm5OXlITc3194LQ2tDRJg7dy7i4uKwYMECj9F2/fp1eyvX2tpa/PDDDzAYDG7XtXjxYhQU\nFCAvLw+bNm3ChAkT8Le//c3tumpqalBVVQUAMBqN2LVrF3Q6ndt1RUREoF+/fjh37hwAYPfu3Rg6\ndChSU1Pdfu0DwMaNG+1pHbb9u1NXbGwsDh8+jNraWhARdu/ejbi4OI85X+2JJ8bstsDVGOwNuBpH\nvQFXY6E34Go88yZa9X9Jq2Zz3yPfffcdRUdH08CBA2nx4sXtuu8nn3ySevfuTT4+PhQZGUl/M+wv\nWgAAB0tJREFU+ctfqLS0lCZOnEiDBw+mSZMmUXl5uX39RYsW0cCBAykmJoZ27tzZZrqysrJIpVKR\nXq+nxMRESkxMpB07drhd28mTJ8lgMJBeryedTkcfffQREZHbdTmzb98+e2tzd+v69ddfSa/Xk16v\np6FDh9qvb3frIiI6ceIEJSUlUUJCAk2fPp0qKio8Qld1dTX17NmTKisr7WWeoGvZsmUUFxdH8fHx\n9Mwzz5DJZPIIXe7AnTG7vWhJDPYm7jaOegOuxkJvwNV41hFpa3+nImomAUoQBEEQBEEQhGbxmNQO\nQRAEQRAEQehIiJEWBEEQBEEQhBYgRloQBEEQBEEQWoAYaUEQBEEQBEFoAWKkhXbjq6++QlxcHCZO\nnAgAmD17NvR6PVatWuVyXaNHjwYAXLx4ERs3bmxVnYIgCAIjcVsQbo/02iG0G4888gjeffddjB49\nGleuXMHYsWORm5t7T3Xu27cPy5cvx7Zt21pJpSAIgmBD4rYg3B55Ii20Ounp6Rg1ahQMBgNefPFF\nWK1WfPDBBzh48CDmzp2LhQsXIjk5GZcvX4bBYMCBAwdw4cIFTJkyBUlJSRg3bhzOnj0LACgpKcH0\n6dORmJiIxMREHD58GADQtWtXAMDbb7+NrKwsGAwGrFy5Er/5zW+Qk5Nj1zJ+/HhkZ2fDaDRizpw5\nGDVqFIYNG4atW7feonvfvn0YN24cUlJSEBsbi/nz59s/j/vSSy9hxIgRiI+Px/vvv2/f5u2338bQ\noUOh1+uxcOFCAPwER6fTITExEQ8++CAAQFEUvPnmmxg5ciT0ej0+//xzAPxFpXHjxsFgMECn0+HA\ngQOt/GsIgiDcGYnbEreFFtI23V8LnZWcnBxKTU0li8VCRETz58+nL7/8koiIxo8fT8eOHSMiovz8\nfIqPj7dvN2HCBMrNzSUiosOHD9OECROIiGjmzJm0atUqIiJSFIVu3LhBRERdu3YlIv5YQEpKir2e\nFStW0HvvvUdEREVFRRQTE0NERO+88w6lp6cTEVF5eTlFR0eT0WhsoD0zM5P8/f0pLy+PFEWhSZMm\n0ZYtW4iIqKysjIiILBYLjR8/nk6ePEnXr1+3109Edm06nY6KiooalK1bt44+/PBDIiKqq6ujpKQk\nysvLo+XLl9OiRYuIiMhqtVJVVZULZ1sQBOHekbgtcVtoOVp3G3nBu9izZw+OHTuGpKQkAPz58IiI\nCPtyuvmkgJwyiqqrq3Ho0CE88cQT9jKTyQQAyMzMRHp6OgBArVYjODi4wf6oUWbSzJkzMXnyZLz/\n/vvIyMiw17lr1y5s27YNH3/8MQCgvr4eBQUFiImJabD9yJEjERUVBYBzAQ8cOIAZM2Zg8+bNWL9+\nPSwWC4qLi3HmzBnExcXB398fc+fORUpKClJSUgBwHuCzzz6LmTNn4rHHHrPv/9SpU9iyZQsAoLKy\nEufPn8eIESMwZ84cmM1mTJs2DXq93qXzLQiCcK9I3Ja4LbQcMdJCq/Pss89i8eLFTS5TqVS3lFmt\nVnTr1g3Hjx9vcpvGQfd29OnTBz179sSpU6eQkZGBdevW2Zd9/fXXGDx48G23d9ZHRFCpVMjPz8fy\n5ctx9OhRhISE4Pnnn0dtbS00Gg2OHDmCPXv2YMuWLVizZg327NmDtWvX4siRI/j2228xfPhwHDt2\nDACwZs0aTJo06ZZ9ZmVlYfv27Xjuuefw2muvIS0t7a6PVxAEoTWQuC1xW2gZkiMttCoTJ07Eli1b\ncO3aNQBAWVkZLl26dNttgoOD0b9/f/tdPxHh5MmT9vrWrl0LgPPVKisrG2wbFBSEqqqqBmWzZs3C\nsmXLUFlZifj4eABAcnIyVq9ebV+nueB/5MgR5Ofnw2q1IiMjA2PHjkVlZSUCAwMRHByMkpIS7Nix\nAyqVCkajERUVFZgyZQo++eQT/PzzzwCACxcuYOTIkfjDH/6A0NBQFBQUIDk5GZ999hksFgsA4Ny5\nc6ipqcGlS5cQGhqKefPmYd68ec3qEgRBaCskbkvcFlqOGGmhVRkyZAg+/PBDTJ48GXq9HpMnT8aV\nK1eaXNf5KcLf//53fPHFF0hMTER8fLy9UcmqVauQmZmJhIQEJCUl4cyZMw221ev10Gg0SExMtHfH\nZHulN3PmTHv97777LsxmMxISEhAfH4/33nuvST0jRozAyy+/jLi4OAwYMADTp09HQkICDAYDYmNj\n8fTTT2PMmDEAgKqqKqSmpkKv12Ps2LFYsWIFAGDhwoVISEiATqfD6NGjodfrMW/ePMTFxWHYsGHQ\n6XSYP38+LBYL9u3bh8TERAwbNgwZGRl45ZVX7vUnEARBcAmJ2xK3hZYj3d8Jwk2kSyZBEISOhcRt\nwd3IE2lBuIlKpWoyF1AQBEHwTCRuC+5GnkgLgiAIgiAIQguQJ9KCIAiCIAiC0ALESAuCIAiCIAhC\nCxAjLQiCIAiCIAgtQIy0IAiCIAiCILQAMdKCIAiCIAiC0ALESAuCIAiCIAhCC/h/ObYI3/Vu/HsA\nAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0xd089e90>"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As is shown above, the SGD solver uses less oracle calls to get to converge. Note that the timing is 2 times slower than they actually need, since there are additional computations of primal objective and training error in each pass. The training errors of both algorithms for each pass are shown in below."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(12,4))\n",
      "\n",
      "terr_bmrm = bmrm.get_helper().get_train_errors()\n",
      "terr_sgd = sgd.get_helper().get_train_errors()\n",
      "\n",
      "xs = range(terr_bmrm.size-1)\n",
      "axes[0].plot(xs, terr_bmrm[1:], label='BMRM')\n",
      "axes[0].plot(range(99), terr_sgd[1:100], label='SGD')\n",
      "axes[0].set_xlabel('effecitve passes')\n",
      "axes[0].set_ylabel('training error')\n",
      "axes[0].set_title('whole training progress')\n",
      "axes[0].legend(loc=1)\n",
      "axes[0].grid(True)\n",
      "\n",
      "axes[1].plot(range(99), terr_bmrm[1:100], label='BMRM')\n",
      "axes[1].plot(range(99), terr_sgd[1:100], label='SGD')\n",
      "axes[1].set_xlabel('effecitve passes')\n",
      "axes[1].set_ylabel('training error')\n",
      "axes[1].set_title('first 100 effective passes')\n",
      "axes[1].legend(loc=1)\n",
      "axes[1].grid(True)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAtYAAAEXCAYAAACJTANKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XtYVNX6B/DvcFOB4a7ITUYBBa+gVl5SsI7QDbTUQguj\nzLS0sosnrfMrS7vQMfOoJyMzy0yxoxmWQqUJaipqhaYYIkoiIoqAgshtWL8/ljPMlbnDnpn38zw8\nsGfv2fOuYeZlsebda4kYYwyEEEIIIYQQkzh0dgCEEEIIIYTYAupYE0IIIYQQYgbUsSaEEEIIIcQM\nqGNNCCGEEEKIGVDHmhBCCCGEEDOgjjUhhBBCCCFmQB1rosbBwQFnz541+H4lJSVwcHBAa2urBaLS\nbd++fYiMjDT7sYQQYmmFhYWIjo6Gh4cHVq5ciWeeeQZLlizp7LAsbtu2bQgJCYFYLMaxY8eUnodV\nq1aZ9bHEYjFKSkrMek5CVFHHmgjCokWLkJKSYtI5xowZg7/++svsxxJCiKV98MEHuPvuu3H9+nU8\n99xzWL16Nf71r38ZdS6JRIJffvlF6/7m5mZMnjwZvXv3hoODA3Jzc9WOefXVV+Hn5wc/Pz8sWLBA\naV9JSQnGjRsHNzc3REVFYffu3UbFCQCvvPIKPv74Y9TW1mLIkCFKz8PcuXONPm9cXBzWrl2rdFtt\nbS0kEonR5yREH9SxJlaBMQZbXMuopaXFrOeTSqVmPR8hpGP8/fff6N+/v17H6sobIpFIZ74cO3Ys\nNmzYgJ49e0IkEintS09PR2ZmJo4fP47jx4/j+++/R3p6unz/1KlTMWzYMFRVVeGdd97B5MmTUVlZ\nqVfsihhjOH/+vFK7DXke2qPaJkI6DCN24fPPP2eJiYny7fDwcDZlyhT5dnBwMDt27BhjjDGRSMQ+\n+eQTFhERwby8vNicOXPkx7W2trLFixez0NBQ1qNHDzZ9+nR27do1xhhj586dYyKRiEmlUsYYYzU1\nNezJJ59kAQEBLCgoiP3rX/+S71OUlZXFXFxcmLOzM3N3d2fR0dGMMcZiY2PZ66+/zkaNGsW6devG\nzpw5wz7//HMWFRXFxGIx69OnD0tPT5efZ8+ePSw4OFi+HRoaypYuXcoGDx7MPD092SOPPMIaGhoM\nPpYxxtLS0uTtWLNmDROJRKy4uFjjcx0bG8sWLFjAbr/9dubh4cEmTJjAqqqqlJ6jtWvXsl69erHY\n2Nh2n1PGGPvyyy9Zr169mK+vr/y43bt3M8YYe/PNN9mkSZPYY489xjw8PNjatWvbfd6LiorY2LFj\nmaenJ/Pz82OPPPKI/Pc6b9481qNHD+bh4cEGDRrETpw4obF9hBDzGjduHHN0dGRdu3ZlYrGYnT59\nmj3++OPsX//6F2OM56ugoCCWlpbGevbsyaZPn84qKyvZ/fffz7y8vJiPjw8bM2YMa21tZY899hhz\ncHBg3bp1Y+7u7uzf//53u48dHBzMcnNzlW4bOXIkW7NmjXz7888/ZyNGjGCMMVZYWMi6dOnC6urq\n5PvHjh3LPvnkE43nb21tZe+99x4LCwtjvr6+7OGHH2ZVVVWsoaGBubm5MZFIxNzc3FhYWBi76667\nlJ6HoqIi1tDQwF5++WXWq1cv5u/vz2bPns1u3rwpP/93333HhgwZwjw8PFhYWBjLzs5mr732mvw8\n7u7u7LnnnmOMMXnePnToEOvZsydrbW2Vn+fbb79lgwcPZowxJpVKNcasiex38+677zI/Pz8mkUjY\n119/Ld//ww8/sOjoaObh4cFCQkLYokWL5Ptu3rzJHn30Uebr68u8vLzYbbfdxioqKhhjjK1bt471\n6dOHicVi1rt3b6Vzrl27lkVFRTFvb2+WkJDA/v77b/k+yuOdjzrWduLs2bPMy8uLMcZYWVkZCw0N\nZSEhIYwxxoqLi5m3t7f8WJFIxBITE9m1a9fY+fPnWffu3Vl2djZjjL+hw8PD2blz51hdXR176KGH\nWEpKCmNMvWM9ceJENnv2bFZfX88uX77Mbr/9dqWOsKJFixbJzyMTGxvLQkNDWUFBAZNKpay5uZnt\n2LGDnT17ljHGWG5uLnN1dWW///47Y0y9syyRSNgdd9zBysvLWVVVFYuKipInf0OOzcrKYj179mQF\nBQWsvr6ePfroo8zBwaHdjnVQUBA7efIku3Hjhrzjq/gcPf7446y+vp7dvHmz3ef05MmTzN3dnf36\n66+sqamJvfLKK8zZ2VmpY+3s7MwyMzMZYzxRt/e8Jycns3fffZcxxlhjYyP79ddfGWOMZWdns2HD\nhsk79H/99RcrLy/X2D5CiPnFxcWxtWvXyrdTU1PZ//3f/zHGeL5ycnJiCxYsYE1NTezmzZtswYIF\nbPbs2aylpYW1tLSw/fv3y+8rkUjkOUIXTR1rT09PdvjwYfn20aNHmVgsZozxDmhUVJTS8c8995y8\n86pq+fLlbOTIkaysrIw1NTWxWbNmsalTp8r3qw5SqD4P8+bNYxMmTGDV1dWstraWJSYmsoULFzLG\nGMvLy2Oenp5s165djDH+t+2vv/7SeB7VxwoLC2M///yzfN/kyZNZWlqaXjErkv1uXn75ZdbU1MRy\nc3OZm5sbKywsZIwxlpOTI+/cHj9+nPn7+7PvvvuOMcbYJ598whITE9nNmzdZa2sr+/3339n169dZ\nXV0d8/DwYKdPn2aMMXbp0iV28uRJxhj/RyI8PJz99ddfTCqVsiVLlrBRo0YxxiiPCwWVgtiJ3r17\nQywW448//sDevXuRkJCAwMBAFBYWIjc3F2PHjlU6fsGCBfDw8EBISAjGjRuHY8eOAQC+/vprvPzy\ny5BIJHBzc8N7772HjIwMtQsWKyoqkJWVhY8++gjdunVD9+7dMW/ePGRkZGiMj2ko9RCJREhNTUVU\nVBQcHBzg5OSE++67D7179wbAP8qMj4/Hvn37tLb7+eefR8+ePeHt7Y3ExETk5+cbfOw333yDJ598\nElFRUejWrRveeuutdj9mFYlEmD59Ovr37w9XV1csXrwY33zzjdJ9Fi1ahG7duqFr165an1OpVIot\nW7YgKSkJo0aNgrOzM95++221jzhHjRqFpKQkAMC1a9fafd5dXFxQUlKCsrIyuLi4YNSoUfLba2tr\ncerUKbS2tqJfv37o2bOn1jYSQsxPNa8objs4OOCtt96Cs7MzunbtChcXF5SXl6OkpASOjo4YPXq0\n2eKoq6uDp6enfNvDwwN1dXUa98n219bWajxXeno6lixZgsDAQDg7O+PNN9/Eli1b2r3IXdZuxhjW\nrFmDZcuWwcvLC+7u7li4cKE8n61duxYzZszA3XffDQAIDAxEv3791M6jydSpU7Fp0yYAvPY6KysL\nU6dONTrmxYsXw9nZGWPHjsX999+Pb775BgAQGxuLAQMGAAAGDRqE5ORkeU27i4sLrl69iqKiIohE\nIsTExEAsFgPgv+8///wTN2/ehL+/v7w85pNPPsHChQvRr18/ODg4YOHChcjPz8f58+cpjwsEdazt\nSGxsLHJycrBv3z7ExsYiNjYWubm52Lt3L2JjY5WOVXwzurq6ypNqeXk5QkND5ft69eqFlpYWVFRU\nKN3/77//RnNzMwICAuDt7Q1vb2/Mnj0bV65cMSjmkJAQpe2srCyMGDECvr6+8Pb2xs6dO3H16lWt\n91dsR7du3eTt0OfYGzduAOBtVowjODjYoLh79eqF5uZmpRpExf3tPafl5eVKj9etWzf4+voqPZbi\nfl3P+wcffADGGG6//XYMHDgQ69atAwCMGzcOc+fOxZw5c+Dv749Zs2Zp/UNJCLGM9uqCu3fvDhcX\nF/n2/PnzER4ejvj4eISFhSEtLc1scbi7u+P69evy7WvXrsHd3V3jPgCoqamBh4eHxnOVlJTgwQcf\nlOej/v37w8nJSe1vhiLZ83DlyhXU19dj2LBh8vvfe++98lx64cIFhIWF6TyPJlOnTsW3336LpqYm\nfPvttxg2bJg8Lxsas7e3N7p16ybfDg0NxcWLFwEAeXl5GDduHHr06AEvLy+kp6fL/2alpKQgISEB\nycnJCAoKwquvvoqWlha4ublh8+bN+OSTTxAYGIgHHngAhYWFAHiOf+GFF+Sxyf4eXLx4kfK4QFDH\n2o7ExsZiz5492LdvH+Li4uQd7dzcXLWOtTaBgYFK0xWdP38eTk5O8Pf3VzouJCQEXbp0wdWrV1Fd\nXY3q6mpcu3YNf/75p8bzOjhofikqJsbGxkZMmjQJ//znP3H58mVUV1fjvvvus/hFjQEBASgtLZVv\nK/6szfnz55V+dnZ2hp+fn/w2xXZpe0579uyJgIAAXLhwQb7v5s2bav9IKJ5L1/Pu7++PTz/9FGVl\nZUhPT8ezzz4rn1rxueeew9GjR1FQUIDTp0/j3//+t852EkIsR/G9rdpJdHd3x9KlS1FcXIzt27dj\n2bJl2LNnj8ZjDTVgwAClT/eOHTuGgQMHyvedPXtWaZDi2LFj8lFZVb169UJ2drY8H1VXV6O+vh4B\nAQE64/Dz80O3bt1QUFAgv29NTY28Yx8SEoIzZ85ovK+u56B///4IDQ1FVlYWNm7ciGnTphkds2y/\nzN9//42goCAAwLRp0zBx4kRcuHABNTU1mD17tnzk28nJCW+88QZOnjyJAwcO4IcffsD69esBAPHx\n8fjpp59w6dIlREZGYubMmfLYPv30U6XYbty4gREjRgCgPC4E1LG2I7KOdUNDAwIDA3HnnXciOzsb\nVVVViImJ0Xo/xTKNqVOn4qOPPkJJSQnq6urw2muvITk5Wa1jHBAQgPj4eLz00kuora1Fa2sriouL\nsXfvXo2P4e/vj5KSknY/Cm1qakJTUxP8/Pzg4OCArKws/PTTT8Y+HTrJHvvhhx/GunXr8Ndff6G+\nvh6LFy/Web8NGzbg1KlTqK+vxxtvvIEpU6ZoTfTtPaeTJk3C999/j4MHD6KpqQmLFi1q9x8JXc/7\n//73P3lH3cvLCyKRCA4ODjh69Cjy8vLQ3NwMV1dXdO3aFY6OjsY8bYQQIym+tzWVxynasWMHzpw5\nA8YYPDw84OjoKM/D/v7+KC4ubvexGhsb0dDQoPYzAEyfPh3Lli3DxYsXUVZWhmXLliE1NRUA0Ldv\nX0RHR+Ott95CQ0MDvv32W5w4cQKTJk3S+DizZ8/Ga6+9Jh9suHLlCrZv367X8+Dg4ICZM2di3rx5\n8k/dysrK5Hl/xowZWLduHX755Re0trairKxMPrKrz3Mwbdo0LF++HPv27cOUKVNMivnNN99Ec3Mz\n9u3bhx07dsjPV1dXB29vb7i4uODw4cPYuHGj/G9BTk4O/vzzT0ilUojFYjg7O8PR0RGXL19GZmYm\nbty4AWdnZ7i5ucnz8ezZs/Huu++ioKAAAP804X//+x8AUB4XCOpY25GIiAiIxWKMGTMGAK+LCwsL\nw+jRo9sdGRGJRPLbnnzySaSkpGDs2LHo06cPXF1dsXLlSo33Xb9+PZqamtC/f3/4+PhgypQpuHTp\nksbYZEnI19cXw4cP13g+sViMFStW4OGHH4aPjw82bdqECRMmqMWqjWI7DDn2nnvuwfPPP49x48ah\nb9++GDlyJACgS5cuWu+bkpKC1NRUBAQEoKmpCStWrND6uO09pwMGDMDKlSuRnJyMwMBAiMVi9OjR\nQ/7Yqm0C2n/ejx49ihEjRkAsFmPChAlYsWIFJBIJrl+/jqeffho+Pj6QSCTw8/PD/PnztT4/hBDz\nU81P7eWroqIijB8/HmKxGKNGjcKcOXPknzwuXLgQS5Ysgbe3N5YtW6bxsfr16wdXV1dcvHgRCQkJ\ncHNzk3ckZ82ahcTERAwaNAiDBw9GYmIinn76afl9MzIycPToUfj4+OD111/H1q1b1UrUZF544QUk\nJSUhPj4eHh4eGDlyJA4fPqy1Xaq3paWlITw8HCNGjICnpyfGjx+P06dPAwBuu+02rFu3Di+++CK8\nvLwQFxcnb8MLL7yALVu2wMfHB/PmzdMY29SpU7F3717cfffd8PHx0TtmVbJrcwIDA5GSkoL09HT0\n7dsXAPDxxx/jjTfegIeHBxYvXoxHHnlEfr9Lly5hypQp8PT0RP/+/REXF4eUlBS0trbio48+QlBQ\nEHx9fbFv3z6sXr0aADBx4kS8+uqrSE5OhqenJwYNGoQff/wRACiPC4SIWfpzdEJszKlTpzBo0CA0\nNTVpLGEZN24cUlJS8OSTT5r9sWWjH2fOnFGqyyaEENLxcnJykJKSoleJILEPFh2xzs7ORmRkJCIi\nIjReWFFZWYl77rkH0dHRGDhwIL744gtLhkOI0bZt24bGxkZUV1fj1VdfRVJSkta6cKD9q9EN9f33\n36O+vh43btzAK6+8gsGDB1OnmlgE5WxCCDGNxTrWUqkUc+fORXZ2NgoKCrBp0yacOnVK6ZhVq1Yh\nJiYG+fn5yMnJwcsvv2z2legIMYdPP/0U/v7+CA8Ph7Ozs/xjOW3MuerX9u3bERQUhKCgIBQXF2ud\nspAQU1DOJsQ4tMojUeRkqRMfPnwY4eHhkEgkAIDk5GRkZmYiKipKfkxAQACOHz8OgNcG+fr6wsnJ\nYiERYrSsrCy9j5VdmW8ua9aswZo1a8x6TkJUUc4mxHCKdd2EABbsWJeVlanN/ZuXl6d0zMyZM3HX\nXXchMDAQtbW18gnVFdF/goQQa2Ytl7FQziaEEM6UvG2xUhB9kuu7776L6OhoXLx4Efn5+ZgzZ47G\nycxl0w4J6evNN9/s9BgoLoqL4hJ2XNbE1nO2Pb7+qM3UZmqz4V+msljHOigoSG1RDdUV6w4cOCCf\nZi0sLAy9e/eWz0EpdIoLeggJxWUYisswFJftsvWcbUn2+PqjNtsHe2yzqSzWsR4+fDiKiopQUlKC\npqYmbN68GUlJSUrHREZGYteuXQCAiooKFBYWok+fPmrnioiwVJSEEEIA8+ZsmWPHgAcfBBQWDyWE\nEJtmsRprJycnrFq1CgkJCZBKpZgxYwaioqKQnp4OgE9A/9prr+GJJ57AkCFD0Nraig8++EBpknaZ\nM2eAlhZASNfIyFahEhqKyzAUl2EoLttlzpwts20bcO4cMHw4kJ4OqKznZDPs8fVHbbYP9thmUwl+\ngRhe98fw119Av36dHQ0hhOhPJBKZpWbPmii2OS4OWLAA8PAApk0DUlOBRYs6MzpCCGmfqXnbapY0\nr6zs7AiU5eTkdHYIGlFchqG4lPn4+MiXUqYv/b/aG7W1Vw0NwNGjwOjRwKhRwP79wH/+A0ilnR2Z\n+Qk1j1gStVk4KG8LK28LqLiifQ0NnR0BIbavurra7kZYzUEkoinmVB06BAwcCIjFfDs4GAgIAPLz\ngWHDOjc2QmwJ5W3jWCpvW00pyA8/APff39nREGLbRCL7K10wB23Pmz0+n7I2v/km0NgIvP9+2745\nc4DevYFXXum8+AixNfaYZ8zBUnnbakpBaMSaEEKsR04OMG6c8m3jxgFmXpiUEEIEhTrWRhJqrRXF\nZRiKixDzu3kT+O03Xl+tKC6O11q3tHRKWBZjj+9XajMhmlHHmhBCiFkdPAgMGgS4uyvf7ucHhIby\nTjchhNgiq6mxXrWK1+cRQiyHavWMQzXWbUQiEf7v/xiam4H33lPf/8IL/CLGBQs6PjZCbJE95hlz\noBprGrEmxK5JJBK4urpCLBbDx8cHDzzwAC7cWtIvNTUVDg4O2L59u9J9XnzxRTg4OODLL78EAHzx\nxRdwdHSEWCyGp6cnBg8ejG3btsmPLykpgYODA4YOHap0nsrKSri4uKB3794WbqVt0FRfLUN11oTY\nB3vN2dSxNpJQa60oLsNQXNZDJBLhhx9+QG1tLcrLy+Hv74/nnntOvr9v375Yv369fLulpQXffPMN\nwsPDlaZVGj16NGpra1FTU4O5c+di2rRpqK6uVnqsmzdv4uTJk/LtjRs3ok+fPjStnp5+/53PXa1J\nbCwvFWlq6tiYLMke36/UZqKLveZsq+hYi8X8YhhCCAGALl26YNKkSSgoKADAE3hiYiL279+Pmpoa\nAEB2djaGDBkCf39/pfvKPuITiUR47LHH0NjYiOLiYqVjUlJS5CMmAPDVV19h+vTp9HGrngYPVq+v\nlvH2BiIigCNHOjYmQkjnsaecbRUd60cfFd6IdVxcXGeHoBHFZRiKy3AikXm+jCFLkvX19di8eTNG\njhwp39e1a1dMmDABGRkZAID169dj+vTpWs8llUqxbt06eHl5oV+/fkr7Hn30UWRkZIAxhoKCAtTV\n1eGOO+4wLmg7tHJl+/ttrRxEyO9XS6E2Ww/K2R3LKjrW/fvTiDUhQsGYeb4Mf1yGiRMnwtvbG15e\nXti9ezdeUVlpZPr06Vi/fj2uXbuGvXv3YuLEiWrnOXToELy9vdGtWzfMnz8f33//PcSy5QFvCQ4O\nRr9+/fDzzz/rTPZEna6VFW2tY02IkFHO7lhW0bF2cRHevKdCrbWiuAxDcVkPkUiEzMxMVFdXo7Gx\nEStXrkRsbCwqKirk+0ePHo0rV65gyZIlSExMRNeuXdXOM2LECFRXV6O6uhpJSUlIS0vT+FjTp0/H\nunXrkJGRgZSUFCoDMaNBg4DCws6Ownzs8f1KbSa62GvOtoqOtaOj8DrWhJDOIxKJ8OCDD8LR0RH7\n9+9X2vfYY49h2bJlOkcs3NzcsHr1auTm5iI3N1dt/0MPPYSdO3ciLCwMwcHBZo3f3gUEAJcvU14n\nxF7YU862io61kxMglXZ2FMqEWmtFcRmG4rIushEIxhgyMzNRU1OD/v37gzEm3/f8889j165dGDNm\njM7zeXt74+mnn8b777+vts/NzQ179uzBZ599Zt5GEDg7A927A+XlnR2Jedjj+5XaTPRhjznbqVMf\nXU80Yk0IAYDExEQ4OjpCJBJBIpHgyy+/RFRUFEQikXxaJW9vb4zTMomy4nEy8+bNQ1hYGI4fPw4P\nDw+l/apzo9J0e+YTEgKUlvLvhBDbZI852ypWXty4kWH7dmDTps6Opk1OTo4g/3uluAxDcSmjFbyM\nQysvttG3zVOmAJMnA4880gFBWZhQ84glUZuFwx7zjDnY9cqLTk40Yk0IIbYkOBi4tQgbIYTYDKsY\nsd66leGrrwCFVSwJIRZAIx/GoRHrNvq2edky4Px5YPnyDgiKEBtmj3nGHKxyxDo7OxuRkZGIiIjQ\nOD3K0qVLERMTg5iYGAwaNAhOTk7yFXgUCfHiRUIIsTXmytn6kNVYE0KILbFYx1oqlWLu3LnIzs5G\nQUEBNm3ahFOnTikd88orr+CPP/7AH3/8gffeew9xcXHw8vJSO5cQL14U6nyWFJdhKC5COHPmbH3Y\nUimIPb5fqc2EaGaxjvXhw4cRHh4OiUQCZ2dnJCcnIzMzU+vxGzduxNSpUzXuoxprQgixLHPmbH3Q\niDUhxBZZbLq9srIyhCjMoxQcHIy8vDyNx9bX1+PHH3/Exx9/rHH/smWpOHNGgkWLAC8vL0RHR8uv\nzJX9B0nbfFt2m1DiEfq27DahxNPZ28Q0y5cvR35+PiQSSWeHYjBz5uzU1FT5c6AtZ48ZE4fKSuDn\nn3Pg7Cyc94Cx2zJCiYe2zb8dFxcnqHgUt4lxcnJykJ+fLy9pKykpMfmcFrt4cevWrcjOzsaaNWsA\nABs2bEBeXh5WrlypduzmzZuxceNGjaMjIpEIe/YwvPkmoGGhHUKIGdFFMMaxhYsXzZmz9W1zSAiw\nbx9ghf+HECIY1pRnhMTqLl4MCgpCqcLnfKWlpVqXmMzIyGj3I0UhXrwo1P8SKS7DUFyEcObM2foK\nCbGNOmt7fL9SmwnRzGId6+HDh6OoqAglJSVoamrC5s2bkZSUpHbctWvXsHfvXkyYMEHruYR48SIh\nhNgSc+ZsfVGdNSHE1lisY+3k5IRVq1YhISEB/fv3xyOPPIKoqCikp6cjPT1dftx3332HhIQEdOvW\nrZ1zCa9jLdSaVIrLMBSXddm/fz9GjRoFLy8v+Pr64s4778TRo0cBAOXl5Zg5cyaCgoIgFosRFhaG\nJ554AoWFhQB47ZyDgwPEYjHEYjF69uyJxMRE7Nq1qzObJBjmzNn6spWOtT2+X6nNRB92mbOZwAFg\nv//OWHR0Z0dCiO0Tckq4du0a8/T0ZBkZGay1tZXdvHmT/fTTT+z48eOssrKSSSQS9thjj7GzZ88y\nxhirqalh69atYytXrmSMMXbu3DkmEomYVCpljDFWUVHB/vOf/zB3d3f2xRdfmBSbtudNyM+npRjS\n5o8+Yuy55ywYDCF2QKh5Rsg5mzHL5W1h/jYUAGDHjjE2cGBnR6Jsz549nR2CRhSXYSguZUJN0Iwx\nduTIEebl5aVx3+uvv86idfz3rZqkZZYuXcr8/f1Nio061m0MafOWLYxNnGjBYDqIUPOIJVGbhUOo\neUbIOZsxy+Vti023Z05CvHiREHslektklvOwNw276rpfv35wdHREamoqkpOTcccdd8Db2xsAsGvX\nLjz44INGxfHggw9i/vz5KCwsRL9+/Yw6BzGOrZSCECJklLM7mEnd8g4AgBUWMhYR0dmREGL7hJ4S\nTp06xVJTU1lwcDBzcnJiSUlJ7NKlSyw8PJylp6fLj8vMzGReXl5MLBaz+Ph4xpj20Y+bN28ykUjE\nDhw4YHRc2p43oT+flmBIm8vKGDPDwBMhdk3IeUaoOZsxy+Vti128aE40KwghBAAiIyOxbt06lJaW\n4sSJE7h48SJefPFF+Pr64uLFi/LjkpKSUF1djY8++ghNTU3tnrOsrAwA4OPjY9HYiTp/f6C6Gmhs\n7OxICCGWYI852yo61kIsBRHqfJYUl2EoLuvVr18/PP744zhx4gTuvvtufPfdd2qT+qtua7Jt2zb4\n+/sL8yNFG+foCAQEALf+Tlote3y/UpuJoewlZ1tFx5pGrAkhhYWFWLZsmXy0orS0FJs2bcLIkSPx\n0ksvobq6GikpKTh79iwYY6itrUV+fj5EIuX6QlnirqiowKpVq/D222/jvffe6/D2EC44mOqsCbFF\n9pqzraJjLcQRa6HOZ0lxGYbish5isRh5eXm444474O7ujpEjR2Lw4MH48MMP4evri0OHDqFr1664\n88474eFV3XilAAAgAElEQVThgZiYGNy4cQOrV69WOo+Xlxfc3d0xePBgZGdnY8uWLUhNTe2cRhGb\nWH3RHt+v1Gaii73mbBHTZ9y9E4lEIly5whAZCVRWdnY0hNg2kUik10dxRJm2580en09D2/zPfwI+\nPsCCBRYMihAbZo95xhwslbetYsRaiKUgQq21orgMQ3ER0rlsoRTEHt+v1GZCNLOKjrWsFCQ+HkhM\n7OxoCCGEmIuhpSCtrfw+9fWWi4kQQoxlFaUgN24w+PoCDQ2AszOgYyYWQoiRfHx8UF1d3dlhWB1v\nb29UVVWp3W6PH9Ea2uajR4GZM4E//tDv+AsXeMf6zBkgLMzIIAmxIZS3jWOpvG0VKy926ULznBLS\nETQlGUIsKSoKOHuWz2d9a1G2dp07x7+Xl1PHmhCA8rbQWEUpiKMj4OLCfxaZZ2VOkwm11oriMgzF\nZRiKi5ibmxtw113A9u36HX/2LP9+6ZLlYjKUPb7+qM32wR7bbCqr6FgDgKtrZ0dACCHEEiZPBrZu\n1e9YxRFrQggRGquosWaMITiYr87VpQuvtSaEEKGjGmv9XLvWdhGjh0f7xz7+OLBvHzB1KvDOOyYE\nSgghGtjFdHsA/7gQEE4pCCGEEPPw9ATGjgV++EH3sWfPAiNHCqsUhBBCZKymYy0rBRFKx1qodUcU\nl2EoLsNQXMRS9C0HOXcOGDVKWKUg9vj6ozbbB3tss6mspmNNI9aEEGK7kpKAXbuAujrtxzQ0AFeu\nAMOH04g1IUSYrKbGOj4e+PlnPnJ940ZnR0UIIbpRjbVhEhKAp54CpkzRvL+wELj/fiAnB7j9duDi\nRePjJIQQTQRdY52dnY3IyEhEREQgLS1N4zE5OTmIiYnBwIEDERcXp/VcNGJNCCGWZc6cbQxd5SDn\nzgF9+gD+/nzkWio168MTQojJLNaxlkqlmDt3LrKzs1FQUIBNmzbh1KlTSsfU1NRgzpw5+P7773Hi\nxAls2bJF6/m6drVUpMYRat0RxWUYisswFJftMnfONsb99/NyEG3OngV69+Yr8Hp5AZWVZn14o9nj\n64/abB/ssc2mstjKi4cPH0Z4eDgkEgkAIDk5GZmZmYiKipIfs3HjRkyaNAnBwcEAAD8/P43nSk1N\nRUEBP09zsxdycqLlIyWyX3pHb8t01uNr287PzxdUPPR8GbctI5R46PnSb3v58uXIz8+X5z1rYu6c\nLTuPl5cXoqP1y9kBAcC1azn46ScgPl59/7lzAGM5yMkBevaMQ3k5cOqU9vN11HZ+fn6nvwfs9T1H\n25bdFmrON/f7t6amBgBQUlICU1msxnrLli348ccfsWbNGgDAhg0bkJeXh5UrV8qPefHFF9Hc3IyT\nJ0+itrYWL7zwAlJSUpQDvFXrkpoKfPklIBYD169bImJCCDEva6qxNnfONpZEAvzyCy/5UDV5MvDw\nw/wrPh546SXgnnuMfihCCFFjag6z2Ii1SI9i6ObmZvz+++/YvXs36uvrMXLkSIwYMQIRERFqxzo5\nyc5r7kgJIYSYO2cbKyiILwamqWMtKwUBgJ49aWYQQojwOFjqxEFBQSgtLZVvl5aWyj8+lAkJCUF8\nfDy6desGX19fjB07FseOHdN4PkdH/l0oHWvVj8OEguIyDMVlGIrLdpk7Zxsfh/bZPs6dU+5YC2Uu\na3t8/VGb7YM9ttlUFutYDx8+HEVFRSgpKUFTUxM2b96MpKQkpWMmTJiA/fv3QyqVor6+Hnl5eejf\nv7/G88k61gDw3HPA119bKnJCCLE/5s7ZxgoM5CPWqqqr+Swgvr58OyDAuBHrN94AzpwxLUZCCNHG\nYqUgTk5OWLVqFRISEiCVSjFjxgxERUUhPT0dADBr1ixERkbinnvuweDBg+Hg4ICZM2dqTdKKpSCr\nVgGHDwOPPmqp6HWTFb4LDcVlGIrLMBSX7TJ3zjaWrBRElWy0WvapZc+ewKFDhp9/2zYgJgYIDzct\nTkX2+PqjNtsHe2yzqaxmgZgXXwSWL+dTLNXU8MUB8vI6OzpCCNHOmi5eNBdT27xxI7B9O5CRoXz7\n1q3Ahg28YwwAOTl89HnvXsPO36MH8O67fCEaQghRJegFYsxJ9eLFzq61FmrdEcVlGIrLMBQXsbTA\nQM011ooXLgLGXbzY0sLnvr561bQYVdnj64/abB/ssc2mspqOterFi53dsSaEEGJ+ukpBZIy5eLGy\nEmBMOAvLEEJsj9WUgrz+Ov/4zteXjzaMGAEcPNjZ0RFCiHZUCmK4GzcAPz+gvl55AOXee4G5c/nq\njADvILu68qXN3d31O/fx48CQIcATTwCff250iIQQG2Z3pSCEEEJsl5sb0KULnwVEkWopiEjER60r\nKvQ/t+xYc5eCEEKIjNV0rGkea/1QXIahuAxDcZGOoFoOIpUC58/zVRkVBQQYVg5SUQEEB1ONtTlQ\nm+2DPbbZVFbXsXa4FbFQOtiEEELMS7VjXVzMR6ddXZWPM/QCxooKoH9/qrEmhFiO1XSsZaUgDgKJ\nWKhzO1JchqG4DENxkY6gOjOIrDZalbEda3OPWNvj64/abB/ssc2mEkg3VTfZiLXQSkIIIYSYl+qI\n9bFjwODB6scZWgpy+TIQFcXrt1tbTY+TEEJUWV3HWiilIEKtO6K4DENxGYbiIh1BU8faXCPWQUH8\nAslr10yPU8YeX3/UZvtgj202ldV2rAkhhNgmTaUgmkasDZ3LuqIC8Pfn0/lRnTUhxBKsZh7r//wH\nmDcPCAkBSkuB0aOB/fs7OzpCCNGO5rE2zpEjwOzZwG+/ATU1fCaP69fVB1aOHgWefhr4/Xf9zhsU\nBBw6BEyaBKxYwddDIIQQRXYzj7WsHk4q7dw4CCGEWJZiKcjx48DAgZo/rTSkFKS1lS8m06NH20Jj\nhBBiblbbsaYaa80oLsNQXIahuEhH6NGDd3ybm7XPCALwso4rV/QbcKmu5tP1deli/o61Pb7+qM32\nwR7bbCqr6VjLEmdLS+fGQQghxLKcnHjn+tIl7RcuAoCzM+DtzWf70OXyZd4RB6jGmhBiOVbTsRba\niLVQ53akuAxDcRmG4iIdRVYOou3CRZlevfiqjLrILlwEzD9ibY+vP2qzfbDHNpvKajrWNGJNCCH2\nIzCQX6h+4gQwaJD24yQSoKRE9/ks2bEmhBAZq+lYC+3iRaHWHVFchqG4DENxkY4SFATk5vKSEE9P\n7ceFhgJ//637fJbsWNvj64/abB/ssc2mardj3draigMHDnRULO2SdaxlI9adXQpCCCFCI6Scbaqg\nICArS3t9tYwhHesePfjPvr5UY00IsYx2O9YODg549tlnjT55dnY2IiMjERERgbS0NLX9OTk58PT0\nRExMDGJiYrBkyRKt55KNVAtlxFqodUcUl2EoLsNQXMImpJxtqsBA4OxZ3R1rfUtBVC9epBpr01Cb\n7YM9ttlUTroO+Mc//oEtW7Zg0qRJEBkwTCyVSjF37lzs2rULQUFBuO2225CUlISoqCil42JjY7F9\n+3Y9zse/y0auacSaEELUCSVnmyooiH9v78JFQBilIIQQIqOzxvqTTz7Bww8/DBcXF4jFYojFYnh4\neOg88eHDhxEeHg6JRAJnZ2ckJycjMzNT7Th9V7eRdahlOrtjLdS6I4rLMBSXYSgu4RNKzjaVrGOt\nbymIalgjRgAnT7Zta+pYm6sp9vj6ozbbB3tss6l0jljX1dUZdeKysjKEhITIt4ODg5GXl6d0jEgk\nwoEDBzBkyBAEBQVh6dKl6N+/v9q5UlNTceOG5NaWF4BoAHEA2n7pso8rOmpbprMeX9t2fn6+oOKh\n58u4bRmhxEPPl37by5cvR35+PiQSCTqLUHK27Dnw8vJCdHS0wc/psGFx6NsXOH8+BxcutPeazAFj\nQFVVHHx9+f7GRuDIkThkZQFXrvDjKyri4O/fdn+RKA43bgBHj+oXT3vb+fn5nf4esNf3HG1bdluo\nOd+c2/n5+aipqQEAlOhTV6aDiOkx/JCZmYm9e/dCJBIhNjYWiYmJOk+8detWZGdnY82aNQCADRs2\nIC8vDytXrpQfU1tbC0dHR7i6uiIrKwsvvPACTp8+rRygyprtspHquDhgzx59mkgIIZ1DNX91FCHl\n7I4wZAiwbh0wdCjf/uMP/nN8PPDjj3xk2tWVX7Do5saPCQkB9u/nI96EECJjag5z0HXAggULsGLF\nCgwYMABRUVFYsWIFFi5cqPPEQUFBKC0tlW+XlpYiODhY6RixWAxXV1cAwL333ovm5mZUVVUZ2gZC\nCCG32GPOlkiU66xPnADuuQc4cABoaADq6gAHh7ZONUB11oQQy9DZsd6xYwd++uknPPnkk5gxYway\ns7Pxww8/6Dzx8OHDUVRUhJKSEjQ1NWHz5s1ISkpSOqaiokL+X8Hhw4fBGIOPj49egVONtWYUl2Eo\nLsNQXMIn1JxtSaGhyjODnDgBjB4NDBzIO9eK9dUy5uxY2+Prj9psH+yxzabSWWMtEolQU1MDX19f\nAEBNTY1eV5o7OTlh1apVSEhIgFQqxYwZMxAVFYX09HQAwKxZs7BlyxasXr0aTk5OcHV1RUZGhonN\nIYQQ+2aPOVt1ZpCTJ4GnngIaG4Fdu4D779fcsaa5rAkh5qazxnrTpk1YsGABxo0bB8YYcnNz8f77\n7yM5ObljAtRSY33XXcDu3R0SAiGEGKUz6o2FlrM7wtatwIYNwLZtfFsi4R3qsjLglVeAhQuB9euB\n775ru8+zzwL9+wNz53ZoqIQQgTM1h7U7Yt3a2goHBwccPHgQR44cgUgkwvvvv4+AgACjH9BcOrsU\nhBBChEbIOduSFEtBrl8HrlwBevcGevUCCguBU6csWwpCCCEy7dZYOzg44IMPPkBgYCAmTJiApKQk\nm0/Q+hJq3RHFZRiKyzAUl7DZa85WLAUpKAAiIwFHR8DFBbjzTiAjo205cxmqsTYNtdk+2GObTaXz\n4sXx48dj6dKlKC0tRVVVlfyrs+3eDdy8yX8WiQCV6VYJIcQuCTVnW5KfH6+nvn6dX7g4cGDbvvHj\n+W26aqyzsvRbwdFY+/YBzz1nufMTQoRBZ421RCJRu/BFJBLh7NmzFg1M8bE01VjLMMZv+/RTYObM\nDgmJEEL00hn1xkLL2R2lf39g82Zg7Vq+auP8+fz2P//ky6J/8w0wZUrb8VlZwPLlfJ5rqZSPei9c\nCMyZo3zezz7jc14nJJgWX3o6r/P+9VfTzkMIsSyL11inpaXhkUceMfoBCCGEdAx7ztmycpCTJ5U7\nwQMHAj17tl9jvWcPv9BR0/8e338PDBpkese6ogK4fNm0cxBChE+vGmtr0NEDJEKtO6K4DENxGYbi\nEjZrytnmJlskRrUURCQCdu4ERo5UPl6xY/3ll8DYsUBxsfp5z5wBLl5s/7H1ef1VVPCLKm2FPb7n\nqM1EH1ZZY62YNAkhhLQRYs7uCKGhwG+/AfX1gMqCkYiJAZydlW+Tdaxra/mo9L/+pT5i3drKbysr\nMz2+igrg2jWgqcn0cxFChMuoGmsAOHfunMWCUqSpxjoiAigq4tuyGuv0dODppzskJEII0YtQaqyB\nzsvZHSUjA5g3D+jTh6+2qAtjfNaQ//4X2LGD1z8HBfGOtuzpu3CB11cPGMBHwk0xZgywfz8/Z1CQ\naecihFiORWusAaBEcZ1YgWhs7OwICCFEmISYsztCaCgfFVZZhV0rkQjw8eEXMC5eDHh6Al278jpo\nWT12cTGfuk9XKYg+KiqAbt14OQh1rAmxXTpLQW7cuIHFixdj5q0pN4qKivDDDz9YPLD2CKFjLdS6\nI4rLMBSXYSgu4RNizu4IoaH8uyGlgr6+wKVLwAMP8O2wMOU66zNngNtvB27caJveVRN9a6z797ed\nCxjt8T1HbSb60NmxfuKJJ+Di4oIDtz5bCwwMxOuvv27xwNrT0ND2c2tr58VBCCFCI8Sc3RF69uSl\nHYZ2rKdOBbp04dt9+ijXWRcX89LDwEDT6qwbGvhXRIRtXcBICFGns2NdXFyMV199FS4uLgAANzc3\niwfVnmnT+BXcMsOGdU4ccXFxnfPAOlBchqG4DENxCZ/QcnZHcXAA5s417G9CSoryoi1hYcod6zNn\ngPBwXrrRXjnIyZNxWLxY+/6KCr7yY48ettOxtsf3HLWZ6ENnjXWXLl1wU+EzsOLiYnSR/XvfCb7+\nmn+XXdGdn8+3O+FaGUIIERyh5eyO9OGHhh2vesF7nz5Abm7b9pkzvLOta8R6w4b266YrKnjddvfu\ntlMKQgjRTOeI9aJFi3DPPffgwoULmDZtGu666y6kpaV1RGyCJtS6I4rLMBSXYSgu4aOcbTzFEWvG\neCmIrhHrixeBQ4dyUFqq/byyjrUtjVjb43uO2kz0oXPEOj4+HkOHDsWhQ4cAACtWrICfn5/FAyOE\nEGI4ytnG69On7eLFykrA0RHw9uYda20j1pmZwJAhfBo9bWjEmhD7oXMe686mbT5BP7+2VbMA4JNP\ngFmzOjAwQgjRobPmdO5M1txmqRRwcwOqqoDjx4HnnwcOHwY2buQd6M2b1e+TkAA8+SSv175xQ30h\nGgB45x2gro7PPjJ/vn7zbBNCOoepOUxnKYi1sNI8TgghRCAcHfnS6OfOtdVXA9pLQWpqgIMHgfvv\n52Ue2spFFEesbaUUhBCimc10rDuaUOuOKC7DUFyGobiIrZNNuSerrwa0X7y4cycQFwccPZqDkBDt\n5SC2WApij+85ajPRh84a66qqKrXbxGIxnDV93kUIIaRTUc42jWyRmDNngH/8g98mG7FmrG25cwDY\ntg2YOJH/HBICrRcwyjrWXl58oZnGxra5swkhtkXniPXQoUPh5+eHiIgIREREwM/PD6GhoRg6dCh+\n++23du+bnZ2NyMhIREREtHtV+pEjR+Dk5IRvv/1W78BVSz86uhREqHM7UlyGobgMQ3EJn1BztrVQ\nHLGWlYK4uvLlyBX/Z2loAH76CUhM5K+/4GDdHWuRiF8fZAvlIPb4nqM2E33o7FiPHz8eWVlZuHr1\nKq5evYrs7Gw88MAD+O9//4tnnnlG6/2kUinmzp2L7OxsFBQUYNOmTTh16pTG41599VXcc889JhWL\nU401IYRYT84WKsURa1kpCKBeDrJrFxAdzcs7AOhVCgLY1pR7hBB1OjvWBw8eREJCgnw7Pj4eBw8e\nxMiRI9HU1KT1focPH0Z4eDgkEgmcnZ2RnJyMzMxMteNWrlyJyZMno7ssOxlJKjXp7gYTat0RxWUY\nisswFJfwWUvOFqo+ffjCY/X1bZ1hQP0Cxp07+Wg1wF9/2kpBmpqA2lrAx4dv20qdtT2+56jNRB86\na6wDAgKQlpaG5ORkMMbwzTffwN/fH1KpFA4O2vvlZWVlCAkJkW8HBwcjLy9P7ZjMzEz88ssvOHLk\nCESKxWsKUlNTIZFIAABeXl6Ijo4GEHdrbw4A4Pnn49CzJ9C9O9+WfXwhe1GYe1vGUuc3djv/1lKU\nQomHni/jtmWEEg89X/ptL1++HPn5+fJ81RmEmrM7+zWi7/aFCzm4eBEYPDgOIlHb/sDAOJSVtW3/\n8kscnn6ab+fn52PEiDiUlqqfLzMzBx4egIMD325tzcHevUBCgvrjX7wIzJyZg/nzhfN8CP09R9uW\n3RZqzjfndn5+PmpqagAAJSUlMBnT4fLly2zOnDksOjqaRUdHszlz5rDLly+zxsZGVlRUpPV+W7Zs\nYU899ZR8+6uvvmJz585VOmby5Mns0KFDjDHGHn/8cbZlyxa182gL0ceHMV4A0vaVmKirNYQQ0nH0\nSLFmJ9ScbU169mTswQeVb3vtNcbeeov/fOECY76+jEmlbfsvXGDM31/9XEePMhYd3bb9wguMLVum\n+XFzcxlzdmassdG0+AkhxjM1h+kcse7evTtWrVqlcV+4YgGaiqCgIJQqfC5WWlqK4OBgpWN+++03\nJCcnAwAqKyuRlZUFZ2dnJCUl6fyHQJOWFqPuRgghNsOacrZQ9emjXF8N8FKQ48f5z3v2AHFxgIPC\nBwA9e/KLG5uaABeXttsrKnhdtUx7pSBVVUBzM1BQwOu3CSHWR/vngrcUFhZi5syZGD9+PMaNG4dx\n48bhrrvu0nni4cOHo6ioCCUlJWhqasLmzZvVku/Zs2dx7tw5nDt3DpMnT8bq1av1TtBjx6rf1pEd\na9WPw4SC4jIMxWUYikv4hJqzrcmAAfxLkeLFi7/8Aig+pTk5OXB0BAIC1Oe7VrxwEWj/4kXZrCO/\n/25a/B3BHt9z1GaiD50j1lOmTMEzzzyDp556Co6OjgCgta5O6cROTli1ahUSEhIglUoxY8YMREVF\nIT09HQAwy8T1x7duBW6/HVCcPYpGrAkh9k6oOduafPyx8mg0wEesy8p44eHu3cA//6l+P9mUe717\nt92m2rFub8S6uhro2pV3rJ980vR2EEI6nuhWPYlWw4YN0zn3qSW1t2b7yJHAoUNt22PGAHv3dlBg\nhBCiQ3v5y1KEnLOtWVkZMGwYcOAAcOedfFv1/5XkZCApCZg2re22F1/knfJXXuHbv/7Kfz54UP0x\nXn8dOHIEqKvjj0MI6Xim5jCdpSCJiYn473//i/LyclRVVcm/hODWYIzcvn180n5CCLFXQs7Z1szf\nH7h6FfjxR14GoulDAE1T7hlaCjJuHK/lVpxCtqEBWLHC9DYQQixPZ8f6iy++wNKlSzFq1CgMGzZM\n/iUEThoKWf76q2MeW6h1RxSXYSguw1BcwifknG3NnJx4p3jjRuX6aqDt9adp9UVDSkGqqgCJhNdz\nFxa23Z6VBbz8srDKHe3xPUdtJvrQWWNtljn9LER1xBpQvhqbEELsjZBztrULDAT27we++krz/pAQ\nfmGjItWOtacnH4FuaOD11Iqqq/lCMkOH8jrr/v357Vu38k71+fN8xhJCiHBprbHevXs37r77bmzd\nulXjhS8PPfSQxYMD2q91iY8Hfv5Z+bbCQqBv37bte+8FYmKAd9+1YJCEEKJBR9YbW0POtnYTJ/Iy\njbNnNe8/cgSYPVv5ovru3YE//+TT8ckEBfHrgxTW4wEADB8OrF7Np/O7dAlYtgxobOT3DQoCPvoI\nGD/e/O0yp88+46tWPv98Z0dCiHFMzWFaR6z37t2Lu+++G99//32nJun2aCoFUf2oLDsbKCqijjUh\nxLZZQ862doGBgJ+f9v2qNdYtLXwUWvU+3bvzOmvVjnVVFeDtzUeslyzht+3aBQwcCERFAWfOCL9j\n/eefvBadELtl0vIyHaC9EPfvZywiQnn1xRdfVL0/Y717mz+uPXv2mP+kZkBxGYbiMgzFZRgrSLFm\nZ8ttPnGCscJC9dtlrz+plDEXF8Zu3uS3l5cz1r27+vHjxzOWna1+u5cXY1evMlZZyZiHBz9faipj\ny5czlpbG2MsvKx/f2srYp5/y7x1N23tu6lTGRo/u2Fg6ilDzjCXZY5tNzWE6a6wbGhqwdetWlJSU\noOXWcLBIJMIbb7xh0Q6/PkaP5vVoij76iH98pshGP5UkhBA1Qs7Z1k510RhVDg5tC8mEhanXV8to\nuoBRKgVqa3kNtqMjH7kuLAS2bwfefhs4fFh9Cr4LF4Cnn+ZTzw4caFrbzOXyZYDK/Ik909mxnjBh\nAry8vDBs2DB0Vb3SQgD0WPfAIh3ruLg485/UDCguw1BchqG4hE/oOdsWKb7+ZOUg7XWsNU25V1MD\niMVtF+UPHQp8+CEQEcHPefUqUFysfB/ZEutZWR3fsdb2nrtyBbh4kdeGd+nSsTFZmj3mGXtss6l0\ndqzLysrw448/dkQsRmls1H1Ma6vl4yCEECEQes62dbIp965c4R3jqCj1YzSNWFdVKX8CO3QosGgR\n8N57fDssjF80yVjbgNLx40BkJO9Yz59vkeYY7MoVPttJaSkQHt7Z0RDS8XTOYz1q1Cgcl/1bLEBF\nReq3qY5QW2LEWqhzO1JchqG4DENxCZ/Qc7YtUnz9hYTw6fFiYoDoaN65VqVpxFo21Z7M0KG8PGTS\nJL4tFgPu7kB5edsxx4/z2TeOHAGuXzdfe/Sh6T3HGFBZyWM/d65j4+kI9phn7LHNptLZsd63bx+G\nDRuGvn37YtCgQRg0aBAGDx7cEbHpJShI/bamJuVtqrEmhNgLoedsWxcWxqfS+/xzIC1N89oK+oxY\njxwJzJunPG91eLhyOcjx4/y4UaOA3bvN2w5jXLvGR6v79aM6a2K/tM5jLaNtsQGJRGKBcNTpmk+w\nvh7YtAl46qm226qrAS8v2f2BgABe80UIIR2pM+Z0FnrOtnXNzXxwx81N+zGHDwPPPgscPdp228aN\n/ELFjAzt95s+nS95/sQTfIEZb29em716NXDyJLBmjfnaYYzTp4H77gMef5zH9847nRsPIcaw2DzW\n169fh4eHBzw8PIw+eUdwdQW6dVO+7ebNto41QCPWhBDbZy0529Y5O/Ov9gQH8xk9FKmWgmgSFtY2\nYn3qFN/u0oUvhLZ0qXL9dWe4coWPxkskfA0JQuyR1lKQqVOnAgCGDh2KYcOGqX0JierS5jdvKm9T\njXXno7gMQ3EZRqhxdSRrytm2xtDXn78/L/1QLFtULQXRJDycLxID8DIQWYVP3768g33ihEFhmERT\nmxU71rZYCmKPecYe22wqrSPWO3bsAKD9Y0UhUe1Y19crb9OsIIQQW2dNOdveOTq2lSjKKnSqqtRX\nYlSlOGKt2LEWifio9c6dwKBB6veTStX/TlqCrXesCdGHzhprAKiurkZRUREaGhrkt40dO9aigcno\nU+uybRugulrvmTM8CYlEfDlZ1SuwCSHE0jqr3ljoOZvwBc7efx8YM4ZvP/44EBfH66e1qazko9NV\nVUB8PL+48b77+L4dO4B//xtQHWBsbuYdXWdnYMQIYOxYYNYsy3S033kHqKvjy7G7uvKZSmxtLmti\n+yxWYy2zZs0arFixAqWlpYiJicGhQ4cwcuRI/PLLL0Y/qLlpShC//MI71gDVWBNC7Ic15GyiXmet\nT1hqT4AAACAASURBVI21ry8ffa6qUh6xBvhFjVOnto0ay/z4IxAaCnzxBZ+tZNEivpiMJf7PunIF\n6NWL/00ODgbOn+cL3BBiT3ROt/ef//wHhw8fhkQiwZ49e/DHH3/A09OzI2LTm6aOdXNz28+WKAUR\nat0RxWUYisswFJfwWUPOtjXGvP5UO9b61FiLRHzA6MABXp+tON2sqysweTLw2WfK9/nqKz6bSN++\n/PvddwMFBQaHq6a9GmvANstB7DHP2GObTaWzY921a1d0uzXtRkNDAyIjI1FYWGjxwAyhqWOteFEI\njVgTQuyFNeRsYlzHGuAXMG7bxkerVWcAmTuXT73X0sK3a2r47BwPP9x2TP/+fGo+S7hyhS9+A9hm\nx5oQfejsWAcHB6O6uhoTJ07E+PHjkZSUpPd8qNnZ2YiMjERERATS0tLU9mdmZmLIkCGIiYnBsGHD\njP6oUteItSU61nFxceY/qRlQXIahuAxDcQmfNeRsW2PM6y8kRL1j7e2t+35hYXy+a01r/gwdys+b\nmcm3//c/YPx45Q77gAHm6VhravPly7Y9Ym2PecYe22wyZoA9e/awzMxM1tjYqPPYlpYWFhYWxs6d\nO8eamprYkCFDWEFBgdIxdXV18p+PHz/OwsLC1M6jT4i7dzPGu89tX++8I7s/Y2KxzlMQQojZGZhi\nzU6oOZswdvAgY7fdxn9ubWXMxYWx+nrd9/vsM/53bc0azfs3bWIsLo7/PGYMY5mZyvsvXGCsRw/j\n425PYCBj58/zn9evZ2zaNMs8DiGWZGoOa3fEuqWlBZGRkfLtuLg4JCUlwUXTGq0qDh8+jPDwcEgk\nEjg7OyM5ORmZsn+jb3FTWJqqrq4Ofn5+hvxPINcZpSBCrTuiuAxDcRmG4hI2a8nZtsbUGuv6ev53\nTHWxM01kF+VrW6X+oYeAwkI+qv3XX8A99yjvDwwEGhv5DCOmUG0zY/yctjxibY95xh7bbKp2ZwVx\ncnJCv3798PfffyM0NNSgE5eVlSFEYVLO4OBg5OXlqR333XffYeHChSgvL8dPP/2k8VypqanyjzK9\nvLwQHR0t/3giJycHf/4JAHG3js4BADQ3t23zerO24wEo3d+YbRlznc9c2/n5+YKKh54v47ZlhBIP\nPV/6bS9fvhz5+fkdtny4KmvJ2UDnv2bMuZ2fn2/w/e+8Mw6VlcCuXTm3ykD0u//ly3x7wADN+w8c\nyEFCApCSEofp0/m24v7c3BwEBQEFBXEYO9Z877mYmDh06QIcOsS3w8LiUFIijN8Pbdtezjf3+7em\npgaAedYB0DmP9ZgxY/DHH3/g9ttvl49WiEQibN++vd0Tb926FdnZ2VizZg0AYMOGDcjLy8PKlSs1\nHr9v3z489dRTahfZ6DOf4MGDwKhRfCqh+fOBffv49w8+4Bd3dOumvmgMIYRYWmfM6WwNOZtwwcF8\nho+aGuDRR3FrkKh9jAG5ucCtfoFG5eV8ir1ffwVuu019/8yZvB77mWeMDl3NmTNAQkLbAjZSKc1l\nTayTxeexXrJkidoDiFQvRdYgKCgIpaWl8u3S0lIEBwdrPX7MmDFoaWnB1atX4evrq/P8imSlIH36\n8Dk7335beVlzyvGEEHthDTmbcLILGJua9JsRBOCDRe11qgG+qmNJCS/70MRcFzAqUrxwEeB/l0NC\naC5rYn8cdB2wY8cOxMXFKX3t3LlT54mHDx+OoqIilJSUoKmpCZs3b0ZSUpLSMcXFxfI/AL///jsA\nGJWgZR1rBwc+Oh0UZPlZQVQ/DhMKisswFJdhKC7hs4acbWuMff3J6qz1nWrPENo61QCfcs/UuaxV\n26y6MA3A66zPnTPtcYTEHvOMPbbZVDpHrH/++We123bu3KlxKialEzs5YdWqVUhISIBUKsWMGTMQ\nFRWF9PR0AMCsWbOwdetWrF+/Hs7OznB3d0dGRoZRjVDsWAN86dbmZv5RFEAj1oQQ+2ENOZtwwcFA\naSng6anfVHvmYsiItVSq3/LninNYy9jiBYyE6KK1xnr16tX4+OOPUVxcjDDZZcgAamtrMXr0aHz9\n9dcdE6AetS5//smvkL52DfDwAD7/HPj2W6CoCDh9GnByUh7BJoSQjtCR9cbWlLMJ9+GHfMQ6MBCo\nqACWLu2Yx2WMd+TPnAHam9hFKuVLnz/zDPDYY+2f8733+N/g999vu23JEuDGDb6PEGthsRrradOm\n4d5778WCBQuQlpYmfxCxWCy4j/5UR6xdXHhdV3k535atQgXwhOLgwJc516PskBBCrII15WzCBQfz\ni+67djV/KUh7RKK2FRhjY7Uft2YNH7jaulV3x/ryZd4eRTExwEcfmR6vKRgDJk4EMjL0m86QEFNp\nrbH29PSERCJBRkYGQkNDIZFIIJFIBJmgNZWClJfzmUAcFFp482ZbJ1tWJmIsodYdUVyGobgMQ3EJ\nlzXlbFtj7OtPdvGiJWqsdRkwoP0664oK4I03+CqOv/zC575WpKnGWrUUZPRoIC9PeV0JY924Ydz9\nrl3jc3qbWlMO2Geescc2m0rnxYvWQNOIdWUl7zwrjkq7urZNBUSlIYQQQjqT7OLF6uqOrbEGdNdZ\nz58PPP44MG4cP3bv3vbPp+niRS8vIDwcuHWdq9F+/RW4/Xbj7nv+PP+uz1SGhJiDTXWsZd+dndv2\nqY5MX7/Ov5v6H3ScrvmOOgnFZRiKyzAUFyHqjH39BQTwkeHLlztnxFpbxzo3F8jJAd58k2/ffz+w\nY4fyMapt1tSxBoAxY3R3ynXZsgU4dUp5Gl19yWaQNEfH2h7zjD222VQ21bFWLAVRdeSI8jaNWBNC\nCOlMzs784sGCgo7vWMtqrFU1NwPPPgssXw64u/PbNHWs6+uV/46qzmMtM3YsX7RNUVmZ+m3aMAZ8\n9x0gFhtXznH+PP9kgEasSUexiY61061LMBVLQVTJPkaS1YmZOmIt1LojisswFJdhKC5C1Jny+gsO\n5qO9HV0KEhjI/w5euaJ8+4oVQK9ewIMPtt02ZAgfLT59mm+3tADDh+dgwQK+zVj7I9b79yt/evzW\nW8BLL+kX5/HjvKTz/vuBEyf0b5/M+fPAffeZp2Ntj3nGHttsKpvoWMs61LJ6aldX7cfKPkoyx8UU\nhBBCiClCQvj3jh6xlq3g+NZbbWs9XLzIp8ZbsUL5+iSRiHdOZaPWH3zAv69bx8sra2v5gJamWTf8\n/flFjbJO8bVrwP/+x0efa2t1x/ndd7yTP2iQcR3r0lJg1Cj+t7+y0vD7E2Iom+hYq5IlKk1kHWvF\nN/Q33/CPvQwh1LojisswFJdhKC5C1Jny+gsO5oNDHh7mi0dfX37JZ+2YP593rl95BZg9W/MS5LJy\nkGPH+BR62dlxiI8H1q7VPlotM3ZsW531hg3A+PHA8OH8okRFTU3q5R7btvHp8gYONH7EOjSU37+9\nUev6et3nMleeyc3l/1xYA8qthrOJjrXqfNQ9e2o/VvbmGTwY+OMP/vNLLwEvvmiZ2AghhBBtgoN5\nGYhDJ/w19vQEfvwR2LULmDyZd3Rfe03zsXffDRw+DDz6KPDvf/NykZde4qPb5eX6dawZAz75hHfe\nY2PVL2pct453uI8f59vnzvFR9FGj+Ii1MeUc58/zWNu7P2O8T3DwoOHnN8aOHXxAj9gmm+xYt5eg\nFK8qrqjQfH99CLXuiOIyDMVlGIqLEHWm1lh3dH21Ih8f4Oef+QWFq1ZpL6V0d+cd3D59+DR8OTk5\nuP12Xqu9Zo36HNaKZBcw/vorH5UeN47flpurfNyWLUBSEjBpElBTw+fQTkriExT06sXLSKqr9W+b\nVMo7/UFB7XesS0r4VLzbtrV/PnPlmeJi4OxZs5zK4ii3Gs4mOtaa6roYa/uvV5HifJq06i4hhJDO\nFBGhvmJhR+vena8AmZjY/nHr1gEbNyoPRr34IvDVV+2PWIeG8hrs+fP5aLVIBIwcCeTnt32KXFnJ\nR8Q//xyIjwdSU4Fvv+VlIAAfMFOdIpCxtgsqNbl0if/j0KVL+x3rPXuAqCjeke8IxcX8i/ogtknE\nTFkQvQPou2Z7S0vb7CAysuXLtdmxgy8l+/jjfJJ+YT8ThBBro2/+siX22GZTafr7ZS1aWvgiMI88\nAqSlaT/uscf4iPTFi20Xao4aBSxZAtx1F6/V/vFHXiLR1MRLRf78k3e4u3blxz/1FC8VmT2bb//6\nK78A8/x5Pie4qoMHgXnzeB15dXXbqLdqvyAlBbjzTuDtt/kqk/36GfYcHD/O/7HQFIMqxngJTlOT\n8nNBhMPUHGYTI9aA5qQkEgH33qv9PqWlvG5Mcbqhujrzx0YIIYRoY62daoDH/uGH/ILE9jz0EJ8f\nW7EjqVgOsmULr/MG+Oj21q3AZ5+1daoB9QsQMzL4J9br1ml+zNLStskMvL35SpB//618DGN8xPqu\nu3jZyfbtyvsvXwZaW9tv21tvAStXtn+MTGUlf86iotpWgjaFPjOrkI5lMx1rbXbu1P7xj+y/3pYW\n/n3qVD4JvT6EWndEcRmG4jIMxUWIOnt8/Sm2edIk4B//aP/4hx4Cli1Tvk12AWN1NXDgAJ/STyYw\nEEhOVj5eccq9lhY+s8aaNfxLU+dXduGijKaZQWSd2/Bw3rFWLAepqeGP+dVX6m1WdPo0r1PXR3Ex\nEBbGv9qrs/77b36hZ3uOHuXxWfIDInt8bZvK5jvW+P/2zjwuynr7459hGPZNFETBHUURGFAQTTNy\nLUNzublUlqZZll2tbla32225WdnNq5Yttmf6c8k2NTVLJW0RSsUlXFBBcQGVRQGHWb+/P47PLOwj\nM8wMc96v17zm2Z/zfebhy3nO8/meA9JX1YeUuH7NGvvbwjAMwzAMMHAgVUX+8kt6eyxVeqwLKeWe\nEFRyvUMHkqCEhgLbttXcvrpjXZvOeudOkpPIZBS1PnTI9Bb73/+mSPfKlXXbZDCQs3z0KFBc3HCb\nJce6a9f6I9bffAPMng0sX173Nr/+Sg74jaQhZOyHWzjW0qskW6YzctbcjmyXdbBd1sF2MUxN3PH+\ns0Wbg4KAnj1JSiHJQOojPJz+jxcWUiBsyhRa/tBDtTug5lIQoG7H+tZbadrbmyQtmzZROt61a4Ht\n24G9e0kPXVubz54licktt9C2DXHqFDnVDUWs9+0Dnn0WeOEF0p7Xxp49pO3esqXh894o7nhvNxW3\ncKwl/dqNpNVjGIZhGMY+DB5MuuP09Ia3lckoar13L0V0J06k5VOmkFb7/HnL7RuKWEuRb8mxBoA7\n76Rqj48+CixYQKn6xo6t+412bi7Qowc55I2Rg5hHrBtyrCdOJO351Km1R6UzM4H58+3rWDPW4xaO\ntaSbliQfDdEYvZKz6o7YLutgu6yD7WKYmrjj/WerNo8eTRlDGlt5Mj4eePNNcrClNIWBgeSEfvyx\n5bbVHetevciZLSmh+ePHAYUC6NLFtM2oUTQ2S68HHniAlt17L8lBamvz8eOUMlFyrBvyHxojBbl2\njeyMjaVsJQsXUkYUcy5eJG36ww+T1vrq1frPe6O4473dVNzCsQ4IsE7cr9HYzxaGYRiGYYhbb63p\nENdHXBxFp6sPbJw1i7KISAE0lYoyZpjn1/b2Bv7+dyA1lSLAkgzE/G1269bA3LkkLZHko7fcQo5s\nfn5Ne6SIda9eNKDyxIn67ZekIB07kqSlNn/j4EFyqr28aP7ee8mBv3DBtE1mJpCSQv7NgAGNk6Ew\nzYNdHeutW7eiZ8+e6N69OxbWkuBy1apVUCqVSEhIwMCBA3GwtooudqSu/JEqFf2hVU/LAwBaLaX3\ncVbdEdtlHWyXdbBdLRtn77OdFXe8/xzV5rg4qsRYXZPdpw85xTt30nxBAUW0q4+tWriQBiXeeiul\nyKutGW++CSQmmublcpKb5ObW3Dg3lyLWMlnDchCViqLlkZEUKY+MrN1Z37eP2iOhUAAjR5L2WyIz\nE+jfn6Zvv71pcpCff6asKIWFNde5+r1dUgLcc0/zntNujrVer8ecOXOwdetW5OTkYPXq1Thy5IjF\nNl27dsWuXbtw8OBBPP/885g1a5a9zAFgqa3q0cP0NFgd6TVRZWXNdWo1UFXVcF5LhmEYV8IZ+2yG\nqU5yMg0qrK3S47RpwOef03R1GYg5U6dSFpHg4Ibzb0vcey+walXN//3Hj5M/AdCxastOInHqFFWh\nlMtpvq4BjHv3WjrWQM0c25mZFHkHTI71jaTd++ILktF4eVGWkZbGsWNULbShNwm2xG6OdVZWFqKj\no9G5c2coFApMnjwZ31WrFzpgwAAEBwcDAFJTU3H27Fl7mQOAnnSl8ufHjtWdhs88T2Z1pD+qbdsy\nbG6fLXBWPRTbZR1sl3U4q12uhDP22a6CO95/jmqzlxflza6Nu+8GNm4kvXFBQd2ONQAkJVHu7MjI\nxp03IQHw8MjAb7+Zlul09Ga7a1eaHzaMBkPW5jsAJhmIRF0DGKtHrAHgttsoslxZSX5IVhbQrx+t\ni4mhJA3m5d4bw3/+Q1lHdu6kqO7vv9fcxtXvbamLas4Bnnar93Tu3Dl0MMtzExUVhczMzDq3//jj\njzHKPDu8GdOmTUPnzp0BACEhIUhMTDS+npB+9MbOe3pmXD9qGhQKADDNExnXnzjTkJEBlJTUfjxJ\nF2Xt+e09n52d7VT2VP+jdBZ7+Hrd2Dxfr/rnlyxZguzsbGN/5Uo4a5/tCvPZ2dlOZU9zzEs4iz1p\naWlo0waIj8/AK68A/v5p6NDBtscfOBB4//0M6HQ0n58PhIRkYM8emg8PB9q0ycB77wGPPVZz/5Mn\nAR+fDGRk0Hy3brQ+NtZ0vh9/zEBODhAfX3P/lBRgyZIMREYCYWFpCAszrR81Kg1btgCXLzeuPUpl\nGt54A1ixIgMXLwIDBqTh+edrbr9xYza0WmD48KZfP0fNh4cD33+fhsceq/vvt6ysDACQX5s2x1qE\nnVi/fr2YOXOmcf6LL74Qc+bMqXXbHTt2iF69eomSkpIa62xtYvv2QkiH7NmTpuv7VOfiRVp+5owQ\n164JUVhoU/MYhmlB2LGLtTnO2mczjDV8+60QgwYJ8cADQnzwgW2P/fPPQiQnm+Y3bxZi+HDLbd56\nS4iAACGGDRNi8WIhLl82rZszh5ZJfPmlEGPHWu7/559CxMfXfv4lS6hdH38sxN13W67buFGItLSa\n+5SWCnHXXULo9ZbLN28W4tZbTfPl5UL4+QlRVWVaplYLERgoxGuv1W6PK/D440L861/0m1RUNG6f\npvZhHk13zWsnMjISBQUFxvmCggJESblxzDh48CAefPBBbNiwAa1atbKXOUYkKQhgym9dHyqVaTAE\nYHrFo1JRmpuICNvaxzAM4wictc9mGGsYNYqknrt21S8FuRH69ydN9eXLNC+l2jPnsccon/ajjwIZ\nGcCDD5rW1SYFqZ5yrzYZiMTo0TSA8fffTfpqiaFDaRxZ9YDrqlVU2fLAAcvlv/1GlS8lAgJIK75/\nv2nZzz9TUZ4336SsKNaydy9JchzJ2bNA796kzTf35eyJ3Rzr5ORk5ObmIj8/HxqNBmvXrsWYMWMs\ntjlz5gzGjx+PlStXIjo62l6mWGCtY710KZU5lZBS+ezalVFr1hBHU/01nbPAdlkH22UdzmqXK+Gs\nfbYr4I73n7O2WaEgvfCJE5ZVF23Bb79lWFRYlFLtVScwkIrKrF5N1RElZ1XKYS0hDV40H3RYn2Pd\ntSsN2ly92pQRRMLXlwZlfvCBaZkQwIcfkmO5Y4fl9r/+Ctx0k+WyAQMsddYbNgC33pqB++4jLba1\nPPkkZV5xJGfPUnaYO+4Avv++ec5pN8fa09MTy5Ytw8iRIxEbG4tJkyahV69eWL58OZZfrz368ssv\no7S0FLNnz0ZSUhL6SUp8O9KmjbmNDW9/6ZLlvBSxVqsbX3CGYRjG2XHWPpthrGXaNPq2tWMNACNG\nmDJ/SKn26sLXF3jmGeDFF8lfyM+3LEYTHExJFMyjwfU51gBlB9FqAaWy5rqHH6ac4Go1ze/dSwM5\nX3jB0rHW6YA//iBH2hxzx1oIcqxvugn417+Ar76ybnBkeTlFxesqx95cSGXtpcI/N5I5xWqaJCRp\nBmxt4qVLQhw9StP9+zessZ4+3VJrfeIEzW/fLsSAAbXrsBmGYYRwT72xO7aZcT4yMuxz3GPHhIiK\nEsJgEKJzZyFyc+vfXqUSIjJSiK+/FqJdu5rrU1KE+O03mtZoSOd89Wrdx8vOJs10XQwbJsSqVTQ9\na5YQr7xCOu+gIDq+EKTj7t275r65udQ2IYQ4cECILl2onUKQNvz22+tvqzkbNwoxeLAQISFCnD/f\n+P1siVYrhEJBWnGDQYhOnYQ4fLjh/Zrah9ktYu2stGlDqWmAmhHr1aspBY85RUWW81LE+pFHakas\nz583PSkzDMMwDOMYbrnFPsft3p3yUB84QJUQG0oA5OMDPPssaa7NZSAS5in3jh6l6GpgYN3HUyqB\ndevqXj97NvDuu0BFBWmrp0+nwjldu1Lpc4AiydVlIADZV1VF8okNGyg6LlWlfOQRsnP+fFNEvD62\nbaMo8dCh9RfNsSdFRdR2Ly9qx6hRJAfR6cjfu/POulMjNgW3c6zNqe5Yh4RYljYFLF995OebnOlj\nxzKQlWW57Y4dpuT0jsJZdW9sl3WwXdbhrHYx7oE73n/u2maZjOQg771HgyMbIymdOZOccfOBixLd\nulGO5XnzyPEbObJpNo4ZQ77Kv/4F3Hwz0L49LR8yxCQHqT5wUUImM8lBJMda+p29vGhA6LFjNHCy\nIVnItm10nUaObLocRKu9scGTkgxE4o47gPffpyqT779Pv4uHHbxgdqxBFxugEqfVHWvzAYpdutQc\nwWuOVE2p+jEYhmEYhmkZjBgBrFxZ+8DF2vD2Jkdu4sSa6/r3B44cAUJDga1bgSVLmmabpydlIlm6\nlBxHCXPHuraBixIDBpCeOjeXHHNzwsOBb78F5syhNwL//W/tY81On6YK1kolXasff2xater33yf7\nrdVHSwMXJYYMoUGl69ZRxpPRo+3jWDu9GA6A2HB0g12OfdttNTXSy5bVrrXW6eh7xYqa+a8l1q41\nLdNq7WIywzAuhAt0sTYHgHhttwsnvmWYBiguFsLDQ4h58xxtSe2cOydEerqlH3LlCuVyzs0VIizM\npJ2uzs6d5MNMnlz/OU6eFOKWW2is2pEjlus++MAyz3ZMjBB7995IS4hx40grvWmTdfstXizEY49Z\nf76m9tsuEbE+UHSg4Y1ugOuVeS149NHat62svG7LAcuUfeZIEWuARsQyDMO4I2/8+gYqNBWONoNh\n7EJoKJCSUn9GEEfSvj2VdjeXqQQFAXFxlJP6ppvqfrOekkK+TLVMmzXo2pUi4PfeCwwaBHzzjWmd\nJAORMM+kYi0GA0WXX3sNeOONurfT6YAzZyyXVZeCNBcu4ViXa+zjpb73HnD4cN3rN240TQ8aRN+L\nFlFOyt69M4zrvv2Wvp3BsXZW3RvbZR1sl3U4q13uys2dbsaaw2scbUaz4Y73n7u3+YMPgMmTHWfL\njTBkCPDpp3XLQADA3x94/XUgPZ3m6/udPTwoGLltGzBrFmmz9XrK8z18uGm7+nTWV66QU/7QQ0Cn\nTlRgx5xDhyjpxNy55Djv2VP7cd54g+S85lSXgjQXLuFY2yvy0aoVJU6vjREjTDcWQD+uOeZaqXHj\ngK+/BiZNMi2bNw8YNsx2tjIMw7gKD/V9CB/s/aDhDRnGRUlIoMi1KzFkCKDR1D5w0Zx//KP+zCTV\n6dMHWLECGD+esm1ERpoGTQKkx/7zT8pUoteTxvvf/yY9d1QUBTl79gQ++YT212hM+2ZkAGlpFH1/\n8snao9anTlHQ8+JFy9ojjnKsZdf1JE6LTCbD1K+nYsW4Fc12zuRkqmA0d27dr0u2bAFuv73hYzn3\n1WUYxp7IZDI4eRdrc2QyGXR6Hbq+1RXfTvoWSe2SGt6JYRi7o1JRtPr33ykNoK356COKXD/+ODm6\n5gwZYkpT2K4d+U/Dh5OTb27LoEHAP/9JGVIAGmw4aRIwZQpJcrt0AXbvNqVNFoKCoIMGUcnyv//d\nFBTt2JEymTSUErE6Te23G5EoxvHYSwpSF1KuR4CeoH79lUbI7tplWu7n17hjCcFZQhiGcS/kHnLM\nTJqJ5XuX4/309x1tDsMwoPFhUnl1ezBzJkWkq2cTASj93+HDJKExrz5ZncmTgTVryLE2GMjveu89\nWufvT9KTRx8F3noLiI0lGUleHn2rVCQVSU8nOwoLLSPnzYVLSEHK1Y4bCTh9Oj2FVX/lc/RoRqP2\nV6lIF3TyJL1auXbN9jaa46y6N7bLOtgu63BWu9yZGX1mYN1f69xiEKM73n/cZvfA2jY/9BA5vNUZ\nMoSiyfU51QDwt7/R+DaViqLb4eEU4ZZ46ilg8GA63qhRpCx4913Ks92/v0mDXVREfpuXl1Xm2wSX\ncKydoWM2H5hoDVeukPZIqoRUXGxbuxiGYZyR9oHtcUvnW7D60GpHm8IwjIsQEQH07Qts3kz66uoD\nEv38SJ+dnw9MmECDHdPSaF1qKvDHHxStPnvWMRlBABfRWMe8HYOjc4461I7Jk4G1a2l06uXLJJJP\nTaVXEADw3HPAggU198vJAVatonLnn34KnDhRe1lThmFaHu6qsZba/NOpn/Dwpoex76F9CPIOcrBl\nDMO4Ah99RFlENBrg7rstE0M0RPfuJAs5doyK+JinAWwsTe23XSJifbHyBmpZ2hgpYi19h4UBBw+a\n1tf1ZBQbS/IPaaRqRQUJ7BmGYVo6w7oOw7CuwzBr4yy3e8BgGObGGD+eUvj9/DNlFLGG/v2BzEzH\nRqxdwrGu1FZCrVM71IbqjnVGRgYUCpr+v/8jXVB1nn2Wvi9dMtW5P3mStEH2wlk1YGyXdbBd1uGs\ndjHA4pGLcfTyUSzfu9zRptgNd7z/uM3ugSPaHBpKAyDbtydpiDVIOmtHpdoDXMSxbuvfFkWV9rYp\nLgAAIABJREFURQ614e67qRKRudZacqyHDwcCAmruk5xMP+zFiybHevt2+tZq7WsvwzCMM+Cr8MW6\nu9bh+Z3PY/8FO6YkYBimxTBvHvDww9bvJznWBQWOc6xdQmOd8kEK3r79baRGpTraHHTtSrpq6arJ\nZEBVFY089aj2mLJxI0Wt5XJK12eeEaS0lEqqv/EG8PTTzWc/wzDNh7trrM1Zc3gNntvxHPbM2IMw\n/zAHWMYwTEtHq6Xif127Au+8U3vqv4ZwC411+8D2OFd+ztFmAKiZHcRgALy9a89V7eVFznNRUc00\nexUVgFoNPPMMR68Zhmn5TI6bjEm9J2Hs2rGo0lU52hyGYVogCgWQmEjVslkKUg+RQZE4d9U5HGvP\n6yV1JN2RuUM9e7bltl5elJC9sLDmcY4epTKcAOVrrI8LF+h740Zg3776t3VWDRjbZR1sl3U4q12M\nJa8MeQWRgZF44LsHWlQk3x3vP26ze+CKbe7fn74dURwGcBHHOiowCmfLzzraDAD157N+912gvNzk\nCHt5AT/9ZFrftq1pevhwytUINOxYt29P1SDHjAHuu+/G7GYYhnE0HjIPfD72c+SV5eG5Hc+1KOea\nYRjnoH9/8re8vR1zfrs61lu3bkXPnj3RvXt3LFy4sMb6o0ePYsCAAfDx8cGi6oXlzegU0gmny07b\n09RGs2QJlddMkzKSVyMgwORAy+XAunWmddXr1Vddfxta1+uK/HxTFSGpsIxOV799ddnlaNgu62C7\nrMNZ7XI1bNVn14evwhffTf4Om45vwtytc6E36JtqtsNxx/uP2+weuGKbhwwBnnjCcee3m2Ot1+sx\nZ84cbN26FTk5OVi9ejWOHDlisU3r1q3x9ttv4x//+Ee9x4oOjcaJkhP2MtUqhg1reKSqJA+RyYCR\nI03LNZrat9fpSKtdnSlTgAEDLPfVu/7/IIZhnBBb9tkNEe4fjt3Td+PwxcOYuH4iVNoGXtsxDMM0\nktBQYP58x53fbo51VlYWoqOj0blzZygUCkyePBnfffedxTZhYWFITk6GQspbVwfRodHILcl1qteG\njdUdBQaapjUacpZr4+RJy/nSUssBj5Jj3VDE2ln1UGyXdbBd1uGsdrkStuyzG0OwTzC23LMFXnIv\nDFkxBKdKTzX5mI7CHe8/brN74I5tbiqe9jrwuXPn0MGs7E1UVBQyMzNv6FhPzH4ChgsGPHHlCXRq\n2wmJiYnG1xPSj97c8xJ1r6f5n382zbdqBcyalYFZswAfn7Tr0Wha36NHGjZsALZty8CAAcA996RJ\nZwAAqNU0X1mZgYyM2u1bsAD4v//LxqJFwG23Ofb6WHu9HDWfnZ3tVPbw9bqxeQlH27NkyRJkZ2ej\nc3Xdlwtgyz572rRpxmsQEhJSZ5/t7emNB0MfxJcXvkTqR6lYMGQBul/tDplM5vB7ypr57Oxsp7Kn\nOeYlnMUenrfPvLP2+bb++y0rKwMA5Ofno6nYLY/1V199ha1bt+LDDz8EAKxcuRKZmZl4++23a2z7\n0ksvISAgAE8++WRNA6/nE7xt5W14NOVRjI4ZbQ9zbY5MBuTkAL16kS778ceBqVOBFStM28THA4cP\n19w3IqL2TCLSurNnKV3f668Dc+YAkZGmcwLA+vXAhAm2bQ/DMNbjSnmsbd1nW0vOpRzc/+39aO3b\nGp+N/QwRAVaWXGMYhrEBTpvHOjIyEgUFBcb5goICRDUhqWBC2wQcunjIFqY1C4WF5FQDVEFo+3Zg\n8WLLbby86t63LnQ64NNPgZAQcqw3bKi5TW16bYZhmPqwdZ9tLbFhsfjtgd+QGpWKpOVJ2Jy7udnO\nzTAMYyvs5lgnJycjNzcX+fn50Gg0WLt2LcaMGVPrto15MogLj3Mqx7r667DqmKfWA2iUauvWlsuU\nypr7bd5cf6Ugvd6UTQQg5zwvjz7XLYMQFL0uKanXxGaloevlKNgu62C7Wi627rNvBIVcgZfSXsLa\nv63Fw5sextytc1GuLrfLuWyJO95/3Gb3wB3b3FTs5lh7enpi2bJlGDlyJGJjYzFp0iT06tULy5cv\nx/LlywEAhYWF6NChAxYvXoxXXnkFHTt2REVFRa3HS4pIwr4LDVRHcTGWLwe6dwcefdS07PbbgV27\ngL/+qn2f0lLLgY4zZwIjRgDvv29aJq2/LhliGIZpEFv32U1hcKfByH44G1eqriBmWQw+3vdxi0jL\nxzBMy8duGmtbIWld9AY9Wi1shfx5+Qj1DXW0WTZnwgTg668B6deorKSc2FFRpKmWiI4mOUh1ff20\nacBnn1ku278fOH6c8jmedY76OgzjVriSxtpW2LrNf5z7A4//8DhUOhXW/m0tokOjbXZshmGY6jit\nxtrWyD3kiA2LRc6lHEebYheq66L9/em7spK+T1xP4z1woKVTHXF9fE91pxqgiPXatcC569XgFy0C\ntm1rnD16PXDIeZQ3DMO4KSmRKdg9fTdmJs3EwE8G4ocTPzjaJIZhmDpxGccaAHq26Yljl4852gwA\nttcdvfYasHKl5TIfHyAmhqbbtaPve+81rQ8IoAGM1SwzTpWVmcqrb9oE/OMfjU+avmYNkJDQWOsb\nxll1WmyXdbBdjCOQyWSYnTIbX038CtO/m47Xf3ndqbTX7nj/cZvdA3dsc1NxKce6X2Q//HjqR0eb\nYRd69gTuucdymUoFDB5M035+wH/+Q4MgVSrgwQeBJ58EevQA+vWz3E+q9rhjB3D5Mk2Pvp6l0LzA\nzJUrwIEDlFnkwgXgerpKtGkDfPGFbdvHMAzTVAZ1HITMmZn45cwvaP+/9hj+xXAs3bMU566ec7Rp\nDMMwAFxIYw0AZVVl6LSkE/Lm5rVInXVt/Oc/wL//bdJe18Xy5aZS60JQdHrRotq3zcgArl4FzAf8\n//OfJDdZu5YyioSEUMTbue8OhnFuWGNtP8rV5fjp1E/YcHwDvjv6HZQRSkyJm4K7Yu9CK99Wdj8/\nwzAtk6b2YS7lWAPAXV/ehRFdR+DBvg860KrmQ6WiDCHJyfVvV1oKhIZSFDooiDTbe/YAu3cDHTsC\nd99d//4pKVR05rXXgLFjKV1gURF9wsMtt9XpALmcHPDcXKB9e5MmnGEYE+xYNw9VuipsPbEVqw6t\nwraT2zCs6zDcG38v7uhxB7zkdRQMYBiGqQW3GbwoMbn3ZKz5a42jzWg23ZGvb8NONQAEB9N3VlYG\nAMDDA7jpJuDpp4Fx40zbSYMcH37YVLERAP74AzhyhJxqgBx1gBzsqiqgTx9Aqm6sUNAxhSApyvPP\nN2yfs+q02C7rYLsYZ8TH0wdje47Fl3d9idPzTmNU9CgsyVyCyP9FYu7Wudh3YR8Mwn6Vs9zx/uM2\nuwfu2Oam4nKO9e3db8dfF//C9lPbHW2KU+HhQY6up2fNdT4+QFoaTY8ZA9xyC/DOOzSgUUKqEimh\n0ZimfX0pdd/GjaZl331nKse+fXvtpdkZhmGamxCfEMzoMwM/T/sZmTMzEeITgolfToTvAl9EvxWN\nYSuGYf6P87Hp+CaUqkodbS7DMC0Ml5OCAMCGYxvwxA9PYNX4VUiNSnWQZa7FN98AO3cCb71lWnbm\nDNCpExWkadvWlIGkLh58kPTeHTrQ/Pvvm3Tdzz0HvPKKfWxnGFeFpSDOg0qrwpkrZ3Cq9BSyzmVh\n95ndyDyXiXD/cMSGxaJ3WG+08WsDb7k3vORe8FP4IdA7EAFeAegd1hvtAts5ugkMwzQDbqexlng7\n8238c8c/8dcjf6FjcEcHWOb6XLtG2miDgfTS+flAly5UJt08Yj18OPBjA8lYpk0D0tMpMi6Vbq+q\nonzYrL9m3BVndTLtiSu1WWfQ4WTJSeRcykHOpRyUVJVAq9dCrVfjmvYaytXluKq+iv2F+9GjdQ/c\nGXMn+rbri2CfYAR7B6N9YHsE+wQ7uhkMw9gQt3WsAWDBrgX472//xcJhCzGr7yzIZLJmsysjIwNp\nkr7CiWiqXWfOkDP8+eckD7njDsoi8tFHFNH+6Sdgn1ll+d69gaNHKUVfUREty8ykipEPPUTp/GbM\nAAYOzMDAgWnw9W1S82xOS/0d7QXbZR2u5GTaipbYZo1eg12nd2HDsQ04evkoyqrKcEV9BeeunoOf\nwg9tL7VF1z5dodapUaWrgr+XP2JaxyCmdQw6BHeAt9wb3p7eaO3bGt1bd4enRy2aPRfDWf/m7Am3\n2T1oah/m0n/dzw1+DgltE/D4D49j15ld6BHaA8oIJdJ7pLeIjssRdLwe/H/xRdOy9HT6SDz9NPDG\nG+RM//EHSUzuuIPyah8/DqRWU+e8+CIwfjxFvqOigPfeA5RK4NgxytPtxYP2GYZxYrzkXhjWdRiG\ndR1msVwIgQsVF7Du+3XonNjZ6ECXq8txrPgYss5n4dtj30KtU0OtV6Ooogjny8+jZ5ueiGkTg2Dv\nYAR6ByLQKxA+nj7w8fSBXCZHuYYi5VW6KgR6ByLEOwQBXgFQ68lx1+q1RqlKoBfJVfy9/BHgFYBg\n72C08m2FIO8g/j/IMA7ApSPWEkUVRXjnj3dw+dplHCg6gDNXzmD+TfMxO2U2dyx2QK8H1GoqWiOx\ncSMNitTrKUq9dy/l0q6LqCjg7FmSoLz2GjB5MtC5M8lTLl+mdQMG2L0pDGNXWmL0tiHcsc3WUKGp\nwOGLh3Gi5ATK1eUo15SjQlOBKl0VqnRV0Bl0CPIOMjrb5ZpylFWVoUJTAW9Pb/h6+sLTw5OkKppy\nlKvLUamtRKWmEuWaclypumKMqPt4+iDQKxCB3oHw9fSFr8IXvp6+CPMPQ1v/togIiEBr39YI9Q1F\nK99WCPUNNX4CvQIh95A7+nIxTLPj1lKQuth3YR+e+vEpZJ3LQmRgJCbETkDfdn2hbKtEt9BuEEIY\nZSNCCOiFnh1wO/Dnn0B8PFV9NNdoDxlCVSElZDJTIZqPPgJmzjQt/9e/KDd3URHw6qtA1660Tq0m\n57tbt+ZpC8PcCO7oZLpjm50RgzBY6MSrdFVQ6VS4pr2Gy9cuo7CiEIUVhShRlaBEVYLSqlLjdImq\nhBx5uTf8vfwhl8lhEAbohR4yyOAh84DcQw5PD0/jx8fTBwFeARZR81Y+rRDgFQBPD08oPBTwknvB\nx9MHvgpfeMu94enhCbmHHB4yDxiEwXjfeHt6w1vuDT+FH8L9wxHuH44ArwCbyD3N///bC71BD61B\nC61eC29Pb86l7mKwY10PxdeKkV+Wj7V/rcWx4mPYfXo3rqqvAgAigyIRERCBvef3QiFXQNlWCa1B\ni4uVFxHoFYghXYYgrXMaokOjofBQIDYs1uKP0Vl1R85q17ZtGVi8OA0dOlA2katXAa0WKCigkuxp\nacC5c6TXDgoCystNzrbkeAcFAUlJQHExEBcHrDFLZ/7vfwPPPEOpAQ0G0ohPn25aLwRpv/v3t7TL\nWa8X22UdzmqXOzqZ7thmZ73/moIQAiqdChWaChiEAR4yD3jIKEOv3qDH7l27kTowFTqDDlqDFmqd\nGhWaCmPUvLSqFKWqUlRoK6A36KEz6KDWq6HSqqDSqaDWqaEXtNz8+EIIaPQaqPVqVGoqcenaJVys\nvAi1Tg1/L3/4KfzgJffCNe01VGoqodKpas1RLoPM6LTLZXLoDDroDDoICHjIPOAl94KX3Au+nr7w\n9/KHv8IfCrmCnH2ZHHIPufFbandJTgnkXeUoVhWj+FoxVDoV9AY99EJf49wKuQIKDwXUejW85d4I\n9Q2Fj6cP9EIPgzBA4aFAiE8IQnxC4OPpA61Ba7RR+kjtkkEGgzBAo9fU+OgMOmNbvOReRrs9ZB4W\n2/gp/EwPPj7BCPEJQZB3EOQyeishk8nQyqcV2vi1QRu/NvCQeUBv0ONw1mEkpCYYHxAkuZO33Bty\nDzlkkEEmk0GG68FKCAghLB7EJJuCvIMQERABb09vu923tsCtNdYN0dqvNVr7tUbf9n0BAJWaSshk\nMnh6eCKvNA/nys+hX2Q/VGoqcejiIQR5B6G1b2tcunYJ3+d+j8+yP8OZK2dQrCpGx+COaOPXBleq\nrkCtVyPsYliL60jtiZcXsGWLaT4khL7DwihHdlwccPo0sHIl8MQTlJUkKwuIiKDiN8XFNAjy559p\nv8OHgQceAD75hOZffpmc8oQE2m/DBhpYWVhImU4yMyldYGUlOfSnTwPnz1OOb4ZhGMYSmUwGP4Uf\n/BR+ta5v49cGHYI7NJs9ah1larmmvQa1Xg0/hR/8Ff7wVfjCQ+ZhdPCEEBAQxgi45MjKZXIo5Aqj\nwyg579IxKzWV0Bq0xocAvdAbnWbJyTogP4BBgwehjV8btPZtDV+Fr9H5lhxLABZBOCEEKjQVKFGV\noEpXZXQyNXoNrqhJtqPSqoyOuNxDDoUHOfgeMg8I0LllkFk40JKjK5fJoTVoqT3XH1b0BmqztJ1c\nJodKpzJKj65UXcEV9RVcVV81Ou8GYUBpVSkOFh1EsarYeM0uF1zGkYAjxusljReQziVdb3OkhyTp\nQUmyqayqjIKX3oFo49cGIT4hCPYOho+nj8U1k64pAIvfwyAMxt9Eq9caHz6ka+Lp4Wl80NMbTA87\nAoKW67UWD0E+nj7InJlpq1vU9Pu35Ii1rVDr1NiRtwNqvRqBXoHG11PdQlmH0NzodCQryckB5s6l\n6VGjgKeeotLvmzdbd7zp06k8+6FD5MwHBgLPPkuR78pKOnZtGAxUlMec4mKgVauayxn3xRn6r+bG\nHdvMMEzjMAgDLl+7jOJrxbiivoIrVVdQpasyrhcQxgcDAVHjDYIkQ1J4KKCQKyCDzPjGRGfQWbx1\nMHfWpYcV84cgmUyGhLYJNWxkKQjDmJGXR06xRkPSkEOHgIkTSXIipQX85hvaNjKSHOcPPwRGjCAt\n+GOPWR5v/HjgrruAr76i9INDhtDyceOAqVOp9HtCApWdl9YVFlLBHYAkKNXlfDpd7RUymZaHO/Zf\n7thmhmFaDuxYOwhn1dSxXQ0jBPD77yQROXYsA337Un5tT0/KaOLrC/ToQWXfi4uBPXssZSzmDBhA\nx5Jo1w64cAG4+WaTE//yy5QF5fnnyZnPygIWLADuvx9Yu5amP/kEKCszSVcOHMhAnz5pmDCBouMV\nFYBKBYSGmpxyrdaUqlCjIQdeobDvtXOm39EcZ7XLWfsve+KObXbW+8+ecJvdA3dsM2usHUR2drZT\n3mxsV8PIZMBNN9H02rWWdimVpu0efdQ0rdVSxpLERIo4h4SQLlyKRr/5JjBrFjnlp0+Tbjw/nxze\nr74ihz0ykjKZTJoELF1KZeABco7nzgXCwynCXlUFANkA0tCpEznqGg1JTAwGct779QO++45yhO/Z\nA2zdSpKW2bMpil5cTMuGDiWbUlLoXM89B2Rn07FCQ+mcOh3g7U0R/U6dSNNeWEgpEwcMoHVFRRSF\nr/471haRdwTOdH8x7oc73n/cZvfAHdvcVOzqWG/duhXz5s2DXq/HzJkz8fTTT9fY5u9//zu2bNkC\nPz8/fPbZZ0hKSrKnSTajrKzM0SbUCttlHY21S6Egp7UuzHN2R0fTByDttxCm8vESQlAFy/BwoIPZ\n+B+1GjhyBHjrrTIkJ5NTPWECRam1Wsp2kp9PzvK0aeQAnzwJjBlDAzZlMop8K5Ukf1EoSB5jMJD+\n/PPPgVOn6Fy+vhQFNyc4mIoEHTpE897e9LlKyXQQG1uGI0co5/ju3RTJ79uXbNPp6AGgqooeXM6f\nB375hWQzPj60XKkEtm0Dunen1Ilt29K6bt3ooUUIuk7nz9O6vDzaLyaG2lJVZZL6VFXRQ4mvL/2O\nR4/SdWeZzY3Tkvtse+Ks/Zs94Ta7B+7Y5qZit39Ber0ec+bMwU8//YTIyEikpKRgzJgx6NWrl3Gb\nzZs348SJE8jNzUVmZiZmz56NPXv22MskhnEIMpmlUy0t69u35rbe3uRgduwIPPJIzfX1+TDVo8ev\nv265vrCQotXJyeRQR0bSsqoqyr6iVtOAzR9/JAc1Lo6i3/n5VPgnKgp4/HGSvly6RKkLIyOBYcPo\nASAjA+jThxxlX19ytkNCyElOTKTj+PlRGy9doqh+ly50zitXKKqv0Zjs7dCBHkh0Olov0bkzFRHy\n8KB2+PpSu196iRz28HB6MxATQw8UlZW0T0QEOfrR0bSvlxcQEEDR/dOnyXHXaml9ZCTZ6OVF19XX\nl2w1f4vR0uA+m2EYpunYzbHOyspCdHQ0OnfuDACYPHkyvvvuO4tOesOGDbj//vsBAKmpqSgrK0NR\nURHaSiO/nJj8/HxHm1ArbJd1tCS7GpJkREQAt91muax9e9O0nx9lNTHP/92qlakoDx0jH1u31n78\nefNqLnviifptMkcI0pLn55Pc5a+/qOS9TgdcvGiKnmdlkdMcFUXbl5UBjz6ajzVrSP4SGEjrDh6k\n7Vu1As6cIed51Chy4MPCKHquVtPDio+PSbN++DDtFx8PHD9uipC3adP4trgiLb3PtifO2o/YE26z\ne+CObW4ywk58+eWXYubMmcb5L774QsyZM8dim/T0dPHrr78a54cOHSr+/PNPi20A8Ic//OGPy35c\nBe6z+cMf/vCHPk3BbhHrxpYMFdVGXlbfr/p6hmEYxvZwn80wDNN07FbKIjIyEgUFBcb5goICREVF\n1bvN2bNnERkZaS+TGIZhmDrgPpthGKbp2M2xTk5ORm5uLvLz86HRaLB27VqMGTPGYpsxY8ZgxYoV\nAIA9e/YgJCTE7bV6DMMwjoD7bIZhmKZjNymIp6cnli1bhpEjR0Kv12PGjBno1asXli9fDgB46KGH\nMGrUKGzevBnR0dHw9/fHp59+ai9zGIZhmHrgPpthGMYGNEmhbWe2bNkiYmJiRHR0tHj99deb9dzT\np08X4eHhIi4uzrisuLhYDBs2THTv3l0MHz5clJaWGte9+uqrIjo6WsTExIgffvjBbnadOXNGpKWl\nidjYWNG7d2+xdOlSp7BNpVKJfv36CaVSKXr16iWeeeYZp7BLQqfTicTERJGenu40dnXq1EnEx8eL\nxMREkZKS4jR2lZaWigkTJoiePXuKXr16iT179jjcrqNHj4rExETjJygoSCxdutThdknniY2NFXFx\ncWLKlCmiqqrKKexyBI7ss5uLG+mDWwrW9KMtAWv7QlfH2r7MFWkO385pHWudTie6desm8vLyhEaj\nEUqlUuTk5DTb+Xft2iX27dtncfGfeuopsXDhQiGEEK+//rp4+umnhRBC/PXXX0KpVAqNRiPy8vJE\nt27dhF6vt4tdFy5cEPv37xdCCFFeXi569OghcnJynMK2yspKIYQQWq1WpKamit27dzuFXUIIsWjR\nInH33XeL0aNHCyGc47fs3LmzKC4utljmDHbdd9994uOPPxZC0G9ZVlbmFHZJ6PV6ERERIc6cOeNw\nu/Ly8kSXLl1EVVWVEEKIiRMnis8++8zhdjkCR/fZzYW1fXBLorH9aEvBmr7Q1bG2L3NVmsO3c1rH\n+rfffhMjR440zr/22mvitddea1Yb8vLyLC5+TEyMKCwsFEJQ5xoTEyOEoCca8+jMyJEjxe+//94s\nNt55553ixx9/dCrbKisrRXJysjh8+LBT2FVQUCCGDh0qduzYYYy0OINdnTt3FpcvX7ZY5mi7ysrK\nRJcuXWosd7Rd5vzwww9i0KBBTmFXcXGx6NGjhygpKRFarVakp6eLbdu2OdwuR+AMfbYjaKgPbilY\n04+2BKztC10da/syV8bevp3dBi82lXPnzqGDWa3nqKgonDt3zoEWwaIQQtu2bVFUVAQAOH/+vMXo\n+eayNT8/H/v370dqaqpT2GYwGJCYmIi2bdvi1ltvRe/evZ3Crscffxz//e9/4eFhut2dwS6ZTIZh\nw4YhOTkZH374oVPYlZeXh7CwMEyfPh19+vTBgw8+iMrKSofbZc6aNWswZcoUAI6/XqGhoXjyySfR\nsWNHtG/fHiEhIRg+fLjD7XIEzthn25vG9MEtBWv60ZaAtX2hq2NtX9aSsHV/7bSOdWNzqjoKmUxW\nr432tr+iogITJkzA0qVLERgY6BS2eXh4IDs7G2fPnsWuXbuwc+dOh9u1adMmhIeHIykpqc78uo66\nXr/++iv279+PLVu24J133sHu3bsdbpdOp8O+ffvwyCOPYN++ffD398fr1WqjO/Le12g02LhxI+66\n665az9vcdp08eRJLlixBfn4+zp8/j4qKCqxcudLhdjmCltKOxtKUPtjVsEU/6mrYoi90JWzRl7UE\nbNFfO61j3Zicqs1N27ZtUVhYCAC4cOECwsPDATR/bletVosJEyZg6tSpGDt2rFPZBgDBwcG44447\nsHfvXofb9dtvv2HDhg3o0qULpkyZgh07dmDq1KkOtwsA2rVrBwAICwvDuHHjkJWV5XC7oqKiEBUV\nhZSUFADA3/72N+zbtw8REREOv14AsGXLFvTt2xdhYWEAHH/f//nnn7jpppvQunVreHp6Yvz48fj9\n99+d5no1J87YZ9sLa/rgloC1/WhLwNq+0NWxti9rSdj6/4jTOtaNyana3IwZMwaff/45AODzzz83\ndqhjxozBmjVroNFokJeXh9zcXPTr188uNgghMGPGDMTGxmLevHlOY9vly5dRVlYGAFCpVPjxxx+R\nlJTkcLteffVVFBQUIC8vD2vWrMGQIUPwxRdfONyua9euoby8HABQWVmJbdu2IT4+3uF2RUREoEOH\nDjh+/DgA4KeffkLv3r0xevRoh9/7ALB69WqjDEQ6vyPt6tmzJ/bs2QOVSgUhBH766SfExsY6zfVq\nTpyxz7YH1vbBLQFr+9GWgLV9oatjbV/WkrD5/xGbqcHtwObNm0WPHj1Et27dxKuvvtqs5548ebJo\n166dUCgUIioqSnzyySeiuLhYDB06tNaULAsWLBDdunUTMTExYuvWrXaza/fu3UImkwmlUmlMPbZl\nyxaH23bw4EGRlJQklEqliI+PF2+88YYQQjjcLnMyMjKMo9kdbdepU6eEUqkUSqVS9O7HR73rAAAG\n+ElEQVTd23h/O9ouIYTIzs4WycnJIiEhQYwbN06UlZU5hV0VFRWidevW4urVq8ZlzmDXwoULjSmq\n7rvvPqHRaJzCLkfgyD67ubiRPrgl0dh+tCVgbV/o6ljbl7kizeHbyYSoQzDFMAzDMAzDMEyjcVop\nCMMwDMMwDMO4EuxYMwzDMAzDMIwNYMeaYRiGYRiGYWwAO9YMwzAMwzAMYwPYsWYcxpdffonY2FgM\nHToUADBlyhQolUosXbrU6mMNHDgQAHD69GmsXr3apnYyDMMwBPfbDFM/nBWEcRi33XYbnn/+eQwc\nOBCFhYW4+eabkZub26RjZmRkYNGiRdi4caONrGQYhmEkuN9mmPrhiDVjd1auXInU1FQkJSXh4Ycf\nhsFgwMsvv4xff/0VM2bMwPz58zFy5EicO3cOSUlJ+OWXX3Dy5EncfvvtSE5OxuDBg3Hs2DEAQFFR\nEcaNG4fExEQkJiZiz549AICAgAAAwDPPPIPdu3cjKSkJS5YswYABA5CTk2O0JS0tDfv27UNlZSUe\neOABpKamok+fPtiwYUMNuzMyMjB48GCkp6ejZ8+emD17trGc7yOPPIKUlBTExcXhxRdfNO7zzDPP\noHfv3lAqlZg/fz4AivDEx8cjMTERt9xyCwBAr9fjqaeeQr9+/aBUKvHBBx8AoKpPgwcPRlJSEuLj\n4/HLL7/Y+NdgGIZpGO63ud9mbhDbp99mGBM5OTli9OjRQqfTCSGEmD17tlixYoUQQoi0tDSxd+9e\nIYQQ+fn5Ii4uzrjfkCFDRG5urhBCiD179oghQ4YIIYSYOHGiWLp0qRBCCL1eL65cuSKEECIgIEAI\nQcUL0tPTjcdZvHixeOGFF4QQQpw/f17ExMQIIYR49tlnxcqVK4UQQpSWlooePXqIyspKC9t37twp\nfHx8RF5entDr9WL48OFi/fr1QgghSkpKhBBC6HQ6kZaWJg4ePCguX75sPL4QwmhbfHy8OH/+vMWy\n5cuXi1deeUUIIURVVZVITk4WeXl5YtGiRWLBggVCCCEMBoMoLy+34mozDMM0He63ud9mbhxPRzv2\nTMtm+/bt2Lt3L5KTkwFQufOIiAjjenE9kiDMFEkVFRX4/fffcddddxmXaTQaAMDOnTuxcuVKAICH\nhweCgoIszieqKZsmTpyIESNG4MUXX8S6deuMx9y2bRs2btyIN998EwCgVqtRUFCAmJgYi/379euH\nzp07AyAt4S+//IIJEyZg7dq1+PDDD6HT6XDhwgUcOXIEsbGx8PHxwYwZM5Ceno709HQApCO8//77\nMXHiRIwfP954/kOHDmH9+vUAgKtXr+LEiRNISUnBAw88AK1Wi7Fjx0KpVFp1vRmGYZoK99vcbzM3\nDjvWjN25//778eqrr9a6TiaT1VhmMBgQEhKC/fv317pP9U64Ptq3b4/WrVvj0KFDWLduHZYvX25c\n9/XXX6N79+717m9unxACMpkM+fn5WLRoEf78808EBwdj+vTpUKlUkMvlyMrKwvbt27F+/XosW7YM\n27dvx3vvvYesrCx8//336Nu3L/bu3QsAWLZsGYYPH17jnLt378amTZswbdo0PPHEE5g6dWqj28sw\nDGMLuN/mfpu5MVhjzdiVoUOHYv369bh06RIAoKSkBGfOnKl3n6CgIHTp0sUYFRBC4ODBg8bjvffe\newBI73b16lWLfQMDA1FeXm6xbNKkSVi4cCGuXr2KuLg4AMDIkSPx1ltvGbep659BVlYW8vPzYTAY\nsG7dOtx88824evUq/P39ERQUhKKiImzZsgUymQyVlZUoKyvD7bffjv/97384cOAAAODkyZPo168f\nXnrpJYSFhaGgoAAjR47Eu+++C51OBwA4fvw4rl27hjNnziAsLAwzZ87EzJkz67SLYRjGXnC/zf02\nc+OwY83YlV69euGVV17BiBEjoFQqMWLECBQWFta6rXmUYdWqVfj444+RmJiIuLg44yCVpUuXYufO\nnUhISEBycjKOHDlisa9SqYRcLkdiYqIx/ZP0CnDixInG4z///PPQarVISEhAXFwcXnjhhVrtSUlJ\nwZw5cxAbG4uuXbti3LhxSEhIQFJSEnr27Il77rkHgwYNAgCUl5dj9OjRUCqVuPnmm7F48WIAwPz5\n85GQkID4+HgMHDgQSqUSM2fORGxsLPr06YP4+HjMnj0bOp0OGRkZSExMRJ8+fbBu3TrMnTu3qT8B\nwzCMVXC/zf02c+Nwuj2GqQNOAcUwDONacL/NOBqOWDNMHchkslq1hAzDMIxzwv0242g4Ys0wDMMw\nDMMwNoAj1gzDMAzDMAxjA9ixZhiGYRiGYRgbwI41wzAMwzAMw9gAdqwZhmEYhmEYxgawY80wDMMw\nDMMwNoAda4ZhGIZhGIaxAf8POgwi420Z4eAAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0xcb4cfd0>"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Interestingly, the training errors of SGD solver are lower than BMRM's in first 100 passes, but in the end the BMRM solver obtains a better training performance. A probable explanation is that BMRM uses very limited number of cutting planes at beginning, which form a poor approximation of the objective function. As the number of cutting planes increasing, we got a tighter piecewise lower bound, thus improve the performance. In addition, we would like to show the pairwise weights, which may learn important co-occurrances of letters. The hinton diagram is a wonderful tool for visualizing 2D data, in which positive and negative values are represented by white and black squares, respectively, and the size of each square represents the magnitude of each value. In our case, a smaller number i.e. a large black square indicates the two letters tend to coincide.  "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def hinton(matrix, max_weight=None, ax=None):\n",
      "    \"\"\"Draw Hinton diagram for visualizing a weight matrix.\"\"\"\n",
      "    ax = ax if ax is not None else plt.gca()\n",
      "\n",
      "    if not max_weight:\n",
      "        max_weight = 2**np.ceil(np.log(np.abs(matrix).max())/np.log(2))\n",
      "\n",
      "    ax.patch.set_facecolor('gray')\n",
      "    ax.set_aspect('equal', 'box')\n",
      "    ax.xaxis.set_major_locator(plt.NullLocator())\n",
      "    ax.yaxis.set_major_locator(plt.NullLocator())\n",
      "\n",
      "    for (x,y),w in np.ndenumerate(matrix):\n",
      "        color = 'white' if w > 0 else 'black'\n",
      "        size = np.sqrt(np.abs(w))\n",
      "        rect = plt.Rectangle([x - size / 2, y - size / 2], size, size,\n",
      "                             facecolor=color, edgecolor=color)\n",
      "        ax.add_patch(rect)\n",
      "\n",
      "    ax.autoscale_view()\n",
      "    ax.invert_yaxis()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 31
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# get pairwise parameters, also accessible from\n",
      "# w[n_dims*n_stats:n_dims*n_stats+n_stats*n_stats]\n",
      "model.w_to_fparams(w_sgd) # update factor parameters\n",
      "w_p = ftype_all[1].get_w()\n",
      "w_p = np.reshape(w_p,(n_stats,n_stats))\n",
      "hinton(w_p)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAO0AAADtCAYAAABTTfKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEx5JREFUeJztncuS2zAORdFTU518sisbd2+68sntbDKLjBJFkYhL4Qoi\n5Hs2adsMCL5skXjw5efPnz9NCFGG/5ytgBCiDy1aIYqhRStEMbRohSiGFq0Qxfjv1gcvLy+Zeggh\nZrSMOpuLdus/vr+///777e1t9f/N318rs/bex8eH/fjxw15fX+3bt29/fXa/31frmesyl2FmsJyl\njNvtZl++fDEzs8fjYd+/f++WgYD0i/ca0WXeJ2b7+8XrW7M/fbfWb2v6r7231GWpxx4Za3K8cd6r\ny/L9Pbp8/fp1Ve7EMI/H04SYT7C9MiJyps5b/r2H2+1mt9stJCPKsh/29gvSt1N/RfstA+Y4R+nV\nZZhF+/r6+te/Z/F4PFb/Fj5Tf6nf+uidc83H45Zws18LbPmtu1x0yzJbi3LtcWte7/IbaK1x87rW\n6lnKWZOx9pjUKwORhfQLoz1IPb1ytsbQ67teXY5sj1cPWsaDMeeWvGy5Mb68vGw+swsh/j0vMNve\n7/fw9va2/yBKCLFNdHHuZZg9rRACQ4tWiGI0H48RWxZiv2OA6IKUqcJR+6UzyZorrLpGnU/NRYvY\njxD7nWd0n8qYbZ+kIbogZVBdvDIfHx9m1t7X3O/3puNFq561vlx7j9meLeeWuRyz9mlnqwxqR/fq\nQRYTY16i9lNU38j8n5NyEDWS0R3RJUtfRj2s9iDOLSOMnxnPMaLSOM9pLlrEToXY7yY5e22aKD22\nt5YuSBnkcctzb0Tq8WC1ZxrHlnMLoidqm27V02MrP3JeonZaVF/W/C9jpx11f3EUV9zTCozL2Gmv\nvkiXaHGKLWTyEaIYWrRCFKPM43EWSPypOA+Nj7NoewKfzdYPiNBORmxZGbDiTz2QfmEEniNlWPVk\ngI7PKPoeQXPR9gQ+L//e+n+RAGqWETs6mKgM1NFg7fXyvUi/eWVY9bD6v8fRYAvGfELKnNFm7WmF\nKEbzl7bHQD393ZKByIkYoJFvKcajEirj6CB4M45zRaaDTNYYZel7RpvLOFdkoYOOsXmG8bmMc0UW\nV5sAV0Pjoz2tEOXQohWiGOEgeAZZ+5Rn2A+J6xMOgkfw7FSoLdcLPGfVw3BG8MjMxMFyNIjaT1lt\nZgb1V3TAoDwee5n0WUHAP378gIK0o/UwnBHMtq8zWcqOZOIw4/T/x8fH7y/Flhyvb1u6sNrMDuqP\n9C1apjUXUBkTKXtaVub519dXKEg7Wo8y5a/zeDwO7xMk2z4yPsiNFVXHuWmn/fz81J7WkfNMj8dR\nni2RwV48O62cK4QYDG/RyuQjRDG0aIUohhatEMUI+R7PDxbM9gfBI2UYBzeIvh4MGWa52fYZMA6R\nWIH/Pbpu6cs4lOydt1u69BLyiFrat/YGwSNlGDY+RF9Gm818435PBvwtXZB60DKMbPs9bV57vXyv\n5WTTWtiseen1f++83dJlWY93ANx8PGZ5RFWC1WbEuM/QBamH5WjgwWhzTz1H15U1/3vraS5axNCd\nRZYurHo84/78/VbguacL4kSQ5WiA1MMA6TsGo8655uNxz/UMWxX2ZmjYKoPsA7wsGoi+jDab+XGf\nyP4JaTMih1EXckWGVw8rW4dXD2teen2CyGDMuSVyrhBiMORcIcTF0KIVohhatEIUY4jMFQiebY4V\nwdNjG43Uw9KlRw7SHrOcNl0hauwswpkrmBnW0awTex00kHp6bKOtehAYukxyzLZPIXuC+tdeo/Uw\ndEFkMDOUtOphlemd/yHnCgRkEFhlGGTVgzCSLpWo1m/s+d/8pUVsc4wM90gZxH6H4NWDtJkFSxfU\nxhptDyNTPiPzf89cmV7vqYdVhrVGJi5jp83cx1Rz9ve4+h6wGspcIUQx5FwhxMXQohWiGLqAa4H2\nd2J0ws4VWXZaBIYTAWrjy8A78EKzIjD7tlWPBztDydHzCeGM9LRh54pMO613LQjLiQDB0wUh6lCC\nZtFA+7aVdYLh6MHOUBKdT6M4i6BlJsJ7WiR4mlVGiNE4Y96GnSsyr69Hg58jRnfUMM/Y5/YEWe8N\nsJ6Xa02snsDyvY4RvUHjRzv0MOZl5vyfkJ1WiMGQnVaIi6FFK0QxtGiFKIacK4Q4kCOcdZqLFo1m\nYQQBe2Tq0uOkERkEr54sR4OR+hapB9GX5fTQYz9d4whnneaiRTM0eIG7DAN0pi49ThqMGwQ8JwKk\nTEZ7MvoWqQfRl+X0gASlRxd2L5Q97ePxONxxAs0qz9DFK8PKcO/Vg2Sez2xPRt8i9SCwnB4YurBp\n2mmXxn05zosRqJSEYM+eVkHwQhRDzhVCXAwtWiGKITvthVnupyZG3wc+E1tj1IJip+1Ryrsd4Mh6\nRiLjJoOtybDHTMUIgu9ts9n++TLS7Rgt9oxF2E7LyPbeU4/Z+iCgRmwv2HtelzexvIXUst+xbjJg\nOREwguDZtzesvV6+59lpt+piLfwzsmik3DDAAM2c4DENFOJE4E2sM1PRTLD6n9GmSpn/GQt//n7U\noaSH5qJFjO5ZjhOIo0FPXZ4TQaseRIbZr1/YrW9WVntYTgRemxiOHuwxXP7dUxdrzp2RleVwO23W\nXrPanhYhuqeteBB1xXFssTVGcq4QohByrhDiYmjRClEMLVohihG+YSCLSpEd4lyufpjV/KXtySo/\nORJscb/fm597MtBAba8ehi5omax6snTJItpm1NHmdrvZ/X5PaTcyL1H0eCxEMZomn8/PTz0ei3Ks\nmSormS89k09zTzuSk7UWqUBBr3apikLzxOW4+he89rRCFEOLVohilHg8HsnutubgrYMxkUk4c0VG\nEHCP3Y0ZbIzo0tKnRU+ANVImkgki04mGccNAxjijnKFL8/GYlckdzfYeNXKj9XgG9QxHg54Aa6RM\nJBNEphMNOl8Y84kxzl4ZNMCd6dwS3tOeEQR8dj0jUa3NjBsGRmrzGbqUuGHginta1uNxr75nPx4L\nH90wIEQxFAQvxMXQohWiGCXstEJU5KjEekPcMNAr5woHUb11jX7DA4srHYoxb3iYM8QNA71yIs4V\n0TIs5wpElxFveIj0LbIgGTcZsMuYHe/o0XObfPjxmJU9nSGH5eiRfWvC0W1mwOg31i0RmePs6cme\n/wgpzhVZcliOHpWcQUbSNeuGgcxxznL08OqZ0/ylXTpXrOH9nKMByT1y9spglFm2p6VPVJes/s/q\nW0TG4/H46xH6KF1YZVh77h45cq4Q4iD2nh6H0s0IIfZz1Cm8nCuEKIZ+aYWLAv/HIrxoRzHMjxQJ\nlAkjUcHcfmr2rw01M/CfxUiB8mzcRTtNiq0Jken00IKZ3QLNBuF9MUQN88gkZyQqWL4fsTm22twT\nbI/0S2sM0UD5LV2ZZVj1TLh72mkiRFyvWIbu+/0evl6BoQujT5B6WM4I1UD75Zn6ZI77SzvZ8CIJ\nnyfbm2ek9sq8v7/v1oGpC9on3remVw9is0RsrEibWbTajLRnXs7rl+h8GsmWm2qn1Z62Pt6eVgdR\nuShzhRDFUOYKIS6GFq0QxXga5wrty2owytnEKHqs0Vy0WRnuWR3U0rfHQaCnTUfoipbxDpCYeLr0\npmrdkoPY25H+92zpngyW3R9p87JvvbOk5uNxVob7ng5Csr239EHoaVNL11aG+56+3SrT4xQRzXDv\n6dJ7k8GWHASk/z1bOiIDgeG40jtvtacVohjNx2PEGI46Isxf7yljhjsrtPRF6GnTkbp6Zeaft+SY\n+Y4pUX2RMezRtwXS/54DjCcDnZM9DjLT61YZpE+exk6rg6gajHwAlIWC4P/Psw18VTROPtrTClEM\nLVohilHi8Xhp6zKrn33+Gam2Xx0lGGZJ2LmiJ3PClhzvc8RGichh6Dovw3Io2esAgJTJvJ+WcdsB\nw+mE1f9n3bxwuHNFT+aELTlsp4gtOQxd5++zHEoiDgBemV6nh1b/v729NScUIzid4XTC6n8ERlKF\n3vlf4vF4NLKCyhF7ZCbeL0BmsP0oMNrca6cNO1f0ZE7YktPrRLBXDkNXs7ys/sgeiuEkwHJKYfQL\nw+mE1f+smxe8hd27HXka5wohqqAgeCEuhhatEMXQQdQM+SeLCmjRzkAD5bMy5bOD7c3OdUrJSHbw\nDLiLlpVhPWqARmA4RiCw7Mo9zgh7jftIEDYjEwQCmuzAA+n/rDk3yTGL3zAw6eIdAFN+ab2JyzBA\nm/2xE241iuEYkQnTGSEig5EJYmLKkBFJLM+QwZpzDF0QesbRXbSoHcr73DNAI2VQ435LpzOM4Z6c\ns2UwgsonGJPbk8EaZ6QM0h7GDQM94yg77YzRDqK0p31OdMOAEMWQc4UQF0OLVohiyE4rLsXauYTZ\ntfbP4SD4LEbSRYwLEndcneaiRZ0IWM4VLRlMXVqfozJYRvlov5j9siW2TBMsRwNW30VloGU8WPV4\n/c/SZUJ72h2wHDSyHD1YjgZiDJomn8/Pz2EeSUd6PM78pWWQ6dJ3Ni0zZRUTpuy04qm4wkGUbhgQ\nT0WVhRlBe1ohiqFFK0Qxmo/HcvAWYjyaixYJWu7J2p8VBM/M9r73VgUPVuZ/RgB71i0ErBseGHMu\n0xrBPpkPPx73ZO2P2glvt9tvI/RR9SBOHGh2+imA2pOxJYelixfAzryFoDVGrBseGHOO0R60DDq3\n7/e7W5fZQHvaKfj36Oz0WfWIcxlpnNm6NB+PWZnakah85LHBK8Ooh3Wrglk76wEr2z4j6wTzFoJW\n/7JueGBk/me0By3DkDFHzhVCDIaC4IW4GFq0QhRDi1aIYlzG97jX1mi2/xZx8S9H9P/em+2vTjhz\nBTPAOhKE3WtrXHuNymEZ5r02M5w4kHoY43xE/++92R7RFy3DgJVgYCKcuYLlONFT5mxQwzwqJ+rE\ngWb9iDgaVAuSZzr0mMWyW7DndnNPO7dftTK5tz5nlxnBWM5iFIcS1jiPBGvOZeqC6tH8pWUZhTPK\n9DoITK+jciID7rUZdeKIGvcZ43NE/3vOPFtlEH0zy8i5QoiLI+cKIS6GFq0QxdCiFaIYZZwrsozu\nnpzRrsNkkBUQzrI7s+RkcIQzCJxuplUZ64YB1Hh/pNHdk4PUjZLh9ICUQey0GWM4l2O2PZ8QOVm3\nKjCcTpZyvANgON1Mq5MZBveRsu1nkeX0wGhz5i0FWfOJ4fTAbjMiBw6Cb9kJPVsl+/r6CFn1MHRB\n7MFIe0aphzWfWIkX0DItWPOpR04ZO632tKIie+alrgURohhyrhDiYmjRClGMMnZacX2e8bxg65a/\nFrQgeKTMkZn/kQ1/ViZ9FEYQPOOAjjXOHp6uqA3c0wXJkIGQMRf22PhpQfBImYjNzKtnjxNENJO+\nWcyhhBEEz3A6YY0zy7nCw9MFyZBh9utLxGz7eszMudCD9rRCFKP5S9tjdEfKHOkA0HsbwlYZVuZ5\nhkNJz+0BW2VYbUbKsIL6PTxdkGB7M/8C6sy50IPstGIYdBD1BzlXCFEIOVcIcTG0aIUoRhnnikqB\nz+Jcrh70EXauQGDYqVg2PgaZXyA9medbMljOLVFbI0POSIkMzqD5eNxjXGZcX49cXe/hyUHq8cqg\nXyD3+z1Uj9mvPvMCoxn9jzq3ZAR7M9rDgjFfGDLmpOxpq2WnHwnGrQqMbPusMWTIefb51DT5fH5+\npvnheoy0px1JF/Ev3p52y5Q5ionTM/mErwXJYqSFMZIu4l+88Vl6iU3vVaHM6bEQLKp/6cpOK0Qx\ntGiFKAacrNxMhy7ZZB14aZxrAScrX3s9kREEjDp6ILq0PkdlZDga9GTSN9ufOYQ1zggZWf1Z9aBl\nPNB6KDcMoDCysE+OCO/v724djIzwkTJnZJX3ZLTkZDojeGRl9WfUg5bpmbtePeEbBlAYWdi3GryU\n4dWH6OIxYlZ5T8b091H1mHFMgFlZ/Rn1oGWQuYvUQ7lhYC3rgfY6eWhP+5zohgEhiqEgeCEuhhat\nEMWQG6NwqR40fjVSguCvxDNO4OpB41cjfMOAGWY8vt/vzaPx6LUgqC5Ro3vPBK7WZpbDiKdL63NW\nmUzniixdJobZ00avBalIVptZzgiVuFp75oRvGDDDvh1Q54lIhntEF0RX1jZglDaznBEQsvqf0eYs\nXdhzTnbaTp5xT/uMbT6TUOYK8S/POFGfsc0jM8yeVgiBoUUrRDH0eDwo2keKLdxFm5ERPksGiwyn\nE9QejEQCsezBYgzcRYtmhDfbPrbOkoHIYRi6WfbTrOtSWPbgkRwNWNlFEAeYDF2ozhWjZIR/9qzy\nEdR310J22h1kPE6iWfAZgfJ6PB4L2WkPIGNSo1nwGQdTWqS10KIdFJ0Siy1kpxWiGFq0QhRDj8c7\n0FWXgsWaE42Hu2g/Pj7MLLbHYgVhe7pkBXujN8F7eLqwvhwyg+BZupiNcQuER7Rv98wfd9EiQhmO\nEUgZTxdWPYyrIBA8XdAvh6z+ZyymrMz/I7aHFZDv7mlfX1/DF+4ixn2kjKcLq54sPF3mbY2MwUj9\nguqC3DJQqT1MXeRcsQPtaQWLrTW2+4YBIcQ57PKIav0nIcR5yE4rRDG0aIUohhatEMXQohWiGFq0\nQhRDi1aIYvwP1bMkvt8s3zwAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0xd0a03d0>"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Inference"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Next, we show how to do inference with the learned model parameters for a given data point."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# get testing data\n",
      "samples_ts, labels_ts = prepare_data(p_ts, l_ts, ftype_all, n_ts_samples)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from modshogun import FactorGraphFeatures, FactorGraphObservation, TREE_MAX_PROD, MAPInference\n",
      "\n",
      "# get a factor graph instance from test data\n",
      "fg0 = samples_ts.get_sample(100)\n",
      "fg0.compute_energies()\n",
      "fg0.connect_components()\n",
      "\n",
      "# create a MAP inference using tree max-product\n",
      "infer_met = MAPInference(fg0, TREE_MAX_PROD)\n",
      "infer_met.inference()\n",
      "\n",
      "# get inference results\n",
      "y_pred = infer_met.get_structured_outputs()\n",
      "y_truth = FactorGraphObservation.obtain_from_generic(labels_ts.get_label(100))\n",
      "print y_pred.get_data()\n",
      "print y_truth.get_data()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[14 12 12  0 13  3  8 13  6]\n",
        "[14 12 12  0 13  3  8 13  6]\n"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Evaluation"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the end, we check average training error and average testing error. The evaluation can be done by two methods. We can either use the apply() function in the structured output machine or use the <a href=\"http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CSOSVMHelper.html\">SOSVMHelper</a>."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from modshogun import LabelsFactory, SOSVMHelper\n",
      "\n",
      "# training error of BMRM method\n",
      "bmrm.set_w(w_bmrm)\n",
      "model.w_to_fparams(w_bmrm)\n",
      "lbs_bmrm = LabelsFactory.to_structured(bmrm.apply())\n",
      "acc_loss = 0.0\n",
      "ave_loss = 0.0\n",
      "for i in xrange(n_tr_samples):\n",
      "\ty_pred = lbs_bmrm.get_label(i)\n",
      "\ty_truth = labels.get_label(i)\n",
      "\tacc_loss = acc_loss + model.delta_loss(y_truth, y_pred)\n",
      "\n",
      "ave_loss = acc_loss / n_tr_samples\n",
      "print('BMRM: Average training error is %.4f' % ave_loss)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "BMRM: Average training error is 0.0067\n"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# training error of stochastic method\n",
      "print('SGD: Average training error is %.4f' % SOSVMHelper.average_loss(w_sgd, model))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "SGD: Average training error is 0.0291\n"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# testing error\n",
      "bmrm.set_features(samples_ts)\n",
      "bmrm.set_labels(labels_ts)\n",
      "\n",
      "lbs_bmrm_ts = LabelsFactory.to_structured(bmrm.apply())\n",
      "acc_loss = 0.0\n",
      "ave_loss_ts = 0.0\n",
      "\n",
      "for i in xrange(n_ts_samples):\n",
      "\t#result = model.argmax(w,i)\n",
      "\t#y_pred = FactorGraphObservation.obtain_from_generic(result.argmax)\n",
      "\ty_pred = lbs_bmrm_ts.get_label(i)\n",
      "\ty_truth = labels_ts.get_label(i)\n",
      "\tacc_loss = acc_loss + model.delta_loss(y_truth, y_pred)\n",
      "\n",
      "ave_loss_ts = acc_loss / n_ts_samples\n",
      "print('BMRM: Average testing error is %.4f' % ave_loss_ts)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "BMRM: Average testing error is 0.2394\n"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# testing error of stochastic method\n",
      "print('SGD: Average training error is %.4f' % SOSVMHelper.average_loss(sgd.get_w(), model))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "SGD: Average training error is 0.1997\n"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "References"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[1] Kschischang, F. R., B. J. Frey, and H.-A. Loeliger, Factor graphs and the sum-product algorithm, IEEE Transactions on Information Theory 2001.\n",
      "\n",
      "[2] Teo, C.H., Vishwanathan, S.V.N, Smola, A. and Quoc, V.Le, Bundle Methods for Regularized Risk Minimization, JMLR 2010.\n",
      "\n",
      "[3] Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y., Support Vector Machine Learning for Interdependent and Structured Ouput Spaces, ICML 2004.\n",
      "\n",
      "[4] Lacoste-Julien, S., Jaggi, M., Schmidt, M., Pletscher, P., Block-Coordinate Frank-Wolfe Optimization for Structural SVMs, ICML 2013."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}