msakai/svm_voting.ipynb

## svm_voting.ipynb
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# one-vs-one による多クラス分類の実現\n",
    "\n",
    "Coursera の Machine Learning のコースなどでは、SVMでの多クラス分類はone-vs-restの分類器をクラスの数だけ作る方法があるという説明をしてたけど、ふと[LIBSVMのドキュメント](https://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f419)を読むと、任意の2クラスの組み合わせ毎に分類器を作って、それら分類器の投票によって、多クラス分類を実現しているらしく、[A comparison of methods for multiclass support vector machines](https://www.csie.ntu.edu.tw/~cjlin/papers/multisvm.pdf) によると、十分な精度が出るらしい。\n",
    "\n",
    "そんな簡単な方法で本当に良いのかと気になったので、自分の理解があっているかどうか、LIBSVMを使っている[scikit-learn](http://scikit-learn.org/)の多クラス分類SVMと、scikit-learnの2値SVMを自分で多クラス分類器に持ち上げたものを、動かして比較してみることにした。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二値分類器の「投票」(voting)による多クラス分類器への持ち上げ\n",
    "\n",
    "任意の二値分類器を「投票」を用いて多クラス分類器に持ち上げるためのクラスを書いてみた。\n",
    "scikit-learnのコードを適当に流用して一応それっぽいインターフェースにしてみたつもり。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.utils import column_or_1d, check_X_y\n",
    "from sklearn.utils.multiclass import check_classification_targets\n",
    "\n",
    "class Voting:\n",
    "    def __init__(self, BinaryClassifier, *arguments, **keywords):\n",
    "        self._BinaryClassifier = BinaryClassifier\n",
    "        self._arguments = arguments\n",
    "        self._keywords = keywords\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        \"\"\"Fit the model according to the given training data.\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : {array-like}, shape (n_samples, n_features)\n",
    "            Training vectors, where n_samples is the number of samples\n",
    "            and n_features is the number of features.\n",
    "        y : array-like, shape (n_samples,)\n",
    "            Target values (class labels)\n",
    "        Returns\n",
    "        -------\n",
    "        self : object\n",
    "            Returns self.\n",
    "        \"\"\"\n",
    "        X, y = check_X_y(X, y, dtype=np.float64, order='C', accept_sparse=None)\n",
    "        y = self._validate_targets(y)\n",
    "\n",
    "        n_classes = self.classes_.size\n",
    "        classifiers = []\n",
    "        for k1 in range(n_classes):\n",
    "            for k2 in range(k1+1,n_classes):\n",
    "                classifier = self._BinaryClassifier(*self._arguments, **self._keywords)\n",
    "                index = np.any([y==k1, y==k2], axis=0)\n",
    "                Xtrn = X[index, :]\n",
    "                Ytrn = y[index]\n",
    "                Ytrn[Ytrn==k2] = -1\n",
    "                Ytrn[Ytrn==k1] = +1\n",
    "                classifier.fit(Xtrn,Ytrn)\n",
    "                classifiers.append((k1,k2,classifier))\n",
    "        self._classifiers = classifiers\n",
    "\n",
    "    def predict(self, X):\n",
    "        votes = np.zeros((X.shape[0], self.classes_.size))\n",
    "        for (k1,k2,classifier) in self._classifiers:\n",
    "            y = classifier.predict(X)\n",
    "            votes[y==+1, k1] += 1\n",
    "            votes[y==-1, k2] += 1\n",
    "        return self.classes_[np.argmax(votes, axis=1)]\n",
    "\n",
    "    def _validate_targets(self, y):\n",
    "        y_ = column_or_1d(y, warn=True)\n",
    "        check_classification_targets(y)\n",
    "        cls, y = np.unique(y_, return_inverse=True)\n",
    "        if len(cls) < 2:\n",
    "            raise ValueError(\n",
    "                \"The number of classes has to be greater than one; got %d\"\n",
    "                % len(cls))\n",
    "        self.classes_ = cls\n",
    "        return np.asarray(y, dtype=np.float64, order='C')\n",
    "\n",
    "    def score(self, X, y):\n",
    "        z = self.predict(X) == y\n",
    "        return sum(1.0 for b in z if b) / y.size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## scikit-learnの多クラス分類と同じ結果になるかirisで確認\n",
    "\n",
    "自分で実装したものがscikit-learnの多クラス分類と同じ結果になるかirisで確認してみる。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scikit-learnによる予測結果など:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.993333333333\n",
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "svc1 = SVC(kernel=\"linear\")\n",
    "svc1.fit(iris.data, iris.target)\n",
    "print(svc1.score(iris.data, iris.target))\n",
    "p1 = svc1.predict(iris.data)\n",
    "print(p1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "自前の「投票」による予測結果など:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9933333333333333\n",
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2]\n"
     ]
    }
   ],
   "source": [
    "svc2 = Voting(SVC, kernel=\"linear\")\n",
    "svc2.fit(iris.data, iris.target)\n",
    "print(svc2.score(iris.data, iris.target))\n",
    "p2 = svc2.predict(iris.data)\n",
    "print(p2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "予測結果が一致しているかどうか確認:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "print(np.all(p1 == p2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 別の例でも確認してみる\n",
    "\n",
    "適当なデータを作ってみる。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x117bebeb8>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXt4VNW9x/3ZuY6ZTKsiCgSDhsR4q7TeoBotalWYKGpA\no0WgfbXtqbRKaIVIaUuVvlLetoqAHs30nGq9RbkUIVGkVZQ5VatwsBUFgZSbQau2PexMmMltvX+s\nmWQue/bc9mQmyfo8zzxhZvbstWbPZl1+l+9PE0KgUCgUiqFJTqY7oFAoFIrMoSYBhUKhGMKoSUCh\nUCiGMGoSUCgUiiGMmgQUCoViCKMmAYVCoRjCWDIJaJr2W03TPtE07a9R3v+Gpmnv+h9uTdO+ZEW7\nCoVCoUgNq3YC/w1cbfJ+C3CpEGIcsBhosKhdhUKhUKRAnhUnEUK4NU0bY/L+m0FP3wRKrGhXoVAo\nFKmRCZ/A7cCLGWhXoVAoFGFYshOIF03TLgO+BVT1Z7sKhUKhMKbfJgFN084BHgMmCSH+ZXKcEjNS\nKBSKBBFCaMl8zkpzkOZ/RL6haaXAamCGEGJvrBMJIbLq8bOf/SzjfVB9Gjx9ytZ+qT4N3D6lgiU7\nAU3TngYmAsM0TTsA/AwoAIQQ4jHgJ8DxwMOapmlApxDiQivaVigUCkXyWBUd9I0Y738b+LYVbSkU\nCoXCOlTGcBxMnDgx012IQPUpPrKxT5Cd/VJ9io9s7FMqaKnak6xG0zSRbX1SKBSKbEbTNEQWOIYV\nCoVCMcBQk4BCoVAMYdQkoFAoFEMYNQkoFArFEEZNAgqFQjGEUZOAQqFQDGHUJKBQKBRDGDUJKBQK\nxRBGTQIKhUIxhFGTgEKhUAxh1CSgUCgUQxg1CSgUCsUQRk0CCoVCMYRRk4BCoVAMYdQkoFAoFEMY\nNQkoFArFEEZNAgqFQjGEUZOAQqFQDGHUJKBQKBRDGDUJKBQKxRBGTQIKhUIxhLFkEtA07beapn2i\nadpfTY55SNO03Zqmbdc07ctWtKtQKBSK1Miz6Dz/DSwHnjB6U9O0ycBYIUSFpmnjgf8EJljUtkKR\ndei6TmNjIy27d1NWUUFtbS0OhyPT3UqJwfidFKAJIaw5kaaNAdYLIc4xeO8/gVeFEI3+5x8AE4UQ\nnxgcK6zqk2JokunByu12U+N0UtXTw3keD1vtdtw5Oaxpbqaqqqrf+mEl/f2dMv0bDjQ0TUMIoSX1\nYSGEJQ9gDPDXKO+tBy4Kev5H4NwoxwqFIlm2bNkihjsc4ga7XSwGcYPdLoY7HGLLli390v6RI0fE\ncIdDvAxCBD1eBjHc4RC6rvdLP6ykv79Tpn/DgYh/3Exq7LbKHGQpixYt6v33xIkTmThxYsb6ohg4\n6LpOjdPJU7rOlYEXPR42ATVOJy2trRQXF6e1D42NjVT19PS17+dKoKqnh8bGRm677ba09sGIVFbW\n/fmddF2nZvJknmpry9hvOBDYvHkzmzdvtuRc/TUJfAScHPR8tP81Q4InAYUiXmINVo8//jiFhYXs\n3t1CRUVZWkwMLbt3c57HA4AONAK7gQrgLI+H/2poQAgRs20rzSHhppxmu50Fc+fGNOUE+vDfDQ2c\n6PGgA+E9ONfjoWXPnqT6ZcTixYv5SvAE4CfTk2i2Eb44/vnPf578yZLdQoQ/gFOAv0V5zwk0+f89\nAXjT5Dzp2C0phgD3zJsnFoeZLAKP+0AU5h8j7PYbBCwWdvsNwuEYbrmJoaGhQdxgt4stIBx5CHsZ\ngsvl39w8xEUgvlJQIOz5+WLevHniyJEjEeew0hySrCknvA+TQQwHsSXsPDfY7cLlciXcLyN27dol\nCjTN9DdcUF9vSVuDDVIwB1k1ATwNtAI+4ADwLeC7wHeCjlkB7AHeJYo/QKhJQJECgQHYaAC5CgTU\nhb38snA4hltq0z5y5IgYVlwsjslDMAPBoqDHDISWh7gWxGJ/n4YXF4cM7lbb382uSbQB3KwPx4L4\nIYgGEH9IwSdw5MgR0dDQIO6ZN080NDSIX//618IG4iwQ1VEmgatArFixIuG2hgKpTAKWmIOEEN+I\n45jvW9GWQhGN2tpaFsydyyYIMSdsAl4nD7g37BNX0tNTZamJweFwcNsdd7B09VIYG/bmWCgsheta\n4Dbgx8CmtrYQW7fV9vdg81Q4waacYPPTRx9/zMXd3YZ9mADsBd4HNgO/WrIkwkYfy5QVME+N7+4m\nr72dVbm5/L27m1XA5fIyGf6GfwamxP3NFfGiMoYVgwaHw8Ga5mamOxzU2O0sBmrsdqYVFODlm0Ck\nQ9HjOZc9e1qs7YiGjJUzwDtGbocDBA/uEP+gHS9lFRVstdsN39tmt1NWXo7b7WZsSQnNc+ZgX7qU\nt55+mvPb2w0/czFwJtAMrAMW1dfT1tYGyMF//vz5jBw2jIdnzyZv6VKa58xhbEkJbre795iayZO5\nR9d5q70dDZjln3C+jTQTrAGmAzXAYuBa//ObgdZDhxL6/orYZGV0kEIRL0arzpbWVvnanj1Ul5dz\n+dGj1Ne/gtHYardvo7y82tI+VZRXYH/RjofIBu37oTzsteDBvayigma7HaPObrPbqS4P/7Q5Zrsj\nd04OK51OxlVWhkRUndTVxfoo59sGBK5W8ARWWVlJzeTJfKWtjXuArciM0DUeD9+jL7Jn8eLFfKmt\njfuBpwz6VAO0+B+NwO8Bu//5DLudCQl+f0UcJGtHStcD5RNQxEm8DtQjR44Ih2O4gJfT7hPobe84\nh6FPwJGH0E1s86n6BMJt7UeOHAm5TveFXScjn8ERvxPYsA+E9v8+ED+sq4veZ//xN9jtYsWKFcKe\nny+uBzElit1/CghX8LXxPx/IeRb9AZn2CSgU/Y1pTsDkyfz0/vtpPXiwd3fQ3LwGp7OGnp4qPJ5z\nsdu3kZPjprl5jeVx5w6Hg+YXmnFOcdJzcg+eYR4ZLrEf7umCZ5Er2zJgOHJF/kRtbe9n1zQ394Z0\nnuvxsC0oO9esr2ahoOG7oydqaykuLqZ5/foI85MDaZK5BqjKz2dCZyd/Bt7xvx7cg212O47PP4/w\nY+jy63I8MBsZHrvuD39grBDsAGZF+Q4XADv9/94EvAK0AW/bbKyP8f0VyWGZbIRVKNkIRTy4XC6a\n58xhjYHZ5Grg04ICpnZ0hMgbfPnLX6axsZE9e1ooL5d5AukcVNra2mR7e/cwumQ0r7/+Ouufe46J\nSNv6/+B3rq5cyR133GH42ZY9eygrL4/ZV13XGVtSEjopIgfS6Q5H1CQrs+s4paiI42+8keHDhvGf\nK1fylM8X4pjdBNxks1F51llcu3UrP/a/7kaadaqA84A3/N+zMyeHOT09/Ab4OvCiwfeYDHyMjDff\nDJwPvFlYyO59+xgxYkTU7z/UyQrZCKseKHOQIg5i5QQssCC80krSLb2QTChorH4da7OJ1tZWIYSI\nMClNsdnEMSAusdnE9cg8glimpGIQdhDlIL4Y5ZgvgjgOxPUgrikqUnIRcUIK5iAVHaQYkJhGvSBN\nLQHCI3AyQTyhn6mQbFRRcETVdTYbi5GZnTcCXwLGVVbidrupqqri3V27cEydyobzz2dTVxfPAq97\nvTyBNBVtQjpzq8Dwe14EfAeYD5xDaARQjf/5OTYbU2bN4sz6eq5/6CFaWlsHrOjeQEH5BBQDEtOo\nFyI1za2WN0gUq0M/wzGLKnozP59hra3oum4oPREY4E879VTswFTgOaDY62WT10uN08mTq1Zx67Rp\nVPX0MNLj4QvAP4AFyAn3SeQg/kXgm1H6eAlwFKj1f84FfIr0j1T7P3d7fj7NK1Yo238/onYCigGJ\nUU7AtYWF1BDpvIS+mPhMEU+8firU1tbizslhU9jrmwB3Zyf/WrUqJF4/nKamJq7My+NpZCJb4Ppd\nCYzv7qb2uut4StdZ4/FQDPwFmStg9/+9FTkRFCMnYSPeRE4YAcfz7UATcAywNj+f2/2/qZoA+plk\n7UjpeqB8AooE0HVduFwusaC+XqxYsUKcUFyclTLO/SHHHLDbX19UJO32hOr9mLVl5mO5DkR1fn6v\nzf9Yk/DRySbvF4eFl+r+8M/KvDwxa9YsFf6ZAqTgE8j4oB/RITUJKFLALCY+05j1zSi+P5hY7wfQ\ndV3MnDlTVOblCRfmOQnBmDmWK/PyeieIBr/TNlqMfzGIjf4J4Rq/k/4G//Nok8PxRUVqAkiRVCYB\n5RNQDCqqqqqixsRnmmh92759O2NLSqJKPSciBV1cXEzJiBHM6OrCSGEomv/BzMfyUW4u7xQWgsdD\nCzJs04gLkD6Bq5B2/suR1aNmIH002+kLHT0XaR56A+gRKiQ8oyQ7e6TrgdoJKAyIdyU80IhlJmpt\nbU3YjJRsuOiWLVvEsOJi8ZWCAvE1pOT1sOJisXHjxt4+NPhX9tFUPr9SUNC7yykuLBRTbLaQY3pN\nQCBm0ZdNbJUc9VAFFSKqGMyEC5yFi5INZGKFjtbX1yccWmrqJM7JodafnWxEDlCiaVzp/5sDFBUV\n9Trh1xUV8Yr/XDoywmcBMBd4x27n9KlTeXH8eBxTp7Ltb3/jjfz8kH4UA6XAP5Ha8sVkPnJrqKPM\nQYqsJhtKRqaTWKGjL+7ahTPB0NJEpCcCAnw7d+zg0Uce4Umfj+sCb/p8bPL5eq9zwJR13KuvUvP8\n89DRwdeAryIF4456PBxauxan18vW997j4rVrWbRkCdPr6xnf0cF4n49tyOih4AiuZITxFBaS7BYi\nXQ+UOUgRRLKmjYFCrO83c+bMpL9/cOSUy+WKMB0lWz3syJEjYni0KKwgZ3TAZHX48OFe8bi6MGd1\nNkRuDQZQjmHFYCXdSVaZJpbU87tLljBu7Vo2IQu6NCKdrl7gdU3rFZ4zori4OGoBGsMdlr/d64FF\nyFKBZUjxt+Dr3NjYSJUQxiYqfx9vo89k1dTUxOzZsxk3bhw1Tif7EhTGU6QXNQkoshqr9fWzjVim\nm5EjR7KmuZlrr7qKrqNHQ8wvoqeH7du3U1VVlXBh+mi+iGOQNWI3IiedZuTEMFv0RfCYTszISar3\nedAEks2RW0OaZLcQ6XqgzEGKIPojySobMDPdmJpfHI7e6J1ECtMbJYeZ1hEoLu7tk6kJi7B6AIPA\nZDcQIAVzUMYH/YgOqUlAEUY2J4D1B2aD7vVFRaK4sDDhSdLonGbhn3EXvjHwCQyWiTqbSWUSUCGi\niqwnYEaoXraMo/X1VC9bNqTUJc3ML+e1t1MSpSi8mTqpURhpC1L/34hgs46RbtN1NhvXAafbbDyI\nrO08XWkBDQiUTyCLSdTOO5gxc3IOdmIphJ7R2Wn4uXgkpIN9ES8WFHBiR0dvcZhg3rbZmBLkfwm3\n708pL+fR6mqampoStver+zyzqMpiWUq4VEBwhayhsgJWSMyqht1ks3FJTg4vtLdHfK7Gbqd62TLT\nyTNQwey1V19l9fPP09PRwQtERipdB7QcPsyIESMsHbTVfW4NGa8sBkxClgb9EJhv8P4XgBeQ8iF/\nA75pcq50mMwGFEPFGaqIn2h+kWBJh2j3SjzidMMdDlEH4hK/Xf8GQsXfLrHZhMvlisgtSMU/o+5z\n6yCTjmFkpvkeYAyQ7x/oTw875h7gfv+/TwA+B/KinC99V2qAMNgTpBTJES2CyMxxHs+gHbjf7gGx\nmD59nwX+v7p/QvhhXZ2lg3ZDQ0OEtlDgMcU/6SjiI5VJwAqfwIXAbiHEfgBN055F7h53Bm84kLUk\n8P/9XAjRZUHbg5LBniCVDQRMGrv37Kai3Do7dO95d7dQUVFmyXnDzS/1CxaEnDNa/L0QItKMZCC5\nEbjfTkLmBRRDhALpNrsdx+efx9QxSsRvs3PHDi70eg3fu8DrZef778d9LkXyWDEJlAAHg54fQk4M\nwawAXtA0rRV5j0VPc1QM+gSpTON2u3FOcdJzcg+eEzzYX7Qzd95cml9IzA4dPjiXlpYybdqt9PRU\n4fGch93ezNy5C2huXpO0fTteGWkjx7nL5Ypr0A7cb497PCyAqNnLM4cN4zQLFyef/vOfRBvm/wyc\n+PnnCZ1PkRz9FR10NfC/QojLNU0bC2zSNO0cIUSb0cGLFi3q/ffEiROZOHFiv3QyW4glJWAmFaAw\nR9d1nFOc6NfoMFa+5sEDe8E5xUnrgfgE6cIH56aiIl5qP4qXXwN18rwegE04nTW0trYkHCqZqnhe\nvDvKwP3mAjqRmv9fQ2YMvw5sPeYY1jU3s3PnTksXJyccdxzrMJ50/gx8e9iwhM43lNi8eTObN2+2\n5mTJ2pECD+S98lLQ83rCnMPABuDioOd/As6Pcj7rDWYDkKGeIJUuGhoahP0cu2AREQ/7OfH5W8wc\nmjYcAvQQE7fdfkNS9u1UfUOJfH7jxo3iGP93CPYJ1IE4wZ8tbPa9j7XZxA/nzOl1PMdT/2H58uXi\ngiiO6AtArFixIuFrNlQhw8libwPlmqaN0TStALgZGQkUzH7g6wCapp0EnEaoxIgijKGeIJUudu/Z\njecE49WxZ5iHPXtjmzTMagBcQg9SQi3ovJ5z2bMn8ds9Vd9QInUFDhw4wNV2O1fS5xP4BfAb4BIh\naGxsNE0S+xJw3IMP0jxnDqUnncQpI0bEVf/hA2RNgmrgqP+vy/+6on9I2RwkhOjWNO37wMvISKHf\nCiE+0DTtu/Jt8RiwGPidpml/9X9snhDin6m2PdgZyglS6aKivAL7i3ZpAgrD/rmd8rGxTRpmg/Ol\nePgjewjOdLHbt1FeXp1wX1P1DSVSV6Bl927Oj2PCCXZC73z/fV5ZuZJngSl+B6/u8XAK8CzENGG1\nHjzIzcDt9JWcbELWG7gZaD10KI6rpEgVS2QjhBAvCSEqhRAVQogl/tce9U8ACCEOCyGuFkKc4388\nY0W7CkWi1NbWknMwB/aGvbEXcg6aV90KUFZRwVa73fC917EhCB6cN5GT447rvEZ9TbZCWIB4d5Rm\n32mb3U5Z0IQTWJxUnnEGV+blMSXo2Eb6/AmBqmMu//NwGYuyigo+t9tpIXQn0AJ8HtamIo0ka0dK\n1wPlExBCDN6autnAli1bhOM4h/QNXCZ9AY7j4ve3mPsENFFUdK2A+4TdfoNwOIan5MfpL99QMolb\nRkqk94C4PcjOvzjIzn87iAX19Sm1qTCGDOcJKCwm3rBARXJUVVXRekCaNPbs3UP52HJqE9C1NzOz\nrFu1ioMHD7JnTwvl5dXU1j6RkoBaf2nwJ2I6CmBkrhoFLEeWjwyP+KkBlowenVKbCutR2kFZhplO\nzHSHI+tr6qYrCSsd/Ug1sSugu9OyZw9l5eU4nU6amprS2qZVROtH+HcymxyN7tUVwHpkUZpwrgam\nrFjB7NmzQ15PpE2FMRnXDrLywRA3Bw1kyYgQM8vliZtZ+rMfW7ZsEQ7HcGG33yBgccqmm0y0mSxW\n9iPcXFWZlxdhIgo87gszBymsgxTMQWonkGUsmD8f+9KlhnK+i4Gj9fX84v77+7tbMdF1nZIxJSFJ\nWADsBccGR9xJWP3RDyEEJSVj0fWnCDdaOBzTE07sykSbyaLrekr9MFIQ1TStdyV/qLWVI6tWsTZJ\nVVNFcqSyE1BFZbKMRKI0sonGxkZ6Tu4JHQQBxkLPydGLm2SiH42NjfT0VIFBpH9PT1XCfc1Em8mS\nSj/cbjdjS0oi4v+3b9/Obbfdxi/uv5/p06ezsb09pYgmRf+iHMNZhlWSEf1te7YiCau/+tHTreHx\nGNfQSiaxKxNtJsvu3S1J9SMeCQshBLdOm8YvgOn0xf6/jbx//7BqFUIIXC5Xxn1Gij7UJJBlWBEx\n4Xa7cTprLBUyi4UVSVj91Q8hBHZ7s1EOVlKJXelu08oiLhUVZXH3I7jdjz7+mPExylgKIajq6aEO\n+DYyZ6AFmAKIoiJeeeUVpt08LWXhPoW1KJ9AlpJsxESqNt9kUT6B9LRpdeWteO+P8HbfyM/H3dnJ\nBuQKP5iAr0r09ET1Z/0EWFpUQMfUjozeH4MVFR2k6KWhocEf9REZoJGskFm8pJqE1Z/9CI2QST2x\nKx1tpiuZKlY/TNv1C8wZRa2ZRbZ9paBAFJ5ZmJJwnyI6qGQxRYBkbb5WkGoSVn/2o6qqitbWFnmM\nBYld6WjTTKgumSIu8fbDrN2vIs08gVaDfVVCiKj+rJ2awDeiw7A//ekzUkSiJoFBRiI233SQLaJ3\n8fTD6r5a3WY6K8yZ9cOs3QuA/y8vj8NdXYa+qmj+rB9873usfGllxn1GikjUJDDIqK2tZe5c4/pQ\nUsjsiaifzZZs1v7EqgzndFy7dFSYi8fJbNbuVrudCdOmcXTkSK4YPZrLhWD1mtU0NDRwwnHHccbZ\nZ/Purl00NzdHlLp8pOERKdwX5hOIV7hPkSaStSOl64HyCaRMMvbubMlm7U+synBO17Wz2icQT9H5\neNsNXDvbWTbB5YjcsYiCPMQlNlvEOQNiiLd+4xvCVmwTRecUZdRnNBhBZQwrwglEF0mbb1lMDZh0\nRBRl887CqmimdEZj6brO4sWL+c8HH+RSTWO8z8c2u53XNY3b7riDXIg7ZFTXdU4dNYqZbW3kAxXI\nQt9vYqxJFRwdFB6mPG7cuOjX7hl4sgtu959z+/btIVFGbxUV8Wp3N9fdOI3LJl6mdIIsQkUHKVIi\nHRFF2b6zMCszWXhGYdylDdMVjRW8av+JP7rGnp8vpk+fLoYXF8dczYczb948kZeHsJchdz1lCEce\nYouJJpWu68LlcokF9fXC5XL17jxMS3SWydKUN9jtYsWKFUoqup9ARQcpUsHqiCJd13E6a0JWx6kW\nXbcasyxf3wgfd8+tY9y4cTFj8dMRjWWUnXtvRwfrgFueeop1xK7aFX6+B1c8SNct0OVfuXsuBfaC\n8xm4K4qTOZrz2DRDegzsaZGO6+YNG9IS3aSwFqUdZDG6ruNyuVgwfz4ulwtd1zPdpZjIiKKthu/J\niKKyhM6XLTo5ZlSUV2D/zFijyb4f/qOjkxqnk7a2NvPzWHztIHqI5qfARIyuamTVrvDzaWM0Y22j\nUmgqKEhIk6qivALbJzbD9+z7oRzpuAbSFt2ksA41CVhINIEtowLb2URtbS05OW4wkP1KpjRiJnMV\n4sW0zOQBuBfzgTXkPBZeO4geotkCXBzlM2aD6u49u/GN9Bm+5xkDOxEJ9dPpdOLb5Y167YYjcwcm\nV1eHiCHq9JWcXFNQwKigAjOKzKHMQRYRj8BWpk0g0XA4HDQ3rwnSGzoXu30bOTlumpvXJNzv/spV\nSMXx7HA4aH6hmSuuvgJGdNAxRq5icw5AcxcUE99q1cprF/g+2/72Lv8oKODOjg6Cv00ZsmKXEWYh\no2baRhyAO+vmJtTP5uZmqvJsbH/GS2cpeMdA7n7IPQBndsEsm431fgfyvQsWsAk4BllZrAo4Dxje\n0cHP6+vjMrkp0kyyzoR0PRigjuGBXAwmQMARWF+/IMQRmChHjhwRDsdwAS+HXYqXhcMx3BKHoFWO\n5+XLl4uvFBSIeqRDU0/yd0v12oWHqzIWcYzfcRvozx9AHON3rCbiaD1y5IhwHOcQzAhz5M5AFB9b\nnHBfA7WFdf81mwtiFogfgbgexI/mzg35XsOKi0VxEv1WxA8pOIYzPuhHdGiATgJGRbeHckUlq7V5\ngrFyksmGYudmg3ReHmJhUBTQypUrkyo8b6WuU6ILnuXLl4trCgsH9AIp20llElDmIItIR3bnQMZq\nbZ5g4nE8xxt1kg3Fzs2K0uSeVsi2MZdTM3Vqb4H5mTNnJlx43kpdp0RrXrQePMgEn7FPQjmIM48l\nk4CmaZOAB5GO5t8KIX5pcMxE4AEgH/hUCHGZFW1nC1YVgxlMpEtHyGrHc1VVFS2trQkPrFZhGq56\nko9zvjwu5Dome12t+j0SnTjVAim7SXkS0DQtB1gBXAG0Am9rmrZOCLEz6JgvAiuBq4QQH2madkKq\n7WYb2bCiTCfZlP2bDsdzJoXvsqUgjxHRtJUSmThjLZBWOp24XC5LiuYoEidl2QhN0yYAPxNCTPY/\nr0fap34ZdMz3gJFCiJ/GcT6Rap8ySbLFYKwgXQN1ZKWyrb3RL5mI7MhU4Zx0YZWExYcffsg3v/lN\nDv3974w+9VR+97vfcdpppyXdL7fbjXOKs68S2Gd2cg7mJFUJzEiG4nVN4+opU1i7+nlOFxrXdHTw\nXopFc4YqGZWNAKYCjwU9vxV4KOyYB5C7hVeRJUdnmJzPQnfJ0CGdImbpjvRJhnQ6njNBqo7buro6\nYQMxCcRi/18biLq6uqT6Y+asdhyXnMM8WIZi/vz5ovjYYsHYSBkLFTWUOAwAx3Aesub05YAdeEPT\ntDeEEIYeoUWLFvX+e+LEiUycOLEfujhwSadMg5VO2FgkIuucTsdzJkjGcRu4Xm+++TZP/fYxXiDS\n3DLlgQeYPXs2Y8eGe53NMXNW95ycnORDwOQW2Pm0XdvWe/5gGYtdXTDa56P2xhu5YepUZR4yYPPm\nzWzevNmakyU7ewQewATgpaDn9cD8sGPmI01GgecuYGqU86VlphzMpLOk5Lx59whYbHhuuE/U1y+w\n5DtYJes8VAjdCZ0sJkUJT54E4qKLLor7vAHZ5wkTLpT5CgYicVyGqL8n+ZBnMwE6WxmiGES1f0cT\nbwjsUIcM7wTeBso1TRsDHAZuBm4JO2YdsFzTtFygEBgP/MaCthWkV6ahP7J/dV3HOcUZYhP34JEr\nwylOVYQ8jPCdXw4jI4q/B7gYeGzfvrjOG2y3H+HxkDsWug2OS9VZveP9HVGjobxjoLYFng28MECy\n7gcyKWsHCSG6ge8DLwM7gGeFEB9omvZdTdO+4z9mJ7AR+CtSwvwxIcT7qbatkKRDxCxAOrRxwonH\n9KDoI9xE18OpRFOn+h/g5FNOiXnOYNmTNR4PTwBF+zHWB0qhEpjb7cb18CPk7jd+P3d/4gJ5itSw\nxCcghHgJqAx77dGw578CfmVFe4pQUikpGQurdYWMMJUmVkXII4jc+f2OzVQahmBuBt57IvbvH65c\n6kBqKDmfgfYx0F0qdwCB6KBkiwzVOJ086fMxYz/oBqUmcw/IYjfhqKSy9KEyhgcB6R6o0+2ETSRO\n3qqawNlFapxJAAAgAElEQVSOWbhvpInuNLzUMYUHmIg0Af0PcgL4Xl1dXE5hI+XSKqC1C2bshU9O\nnMBtd96eUshzYKK5jr4JpqdUKpnmHgBxMIfztVyK6Yz4rEoqSyPJOhPS9UA5hpPGKgG4/ibecMSh\n4jyOFe4bPWz3dyInJ0+MHjFCXHTRRWLPnj1xt9kfAojh+loBAbp6v/DcD2bPzriO00AFJSCnGOjE\nipNPR9x6NhJvXkY8eRKBSJ975s0TDQ0N4siRI6btpnsAjmeiCS6rmYhA3lAnlUlAFZpXZA2BbGuj\nOHmXy8Wc5XPw1BiYjNbYWXbnskFRqtDlcjFnTjMeT2TlALu9hmXLqnu/Z+/12tNCeXlZyPUKjvQ5\nz+NhaxyZuGbF5a3I3m1tbaWyrIxrfD6uQNr+HUjfRXCx+0xm3Q9UUskYVj4BRdZgpt/TX87jeKQ3\ndF3n8ccfZ0PzBnJ6wHnNNcyaNcsS30Qi4b7RrleyBY7SKaQXmGAu1zTORhbH+RFwjs3Gzvz8EH2t\nTOo4DUXUJKAYEPSHyFqkRlIzc+cuCNFIcrvdXF19Ne3D22EMFOyDTXUb+cm8eax/+eWUV8xW5GVE\nq1EcT4H3dAzAhpMScgdwE/DBhx8yYsQIS9tUxI+qMawYEJjWBE4hbj1AcAKWNMX8GI9nDbr+FE5n\nDW1tbei6zuRrJ9N+XTvMAC6FjpnQdQv4Oo9yw+TJMQvTx8KKvIxoNYohM6GWZpPSZbm5NDU19Wt/\nFKGonYBiQBCoCdyrajnMk3LcejDxaCQJIegc2WmY1JZTCicf6khZS8mKcN9s0+/PtklJEYqaBBQD\nBiurY4UTjy2+R3ThG2lcIcszBopbOiwZ0FLNy8i2AkfZNikpQlGTgGJAkS6nYTy2eCEEhS8U4iNy\nIrDvh7aCAsosGtBS+Z7ZVuAo2yYlRSgqRFShIL5CNUIIRpWOCpFABmAvHPMM2G3F/P3w4awJZ8ym\nUMt0h58OdVIJEVWTgELhx+12M3ny9XR0lNLR8QUKCo5QUHCAF1/8Q+9A9fDDDzP7rtkwGigE7RPg\nCNjzCnjxT39SA5oJ2TQpDTZUnoDCkGyqC5yN/TEmB00rASagaW8Ch3rf0XWdRfX13N8Fiw5BVyl0\nnwt5+8BzoIP29vZMddqQbNNZCmSodomeYIUARaZJNtU4XQ+UbIQlpKvc5GDpTzjxyDU0NDSIa4qK\nhCMPQ/kKW7Eta+Qrsk1nKdv6M9hAyUYogsm2QuxW9iewum3ZvZuyCutWt/HINez98EM+WLqUTWXg\nmRl5jvxn83nkR49kPNvVqsL1g7U/g5FUzEEqWWwQEk/M+0Dsj9vtZmxJCc1z5mBfupTmOXMYW1KC\n2x2tpEr8xBMiWlZRwY6CPDynGJ+jc2RnVtQ+yLYiPdnWH0UoahIYhKSz3GSm+hNe+erHwBqPh6f8\nr6eaqRtPdbba2loOa7nY9hmfwyr5ilTJtiI92dYfRShqEhiEpLPcZKb6E48eTirEI9fgcDhY/cIL\n+A6QNvkKK6gor8D+md3wvUxMVNnWH0UoahIYhDidTrq7XyWddYF1XcflcjF//gJcLhe6rkc9diDo\n4QTkGhyO6djtNcBi7PYaHI7pIXINV111FS81bcS21kb+s/nwmpSydmxwWCJfYQWp6Cz1/q7182P+\nrlb0p7ulG6fTmXIbihRI1qOcrgcqOigmgWIh8+bdE1EsJBCFY7NdIuCLAiYLuE/YbFMsi8ZJJtIn\n1SIo/VH5Soj4q7P1HndPfVZWcYtVpCfmZyyO4Amc23aWTXAZggoEhQjbWFtEG7339/zYxXAUElR0\n0NAhUu54a6+42Lhx48KicNqARuBPFBauZ9++3SlL9qYS6ZNKERRd1xlbUmIoRxxckETRh1mRnnD6\nI4Ln8OHDnFp+Kr4yH5QBZwGFoW1s3769TyTwBA/2z/pEAlUiXnRSiQ7K+Mo//IHaCUQlViz78uXL\n/SvtyAWz3X6DJavlhoYGy9oIrPjuumuuONZmi1naUJUeNCeVFXRDQ4PcASwi4mE/x5qdllkbhWcU\nil/96ldDooRoOiCFnYDKGB5AmIVadnSM5ze/eRCP51uGn7UqKsiqyCMp0XADXu8FdHUVciWdMYug\npLPy1UDH7XYz+drJeE/y0lXSRd4f8qi7u44X178Y1wq6PyJ4zNrwjfBx990/Iq8y3zSUNNM5GIMR\nSyYBTdMmAQ8iHc2/FUL8MspxFwB/BmqFEJFZOQpTzAZgn288f//7buB/DN+PtypVLKyofKXrOl//\n+rX4fM8BV6Ixn6/RbXhsuNNXlR6MRNd1vj7p6/hqfL0DaBddtO1t4+uTvs5nH38Wc6I0q9xW+HEh\nf/3f7bhcrpSS88za4CCI46FzVKfhZ1UoafpIOTpI07QcYAVwNdLKd4umaadHOW4JsDHVNocqZqGW\nsA34AfAO6YwKsiLS59FHH8Xnu4DAjkZQwWsYhxBus9stk2cerDz66KP4RvgMV9C+ET4ee+yxmOcw\njeDZ7eO8l15KOTnPrA1agQv9fw1QoaTpI2XHsKZpE4CfCSEm+5/XI+1Tvww77i6gA7gA2BBtJ6Ac\nw9Exc8rCdKAF2A7UAOcDF4VUpbLKsRbqnD434TbOPnscO3bcBPw48M2wUcILKKdvMpz9pbPZceIO\nuNTgzdfg7M/O5m9//VvM87jd7pDKbRyAY/bDy10Q+FVT/U3cbjdXXH0FjOigYwxwEDnw1wIjgGXA\nVJS8RIJkWjaiBPlTBjjkf60XTdNGAdcLIR4BkvNgKwxj2eEa5ASwBihG/ndtAQqZMOElli2rprW1\nxdLIikDlq2XLqqmvP5pwGx6PB3gj+JvhpZkpOLiKXBYDNXY70/3FUdR/fHM87Z5gsdNQDhG3ummg\nctuyO5cxyTaJrxws4B9BEwCknpxXVVXFr3/5a846VMD4V5EG6buAMchIoVpgNfAUWZmDMRjpL8fw\ng8D8oOemE8GiRYt6/z1x4kQmTpyYlk4NRIJLD65atYZXXjlMR0cLcgIIUIzdrnH77d9Km/08Fdv8\nV796Ifv2rYeQWlNVeHmSTdzE4TMrmTN3rnL6xslXx3+Vfav3SbNK2Aqa/TBh2oS4zxX4Xfd++CH2\nF1/C6Oqnmpw3a9Ys7l2wgFs7OnivAzyFQW+OAe6Cgt8XcMUxVzD1zqmq7oABmzdvZvPmzZacyypz\n0CIhxCT/8whzkKZpgZARDTgB8ADfEUK8YHA+ZQ6Kk1gx+7t2vUtTU1PW6fe3trZSUnIqcuL6GnAu\n0qfxGtDG4cP7U85nGIgkq5Da2tpKSWkJFCAH0ZHAYWA/0AGHDx1O+Hq6XC6a58xhjUEEQI3dTvWy\nZQkvAoLrGyDAtXIlR3weum5BmX9SJKN5AkAusAd5+xUgjdJnmBz/30CNyfsWRM0OHaJl4q5cuTKr\n9ftXrlwpwCbgbAFf8/+1iZUrV2a6axkhOAdicRI5ECtXrhTkITgRwRj/3zySvp5HjhwRwx2OmLkb\n8WKUjVx8bLGYPn26KCgqEIVnFsad2ayIhExnDPtDRJfRFyK6RNO07/o79ljYsf/FIHYMZ6KaU3gm\nrtPppLJyXMJZvf1d+evjjz+mvr6eXbv2UFlZzpIlS4bsDsCKbOje67l7F5UVlSlfz1h1geO912Nl\nI3/4/oc0NTXFldlsFQOjyl38qIzhLCFbqiclk9Wb7ZW/0k2y2bZmOk7x0l+6SMkQ0EhaUB+qkZTI\nvd4f2ciJMBjvdVTGcObRdR3nFGfIaseDB/aCc4qzX+2biWb16rqO01kTsnOQpuBNOJ01GalEZrRK\nS9fqLSQ08gQP9hftzJ03N6ZeTaSOUzNz5y5IOBw3mkKqDvR4PLh+24AQIvI6JLjbTMbnYBQAkOi9\nnk31BLLtXs8GlJS0RWRT9aRE9fuzqRKZ2+2mpGQsc+Y0s3SpnTlzmikpGcvDDz9s+HqqVcWCBzRP\njQcuBU+NB/0a+Xq0YjXBg4ksSfljPJ416PpTOJ01CRW5KauoYKs9NFnODZTkwYax8OYxbzFn+RxK\nxpTI6zCmhDnL57D07aW9r8e6DlZWZUv0Xs+megLZdK9nC2oSsIhsWu2YZfV2d78aod/+3nsf4PEI\nYAHgQq5BJclqDiVSbyD4M9EG1tmzf4Suu1IecMNJdvK2cjCpra3FnZPT+2vpgDMP9FugewZ9E9PV\nOrPnzE5qwgpUZXvc4+EkoMLj4VZd5/rJkxO+fhH3ug/YCvwRPD4PO97fEXK80+mku6U7KwrxZFvV\nvWxATQIWkU2rneCkMpvtOmRSmRO4EfgSlZXjeleAbrebRx/9L+T/ZDvQjBwR5fvJVCKLtpqPteo0\nG1hhIvBpxOuprt52vL8jqcnbysHE4U+Km+5wUGO3MwNoH0PkxOQBTjF4PY4Jq6qnh2OQu4s5ZbD0\ncnisDP7P28bixYvj7ivIe932iU0+2Y8MCdmNjA3shkceeyTk/jr97NPhJOA5epPAbM/bMpIElm1V\n97IBNQlYRCrVnNJBVVUVu3a9ixCvAO8jc/EP4fW+3ruCPnz4ME5nDV7vc8jB/8fIzOOnkNITLySs\nOZSKmcRsYIWLkJnQoaSyenO73bgefoTc/cbvm03eVg8mAYXU6mXL+OSr4+keY3DQv4CTjT9vNmG1\n7N7N2R5P7+7CMxO5i5gJXbfAA8sfSGg34HQ68e3ywk5kuYqpwM3ynNwK3hu8OKc45f3lN7V5Z3hh\nLnAG8CmIvws+fP/Dfq8RYIX21WBDTQIW4XDIVY1jgwP7GntWpLw3NTWRl3cl8DRwG31ZxXIFXV9f\nb7LyPp/CwtDSivGQipnEXCDvz8hKJKEku3oLmEie9Pko2k/Ck3c6BpOAE/a2/+d2413lcYQKtARh\nNmGVVVSwoaCAnlIMdxHaKVpCu6nm5maq8mwUPgeMMj5nz8k98v4KNrUVIvMCp0FeeR5NTU1xt2kV\n8ZYRHUqoScBCgrVX6ifUs+zOZbQeaM1YRaRYJotdu/aYrrxnz/6PhPueipnEbGCFzcDwiNeTHXAD\nJpLrgOYucDwD9ieA1yD392BbazOdvNM5mETdVdqBfSQ1Ye3UBJ5TjNvzneRLyGfVsns3V3u93NED\nlBof4xnmYdfuXVnjJwsmVe2rwYYKEbWYbNK7j6X9X1lZznvvbY36/plnJl5/IJV6A4GBNVyhVNPc\nTJlyA88//w007TQ6Oqqx23f0KpcmM+AGh2VWAa1d0NgCe1qklaN87h0xB4VgHSeZqFdNbe0TKa8m\nA7vK3rDVL3ooaClA/FNw44038sL6FxClAs8wD/bP+8ovmk1YP7irjqWrlxq+n6jPqqyigma7HafH\ng30feAzUS+2f26n8ciXvbX/PsH5ApqWhs+n/aaZRNYYHMfFoCyWSWRxPnH4qNYgDBGdAC9HNypW/\nRYhL8HjOo7DwTYR4nbq677Fw4cKkB9x0aONYTVubdNo+sPwBtFM0fCN82D+zox3QmP3d2Wg5WtwZ\ntrquM6p0FG3XtsWt0xMtHyGQ4dyg68zw+xmMzrlrxy4qz6pMa91ihURlDCuiEk1bKJAdGet94/OY\nZ1nGe85YxKqpnErNWau1cdLBkSNHLK25G5LlG0OnJ1ZGcEDr6BKbTRTkIXLHIrgMYTvLFnFccJtF\n5xQJW7FN3PqNbySdYa2IhExrB1mJ2glYT7i2UPjKMdb7yazuY50zHlwuF3PmNPujjEKx22tYtqw6\npdV6LG2cTONyuZizfI7MCQjDvsbOsjsT3630/i4mOj2xtH4CK/jAuXa+/z6ffv45w04YxplnnBn1\n/np186use34Vl+XmMr69na1Zdr0HMqnsBJRPYAgQbv8MJHIFm3WMpAECpp+PP/6I7u6LMYv4Cf+8\nFTbXdCf2ZHvh+nQkIMbzu5gl0HWf3M3s2bMZMXJEVMmK3vsryIx00003cU9dHWt8vr67yONhE1Dj\ndKrqcRlETQJDjHj0bsKPyct7i66uGYbnS2eWZSpO5nh1cqx0ECZbDyAaZoXZ0+lYNZt82oe18/RL\nT9N1dpehxlI0Habvfft7VPX0GAYjByqVZdoHM1RR5qAhRDxmHSGEwTEuYD2wLuKc4WYZKwfCZJ3M\nwWae8zyefjE7pKPNeM0yVmNmhuJp4HRkvH9YX4QQUftbsLqA+e0d3GvQ3mLgaH09v7j/fqu/ypBB\nmYMUcRFPIpcQwuCYWqSu0CbCB2MZp/8EEDkQNtvtLJg7N+mBMFrIqFloaLBOTn+ZHdLVZkSoaJwh\noalSW1vL3HlzjctVHkJmCAcIkqwQQkQ1I2ljNDbsLeDejo6I9rbZ7VSXZy5cdKijJoEsJR3FaaSN\n/Wzkyr4FmYFbCzh6zTo9PcLADu9AyklcQ35+FZ2dEyIG43QNhInG4geSwPrT7JDONgMJiLGcuVZi\nNPnkH86n8++d8A1k5m8QAf9ET09PnxnJB7yHlLo4Dnwn+Ni5N99gGQHunByeiDPhb7AVg8kG1CSQ\nhSSrbx+bbmA58r/heUi9oAXAml4buxAiih2+iqKiS7nxxuMZOfJoxGCczoEwEbt9NG1+SL1Aeqba\nzERiU/jk0/pRK6tyV9E+pj3i2IB/QgghfRj7PVJTqBQpK7EbaIGaG29i+gsvGEZjxTOpWVW/QRGK\nmgSyjHQVp9F1nYcf/m/kij58LXYdmpZDbe0TCCGYO9fY9JOb+yYrVhjb4TMx+BoRyGY18iany+yQ\niTb7g+DJR9d11o5Za2giCkhWCCGou7tODvrTIo9bt34duz/cTVNTU8LRWKoYTPpQ2kFZRrqK00ib\nbXSxuKqq87nvvv+XxsZGVq16MmFNHKPCKAG22e2U9dNAGK7NHyBgdkiHSmQm2uxvzAQSVz27imef\nfZbFv1jMxeMvjioqJ0oFTU1N3Hbbbfzi/vu57bbb4h64VTGY9KF2AllGuorTmMs0X8If//gkL71U\nhd3eTE6Om1WrnuTgwYNxa+LU1tayYO7clG2+qRLQ5o+WBJaO1WIm2swERv6Jk08+mWk3T+s1Xea9\nlwdnG38+XffvUC0GYxVqEsgykokNj8dZVlFRhs22Hq/XqNW36Oq6G7itd4s9bVr0EMxo7WXLQJhs\nEliiTsdw5/27u3bR3NyclYln8RIrICHcRBQeEtpl74Jdxue2/cNmfv+aBEGkkjOiMEflCWQZicaG\nRzrLtvZG7QQ7y1pbWykpKUfG+kf6BGS00IjeV6PJMsRqLyAR0LJnD2Xl1kSypCMiJDyfobS0lGnT\nbo15HQNEOO8/6wvdtNpJmY5IMSMS/U6G+QRHgIcAA1E5noXDBw4zYkTffRZvm1YIEw5mlIDcICNe\noa9EBNYaGhqEzXaJgOECpLCb/DtcwCUCXGHnuE/U1y9Iuj1Lr0WcwnWJnHO4wyFusNvFYhDXFxUJ\nG5qA38T1vawWdovVVzMhN6tI5jvNmz9PcHnY8dciKEVQhOB0KSrH6fK5baxNuFyupNu0SphwMEIK\nAnKWOIY1TZukadpOTdM+1DRtvsH739A07V3/w61p2pesaHewEm9xmkScZbt3t+D1Xo1c8VcDR/1/\nW4CrCC/daFSxK1nnXDJF5wOfS7ZUpdk5A/kMazwefgysbW/nBQQ2fgYEn9P4e6XLeW/U10CkWCKF\n5Y3O43K5mF8/P+r1T+Y7GdbV/hdQDtwFnAZ0+f/eBd5Sb4hPINE2VTGY9JCyT0DTtBxgBXAF0Aq8\nrWnaOiHEzqDDWoBLhRD/p2naJKABmJBq24OZeGLD+5xlOjIwuy8BLNxZ1mdTLUaWmgzmbWBK0PN1\ndHVtYseOMlwuV6/5IRnnXCqx3fFMOonGz5vlM1xCD5toJPj6GH2vdDnvjfoaa5CM9f3jzTlJ5DsF\nzFM73t9B154uWYXndP+bxyF9ApfSJy3hJ9yn1dtmWGIZZ0e/jqoYjPVY4Ri+ENgthNgPoGnas0gj\nc+8kIIR4M+j4N4ESC9od8khn7xN4vQ8gM3O+ALwE1GOznUl5+azeY2tra6PG/8MmiooE7e2Hsdle\nxut9B027kgceOC5k0DZzzhUWvsXo0ZNCXks1tjsdESFm+QyX4uGP7CHYI2XkdOwvYbdUJ5tEck4q\nyisoerGIdqIng0HkpGI71QarwVZqw1vqpegfRbT/vd00nyBARXkFtlU2vG944YvITOQ9wB/BNspG\n+fSBmV8x0LDCHFRCaPnrQ5gP8rcDL1rQ7qAn1jbe6XTi9b4FCOAU5EB7CiDwet+iurpv8DKribtx\n4x946KHrqKv7N/A3YB1e7zrCzS/V1dX+GsDrkNITC/x/1+Hzbaa+fhFut7u3zVRju80KzydbYN4s\nn+F1bAiCBx7jGsZRawDHqPWbKIbmFj/xTDaJmFtKS0tp/8A/ePuArcAfgZdAO6BRW1traJ7y3uiF\nm4FPYO65c3mo7iE2bthomE8QrnfkdDrx7vOHqx0r+8Wx8ql3nzfk/oXkzYoKc/o1RFTTtMuAbyHL\nukZl0aJFvf+eOHEiEydOTGu/spF4tvFr1qwBCoBniVzd17B69Wpmz57d+2osHR6Xy0Vu7mVEG7Sb\nm5tZsmQRs2ffAnwN+TOuQRaB/xVtbRUhK/xUV/Jmu5dg4bpEMMtn2IKPoqJ1tLcfNhWq6y9hNzMh\nt3gmm3h3ErquM+3maXA58BxyTTEGOBk4DN3d3Wzfvp2dO3dGnVRyy3I584wze0018egdrVmzBnIx\nzC6mkZD7V0lGhLJ582Y2b95sybmsmAQ+QtoiAoz2vxaCpmnnAI8Bk4QQ/zI7YfAkMBSJdxu/YUMz\ncjA2snB/jaam5pBJAMxtqrEG7fff30lDw+MYh5lOB1pCbPWpxnYnoyIaC7N8hnWrVsWdINcfwm6p\nTjbxmq16dwznAluQKqFBg/LRvUdxTnFy+7duj9s8FY/tfkPTBrlxNZhUGANNzU3Mnj1bSUYYEL44\n/vnPf570uayYBN4GyjVNGwMcRm4Obwk+QNO0UmA1MEMIEb6JVoQRv0NQA74a5SwTkO6X+Ik1aH/2\nmSOqeUfuChpDVvhWrOQTVRGNB6sqivWHkzKVySbenUTvjuE95HIuyn33+WefY//MQl+IhlwyGhH0\nejoCBBR9pDwJCCG6NU37PvAy0sfwWyHEB5qmfVe+LR4DfgIcDzysaZoGdAohLky17cFKvNv4a66Z\nzMaN64EfGxy5herqKQavRyfWoH388TPxeE7DKBpJLiNbsNs/6F3hx7OSjycRLB2D7UCKMomnr9ES\nyuLZSfTuGDo8UvcnSrTOsBOG9flCkjBPhXON8xo2PrjR+M0DUF0n7yMlGZFeVMZwFhJvgXFd1xkx\n4hTa2yN9AkVFN/PJJ/sTXt2G2l5DB+2dO3fygx88gde7E7nyPw/pQXQjYwTPx+F4MmJ7Hq3ofKzs\nY6UdH0q0gT5W1m2s4vK9WeqVugzy/ow+GehW4ADYRtpYcc8KKisro04qidrmdV1nxOgRtF/XHjGp\nFK0r4pOPPqG4uFj+f5jT7M8VCSVaZvtQQ2UMDzISyaTcsmWLKC4eLgoLrxFwnygsvEYUF6eWRanr\nunC5XKK+foFwuVy97X300UcCjjHMGAabsNuHxd1urOzjjRs3Wp4pPJCJljm8ceNGS7KXt2zZIuxf\ntAvyMDwX+YjDhw8LIYLuj3vqQ+6PZL9X8bHFovDMQsFliMIzC0XxscUhv3MmMtUHGqSQMax2AllK\nyOouxoor2ko7EeJZdbtcLn7wg/X+8NFQ8vOrufHGYxk9ekxcq3az1V1R0bX09Ljxep9D6cSY60nZ\n1trIGZND+40G8f1Bu0azcwd2FwcPHGTVtlV03tIZcZzteRsr6laEnMsqTaNYOxUw36EOxeigcFSN\n4UFIIg7BZO3bgf/Er776GmvWbCAnZyLt7edHDb+T0hPGrpzOzq/y3HNP0tV1Zlzhe2Z23vb2XPLz\nLyZ0AtCB/Rz1Hcvs2bNZsWLFkDAN6brO97//fbwneQ0dtt2ju/F2G0rDxkwoCzcj5b2XR9fZXYbH\nek8MlXwwCmGuu7uOO75zB2gkNCnEc/+mI0BAIVGTQBaTTudlYGXV3T2e9vYtwPPECr8zix6CtyPk\nqM3C98zOlZe3k87OGcG9hTwnlPbQdYqHZ/6yj7Vj1qZFsTObCPxGR33H0lUVuToH6BzZSd6OPLqI\nHLzNInaMwpDNZKCDz2UWwrz0waUwAQtLovYxkJz5AwlVWWwIEhx33d5+HTJLKHZWb21tLZrmBsMa\nWm8go4Sifz6Y2tpaf/Zx5Llycz/Cbn8n0Fs5Adyiw0yZpdp5c2fCImoDjeDfqKtjHuwzzhwu+ryI\n3LbchLOXDcOQz0bm+xucK5A1HPWzIJ+XAccmJ3SnyAxqJzBIMbPxh8ZdL0BG+UQSHn7ncDi4445v\nsXRpjf+z5yJzEdzABqDY9PPBmIWPrlq1mmnTbkVOEPuhNHrOxOOPP05hYeGAjyAKt697vd6g30iH\nA3WGoZm5B3NZu3ZtX3WvOBPKDMOQC5Hz+NPI6KDRyMyfFphdN7v3XGYhzIxEhpaCodBdf9VGUMSP\nmgQGIbFS7EPt8WVAs+F5jLN6c4E6pK5ACzAMKRkZueWPlRVsZudtbl7D1VdfR3v7F+CU6DkTdXPv\npiB/0oCWEjCyr3fu6aSjfab/iHdBdEt1kFORylwHoOjTIpqbmpNKKIuaTTzG/ziGPhno4aDlaLE/\nC3LSOK3vabBfIl5FU0X/oqKDBhnxVGB69tlngyJzdOTyMvJ4m+0mWlreZ+TIkb2vRkb1RP98KpE8\nuq4zalQZbW1fhrLXYKaBTfz3wN464DeWtdvfmEX+8EwedB2AvEppDhsN7ECutDuheGcxhw8eTvr6\nRm13NbIeQKF8KTzKSNd1RpWOou3atrg/e9NNNyVUMU+RGKlEBymfwCDDOMXeH1lz9Hhmz56N0+kM\nssc7kCJw04FrgcWAE7gRIb5EWdmZTJ8+o1e1MdKWH/j8zcDVBKuThuv7JKIC2djYiBCXyHMfsBna\nqfQLtXoAABgsSURBVNmfB9wb9kZ8CqXZgql9fUwXMLPPHFaItMBdAUwCUSpMv6eZCm0gmzhY7ZPf\nA6uQJiH/IB7sWwic7z++9x90+Dpk0vjT9H32OePPOp1Ovv/973M0/yj8G5mRHPQ9rSzEo0gcZQ4a\nZESGXrqBGqCKrq4ZPPPMG6xd+2WWLFnE/PnT6ewcj883nvz8cXR2bkQO6lOB5/D5ioFNPP30jfzh\nD//Xa2oxsuVrWg6zZ38FTTtqGL6XqApk3/dwQFczPCOjgxjjgf355HzUTU/XNwn3Q8DAkhIwta+X\ngrbvVcQp3YZvm4WAxmN6CTcjiQsFKx9didgq8OwL9S1s374d5xQn3aO6ad/TDjcRujMZDuyDgj8X\n0LGvg8KPC8k/nM+SXyyh8qxKvCd5ZfjpbuBPyMliTOzvoUg/ahIYZISGXurICaDPVNPZCZ2dm7j7\n7pv9W8iPgD/S09MCXIZc2gVzJXA57e3VwA96wz537XqX+vp6du16kcrKcpYs+WuI2SiYZFQgQ79H\nFXS1QksjtOyhoGAjU6eezgsvfJ60Qmm2EEvp87ra61i9fTW+kOVz3/tGIaCJFJMJD7u866675O+6\nexeVX65kyUtLKC4u7jPl/Bvopm/nElw97FPo/kc3eEF0y2zUu++521AWgkZ6zUZWFuJRJI4yBw0y\n+sw164DvI3X79iMnhABX0t5+Ph7PTDo6tgGb6e6eiHTwGiHF4QKmlsWLF1NZOY7Vq3XeesvJ6tU6\nlZXjQgrKBJNMcZlIs1OgLOblFBYe4te//nXUEFNNe51//etfTHJOYtIkJytWrMiaAiS6rrNixYre\nvv373/9GO6BFDfH81a9+RcHhgoRCQJOtgex2u6k8q5LV21fz1jFvsXr7airPqmTx4sV95/sXUlPI\niFLoPrkbvgUdt3fgOcND+/CwCcCHnEiOQcYj7LS2EI8icdROYJDhcDiCCr9cCsxA/m9bgLTdj0Mu\nw3zIzKDA4LgWmS9gxFbgGkCaWn7zmyV0dq4l3lV9MiqQsRRIR44cafh+T8+rdIqj3P2fd0tzw74C\nNtZtYt68n/Dyy+szGoXidru5uvpqOTAG+vanTRTk2ChaV4R2ihYR4jly5MiEawokU5bSbPfwwPIH\n6Pyq3zF/HNKkY8RHwD76bP6fISOMtiJzED5G3nqlyNtwP7Aaljy4RDmFM4iaBAYZuq5TX78I48Iv\n1wL5yHDOK5Ex/mORxd4mIhO+jGpubUZ6/mQtYSFOIxFtd+PsYClHnZf3JK2t49F1PSJePJZUQPj7\no0dfQf3CTRy91te3+ry0A/bC0Wd8TJ58A4cP/z0jA46u60y+dnKoacTft45nusnvLuKX3/k5hz46\nFBHimUgIqK7rfHz4YykBYe+Sg29h3/vRTC9muwfteI3CjwulSeps5C1hkLPAIWSRmNeB/0X6DMbS\n5wfoRvoSwj5Xv7CemTNnqokgQ6hJYBAR0Jk5ejTYBBQYWCcgl2WRomyyvt9dwHwCTmRpAtoGvAJM\nQZpjNiHE63R03GXYfrRVfWSdgoCz+qt0dc1g9ep3WLt2rKGTOJZUQEAJsUd08Ze//IWOEzuMI21K\nc+g4dHLGCpA0NjbSObIzat86D5Xwl7/8hREjSoIVdXuJRzIh4AzuPrlbOmF3EeqENTEhme0eOso6\nKNha0DfwVyBvo1ORyWGHgQP+dvYAf8FwsKeR0CIyftPQ0byjQ0oPKttQPoFBgtvtZtSoU/n977fR\n1TUCeBj5vzRgp28keinKi4DHkXv0FqAaOOr/eyGg94Z9zpnzH9jt7xn2IVrx9+Ai90VF1yJNS08h\ndys/xuNZ21vMPhGJAbfbTcmYEuYsn8PSt5fyzF+eoWN/h5z/whnjoaOjOGNRQ7v37MY3MtK5C/j7\ntpNnnvmcpUvtzJnTTEnJ2Kg+FiOCzTntNe3SEvgNZKDX01D0fJFhsfcApkXt/89O3Q/q+sJJO5CD\n+Wn0JZTdBYwA3kXWJjaa7E5BRhOB/I0eBN6GrqIuntz4JKNKRyX0nRXWoCaBQYCu61x11bW0tQmE\nqEAO7KcgK4ZfC7QhB/dopSgvRm4KTwG2Ix2wvwBOIC/Pzfnnf8zUqQ527XqXhQsXhjlkdcAF3EJX\n1yacTqdhCwHTzbRpx5Ofn5iTONp3Dgx6npo+TSFuos/lEcx+OwUFbYaTVH9QUV5B4eFC4zf3A8ym\ns3MDclJck/CkaGbOyT81nxvPv5HWA61RfSJOp5OuPV0yT2ArfdfPv3tYuHAhu3bsYuqXp3L+F88n\n71AeFCFzFs5F2vsfRCaUlxo2ITOd/+U/dyAI7VjZx57jemjztXGV8yqlNdTPKHPQAETXdR5//HE2\nbHgREDgcdo4e7UI6fsNNPTXIVb4X+b/bqBTlNqTjuBSYDMzBZnsNr/cd8vKu5J13LuSDD7aydu24\nkDyBzs7T8Xr/itxJXIymtVNZOS5q7H9xcTEjRpTQ2XlaxHuQWHy/aZJVIH49EL64FzjQQ4HtYMai\nUGpra6m7uw7fXl+kmeSAedJbPOYrM3NO58hORo4aGdXmHjAjaadqcCKwE3gZbKNt5P8zPyRPIJB3\nYCu30bW6C1upDe8oL7yFNAf9m+iO4wPIfILtyPXJNCKuxdHGo1xw4QWMOWUM1zivYdasWcpElGbU\nJDDAePnll7n22hvo6BgBnIFcRu4EJmFs6vkasAi5/OrB2PHrBp4AiiksnMill/6FLVv+BqzD6w2O\nAFrH178+mTvu+C4//ek8Fi5cTLAEtdcLXm/yEtKJxPebJlmVAFvwJ0rnw4EecoXGHXf8PxG29v7k\nju/cwW8e+g1dJV1yvt2fCwd6wIKkt1j5BolISgOwF8Qawc9+/jNWr1nNI48+gm+KT1YRBbx45QS2\nFq4su5LXy1/HN9Ynb7M/Yew4PgAcpG/TaZglDTv37GRn1042LtpI3Y/qmHvXXBYuXKgmgzShzEED\niJdffpmrr76ejo4rkOGcryFHvHKim3omAJ3AaHJyOpFG4muQ8hA1SLmINQQGIZ9vPLm5eeTmXkbo\nZOEGvo3PdwkPPHAcCxe+hs/XjQz4DibUrBMuFREqWRGMLGYfa6UeON+72/8W1bxi+4eNPL0Q7bUT\noeVU6DqN7u48HnrojYRt7akQ6OuM6dM5cdSJLFu9jK6eLjk5tUCOR5Cbl4PNZiTir1NQsIbt726L\nKbEB/ryKQBH4YIKkG4wkJKLuqPLA1+njx//1Yx5890F8JT54gVB/y1jILcslNy8X3wi//SigRLoa\nKXj3GtL90wg5Ige+gIwSCnYQBzMaeUwPoEPXWV0sXb1U+QvSiBKQGyDous6JJ5b6Sy5OIFS0bQWw\nHtho8MmrgXxyc1/hl7+8j5/+9DXa249D7t/vRv6P7VuF2u01XHKJj5deuog+01F0kTg5ibQQupJd\nTH39UaqrJxsWkl+yZBH19YsSLhUYKj1xNuTdD7d0GQivadD1B2RUU3hfXTgct6ddYM7tdlPjdDK+\nu5uXO9rpmIYcRKcSR3/dkHcVjDkKY4goHG/WplE+wZLFS6hfWG9YiH79hvUsfXtpaJ6gD1gWpa9h\nAnG8BpNtk3n9o9elbyb4HDuA/wFGQ1FXEfd+617mL5hP95huOcgH1w0K8HugEhmPENzeISU0Z4YS\nkBsCNDY20tV1EXIQbqQv1h9gFvAOxsVe3gGO8qMf3cl3vvMdcnPfRO4A/om0SRSHHJ+T46a6ejJ2\n+9bg1sPaC3Cl//VQZ67dvo3Ro0f1SkVIxdE+h2d9/SI+/PCvLFtWTX39UZYtq6a1tcV0gAuWnpDn\nuxe6XoVnjpEDx2tSsdK21oYtr4rQCSC4r5+mXWBO13VqnE6e0nWua28nvxTwIC+3gQnEVllIYeF0\n7PYa4CeQdxncclQOkpfGX6AlkE+w7M5l1E+oZ9mdy9i1Yxf1C+tDHOjB5zt59MmRUUHvRe8rpfRF\n+CBNTdXV1ZG7kELgi8ggszMgtzWX7373u/xwzg9lRFErxqKArcggtfD2lNBc2lA+gQHC7t0t/klA\nR5pv2pFRObXIXIB1yMH9fKSj9i3/YyHFxfezcOG6Xp3+Pqfujf5jL8Jme5v8/Ddobl7DuHHjWLDg\nXvr8By1EKzzTJykRQE4kQlxuKhXR1NSUULy+sfREFXT9g4KDl3JF5Qim3jmV9977gAcfLPJfm93I\noPbANToX+ACPp4eGhv9CCJGWoiaNjY2M7+5mP9AAeAqAT4kqt+A90cvcSXM584wzWb16Da/sz8U3\nNqxcpEGBFiPC8wlcLpephISmaX0DeOAYM2mI4KIxflPTrFmzGDduHM4pTjpHdeI90SsTxw74ncsb\n83tDUxcuXMjDjz1M26VtcpU/GmnRPIicAG5GTiA+5GR0FPgAOEsJzaULSyYBTdMmIQPEcoDfCiF+\naXDMQ8jQEw/wTSHEdivaHipUVJSRm/tburuXAeORg2GwHEQVcjAeT07O/9LTM4Giognk5t4fIukc\nnGX7/vs7+fzzTxk27P8488wp1NY+1XuclJ64DplJXEBkzGWALRQUfEpHxzEhZp3165sTloowI7r0\nRDEdHTX8/+2dbYxU5RXHfwdwWZmZIgii5UUL2AK2q7Gtu0ZSV5u2sLRWbVKjjVW6m/qhUVNTu9iQ\nSNIP4pe2tmiNi7ba1mpjTUQNitbdOiZCtYCAgi6+oItIK2nLLriu4OmH5y47zt65c3fmzp3ZveeX\nTPbembPP87/PnJkz9zxvZzZ8QGtrK+3t7S5NNGei24zmrRS8fYNbiZQNwHbgXDZtamHHjspsQvP3\nzk42DBymcy4cOg3XIboVN/LGb3mmt+G4c46jtbWV17pfY/2H/pv8lPIlWGwJiZ69PcOWpah7s46B\n+gH/At/BDQP9E/DW0JIPubOaX9n5CgfmHODEi05k0cJFn5jdnMlkWP/oejep7TNHOXzkMONeHMfH\n9R8PpZn2MLS8xDxcQLkNxk8bz/zv2UJzUVN2EBCRcbik9FdxsfwFEXlEVXfl2CwF5qnq6SLSCNyJ\nS2wbIWlpaeHo0evwXw7iUlwAeJ50+t+sXn0zPT3vMn/+XC677P5hOdRis0+Hlp54APcTdidwN34j\ni9LpLTn1DS3rsGvXrkhGAQ0SZlRRb28vd9x1h9dP4P2S/sohL+/+dTiiuMR8uDWPSqG3t5eH1j3M\nwOUwMCz3jxvItSDv+T3Hc/vtd7Ny5cqSR/kUIkx5+ctSzPrOLFasXEHf6z6bxvTgbqimAV/85JIP\nYTeCH1bfzFnceNONfNDzgbszeBDf/oijfz7KsmWjY3XYUcXgFPVSH7gv8/U55yuA9jybO4HLcs53\nAjMKlKfGcDo6OrS+/lsK6vNYprBI0+npms1mI6krlbokr46swnSFpQo/11TqEs1kCtd38OBBzWSm\nK2zIK2eDZjLTtbe3d0SawpTX0dGhqYaUsorhj7kofN63/VKpS3Tt2rVlt5uqa7tJDZP8NcxHmeD9\nvQBlbr0yIaOQPabh4MGDmpmSUa7M+98r0cyUTGntVkJ52WxWM1Myrj0vQDkdZRLK8k+Wk2pIRdJ2\n2WxW0yekleleXT7tV39GfWTv01jD+94s6Ts8io7hwYzeID3ec0E2e31sjAC6u9+gv7+xwKtNNDWd\nwL59wZ2rI6lreOplMN00kaamJ4p25uYuFeE6PAvvOBaGMOUFzh04FdwSmMOJchOa7t3dHJ522P/F\n2cCRabD7Kui8Ad5Y4/ZJYPExDX47fqUeTgUu+RBEqeXldjI39je6FND1HNsIZpCo8vSLFy9m3zv7\naJzX6NrJh/6T+q1PoALUZMfwqlWrjh03NzfT3NxcNS21QnA65J+0tf0gsqFzhetKk0oJbW3Lw9/2\nB6wCOlKKlReU+pj43kS0ro8Bn1R3lJvQBGmoe68O6mYzMPD7QA2lbBwfRKnlDaZ3VJUdv9nBoYnR\npKiC6mtrbXN1RZQOG6t0dXXR1dUVSVllzxMQkSZglaou8c5X4G5Nbs2xuRPoVNUHvfNdwPmqut+n\nPC1X01gkzAbyUQWBOOuKkqDN09OPpuFIPX1991PJa6oFDVETdE1Rj92Ps66xRDnzBKLoExiPW0D2\nVNwwkq3AwjybFuBxHepD2BhQXqS5srFENpvVTGa6l68vnpcfLXVFSX4uO9WQ0syUjGaz2diuqRY0\nRE3QNY3musYKlNEnEMmMYW+I6G0MDRFdLSLXeMLu8mzW4Ba4OQQsV9XNBcrSKDSNVfr6+nLSIXPL\nShPUUl1Rcky3T+ojrmuqBQ1RE3RNo7musUA5dwK2bIRhGMYox5aNMAzDMErCgoBhGEaCsSBgGIaR\nYCwIGIZhJBgLAoZhGAnGgoBhGEaCsSBgGIaRYCwIGIZhJBgLAoZhGAnGgoBhGEaCsSBgGIaRYCwI\nGIZhJBgLAoZhGAnGgoBhGEaCsSBgGIaRYCwIGIZhJBgLAoZhGAnGgoBhGEaCsSBgGIaRYCwIGIZh\nJBgLAoZhGAnGgoBhGEaCKSsIiMgUEdkgIq+KyJMiMtnHZpaIPCMiL4vIdhG5rpw6DcMwjOgo905g\nBfC0qn4OeAa4ycfmCHCDqp4BnAv8SEQWlFlvrHR1dVVbwjBMUzhqURPUpi7TFI5a1FQO5QaBbwP3\nesf3AhfnG6jqe6q61TvuA3YCM8usN1Zq8U03TeGoRU1Qm7pMUzhqUVM5lBsETlLV/eC+7IGTgoxF\n5DTgLGBTmfUahmEYETChmIGIPAXMyH0KUGClj7kGlJMGHgKu9+4IDMMwjCojqgW/t4v/s8hOoFlV\n94vIyUCnqi70sZsAPAasV9XbipRZuiDDMIyEoqpSyv8VvRMowjrgauBW4CrgkQJ29wCvFAsAUPqF\nGIZhGCOn3DuBqcBfgNnAHuC7qvpfETkF6FDVb4rIecCzwHZcukiBn6nqE2WrNwzDMMqirCBgGIZh\njG6qOmM4zGSzHNtxIrJZRNbVgq64JsGJyBIR2SUir4lIewGbX4tIt4hsFZGzKqFjJJpE5AoRecl7\nPCciX6i2phy7L4vIRyJyaS1oEpFmEdkiIjtEpLPamkTkUyKyzvOl7SJydQya7haR/SKyLcAmbh8P\n1FQNHw+jK8cuvJ+ratUeuL6En3rH7cDqANsfA38E1tWCLuBk4CzvOA28CiyIWMc4YDdwKnAcsDW/\nDmAp8Lh33AhsrHDbhNHUBEz2jpfUgqYcu7/hBilcWm1NwGTgZWCmdz6tBjTdBNwyqAc4AEyosK7F\nuKHj2wq8HquPh9QUq4+H1ZXzPof282qvHVR0shm4X91AC7C2VnRpPJPgzgG6VXWPqn4EPOBpy9d6\nn6djEzBZRGZQOYpqUtWNqvo/73QjlZ8cGKadAK7FDVP+V4X1hNV0BfBXVd0LoKrv14AmBTLecQY4\noKpHKilKVZ8D/hNgErePF9VUBR8PpctjRH5e7SAQdrLZL4EbCZiHUCVdQEUnwc0E3sk572G4s+Xb\n7PWxiVtTLm3A+grqgRCaROTTwMWq+lvcXJdKE6adPgtMFZFOEXlBRK6sAU1rgEUi8i7wEnB9hTWF\nIW4fHylx+HgoSvHzcoeIFqXcyWYisgzYr6pbRaSZiD7ANgmu8ojIBcBy3C1stfkVLrU3SC0MRZ4A\nnA1cCKSA50XkeVXdXUVN3wC2qOqFIjIPeEpEGsy3/akxH4cS/LziQUBVv1boNa+DY4YOTTbzu305\nD7hIRFqA44GMiNynqt+vsq7BSXAPAX9Q1UJzJMphLzAn53yW91y+zewiNnFrQkQagLuAJapa7PY1\nDk1fAh4QEcHlupeKyEeqWqmBBmE09QDvq2o/0C8izwJn4vL21dK0HLgFQFVfF5E3gQXAixXSFIa4\nfTwUMft4WEbu53F0ZgR0YNwKtHvHgR3Dns35xNcxXFQXLk/5iwrqGM9QR14driNvYZ5NC0OdZk1U\nvhM2jKY5QDfQFJMfFdWUZ/87Kt8xHKadFgBPebaTcHNpFlVZ0+3Azd7xDFwaZmoM7+FpwPYCr8Xq\n4yE1xerjYXXl2YXy81jF+4icCjyNG1mzATjBe/4U4DEf+7iCQFFduDuUo94HaQuwGfeLIGotSzwd\n3cAK77lrgB/m2KzxPtwvAWfH0D6BmoAO3KiSzV7b/KPamvJs76l0EBjBe/cT3AihbcC11dbk+fiT\nnp5twOUxaLofeBf4EHgbdzdSbR8P1FQNHw/bVjm2ofzcJosZhmEkmGqPDjIMwzCqiAUBwzCMBGNB\nwDAMI8FYEDAMw0gwFgQMwzASjAUBwzCMBGNBwDAMI8FYEDAMw0gw/wdb+jKh0/XjoQAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x117966550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import math\n",
    "import pandas as pd\n",
    "\n",
    "np.random.seed(1234)\n",
    "angle = np.random.rand(100) * math.pi/2\n",
    "r = np.abs(np.random.normal(0, 0.5, size = 100))\n",
    "df0 = pd.DataFrame(np.c_[r * np.cos(angle), r * np.sin(angle), np.repeat(0, 100)], columns=[\"x1\",\"x2\", \"class\"])\n",
    "df1 = pd.DataFrame(np.c_[1 - r * np.cos(angle), 1 - r * np.sin(angle), np.repeat(1, 100)], columns=[\"x1\",\"x2\", \"class\"])\n",
    "df2 = pd.DataFrame(np.c_[1 - r * np.cos(angle), r * np.sin(angle), np.repeat(1, 100)], columns=[\"x1\",\"x2\", \"class\"])\n",
    "\n",
    "X = np.vstack((df0.ix[:,:-1], df1.ix[:,:-1], df2.ix[:,:-1]))\n",
    "y = np.concatenate((df0.ix[:,-1], df1.ix[:,-1], df2.ix[:,-1]))\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.scatter(df0[\"x1\"], df0[\"x2\"], c=\"b\", s=50)\n",
    "plt.scatter(df1[\"x1\"], df1[\"x2\"], c=\"r\", s=50)\n",
    "plt.scatter(df2[\"x1\"], df2[\"x2\"], c=\"g\", s=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scikit-learnによる予測結果など:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.876666666667\n",
      "[ 0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "  0.  0.  0.  0.  1.  1.  0.  1.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.\n",
      "  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  1.  1.  1.  1.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  1.  0.  1.  0.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "  1.  0.  1.  0.  1.  1.  0.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  0.  1.  1.  0.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  0.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.\n",
      "  1.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  0.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  0.  1.  1.  1.  0.  1.  1.  1.  0.  1.]\n"
     ]
    }
   ],
   "source": [
    "svc1 = SVC(kernel=\"linear\")\n",
    "svc1.fit(X, y)\n",
    "print(svc1.score(X, y))\n",
    "p1 = svc1.predict(X)\n",
    "print(p1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "自前の「投票」による予測結果など:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8766666666666667\n",
      "[ 0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "  0.  0.  0.  0.  1.  1.  0.  1.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.\n",
      "  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  1.  1.  1.  1.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  1.  0.  1.  0.\n",
      "  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "  1.  0.  1.  0.  1.  1.  0.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  0.  1.  1.  0.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  0.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.\n",
      "  1.  0.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.  0.  1.\n",
      "  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  0.  1.  1.  1.  0.  1.  1.  1.  0.  1.]\n"
     ]
    }
   ],
   "source": [
    "svc2 = Voting(SVC, kernel=\"linear\")\n",
    "svc2.fit(X, y)\n",
    "print(svc2.score(X, y))\n",
    "p2 = svc2.predict(X)\n",
    "print(p2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "予測結果が一致しているかどうか確認:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "print(np.all(p1 == p2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## まとめ\n",
    "\n",
    "任意の二値分類器を「投票」を用いて多クラス分類器に持ち上げるためのクラスを書いてみて、scikit-learnの標準のSVCクラスによる多クラス分類と結果が一致することを確認した。 ほんとこんな簡単な方法が使われているんだなぁ。ビックリした。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}