Code for the blog post "Improving trust in text classification using HF and BaaL"

README.md

The code should run as is with the following dependencies:

pip install transformers datasets baal matplotlib tqdm

notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fa54f9f8",
   "metadata": {},
   "source": [
    "# Improve trust in text classification models using HuggingFace and BaaL\n",
    "\n",
    "As we introduce more deep learning models in production, it is essential that users trust decisions' made by our models. \n",
    "\n",
    "One of the worst experience for a user is when a model makes a wrong prediction with high confidence, they would stop trusting the model as the confidence does not match the expected performance.\n",
    "\n",
    "This is what we call **calibration**, we can compute how well a model is calibrated using the **Expected calibration error** or **ECE**. This metric can be summarized as the weighted average of the difference between a model's confidence and its accuracy at multiple bins of confidence. Below, we have a visual explanation coming from the excellent paper of Guo et al. 2017.\n",
    "\n",
    "![](https://i.imgur.com/WZCdroM.png)\n",
    "\n",
    "We want to minimize the gaps in this diagram. In this post, we will improve a model's calibration using HuggingFace and BaaL."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95936ab2",
   "metadata": {},
   "source": [
    "#### Load our HuggingFace Pipeline and Dataset.\n",
    "\n",
    "The HuggingFace ecosystem is simple to use and in just a few lines of code, we can have a pretrained model and its associated dataset. We will use the well-known SST2 dataset along with a DistilBERT model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "5abd21ca",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Reusing dataset glue (/home/fred/.cache/huggingface/datasets/glue/sst2/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a089f827c6be4730ae9a4a0121057a38",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from datasets import load_dataset, Dataset\n",
    "from transformers import AutoTokenizer, TextClassificationPipeline, AutoModelForSequenceClassification\n",
    "import numpy as np\n",
    "\n",
    "def load_model(checkpoint_path, use_cuda=False):\n",
    "    model = AutoModelForSequenceClassification.from_pretrained(\n",
    "        checkpoint_path\n",
    "    )\n",
    "    tokenizer = AutoTokenizer.from_pretrained(\n",
    "        checkpoint_path, use_fast=False\n",
    "    )\n",
    "    device = 1 if use_cuda else 0\n",
    "    # We set return_all_scores=True to get all softmax outputs\n",
    "    return TextClassificationPipeline(\n",
    "        model=model, tokenizer=tokenizer, device=device, return_all_scores=True\n",
    "    )\n",
    "\n",
    "pipeline = load_model(\"distilbert-base-uncased-finetuned-sst-2-english\", use_cuda=False)\n",
    "\n",
    "TEXT_COLUMN = \"sentence\"\n",
    "LABEL_COLUMN = \"label\"\n",
    "dataset =  load_dataset(\"glue\", \"sst2\")[\"validation\"]\n",
    "\n",
    "def output_to_probs(predictions):\n",
    "    # Get the score directly\n",
    "    return np.array([[p['score'] for p in pred] for pred in predictions])\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c707e50",
   "metadata": {},
   "source": [
    "#### Use MC-Dropout for better predictions with BaaL.\n",
    "\n",
    "BaaL is a Bayesian active learning library that will help us improve ECE.\n",
    "\n",
    "To do so, we will use Bayesian deep learning to gather multiple predictions for the same input. The key idea is that by drawing multiple sets of weights from the posterior distribution, the average prediction will be better than a single. This is not unalike Ensembles, but without retraining, we will call this a Bayesian Ensemble. Generally Ensembles are better, but require more computational power.\n",
    "\n",
    "While we have ways to separate the model's uncertainty from the data's uncertainty, we will focus on the predictive uncertainty which is ultimately what will affect the calibration of the model.\n",
    "\n",
    "Next, we will compare the regular model's ECE and its Bayesian alternative. BaaL will help us prepare the model and compute the ECE.\n",
    "\n",
    "To prepare the model, we simply do:\n",
    "\n",
    "```python\n",
    "from baal.bayesian.dropout import patch_module\n",
    "pipeline.model = patch_module(pipeline.model)\n",
    "```\n",
    "\n",
    "This will modify the model of our loaded pipeline to use Dropout at test time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "733246b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from baal.bayesian.dropout import patch_module\n",
    "from baal.utils.metrics import ECE, Accuracy\n",
    "from functools import partial\n",
    "from pprint import pprint\n",
    "\n",
    "def evaluate_pipeline_on_dataset(pipeline: TextClassificationPipeline, dataset: Dataset):\n",
    "    predictions = output_to_probs(pipeline(dataset[TEXT_COLUMN]))\n",
    "    labels = np.array(dataset[LABEL_COLUMN])\n",
    "    ece = ECE(n_bins=20)\n",
    "    ece.update(output=torch.from_numpy(predictions), target=torch.from_numpy(labels))\n",
    "    accuracy = Accuracy()\n",
    "    accuracy.update(output=torch.from_numpy(predictions), target=torch.from_numpy(labels))\n",
    "    return {\"ece\":ece.value, \"accuracy\": accuracy.value}\n",
    "\n",
    "standard_metrics = evaluate_pipeline_on_dataset(pipeline, dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "59062120",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 14%|███████████████████████████▎                                                                                                                                                                   | 1/7 [09:57<59:46, 597.69s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'bayesian': {'accuracy': 0.9036697149276733, 'ece': 0.0636752943077341},\n",
      " 'standard': {'accuracy': 0.9105504155158997, 'ece': 0.08026097557686887}}\n"
     ]
    }
   ],
   "source": [
    "def evaluate_pipeline_on_dataset(pipeline: TextClassificationPipeline, dataset: Dataset):\n",
    "    predictions = pipeline(dataset[TEXT_COLUMN])\n",
    "    labels = np.array(dataset[LABEL_COLUMN])\n",
    "    ece = ECE(n_bins=20)\n",
    "    ece.update(output=torch.from_numpy(predictions), target=torch.from_numpy(labels))\n",
    "    accuracy = Accuracy()\n",
    "    accuracy.update(output=torch.from_numpy(predictions), target=torch.from_numpy(labels))\n",
    "    return {\"ece\":ece.value, \"accuracy\": accuracy.value}\n",
    "\n",
    "# We now modify the model to do MC-Dropout\n",
    "pipeline.model = patch_module(pipeline.model)\n",
    "\n",
    "# We need a custom inference function to aggregate predictions\n",
    "def inference(sentences, pipeline, iterations, **kwargs):\n",
    "    preds = np.array([output_to_probs(pipeline(sentences, **kwargs)) for _ in range(iterations)])\n",
    "    return preds.mean(0)\n",
    "\n",
    "bayesian_pipeline = partial(inference, pipeline=pipeline, iterations=20)\n",
    "bayesian_metrics = evaluate_pipeline_on_dataset(bayesian_pipeline, dataset)\n",
    "\n",
    "\n",
    "pprint({\"standard\": standard_metrics,\n",
    "       \"bayesian\": bayesian_metrics})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bf49eb7",
   "metadata": {},
   "source": [
    "## Impact of the iterations parameters\n",
    "\n",
    "Using 20 iterations, we improved our model's calibration by a significant margin. This is quite good!\n",
    "\n",
    "Let's investigate how more iterations means better ECE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3f79ccf1",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [27:11<00:00, 233.01s/it]\n"
     ]
    }
   ],
   "source": [
    "# Get the metrics at different number of samplings, we get the average over 3 runs.\n",
    "from tqdm import tqdm\n",
    "metrics = {it: [evaluate_pipeline_on_dataset(partial(inference, pipeline=pipeline, iterations=it), dataset) for r in range(3)]\n",
    "           for it in tqdm([1, 5, 10, 20, 40, 60, 80])}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "f77f7c12",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA4TElEQVR4nO3deXhV1bn48e+bOWHIDBKSGCogg5CAiOAIYhUHRL1VnFutV9te7eRtqx3Ueuu97a2/Vm1rK9exWqfaWlERrAM4IaMEGUJBRBLCEIaEMUCS9/fHWiccjgnJSXJyMryf5zlP9rzffc7Oec9aa++1RVUxxhhjmism2gEYY4zpXCxxGGOMCYslDmOMMWGxxGGMMSYsljiMMcaExRKHMcaYsFji6OBEZI6I3BjtOLoLETldRFZHO45IEZEnROQXfrhTHKuIfE1E3j/K/EtEpFRE9ojIqPaMra2JyOsi8tVox9EUSxxdWGdIOiKyXkT2+3/6nSLymojkRSseVX1PVY+P1v7bUxc61vuAW1S1p6p+HO1gmktE7haRp4Onqep5qvpkM9ZVERkYueiOzhKH6QimqGpPoB+wBfhdlOMxncuxwIqGZohIXDvH0i1Y4mgFEckRkb+JSIWIfCYi3w6aFysiPxaRT0Vkt4gsDvySFpEhIvJPEdkhIqtF5PImdnWciCwQkV0i8rKIZATtZ5yIfCgilSJSLCIT/PR7gdOB3/tf878XkZ+LyO/8/HgR2Ssiv/bjySJSHdh2Y9v181JF5FER2SQiG0XkFyIS6+d9TUTeF5H7fAniMxE5rznvp6pWAy8Cw4L2dYGIfOyPvVRE7g6a95qI3BrymSwTkUuaep9F5HwRWek/m40i8p9++gQRKQta7vagz3BlYNstOVYR+ZHf124fzyQ/fayIzPPv9Sb/WSUEraci8i0RWePX/S8ROc5/PrtE5IXA8oH4/bm3zZform4kntBjXS8i/+nfwyoReV5EkoLm/9DHVy4iN8pRfvWKyPUissrHu05Ebg7dr4jcJiJb/TavD5qfKSIz/LEtAI5rZB+JIrIHiAWKReTToOP4kYgsA/aKSFwT5/MAEZnrY/2nf/+fbug9Ctr+2X44Jugc2e4/i8D/UIF/j74qIhv85/ETP28y8GNgmrj/z2I/vb6WQEQG+riq/LrP++nv+lCK/brTGnp/IkpV7dWCFy7pLgbuBBKALwHrgHP9/B8AnwDHAwIUAplAD6AUuB6IA0YB24BhjexnDrAROMGv+zfgaT+vP7AdON/H82U/nh207o1B2zoL+MQPnwJ8CswPmlfczO2+BDzs4+kDLABu9vO+BhwC/h33D/1NoByQRo5vPXC2H04BngT+HDR/AjDCxzESVyK52M+7PBC/Hy/0cSY09T4Dm4DT/XA6MDpof2VB27wMyPH7nwbsBfqFe6z+PCgFcvx4AXCcHz4RGOfjLABWAd8NWleBl4HewHDgAPAW7pxLBVYCXw2Kvwb4DZAInOljPt7PfwL4RSPHut5/ljlAho/jG37eZGCz338K8LSPa2Ajn+sFuC988THsC3mPa4B7gHjcebYPSPfznwNe8J/hCbjz//2j/C8eEYc/jqVAHpBM0+fzvKD36wxgN4f/x454jxo4Z78DfATk+vUfBp4N+owV+D8fR6H/7Ib6+XcH9hPy/36jH34W+ImPOQk4rbFjbvfvv2jtuLO/gJOBDSHT7gAe98OrgakNrDcNeC9k2sPAXY3sZw7wy6DxYcBB3BfVj4CnQpafzeEvkfqT0I8nA9W4BHY77hdPGdAT+DnwoF+u0e0Cff3Jnxw070rgHT/8NWBt0LwUf5If08jxrQf2AJW4L+FyYMRR3vf7gd/64SRgJzDIj98HPNSc9xnYANwM9A5ZZgIhXxQh85cGPtdwjhUYCGwFzgbimzi3vgu8FDSuwKlB44uBHwWN/z/g/qD4a4AeQfNfAH7mh5/g6InjmqDx/wX+5IcfA/4n5Hia/eUF/AP4TtB+9wNxQfO34pJnrD8PhgTN+2/CTxw3BI0f7XzOb+D9eobmJ45VwKSgef18/IEfAQrkBs1fAFzhh+/m6Injz8D04PUbO+b2fllVVcsdC+T4om+liFTivoj7+vl5uF/0Da13csh6VwPHHGVfpUHDn+N+pWX5bV0Wsq3TcCfvF6jqfmAR7hfgGcBc4EPgVD9tblCMjW33WL//TUHzHsaVPAI2B+1znx/seZTju1hV03CJ4BZgrogcAyAiJ4vIO+KqA6uAb/hjR13V1vPANSISg0tgTwUdw9He53/D/QL93FcHjG8oMBG5TkSWBm3jhMD+wzlWVV2LSwh3A1tF5DkRyfH7GCwir4rIZhHZhfuizArZxJag4f0NjAfvc6eq7g0a/xxXimiOzUHD+4K2m8OR52Hw8BeIyHki8pG4asJK3HsdfEzbVbWmgX1l4750Q8/5cAWvf7TzOYeG36/mOhZ4KWi7q4BaDn8PQOPvaVN+iCuxLRCRFSJyQxhxRZQljpYrBT5T1bSgVy9VPT9ofkN1s6XA3JD1eqrqN4+yr+CrjPJxv2i2+W09FbKtHqr6S7+sNrCtubhqqVHAQj9+LjAWCNSdHm27pbgSR1bQvN6qOvyo71YzqGqtqv4d9493mp/8DDADyFPVVOBPuH+mgCdxCWESsE9V5wUdQ6Pvs6ouVNWpuIT3D9yv8iOIyLG4aoZbgEyf3JaH7D+c43tGVU/Dfdko8Cs/649ACa7k1Bv3A6RF+/DSRaRH0Hg+riTXGptw1TEBjV75JiKJuCrV+4C+/n2bSfOOqQJXAgg958MVfO4f7XzeRMPvV8BeXEkScG2XuOQWvO3zQradpKobw4zxizNVN6vqv6tqDq50/FBjbUrtzRJHyy0AdvtGuGRxjeEniMhJfv4jwH+JyCBxRopIJvAqMFhErhXXQB0vIieJyNCj7OsaERkmIim4euEXVbUWV888RUTO9ftP8o15gX/wLbh68GBzgeuAlap6EF80xiXBCr9Mo9tV1U3AG8D/E5HevnHwOBE5sxXvJQD+fZqKa3NY5Sf3AnaoarWIjAWuCl7HJ4o6XHXNU0GzGn2fRSRBRK4WkVRVPQTs8tsI1QP3z13h47seV+JoybEdLyJn+S/ValwpIbDPXj6GPSIyBNdW0lo/98d5OnAh8NdWbu8F4Hr//qUAPzvKsgm4+v4KoEbcBQPnNGcn/rz+O3C3iKSIyDBclVJrHO18/hxXCg+8X6cBU4LW/ReQJO4ijXjgp/7YAv4E3Ot/ZCAi2f4cbo4tQIEvLX+BiFwW9L+8E3cu1gWtG/q/3W4scbSQP8EvBIqAz3AlgEdwjZXgGttewH3J7gIexbUL7Mb9E12B+xW4GffLM/hkDPUUrm56M64659s+hlJgKu4XagXu188POPy5PgB8RdwVPw/6aR/i2joCpYuVuC+ywHhztnsd7sthJe6EfpFGqsea6RVxV8fsAu7FtdEELq/8FnCPiOzGXYjwhZIBri54BO4LInAMTb3P1wLrfdXQN3ClliOo6kpcQpqH+0cdAXzQwmNMBH6JO08240o6d/h5/4lLiLtxJZznW7iPgM24z6Uc+AuugbukNRtU1deBB4F3gLW4BmFwpc/QZXfjztEXfBxX4UqNzXULrjpnM+68f7ylcft4mjqfr8K1We4A7sKdT4F1q3Dn4CO4Rvq9uHbBgAdwx/aGP0c/8ttqjkAy3y4iSxqYfxIw3/9vzMC1Ea3z8+4GnvRVZE1dldnmxDe0GNNpich1wE2+GqhbE3eZ6dOqmtvEoq3dz1BctV1iSFtFpyfuku+BqnpNtGPpqKzEYTo1X23yLdzVJyaCxHXtkSgi6bjS2ytdLWmY5rHEYTotETkXV/WwBdeIbiLrZtxls5/iLmBoi7YY0wlZVZUxxpiwWInDGGNMWLpFB2BZWVlaUFAQ7TCMMaZTWbx48TZVzQ6d3i0SR0FBAYsWLYp2GMYY06mISIN30VtVlTHGmLBY4jDGGBMWSxzGGGPCEtE2DnEPK3kA11XyI0Gd7wXmJ+Ju7z8R1z/+NFVd7/uEeQQY7WP8s6r+j19nPa5rhlqgRlXHRPIYjDEd26FDhygrK6O6ujraoXRaSUlJ5ObmEh8f36zlI5Y4fC+Sf8A9NKUMWCgiM3z/PwFfx3VpPFBErsDdjToN9/CcRFUd4e8MXikiz6rqer/eRFXdFqnYjTGdR1lZGb169aKgoACR1nQq3D2pKtu3b6esrIwBAwY0a51IVlWNxT3kZp3vhfU5XEdjwabiusUG11HeJHGfvAI9xD0vOBn34KJdEYzVGNNJVVdXk5mZaUmjhUSEzMzMsEpskUwc/TnyYSplflqDy/g+b6pwT6d7EdcL5Sbck9ruU9Udfh3F9US5WERuamznInKTiCwSkUUVFRWNLWaM6QIsabROuO9fR20cH4trw8gBBgC3iUig7/nTVHU0cB7wHyJyRkMbUNXpqjpGVcdkZ3/h/hVjjDEtFMnEsZEjn+KV66c1uIyvlkrFNZJfBcxS1UOquhX3DIQxAIEna/npL+GSTLuY9vA8pj08r+kFjTHdzr333svw4cMZOXIkRUVFzJ8/n/vvv599+/Y1vXIzFRQUsG1by5t3n3jiCW655ZZWxxHJxLEQGCQiA0QkAfdAndCHuczg8NO9vgK8ra7XxQ24x5si7pGO44ASEekhIr2Cpp+DeyaAMcZEzbx583j11VdZsmQJy5Yt48033yQvL6/NE0e4amtrI7LdiCUO32ZxCzAb9xjQF1R1hYjcIyIX+cUeBTJFZC3wfeB2P/0PQE8RWYFLQI+r6jLcA+DfF5Fi3KNbX1PVWZE6BmOMaY5NmzaRlZVFYqJ7wGRWVhYvvvgi5eXlTJw4kYkTJwLwzW9+kzFjxjB8+HDuuuuu+vULCgq46667GD16NCNGjKCkxD2wcfv27ZxzzjkMHz6cG2+8keDezC+++GJOPPFEhg8fzvTphx9H07NnT2677TYKCwuZN28ejz/+OIMHD2bs2LF88EFLH2B5pG7RrfqYMWO0LfqqClRTPX/z+FZvyxjTNlatWsXQoUMPT5gwoW13MGdOk4vs2bOH0047jX379nH22Wczbdo0zjzzzPp+8rKysgDYsWMHGRkZ1NbWMmnSJB588EFGjhxJQUEBt912G7feeisPPfQQS5Ys4ZFHHuHb3/42WVlZ3Hnnnbz22mtceOGFVFRUkJWVVb+t/fv3c9JJJzF37tz6q8uef/55Lr/8cjZt2sTJJ5/M4sWLSU1NZeLEiYwaNYrf//73XziGL7yPgIgsbuheuY7aOG6MMZ1Gz549Wbx4MdOnTyc7O5tp06bxxBNPfGG5F154gdGjRzNq1ChWrFjBypWHb2u79NJLATjxxBNZv349AO+++y7XXOOeYHvBBReQnp5ev/yDDz5IYWEh48aNo7S0lDVr1gAQGxvLv/3bvwEwf/58JkyYQHZ2NgkJCUybNq1Njrdb9I5rjOlGmlFCiITY2FgmTJjAhAkTGDFiBE8++eQR8z/77DPuu+8+Fi5cSHp6Ol/72teOuHciUM0VGxtLTc3Rn8g7Z84c3nzzTebNm0dKSgoTJkyo31ZSUhKxsbFtfHRHshKHMca00urVq+t/8QMsXbqUY489ll69erF7924Adu3aRY8ePUhNTWXLli28/vrrTW73jDPO4Jln3FORX3/9dXbu3AlAVVUV6enppKSkUFJSwkcffdTg+ieffDJz585l+/btHDp0iL/+9a+tPVTAShzGGNNqe/bs4dZbb6WyspK4uDgGDhzI9OnTefbZZ5k8eTI5OTm88847jBo1iiFDhpCXl8epp57a5HbvuusurrzySoYPH84pp5xCfn4+AJMnT+ZPf/oTQ4cO5fjjj2fcuHENrt+vXz/uvvtuxo8fT1paGkVFRW1yvNY4HgZrHDem42moUdeEzxrHjTHGRIwlDmOMMWGxxGGMMSYsljiMMcaExRKHMcaYsFjiMMZ0O9bTdetY4jDGmFaKjY2lqKiIwsJCRo8ezYcfftgu+73xxhuP6LakvdgNgMYY00rJycksXboUgNmzZ3PHHXcwd+7ciO/3kUceifg+GmIlDmOMaUO7du2q74xwz549TJo0qb679JdffhmAO++8k/vvv79+nZ/85Cc88MADAPz617/mpJNOYuTIkfVdr+/du5cLLriAwsJCTjjhBJ5//nkAJkyYQODm5nC7bG8NK3EYY7qMn7+ygpXlu5pcbuUmt0xz2jmG5fTmrinDj7rM/v37KSoqorq6mk2bNvH2228DrsPBl156id69e7Nt2zbGjRvHRRddxA033MCll17Kd7/7Xerq6njuuedYsGABb7zxBmvWrGHBggWoKhdddBHvvvsuFRUV5OTk8NprrwGur6pQ99577xFdti9btoyRI0cC7vkgS5Ys4aGHHuK+++5rdUnFShzGGNNKgaqqkpISZs2axXXXXYeqoqr8+Mc/ZuTIkZx99tls3LiRLVu2UFBQQGZmJh9//DFvvPEGo0aNIjMzkzfeeKN+fPTo0ZSUlLBmzRpGjBjBP//5T370ox/x3nvvkZqa+oUYwu2yvTWsxGGM6TKaKhkERLLfufHjx7Nt2zYqKiqYOXMmFRUVLF68mPj4eAoKCuq7P7/xxht54okn2Lx5MzfccAMAqsodd9zBzTff/IXtLlmyhJkzZ/LTn/6USZMmceedd9bPa8su25vDShzGGNOGSkpKqK2tJTMzk6qqKvr06UN8fDzvvPMOn3/+ef1yl1xyCbNmzWLhwoWce+65AJx77rk89thj7NmzB4CNGzeydetWysvLSUlJ4ZprruEHP/gBS5YsOWKfLemyvTWsxGGMMa0UaOMAV2p48skniY2N5eqrr2bKlCmMGDGCMWPGMGTIkPp1EhISmDhxImlpafUPXjrnnHNYtWoV48e7klDPnj15+umnWbt2LT/4wQ+IiYkhPj6eP/7xj0fsv7CwMOwu21vDulUPg3WrbkzH05Ju1TvC/3JdXR2jR4/mr3/9K4MGDYpaHAHhdKtuJQ5jTLcT7R9/K1eu5MILL+SSSy7pEEkjXJY4jDGmnQ0bNox169ZFO4wWs8ZxY0yn1x2q3CMp3PfPEocxplNLSkpi+/btljxaSFXZvn07SUlJzV7HqqqMMZ1abm4uZWVlVFRURDuUTispKYnc3NxmLx/RxCEik4EHgFjgEVX9Zcj8RODPwInAdmCaqq4XkXjgEWC0j/HPqvo/zdlmpNmvGmM6lvj4eAYMGBDtMLqViCUOEYkF/gB8GSgDForIDFUN7gP468BOVR0oIlcAvwKmAZcBiao6QkRSgJUi8ixQ2oxttq0JEwDYHZvA6hE3cPnqd+G5H0dsd8YY02bmzInIZiPZxjEWWKuq61T1IPAcMDVkmanAk374RWCSiAigQA8RiQOSgYPArmZuMyJ61h7kQGw8m3tktMfujDGmw4pkVVV/XAkhoAw4ubFlVLVGRKqATFwSmQpsAlKA76nqDhFpzjYBEJGbgJsA8vPzW34UPmMLIHfO4t2RZ8Cdv2j59owxppPrqFdVjQVqgRxgAHCbiHwpnA2o6nRVHaOqY7Kzs9skqMS4GA4cqmuTbRljTGcVycSxEcgLGs/10xpcxldLpeIaya8CZqnqIVXdCnwAjGnmNiMmMS6GAzW11kBujOnWIpk4FgKDRGSAiCQAVwAzQpaZAXzVD38FeFvdt/IG4CwAEekBjANKmrnNiEmMj6VOYdueg+21S2OM6XAiljhUtQa4BZgNrAJeUNUVInKPiFzkF3sUyBSRtcD3gdv99D8APUVkBS5ZPK6qyxrbZqSOIVRinHu7Snfua69dGmNMhxPR+zhUdSYwM2TanUHD1bhLb0PX29PQ9Ma22V6S4lzXx6U79jE6Pz0aIRhjTNR11MbxDqm+xLHDShzGmO7LEkcYYmKE+FhhgyUOY0w3ZokjTIlxsZY4jDHdmiWOMCXGxVC6Y3+0wzDGmKixxBGmpPgYNlXt51Ct3QhojOmeLHGEKTHO3ctRXmmlDmNM92SJI0yBK6usncMY011Z4ghTYnzgklwrcRhjuidLHGFKiI0hITbGShzGmG7LEkeYRIT+6cnW7YgxptuyxNECeRkpdve4MabbssTRAnnpyZY4jDHdliWOFsjPSGHnvkPsrj4U7VCMMabdWeJogbyMFMCurDLGdE+WOFog3ycOu7LKGNMdWeJogbz0QInDEocxpvuxxNECqSnx9E6Ks0tyjTHdkiWOFsrLSLGqKmNMt2SJo4Xy7V4OY0w3ZYmjhfIyUijduZ+6Oo12KMYY064scbRQXkYKB2vqqNhzINqhGGNMu7LE0UJ2Sa4xpruyxNFCeenJgF2Sa4zpfuKiHUBn8vzN4+uH+6cnI2IlDmNM92MljhZKjIvlmN5J1u2IMabbscTRCta9ujGmO4po4hCRySKyWkTWisjtDcxPFJHn/fz5IlLgp18tIkuDXnUiUuTnzfHbDMzrE8ljOJq8dLsJ0BjT/UQscYhILPAH4DxgGHCliAwLWezrwE5VHQj8FvgVgKr+RVWLVLUIuBb4TFWXBq13dWC+qm6N1DE0JT8jhS27q6k+VButEIwxpt1FssQxFlirqutU9SDwHDA1ZJmpwJN++EVgkohIyDJX+nU7nLyMZFRhY6W1cxhjuo9IJo7+QGnQeJmf1uAyqloDVAGZIctMA54Nmfa4r6b6WQOJBgARuUlEFonIooqKipYew1HlZ1gvucaY7qdDN46LyMnAPlVdHjT5alUdAZzuX9c2tK6qTlfVMao6Jjs7OyLx5VniMMZ0Q5FMHBuBvKDxXD+twWVEJA5IBbYHzb+CkNKGqm70f3cDz+CqxKIiu2ciiXExlO60qipjTPcRycSxEBgkIgNEJAGXBGaELDMD+Kof/grwtqoqgIjEAJcT1L4hInEikuWH44ELgeVESUyMkJuezIbtVuIwxnQfEbtzXFVrROQWYDYQCzymqitE5B5gkarOAB4FnhKRtcAOXHIJOAMoVdV1QdMSgdk+acQCbwL/F6ljaI78jBR7oJMxpluJaJcjqjoTmBky7c6g4WrgskbWnQOMC5m2FzixzQNthbyMFBZ9vjPaYRhjTLvp0I3jnUF+Rgq7q2uo2nco2qEYY0y7sMTRSrnp1r26MaZ7scTRSvX3clg7hzGmm7DE0Up5Ge65HFbiMMZ0F5Y4WqlXUjzpKfGWOIwx3YYljjaQb92rG2O6EUscbSDXEocxphuxxNEG8jNS2Fi5n9o6jXYoxhgTcZY42kBeegqHapXNu6qjHYoxxkScJY42YN2rG2O6E0scbcAuyTXGdCeWONpATloyMQJlljiMMd2AJY42EB8bQ7/UZCtxGGO6BUscbcR1r24PdDLGdH2WONpIXoaVOIwx3YMljjaSn5FCxe4D7D9YG+1QjDEmoixxtJE8f0lumfWSa4zp4ixxtJFA4rDqKmNMV2eJo43kpdtNgMaY7sESRxvJ6plAcnwsG3bYlVXGmK6t2YlDRJJF5PhIBtOZiYi/JNdKHMaYrq1ZiUNEpgBLgVl+vEhEZkQwrk4pLyPZqqqMMV1ec0scdwNjgUoAVV0KDIhIRJ1Ynn8uh6p1r26M6bqamzgOqWpVyDT7dgyRl57C3oO17Nh7MNqhGGNMxDQ3cawQkauAWBEZJCK/Az6MYFydUn336tb1iDGmC2tu4rgVGA4cAJ4BqoDvRiimTsvu5TDGdAfNShyquk9Vf6KqJ/nXT1W1ycfdichkEVktImtF5PYG5ieKyPN+/nwRKfDTrxaRpUGvOhEp8vNOFJFP/DoPioiEd8iRE3guhzWQG2O6suZeVfVPEUkLGk8XkdlNrBML/AE4DxgGXCkiw0IW+zqwU1UHAr8FfgWgqn9R1SJVLQKuBT7zDfIAfwT+HRjkX5ObcwztISUhjqyeCZY4jDFdWnOrqrJUtTIwoqo7gT5NrDMWWKuq61T1IPAcMDVkmanAk374RWBSAyWIK/26iEg/oLeqfqTu0qU/Axc38xjaRZ7dy2GM6eKamzjqRCQ/MCIix9L0VVX9gdKg8TI/rcFlVLUG13aSGbLMNODZoOXLmthmIMabRGSRiCyqqKhoItS2k5eeYm0cxpgurbmJ4yfA+yLylIg8DbwL3BG5sBwRORnYp6rLw11XVaer6hhVHZOdnR2B6BqWn5FCeWU1NbV17bZPY4xpT81NHLOBnwJDcNVGpwM7m1hnI5AXNJ7rpzW4jIjEAanA9qD5V3C4tBFYPreJbUZVXkYytXXKpqomrx0wxphOqbmJ4yHgZKCnqr4K7MY1fB/NQmCQiAwQkQRcEgjtpmQG8FU//BXgbd92gYjEAJfj2zcAVHUTsEtExvm2kOuAl5t5DO3CLsk1xnR1cc1c7mRVHS0iH4NrHPfJoFGqWiMit+BKK7HAY6q6QkTuARap6gzgUeApEVkL7MAll4AzgFJVXRey6W8BTwDJwOv+1WFY9+rGmK6uuYnjkL+8NlAayAaarMRX1ZnAzJBpdwYNVwOXNbLuHGBcA9MXASc0M+521y81ibgYsRKHMabLam5V1YPAS0AfEbkXeB/474hF1YnFxcaQk5Zs3Y4YY7qsZpU4VPUvIrIYmAQIcLGqropoZJ1YfoZdkmuM6bqaW1WFqpYAJRGMpcvIy0jhjRWbox2GMcZEhD06NgLyMpLZvvcgew/URDsUY4xpc5Y4IuBw9+pWXWWM6XoscURA4JLcDdstcRhjuh5LHBGQbzcBGmO6MEscEZCWEk/PxDjK7JJcY0wXZIkjAkSEPLsk1xjTRVniiJC89GTrdsQY0yVZ4oiQfP9AJ99nozHGdBmWOCIkLyOF6kN1VOw5EO1QjDGmTVniiJD6ezmsusoY08VY4oiQvIxkAEp32JVVxpiuxRJHhOSm270cxpiuyRJHhCTFx9KnV6JVVRljuhxLHBFk3asbY7oiSxwRlJeRYnePG2O6HEscEZSXkUJ51X4O1jT5lF1jjOk0LHFEUH5GCqqwsXI/0x6ex7SH50U7JGOMaTVLHBGUlx64JNfaOYwxXYcljgjKz7RLco0xXY8ljgjq2yuJhNgYexKgMaZLscQRQTExQq71kmuM6WIscURYbkaKdTtijOlSIpo4RGSyiKwWkbUicnsD8xNF5Hk/f76IFATNGyki80RkhYh8IiJJfvocv82l/tUnksfQWvkZydbGYYzpUuIitWERiQX+AHwZKAMWisgMVV0ZtNjXgZ2qOlBErgB+BUwTkTjgaeBaVS0WkUzgUNB6V6vqokjF3pby0lOo2n+Imto64mKtgGeM6fwi+U02FlirqutU9SDwHDA1ZJmpwJN++EVgkogIcA6wTFWLAVR1u6rWRjDWiAl0r37AbgI0xnQRkUwc/YHSoPEyP63BZVS1BqgCMoHBgIrIbBFZIiI/DFnvcV9N9TOfaL5ARG4SkUUisqiioqItjqdF8ixxGGO6mI5adxIHnAZc7f9eIiKT/LyrVXUEcLp/XdvQBlR1uqqOUdUx2dnZ7RFzgw4njk5ZYDLGmC+IZOLYCOQFjef6aQ0u49s1UoHtuNLJu6q6TVX3ATOB0QCqutH/3Q08g6sS67BSk+PpnRRH9SErcRhjuoZIJo6FwCARGSAiCcAVwIyQZWYAX/XDXwHeVlUFZgMjRCTFJ5QzgZUiEiciWQAiEg9cCCyP4DG0ifzMFKuqMsZ0GRG7qkpVa0TkFlwSiAUeU9UVInIPsEhVZwCPAk+JyFpgBy65oKo7ReQ3uOSjwExVfU1EegCzfdKIBd4E/i9Sx9BW8tJTWLt1T7TDMMaYNhGxxAGgqjNx1UzB0+4MGq4GLmtk3adxl+QGT9sLnNj2kUZWfkYKBw7V4QpTxhjTuXXUxvEuJTcjBQUO1lriMMZ0fpY42kG+XVlljOlCLHG0g/rEYVdWGWO6AEsc7SAnLQmwEocxpmuwxNEOEuNiSYiNoXL/IdZu3R3tcIwxplUscbSTY1KT2HewlrN/8y7XPjqft1Ztoa7OGsuNMZ1PRC/HNYf1S00iq2cCk4b25al5n/P1JxdxbGYK140v4LIxufROio92iMYY0yxW4mhH8bEx/MfEgbz3o4n8/qpRZPVM5L9eXcn4/36Lu15ezqcVzbtJcNrD85j28LwIRxu+jhqXMaZtWYkjCuJjY7hwZA4XjsxhWVklT3y4nmcXlPLkvM85c3A2Xzu1gDMHZRMT02DHv8YYE1VW4oiykblp/ObyIj64/Sy+d/ZgVm7axfWPL2TSb+byxAefsedATbRDNMaYI1ji6CCyeyXynbMH8cGPzuKBK4pITY7n7ldWMu6/3+Lnr6xg/ba90Q7RGGMAq6rqcBLiYpha1J+pRf35eMNOnvxwPU9/9DlPfLieicf34fpTC1BVGnl+lTHGRJwljg5sVH46o/LT+fH5Q3l6/gaemf851z66gKT4GHonxfPTf3xCnYKqUlun1CnUqaL+7+Fxpa7u8DRVDZnv/tbWBa8bvGzT69apsmXXARJiY/jDO2s5a0gfhhzTyxKcMV2QdIceW8eMGaOLFi2KagyBq42ev3l8i7dxoKaW15Zt4mcvL+fAoTrSUuIREWIEYkSIEUHqhwkZDxqOOTytwXVjAusGz6fRfcXGuOF3/1XB/oO17D3o7pDPSU1i4pA+nDWkD6ccl0VyQmybvJfGmPYhIotVdUzodCtxdCKJcbFcOjqX5xe6R7m3JglFQiA5/u7KUbyzeitvl2zlHx9v5C/zN5AYF8P44zKZNKQPE4f0ITc9JcrRGmNayhKHaXN9eicx7aR8pp2Uz4GaWhZ8toO3S1wi+dnLK+DlFQzu25OzhvTlrCF9GJ2fRlxs97tOoy1KocZEgyUOE1GJcbGcPiib0wdlc+eFw1i3bS/v+CTyyHvr+NPcT0lNjufMwdmcNaQPZw7OJr1HQrTDNh2QJdqOwxKHaTciwnHZPTkuuyc3nv4ldlUf4v0123i7ZCtzVm9lRnE5MQKj89Pr20asgd2YjscSh4ma3knxnD+iH+eP6EddnbJsY5Wv0trCr2ev5tezV1sDu6Fi9wGKSysp3bGPOoWF63dwYn669awQRZY4OqGOWlRvTVwxMUJRXhpFeWl8/8uD2bKrmjmrt/LWqq28FNTAfspxmZxlDexd1r6DNXxSVkVxWSXFpVUsLa1kY+X++vkicNmf5pGTmsSUwhymFOYwPKe3lUrbmSUO0yH1baCB/a1VW3ln9Vbe8Q3sx/ftxcQhfZg0tA+j8rpnA3tnVlNbx5qteygurWSpf/1ry24CTxvITU+mKD+N608toDAvjV+9XgLANeOOZUZxOY++/xkPv7uOL2X1qE8iA/v0jOIRdR+WOEyHF9zAfpe6Bva3VzXcwD5pqGtgT0uxBvaORFUpr6pm6YZKistckli+sYp9/p6f1OR4CvPSOGdYXwrz0ijMSyOrZ+IR24j1VVMXj+rPxaP6U7nvIK8v38wrxeU8+PYaHnhrDcP69fZJpJ+VSCPIbgBsJ3ZFSGTsqj7Ee/863MC+fe/B+gb2s4a6tpHj+3bMBvaufE5U7T/EsrJKX5pwVU7b9hwAICE2hmE5veurJgvz0ijITGnVZ7R1VzWvLtvEK8vK+XhDJQCj89O4qDCHC0bmkN0r8egbMA1q7AZASxztpCt/SXQUdXVKcVmlu9x39VaWb9wFQP+0ZCYOya5vYE+K7xgN7F3lnDhQU0vJpt2uJLGhkqVllayrONwp55eyexxOErlpDO3Xm4S4yFUrlu7Yx4zicl4pLqdk825iBMYfl8lFhTlMHt6P1BR7aFpzWeKwxNHtbNlVXX/PyPtrt7HvYC2JcTGcOjCr/kqt/mnJUYuvM54Tqsr67ftYWrqzvvF6ZfkuDtbWAZDVM5GivDRG5bskMSI3ldTk6H1Rr9mym1eKy5lRXM767fuIjxXOHJzNlMIczh7alx6JVlt/NJY4opw4THQdqKll/rrDd7Bv2LEPgCHH9KpPIu3dwN4ZEse2Pe5S2OLSSj4urWRZWRVV+w8BkJIQy4j+qfXVTUV5afRLTeqQ1YKqyvKNu5hRvJFXijexeVc1yfGxTBrahymFOUw4PpvEuI5REu1IopI4RGQy8AAQCzyiqr8MmZ8I/Bk4EdgOTFPV9X7eSOBhoDdQB5ykqtUiciLwBJAMzAS+o00chCUOE0xV+bTC3cH+VskWFq3fSU2dkpZy5B3skW5g72iJY//BWpaXV9VXNxWXVlK2010KGyNw/DG9Kco7nCgGZvfslFey1dUpiz7fyYzijcz8ZDM79h6kV1Ick4cfw5TCHE45LrNTHlcktHviEJFY4F/Al4EyYCFwpaquDFrmW8BIVf2GiFwBXKKq00QkDlgCXKuqxSKSCVSqaq2ILAC+DczHJY4HVfX1o8ViicMcTWMN7Cce6+5gnzSkL4P79mzzX9LRTBy1dcqarbuPaLz+15bd1PprYfunuUthi3Jdkjihf29SErpetU5NbR0ffLqdGUvLeWPFZnYfqCGrZwLnj+jHlMKcbn+jYTQSx3jgblU914/fAaCq/xO0zGy/zDyfLDYD2cB5wFWqek3INvsB76jqED9+JTBBVW8+WiyWOExzBTewv1WylRXlRzawTxrSl/HHZbZJA3t7JQ5VZVNV9RH3S3wSdCls76S4+qqmorw0RuamdcurkKoP1TJndQWvFJfz5qotHKipIyc1iQsLc7iom95oGI1u1fsDpUHjZcDJjS2jqjUiUgVkAoMB9YklG3hOVf/XL18Wss3+De1cRG4CbgLIz89v9cGY7iEmRuofoPX9c46vb2B/q2Qrf1+ykac/2kBSfAynHJfFWb5tJCeKDewN2VV9iGWlVfX3SywtraRi9+FLYYfm9ObyMXkU5qVSmJvGgKwe3e4LsSFJ8bFMPuEYJp9wDHsO1PDmyi28UlzOY+9/xnR/o2EgiXT3Gw07atkzDjgNOAnYB7wlIouBquZuQFWnA9PBlTgiEaTp+vr2TuKKsflcMTb/Cw3sb5dsBVwDeyCJjMpPr79RrT0crKmjZPOu+sbr4tJKPg25FPb0gVn1JYoh/XpZI3Az9EyMO+JGw1nLNzOjuJzfvb2GB99aw9B+vbmoMIcLR/YjL6P73WgYycSxEcgLGs/10xpapsxXVaXiGsnLgHdVdRuAiMwERgNP++0cbZvGRERiXCxnDM7mjMHZ3DVlGJ9W7OXtki28XbKV6e+u46E5n5KWEs+EwdlMjEADu6ry+fZ9FJdV8rG/A3tF+S4O1gQuhU2gKC+NS0b1pzAvjZH90+yehTaQlpJQ/+Nh665qXvtkEzOKy/nVrBJ+Nauk/kbD80f2o0+vpGiH2y4i2cYRh2scn4T7cl+Ia7dYEbTMfwAjghrHL1XVy0UkHXgLV+o4CMwCfquqrzXQOP47VZ15tFisjcNEWtX+Q7y3psI3sFewwzewjzk2o/5y39AG9qbaOLbvOeCrm6rcJbFllVTuc5fCJsfHMiI3tf6muqL8NHI66KWwXVXpjn28sqycGUuPvNFwysgczjuha9xoGK3Lcc8H7sddjvuYqt4rIvcAi1R1hogkAU8Bo4AdwBWqus6vew1wB6DATFX9oZ8+hsOX474O3GqX45qOpDb4DvaQBvZAldb44zL56mMLAJc49h+sZUV5VX2bRHFZJaU7Dl8KO7hvryO66BjUp3NeCttVddUbDe0GQEscJko2V1XXP4P9/TXb2H+olqT4GJLiYkmIiyG7VyIlm0Muhc1Lq2+8HpGb2iUvhe2Kgm80fHXZJjZVVZMUH8OkoX25qBPeaGiJwxKH6QCqD9Uy/7MdvFOylWcXbKCmTjnluExX3ZSXxsi81G5TT97VNXaj4bnDj+GiTnKjoSUOSxymg+lod46byAncaPhKcTmzl7sbDTN7uBsNLyrquDcaRuM+DmOMMUBcbAxnDs7mzMHZ/OLiE9yNhsvK+eviUp766PP6Gw2njMzhhP4d/0ZDSxzGGNOOQm80fGvVFmYsLefxD9yNhgP8Ew0vKuzHwD69oh1ugyxxGGNMlPRMjGNqUX+mFjV+o+GUwn5MGZnToW40tMRhjDEdQGM3Gv7vrNX876zVjM5PY0phDhc080bDSLahWeIwxpgOpk/vJK4/dQDXnzqg/kbDV4o38fNXVvJfr65k3Jf8Ew1POCbi3f83xBKHMcZ0YHkZKXxrwkC+NWHgETca3v73T/jZy8s5Y1A2FxW1742GljiMMaaTGNS3F98/53i+9+XBLN+4y5dEynmrZGv9jYZTRrobDSPJEocxxnQyIsKI3FRG5KZy++QhLPp8J68UlzPzk028tmwTvRLjSIyPIbNHAjW1dW1+o6ElDmOM6cRiYoSxAzIYOyCDu6YM48NPtzOjuJyXlmxk+56D7KquIaNH27aDWOIwxpguIi42pr7r/8+37WXvwbZPGgAdu6MUY4wxLRITI/RKikzX7pY4jDHGhMUShzHGmLBYG4cxxnRBkex12UocxhhjwmKJwxhjTFgscRhjjAmLJQ5jjDFhscRhjDEmLHZVlTFRYs8aN52VlTiMMcaExRKHMcaYsFjiMMYYE5aIJg4RmSwiq0VkrYjc3sD8RBF53s+fLyIFfnqBiOwXkaX+9aegdeb4bQbm9YnkMRhjjDlSxBrHRSQW+APwZaAMWCgiM1R1ZdBiXwd2qupAEbkC+BUwzc/7VFWLGtn81aq6KEKhG2OMOYpIljjGAmtVdZ2qHgSeA6aGLDMVeNIPvwhMEhGJYEzGGGNaKZKJoz9QGjRe5qc1uIyq1gBVQKafN0BEPhaRuSJyesh6j/tqqp9ZojHGmPbVURvHNwH5qjoK+D7wjIj09vOuVtURwOn+dW1DGxCRm0RkkYgsqqioaJegjTGmO4hk4tgI5AWN5/ppDS4jInFAKrBdVQ+o6nYAVV0MfAoM9uMb/d/dwDO4KrEvUNXpqjpGVcdkZ2e32UEZY0x3F8k7xxcCg0RkAC5BXAFcFbLMDOCrwDzgK8Dbqqoikg3sUNVaEfkSMAhY55NLmqpuE5F44ELgzaYCWbx48TYR+byFx5EFbGvhupFkcYXH4gqPxRWerhrXsQ1NjFjiUNUaEbkFmA3EAo+p6goRuQdYpKozgEeBp0RkLbADl1wAzgDuEZFDQB3wDVXdISI9gNk+acTiksb/NSOWFhc5RGSRqo5p6fqRYnGFx+IKj8UVnu4WV0T7qlLVmcDMkGl3Bg1XA5c1sN7fgL81MH0vcGLbR2qMMaa5OmrjuDHGmA7KEkfTpkc7gEZYXOGxuMJjcYWnW8UlqhqJ7RpjjOmirMRhjDEmLJY4jDHGhMUSRyOa6tm3nWN5TES2isjyoGkZIvJPEVnj/6a3c0x5IvKOiKwUkRUi8p2OEJePIUlEFohIsY/t5376AN8L81rfK3NCFGKL9V3pvNpRYvJxrBeRT3xXPov8tI7wWaaJyIsiUiIiq0RkfLTjEpHjg3rnXioiu0Tku9GOy8f2PX/OLxeRZ/3/QpufY5Y4GhDUs+95wDDgShEZFsWQngAmh0y7HXhLVQcBb/nx9lQD3Kaqw4BxwH/49yjacQEcAM5S1UKgCJgsIuNwvS//VlUHAjtxvTO3t+8Aq4LGO0JMARNVtSjouv+O8Fk+AMxS1SFAIe69i2pcqrrav09FuNsD9gEvRTsuEekPfBsYo6on4O51C/Q63rbnmKraK+QFjAdmB43fAdwR5ZgKgOVB46uBfn64H7A6yvG9jOtCv6PFlQIsAU7G3UEb19Bn3E6x5OK+UM4CXgUk2jEFxbYeyAqZFtXPEtcF0Wf4i3g6SlwhsZwDfNAR4uJwp7EZuHv0XgXOjcQ5ZiWOhjWnZ99o66uqm/zwZqBvtAIR9wCuUcD8jhKXrxJaCmwF/onr76xSXS/MEJ3P9H7gh7jeEMD1BB3tmAIUeENEFovITX5atD/LAUAFrjfsj0XkEd97RLTjCnYF8Kwfjmpc6vrxuw/YgOsotgpYTATOMUscXYC6nxJRua5aRHri7vL/rqru6ihxqWqtuqqEXFxHmEOiEUeAiFwIbFXXaWdHdJqqjsZVz/6HiJwRPDNKn2UcMBr4o7qesvcSUv0T5XM/AbgI+GvovGjE5dtUpuISbg7Qgy9WcbcJSxwNa07PvtG2RUT6Afi/W9s7AN9n2N+Av6jq3ztKXMFUtRJ4B1dET/MdZUL7f6anAheJyHrcQ83OwtXfRzOmenq41+mtuPr6sUT/sywDylR1vh9/EZdIoh1XwHnAElXd4sejHdfZwGeqWqGqh4C/4867Nj/HLHE0rL5nX/+r4gpcT74dSaBnYfzfl9tz5yIiuE4qV6nqbzpKXD62bBFJ88PJuLaXVbgE8pVoxKaqd6hqrqoW4M6nt1X16mjGFCAiPUSkV2AYV2+/nCh/lqq6GSgVkeP9pEnAymjHFeRKDldTQfTj2gCME5EU//8ZeL/a/hyLVqNSR38B5wP/wtWN/yTKsTyLq7M8hPsV9nVc/fhbwBpcL8EZ7RzTabii+DJgqX+dH+24fGwjgY99bMuBO/30LwELgLW46oXEKH2eE4BXO0pMPoZi/1oRON87yGdZBCzyn+U/gPQOElcPYDuQGjStI8T1c6DEn/dPAYmROMesyxFjjDFhsaoqY4wxYbHEYYwxJiyWOIwxxoTFEocxxpiwWOIwxhgTFkscxjRBRPb4vwUiclUbb/vHIeMftuX2jYkESxzGNF8BEFbiCLpjtzFHJA5VPSXMmIxpd5Y4jGm+XwKn+2cwfM93pPhrEVkoIstE5GYAEZkgIu+JyAzcnbuIyD98B4IrAp0IisgvgWS/vb/4aYHSjfhtL/fPyZgWtO05Qc+o+Iu/SxgR+aW456MsE5H72v3dMd1GU7+GjDGH3Q78p6peCOATQJWqniQiicAHIvKGX3Y0cIKqfubHb1DVHb4LlIUi8jdVvV1EblHXGWOoS3F3TRcCWX6dd/28UcBwoBz4ADhVRFYBlwBDVFUDXa4YEwlW4jCm5c4BrvPdt8/HdTkxyM9bEJQ0AL4tIsXAR7gONAdxdKcBz6rr5XcLMBc4KWjbZapah+vqpQDXhXY18KiIXIp7uJAxEWGJw5iWE+BW9U+DU9UBqhooceytX0hkAq7n0vHqnkr4MZDUiv0eCBquxT2kpwbXo+2LwIXArFZs35ijssRhTPPtBnoFjc8Gvum7l0dEBvveZUOlAjtVdZ+IDME9ajfgUGD9EO8B03w7SjZwBq6jugb556KkqupM4Hu4Ki5jIsLaOIxpvmVAra9yegL3PI0CYIlvoK4ALm5gvVnAN3w7xGpcdVXAdGCZiCxR1816wEu4Z4gU43oh/qGqbvaJpyG9gJdFJAlXEvp+i47QmGaw3nGNMcaExaqqjDHGhMUShzHGmLBY4jDGGBMWSxzGGGPCYonDGGNMWCxxGGOMCYslDmOMMWH5/2O5VlTB5j3BAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA+5ElEQVR4nO3dd3gU1frA8e9LKiGQDgIBEiCU0ANSpAgWwIpgwYK9INfGtXcE9aq/i/V6Rbh2UZq9IIpKVRRI6D30UENIQguknd8fM8ElJmSXZJlN8n6eJ092+ju7s/PuOWfmjBhjUEoppdxVw+kAlFJKVS6aOJRSSnlEE4dSSimPaOJQSinlEU0cSimlPKKJQymllEc0cVQDIjJbRG5zOo7qQkR6i8g6p+PwFhH5QESes19Xin0VkZtEZP5Jpg8Wke0ickhEOp3O2CqaiPwgIjd6cxuaONRJVYakIyJbRCTH/tJnisj3ItLIqXiMMfOMMS2d2v7pVIX2dSxwtzEm1BizxOlg3CUiz4jIRNdxxpgLjDEfurGsEZHmp7JdTRwOEYu+/xXnEmNMKFAf2AP8x+F4VOXSBFhV0gQR8T/Nsfi8an3iEpFHRWSjiBwUkdUiMrjY9NtFZI3L9CR7fCMR+UJE0kUkQ0TetMefkP1FJM7O6v728GwReV5EfgOOAE1F5GaXbWwSkeHFYhgkIktF5IAd60ARuVJEkovNd7+IfH2S3W0mIgvt9XwtIpEuy3YXkd9FJEtElolIX3v880Bv4E371/ybIjJaRP5jTw8QkcMi8m97uKaIHC1ad2nrtaeFici7IrJLRHaIyHMi4mdPu0lE5ovIWLsEsVlELijzAwWMMUeBz4BEl21dJCJL7H3fLiLPuEz7XkTuKfZeLi86FkSklYjMFJH9IrJORK5yme9C+7g4aO/Dg/b4viKS5jJfqceZp/sqIo/Y2zpox3OuPb6riCyw3+td9mcV6LKcEZF/iMgGe9lnRaSZ/fkcEJGpRfMXxS8ij4vIPrFKdNeVEk/xfd0iIg/a72G2iEwRkWCX6Q/b8e0UkdvkJL965STfDZcYHxCRvfY6b3aZHiUi39j7thBoVso2gkTkEOAHLBORjS778YiILAcOi4h/GcdzvIjMsWOdab//E0t6j1zWf579uobLMZJhfxZF36Gic8iNIrLN/jyesKcNBB4Hhor1/Vxmjz9eSyAize24su1lp9jj59qhLLOXHVrS+1MqY0y1/QOuBBpgJdChwGGgvsu0HcCZgADNsX6V+AHLgFeBWkAw0Mte5hlgosv64wAD+NvDs4FtQBvAHwgALsI6qAU4GyuhJNnzdwWygfPtGBsCrYAgYD/Q2mVbS4DLS9nP2fa+tLVj/rwoTnudGcCF9jbOt4djXJa9zWVd5wAr7NdnARuBP12mLXNzvV8C4+146gILgeH2tJuAPOB2+/0eAewEpJT92wKcZ78OAT4EPnKZ3hdoZ8fRHqtEcpk97aqi+O3hDnacgXZs24Gb7c+rE7APSLTn3QX0tl9HuHxufYE0N48zt/cVaGnH08Dl+Gpmv+4MdLfjjAPWACNdljXA10AdrOPvGPAL0BQIA1YDN7rEnw+8gnWsnW3H3NKe/gHwXCn7usX+LBsAkXYcd9rTBgK77e2HABPtuJqX8rme7LtRFOMYrO/Rhfb0CHv6ZGCq/Rm2xTr+55/kXHBCHPZ+LAUaATUp+3he4PJ+9QEO8td37IT3qIRj9j7gDyDWXn48MKnYOeR/dhwd7M+udUnnnOLfWWAS8IQd8/FzVUn77NG50+mTty/92QfKIPv1j8B9JczTA0jHTgbFpp3wIVJy4hhTRgxfFW3XPoBeLWW+ccDz9us2QCYQVMq8s4EXXYYTgVysE9UjwMfF5v+Rv04ixw9Ce7gmcBSIAh7F+sWTBoQCo4E37PlKXS9Qzz74a7pMuwaYZb++CUh1mRZiv49nlLJ/W4BDQBbWSXgn0O4k7/FrRe+r/WXKBBLs4bHAW/brocC8YsuOB0bZr7cBw4E6xebpS7ETxUmOM7f3FevHy17gPCCgjONoJPCly7ABeroMJwOPuAy/DLzmEn8+UMtl+lTgKfv1B5w8cQxzGf4/4G379XvAC8X2x+2TFyd+N/oCObh8D+33pjvWcZ0HtHKZ9i88Txy3uAyf7HhuXML79SnuJ441wLku0+rb8Rf9CDBArMv0hcDV9utnOHni+AiY4Lp8afvsyV91r6q6QaxqoCwRycL6ZRJtT26E9Wu6uEbAVmNM/iludnuxGC4QkT/EqgrJwvpFU1YMYP2qvlZEBLgemGqMOebmdrdi/UqLxipFXVn0Htgx9MI6eP/GGJMDLMb6BdgHmAP8DvS0x82xZz3ZepvY29/lMm08VsmjyG6XbR6xX4aeZP8uM8aEYyWCu4E5InIGgIh0E5FZYlUtZgN32vuOsaq2pgDDxGpzugb42GUfuhXbh+uAM+zpl2N9Xlvt6oAeJQVWxnHm9r4aY1KxEsIzwF4RmSwiDexttBCR70Rkt4gcwDpRRhdbxR6X1zklDLtuM9MYc9hleCtWKcIdu11eH3FZbwNOPA5P+C4UV8Z3AyCj2PewaFsxWCfd4se8p1yXP9nx3ICS3y93NQG+dFnvGqAA6wdWkdLe07I8jFViWygiq0TkFg/iKlW1TRwi0gSr+Hc3EGWfdFZivclgHTQl1YtuBxpLyQ1mh7F+MRY5o4R5jEsMQVjVRmOBenYM092IAWPMH1ilht7Atfx1siuN61VGjbF+0eyzt/GxMSbc5a+WMebF4vG6mINVLdUJWGQPD8CqWiuqOz3ZerdjlTiiXabVMca0KWMfymSMKTDGfIH1xetlj/4U+AZoZIwJA97mr/cYrCR8HXAucMQYs8BlH+YU24dQY8wIe1uLjDGDsBLeV1i/yk/gxnHm6f59aozphXWyMcBL9qRxwFqsklMdrJLgKW3DFiEitVyGG2OV5MpjF1Z1TJFSr3xz47txMulYJYDix7ynXI/9kx3Puyj5/SpywnlBrLa8mGLrvqDYuoONMTs8jPHvE43ZbYy53RjTAKt0/Jac4pVUrqpt4sCq+zRYBxl2w1pbl+nvAA+KSGexNLdPAguxDpQXRaSWiASLSE97maVAHxFpLCJhwGNlxBCIVaeZDuSL1Sja32X6u8DNInKu3YDWUERauUz/CHgTyDPGlHqNum2YiCSKSAhWvfBnxpgCrHrmS0RkgIj42fvTV0SKvuB7sOrBXc0BbgBWG2NysYvGwGZjTLo9T6nrNcbsAn4CXhaROva+NRORs8vYhzLZn9UgrDaHNfbo2sB+Y8xREemKlWiPsxNFIVZ1jWsC/g5oISLXi3UhQICInCkirUUkUESuE5EwY0wecMBeR3FlHWee7FtLETnHPqkexSolFG2zth3DIfsYGXEq2yhmtL2fvYGLgWnlXN9UrOO5tX0cPnWSecv6bpTKPq6/AJ4RkRARScSqUiqPkx3PW7FK4UXvVy/gEpdl1wPBYl2kEQA8ae9bkbeB5+3zCyISYx/D7tgDxEkpV2iKdSFN0Xc5E+tYLHRZtvh32y3VNnEYY1ZjnSgWYL2B7YDfXKZPA57H+rV6EOsXZaR9UF6CVT+7Dat+f6i9zEysao/lWHXI35URw0HgXqwvVCbWCe0bl+kLsRpmX8VqJJ+D9UuzyMdYJ6ETruMuxcdYddO7sapz7rW3sR0YhPULNR3r189D/HVsvA5cIdYVP2/Y437HausoKl2sxjqRFQ27s94bsE4Oq+19/4xSqsfc9K1YV8ccwPrcbjTGFF1e+Q9gjIgcBJ6mhJIBVhJuh8t7aX8+/YGrsX5t78b6hV/0pb8e2GJXDd2JVWo5QVnHmYeCgBexSoq7sUo6RT9OHsQ6fg5ilXCmnOI2iuzG+lx2Ap9gNXCvLc8KjTE/AG8As4BUrAZhsEqfxec96XfDDXdjVefsxjru3z/VuO14yjqerwW6YV20MgrreCpaNhvrGHwHq5H+MNZ5o8jrWPv2k32M/mGvyx1FyTxDRFJKmH4m8Kf93fgGq41okz3tGeBDu4rsqhKWLZXYjSSqEhKRmlgNgknGmA1Ox1OZicgNwB12NVC1JtZlphONMbFlzFre7bTGqrYLKkeboU8S65Lv5saYYU7H4g3VtsRRRYwAFmnSKB+72uQfWFefKC8Sq2uPIBGJwCq9fVvVkkZ1oImjkhKRLVjXfz/gcCiVmogMwKp62INVLam8azhWKXkj1gUMFdEWo04zrapSSinlES1xKKWU8ki16LwrOjraxMXFOR2GUkpVKsnJyfuMMTHFx1eLxBEXF8fixYudDkMppSoVESnxDnitqlJKKeURTRxKKaU8oolDKaWUR6pFG4dSqurKy8sjLS2No0ePOh1KpRUcHExsbCwBAQFuza+JQylVqaWlpVG7dm3i4uIQKU+HwNWTMYaMjAzS0tKIj493axmtqlJKVWpHjx4lKipKk8YpEhGioqI8KrFp4lBKVXqaNMrH0/dPE4dSSimPaOJQSlUKQ8cvYOj4BWXP6JDnn3+eNm3a0L59ezp27Miff/7Ja6+9xpEjR8pe2E1xcXHs27fvlJf/4IMPuPvuu8sdhzaOK6VUOS1YsIDvvvuOlJQUgoKC2LdvH7m5uQwdOpRhw4YREhJS9kq8oKCgAD8/vwpfr5Y4lFKqnHbt2kV0dDRBQdbDIaOjo/nss8/YuXMn/fr1o1+/fgCMGDGCLl260KZNG0aNGnV8+bi4OEaNGkVSUhLt2rVj7VrrYYsZGRn079+fNm3acNttt+Ham/lll11G586dadOmDRMm/PUomdDQUB544AE6dOjAggULeP/992nRogVdu3blt99O9eGTJ9ISh1Kqaunbt2LXN3t2mbP079+fMWPG0KJFC8477zyGDh3KvffeyyuvvMKsWbOIjo4GrOqsyMhICgoKOPfcc1m+fDnt27cHrGSTkpLCW2+9xdixY3nnnXcYPXo0vXr14umnn+b777/n3XffPb7N9957j8jISHJycjjzzDO5/PLLiYqK4vDhw3Tr1o2XX36ZXbt2ce2115KcnExYWBj9+vWjU6dO5X5LtMShlFLlFBoaSnJyMhMmTCAmJoahQ4fywQcf/G2+qVOnkpSURKdOnVi1ahWrV68+Pm3IkCEAdO7cmS1btgAwd+5chg2znj570UUXERERcXz+N954gw4dOtC9e3e2b9/Ohg3Wg0D9/Py4/PLLAfjzzz/p27cvMTExBAYGMnTo0ArZXy1xKKWqFjdKCN7g5+dH37596du3L+3atePDDz88YfrmzZsZO3YsixYtIiIigptuuumEeyeKqrn8/PzIzz/503Rnz57Nzz//zIIFCwgJCaFv377H1xUcHOyVdg1XWuJQSqlyWrdu3fFf/ABLly6lSZMm1K5dm4MHDwJw4MABatWqRVhYGHv27OGHH34oc719+vTh00+tJxr/8MMPZGZmApCdnU1ERAQhISGsXbuWP/74o8Tlu3Xrxpw5c8jIyCAvL49p06aVd1cBLXEopVS5HTp0iHvuuYesrCz8/f1p3rw5EyZMYNKkSQwcOJAGDRowa9YsOnXqRKtWrWjUqBE9e/Ysc72jRo3immuuoU2bNpx11lk0btwYgIEDB/L222/TunVrWrZsSffu3Utcvn79+jzzzDP06NGD8PBwOnbsWCH7Wy2eOd6lSxejD3JSqnIruodjyvAeJ4xfs2YNrVu3diKkKqWk91FEko0xXYrPq1VVSimlPKKJQymllEc0cSillPKIJg6llFIe0cShlFLKI5o4lFLVjq/3tOvr9D6OslR0vzdKqVPy9M4D1otJdU6cMGoU1PDwN3CO3dX5unXlDwzwS0ykXYsWGGPw8/PjzSef5KykpApZ98nc9uST3H/TTSQ2b17yDC1bemW7mjiUUqqcagYHs/SrrwD4cd48HnvlFeZMnOj17b7z3HNe30ZJNHGUxaF+b5RSJxpTyg2ArFnj+S/r2fut/xX1i1zk+LoOLF9ORMOG0LIlhw4dYtCgQWRmZpKXl8dzzz3HoEGDePrpp4mMjGTkyJEAPPHEE9StW5f77ruPf//730ydOpVjx44xePBgRo8ezeHDh7nqqqtIS0ujoKCAp556iqFDh9K3b1/Gjh1Lly5dGDFiBIsWLSInJ4crrriC0aNHA1aX7TfeeCPffvvt8W5HWrVqVa7d1cShlKoyRn+7itVFVVonsXqXNY877RyJDeow6pI2J50nJyeHjh07cvToUXbt2sWvv/4KWB0Ofvnll9SpU4d9+/bRvXt3Lr30Um655RaGDBnCyJEjKSwsZPLkySxcuJCffvqJDRs2sHDhQowxXHrppcydO5f09HQaNGjA999/D1h9VRXnaZft5aGN40opVU41a9Zk6dKlrF27lhkzZnDDDTdgjMEYw+OPP0779u0577zz2LFjB3v27CEuLo6oqCiWLFnCTz/9RKdOnYiKiuKnn346PpyUlMTatWvZsGED7dq1Y+bMmTzyyCPMmzePsLCwv8XgaZft5aElDqVUlVFWyaBIaf1eVYQePXqwb98+0tPTmT59Ounp6SQnJxMQEEBcXNzx7s9vu+02PvjgA3bv3s0tt9wCgDGGxx57jOHDh/9tvSkpKUyfPp0nn3ySc889l6effvr4tIrsst0dWuJQSqkKtHbtWgoKCoiKiiI7O5u6desSEBDArFmz2Lp16/H5Bg8ezIwZM1i0aBEDBgwAYMCAAbz33nscOnQIgB07drB371527txJSEgIw4YN46GHHiIlJeWEbZ5Kl+3loSUOpZQqp6I2DrBKDR9++CF+fn5cd911XHLJJbRr144uXbqc0CgdGBhIv379CA8PP/7gpf79+7NmzRp69LBKQqGhoUycOJHU1FQeeughatSoQUBAAOPGjTth+x06dPC4y/by0G7VlVKVQkV2q+7Nqip3FRYWkpSUxLRp00hISHAsjiKedKuuJQ6lVLXjZMIAWL16NRdffDGDBw/2iaThKa8mDhEZCLwO+AHvGGNeLDa9CfAeEAPsB4YZY9LsaTOA7sB8Y8zFLsvEA5OBKCAZuN4Yk+vN/VBKqYqUmJjIpk2bnA7jlHmtcVxE/ID/AhcAicA1IpJYbLaxwEfGmPbAGOAFl2n/Bq4vYdUvAa8aY5oDmcCtFR27UqpyqQ5V7t7k6fvnzauqugKpxphNdolgMjCo2DyJwK/261mu040xvwAHXWcWEQHOAT6zR30IXFbhkSulKo3g4GAyMjI0eZwiYwwZGRkEBwe7vYw3q6oaAttdhtOAbsXmWQYMwarOGgzUFpEoY0xGKeuMArKMMUUXIqfZ2/kbEbkDuAM4/oB3pVTVExsbS1paGunp6U6HUmkFBwcTGxvr9vxON44/CLwpIjcBc4EdQEFFrNgYMwGYANZVVRWxTqWU7wkICCA+Pt7pMKoVbyaOHUAjl+FYe9xxxpidWCUORCQUuNwYk3WSdWYA4SLib5c6/rZOpZRS3uXNNo5FQIKIxItIIHA18I3rDCISLSJFMTyGdYVVqYxViTkLuMIedSPwdYVGrZRS6qS8ljjsEsHdwI/AGmCqMWaViIwRkUvt2foC60RkPVAPeL5oeRGZB0wDzhWRNBEZYE96BLhfRFKx2jze9dY+KKV8Q25+IekHj7ErO4ejeRVSm63KQe8cV0r5rEPH8pm8cBvvzt/Mrmyr076EuqG8OrQjbRv+vYdYVbH0znGlVKWRfvAYH/y+mY8XbOXA0Xy6N42kTrA/IGTl5HLZf39j5HkJ3Hl2M/z9tK/W000Th1LKZ2zed5gJczfxeUoaeQWFDGxzBsPPbkbHRuEu/Ut156mvVzH2p/X8unYvr1zVkbjoWg5HXr1o4lBKOW7p9izGz9nIjFW7CfCrwRWdY7m9d1PiS0gI4SGB/OeaTpzXui5PfbWSC9+YxxMXtebaro2x7hFW3qaJQynlCGMMs9enM37ORv7YtJ86wf78o28zbjornpjaQWUuP6hjQ7rGR/LQtOU88eVKfl69h5euaE/d2u7fAa1OjSYOpdRplVdQyHfLdzJ+zibW7j5I/bBgnryoNVd3bUxokGenpPphNfnolq58tGALL/ywlgGvzuWFIe0Y2La+l6JXoIlDKXWaHD6Wz+RF23l33iZ2Zh+lRb1QXr6yA5d0aECg/6k3cNeoIdzUM55eCTH8c8pS7pyYwuVJsYy6NJE6wQEVuAeqiCYOpZRX7Tt0jA9/38JHC7aSnZNH1/hInhvcln4t61Zom0TzuqF88Y+z+M+vqfx3Vip/bMpg7JUd6NEsqsK2oSyaOE4TX3jimFKn09aMw/xv3iamLU4jt6CQ/on1GH52M5IaR3htmwF+Nbj//Bb0bRnD/VOWcu07f3Bbr3ge6N+S4AA/r223utHEoZSqUMvTshg/ZxM/rNyFf40aXN65Ibf1bkqzmNDTFkNS4wim39ebf01fw//mbWbu+n28MrQDbRroTYMVQROHUqrcjDHM3bCP8XM28vvGDGoH+zP87GbcfFYcdes4c5VTSKA/z13WjnNb1+Phz5Zz2X9/4/7zW3JHn6b41dDLdstDE4dS6pTlFxTy/YpdvD1nE2t2HaBenSAev7AV13RtTG0faZju17IuP43swxNfreClGWv5de0eXr6yI42jQpwOrdLSxKGU8tiR3HymLNrOO/M2syMrh+Z1Q/n3Fe0Z1LFhua6Q8paIWoH899okvlq6g6e/WsUFr8/l6UsSuapLI71p8BRo4lBKuS3j0DE+XLCVjxZsIetIHmfGRTD60jac06ouNXy8+kdEGNwplq7xUTw4dRmPfL6Cmav38MKQ9m7dcKj+oolDKYdUpivttmUc4Z35m5i6eDtH8wo5P7Eed57dlM5NIp0OzWMNw2vyyW3deP/3Lbw0Yy0DX7NuGuzf5gynQ6s0NHEopUq1ckc24+du4vvlO/GrIQzpFMvtfZrSvO7pu0LKG2rUEG7tFU/vhGj+OWUpd3yczFVdYnnq4kSfaZvxZZo4lFInMMYwP3Uf4+dsYn7qPmoH+XN7n6bc0jOeeg5dIeUtLerV5st/9OT1X9YzbrZ1RdgrV3Wka3zlK0mdTpo4lFKAdYXU9JW7GT9nI6t2HqBu7SAevaAV13ZrXKW77gj0r8FDA1pxTqu6/HPKMoZOWMAdfZpy//ktCPLXmwZLoolDqWouJ7eAacnb+d+8TWzfn0PTmFq8dHk7LuvUsFqdODs3ieSH+3rz3PdrGD9nE3PWpfPa1R1pdUYdp0PzOZo4lKqm9h/O5aMFW/jw9y1kHskjqXE4T12UyHmt6/n8FVLeUivInxeGtOO81nV55PMVXPqf33hwQAtu7aU3DbrSxKFUNbN9/xHenb+ZKYu2k5NXwHmt6zL87GacGaf1+kXObV2PH0eG8/iXK/jX9LX8vGYvL1/ZgUaRetMgaOJQqtpYtTOb8XM28f2KXdQQ60FIw/s0JaFebadD80lRoUG8Pawzn6fs4JlvVnHB6/MYdUkiV3SOrfY3DWriUKoKM8awYGMG4+ZsZN6GfYQG+XNrr3hu7hlH/bCaTofn80SEKzrH0i0+kgemLeOhz5bz85o9/GtwO6JCq+9Ng5o4lKqCCgoNP6zcxfg5m1ixI5vo0CAeHtiS67o1Iaxm1b1CylsaRYYw6fbuvDt/E2N/XM+A1+bx0uVWB4rVkSYOpaqQo3kFTEtO439zN7Ft/xHio2vxwpB2DO7UUJ9HUU5+NYQ7+jSjT4sYRk5eyq0fLuaaro148qJEann4yNvKrnrtrVJVVNaRXD5esJUPft9CxuFcOjYK5/ELW3N+Yj29GqiCtTqjDl/f3ZNXZ25g/NyN/JaawatDO1TK7ldOlSYOpSqxtMy/rpA6klvAOa3qMrxPU7rGR1b7BlxvCvL349ELrJsG75+6lCvfXsCIvs2479wWPtk7cEXTxKFUJbRm1wEmzN3EN8t2IsClHRtwR5+merPaadY1PpIZI/vw7Ler+e+sjcxel86rQzvSoopfqaaJQ6lKwhjDH5v28/acjcxZn05IoB83nRXHrb3iaRCuV0g5JTTIn5euaM+5revy2BcruPg/83l4QEtu6Rnv6I2U3ux9WROHUj6uoNDw4yqrD6lladlEhwby0ICWDOvWhLAQvULKV/RvcwZJTSJ49PMVPPf9Gn5Zs5exV3WgYRVM6po4lPJRR/MK+DzFukJqS8YR4qJCeH5wWy5PitUrpHxUdGgQ/7uhM9MWpzH621UMfHUuYy5rw2UdG1apNidNHEr5mOwjeXz8xxY++H0L+w7l0iE2jHHXJdG/zRl6hVQlICJcdWYjujeN4oFpS/nnlGXMXL2H5y9rR0StQKfDqxCaOJTyETuzcnh3/mYmLdzGkdwCzm4Rw51nN6N7U71CqjJqHBXC5Dt6MGHuJl6ZuY5FWzL5vyva069lXadDKzdNHEo5bN3ug4yfu5Fvlu7EAJd2sK6Qal1fr5ByVRkesVucXw1hRN9m9GkRzf1TlnHz+4u4rltjnrioNSGBlff0W3kjV6qSO5Kbz/b9OQx4bS41A/y4vkcTbu0VT2yE9sBa1bRpEMbXd/fklZnr+d+8TfaTBjvQqXGE06GdEk0cSjlgz4GjrN19EGPg/vNbcH33JlWm/luVLDjAj8cvbM05rerywNRlXPH2Au7q24x7zk0gwK9y3TRYuaJVqgrIzS9kxMRkCgoNrevX4d5zEzRpVCPdm0bxw8jeXNaxIW/8msqQt34nde9Bp8PyiCYOpU6zZ79bTcq2LJrGhBISqJfVVkd1ggN4+aoOvD0sibTMI1z0xnze/20zhYXG6dDc4tXEISIDRWSdiKSKyKMlTG8iIr+IyHIRmS0isS7TbhSRDfbfjS7jrxGRFfYyM0Qk2pv7oFRF+iw5jY//2ModfZoSpaWMam9g2/r8+M8+9GwezehvV3PDewvZlZ3jdFhl8lriEBE/4L/ABUAicI2IJBabbSzwkTGmPTAGeMFeNhIYBXQDugKjRCRCRPyB14F+9jLLgbu9tQ9KVaSVO7J54ssV9GgaxcMDWjodjvIRdWsH8+6NXfjX4HakbMtkwKtz+XrpDqfDOilvlji6AqnGmE3GmFxgMjCo2DyJwK/261ku0wcAM40x+40xmcBMYCAg9l8tsS5srwPs9OI+KFUhMg/ncufEZCJrBfKfazvhX8kaQ5V3iQjXdmvM9Ht707xuKPdNXsrdn6aQdSTX6dBK5M2jtyGw3WU4zR7nahkwxH49GKgtIlGlLWuMyQNGACuwEkYi8G5JGxeRO0RksYgsTk9PL+++KHXKCgoN905ewt4Dxxg3rDPR1fiRo+rk4qJrMXV4Dx4a0JIZK3cz4LW5zF3ve+cvp3/2PAicLSJLgLOBHUBBaTOLSABW4ugENMCqqnqspHmNMROMMV2MMV1iYmIqPHCl3PXqzPXM27CP0YPa0LFRuNPhKB/n71eDu/o156u7elInOIAb3lvI01+vJCe31FPjaefNxLEDaOQyHGuPO84Ys9MYM8QY0wl4wh6XdZJlO9rzbDTGGGAqcJaX4leq3H5atZs3Z6UytEsjruna2OlwVCXStmEY397Ti1t6xvPRgq1c9MY8lm3PcjoswLuJYxGQICLxIhIIXA184zqDiESLSFEMjwHv2a9/BPrbDeIRQH973A4gUUSKihDnA2u8uA9KnbJN6Yd4YOoy2seGMXpQG6fDUZVQcIAfT1+SyCe3dSMnr4Ah437ntZ/Xk1dQ6GhcXkscxph8rCuefsQ6uU81xqwSkTEicqk9W19gnYisB+oBz9vL7geexUo+i4AxdkP5TmA0MFdElmOVQP7lrX1Q6lQdPpbP8I+TCfCvwbhhnbUbdFUuPZtHM2NkHy7t0IDXft7AFW8vYFP6IcficavLERH5AqsR+gdjjNupzhgzHZhebNzTLq8/Az4rZdn3+KsE4jr+beBtd2OoSN58opaqOowxPPzZcjamH+LjW7tVyQf5qNMvrGYArw7tyHmt6/HEVyu48I15PH5ha67v3uS0957sbonjLeBaYIOIvCgiehG6UqV4Z95mvl+xi4cHtqJnc70/VVWsi9rX58eRfegaH8XTX6/ixvcXsefA0dMag1uJwxjzszHmOiAJ2AL8LCK/i8jN9pVOSing9437eHHGWi5oewbD+zR1OhxVRdWrE8yHN5/Js5e1ZeHmDPq/Opfvlp++W9rcbuOw76+4CbgNWIJ1B3cS1s15SlV7O7NyuOfTJcRFhfDvKzvow5eUV4kI13dvwvR7exMXXYu7P13CyMlLyM7J8/q23W3j+BJoCXwMXGKM2WVPmiIii70VnFKVxbH8AkZ8ksKx/ELGX9+F0CB9YoE6PZrGhPL5nT14a/ZGXv9lA39u3s/YKzt4dZvuljjeMMYkGmNecEkaABhjunghLqUqlWe+Wc2y7VmMvbI9zeuGOh2Oqmb8/Wpw77kJfPmPs6gZ6Md17/zJ1ozDXutt193EkSgi4UUD9v0V//BKREpVMlMWbWPSwm2M6NuMgW3rOx2Oqsbax4bz/T29uemsOHYfOMaKndnszq74hnN3E8ft9h3dANgdD95e4dEoVcksT8viqa9X0at5NA/214sNlfNqBvrxzKVtaHVGbUIC/YmpXfF9o7lbEesnImJ381HUZbo+TEBVa/sP5zJiYgoxoUG8cU0n/GpoY7jyHWE1AwirGeCV49LdxDEDqyF8vD083B6nVLWUX1DIPZNSSD90jM/vPItIfSiTqkbcTRyPYCWLEfbwTOAdr0SkVCUw9qf1/Jaawf9d0Z52sWFOh6PUaeVW4rC7GRln/ylVrf2wYhdvz9nItd0ac1WXRmUvUArtukZVVu7ex5GA9VjXRCC4aLwxRm+NVdVK6t6DPDhtGR0bhTPqkuJPQlaqenD3qqr3sUob+UA/4CNgoreCUsoXHTyax/CPkwkO8GPcsCSC/LXHW1U9uZs4ahpjfgHEGLPVGPMMcJH3wlLKtxhjeGjacrZkHOHNa5OoH6Y93qrqy93G8WP2A5c2iMjdWA9U0ttjVbXx9pxNzFi1mycvak2PZlFOh6OUo9wtcdwHhAD3Ap2BYcCN3gpKKV8yf8M+/v3jWi5uX59be8U7HY5SjiuzxGHf7DfUGPMgcAi42etRKeUj0jKPcM+kFJrXDeWly9trj7dK4UaJwxhTAPQ6DbEo5VOO5hUwYmIK+QWG8dd3oZb2eKsU4H4bxxIR+QaYBhwuGmmM+cIrUSnlMGMMT3+9khU7svnfDV2Ij67ldEhK+Qx3E0cwkAGc4zLOAJo4VJU0aeF2pi5O455zmnN+Yj2nw1HKp7h757i2a6hqY8m2TJ75ZhV9WsQw8rwWToejlM9x987x97FKGCcwxtxS4REp5aB9h44xYmIK9cKCeOPqjtrjrVIlcLeq6juX18HAYOD0PRm9CjiWX4ifnoN8Wn5BIXd/mkLmkVw+H3EW4SHa461SJXG3qupz12ERmQTM90pEVUz2kTxe/2UDy7ZnEVYzwOlw1Em8NGMtf2zaz8tXdqBtQ+3xVqnSnOr1hQlA3YoMpKopKDRMXrSNl39aT+aRXAL9a5B9NI+jeQUEB2gfR77mu+U7+d+8zdzQowmXd451OhylfJpbd46LyEEROVD0B3yL9YwOVYIFGzO46I15PPHlShLqhvLdPb1oEhWCMbB4S6bT4ali1u85yMOfLadzkwievEh7vFWqLO5WVdX2diBVwfb9R/jX9DX8sHI3DcNr8tZ1SVzQ9gxEhDrBAQgwLzWdXgnRToeqbAfsHm9DAv1567okAv3d7YVHqerL3auqBgO/GmOy7eFwoK8x5ivvhVZ5HD6Wz7jZG5kwbxN+Ijxwfgtu79P0hCopvxpCaLA/8zfsgwscDFYdV1hoeGDqMrbvP8Knt3enXp3gshdSSrndxjHKGPNl0YAxJktERgFfeSWqSqKw0PD1sh28+MNa9hw4xmUdG/DIBa1K7XI7rGYAq3YeIOPQMaJCg05ztKq4t2anMnP1HkZdkkjX+Einw1Gq0nA3cZRUfq/WHfcs3Z7F6G9XsWRbFu1jw3jruiQ6Nzn5ySesZgBpmTn8tjGDSzs0OE2RqpLMWZ/OyzPXM6hjA246K87pcJSqVNw9+S8WkVeA/9rDdwHJ3gnJt+XmF/LA1GV8npJGTO0gxl7ZgSGdGlLDjRvFagX6EVYzgHnr0zVxOGj7/iPcO2kJLevV5oUh7bTHW6U85G7iuAd4CpiCdQf5TKzkUW0czStgZ1YOO7JyWLXzACP6NuOufs0J9aDHVBGhZ/Mo5qfuwxijJywH5OQWMPzjZIwxjL++MyGB1brgrKqwKcN7eG3d7l5VdRh41GtRVAKHj+WzM/soYTUD+OqunjSJOrXeUns1j2H6it1sTD9M87r6EMXTyRjDE1+tYPWuA7x3U5dT/gyVqu7cvY9jpn0lVdFwhIj86LWofFBUaBDtG4bRol7tcp1wetuX4s7fkF5RoSk3TfxjK1+k7GDkeQmc00p7vFXqVLl70Xq0MSaraMAYk0k1vHO8Iq7xbxQZQlxUCPM27KuAiJS7krfuZ8x3qzmnVV3uPSfB6XCUqtTcPRMWikjjogERiaOE3nKVe3olRPPHpgzyCgqdDqVa2HvwKCMmplA/rCavXtXRrQsZlFKlczdxPAHMF5GPRWQiMAd4zHthVW29msdwOLeAJduynA6lyssrKOTuT5Zw4Gge46/vTFiIdjSpVHm5lTiMMTOALsA6YBLwAJBT1nIiMlBE1olIqoj8rXFdRJqIyC8islxEZotIrMu0G0Vkg/13o8v4QBGZICLrRWStiFzuzj74kh7Noqgh2s5xOvxr+hoWbtnPS5e3p3X9Ok6Ho1SV4G7j+G3AL1gJ40HgY+CZMpbxw7rv4wIgEbhGRIr3IDcW+MgY0x4YA7xgLxsJjAK6AV2BUSISYS/zBLDXGNPCXu8cd/bBl4TVDKBDo3DmajuHV329dAfv/7aFm3vGMahjQ6fDUarKcLeq6j7gTGCrMaYf0AnIKmOZrkCqMWaTMSYXmAwMKjZPIvCr/XqWy/QBwExjzH67IX4mMNCedgt2gjHGFBpjKuXZt3dCDMvTssg+kud0KFXSml0HeOTz5XSNi+TxC1s7HY5SVYq7ieOoMeYogIgEGWPWAi3LWKYhsN1lOM0e52oZMMR+PRioLSJRpS3rcknwsyKSIiLTRKTE6ypF5A4RWSwii9PTfa9KqHdCNIUGFmyqlHnPp2Xn5HHnxGTqBAfw5nWdCPDTHm+VqkjufqPS7JP2V8BMEfka2FoB238QOFtElgBnAzuAgpPM7w/EAr8bY5KABVjVXX9jjJlgjOlijOkSExNTAaFWrI6NwgkN8tfLcitYYaHhn1OWsjMrh3HDkqhbW3u8VaqiuXvn+GD75TMiMgsIA2aUsdgOoJHLcKw9znW9O7FLHCISClxu97y7A+hbbNnZQAZwBPjCHj8NuNWdffA1AX416N40UhNHBXvj1w38unYvYwa1KbPTSaXUqfG4DG+MmWOM+cZutziZRUCCiMSLSCBwNfCN6wwiEi0iRTE8Brxnv/4R6G/foR4B9Ad+NMYYrKcP9rXnOxdY7ek++IreCTFs23+EbRlHnA6lSvh17R5e/2UDQ5Iacn33Jk6Ho1SV5bXKX2NMPnA3VhJYA0w1xqwSkTEicqk9W19gnYisB+oBz9vL7geexUo+i4Ax9jiwHln7jIgsB67HutKrUip6EuC8VN9rg6lstmYcZuTkpbQ+ow7/Gqw93irlTV7tGtQYMx2YXmzc0y6vPwM+K2XZ9/irBOI6fivQp2IjdUbT6Fo0CAtm/oZ9XNdNfyGfqqIeb0WE8dd3PuHJi0qpiqeXmzhIROiVEM1vqfsoKNQeXE6FMYZHv1jOuj0HeeOaTjSKDHE6JKWqPE0cDuudEMOBo/ksT8tyOpRK6YPft/D10p3cf14Lzm7he1fPKVUVaeJwWM/m0YjAfL26ymMLN+/n+e/XcF7retzVr7nT4ShVbWjicFhkrUDaNKjDvFRNHJ7Yc+Ao//gkhUaRIbwytIP2eKvUaaSJwwf0ah5DytZMDh3LdzqUSiE3v5B/fJLCkdx83h7WmTrB2uOtUqeTJg4f0DshmvxCw5+bMpwOpVJ47vvVJG/N5P+uaE/LM2o7HY5S1Y4mDh/QuUkEwQE1Kv1d5EPHL2Do+AVe3cYXKWl8tGArt/eO5+L2Dby6LaVUyTRx+IDgAD+6xkcxX9s5Tmrljmwe+2IF3ZtG8sjAVk6Ho1S1pYnDR/RuHk3q3kPsyi7z+VjVUtaRXEZ8kkxESCBvXpuEv/Z4q5Rj9NvnI453P1LJq6u8oaDQcO/kpezJPsa4YUlEhwY5HZJS1ZomDh/R6ozaRIcG6f0cJXjt5/XMXZ/OqEsT6dQ4ouwFlFJepYnDR4gIve3uRwq1+5HjZq7ew39+TeWqLrFc27Wx0+EopdDE4VN6NY8m43Auq3cdcDoUn7Ap/RD3T1lKu4ZhjBnUVnu8VcpHaOLwIUXtHHp1FRw+ls+dE5Px9xPGDUvSHm+V8iGaOHxIvTrBtKgXWu3bOYwxPPL5clL3HuKNazoRG6E93irlSzRx+JjeCTEs3LKfo3kne/R61fbu/M18t3wXDw5oSe8E7fFWKV+jicPH9EqIJje/kIWb95c9cxW0YGMGL/ywlgFt6jHi7GZOh6OUKoEmDh/TLT6SQL8a1bKdY1d2Dnd/mkJcVAhjr+ygjeFK+ShNHD4mJNCfpCbh1e5GwGP5BYyYmMLRvALGX9+Z2trjrVI+y6vPHK9qpgzvcVq20zshhn//uI70g8eIqV097pIe8+1qlm7PYtx1STSvqz3eKuXLtMThg3rbl+X+Vk2qq6Yu3s4nf25j+NlNuaBdfafDUUqVQROHD2rTIIzwkIBqUV21Ii2bJ79aSc/mUTzUv6XT4Sil3KCJwwf51RB6Notmfmo6xlTd7kf2H87lzonJRNcK5I2rO2mPt0pVEvpN9VG9E6LZc+AYqXsPOR2KVxQUGu6dtIT0Q8d4+/rORGmPt0pVGpo4fFRR9yNzq2h11dif1jE/dR/PDmpD+9hwp8NRSnlAE4ePio0IIT66FvM3pDsdSoWbsXI342Zv5JqujRl6pvZ4q1Rlo4nDh/VqHs2fm/eTm194wvjT8Wxvb0nde4gHpy2jQ6Nwnrk00elwlFKnQBOHD+uVEM2R3AJStmU6HUqFOHQsn+EfLybIvwbjrksiyF97vFWqMtLE4cN6NIvCr4YwrwpUVxljeGjaMjbvO8x/ru1Eg/CaToeklDpFmjh8WJ3gADo2Cq8S3ayPn7uJH1bu5tELWnFWs2inw1FKlYMmDh/Xq3k0y3dkk3Uk1+lQTtlvqfv4vxlruahdfW7v3dTpcJRS5aSJw8f1TojGGPh9Y4bToZySHVk53DNpCc1iQvm/K9prj7dKVQGaOHxch0bh1A7yr5TtHEfzChgxMZnc/ELevr4ztYK0T02lqgJNHD4uwK8G3ZtFMW/DvkrX/cgz36xieVo2L1/VgWYxoU6Ho5SqIJo4KoHeCdGkZeawNeOI06G4bdLCbUxetJ27+jVjQJsznA5HKVWBNHFUAr2aW1chzask3awv3Z7FqK9X0TshmvvP1x5vlapqNHFUAvHRtWgYXrNSdD+SV1DIiInJ1K0TxBtXd8KvhjaGK1XVeDVxiMhAEVknIqki8mgJ05uIyC8islxEZotIrMu0G0Vkg/13YwnLfiMiK70Zv68QEXonRPN7agb5BYVlL+AQYwypew+x/3Aubw/rTEStQKdDUkp5gdcSh4j4Af8FLgASgWtEpHjnRGOBj4wx7YExwAv2spHAKKAb0BUYJSIRLuseAlTN/sZL0SshmoPH8lmWlu10KCcoLDSs3X2AiX9sZd2eQxw4ms/zg9vRtmGY06EppbzEm9dHdgVSjTGbAERkMjAIWO0yTyJwv/16FvCV/XoAMNMYs99ediYwEJgkIqH2MncAU70Yv0/p2SwaERy/izwnt4Cl27NI3rqfRVsySdmWycGj+QAE+Amx4TW5onNsGWtRSlVm3kwcDYHtLsNpWCUIV8uAIcDrwGCgtohElbJsQ/v1s8DLwEkvMRKRO7CSC40bO99195ThPcq1fEStQNo2CGN+ajo1TuNNdHsPHiV5SyaLtmSSvHU/q3YeIL/Quiw4oW4oF7evT5cmkXSJi+Chacv0Bj+lqgGn78h6EHhTRG4C5gI7gILSZhaRjkAzY8w/RSTuZCs2xkwAJgB06dKlct0AUYreCdGMn7uJjo3C8fdCo3NhoWHD3kMs3rqf5C2ZLN6aybb9Vn4O8q9Bh0bh3NGnKV3iIkhqHEF4yIltGJo0lKoevJk4dgCNXIZj7XHHGWN2YpU4sKugLjfGZInIDqBvsWVnAz2ALiKyBSv2uiIy2xjjOm+V1Sshmrdmb+RgTl6FNDzn5BawLC2L5K2ZLN6yn+StmRywq52iQwPp3CSC67s3oXNcBG0bhBHorxfhKaW8mzgWAQkiEo+VMK4GrnWdQUSigf3GmELgMeA9e9KPwL9cGsT7A4/ZbR7j7GXjgO+qS9IA6NwkgpoBfmQfPbXEkX7w2PG2icVbM1m1I/t4tVPzuqFc2K4+XeIi6dIkgiZRIVqCUEqVyGuJwxiTLyJ3YyUBP+A9Y8wqERkDLDbGfINVqnhBRAxWVdVd9rL7ReRZrOQDMKaoobw6C/L3o2t8JH9uLrvDw8JCQ2r6IRZvybSqnrZmHr/zPNC/Bh1jw7m9T1O6NLGqnfTSWaWUu7zaxmGMmQ5MLzbuaZfXnwGflbLse/xVAilp+hagbYUEWon0Tohmzvp0juWf2BR0NK+AZduzWLw1k2T7LzsnD4CoWla103XdGtO5SSRtG9bRp+8ppU6Z043jykO9E2KANWQcymXGyt3Hq55W7cwmr8CqdmoWU4uBbc6gc1wEZ8ZFEqfVTkqpCqSJo5JpUS+UAD9he2YOd05MJtC/Bu0bhnFrL7vaqUkEkVrtpJTyIk0clYyI0DS6FkdyC3jt6o60bRim1U5KqdNKE0clFB4SSHgIdG4S6XQoSqlqSC/MV0op5RFNHEoppTyiiUMppZRHNHEopZTyiCYOpZRSHtHEoZRSyiOaOJRSSnlEE4dSSimPaOJQSinlEU0cSimlPKKJQymllEc0cSillPKIJg6llFIe0cShlFLKI5o4lFJKeUQTh1JKKY9o4lBKKeURTRxKKaU8oolDKaWUR/SZ45XQlOE9nA5BKVWNaYlDKaWURzRxKKWU8ohWVakKo1VoSlUPWuJQSinlEU0cSimlPKKJQymllEc0cSillPKIJg6llFIe0cShlFLKI5o4lFJKeUQTh1JKKY9o4lBKKeURMcY4HYPXiUg6sPUUF48G9lVgOBVF4/KMxuUZjcszVTWuJsaYmOIjq0XiKA8RWWyM6eJ0HMVpXJ7RuDyjcXmmusWlVVVKKaU8oolDKaWURzRxlG2C0wGUQuPyjMblGY3LM9UqLm3jUEop5REtcSillPKIJg6llFIe0cRRChEZKCLrRCRVRB51OJb3RGSviKx0GRcpIjNFZIP9P+I0x9RIRGaJyGoRWSUi9/lCXHYMwSKyUESW2bGNtsfHi8if9mc6RUQCHYjNT0SWiMh3vhKTHccWEVkhIktFZLE9zhc+y3AR+UxE1orIGhHp4XRcItLSfp+K/g6IyEin47Jj+6d9zK8UkUn2d6HCjzFNHCUQET/gv8AFQCJwjYgkOhjSB8DAYuMeBX4xxiQAv9jDp1M+8IAxJhHoDtxlv0dOxwVwDDjHGNMB6AgMFJHuwEvAq8aY5kAmcKsDsd0HrHEZ9oWYivQzxnR0ue7fFz7L14EZxphWQAes987RuIwx6+z3qSPQGTgCfOl0XCLSELgX6GKMaQv4AVfjjWPMGKN/xf6AHsCPLsOPAY85HFMcsNJleB1Q335dH1jncHxfA+f7YFwhQArQDesOWv+SPuPTFEss1gnlHOA7QJyOySW2LUB0sXGOfpZAGLAZ+yIeX4mrWCz9gd98IS6gIbAdiAT87WNsgDeOMS1xlKzoAyiSZo/zJfWMMbvs17uBek4FIiJxQCfgT1+Jy64SWgrsBWYCG4EsY0y+PYsTn+lrwMNAoT0c5QMxFTHATyKSLCJ32OOc/izjgXTgfbt67x0RqeUDcbm6Gphkv3Y0LmPMDmAssA3YBWQDyXjhGNPEUQUY66eEI9dVi0go8Dkw0hhzwFfiMsYUGKsqIRboCrRyIo4iInIxsNcYk+xkHCfRyxiThFU9e5eI9HGd6NBn6Q8kAeOMMZ2AwxSr/nH42A8ELgWmFZ/mRFx2m8ogrITbAKjF36u4K4QmjpLtABq5DMfa43zJHhGpD2D/33u6AxCRAKyk8Ykx5gtficuVMSYLmIVVRA8XEX970un+THsCl4rIFmAyVnXV6w7HdJz9axVjzF6s+vquOP9ZpgFpxpg/7eHPsBKJ03EVuQBIMcbssYedjus8YLMxJt0Ykwd8gXXcVfgxpomjZIuABPtqhECs4ug3DsdU3DfAjfbrG7HaGE4bERHgXWCNMeYVX4nLji1GRMLt1zWx2l7WYCWQK5yIzRjzmDEm1hgTh3U8/WqMuc7JmIqISC0RqV30GqvefiUOf5bGmN3AdhFpaY86F1jtdFwuruGvaipwPq5tQHcRCbG/n0XvV8UfY041Kvn6H3AhsB6rbvwJh2OZhFVnmYf1K+xWrPrxX4ANwM9A5GmOqRdWUXw5sNT+u9DpuOzY2gNL7NhWAk/b45sCC4FUrOqFIIc+z77Ad74Skx3DMvtvVdHx7iOfZUdgsf1ZfgVE+EhctYAMIMxlnC/ENRpYax/3HwNB3jjGtMsRpZRSHtGqKqWUUh7RxKGUUsojmjiUUkp5RBOHUkopj2jiUEop5RFNHEqVQUQO2f/jROTaCl7348WGf6/I9SvlDZo4lHJfHOBR4nC5Y7c0JyQOY8xZHsak1GmniUMp970I9LafwfBPuyPFf4vIIhFZLiLDAUSkr4jME5FvsO7cRUS+sjsQXFXUiaCIvAjUtNf3iT2uqHQj9rpX2s/JGOqy7tkuz6j4xL5LGBF5UaznoywXkbGn/d1R1UZZv4aUUn95FHjQGHMxgJ0Aso0xZ4pIEPCbiPxkz5sEtDXGbLaHbzHG7Le7QFkkIp8bYx4VkbuN1RljcUOw7pruAETby8y1p3UC2gA7gd+AniKyBhgMtDLGmKIuV5TyBi1xKHXq+gM32N23/4nV5USCPW2hS9IAuFdElgF/YHWgmcDJ9QImGauX3z3AHOBMl3WnGWMKsbp6icPqQvso8K6IDMF6uJBSXqGJQ6lTJ8A9xn4anDEm3hhTVOI4fHwmkb5YPZf2MNZTCZcAweXY7jGX1wVYD+nJx+rR9jPgYmBGOdav1Elp4lDKfQeB2i7DPwIj7O7lEZEWdu+yxYUBmcaYIyLSCutRu0XyipYvZh4w1G5HiQH6YHVUVyL7uShhxpjpwD+xqriU8gpt41DKfcuBArvK6QOs52nEASl2A3U6cFkJy80A7rTbIdZhVVcVmQAsF5EUY3WzXuRLrGeILMPqhfhhY8xuO/GUpDbwtYgEY5WE7j+lPVTKDdo7rlJKKY9oVZVSSimPaOJQSinlEU0cSimlPKKJQymllEc0cSillPKIJg6llFIe0cShlFLKI/8P+wFeEXZVUYwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Merge metrics together.\n",
    "metrics = {it: {m: (np.mean([v[m] for v in vals]),\n",
    "                    np.std([v[m] for v in vals]))\n",
    "                for m in vals[0].keys()} for it, vals in metrics.items()}\n",
    "\n",
    "for met in [\"ece\", \"accuracy\"]:\n",
    "    iterations = list(metrics.keys())\n",
    "    means = [v[met][0] for v in metrics.values()]\n",
    "    std = [v[met][1] for v in metrics.values()]\n",
    "    plt.errorbar(iterations, means, yerr=std, label=\"Bayesian\")\n",
    "    plt.hlines(standard_metrics[met], xmin=0.0, xmax=max(iterations), color=\"red\", label=\"Standard\")\n",
    "    plt.xlabel(\"Iterations\")\n",
    "    plt.ylabel(met)\n",
    "    plt.title(f\"{met} between Bayesian sampling and frequentist\")\n",
    "    plt.legend()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8434f62b",
   "metadata": {},
   "source": [
    "#### Discussion\n",
    "\n",
    "Testing our ECE at multiple iterations, we see that it converges quickly after ~40 iterations. While the accuracy takes a hit in the beginning, it quickly comes back. Of course, sampling brings **noise** to the prediction, but it stabilizes quickly with enough iterations.\n",
    "\n",
    "\n",
    "\n",
    "### Conclusion\n",
    "\n",
    "Using a couple of line of code, we can improve our model's calibration. While we now require multiple predictions per input, the cost should not be too prohibitive for most cases. If you have access to large GPUs, I suggest duplicating your dataset and aggregate the predictions at end. \n",
    "\n",
    "I have gone quickly over Bayesian deep learning and MC-Dropout so here are some resources if you want to know more:\n",
    "1. [BaaL background literature](https://baal.readthedocs.io/en/latest/literature/core-papers.html)\n",
    "2. [BaaL user guide](https://baal.readthedocs.io/en/latest/user_guide/index.html)\n",
    "\n",
    "\n",
    "Earlier, I mentioned model uncertainty versus data uncertainty, if you would like to know more I would recommend the following resources:\n",
    "* [Bayesian active learning for production, a systematic study and a reusable library\n",
    "](https://arxiv.org/abs/2006.09916) (Atighehchian et al. 2020)\n",
    "* [Synbols: Probing Learning Algorithms with Synthetic Datasets (Section 3.3)\n",
    "](https://nips.cc/virtual/2020/public/poster_0169cf885f882efd795951253db5cdfb.html) (Lacoste et al. 2020)\n",
    "\n",
    "\n",
    "If you have any question or suggestion, please contact me at:\n",
    "1. @Dref360 on [Slack](https://join.slack.com/t/baal-world/shared_invite/zt-z0izhn4y-Jt6Zu5dZaV2rsAS9sdISfg)\n",
    "2. frederic.branchaud.charron@gmail.com\n",
    "\n",
    "I'm thinking of more blog posts combining HuggingFace and BaaL, let me know if that interest you!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7cc9c342",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12130668",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}