pyarrow-string-slicing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a1d7c324-3e74-4ea3-8760-f680d8ab027d",
   "metadata": {
    "tags": []
   },
   "source": [
    "# PyArrow String Array Serialization\n",
    "\n",
    "Traditionally, pandas has used Python strings for storing string data in DataFrames/Series\n",
    "due to NumPy's lack of support for string dtypes.\n",
    "However, recent versions of pandas now support Series backed by PyArrow strings.\n",
    "PyArrow strings are intended to be much faster and have a much lighter memory footprint than Python strings,\n",
    "especially for array-based data.\n",
    "\n",
    "However, there is a somewhat long-standing [bug](https://issues.apache.org/jira/browse/ARROW-10739) in PyArrow\n",
    "which prevents efficient slicing of PyArrow string arrays, and limits their usefulness in tools like Dask.\n",
    "By default, a string array slice serializes the entre buffer, making it quite expensive to distribute.\n",
    "\n",
    "Hopefully this PyArrow bug will be rectified some time soon,\n",
    "but in the meantime a number of downstream libraries have attempted to work around it, including:\n",
    "\n",
    "* `dask`: [registers a copyreg serializer for `ArrowStringArrays`](https://github.com/dask/dask/pull/9024/files)\n",
    "* `vaex`:  [uses pyarrow's IPC](https://github.com/vaexio/vaex/blob/caed2cf106007c6a0141a02a5dfdf823fa38799e/packages/vaex-core/vaex/arrow/convert.py#L220-L262) to trim the buffers\n",
    "* `pandas`: [Sets a custom reducer for `ArrowStringArrays`](https://github.com/pandas-dev/pandas/pull/49078/files) (merged this morning!)\n",
    "\n",
    "Given the changing landscape of PyArrow strings, we should check on the current state of things,\n",
    "and what approach we should take in the future. A few questions I would like to evaluate:\n",
    "\n",
    "1. Does the current Dask approach work as intended? Do we see the correct slicing and scaling performance, including at scale?\n",
    "1. The brand-new pandas fix should be in the next major release (2.0). Does it work well for Dask?\n",
    "1. If the pandas fix works well, should we remove Dask's copyreg? Do they conflict at all?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9808574d-b86e-4195-a37a-9f3c26f82261",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "\n",
    "# Ensure we start with 1.5.0\n",
    "%pip install pandas==1.5.0\n",
    "\n",
    "import IPython\n",
    "IPython.get_ipython().kernel.do_shutdown(restart=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e86d944-d9be-4f72-b75a-6bb369b4f1fe",
   "metadata": {},
   "source": [
    "## Reproduce the broken PyArrow State and confirm copyreg approach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c50b16b3-8d55-42d3-ba1f-70c5ccaf087e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.5.0'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import copyreg\n",
    "import contextlib\n",
    "\n",
    "import dask\n",
    "import dask.dataframe as dd  # Trigger copyreg\n",
    "import pandas\n",
    "\n",
    "pandas.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "73248c0b-65c7-42f3-9f1b-b2a2e5a23ca4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove the dask copyreg\n",
    "\n",
    "@contextlib.contextmanager\n",
    "def disable_copyreg(klass):\n",
    "    disp = copyreg.dispatch_table.pop(klass, None)\n",
    "    try:\n",
    "        yield\n",
    "    finally:\n",
    "        copyreg.dispatch_table[klass] = disp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "568e9882-a4bd-40a5-ab3f-8b225817d356",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    ACTwdgXFQdhYmJyRpstdSaQoLMfRFBdBadSFUdjnMgLoFT...\n",
       "1    ZOKJxDwgVxUmcckhmBnRmFPhmgXFtPXoSGOuJqHXjuJoxo...\n",
       "2    enWPALborbSrBXSascFPuXcFaZtqimqMhUOeeaJUsBLvpA...\n",
       "3    YvkriIFHHOJfYaLpiZJEbrKmhLEFtKxBOUmWGbUZCjnhWw...\n",
       "4    NqjlVbUfNxJRWBjtHZVCOIIcgwnsCwJbEzYnUsPHEiBBKP...\n",
       "Name: data, dtype: string"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pickle\n",
    "import random\n",
    "import string\n",
    "\n",
    "# Create ~50 MiB of sample string data\n",
    "s = pandas.Series(\n",
    "    [\n",
    "        \"\".join(random.choices(string.ascii_letters, k=random.randint(100, 1000)))\n",
    "        for _ in range(100_000)\n",
    "    ],\n",
    "    dtype=\"string[pyarrow]\",\n",
    "    name=\"data\",\n",
    ")\n",
    "s.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "380e35a9-0e1b-4e1c-bb97-09924d78a5bd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "52\n",
      "52\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "print(len(pickle.dumps(s)) // 1024**2)   # 52 MiB\n",
    "\n",
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    print(len(pickle.dumps(s[:10_000])) // 1024**2)   # 52 MiB, oops!\n",
    "    \n",
    "print(len(pickle.dumps(s[:10_000])) // 1024**2)   # 5 MiB, good"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0735961f-a29f-4ad5-8f7c-f284fd68d767",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[52, 52, 52, 52]\n",
      "[13, 13, 13, 13]\n"
     ]
    }
   ],
   "source": [
    "# Now use Dask to partition the data. Each partition will get the full dataset!\n",
    "@dask.delayed\n",
    "def get_partition_serialized_size(df):\n",
    "    return len(pickle.dumps(df)) // 1024**2\n",
    "\n",
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    result, = dask.compute(\n",
    "        [get_partition_serialized_size(p) for p in dd.from_pandas(s, npartitions=4).to_delayed()],\n",
    "        scheduler=\"processes\",\n",
    "    )\n",
    "    print(result)  # [52, 52, 52, 52]\n",
    "\n",
    "result, = dask.compute(\n",
    "    [get_partition_serialized_size(p) for p in dd.from_pandas(s, npartitions=4).to_delayed()],\n",
    "    scheduler=\"processes\",\n",
    ")\n",
    "print(result)  # [13, 13, 13, 13]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "145e4fef-1b1f-4095-a6fb-87db10ad061d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Do we get the same data back after round-tripping?\n",
    "dd.assert_eq(s, dd.from_pandas(s, npartitions=4), scheduler=\"processes\")\n",
    "dd.assert_eq(\n",
    "    s.str.upper(), \n",
    "    dd.from_pandas(s, npartitions=4).str.upper(),\n",
    "    scheduler=\"processes\",\n",
    ")\n",
    "\n",
    "dd.assert_eq(\n",
    "    s[50_000:].str.slice(0,10), \n",
    "    dd.from_pandas(s, npartitions=4)[50_000:].str.slice(0,10),\n",
    "    scheduler=\"processes\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f29b6710-bc75-46ed-95e9-733cbbbe266b",
   "metadata": {
    "tags": []
   },
   "source": [
    "## How does this affect things in a distributed context?\n",
    "\n",
    "Let's try a shuffling workload, which should involve a lot of sharding of the arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "00dce09f-45db-4c97-8cfe-20a09b8b70a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.5.0'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import dask.dataframe as dd\n",
    "import distributed\n",
    "import pandas\n",
    "import numpy\n",
    "\n",
    "pandas.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b5d36579-c141-4bda-94c7-237108df8fa2",
   "metadata": {},
   "outputs": [],
   "source": [
    "client = distributed.Client()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0b9e3a2b-2931-422a-900c-cb5bb216be15",
   "metadata": {},
   "outputs": [],
   "source": [
    "def disable_copyreg():\n",
    "    import copyreg\n",
    "    import dask.dataframe\n",
    "    del copyreg.dispatch_table[pandas.arrays.ArrowStringArray]\n",
    "    return copyreg.dispatch_table.get(pandas.arrays.ArrowStringArray)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "88b57b50-7693-419d-b1d5-f8e3d5b9a566",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import string\n",
    "\n",
    "def make_partition(_=None):\n",
    "    # Create ~50 MiB of sample string data\n",
    "    s1 = pandas.Series(\n",
    "        [\n",
    "            \"\".join(random.choices(string.ascii_letters, k=random.randint(100, 1000)))\n",
    "            for _ in range(1_000_000)\n",
    "        ],\n",
    "        dtype=\"string[pyarrow]\",\n",
    "        name=\"label\",\n",
    "    )\n",
    "    df = pandas.DataFrame(numpy.random.randint(0, 100, size=(len(s1), 10)))\n",
    "    df.insert(0, \"label\", s1)\n",
    "    return df\n",
    "meta = make_partition()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "88aaa6a5-78a2-4c59-8536-dae189ac0d90",
   "metadata": {},
   "outputs": [],
   "source": [
    "ddf = dd.from_pandas(meta, npartitions=100)\n",
    "#ddf = dd.from_map(make_partition, range(200), meta=meta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "091d4537-aea1-4a3a-b60d-c844a3bed468",
   "metadata": {},
   "outputs": [],
   "source": [
    "sampler = distributed.diagnostics.MemorySampler()\n",
    "\n",
    "with sampler.sample(\"copyreg\"):\n",
    "    distributed.wait(ddf.set_index(\"label\").persist())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "42a8d9e6-65e5-41c6-97ad-557d328e26f7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'tcp://127.0.0.1:35063': None,\n",
       " 'tcp://127.0.0.1:39933': None,\n",
       " 'tcp://127.0.0.1:40263': None,\n",
       " 'tcp://127.0.0.1:41969': None}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.restart()\n",
    "client.run(disable_copyreg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "35342c0e-3820-4c25-8637-fd4504eef1fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "with sampler.sample(\"nocopyreg\"):\n",
    "    distributed.wait(ddf.set_index(\"label\").persist())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "809100b4-7aa3-43f2-805e-2834ca81c130",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='0'>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGyCAYAAAC4Io22AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAA9hAAAPYQGoP6dpAABgV0lEQVR4nO3dd3hUVcIG8HeSSe8JCYEQUugQeg0gRUDFlQU7ihQVey+L67q7WEDWta8IKt+KgAqCCoJiYUFAeu89BBJIIAmkl0mZ8/1xcicZUmYmmZk75f09T557M3Nn5uQwTN6cqhFCCBARERFZgYfaBSAiIiLXwWBBREREVsNgQURERFbDYEFERERWw2BBREREVsNgQURERFbDYEFERERWw2BBREREVsNgQURERFbDYEFERERWo1qw2Lx5M8aNG4fWrVtDo9Fg1apVFj/H8uXL0atXL/j7+yMuLg5vv/229QtKREREZlMtWBQXF6Nnz56YO3dukx7/888/Y9KkSXj00Udx5MgRzJs3D++9916Tn4+IiIiaT+MIm5BpNBqsXLkSEyZMMNxWXl6Ov//97/jqq6+Ql5eHpKQkvPXWWxgxYgQA4N5770VFRQVWrFhheMwHH3yAd999F2lpadBoNHb+KYiIiMhhx1jcf//92Lp1K5YtW4ZDhw7hzjvvxE033YTTp08DAHQ6HXx9fY0e4+fnhwsXLuD8+fNqFJmIiMjtOWSwSElJwdKlS7FixQpcd911aNeuHV588UUMHToUCxcuBADceOON+P7777F+/Xro9XqcOnUKH3zwAQAgMzNTxdITERG5L63aBajPvn37IIRAx44djW7X6XSIiIgAADz00ENISUnBLbfcgoqKCgQHB+OZZ57Bq6++Ck9PTzWKTURE5PYcMljo9Xp4enpi7969dUJCYGAgADku46233sKbb76JS5cuITIyEuvXrwcAxMfH27vIREREBAcNFr1790ZVVRWysrJw3XXXNXqtp6cnYmJiAABLly5FcnIyoqKi7FFMIiIiuoZqwaKoqAhnzpwxfJ+amooDBw4gPDwcHTt2xKRJkzBlyhS8++676N27N3JycrBhwwZ0794dN998M3JycvDtt99ixIgRKCsrw8KFC7FixQps2rRJrR+JiIjI7ak23XTjxo0YOXJkndunTp2KL774AhUVFZg1axYWL16MixcvIiIiAsnJyXjttdfQvXt35OTkYNy4cTh8+DCEEEhOTsbs2bMxcOBAFX4aIiIiAhxkHQsiIiJyDQ453ZSIiIicE4MFERERWY3dB2/q9XpkZGQgKCiIy24TERE5CSEECgsL0bp1a3h4NNwuYfdgkZGRgdjYWHu/LBEREVlBeno62rRp0+D9dg8WQUFBAGTBgoOD7f3yRERE1AQFBQWIjY01/B5viN2DhdL9ERwczGBBRETkZEwNY+DgTSIiIrIaBgsiIiKyGgYLIiIishqH3IRMr9ejvLxc7WKQjXh5eXFreyIiF+VwwaK8vBypqanQ6/VqF4VsKDQ0FNHR0VzLhIjIxThUsBBCIDMzE56enoiNjW10AQ5yTkIIlJSUICsrCwDQqlUrlUtERETW5FDBorKyEiUlJWjdujX8/f3VLg7ZiJ+fHwAgKysLUVFR7BYhInIhDtUkUFVVBQDw9vZWuSRka0pwrKioULkkRERkTQ4VLBTsd3d9/DcmInJNDhksiIiIyDk51BgLIiIityIEUFEClOUDpXmArgCI7AT4haldsiZjsCAiImoqfRVw5YwMCABQWVoTEsryap3ny+/rO9dfM9asZRLw2FY7/hDWxWDhoioqKuDl5aV2MYiIXFd5MbDwZiDzQPOfy0ML+IYCJVeAy0eA3PNAWFzzn1cFHGNhJXq9Hm+99Rbat28PHx8ftG3bFrNnzwYAHD58GNdffz38/PwQERGBhx9+GEVFRYbHTps2DRMmTMBrr72GqKgoBAcH45FHHjGsPrp48WJERERAp9MZvebtt9+OKVOmAABeffVV9OrVC59//jkSExPh4+MDIQTy8/Px8MMPG573+uuvx8GDB42eZ9asWYiKikJQUBCmT5+Ov/71r+jVq5cNa4uIyAWsnSFDhac34B8hv4JaA1FdgbbJQMexQI+JwMBHgeF/BW6cA0yYD0xcCkxbCzy2DXjuGPDyReAfOcCMFKBNf/nc59liYRNCCJRWVKny2n5enhbNXHj55ZexYMECvP/++xg6dCgyMzNx4sQJlJSU4KabbsKgQYOwe/duZGVlYfr06XjyySfxxRdfGB6/fv16+Pr64vfff8e5c+dw//33o0WLFpg9ezbuvPNOPP3001i9ejXuvPNOAEBOTg5+/PFH/PLLL4bnOHPmDJYvX47vvvvOsDbEn/70J4SHh2Pt2rUICQnBp59+ilGjRuHUqVMIDw/HV199hdmzZ2PevHkYMmQIli1bhnfffRcJCQnWqUgiIlcjBLB/CXDgS0DjAdz3PZBwnXWeO34ocGEXcG4L0Ote6zynnWmEUDqG7KOgoAAhISHIz89HcHCw0X1lZWVITU1FQkICfH19UVJeia7//NWexTM49vqN8Pc2L3cVFhYiMjISc+fOxfTp043uW7BgAV566SWkp6cjICAAALB27VqMGzcOGRkZaNmyJaZNm4Y1a9YgPT3dsL7DJ598gr/85S/Iz8+Hh4cHHn/8cZw7dw5r164FAHz44Yf4z3/+gzNnzkCj0eDVV1/Fm2++iYsXLyIyMhIAsGHDBtx6663IysqCj4+PoUzt27fHjBkz8PDDD2PQoEHo168f5s6da7h/6NChKCoqwoEDB5pcf6Zc+29NROTwCjKAg8uAg0uBnFPythEvAyP+ar3XOLMe+PI2ILQt8Oxh6z2vFTT2+7s2doVYwfHjx6HT6TBq1Kh67+vZs6chVADAkCFDoNfrcfLkScNtPXv2NFptNDk5GUVFRUhPTwcAPPTQQ/jtt99w8eJFAMDChQsxbdo0o1aVuLg4Q6gAgL1796KoqAgREREIDAw0fKWmpiIlJQUAcPLkSQwYMMCozNd+T0Tk9k79CnzQHVj/mgwVWj9gwMPAsL9Y93ViB8rxFnlpcpyFE3LorhA/L08ce/1G1V7b7Gurl6iujxCiwS4Vc7palGt69+6Nnj17YvHixbjxxhtx+PBhrFmzxuja2uEFkOM+WrVqhY0bN9Z53tDQ0AbLYedGLCIix6avAn59BdBXAq37AP0eALqOB3wb/qu9yXwC5Wtc2AWc+8MpB3A6dLDQaDRmd0eoqUOHDvDz88P69evrdIV07doVixYtQnFxseEX/9atW+Hh4YGOHTsarjt48CBKS0sNIWXHjh0IDAxEmzZtDNdMnz4d77//Pi5evIjRo0cjNja20XL16dMHly5dglarRXx8fL3XdOrUCbt27cLkyZMNt+3Zs8ein5+I3IxeD5TmAkWX5VdAJBCdpHapbOfYD8CV03LWxpQfbBMoams/SgaLw98Cve+z7WvZgOP/1nYCvr6+eOmllzBjxgx4e3tjyJAhyM7OxtGjRzFp0iTMnDkTU6dOxauvvors7Gw89dRTmDx5Mlq2bGl4jvLycjz44IP4+9//jvPnz2PmzJl48sknjXZ4nTRpEl588UUsWLAAixcvNlmu0aNHIzk5GRMmTMBbb72FTp06ISMjA2vXrsWECRPQr18/PPXUU3jooYfQr18/DB48GN988w0OHTqExMREm9QVETkZvR44txk4sBTIPgEUZQHFWfKvd4XGA3h6PxAWr1oxbUavBza/I88HPWb7UAEAPScCG+cAZzcCeelAaON/RDoaBgsr+cc//gGtVot//vOfyMjIQKtWrfDoo4/C398fv/76K5555hn0798f/v7+uP322/Hee+8ZPX7UqFHo0KEDhg0bBp1Oh4kTJ+LVV181uiY4OBi33347fvrpJ0yYMMFkmTQaDdauXYtXXnkFDzzwALKzsxEdHY1hw4YZQs2kSZNw9uxZvPjiiygrK8Ndd92FadOmYdeuXdaqGiJyRlWVwLYPgb2LgLwG+vr9woHyIqCqXC4S5YrB4tTPQNZRwDsIGPiIfV4zLB6Iv052hRxcCgyfYZ/XtRKHnhXiLqZNm4a8vDysWrXK5LVjxoxBly5d8J///Mdm5RkzZgyio6OxZMkSm72Gu/5bEzmN3+cAm/4lz32Cge53AO3HAEHRQGBL2f2h9Qa+uEX+Arz9v/IaV5F9Eti3GDjwlez2Gfo8MHqm/V7/4DJg5SMyZDy1H/BQf66FubNC2GLhJK5evYrffvsNGzZsMJoa2lwlJSX45JNPcOONN8LT0xNLly7F//73P6xbt85qr0FETqYsH9g5X56PmikXePL2r/9aZU+L0lz7lM0ezm2RgQnVf3e36AgkP2nfMnT5M/DTi0DuOSBtm1zfwkkwWDiJPn36IDc31zBWwlqU7pJZs2ZBp9OhU6dO+O677zB69GirvQYROZldC2S4aNEJGPJs438tu2KwSN8JQMgVNEfNBNqPBjzt/OvS2x9Iug3YtwjY/yWDBVmm9gqcDTl37pxNXtvPzw//+9//bPLcROSEyouB7R/L82Evmm6Cd8VgkSfXD0LnW4BON6lXjp4TZbA4/Zt6ZWgC9TttiIjIcexZCJReBcISgG63mb7eFYNFfnWwCGnT+HW21qoXAI3cmKw4R92yWIDBgoiIpIoyYFv1wPDrnjev+d8Vg4XSYqH2NE9v/5oyZJ9s/FoHwmBBRETSke/kglfBMXJXTnO4WrAQAsi/IM9D2qpbFgCI7CyPOQwWRETkTISomQnSf7qcSmoOVwsWpblARbE8V7srBJAzUgC2WBARkZNJ2w5cOiw31+o7zfzHuVqwyEuTx4AowMsB1thRWiyyT6hbDgswWBAREbCjurWix12Af7j5j6sdLFxhA0NHGbipMASLU+qWwwIMFkRE7i4vDTjxozwf+Khlj1WChb5SLu/t7JTxFWoP3FREVneFFGbItUWcANexICJyRkIAlWWArrD6qwDQFV3zfWHdr/J6bqsokc+ZMBxo2dWycnj5AZ4+QJVOtlr4BFn/Z7UnZUZIiIMEC98QIKgVUJgJ5JwG2vRTu0QmMVhQvYQQqKqqglbLtwiRTVXqAE9vQKORYSF9F3B0ZfVf/wIoL6kOBEV1A0PtHUabS+sHDPuL5Y/TaGTXSWGmDBahDjCTojnyq8dYONLP0aKjrN/sE04RLNgVYgUjRozA008/jRkzZiA8PBzR0dFGO5OmpaVh/PjxCAwMRHBwMO666y5cvnzZ6DlWr16Nfv36wdfXFy1atMBtt9UsTJObm4spU6YgLCwM/v7+GDt2LE6fPm24/4svvkBoaChWrVqFjh07wtfXF2PGjEF6ukze586dg4eHB/bs2WP0mh999BHi4uIghMDGjRuh0Wjw66+/ol+/fvDx8cEff/wBIQT+/e9/IzExEX5+fujZsye+/fbbOmXv0KED/Pz8MHLkSCxatAgajQZ5eXlWqmEiF3RxH7D0HmBWFPCfXsDPLwHzhwCf3yBnZ+xfIpdyPvo9cGadHFx5+YjstijNrRUqNHKTsOAY2R/fpj+QOFLuNdHrPmDgY8CwGcCYN4BbPpCbhd27Arj/F+DRLcAzB4EZqcDL6UDCdU37WZTukJKrVqgYlTlaiwVQa5yFc8wMcew/R4WoaaKzNy9/mcTNtGjRIjz//PPYuXMntm/fjmnTpmHIkCEYPXo0JkyYgICAAGzatAmVlZV4/PHHcffdd2Pjxo0AgJ9++gm33XYbXnnlFSxZsgTl5eX46aefDM89bdo0nD59GqtXr0ZwcDBeeukl3HzzzTh27Bi8vLwAyM3EZs+ejUWLFsHb2xuPP/44Jk6ciK1btyI+Ph6jR4/GwoUL0a9fTdpduHAhpk2bBk2tn3PGjBl45513kJiYiNDQUPz973/H999/j/nz56NDhw7YvHkz7rvvPkRGRmL48OE4d+4c7rjjDjzzzDOYPn069u/fjxdffLGZlU/kovR64OwGYMcnMiwocs8BOz+R51o/uUdERDv5vVeA7F7wCQJ8AmWIMHwfJO9Xe+dLV5oZ4miDNwEgsnp/KAYLK6goAd5src5r/y0D8A4w+/IePXpg5ky5pW6HDh0wd+5crF+/HgBw6NAhpKamIjZWJuAlS5agW7du2L17N/r374/Zs2dj4sSJeO211wzP17NnTwAwBIqtW7di8ODBAICvvvoKsbGxWLVqFe68804AQEVFBebOnYuBAwcCkEGnS5cu2LVrFwYMGIDp06fj0UcfxXvvvQcfHx8cPHgQBw4cwPfff2/0c7z++usYM2YMAKC4uBjvvfceNmzYgOTkZABAYmIitmzZgk8//RTDhw/HJ598gk6dOuHtt98GAHTq1AlHjhzB7NmzLahsIjdwdBXwv1eB3FT5vcYD6H4XkPy47Ds/uxFo2U3uD6H8onYWrhIsykvk8tmA4wzeBGoFC+eYcsquECvp0aOH0fetWrVCVlYWjh8/jtjYWEOoAICuXbsiNDQUx48fBwAcOHAAo0aNqvd5jx8/Dq1WawgMABAREYFOnToZHg8AWq3WqDWic+fORq8xYcIEaLVarFy5EgDw+eefY+TIkYiPjzd6vdrPcezYMZSVlWHMmDEIDAw0fC1evBgpKSkAgJMnT6J///5GzzFgwIDGK4vIneirgHUzgRVTZajwCQYGPAI8uQe47VOgVU+g+x3A+LnAoMecL1QAgF+oPDp7sFBmhHgHAb6hqhbFiNIVkpcmw4+Dc+wWCy9/2XKg1mtbcnl1l4RCo9FAr9dDCGHU1aCofbufn1+DzysamBde3/PW9zrKbd7e3pg8eTIWLlyI2267DV9//TU++OCDOtcHBNS00uj1egCyqyYmJsboOh8fnwbL0VCZidxORRmwfHLN7pSDnwZG/NWi1lCn4CotFoaBm7EWdYXbXEALwC9cbg535bQMow7MsYOFRuP0/wG7du2KtLQ0pKenG1otjh07hvz8fHTp0gWAbO1Yv3497r///nofX1lZiZ07dxq6Qq5cuYJTp04ZHg8AlZWV2LNnj6G14OTJk8jLy0Pnzp0N10yfPh1JSUmYN28eKioqjAaINlR2Hx8fpKWlYfjw4fVe07lzZ6xdu9botmsHiRK5ra0fylCh9ZMtEt3vULtEtmEIFnmqFqPZHHHgpqJlN+DcH8DhFQ4fLNgVYmOjR49Gjx49MGnSJOzbtw+7du3ClClTMHz4cEO3w8yZM7F06VLMnDkTx48fx+HDh/Hvf/8bgByvMX78eDz00EPYsmULDh48iPvuuw8xMTEYP3684XW8vLzw1FNPYefOndi3bx/uv/9+DBo0yKhbokuXLhg0aBBeeukl3HPPPY22lABAUFAQXnzxRTz33HNYtGgRUlJSsH//fnz88cdYtGgRAOCRRx7BiRMn8NJLL+HUqVNYvnw5vvjiCwD1t6AQuY28NGDLe/LclUMF0HiLhTO1YDra4li1DX5aHnd8IsfkODAGCxvTaDRYtWoVwsLCMGzYMIwePRqJiYn45ptvDNeMGDECK1aswOrVq9GrVy9cf/312Llzp+H+hQsXom/fvrjllluQnJwMIQTWrl1r1P3i7++Pl156Cffeey+Sk5Ph5+eHZcuW1SnPgw8+iPLycjzwwANmlf+NN97AP//5T8yZMwddunTBjTfeiDVr1iAhIQEAkJCQgG+//Rbff/89evTogfnz5+OVV14BUNNdQuQWso4DZ/4nv66mAr/9Qy5gFTcESLpd7dLZVn3BQq8HfngSeLdzzf4bjs4RZ4QoOt4AdLgR0FcAv/zVoQObRti5Q7ygoAAhISHIz89HcHCw0X1lZWVITU1FQkICfH0dYPMXJ/HFF1/g2WefNWvdiNmzZ2PZsmU4fPiwzcoze/ZsfPLJJ4Z1NOrDf2tyGUIAm98Bfp9V60YNACFnfjyyGYjurlbp7OPsRmDxeCCyC/DEDnnb/14Ftrwvz8e8Dgx5Rq3Sme+/NwDpO+VaH47YwnQlBfh4oAwX93wDdLrJri/f2O/v2thi4SaKioqwe/dufPTRR3j66aet+tzz5s3D7t27cfbsWSxZsgRvv/02pk6datXXIHJIQgDrX6sJFVHdgKiuAKr/Xut7v+uHCqBui8W+JTWhAgDObrJ/mSxVfAW4UD0+LKaPumVpSEQ7IPkJeb71Q3XL0gjHHrxJVvPkk09i6dKlmDBhgtndIOY6ffo0Zs2ahatXr6Jt27Z44YUX8PLLL1v1NYgcjhCySVpZ2OqG2cDgJ+V5xgHgwm6g932qFc+uageLnDPAj8/K75NuB458B5zfJpcu1zpw9+iJNYCokgMjwxPVLk3Duo4Htn5Qsx6KA2JXCKmC/9bk9H7+q1x6GwD+9C7Qf7q65VGTrhCYUz0uofdkuRx5u+uBSd8B73YCirOAqT/Wv2S4EMDXd8tNzO77HvDwtG/ZFYvHyy6dUTOB655XpwzmKLwk6xQa4B/ZgKeXyYdYC7tCiIhs5fKxmlAxfp57hwoA8A4EPKobwA8ulcfkJ+VS44nVU9VTG+gOyTkFnP5V/lJXa8nq4itA6h/yvNsEdcpgroDI6roWQNFlk5erwaJg8eqrr0Kj0Rh9RUdHW71QXGDJ9fHfmJyaMo20y5+B3pPULYsj0GhqukP0lUB4O7kRGgAkjpDHsxvrf2za9przS7YbVN4oZ+kGAWSLTlAreV6QqW5ZGmBxi0W3bt2QmZlp+LLm7AJPT9kEVl5ebrXnJMdUUiKXpb12xVIih1VVCVSWA9mn5LgBABjGDfcMai9F3v/Bmo3REqpbLC7uA8ry6z7ufO1gcch25WvMUbnVAbpOUOf1LRVcvYdWwUV1y9EAiwdvarVam7RSKM/t7++P7OxseHl5wUPtHfvI6oQQKCkpQVZWFkJDQw1hkshhCQFseEOOwjdsVQ65poCDr4BoV0qw0PoBve6tuT00VrZgXE0Bzm0FOt9s/Lg0M4KFvqp5Yy+EkKFG2dOkNmfqBlEYgoVKW16YYHGwOH36NFq3bg0fHx8MHDgQb775JhITG2460ul00Ol0hu8LCgoavFaj0aBVq1ZITU3F+fPnLS0aOZHQ0FCbBVQiq/rjXflVm5c/MJIzn4z4hctj99vrbqSWOFwGi9RNxsGiIAPIq/VZf+mwDAG1V+394z35Ne1HoHWvppVt12fAzzOAuxbLWRW1Xdglu0Eiuzh+N4giyIVaLAYOHIjFixejY8eOuHz5MmbNmoXBgwfj6NGjiIiIqPcxc+bMMdoO3BRvb2906NCB3SEuzMvLiy0V5Bz2fiFbKwA5nbTPZHmu9XXsqZNqGPiIPA7/a9374q8D9nxu3DoB1Hwf2UVurlWaK5fVrr2k9qHlQHmhDCVNDRYnfpTHvV/UDRa556rL0LFpz60GV2qxGDt2rOG8e/fuSE5ORrt27bBo0SI8/3z903Nefvllo/sKCgqMthCvj4eHB6cgEpG60nYAPz4nz697oWaNCqpfu5Hyqz7KglOXjxmvZ6GMr0gcLrs6Lh+RrRZKsNAVATnVM0XyGl7J18jVs8Dp/wFJt8ldQYUAMqu7WFL/kF0iviE11yvBIizevOd3BEqwKHSRwZu1BQQEoHv37jh9uuENUXx8fBAcHGz0RUTk8LZ+CAi9XOTp+n+oXRrnFhoH+IbKpaizjtXcnla9/Hfb5JoVSmuPs7h0SP4bAKb3G8lLA1Y+BnzUD/j5L8C6mfL2/HSgLE+e6yvkXi61OWWwiJFHB+0KaVaw0Ol0OH78OFq1amWt8hARqS/3HHDyZ3k+/K/Gff5kOY2mphsj44A8lubJFgqgOlj0kOe1p5xe3Fdznt9Ii0VFGfD5WODg13K8BACkbq77fABwYq3x904ZLJSukEy52ZuDsShYvPjii9i0aRNSU1Oxc+dO3HHHHSgoKOC+EETkOirLge3zAAi5FoMz9b07sla95DHzoDxe2A1AyAGTQS1rWiwya7VYZOyvOc9Lb3hHz4NLgYILcn2HKasBjSeQnybHayjP16L63/H0OqCqQp4L4ZzBIigagEa2wJTkqF2aOiwaY3HhwgXcc889yMnJQWRkJAYNGoQdO3YgLi7OVuUjIrK+qgog97ycqXAlxfiYf6Gm+V0ZkEjNp7RYZB6Qx9Pr5DFusDxGJ8ljfpocxOkXBmTUarEoL5S3+4cbP6++Ctj2H3k++Gk5XiO6u3ydtB01LRZ9pso9NoqzgXNb5HiQostya3uNBxDS+Ng/h+LpBQRGyfIXZMhzB2JRsFi2bJmtykFEZB/HVgM/vSD3r2iIVwDQYTTQ4Qb7lcvVKS0Wl4/Krotjq+T3Xf4sj35hQGhbOVbi0mEZDq6elfd5BwLlRbI75NpgcXyNvM43FOgzRd4WN1gGi/PbasZstO4FdLwR2P8lcHKtDBZKa0VIG7vuuWEVwa1rgkVTZ8vYCHc3JSL3UHJVrmVweIX83stfLtwUniC3ow5vV3MMjOK4CmsLi5ezMcry5dTTossyDCTWmkkS3UMGi/SdsiVCeZx/BHBxr7yv9qJkQtRszz7gYcAnUJ63TQZ2zANO/Sq7SAAZVMoKqoPFL8DYfztnN4giOEZ2FTngAE4GCyJyfad+BVY/DRRdks3eQ58Dhr/EtSjsSaORoSB1M7D5bXlbl1sArXfNNe1HyzUntn4E9Jwob2vdWwaIi3trppzqq4Az62VAyTwgV/us3W3VNlkelVChhJrE4XIDr/w0GVKcOlg47loWDBZE5DpyzgAX9xjflroZOPCVPI/oANz6CdCmn/3LRrI7JHUzUHpVft/tNuP7+0wB9i2WYyt2fSpva91HjosAamaGrHocOFSra374X+SaFYrASPlvfaV6KQRlYKh3gAwqF3YD57cyWNgIgwURORa9XrYsCAum0ekrgV0LgB3za6YbGtEAyU8A1/8d8PKzWlHJQrXHAvhH1GxQpvDwBP78H+DT4TX/jjF9gKzj8jwvTQ68Pb5Gft9/uuwCiexU97XaDqoVLGp1n8QNkcHinJMHC2VZ70IGCyKi+hVkAge+lH+xmloMqTFt+gM+QTXfe/kDgx4H4oc0v4zUPMoATkAO2vSs51dQdHdgyNPVYyequ090RfK+vDQ5XbWiWA72HPt2zS6q14obDOxfUvOchtuHyNkh57fKGSGAcwYLtlgQEdVDXyVXQty7CDj1S81fqRoP2RduifB2wA2z5GwOckzhiXKzstKrcsnthgx/CbiaKgfT+gTJ2SKA7Ao5t0Wexw1pOFQANeMsAKBVj1q3D5Lvr9zUmtvCEiz/WdRWO1hcu3GbyhgsiMh8RdnAb6/Irofb/q/xD/ZGnydLDrzbt6RmgB0gfxn0nSY3imKXhevRaIA7F8rQEH9dw9d5+QF3Lar5Xtk7pDRXDsQFata/aEh4AjDib/I1lV/CAOAbLGefKOtp+ATX3Y3VGSg/U0WJXLLcgX4GBgsiMs/pdcCqx2oG0g2bAUR1tvx5jq6Um3uV5srv/cKAnvfKgXtNeT5yLokj5JclfILk+6Q0F0jbJm+LM6Nra8RL9d8eP7QmWITFOdRf+2bz8qtp/cm/yGBBRE4k4wCwcY7sqqgt56TpICCEHDOxd6FcKltfAeSckve17A4MeQboMg7w4m7GZEJIbE0Y9QkxHjdhqbghwPa58twZx1cowhOAi1eB7BM1K5c6AAYLImrY73OATf+S5xpPuVZA4SXg6PdA9snGH1t8BVjztFyXoDaNh9yGfNgM4zUMiBoT2rZmFc22g+QMkqaKSwagASCcO1jE9JXre1zcC3S/Q+3SGDBYEFH9KnVy9DwAJN0BjPybHEy35YPqYHGi4ceW5gKfDpPjJzy8gJEvyw9BQH6QO/OHOalDGcAJmB5fYYpfGNAyCbh82Lnfi8r/qQt7Gr/OzhgsiKh+GQfkdDz/COD2/6vph1bWDMg+1fBjj62WoSK4DXDP18bLMBM1Re1NwuKHNv/5Rs+U3XRJtzf/udQSU73QW+ZBub6Hg+x3wmBBRPU7v1Ue4wYbD25TgkXOKTldtL4maWXkft9pDBVkHcrMEK8A67ynOoyRX84sop3cb6UsD7h8RK4q6gCaOFeMiFze+QZG34fGAZ4+QJUOyDtf93EVZcDZ3+V5xxttW0ZyH/HXyWmiyY87zF/mqtNoarpDLu5Vtyy1MFgQUV36KrnDJGC80BAgWyhadJTn9Q3gPPeHnFsfHNO8kftEtfmFAo/+IZdlpxqGcRYMFkTkyC4fAXQFgHdQ/eHAMM6inmBx8md57Hijc64PQORMlA31rt18T0UMFkRUl9IN0tC0voaChRA14ys63mS78hGRpLRY5JwCyvLVLUs1BgsiqsswvqKBaX2GAZzVwSJjP3BmPZC2Q84G0foBCcNsX04idxfQQo57AoCL+9QtSzXOCiEiY0KYDhYtarVYpO8GPr/ReLvyxBHc64PIXtr0kwOpL+4F2o1UuzRssSCia+ScBkpyAK1vw9PXwhPl7qPlRcDyyTJUeNfaqtyZ1wYgcjat+8jjpcPqlqMaWyyIyFjKBnmMHQBofeq/Rustw0XOKaAwEwhqDTy+DSgvBgovAzF97FdeInenrPFRkKFuOaqxxYKIjJ2uHnzZ4YbGr1PGWQDAhI/lMskhbYA2fTkbhMiegqq3UC+8pG45qjFYEFENXRFwbos872Bicau46mWVBzwCtLvetuUiooYFRctjYSag16tbFrArhIhqO7sRqCqXGzO16ND4tQMekjM/orrYo2RE1JCgaAAaQF8BlFwBAiNVLQ5bLIiohqEbxIzFrTw8gZZd2e1BpDZPLyCgOkwUZqpbFjBYEJFCCOD0Onne0cT4CiJyLLW7Q1TGYEFE0qVD8kPJy79m/AQROYfg6gGcDjAzhMGCiKRTv8lj4kjAy1fdshCRZYJayaMDzAxhsCAiuf7E/iXynN0gRM7HECzYYkFEjuD3N+WSwMFtuGomkTMKZosFETmKC3uAHfPk+S3vAz5BjV9PRI5HabEo4OBNIlJTVSXww5OA0AM97mY3CJGzYlcIETmEzANA9nHAJwS46V9ql4aImkqZFVJyBajUqVoUBgsid5Z7Th5bdgP8w1UtChE1g18Y4Fm9aaDK4ywYLIjcWd55eQyLU7ccRNQ8Gk2tRbIYLIhILXlp8hjKYEHk9JTuEJXHWTBYELmz3OoWi9C26paDiJpPabFQeWYIgwWRO1NaLNgVQuT8gpQWCwYLIlKDXg/kp8tztlgQOT8H2YiMwYLIXRVmAlXlgIe25i8dInJehjEWHLxJRGpQukGCYwBPrbplIaLmM6y+ycGbRKQGTjUlci21u0KEUK0YDBZE7opTTYlci9IVUlEC6ApUKwbbP4nclWGqKYMFkUvw8gMCowGtD1CaB/iGqFIMBgsid8WuECLX88IJuQqnitgVQuSu8rg4FpHLUTlUAM0MFnPmzIFGo8Gzzz5rpeKQs6nSC3y79wIe/GI3bp+/DTd9sBkfrT+tdrHIlKpKIP+iPGdXCBFZUZO7Qnbv3o3PPvsMPXr0sGZ5yIlsOpWNWT8ew+msIqPbT1wqxHUdI9ErNlSdgpFpBRcBUSV3QwxsqXZpiMiFNKnFoqioCJMmTcKCBQsQFhZm7TKREzh1uRAPfLEbp7OKEOLnhRfGdMQn9/XBTd3kdKfX1hyFUHG6E5lg6AaJBTzYI0pE1tOkT5QnnngCf/rTnzB69GiT1+p0OhQUFBh9kfP7z/rTqNILXNehBTbPGImnRnXATUmt8Nr4bvD39sT+tDz8cEDdRVqoEZxqSkQ2YnGwWLZsGfbt24c5c+aYdf2cOXMQEhJi+IqNjbW4kORYTl0uxE+H5Vr0L4/tghA/L8N9LYN98cTI9gCAOT8fR06Rrs7jL+aV4tu9F3Axr9Q+Baa6uKspEdmIRcEiPT0dzzzzDL788kv4+vqa9ZiXX34Z+fn5hq/09PQmFZQcx3/Wn4YQwE3dotG1dXCd+x8cmoDYcD9cLtBhzHubsOZghlG3yIbjl/HiioOY8e1Bexabass9J4+cakpEVmZRsNi7dy+ysrLQt29faLVaaLVabNq0Cf/5z3+g1WpRVVVV5zE+Pj4IDg42+iLnVbu14pnRHeq9xtfLE/+d2h+do4OQW1KBp5bux4e1ZopsP3sFAJCcGGH7AlP9so/LY4tO6paDiFyORcFi1KhROHz4MA4cOGD46tevHyZNmoQDBw7A09PTVuUkB1BaXoXnlx8wtFZ0adVwSOzYMgirnxyKR4e3AwAs2nYOVXoBvV5gx9mrAIBBDBbq0FcBOdVBL6qzumUhIpdj0XTToKAgJCUlGd0WEBCAiIiIOreTaxFCYMZ3h3DkYgHCA7zxyp+6mHyMt9YDL9zQEV/tPI/ckgocvJCHAG8trhaXw8/LEz3ahNq+4FRX7jmgsgzQ+gGh8WqXhohcDOeZkVk+3XwWaw5mQOuhwbxJfRAb7m/W47w8PTCsYyQAYOOJLGxPyQEA9IsPg7eWbz9VZFV3g0R25FRTIrK6Zu8VsnHjRisUgxyZEAJzN5wBAMwc19XiLoyRnaLw06FMbDiZhTahMpCwG0RFyviKSNOtTkREluImZGRSVqEORbpKeHpocHd/y6cnjugkWyyOXCxAanYxAAYLVWWdkEeOryAiG2A7KJmUmiPDQEyoX5O6L1oE+qBnG7l9b3F5Ffy9PdGjjTrb+RKA7OpgwRYLIrIBBgsy6fwVGSziWwQ0+TlGdo4ynPeLD4eXJ996qqiqBHJOyXO2WBCRDfDTnUxKzSkBAMRHmDdgsz4jO9UEC65foaKrZ4GqcsDLHwjhqptEZH0MFmSSocUiouktFt1jQhAdLFdrva5DC6uUi5rAMHCzE2eEEJFNcPAmmaSMsYhv0fQWCw8PDRbe3x8Xc0uRFMPxFarJ4vgKIrItBgtqlBAC56/IrpC4ZrRYAECXVsGNrtZJdqC0WHB8BRHZCNtCqVFZhTqUVlTBQwPEhjW9xYIcBFssiMjGGCyoUeeUqaZhTZtqSg6kqgK4Ihc6Y4sFEdkKf1NQo5RukOYM3CQHkZcG6CvkHiEhsWqXhohcFIMFNSrVCjNCyEHknZfH0LaARqNuWYjIZTFYuLmz2UVYsuM8qvSi3vutsTgWOYjc6mARFqduOYjIpXFWiBvLL6nApP/bicz8MgghMCU5vs411lgcixyEocWCwYKIbIctFm7sn6uPIDO/DACwePt5CGHcaiGnmrLFwmWwxYKI7IDBwk2tOZiBHw5kwEMD+Hp54ExWEbanXAEA5BTpcDA9D9mFOpSUc6qpy2CLBRHZAbtC3FBeSTn+vuoIAODJke2RW1KBJTvOY9H2c2gV6oe7Pt2O7EIdYsP9AHCqqcvIS5NHtlgQkQ0xWLih345eRn5pBdpFBuCpUR1wLqcYS3acx7pjl3HoQj6yC3UAgPSrpQA4I8QllBcDxdnynC0WRGRD/DPUDf18JBMAMKFXDLw8PdChZRCSEyOgF0BmfhkSWwTg12eH4YmR7RAb7ofb+sSoXGJqNqW1wjcE8AtVtShE5NrYYuFmCsoqsPWMHEtxU1K04fYHhiZg+9kriAn1w5fTB6J1qB/+Et0Zf7mRKzS6hNxaa1gQEdkQg4Wb+f1EFsqr9EiMDED7qEDD7WO6tsSyhwehY8sghAd4q1hCsgkO3CQiO2GwcDO/Hr0EALipWzQ016y+OCgxQo0ikT0YpprGq1oMInJ9HGPhRsoqqvD7CTmAr3Y3CLkBtlgQkZ0wWLiRzaeyUVpRhZhQP3SPCVG7OGRPeVwci4jsg8HCjfxwMAMAcGM93SDk4nKrZ4WwxYKIbIzBwk1k5pfilyNyfAWnj7qZ0lxAly/POSuEiGyMwcJNLN4udzAdmBCOJHaDuBdl4GZAFODNpdmJyLYYLNxAaXkVvt4pm8IfHJqgcmnI7ji+gojsiMHCDXy//wLySyvQNtwfo7q0VLs4ZG9cHIuI7IjBwsUJIbBw6zkAwLTB8fD04KBNt3Nxjzy26KRuOYjILTBYuLjzV0pwJqsI3loP3NmvjdrFIXurqgRSNsrzdterWhQicg8MFi7ueGYBAKBTyyAE+XqpXBqyu4t75IwQvzAgpo/apSEiN8Bg4eKUYNGlVZDKJSFVnPmfPLa7HvDwVLcsROQWGCxc3LHMQgBAl1bBKpeEVKEEi/aj1S0HEbkNBgsXV9NiwWDhdoqygYz98rzdKHXLQkRug8HCheWXVuBiXikAoEs0g4XbSdkgj9E9gCBOMyYi+2CwcGEnL8lukNYhvgjx58BNt8NuECJSAYOFC2M3iBvLvwCc+lWeM1gQkR0xWLgwBgs3VVEGfDNZTjON7gHEDlS7RETkRhgsXBiDhRsSAlj7ApCxT65dcfeXgKdW7VIRkRthsHBRVXqBk5eVqaZcw8JtnF4H7P8S0HgAd3zOjceIyO4YLFxUak4xyir08PPyRFxEgNrFIXs594c89prEJbyJSBUMFi7KsJR3dBA3HnMnl4/IY0xfdctBRG6LwcIFFZZV4P/+OAsA6Nqa4yvcyuWj8hjdXd1yEJHbYrBwMcW6SkxbuBsHL+Qj1N8LDwxJULtIZC9F2UDRZQAaIKqL2qUhIjfFYOFChBB4/Kt92Hs+F8G+Wnz54EC0jwpUu1hkL0o3SHgi4M1xNUSkDgYLF7I/PQ+bTmXDW+uBxQ8ORFJMiNpFIntSukFadlO3HETk1iwKFvPnz0ePHj0QHByM4OBgJCcn4+eff7ZV2chCi7edAwD8uWdr9IoNVbUspAJDsEhStxxE5NYsChZt2rTBv/71L+zZswd79uzB9ddfj/Hjx+Po0aO2Kh+ZKbtQh7WHLwEApiRz7QK3dPmwPEYzWBCReixakm/cuHFG38+ePRvz58/Hjh070K0bm1/V9M3uNJRX6dErNhQ92oSqXRyyt6oKIPukPGdXCBGpqMlr/VZVVWHFihUoLi5GcnJyg9fpdDrodDrD9wUFBU19SWpAZZUeX+5IAwBMHczWCrd05QxQVQ54BwGhfA8QkXosHrx5+PBhBAYGwsfHB48++ihWrlyJrl27Nnj9nDlzEBISYviKjY1tVoGprvUnsnCpoAwRAd64uXsrtYtDarhUPSOkZTdAwwXRiEg9FgeLTp064cCBA9ixYwcee+wxTJ06FceOHWvw+pdffhn5+fmGr/T09GYVmOr69YgcW3Fr7xj4aD1VLg2p4nKtYEFEpCKLu0K8vb3Rvn17AEC/fv2we/dufPjhh/j000/rvd7Hxwc+Pj7NKyU1qLJKjw0nswAAY7q2VLk0pBrDipscuElE6mr2OhZCCKMxFGRf+9LykFdSgRA/L/SNC1O7OKQGXSGQtl2et+qpblmIyO1Z1GLxt7/9DWPHjkVsbCwKCwuxbNkybNy4Eb/88outykcm/O/4ZQDA9Z2joPXkemdu6dA3QHkR0KIj0LqP2qUhIjdnUbC4fPkyJk+ejMzMTISEhKBHjx745ZdfMGbMGFuVj0xQgsWoLlEql4QsUngZ+PE5oP8DQPvRTX8eIYDdn8vzfg9w4CYRqc6iYPHf//7XVuWgJjibXYSz2cXQemgwrGOk2sUhS5xYA5z8CRBVzQsW6buArKOA1g/oOdF65SMiaiK2nTux9cfloM1BiREI9vVSuTRkkaJsedQVNe959lS3ViTdDvhxjA0RqY/BwomxG8SJFVcHi4qSpj9HyVXg6Ep53u+B5peJiMgKGCycVFZhGXafuwqA00ydkjWCxbFVQJUOiO4OxHDQJhE5BgYLJ/XLkUvQC6BXbCjahPmrXRyylBIsypsRLI6vkcdut3HQJhE5DAYLJ/XjoUwAwC09uIS3U2pui0VpHpC6WZ53GdfopURE9sRg4YQuF9R0g3BvECfV3GBx6ldAXwlEdgZadLBeuYiImonBwgmtPZwJIYC+cWFoHeqndnHIUpXlQFm+PK8oAfR6y5/jRHU3SOdbrFcuIiIrYLBwQj9Vd4P8ia0Vzqkkx/j7ylLLHl9eApxZL8/ZDUJEDobBwslk5pdiz/lcaDTsBnFaSjeIosLCYJGyQbZ0hLTl3iBE5HAYLJzMjrNXAAA924QiOsRX5dJQk1wbLMqLLXv8iR/lscstnA1CRA6HwcLJHLlYAEBOMyUnVXRti4UFAzj1euD0b/K8083WKxMRkZUwWDiZIxfloL+kmBCVS0JNVqcrxIJgkbEfKLkC+AQDbQdZt1xERFbAYOFE9HqBYxmyxSIpJljl0lCT1ekKsSBYKK0ViSMAT+4PQ0SOh8HCiaRdLUGhrhI+Wg+0jwxUuzjUVMXXzAqxpMVCCRYdbrBeeYiIrIjBwokcyZDdIJ1bBUPryX86p9XUrpCibNkVAjRvq3UiIhvibycnogzcTGrNbhCnpgQLTfV/P3O7QlLWAxBAdA8gmFONicgxMVg4kaPVLRbdWnPgplNTukKC28ijuS0W7AYhIifAYOEkhBC1ZoSwxcJpCQEUZ8nz0LbyaM46FlWVNattdhhjm7IREVkBg4WTyMgvQ25JBbQeGnRsGaR2caipdAVAVbk8D4uTR3NW3sw6CpTlAT4hQEw/mxWPiKi5GCychNJa0aFlEHy9PFUuDTWZ0g3iHQj4h8vzCjNaLLJPymPLboCn1jZlIyKyAgYLJ3FU6QbhwE3npgzcDIgEvPzluTmDN5VgEdnRNuUiIrISBgsnsT89DwDQjcHCudUXLMzpCslRgkVn25SLiMhKGCycQF5JObanyM3HhnZooXJpqFlqBwvvAHluSVdIC7ZYEJFjY7BwAr8cuYRKvUDn6CC0j+LATaemjLEIaAF4+clzU10hVRXA1bPyPLKT7cpGRGQFDBZO4MdDmQCAcT1bq1wSqqOqwrJtz5vSFXL1LKCvlAM+g2OaVk4iIjthsHBwOUU6bEuRf+Xe0oOrLTqUlN+BdzoC8wcDukLzHlNUvYaFJV0h2SfksUVHQKNpWlmJiOyE89Yc3M9HLkEvgB5tQhAXEaB2cdzbsR+Ac1uAqC5AaR6wYRYgqoDSq8D+r4BBj5p+jqZ0hWSfkkd2gxCRE2CwcHBrDmYAYGuF6sryge8eAqp0xrdHdQWyjgE75gEDHgI8TKwxkndeHoNbA54+8txUV0gOB24SkfNgV4gDyy7UYfe5qwCAP/Xg+ApVHV8jQ0VQK6DdKBkobvoXMH094BcuA8OJHxt/jpKrQH66PG/ZDfBWxliY6grhVFMich5ssXBghy7kQQigY8tAxIT6qV0c93Z4hTz2fxAY9hfj+/o/CGx+G9g2F+g6vuHnuHRIHsMSAN8QoDRXft9YV4heD+SclufsCiEiJ8AWCwd24pIcENi1FRfFUlXhJSB1szzvfmfd+/s/BHh6Axd2Aem7Gn6ezIPy2KqHPHpVj5mpLJUBoj75afJ+T28gNK5p5ScisiMGCwd2LLMAANCZwUJdR74DhB5oMwAIi697f1BLoPtd8nz73IafxxAsesqj0hUCyPBQH2XgZkR77hFCRE6BwcKBnagOFl0YLNSldIP0uKvha5KfkMfja4CrqfVfk1ndFRJdHSy0tbq3GuoOMSzlzW4QInIODBYOqqyiCqk5clBfl2iutqmanDNAxn5A4wl0ndDwdS27ykGdQg+seaZu14auCLhyRp4rXSEeHjXhoqEBnFnH5bEFgwUROQe2rTbD8j3p+HxLKvy8PdE61A+tQ3zRKsQPnaKDMLhdBDTNWMzo1OVC6AUQEeCNyCAfK5aaLJK6SR7jhwKBkY1fe9O/gM+Gy8esmGI8JqI4B4CQs0oCo2pu9/aX3SANTTm9sFseW/du8o9ARGRPDBZNIITARxvO4L11pwy37U/LM7pm4bT+GNk5Ck113DC+IqhZAYWaSZmREd3d9LWRHYGb3wF+eFx2idQnpq/x917+AK7U3xVSchXIqX6PtelvdpGJiNTEYNEEc34+gc82y02hHhmWiF6xocjIL0NmXil+O3YZaVdLcDanGCOb8RrHM+WMkC7RHF+hKuUXu7mLU/W61/hxtXl619yv8GpkLYuLe+UxPBEIiDDv9YmIVMZgYaHsQp0hVLw6riumDUkwur+8So/F288jr6S8Wa9znDNCHIPSYmFusNBogN6TzH9+70Y2IlO6QdoMMP/5iIhUxsGbFtp7Xi5q1Dk6qE6oAIBQf28AQG4zgoUQwhAsurTiwE3VlJfUrJTZooNtXkNZy6K+HVKVNTFi2Q1CRM6DwcJCe8/LJbb7xIXVe3+YvxcAILekosmvkZlfhoKySmg9NGgfFdjk56FmupoCQAB+YYC/jboilI3IKq4ZY6HX13SFsMWCiJwIg4WF9lS3WPRrIFiEVgeL/GYEC6W1ol1kIHy0Jja1ItupPb7CVgNola6QawdvZp8AdAWyRSOqq21em4jIBhgsLFBWUYUjF/MBAP3iwuu9xhpdIcpS3p3ZDaIuw/gKG3WDADVdIde2WFyo7gaJ6cMVN4nIqTBYWODQhXxUVAm0CPRBbHj9m4KFVQeLvGa0WBxIzwPAPUJUpwSLCFsGiwa6QtKrB27GshuEiJwLg4UF9tbqBmlobYmaMRZNa7Go0gvsPHsFADAwkVMMVWXpVNOmMHSFXDN4U2mx4PgKInIyDBYWUAZu9ouvf3wFUNMVUlJeBV1llcWvcSyjAAVllQj00SKpNVssVKPX1yzBbctgYegKqTXdlAtjEZETY7AwkxDC0GLRt4GBmwAQ5KOFR3VjRlMGcG5LyQEADEwIh9aT/zyqKcyQ3RMeXkCYDbcrr68rJG27PLboxIWxiMjpWPSba86cOejfvz+CgoIQFRWFCRMm4OTJk7Yqm0NJyS5GbkkFfLQe6NY6pMHrPDw0tQZwWh4stld3gyS34y8UVSktBuEJgKeX7V7Hu551LM5tkcf4IbZ7XSIiG7EoWGzatAlPPPEEduzYgXXr1qGyshI33HADiosb2JnRheyrbq3o2SYU3trGqy20ieMsKqr02JUqu1sYLFRm6YqbTeVVz8qbSrCIY7AgIudj0Ty2X375xej7hQsXIioqCnv37sWwYcOsWjBHsz9dBovecaEmr5UzQ4rNWtb71OVCvLbmKB4cmoAQP2+UlFch1N+Le4SozTBw04YzQoC6XSGlecClw/I8fqhtX5uIyAaaNUE+P1+u6RAeXv+aDgCg0+mg0+kM3xcUFDTnJVWj7F7aOzbU5LWhfrLFwpwppz8cuIitZ65gd2ouruvQAgCQnBgBDw/uaKoqe0w1Bep2haTtACCAiPZAULRtX5uIyAaaPDpQCIHnn38eQ4cORVJSUoPXzZkzByEhIYav2NjYpr6kakrKK3Hqsly0qldswwM3FZaMsbhaLFs1yqv0WH8iCwC7QRxC0WV5DGlj29e5tivk3B/yyG4QInJSTQ4WTz75JA4dOoSlS5c2et3LL7+M/Px8w1d6enpTX1I1hy/kQy+A6GBfRIf4mrxeWcvCnK6Q3GIZPvy9a5buHsxgoT6dDJLwtXGXlCFYVHeFnN8qj+wGISIn1aSukKeeegqrV6/G5s2b0aZN43/R+fj4wMfHp0mFcxTKSpi9zOgGAYCwAPOX9b5afc0/bumKX49eQqCPFu0iufGY6pRg4WPjYFF7gayyAiDzoPyeLRZE5KQsChZCCDz11FNYuXIlNm7ciISEutuGuyJDsGgbatb1IX7m73CqtGq0DffHF/dzlUWHoNfXChY23q+ldotF2g5A6IGwBCAkxravS0RkIxYFiyeeeAJff/01fvjhBwQFBeHSpUsAgJCQEPj51b93his4WB0serYJNet6Zb8QcxbIulrdFaJMUSUHUFEMQMhzewWLyjJg+0fynN0gROTELBpjMX/+fOTn52PEiBFo1aqV4eubb76xVflUl1VQhoz8MnhogB5tGl4YqzZz9wsRQhhaLMKru0/IASitFR5aQGt6TE2zKF0hAJC6WQaNIc/a9jWJiGzI4q4Qd7O/urWiY8sgBPiYV13mzgop1FWiUi/rVGnlIAdQuxukgc3mrEZ7TUvf2LeAFu1t+5pERDbEzShMsHTgJgCEBdTMCmksjOVVd4P4eXnC18uzwevIzuw1vgIAPDxqukO6/BnoPdn2r0lEZEMMFiYcqF4Yy5JgEeonWx8q9QJFusoGr1NmhIRxfIVj0VUv4mbrGSGKPlOAuKHAuA9t30JCRGRjzVp50x0cvyR/yXQ3c3wFAPh5e8JH6wFdpR55JRUI8q0/OChjMMI4vsKx2LPFApDdH0RELoItFo0oq6gyLMsdE2rZrBdlzERjy3rnFpcbXUsOQlckj/YKFkRELoTBohE5RXKPE2+th2FtCnOZs8OpMriTLRYOxt4tFkRELoTBohFZhTJYRAb6QGNh37dZwaKYYywcEoMFEVGTMVg0IqugOlgEWb4kuVldISXsCnFIhsGbDBZERJZisGhEdnVXSFQTgkWoRcGCLRYOxV77hBARuSAGi0ZkF5QBAKKCm9JiYU5XCMdYOCR2hRARNRmDRSNqxlhYvqxzqBlbp7MrxEExWBARNRmDRSOyq4NFU1oszFnW+2ox9wlxSEqw8Ob29URElmKwaITSYtGUMRY1gzfrb7GQG5BxZ1OHxBYLIqImY7BoRFahHGPRtFkh1V0hpfW3WBSXV6G8Sg+ALRYOx95LehMRuRAGiwbo9QI5RbK1ISrI8jEWyoJa+Q0EC2UNCx+tB/y4AZljYYsFEVGTMVg04GpJOar0AhoNEBFoeYtCiH9NsNDr6+5wWnvgpqWLb5ENCcFgQUTUDAwWDVAWxwr394aXp+XVpLRYCAEUltXd4ZTLeTuoSh2gr25lYrAgIrIYg0UDlMWxmjK+AgB8tJ6GLo76ukO4nLeDUlorAM4KISJqAgaLBmQVNH3gpsKwlkVp3Zkh3DLdQSkDN72DAA/+9yAishQ/ORtQM9XU8oGbCqU7pL5lvdli4aA4voKIqFkYLBrQnMWxFKGNTDlVxliEc9VNx8JgQUTULAwWDciutWV6UzU25fRqdVdIKIOFY2GwICJqFgaLBiiLYzWrxcJPhob8elbfVFbk5OJYDobBgoioWRgsGpBthTEWNRuR1dNiUczlvB2SYdVNBgsioqZgsGiAYWfTZswKCW6kK4QtFg7K0GLB5byJiJqCwaIeRbpKlJRXAWjaBmSKhgZvllVUWWXWCdkAu0KIiJqFwaIeSjdIgLcnAny0TX6emjEWxsHi5KVCVOkFwgO80bIZYzjIBhgsiIiahcGiHtZYHAtoeIGsIxn5AIBurYO5T4ijYbAgImoWBot6ZOZXzwhpZjdFQ9NNj1yUAwSTYkKa9fxkAwwWRETNwmBRj61ncgAAXVs3bwBfQytvHqvVYkEOhrNCiIiahcHiGnq9wO8nswAAo7u0bNZzKV0huko9yirkYNCKKj2OX5J/FSe1ZouFw2GLBRFRszBYXOPAhTzkFJUjyEeLAQnhzXquQB8tPD3kGAql1eJMVhHKK/UI8tGibbh/s8tLVsZgQUTULAwW11h//DIAYFinSHhrm1c9Go2mzjiLoxmyqb1r62B4eHDgpsNhsCAiahYGi2usP650g0RZ5flCDeMs5MyQIxeV8RXsBnFIDBZERM3CYFHLhdwSnLhUCA8NMKKjdYJFyDWLZB2tHriZFMOBmw6nqgKoLJXnXHmTiKhJGCxqUVor+sWFI8xKS23X7grR6wWOZXCqqcNSWisAwDtQvXIQETkxBotaNpyQwWKUlbpBgJqukPySCpy7Uozi8ir4aD2Q2CLAaq9BVqIEC60voOUeLkRETcFgUYsysHJQYoTVnjPUX/6CyistNzx/51bB0Hqy6h0Ox1cQETUbf7tVyy+pQE6R3COkXZT1msGDay2SdSFX9t+ztcJBlcnxLwwWRERN1/QdtlzMmewiAECrEF8ENmPjsWuF1hpj4eUplwqPDuGOpg4pP10eg2PULQcRkRNjsKiWUh0s2kVad9CesvpmfmkFKqsEACA6mMHCIV1NlceweFWLQUTkzBgsqtUEC+t2Uxh2OC2pQGFZJQCgJYOFY8qtDhbhieqWg4jIiTFYVEvJksGivRXHVwC1NiIrLa9psWBXiGNSWizCE9QtBxGRE2OwqJaSXQzA+l0hIX5yVkhucQVKqzciY1eIg1JaLMIYLIiImorBAoCusgppV0sAWHdGCFDTFVKkk90gHhqgRSDXSHA4ukKgOFues8WCiKjJON0UQNqVElTpBQJ9tIgK8rHqcytdIYrIIB+uYeGIlG4Qv3DAl6uiEhE1FX/DQW5lDsjWCo3GujuOenl6IMDb0/A9u0EcVC7HVxARWYPFwWLz5s0YN24cWrduDY1Gg1WrVtmgWPZlqxkhCmX1TYAzQhzWVc4IISKyBouDRXFxMXr27Im5c+faojyqsNXATUXt7hDOCHFQHLhJRGQVFg/eHDt2LMaOHWuLsqjGVotjKRgsnACnmhIRWYXNZ4XodDrodDrD9wUFBbZ+SYsIIWy2hoVCmRkCcIyFw2KLBRGRVdh88OacOXMQEhJi+IqNjbX1S1rkUkEZisuroPXQIC7C3yavwWDh4CrLgfwL8pwtFkREzWLzYPHyyy8jPz/f8JWenm7rl7RIao4cXxEb7g8vG00DDa7VFdKSXSGOJy8NEHrAyx8IbKl2aYiInJrNu0J8fHzg42PdtSGsKatAdtO0suEv/FC/mlkhbLFwQLW7Qaw83ZiIyN24/ToWlwvkVua2nAaqdIUE+WgRYMUt2clKOHCTiMhqLP4tV1RUhDNnzhi+T01NxYEDBxAeHo62bdtatXD2kFUoWyysveJmbaHVXSHsBnFQudwunYjIWiwOFnv27MHIkSMN3z///PMAgKlTp+KLL76wWsHsRWmxiLJhi8WgxAj0bBOCCb1jbPYa1ERCAOe3yvOIduqWhYjIBVgcLEaMGAEhhC3KogpljIUtWyzCArzxw5NDbfb81AwnfwYyD8qBm53HqV0aIiKn5/ZjLLIKbT/GghyUXg/8/qY8H/AwEBipbnmIiFyAW48kFELgsh1aLMiBCFH9pQdOrAEuHwa8g4Ahz6hdMiIil+DWwaJIV4nSiioAQFQwg4XLeL87UHRJBghUhwjlvD6DHgP8w+1ZQiIil+XWwUJprQjy1cLf262rwrVU6YCqcvOuDW0LJD9h2/IQEbkRt/5tqoyvYDeIi3l4o2ylgAbQeMhFrzQe1d8r55DnPsGAh6eKhSUici3uHSyqWyw4cNPFBLdWuwRERG7LrWeFsMWCiIjIutw6WFxmiwUREZFVuXmwkC0WkWyxICIisgq3DhbKPiFssSAiIrIO9w4WBRxjQUREZE1uGyyEEGyxICIisjK3DRZFukqUlHPVTSIiImty22ChtFYE+XDVTSIiImtx22ChzAhhawUREZH1uG2wyC5UdjXl+AoiIiJrcdtgobRYtGSLBRERkdW4cbCobrHgjBAiIiKrcdtgcTa7CADXsCAiIrImtwwWv5/Mwu8ns6HRAIMSI9QuDhERkctwu2CRX1qBl787DAC4f3ACkmJCVC4RERGR63CZBRzmbjiNIl2VyesOXcjDpYIyxEf44y83drJDyYiIiNyHywSLxdvPGxa9MkWjAd6+syf8vD1tXCoiIiL34jLB4p4BbVGsqzTr2r5xYegfH27jEhEREbkflwkWz43pqHYRiIiI3J7bDd4kIiIi22GwICIiIqthsCAiIiKrYbAgIiIiq2GwICIiIqthsCAiIiKrYbAgIiIiq2GwICIiIqthsCAiIiKrYbAgIiIiq2GwICIiIqux+14hQggAQEFBgb1fmoiIiJpI+b2t/B5viN2DRWFhIQAgNjbW3i9NREREzVRYWIiQkJAG79cIU9HDyvR6PTIyMhAUFASNRmPPl26SgoICxMbGIj09HcHBwWoXxyGxjszDejKNdWQa68g01pF5LK0nIQQKCwvRunVreHg0PJLC7i0WHh4eaNOmjb1fttmCg4P5BjWBdWQe1pNprCPTWEemsY7MY0k9NdZSoeDgTSIiIrIaBgsiIiKyGgYLE3x8fDBz5kz4+PioXRSHxToyD+vJNNaRaawj01hH5rFVPdl98CYRERG5LrZYEBERkdUwWBAREZHVMFgQERGR1TBYEBERkdUwWBAREZHVMFjA9IYqJLGeTGMdmYf11DjWj3lYT6apUUduHSzOnj0LAE6xZ4la5s6di3nz5gFgPTWG7yXT+F4yje8j0/g+Mo+a7yW3DBbr1q1Dt27dcMstt2DUqFH49NNP1S6Sw1m7di3i4uLwxhtvoFevXmoXx2HxvWQa30um8X1kGt9H5nGI95JwM5s2bRJt27YVb775plizZo14+umnhUajEQsWLBBFRUVqF0912dnZYuzYscLb21v861//Urs4Do3vpcbxvWQevo8ax/eR+RzlveQ2wUKv1wshhHj99ddF//79RXFxseG+GTNmiE6dOolVq1apVTyH8fvvvwuNRiPee+89w21r1qwRO3fuFOfPnxdC1NSlu+J7yTx8LzWO7yPz8H1kmqO9l9xuSe+77roLFRUVWLlyJcrLy+Ht7Q29Xo/BgwejY8eOmDNnDmJiYtQupl3l5+cbbYV7zz33IC8vDwMHDsSXX36JiIgInD17FqGhofj4449xww03qFhax8H3kml8L5nG91Fd/ExqGod5L9ktwtjZ9u3bxZNPPineeecdsX79esPtCxYsEMHBwYZEp9PphBBCLFu2TLRs2VJs3LhRlfKq4cyZM6J///7iueeeE3l5eYbbT548KcLCwkRSUpL47LPPxLlz58SWLVvEQw89JKKiosTZs2dVLLX9bdmyRTzxxBPi008/FTt37jTc/tlnn/G9VG337t3i3LlzQgj511NlZaUQgu+l2viZZBo/k8zj6J9JLhcsrly5Iu644w4REhIiJk6cKAYNGiS8vb3F8ePHhRBCbNu2TSQlJYlXXnlFCCEMH4BCCBEfHy9ee+01IYR7NK199dVXQqPRiJiYGLF582YhRM3PvXDhQrF06VKj68vKykTLli3FW2+9ZXStK9Lr9aK8vFy8+OKLwt/fX9x6662iV69ews/PTyxfvlyUl5eLAwcOiC5durj1e6mgoEA89thjQqPRiKFDhxrdV1VVJYTge4mfSebjZ1LDnOkzyaVmhVy6dAnTpk2DXq/Hnj17sHTpUqxfvx7x8fFYvHgxACApKQnjxo3D0qVLcfLkSXh6eqKyshIA0K1bN6SkpABw7WlMorr3Kzc3Fx999BFiYmLw73//G1euXDH83FOnTsXEiRONHqfRaBAfH48TJ04YvndVGo0GGRkZWL16Nb777jt8//332L9/P+677z68+eab+OGHH9CzZ0+MHz/ebd9LRUVFeP/993HixAnMmjULe/fuxXfffQcA0Ov1fC+Bn0nm4meSac70meRSwSI6Ohq33nor3njjDbRv3x4AoNVq0b59e9x5550QQiAoKAi333474uLiMHXqVJSUlECr1eLq1as4f/48br75ZpV/CttT3lRHjx5FRUUFFixYgJ9++gnr1q0zukav1xs97tixY8jPz8fdd99t1/KqZceOHSgtLUVCQoLhtnfeeQetWrXC4sWLkZmZifvvvx+xsbFu+V4KDAxEfHw8nnnmGTz11FOYOnUqXnjhBQCAh4eH4X1W34eYu7yX+JlkHn4mmcdpPpNs2h5iR0rTTkVFheG2/fv3i+7duwt/f38xaNAgMWHCBJGamiqEEOLIkSMiNjZWdOzYUUyaNEl06NBBDBw40DDK2B1MmDBBrFy5UgghxB133CEGDBggfvzxR0NzmRBC5Ofni9TUVLFq1SrRpUsXcffdd4ucnByVSmxfGzduFJ6enuLixYtCiJr+yhUrVohevXqJ+fPnCyHke6lt27Zu9V5S/r8pdSKEEEePHhUtWrQQs2bNEkLUdIUoCgoK3Oq9xM8ky/EzqXHO8pnk9MGiob6iCxcuiClTpojp06eL7du3i99++020a9dO3HbbbSI7O1sIIcSJEyfE/PnzxfTp08UHH3xgz2Lb1bV1pHzQjRs3ztBnmZKSIjw9PYVGoxHPPPOMqKioEIWFhWL58uViwIABIjIyUsyZM8fuZbena+upuLhYtGvXTjz//PNCCCHKy8sN940ZM0ZMmjTJ8B/75MmTbvleqn1bVVWV+Pe//y38/PzEpUuXjK4pLi52m/cSP5NM42eSeZz1M8npgkVKSkq9t2/YsKHObcqoYuUvp5UrV4rAwECRm5trdJ2rDfgxp46Ki4vF6NGjxdmzZw2/DOLj40VYWJg4dOiQ4brz58+LpUuXGs2LdhW1/9quTaknnU5nqJv09HSjxyxYsEBER0fXeayrvZdM1dG1MjIyRI8ePcS9995ruE1ZmOfChQvi66+/drn3kiV15K6fSebUET+ThCgpKan3dmf7THKaYLFy5UrRq1cv0atXL3HDDTeI5cuXCyFk0l2xYoXQaDRiz5499T5Wr9cLvV4v5syZI7p37y6ysrLsWXS7saSOrl69Kvr27Ss0Go2Ii4sT33zzjRBCiNjYWDF58mSjqV6uZuXKlWLYsGHinnvuEe+9956hKbp2Pe3evVsIIcSpU6dEcnKyGD16tNFzvPLKK2LYsGGiuLjY5X4JCGFeHTX0/+3bb78VWq1WrFu3Trzxxhvi7rvvFmlpaXYsvX00p47c6TPJ3Dpy98+kIUOGiPHjx4u33nrL8Mehs34mOXywqKysFK+++qqIiooSH374oVi+fLmYOHGi6Ny5sygtLRVCyCVfX3rpJXHkyJE6j1f+Mvj1119F3759jVZvcxVNqaOSkhLxj3/8Q8ydO1dcuXLF8FxffPGF6Ny5s6Fp1lUoU7VmzJghoqKixJtvvileeeUVMWDAADFgwABRUFAghBAiKytLvPTSS+Lw4cOGx+3YsUP4+PiIe++9V3z99dfit99+E4mJieLNN99U80eyOkvrqL7/b0IIkZOTI7p37y40Go0IDQ0Vy5Yts+ePYVPWqCNX/0xqah2522eS4t133xVhYWHi9ddfF88++6zo1q2b6Nu3r1N/Jjl8sMjIyBC9e/c2+nBasmSJuO6660wm2NOnT4s5c+aICRMmCF9fXzF79mxbF1cVltaRkmbLysrsVkZHkJqaKnr27GnU/Lpx40YREhIiHnjggUYf+9NPP4mbb75ZdO7cWURHR4u//e1vti6uKppTR0IIcfbsWTFs2DCh1WrFO++8Y8uiqqY5deQun0mW1pG7fiYVFBSIYcOGGY0V2blzp0hKShK33XZbo4915M8kh59uqtPpcP78eVRUVBhu27VrF9q2bYuzZ88iLy8PgPGe88qUpODgYFy4cAExMTFIT0/H3/72N7uW3V4srSNl2paPj0+d5xIuvML7yZMnkZubi+DgYMNtiYmJAICFCxdi/fr1AICqqirD/cr5zTffjJ9++gk//fQTTp06hdmzZ9ux5PbTlDqqPQVw7969SEpKQkZGhmHqqatpTh25y2eSpXXkrp9JXl5eOHHiBFq1amW4rV+/fnj77bfxww8/4NdffwVg/H/MKT6T1M01xk6ePGlIrEpz4cWLF8WDDz4ovLy8xN///neRmJgoIiMjxahRo0SHDh3E0KFDDauLlZWVienTp4uZM2canrOhwTDOyhZ15Ipq15Pys2/atEl07txZfP7554a/kJYuXSruuOMOMWbMGDFkyBDD46+tp2unTroCa9dR7edxFbaoI1f+TLJWHbmiPXv2iA0bNoiUlBTD50laWpoYOXKkeOGFF4z+75SVlYk777xT9OvXz3BbaWmp03wmOUSwyMrKEnfffbcICgoSH330kRDCeCRrXl6e+PHHH8Vdd90lJk6cKK5cuSKKi4tFamqqCAwMFB9//LEQQv5jvPDCC6JDhw4uN2KYdWSe+uqp9n/ABx98UCQkJIjx48eLgQMHioCAALFkyRLx+eefi3bt2olTp04JIeQHpKvWE+vINNaRaawj82RkZIibb75ZREVFiT59+ojIyEjx/vvvG+5/8MEHxejRo8W+ffsMt+n1erF27VrRsmVLw14gFRUVTlNPqgeL1NRUccMNN4jhw4eLMWPGiJtvvtmwmVHtBFdQUCDatGkjfv75ZyFEzajq66+/Xtxxxx1G17ka1pF5GqsnZUpWdna2WLlypXjkkUfEjBkzREZGhhBCjspOSEgw+g/rivXEOjKNdWQa68g8e/fuFQMHDhR33323SElJEenp6eLpp58WQ4YMEZs2bRJCyEXT2rZtK9544w2jetixY4do27at2L59u+E2Z6kn1cdYtGnTBp06dcK//vUvPPLII8jLy8N///tfAICnp6fhuuzsbMTFxeHq1asAZJ9cWloacnJycMcddxiuCwoKsu8PYAesI/M0Vk/e3t4AgBYtWmDChAn4+OOP8dZbb6FVq1YQQmDbtm1o06YNNBqNoU/XFeuJdWQa68g01pF5jh07hp49e2LOnDlITExEmzZt8Pjjj+Ps2bPw9/cHAPTq1QsTJ07E6tWr8dVXXxkeW1JSAg8PD0RGRhpuc5p6Ui/T1Kidwp566ikxdOhQQ0pTVmQrLCwU48aNE3369BGvv/66+OSTT0THjh3F2LFjDcubujLWkXkaq6f6lpjOzs4Wy5YtE4mJiWLhwoX2LKpqWEemsY5MYx01TOmmPnr0aJ0t3fPy8kR8fLzYsmWL4baCggLx8MMPi4iICDFlyhTx5ptvirZt24qHHnrIsGSAM1ElWNQ3wEu5bdu2bWLUqFHioYceMtyn/OLct2+fePTRR8XgwYNF7969xYcffmifAquAdWQeS+tJodPpxG+//Sb69esnwsLCDH3Eroh1ZBrryDTWkXkaGsCshK2NGzeKiIgIw7ocyvVFRUVi4cKFYtKkSWLQoEFOvaS7qi0Wyupi1/5DvPHGG2LgwIHi22+/FULUXZI0OzvboUfEWhPryDzm1lNtGRkZYsmSJUbr7bsy1pFprCPTWEfmqb16Zm2vvvqqGD9+vBCi4eW21V45s7lsGiyWL19u2ACl9lrveXl5YvLkyWLAgAFGa8grvwhTUlLE+PHjxZ133inOnDkjXnjhBbFkyRJbFlU1rCPzWLOeFi9ebPfy2wPryDTWkWmsI/NYWk+KP//5z+Ltt982fL9gwQKxceNGu5TZXmwyePPKlSu488478fTTT0Or1eLbb7/FTTfdhEWLFgEAQkJC0L17dwwePBj5+fmGx3l4yOIkJiZi7Nix2LZtG7p3747PPvsMYWFhtiiqalhH5rFFPYWHh6vys9gK68g01pFprCPzNLWeACA/Px8HDx7E8OHDsWnTJrRr1w4zZ86sd2Ewp2aLtLJixQoxYMAAceHCBcNt48ePFwkJCWLFihVCCNHgPFy9Xi927dolBgwYIPz9/Y3m+7oS1pF5WE+msY5MYx2ZxjoyT3PqafXq1cLX11f07dtXaLVal10UzCbB4tZbbzWsc15YWCiEkBvJaDQaMWrUKMOglYb6kcaNGyduvfVWp5mz2xSsI/OwnkxjHZnGOjKNdWSe5tTT3LlzhUajEQ888IDIz8+3X6HtrNldIZs3b8avv/6KyspKw20dOnTA0aNHAQCBgYEAgBMnTuD6669HWVkZVq5cCUCus5Ceno4ZM2Zg+/bthsd/8803+P77751nzq4JrCPzsJ5MYx2ZxjoyjXVkHmvV09atWwEAQ4cOxenTp/Hf//7XaB8Vl9PURJKdnS2mTJkiNBqN6Nmzp0hNTTXcl5KSIiIjI8Xw4cPFW2+9JZKTk0VCQoJYv3696Nmzp/jHP/5huHbLli2iXbt24rnnnmtWQnJErCPzsJ5MYx2ZxjoyjXVkHmvX07PPPqvCT6GeJgWLiooKMW/ePHHjjTeKZcuWCX9/fzFnzhyjLW+3bNkiHnroIdGnTx/x5JNPGpqHJk+eLG6//Xaj56u9UIirYB2Zh/VkGuvINNaRaawj87Cemq/JLRY7duwQa9asEUII8dprr4nIyEixf//+OtfVnm5z+fJlkZSUJGbNmiWEqDu/19WwjszDejKNdWQa68g01pF5WE/N0+Rgce3AlNatW4uHH37YMHCn9v2lpaWivLxczJs3T/Tu3dtozq8rYx2Zh/VkGuvINNaRaawj87CemqfZs0KUxLZ8+XKh1WrFb7/9ZnT/hQsXxLx580S/fv1EeHi4+Prrr5v7kk6HdWQe1pNprCPTWEemsY7Mw3pqGo0Q1dvLWcHgwYMREBCAr776ClFRUcjOzkZkZCSWLl2KjIwMvPDCC9Z6KafFOjIP68k01pFprCPTWEfmYT2ZzyrBorKyElqtFkePHkXPnj3x3nvvISUlBVu2bMGiRYuQlJRkjbI6NdaReVhPprGOTGMdmcY6Mg/rqQms3QTSv39/odFoRFxcnPjll1+s/fQugXVkHtaTaawj01hHprGOzMN6Mo/VgsWZM2dEUlKS8Pf3F//3f/9nrad1Kawj87CeTGMdmcY6Mo11ZB7Wk2WstgmZp6cnbr/9duTk5ODBBx+01tO6FNaReVhPprGOTGMdmcY6Mg/ryTJWHbxJRERE7s0m26YTERGRe2KwICIiIqthsCAiIiKrYbAgIiIiq2GwICIiIqthsCAiIiKrYbAgIiIiq2GwICIiIqthsCAiq5g3bx4SEhLg6+uLvn374o8//lC7SESkAgYLImq2b775Bs8++yxeeeUV7N+/H9dddx3Gjh2LtLQ0tYtGRHbGJb2JqNkGDhyIPn36YP78+YbbunTpggkTJmDOnDkqloyI7I0tFkTULOXl5di7dy9uuOEGo9tvuOEGbNu2TaVSEZFaGCyIqFlycnJQVVWFli1bGt3esmVLXLp0SaVSEZFaGCyIyCo0Go3R90KIOrcRketjsCCiZmnRogU8PT3rtE5kZWXVacUgItfHYEFEzeLt7Y2+ffti3bp1RrevW7cOgwcPVqlURKQWrdoFICLn9/zzz2Py5Mno168fkpOT8dlnnyEtLQ2PPvqo2kUjIjtjsCCiZrv77rtx5coVvP7668jMzERSUhLWrl2LuLg4tYtGRHbGdSyIiIjIajjGgoiIiKyGwYKIiIishsGCiIiIrIbBgoiIiKyGwYKIiIishsGCiIiIrIbBgoiIiKyGwYKIiIishsGCiIiIrIbBgoiIiKyGwYKIiIishsGCiIiIrOb/AUjIVLwy0p1xAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sampler.to_pandas().plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac77cec5-a4d0-41cb-8bdc-d293485ce417",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Try with pandas 2.0dev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "21847024-33f5-4b33-aca8-1296081ff228",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "%pip install https://anaconda.org/scipy-wheels-nightly/pandas/2.0.0.dev0%2B404.g890d097534/download/pandas-2.0.0.dev0%2B404.g890d097534-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl\n",
    "\n",
    "import IPython\n",
    "IPython.get_ipython().kernel.do_shutdown(restart=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0bf0f8fe-970b-4fb6-88e8-d8427be48cba",
   "metadata": {},
   "outputs": [],
   "source": [
    "import copyreg\n",
    "import contextlib\n",
    "\n",
    "import dask\n",
    "import dask.dataframe as dd  # Trigger copyreg\n",
    "import pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7ebe26e5-f10f-4546-a413-e21e44a86769",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.0.0.dev0+404.g890d097534'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pandas.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fe1c677f-4570-4681-aad7-99e88083c276",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove the dask copyreg\n",
    "\n",
    "@contextlib.contextmanager\n",
    "def disable_copyreg(klass):\n",
    "    disp = copyreg.dispatch_table.pop(klass, None)\n",
    "    try:\n",
    "        yield\n",
    "    finally:\n",
    "        copyreg.dispatch_table[klass] = disp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "72cc02fb-2642-4cb9-8caa-5a136da4e23e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    hgyKNYMNNjZYniPdgnBqhTkUYTsdsCBQFFdOaklVZlGAXt...\n",
       "1    zzddGphWWZtbImSGkUyhcHRkYGyUXlSlGKLKireQqynVir...\n",
       "2    LgYaqaYOVhnldETsnSeGEuWkUZqRDLSENMCLydBulxcIFi...\n",
       "3    pnrvaWgLuaqrIUzrqiVwBImxuRKdYKocwgTfXgkmZWJTVT...\n",
       "4    jqfurqjahQNEhlZEijOmeOauUCNJbnOXLFjfiBFpYmSABU...\n",
       "Name: data, dtype: string"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pickle\n",
    "import random\n",
    "import string\n",
    "\n",
    "# Create ~50 MiB of sample string data\n",
    "s = pandas.Series(\n",
    "    [\n",
    "        \"\".join(random.choices(string.ascii_letters, k=random.randint(100, 1000)))\n",
    "        for _ in range(100_000)\n",
    "    ],\n",
    "    dtype=\"string[pyarrow]\",\n",
    "    name=\"data\",\n",
    ")\n",
    "s.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c372c5df-4adc-44a2-abd8-ab0894130aee",
   "metadata": {},
   "source": [
    "If we disable the copyreg approach with pandas `main`, things seem to work correctly!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e8de7d90-a578-4a10-bdb9-913893e27030",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "52\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    print(len(pickle.dumps(s)) // 1024**2)   # 52 MiB    \n",
    "    print(len(pickle.dumps(s[:10_000])) // 1024**2)   # 5 MiB, good"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ffce15af-30c0-42d0-a634-4658e47c2558",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[13, 13, 13, 13]\n"
     ]
    }
   ],
   "source": [
    "# Now use Dask to partition the data.\n",
    "@dask.delayed\n",
    "def get_partition_serialized_size(df):\n",
    "    return len(pickle.dumps(df)) // 1024**2\n",
    "\n",
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    result, = dask.compute(\n",
    "        [get_partition_serialized_size(p) for p in dd.from_pandas(s, npartitions=4).to_delayed()],\n",
    "        scheduler=\"processes\",\n",
    "    )\n",
    "    print(result)  # [13, 13, 13, 13]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b8c8fc7f-5abf-449c-b250-d56c636a5a23",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pickling using dask\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "# Which takes priority?\n",
    "disp = copyreg.dispatch_table.pop(pandas.arrays.ArrowStringArray, None)\n",
    "try:\n",
    "    def wrap(*args, **kwargs):\n",
    "        print(\"pickling using dask\")\n",
    "        return disp(*args, **kwargs)\n",
    "    copyreg.dispatch_table[pandas.arrays.ArrowStringArray] = wrap\n",
    "    \n",
    "    print(len(pickle.dumps(s[:10_000])) // 1024**2)   # 5 MiB, good\n",
    "\n",
    "finally:\n",
    "    copyreg.dispatch_table[pandas.arrays.ArrowStringArray] = disp\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e030a568-42af-4fb5-afe9-1358361ff493",
   "metadata": {},
   "source": [
    "The `dask` copyreg takes priority over the pandas reducer: as soon as this is released we should probably disable the dask version in favor of the upstream one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c4f4e034-2095-49f7-aa15-732bbcf94720",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Do the dask and pandas approaches get the same answer?\n",
    "\n",
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    a = dd.from_pandas(s, npartitions=4).str.upper().compute(scheduler=\"processes\")\n",
    "b = dd.from_pandas(s, npartitions=4).str.upper().compute(scheduler=\"processes\")\n",
    "\n",
    "dd.assert_eq(a, b)\n",
    "\n",
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    a = dd.from_pandas(s, npartitions=4)[50_000:].str.slice(0,10).compute(scheduler=\"processes\")\n",
    "b = dd.from_pandas(s, npartitions=4)[50_000:].str.slice(0,10).compute(scheduler=\"processes\")\n",
    "\n",
    "dd.assert_eq(a, b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "53691870-53ef-4d61-8ae7-4ae1dbaa79d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15 ms ± 322 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n",
      "9.92 ms ± 361 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "# Which approach is faster?\n",
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    %timeit pickle.dumps(s)\n",
    "%timeit pickle.dumps(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e90db760-9b90-4c6c-9056-b0826dcc3685",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pandas  55275647\n",
      "Dask  55275489\n"
     ]
    }
   ],
   "source": [
    "# Which approach is more space efficient?\n",
    "with disable_copyreg(pandas.arrays.ArrowStringArray):\n",
    "    print(\"Pandas \", len(pickle.dumps(s)))\n",
    "print(\"Dask \", len(pickle.dumps(s)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}