Skip to content

Instantly share code, notes, and snippets.

@gakuba

gakuba/MultiLabelClassification-with-nltk.ipynb

Last active April 1, 2020 16:11
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save gakuba/9e3fd0cf2ce026b3a53ebde5b5f58534 to your computer and use it in GitHub Desktop.
Save gakuba/9e3fd0cf2ce026b3a53ebde5b5f58534 to your computer and use it in GitHub Desktop.
Download ZIP
In this Tutorial, I explain how to do Multi label classification of texts using nltk
Display the source blob
Display the rendered blob
Raw
MultiLabelClassification-with-nltk.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# MULTI LABEL CLASSIFICATION WITH NLTK"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this tutorial, I will show you how to predict tags for a text. In this post, we will build a multi-label model that’s capable of detecting different types of toxicity in a large number of Wikipedia comments which have been labeled by human raters for toxic behavior. The types of toxicity are: \n",
"- toxic\n",
"- severe_toxic\n",
"- obscene\n",
"- threat\n",
"- insult\n",
"- identity_hate\n",
"\n",
"The data set used can be downloaded at [Kaggle](https://www.kaggle.com/jhoward/nb-svm-strong-linear-baseline/data) . there is a disclaimer on the data set that the dataset contains texts that may be considered profane, vulgar, or offensive. this is applicable to this tutorial as well.\n",
"\n",
" I would like to acknowledge the National Research University Higher School of Economics of Moscow for most of my codes borrowed from their [github](https://github.com/hse-aml/natural-language-processing/blob/master/Docker-tutorial.md). I would also like to acknowledge Susan Li for her article from which I borrowed the codes used to do the analysis of the tags. [Susan Li's Article](https://towardsdatascience.com/multi-label-text-classification-with-scikit-learn-30714b7819c5).\n",
"\n",
"To solve this task you will use multilabel classification approach.\n",
"\n",
"### Libraries and file used\n",
"\n",
"In this task I will need the following libraries:\n",
"- [Metrics](https://gist.github.com/gakuba/668b42d853ad9471e2e5675f5601a312) file used to plot the roc_au curves.\n",
"- [Numpy](http://www.numpy.org) — a package for scientific computing.\n",
"- [Pandas](https://pandas.pydata.org) — a library providing high-performance, easy-to-use data structures and data analysis tools for the Python\n",
"- [scikit-learn](http://scikit-learn.org/stable/index.html) — a tool for data mining and data analysis.\n",
"- [NLTK](http://www.nltk.org) — a platform to work with natural language."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's first download the list of stopwords"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
"[nltk_data] Package stopwords is already up-to-date!\n"
]
}
],
"source": [
"import nltk\n",
"nltk.download('stopwords')\n",
"from nltk.corpus import stopwords"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The data set contains text data and corresponding tags. for easy operation on the data, let's first load pandas and numpy that we will to structure our data and do operations on the data. let's also deploy matplotlib that we will use for visualisation"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import pandas as pd\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's create a function and call it read_data that we can use to load data.\n",
"I've already split the data into training data set and test data set that I then saved as Pickle file. you can see the whole process [Here.](https://gist.github.com/gakuba/a1534820442a42512d43d43584e01ca3)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"def read_data(filename):\n",
" data = pd.read_pickle(filename)\n",
" return data"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"train = read_data('data/train.pkl') \n",
"test = read_data('data/test.pkl')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that our data is deployed, let's see some statics about our data."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>toxic</th>\n",
" <th>severe_toxic</th>\n",
" <th>obscene</th>\n",
" <th>threat</th>\n",
" <th>insult</th>\n",
" <th>identity_hate</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>count</th>\n",
" <td>143613.000000</td>\n",
" <td>143613.000000</td>\n",
" <td>143613.000000</td>\n",
" <td>143613.000000</td>\n",
" <td>143613.000000</td>\n",
" <td>143613.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>mean</th>\n",
" <td>0.096189</td>\n",
" <td>0.010076</td>\n",
" <td>0.053011</td>\n",
" <td>0.003071</td>\n",
" <td>0.049341</td>\n",
" <td>0.008760</td>\n",
" </tr>\n",
" <tr>\n",
" <th>std</th>\n",
" <td>0.294851</td>\n",
" <td>0.099871</td>\n",
" <td>0.224055</td>\n",
" <td>0.055329</td>\n",
" <td>0.216580</td>\n",
" <td>0.093183</td>\n",
" </tr>\n",
" <tr>\n",
" <th>min</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>25%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>50%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>75%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>max</th>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" toxic severe_toxic obscene threat \\\n",
"count 143613.000000 143613.000000 143613.000000 143613.000000 \n",
"mean 0.096189 0.010076 0.053011 0.003071 \n",
"std 0.294851 0.099871 0.224055 0.055329 \n",
"min 0.000000 0.000000 0.000000 0.000000 \n",
"25% 0.000000 0.000000 0.000000 0.000000 \n",
"50% 0.000000 0.000000 0.000000 0.000000 \n",
"75% 0.000000 0.000000 0.000000 0.000000 \n",
"max 1.000000 1.000000 1.000000 1.000000 \n",
"\n",
" insult identity_hate \n",
"count 143613.000000 143613.000000 \n",
"mean 0.049341 0.008760 \n",
"std 0.216580 0.093183 \n",
"min 0.000000 0.000000 \n",
"25% 0.000000 0.000000 \n",
"50% 0.000000 0.000000 \n",
"75% 0.000000 0.000000 \n",
"max 1.000000 1.000000 "
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train.describe()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's now print a snapshot of our data to see the structure"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>id</th>\n",
" <th>comment_text</th>\n",
" <th>toxic</th>\n",
" <th>severe_toxic</th>\n",
" <th>obscene</th>\n",
" <th>threat</th>\n",
" <th>insult</th>\n",
" <th>identity_hate</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>34117</th>\n",
" <td>5b02208daa29a40f</td>\n",
" <td>Outrageous!!!!! \\n\\nThis block is outrageous a...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6579</th>\n",
" <td>1190ddc487465bd2</td>\n",
" <td>Except that you would never dare say something...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>59858</th>\n",
" <td>a0473abe447e04e3</td>\n",
" <td>Thanks for your reply and your explanation; yo...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>86152</th>\n",
" <td>e6763dac9d770096</td>\n",
" <td>or attempted generalization</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7620</th>\n",
" <td>1446437fe8605add</td>\n",
" <td>You seem to be vandalising the article. Why a...</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" id comment_text \\\n",
"34117 5b02208daa29a40f Outrageous!!!!! \\n\\nThis block is outrageous a... \n",
"6579 1190ddc487465bd2 Except that you would never dare say something... \n",
"59858 a0473abe447e04e3 Thanks for your reply and your explanation; yo... \n",
"86152 e6763dac9d770096 or attempted generalization \n",
"7620 1446437fe8605add You seem to be vandalising the article. Why a... \n",
"\n",
" toxic severe_toxic obscene threat insult identity_hate \n",
"34117 0 0 0 0 0 0 \n",
"6579 0 0 0 0 0 0 \n",
"59858 0 0 0 0 0 0 \n",
"86152 0 0 0 0 0 0 \n",
"7620 1 0 0 0 0 0 "
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we have a slight idea of what our data contains. let's first split the training set into train and validation set. the validation set will help us to verify whether our model is learning before we can fit the model to the test set. this is crucial since we should not use the test set to adjust the learning parameters of our models. doing so will leak some features of the test set and our model will fit well the test set and might fail to generalise to unknow data. I've chosen to use a sample of 10% of the training set as the validation data set."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.model_selection import train_test_split"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"train, validation = train_test_split(train, random_state=42, test_size=0.1, shuffle=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we have split the train and validation sets, let's explore our data and find out whether the split operation haven't altered the distribution of our data. we will first examine whether the mean and standard deviation of our training data has remained in the same range."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>toxic</th>\n",
" <th>severe_toxic</th>\n",
" <th>obscene</th>\n",
" <th>threat</th>\n",
" <th>insult</th>\n",
" <th>identity_hate</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>count</th>\n",
" <td>129251.000000</td>\n",
" <td>129251.000000</td>\n",
" <td>129251.000000</td>\n",
" <td>129251.000000</td>\n",
" <td>129251.000000</td>\n",
" <td>129251.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>mean</th>\n",
" <td>0.096146</td>\n",
" <td>0.010182</td>\n",
" <td>0.053083</td>\n",
" <td>0.003095</td>\n",
" <td>0.049454</td>\n",
" <td>0.008774</td>\n",
" </tr>\n",
" <tr>\n",
" <th>std</th>\n",
" <td>0.294793</td>\n",
" <td>0.100390</td>\n",
" <td>0.224199</td>\n",
" <td>0.055545</td>\n",
" <td>0.216815</td>\n",
" <td>0.093256</td>\n",
" </tr>\n",
" <tr>\n",
" <th>min</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>25%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>50%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>75%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>max</th>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" toxic severe_toxic obscene threat \\\n",
"count 129251.000000 129251.000000 129251.000000 129251.000000 \n",
"mean 0.096146 0.010182 0.053083 0.003095 \n",
"std 0.294793 0.100390 0.224199 0.055545 \n",
"min 0.000000 0.000000 0.000000 0.000000 \n",
"25% 0.000000 0.000000 0.000000 0.000000 \n",
"50% 0.000000 0.000000 0.000000 0.000000 \n",
"75% 0.000000 0.000000 0.000000 0.000000 \n",
"max 1.000000 1.000000 1.000000 1.000000 \n",
"\n",
" insult identity_hate \n",
"count 129251.000000 129251.000000 \n",
"mean 0.049454 0.008774 \n",
"std 0.216815 0.093256 \n",
"min 0.000000 0.000000 \n",
"25% 0.000000 0.000000 \n",
"50% 0.000000 0.000000 \n",
"75% 0.000000 0.000000 \n",
"max 1.000000 1.000000 "
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train.describe()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see that the mean and std for all our class columns have remained in the same range. this a first good sign. However we need to explore deeper and try to understand whether the validation set represents the sample population of our training. to do that we will explore the mean and std and then we will check how each individual class is distributed in both the training set and the validation set"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>toxic</th>\n",
" <th>severe_toxic</th>\n",
" <th>obscene</th>\n",
" <th>threat</th>\n",
" <th>insult</th>\n",
" <th>identity_hate</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>count</th>\n",
" <td>14362.000000</td>\n",
" <td>14362.000000</td>\n",
" <td>14362.00000</td>\n",
" <td>14362.000000</td>\n",
" <td>14362.000000</td>\n",
" <td>14362.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>mean</th>\n",
" <td>0.096574</td>\n",
" <td>0.009121</td>\n",
" <td>0.05236</td>\n",
" <td>0.002855</td>\n",
" <td>0.048322</td>\n",
" <td>0.008634</td>\n",
" </tr>\n",
" <tr>\n",
" <th>std</th>\n",
" <td>0.295387</td>\n",
" <td>0.095072</td>\n",
" <td>0.22276</td>\n",
" <td>0.053355</td>\n",
" <td>0.214453</td>\n",
" <td>0.092520</td>\n",
" </tr>\n",
" <tr>\n",
" <th>min</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.00000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>25%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.00000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>50%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.00000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>75%</th>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.00000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>max</th>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.00000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" <td>1.000000</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" toxic severe_toxic obscene threat insult \\\n",
"count 14362.000000 14362.000000 14362.00000 14362.000000 14362.000000 \n",
"mean 0.096574 0.009121 0.05236 0.002855 0.048322 \n",
"std 0.295387 0.095072 0.22276 0.053355 0.214453 \n",
"min 0.000000 0.000000 0.00000 0.000000 0.000000 \n",
"25% 0.000000 0.000000 0.00000 0.000000 0.000000 \n",
"50% 0.000000 0.000000 0.00000 0.000000 0.000000 \n",
"75% 0.000000 0.000000 0.00000 0.000000 0.000000 \n",
"max 1.000000 1.000000 1.00000 1.000000 1.000000 \n",
"\n",
" identity_hate \n",
"count 14362.000000 \n",
"mean 0.008634 \n",
"std 0.092520 \n",
"min 0.000000 \n",
"25% 0.000000 \n",
"50% 0.000000 \n",
"75% 0.000000 \n",
"max 1.000000 "
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"validation.describe()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see that our validation set is indeed doing a good job in representing our sample population. Good, but let's see if it is really good as it shows. let's now list how many comments are tagged by each class(tag). I will call this tags a category. the rule is that the ratio between the classes will remain almost the same on both data sets."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"def count_labels_per_category(df):\n",
" \n",
" df_toxic = df.drop(['id', 'comment_text'], axis=1)\n",
" counts = []\n",
" categories = list(df_toxic.columns.values)\n",
" for i in categories:\n",
" counts.append((i, df_toxic[i].sum()))\n",
" df_stats = pd.DataFrame(counts, columns=['category', 'number_of_comments'])\n",
" return df_stats"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's display the count of comments tagged with each category on the training set."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>category</th>\n",
" <th>number_of_comments</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>toxic</td>\n",
" <td>12427</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>severe_toxic</td>\n",
" <td>1316</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>obscene</td>\n",
" <td>6861</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>threat</td>\n",
" <td>400</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>insult</td>\n",
" <td>6392</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>identity_hate</td>\n",
" <td>1134</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" category number_of_comments\n",
"0 toxic 12427\n",
"1 severe_toxic 1316\n",
"2 obscene 6861\n",
"3 threat 400\n",
"4 insult 6392\n",
"5 identity_hate 1134"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df_stats_train = count_labels_per_category(train)\n",
"df_stats_train"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"let's do the same with the validation set"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>category</th>\n",
" <th>number_of_comments</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>toxic</td>\n",
" <td>1387</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>severe_toxic</td>\n",
" <td>131</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>obscene</td>\n",
" <td>752</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>threat</td>\n",
" <td>41</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>insult</td>\n",
" <td>694</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>identity_hate</td>\n",
" <td>124</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" category number_of_comments\n",
"0 toxic 1387\n",
"1 severe_toxic 131\n",
"2 obscene 752\n",
"3 threat 41\n",
"4 insult 694\n",
"5 identity_hate 124"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df_stats_valid = count_labels_per_category(validation)\n",
"df_stats_valid"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can see that the ration of comments attributed for each tag have remained the same."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's now plot this data since it is easier to see via graphs how comments are assigned to tags than to just read numbers."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"def plot_count_labels_per_category(df_stats):\n",
" df_stats.plot(x='category', y='number_of_comments', kind='bar', legend=False, grid=True, figsize=(8, 5))\n",
" plt.title(\"Number of comments per category\")\n",
" plt.ylabel('# of Occurrences', fontsize=12)\n",
" plt.xlabel('category', fontsize=12)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgcAAAGHCAYAAAAk+fF+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xm8XfO9//HXW2IIQQxxShJCKUK0\nJTXVbU9QYoxbFFdruFq3qmirVAelxS0tV2nVrV+pqRVjUUONDVU1hRLjlZoiphAiiTF8fn+s72at\n4wz7ZO9z1t77vJ+Px3mcvb5rre/+7LV3sj/nOy1FBGZmZmYVC5UdgJmZmTUWJwdmZmZW4OTAzMzM\nCpwcmJmZWYGTAzMzMytwcmBmZmYFTg7MaiTpbEnHlvTckvR7Sa9KuquMGMys9Tg5sJYj6SlJL0la\nIlf2VUmTSwyrr2wGfAEYGREblh1MX5M0WlJIGlx2LPUiqV3Ss2XHYZbn5MBa1SDgkLKD6C1Jg3p5\nyirAUxExry/isc61SnLSKq/D6s/JgbWqXwDflTSs447O/vqUNFnSV9PjfST9XdLJkl6T9ISkTVP5\n9NQqsXeHapeXdIOkOZJukbRKru610r5Zkh6T9KXcvrMlnS7pGknzgPGdxLuSpCvT+dMkfS2V7wf8\nDthE0lxJP+nsQkj6mqRHUmwPS1o/la+dXvdrkh6StGOHuH4j6dpU998lfUzSL1MXxqOSPp07/ilJ\nh0l6QNI8SWdKakvnz5F0o6RlcsdvLOn29Nz3S2rv8F4ck55zjqTrJS2fdt+afr+W4tpE0urpms+W\n9LKkC7u4DpX3fX9Jz0l6XtJ3c/sXknSEpH9JekXSRZKW7XDufpKeAW7u4jkmSvqnpNdTPRNS+b65\n9+AJSf+VypcArgVWSq9nbnq/u4wlnbeXpKfTviPT9d8y7Vs0vU/PpZ9fSlo07WuX9Kyk70l6Afi9\npAcl7ZCre+F0HT+NDVwR4R//tNQP8BSwJXAZcGwq+yowOT0eDQQwOHfOZOCr6fE+wHxgX7IWiGOB\nZ4DTgEWBrYA5wNB0/Nlp+3Np/ynAbWnfEsD0VNdg4NPAy8CY3Lmzgc+SJeuLdfJ6bgV+AywGfAqY\nCWyei/W2bq7FrsAM4DOAgNXJWhsWBqYBPwAWATZPr2HNXFwvAxuk570ZeBLYK3dN/trhmt8BtAEj\ngJeAe9PrrZx/VDp2BPAKsG16zV9I28Nz78W/gE8AQ9L28d28dxcAP6xcP2CzLq5F5dwL0vsyNl3L\nLdP+Q9JrGJnex98CF3Q499x07pBO6t8wvZdfSLGMANZK+7YDPp7eg88DbwDrp33twLMd6uouljHA\nXLIupUWAE4F3c6/jp+ncFYDhwO3AMbnnmg+ckOodAhwOXJh77onA1LL/Hfun3J/SA/CPf+r9w4fJ\nwbrpP+vh9D45eDy3b2w6vi1X9grwqfT4bGBSbt9Q4D1gFLAb8LcO8f2WD78ozwbO7ea1jEp1LZkr\n+xlwdi7W7pKD64BDOin/N+AFYKFc2QXA0bm4/l9u30HAIx2uyWsdrvmeue1LgdM7nH95evw94LxO\n4tw79178KLfvG8BfunnvzgXOIBt30d3nonLuWrmynwNnpsePAFvk9q1I9qU7OHfuat3U/1vg5Co/\no5dX3hc6Tw66i+XHpEQh7VsceIcPk4N/Advm9m9N1vVUea53yCWhwEpkieFSafsS4PD+/Dfrn8b7\ncbeCtayIeBC4CjhiAU5/Mff4zVRfx7Khue3pueedC8wi+093FWCj1Hz+mqTXgD2Bj3V2bidWAmZF\nxJxc2dNkf5VWYxTZl0Vn9U6PiPe7qbfj6+3u9ffm+FWAXTtck83IvgArXsg9fqOT58o7nOwv8rtS\n98h/dnMsFK/302TXohLXn3IxPUKWmLV1cW5HXV1rJG0j6Y7UNfQaWavJ8p0dW0UsK1H8vL1BlqxW\nrJReV2evEWBmRLyVO/854O/Azsq64bYB/tBNbDYAeDCKtbqjyJq3T8qVVQbvLQ68nh7nv6wXxKjK\nA0lDgWWB58j+E78lIr7Qzbnd3Rr1OWBZSUvmEoSVyboKqjGdrDm7s3pHSVoolyCsDPxflfXWYjpZ\ny8HXFuDcj1yriHgBqIzD2Ay4UdKtETGtizpGAY+mxyuTXYtKXP8ZEX/veIKk0V09f06n1zr1919K\n1iVzRUS8K+lysoSmqzq7i+V5YM3c9hBgudwhz5ElFw+l7fxr7Or5ziFrXRsM/CMiqv18WYtyy4G1\ntPQFcSFwcK5sJtmX65clDUp/aXb2Bdob20raTNIiwDHAHRExnazl4hOSvpIGei0s6TOS1q4y/ulk\nfcY/k7SYpPWA/YDzq4zrd2QDMzdQZnVlgyXvJPuL/PAUUzuwAzCpNy96AZ0P7CBp63T9F0sD5UZW\nce5M4H1gtUqBpF1z575K9uX3fifnVhwpaXFJ65CNBakMYPxf4Lh0fZA0XNLEXryuM4F9JW2RBhSO\nkLQW2biARVPs8yVtQzZupeJFYDlJS+fKuovlErLrt2n6vB3Nh4kGZN1DP0rnLE/WDdHT5+VyYH2y\nsQ7n9uI1W4tycmADwU/JBpHlfQ04jKw5dh2yL+Ba/JGslWIW2SC+LwOkv/a3AnYn++vtBT4cDFat\nPcj6vJ8D/kQ2XuHGak6MiIuB41J8c8i+BJaNiHfIkoFtyAYe/gbYKyIe7aquekkJz0SywZAzyf5K\nPowq/j9KTejHAX9PTe4bkw22vFPSXOBKsr78J7qp5haywZg3ASdGxPWp/JR0/vWS5pAN6tuoF6/r\nLrJk42SysS63AKukz8DBwEVkyct/pOepnPco2Rf6E+k1rdRdLBHxENkYjknA82SDE18C3k5VHgvc\nAzwATCVrOet2ka6IeJOsdWNVsoG8NsAportWMjOz1pC6Bp4EFo6I+eVGUz+pG+s1YI2IeLKGen4M\nfCIivly34KxpueXAzKzJSNohdY0sQTaVcSrZjJEFrW9Zsu6qM+oToTU7JwdmZs1nIlk303PAGsDu\nsYDNwMoW1ZoOXBsRt/Z0vA0M7lYwMzOzArccmJmZWcGAXudg+eWXj9GjR5cdRsG8efNYYomOA+ut\nI1+n6vlaVcfXqXq+VtVpxOs0ZcqUlyNieE/HDejkYPTo0dxzzz1lh1EwefJk2tvbyw6j4fk6Vc/X\nqjq+TtXztapOI14nSU/3fJS7FczMzKwDJwdmZmZW4OTAzMzMCpwcmJmZWYGTAzMzMytwcmBmZmYF\nTg7MzMyswMmBmZmZFTg5MDMzswInB2ZmZlbg5MDMzMwKBvS9Fepl9BFX162uQ8fOZ5861ffU8dvV\npR4zMxtY3HJgZmZmBU4OzMzMrMDJgZmZmRU4OTAzM7MCJwdmZmZW0C/JgaSzJL0k6cFc2S8kPSrp\nAUl/kjQst+/7kqZJekzS1rnyCalsmqQjcuWrSrozlV8oaZH+eF1mZmatqL9aDs4GJnQouwFYNyLW\nA/4P+D6ApDHA7sA66ZzfSBokaRBwGrANMAbYIx0LcAJwckSsDrwK7Ne3L8fMzKx19UtyEBG3ArM6\nlF0fEfPT5h3AyPR4IjApIt6OiCeBacCG6WdaRDwREe8Ak4CJkgRsDlySzj8H2KlPX5CZmVkLa5RF\nkP4TuDA9HkGWLFQ8m8oApnco3whYDngtl2jkj/8ISfsD+wO0tbUxefLkWmPn0LHzez6oSm1D6ldf\nPV5bo5o7d25Lv7568rWqjq9T9XytqtPM16n05EDSD4H5wB/64/ki4gzgDIBx48ZFe3t7zXXWa0VD\nyBKDk6bW5215as/2utTTiCZPnkw93ruBwNeqOr5O1fO1qk4zX6dSkwNJ+wDbA1tERKTiGcCo3GEj\nUxldlL8CDJM0OLUe5I83MzOzXiptKqOkCcDhwI4R8UZu15XA7pIWlbQqsAZwF3A3sEaambAI2aDF\nK1NS8Vdgl3T+3sAV/fU6zMzMWk1/TWW8APgHsKakZyXtB/waWBK4QdI/Jf0vQEQ8BFwEPAz8BTgw\nIt5LrQLfBK4DHgEuSscCfA/4jqRpZGMQzuyP12VmZtaK+qVbISL26KS4yy/wiDgOOK6T8muAazop\nf4JsNoOZmZnVyCskmpmZWYGTAzMzMytwcmBmZmYFTg7MzMyswMmBmZmZFTg5MDMzswInB2ZmZlbg\n5MDMzMwKnByYmZlZgZMDMzMzK3ByYGZmZgVODszMzKzAyYGZmZkVODkwMzOzAicHZmZmVuDkwMzM\nzAqcHJiZmVmBkwMzMzMrcHJgZmZmBU4OzMzMrMDJgZmZmRU4OTAzM7MCJwdmZmZW4OTAzMzMCpwc\nmJmZWYGTAzMzMytwcmBmZmYFTg7MzMyswMmBmZmZFTg5MDMzswInB2ZmZlbg5MDMzMwKnByYmZlZ\nQb8kB5LOkvSSpAdzZctKukHS4+n3Mqlckk6VNE3SA5LWz52zdzr+cUl758o3kDQ1nXOqJPXH6zIz\nM2tF/dVycDYwoUPZEcBNEbEGcFPaBtgGWCP97A+cDlkyARwFbARsCBxVSSjSMV/LndfxuczMzKxK\n/ZIcRMStwKwOxROBc9Ljc4CdcuXnRuYOYJikFYGtgRsiYlZEvArcAExI+5aKiDsiIoBzc3WZmZlZ\nLw0u8bnbIuL59PgFoC09HgFMzx33bCrrrvzZTso7JWl/shYJ2tramDx58oK/guTQsfNrrqOibUj9\n6qvHa2tUc+fObenXV0++VtXxdaqer1V1mvk6lZkcfCAiQlL003OdAZwBMG7cuGhvb6+5zn2OuLrm\nOioOHTufk6bW5215as/2utTTiCZPnkw93ruBwNeqOr5O1fO1qk4zX6cyZyu8mLoESL9fSuUzgFG5\n40amsu7KR3ZSbmZmZgugzOTgSqAy42Bv4Ipc+V5p1sLGwOzU/XAdsJWkZdJAxK2A69K+1yVtnGYp\n7JWry8zMzHqpX7oVJF0AtAPLS3qWbNbB8cBFkvYDnga+lA6/BtgWmAa8AewLEBGzJB0D3J2O+2lE\nVAY5foNsRsQQ4Nr0Y2ZmZgugX5KDiNiji11bdHJsAAd2Uc9ZwFmdlN8DrFtLjGZmZpbxColmZmZW\n4OTAzMzMCpwcmJmZWYGTAzMzMytwcmBmZmYFTg7MzMyswMmBmZmZFTg5MDMzswInB2ZmZlbg5MDM\nzMwKnByYmZlZgZMDMzMzK3ByYGZmZgVODszMzKzAyYGZmZkVODkwMzOzgqqSA0l7SFo7PV5T0q2S\n/ipprb4Nz8zMzPpbtS0HxwKz0uMTgbuAW4Df9EVQZmZmVp7BVR43PCJelLQYsBmwC/Au8HKfRWZm\nZmalqDY5mClpdWAscHdEvC1pcUB9F5qZmZmVodrk4BhgCvAesFsq2xK4vy+CMjMzs/JUlRxExNmS\nLkqP30jFdwC791VgZmZmVo7eTGUcAuws6fC0PZjqWx7MzMysSVQ7lfHzwGPAnsCRqXgN4PQ+isvM\nzMxKUm3LwS+B3SJiAjA/ld0JbNgnUZmZmVlpqk0ORkfETelxpN/v4G4FMzOzllNtcvCwpK07lG0J\nTK1zPGZmZlayav/yPxS4StLVwBBJvwV2ACb2WWRmZmZWiqpaDiLiDmA94CHgLOBJYMOIuLsPYzMz\nM7MSVNVyIGlRYGZE/DxXtrCkRSPi7T6LzszMzPpdtWMObgA26FC2AXBdfcMxMzOzslWbHIwlm7qY\ndxfwyfqGY2ZmZmWrNjmYDbR1KGsD5tUagKRvS3pI0oOSLpC0mKRVJd0paZqkCyUtko5dNG1PS/tH\n5+r5fip/rJOZFWZmZlalapODS4E/SlpX0uKSxgLnAhfV8uSSRgAHA+MiYl1gENn9Gk4ATo6I1YFX\ngf3SKfsBr6byk9NxSBqTzlsHmAD8RtKgWmIzMzMbqKpNDn4IPELWlTCH7KZLjwE/qEMMg8mmRw4G\nFgeeBzYHLkn7zwF2So8npm3S/i0kKZVPioi3I+JJYBpevdHMzGyBKCJ6PqpycPZFvDzwcvTmxO7r\nPAQ4DngTuB44BLgjtQ4gaRRwbUSsK+lBYEJEPJv2/QvYCDg6nXN+Kj8znXNJJ8+3P7A/QFtb2waT\nJk2q+TVMnTG75joq2obAi2/Wp66xI5auT0UNaO7cuQwdOrTsMJqCr1V1fJ2q52tVnUa8TuPHj58S\nEeN6Oq7q5Y8lLQ2sCQxN2wBExM0LGCOSliH7q39V4DXgYrJugT4TEWcAZwCMGzcu2tvba65znyOu\nrrmOikPHzuekqfVZlfqpPdvrUk8jmjx5MvV47wYCX6vq+DpVz9eqOs18napd52Af4DRgLvBGblcA\nq9Xw/FsCT0bEzPQ8lwGfBYZJGhwR84GRwIx0/AxgFPBs6oZYGnglV16RP8fMzMx6odoxB8cBu0RE\nW0SsmvupJTEAeAbYOA1yFLAF8DDwV2CXdMzewBXp8ZVpm7T/5tS9cSWwe5rNsCrZ7aTvqjE2MzOz\nAana9uvBZOMB6ioi7pR0CXAv2a2g7yNr8r8amCTp2FR2ZjrlTOA8SdOAWWQzFIiIhyRdRJZYzAcO\njIj36h2vmZnZQFBtcnAC8CNJx0TE+/UMICKOAo7qUPwEncw2iIi3gF27qOc4shYOMzMzq0G1ycG3\ngY8Bh0t6Jb8jIlaue1RmZmZWmmqTgy/3aRRmZmbWMKpKDiLilr4OxMzMzBpDVbMV0iyA4yQ9IWl2\nKttK0jf7NjwzMzPrb9VOZTwZWBfYk2xtA4CHgAP6IigzMzMrT7VjDv4dWD0i5kl6HyAiZqQbJ5mZ\nmVkLqbbl4B06JBKShpOtTmhmZmYtpNrk4GLgnLT6IJJWBH4N1H7XIjMzM2so1SYHPwCeBKYCw4DH\ngeeAn/RRXGZmZlaSHsccSFoI2Aw4IiK+nboT6nbLZjMzM2ssPbYcpOWSr4iIt9P2TCcGZmZmrava\nboVbJW3cp5GYmZlZQ6h2KuPTwLWSrgCm8+FaB0TEj/siMDMzMytHtcnBEODy9HhkrtzdC2ZmZi2m\n2gGJ5wF/r4w7MDMzs9bV6wGJZmZm1to8INHMzMwKPCDRzMzMCmodkGhmfWDsOWPrVtcBQw/goHMO\nqrmeqXtPrUM0ZtYMqkoOImLfvg7EzMzMGkNVyYGk1braFxFP1C8cMzMzK1u13QrTyMYZKFdWGXcw\nqK4RmZmZWamq7VYozGqQ9DHgKOBvfRGUmZmZlafaqYwFEfEC8C3gZ/UNx8zMzMq2QMlBsiaweL0C\nMTMzs8ZQ7YDEv1G8j8LiwDrAT/siKDMzMytPtQMSf9dhex5wf0Q8Xud4zMzMrGTVDkg8p68DMTMz\ns8ZQbbfCZcDJEfG3XNm/AYdExC59FZyZmdXHaV+/uW51rbDpvLrUd+D/bl6HaKwvVDsg8fPA7R3K\n/gGMr284ZmZmVrZqk4O3gCU6lA0F3q1vOGZmZla2apOD64DfSloKIP3+NfCXvgrMzMzMylFtcnAo\nsBQwS9JLwCxgabKFkMzMzKyFVJUcRMSrEbEd2e2atwNGRsQOEfFarQFIGibpEkmPSnpE0iaSlpV0\ng6TH0+9l0rGSdKqkaZIekLR+rp690/GPS9q71rjMzMwGqqqSA0lbSfpERLwQEXdHxAuS1pT0hTrE\ncArwl4hYC/gk8AhwBHBTRKwB3JS2AbYB1kg/+wOnp/iWJbvXw0bAhsBRlYTCzMzMeqfaboXTgDkd\nyuak8gUmaWngc8CZABHxTmqNmAhU1lY4B9gpPZ4InBuZO4BhklYEtgZuiIhZEfEqcAMwoZbYzMzM\nBipFRM8HSbMjYukOZQJmR8RSC/zk0qeAM4CHyVoNpgCHADMiYljueV6NiGGSrgKOj4jb0r6bgO8B\n7cBiEXFsKj8SeDMiTuzkOfcna3Wgra1tg0mTJi1o+B+YOmN2zXVUtA2BF9+sT11jRyzd80FNau7c\nuQwdOrTsMPrMw688XLe6hg8azsz3ZtZcz5jlxtQhmsbV6p+pmc90/PtuwQ1e4n3mz6vl1jyZ4Ssv\nWYdoGlcjfqbGjx8/JSLG9XRctcsnPyFp84jIr3rRDjy5IMF1eP71gYMi4k5Jp/BhFwIAERGSes5g\nqhQRZ5AlJIwbNy7a29trrnOfI66uuY6KQ8fO56Sp1b4t3Xtqz/a61NOIJk+eTD3eu0Z10DkH1a2u\nA4YewOlzT6+5nqk7T61DNI2r1T9T9V4E6aXbO85u771d92qvPZgG1syfqWq/hY4GLpN0JvAv4OPA\nvumnFs8Cz0bEnWn7ErLk4EVJK0bE86nb4KW0fwYwKnf+yFQ2gyxZyZdPrjE2MzOzAana2QpXAFuR\nLYS0Xfq9dSpfYBHxAjBd0pqpaAuyLoYrgcqMg72ByvNcCeyVZi1sTNat8TzZOgxbSVomDUTcKpWZ\nmZlZL1Xdfh0RdwF39UEMBwF/kLQI8ARZa8RCwEWS9gOeBr6Ujr0G2BaYBryRjiUiZkk6Brg7HffT\niJjVB7GamZm1vB6TA0mjyboVvgAsD7wM3Aj8JCKeqDWAiPgn0NngiC06OTaAA7uo5yzgrFrjMTMz\nG+i67VaQtDZwL7AC8ENgx/R7OHBP2m9mZmYtpKeWg+OB0yLiyA7lZ0s6Fvg5sEOfRGZmZmal6Ck5\n+BwfDgzs6CRqn8poZmZmDaan2QqD6Pq2zO+m/WZmZtZCekoO7qbrtQz2Ae6pazRmZmZWup66FY4E\nrkvrEFwCPA+sCOxK1t2wdd+GZ2ZmZv2t25aDiLidbEGhT5LdHfHR9PuTwIS038zMzFpIj+scRMQ/\ngM9JGgIsS3YTpDf6PDIzMzMrRW9WSHyT7B4GZmZm1sJqv+emmZmZtRQnB2ZmZlbQZXIg6Re5x5v3\nTzhmZmZWtu5aDvbPPb68rwMxMzOzxtDdgMT7JV0CPAwsKumnnR0UET/uk8jMzMysFN0lB7uQtR6s\nAggY1ckx0RdBmZmZWXm6TA4i4iXgWABJgyOiq2WUzczMrIVUtc5BROwraRmy2zOPIFvv4KqImNWX\nwZmZmVn/q2oqo6RNgH8BXwfWA/4LmJbKzczMrIVUu0LiL4FvRMSkSoGk3YBTgc/0RWBmZmZWjmoX\nQfoEcFGHskuA1esbjpmZmZWt2uTgcWD3DmW7knU1mJmZWQuptlvhW8BVkg4GngZGA2sA2/dRXGZm\nZlaSamcr3C7p48B2wErAn4FrPFvBzMys9fTmls2vAuf3YSxmZmbWAHxXRjMzMytwcmBmZmYFTg7M\nzMysoOrkQNIqfRmImZmZNYbetBzcB5CmM5qZmVmL6na2gqQpwBSyxGBQKj6abNlkMzMza0E9tRzs\nAlwPrAIsLuleYFFJ4yUt3efRmZmZWb/rKTkYFBGXRMQRwBxgIiDgIOCfkh7v6wDNzMysf/W0CNIf\nJK0MPAwsBiwDvBURXwSQtGwfx2dmZmb9rNuWg4jYCBgFfBcI4NfAkpJOl/Q1YNV6BCFpkKT7JF2V\ntleVdKekaZIulLRIKl80bU9L+0fn6vh+Kn9M0tb1iMvMzGwg6nG2QkTMj4j7gHci4nPAPGAy2Y2X\nTqhTHIcAj+S2TwBOjojVgVeB/VL5fsCrqfzkyvNLGkN218h1gAnAbyQNwszMzHqtN1MZv51+R0Rc\nGBGHR8SWtQYgaSTZDZ1+l7YFbA5ckg45B9gpPZ6Ytkn7t0jHTwQmRcTbEfEkMA3YsNbYzMzMBiJF\nRO9OkJZJN2GqTwDSJcDPgCXJui/2Ae5IrQNIGgVcGxHrSnoQmBARz6Z9/wI2IpteeUdEnJ/Kz0zn\nXNLh6ZC0P7A/QFtb2waTJk2q+TVMnTG75joq2obAi2/Wp66xI1p3QsncuXMZOnRo2WH0mYdfebhu\ndQ0fNJyZ782suZ4xy42pQzSNq9U/UzOfmVO3ugYv8T7z59W+wO7wlZesQzSNqxE/U+PHj58SEeN6\nOq7quzJW1Dkx2B54KSKmSGqvV73diYgzgDMAxo0bF+3ttT/tPkdcXXMdFYeOnc9JU3v9tnTqqT3b\n61JPI5o8eTL1eO8a1UHnHFS3ug4YegCnzz295nqm7jy1DtE0rlb/TJ329ZvrVtcKm87jpduXqLme\nXfdqrz2YBtbMn6n6fAstuM8CO0ralmw2xFLAKcAwSYMjYj4wEpiRjp9BNkDyWUmDgaWBV3LlFflz\nzMzMrBdKvfFSRHw/IkZGxGiyAYU3R8SewF/JFmAC2Bu4Ij2+Mm2T9t8cWb/IlcDuaTbDqmSDJe/q\np5dhZmbWUspuOejK94BJko4lW7r5zFR+JnCepGnALLKEgoh4SNJFZOsxzAcOjIj3+j9sMzOz5tcw\nyUFETCabIklEPEEnsw0i4i1g1y7OPw44ru8iNDMzGxhK7VYwMzOzxuPkwMzMzAqcHJiZmVmBkwMz\nMzMrcHJgZmZmBU4OzMzMrMDJgZmZmRU4OTAzM7MCJwdmZmZW4OTAzMzMCpwcmJmZWYGTAzMzMytw\ncmBmZmYFTg7MzMyswMmBmZmZFTg5MDMzswInB2ZmZlbg5MDMzMwKnByYmZlZgZMDMzMzK3ByYGZm\nZgVODszMzKzAyYGZmZkVODkwMzOzAicHZmZmVuDkwMzMzAqcHJiZmVmBkwMzMzMrcHJgZmZmBU4O\nzMzMrMDJgZmZmRU4OTAzM7OCUpMDSaMk/VXSw5IeknRIKl9W0g2SHk+/l0nlknSqpGmSHpC0fq6u\nvdPxj0vau6zXZGZm1uzKbjmYDxwaEWOAjYEDJY0BjgBuiog1gJvSNsA2wBrpZ3/gdMiSCeAoYCNg\nQ+CoSkJhZmZmvVNqchARz0fEvenxHOARYAQwETgnHXYOsFN6PBE4NzJ3AMMkrQhsDdwQEbMi4lXg\nBmBCP74UMzOzlqGIKDsGACSNBm4F1gWeiYhhqVzAqxExTNJVwPERcVvadxPwPaAdWCwijk3lRwJv\nRsSJnTzP/mStDrS1tW0wadKkmmOfOmN2zXVUtA2BF9+sT11jRyxdn4oa0Ny5cxk6dGjZYfSZh195\nuG51DR80nJnvzay5njHLjalDNI2r1T9TM5+ZU7e6Bi/xPvPn1f635fCVl6xDNI2rET9T48ePnxIR\n43o6bnB/BNMTSUOBS4FvRcTrWT6QiYiQVLcMJiLOAM4AGDduXLS3t9dc5z5HXF1zHRWHjp3PSVPr\n87Y8tWd7XeppRJMnT6Ye713lZh+UAAAUVUlEQVSjOuicg+pW1wFDD+D0uafXXM/UnafWIZrG1eqf\nqdO+fnPd6lph03m8dPsSNdez617ttQfTwJr5M1X2mAMkLUyWGPwhIi5LxS+m7gLS75dS+QxgVO70\nkamsq3IzMzPrpbJnKwg4E3gkIv4nt+tKoDLjYG/gilz5XmnWwsbA7Ih4HrgO2ErSMmkg4lapzMzM\nzHqp7G6FzwJfAaZK+mcq+wFwPHCRpP2Ap4EvpX3XANsC04A3gH0BImKWpGOAu9NxP42IWf3zEszM\nzFpLqclBGlioLnZv0cnxARzYRV1nAWfVLzozM7OBqfQxB2ZmZtZYnByYmZlZgZMDMzMzK3ByYGZm\nZgVODszMzKzAyYGZmZkVODkwMzOzAicHZmZmVuDkwMzMzArKXj7ZBpqj63Qb6TV/AkdPrE9dR9fv\nlttmZq3ALQdmZmZW4OTAzMzMCtytYGZmlpy02/Z1q2vk1jtx0ukn1lzPoRdeVYdoesctB2ZmZlbg\n5MDMzMwKnByYmZlZgZMDMzMzK3ByYGZmZgVODszMzKzAyYGZmZkVODkwMzOzAicHZmZmVuDkwMzM\nzAqcHJiZmVmBkwMzMzMrcHJgZmZmBU4OzMzMrMDJgZmZmRUMLjsAM7NaPLLW2nWp562DvskjXz+g\nLnWt/egjdanHrCxuOTAzM7MCJwdmZmZW4OTAzMzMCpwcmJmZWUFLJQeSJkh6TNI0SUeUHY+ZmVkz\napnkQNIg4DRgG2AMsIekMeVGZWZm1nxaJjkANgSmRcQTEfEOMAmYWHJMZmZmTUcRUXYMdSFpF2BC\nRHw1bX8F2CgivtnhuP2B/dPmmsBj/Rpoz5YHXi47iCbg61Q9X6vq+DpVz9eqOo14nVaJiOE9HTTg\nFkGKiDOAM8qOoyuS7omIcWXH0eh8narna1UdX6fq+VpVp5mvUyt1K8wARuW2R6YyMzMz64VWSg7u\nBtaQtKqkRYDdgStLjsnMzKzptEy3QkTMl/RN4DpgEHBWRDxUclgLomG7PBqMr1P1fK2q4+tUPV+r\n6jTtdWqZAYlmZmZWH63UrWBmZmZ14OTAzMzMCpwcmJmZWYGTAzMbkCR9tpoys4HIyUHJJP27pKVz\n28Mk7VRmTI1K0hKSFsptLyRp8TJjamS+Nj36VZVlBkjatZoyA0lDJK1Zdhy1cHJQvqMiYnZlIyJe\nA44qMZ5GdhOQ/8JbHLixpFgalqRNJT0MPJq2PynpNyWH1TAkbSLpUGC4pO/kfo4mmwZtnft+lWUD\nmqQdgH8Cf0nbn5LUdGvutMw6B02sswTN70vnFouIuZWNiJjrv447dTKwNWkRsIi4X9Lnyg2poSwC\nDCX7d7Zkrvx1YJdSImpgkrYBtgVGSDo1t2spYH45UTW0o8luBDgZICL+KWnVMgNaEP4SKt89kv6H\n7HbTAAcCU0qMp5HNk7R+RNwLIGkD4M2SY2pIETFdUr7ovbJiaTQRcQtwi6SzI+LpsuNpAs+R/Z+0\nI8X/m+YA3y4losb2bkTM7vDvr+kWFHJyUL6DgCOBC9P2DWQJgn3Ut4CLJT0HCPgYsFu5ITWk6ZI2\nBULSwsAhwCMlx9SI3pD0C2AdYLFKYURsXl5IjSci7gful3R+RLiloGcPSfoPYJCkNYCDgdtLjqnX\nvEKiNZX0ZVcZ6PNYRLxbZjyNSNLywCnAlmRJ1PXAIRHxSqmBNRhJ15Ml5d8Fvg7sDcyMiO+VGliD\nkTSVbv7yjYj1+jGchpe6On8IbJWKrgOOiYi3y4uq95wclETSLyPiW5L+TCf/8CJixxLCakiSNo+I\nmyV9sbP9EXFZf8dkzU/SlIjYQNIDlS84SXdHxGfKjq2RSFqlu/3umimStGtEXNxTWaNzt0J5zku/\nTyw1iubweeBmYIdO9gXg5CBH0nDga8Bocv/GI+I/y4qpQVVanZ6XtB1Z3/qyJcbTkPzl32vfBzom\nAp2VNTS3HJRM0goR8VKHsjUj4rGyYrLmJul24G9kg8c+GIgYEZeWFlQDkrQ92XUaRba+wVLATyKi\n6aad9QdJc/iwlXMRYGFgXkQsVV5UjSM3q+NLfDiGDLLP1ZiI2LCUwBaQWw7K9zdJR0bERQBp/vV+\nwJhyw2o8ks4DvllZFyI1d54VEVuUG1nDWdz95j2LiKvSw9nA+DJjaQYR8cG0T2VD8ScCG5cXUcN5\nDriHFpnV4ZaDkklakeye328BbWSjyg/Nz+e3jKT/IvtH9h1gBHAY2bX6c6mBNRhJxwK3R8Q1ZcfS\nyCR9AjgdaIuIdSWtB+wYEceWHFrTkHRfRHy67DgaiaSFW2GgtJODBiDpQLI+qfeB3SOi6aa99BdJ\nmwF/BV4GPh0RL5QcUsNJzb9LAO+kHwHh5t8iSbeQJZi/rXzBSXowItYtN7LG1GFA8ELAOODzEbFJ\nSSE1pDR98Wdkrb/5KbKrlRbUAnC3Qskk3UjWHLUuWd/nmZJujYjvlhtZ45H0FbI1IfYC1gOukbRv\nmodtSb7517q1eETc1WGxGs/j71p+QPB84CmyrgUr+j3ZEvgnk3VX7UsT3qrAyUH5fh0Rl6fHr6XF\na7xeeed2BjZLAzgvkPQn4GzAzZo5qT94T2DViDhG0ihgxYi4q+TQGs3Lkj5OGmQnaRfg+XJDalwR\nsW/ZMTSJIRFxkySlmR5HS5oC/LjswHrD3QoNQFIbUJlbfVfH2QvWNUmLRMQ7ZcfRSCSdTtZFtXlE\nrC1pGeB6z98vkrQa2XifTYFXgSeBPT11r3OSfg4cS7Zk+V/IWu++HRHnlxpYg0mzhTYDLiGbgj0D\nOD4imuoujU3X1NFqJH0JuAvYlWwKzJ3pLxjrQNJISX+SNFPSS5IuBVYoO64GtFFEHEg2yJWIeJVs\n6pkl6dbf4yJiS2A4sFZEbObEoFtbRcTrwPZkXQqrk43ZsKJDyO4YezCwAfAVstU3m4q7Fcr3Q+Az\nldaCtIDNjWRZpxX9HvgjWSIF8OVU9oXSImpM70oaxIfN5cPJWhIsiYj3JR0OXBQR88qOp0lUvi+2\nAy7u5OZCBkTE3enhXLLxBk3JyUH5FurQjfAKbtHpyvCI+H1u+2xJ3yotmsZ1KvAnYAVJx5HdhvhH\n5YbUkG6U9F2yBWs+SBAiYlZ5ITW0qyQ9StatcEBKOt8qOaaGk6bIHgasQnGF0qa6oZfHHJQs9eN9\nErggFe0GPOBFbD5K0k1kLQWVa7UHsK8XQfooSWsBW5BNY7wpInxXxg4kPdlJcTTblLP+JGlZYHZE\nvJduMLSUpxMXSbof+F8+ukLplC5PakBODkom6QTgTrIBLJAt57qxk4OPSisi/grYhKzJ/HbgoIiY\nXmpgDUbSxsBDETEnbS8FrB0Rd5YbmTW7NJtqNMW/iM8tLaAGVLmhV9lx1MrJQckk3RsR63co++Au\ncfYhSZ+NiL/3VDbQSboPWD/SP+40+O6ejp8z85ddb6Tlyz8O/JMP/yKOiDi4vKgaR2pVgWwg4ktk\nXXsf3Ka52bqrPOagJJIOAL4BrCbpgdyuJQF/2XXuV0DHL7jOygY6RS7rT4Pv/G+9g66+7AAnB50b\nR3YDIf9F2bkpZJ+fyijN/EyOAJqqu8r/YZTnj8C1ZMtsHpErn9NsGWZfk7QJ2Vz04ZK+k9u1FDCo\nnKga2hOSDia7bwBkSegTJcbTqPxl1zsPAh/DC0V1KiJWreY4SV+IiBv6Op5aOTkoSbqz4GyyQXXW\nvUWAoWSf1/zSwK+TjcS3oq+TzVj4EdlfLDcB+5caUWPyl13vLA88LOkuis3lO5YXUlM6AWj45MBj\nDqxpSFqlu0VqJP0qIg7qz5is+Uj6M1nStCTwKbJFyPxl1wNJn++sPCJu6e9Ymlmz3MnSLQfWNKpY\nve6z/RJIg/Mytz06kaxf+ARgp1x5pcw64SSgbpriL3InB2atZ6uIOFzSv5Mtc/tF4FbAyQEffslJ\nWrjjF56kIeVE1bgk3RYRm6Vbgee/2Hwr8Bbm5MCs9XiZ2254plDvRMRm6bdvBV4fT5UdQDWcHFgr\n8Tdgxsvcds8zhazPpNsznwX8Md30rCAivtj/UfWeByRa05G0eES80Un5PhFxdgkhNRwvc2tWDkmr\nk91waTfgHrIl369vtimzTg6saaTV7H4HDI2IlSV9EviviPhGyaE1FEmLkTWbb0bWR3wbcHpEuPXA\nrJ+klUm3J1tv5D2yJOGUZmmd8t3/rJmcDGxNdudKIuJ+4HOlRtSYzgXWIVs98tfAGOC8UiMyG0Ak\nrQecBPwCuJTsNvOvAzeXGVdveMyBNZWImN5hcN17XR07gK0bEWNy23+V9HBp0ZgNIGnMwWvAmcAR\nEVFZQ+NOSU0z3drJgTWT6alrISQtDBwC+FbEH3WvpI0j4g4ASRuR9X2aWd/bNSIKy5VLWjUinmyW\nwYjgMQfWRCQtD5wCbEk2M+F64JCIeKXUwBqEpKlkYwwWBtYEnknbqwCPdmhNMLM+0MWddpvuNs5u\nObCmIGkQ8JWI2LPsWBrY9rnHywD/lh7fStbMaWZ9RNJaZGN9lpaUbyFYClisnKgWnAckWlOIiPeA\n/yg7jkYWEU+nJaZ3IhuAuDwwPD32/QLM+taaZAn6MGCH3M/6wNdKjGuBuFvBmoakk8mazC8E5lXK\nI+Le0oJqQGnVv00iYl7aXgL4R0SsV25kZq1P0iYR8Y+y46iVuxWsmXwq/f5priyAzUuIpZGJ4iyO\n9/DqkWZ9StLhEfFz4D8k7dFxf0QcXEJYC8zJgTWNiBhfdgxN4vdk06b+lLZ3IptWZWZ9pzJzqiVm\nBrlbwZqGpDbgv4GVImIbSWPIms/9xdeBpPXJVkgE+FtE3FdmPGYDhaRdI+LinsoanZMDaxqSriX7\nq/iHEfFJSYOB+yJibMmhmZkBXU5l/EhZo3O3gjWT5SPiIknfB4iI+ZK8QqKZlU7SNsC2wAhJp+Z2\nLQXMLyeqBefkwJrJPEnLkQ1CRNLGwOxyQzIzA+A5svEGOwJTcuVzgG+XElEN3K1gTUPSBsCpwLrA\ng2Rz+HeJiAdKDczMLJG0cES8W3YctXJyYE0ljTNYk2xq3mOt8I/QzFpHurnS0WTLlg8m+78qImK1\nMuPqLScH1jTS4j6TgAsj4l9lx2Nm1pGkR8m6EaaQW2+k2e4B4+TAmoakVYDd0s/7ZCslXhQRz5Qa\nmJlZIunOiNio7Dhq5eTAmpKkNYAjgT0jYlDZ8ZiZAUg6HhgEXAa8XSlvtmXePVvBmkqH1oP3gMPL\njcjMrKDSajAuV9Z0y7y75cCahqQ7yW68dDHZuIMnSg7JzKwlOTmwpiFpzYh4rOw4zMy60irLvC9U\ndgBmvfCapDPTMspIGiNpv7KDMjPLORu4Dlgpbf8f8K3SollATg6smZxNC/yjM7OWtnxEXEQ2o4qI\nmE/xFupNwcmBNZOW+EdnZi2tJZZ592wFayYt8Y/OzFrad4ArgY9L+jtpmfdyQ+o9D0i0piFpfeBX\n+N4KZtbAWmGZd7ccWDP5OLANMArYmWw+sT/DZlY6SV/sYtcnJBERl/VrQDXyf6zWTI6MiIslLQOM\nB04ETufDRUfMzMqyQ/q9ArApcHPaHg/cTrZiYtPwgERrJpXBh9sB/y8irgYWKTEeMzMAImLfiNiX\nbKG2MRGxc0TsDKyTypqKkwNrJjMk/ZZs6eRrJC2KP8Nm1lhGRcTzue0XgZXLCmZBeUCiNQ1JiwMT\ngKkR8bikFYGxEXF9yaGZmQEg6dfAGsAFqWg3YFpEHFReVL3n5MDMzKyO0uDEf0ubt0bEn8qMZ0E4\nOTAzM7MCz1YwMzOrkaTbImIzSXNIC7VVdgEREUuVFNoCccuBmZmZFXikt5mZmRU4OTAzM7MCJwdm\nZmZW4OTAzGom6SlJW5Ydh5nVh5MDM2ta6e53ZlZnTg7MrEDSKEmXSZop6RVJv5b0cUk3p+2XJf1B\n0rB0/Hlky8P+WdJcSYen8o0l3S7pNUn3S2rPPceqkm6VNEfSjZJOk3R+bv+Okh5K506WtHZu31OS\nvifpAWCepMMkXdrhNZwq6ZS+vVJmrcvJgZl9QNIg4CrgaWA0MAKYRDZX+2fASsDaZLfNPhogIr4C\nPAPsEBFDI+LnkkYAVwPHAssC3wUulTQ8PdUfgbuA5VI9X8nF8AmypWe/BQwHriFLPPI32dqD7AZc\nw4DzgQm5ZGUwsDtwbn2uitnA4+TAzPI2JEsADouIeRHxVkTcFhHTIuKGiHg7ImYC/wN8vpt6vgxc\nExHXRMT7EXEDcA+wraSVgc8AP46IdyLiNuDK3Lm7AVen53uX7NbcQ8hug1txakRMj4g3001ubgV2\nTfsmAC9HxJSar4bZAOXkwMzyRgFPR8T8fKGkNkmTJM2Q9DrZX+vLd1PPKsCuqVvgNUmvAZsBK5Il\nH7Mi4o3c8dNzj1cia7kAICLeT/tHdHE8wDlkCQnp93k9vE4z64aTAzPLmw6s3MlAv/8mWxJ2bFoG\n9stkXQ0VHZdanQ6cFxHDcj9LRMTxwPPAsukumxWjco+fI0suAJCktH9GN893ObCepHWB7YE/VPFa\nzawLTg7MLO8usi/v4yUtIWkxSZ8FlgTmArPTeILDOpz3IrBabvt8YAdJW0salOpplzQyIp4m62I4\nWtIikjYBdsidexGwnaQtJC0MHAq8DdzeVdAR8RZwCWksQ0Q8U8M1MBvwnByY2Qci4j2yL+rVyQYZ\nPks2BuAnwPrAbLKBhpd1OPVnwI9SF8J3I2I6MBH4ATCTrCXhMD78P2dPYBPgFbJBixeSJQBExGNk\nLRO/Al5O8ewQEe/0EP45wFjcpWBWM994ycxKJ+lC4NGIOKqGOlYGHgU+FhGv1y04swHILQdm1u8k\nfSatnbCQpAlkrQyX11DfQsB3gElODMxq59XFzKwMHyPrmliOrOvigIi4b0EqkrQE2ZiHp8mmMZpZ\njdytYGZmZgXuVjAzM7MCJwdmZmZW4OTAzMzMCpwcmJmZWYGTAzMzMyv4/x6sB/qAcpiIAAAAAElF\nTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f6ab0dfc6d8>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_count_labels_per_category(df_stats_train)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgEAAAGHCAYAAAAp54E5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcHFW5//HPl4QlJEBYwgAhEDaR\nSFAhsok6AWSHcAUELrJdNBdEQEEgrqDiT1y4CIpcUTBBlICAiCwCggMisgWFsF4iWwhb2EIS1sDz\n+6POSGcyM6lkuruqp77v16tf03Wq+vTTZ3qmnz7n1ClFBGZmZlY9SxQdgJmZmRXDSYCZmVlFOQkw\nMzOrKCcBZmZmFeUkwMzMrKKcBJiZmVWUkwCznCRNlHRKQc8tSb+S9LKkO4qIwcz6HycB1rIkPS7p\neUmDa8o+K6mjwLAaZRvgk8CaEbF50cE0mqSRkkLSwKJjqRdJ7ZKeKjoOs1pOAqzVDQCOKTqIRSVp\nwCI+ZG3g8YiY24h4rHv9JQnpL6/D6s9JgLW6HwJfljS0647uvk1K6pD02XT/EEl/k3S6pFckPSpp\n61Q+PfUyHNyl2lUkXS9ptqSbJK1dU/f7076XJD0s6dM1+yZKOlvS1ZLmAmO7iXcNSVekx0+T9LlU\nfhjwS2ArSXMkfau7hpD0OUkPptgekLRpKt8ove5XJN0vaY8ucf1M0jWp7r9JWk3Sj9PQw0OSPlxz\n/OOSjpd0r6S5ks6V1JYeP1vSnyWtWHP8lpJuTc99j6T2Lr+L76TnnC3pOkmrpN03p5+vpLi2krR+\navNZkl6QdFEP7dD5ex8v6WlJz0j6cs3+JSRNkPQvSS9KuljSSl0ee5ikJ4Ebe3iOcZL+KenVVM9O\nqfzQmt/Bo5L+O5UPBq4B1kivZ076ffcYS3rcQZKeSPu+kdp/+7Rv6fR7ejrdfixp6bSvXdJTkk6U\n9CzwK0n3Sdq9pu4lUzt+GKuuiPDNt5a8AY8D2wOXAaekss8CHen+SCCAgTWP6QA+m+4fAswDDiXr\nUTgFeBI4C1ga2AGYDQxJx09M2x9P+88Abkn7BgPTU10DgQ8DLwCjah47C/goWfK9TDev52bgZ8Ay\nwIeAmcC2NbHe0ktb7APMAD4CCFifrPdgSWAa8FVgKWDb9Bo2rInrBWCz9Lw3Ao8BB9W0yV+6tPlt\nQBswHHgeuDu93s7Hn5SOHQ68COySXvMn0/awmt/Fv4D3AYPS9qm9/O4uBL7W2X7ANj20RedjL0y/\nl9GpLbdP+49Jr2HN9Hv8OXBhl8eenx47qJv6N0+/y0+mWIYD70/7dgXWS7+DTwCvAZumfe3AU13q\n6i2WUcAcsqGgpYAfAW/XvI5vp8euCgwDbgW+U/Nc84Dvp3oHAScAF9U89zhgatF/x74Veys8AN98\nW9wb7yUBG6d/ysNY9CTgkZp9o9PxbTVlLwIfSvcnApNr9g0B3gFGAPsCf+0S38957wNxInB+L69l\nRKpruZqy7wETa2LtLQm4Fjimm/KPAc8CS9SUXQicXBPXL2r2HQU82KVNXunS5gfUbF8KnN3l8Zen\n+ycCv+4mzoNrfhdfr9n3eeBPvfzuzgfOIZsX0dv7ovOx768p+wFwbrr/ILBdzb7VyT5cB9Y8dt1e\n6v85cHrO9+jlnb8Xuk8Ceovlm6SEIO1bFniL95KAfwG71OzfkWzIqPO53qIm2QTWIEsAl0/blwAn\nNPNv1rfy3TwcYC0vIu4DrgQmLMbDn6u5/3qqr2vZkJrt6TXPOwd4ieyf69rAFqnb+xVJrwAHAKt1\n99hurAG8FBGza8qeIPuWmccIsg+F7uqdHhHv9lJv19fb2+tflOPXBvbp0ibbkH3QdXq25v5r3TxX\nrRPIvmHfkYY1/quXY2H+9n6CrC064/p9TUwPkiVgbT08tque2hpJO0u6LQ3pvELWC7JKd8fmiGUN\n5n+/vUaWlHZaI72u7l4jwMyIeKPm8U8DfwP2UjZ8tjPwm15iswrwZBHrL04i65Y+raascxLdssCr\n6X7th/LiGNF5R9IQYCXgabJ/1jdFxCd7eWxvl+x8GlhJ0nI1icBaZF38eUwn64burt4RkpaoSQTW\nAv4vZ719MZ2sJ+Bzi/HYBdoqIp4FOudJbAP8WdLNETGthzpGAA+l+2uRtUVnXP8VEX/r+gBJI3t6\n/hrdtnUaj7+UbCjlDxHxtqTLyRKXnursLZZngA1rtgcBK9cc8jRZEnF/2q59jT093ySy3rKBwN8j\nIu/7y/op9wRYv5A+CC4Cjq4pm0n2IfoZSQPSN8fuPigXxS6StpG0FPAd4LaImE7WE/E+SQemCVdL\nSvqIpI1yxj+dbEz3e5KWkbQJcBhwQc64fkk2QXIzZdZXNmnxdrJv2CekmNqB3YHJi/KiF9MFwO6S\ndkztv0yasLZmjsfOBN4F1u0skLRPzWNfJvuQe7ebx3b6hqRlJX2AbK5G50TC/wW+m9oHScMkjVuE\n13UucKik7dLEvuGS3k82br90in2epJ3J5pV0eg5YWdIKNWW9xXIJWfttnd5vJ/NeQgHZsM7X02NW\nIRs+WNj75XJgU7K5COcvwmu2fspJgPUn3yabzFXrc8DxZN2oHyD7oO2L35L1OrxENpnuMwDp2/sO\nwH5k38ae5b1JWXntTzYm/TTwe7L5BH/O88CI+B3w3RTfbLJ/9itFxFtkH/o7k00A/BlwUEQ81FNd\n9ZISm3FkkxJnkn3rPZ4c/3dS1/d3gb+lrvItySY93i5pDnAF2Vj7o71UcxPZpMgbgB9FxHWp/Iz0\n+OskzSabXLfFIryuO8iSitPJ5qLcBKyd3gNHAxeTJSn/mZ6n83EPkX1wP5pe0xq9xRIR95PNsZgM\nPEM2SfB54M1U5SnAXcC9wFSynrBeF7OKiNfJeivWIZtQaxWniN56vczMWkvq0n8MWDIi5hUbTf2k\n4adXgA0i4rE+1PNN4H0R8Zm6BWctyz0BZmYlJWn3NKQxmOwUwalkZ2gsbn0rkQ0znVOfCK3VOQkw\nMyuvcWTDQ08DGwD7xWJ23ypbfGo6cE1E3Lyw460aPBxgZmZWUe4JMDMzq6h+v07AKqusEiNHjiw6\njAXMnTuXwYO7TmS3rtxO+bid8nNb5eN2yq+MbTVlypQXImLYwo7r90nAyJEjueuuu4oOYwEdHR20\nt7cXHUbpuZ3ycTvl57bKx+2UXxnbStITCz/KwwFmZmaV5STAzMysopwEmJmZVZSTADMzs4pyEmBm\nZlZRTgLMzMwqqilJgKTzJD0v6b5u9h0nKdKlMEmXQT1T0jRJ90ratObYgyU9km4HNyN2MzOz/qpZ\nPQETgZ26FkoaQXb51SdrincmWyN7A2A8cHY6diWyS7huAWwOnCRpxYZGbWZm1o81JQlIF6t4qZtd\npwMnALUXMBgHnB+Z24ChklYHdgSuj4iXIuJl4Hq6SSzMzMwsn8JWDJQ0DpgREfdIqt01nOxKV52e\nSmU9lXdX93iyXgTa2tro6OioX+B1MmfOnFLGVTZup3zcTvm5rfJxO+XXym1VSBIgaVngq2RDAXUX\nEeeQrpc9ZsyYKNtyjlDOZSbLyO2Uj9spP7dVPm6n/Fq5rYrqCVgPWAfo7AVYE7hb0ubADGBEzbFr\nprIZQHuX8o4mxPpvIydcVbe6jhs9j0PqUN/jp+5ah2jMzKyKCjlFMCKmRsSqETEyIkaSde1vGhHP\nAlcAB6WzBLYEZkXEM8C1wA6SVkwTAndIZWZmZrYYmnWK4IXA34ENJT0l6bBeDr8aeBSYBvwC+DxA\nRLwEfAe4M92+ncrMzMxsMTRlOCAi9l/I/pE19wM4sofjzgPOq2twZmZmFeUVA83MzCrKSYCZmVlF\nOQkwMzOrKCcBZmZmFeUkwMzMrKKcBJiZmVWUkwAzM7OKchJgZmZWUU4CzMzMKspJgJmZWUU5CTAz\nM6soJwFmZmYV5STAzMysopwEmJmZVZSTADMzs4pyEmBmZlZRTgLMzMwqykmAmZlZRTkJMDMzqygn\nAWZmZhXlJMDMzKyinASYmZlVlJMAMzOzinISYGZmVlFOAszMzCrKSYCZmVlFOQkwMzOrKCcBZmZm\nFdWUJEDSeZKel3RfTdkPJT0k6V5Jv5c0tGbfVyRNk/SwpB1ryndKZdMkTWhG7GZmZv1Vs3oCJgI7\ndSm7Htg4IjYB/g/4CoCkUcB+wAfSY34maYCkAcBZwM7AKGD/dKyZmZkthqYkARFxM/BSl7LrImJe\n2rwNWDPdHwdMjog3I+IxYBqwebpNi4hHI+ItYHI61szMzBbDwKIDSP4LuCjdH06WFHR6KpUBTO9S\nvkV3lUkaD4wHaGtro6Ojoy5BHjd63sIPyqltUH3qq9drK6s5c+b0+9dYD26n/NxW+bid8mvltio8\nCZD0NWAe8Jt61RkR5wDnAIwZMyba29vrUu8hE66qSz2QJQCnTe178z9+QHvfgymxjo4O6vX768/c\nTvm5rfJxO+XXym1VaBIg6RBgN2C7iIhUPAMYUXPYmqmMXsrNzMxsERV2iqCknYATgD0i4rWaXVcA\n+0laWtI6wAbAHcCdwAaS1pG0FNnkwSuaHbeZmVl/0ZSeAEkXAu3AKpKeAk4iOxtgaeB6SQC3RcTh\nEXG/pIuBB8iGCY6MiHdSPV8ArgUGAOdFxP3NiN/MzKw/akoSEBH7d1N8bi/Hfxf4bjflVwNX1zE0\nMzOzyvKKgWZmZhXlJMDMzKyinASYmZlVlJMAMzOzinISYGZmVlFOAszMzCrKSYCZmVlFOQkwMzOr\nKCcBZmZmFeUkwMzMrKKcBJiZmVWUkwAzM7OKchJgZmZWUU4CzMzMKspJgJmZWUU5CTAzM6soJwFm\nZmYV5STAzMysopwEmJmZVZSTADMzs4pyEmBmZlZRTgLMzMwqykmAmZlZRTkJMDMzqygnAWZmZhXl\nJMDMzKyinASYmZlVlJMAMzOzimpKEiDpPEnPS7qvpmwlSddLeiT9XDGVS9KZkqZJulfSpjWPOTgd\n/4ikg5sRu5mZWX/VrJ6AicBOXcomADdExAbADWkbYGdgg3QbD5wNWdIAnARsAWwOnNSZOJiZmdmi\na0oSEBE3Ay91KR4HTEr3JwF71pSfH5nbgKGSVgd2BK6PiJci4mXgehZMLMzMzCyngQU+d1tEPJPu\nPwu0pfvDgek1xz2VynoqX4Ck8WS9CLS1tdHR0VGXgI8bPa8u9QC0DapPffV6bWU1Z86cfv8a68Ht\nlJ/bKh+3U36t3Fa5kgBJ+wP/jIgHJW0I/AJ4BzgiIh7qaxAREZKir/XU1HcOcA7AmDFjor29vS71\nHjLhqrrUA1kCcNrUvudgjx/Q3vdgSqyjo4N6/f76M7dTfm6rfNxO+bVyW+UdDjiF97rzfwTcAdwE\n/KwPz/1c6uYn/Xw+lc8ARtQct2Yq66nczMzMFkPeJGBYRDwnaRlgG+BrwLeBD/Xhua8AOmf4Hwz8\noab8oHSWwJbArDRscC2wg6QV04TAHVKZmZmZLYa8/dEzJa0PjAbujIg3JS0LKM+DJV0ItAOrSHqK\nbJb/qcDFkg4DngA+nQ6/GtgFmAa8BhwKEBEvSfoOcGc67tsR0XWyoZmZmeWUNwn4DjCFbB7Avqls\ne+CePA+OiP172LVdN8cGcGQP9ZwHnJfnOc3MzKx3uZKAiJgo6eJ0/7VUfBuwX6MCMzMzs8ZalHUC\nBgF7STohbQ+k2FMMzczMrA9yJQGSPgE8DBwAfCMVb0Bazc/MzMxaT96egB8D+0bETkDnCje3ky3f\na2ZmZi0obxIwMiJuSPc7F/V5Cw8HmJmZtay8ScADknbsUrY9MLXO8ZiZmVmT5P0mfxxwpaSrgEGS\nfg7sTnaxHzMzM2tBuXoC0tX8NgHuJztP/zFg84i4s9cHmpmZWWnlvYDQ0sDMiPhBTdmSkpaOiDcb\nFp2ZmZk1TN45AdcDm3Up2wyv3W9mZtay8iYBo8lOCax1B/DB+oZjZmZmzZI3CZgFtHUpawPm1jcc\nMzMza5a8ScClwG8lbSxpWUmjgfOBixsXmpmZmTVS3iTga8CDZEMAs8kuHvQw8NUGxWVmZmYNlvcq\ngm8AR0r6ArAK8EK65K+ZmZm1qNzL/kpaAdgQGJK2AYiIGxsSmZmZmTVU3nUCDgHOAuYAr9XsCmDd\n+odlZmZmjZa3J+C7wN4RcU0jgzEzM7PmyTsxcCBwXSMDMTMzs+bKmwR8H/i6pLzHm5mZWcnlHQ74\nErAacIKkF2t3RMRadY/KzMzMGi5vEvCZhkZhZmZmTZd3nYCbGh2ImZmZNVeuMX5JS0v6rqRHJc1K\nZTukxYPMzMysBeWd6Hc6sDFwANnaAAD3A0c0IigzMzNrvLxzAv4DWD8i5kp6FyAiZkga3rjQzMzM\nrJHy9gS8RZeEQdIw4MXuDzczM7Oyy5sE/A6YJGkdAEmrAz8FJjcqMDMzM2usvEnAV4HHgKnAUOAR\n4GngWw2Ky8zMzBpsoUlAWiVwG2BCRAwB2oDlIuJLEfFWXwOQ9CVJ90u6T9KFkpaRtI6k2yVNk3SR\npKXSsUun7Wlp/8i+Pr+ZmVlVLTQJiIh3gT9ExJtpe2ZExEIelkuaWHg0MCYiNgYGAPuRLVN8ekSs\nD7wMHJYechjwcio/PR1nZmZmiyHvcMDNkrZsUAwDgUGSBgLLAs8A2wKXpP2TgD3T/XFpm7R/O0lq\nUFxmZmb9mvJ8qZf0M2B/4A/AdN5bK4CI+GafApCOIbtU8etkVyo8BrgtfdtH0gjgmojYWNJ9wE4R\n8VTa9y9gi4h4oUud44HxAG1tbZtNnlyf+YtTZ8yqSz0AbYPgudf7Xs/o4Sv0vZISmzNnDkOGDCk6\njNJzO+XntsrH7ZRfGdtq7NixUyJizMKOy7tOwCDg8nR/zZryPg0LSFqR7Nv9OsArZGch7NSXOgEi\n4hzgHIAxY8ZEe3t7X6sE4JAJV9WlHoDjRs/jtKl5m79njx/Q3vdgSqyjo4N6/f76M7dTfm6rfNxO\n+bVyWy30UyhNDPw18LfOeQF1tD3wWETMTM91GfBRYKikgRExjyzpmJGOnwGMAJ5Kwwcr4LUKzMzM\nFssiTwyssyeBLSUtm8b2twMeAP4C7J2OOZhsGALgirRN2n9jvSYpmpmZVU2hEwMj4nayCX53k61B\nsARZN/6JwLGSpgErA+emh5wLrJzKjwUm1DsmMzOzqsg7KP0EcI2kuk8MjIiTgJO6FD8KbN7NsW8A\n+/Tl+czMzCzT14mBZmZm1qJyJQERcWijAzEzM7PmypUESFq3p30R8Wj9wjEzM7NmyTscMI1sHkDt\n6nyd8wIG1DUiswoZPWl0Xeo5YsgRHDXpqLrUNfXgqXWpx8zKL+9wwHxnEUhajWwy318bEZSZmZk1\nXt5TBOcTEc8CXwS+V99wzMzMrFkWKwlINiS74I+ZmZm1oLwTA//K/NcJWBb4APDtRgRlZmZmjZd3\nYuAvu2zPBe6JiEfqHI+ZmZk1Sd6JgZMaHYiZmZk1V645AZIuk/SxLmUfk3RJY8IyMzOzRss7MfAT\nwK1dyv4OjK1vOGZmZtYseZOAN4DBXcqGAG/XNxwzMzNrlrxJwLXAzyUtD5B+/hT4U6MCMzMzs8bK\ne3bAccAFwEuSXgJWAq4BDmxUYGZmtujOOvzGutSz6tZz61bXkf+7bV3qsfrLe3bAy8CuabngEcD0\ntGqgmZmZtai8iwXtADweEf8HPJvKNgTWiojrGxifmZmZNUjeOQFnAbO7lM1O5WZmZtaC8iYBq0bE\nM13KngFWq3M8ZmZm1iR5k4BHJXWd2dEOPFbfcMzMzKxZ8p4dcDJwmaRzgX8B6wGHppuZmZm1oFw9\nARHxB2AHsgWDdk0/d0zlZmZm1oLy9gQQEXcAdzQwFjMzM2uihfYESBopaaKkGZLeTD8nSVq3GQGa\nmZlZY/SaBEjaCLgbWBX4GrBH+jkMuCvtNzMzsxa0sOGAU4GzIuIbXconSjoF+AGwe0MiMzMzs4Za\nWBLwceDgHvadhk8RNDMza1kLmxMwgJ4vF/x22m9mZmYtaGFJwJ30vBbAIcBdfQ1A0lBJl0h6SNKD\nkraStJKk6yU9kn6umI6VpDMlTZN0r6RN+/r8ZmZmVbWw4YBvANemiwVdQrZU8OrAPmTDBDvWIYYz\ngD9FxN6SlgKWBb4K3BARp0qaAEwATgR2BjZIty2As9NPMzMzW0S99gRExK1kiwR9ELgBeCj9/CCw\nU9q/2CStQDbv4Nz0fG9FxCvAOGBSOmwSsGe6Pw44PzK3AUMlrd6XGMzMzKpKEZHvQGkQsBLwckS8\nVpcnlz4EnAM8QJZYTAGOAWZExNB0jNJzDpV0JXBqRNyS9t0AnBgRd3WpdzwwHqCtrW2zyZMn1yNc\nps6YVZd6ANoGwXOv972e0cNX6HslJTZnzhyGDBlSdBgN88CLD9SlnmEDhjHznZl1qWvUyqPqUk9Z\n9ff31Mwnu17wdfEMHPwu8+bmvbxM74attVxd6imrMr6nxo4dOyUixizsuEVZMfB1YEafour++TcF\njoqI2yWdQdb1X/u8ISlfpvLeY84hSy4YM2ZMtLe31yXYQyZcVZd6AI4bPY/TpuZu/h49fkB734Mp\nsY6ODur1+yujoyYdVZd6jhhyBGfPObsudU3da2pd6imr/v6eOuvwG+tSz6pbz+X5WwfXpa59Dmqv\nSz1l1crvqfqkeYvvKeCpiLg9bV9ClhQ819nNn34+n/bPAEbUPH5N6p+YmJmZVUKhSUBEPAtMTxMP\nAbYjGxq4gvfWJzgY6LxQ0RXAQeksgS2BWRHxTDNjNjMz6y967I+W9MOIOD7d3zYi6tPHtKCjgN+k\nMwMeJTslcQngYkmHAU8An07HXg3sAkwDXsOXMjYzM1tsvQ1KjweOT/cvB5ZvRAAR8U+gu8kL23Vz\nbABHNiIOMzOzquktCbhH0iVk3fNLS/p2dwdFxDcbEpmZmZk1VG9JwN5kvQFrA2L+CXmdFmnWvpmZ\nmZVHj0lARDwPnAIgaWBEePzdzMysH8l1onpEHJrW798dGE52Wt6VEfFSI4MzMzOzxsl1iqCkrYB/\nAYcDmwD/DUxL5WZmZtaC8i5Z92Pg8xHx7/V3Je0LnAl8pBGBmZmZWWPlXSzofcDFXcouAdavbzhm\nZmbWLHmTgEeA/bqU7UM2RGBmZmYtKO9wwBeBKyUdTbaC30hgA2C3BsVlZmZmDZb37IBbJa0H7Aqs\nAfwRuNpnB5iZmbWuRbmU8MvABQ2MxczMzJqo6EsJm5mZWUGcBJiZmVWUkwAzM7OKyp0ESFq7kYGY\nmZlZcy1KT8A/ANJpgmZmZtbiej07QNIUYApZAjAgFZ9MtlywmZmZtbCF9QTsDVwHrA0sK+luYGlJ\nYyWt0PDozMzMrGEWlgQMiIhLImICMBsYBwg4CvinpEcaHaCZmZk1xsIWC/qNpLWAB4BlgBWBNyLi\nUwCSVmpwfGZmZtYgvSYBEbGFpIHAaOAW4KfAcpLOBu5ONy8dbGZm1oIWenZARMyLiH8Ab0XEx4G5\nQAfZBYS+39jwzMzMrFFyXzsA+FL6GRFxEXBRA+IxMzOzJsm9TkBETEx3121MKGZmZtZMi7xscLqa\noJmZmbU4XzvAzMysopwEmJmZVZSTADMzs4oqRRIgaYCkf0i6Mm2vI+l2SdMkXSRpqVS+dNqelvaP\nLDJuMzOzVlaKJAA4BniwZvv7wOkRsT7wMnBYKj8MeDmVn47XKTAzM1tshScBktYEdgV+mbYFbAtc\nkg6ZBOyZ7o9L26T926XjzczMbBEpIooNQLoE+B6wHPBl4BDgtvRtH0kjgGsiYmNJ9wE7RcRTad+/\ngC0i4oUudY4HxgO0tbVtNnny5LrEOnXGrLrUA9A2CJ57ve/1jB7evy/mOGfOHIYMGVJ0GA3zwIsP\n1KWeYQOGMfOdmXWpa9TKo+pST1n19/fUzCdn16WegYPfZd7c+nxPHLbWcnWpp6zK+J4aO3bslIgY\ns7DjFmXFwLqTtBvwfERMkdRer3oj4hzgHIAxY8ZEe3t9qj5kwlV1qQfguNHzOG1q35v/8QPa+x5M\niXV0dFCv318ZHTXpqLrUc8SQIzh7ztl1qWvqXlPrUk9Z9ff31FmH31iXelbdei7P3zq4LnXtc1B7\nXeopq1Z+TxWaBAAfBfaQtAvZVQqXB84AhkoaGBHzgDWBGen4GcAI4Kl0YaMVgBebH7aZmVnrK3RO\nQER8JSLWjIiRwH7AjRFxAPAXYO902MHAH9L9K9I2af+NUfR4hpmZWYsqfGJgD04EjpU0DVgZODeV\nnwusnMqPBSYUFJ+ZmVnLK3o44N8iooPsEsVExKPA5t0c8wawT1MDMzMz66fK2hNgZmZmDeYkwMzM\nrKKcBJiZmVWUkwAzM7OKchJgZmZWUU4CzMzMKspJgJmZWUU5CTAzM6soJwFmZmYV5STAzMysopwE\nmJmZVZSTADMzs4pyEmBmZlZRTgLMzMwqykmAmZlZRTkJMDMzqygnAWZmZhXlJMDMzKyinASYmZlV\nlJMAMzOzinISYGZmVlFOAszMzCrKSYCZmVlFOQkwMzOrKCcBZmZmFeUkwMzMrKKcBJiZmVWUkwAz\nM7OKchJgZmZWUYUmAZJGSPqLpAck3S/pmFS+kqTrJT2Sfq6YyiXpTEnTJN0radMi4zczM2tlRfcE\nzAOOi4hRwJbAkZJGAROAGyJiA+CGtA2wM7BBuo0Hzm5+yGZmZv1DoUlARDwTEXen+7OBB4HhwDhg\nUjpsErBnuj8OOD8ytwFDJa3e5LDNzMz6BUVE0TEAIGkkcDOwMfBkRAxN5QJejoihkq4ETo2IW9K+\nG4ATI+KuLnWNJ+spoK2tbbPJkyfXJcapM2bVpR6AtkHw3Ot9r2f08BX6XkmJzZkzhyFDhhQdRsM8\n8OIDdaln2IBhzHxnZl3qGrXyqLrUU1b9/T0188nZdaln4OB3mTe3Pt8Th621XF3qKasyvqfGjh07\nJSLGLOy4gc0IZmEkDQEuBb4YEa9mn/uZiAhJi5SpRMQ5wDkAY8aMifb29rrEeciEq+pSD8Bxo+dx\n2tS+N//jB7T3PZgS6+jooF6/vzI6atJRdanniCFHcPac+oyOTd1ral3qKav+/p466/Ab61LPqlvP\n5flbB9elrn0Oaq9LPWXVyu9aUoGVAAAT3klEQVSpoucEIGlJsgTgNxFxWSp+rrObP/18PpXPAEbU\nPHzNVGZmZmaLqOizAwScCzwYEf9Ts+sK4OB0/2DgDzXlB6WzBLYEZkXEM00L2MzMrB8pejjgo8CB\nwFRJ/0xlXwVOBS6WdBjwBPDptO9qYBdgGvAacGhzwzUzM+s/Ck0C0gQ/9bB7u26OD+DIhgZlZmZW\nEYXPCTAzM7NiOAkwMzOrKCcBZmZmFeUkwMzMrKKcBJiZmVWUkwAzM7OKchJgZmZWUUUvFmT90cl1\nvKjRht+Ck8fVp66T63cBKDOz/sBJgJmZVc5p++5Wt7rW3HFPTjv7R32u57iLrqxDNIvGwwFmZmYV\n5STAzMysopwEmJmZVZSTADMzs4pyEmBmZlZRTgLMzMwqykmAmZlZRTkJMDMzqygnAWZmZhXlJMDM\nzKyinASYmZlVlJMAMzOzinISYGZmVlFOAszMzCrKlxI2s9J78P0b1a2uN476Ag8efkSf69nooQfr\nEI1ZsdwTYGZmVlFOAszMzCrKSYCZmVlFOQkwMzOrqJZMAiTtJOlhSdMkTSg6HjMzs1bUckmApAHA\nWcDOwChgf0mjio3KzMys9bRcEgBsDkyLiEcj4i1gMjCu4JjMzMxajiKi6BgWiaS9gZ0i4rNp+0Bg\ni4j4Qs0x44HxaXND4OGmB7pwqwAvFB1EC3A75eN2ys9tlY/bKb8yttXaETFsYQf1y8WCIuIc4Jyi\n4+iNpLsiYkzRcZSd2ykft1N+bqt83E75tXJbteJwwAxgRM32mqnMzMzMFkErJgF3AhtIWkfSUsB+\nwBUFx2RmZtZyWm44ICLmSfoCcC0wADgvIu4vOKzFUerhihJxO+XjdsrPbZWP2ym/lm2rlpsYaGZm\nZvXRisMBZmZmVgdOAszMzCrKSYCZmVlFOQkws35N0kfzlJlVkZOAJpH0H5JWqNkeKmnPImMqK0mD\nJS1Rs72EpGWLjKnM3DYL9ZOcZZUnaZ88ZZaRNEjShkXH0RdOAprnpIiY1bkREa8AJxUYT5ndANR+\nsC0L/LmgWEpL0taSHgAeStsflPSzgsMqDUlbSToOGCbp2JrbyWSnF9uCvpKzrPIk7Q78E/hT2v6Q\npJZbs6bl1gloYd0lXG7/7i0TEXM6NyJijr/tdut0YEfSYlkRcY+kjxcbUqksBQwh+ztbrqb8VWDv\nQiIqKUk7A7sAwyWdWbNreWBeMVGV3slkF7TrAIiIf0pap8iAFoc/hJrnLkn/Q3YZZIAjgSkFxlNm\ncyVtGhF3A0jaDHi94JhKKSKmS6oteqeoWMomIm4CbpI0MSKeKDqeknua7P/RHsz/f2k28KVCIiq/\ntyNiVpe/v5ZbeMdJQPMcBXwDuChtX0+WCNiCvgj8TtLTgIDVgH2LDamUpkvaGghJSwLHAA8WHFMZ\nvSbph8AHgGU6CyNi2+JCKpeIuAe4R9IFEeFv/vncL+k/gQGSNgCOBm4tOKZF5hUDrZTSh1rnhJuH\nI+LtIuMpI0mrAGcA25MlS9cBx0TEi4UGVjKSriNLvr8MHA4cDMyMiBMLDaxEJE2ll2+xEbFJE8Np\nCWmI8mvADqnoWuA7EfFmcVEtOicBDSbpxxHxRUl/pJs/sojYo4CwSknSthFxo6RPdbc/Ii5rdkzW\n+iRNiYjNJN3b+WEm6c6I+EjRsZWFpLV72+/hlAVJ2icifrewsrLzcEDj/Tr9/FGhUbSGTwA3Art3\nsy8AJwE1JA0DPgeMpOZvOSL+q6iYSqqzF+kZSbuSjX+vVGA8peMP+cXyFaDrB353ZaXmnoAmkbRq\nRDzfpWzDiHi4qJistUm6Ffgr2USuf08IjIhLCwuqhCTtRtZOI8jWB1ge+FZEtNzpXI0maTbv9Vgu\nBSwJzI2I5YuLqlxqzqT4NO/N8YLsfTUqIjYvJLDF5J6A5vmrpG9ExMUA6fzlw4BRxYZVPpJ+DXyh\nc12F1FV5XkRsV2xkpbOsx7UXLiKuTHdnAWOLjKXsIuLfp1Iqm/Y+DtiyuIhK6WngLvrJmRTuCWgS\nSauTXXP6DaCNbBb3cbXnw1tG0n+T/TEdCwwHjidrqz8WGljJSDoFuDUiri46ljKT9D7gbKAtIjaW\ntAmwR0ScUnBoLUHSPyLiw0XHUTaSluwPE5adBDSRpCPJxozeBfaLiJY7naRZJG0D/AV4AfhwRDxb\ncEilk7puBwNvpZuAcNft/CTdRJZI/rzzw0zSfRGxcbGRlU+XSblLAGOAT0TEVgWFVFrptMDvkfXm\n1p56um5hQS0GDwc0iaQ/k3UjbUw2NnmupJsj4svFRlY+kg4kW1PhIGAT4GpJh6ZzmS2p7bq1Xi0b\nEXd0WdTF58J3r3ZS7jzgcbIhAVvQr8iWfj+dbJjpUFpwKX4nAc3z04i4PN1/JS3y4jW5u7cXsE2a\nSHmhpN8DEwF3SdZIY7YHAOtExHckjQBWj4g7Cg6tbF6QtB5pwpukvYFnig2pnCLi0KJjaCGDIuIG\nSUpnV5wsaQrwzaIDWxQeDmgiSW1A57nJd3Q9W8B6JmmpiHir6DjKRNLZZENL20bERpJWBK7z+e/z\nk7Qu2XycrYGXgceAA3xa3IIk/QA4hWyZ7j+R9cR9KSIuKDSwEkpn52wDXEJ2avMM4NSIaKmrCrZc\n10WrkvRp4A5gH7JTS25P30isC0lrSvq9pJmSnpd0KbBq0XGV0BYRcSTZZFMi4mWy07osSZekHhMR\n2wPDgPdHxDZOAHq0Q0S8CuxGNhSwPtl8ClvQMWRXOD0a2Aw4kGw1ypbi4YDm+Rrwkc5v/2mhlz+T\nZZE2v18BvyVLmAA+k8o+WVhE5fS2pAG81809jKxnwJKIeFfSCcDFETG36HhaQOdnwq7A77q5QI4l\nEXFnujuHbD5AS3IS0DxLdOn+fxH3xPRkWET8qmZ7oqQvFhZNeZ0J/B5YVdJ3yS6P+/ViQyqlP0v6\nMtnCLv9OBCLipeJCKq0rJT1ENhxwREos3yg4plJKp54eD6zN/Ct2ttSFqTwnoEnSWNsHgQtT0b7A\nvV7sZUGSbiD75t/ZVvsDh3qxoAVJej+wHdnpgTdEhK8i2IWkx7opjlY7latZJK0EzIqId9JFcpb3\nKboLknQP8L8suGJnS10i3klAk0j6PnA72UQSyJYx3dJJwILSCoE/AbYi6+q+FTgqIqYXGljJSNoS\nuD8iZqft5YGNIuL2YiOzVpbOXBrJ/N9uzy8soJLqvDBV0XH0lZOAJpF0d0Rs2qXs31c1s/dI+mhE\n/G1hZVUn6R/AppH+iNMkuLu6vs/MH2x5pSW71wP+yXvfbiMiji4uqnJJPSWQTQh8nmxI7t+XD261\nYSbPCWgwSUcAnwfWlXRvza7lAH+ode8nQNcPsu7Kqk5Rk8WnSXD+m+6ipw82wEnAgsaQXQTH3w57\nNoXs/dM5Y7L27IkAWmqYyf8wGu+3wDVky0tOqCmf3WoZY6NJ2orsXO5hko6t2bU8MKCYqErtUUlH\nk62LD1my+WiB8ZSVP9jyuw9YDS+m1KOIWCfPcZI+GRHXNzqevnIS0GDpSnizyCa3We+WAoaQvS9r\nl8R9lWzmu83vcLIzBL5O9g3kBmB8oRGVkz/Y8lsFeEDSHczfxb1HcSG1rO8DpU8CPCfASkfS2r0t\n5iLpJxFxVDNjstYj6Y9kydFywIfIFuvyB1svJH2iu/KIuKnZsbS6Vrn6onsCrHRyrOb20aYEUnJe\n4nWhfkQ2bvt9YM+a8s4y68If9nXVEt+wnQSYta4dIuIESf9BtsTrp4CbAScBvPeBlq77Pt+Hm6RB\nxURVTpJuiYht0uWpaz+8fHnqfs5JgFnr8hKvvfCZOflFxDbppy9PXT+PFx1AHk4CrBX5ky7jJV57\n5zNzrGHSZYPPA36bLt41n4j4VPOjWnSeGGilJWnZiHitm/JDImJiASGVjpd4NSuGpPXJLhy0L3AX\n2VLn17XaqahOAqx00upuvwSGRMRakj4I/HdEfL7g0EpF0jJk3d3bkI3j3gKcHRHuDTBrkrRS525k\n63W8Q5YMnNEqvU2+ip2V0enAjmRXWiQi7gE+XmhE5XQ+8AGy1RR/CowCfl1oRGYVImkT4DTgh8Cl\nZJc/fxW4sci4FoXnBFgpRcT0LpPc3unp2ArbOCJG1Wz/RdIDhUVjViFpTsArwLnAhIjoXIPidkkt\ncxqzkwAro+lpSCAkLQkcA/gSuQu6W9KWEXEbgKQtyMYmzazx9omI+ZbplrRORDzWKpMCwXMCrIQk\nrQKcAWxPdibAdcAxEfFioYGVhKSpZHMAlgQ2BJ5M22sDD3XpHTCzBujhyrAtd3lh9wRYqUgaABwY\nEQcUHUuJ7VZzf0XgY+n+zWTdk2bWIJLeTzYXZwVJtd/4lweWKSaqxeeJgVYqEfEO8J9Fx1FmEfFE\nWlp5T7KJgKsAw9J9r4dv1lgbkiXiQ4Hda26bAp8rMK7F4uEAKx1Jp5N1dV8EzO0sj4i7CwuqhNIq\neFtFxNy0PRj4e0RsUmxkZv2fpK0i4u9Fx9FXHg6wMvpQ+vntmrIAti0gljIT85818Q5eTdGsoSSd\nEBE/AP5T0gKXiI+IowsIa7E5CbDSiYixRcfQIn5FdjrS79P2nmSnK5lZ43SeqdQvzsTxcICVjqQ2\n4P8Ba0TEzpJGkXV7+wOuC0mbkq0YCPDXiPhHkfGYVYWkfSLidwsrKzsnAVY6kq4h+5b7tYj4oKSB\nwD8iYnTBoZmZAT2eIrhAWdl5OMDKaJWIuFjSVwAiYp4krxhoZoWTtDOwCzBc0pk1u5YH5hUT1eJz\nEmBlNFfSymSTAZG0JTCr2JDMzAB4mmw+wB7AlJry2cCXComoDzwcYKUjaTPgTGBj4D6yc+D3joh7\nCw3MzCyRtGREvF10HH3lJMBKKc0D2JDslLeH+8Mfm5n1H+kiQSeTLdc9kOx/VUTEukXGtaicBFjp\npEVwJgMXRcS/io7HzKwrSQ+Rdf9PoWa9jla7xomTACsdSWsD+6bbu2QrB14cEU8WGpiZWSLp9ojY\noug4+spJgJWapA2AbwAHRMSAouMxMwOQdCowALgMeLOzvNWWN/fZAVZKXXoD3gFOKDYiM7P5dPYC\njKkpa7nlzd0TYKUj6XayCwj9jmxewKMFh2Rm1i85CbDSkbRhRDxcdBxmZj3pL8ubL1F0AGbdeEXS\nuWn5YCSNknRY0UGZmdWYCFwLrJG2/w/4YmHRLCYnAVZGE+kHf1xm1q+tEhEXk53BRETMY/5Le7cE\nJwFWRv3ij8vM+rV+sby5zw6wMuoXf1xm1q8dC1wBrCfpb6TlzYsNadF5YqCVjqRNgZ/gaweYWYn1\nh+XN3RNgZbQesDMwAtiL7Hxcv1fNrHCSPtXDrvdJIiIua2pAfeR/rFZG34iI30laERgL/Ag4m/cW\n5zAzK8ru6eeqwNbAjWl7LHAr2QqCLcMTA62MOicB7gr8IiKuApYqMB4zMwAi4tCIOJRsQbNREbFX\nROwFfCCVtRQnAVZGMyT9nGzJ4KslLY3fq2ZWLiMi4pma7eeAtYoKZnF5YqCVjqRlgZ2AqRHxiKTV\ngdERcV3BoZmZASDpp8AGwIWpaF9gWkQcVVxUi85JgJmZ2WJIkwQ/ljZvjojfFxnP4nASYGZmVlE+\nO8DMzCwnSbdExDaSZpMWNOvcBURELF9QaIvFPQFmZmYV5RnXZmZmFeUkwMzMrKKcBJiZmVWUkwAz\ny03S45K2LzoOM6sPJwFmVnrpam1mVmdOAswqStIISZdJminpRUk/lbSepBvT9guSfiNpaDr+12TL\nov5R0hxJJ6TyLSXdKukVSfdIaq95jnUk3SxptqQ/SzpL0gU1+/eQdH96bIekjWr2PS7pREn3AnMl\nHS/p0i6v4UxJZzS2pcz6LycBZhUkaQBwJfAEMBIYDkwmO9f5e8AawEZkl3M+GSAiDgSeBHaPiCER\n8QNJw4GrgFOAlYAvA5dKGpae6rfAHcDKqZ4Da2J4H9mSq18EhgFXkyUYtReL2p/sQlJDgQuAnWqS\nkoHAfsD59WkVs+pxEmBWTZuTfdAfHxFzI+KNiLglIqZFxPUR8WZEzAT+B/hEL/V8Brg6Iq6OiHcj\n4nrgLmAXSWsBHwG+GRFvRcQtwBU1j90XuCo939tkl4weRHZ51k5nRsT0iHg9XazlZmCftG8n4IWI\nmNLn1jCrKCcBZtU0AngiIubVFkpqkzRZ0gxJr5J9+16ll3rWBvZJ3fmvSHoF2AZYnSzJeCkiXqs5\nfnrN/TXIeiIAiIh30/7hPRwPMIks8SD9/PVCXqeZ9cJJgFk1TQfW6mbC3f8jWwp1dFr+9DNkQwSd\nui4xOh34dUQMrbkNjohTgWeAldJVITuNqLn/NFkSAYAkpf0zenm+y4FNJG0M7Ab8JsdrNbMeOAkw\nq6Y7yD6kT5U0WNIykj4KLAfMAWal8f7juzzuOWDdmu0LgN0l7ShpQKqnXdKaEfEE2dDAyZKWkrQV\nsHvNYy8GdpW0naQlgeOAN4Fbewo6It4ALiHNNYiIJ/vQBmaV5yTArIIi4h2yD+T1ySb7PUU2Rv8t\nYFNgFtmEv8u6PPR7wNdT1/+XI2I6MA74KjCTrGfgeN7733IAsBXwItnkwYvIPuiJiIfJehp+AryQ\n4tk9It5aSPiTgNF4KMCsz3wBITNrGkkXAQ9FxEl9qGMt4CFgtYh4tW7BmVWQewLMrGEkfSStPbCE\npJ3Ieg0u70N9SwDHApOdAJj1nVfhMrNGWo1sSGFlsiGHIyLiH4tTkaTBZHMSniA7PdDM+sjDAWZm\nZhXl4QAzM7OKchJgZmZWUU4CzMzMKspJgJmZWUU5CTAzM6uo/w/bapGuj+lMmQAAAABJRU5ErkJg\ngg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f6aaed714a8>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_count_labels_per_category(df_stats_valid)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Although we now know how many comments are assigned to each tag, we still lack the big picture since as we know in any multi label problem, one text can be assigned to multiple tags. so we need to understand how many comments have zero tags (or safe comments), how many comments have only one tag, how many comments assigned just two tags, etc..."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"import seaborn as sns"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Text(0.5,0,'# of categories')"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAg4AAAFQCAYAAADX1/YjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYZVV57/Hvj2aQQeYWmQQUQoKo\nUVrEGIeAwXZAjCGKUUFCJMbZGBE1iiLm4k3idFUSAgioEbEdQIMiURGHMHTjQACVliHdLUIDzSQB\nBN77x14VD2VV9aa7qk511/fzPOc5e689rHefajjvWWvttVNVSJIk9bHOsAOQJElrDhMHSZLUm4mD\nJEnqzcRBkiT1ZuIgSZJ6M3GQJEm9mThIkyRJJdl1gu2XJXlGz3Ndk+SZkxbcDJfkqUl+Ouw4JK2c\niYNmvfYlfU+SrUeV/6AlAzuvwjlPSXLsYFlVPbqqzlutYKdIkmckWTqs+qvqO1W1+7DqFyTZuf17\nX3fYsWhmM3GQOlcDLxlZSfIYYKPhhTN7rAlfVGtCjNJ0MXGQOp8EDhlYPxQ4bXCHJOcl+cuB9Vck\n+e7oEyU5AngpcGSSO5J8uZX/b/dDkncnWZDks0luT3JJkseNFViSdZIcleTnSW5KckaSLce7kCQH\nJvlhktvaMfNb+WFJrmj1XZXkr1r5xsBXge1avHck2W5l9SY5JMm1bds7R13fBkk+lOQX7fWhJBu0\nbc9IsjTJW5P8EvjE6BaPVv/nkyxPcnWS1w9s2zvJwnZ91yf5wDifw0g9b09yY4vvpQPbN0jyj0n+\nu53nn5NsOF6M49TxyoHP9PIkT2jlv9f+vdzSuqieP3DMKUk+nuSr7bP+XpKHt89oRZKfJHn8wP7X\nJHlLkh8n+VWSk5Js046/Pcl/JNliYP99kny/1f2jDHSPtZje2+q8PcnX85uWtvPb+y0triePdc2S\niYPUuQDYtP0Pfw5wMPCpVTlRVZ0AfBr4v1W1SVUdMM6uBwKfA7YE/g34UpL1xtjvdcALgKcD2wEr\ngI+NdcIke9MlPG8BNgeeBlzTNt8APA/YFDgM+GCSJ1TVr4BnA79o8W5SVb+YqN4kewAfp0uQtgU2\nA7YfCOUdwD7A7wOPA/YG/m5g+8Pbde8EHDHqGtYBvgz8qJ1zP+CNSZ7Vdvkw8OGq2hR4FHDGWJ/F\nQD1bt/McCpyQZKRL5Djgd1qMu7Z93tUnxhbnnwHvpks4NwWeD9zU/oZfBr4OPIzuc/z0QL0AL2qf\nx9bA3cB/Ape09QXA6GToT4E/bvEeQJfovR2YS/f/8de3mLYH/h04tsX+t8Dnk8wdONef0/39Hwas\n3/aB7t8KwObt38B/jr5mCUwcpEEjrQ5/DFwBLJvi+hZV1YKq+jXdF8VD6L5sR3sV8I6qWlpVd9N9\nWR2UsZvPDwdOrqpzq+r+qlpWVT8BqKp/r6qfV+fbdF9sT50gvonqPQj4clV9t6ruofvCHXzwzUuB\nY6rqhqpaDrwHePnA9vuBo6vq7qr6n1H1PhGYW1XHVNU9VXUV8K90yRzAr4Fdk2xdVXdU1QUTXAPA\nO1s936b7Un1RktAlA2+qqpur6nbg7wfqWFmMAH9Jlxxe3D7TxVV1Ld3fcBPguBb/N4GvMNAVBnyx\nqhZV1V3AF4G7quq0qroP+Czw+FF1/b+qur6qlgHfAS6sqh8MHD+y/8uAs6vq7Pb3PxdYCDxn4Fyf\nqKqftWs6gy5xknqz3076jU/SNdfuwqhuiimyZGShqu5vTfXbjbHfTsAXk9w/UHYfsA2/ndzsCJw9\nVmVJng0cTferdR26MRyXThDfRPVuNyr+O5PcNLDfdsC1A+vX8sBrW96+9Mard7sktwyUzaH7woQu\nOToG+EmSq4H3VNVXxjnXitaiMjqOuXTXv6jLIQBIq6dPjNB91j8fo3w7YElVDX5u1/LAFpnrB5b/\nZ4z1TUads+/+OwF/lmSwlWs94FsD678cWL5zjLqkCZk4SE1VXdu+iJ5D9+U02q944IDJh090uh5V\n7jiy0JrndwB+McZ+S4C/qKrv9TjnErrm+wdo4ws+T9eicmZV/TrJl+i+LMeLd9x6k1wH7D6wviGw\n1cAuv6D7ErusrT+CB17bRJ/PEuDqqtptrI1VdSXwkvaZvRBYkGSrUQnCiC2SbDyw7RHAfwE30n3h\nPrr9ih+zqgliHInztz5ruuvcMck6A8nDI4CfreR8k2EJ8MmqeuUqHOujktWLXRXSAx0O7DvOl9AP\ngRcm2SjdfA1jJRcjrgceuZK69krywtb0/0a6vu6xmt3/GXhfkp0AksxNcuA45zwJOCzJfukGN26f\n5Hfp+rI3AJYD97bWh/1HxbtVks161rsAOCDJHyRZn64bIwPHfgb4u3bM1nRdGX3HjFwE3N4GJm6Y\nZE6SPZM8scXxsiRz25fySKvE/eOeDd6TZP0kT6Ub4/G5duy/0o3zeFg77/YD4yj6OBH42yR7pbNr\n+6wupPslf2SS9drgxAOA0x/EuVfVp+j+Ls9qn9tD0g303KHHscvpPseV/bvVLGfiIA1oYwAWjrP5\ng8A9dF+yp9INgBzPScAebWT7l8bZ50zgxXSDDl8OvLCNdxjtw8BZwNeT3E6XXDxpnPgvog18BG4F\nvg3s1PrwX0/Xp72CboDcWQPH/YTuy/6qFvN2E9VbVZfRDfo7HbgOuINu8OXd7ZTH0vWt/5iuO+SS\nVrZSrZ//eXR971fTtQ6cSDcAE2A+cFmSO1qMB48zBgG6ZvkVdK0AnwZeNTLmA3grsBi4IMltwH8w\n0IrSI87PAe+jG9h6O/AlYMs25uMAugGnN9INIj1koN4pU1VL6Abdvp0uEVhCN1B2pf+vr6o76a7n\ne+3fwFjjbSRSZeuUNN2SvBvYtapeNuxYJkOSTeh+/e9WVVcPOx7obqkEPlVVfX5tS+rJFgdJqyTJ\nAa3bZmPgH+laFq4ZblSSppqJg6RVdSBdF8AvgN3ougxswpTWcnZVSJKk3mxxkCRJvZk4SJKk3qZl\nAqgkJ9PdXnVDVe3Zyv6B7pale+hmXzusqm5p295Gd4/8fcDrq+qcVj6f7varOcCJVXVcK9+F7raw\nrYBFwMur6p426c1pwF7ATcCLq+qalcW79dZb18477zw5Fy9J0gy3aNGiG6tq7sr3nKYxDkmeRnef\n92kDicP+wDer6t4k7weoqre2h+d8hu6hONvR3Vv9O+1UP6N7jsBS4GLgJVV1eZIzgC9U1elJ/hn4\nUVUdn+TVwGOr6lVJDgb+pKpevLJ4582bVwsXjncrvyRJa5cki6pqXp99p6WroqrOB24eVfb1qrq3\nrV5AN90udCO1T28PlrmaboKWvdtrcVVd1SZYOR04sD2sZl+6meygm5jnBQPnOrUtLwD2y8DE9JIk\n6cGZKWMc/oLuMbHQPQhmycC2pa1svPKtgFsGkpCR8gecq22/lQfOpy9Jkh6EoScOSd4B3MvE0/dO\nRxxHJFmYZOHy5cuHGYokSTPWUBOHJK+gGzT50oGJY5Yx8NRAui6MZROU3wRs3h4UNFj+gHO17Zu1\n/X9LVZ1QVfOqat7cub3Gh0iSNOsMLXFod0gcCTy/PVxlxFnAwUk2aHdL7Eb3tLyLgd2S7NKexncw\ncFZLOL4FHNSOP5Tu4UEj5zq0LR9ENxjTGa8kSVpF03U75meAZwBbJ1kKHA28je4xv+e28YoXVNWr\nquqydpfE5XRdGK9pT8sjyWuBc+huxzy5PaEPuqfcnZ7kWOAHdE8mpL1/MsliusGZB0/5xUqStBZz\nyukxeDumJGk2mXG3Y0qSpLWDiYMkSerNxEGSJPVm4iBJknqblrsq1hZ7veW0YYcwaRb9wyHDDkGS\ntAayxUGSJPVm4iBJknozcZAkSb2ZOEiSpN5MHCRJUm8mDpIkqTcTB0mS1JuJgyRJ6s3EQZIk9Wbi\nIEmSejNxkCRJvZk4SJKk3kwcJElSbyYOkiSpNxMHSZLUm4mDJEnqzcRBkiT1ZuIgSZJ6M3GQJEm9\nmThIkqTeTBwkSVJvJg6SJKk3EwdJktSbiYMkSerNxEGSJPVm4iBJknozcZAkSb2ZOEiSpN5MHCRJ\nUm8mDpIkqTcTB0mS1Nu0JA5JTk5yQ5L/GijbMsm5Sa5s71u08iT5SJLFSX6c5AkDxxza9r8yyaED\n5XslubQd85EkmagOSZK0aqarxeEUYP6osqOAb1TVbsA32jrAs4Hd2usI4HjokgDgaOBJwN7A0QOJ\nwPHAKweOm7+SOiRJ0iqYlsShqs4Hbh5VfCBwals+FXjBQPlp1bkA2DzJtsCzgHOr6uaqWgGcC8xv\n2zatqguqqoDTRp1rrDokSdIqGOYYh22q6rq2/Etgm7a8PbBkYL+lrWyi8qVjlE9UhyRJWgUzYnBk\naymoYdaR5IgkC5MsXL58+VSGIknSGmuYicP1rZuB9n5DK18G7Diw3w6tbKLyHcYon6iO31JVJ1TV\nvKqaN3fu3FW+KEmS1mbDTBzOAkbujDgUOHOg/JB2d8U+wK2tu+EcYP8kW7RBkfsD57RttyXZp91N\nccioc41VhyRJWgXrTkclST4DPAPYOslSursjjgPOSHI4cC3worb72cBzgMXAncBhAFV1c5L3Ahe3\n/Y6pqpEBl6+mu3NjQ+Cr7cUEdUiSpFUwLYlDVb1knE37jbFvAa8Z5zwnAyePUb4Q2HOM8pvGqkOS\nJK2aGTE4UpIkrRlMHCRJUm8mDpIkqTcTB0mS1JuJgyRJ6s3EQZIk9WbiIEmSejNxkCRJvZk4SJKk\n3kwcJElSbyYOkiSpNxMHSZLUm4mDJEnqzcRBkiT1ZuIgSZJ6M3GQJEm9mThIkqTeTBwkSVJvJg6S\nJKk3EwdJktSbiYMkSerNxEGSJPVm4iBJknozcZAkSb2ZOEiSpN5MHCRJUm8mDpIkqTcTB0mS1JuJ\ngyRJ6s3EQZIk9WbiIEmSeuuVOCR5SZLfa8u7Jzk/ybeS/O7UhidJkmaSvi0OxwI3t+V/BC4Cvg18\nfCqCkiRJM9O6PfebW1XXJ3kI8IfAQcCvgRunLDJJkjTj9E0clifZFXgMcHFV3Z1kIyBTF5okSZpp\n+iYO7wUWAfcBL25lzwR+NBVBSZKkmanXGIeqOgXYFtihqs5txRcAB69uAEnelOSyJP+V5DNJHpJk\nlyQXJlmc5LNJ1m/7btDWF7ftOw+c522t/KdJnjVQPr+VLU5y1OrGK0nSbPZgbsfcEPjTJEe29XXp\n32IxpiTbA68H5lXVnsAcumTk/cAHq2pXYAVweDvkcGBFK/9g248ke7TjHg3MBz6eZE6SOcDHgGcD\newAvaftKkqRV0Pd2zKcDPwVeCryzFe8GHD8JMawLbJhkXWAj4DpgX2BB234q8IK2fGBbp23fL0la\n+elVdXdVXQ0sBvZur8VVdVVV3QOc3vaVJEmroG+Lw4eAF1fVfODeVnYh3RfzKquqZXS3d/43XcJw\nK91YiluqaqSepcD2bXl7YEk79t62/1aD5aOOGa/8tyQ5IsnCJAuXL1++OpclSdJaq2/isHNVfaMt\nV3u/h9XvqtiCrgVgF2A7YGO6roZpV1UnVNW8qpo3d+7cYYQgSdKM1zdxuHxwwGHzTODS1az/mcDV\nVbW8qn4NfAF4CrB567oA2AFY1paXATsCtO2bATcNlo86ZrxySZK0CvomDm8GPp3kVLrxCP8CnAK8\nZTXr/29gnyQbtbEK+wGXA9+im2QK4FDgzLZ8Vlunbf9mVVUrP7jddbEL3fiLi4CLgd3aXRrr0w2g\nPGs1Y5Ykadbq1dVQVRckeSzwMuBkunEDe1fV0tWpvKouTLIAuIRu7MQPgBOAfwdOT3JsKzupHXIS\n8Mkki+mmwD64neeyJGfQJR33Aq+pqvsAkrwWOIfujo2Tq+qy1YlZkqTZLN0P9pXslGwA3N+6E0bK\n1gPWqaq7pzC+oZg3b14tXLjwt8r3estpQ4hmaiz6h0OGHYIkaYZIsqiq5vXZt29XxbnAXqPK9qL7\nJS9JkmaJvonDY+huvxx0EfC4yQ1HkiTNZH0Th1uBbUaVbQP8anLDkSRJM1nfxOHzwL8l2bPdAfEY\n4DTgjKkLTZIkzTR9E4d3AFfQdU/cTveAq58Cb5+iuCRJ0gzU93bMu4DXtFsbtwZurD63Y0iSpLVK\n7ymjk2wG7A5s0tYBqKpvTklkkiRpxumVOCR5Bd3jqe8A7hzYVMAjJz8sSZI0E/VtcXgfcFBVfXUq\ng5EkSTNb38GR6wJfn8pAJEnSzNc3cXg/8HdJ+u4vSZLWQn27Kt4EPBw4MslNgxuq6hGTHpUkSZqR\n+iYOL5vSKCRJ0hqh7zwO357qQCRJ0szXa8xCkg2SvC/JVUlubWX7twmhJEnSLNF3sOMHgT2Bl9LN\n3QBwGfDXUxGUJEmamfqOcfgTYNeq+lWS+wGqalmS7acuNEmSNNP0bXG4h1FJRpK5wE1j7y5JktZG\nfROHzwGnJtkFIMm2wEeB06cqMEmSNPP0TRzeDlwNXApsDlwJ/AJ4zxTFJUmSZqCVjnFos0X+IXBU\nVb2pdVH4WG1JkmahlbY4VNX9wJlVdXdbX27SIEnS7NS3q+L8JPtMaSSSJGnG63s75rXAV5OcCSzh\nN3M5UFXvmorAJEnSzNM3cdgQ+FJb3mGg3C4LSZJmkb6DIz8JfG9knIMkSZqdHvTgSEmSNHs5OFKS\nJPXm4EhJktTb6g6OlCRJs0ivxKGqDpvqQCRJ0szXK3FI8sjxtlXVVZMXjiRJmsn6dlUsphvXkIGy\nkXEOcyY1IkmSNGP17ap4wN0XSR4OHA18ZyqCkiRJM1Pf2zEfoKp+CbwR+D+TG44kSZrJVilxaHYH\nNlrdAJJsnmRBkp8kuSLJk5NsmeTcJFe29y3avknykSSLk/w4yRMGznNo2//KJIcOlO+V5NJ2zEeS\nZKw4JEnSyvVKHJJ8J8n5A6+FwIXAByYhhg8DX6uq3wUeB1wBHAV8o6p2A77R1gGeDezWXkcAx7f4\ntqTrOnkSsDdw9Eiy0fZ55cBx8ychZkmSZqW+gyNPHLX+K+BHVXXl6lSeZDPgacArAKrqHuCeJAcC\nz2i7nQqcB7wVOBA4raoKuKC1Vmzb9j23qm5u5z0XmJ/kPGDTqrqglZ8GvAD46urELUnSbNV3cOSp\nU1T/LsBy4BNJHgcsAt4AbFNV17V9fgls05a3p5u5csTSVjZR+dIxyiVJ0iro21XxhSRPHVX21CQL\nVrP+dYEnAMdX1ePpWjKOGtyhtS5M+eO7kxyRZGGShcuXL5/q6iRJWiP1HRz5dOD7o8r+E/ij1ax/\nKbC0qi5s6wvoEonrWxcE7f2Gtn0ZsOPA8Tu0sonKdxij/LdU1QlVNa+q5s2dO3e1LkqSpLVV38Th\nLmDjUWWbAL9encrbbZ1LkuzeivYDLgfOAkbujDgUOLMtnwUc0u6u2Ae4tXVpnAPsn2SLNihyf+Cc\ntu22JPu0uykOGTiXJEl6kPoOjjwH+Jckf1VVtyXZFPgo8LVJiOF1wKeTrA9cBRxGl9CckeRwuidz\nvqjtezbwHLqZLO9s+1JVNyd5L3Bx2++YkYGSwKuBU+ge1PVVHBgpSdIq65s4vBn4FHBzkpuBLem+\ngF++ugFU1Q+BeWNs2m+MfQt4zTjnORk4eYzyhcCeqxmmJEmi/10VK4DntqmmdwSWtG4GSZI0i/R9\nOub+wDVV9TO62yNp4xIeUVXnTmF8kiRpBuk7OPJjwO2jym5v5ZIkaZbomzg8bGBCphHXAQ+f5Hgk\nSdIM1jdxuCrJvqPKngFcPbnhSJKkmazvXRXvBr6Q5CTg58Cj6G6FPGyK4pIkSTNQrxaHqjqTblKl\njYHntvdntXJJkjRL9G1xoKouAi6awlgkSdIMt9IWhyQ7JzklybIkd7f3U5M8cjoClCRJM8eEiUOS\n3wMuAR4GvAN4fnufCyxs2yVJ0iyxsq6K44CPVdU7R5WfkuRY4P8CB0xJZJIkacZZWeLwNH7zlMrR\n/glvx5QkaVZZ2RiHOYz/6Oxft+2SJGmWWFnicDHjz9XwCmDhpEYjSZJmtJV1VbwTOKc90GoB3TTT\n2wJ/RteF8aypDU+SJM0kE7Y4VNX36SZ+ehzwDeAn7f1xwPy2XZIkzRIrnQCqqv4TeFqSDYEtgRVV\ndeeURyZJkmacBzNz5P8Ay6YwFkmSNMP1fTqmJEmSiYMkSepv3MQhyT8MLO87PeFIkqSZbKIWhyMG\nlr801YFIkqSZb6LBkT9KsgC4HNggyTFj7VRV75qSyCRJ0owzUeJwEF2rw05AgB3H2KemIihJkjQz\njZs4VNUNwLEASdatqvGmnpYkSbNEr3kcquqwJFvQPUJ7e7r5HL5SVTdPZXCSJGlm6XU7ZpInAz8H\nXgU8FvgrYHErlyRJs0TfmSM/BLy6qk4fKUjyYuAjwBOnIjBJkjTz9J0A6neAM0aVLQB2ndxwJEnS\nTNY3cbgSOHhU2Z/RdV9IkqRZom9XxRuBryR5PXAtsDOwG/C8KYpLkiTNQH3vqvh+kkcBzwW2A74M\nnO1dFZIkzS4P5rHaK4BPTWEskiRphvPpmJIkqTcTB0mS1JuJgyRJ6q134pBkp6kKIsmcJD9I8pW2\nvkuSC5MsTvLZJOu38g3a+uK2feeBc7ytlf80ybMGyue3ssVJjpqqa5AkaTZ4MC0OPwBot2ROtjcA\nVwysvx/4YFXtCqwADm/lhwMrWvkH234k2YNunolHA/OBj7dkZA7wMeDZwB7AS9q+kiRpFUyYOCRZ\nlOSEJH8NzGnF757MAJLsQHeb54ltPcC+dDNTApwKvKAtH9jWadv3a/sfCJxeVXdX1dXAYmDv9lpc\nVVdV1T3A6W1fSZK0ClbW4nAQ8HVgJ2CjJJcAGyT5oySbTVIMHwKOBO5v61sBt1TVvW19Kd0TOWnv\nSwDa9lvb/v9bPuqY8colSdIqWFniMKeqFlTVUcDtdL/WA7wO+GGSK1en8iTPA26oqkWrc57JkOSI\nJAuTLFy+fPmww5EkaUZa2QRQn07yCOBy4CHAFsBdVfVCgCRbrmb9TwGen+Q57fybAh8GNk+ybmtV\n2AFY1vZfBuwILE2yLrAZcNNA+YjBY8Yrf4CqOgE4AWDevHm1mtclSdJaacIWh6p6Et0X798CBXwU\neGiS45O8EthldSqvqrdV1Q5VtTPd4MZvVtVLgW/RdZMAHAqc2ZbPauu07d+sqmrlB7e7Lnahe47G\nRcDFwG7tLo31Wx1nrU7MkiTNZiu9q6Kq7q2qHwD3VNXTgF8B59F9Ob9/iuJ6K/A3SRbTjWE4qZWf\nBGzVyv8GOKrFeBndY78vB74GvKaq7mstFq8FzqG7a+OMtq8kSVoFvZ9VAbypvVdVfRb47GQGUlXn\n0SUkVNVVdHdEjN7nLrrHeY91/PuA941RfjZw9iSGKknSrNV7HoeqOqUtPnJqQpEkSTPdg55yuj0l\nU5IkzUI+q0KSJPVm4iBJknozcZAkSb2ZOEiSpN5MHCRJUm8mDpIkqTcTB0mS1JuJgyRJ6s3EQZIk\n9WbiIEmSejNxkCRJvZk4SJKk3kwcJElSbyYOkiSpNxMHSZLUm4mDJEnqzcRBkiT1ZuIgSZJ6M3GQ\nJEm9mThIkqTeTBwkSVJvJg6SJKk3EwdJktSbiYMkSerNxEGSJPVm4iBJknozcZAkSb2ZOEiSpN5M\nHCRJUm8mDpIkqTcTB0mS1JuJgyRJ6s3EQZIk9TbUxCHJjkm+leTyJJcleUMr3zLJuUmubO9btPIk\n+UiSxUl+nOQJA+c6tO1/ZZJDB8r3SnJpO+YjSTL9VypJ0tph2C0O9wJvrqo9gH2A1yTZAzgK+EZV\n7QZ8o60DPBvYrb2OAI6HLtEAjgaeBOwNHD2SbLR9Xjlw3PxpuC5JktZKQ00cquq6qrqkLd8OXAFs\nDxwInNp2OxV4QVs+EDitOhcAmyfZFngWcG5V3VxVK4Bzgflt26ZVdUFVFXDawLkkSdKDNOwWh/+V\nZGfg8cCFwDZVdV3b9Etgm7a8PbBk4LClrWyi8qVjlEuSpFUwIxKHJJsAnwfeWFW3DW5rLQU1DTEc\nkWRhkoXLly+f6uokSVojDT1xSLIeXdLw6ar6Qiu+vnUz0N5vaOXLgB0HDt+hlU1UvsMY5b+lqk6o\nqnlVNW/u3Lmrd1GSJK2lhn1XRYCTgCuq6gMDm84CRu6MOBQ4c6D8kHZ3xT7Ara1L4xxg/yRbtEGR\n+wPntG23Jdmn1XXIwLkkSdKDtO6Q638K8HLg0iQ/bGVvB44DzkhyOHAt8KK27WzgOcBi4E7gMICq\nujnJe4GL237HVNXNbfnVwCnAhsBX20uSJK2CoSYOVfVdYLx5FfYbY/8CXjPOuU4GTh6jfCGw52qE\nKUmSmqGPcZAkSWsOEwdJktSbiYMkSerNxEGSJPVm4iBJknozcZAkSb2ZOEiSpN5MHCRJUm8mDpIk\nqTcTB0mS1JuJgyRJ6s3EQZIk9WbiIEmSejNxkCRJvZk4SJKk3kwcJElSbyYOkiSpNxMHSZLUm4mD\nJEnqzcRBkiT1ZuIgSZJ6M3GQJEm9mThIkqTeTBwkSVJvJg6SJKk3EwdJktSbiYMkSerNxEGSJPW2\n7rAD0Jrjv495zLBDmDSPeNelww5BktZItjhIkqTeTBwkSVJvJg6SJKk3xzhI6uXbT3v6sEOYNE8/\n/9vDDkFaY9niIEmSerPFQerpKf/vKcMOYdJ873XfG3YIktZQtjhIkqTeZkXikGR+kp8mWZzkqGHH\nI0nSmmqt76pIMgf4GPDHwFLg4iRnVdXlw41M0prio2/+8rBDmDSv/acDhh2C1nCzocVhb2BxVV1V\nVfcApwMHDjkmSZLWSGt9iwOwPbBkYH0p8KQhxSJJa5z3veygYYcwad7xqQXDDmGNl6oadgxTKslB\nwPyq+su2/nLgSVX12lH7HQEc0VZ3B346rYE+0NbAjUOsf9hm8/XP5msHr9/rn73XP+xr36mq5vbZ\ncTa0OCwDdhxY36GVPUBVnQCcMF1BTSTJwqqaN+w4hmU2X/9svnbw+r3+2Xv9a9K1z4YxDhcDuyXZ\nJcn6wMHAWUOOSZKkNdJa3+JQVfcmeS1wDjAHOLmqLhtyWJIkrZHW+sQBoKrOBs4edhwPwozoMhmi\n2Xz9s/nawev3+mevNeba1/qi0IIYAAAHn0lEQVTBkZIkafLMhjEOkiRpkpg4zCCzfWrsJCcnuSHJ\nfw07lumWZMck30pyeZLLkrxh2DFNpyQPSXJRkh+163/PsGOabknmJPlBkq8MO5bpluSaJJcm+WGS\nhcOOZ7ol2TzJgiQ/SXJFkicPO6aJ2FUxQ7SpsX/GwNTYwEtm09TYSZ4G3AGcVlV7Djue6ZRkW2Db\nqrokyUOBRcALZsvfP0mAjavqjiTrAd8F3lBVFww5tGmT5G+AecCmVfW8YccznZJcA8yrqlk5h0OS\nU4HvVNWJ7e6/jarqlmHHNR5bHGaOWT81dlWdD9w87DiGoaquq6pL2vLtwBV0s57OCtW5o62u116z\n5ldNkh2A5wInDjsWTa8kmwFPA04CqKp7ZnLSACYOM8lYU2PPmi8O/UaSnYHHAxcON5Lp1Zrqfwjc\nAJxbVbPp+j8EHAncP+xAhqSArydZ1GbxnU12AZYDn2hdVScm2XjYQU3ExEGaQZJsAnweeGNV3Tbs\neKZTVd1XVb9PN7vr3klmRXdVkucBN1TVomHHMkR/WFVPAJ4NvKZ1W84W6wJPAI6vqscDvwJm9Bg3\nE4eZo9fU2Fp7tb79zwOfrqovDDueYWnNtN8C5g87lmnyFOD5rZ//dGDfJJ8abkjTq6qWtfcbgC/S\ndd3OFkuBpQMtbAvoEokZy8Rh5nBq7FmsDQ48Cbiiqj4w7HimW5K5STZvyxvSDRL+yXCjmh5V9baq\n2qGqdqb77/6bVfWyIYc1bZJs3AYE05ro9wdmzZ1VVfVLYEmS3VvRfsCMHhQ9K2aOXBM4NTYk+Qzw\nDGDrJEuBo6vqpOFGNW2eArwcuLT18wO8vc16OhtsC5za7i5aBzijqmbdbYmz1DbAF7vcmXWBf6uq\nrw03pGn3OuDT7UfjVcBhQ45nQt6OKUmSerOrQpIk9WbiIEmSejNxkCRJvZk4SJKk3kwcJElSbyYO\nknpLsk2S85PcnuSfhh3PZEnyz0neOew4pDWB8zhIs0CSi4CXAfcCC9r0vqviCOBGuic4Ttq93ElO\noZs97+8m65wPRlW9ahj1SmsiWxyktVybynon4EpgL+CS1TjdTsDlk5k0DFubdEpSTyYO0tpvT37z\nZT+PlSQOSf4gycVJbm3vf9DKTwEOBY5MckeSZ45x7IZJ/inJte3477YppEnyuSS/bOXnJ3l0Kz8C\neOnAeb/cyrdL8vkky5NcneT1o+o5NcmKJFckObLNNjqy/feSnJfkliSXJXn+wLZTkhyf5OwkvwL+\nqJUdO7DP85L8sB3//SSPHdj21iTLWnfNT5Ps9yD+FtKar6p8+fK1Fr7opq29BbgTuKst3wvc3pZ3\nGeOYLYEVdNNfrwu8pK1v1bafAhw7QZ0fA86jeyT8HOAPgA3atr8AHgpsQPcY6R8OHPeA89L9qFkE\nvAtYH3gk3VS8z2rbjwO+DWxB90C4H9N1dQCsBywG3t6O3bdd8+4Ddd1KN833OsBDBuune6T5DcCT\n2jUcClzT4t4dWAJs1/bdGXjUsP/WvnxN58sWB2ktVVWfqKrN6b6A9wEeS/fwoE2ravOqunqMw54L\nXFlVn6yqe6vqM3QPmzpgZfUlWYcuOXhDVS2r7jHZ36+qu1s8J1fV7W393cDjkmw2zumeCMytqmOq\n6p6qugr4V7qHQAG8CPj7qlpRVUuBjwwcuw+wCXBcO/abwFfokqARZ1bV96rq/qq6a1TdRwD/UlUX\ntms4Fbi7nfc+ugRijyTrVdU1VfXzlX020trExEFaCyXZsjWz30r3q/884Kd0v5hXJHnjOIduB1w7\nquxauhaEldma7tf7b32RJpmT5LgkP09yG90v+JFjxrITsF27hluS3ELXgrDNQJxLBvYfXN4OWFJV\n909wDYP7j1X3m0fVvSNdK8Ni4I10ic8NSU5Pst0E55LWOiYO0lqoqm5urQ1/BZzYlr8GHNBaGz40\nzqG/oPviHPQIYFmPam+k6xJ51Bjb/hw4EHgmsBldEz9ARkIetf8S4OoW68jroVX1nLb9OrouihE7\njrqGHVsLyHjXMNHgziXA+0bVvVFrfaGq/q2q/pDucyrg/ROcS1rrmDhIa7fBuygeT9dtMZGzgd9J\n8udJ1k3yYmAPuqb+CbVf+CcDH2gDG+ckeXKSDejGNtwN3ARsBPz9qMOvpxvHMOIi4PY2EHHDdq49\nkzyxbT8DeFuSLZJsD7x24NgL6cZ1HJlkvSTPoOtqOX1l19D8K/CqJE9KZ+Mkz03y0CS7J9m3XdNd\nwP8A9098OmntYuIgrd32Ai5JshVwX1WtmGjnqroJeB7wZrov+SOB51XVjT3r+1vgUuBi4Ga6X+Pr\nAKfRdRcsAy4HLhh13El04wZuSfKlqrqvxfH7wNV0rRkn0rVWABwDLG3b/gNYQJeYUFX30CUKz27H\nfRw4pKp+0ucCqmoh8Ergo3QDQxcDr2ibN6AbmHkj8EvgYcDb+pxXWlukaq25HVvSLJXkr4GDq+rp\nw45FWtvZ4iBpjZNk2yRPSbJOkt3pWki+OOy4pNnAKaclrYnWB/4F2IVuTorT6bokJE0xuyokSVJv\ndlVIkqTeTBwkSVJvJg6SJKk3EwdJktSbiYMkSerNxEGSJPX2/wG60qIVisQY3QAAAABJRU5ErkJg\ngg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f6aae7e9860>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"rowsums = train.iloc[:,2:].sum(axis=1)\n",
"x=rowsums.value_counts()\n",
"plt.figure(figsize=(8,5))\n",
"ax = sns.barplot(x.index, x.values)\n",
"plt.title(\"Multiple categories per comment\")\n",
"plt.ylabel('# of Occurrences', fontsize=12)\n",
"plt.xlabel('# of categories', fontsize=12)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This indeed just revealed something not surprising but important to know. most of our comments are safe and so they don't have a tag. no comment has all tags at a time. looking at the graph we can see that our data set is sparse. so we need to keep this in mind when we evaluate our model. I will talk about this in the section of model evaluation. to understand how sparse our model is, let's calculate the percentage of comments that have no tag on them."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Percentage of comments that are not labelled:\n",
"0.8980975002127644\n"
]
}
],
"source": [
"print('Percentage of comments that are not labelled:')\n",
"print(len(train[(train['toxic']==0) & (train['severe_toxic']==0) & (train['obscene']==0) & (train['threat']== 0) & (train['insult']==0) & (train['identity_hate']==0)]) / len(train))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see that ~ 90% of our data are safe comments. during the evaluation, we need to keep this in mind so that we don't think that our model is doing a good job when in reality it is just assigning all comments to a safe tag. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So far we have been looking at the distribution of the tags. what about the composition of our comments data? as a matter of curiosity, let see how big are texts in general"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.axes._subplots.AxesSubplot at 0x7f6aae7e9278>"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGVZJREFUeJzt3XuMXOWd5vHvs+YyCCfBBKbksT1r\no+1kZWDWg1vgVS5qwsYYGI3JKmKNEDaXpJMFS4nG0qyZSEs2LJJndkh2YbJEnWBhNB4cNoSxRcw6\nHQ81aKQ12A4OtiHEbacR3WtsBROcJhEzzvz2j3rbe/Dbl6Kququ7z/ORSn3qd95z6v212n76XKpL\nEYGZmVnRv2j3BMzMbOpxOJiZWcbhYGZmGYeDmZllHA5mZpZxOJiZWcbhYGZmGYeDmZllHA5mZpY5\nq90TaNRFF10UCxcubGjbd955h/PPP7+1E5oGyth3GXuGcvbtnuuzd+/eX0TExeONm7bhsHDhQvbs\n2dPQttVqla6urtZOaBooY99l7BnK2bd7ro+k1+oZ59NKZmaWGTccJC2Q9KyklyUdlPSlVL9QUq+k\nQ+nrnFSXpAcl9Ul6SdIVhX2tSeMPSVpTqC+VtD9t86AkTUSzZmZWn3qOHE4B6yJiMbAMuFvSYmA9\nsDMiOoCd6TnAdUBHenQDD0MtTIB7gauAK4F7hwMljfl8YbsVzbdmZmaNGjccIuJoRPw4Lf8KeAWY\nB6wENqVhm4Ab0/JK4LGo2QVcIGkucC3QGxEnIuItoBdYkdZ9MCJ2Re3vhz9W2JeZmbXB+7ogLWkh\n8IfA80AlIo6mVW8AlbQ8D3i9sNlAqo1VHxihPtLrd1M7GqFSqVCtVt/P9E8bGhpqeNvprIx9l7Fn\nKGff7rm16g4HSbOBJ4EvR8TJ4mWBiAhJE/6pQRHRA/QAdHZ2RqN3JpTxrgYoZ99l7BnK2bd7bq26\n7laSdDa1YNgcEd9P5WPplBDp6/FUHwQWFDafn2pj1eePUDczszap524lAY8Ar0TE1wurtgHDdxyt\nAbYW6qvTXUvLgLfT6acdwHJJc9KF6OXAjrTupKRl6bVWF/ZlZmZtUM9ppY8BtwL7Je1LtT8DNgBP\nSLoTeA24Ka3bDlwP9AG/Bm4HiIgTku4DdqdxX4uIE2n5LuBR4DzgmfQwM7M2GTccIuIfgNHed3DN\nCOMDuHuUfW0ENo5Q3wNcNt5cJsLC9T84vdy/4YZ2TMHMbMrxO6TNzCzjcDAzs4zDwczMMg4HMzPL\nOBzMzCzjcDAzs4zDwczMMg4HMzPLOBzMzCzjcDAzs4zDwczMMg4HMzPLOBzMzCzjcDAzs4zDwczM\nMg4HMzPLOBzMzCxTz2dIb5R0XNKBQu27kvalR//wx4dKWijpN4V13ypss1TSfkl9kh5MnxeNpAsl\n9Uo6lL7OmYhGzcysfvUcOTwKrCgWIuI/RMSSiFgCPAl8v7D68PC6iPhiof4w8HmgIz2G97ke2BkR\nHcDO9NzMzNpo3HCIiOeAEyOtS7/93wQ8PtY+JM0FPhgRu9JnTD8G3JhWrwQ2peVNhbqZmbVJs9cc\nPgEci4hDhdoiSS9K+ntJn0i1ecBAYcxAqgFUIuJoWn4DqDQ5JzMza9JZTW5/M+89ajgK/H5EvClp\nKfC3ki6td2cREZJitPWSuoFugEqlQrVabWjSQ0NDp7ddd/mp0/VG9zddFPsuizL2DOXs2z23VsPh\nIOks4N8DS4drEfEu8G5a3ivpMPARYBCYX9h8fqoBHJM0NyKOptNPx0d7zYjoAXoAOjs7o6urq6G5\nV6tVhre9bf0PTtf7b2lsf9NFse+yKGPPUM6+3XNrNXNa6d8BP42I06eLJF0saVZavoTahecj6bTR\nSUnL0nWK1cDWtNk2YE1aXlOom5lZm9RzK+vjwP8BPippQNKdadUq8gvRnwReSre2fg/4YkQMX8y+\nC/gO0AccBp5J9Q3ApyUdohY4G5rox8zMWmDc00oRcfMo9dtGqD1J7dbWkcbvAS4bof4mcM148zAz\ns8njd0ibmVnG4WBmZhmHg5mZZRwOZmaWcTiYmVmm2XdIzygLi2+I23BDG2diZtZePnIwM7OMw8HM\nzDIOBzMzyzgczMws43AwM7OMw8HMzDIOBzMzyzgczMws43AwM7OMw8HMzDIOBzMzyzgczMwsU89n\nSG+UdFzSgULtq5IGJe1Lj+sL6+6R1CfpVUnXFuorUq1P0vpCfZGk51P9u5LOaWWDZmb2/tVz5PAo\nsGKE+jciYkl6bAeQtBhYBVyatvmfkmZJmgV8E7gOWAzcnMYC/Hna178C3gLubKYhMzNr3rjhEBHP\nASfq3N9KYEtEvBsRPwf6gCvToy8ijkTEPwJbgJWSBHwK+F7afhNw4/vswczMWqyZz3NYK2k1sAdY\nFxFvAfOAXYUxA6kG8PoZ9auADwO/jIhTI4zPSOoGugEqlQrVarWhiQ8NDZ3edt3lp0Yc0+i+p7Ji\n32VRxp6hnH2759ZqNBweBu4DIn19ALijVZMaTUT0AD0AnZ2d0dXV1dB+qtUqw9veVviAn6L+Wxrb\n91RW7LssytgzlLNv99xaDYVDRBwbXpb0beDp9HQQWFAYOj/VGKX+JnCBpLPS0UNxvJmZtUlDt7JK\nmlt4+hlg+E6mbcAqSedKWgR0AC8Au4GOdGfSOdQuWm+LiACeBT6btl8DbG1kTmZm1jrjHjlIehzo\nAi6SNADcC3RJWkLttFI/8AWAiDgo6QngZeAUcHdE/DbtZy2wA5gFbIyIg+kl/hOwRdJ/BV4EHmlZ\nd2Zm1pBxwyEibh6hPOp/4BFxP3D/CPXtwPYR6keo3c1kZmZThN8hbWZmGYeDmZllHA5mZpZxOJiZ\nWcbhYGZmmWb+fMa0tX/w7VHfGW1mZj5yMDOzETgczMws43AwM7OMw8HMzDIOBzMzyzgczMws43Aw\nM7NMKd/nUI+FhfdB9G+4oY0zMTObfD5yMDOzjMPBzMwyDgczM8uMGw6SNko6LulAofbfJP1U0kuS\nnpJ0QaovlPQbSfvS41uFbZZK2i+pT9KDkpTqF0rqlXQofZ0zEY2amVn96jlyeBRYcUatF7gsIv4A\n+BlwT2Hd4YhYkh5fLNQfBj4PdKTH8D7XAzsjogPYmZ6bmVkbjRsOEfEccOKM2g8j4lR6uguYP9Y+\nJM0FPhgRuyIigMeAG9PqlcCmtLypUDczszZpxTWHO4BnCs8XSXpR0t9L+kSqzQMGCmMGUg2gEhFH\n0/IbQKUFczIzsyY09T4HSV8BTgGbU+ko8PsR8aakpcDfSrq03v1FREiKMV6vG+gGqFQqVKvVhuZd\nOQ/WXX5q/IFJo68z1QwNDc2YXupVxp6hnH2759ZqOBwk3Qb8EXBNOlVERLwLvJuW90o6DHwEGOS9\np57mpxrAMUlzI+JoOv10fLTXjIgeoAegs7Mzurq6Gpr7Q5u38sD++lvvv6Wx15lqqtUqjX7Ppqsy\n9gzl7Ns9t1ZDp5UkrQD+FPjjiPh1oX6xpFlp+RJqF56PpNNGJyUtS3cprQa2ps22AWvS8ppC3czM\n2mTcX58lPQ50ARdJGgDupXZ30rlAb7ojdVe6M+mTwNck/RPwz8AXI2L4YvZd1O58Oo/aNYrh6xQb\ngCck3Qm8BtzUks7MzKxh44ZDRNw8QvmRUcY+CTw5yro9wGUj1N8ErhlvHmZmNnn8DmkzM8s4HMzM\nLONwMDOzjMPBzMwyDgczM8s4HMzMLONwMDOzjMPBzMwyDgczM8s4HMzMLONwMDOzjMPBzMwyDgcz\nM8s4HMzMLONwMDOzjMPBzMwyDgczM8vUFQ6SNko6LulAoXahpF5Jh9LXOakuSQ9K6pP0kqQrCtus\nSeMPSVpTqC+VtD9t82D6nGkzM2uTeo8cHgVWnFFbD+yMiA5gZ3oOcB3QkR7dwMNQCxNqnz99FXAl\ncO9woKQxny9sd+ZrmZnZJKorHCLiOeDEGeWVwKa0vAm4sVB/LGp2ARdImgtcC/RGxImIeAvoBVak\ndR+MiF0REcBjhX2ZmVkbNHPNoRIRR9PyG0AlLc8DXi+MG0i1seoDI9TNzKxNzmrFTiIiJEUr9jUW\nSd3UTlVRqVSoVqsN7adyHqy7/FTd4xt9nalmaGhoxvRSrzL2DOXs2z23VjPhcEzS3Ig4mk4NHU/1\nQWBBYdz8VBsEus6oV1N9/gjjMxHRA/QAdHZ2RldX10jDxvXQ5q08sP99tL7/ndOL/RtuaOg1p4Jq\ntUqj37Ppqow9Qzn7ds+t1cxppW3A8B1Ha4CthfrqdNfSMuDtdPppB7Bc0px0IXo5sCOtOylpWbpL\naXVhX2Zm1gZ1/fos6XFqv/VfJGmA2l1HG4AnJN0JvAbclIZvB64H+oBfA7cDRMQJSfcBu9O4r0XE\n8EXuu6jdEXUe8Ex6mJlZm9QVDhFx8yirrhlhbAB3j7KfjcDGEep7gMvqmYuZmU08v0PazMwyDgcz\nM8s4HMzMLONwMDOzjMPBzMwyDgczM8s4HMzMLONwMDOzjMPBzMwyDgczM8s4HMzMLONwMDOzjMPB\nzMwyLfkkuDJZuP4Hp5en8wf/mJmNxUcOZmaWcTiYmVnG4WBmZhmHg5mZZRoOB0kflbSv8Dgp6cuS\nvippsFC/vrDNPZL6JL0q6dpCfUWq9Ula32xTZmbWnIbvVoqIV4ElAJJmAYPAU8DtwDci4i+L4yUt\nBlYBlwK/B/xI0kfS6m8CnwYGgN2StkXEy43OzczMmtOqW1mvAQ5HxGuSRhuzEtgSEe8CP5fUB1yZ\n1vVFxBEASVvSWIeDmVmbtOqawyrg8cLztZJekrRR0pxUmwe8XhgzkGqj1c3MrE0UEc3tQDoH+L/A\npRFxTFIF+AUQwH3A3Ii4Q9JfAbsi4q/Tdo8Az6TdrIiIz6X6rcBVEbF2hNfqBroBKpXK0i1btjQ0\n5+Mn3ubYbxra9D0un/eh5ncyiYaGhpg9e3a7pzGpytgzlLNv91yfq6++em9EdI43rhWnla4DfhwR\nxwCGvwJI+jbwdHo6CCwobDc/1Rij/h4R0QP0AHR2dkZXV1dDE35o81Ye2N986/23NPb67VKtVmn0\nezZdlbFnKGff7rm1WnFa6WYKp5QkzS2s+wxwIC1vA1ZJOlfSIqADeAHYDXRIWpSOQlalsWZm1iZN\n/fos6Xxqdxl9oVD+C0lLqJ1W6h9eFxEHJT1B7ULzKeDuiPht2s9aYAcwC9gYEQebmZeZmTWnqXCI\niHeAD59Ru3WM8fcD949Q3w5sb2YuZmbWOn6HtJmZZfwnu5tQ/PPd4D/hbWYzh48czMws43AwM7OM\nw8HMzDIOBzMzyzgczMws43AwM7OMw8HMzDIOBzMzyzgczMws43AwM7OMw8HMzDIOBzMzyzgczMws\n43AwM7OMw8HMzDJNh4Okfkn7Je2TtCfVLpTUK+lQ+jon1SXpQUl9kl6SdEVhP2vS+EOS1jQ7r3ZY\nuP4Hpx9mZtNZq44cro6IJRHRmZ6vB3ZGRAewMz0HuA7oSI9u4GGohQlwL3AVcCVw73CgmJnZ5Juo\n00orgU1peRNwY6H+WNTsAi6QNBe4FuiNiBMR8RbQC6yYoLmZmdk4WhEOAfxQ0l5J3alWiYijafkN\noJKW5wGvF7YdSLXR6mZm1gat+Azpj0fEoKTfBXol/bS4MiJCUrTgdUjh0w1QqVSoVqsN7adyHqy7\n/FQrpjSqRuc2kYaGhqbkvCZSGXuGcvbtnlur6XCIiMH09bikp6hdMzgmaW5EHE2njY6n4YPAgsLm\n81NtEOg6o14d4bV6gB6Azs7O6OrqOnNIXR7avJUH9rciF0fXf0vXhO6/EdVqlUa/Z9NVGXuGcvbt\nnlurqdNKks6X9IHhZWA5cADYBgzfcbQG2JqWtwGr011Ly4C30+mnHcBySXPShejlqWZmZm3Q7K/P\nFeApScP7+puI+N+SdgNPSLoTeA24KY3fDlwP9AG/Bm4HiIgTku4DdqdxX4uIE03OzczMGtRUOETE\nEeDfjFB/E7hmhHoAd4+yr43AxmbmY2ZmreF3SJuZWcbhYGZmGYeDmZllJvZ+zhIr/n2l/g03tHEm\nZmbvn48czMws43AwM7OMw8HMzDIOBzMzyzgczMws47uVJoHvXDKz6cZHDmZmlnE4mJlZxuFgZmYZ\nh4OZmWUcDmZmlnE4mJlZxreyTjLf1mpm04GPHMzMLNNwOEhaIOlZSS9LOijpS6n+VUmDkvalx/WF\nbe6R1CfpVUnXFuorUq1P0vrmWjIzs2Y1c1rpFLAuIn4s6QPAXkm9ad03IuIvi4MlLQZWAZcCvwf8\nSNJH0upvAp8GBoDdkrZFxMtNzM3MzJrQcDhExFHgaFr+laRXgHljbLIS2BIR7wI/l9QHXJnW9UXE\nEQBJW9JYh4OZWZsoIprfibQQeA64DPgT4DbgJLCH2tHFW5L+CtgVEX+dtnkEeCbtYkVEfC7VbwWu\nioi1I7xON9ANUKlUlm7ZsqWh+R4/8TbHftPQphPm8nkfmvDXGBoaYvbs2RP+OlNJGXuGcvbtnutz\n9dVX742IzvHGNX23kqTZwJPAlyPipKSHgfuASF8fAO5o9nUAIqIH6AHo7OyMrq6uhvbz0OatPLB/\nat2o1X9L14S/RrVapdHv2XRVxp6hnH2759Zq6n9ISWdTC4bNEfF9gIg4Vlj/beDp9HQQWFDYfH6q\nMUbdzMzaoJm7lQQ8ArwSEV8v1OcWhn0GOJCWtwGrJJ0raRHQAbwA7AY6JC2SdA61i9bbGp2XmZk1\nr5kjh48BtwL7Je1LtT8Dbpa0hNpppX7gCwARcVDSE9QuNJ8C7o6I3wJIWgvsAGYBGyPiYBPzmpb8\n5jgzm0qauVvpHwCNsGr7GNvcD9w/Qn37WNuZmdnk8jukzcwsM7Vu2THAp5jMrP185GBmZhkfOUxx\nPoows3bwkYOZmWUcDmZmlnE4mJlZxtccphFffzCzyeJwmKYcFGY2kRwOM4CDwsxazdcczMws4yOH\nGaZ4FHGmdZef4ra03kcYZjYWh0NJ+VSUmY3Fp5XMzCzjIwfzUYSZZRwO9h4OCjMDh4ONYayL28OK\nAeJgMZs5pkw4SFoB/A9qHxX6nYjY0OYpWR3qCRCHhtn0MyXCQdIs4JvAp4EBYLekbRHxcntnZo0a\nLTTqCRNwiJi125QIB+BKoC8ijgBI2gKsBBwOJVVviIyn+N6OIoeP2dimSjjMA14vPB8ArmrTXKwE\nWhU+U9VooTiT1dNzI9fIRhtXT30iTNYvNoqISXmhMSchfRZYERGfS89vBa6KiLVnjOsGutPTjwKv\nNviSFwG/aHDb6ayMfZexZyhn3+65Pv8yIi4eb9BUOXIYBBYUns9PtfeIiB6gp9kXk7QnIjqb3c90\nU8a+y9gzlLNv99xaU+Ud0ruBDkmLJJ0DrAK2tXlOZmalNSWOHCLilKS1wA5qt7JujIiDbZ6WmVlp\nTYlwAIiI7cD2SXq5pk9NTVNl7LuMPUM5+3bPLTQlLkibmdnUMlWuOZiZ2RRSunCQtELSq5L6JK1v\n93yaIWmjpOOSDhRqF0rqlXQofZ2T6pL0YOr7JUlXFLZZk8YfkrSmHb3US9ICSc9KelnSQUlfSvWZ\n3vfvSHpB0k9S3/8l1RdJej719910QweSzk3P+9L6hYV93ZPqr0q6tj0d1U/SLEkvSno6PS9Dz/2S\n9kvaJ2lPqk3uz3hElOZB7WL3YeAS4BzgJ8Dids+riX4+CVwBHCjU/gJYn5bXA3+elq8HngEELAOe\nT/ULgSPp65y0PKfdvY3R81zgirT8AeBnwOIS9C1gdlo+G3g+9fMEsCrVvwX8x7R8F/CttLwK+G5a\nXpx+7s8FFqV/D7Pa3d84vf8J8DfA0+l5GXruBy46ozapP+NlO3I4/Wc6IuIfgeE/0zEtRcRzwIkz\nyiuBTWl5E3Bjof5Y1OwCLpA0F7gW6I2IExHxFtALrJj42TcmIo5GxI/T8q+AV6i9w36m9x0RMZSe\nnp0eAXwK+F6qn9n38Pfje8A1kpTqWyLi3Yj4OdBH7d/FlCRpPnAD8J30XMzwnscwqT/jZQuHkf5M\nx7w2zWWiVCLiaFp+A6ik5dF6n7bfk3Ta4A+p/RY94/tOp1f2Acep/UM/DPwyIk6lIcUeTveX1r8N\nfJjp1/d/B/4U+Of0/MPM/J6hFvw/lLRXtb8MAZP8Mz5lbmW11ouIkDQjb0eTNBt4EvhyRJys/YJY\nM1P7jojfAkskXQA8BfzrNk9pQkn6I+B4ROyV1NXu+Uyyj0fEoKTfBXol/bS4cjJ+xst25FDXn+mY\n5o6lQ0rS1+OpPlrv0+57IulsasGwOSK+n8ozvu9hEfFL4Fng31I7hTD8S16xh9P9pfUfAt5kevX9\nMeCPJfVTOwX8KWqf+TKTewYgIgbT1+PUfhG4kkn+GS9bOJThz3RsA4bvSlgDbC3UV6c7G5YBb6dD\n1B3Acklz0t0Py1NtSkrnkB8BXomIrxdWzfS+L05HDEg6j9pnn7xCLSQ+m4ad2ffw9+OzwN9F7Srl\nNmBVurNnEdABvDA5Xbw/EXFPRMyPiIXU/q3+XUTcwgzuGUDS+ZI+MLxM7WfzAJP9M97uq/KT/aB2\nZf9n1M7XfqXd82myl8eBo8A/UTufeCe1c6w7gUPAj4AL01hR+0Clw8B+oLOwnzuoXaTrA25vd1/j\n9PxxaudjXwL2pcf1Jej7D4AXU98HgP+c6pdQ+4+uD/hfwLmp/jvpeV9af0lhX19J349Xgeva3Vud\n/Xfx/+9WmtE9p/5+kh4Hh/+fmuyfcb9D2szMMmU7rWRmZnVwOJiZWcbhYGZmGYeDmZllHA5mZpZx\nOJiZWcbhYGZmGYeDmZll/h9qC1rjjVjMdgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f6aae4f0550>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"lens = train.comment_text.str.len()\n",
"lens.hist(bins = np.arange(0,5000,50))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The majority of our texts have between 200 and 300 characters. we can see few texts that have more than 1500 characters."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's now see if no comment is empty. in which we could delete it"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of missing comments in comment text on Training set:\n"
]
},
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"print('Number of missing comments in comment text on Training set:')\n",
"train['comment_text'].isnull().sum()"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of missing comments in comment text on Validation set:\n"
]
},
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"print('Number of missing comments in comment text on Validation set:')\n",
"validation['comment_text'].isnull().sum()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's split the input data from the ground truth labels."
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
"X_train, y_train = train['comment_text'].values, train.iloc[:,2:].values\n",
"X_val, y_val = validation['comment_text'].values, validation.iloc[:,2:].values\n",
"X_test, y_test = test['comment_text'].values, test.iloc[:,2:].values"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"X_train shape (129251,)\n",
"y_train shape (129251, 6)\n",
"X_val shape (14362,)\n",
"y_val shape (14362, 6)\n",
"X_test shape (15958,)\n",
"y_test shape (15958, 6)\n"
]
}
],
"source": [
"print('X_train shape ', X_train.shape)\n",
"print('y_train shape ', y_train.shape)\n",
"print('X_val shape ', X_val.shape)\n",
"print('y_val shape ', y_val.shape)\n",
"print('X_test shape ', X_test.shape)\n",
"print('y_test shape', y_test.shape)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[0, 0, 0, 0, 0, 0],\n",
" [0, 0, 0, 0, 0, 0],\n",
" [1, 0, 1, 0, 1, 0],\n",
" ..., \n",
" [0, 0, 0, 0, 0, 0],\n",
" [1, 1, 1, 0, 1, 0],\n",
" [0, 0, 0, 0, 0, 0]])"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_train"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As it can be seen, our labels are still not yet formatted. we need to list all tags of a comment is easy to understand way. for example:\n",
"**comment1** will have a new tag safe which means safe comment.\n",
"and **comment3** will have an array containing 'severe toxic','obscene','identity_hate'"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Index(['id', 'comment_text', 'toxic', 'severe_toxic', 'obscene', 'threat',\n",
" 'insult', 'identity_hate'],\n",
" dtype='object')"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train.columns"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"classes = {0:'toxic',1:'severe_toxic',2:'obscene',3:'threat',4:'insult',5:'identity_hate'}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a function called convertClass that convert our labels as described above. for comments which are safe, add a new tag called safe"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"def convertClass(tags,classes):\n",
" result = []\n",
" for i,tag in enumerate(tags):\n",
" if tag > 0:\n",
" result.append(classes[i])\n",
" if len(result) == 0:\n",
" result.append('safe')\n",
" \n",
" return result"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's run our function to all labels data"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"y_train = np.array([convertClass(tag,classes) for tag in y_train])\n",
"y_val = np.array([convertClass(tag,classes) for tag in y_val])\n",
"y_test = np.array([convertClass(tag,classes) for tag in y_test])"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([list(['safe']), list(['safe']),\n",
" list(['toxic', 'obscene', 'insult']), ..., list(['safe']),\n",
" list(['toxic', 'severe_toxic', 'obscene', 'insult']), list(['safe'])], dtype=object)"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_train"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Our data is almost ready for training. but one more thing needs to be done. we need to clean the comments by removing wrong characters such as special characters. we also need to change the case of all characters that make the comment to avoid case insensitivity our models. we also need to remove stop words since those are words that are likely to be common to all comments. let's print a sample of 3 comments to see what the comments are like"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array(['Giant Cunt==\\n\\nShe is not liberal but in fact a giant cunt. \\n\\n==',\n",
" \"Sci-Fi Dine-In Theater Restaurant \\n\\nHello, Neelix. I'm not clear on why you are reverting my edits to this page. Please let me know! I'm attempting to adhere to wiki guidelines and am fairly certain that my edits are in lockstep with them. Thanks!\\n\\nJohn\",\n",
" '(only elligable for new accounts)'], dtype=object)"
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X_train[2:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's create a function and call it text_prepare which will be passed a text and return a cleaned version of the text."
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [],
"source": [
"import re"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [],
"source": [
"REPLACE_BY_SPACE_RE = re.compile('[/(){}\\[\\]\\|@,;]')\n",
"NEW_LINE = re.compile('\\n')\n",
"BAD_SYMBOLS_RE = re.compile('[^0-9a-z #+_]')\n",
"STOPWORDS = set(stopwords.words('english'))\n",
"\n",
"def text_prepare(text):\n",
" \"\"\"\n",
" text: a string\n",
" \n",
" return: modified initial string\n",
" \"\"\"\n",
" text = text.lower()# lowercase text\n",
" text = NEW_LINE.sub(' ',text) # replace NEW_LINE symbols in our texts by space\n",
" text = REPLACE_BY_SPACE_RE.sub(' ',text)# replace REPLACE_BY_SPACE_RE symbols by space in text\n",
" text = BAD_SYMBOLS_RE.sub('',text)# delete symbols which are in BAD_SYMBOLS_RE from text\n",
" text = ' '.join([word for word in text.split() if word not in STOPWORDS])\n",
" return text"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's test our function by using a small sample before we can apply the function to all texts."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [],
"source": [
"sample_text = [ 'Possibly related 2 nearby IPs that have also spammed; see talk pages for details:',\n",
" '\"\\nWhen the category was created there was only one person in the category. I don\\'t know anything about the American Protestant groups, so if you need to, you can create the category. Please use Category:Southern Baptist Convention when categorizing individuals who are Southern Baptist.â\\x80\\x94RyÅ«lóng (ç«\\x9cé¾\\x99) \"',\n",
" 'Giant Cunt==\\n\\nShe is not liberal but in fact a giant cunt. \\n\\n==',\n",
" \"Sci-Fi Dine-In Theater Restaurant \\n\\nHello, Neelix. I'm not clear on why you are reverting my edits to this page. Please let me know! I'm attempting to adhere to wiki guidelines and am fairly certain that my edits are in lockstep with them. Thanks!\\n\\nJohn\",\n",
" '(only elligable for new accounts)']"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['possibly related 2 nearby ips also spammed see talk pages details',\n",
" 'category created one person category dont know anything american protestant groups need create category please use categorysouthern baptist convention categorizing individuals southern baptistrylng',\n",
" 'giant cunt liberal fact giant cunt',\n",
" 'scifi dinein theater restaurant hello neelix im clear reverting edits page please let know im attempting adhere wiki guidelines fairly certain edits lockstep thanks john',\n",
" 'elligable new accounts']"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sample_text_clean = [text_prepare(x) for x in sample_text]\n",
"sample_text_clean"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [],
"source": [
"X_train = [text_prepare(x) for x in X_train]\n",
"X_val = [text_prepare(x) for x in X_val]\n",
"X_test = [text_prepare(x) for x in X_test]"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['giant cunt liberal fact giant cunt',\n",
" 'scifi dinein theater restaurant hello neelix im clear reverting edits page please let know im attempting adhere wiki guidelines fairly certain edits lockstep thanks john',\n",
" 'elligable new accounts']"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X_train[2:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can definitely do a better job than I did if you spend more time looking at the data and cleaning accordingly. I am satisfied with the little I did, that I do. so I will move to the next step."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Data transformation\n",
" In this section we will transform our input text into vectors of numbers that represent each word that makes the training corpus"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's first create two dictionaries one to hold all words and the number of times they have been used across the corpus. and another to hold all tags and the number of times they have been assigned a comment. these two will be used later when we want to check what the model is learning about some of the most important words in the corpus."
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [],
"source": [
"# Dictionary of all tags from train corpus with their counts.\n",
"tags_counts = {}\n",
"for tags in y_train:\n",
" for tag in tags:\n",
" if tag in tags_counts:\n",
" tags_counts[tag] += 1\n",
" else:\n",
" tags_counts[tag] = 1\n",
"# Dictionary of all words from train corpus with their counts.\n",
"words_counts = {}\n",
"\n",
"for title in X_train:\n",
" for word in title.split():\n",
" if word in words_counts:\n",
" words_counts[word] += 1\n",
" else:\n",
" words_counts[word] = 1"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Common tags [('safe', 116080), ('toxic', 12427), ('obscene', 6861), ('insult', 6392), ('severe_toxic', 1316)]\n",
"-----------------\n",
"common words [('article', 45147), ('page', 36777), ('wikipedia', 28784), ('talk', 25741), ('please', 24036)]\n"
]
}
],
"source": [
"most_common_tags = sorted(tags_counts.items(), key=lambda x: x[1], reverse=True)[:5]\n",
"print('Common tags',most_common_tags)\n",
"print('-----------------')\n",
"most_common_words = sorted(words_counts.items(), key=lambda x: x[1], reverse=True)[:5]\n",
"print('common words',most_common_words)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are different ways we can represent each word in a numerical way. the most common are bag of words, TFIDF and Word Embeddings. The later has recently attracted attention of many especially when you have a big corpus of millions of words(including n-grams). I will use this technique in another post where I will cover as similar classification exercise using deep learning. for this exercise though, I will use TFIDF which is an improved version of bag of words which uses inversed logarithmic normalisation to penalise those words that are most frequent accross different input text of our corpus."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's use sklearn library to generate tfidf vectors. I will also use n-grams of size 1 and two. you can experiment with size if you have enough RAM on your computer. Also keep in mind that increasing the size of n-grams without increasing the size of samples might generate a lot of features than the size of the sample. this can result into your model being high biased."
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.feature_extraction.text import TfidfVectorizer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a function that convert each set of our traing, validation and test corpus into a TFIDF matrix"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [],
"source": [
"def tfidf_features(X_train, X_val, X_test):\n",
" \"\"\"\n",
" X_train, X_val, X_test — samples \n",
" return TF-IDF vectorized representation of each sample and vocabulary\n",
" \"\"\"\n",
" # Create TF-IDF vectorizer with a proper parameters choice\n",
" # Fit the vectorizer on the train set\n",
" # Transform the train, test, and val sets and return the result\n",
" \n",
" \n",
" tfidf_vectorizer = TfidfVectorizer(min_df=5, max_df=0.9, ngram_range=(1, 2),token_pattern='(\\S+)')\n",
" tfidf_vectorizer.fit(X_train)\n",
" X_train = tfidf_vectorizer.transform(X_train)\n",
" X_val = tfidf_vectorizer.transform(X_val)\n",
" X_test = tfidf_vectorizer.transform(X_test)\n",
" return X_train, X_val, X_test, tfidf_vectorizer.vocabulary_"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the functiion agains the training set, validation and the test. check if most common words of our corpus are still there. this is crucial since the performance our model will depend on the TFIDF representation of the data sets.\n",
"Implement function *tfidf_features* using class [TfidfVectorizer](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html) from *scikit-learn*. Use *train* corpus to train a vectorizer. You can filter out too rare words (occur less than in 5 titles) and too frequent words (occur more than in 90% of the titles). Also, use bigrams along with unigrams in your vocabulary."
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [],
"source": [
"X_train_tfidf, X_val_tfidf, X_test_tfidf, tfidf_vocab = tfidf_features(X_train, X_val, X_test)\n",
"tfidf_reversed_vocab = {i:word for word,i in tfidf_vocab.items()}"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"True\n",
"True\n",
"True\n",
"True\n",
"True\n"
]
}
],
"source": [
"print('article' in tfidf_vocab)\n",
"print('page' in tfidf_vocab)\n",
"print('wikipedia' in tfidf_vocab)\n",
"print('please' in tfidf_vocab)\n",
"print('talk' in tfidf_vocab)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check the size of our training set to find out how many features are we going to train."
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"X_train_tfidf shape (129251, 121181)\n",
"X_val_tfidf shape (14362, 121181)\n",
"X_test_tfidf shape (15958, 121181)\n"
]
}
],
"source": [
"print('X_train_tfidf shape ', X_train_tfidf.shape)\n",
"print('X_val_tfidf shape ', X_val_tfidf.shape)\n",
"print('X_test_tfidf shape ', X_test_tfidf.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training the classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that our different data are ready. let's decide on the training technic. since this exercise requires to predict tags for each comment and since a comment can have zero, one or more than one tags, we will need to use a model that consider each input of tfidf vectors representing words of each comment; and each tag individually and evaluate the probability that the input can be assigned the tag(output 1) or otherwise (output 0). we will use the MultiLabelBinarizer of sklearn to convert each tag into a binary form."
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.preprocessing import MultiLabelBinarizer"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {},
"outputs": [],
"source": [
"mlb = MultiLabelBinarizer(classes=sorted(tags_counts.keys()))\n",
"y_train = mlb.fit_transform(y_train)\n",
"y_val = mlb.fit_transform(y_val)\n",
"#y_test = mlb.fit_transform(y_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that the tags have been converted into binary representation, let's build our classifier. we will use OneVsRestClassifier to train each binary tag individually and we will use basic classify called LogisticRegression as the upper layer. It is one of the simplest methods, but often it performs good enough in text classification tasks."
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.multiclass import OneVsRestClassifier\n",
"from sklearn.linear_model import LogisticRegression, RidgeClassifier"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [],
"source": [
"def train_classifier(X_train, y_train):\n",
" \"\"\"\n",
" X_train, y_train — training data\n",
" \n",
" return: trained classifier\n",
" \"\"\"\n",
" \n",
" # Create and fit LogisticRegression wraped into OneVsRestClassifier.\n",
" \n",
" lr = LogisticRegression(C=4.0,penalty='l2') # use L2 to optimise\n",
" ovr = OneVsRestClassifier(lr)\n",
" ovr.fit(X_train, y_train)\n",
" return ovr"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's run the classifier function against our training set"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {},
"outputs": [],
"source": [
"classifier_tfidf = train_classifier(X_train_tfidf, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's use the model to predict first the output of using our training set. also get the score to be used for evaluation"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {},
"outputs": [],
"source": [
"y_train_predicted_labels_tfidf = classifier_tfidf.predict(X_train_tfidf)\n",
"y_train_predicted_scores_tfidf = classifier_tfidf.decision_function(X_train_tfidf)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's print a sample of 200 comments of our training set and compare what the model predicted vs the true labels"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Title:\tmediamediaexampleogg mediamediaexampleogg mediamediaexampleogg mediamediaexampleogg mediamediaexampleogg mediamediaexampleogg mediamediaexampleogg #redirect insert text\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tstarted insult bullshit quirk transformer article go home sleep\n",
"True labels:\tinsult,obscene,toxic\n",
"Predicted labels:\tobscene,toxic\n",
"\n",
"\n",
"Title:\tconclusion following comment added article removed appropriate talk page article space request posted\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tgood start especially temperament section appearance section needs expanding article needs references pictures colours would nice talk\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tperhaps article time contains quotes reverse everything believe according basic principles physiology declares professor varro tyler purdue university expert herbal remedies believe greater dose greater physiological response believe even drug left still get response advocates claim evidence homeopathys efficacy emerging citing list scientific papers published recent years reputable journals pediatrics british medical journal lancet nature handful reports far definitive ultimate test scientific validity whether results duplicated far belief entire process contend scientists\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tdont disagree wrote earlier comment find web sources articles happy use preference links langans website check links legitimate secondary sources chances tampered remote reasons tampering might occur advanced course anyone access publications may check accuracy bring problems attention editors sources article span various publications television programs whether span web perhaps major consideration nevertheless supporting links available used preference think also pointed already pointed one user removed links claiming improper later restored one links wished use support argument clear evidence inconsistency rejecting links said users appear satisfied links may remain replaced links become available would hope issue need reopened without good reason\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n"
]
}
],
"source": [
"y_train_pred_inversed = mlb.inverse_transform(y_train_predicted_labels_tfidf)\n",
"y_train_inversed = mlb.inverse_transform(y_train)\n",
"for i,text in enumerate(X_train[20:26]):\n",
" print('Title:\\t{}\\nTrue labels:\\t{}\\nPredicted labels:\\t{}\\n\\n'.format(\n",
" text,\n",
" ','.join(y_train_inversed[20:26][i]),\n",
" ','.join(y_train_pred_inversed[20:26][i])\n",
" ))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Our model seems to be doing well on the training set. but that was expected. let's now predict the validation set"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {},
"outputs": [],
"source": [
"y_val_predicted_labels_tfidf = classifier_tfidf.predict(X_val_tfidf)\n",
"y_val_predicted_scores_tfidf = classifier_tfidf.decision_function(X_val_tfidf)"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Title:\tharbulak prod template added article harbulak suggesting deleted according proposed deletion process contributions appreciated article may satisfy wikipedias criteria inclusion deletion notice explains see also wikipedia wikipedias deletion policy may contest proposed deletion removing dated prod notice please explain disagree proposed deletion edit summary talk page also please consider improving article address issues raised even though removing deletion notice prevent deletion proposed deletion process article may still deleted matches speedy deletion criteria sent articles deletion may deleted consensus delete reached endorse deletion article person made substantial edits page please tag\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tgirl ever sees naked cry laughing hard kill\n",
"True labels:\ttoxic\n",
"Predicted labels:\ttoxic\n",
"\n",
"\n",
"Title:\tlot frikking stop use fooking method want test open book light current pissed dont even understand makes tick feel bigger man blocking youre youre small men heck point ill even settle without apology accept aint sock let banter bugs 21741226121\n",
"True labels:\ttoxic\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\talready left page either discuss banned wikipedia\n",
"True labels:\ttoxic\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\trequested move requested move heinrich pette institute institut german located hamburg survey feel free state position renaming proposal beginning new line section support oppose sign comment since polling substitute discussion please explain reasons taking account wikipedias policy article titles discussion additional comments\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tgiving warnings equivalent banning one blocked considering offensive disgusting editor\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n"
]
}
],
"source": [
"y_val_pred_inversed = mlb.inverse_transform(y_val_predicted_labels_tfidf)\n",
"y_val_inversed = mlb.inverse_transform(y_val)\n",
"for i,text in enumerate(X_val[84:90]):\n",
" print('Title:\\t{}\\nTrue labels:\\t{}\\nPredicted labels:\\t{}\\n\\n'.format(\n",
" text,\n",
" ','.join(y_val_inversed[84:90][i]),\n",
" ','.join(y_val_pred_inversed[84:90][i])\n",
" ))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Not bad either. However our training data set is big and has a hundred of thousands of comments. the validation set is also big. we can 't evaluate each example by comparing labels. we need some numbers that can help us to understand how the model performed. we will use a combination of accuracy, f1 score, precision, recall and roc curve. you can read more about the evaluation methods by following the links.\n",
"- [Accuracy](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html)\n",
"- [F1-score](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html)\n",
"- [Area under ROC-curve](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html)\n",
"- [Area under precision-recall curve](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.average_precision_score.html#sklearn.metrics.average_precision_score) "
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.metrics import accuracy_score\n",
"from sklearn.metrics import f1_score\n",
"from sklearn.metrics import roc_auc_score \n",
"from sklearn.metrics import average_precision_score\n",
"from sklearn.metrics import recall_score"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's create a function that prints the metrics that specified above"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {},
"outputs": [],
"source": [
"def print_evaluation_scores(y_val, predicted):\n",
" \n",
" ######################################\n",
" ######### YOUR CODE HERE #############\n",
" ######################################\n",
" print('Accuracy:',accuracy_score(y_val, predicted))\n",
" print('F1 Score:',f1_score(y_val, predicted, average='weighted'))\n",
" print('Precision:',average_precision_score(y_val, predicted,average='weighted'))\n",
" print('Recall:',recall_score(y_val, predicted,average='weighted'))"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy: 0.918256510235\n",
"F1 Score: 0.922387498956\n",
"Precision: 0.879203307783\n",
"Recall: 0.916125405946\n"
]
}
],
"source": [
"print_evaluation_scores(y_val, y_val_predicted_labels_tfidf)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see that the accuracy is quite impressive. but that was not a surprise and it does not mean the model is performing well. You remember when I discussed about sparse data? this is what I was referring too. since one class is 90% dominant, predicting everytime that class will give us a 90% accuracy. almost what we see here. does that mean we are not doing well since we are getting only 91% accuracy? not yet. let's think about the precission and recall. for a quick reminder see this [Wikipedia](https://en.wikipedia.org/wiki/Precision_and_recall).\n",
"It is clear that the precision and recall are not telling all as well. same as F1 score. so we still need to analyse the area under the curve."
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {},
"outputs": [],
"source": [
"from metrics import roc_auc\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXd8VFX2wL8nPSGVHgm9hRq6+AMk\nK4KouyqwiKKgwuoioiIWsLsqqwLqirigWBBdFQsgYlekSFFA6b2T0AMJKaTNnN8fbzIzCZNkgIRJ\n4H4/n/lk3n3n3Xvey5t33j333nNEVTEYDAaDoTj8fK2AwWAwGCo2xlAYDAaDoUSMoTAYDAZDiRhD\nYTAYDIYSMYbCYDAYDCViDIXBYDAYSsQYCkOlR0QeE5G3fdBuPxHZLyIZItL+fLdfGRCRPSJyZQn7\nvxWR286i3mdE5MNz087gLcZQVCBEpLuILBORNBE5LiJLRaSzr/U6W0Rkhog8X97tqOq/VfUf5d2O\nByYBo1Q1XFX/LLpTRFREMh2GJFlEXhER/yIyfxWR3x1yKSLyPxGJKyITKyLviMhBEUkXkS0i8i8R\nqeKhzQaOdgPK/GzPEU8Pd1W9WlXf95VOBu8whqKCICKRwHzgdaAqUAf4F5DjS70MJVIf2FiKTIKq\nhgM9gUHAsIIdIvJ34CPgP0B1oBXW//tXEYlxyFQFlgOhwGWqGgH0BqKBxmV6NgZDcaiq+VSAD9AJ\nSC1hvx/wBLAXOALMBKIc+xoACtwB7AdOACOAzsA6IBWYUqS+YcBmh+z3QP0S2u4KLHPUsxZIdJRX\nBZKAvzm2w4EdwFDgLiAPyAUygK8cMpcAXwBHgd3AfW7tPAN86ji3dKyHcCe3/WOBZMe+rUAvt+M+\ndJO7znFsKrAQaOG2bw/wkOO6pAGzgJAzueZAsOOcFMgEdhZzvAJN3LY/Bd5wfBdHvY94aHMD8Kxj\n+3lgPeDn5X20z9FuhuNzWUn3jofjEx3/00ccsgeBG4BrgG3AceAxN/kZwPNFjy9yva8E+jruhTyH\nXmsd+xcC/yjhfFoBPzraPVzQtof/+WfAIcf/dDHQym3fNcAmx32TDDzkKK+O9XKW6qh/ibfX+WL7\n+FwB83H8IyASSAHeB64GYorsH4b1EG6E9UCeDXzg2NfA8XCYBoQAfYBsYC5QE6t3cgTo6ZC/3lFX\nCyDA8RBZVoxedRx6XeN44PR2bNdw7O/j+IHWBKYDn7sdW/Qh4gesBp4Cghznsgu4yrH/GYfe1wD+\nwAvACse+5lhG8BK3c27sdtyHju/NsB7evYFArAfeDiDIsX8P8DuWwaqKZSxHFHPuxV5zx/5ChsDD\n8c79QDzWQ/cBt20FGno47l/Acsf3FcC/zuA+KrgXArw9jyLHJwL5jv9RIHAnllH/CIjAenCfKtDb\nw/84EQ+Gouj/yW3/QooxFI72DgIPYt3XEcClnupynGMElhH/D7DGbd9BoIfjewzQwfH9BazfTKDj\n0wMQXz8LKuLHuJ4qCKp6EuiO9SOfDhwVkXkiUsshcgvwiqruUtUM4FHgpiK+6OdUNVtVf8B6WH6s\nqkdUNRnrbalgwHUE8IKqblbVfODfQDsRqe9BtVuBb1T1G1W1q+qPwCqshzmOtj4DfnaU/bOE0+yM\nZWCeVdVcVd3lONeb3GR+dbRlAz4AEhzlNqyHQEsRCVTVPaq600Mbg4CvVfVHVc3DGkcIBf7PTWay\nqh5Q1ePAV0C7YvT15pqXxh8ikollkBYC/3WUV3f8PejhmINu+6sVI3MmnOl55AHjHdfvE4cur6lq\nuqpuxHo7Tyjm2LLkr8AhVX3ZcV+nq+pvngRV9V3H/hwsI5IgIlGO3XlY902kqp5Q1T/cymOxetN5\nqrpEHRbEUBhjKCoQjgf37aoaB7TGeuv9j2P3JViugwL2YvUGarmVHXb7fsrDdrjje33gNRFJFZGC\nbrdg9R6KUh8YWCDrkO+O9QMr4C2HvjNUNaWEU6wPXFKkrseKnMMht+9ZQIiIBKjqDmA01kPgiIh8\nIiKXeGij0HVSVTtWT8T93Iq2EY5nvLnmpdHBUf8g4FKgYAD6mONvrIdjYt32pxQjcyac6XmkOAw1\nWPcNFH8vlRkistEx8J8hIj2AuoCnl4Gix/mLyIsislNETmL1YsBlbAdgvcTsFZFFInKZo3wiVk/r\nBxHZJSLjyvSELiCMoaigqOoWrG59a0fRAawHbQH1sFwEhzlz9gP/VNVot0+oqi4rRvaDIrJVVPVF\nsH6kWIZiJjBSRJq4n4aHunYXqStCVa/xRmlV/UhVu2NdBwVe8iBW6DqJiGA9cJK9aaOkujjLa64W\nn2INSj/lKN6KNRYw0F1WRPywHmw/O4p+Avo5yr1qzkNZWd47RckEwty2a5cgW+Lbuqq2UmsGWbiq\nLsG6Xxp5ocNgLHfqlVhjSA0c5eKod6WqXo/lHp2LNVaEowfyoKo2whrXGiMivbxo76LDGIoKgojE\ni8iDBVMjRaQucDOWjxrgY+ABEWkoIuFY7qJZDtfRmTINeFREWjnaihKRgcXIfgj8TUSucry5hYhI\notsUzsewHgDDsN7QZrpNAT1M4R/670C6iIwVkVBHfa29mQIsIs1F5AoRCcYaxzgF2D2IfgpcKyK9\nRCQQy7+dgzUYf6aU5TUHeBG4U0RqO1wcDwFPiMhgx3WtDbyNNV71quOYVxzb7xe4BkWkjmOqbVsP\nbRzFui7u172sz8OdNcA1IlLVof/oEmQPAw3OwOjNB2JFZLSIBItIhIhc6kEuAut/nIJltP5dsENE\ngkTkFhGJcrjSTuK4bxxTk5s4XibSsNybnu6pix5jKCoO6Viuid8cPu0VWLNfHnTsfxfLZ78Ya7ZQ\nNnDv2TSkqnOw3sY/cXTVN2ANoHuS3Y/1tvYY1kNoP/Aw4CciHYExwFCHq+IlLKNR0IV/B8s3nCoi\ncx0yf8UaE9iN5V55G+stsDSCsR60x3ANnj/qQd+tWOMqrztk/4Y1KyvXizaKUmbX3KHbekddDzu2\nZwFDgAewHnKbsMZTuhW48BzjKP+H5U//TUTSsXobaVhuk6JtZAHjgaWO6961rM+jCB9gzYTbA/yA\nNYusOD5z/E0RkT9KkAOsN36sSQl/w/qfbwf+4kF0JpY7LRnrGq4osn8IsMdxr4/AGrMBaIrVY8vA\n6u39V1V/KU2vixExYzcGg8FgKAnTozAYDAZDiRhDYTAYDIYSMYbCYDAYDCViDIXBYDAYSqTCRZgs\njerVq2uDBg18rYbBYDBUKlavXn1MVWuczbGVzlA0aNCAVatW+VoNg8FgqFSIyN7SpTxjXE8Gg8Fg\nKBFjKAwGg8FQIsZQGAwGg6FEjKEwGAwGQ4kYQ2EwGAyGEjGGwmAwGAwlUm7TY0XkXaxIoUdUtbWH\n/QK8hpVQJAu43S3zlOFCJscG+XZQtWLNhvhDoP/pcnaFI5mQa7cyC/gJ1InwXOeJbOtTEOMyOhiq\nhXqW3Znqah+gcbTn9tNzYU+aq87wQGgUXXydqW7tN46GmJDT5XJtsNotDUSgH3QqJoXD/pOQlOHa\njguHupGeZVcdsuouoGNtsmx28vPtBelACQsLJDA9FzYfd8nFBGOPr8aJE6ecl0MEqlULg03HrGta\nQHw1MoL9ycjIddYZHh5ERFAArHRLwhfoD5fGcuBAOjab3VnvJZdEEJCcAftOumTrRpBVM4xDh1zn\nGRYWSO3a4bDiQOFz6hJLckoWmZl5zvbr1IkkPMcGG4655GJCyG9Zja1bjznbDgjwIz6+Oqw/Csfd\nzqlVdQ7l2zh0KMNZZ+3a4cRWDYPlbilMAv2hWx3Wrj1Ebq7NWW+7drUJSs6w7pMC6kdyPDqYLVuO\nOeuMiQmlZcsasDTJuv8L+L86bNhxnJSULJdKrWpSXYF1R11y1UPJia/Kr7/uc7YdFOTP5ZfXh7VH\n4Ogpl2zbGuw4mc2OHced7TdpUpVzoTzXUcwApmCFAPbE1Vhhfptihdee6vh74XP8FCRnWD+CHBuE\nBkD7YpKNfbfburkX7IUOteHG5pBQ83S5U/kw6iew2cGm1gPoXY+Rw+GzrTBtjfUgVoWBzeGeDqfL\nLUuG6+dAbBWHLPDt36Geh4fVr0lwy3yX3P/VgU+v89z+P7+Hr3e5tt/tC39rcrrciWxoM8O1XT0U\nNg/3XOe76+FFtyyZD3SCx7oWElFVRAT6z4UDbg/gNbeRFh5IVlYedrtitysxMaGE/3YQbv7KJXdF\nPewf/42NG4845QDat4+FZ5fBN27nNONqkhJqsG1bCvn5dnbvPkHDhjH0SagN1812ydUIg03DeOqp\nXzh0KAO7XcnPtzNhQm9qfrIFJvzukn2oM6uvrMtDD/2IzWEEOnSozZTHqsETswo/1P/bmwmv/8Zv\nv7kedk8/1ZOuQYEwwe06tatF6l1tufVWl04x0SF89NEAmPJbYaP2UBe+3HqED/+33ll08+A23HpN\nM3joR5dcZAhM680jQ+eQctz1AHt/xg1UX5QEX2x1yfZvxtq6VXj2ucXOos6dLuGZZxLhmR8t41vA\nlCt5a8rv/P6765yefOJy65wmuV2nhFqk3tmGh4bOKXROH37YH6b+Dn+4ndOYzvy07Sj/+8jtnG5u\nbZ3TuNPPaeI93p3T1rpVeM7DOdmf/4ldATXICQy2dmh7/jdvK1u3ugzd4MFtiA8IgM/d6mwUTfrV\nDZkycamzKLxKEDEh3Sy5HamF2v8lOZVfftnjLOrePY5zoVzDjItIA2B+MT2KN4GFqvqxY3srkKiq\nJeYH7tSpk57XBXfpubD9BCSnQ7YN6kZAVw8ZONNyoMl0iK9qPbQzcmHWdac/1Odss94UpvzpKmtf\nE3640XP79/4En2xxbb/eC25qcbpcVh7Uf9O1HRYAe0d4rnPqn/CU64ZjRAI816OwjF0tuZkbYZfb\n29Jvt5JRM4zU1GzsdsVmsxMZGUy1jSkw4EuXXPc6MKcfy5btJyMjl/x8OzabnSuuaEiVUT/DfLcM\nl+/0Zb6fnbfeWo3NZj2Ar7uuGXcPbA0t3nHJVQ2Brf+gf/9ZbN9+3Pmw/OabW2jy5c7ChmJ0R37t\nWYc+fT4gL8+S69atLr/+OgwSZpCemkdqhKPHMes6Hp+8gq+/3g5Anfi6DBrUio5RYfDRJledjaPJ\nHxTP448vcBb5Bwj/frY7zNlU+Mf6t8b8nprF99+7zvOyrnFceVldeHMt+Nkg+hiE+EGfBnz33Q4y\nMlwpM/r2bUL4gZ0gm0DFWa7i6ggB+IlrY68kkOJXlxDNpLzJJYSd/hfHe11l57fZT5OStI4DWxev\nVtVOZ1OHL1dm18FKglNAkqPsNEMhIncBdwHUq1ev7DRIOQW/7INjp2BYGwgq4n7IyIWJv8PUNa6y\n65p4NhRfbIOW1WCTW8roLA8JxGqEwY3xhQ2Fe/e6KAFFhpEy86weSdUibhX/InKOB/OKFUkkJZ3k\n2LEskg5mMeDvrWiOP4QFux9M2qFT9O37ITk5+YTERBFVLYJJk3rDsjQIj3GJJtv5eu5WPvlko7Po\n6muaMLhjHTbd0BW7nyAK1AyFnzKYO/cw6emuB+DO/OpENGwE/dwMaEoVtmTmcDKmPiJ2okOy2ZYm\nvLPkEAx2S+IWlg4fL6NJ21Dimlk9MA0M46UfICKkJjx2rUs20A/7ehjy/M3OIj9/YcxbSWTedS12\ncftfL8onOKET/RNcv6E9ebDnGNCnfeHruiqHLv26FSr6YqUd4uLB/aUtHfCHlte4MpCm4XjxTCzc\n02EPEN8GcSv6/hgQ1JHKgUkKd2b44U8O1YO2lC5aBjSoV4VNi5afUx2+7FHMB15U1V8d2z8DY1W1\nxO5CmfQoTuXDlbNg2wlX2eqhnl0qS5Pghrmu7Z514YNrLXeRO/l2uPVr+Nltlfxn10GiB8O2MxW6\nfujabhINy2/1rOvDC2HGBte2v3Bw6WA+/2Ybuw7lciy/CgA3XN8cXncb4hHh2M3xrFlziL1H8qnT\nogwN7AVEnH3DaWU2ArARSHvb12dWmc3N+Ab5k2dTcgteAqoEniYebMvhZFAkSRHFuwVC8rNZWDeR\ntKBixiYARGh09BQDliSRHeBP66PHCRrTkfGv/+bmphEee6w7l0aHwv82u45tFkPqXxszbNiXBVUR\nFR3Cu+9cD//bBFvdxjMGt+CLjYeZNcv1knBrv4aMu7M93PGtSy46GGZcQ+fO051jDyKwYsU/uGRx\nEsxye0AObM4vsaEMHz7PWZSY2IB3370ehn9X2J02/SrufuJnfv55t6NO4Y03ruHK6DB4wS2pXYda\nHPtnAj17znC2Xa1aGIsW3W7JrTrkkh17KW/+mczUqaucdd55ZwdG3twG7vzeJRcVDO/05dprP+LI\nkUxnvV99dTO1FifDp27nNKAZy+qF89BDPzjrvOyyOCZN6mO5Xd3HSKb14bGXl7F0qeudefz4K+ge\nFly4h9y+Jsf/mcDAgZ85246JCeWzzwZarkQ3F+Gm/lG8vnApu3e7fvO9e1floYeuPesexcXrekr8\nBDa6DYDN6w9tqkN4UGG5dUehV5HsjotugpbVT69z2LfwlZtL5f1r4BoPueEPZVi+d3+BzrHWIOXU\nPmzadJTVa49y8Gg2KSlZ9OzZgPaZNtiSYvUYktKhdwO+yA5m7cGz7wyG63FKfgsUMqQal+Z/VoKM\nZzIlms62OaULlsKJ4BhOBhd5OKoSmXuSRXE9i0gLQWGH8fPPxnuUEL801tZsx65oD/+jM+BUQCgb\nq7Wyfr2ltuqZWbM2kJqajZ+fEBDgx/XXx1O1aK8RSEvLZvXqgwQE+BEQ4EdkZDCtW3sYszJcdGRl\nZfH8888zceJE/P392bBhA02auMb+RKRSup7mAaNE5BOsQey00ozEGbEnzXIHPdjZ8/7LLilsKGZv\ns2bVXBpbWK5qiDUusT8dOtWy3EmejATA413hrgSoVQVC/U9zDx04kM6BA+kktI9ly6q7WLYll9Ag\n4eQpO2v/exzwB2pb/5VaMGcLzAGo4pjp0wQr47Eb6Xu2k39oD8OHOVwkaoOUTZCTCif3kJ1jJ0Qy\nySOIrrbPqalFKvCWRteCwzmyb38aW7ekIGK9LcXVjaR5s2oeDlLWrzvIT3sS8PcXRIT193bhaGgN\nfost3r+dGhzNqcCws9OzAnNNCfsGDTrtXcojUVEhXHFFw7JRyHDB8O2333LPPfewe7f1+x4+fDjV\nqnn6TZ4d5Tk99mMgEaguIknA00AggKpOA77B+u3swJoee0eZNf7+BnhoofX9jtan+/PBMhRvr4MG\nkda4Qd2I040EQFwE/HGbd+02jrE+buTl2UhMfJ9ly/ZTo34tev/zWsJ+P1FMBS4CM5KoEiqEhJzu\nsiAnjUN+TXkxO4FqtZKgFrDaOxVnthxCVkAYOf7BPNxzInn+QaUfdC7081xsMrUbDOdOcnIyo0eP\n5vPPPwegbdu2TJs2jcsuu6xM2ylX11N5UKLryWaHRxZZM3UK+PBaa1wh0K/wgG9mHuTZINrDXPcz\nJPlINr9tOcXc4/4cDBYa7M4jr4gJDvQwrm33U7JqplE1dDPxKVuo7beZ3gfnE83h04U9kBJSlR3R\np08rDc0/xdGwGiyPvYxf6v6FY6HV2Vi9FTa/ihFV/hrgDL3/BoPBAzfccANffvklYWFhPPvss9x/\n//0EBHj+nVdW11PZk5Zjzed3Z1myNXh0c5EppVUCcXRwzhibXfl5TSbzfkwiJ8LlHw4mnwaO754M\nQwFtbD8wMO9pYnXbaa4kd/ZG1OO5rk9il8IzmjKCwvmhfh/SQopZ/FWEa4C1XkkaDIaKTn5+vtMY\nvPTSSwQGBvLyyy+X7YzQIlxYPQqwFlJdNxv2nrR6EWMvhZHtPK+89ZLdh/PZsC+PlTtyOXii+Kms\nKdX9ORWTxfPBW7nkxNfE+B+Hra6B8AByCSDPdUCkY+rkyb3Q9i6w5UDzmyC0GtTq5NXgqMFguDhI\nS0vjiSeeYNu2bXz33XfW4tEzwPQo3LkkHGbfALd9A5MSoWMx4RGAk1l2HItrOZRq43i6nUOpNpJT\nbGxJzkOwfOm5JfQOtnXIJDY7iSZBPzF9+VOFV4YUQSMbWoPC1VtDwj/P5uwMBsNFhqry2WefMXr0\naA4ePIi/vz9r1qyhffv2pR9cRlx4hgKs9RALBnl8I//2j1PMX3WqxId/cTSo+hsn/fPokf0J1x75\n2JoDtLQY4SY3QP4piB8MUY2gTrczfgMwGAwXNzt37mTUqFF89913AFx22WVMmzaNtm3blnJk2XJh\nGgo4zUjsOZLP+M9PehSNCrNk07KUy5oHkZGtxIblkPfrRP4W9g6ghHMcST792NTgKDKCqxF3chd0\nGA3xN0Nsl7I+G4PBcJExadIknnzySbKzs4mOjuall17iH//4B35+5z/od+U2FOm5EFH69M6sHPtp\nRuK+a8NpUDOAiFAPFz3jALxZB4pM5V9Spzs1bbk0b/Z3sNug9R1EV6mFd0PKBoPB4D1ZWVlkZ2cz\nZMgQJk2aRM2avltYWXkNxaEM6PwB9GkIt7e2gtAV49q5/x1XsLZ/XFmFS5sFe5QDYN8C+KxXoaLu\ng5awtE43cvPtBJ7DoLjBYDAUx9GjR9m6dSvdu3cHYOzYsSQmJnL55Zf7WLPKnLhoxkYrmuu8HVbY\n6MHzPYqlZrpCVTS/JKBkI7F9TiEj8UbCSPzG2Fga151aqw+SlZVX/LEGg8FwFtjtdt5++22aN29O\n//79OX7ciq8VHBxcIYwEVNYehWrhWO1gLarzwLd/uGLHP3h94aQ3S5fu46P3fuXuK6dR59h6YnJc\nPY//u2kpy+v8HwCpadlEdfIQMdZgMBjOgQ0bNjBixAiWLrVmxfTu3ZusrCyqVj23RENlTeXsUdgV\nBrd0heAO9Dt9QR2Qkm5jwfocABrXCnDOOlq//jDDhn1J9+7v8UaLW2idvKSQkRh07SdOI3ENVnwd\ng8FgKCsyMzMZO3Ys7du3Z+nSpdSqVYuPP/6Y77//nri4c0syVB5Uzh6Fvx+M6QRX1IURP0LzqlYY\nYDemfpfOH7tcrqLhV1rhuL/5ZjsTJixl2a87yX3p3879eX4BBA5aDLU6Mss/iCLxYg0Gg6HM+Pvf\n/+5cNDdy5EjGjx9PdHTFnRZTOQ1FAe1qwc+D4EhWoeJHP0zl2EnX2MTgy8OoEWUNQrdqVYMPXmtF\n3Z8KxyAMeiDPBKozGAznhbFjx3L48GGmTp3KpZdW/EyBldtQgBWzqWGUczMnTwsZiWkjYvD3c82G\nqh9XBT53rXM4UCWWenft4xpVEzLDYDCUOfn5+bz++uvs2bOH1157DYDExERWrVrlkzURZ0Pl0PIM\nmDjXtV5i+siqhYwEQPKMls7vw/u8TZ0RB7D5BfC1MRIGg6GM+f333+ncuTNjxoxh8uTJbNzoimxd\nWYwEXGCGIulYPnuPWkH76tfwsN4hP5s6qTss2fA6vNtmOFByQhmDwWA4U1JTUxk5ciRdu3ZlzZo1\n1K9fn6+++opWrVr5WrWzovK7ntz45FfXWMXDN1hpNG02OzabEhTkDz+NdO6PG77DjEkYDIYy55NP\nPmH06NEcPnyYgIAAHnzwQZ588kmqVKnia9XOmspnKI5nw0eboE4E1AmH+pEQ6I9dla0HrEh/HRsH\nERxouZKefXYRy5YlMfuDy4nY+B4AR0OrUyPATHk1GAxlzw8//MDhw4fp1q0bU6dOpU2bNr5W6Zyp\nfIbiYAbcv8C1veRmiK/GgeOuPBGDe1hBmmw2O889txhV+P7p6fy9mbX/wSF/MvN86mwwGC5YcnJy\nSE5OplGjRgBMmDCBHj16cNttt1WqcYiSqHxnke+a0YS/QGNr7vGm/a41E5Fh1ml9/PEGrMlMdv7e\nbAkAi+IuZ2ZExVvQYjAYKh8LFiygbdu2XHvtteTm5gJQvXp17rjjjgvGSEBlNBTu1It0Zq7b5nA7\nuQ9iz5q1EVDsE591ls3o+/55VdFgMFx4HD58mCFDhtCrVy+2bdsGQFJSUilHVV4qn+upZhhc18xK\nedrQtZJx71HLUHR1C/o3e/aNBE52neKfNdrxXlSD86aqwWC4sLDb7UyfPp1x48aRmppKSEgITzzx\nBA8//DBBQaWnPKisVD5DERsOU/ucVlyQsa5WtKuTFHjoV+f39MBwOgz908x0MhgMZ02/fv2YN28e\nAFdddRVvvPEGjRs39rFW5U/ldj25kZVjmYBa0W7rJ5aMdX6NvC/9fKtkMBguMPr370/t2rWZNWsW\n33777UVhJKAy9ig8kJPn6idUj3DYvrxMOPgbALM7PugLtQwGQyVn3rx5JCUlMXKktQZr6NCh9O/f\nn4iIiFKOvLC4IAzF7BWuhXaqCgh80sNZdn+H+wGzAttgMHjHvn37uO+++/jyyy8JDg6mb9++NGrU\nCBG56IwEXCCup3V7ramxVYKFyy+fwfz52+DIn9bO2EtJirSSGn3tKwUNBkOlIC8vj5dffpmWLVvy\n5ZdfEhERwYQJE6hfv76vVfMpF4ShCA+xVmFf2dzOsmX7GXfnf5z7Qm5c6COtDAZDZWLFihV06tSJ\nhx56iMzMTAYOHMjmzZu577778Pf3EDvuIqLyGYptJ+Bvs2HQPDhpZa/bc8Ralb3u990AXBO/3Sme\n4wjVYdxOBoOhJJ588knWrVtHw4YN+frrr/n000+pU6eOr9WqEFS+MYpTebDigPXd34+jJ12hO35d\nYBmIxMZ7AFhQ9y8AZkqswWA4DVUlPT2dyEgrgOiUKVOYOXMmjz/+OGFhYT7WrmJR+XoUBfgJhAbw\n1cpTzqIP3r+ewYPbcE0LK5T4T/Wv9JV2BoOhArN161auvPJK+vfv75gAA82bN2f8+PHGSHig8hqK\nsADwE6fbKcAPqlUL43/vuRbjfdPQOJwMBoOL7Oxsnn76adq2bcuCBQtYs2YNe/bs8bVaFZ7K53pq\nGgPT+4HNCg6Yfsr6O6i74y1gz3dO0bU125139QwGQ8Xkxx9/ZOTIkezYYXkchg0bxoQJE6hWrZqP\nNav4lGuPQkT6ishWEdkhIuM87K8nIr+IyJ8isk5ESu8ChAVCtzpweV1sdiUj2+o2xsY4ZiUstlZj\npweGA2YQ22C42FFVhg0bRp/vPQ6FAAAgAElEQVQ+fdixYwctW7Zk8eLFvPPOO8ZIeEm5GQoR8Qfe\nAK4GWgI3i0jLImJPAJ+qanvgJuC/Z9JGQdpTgKaxjs5R2i4APm/2d8CsnTAYLnZEhAYNGhAaGsoL\nL7zAn3/+SY8ePUo/0OCkPF1PXYAdqroLQEQ+Aa4HNrnJKBDp+B4FHDiTBlZsy3F+9/MTUNf8ptc6\n3G96EwbDRcqaNWs4ePAgV199NQBjx45lyJAhNGzY0MeaVU7K0/VUB9jvtp3kKHPnGeBWEUkCvgHu\n9VSRiNwlIqtEZNXRo0ed5ZkOt1PjWo7TWP4v5761NduZ3oTBcJGRnp7OmDFj6NixI7fddhvHjx8H\nIDg42BiJc8DXs55uBmaoahzWcMIHInKaTqr6lqp2UtVONWrUcJZvSbJCd8x6exlxca8UMhQGg+Hi\nQVWZM2cOLVu25NVXXwVg8ODBBAYG+lizC4PydD0lA3XdtuMcZe4MB/oCqOpyEQkBqgNHiq31VD5s\nOgZRwZw8ZfUo0o6cIP+ky2vVZNj24o42GAwXGHv37mXUqFHMnz8fgE6dOvHmm2/SoUMHH2t24VCe\nPYqVQFMRaSgiQViD1fOKyOwDegGISAsgBDhKSWw7Dj0/If/Bhc6iA1v206jqCef2zpgmZaC+wWCo\n6KgqAwYMYP78+URGRjJlyhRWrFhhjEQZU26GQlXzgVHA98BmrNlNG0XkWRG5ziH2IHCniKwFPgZu\nV1WvIm6sr+ma1hYSCJEh1sB2nn9kcYcYDIYLBLvdWj8lIkyaNIlBgwaxZcsW7rnnnos+gF95UK4L\n7lT1G6xBaveyp9y+bwK6nU3dtkDXzZCaOo7c7++BDSB1e56ltgaDoaKTkpLCuHHWkqzp06cDkJiY\nSGJiog+1uvDx9WD2mRMaCC2rcTgsFIAOjazBqqBgK0rsJptJeWowXGioKu+//z7x8fG8/fbbzJw5\nk6SkJF+rddFQ+QxFsxi4ow25bWsBYC9wVG2aCcAbzW8CzIpsg+FCYfPmzfzlL3/h9ttv59ixYyQm\nJrJ27Vri4uJ8rdpFQ+UzFHaFS2PZgNWDiI12uKBOHQNgX2Q9wKzINhgqO6rKk08+SUJCAosWLaJ6\n9eq8//77LFiwgPj4eF+rd1FR+QyFn0CLaoQEWlnt/P0BW65z96I4M0ZhMFwIiAjJycnk5eVx5513\nsnXrVoYOHYqI+Fq1i47KFz3Wwa5D+QDE1wmE1a86y08FmljyBkNl5cCBAxw7doy2bdsCMGHCBIYP\nH063bmc158VQRlS+HoWDGlGWy+nlSb+StPh/PtbGYDCcCzabjSlTptCiRQtuuukmcnMtL0H16tWN\nkagAVFpDcfCEFTn245l/EpC+E4Dnuj4BmIFsg6Ey8ccff9C1a1fuvfdeTp48SePGjTl58qSv1TK4\n4ZWhEJEgEakYy533p5M7dpFzMzM1k6Q0a5HduupWd9UMZBsMFZ+TJ09y//3307lzZ1atWkVcXByz\nZ89m3rx5VK9e3dfqGdwo1VCIyLXAeuBHx3Y7EZlT3ooVy/FTHPrGNX9a7TY61T0IwN7I+r7SymAw\nnAGqyuWXX87kyZMREcaMGcOmTZvo16+fGayugHjTo3gWuBRIBVDVNYBPexfb67siyE5+2TXL6c+a\n7X2hjsFgOENEhAceeIAuXbqwatUqXn75ZSIiInytlqEYvDEUeaqaWqTMq3hM5YW/I192bIw/994W\naxUGhpPvb0IKGwwVkdzcXF588UUmTpzoLBs6dCjLli2jXTuT276i48302M0iciPgJyINgfuAFeWr\nVgnERWDr1wzyIT4uANL3WuV5GT5TyWAwFM+SJUsYMWIEmzZtIjg4mKFDh1KrVi1ExATwqyR406MY\nBXQE7MBsIAe4vzyVKpFqoRxpYYXvCPATyLDyUOyp1spnKhkMhtM5duwYw4YN4/LLL2fTpk00bdqU\n+fPnU6tWLV+rZjhDvOlRXKWqY4GxBQUi0h/LaPiEAD9rsCstyw6ZywE4EGTNfDJTYw0G36KqzJgx\ng4cffpiUlBSCgoJ49NFHGTduHCEhIb5Wz3AWeNOjeMJD2eNlrciZUDBAElfNH9ZboYZ3RjcGzNRY\ng6Ei8OGHH5KSksIVV1zBunXreOaZZ4yRqMQU26MQkauw0pTWEZFX3HZFYrmhfMam/VaubD83NV5v\nf6+v1DEYLnqysrJIS0sjNjYWEeG///0vK1eu5JZbbjHTXS8ASupRHAE2ANnARrfPD8DV5a9a8URX\nsdR++nFX/2Fl7c6+UsdguKj59ttvad26NUOGDKEgQWXz5s259dZbjZG4QCi2R6GqfwJ/isj/VDX7\nPOpUKkfSrPAdIRn7AEjJDAVzQxoM55Xk5GRGjx7N559/DkBERAQpKSlmVfUFiDdjFHVE5BMRWSci\n2wo+5a5Zcaw9Qk5yJgCNqqcB4Cc+XdZhMFxU2Gw2Jk+eTIsWLfj888+pUqUKL7/8MqtXrzZG4gLF\nm1lPM4DngUlYLqc78PGCu5MRVhrUkOxDAKyr0daX6hgMFw12u52ePXuydOlSAG644QZee+016tWr\n52PNDOWJNz2KMFX9HkBVd6rqE/hwjEJxuZj+c38QAH80/z/ATI01GMobPz8/+vTpQ926dfnyyy+Z\nM2eOMRIXAd70KHJExA/YKSIjgGTAZ0FZ1G0oIqiKtXYi3LEq20yNNRjKFlXl008/JSAggAEDBgAw\nduxYxowZQ3h4uI+1M5wvvDEUDwBVsEJ3jAeigGHlqVRJaGsrIGB4iMChVQBsNKuyDYYyZ+fOnYwc\nOZIffviBGjVqcMUVVxATE0NwcDDBwcG+Vs9wHinVUKjqb46v6cAQABGpU55KlayP9TfAHwi3AgKG\nmzhPBkOZkZOTw8SJExk/fjzZ2dnExMQwfvx4oqKifK2awUeUaChEpDNQB/hVVY+JSCusUB5XAHHn\nQb/TyLdbliI1UyHPmv20LaaZL1QxGC44Fi5cyN13382WLVsAGDJkCJMmTaJmzZo+1szgS4odzBaR\nF4D/AbcA34nIM8AvwFrA50/mS6r6w34r0122vwkNYDCcKzabjZEjR7JlyxaaN2/OggULmDlzpjES\nhhJ7FNcDCap6SkSqAvuBNqq66/yo5pl8a60dKYfSyLblEQKkB5mEJwbD2WC328nOziYsLAx/f3+m\nTp3K4sWLeeSRR8w4hMFJSdNjs1X1FICqHge2+dpIANgcrqeDR7MJwUrAvi/STM8zGM6U9evX06NH\nD+691xUnrWfPnjz55JPGSBgKUVKPopGIFIQSF6Ch2zaq2r9cNSuOTCsgoD0rDapaRbuiGpk1FAaD\nl2RmZvLss8/yyiuvkJ+fz+7duzlx4gQxMTG+Vs1QQSnJUAwosj2lPBXxlpxsK2Js6PGNruF0EbOG\nwmDwgq+++opRo0axb98+RISRI0cyfvx4oqOjfa2aoQJTUlDAn8+nIt7ib7cMRfsmVtc4JaSqL9Ux\nGCoF+fn5DBo0iNmzLadAu3btePPNN+nSpYuPNTNUBrwJ4VGhyHfk2O1RNxeAJXV6GLeTwVAKAQEB\nREVFER4ezquvvsrKlSuNkTB4TbkaChHpKyJbRWSHiIwrRuZGEdkkIhtF5KPS6rQHWobCZk8GYEP1\n1sbtZDB44LfffuO3335zbk+cOJHNmzczevRoAgK8CcpgMFh4bShE5IymQYiIP/AGVgDBlsDNItKy\niExT4FGgm6q2AkaXqnCwpXJEjKVOdE7qmahlMFzwpKamcvfdd3PZZZdxxx13kJtr9b6rVatGXJxP\n1skaKjmlGgoR6SIi64Htju0EEXndi7q7ADtUdZeq5gKfYK3NcOdO4A1VPQGgqkdKq7Qgvnmw5ACw\npkY7L1QxGC58VJWPPvqI+Ph4pk2bhr+/P9dddx02m83XqhkqOd70KCYDfwVSAFR1LfAXL46rg7VI\nr4AkR5k7zYBmIrJURFaISF8v6gXA/8RmANRktjMY2L59O3369OGWW27h8OHDdOvWjT///JMXX3yR\n0NBQX6tnqOR446j0U9W9RXLfltUrSgDQFEjEmuy6WETaqGohf5KI3AXcBVCzXmtLqYjacBzC8rLK\nSBWDoXKSl5fHFVdcQVJSElWrVmXChAnccccd+PlVurkqhgqKN3fSfhHpAqiI+IvIaMCbVKjJQF23\n7ThHmTtJwDxVzVPV3Y56mxatSFXfUtVOqtrJjjWY7bf3RwB2xDTxQhWD4cJDHaGUAwMDGT9+PLff\nfjtbtmxh+PDhxkgYyhRv7qa7gTFAPeAw0NVRVhorgaYi0lBEgoCbgHlFZOZi9SYQkepYrqiSw4Q4\nfhyH0q3u9LFQk6PXcHFx+PBhhgwZwvPPP+8sGzp0KO+99x41atTwoWaGCxVvXE/5qnrTmVasqvki\nMgr4HvAH3lXVjSLyLLBKVec59vURkU1Y7qyHVTWlxIodsZ4iOQ5AcrjPUmMYDOcVu93O9OnTGTdu\nHKmpqURHRzN69GgiIkxQTEP54o2hWCkiW4FZwGxVTfe2clX9BvimSNlTbt8Vq7cyxts68bc6QXUj\nLHuS5xfo9aEGQ2Vl7dq1jBgxghUrVgDQt29f3njjDWMkDOeFUl1PqtoYeB7oCKwXkbkicsY9jDLD\nMT82GCtpUVZgmM9UMRjKm7y8PB566CE6duzIihUriI2N5dNPP+Wbb76hUaNGvlbPcJHg1YiXqi5T\n1fuADsBJrIRGvsEx+SqQbACyAoyhMFy4BAQE8Oeff2K327n33nvZvHkzAwcORMy0cMN5pFTXk4iE\nYy2UuwloAXwJ/F8561UiAaI4fybmB2O4wNi3bx82m42GDRsiIkybNo20tDQ6derka9UMFyne9Cg2\nYM10mqCqTVT1QVX9rbSDyhNHXECD4YIiLy+PSZMm0aJFC+68807n9NemTZsaI2HwKd4MZjdSVXu5\na3IG5ORbvYg9kfVN5FjDBcHy5csZMWIE69atA6Bq1apkZWVRpUoVH2tmMJRgKETkZVV9EPhCRLTo\nfp9luAOqhtnglDXjyUSONVRmTpw4wbhx43jrrbcAaNiwIW+88QZXX321jzUzGFyU1KOY5fhbITLb\nuZObY6VDzfMLJC/PRmCg8UUZKh85OTm0a9eOffv2ERgYyMMPP8zjjz9OWJiZoGGoWJSU4e53x9cW\nqlrIWDgW0vksA97xo2kQClXyMsnIyCUmxgQ9M1Q+goODGT58OD///DNTp06lZcuWpR9kMPgAbwaz\nh3koG17WipwRdismoYpQpUqQT1UxGLwlOzubp59+mo8+cuXneuyxx1i4cKExEoYKTUljFIOwpsQ2\nFJHZbrsiAJ9mCwqsWgtyrOx2DYKM28lQ8fnxxx8ZOXIkO3bsoGbNmvTr14/Q0FCTac5QKSjpLv0d\nKwdFHFamugLSgT/LU6nSCDu+B6pAaP4pX6phMJTKoUOHGDNmDB9//DEArVq1Ytq0aSZHhKFSUdIY\nxW5gN/DT+VPHO+o6Zutm+4f4WBODwTM2m40333yTxx57jLS0NEJDQ3n66ad54IEHCAoy7lJD5aIk\n19MiVe0pIidwZSAFK4iGqmrVcteuGPzCrDSoO6JNLgpDxcRms/H666+TlpbGNddcw5QpU2jYsKGv\n1TIYzoqSXE8F6U4rXMIHv2p+cAKictN8rYrB4CQ9PR2bzUZ0dDRBQUFMnz6dw4cP079/fxObyVCp\nKXbWk9tq7LqAv6ragMuAfwI+XS7qj6Xa3oj6vlTDYACsTHOzZ8+mRYsWPPjgg87y7t27M2DAAGMk\nDJUeb6bHzsVKg9oYeA8rVelHJR9SvuTYrJlOm6u18KUaBgN79uzhuuuuY8CAASQnJ7Nhwways7N9\nrZbBUKZ4YyjsqpoH9AdeV9UHAN+mlcvLAEyIcYPvyMvL46WXXqJly5bMnz+fyMhIpkyZwrJlywgJ\nMZMsDBcWXqVCFZGBwBDgBkeZT9PKhZxKBiAtOMqXahguUrKysujatSvr168H4KabbuKVV14hNjbW\nx5oZDOWDN4ZiGDASK8z4LhFpCHxcvmqVjOZmQRCkBkf7Ug3DRUpYWBidOnUiKyuL//73v/Tp08fX\nKhkM5UqphkJVN4jIfUATEYkHdqjq+PJXrXj8wmMgF3L8g32phuEiQVWZOXMmjRs3pnv37gC8+uqr\nBAUFmYVzhosCbzLc9QA+AJKx1lDUFpEhqrq0vJUrjjj7BgCq78uFar7SwnAxsHnzZu6++24WLVpE\nixYtWLNmDUFBQURFGben4eLBG9fTq8A1qroJQERaYBkOn6XcCiQXgMn5Eb5SwXCBc+rUKcaPH8+E\nCRPIy8ujRo0aPProowQG+nR4zmDwCd4YiqACIwGgqptFpELEIOjQ2ax0NZQ93333Hffccw+7du0C\n4M477+TFF1+kalWfBSMwGHyKN4biDxGZBnzo2L4FHwcFtBHAkdAa1PSlEoYLkoyMDIYMGcKxY8do\n3bo106ZNo1u3br5Wy2DwKd4YihHAfcAjju0lwOvlppEXROsh8v1MeGZD2WCz2bDb7QQGBhIeHs5r\nr71GUlISDzzwgHE1GQyUYihEpA3QGJijqhPOj0qlI9iwiclDYTh3Vq9ezT//+U+uv/56nnzySQAG\nDx7sY60MhopFsSuzReQxrPAdtwA/ioinTHc+wQ81PQrDOXHy5Enuv/9+unTpwurVq/nggw/Iy8vz\ntVoGQ4WkpBAetwBtVXUg0Bm4+/yoVDqCjYYn92C3a+nCBoMbqspnn31GfHw8kydPRkQYM2YMf/zx\nh3EzGQzFUNJreY6qZgKo6lER8SYu1HnBDxvLYi8jYNUBunTxbdgpQ+UhPT2dQYMG8e233wJw6aWX\nMm3aNNq1a+djzQyGik1JhqKRW65sARq7585W1f7lqlkJCHbqZCRzwuTLNpwB4eHh5OTkEBUVxYsv\nvshdd92Fn1+Fef8xGCosJRmKAUW2p5SnImeCHzZ+rdOdDsHGUBhKZvHixcTGxtK0aVNEhHfffZeQ\nkBBq1arla9UMhkpDSTmzfz6fipwJfmqnXvo+goPNgLbBM8eOHeORRx7hvffeo1evXvz444+ICPXr\nm2RXBsOZUin73TlShdW1OtKoUYyvVTFUMOx2O++++y7NmzfnvffeIygoiB49emCz2XytmsFQaSlX\nQyEifUVkq4jsEJFxJcgNEBEVEa/iR0XqEXL9KkQUEUMFYuPGjSQmJjJ8+HCOHz9Or169WL9+PU8/\n/TQBAab3aTCcLV7/ekQkWFVzzkDeH3gD6A0kAStFZJ573CiHXARwP/Cb13VjJ8/fTGU0uEhLS6Nr\n165kZGRQs2ZNXnnlFQYPHmzyVRsMZUCpPQoR6SIi64Htju0EEfEmhEcXrNwVu1Q1F/gEuN6D3HPA\nS4DXiYYFJSTf5CU2WOsiAKKiohg7diwjRoxgy5Yt3HLLLcZIGAxlhDc9isnAX7FWaaOqa0XkL14c\nVwfY77adBFzqLiAiHYC6qvq1iDxcXEUichdwF0D1egkIdrIDTF7i80FeXh5JSUlkZ1csw5yfn8+J\nEycIDQ0lPDwcgAEDrIl6hw4d4tChQ75Uz2DwGSEhIcTFxZXpAlJvDIWfqu4t8nZ2ziODjgV8rwC3\nlyarqm8BbwHUqN9O/bBxoMol56qCwQuSkpKIiIigQYMGFeINXVU5cuQIycnJhIWFERwcTHx8fIXQ\nzWDwNapKSkoKSUlJNGxYdmkYvDEU+0WkC6COcYd7gW1eHJcM1HXbjnOUFRABtAYWOn7ktYF5InKd\nqq4qqWLBbtKgnieys7MrjJHIzMxk7969ZGVlARAdHU29evUqhG4GQ0VARKhWrRpHjx4t03q9MRR3\nY7mf6gGHgZ/wLu7TSqCpiDTEMhA3Ac6wnKqaBlQv2BaRhcBDpRkJAD/sVMnP9EIFQ1ng6wexzWYj\nOTmZI0eOABAUFES9evWIjo72qV4GQ0WkPH6vpRoKVT2C9ZA/I1Q1X0RGAd8D/sC7qrpRRJ4FVqnq\nvDPW1oGonZ1Rjc/2cEMlQ0Q4efIkALVr1yY2NhZ/f7Mq32A4X3gz62m6iLxV9ONN5ar6jao2U9XG\nqjreUfaUJyOhqone9CYspe3k+pt1FBcy2dnZ5OfnA+Dn50fDhg1p2bIlcXFxHo3EvHnzePHFF8+3\nmhWOhQsXEhUVRbt27YiPj+ehhx4qtH/u3Lm0bduWFi1a0KZNG+bOnVto/6RJk4iPj6ddu3Z07tyZ\nmTNnnk/1veI///lPhdSrgMWLF9OhQwcCAgL4/PPPi5VbvXo1bdq0oUmTJtx3333OGXzHjx+nd+/e\nNG3alN69e3PixAkA5s+fz1NPPXVezuE0VLXEDzDI7XMb1uyn10s7rrw+1esl6O6X22nX5GVqKH82\nbdpUuKD664U/xfH++sJyD/zsVXs2m02Tk5N11apVunv37rNXvATsdrvabLZyqdsb8vLyyq3uX375\nRa+99lpVVc3KytLmzZvrr7/+qqqqa9as0caNG+uuXbtUVXXXrl3auHFjXbt2raqqTp06Vfv06aNp\naWmqqpqWlqYzZswoU/3y8/PP6fi8vDxt06bNGV3D8rzenti9e7euXbtWhwwZop999lmxcp07d9bl\ny5er3W7Xvn376jfffKOqqg8//LC+8MILqqr6wgsv6COPPKKq1n3brl07zczMLFWH0363qorlyTmr\n526pPQpVneX2eR/oD3QsP9NVOoqQGmz80xca6enpbNq0iQMHDjhv0N27dxMfH8/tt99Os2bNuOWW\nW/jpp5/o1q0bTZs25ffffwdgxowZjBo1CoDDhw/Tr18/EhISSEhIYNmyZezZs4fmzZszdOhQWrdu\nzf79+/n4449p06YNrVu3ZuzYsR512rNnDz169KBDhw506NCBZcuWAXDTTTfx9ddfO+Vuv/12Pv/8\nc2w2Gw8//DCdO3embdu2vPnmm4D1pt+jRw+uu+46WrZsCcANN9xAx44dadWqFW+95eqkv/POOzRr\n1owuXbpw5513Os/r6NGjDBgwgM6dO9O5c2eWLl1a4vUMDQ2lXbt2JCdbc0gmTZrEY4895pwN07Bh\nQx599FEmTpwIwL///W+mTp1KZGQkAJGRkdx2222n1btjxw6uvPJKEhIS6NChAzt37mThwoX89a9/\ndcqMGjWKGTNmANCgQQPGjh1Lhw4dmDhxIl26dCl0fdu0aQNYb9g9e/akY8eOXHXVVRw8ePC0thcs\nWOB8WweYPn06nTt3JiEhgQEDBjgnOtx+++2MGDGCSy+9lEceeYTMzEyGDRtGly5daN++PV9++WWJ\n/99zoUGDBrRt27bEyMQHDx7k5MmTdO3aFRFh6NChzt7dl19+6bzut912m7NcREhMTGT+/PnnrOMZ\nc6aWBSs16s6ztUzn+qleL0GTXm6ujY9vL9WqGs6d89GjyM3N1V27dunKlSt15cqVun79eudb7e7d\nu9Xf31/XrVunNptNO3TooHfccYfa7XadO3euXn/99aqq+t577+k999yjqqo33nijvvrqq6pqvcGm\npqbq7t27VUR0+fLlqqqanJysdevW1SNHjmheXp7+5S9/0Tlz5pymW2Zmpp46dUpVVbdt26YdO3ZU\nVdXZs2fr0KFDVVU1JydH4+LiNCsrS99880197rnnVFU1OztbO3bsqLt27dJffvlFw8LCnG/zqqop\nKSmqar35t2rVSo8dO6bJyclav359TUlJ0dzcXO3evbvzvG6++WZdsmSJqqru3btX4+PjT9PXvUdx\n/Phx7dChgx48eFBVVdu3b69r1qwpJL9mzRpt3769pqWlaXR0dLH/I3e6dOmis2fPVlXVU6dOaWZm\nZqF2VVXvuecefe+991RVtX79+vrSSy859yUkJDivw4svvqjPPfec5ubm6mWXXaZHjhxRVdVPPvlE\n77jjjtPafuqpp3Ty5MnO7WPHjjm/P/744859t912m1577bXOHsyjjz6qH3zwgaqqnjhxQps2baoZ\nGRnF/n+L0r17d01ISDjt8+OPPxZ7nW677bZiexQrV67UXr16ObcXL17svH5RUVHOcrvdXmj7ww8/\n1FGjRhXbZgFl3aModTBbRE4ABank/IDjQLFxm84Xh6rU9rUKhjIgLy+PjRs3kp+fj4gQGxtL7dq1\nC72NNWzY0PnW2apVK3r16oWI0KZNG/bs2XNanQsWLHD6sP39/YmKiuLEiRPUr1+frl27ArBy5UoS\nExOpUaMGALfccguLFy/mhhtuOE2/UaNGsWbNGvz9/dm2zZoZfvXVV3P//feTk5PDd999x+WXX05o\naCg//PAD69atc/qm09LS2L59O0FBQXTp0qXQ3PbJkyczZ84cAPbv38/27ds5dOgQPXv2pGrVqgAM\nHDjQ2eZPP/3Epk2uCDgnT54kIyPDueCwgCVLlpCQkMD27dsZPXo0tWuX3W8lPT2d5ORk+vXrB1iL\nu7xh0KBBzu833ngjs2bNYty4ccyaNYtZs2axdetWNmzYQO/evQFrpltsbOxp9Rw8eJAWLVo4tzds\n2MATTzxBamoqGRkZXHXVVc59AwcOdI5n/fDDD8ybN49JkyYB1hjYvn37uOSSSzz+f4uyZMkSr86z\nrBGRQrOYatasyYEDB867HiUaCrE0TMC1/sHusEw+J8/PxHryCUdHeSc3tLX1KYXAwECio6PJzc2l\nXr16Hh88wcGuNTN+fn7ObT8/P+eAtzdUqVKlVJk5c+bwr3/9C4C3336b+fPnU6tWLdauXYvdbnfq\nFxISQmJiIt9//z2zZs3ippusiYGqyuuvv17ogQWW68m9/YULF/LTTz+xfPlywsLCSExMLHX1u91u\nZ8WKFaU+nHv06MH8+fPZvXs3Xbt25cYbb6Rdu3a0bNmS1atXk5CQ4JRdvXo1rVq1IjIykvDwcHbt\n2kWjRo1KvU5FCQgIwG63O7eLnov7uQ8aNIiBAwfSv39/RISmTZuyfv16WrVqxfLly0tsJzQ0tFDd\nt99+O3PnziUhIYEZM76VhrIAACAASURBVGawcOFCj22qKl988QXNmzcvVN8zzzzj8f9blB49epCe\nnn5a+aRJk7jyyitL1NkTderUISkpybmdlJREnTpWts5atWpx8OBBYmNjOXjwIDVr1nTKZWdnExoa\nesbtnSsljlE4jMI3qmpzfCqEkRAg389EA62M2Gw2kpKSCv3o6tWrR9OmTb1+Oy2NXr16MXXqVGd7\naWlpp8l06dKFRYsWcezYMWw2Gx9//DE9e/akX79+rFmzhjVr1tCpUyfS0tKIjY3Fz8+PDz74oFC4\n8kGDBvHee++xZMkS+vbtC8BVV13F1KlTycvLA2Dbtm1kZp6+5ictLY2YmBjCwsLYsmULK1asAKBz\n584sWrSIEydOkJ+fzxdffOE8pk+fPrz++v+3d97hUVRtH75PAAlICRBBWgIhgdTdUIKUDxGQgNJE\nEEFQg3QQBGnxBRTBV4qKjSaIAoKioJTXAgoGlSZFkF6U0BM6gQAJJHm+P3YzJNmUJSTZbDj3dc11\n7cycOefZs7PzzGm/547M2q5duzKth+rVqxMeHs6UKVMAGDFiBJMmTTJaYceOHePtt99m+PDhALz2\n2msMGjTImIocGxtrM7uoZMmSVKlSxeg3j4+P58aNG3h6erJ//37i4+O5cuUK69ZlHM6mRo0aFCpU\niIkTJxotjVq1anH+/HnDUSS3NNPi5+fHP//8Y+xfu3aNihUrcvv2bRYvXpxhma1ateLjjz82Zhbt\n3LkTINPfNyV//PGHcV+k3LLjJAAqVqxIqVKl2LJlCyLCwoUL6dDBIoXXvn17FixYAMCCBQuM42C5\nnwIDs34By2nskRnfpZSqneuW3CVJ+SeEt8ZOrly5wr59+4iOjubEiRPGn9bFxSVHFwl9+OGHRERE\nEBQURN26dVN11yRTsWJFJk+eTLNmzTCbzdStWzfVHzKZgQMHsmDBAsxmMwcPHkz1lhoaGspvv/3G\n448/zgMPWKZr9+7dG39/f+rUqUNgYCD9+vVLt9XTunVrEhIS8PPzIzw83OgSq1y5Mv/5z3+oX78+\njRs3plq1apQuXRqwdFVt374dk8mEv78/s2fPzrIu+vfvz++//86xY8cIDg5mypQptGvXDl9fX9q1\na8fUqVONmOEDBgygWbNmhISEEBgYSJMmTdIdkP3iiy/46KOPMJlMNGrUiOjoaKpWrUqXLl0IDAyk\nS5cu1K6d+SPj2WefZdGiRXTp0gWwLKJctmwZo0ePxmw2ExwcnO7A8hNPPMHvv/9u7E+cOJFHHnmE\nxo0b4+vrm2F548aN4/bt25hMJgICAhg3bhyQ+e+bXbZt20aVKlVYunQp/fr1IyAgwDiXMj77zJkz\n6d27N97e3tSoUYMnnngCgPDwcH755Rd8fHxYu3Yt4eF3evojIiJo06bNPdt4t6iMGglKqcJiWTS3\nD6gF/Atcx/JCLyJSJ+/MvMNDnsGybVgi1YfuIV80bwo4Bw4cSNUnnB1u3brFiRMnuHLlCgDFixfH\n09MzR/6UBZHkcYeEhAQ6duzISy+9ZIwJaKBjx45MnToVHx8fR5uSp5w9e5bnnnsu09ZaMun9b5VS\nO0TErpg/acms/2YrUAdon52Mc5NEF70q1xkQEc6ePcuZM2dISkrCxcWFypUrU758eYfLguRnxo8f\nz9q1a4mLiyM0NNRmgP1+Z/LkyURFRd13juLEiRO89957Dik7M0ehAETk3zyyxW4SdbeTU5CYmEh0\ndDRJSUmUKVOGqlWrGl00moxJnpmjSZ9atWrZDErfD4SEhDis7MwcxUNKqVczOiki03LBHrtIUi5U\n330WTBUcZYImAxISEnBxccHFxYXChQvj6emJUkoL+Gk0TkxmjqIQUAJryyI/kaRc2H47KeuEmjxD\nRLh06RInT56kfPnyVKpkiRdSpkwZB1um0WjulcwcRZSITMgzS+6CJOVCWTcd4S6/EBcXx/Hjx40p\nr7GxsYiIHofQaAoIWY5R5EcSVSGoXNLRZtz3JCUlER0dTVRUlGWZf+HCVKlShXLlymknodEUIDIb\nFW6RZ1bcJUlKgatecOdIkhdEJQv4lStXjoCAANzd3bWTyEcUKlSI4OBgAgMDadeunTFFGWDfvn00\nb96cWrVq4ePjw8SJE0k5Xf6nn36iXr16+Pv7U7t2bWNhXn5i586d9OrVy9FmZMqkSZPw9vamVq1a\nrFmzJt00yWKHgYGBvPjii8bam5iYGNq1a4fZbCYgIIDPP/8csAhEJi/yzBOyKxLlqM3dwyxR73ln\nroilyTHSiovB+FTbnj175OrVqzbXffLJ9lTp+vRZlVcm3zX3Kn2dn8t/8MEHjc8vvPCCvPXWWyJi\nESL08vKSNWvWiIhF/LB169Yyffp0ERHZs2ePeHl5yYEDBwwbZ86cmaO25YT8d+fOnW2EDnO7zLth\n3759YjKZJC4uTo4ePSpeXl42v3diYqJUqVJFDh06JCIi48aNk08//VRERP773/8aMuPnzp2TMmXK\nSHx8vIiIhIWFGRLyaclzmfH8yF/l891C8QKPiKQbh9ff35+SJXOvG/DYsWN2yYxv3bqVhg0bUrt2\nbRo1asShQ4cAyxTdESNGEBgYiMlkMiQwUkpfL126lF27dtGgQQNMJhMdO3Y0gsWkJT1p8NmzZzNy\n5EgjTUrJ80WLFlG/fn2Cg4Pp16+fIRFRokQJhg8fjtlsZvPmzUyYMMFYEd23b1/jzX7btm2YTCaC\ng4MZOXKkId+QkZx5ZjRs2NCQHP/yyy9p3LgxoaGhgGUR5PTp043gT1OnTmXMmDHGaudChQoxYIBt\nBOTY2Fh69uxJUFAQJpPJkBxJKVS4bNkywsLCAFv572rVqqVq5fj4+HD27Fm7JNWvXbvG7t27De2q\njO6B+fPn0759e5o3b06LFpaOknfeeceouzfeeMPIMyPp9+yycuVKunbtStGiRalevTre3t7GPZvM\nxYsXeeCBB6hZsyYALVu2NOpRKcW1a9cQEWJjYylbtqwhsf7UU09lKluSo2TXwzhqc/cwy/dfPCPH\nj19J15Nqcpb9+/fL9evXZf/+/bJt2zabFkVG5FSLwl6Z8ZiYGONt8ZdffpGnn35aRERmzpwpnTp1\nMs4lS3unlb4OCgqS9evXi4jlje6VV15J1570pMHPnTsnNWrUMNK0bt1a/vjjD9m/f7+0bdtWbt26\nJSIiAwYMkAULFoiICCBff/21Tb4iIj169JBVqyz1FRAQIJs2WYJ0jR49WgICAkREMpQzT0tyiyIh\nIUE6d+4sP/30k4iIDBs2TD744AOb9G5ubhITE5OuJHl6jBo1KlVdXbp0KVW5IiJLly6VF198UURs\n5b+HDBkin332mYiIbNmyxZDetkdS/ddffzV+Z5GM74HPP/9cKleubNTxmjVrpE+fPkYAqzZt2shv\nv/0mIun/vmkZOnRoupLjycGGUjJo0CBD3lxE5KWXXrKRHk9KShIPDw/Ztm2bUSeBgYEiInL16lV5\n7LHH5OGHH5YHH3xQvv/+e+O6U6dOGenSkucy4/kR95sXWLnyIIMHP+JoUwo0sbGxXL582RC1K1LE\nMYq99siMx8TE8OKLL3LkyBGUUoYo39q1a+nfv7/xFpYs3w13pK9jYmK4cuUKTZs2BSzBYp555pl0\nbUlPGrxBgwZ4eXmxZcsWfHx8OHjwII0bN2bGjBns2LHDWCh18+ZNQwm0UKFCdOrUycg3IiKCqVOn\ncuPGDS5dukRAQIChWNqwYUMAnnvuOSNoTUZy5illzJPLTA5e5OfnZ8h45xRr165lyZIlxr4906FT\nyn8/++yzTJgwgZ49e7JkyRLjN7FHUj0qKsqQiYeM7wGwvKUn//Y///wzP//8s6FHFRsby5EjR3j0\n0UfT/X3LlSuXyv7333/fvsqxE6UUS5YsYdiwYcTHxxMaGmrUz5o1awgODubXX3/l33//pWXLljRp\n0oRSpUrlqeS4UzqKfe6BFD7paCsKNitWrGDw4MHMnTsXd3d3ypcvT+XKlRExZ30x0LdvXfr2zZlA\niPbIjI8bN45mzZqxfPlyjh07xmOPPZZlvllpTZ08eZJ27doBFnE9X1/fDKXBu3btyjfffIOvry8d\nO3ZEKYWI8OKLLzJp0iSbvF1dXY2HQVxcHAMHDmT79u1UrVqV8ePHZyk5LpK+nHlaihUrxq5du7hx\n4watWrVixowZDBkyBH9//1TiegBHjx6lRIkSlCpVioCAABtJ8rsh5YSGzCTHGzZsyD///MP58+dZ\nsWIFY8eOBeyTVE8rOZ7ZPZBWcvy1116jX79+qfKzV/p92LBhRERE2Bzv2rVrKgE/sIg8njx552GV\nUk48JQ0bNjRiXvz8889GXIzPP/+c8PBwlFJ4e3tTvXp1Dh48SP369fNUctwpxygSXApT+PrtrBNq\nssXp06fp2rUrp06d4oEHHsDPzw8PDw/jwZYfiYmJMf6AySE4wfIm+cknnxgO5dKlSzbXli5dmjJl\nyhh/1C+++IKmTZtStWpVQ066f//+GUqDg0WobuXKlXz11VdGbIoWLVqwbNkyzp07Z5R9/Phxm/KT\nH0bu7u7ExsYarQQ3NzdKlizJn3/+CZDqzd1eOfNkihcvzkcffcR7771HQkIC3bt3Z8OGDaxduxaw\ntDyGDBnCqFGjABg5ciRvv/228cBKSkpKV622ZcuWzJgxw9hPHtupUKECBw4cICkpyXhDTw+lFB07\nduTVV1/Fz8/PeHu3R1I9reR4RvdAWlq1asVnn31GbGwsYLnfz507l+nvm5L3338/XcnxtE4CLJLh\nS5YsIT4+nsjISI4cOZIqFGwyyfdIfHw8U6ZMoX///oBFgj9ZBPDs2bMcOnTIiBeSl5LjTukobrsU\npmpFvY4iJ7l9+7YxgFq5cmX++9//8tFHH/Hwww87hcrrqFGjeO2116hdu3YqWe/evXvj4eGByWTC\nbDbz5Zdfpnv9ggULGDlyJCaTiV27dvH666/bpMlIGhwsXS5+fn4cP37ceBD4+/vz1ltvERoaislk\nomXLlunGgXZzc6NPnz4EBgbSqlWrVJo+8+bNo0+fPgQHB3P9+nVDctxeOfOU1K5dG5PJxFdffUWx\nYsVYuXIlb731FrVq1SIoKIiQkBBjEN5kMvHBBx/QrVs3/Pz8CAwM5OjRozZ5jh07lsuXLxMYGIjZ\nbDbetCdPnkzbtm1p1KhRupHqUpIsOZ4yCp49kuq+vr7ExMQYCz0zugfSEhoaynPPPUfDhg0JCgqi\nc+fOXLt2LdPfN7sEBATQpUsX/P39ad26NTNmzDBeuJ588kmj6+idd97Bz88Pk8lEu3btaN68OWBp\nJW3atImgoCBatGjBlClTcHd3B/JYcjy7gxuO2tw9zPLh/8JFjurB7Jxi48aNEhQUJAsXLrQ5l96g\nmCbvuHbtmvF50qRJMmTIEAdak/+YNm2azJ0719FmOIQmTZoYkwfSoqfHAlViT0HR/NsN4ixcunSJ\nfv360bhxY/bs2cPMmTONVoUmf/DDDz8YC+b++OMPow9fY2HAgAGpxrDuF86fP8+rr76aZ1pqGQYu\nyq885Bksg+d14/XHRzvaFKdFRFi0aBHDhw/n/PnzFClShFGjRjFmzBibwbGcCFyk0WjylrwMXJRv\n0WFQs8/Zs2fp1q2b0ZfctGlTZs2apZ2BRqPJEKd84iZpLaFs4+bmRlRUFO7u7syfP5+IiAjtJDQa\nTaY4ZYvCSYdWHMYvv/xCnTp1KFeuHEWLFmXp0qVUrFjRZiGRRqPRpIdTPnGTlGL79rxZkejMREVF\n0a1bN0JDQxk9+s6YTmBgoHYSGo3GbpzUUbhw7lzGi4vudxITE5k5cya+vr4sWbKEYsWKUatWLaed\n0VTQpbIzolu3bphMJrslI1LKW+QkIsKQIUPw9vbGZDLx119/pZvu5s2bNG3a1BA+zI+sXr2aWrVq\n4e3tbQggpuX48eO0aNECk8nEY489xqlTpwDLuoXg4GBjc3V1ZcWKFYBlVfaRI0fy7HvkOdmdV+uo\nzd3DLKPXfyg/fb03i1nG9yc7duyQkJAQAQSQNm3aSGRkZLbzyw/rKAq6VHZ6REVFpRIatIeU9ZST\n/PDDD9K6dWtJSkqSzZs3S/369dNNN3369HSFBjMiWZQvr0hISBAvLy/5999/JT4+Xkwmk+zbt88m\nXefOnWX+/PkiIrJu3Trp0aOHTZqLFy9KmTJl5Pr16yIisn79eundu3fufoG7IKfXUTj8wX+3m7uH\nWUb+9rH8vGh3tiuxoJKstApI5cqV5dtvv5WkpKR7yjPlDZdbP2pWpHwAzpo1SwYMGCAiIp9++qk8\n//zzqdL+888/UqVKFRERef7552XevHlZ5n/t2jUJCwuTwMBACQoKkmXLltmUm1YBtV+/flK/fn0Z\nNmyYeHp6yuXLl4203t7eEh0dLefOnZOnn35a6tWrJ/Xq1Us3dsDNmzeNsoODg+XXX38VEYuaraur\nq5jNZvn9999TXRMdHS1PPfWUmEwmMZlMsnHjxlT2Xrt2TZo3by61a9eWwMBAWbFihYiIxMbGypNP\nPikmk0kCAgJkyZIlImJRpfXz85OgoCAZPny4jY19+/aVL7/80tivWbOmnDlzxiZdw4YNjZeSjGyI\njIyUmjVryvPPPy/+/v5y7NgxWbNmjTRo0EBq164tnTt3NhYZvvnmm1KvXj0JCAgw1F7vhU2bNklo\naKix//bbb8vbb79tk87f319OnDghIhZnVrJkSZs0n3zyiTz33HPGfmJiolSrVi3P411khFM5CqA1\ncAj4BwhP5/yrwH5gN7AO8MwqT3cPs7z741g5uyPqniuzINK7d28ZNmxYusGEskN+chQFUSr73Xff\nlZ49e4qIyIEDB6Rq1apy8+ZNiYyMNCTF09KlSxd5//33jTq5cuVKKntv374tMTExIiJy/vx5qVGj\nhiQlJcmyZctSvfVeuXJFLly4IDVr1jQewikdXjJt2rQxvoeISPPmzQ1J7GTi4+OlQoUKxn5GNkRG\nRopSSjZv3myca9KkicTGxoqIyOTJk+XNN98UkYyl11OyaNGidCW/O3XqZJN26dKl0qtXL2N/4cKF\nMmjQIJt03bp1M+6rb7/9VgAbufFmzZrJ//73v1THHn/8cdm+fbtNfo7AaWTGlVKFgBlAS+AUsE0p\ntUpE9qdIthOoJyI3lFIDgKnAs7a5peZo6eqUdy+eG2Y7FceOHWPw4MGMGDHCkMieM2dOroUiddQI\nR0GWyt6wYQODBw8GLNpFnp6eHD58mFKlSmVY9q+//srChQsBy/hNsv5TMiLCf/7zH37//XdcXFw4\nffo0Z8+eJSgoiOHDhzN69Gjatm1LkyZNSEhIwNXVlV69etG2bVvatm2b5XdPjwsXLuDm5palDQCe\nnp6GjtKWLVvYv38/jRs3BuDWrVuGrHp60uvJar7JdO/ene7du2fL5ox49913efnll5k/fz6PPvoo\nlStXTiWIGRUVxZ49e2yUe5Nlv+vWzRnV5PxEbk6PrQ/8IyJHAZRSS4AOWFoQAIhISq3eLUAPezIW\n5QJlM5YfLujcvn2badOm8eabb3Lz5k0uXLjA5s2bAQpkvOqCLJWdGyxevJjz58+zY8cOihQpQrVq\n1YiLi6NmzZr89ddf/Pjjj4wdO5YWLVrw+uuvs3XrVtatW8eyZcuYPn06v/76a6r87JHKTiv5nZEN\nYCv53bJlS7766qtU+dkrvb548WLeeecdm+Pe3t6GCu/dfA+ASpUq8d133wGWWBXffvttKif4zTff\n0LFjR5v4LHkp+53X5Oasp8pAyqgRp6zHMqIX8FN6J5RSfZVS25VS28G64K7EAzlmqDOxYcMGateu\nTXh4ODdv3qRr167GTV3QKYhS2U2aNDHCWR4+fJgTJ05Qq1atTOuhRYsWzJo1C7DMcIuJiUl1PiYm\nhvLly1OkSBEiIiIMafMzZ85QvHhxevTowciRI/nrr7+IjY0lJiaGJ598kvfff5+///7bprz27duz\ncOFCRIQtW7ZQunRpG0XYMmXKkJiYaDzMM7IhLQ0aNGDjxo2GXPj169c5fPhwhtLraenevXu6kt/p\npQ8JCeHIkSNERkZy69YtlixZQvv27W3SXbhwgaSkJAAmTZrESy+9lOr8V199Rbdu3Wyuy0vZ7zwn\nu31WWW1AZ+DTFPvPA9MzSNsDS4uiaFb5unuYpc8WW5XTgs6lS5ekV69exmymGjVqGLN9cpP8NutJ\nRKRt27aG0u3u3buladOmUrNmTalRo4aMHz8+1aDn//73P6lTp474+vqKn5+fjBw50ib/a9euyQsv\nvCABAQFiMpnk22+/FRFLn7aXl5c88sgjMmjQoFRjFGnDWVrCxGLMlhGx9L936dJFgoKCxM/PT/r1\n62dTdkaD2ZmNUURHR0v79u0lMDBQzGazESo1uZ7Onz8vDRo0kMDAQAkLCxNfX1+JjIyU1atXS1BQ\nkJjNZqlXr55s27ZNzpw5IyEhIRIUFCSBgYGp7E8mKSlJBg4cKF5eXhIYGGgzPpHMSy+9JL/88kum\nNqT3vdatWyf16tWToKAgCQoKkpUrV4qIyJgxY8TLy0saNWokYWFh8sYbb6Rb7t3www8/iI+Pj3h5\neRmz50Qs4W+Ty126dKl4e3uLj4+P9OrVS+Li4ox0kZGRUqlSJZvZWtHR0RISEnLP9uUUTjOYDTQE\n1qTYfw14LZ10jwMHgPL25OvuYZZ3/lx0r/XodFy4cEHc3d2lSJEiMm7cOLlx40aelJsfHIXGOdix\nY0e6U0nvB6ZNmyaffvqpo80wcJrBbGAb4KOUqg6cBroCz6VMoJSqDXwCtBaRc/Zm3On6LW7fTqRI\nkYItNX7w4EGqV69O0aJFKVeuHIsXL8bDwwNfX19Hm6bR2FCnTh2aNWtGYmJivo6GmBu4ubnx/PPP\nO9qMXCPXxihEJAF4GViDpcXwjYjsU0pNUEoldwy+A5QAliqldimlVtmT94QJG9i1KzpX7M4P3Lhx\ngzFjxmAymZg6dapxPDQ0VDsJTb7mpZdeuu+cBEDPnj0pXNgppfPsIle/mYj8CPyY5tjrKT4/np18\nE5MULgVvcg9gkRgYOHAgkZGRgGVgTaPRaByJU7pAEVAXbjrajBzlzJkzDB06lKVLlwIQFBTE7Nmz\nadSokYMt02g09ztO6SgeFBeKFHZKPcN0OXz4MPXq1ePatWsUL16c8ePHM3ToUJt52hqNRuMInNJR\n/LdUUcr5l3e0GTmGj48PISEhPPjgg3z88cd4eno62iSNRqMxcMrXciVOabbB1atXGTp0qLEQTCnF\nqlWrWLVqlXYS6aBlxh0rM37w4EEaNmxI0aJFeffddzNMJyI0b96cq1ev5oodOcGOHTsICgrC29ub\nIUOGpLpXkrl8+TIdO3bEZDJRv3599u7dC8ChQ4dSyYyXKlWKDz74AIARI0bYrGgvUGR3Xq2jNncP\ns1zc89PdTivOFyQlJck333wjFStWFEBatWrlaJOyJD+so9Ay4/aRWzLjZ8+ela1bt8p//vMfeeed\ndzJM9/3338vQoUPvKu9kYcW8IiQkRDZv3ixJSUnSunVr+fHHH23SjBgxQsaPHy8iFqHG5s2b26RJ\nSEiQChUqyLFjx0RE5NixY9KyZcvcNf4uyOl1FE76au58U56OHj1KmzZt6NKlC1FRUTRo0IApU6Y4\n2qy74z2VO9td0LBhQ06fPg3Al19+SePGjQkNDQUsEh/Tp083AtJMnTqVMWPGGFOKCxUqxIABA2zy\njI2NpWfPngQFBWEymfj222+B1G/oy5YtIywsDICwsDD69+/PI488wqhRo6hWrVqqVo6Pjw9nz57l\n/PnzdOrUiZCQEEJCQti4caNN2XFxcUbZtWvXJiLCIn8WGhrK6dOnCQ4O5o8//kh1zdmzZ+nYsSNm\nsxmz2cymTZtsvk+LFi2oU6cOQUFBrFy5ErDIY7Rp0waz2UxgYCBff/01AOHh4fj7+2MymRgxYoSN\njeXLlyckJCTLMbPFixfToUMHY/+pp56ibt26BAQEMGfOHON4iRIlGD58OGazmc2bN7Njxw6aNm1K\n3bp1adWqFVFRUQDMnTuXkJAQzGYznTp14saNG5mWnxVRUVFcvXqVBg0aoJTihRdeMAIPpWT//v00\nb94csAg1Hjt2zBA0TGbdunXUqFHD6AHw9PTk4sWLREcXzGn7TjlGgYvz+Ldbt27x7rvvMnHiROLi\n4nBzc2Py5Mn06dMHFyf6HvmBxMRE1q1bR69evQBLt1Napc4aNWoQGxvL1atX2bt3r11dTRMnTqR0\n6dLs2bMHuKP1lBmnTp1i06ZNFCpUiMTERJYvX07Pnj35888/8fT0pEKFCjz33HMMGzaM//u//+PE\niRO0atWKAwcOpMpnxowZKKXYs2cPBw8eJDQ0lMOHD7Nq1Sratm2brj7UkCFDaNq0KcuXLycxMZHY\n2NhU511dXVm+fDmlSpXiwoULNGjQgPbt27N69WoqVarEDz/8AFj0mC5evMjy5cs5ePAgSqlUDu9u\n2bhxI5988omx/9lnn1G2bFlu3rxJSEgInTp1oly5cly/fp1HHnmE9957j9u3b9O0aVNWrlzJQw89\nxNdff82YMWP47LPPePrpp+nTpw8AY8eOZd68eYbSbjIREREMGzbMxpbixYvbONDTp09TpUoVY79K\nlSrGS0dKzGYz3333HU2aNGHr1q0cP36cU6dOUaFCBSPNkiVLbPSe6tSpw8aNG+nUqdNd1Jpz4JyO\nwokaQidPnmTChAnEx8fTvXt33nvvvVQ3nFMx3DFC41pmPDX5UWYc4NKlS5QsWdLY/+ijjwwxxZMn\nT3LkyBHKlStHoUKFjIfpoUOH2Lt3r/GbJiYmGoKDe/fuZezYsVy5coXY2FgbWW+AZs2apetM74Xw\n8HBeeeUVgoODjZZeykWEt27dYtWqVUyaNCnVdcky4wURp3QUy5cfoEP5hrjn05gUly9fxs3NDaUU\nNWrU4MMPP8Tb0r47/QAAEU9JREFU25sWLVo42jSnRMuM3x05LTNuL4ULFyYpKQkXFxfWr1/P2rVr\n2bx5M8WLF+exxx4z6tDV1dV48IoIAQEBhkx+SsLCwlixYgVms5n58+ezfv16mzR306KoXLmyEf8a\nMpYZL1WqFJ9//rlhX/Xq1fHy8jLO//TTT9SpU8fmhU/LjOczln17kIsX762/MjdISkris88+w9vb\nm0WLFhnH+/Xrp51EDqBlxi3ktcy4vdSqVYujR48aNpQpU4bixYtz8OBBtmzZkuE158+fNxzF7du3\n2bdvHwDXrl2jYsWK3L5926ijtCS3KNJuaZ0EQMWKFSlVqhRbtmxBRFi4cGGqMZVkrly5wq1btwD4\n9NNPefTRR1O18LTMuBNs7h5mCQ1+WQ7vis7WbIDcYu/evdKkSRNDBrxbt26ONilHyG+znkS0zHhe\ny4xHRUVJ5cqVpWTJklK6dGmpXLmyEeY0JRMmTJC5c+eKiEhcXJy0bt1afH19pUOHDtK0aVOJiIhI\nZWcyO3fulCZNmojJZBJ/f3+ZM2eOiIjMnDlTqlWrJiEhIfLyyy8b9X8vbNu2TQICAsTLy0sGDRpk\n3CuzZs2SWbNmiYgltraPj4/UrFlTOnbsaITGFbHEHS9btqwRfjaZW7duia+vr46ZnV82dw+zPG4e\nLEc2nch2JeYk169fl/DwcClcuLAAUr58eVm8ePE9B4LPL+QHR6FxDs6cOSOPP/64o81wCN99952M\nHTvW0WYYOJPMeK7RqcgDlHVzfCjUw4cP06pVK44dO4ZSiv79+/P222/bNRiq0RQ0KlasSJ8+fbh6\n9Wqmg/EFkYSEBKdazHm3OKWj6Fq0KG5lHD+Q7enpiaurK2azmdmzZxsB4zWa+5UuXbo42gSH8Mwz\nzzjahFzFKQezaVYNiue9j0tISGD69OlcvHgRgKJFi7J69Wq2b9+unYRGoymwOKejCDNBqaJ5WuTW\nrVupX78+gwcPZvTo0cZxT0/PAh2wRKPRaJzTUeQhMTExvPzyyzRo0ICdO3fi4eGR7pQ6jUajKag4\np6Nwyf1QiyLCkiVL8PX1ZcaMGRQqVIhRo0axf/9+2rVrl+vlazQaTX7BKR2FKpI7csop+fvvv+nW\nrRvR0dE0atSIv/76iylTpqRakavJG7TMuGNlxhcvXozJZCIoKIhGjRpluChPpGDLjAO8//77BAQE\nEBgYSLdu3YzV5l27duXIkSN59j3ynOzOq3XU5u5hlmYhb0pU1LVszS/OjLSSx8OGDZO5c+dKYmJi\njpflLOSHdRRaZtw+cktmfOPGjcaisx9//FHq16+fbrqCLjN+6tQpqVatmty4cUNERJ555hn5/PPP\nRURk/fr10rt377z5Enag11EA5y7cJCkpZwXqIiIiGDhwIJ988gmPPvooANOmTcvRMpydPjMv5Uq+\ncweWtTttw4YN2b17N5CxzPhjjz3GoEGD7kpmfPDgwWzfvh2lFG+88QadOnWiRIkShjLrsmXL+P77\n75k/fz5hYWG4urqyc+dOGjduzHfffceuXbtwc3MDLDLjGzZswMXFhf79+3PixAkAPvjgAxo3bpyq\n7Li4OAYMGMD27dspXLgw06ZNo1mzZqlkxj/++GOaNGliXHP27Fn69+9vyGXMmjUrVWz12NhYOnTo\nwOXLl7l9+zZvvfUWHTp04Pr163Tp0oVTp06RmJjIuHHjePbZZwkPD2fVqlUULlyY0NBQm+BEKfNu\n0KBBKr2klCxevJi+ffsa+0899RQnT54kLi6OV155xThXokQJ+vXrx9q1a5kxYwbFihXj1VdfJTY2\nFnd3d+bPn0/FihWZO3cuc+bM4datW3h7e/PFF19QvHj2p8WnlBkHDJnxJ554IlW6/fv3Ex4eDtjK\njCckJHDz5k2KFCnCjRs3qFSpEmCRYgkLCyMhIaFATm5xym8kSUKhq/FQqWTWibPg3LlzjBw50lDj\nnDZtmuEoNPkLLTNuwZEy4/PmzbN5sCZT0GXG69aty4gRI/Dw8KBYsWKEhoYaLykuLi54e3vz999/\n29yTBQHndBQoXO6xQZGUlMS8efMYPXo0ly9fpmjRoowdO5aRI0fmjJEFkLt5889JtMx4ahwlMx4R\nEcG8efPYsGFDuucLusz45cuXWblyJZGRkbi5ufHMM8+waNEievToAdyRGdeOIp+wokQpypbJvoRH\nZGQkPXr0MN44QkNDmTFjBt7e3jlloiYH0TLjd0duyIzv3r2b3r1789NPPxnquGkp6DLja9asoXr1\n6jz00EMAPP3002zatMlwFFpmPJ/xsHKhUKHsm16qVCkOHz7Mww8/zJIlS1i9erV2Ek6Alhm3kNcy\n4ydOnODpp5/miy++oGbNmhnaVdBlxj08PNiyZQs3btxARFi3bh1+fn7GdVpmPB9t7h5mifljh8it\nu5stsXr1aomLizP2N23aZCMVrLElv816EtEy43ktM96rVy9xc3MTs9ksZrNZ6tatm65d94PM+Ouv\nvy61atWSgIAA6dGjh/FMiY6OlpCQkHu2L6fQMuMeZok5e9zuCjtx4oQ89dRTAsjEiRPtvk5jIT84\nCo1zcD/LjE+bNk0+/fRTR5thkNOOwim7nlTWSUhISGDatGn4+fmxYsUKSpQoQdmyjhmM1WjuB1LK\njN9vuLm58eKLLzrajFzDKQezs3IVW7ZsoX///kZ/a6dOnfjwww/THbjSaDQ5x/0qM96zZ09Hm5Cr\nOKejUBk3hP78808aNWqEiFCtWjWmT59OmzZt8tC4goeIpJoBpNFo8i+WXqacxUkdRcYPrfr169Oq\nVStq167N2LFj72klp8YylfHixYuUK1dOOwuNJp8jIly8eDHHp2Q7p6PYfxEaVoAihThy5AjDhg1j\n2rRp1KxZE6UUP/zwAy4uTjn8ku+oUqUKp06d4vz58442RaPR2IGrq2uqFeg5gXM6ir6/EB9Rlclz\nPmTSpEnEx8fj6urKsmXLALSTyEGKFClC9erVHW2GRqNxILn6RFVKtVZKHVJK/aOUCk/nfFGl1NfW\n838qparZk+9v1w9jalKf8ePHEx8fT8+ePdNdSKXRaDSae0flxsAHgFKqEHAYaAmcArYB3URkf4o0\nAwGTiPRXSnUFOorIs5nl6/pgWYm/YVk56+fnx+zZs7WIn0aj0WSBUmqHiNTLzrW52aKoD/wjIkdF\n5BawBEi7Xr4DsMD6eRnQQmUxYhp/4wquLkV4+/UJ7Nq1SzsJjUajyWVys0XRGWgtIr2t+88Dj4jI\nyynS7LWmOWXd/9ea5kKavPoCyUL3gcBeNADuwIUsU90f6Lq4g66LO+i6uEMtEclWbAanGMwWkTnA\nHACl1PbsNp8KGrou7qDr4g66Lu6g6+IOSqnt2b02N7ueTgNVU+xXsR5LN41SqjBQGriYizZpNBqN\n5i7JTUexDfBRSlVXSj0AdAVWpUmzCkgWSOkM/Cq51Rem0Wg0mmyRa11PIpKglHoZWAMUAj4TkX1K\nqQlYVAxXAfOAL5RS/wCXsDiTrJiTWzY7Ibou7qDr4g66Lu6g6+IO2a6LXBvM1mg0Gk3BQC9h1mg0\nGk2maEeh0Wg0mkzJt44it+Q/nBE76uJVpdR+pdRupdQ6pZSnI+zMC7KqixTpOimlRClVYKdG2lMX\nSqku1ntjn1Lqy7y2Ma+w4z/ioZSKUErttP5PnnSEnbmNUuozpdQ56xq19M4rpdRH1nrarZSqY1fG\n2Q2Nl5sblsHvfwEv4AHgb8A/TZqBwGzr567A146224F10Qwobv084H6uC2u6ksDvwBagnqPtduB9\n4QPsBMpY98s72m4H1sUcYID1sz9wzNF251JdPArUAfZmcP5J4Ccs0d8aAH/ak29+bVHkivyHk5Jl\nXYhIhIjcsO5uwbJmpSBiz30BMBGYAsTlpXF5jD110QeYISKXAUTkXB7bmFfYUxcClLJ+Lg2cyUP7\n8gwR+R3LDNKM6AAsFAtbADelVMWs8s2vjqIycDLF/inrsXTTiEgCEAOUyxPr8hZ76iIlvbC8MRRE\nsqwLa1O6qoj8kJeGOQB77ouaQE2l1Eal1BalVOs8sy5vsacuxgM9lFKngB+BwXljWr7jbp8ngJNI\neGjsQynVA6gHNHW0LY5AKeUCTAPCHGxKfqEwlu6nx7C0Mn9XSgWJyBWHWuUYugHzReQ9pVRDLOu3\nAkUkydGGOQP5tUWh5T/uYE9doJR6HBgDtBeR+DyyLa/Jqi5KYhGNXK+UOoalD3ZVAR3Qtue+OAWs\nEpHbIhKJRfbfJ4/sy0vsqYtewDcAIrIZcMUiGHi/YdfzJC351VFo+Y87ZFkXSqnawCdYnERB7YeG\nLOpCRGJExF1EqolINSzjNe1FJNtiaPkYe/4jK7C0JlBKuWPpijqal0bmEfbUxQmgBYBSyg+Lo7gf\n4/uuAl6wzn5qAMSISFRWF+XLrifJPfkPp8POungHKAEstY7nnxCR9g4zOpewsy7uC+ysizVAqFJq\nP5AIjBSRAtfqtrMuhgNzlVLDsAxshxXEF0ul1FdYXg7creMxbwBFAERkNpbxmSeBf4AbQE+78i2A\ndaXRaDSaHCS/dj1pNBqNJp+gHYVGo9FoMkU7Co1Go9FkinYUGo1Go8kU7Sg0Go1GkynaUWjyHUqp\nRKXUrhRbtUzSVstIKfMuy1xvVR/92yp5USsbefRXSr1g/RymlKqU4tynSin/HLZzm1Iq2I5rhiql\nit9r2Zr7F+0oNPmRmyISnGI7lkfldhcRMxaxyXfu9mIRmS0iC627YUClFOd6i8j+HLHyjp0zsc/O\noYB2FJpsox2Fximwthz+UEr9Zd0apZMmQCm11doK2a2U8rEe75Hi+CdKqUJZFPc74G29toU1hsEe\nq9Z/UevxyepODJB3rcfGK6VGKKU6Y9HcWmwts5i1JVDP2uowHu7Wlsf0bNq5mRSCbkqpWUqp7coS\ne+JN67EhWBxWhFIqwnosVCm12VqPS5VSJbIoR3Ofox2FJj9SLEW303LrsXNASxGpAzwLfJTOdf2B\nD0UkGMuD+pRVruFZoLH1eCLQPYvy2wF7lFKuwHzgWREJwqJkMEApVQ7oCASIiAl4K+XFIrIM2I7l\nzT9YRG6mOP2t9dpkngWWZNPO1lhkOpIZIyL1ABPQVCllEpGPsEhqNxORZlYpj7HA49a63A68mkU5\nmvucfCnhobnvuWl9WKakCDDd2iefiEW3KC2bgTFKqSrAdyJyRCnVAqgLbLPKmxTD4nTSY7FS6iZw\nDIsMdS0gUkQOW88vAAYB07HEupinlPoe+N7eLyYi55VSR606O0cAX2CjNd+7sfMBLLItKeupi1Kq\nL5b/dUUsAXp2p7m2gfX4Rms5D2CpN40mQ7Sj0DgLw4CzgBlLS9gmKJGIfKmU+hNoA/yolOqHJZLX\nAhF5zY4yuqcUEFRKlU0vkVVbqD4WkbnOwMtA87v4LkuALsBBYLmIiLI8te22E9iBZXziY+BppVR1\nYAQQIiKXlVLzsQjfpUUBv4hIt7uwV3Ofo7ueNM5CaSDKGj/geSzib6lQSnkBR63dLSuxdMGsAzor\npcpb05RV9scUPwRUU0p5W/efB36z9umXFpEfsTgwczrXXsMie54ey7FEGuuGxWlwt3ZaBe3GAQ2U\nUr5YorddB2KUUhWAJzKwZQvQOPk7KaUeVEql1zrTaAy0o9A4CzOBF5VSf2PprrmeTpouwF6l1C4s\ncSkWWmcajQV+VkrtBn7B0i2TJSISh0Vdc6lSag+QBMzG8tD93prfBtLv458PzE4ezE6T72XgAOAp\nIlutx+7aTuvYx3tYVGH/xhIf+yDwJZburGTmAKuVUhEich7LjKyvrOVsxlKfGk2GaPVYjUaj0WSK\nblFoNBqNJlO0o9BoNBpNpmhHodFoNJpM0Y5Co9FoNJmiHYVGo9FoMkU7Co1Go9FkinYUGo1Go8mU\n/wfV/52ANExJ9wAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f6aa9baeda0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"n_classes = len(tags_counts)\n",
"roc_auc(y_val, y_val_predicted_scores_tfidf, n_classes)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Our ROC curve looks good. there is a good trade-off between sensitivity and specificity. that can be told by looking how closely the curve is approaching both the left side and the top side. our model is not bias nor it has a high variance. well done."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's apply the model on the test set. this will allow us to see whether our model is generalising"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {},
"outputs": [],
"source": [
"y_test_predicted_labels_tfidf = classifier_tfidf.predict(X_test_tfidf)\n",
"y_test_predicted_scores_tfidf = classifier_tfidf.decision_function(X_test_tfidf)"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Title:\tgay douche like gay porno ahaghagagagaa\n",
"True labels:\ttoxic,obscene,insult,identity_hate\n",
"Predicted labels:\tidentity_hate,insult,toxic\n",
"\n",
"\n",
"Title:\twell thinking good article could fall im little unsure start ive found couple books mine though theres definitely potential youre right list converted prose demolition plans opposition also needs expansion ill tinker bit later today ive got mo talk cont\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tpreceding unsigned comment added talk contribs 3 april 2006 utc\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\tthank thank constructive contribution update jodeci article kci jojo however commet uncalled copyedit mr treason additions\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\twikipedia blocks paedo article go http enwikipediaorg wiki virgin_killer see people wikipedia protecting photo shows real underage girl 12 bondage however wikipedia blocks people protecting photo guess wikipedia\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n",
"Title:\toh see working different definitions nationality thank bringing example holding american citizenship make person american else eg henry kissinger born germany german parents naturalised us citizen german politician tricky must say anyways nice talking\n",
"True labels:\tsafe\n",
"Predicted labels:\tsafe\n",
"\n",
"\n"
]
}
],
"source": [
"y_test_pred_inversed = mlb.inverse_transform(y_test_predicted_labels_tfidf)\n",
"y_test_inversed = y_test#mlb.inverse_transform(y_test)\n",
"for i,text in enumerate(X_test[84:90]):\n",
" print('Title:\\t{}\\nTrue labels:\\t{}\\nPredicted labels:\\t{}\\n\\n'.format(\n",
" text,\n",
" ','.join(y_test_inversed[84:90][i]),\n",
" ','.join(y_test_pred_inversed[84:90][i])\n",
" ))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A quick look at the examples, shows that the model is doing well on the test set."
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy: 0.921042737185\n",
"F1 Score: 0.923328020394\n",
"Precision: 0.881731181677\n",
"Recall: 0.914824374369\n"
]
}
],
"source": [
"y_test_binary = mlb.fit_transform(y_test)\n",
"print_evaluation_scores(y_test_binary, y_test_predicted_labels_tfidf)"
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXd8FNX2wL9nN500ehUICAQQQhcf\noigKij4LPMSngqI/BBGf2MDenjwVsaOABWxPRREBeTYUEVRQQOlNCAgJkFBDCkk2u+f3x0w2m7BJ\nlhI2gfv9fPazM/eeuXNmdvaeue0cUVUMBoPBYCgNR7AVMBgMBkPlxhgKg8FgMJSJMRQGg8FgKBNj\nKAwGg8FQJsZQGAwGg6FMjKEwGAwGQ5kYQ2Go8ojIgyLyVhDOe7WI7BCRLBHpeLLPXxUQkW0iclEZ\n+V+JyI3HUO7jIvLB8WlnCBRjKCoRInKuiPwiIhkisl9EfhaRrsHW61gRkXdE5KmKPo+q/kdV/6+i\nz+OHCcAoVY1W1T9KZoqIiki2bUhSReQFEXGWkLlcRH6z5faJyH9FpFEJmfoi8raI7BKRTBHZICJP\niEg1P+dsap835IRf7XHir3JX1UtV9d1g6WQIDGMoKgkiEgvMBV4FagANgSeAvGDqZSiTJsDacmSS\nVDUaOB8YBNxcmCEi/wA+BF4CagFtsX7vn0Skui1TA1gMRALnqGoMcDEQDzQ/oVdjMJSGqppPJfgA\nXYCDZeQ7gIeBv4B04D0gzs5rCigwFNgBHABGAF2BVcBBYGKJ8m4G1tuy3wBNyjh3d+AXu5yVQC87\nvQaQAvzd3o8GNgNDgFsBF5APZAFf2DINgM+APcBW4F8+53kc+MS+tkysSriLT/5YINXO2wj09jnu\nAx+5K+xjDwILgNY+eduAe+37kgFMByKO5p4D4fY1KZANbCnleAXO9Nn/BHjN3ha73DF+zrkGeNLe\nfwpYDTgCfI622+fNsj/nlPXs+Dm+l/2bjrFldwFXAf2ATcB+4EEf+XeAp0oeX+J+XwRcYj8LLluv\nlXb+AuD/yrietsA8+7xphef285t/Cuy2f9OFQFufvH7AOvu5SQXutdNrYb2cHbTLXxTofT7dPkFX\nwHzsHwJigX3Au8ClQPUS+TdjVcLNsCrkmcD7dl5Tu3KYDEQAfYBcYBZQB6t1kg6cb8tfaZfVGgix\nK5FfStGroa1XP7vCudjer23n97H/oHWAN4EZPseWrEQcwHLgUSDMvpZkoK+d/7itdz/ACTwNLLHz\nWmEZwQY+19zc57gP7O2WWJX3xUAoVoW3GQiz87cBv2EZrBpYxnJEKdde6j2384sZAj/He/OBRKxK\n9y6ffQUS/Bz3BLDY3l4CPHEUz1HhsxAS6HWUOL4XUGD/RqHAMCyj/iEQg1VxHy7U289v3As/hqLk\n7+STv4BSDIV9vl3APVjPdQxwtr+y7GuMwTLiLwErfPJ2AT3t7epAJ3v7aaz/TKj96QlIsOuCyvgx\nXU+VBFU9BJyL9Sd/E9gjInNEpK4tcj3wgqomq2oW8ABwbYm+6H+raq6qfotVWX6kqumqmor1tlQ4\n4DoCeFpV16tqAfAfoIOINPGj2g3Al6r6pap6VHUesAyrMsc+16fA93ba8DIusyuWgXlSVfNVNdm+\n1mt9ZH6yz+UG3geS7HQ3ViXQRkRCVXWbqm7xc45BwP9UdZ6qurDGESKBv/nIvKKqO1V1P/AF0KEU\nfQO55+Xxu4hkYxmkBcDrdnot+3uXn2N2+eTXLEXmaDja63AB4+z797Gty8uqmqmqa7HezpNKOfZE\ncjmwW1Wft5/rTFX91Z+gqk618/OwjEiSiMTZ2S6s5yZWVQ+o6u8+6fWxWtMuVV2ktgUxFMcYikqE\nXXHfpKqNgLOw3npfsrMbYHUdFPIXVmugrk9ams/2YT/70fZ2E+BlETkoIoXNbsFqPZSkCTCwUNaW\nPxfrD1bIG7a+76jqvjIusQnQoERZD5a4ht0+2zlAhIiEqOpmYDRWJZAuIh+LSAM/5yh2n1TVg9US\n8b22kueIxj+B3PPy6GSXPwg4GygcgN5rf9f3c0x9n/x9pcgcDUd7HftsQw3WcwOlP0snDBFZaw/8\nZ4lIT+AMwN/LQMnjnCLyjIhsEZFDWK0YKDK2A7BeYv4SkR9F5Bw7/Tmslta3IpIsIvef0As6hTCG\nopKiqhuwmvVn2Uk7sSraQhpjdRGkcfTsAIararzPJ1JVfylF9v0SstVU9Rmw/qRYhuI9YKSInOl7\nGX7K2lqirBhV7ReI0qr6oaqei3UfFHjWj1ix+yQiglXhpAZyjrLK4hjvuVp8gjUo/aidvBFrLGCg\nr6yIOLAqtu/tpO+Aq+30gE7nJ+1EPjslyQaifPbrlSFb5tu6qrZVawZZtKouwnpemgWgw3VY3akX\nYY0hNbXTxS53qapeidU9OgtrrAi7BXKPqjbDGte6W0R6B3C+0w5jKCoJIpIoIvcUTo0UkTOAf2L1\nUQN8BNwlIgkiEo3VXTTd7jo6WiYDD4hIW/tccSIysBTZD4C/i0hf+80tQkR6+UzhfBCrArgZ6w3t\nPZ8poGkU/6P/BmSKyFgRibTLOyuQKcAi0kpELhSRcKxxjMOAx4/oJ8BlItJbREKx+rfzsAbjj5YT\nec8BngGGiUg9u4vjXuBhEbnOvq/1gLewxqtetI95wd5/t7BrUEQa2lNt2/s5xx6s++J730/0dfiy\nAugnIjVs/UeXIZsGND0KozcXqC8io0UkXERiRORsP3IxWL/xPiyj9Z/CDBEJE5HrRSTO7ko7hP3c\n2FOTz7RfJjKwujf9PVOnPcZQVB4ysbomfrX7tJdgzX65x86fitVnvxBrtlAucMexnEhVP8d6G//Y\nbqqvwRpA9ye7A+tt7UGsSmgHcB/gEJHOwN3AELur4lkso1HYhH8bq2/4oIjMsmUuxxoT2IrVvfIW\n1ltgeYRjVbR7KRo8f8CPvhuxxlVetWX/jjUrKz+Ac5TkhN1zW7fVdln32fvTgcHAXViV3Dqs8ZQe\nhV149jjK37D6038VkUys1kYGVrdJyXPkAOOAn+373v1EX0cJ3seaCbcN+BZrFllpfGp/7xOR38uQ\nA6w3fqxJCX/H+s3/BC7wI/oeVndaKtY9XFIifzCwzX7WR2CN2QC0wGqxZWG19l5X1R/K0+t0RMzY\njcFgMBjKwrQoDAaDwVAmxlAYDAaDoUyMoTAYDAZDmRhDYTAYDIYyqXQeJsujVq1a2rRp02CrYTAY\nDFWK5cuX71XV2sdybJUzFE2bNmXZsmXBVsNgMBiqFCLyV/lS/jFdTwaDwWAoE2MoDAaDwVAmxlAY\nDAaDoUyMoTAYDAZDmRhDYTAYDIYyMYbCYDAYDGVSYdNjRWQqlqfQdFU9y0++AC9jBRTJAW7yiTxl\nOJXxaNHHrRDmAKefdxaPQmY+qFo+aQWIj/BfZlY+ZLkAWzY6DGLC/MumZEKB7U3a7YEmcRDi5/xZ\n+bAzq2i/Wig0jPFf5o5D1vkLnWyeEev//C43bNhftB/igNY1/Ze5OwvScor260ZBvVLiBa3aY11T\n4fnb1YYwJ263h717i8pwHMqn9oG8ouPiw6FlDQAOHcojMzPPW0Ts7mxiXT5OQ1tVhxqRAPz110Hc\nbjvUaJ6bpnsP48x1Q4FCjQjobsWUyszMY+fOTOsnVCXmQB6NfMtsEgtNLefByckHyMrKLwxfSrP0\nw8SEOotkz2kI4U5ycwtYtmynNzk8x0XXsNAiuZqR1vUDW7ceYMeOQ96spjkFNA73kW1XG2pZ1/TT\nT9spKHwu8t30ECehhc9FmBN6WLGv0tOzWbs23VtE7Vw3Z/mWmRAPCdY1/fHHLvbvP+zN6nDYQ81I\nH9lzG0G4k8OHXfz003ZvcsRhNz2rhRfJ1YqEpDoAbNq0j61bD3izWriUZhE+z1pSbahthQj57rtk\nXC43kb7nPAYqch3FO8BELBfA/rgUy81vCyz32pPs71Of3ALYnQ35bjiUD+FO74N9BB+sgz/3Ww+/\nW+HSBEj0U7Fku+CxnyyZAg9EhMBzvfyX+elG+GCtde41e+E/PWFYKZEtL/0UdmVDql1hrr7Jf2U1\n/y8Y+pVVSavCBY3hvcv8lznkf/DNNp9rvAz6Jhwpl54N7d4p2q8bBWtuBiAvr4B9+w7jdntwu5Xw\nN1dS/43VRbL3dYUxZ+P2KH9uPsCB/YdxexSPR2kxfB41fCvL326AutFkZeXx+utL8XgUVegREcZ5\nL/1RJNe7CUy1vLFPm/YHH3+8Fo9d5i0eJ9clZxbJvn0JXNQUgL59P2D//hyrsnS5+W6/Ut1hV0B1\nomDpEOsWzk/mrru+8RZxQY1qvJTsYyhGd4a7rNAdt4/8H4t8KpbXCpz0zPaQERPGb53qoi9fCNUj\nSU/PZvjwuV65OnERTFGfSqNDHW+Z789Yz8yZ671ZNzStwYD9Lp/zd4GOlqEePPZ7srKKPLe/Hx1N\ntEOsneoR0NkKyrf4jzTGj//ZK3dO43jGHHQXldm/BTSMBeDxqX+wckUagocQlIdrR5OUX2RUMhvG\nkRcSyoEDh3nlzbXe9LjoUEarj0E5szpEW4EEv/sulZ9/2uHNurBuND2zfMJwuEOguXWOZ/+7kdyc\norwxUeFEin1N0WHQoDoA69bt5dNPNnjlWteJ4ppsnzAWPRpiheCG97/YRvKWg0X3NC6S5gU+hrJu\nHFQLs67pg01lX1Os9b/7ecFm9q4qei47VI/grMM+589qBA0s2U++Wkt+npuISOF4qFA34yLSFJhb\nSotiCrBAVT+y9zcCvVS1zPjAXbp00ZO+4C4r36rY9+VCXJj/ihrgypnw1yHLABwugK/+4V928U64\nYmbRftd68OU/jpRLz4G2U4unTb4YBrQ6UjYjD858s2g/JgySb/Wv50vLYJyPy/7RneGhc/zLdnzX\negMvZMWN/t+q522D64oqJC5sDNOvAODgwVy2bNmPy+WhoMBD/H+W0GLFPrTwT/hmX7i4KR6P8tZb\ny1m+LouIqDB69WqF86ONSOEzGhUKQ9sB1pvizM/WUTcmi5AQoUa7bjQ96BOFtWYk+yJjycyvcmtK\nDYYTyq8zH2Nfyip2bly4XFW7HEsZwfwXNcQKglNIip12hKEQkVuBWwEaN258YrVQhb2HvU21Yvyc\nCm+uhP8lF6Vd0wpeu9h/WdmuojdvsIyFP8KdxffddkXocoNvU9vjgQEt4bNNR8qWxFnijaE0OTii\nm2dHtVCuzi1gT6iDAo/icnnst2olbG5/wjy+b0DV4LALIkPZty/H6n7wKNq2JnV+H0KIW4k9CA12\nQcHPOTTYkEdOtFDgKuoyqnX5RVanZCHbgDftprQ0J6ytFWZs/l/A31oX131Frr0RSbuLOhfL2lan\nRMhvn1BFYZqDoWpR8gn2EIJbwmjjmV9OUFXsIKh+CilNrqJkK6hMBx72OqujgFOEYjVKiAPsl7B8\nl5umDSJY9+PicgovmyrxuqWqb2DFZaZLly7H3wTyKLy8HL7ZCsvTrD7ybcOLV9IAqZkwsFVxQ5Ht\nolSiS/RJ55RiKMJKnMflgQ/XWf2VjWOL0utFW/3ivhQcGamxAJjo8rD/jo5EFb6ll+hznz9/Kzk5\nLvLz3eRLAf3v6ESYLfrh1S3ZiMOKoYYQkyVEZ1mDAo3/clAQAq3W53Ew3okViM8/tfa6S6RY+2F5\npf9kYZrt3Rb7n5In0YTqYRI9izggDeju/rS0wwHIcUayq1oDqjn2EOss/p4hFFDLkUx0QRazzryq\nzHKOhsiCwyyt15W0qLonrMwTRXxWPtPdSlRcOGnpWQwb9oU3r3ZMOG83qFEk3Lqmt5X2wX9X8dln\nVteTCFzXvBb/EJ9ndchZ0LYWALfcMpvMzHyrPlJ4s0lNYkOcVit472GY3R+AX39L4eWXf/WW2bVe\nDKMjIovK7N0YLrDCeU+Y8Atr1qTjUWH9/ia81KAuPSJ9+unf6gvxEeza1ZeBA4ueiXphIcyo4RMk\nsUs9uN/qxZ4yZRn//W9Rl+SwunEM9q1W7z8bOlthvv/+94/IzLS7JF0e5tSOJ7awTqgRAVP6AvDj\nj9t4/PEfvUWcV7MaT4jP//SaRKveAMaMmcfy5UXP5LMx1ejiO57yRh+Ij2D37iwGD/7cm1w3xMEH\nMT4t9y51YYx1TVOn/sH06WspDHk+tEY013qK/u/rBsbze+52brjhBq655lPyDyVx5RUTmTlzOMfK\n6dv11G8GLN1dtL/on0d2E6nCL6lw1ayitF5nwKdX+i/zhrnF+94//rvVr12S5IMwcLZlMMKd4HDA\nPV3gsubFxDweZe+7a6i5bDfOjfvh/DPg782hg1U5rV+/h3feWcEbmflkvdyPBikuHKVE/HWom9g8\na1AvNB+a7MghWvcTkupnbOAEcVHBJOJ1F1GaQWPP6mJ51XUnsezxe9yh/HDeXns2+3Or8e76rox6\n6EoGDSr+CP3++05GjfoSJARxOunUqQGvvuo3mivjx//MokXbcTgEh0O4++7u9Ox55O+Sk5PPxIlL\ncTgEEatiu/LKRJo3r3GE7I4dGaSmHvKWWb9+DA0bxh4hB/Dnn/twuz2IWOUmJFQntORLCZCdnU9a\nWpHhjIoKoV49/4Pnu3dnctinxVq3bjWioqwXldpYgaMNpxc5OTk89dRTPPfcczidTtasWcOZZ57p\nzReRY+56CqahuAwYhTXr6WzgFVXtVl6ZARuK9ByrO6ZmpP/8G7+EL31aClP6QP+WR8qt3Qu9Pra2\nO9W1xhOe6um/zK0Z1kB1fLjVEqgW6n82TxmoKj16TCUjI49MdyjRcaG8/PIlVI8vug4FftroYdtu\nFztTDhJdL44QOb5ZDQBRetAuXzgscXRwf4kiFBBGG88CqmsqdT3JZZYRSQa11R5kbW+9wezfuYPb\n3+vEvvxaOEOEjh3r859xvf0e/+7HW0nfm0vTpvFERobSsmVNWrYsZUzIYDAA8NVXX3H77bezdetW\nAIYPH87TTz9N9erVvTKV0lCIyEdAL6AWVhvpMeypAKo62Z4eOxG4BGt67FBVLdcClGsosvKtgdr/\nroN/tIQXLvQvd+8P8G7RzAn+1Qke+duRcoX9847AZg1kHvZ4D3G7leQ0N6rKwRxlW3oBkWECqqSl\nHaKWbMPpzgGxjIkC6bmZbMgpxRAFQLzuopmn9PuT67J6Gz3hcdRzr6de7Tiindl0c36HNDyHNdtC\n+f77ZCIiQggLd9K6dW26n93Ib1lr1+3BVaAcrn42oRERtGldy3qrdYRCdP1jvgaDwRAYqampjB49\nmhkzZgDQvn17Jk+ezDnnHDk5pVIaioqiTEOx4xB08pmNK8D3g/xPPf05BXZkQptakFjjyHGDMihw\nKyu3uShwK6v/chEeahmRhevyyjny2OjonoNHiusXSi51dAuJ+iNO8okpyGRpThP+3u/viDMcGhU3\nNjNmrGNHZi2iYqKJjg6jX78WVK9eSmvLYDBUCa666ipmz55NVFQUTz75JHfeeSchIf6Hno/HUFSJ\nweyA2ZFpGYdC26fAQ4tg9tXeWQBeevh/Sy6N7FwPo6cepE6cg/SMUgYCfIiNBAoOk10QRrw7hQTP\n72RLDZp5lhKvRWMjKg5odhkiDgpHQkLdBfy56CB7c3JZf6gaDR+5nFatah1xjj2ZeXz47RYio0Jp\nHBHCmK4NkZID6jb/GN7mqK7XYDBUTgoKCrzG4NlnnyU0NJTnn3/+xM8I9eHUalGANaX1urmQY89O\n+ve5MKz9UY0VvP1dFks351Mj2jom16VkHvZ/n7qeGUZ+gdKuSSi5hwvI3H8IzTjAwO1HDMsAMO67\nnoSHFPDEvF58+OmN/PpbKjufvIBfRVhny4zfsJf4RX9x4YUJNGtWHSlp5AwGw2lHRkYGDz/8MJs2\nbeLrr78+6nrBdD2VZPFOeHcNPPo37wrFsvB4lJ835PPeguxyZVs1DGHw+dWIjhCqhQO5B9BNM5j9\n+tuk7fNQLyaLK8/a6JV3h0azpu1N5FSrx28d72X0vfO9efHVI2h/QVMWXlA086g58CfFp1cbDIbT\nF1Xl008/ZfTo0ezatQun08nSpUvp2LHjUZVjDMUx4FHlrXnZOB2wZFO+X5lHBsYSYS82EKBmrANH\noRXfuQQ+KmU1sw8R9yiBjlysARIwUxsNBoPFli1bGDVqFF9//TUA55xzDpMnT6Z9+/ZHXZYZozgK\nVJWXvshkXYr/xXAD/xZJ5+Zh1Ih2FGvape0+xKy33uLq5guR/evhQNFq6UPh8cTmHeTu858nO7Qa\nIZ4Cfmh8AetrFo0LjCpHr8uBtsdzYQaD4ZRiwoQJPPLII+Tm5hIfH8+zzz7L//3f/+FwnHyn31XX\nUKhCrhsiA7+EnDwPd7598Ij0oRdWo1FNJ41rlyhr0ww2fTGJjF3b6Fo/mf4AW4qLPHHlbB4/0/Jp\nVLhq2lqsVbSsfgDwasBaGgwGg7WALjc3l8GDBzNhwgTq1KkTNF2qrqGYtRkm/m6tVj6vEZzdoEyj\nUeDWI4zE8zfFExtVwjq7XbDmbVhwDxTk0BKgxJKAj1pdy4etr2Nx/XPYF2XNRmrr9vCHqt8VtwaD\nwVAee/bsYePGjZx77rkAjB07ll69enHeeecFWbOqOkaxdCn0/Ag2+vj1L8tZHzDs9SLZtmeEMvrv\nJVwjuHLguxGw7v0jjp2yuDNfbzyTjZHNWP/bfwC4D8s/eiEXAGcecaTBYDCUjcfjYerUqYwZM4aQ\nkBA2bNhAjRpHuo05Xk6/MYqV6cWNhENgeIcjxDweZcHaPD5aVNxz6B2XFZ8JtXnmM5y59YEjjp/8\ntyeZ1nYUv/W0j+9W5J30GUx4QIPBcHysWbOGESNG8PPPVsyOiy++mJycnAoxFMdD1TQUa/ZZ3UyF\nTtEubAzti6++XrvdxUtzj/R0OuW26t6ZS/t2pzP57vt5qPM0b75bHJx14xo21PRxb92terEyhmOM\nhMFgOHays7N58skneeGFFygoKKBu3bq89NJLDBo0qFKum6qahuKGNpYb36W7YMEOy1FfCXyNhEOg\nT4cI+neP9P4I+ryDmigP+YQ02HjJFhLbNgNgLpYXzpKEAKXEgjMYDIaA+Mc//uFdNDdy5EjGjRtH\nfHx8sNUqlappKMByz31uI+tTYpzlr/Siqa+j+kWT1LSEW4tZV3ljHwAkRzTmtosn8W3LZt60UoJ4\nGgwGw3EzduxY0tLSmDRpEmefXfkjQFddQ+GLT1Nt7yE3T80oCqZ+hJFYOBa2zPbuOt0/4bm9RzER\n/1ENDAaD4egpKCjg1VdfZdu2bbz88ssA9OrVi2XLlgVlTcSxcGoYCh8e+CDDu31z72pHCiwd791c\nOSoDT7gVbCbJo/xquxL371bPYDAYjo7ffvuN4cOHs2LFCgBuvfVW2ra1ltZWFSMBp9iYbK6rqDtp\n4N8iOadVeHGBv77zbn4xZBUdwosikn3iEMKBcIyfJYPBcHwcPHiQkSNH0r17d1asWEGTJk344osv\nvEaiqlH1DEUZyz6SdxeNTfTpUCLWwqIHYUbROosrarfzbj8N+IltZzAYDEfNxx9/TGJiIpMmTcLp\ndDJ27FjWrl3L5ZdfHmzVjpmq1/W0fh+MWQDDk6B58WmrL35hzXRqWKPE6uhvh8Hqt7y7qVf/z7v9\nPnBDRelqMBhOO7799lvS0tLo0aMHkyZNol27duUfVMmpei0KlxumrYHu/4X/+9qvSMuGPvZv9dRi\nRuKF/W+RekZf7/71FaaowWA4HcjLyyM5uSiW/Pjx45k6dSoLFy48JYwEVMUWhS+bD3g33/2hKJbE\ntefajrrVA9/e4k2v98Q9pGWmUL/5Kri5I+dgxiMMBsOxM3/+fG677TYcDgcrV64kLCyMWrVqMXTo\n0GCrdkKpei0KXxpa/prWbM/np/VW1AcRimJGvFDUBXXpW9eTlhkD/Vuz62Yr4EfWydXWYDCcIqSl\npTF48GB69+7Npk1WyIGUlJQga1VxVD1DUTsKBiVCn6bQvQEA81cVhQZ67kZ7dWPWLm+aRtZmq3aH\n2HD47Bpv+sMnRWGDwXCq4PF4mDJlComJiXzwwQdERETw1FNPsXLlSpo1a1Z+AVWUqtf11CAaJl5U\nLGn1dis+dv/ukcQVug3fu8qbLyPTeTR+Nfe9u5Kddtok4MqToK7BYDh1uPrqq5kzZw4Affv25bXX\nXqN58+ZB1qriqXotihK4Cormy7ZpFFqU8dkl1ndjy6gMGtSWvrMGebNHYK2ZMBgMhkDp378/9erV\nY/r06Xz11VenhZGAqtiiKMG+TI93u0kd+3LyilZnUyMRAKfTwbRIyy5ecNK0MxgMVZk5c+aQkpLC\nyJEjARgyZAj9+/cnJiamnCNPLaq8odi2x0/s6+0/FG1f+AoAHp/s1ytWJYPBUMXZvn07//rXv5g9\nezbh4eFccsklNGvWDBE57YwEnAJdT4fzrK6nevHWpWzZsp/9n98GgKdWB6/DwHE+xySeVA0NBkNV\nweVy8fzzz9OmTRtmz55NTEwM48ePp0mTJsFWLahU+RbFh3b0uhYNrPGJ2bPWcrdzNwBTvwrjfzHr\n6dq/NY/a8sELT24wGCozS5YsYfjw4axaZU2EGThwIC+++CINGzYs58hTn6pnKDbuhz6fQJiT9Hev\n8CbXiLZaFLt+nQO2e/dRW28kr39rZvkc/ttJVNVgMFQdHnnkEVatWkVCQgITJ06kX79+wVap0lD1\nDEVuAfyRDsDGNLc3+dJOEezZk83WjTu8hiJvQFEsugeANsDp3YA0GAyFqCqZmZnExlpepCdOnMh7\n773HQw89RFRUVJC1q1xUPUNRSJiDXQesIeqEOk6cDqFatTDe+r+VkA+zN3ZEJnZHgc7Af4KqrMFg\nqExs3LiRkSNHIiLMmzcPEaFVq1aMGzeu/INPQ6ruYHaYk5w8y1CcUcuyd1FkEJ+/AYBLe8eiYZYL\njzuCo6HBYKhk5Obm8thjj9G+fXvmz5/PihUr2LZtW7DVqvRUvRZFyxrw3j/ArWzcZE2NbVLH9umU\nttQrlnr1F95t4yHWYDDMmzePkSNHsnnzZgBuvvlmxo8fT82aNYOsWeWnQlsUInKJiGwUkc0icr+f\n/MYi8oOI/CEiq0Sk/NGjyBDj+dqAAAAgAElEQVToXA+61Scmwpr6Gh1hX8aaqbZMLV4Ms+Y6t6Uq\nWkODwXCiUFVuvvlm+vTpw+bNm2nTpg0LFy7k7bffNkYiQCrMUIiIE3gNuBRrHPmfItKmhNjDwCeq\n2hG4lqNcC7c13RrMrhHtgMwU2DQDgIKEfrxqy5ghKYPh9EZEaNq0KZGRkTz99NP88ccf9OzZM9hq\nVSkq8mW7G7BZVZMBRORjLD9863xkFCgMXB0HXp995bL3UNGMpzpxDkj+0bs/7eI3vNvvHLXaBoOh\nqrNixQp27drFpZdeCsDYsWMZPHgwCQkJQdasalKRXU8NgR0++yl2mi+PAzeISArwJaWMO4vIrSKy\nTESW7dmzB4Af1xa5Fp/65jJ+/NyKdleQcBXzQix3f9WA1ifgQgwGQ9UgMzOTu+++m86dO3PjjTey\nf/9+AMLDw42ROA6CPevpn8A7qtoI6Ae8LyJH6KSqb6hqF1XtUrt2bQC2plkD2U3rOLnjjq/YsWYl\nAHO+2ET25n0ADMJEsDMYTgdUlc8//5w2bdrw4osvAnDdddcRGhpazpGGQKjIrqdU4Ayf/UZ2mi+3\nAJcAqOpiEYkAagHppZaa44LVe8jJsmY6dUywHoRWdSzj8Oee6kTUrgZY6ycMBsOpzV9//cWoUaOY\nO3cuAF26dGHKlCl06tQpyJqdOlRki2Ip0EJEEkQkDGuwek4Jme1AbwARaQ1EAHvKLPXPA3DhdA7s\nzQcg3J0LQNczrOGNpTsaMDMuAgh+c8lgMFQsqsqAAQOYO3cusbGxTJw4kSVLlhgjcYKpsLpUVQuA\nUcA3wHqs2U1rReRJESl00nQPMExEVgIfATepqvovsQiPCFlhYQA0rB3Gqy+e783bE9mDwp7Is07Y\n1RgMhsqEx2MtthURJkyYwKBBg9iwYQO33347TqczyNqdekgA9XKloktoY/2x3kOMfmAgAG/cVh05\nsAmm2c7D71HvuMQW4NSNYmswnH7s27eP+++3lmS9+eabQdamaiEiy1W1y7EcW/V6ZyJDOdCpAQDh\nodYbBSsne7N3+YiaUKcGw6mBqvLuu++SmJjIW2+9xXvvvUdKSkqw1TptqHqGomV1DrzSFwCHHZSI\nHfOt79od+MBH1HiRNxiqPuvXr+eCCy7gpptuYu/evfTq1YuVK1fSqFGjYKt22lD1DAWQbE+NjYuy\nDcUeK9AIF7zIE7ZM/ZOvlsFgOIGoKo888ghJSUn8+OOP1KpVi3fffZf58+eTmGjiVJ5MqqSh8NjD\nKrVineB2edN3Nvgb2fb2MydfLYPBcAIREVJTU3G5XAwbNoyNGzcyZMgQq7vZcFKpkoZi+RZramyD\nGDc5m+d70//tDPNuDzrpWhkMhuNl586d3lCkAOPHj+enn37ijTfeoEaNGkHU7PSmShqK6HBL7Xff\nXcE3T48G4HBBKJ/lW/6fumMGsg2GqoTb7WbixIm0bt2aa6+9lvx862WwVq1a9OjRI8jaGaqkoSiw\n+562rdlBZKjV9TR7dUtcTqtJOiJomhkMhqPl999/p3v37txxxx0cOnSI5s2bc+jQoWCrZfAhIEMh\nImEicmZFKxMQOzL5a4/Vcti/N4sWtSynX5+sbMtBp3U5bYOmnMFgCJRDhw5x55130rVrV5YtW0aj\nRo2YOXMmc+bMoVatWsFWz+BDuYZCRC4DVgPz7P0OIvJ5RStWKvsPE55ntSIa1A7nUH4kAAfiGnhF\nmgdFMYPBECiqynnnnccrr7yCiHD33Xezbt06rr76ajNYXQkJpEXxJHA2cBBAVVcAQW1d5IVbjgB/\n+XkoHRtYPp4ueaso4GlcULQyGAyBIiLcdddddOvWjWXLlvH8888TExMTbLUMpRCI91iXqh4sYeWD\n5vdDffQIcQKh0eDK4udG1vK686miAy8GwylMfn4+L7zwAk6nk/vuuw+AIUOGcMMNNxjfTFWAQAzF\nehG5BnCISALwL2BJxapVOp6G1ltHhFNx5GeCKwuAryOt2LdXlHqkwWAIBosWLWLEiBGsW7eO8PBw\nhgwZQt26dRERYySqCIG8fI/CCu3gAWYCecCdFalUWRTEWhNfVQS2fe1Nd9lrKK4NilYGg6Eke/fu\n5eabb+a8885j3bp1tGjRgrlz51K3bt1gq2Y4SgIxFH1VdayqdrQ/9wOXVrRipVHo7DYmygH71gOQ\nFdsUgOuBBv4PMxgMJwlVZdq0aSQmJjJt2jTCwsJ47LHHWLVqFRdddFGw1TMcA4EYiof9pD10ohUJ\nlALLDT11Yh2w/XsAUpr2AcwiO4OhsvDBBx+wb98+LrzwQlatWsXjjz9OREREsNUyHCOljlGISF+s\nMKUNReQFn6xYrG6ooOC2F9utWJXOWsmjbQR8G1YPgLCyDjQYDBVGTk4OGRkZ1K9fHxHh9ddfZ+nS\npVx//fVmuuspQFmD2enAGiAXWOuTngncX5FKlUVh19P2zenkVd8OjWBuE2uJf3SwlDIYTmO++uor\nbr/9dpo1a8a8efMQEVq1akWrVq2CrZrhBFGqoVDVP4A/ROS/qpp7EnUqk7wCy1IcTDtAnTMsX7GF\nMe2GBU0rg+H0IzU1ldGjRzNjxgwAYmJi2Ldvn1lVfQoSyBhFQxH5WERWicimwk+Fa1YKzoN59pbg\nVkv9vZHWg9kiSDoZDKcTbrebV155hdatWzNjxgyqVavG888/z/Lly42ROEUJZB3FO8BTwASs2U5D\nCeKCu0Iub1OdJtUzAMgIt9Zim55Qg6Fi8Xg8nH/++fz8888AXHXVVbz88ss0btw4yJoZKpJAWhRR\nqvoNgKpuUdWHCeb0WNsaXNmtAYREAbAnqrZpTRgMJwGHw0GfPn0444wzmD17Np9//rkxEqcBgbQo\n8kTEAWwRkRFAKhA0pyyFLjycDgG31Q11OCQyeNOwDIZTGFXlk08+ISQkhAEDBgAwduxY7r77bqKj\nzfSR04VADMVdQDUs1x3jsHzu3VyRSpVFQbQ1CdbZuTZsd1MgTtyOEEzPqMFwYtmyZQsjR47k22+/\npXbt2lx44YVUr16d8PBwwsPNqqXTiXINhar+am9mAoMBRKRhRSpVtkLWl6egqDUBJvSpwXCiyMvL\n47nnnmPcuHHk5uZSvXp1xo0bR1yc8ct8ulKmoRCRrkBD4CdV3SsibYGxwIVAo5Og3xE47DGKKD0I\nWOMTYK0MNBgMx8eCBQu47bbb2LBhAwCDBw9mwoQJ1KlTJ8iaGYJJqYPZIvI08F8sF0pfi8jjwA/A\nSqDlSdHOD4XTrZz5VmS7PZGWoUgMkj4Gw6mC2+1m5MiRbNiwgVatWjF//nzee+89YyQMZbYorgSS\nVPWwiNQAdgDtVDX55KhWNg+Ons70fpAeZT3EZmqswXD0eDwecnNziYqKwul0MmnSJBYuXMiYMWPM\nOITBS1nTY3NV9TCAqu4HNlUGI+H1HhuSCVgtikaeoC/rMBiqHKtXr6Znz57ccccd3rTzzz+fRx55\nxBgJQzHKalE0E5GZ9rYACT77qGr/CtWsFFxuyyjUiLQMRXpUHQaa5oTBEDDZ2dk8+eSTvPDCCxQU\nFLB161YOHDhA9erVg62aoZJSlqEYUGJ/YkUqEjB246FLgxTAGsyub7xTGgwB8cUXXzBq1Ci2b9+O\niDBy5EjGjRtHfHx8sFUzVGLKcgr4/clUJFAcdjfTxR3qQSaEeArYH2SdDIbKTkFBAYMGDWLmTKtT\noEOHDkyZMoVu3boFWTNDVSAQFx6VC7vxEO6wHNr+Wb0FbYOojsFQFQgJCSEuLo7o6GhefPFFli5d\naoyEIWAq1FCIyCUislFENouI3xgWInKNiKwTkbUi8mF5ZRa68PDkrgCsBXdNTqTSBsMpwq+//sqv\nv/7q3X/uuedYv349o0ePJiQkEKcMBoNFwIZCRI5qGoSIOIHXsBwItgH+KSJtSsi0AB4AeqhqW2B0\n+QVbX3mR1QBwOUJJOhrFDIZTnIMHD3LbbbdxzjnnMHToUPLz8wGoWbMmjRoFZZ2soYpTrqEQkW4i\nshr4095PEpFXAyi7G7BZVZNVNR/4GGtthi/DgNdU9QCAqqaXV2jhRNj4g9bK0XYxjYkNQBmD4VRH\nVfnwww9JTExk8uTJOJ1OrrjiCtxud7BVM1RxAmlRvAJcDuwDUNWVwAUBHNcQa5FeISl2mi8tgZYi\n8rOILBGRgD1x5ERaU/lyQqsFeojBcMry559/0qdPH66//nrS0tLo0aMHf/zxB8888wyRkZHBVs9Q\nxQmko9Khqn+VCJB+ol5RQrAC0/XC8h21UETaqdqOnGxE5FbgVoBaja2OppD8wwAki/kTGE5vXC4X\nF154ISkpKdSoUYPx48czdOhQHI6qN1fFUDkJ5EnaISLdABURp4iMBgIJhZoKnOGz38hO8yUFmKOq\nLlXdapd7RAwiVX1DVbuoaheAAlcBkW4rXnb01sMBqGIwnHqo7aYgNDSUcePGcdNNN7FhwwZuueUW\nYyQMJ5RAnqbbgLuBxkAa0N1OK4+lQAsRSRCRMOBaYE4JmVlYrQlEpBZWV1T5bkI8RWGKNDoqAFUM\nhlOHtLQ0Bg8ezFNPPeVNGzJkCNOmTaN27dpB1MxwqhJI11OBql57tAWraoGIjAK+AZzAVFVdKyJP\nAstUdY6d10dE1mF1Z92nqvvKK9udud8KpQREhJtpfobTA4/Hw5tvvsn999/PwYMHiY+PZ/To0cTE\nBC3gpOE0IZBadqmIbASmAzNVNTPQwlX1S+DLEmmP+mwrVmvl7kDLBKgZ4gLg66Z9OSvSGArDqc/K\nlSsZMWIES5YsAeCSSy7htddeM0bCcFIot+tJVZsDTwGdgdUiMktEjrqFcSKJF6tvtl72biKrm8Fs\nw6mLy+Xi3nvvpXPnzixZsoT69evzySef8OWXX9KsWbNgq2c4TQhoxEtVf1HVfwGdgENYAY2ChsOe\ndPVlQj/alCNrMFRlQkJC+OOPP/B4PNxxxx2sX7+egQMHIsYRpuEkUm6/jYhEYy2UuxZoDcwG/lbB\nepWtU3UPZEN2aDXMKgrDqcb27dtxu90kJCQgIkyePJmMjAy6dOkSbNUMpymBtCjWYM10Gq+qZ6rq\nPar6a3kHVSSOsFAAogpyqBdMRQyGE4jL5WLChAm0bt2aYcOGeae/tmjRwhgJQ1AJZCS4map6yhc7\neUh+BgDra7Q+ctGFwVAFWbx4MSNGjGDVqlUA1KhRg5ycHKpVM21mQ/Ap1VCIyPOqeg/wmYgcEWs0\nWBHuAMRhqR2OiZVtqNocOHCA+++/nzfeeAOAhIQEXnvtNS699NIga2YwFFFWi2K6/V05Itv5cDjH\nikWxIS4hyJoYDMdOXl4eHTp0YPv27YSGhnLffffx0EMPERVlFpEaKhdlRbj7zd5srarFjIW9kC5o\nEfBinFngBk0vONLNoMFQRQgPD+eWW27h+++/Z9KkSbRpY+bwGSongQxm3+wn7ZYTrcjRsMMOa5Hr\nNGsoDFWH3NxcHnvsMT78sCg+14MPPsiCBQuMkTBUasoaoxiENSU2QURm+mTFAAf9H3VyOFOXAnAw\nvFYw1TAYAmbevHmMHDmSzZs3U6dOHa6++moiIyNNpDlDlaCsp/Q3rBgUjbAi1RWSCfxRkUqVhwNr\nElZEwwbBVMNgKJfdu3dz991389FHHwHQtm1bJk+ebGJEGKoUZY1RbAW2At+dPHUCw4EblyMER3RE\nsFUxGPzidruZMmUKDz74IBkZGURGRvLYY49x1113ERYWFmz1DIajoqyupx9V9XwROUBRBFKwZqSq\nqtaocO1KQdRNTkgUacFSwGAoB7fbzauvvkpGRgb9+vVj4sSJJCSYWXqGqklZXU+F4U4r3UCAAzcF\njhAz4clQqcjMzMTtdhMfH09YWBhvvvkmaWlp9O/f3/hmMlRpSp315LMa+wzAqapu4BxgOATXxZID\nD25x0i+YShgMNqrKzJkzad26Nffcc483/dxzz2XAgAHGSBiqPIFMj52FFQa1OTANK1Tph2UfUrE4\nKKDAEYLp6TUEm23btnHFFVcwYMAAUlNTWbNmDbm5ucFWy2A4oQRiKDyq6gL6A6+q6l0EeZmbAzfV\n8w6Qn+8OphqG0xiXy8Wzzz5LmzZtmDt3LrGxsUycOJFffvmFiAgzycJwahFQKFQRGQgMBq6y00Ir\nTqXy2SMJ5DnCifhlB/RqGkxVDKchOTk5dO/endWrVwNw7bXX8sILL1C/fv0ga2YwVAyBGIqbgZFY\nbsaTRSQB+Khi1SqbBrqBDTUTaZOTH0w1DKcpUVFRdOnShZycHF5//XX69OkTbJUMhgqlXEOhqmtE\n5F/AmSKSCGxW1XEVr1rpOHFRNzuNjJoBBegzGI4LVeW9996jefPmnHvuuQC8+OKLhIWFmYVzhtOC\nQCLc9QTeB1Kx1lDUE5HBqvpzRStXGg51s7VaE+LiTF+woWJZv349t912Gz/++COtW7dmxYoVhIWF\nERcXF2zVDIaTRiBdTy8C/VR1HYCItMYyHEELuSV42BPbgEHdGwVLBcMpzuHDhxk3bhzjx4/H5XJR\nu3ZtHnjgAUJDgzo8ZzAEhUAMRVihkQBQ1fUiEtSZqQ48nHHor2CqYDiF+frrr7n99ttJTk4GYNiw\nYTzzzDPUqBE0ZwQGQ1AJxFD8LiKTgQ/s/esJulNAN+l1uwVTBcMpSlZWFoMHD2bv3r2cddZZTJ48\nmR49egRbLYMhqARiKEYA/wLG2PuLgFcrTKMAEDx4QsODqYLhFMLtduPxeAgNDSU6OpqXX36ZlJQU\n7rrrLtPVZDBQjqEQkXZAc+BzVR1/clQqHwdu3E6zLttw/Cxfvpzhw4dz5ZVX8sgjjwBw3XXXBVkr\ng6FyUer8UhF5EMt9x/XAPBHxF+kuKDhwow7zpmc4dg4dOsSdd95Jt27dWL58Oe+//z4ulyvYahkM\nlZKyFiJcD7RX1YFAV+C2k6NS+QjK9tQcXC7jwsNwdKgqn376KYmJibzyyiuICHfffTe///676WYy\nGEqhrK6nPFXNBlDVPSJSaVa3ZUgdsvZsNl45DUdFZmYmgwYN4quvvgLg7LPPZvLkyXTo0CHImhkM\nlZuyDEUzn1jZAjT3jZ2tqv0rVLMyqO/ZxFx3M0JCKo3tMlQBoqOjycvLIy4ujmeeeYZbb70Vh8M8\nQwZDeZRlKAaU2J9YkYocDaHkkrfbdDsZymfhwoXUr1+fFi1aICJMnTqViIgI6tatG2zVDIYqQ1kx\ns78/mYocDSHk0yypfbDVMFRi9u7dy5gxY5g2bRq9e/dm3rx5iAhNmjQJtmoGQ5WjSra7c4mmbZe2\nwVbDUAnxeDxMnTqVVq1aMW3aNMLCwujZsydut2mBGgzHSoUaChG5REQ2ishmEbm/DLkBIqIiEpD/\nqHjdjTMk5sQpajglWLt2Lb169eKWW25h//799O7dm9WrV/PYY48REhLI2lKDweCPgP89IhKuqnlH\nIe8EXgMuBlKApSIyx9dvlC0XA9wJ/Bpw2XhIaGT6mA1FZGRk0L17d7KysqhTpw4vvPAC1113nZkZ\nZzCcAMptUYhINxFZDfxp7yeJSCAuPLphxa5IVtV84GPgSj9y/waeBQIONOxyhpgKwABY6yIA4uLi\nGDt2LCNGjGDDhg1cf/315hkxGE4QgbQoXgEux1qljaquFJELAjiuIbDDZz8FONtXQEQ6AWeo6v9E\n5L7SChKRW4FbAWo1TiLCfZg6AShgOH5cLhcpKSnk5gZsx08KBQUFHDhwgMjISKKjowEYMMCaqLd7\n9252794dTPUMhqARERFBo0aNTugC0kAMhUNV/yrxdnbcI4P2Ar4XgJvKk1XVN4A3AGo36aC5IeHU\nO14FDAGRkpJCTEwMTZs2rRRv6KpKeno6qampREVFER4eTmJiYqXQzWAINqrKvn37SElJISEh4YSV\nG8hg9g4R6QaoiDhFZDSwKYDjUoEzfPYb2WmFxABnAQtEZBvQHZgTyID23shaDL3m0wBUMBwvubm5\n1KxZs1JUxNnZ2axfv54dO3bg8XiIj4+nVatWlUI3g6EyICLUrFnzhPcABNKiuA2r+6kxkAZ8R2B+\nn5YCLUQkActAXAt43XKqagZQq3BfRBYA96rqsvIKdjsceLKNA7eTRbArYrfbTWpqKunp6QCEhYXR\nuHFj4uPjg6qXwVAZqYj/a7mGQlXTsSr5o0JVC0RkFPAN4ASmqupaEXkSWKaqc45a26KycXr0WA83\nVDFEhEOHDgFQr1496tevj9PpDLJWBsPpQyCznt4UkTdKfgIpXFW/VNWWqtpcVcfZaY/6MxKq2iuQ\n1gRA/ZxdOAo8gYgaqii5ubkUFBQA4HA4SEhIoE2bNjRq1MivkZgzZw7PPPPMyVaz0rFgwQLi4uLo\n0KEDiYmJ3HvvvcXyZ82aRfv27WndujXt2rVj1qxZxfInTJhAYmIiHTp0oGvXrrz33nsnU/2AeOml\nlyqlXoUsXLiQTp06ERISwowZM0qVW758Oe3atePMM8/kX//6l3cG3/79+7n44otp0aIFF198MQcO\nHABg7ty5PProoyflGo5AVcv8AIN8PjdizX56tbzjKupTq3GSfvj+KM1fv0cNFc+6deuKJ9R6tfin\nNN5dXVzuru8DOp/b7dbU1FRdtmyZbt269dgVLwOPx6Nut7tCyg4El8tVYWX/8MMPetlll6mqak5O\njrZq1Up/+uknVVVdsWKFNm/eXJOTk1VVNTk5WZs3b64rV65UVdVJkyZpnz59NCMjQ1VVMzIy9J13\n3jmh+hUUFBzX8S6XS9u1a3dU97Ai77c/tm7dqitXrtTBgwfrp59+Wqpc165ddfHixerxePSSSy7R\nL7/8UlVV77vvPn366adVVfXpp5/WMWPGqKr13Hbo0EGzs7PL1eGI/62qYvXkHFO9W26LQlWn+3ze\nBfoDnSvOdJVPfqiT0FDT9XCqkZmZybp169i5c6f3Ad26dSuJiYncdNNNtGzZkuuvv57vvvuOHj16\n0KJFC3777TcA3nnnHUaNGgVAWloaV199NUlJSSQlJfHLL7+wbds2WrVqxZAhQzjrrLPYsWMHH330\nEe3ateOss85i7NixfnXatm0bPXv2pFOnTnTq1IlffvkFgGuvvZb//e9/XrmbbrqJGTNm4Ha7ue++\n++jatSvt27dnypQpgPWm37NnT6644gratGkDwFVXXUXnzp1p27Ytb7xR1Eh/++23admyJd26dWPY\nsGHe69qzZw8DBgyga9eudO3alZ9//rnM+xkZGUmHDh1ITbXmkEyYMIEHH3zQOxsmISGBBx54gOee\new6A//znP0yaNInY2FgAYmNjufHGG48od/PmzVx00UUkJSXRqVMntmzZwoIFC7j88su9MqNGjeKd\nd94BoGnTpowdO5ZOnTrx3HPP0a1bUbz7bdu20a5dO8B6wz7//PPp3Lkzffv2ZdeuXUece/78+d63\ndYA333yTrl27kpSUxIABA8jJyfH+HiNGjODss89mzJgxZGdnc/PNN9OtWzc6duzI7Nmzy/x9j4em\nTZvSvn37Mj0T79q1i0OHDtG9e3dEhCFDhnhbd7Nnz/be9xtvvNGbLiL06tWLuXPnHreOR83RWhas\n0KhbjtUyHe+nVuMkfffDe1XTyreqhuPnZLQo8vPzNTk5WZcuXapLly7V1atXe99qt27dqk6nU1et\nWqVut1s7deqkQ4cOVY/Ho7NmzdIrr7xSVVWnTZumt99+u6qqXnPNNfriiy+qqvUGe/DgQd26dauK\niC5evFhVVVNTU/WMM87Q9PR0dblcesEFF+jnn39+hG7Z2dl6+PBhVVXdtGmTdu7cWVVVZ86cqUOG\nDFFV1by8PG3UqJHm5OTolClT9N///reqqubm5mrnzp01OTlZf/jhB42KivK+zauq7tu3T1WtN/+2\nbdvq3r17NTU1VZs0aaL79u3T/Px8Pffcc73X9c9//lMXLVqkqqp//fWXJiYmHqGvb4ti//792qlT\nJ921a5eqqnbs2FFXrFhRTH7FihXasWNHzcjI0Pj4+FJ/I1+6deumM2fOVFXVw4cPa3Z2drHzqqre\nfvvtOm3aNFVVbdKkiT777LPevKSkJO99eOaZZ/Tf//635ufn6znnnKPp6emqqvrxxx/r0KFDjzj3\no48+qq+88op3f+/evd7thx56yJt344036mWXXeZtwTzwwAP6/vvvq6rqgQMHtEWLFpqVlVXq71uS\nc889V5OSko74zJs3r9T7dOONN5baoli6dKn27t3bu79w4ULv/YuLi/OmezyeYvsffPCBjho1qtRz\nFnKiWxTlDmaLyAGgcOTYAewHSvXbdDKoHlYAdaKCqYLhBOFyuVi7di0FBQWICPXr16devXrF3sYS\nEhK8b51t27ald+/eiAjt2rVj27ZtR5Q5f/58bx+20+kkLi6OAwcO0KRJE7p37w7A0qVL6dWrF7Vr\n1wbg+uuvZ+HChVx11VVH6Ddq1ChWrFiB0+lk0yZrZvill17KnXfeSV5eHl9//TXnnXcekZGRfPvt\nt6xatcrbN52RkcGff/5JWFgY3bp1Kza3/ZVXXuHzzz8HYMeOHfz555/s3r2b888/nxo1agAwcOBA\n7zm/++471q0r8oBz6NAhsrKyvAsOC1m0aBFJSUn8+eefjB49mnr1Ttyqo8zMTFJTU7n66qsBa3FX\nIAwaNMi7fc011zB9+nTuv/9+pk+fzvTp09m4cSNr1qzh4osvBqyZbvXr1z+inF27dtG6dWvv/po1\na3j44Yc5ePAgWVlZ9O3b15s3cOBA73jWt99+y5w5c5gwYQJgjYFt376dBg0a+P19S7Jo0aKArvNE\nIyLFZjHVqVOHnTt3nnQ9yjQUYmmYRNH6B49tmQynK3tGBSY35CzrUw6hoaHEx8eTn59P48aN/VY8\n4eHh3m2Hw+Hddzgc3gHvQKhWrVq5Mp9//jlPPPEEAG+99RZz586lbt26rFy5Eo/H49UvIiKCXr16\n8c033zB9+nSuvdaaGKY6LfQAACAASURBVKiqvPrqq8UqLLC6nnzPv2DBAr777jsWL15MVFQUvXr1\nKnfuu8fjYcmSJeVWzj179mTu3Lls3bqV7t27c80119ChQwfatGnD8uXLSUpK8souX76ctm3bEhsb\nS3R0NMnJyTRr1qzc+1SSkJAQPJ6iCSYlr8X32gcNGsTAgQPp378/IkKLFi1YvXo1bdu2ZfHixWWe\nJzIysljZN910E7NmzSIpKYl33nmHBQsW+D2nqvLZZ5/RqlWrYuU9/vjjfn/fkvTs2ZPMzMwj0idM\nmMBFF11Ups7+aNiwISkpKd79lJQUGjZsCEDdunXZtWsX9evXZ9euXdSpU+SHIjc3l8jIyKM+3/FS\n5hiFbRS+VFW3/akURiIuJz3YKhiOEbfbTUpKSrE/XePG/9/emcdFVb1//H1AE80FkyxDQRGUdQYX\nDPVnpiZaLuWSS1ppam5lmRt91bLsW2qlLW5Z5pKWpebybdHSKEsllzS3TEvMDRU3FBUUeH5/zHAF\nhmUkYGbwvF+v+3rNvffcc545c+c+92yfx4eAgAC7307zo1WrVsyaNcsoLzEx0SZNo0aN+Omnnzhz\n5gxpaWl89tlnNG/enE6dOrFz50527txJw4YNSUxMpFq1ari5ufHJJ59kkSvv3r078+bN4+eff6Zt\n27YAtGnThlmzZnH9umWdz4EDB7h8+bJN+YmJiVSuXJly5cqxf/9+YmNjAYiIiOCnn37i/PnzpKam\nsnz5cuOaqKgo3n//hszazp0786yHWrVqER0dzeTJkwEYOXIkb7zxhtEKO3z4MK+//jojRowA4MUX\nX2To0KHGVOSkpCSb2UUVKlSgevXqRr95SkoKV65cwdfXl3379pGSksKFCxdYvz73cDa1a9fG3d2d\niRMnGi2NunXrkpCQYDiKjJZmdoKCgvjrr7+M/UuXLlGtWjWuX7/O4sWLcy2zTZs2vP/++8bMoh07\ndgDk+ftm5ueffzbui8xbQZwEQLVq1ahYsSKxsbGICAsXLuThhy1SeB07dmTBggUALFiwwDgOlvsp\nNDT/F7DCxp6V2TuVUvWK3JKb4GAZP377zXagS+PcXLhwgb1793Ly5EmOHDli/Gnd3NwKdZHQu+++\nS0xMDGFhYTRo0CBLd00G1apVY9KkSbRo0QKz2UyDBg2y/CEzGDJkCAsWLMBsNrN///4sb6lRUVH8\n9NNPPPDAA9x2220A9O/fn+DgYOrXr09oaCgDBw7MsdXTtm1bUlNTCQoKIjo62ugS8/b25j//+Q+N\nGjWiadOm1KxZk0qVKgGWrqpt27ZhMpkIDg5m9uzZ+dbFoEGD2LBhA4cPHyY8PJzJkyfToUMHAgMD\n6dChA1OmTDFihg8ePJgWLVoQERFBaGgozZo1y3FA9pNPPuG9997DZDLRpEkTTp48SY0aNejWrRuh\noaF069aNevXyfmR0796dRYsW0a1bN8CyiHLZsmWMGTMGs9lMeHh4jgPLDz74IBs2bDD2J06cyL33\n3kvTpk0JDAzMtbzx48dz/fp1TCYTISEhjB8/Hsj79y0oW7dupXr16ixdupSBAwcSEnIjdk7m+Owz\nZ86kf//++Pv7U7t2bR588EEAoqOj+f777wkICGDdunVER9/o6Y+JiaFdu3b/2sabReXWSFBKlRLL\norm9QF3gb+AylvjZIiL1i8/MG9zpGy4joqM4f7g1kye3doQJtxR//PFHlj7hgnDt2jWOHDnChQsX\nAChXrhy+vr6F8qcsiWSMO6SmptKpUyeeeuopY0xAA506dWLKlCkEBAQ42pRi5dSpUzz22GN5ttYy\nyOl/q5TaLiJ2xfzJTl5jFFuA+kDHgmRclKSJwt3dJYPz3VKICKdOneLEiROkp6fj5uaGt7c3VatW\ndbgsiDMzYcIE1q1bR3JyMlFRUTYD7Lc6kyZNIj4+/pZzFEeOHOHtt992SNl5OQoFICJ/F5MtdpOG\nG25JWuvJ2UlLS+PkyZOkp6dTuXJlatSoYXTRaHInY2aOJmfq1q1rMyh9KxAREeGwsvNyFHcqpV7I\n7aSITC0Ce+yiPKWoGeSVf0JNsZOamoqbmxtubm6UKlUKX19flFJawE+jcWHychTuQHmsLQtnoqpy\no3Mbf0ebocmEiHDu3DmOHj1K1apVueeeewCoXLmygy3TaDT/lrwcRbyIvFpsltwEZUpfdLQJmkwk\nJyfzzz//GFNek5KSEBE9DqHRlBDyHaNwRty9A8C7gqPNuOVJT0/n5MmTxMfHW5b5lypF9erVnSbQ\nkUajKRzymjrUqtisuEnU7R5QRosCOpKMBVEZAn5VqlQhJCQELy8v7SScCHd3d8LDwwkNDaVDhw7G\nFGWAvXv30rJlS+rWrUtAQAATJ04k83T5b7/9loYNGxIcHEy9evWMhXnOxI4dO+jXr5+jzciTN954\nA39/f+rWrcvatWtzTJMhdhgaGsqTTz5prL1JTEykQ4cOmM1mQkJCmDdvHmARiMxY5FksFFQkylGb\nl49Zvvx5Xp6CWJrCI7u4GEzIsu3evVsuXrxoc90HH2zLkm7AgNXFZfJN82+lr525/Ntvv934/MQT\nT8hrr70mIhYhQj8/P1m7dq2IWMQP27ZtK9OnTxcRkd27d4ufn5/88ccfho0zZ84sVNsKQ/67a9eu\nNkKHRV3mzbB3714xmUySnJwshw4dEj8/P5vfOy0tTapXry5//vmniIiMHz9ePvroIxER+e9//2vI\njJ8+fVoqV64sKSkpIiLSp08fQ0I+O8UuM+6MuCmXNNulERESEhJsjgcHB1OhQtF1Ax4+fNgumfEt\nW7bQuHFj6tWrR5MmTfjzzz8ByxTdkSNHEhoaislkMiQwMktfL126lJ07dxIZGYnJZKJTp05GsJjs\n5CQNPnv2bEaNGmWkySx5vmjRIho1akR4eDgDBw40JCLKly/PiBEjMJvNbN68mVdffdVYEf30008b\nb/Zbt27FZDIRHh7OqFGjDPmG3OTM86Jx48aG5Pinn35K06ZNiYqKAiyLIKdPn24Ef5oyZQpjx441\nVju7u7szeLBtBOSkpCT69u1LWFgYJpPJkBzJLFS4bNky+vTpA9jKf9esWTNLKycgIIBTp07ZJal+\n6dIldu3aZWhX5XYPzJ8/n44dO9KyZUtatbJ0lLz55ptG3b388stGnrlJvxeUVatW0aNHD8qUKUOt\nWrXw9/c37tkMzp49y2233UadOnUAaN26tVGPSikuXbqEiJCUlMQdd9xhSKw/8sgjecqWFCoF9TCO\n2rx8zDJq2iT5/PM9OXpSTeGyb98+uXz5suzbt0+2bt1q06LIjcJqUdgrM56YmGi8LX7//ffSuXNn\nERGZOXOmdOnSxTiXIe2dXfo6LCxMfvzxRxGxvNE999xzOdqTkzT46dOnpXbt2kaatm3bys8//yz7\n9u2T9u3by7Vr10REZPDgwbJgwQIREQHk888/t8lXRKR3796yerWlvkJCQmTTpk0iIjJmzBgJCQkR\nEclVzjw7GS2K1NRU6dq1q3z77bciIjJ8+HB55513bNJ7enpKYmJijpLkOTF69OgsdXXu3Lks5YqI\nLF26VJ588kkRsZX/HjZsmHz88cciIhIbG2tIb9sjqf7DDz8Yv7NI7vfAvHnzxNvb26jjtWvXyoAB\nA4wAVu3atZOffvpJRHL+fbPz/PPP5yg5nhFsKDNDhw415M1FRJ566ikb6fH09HTx8fGRrVu3GnUS\nGhoqIiIXL16U+++/X+6++265/fbb5auvvjKuO3bsmJEuO8UuM+6MXDhzldOlbIXWNIVLUlIS58+f\nN0TtSpcu7RA77JEZT0xM5Mknn+TgwYMopQxRvnXr1jFo0CDjLSxDvhtuSF8nJiZy4cIFmjdvDliC\nxTz66KM52pKTNHhkZCR+fn7ExsYSEBDA/v37adq0KTNmzGD79u3GQqmrV68aSqDu7u506dLFyDcm\nJoYpU6Zw5coVzp07R0hIiKFY2rhxYwAee+wxI2hNbnLmmWXMM8rMCF4UFBRkyHgXFuvWrWPJkiXG\nvj3ToTPLf3fv3p1XX32Vvn37smTJEuM3sUdSPT4+3pCJh9zvAbC8pWf89t999x3fffedoUeVlJTE\nwYMHue+++3L8fatUqZLF/mnTptlXOXailGLJkiUMHz6clJQUoqKijPpZu3Yt4eHh/PDDD/z999+0\nbt2aZs2aUbFixWKVHHdJR5F+JR33dKcQsi2xrFy5kmeffZYPP/wQLy8vqlatire3NyLm/C8Gnn66\nAU8/XTiBEO2RGR8/fjwtWrRgxYoVHD58mPvvvz/ffPPTmjp69CgdOnQALOJ6gYGBuUqD9+jRgy++\n+ILAwEA6deqEUgoR4cknn+SNN96wydvDw8N4GCQnJzNkyBC2bdtGjRo1mDBhQr6S4yI5y5lnp2zZ\nsuzcuZMrV67Qpk0bZsyYwbBhwwgODs4irgdw6NAhypcvT8WKFQkJCbGRJL8ZMk9oyEtyvHHjxvz1\n118kJCSwcuVKxo0bB9gnqZ5dcjyveyC75PiLL77IwIEDs+Rnr/T78OHDiYmJsTneo0ePLAJ+YBF5\nPHr0qLGfWU48M40bNzZiXnz33XdGXIx58+YRHR2NUgp/f39q1arF/v37adSoUbFKjrtkZ3+aaAmP\nouT48eP06NGDY8eOcdtttxEUFISPj4/xYHNGEhMTjT9gRghOsLxJfvDBB4ZDOXfunM21lSpVonLl\nysYf9ZNPPqF58+bUqFHDkJMeNGhQrtLgYBGqW7VqFZ999pkRm6JVq1YsW7aM06dPG2X/888/NuVn\nPIy8vLxISkoyWgmenp5UqFCBX3/9FSDLm7u9cuYZlCtXjvfee4+3336b1NRUevXqxS+//MK6desA\nS8tj2LBhjB49GoBRo0bx+uuvGw+s9PT0HNVqW7duzYwZM4z9jLGdu+66iz/++IP09HTjDT0nlFJ0\n6tSJF154gaCgIOPt3R5J9eyS47ndA9lp06YNH3/8MUlJSYDlfj99+nSev29mpk2blqPkeHYnARbJ\n8CVLlpCSkkJcXBwHDx7MEgo2g4x7JCUlhcmTJzNo0CDAIsGfIQJ46tQp/vzzTyNeSHFKjruko2hd\n2p1Homo72owSxfXr140BVG9vb/773//y3nvvcffdd7uEyuvo0aN58cUXqVevXhZZ7/79++Pj44PJ\nZMJsNvPpp5/meP2CBQsYNWoUJpOJnTt38tJLL9mkyU0aHCxdLkFBQfzzzz/GgyA4OJjXXnuNqKgo\nTCYTrVu3zjEOtKenJwMGDCA0NJQ2bdpk0fSZO3cuAwYMIDw8nMuXLxuS4/bKmWemXr16mEwmPvvs\nM8qWLcuqVat47bXXqFu3LmFhYURERBiD8CaTiXfeeYeePXsSFBREaGgohw4dsslz3LhxnD9/ntDQ\nUMxms/GmPWnSJNq3b0+TJk1yjFSXmQzJ8cxR8OyRVA8MDCQxMdFY6JnbPZCdqKgoHnvsMRo3bkxY\nWBhdu3bl0qVLef6+BSUkJIRu3boRHBxM27ZtmTFjhvHC9dBDDxldR2+++SZBQUGYTCY6dOhAy5Yt\nAUsradOmTYSFhdGqVSsmT56Ml5dFvqg4JcdzlRl3Vu70DZclj3Wk1aCR4FvR0eaUCDZt2sSgQYMY\nNWoUjz/+eJZzhSEzrik4mfvlM1RT3333XQdb5TxMmzaNChUq0L9/f0ebUuzcd999rFq1KsdxocKW\nGXfJFsX1SnfCbS5pulNx7tw5Bg4cSNOmTdm9ezczZ87E1V4cSjpff/21sWDu559/NvrwNRYGDx6c\nZQzrViEhIYEXXnih2LTUXLJF8cmK/9K2fvFHeSopiAiLFi1ixIgRJCQkULp0aUaPHs3YsWNtBsd0\ni0KjcT2KM3CR86IlIgrMqVOn6Nmzp9GX3Lx5c2bNmqWdgUajyRWX7L9xSaOdBE9PT+Lj4/Hy8mL+\n/PnExMRoJ6HRaPLEJVsUZ89dJTExmUqVcp9jrbnB999/T/369alSpQplypRh6dKlVKtWzWYhkUaj\n0eSES76cL1+2j3XrbKfqabISHx9Pz549iYqKYsyYMcbx0NBQ7SQ0Go3duKSjEAE3Nz1OkRtpaWnM\nnDmTwMBAlixZQtmyZalbt67Lzmgq6VLZudGzZ09MJpPdkhGZ5S0KExFh2LBh+Pv7YzKZ+O2333JM\nd/XqVZo3b24IHzoja9asoW7duvj7+xsCiNn5559/aNWqFSaTifvvv59jx44BlnUL4eHhxubh4cHK\nlSsBy6rsgwcPFtv3KHYKKhLlqM3Lxyydn35VVi36PUcxrFud7du3S0REhAACSLt27SQuLq7A+eUk\nLlbclHSp7JyIj4/PIjRoD5nrqTD5+uuvpW3btpKeni6bN2+WRo0a5Zhu+vTpOQoN5kaGKF9xkZqa\nKn5+fvL3339LSkqKmEwm2bt3r026rl27yvz580VEZP369dK7d2+bNGfPnpXKlSvL5cuXRUTkxx9/\nlP79+xftF7gJClsU0OEP/pvdvHzM0n/ga/LjF7Y/8K1OhtIqIN7e3rJ8+XJJT0//V3lmvuGK6kfN\nj8wPwFmzZsngwYNFROSjjz6Sxx9/PEvav/76S6pXry4iIo8//rjMnTs33/wvXbokffr0kdDQUAkL\nC5Nly5bZlJtdAXXgwIHSqFEjGT58uPj6+sr58+eNtP7+/nLy5Ek5ffq0dO7cWRo2bCgNGzbMMXbA\n1atXjbLDw8Plhx9+EBGLmq2Hh4eYzWbZsGFDlmtOnjwpjzzyiJhMJjGZTLJx48Ys9l66dElatmwp\n9erVk9DQUFm5cqWIiCQlJclDDz0kJpNJQkJCZMmSJSJiUaUNCgqSsLAwGTFihI2NTz/9tHz66afG\nfp06deTEiRM26Ro3bmy8lORmQ1xcnNSpU0cef/xxCQ4OlsOHD8vatWslMjJS6tWrJ127dpVLly6J\niMgrr7wiDRs2lJCQEEPt9d+wadMmiYqKMvZff/11ef31123SBQcHy5EjR0TE4swqVKhgk+aDDz6Q\nxx57zNhPS0uTmjVrFnu8i9xwKUcBtAX+BP4ConM4/wKwD9gFrAd888vTy8cs6559U+SobbAcjUj/\n/v1l+PDhOQYTKgjO5ChKolT2W2+9JX379hURkT/++ENq1KghV69elbi4OENSPDvdunWTadOmGXVy\n4cKFLPZev35dEhMTRUQkISFBateuLenp6bJs2bIsb70XLlyQM2fOSJ06dYyHcGaHl0G7du2M7yEi\n0rJlS0MSO4OUlBS56667jP3cbIiLixOllGzevNk416xZM0lKShIRkUmTJskrr7wiIrlLr2dm0aJF\nOUp+d+nSxSbt0qVLpV+/fsb+woULZejQoTbpevbsadxXy5cvF8BGbrxFixbyv//9L8uxBx54QLZt\n22aTnyNwGZlxpZQ7MANoDRwDtiqlVovIvkzJdgANReSKUmowMAXobptbtrxxzb72wubw4cM8++yz\njBw50pDInjNnTpGFInVUrZdkqexffvmFZ599FrBoF/n6+nLgwAEqVsxdnuaHH35g4cKFgGX8JkP/\nKQMR4T//+Q8bNmzAzc2N48ePc+rUKcLCwhgxYgRjxoyhffv2NGvWjNTUVDw8POjXrx/t27enffv2\n+X73nDhz5gyenp752gDg6+tr6CjFxsayb98+mjZtCsC1a9cMWfWcpNcz1Hwz6NWrF7169SqQzbnx\n1ltv8cwzzzB//nzuu+8+vL29swhixsfHs3v3bhvl3gzZ7wYNCkc12ZkoyumxjYC/ROQQgFJqCfAw\nlhYEACKSWas3FuhtV87/VwM8b71l+xlcv36dqVOn8sorr3D16lXOnDnD5s2bAUpkvOqSLJVdFCxe\nvJiEhAS2b99O6dKlqVmzJsnJydSpU4fffvuNb775hnHjxtGqVSteeukltmzZwvr161m2bBnTp0/n\nhx9+yJKfPVLZ2SW/c7MBbCW/W7duzWeffZYlP3ul1xcvXsybb75pc9zf399Q4b2Z7wFwzz338OWX\nXwIWra3ly5dncYJffPEFnTp1sonPUpyy38VNUc568gaOZto/Zj2WG/2Ab3M6oZR6Wim1TSm1DUAF\nVYHytxWaoa7EL7/8Qr169YiOjubq1av06NHDuKlLOiVRKrtZs2ZGOMsDBw5w5MgR6tatm2c9tGrV\nilmzZgGWGW6JiYlZzicmJlK1alVKly5NTEyMIW1+4sQJypUrR+/evRk1ahS//fYbSUlJJCYm8tBD\nDzFt2jR+//13m/I6duzIwoULERFiY2OpVKmSjSJs5cqVSUtLMx7mudmQncjISDZu3GjIhV++fJkD\nBw7kKr2enV69euUo+Z1T+oiICA4ePEhcXBzXrl1jyZIldOzY0SbdmTNnSE9PB+CNN97gqaeeynL+\ns88+o2fPnjbXFafsd7FT0D6r/DagK/BRpv3Hgem5pO2NpUVRJr98vXzMErP7+4J23bks586dk379\n+hmzmWrXrm3M9ilKnG3Wk4hI+/btZeHChSIismvXLmnevLnUqVNHateuLRMmTMgy6Pm///1P6tev\nL4GBgRIUFCSjRo2yyf/SpUvyxBNPSEhIiJhMJlm+fLmIWPq0/fz85N5775WhQ4dmGaPIHs7SEiYW\nY7aMiKX/vVu3bhIWFiZBQUEycOBAm7JzG8zOa4zi5MmT0rFjRwkNDRWz2WyESs2op4SEBImMjJTQ\n0FDp06ePBAYGSlxcnKxZs0bCwsLEbDZLw4YNZevWrXLixAmJiIiQsLAwCQ0NzWJ/Bunp6TJkyBDx\n8/OT0NBQm/GJDJ566in5/vvv87Qhp++1fv16adiwoYSFhUlYWJisWrVKRETGjh0rfn5+0qRJE+nT\np4+8/PLLOZZ7M3z99dcSEBAgfn5+xuw5EUv424xyly5dKv7+/hIQECD9+vWT5ORkI11cXJzcc889\nNrO1Tp48KREREf/avsLCZQazgcbA2kz7LwIv5pDuAeAPoKo9+Vocxbp/W48ux5kzZ8TLy0tKly4t\n48ePlytXrhRLuc7gKDSuwfbt23OcSnorMHXqVPnoo48cbYaBywxmA1uBAKVULeA40AN4LHMCpVQ9\n4AOgrYictjfj9ev+xvf2+tSqVTwSu45i//791KpVizJlylClShUWL16Mj48PgYGBjjZNo7Ghfv36\ntGjRgrS0NKeOhlgUeHp62sRyKUkU2RiFiKQCzwBrsbQYvhCRvUqpV5VSGR2DbwLlgaVKqZ1KqdX2\n5L1j50lOn8497KOrc+XKFcaOHYvJZGLKlCnG8aioKO0kNE7NU089dcs5CYC+fftSqpRLSufZRZF+\nMxH5Bvgm27GXMn1+oGD5Qsmb22NhzZo1DBkyhLi4OMAysKbRaDSOxCVdoIjC7bztVDlX5sSJEzz/\n/PMsXboUgLCwMGbPnk2TJk0cbJlGo7nVcUlH0aZUaXy8S0687AMHDtCwYUMuXbpEuXLlmDBhAs8/\n/7zNPG2NRqNxBC7pKCLc3KnqVc7RZhQaAQEBREREcPvtt/P+++/j6+vraJM0Go3GwCVlxsslO6+M\nsT1cvHiR559/3lgIppRi9erVrF69WjuJHNAy446VGd+/fz+NGzemTJkyvPXWW7mmExFatmzJxYsX\ni8SOwmD79u2EhYXh7+/PsGHDstwrGZw/f55OnTphMplo1KgRe/bsMc5NmzaNkJAQQkND6dmzp7Ew\nUMuMO9nm5WOWHft+utlpxU5Benq6fPHFF1KtWjUBpE2bNo42KV+cYR2Flhm3j6KSGT916pRs2bJF\n/vOf/8ibb76Za7qvvvpKnn/++ZvKO0NYsbiIiIiQzZs3S3p6urRt21a++eYbmzQjR46UCRMmiIhF\nqLFly5YiInLs2DGpWbOmsYbp0UcflXnz5olIyZcZd8kWhSvOeDp06BDt2rWjW7duxMfHExkZyeTJ\nkx1t1s3xtiqa7SZo3Lgxx48fB+DTTz+ladOmREVFARaJj+nTpxsBaaZMmcLYsWONKcXu7u4MHjzY\nJs+kpCT69u1LWFgYJpOJ5cuXA1nf0JctW0afPn0A6NOnD4MGDeLee+9l9OjR1KxZM0srJyAggFOn\nTpGQkECXLl2IiIggIiKCjRs32pSdnJxslF2vXj1iYizyZ1FRURw/fpzw8HB+/vnnLNecOnWKTp06\nYTabMZvNbNq0yeb7tGrVivr16xMWFsaqVasAizxGu3btMJvNhIaG8vnnnwMQHR1NcHAwJpOJkSNH\n2thYtWpVIiIi8h0zW7x4MQ8//LCx/8gjj9CgQQNCQkKYM2eOcbx8+fKMGDECs9nM5s2b2b59O82b\nN6dBgwa0adOG+Ph4AD788EMiIiIwm8106dKFK1eu5Fl+fsTHx3Px4kUiIyNRSvHEE08YgYcys2/f\nPlq2bAlYhBoPHz5sCBqmpqZy9epVUlNTuXLlCvfccw9gkWJZt24dqamp/8pGZ8UlxyhcSffu2rVr\nvPXWW0ycOJHk5GQ8PT2ZNGkSAwYMwM3NJf20w0hLS2P9+vX069cPsHQ7ZVfqrF27NklJSVy8eJE9\ne/bY1dU0ceJEKlWqxO7du4EbWk95cezYMTZt2oS7uztpaWmsWLGCvn378uuvv+Lr68tdd93FY489\nxvDhw/m///s/jhw5Qps2bfjjjz+y5DNjxgyUUuzevZv9+/cTFRXFgQMHWL16Ne3bt89RH2rYsGE0\nb96cFStWkJaWRlJSUpbzHh4erFixgooVK3LmzBkiIyPp2LEja9as4Z577uHrr78GLHpMZ8+eZcWK\nFezfvx+lVBaHd7Ns3LiRDz74wNj/+OOPueOOO7h69SoRERF06dKFKlWqcPnyZe69917efvttrl+/\nTvPmzVm1ahV33nknn3/+OWPHjuXjjz+mc+fODBgwAIBx48Yxd+5cQ2k3g5iYGIYPH25jS7ly5Wwc\n6PHjx6levbqxX716deOlIzNms5kvv/ySZs2asWXLFv755x+OHTtGgwYNGDlyJD4+PpQtW5aoqCjj\nJcXNzQ1/f39+KOl8ygAAEUpJREFU//13rR7rNLiQpzh69CivvvoqKSkp9OrVi7fffpu77rrL0WYV\njBGOERrXMuNZcUaZcYBz585RoUIFY/+9994zxBSPHj3KwYMHqVKlCu7u7nTp0gWAP//8kz179hi/\naVpamiE4uGfPHsaNG8eFCxdISkqykfUGaNGiRY7O9N8QHR3Nc889R3h4uNHSc3d35/z586xatYq4\nuDg8PT159NFHWbRoEb17W0Svtcy4kzFixHe8/1YQgYFejjYlR86fP4+npydKKWrXrs27776Lv78/\nrVq1crRpLomWGb85Cltm3F5KlSpFeno6bm5u/Pjjj6xbt47NmzdTrlw57r//fqMOPTw8DCcrIoSE\nhBgy+Znp06cPK1euxGw2M3/+fH788UebNDfTovD29jbiX0PuMuMVK1Zk3rx5hn21atXCz8+PtWvX\nUqtWLe68804AOnfuzKZNmwxHoWXGnQxJd85GRXp6Oh9//DH+/v4sWrTIOD5w4EDtJAoBLTNuobhl\nxu2lbt26HDp0yLChcuXKlCtXjv379xMbG5vrNQkJCYajuH79Onv37gXg0qVLVKtWjevXrxt1lJ2M\nFkX2LbuTAKhWrRoVK1YkNjYWEWHhwoVZxlQyuHDhAteuXQPgo48+4r777qNixYr4+PgQGxvLlStX\nEBHWr19PUFCQcZ2WGXeizcvHLC3bjpM/d8QXZDJAkbFnzx5p1qyZIQPes2dPR5tUKDjbrCcRLTNe\n3DLj8fHx4u3tLRUqVJBKlSqJt7e3EeY0M6+++qp8+OGHIiKSnJwsbdu2lcDAQHn44YelefPmEhMT\nk8XODHbs2CHNmjUTk8kkwcHBMmfOHBERmTlzptSsWVMiIiLkmWeeMer/37B161YJCQkRPz8/GTp0\nqHGvzJo1S2bNmiUiltjaAQEBUqdOHenUqZMRGldE5KWXXpK6detKSEiI9O7d25Ag1zLjTrZ5+Zil\nZZuxcmDj0QJXYmFy+fJliY6OllKlSgkgVatWlcWLF//rQPDOgjM4Co1rcOLECXnggQccbYZD0DLj\nTsi028vhX8sz/4RFzIEDB2jTpg2HDx9GKcWgQYN4/fXX7RoM1WhKGtWqVWPAgAFcvHgxz8H4kkhJ\nlxl3SUdRStycIja0r68vHh4emM1mZs+ebQSM12huVbp16+ZoExxC3759HW1CkeKSg9kqshqUK34f\nl5qayvTp0zl79iwAZcqUYc2aNWzbtk07CY1GU2JxSUdB50CoWKZYi9yyZQuNGjXi2WefZcyYMcZx\nX1/fEh2wRKPRaFzSUSi5XmxlJSYm8swzzxAZGcmOHTvw8fHJcUqdRqPRlFRc0lFI6aJRycxShghL\nliwhMDCQGTNm4O7uzujRo9m3bx8dOnQo8vI1Go3GWXBJR7Hhx3+4fPlakZbx+++/07NnT06ePEmT\nJk347bffmDx5cpYVuZriQcuMO1ZmfPHixZhMJsLCwmjSpEmui/JEtMx4iaWg82odtXn5mKVh05fl\n+PGLBZlenCfZJY+HDx8uH374oaSlpRV6Wa6CM6yj0DLj9lFUMuMbN240Fp1988030qhRoxzTaZnx\nkisz7pKjsCKCu3vhTo+NiYlhyJAhfPDBB9x3330ATJ06tVDLcHUGzDxXJPl+OOQOu9M2btyYXbt2\nAbnLjN9///0MHTr0pmTGn332WbZt24ZSipdffpkuXbpQvnx5Q5l12bJlfPXVV8yfP58+ffrg4eHB\njh07aNq0KV9++SU7d+7E09OyticgIIBffvkFNzc3Bg0axJEjRwB45513aNq0aZayk5OTGTx4MNu2\nbaNUqVJMnTqVFi1aZJEZf//992nWrJlxzalTpxg0aJAhlzFr1qwssdWTkpJ4+OGHOX/+PNevX+e1\n117j4Ycf5vLly3Tr1o1jx46RlpbG+PHj6d69O9HR0axevZpSpUoRFRVlE5woc96RkZFZ9JIys3jx\nYp5++mlj/5FHHuHo0aMkJyfz3HPPGefKly/PwIEDWbduHTNmzKBs2bK88MILJCUl4eXlxfz586lW\nrRoffvghc+bM4dq1a/j7+/PJJ59QrlzBI1tmlhkHDJnxBx98MEu6ffv2ER0dDeQuM166dGkbmfE+\nffqQmppaIie3uOQ3SheFW2IK3PXvm9qnT59m1KhRhhrn1KlTDUehcS60zLgFR8qMz5071+bBmoGW\nGdcy405F89Kl8Sjt/q/ySE9PZ+7cuYwZM4bz589TpkwZxo0bx6hRowrJypLHzbz5FyZaZjwrjpIZ\nj4mJYe7cufzyyy85ntcy41pm3KkYcpsHFcrfVuDr4+Li6N27t/HGERUVxYwZM/D39y8sEzWFiJYZ\nvzmKQmZ8165d9O/fn2+//dZQx82OlhnXMuNOhZv8u/GJihUrcuDAAe6++26WLFnCmjVrtJNwAbTM\nuIXilhk/cuQInTt35pNPPqFOnTq52qVlxrXMuNNsXj5m+WvjFpFrNzdbYs2aNYYksIhFSvjChQs3\nlcetiLPNehLRMuPFLTPer18/8fT0FLPZLGazWRo0aJCjXVpmXMuMO83m5WOWv48fsLvCjhw5Io88\n8ogAMnHiRLuv01hwBkehcQ20zHjJlRl3ya4nVP5mp6amMnXqVIKCgli5ciXly5fnjjscMxir0dwK\nZJYZv9Xw9PTkySefdLQZRYZLDmbnJzEeGxvLoEGDjP7WLl268O677+Y4cKXRaAoPLTNeMnFNR0Hu\njuLXX3+lSZMmiAg1a9Zk+vTptGvXrhitK3mIiFPE/9BoNPlj6WUqXFzSUeTV9dSoUSPatGlDvXr1\nGDdu3L9ayamxTGU8e/YsVapU0c5Co3FyRISzZ88W+pRsl3QUbn+dgzuqQ2l3Dh48yPDhw5k6dSp1\n6tRBKcXXX3+Nm5trDr84G9WrV+fYsWMkJCQ42hSNRmMHHh4eWVagFwYu6SjUyA2kfFmLSXPe5Y03\n3iAlJQUPDw+WLVsGoJ1EIVK6dGlq1arlaDM0Go0DKdInqlKqrVLqT6XUX0qp6BzOl1FKfW49/6tS\nqqY9+f6SeABTs0ZMmDCBlJQU+vbtm+NCKo1Go9H8e1RRDHwAKKXcgQNAa+AYsBXoKSL7MqUZAphE\nZJBSqgfQSUS655Wvx+13SMoVy8rZoKAgZs+erUX8NBqNJh+UUttFpGFBri3KFkUj4C8ROSQi14Al\nQPb18g8DC6yflwGtVD4jpilXLuDhVprXX3qVnTt3aieh0Wg0RUxRtii6Am1FpL91/3HgXhF5JlOa\nPdY0x6z7f1vTnMmW19NAhtB9KLAHDYAXcCbfVLcGui5uoOviBroublBXRCrkn8wWlxjMFpE5wBwA\npdS2gjafShq6Lm6g6+IGui5uoOviBkqpbQW9tii7no4DNTLtV7ceyzGNUqoUUAk4W4Q2aTQajeYm\nKUpHsRUIUErVUkrdBvQAVmdLsxrIEEjpCvwgRdUXptFoNJoCUWRdTyKSqpR6BlgLuAMfi8hepdSr\nWFQMVwNzgU+UUn8B57A4k/yYU1Q2uyC6Lm6g6+IGui5uoOviBgWuiyIbzNZoNBpNyUAvYdZoNBpN\nnmhHodFoNJo8cVpHUVTyH66IHXXxglJqn1Jql1JqvVLK1xF2Fgf51UWmdF2UUqKUKrFTI+2pC6VU\nN+u9sVcp9Wlx21hc2PEf8VFKxSildlj/Jw85ws6iRin1sVLqtHWNWk7nlVLqPWs97VJK1bcr44KG\nxivKDcvg99+AH3Ab8DsQnC3NEGC29XMP4HNH2+3AumgBlLN+Hnwr14U1XQVgAxALNHS03Q68LwKA\nHUBl635VR9vtwLqYAwy2fg4GDjva7iKqi/uA+sCeXM4/BHwLKCAS+NWefJ21RVEk8h8uSr51ISIx\nInLFuhuLZc1KScSe+wJgIjAZSC5O44oZe+piADBDRM4DiMjpYraxuLCnLgSoaP1cCThRjPYVGyKy\nAcsM0tx4GFgoFmIBT6VUtfzydVZH4Q0czbR/zHosxzQikgokAlWKxbrixZ66yEw/LG8MJZF868La\nlK4hIl8Xp2EOwJ77og5QRym1USkVq5RqW2zWFS/21MUEoLdS6hjwDfBs8ZjmdNzs8wRwEQkPjX0o\npXoDDYHmjrbFESil3ICpQB8Hm+IslMLS/XQ/llbmBqVUmIhccKhVjqEnMF9E3lZKNcayfitURNId\nbZgr4KwtCi3/cQN76gKl1APAWKCjiKQUk23FTX51UQGLaOSPSqnDWPpgV5fQAW177otjwGoRuS4i\ncVhk/wOKyb7ixJ666Ad8ASAimwEPLIKBtxp2PU+y46yOQst/3CDfulBK1QM+wOIkSmo/NORTFyKS\nKCJeIlJTRGpiGa/pKCIFFkNzYuz5j6zE0ppAKeWFpSvqUHEaWUzYUxdHgFYASqkgLI7iVozvuxp4\nwjr7KRJIFJH4/C5yyq4nKTr5D5fDzrp4EygPLLWO5x8RkY4OM7qIsLMubgnsrIu1QJRSah+QBowS\nkRLX6razLkYAHyqlhmMZ2O5TEl8slVKfYXk58LKOx7wMlAYQkdlYxmceAv4CrgB97cq3BNaVRqPR\naAoRZ+160mg0Go2ToB2FRqPRaPJEOwqNRqPR5Il2FBqNRqPJE+0oNBqNRpMn2lFonA6lVJpSamem\nrWYeaWvmppR5k2X+aFUf/d0qeVG3AHkMUko9Yf3cRyl1T6ZzHymlggvZzq1KqXA7rnleKVXu35at\nuXXRjkLjjFwVkfBM2+FiKreXiJixiE2+ebMXi8hsEVlo3e0D3JPpXH8R2VcoVt6wcyb22fk8oB2F\npsBoR6FxCawth5+VUr9ZtyY5pAlRSm2xtkJ2KaUCrMd7Zzr+gVLKPZ/iNgD+1mtbWWMY7LZq/Zex\nHp+kbsQAect6bIJSaqRSqisWza3F1jLLWlsCDa2tDuPhbm15TC+gnZvJJOimlJqllNqmLLEnXrEe\nG4bFYcUopWKsx6KUUput9bhUKVU+n3I0tzjaUWickbKZup1WWI+dBlqLSH2gO/BeDtcNAt4VkXAs\nD+pjVrmG7kBT6/E0oFc+5XcAdiulPID5QHcRCcOiZDBYKVUF6ASEiIgJeC3zxSKyDNiG5c0/XESu\nZjq93HptBt2BJQW0sy0WmY4MxopIQ8AENFdKmUTkPSyS2i1EpIVVymMc8IC1LrcBL+RTjuYWxykl\nPDS3PFetD8vMlAamW/vk07DoFmVnMzBWKVUd+FJEDiqlWgENgK1WeZOyWJxOTixWSl0FDmORoa4L\nxInIAev5BcBQYDqWWBdzlVJfAV/Z+8VEJEEpdciqs3MQCAQ2WvO9GTtvwyLbkrmeuimlnsbyv66G\nJUDPrmzXRlqPb7SWcxuWetNockU7Co2rMBw4BZixtIRtghKJyKdKqV+BdsA3SqmBWCJ5LRCRF+0o\no1dmAUGl1B05JbJqCzXCIjLXFXgGaHkT32UJ0A3YD6wQEVGWp7bddgLbsYxPvA90VkrVAkYCESJy\nXik1H4vwXXYU8L2I9LwJezW3OLrrSeMqVALirfEDHsci/pYFpZQfcMja3bIKSxfMeqCrUqqqNc0d\nyv6Y4n8CNZVS/tb9x4GfrH36lUTkGywOzJzDtZewyJ7nxAoskcZ6YnEa3KydVkG78UCkUioQS/S2\ny0CiUuou4MFcbIkFmmZ8J6XU7UqpnFpnGo2BdhQaV2Em8KRS6ncs3TWXc0jTDdijlNqJJS7FQutM\no3HAd0qpXcD3WLpl8kVEkrGoay5VSu0G0oHZWB66X1nz+4Wc+/jnA7MzBrOz5Xse+APwFZEt1mM3\nbad17ONtLKqwv2OJj70f+BRLd1YGc4A1SqkYEUnAMiPrM2s5m7HUp0aTK1o9VqPRaDR5olsUGo1G\no8kT7Sg0Go1GkyfaUWg0Go0mT7Sj0Gg0Gk2eaEeh0Wg0mjzRjkKj0Wg0eaIdhUaj0Wjy5P8BA+Ej\nlICqknsAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f6aaed546a0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"n_classes = len(tags_counts)\n",
"roc_auc(y_test_binary, y_test_predicted_scores_tfidf, n_classes)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The ROC curve also suggest that things are good."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Some things you can do"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Pheeeew, we have now built our model and we are happy with the level of performance it shows on the comment data set. let's now do something interesting. I found this a bit interesting and even funny. it is my favorite part for NLP models. let's see what our models did actually learn. let's ask our model to tell us the most positive words and the most negative words associated with each tag. sounds fun. Yuuuhuu."
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [],
"source": [
"def print_words_for_tag(classifier, tag, tags_classes, index_to_words, all_words):\n",
" \"\"\"\n",
" classifier: trained classifier\n",
" tag: particular tag\n",
" tags_classes: a list of classes names from MultiLabelBinarizer\n",
" index_to_words: index_to_words transformation\n",
" all_words: all words in the dictionary\n",
" \n",
" return nothing, just print top 5 positive and top 5 negative words for current tag\n",
" \"\"\"\n",
" print('Tag:\\t{}'.format(tag))\n",
" \n",
" # Extract an estimator from the classifier for the given tag.\n",
" # Extract feature coefficients from the estimator. \n",
" est = classifier.estimators_[tags_classes.index(tag)]\n",
" ######################################\n",
" ######### YOUR CODE HERE #############\n",
" ######################################\n",
" \n",
" top_positive_words = [index_to_words[index] for index in est.coef_.argsort().tolist()[0][-5:]]# top-5 words sorted by the coefficiens.\n",
" top_negative_words = [index_to_words[index] for index in est.coef_.argsort().tolist()[0][:5]]# bottom-5 words sorted by the coefficients.\n",
" print('Top positive words:\\t{}'.format(', '.join(top_positive_words)))\n",
" print('Top negative words:\\t{}\\n'.format(', '.join(top_negative_words)))"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {},
"outputs": [],
"source": [
"DICT_SIZE = len(words_counts)\n",
"WORDS_TO_INDEX = {word[0]:i for i,word in enumerate(sorted(words_counts.items(), key=lambda x: x[1], reverse=True)[:DICT_SIZE])}\n",
"INDEX_TO_WORDS = {WORDS_TO_INDEX[i]:i for i in WORDS_TO_INDEX}####### YOUR CODE HERE #######\n",
"####### YOUR CODE HERE #######\n",
"ALL_WORDS = WORDS_TO_INDEX.keys()"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Tag:\tsafe\n",
"Top positive words:\tagree, best, thank, redirect, thanks\n",
"Top negative words:\tfuck, fucking, shit, idiot, stupid\n",
"\n",
"Tag:\tidentity_hate\n",
"Top positive words:\tjew, nigga, gay, homosexual, nigger\n",
"Top negative words:\tarticle, talk, better, source, see\n",
"\n",
"Tag:\tinsult\n",
"Top positive words:\tidiots, bitch, stupid, asshole, idiot\n",
"Top negative words:\tredirect, thanks, talk, could, article\n",
"\n",
"Tag:\tobscene\n",
"Top positive words:\tbitch, shit, asshole, fucking, fuck\n",
"Top negative words:\tthanks, manners, article, section, cheers\n",
"\n",
"Tag:\tsevere_toxic\n",
"Top positive words:\tfuckin, motherfucker, fcking, fucking, fuck\n",
"Top negative words:\tarticle, please, talk, ass fuck, redirect\n",
"\n",
"Tag:\tthreat\n",
"Top positive words:\tdestroy, rape, death, die, kill\n",
"Top negative words:\tarticle, please, thanks, dont u, isnt\n",
"\n",
"Tag:\ttoxic\n",
"Top positive words:\tstupid, idiot, shit, fucking, fuck\n",
"Top negative words:\tthanks, thank, redirect, best, may\n",
"\n"
]
}
],
"source": [
"print_words_for_tag(classifier_tfidf, 'safe', mlb.classes, tfidf_reversed_vocab, ALL_WORDS)\n",
"print_words_for_tag(classifier_tfidf, 'identity_hate', mlb.classes, tfidf_reversed_vocab, ALL_WORDS)\n",
"print_words_for_tag(classifier_tfidf, 'insult', mlb.classes, tfidf_reversed_vocab, ALL_WORDS)\n",
"print_words_for_tag(classifier_tfidf, 'obscene', mlb.classes, tfidf_reversed_vocab, ALL_WORDS)\n",
"print_words_for_tag(classifier_tfidf, 'severe_toxic', mlb.classes, tfidf_reversed_vocab, ALL_WORDS)\n",
"print_words_for_tag(classifier_tfidf, 'threat', mlb.classes, tfidf_reversed_vocab, ALL_WORDS)\n",
"print_words_for_tag(classifier_tfidf, 'toxic', mlb.classes, tfidf_reversed_vocab, ALL_WORDS)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Impressive , Hein!!!** our model can learn words that are most positive for each category. a few take on though. first some classes are ambigious and negative words are sometimes words that have nothing to do with the tag. for instance if a comment has a word article then the model might this is not offensive even when it is. depending on other words that make the comment,then the model might classify as a toxic comment. it is good to see what the model learn since that way you can improve your model by providing more examples."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Save the model to the disk for later use"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {},
"outputs": [],
"source": [
"import pickle"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {},
"outputs": [],
"source": [
"# save the model to disk\n",
"filename = 'classify_wiki_comments.sav'\n",
"pickle.dump(classifier_tfidf, open(filename, 'wb'))"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {},
"outputs": [],
"source": [
"# in case you would like to reload the model from the disk. just do\n",
"loaded_model = pickle.load(open(filename, 'rb'))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment