Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save lukemerrick/e31d9a8ec1b42767ad09155eb2f9b231 to your computer and use it in GitHub Desktop.
Save lukemerrick/e31d9a8ec1b42767ad09155eb2f9b231 to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import pathlib\n",
"\n",
"import category_encoders\n",
"import lightgbm as lgb\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import pandas as pd\n",
"import seaborn as sns\n",
"import sklearn.impute\n",
"import sklearn.linear_model\n",
"import sklearn.metrics\n",
"import sklearn.model_selection\n",
"import sklearn.pipeline\n",
"import tqdm"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"%load_ext autoreload\n",
"%autoreload 2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Ames Housing Data\n",
"Used as the example in the popular Kaggle [House Prices: Advanced Regression Techniques](https://www.kaggle.com/c/house-prices-advanced-regression-techniques) playground competition, the Ames housing data serves as a beefed-up version of the classic [Boston Housing dataset](https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html) with many more features and examples. We will use this dataset to learn a system that estimates the sale price of a home.\n",
"\n",
"Note: we'll do a random 80%-20% split of the data, rather than using the pre-split version from Kaggle."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"ames_housing_dataset_url = 'http://jse.amstat.org/v19n3/decock/AmesHousing.txt'\n",
"target_column = 'SalePrice'\n",
"\n",
"# download the dataset\n",
"df = pd.read_csv(ames_housing_dataset_url, sep='\\t')\n",
"\n",
"# drop where target is missing\n",
"df = df[df[target_column].notna()]\n",
"\n",
"# convert categorical datatype in Pandas\n",
"for col in df.select_dtypes('object'):\n",
" df[col] = df[col].astype('category')\n",
"\n",
"# split x/y and train/test\n",
"df_x = df.drop(columns=[target_column])\n",
"df_y = df[target_column]\n",
"x_train, x_test, y_train, y_test = sklearn.model_selection.train_test_split(df_x, df_y, test_size=0.2, shuffle=True, random_state=0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gradient Boosting (GBM) + Regularized Linear Model (LASSO)\n",
"Here we construct a complex model by averaging the predictions of two base models:\n",
"\n",
"1. a linear model characterized by its use of one-hot encoding, mean imputation, standard scaling, and LASSO (l1) regularization (with regularizatoin tuned by cross-validation).\n",
"2. a gradient boosted machine using the default configuration provided by the powerful `lightgbm` library.\n",
"\n",
"This ensemble happens to do pretty well at Mean Absolute Error (MAE), so it's arguably better to use this ensemble than either of the base models, when it comes to estimating the median price (see [Wikipedia](https://en.wikipedia.org/wiki/Mean_absolute_error#Optimality_property) for info on the connection between MAE and median prediction). With a modicum of feature engineering and hyperparameter tuning, I'm sure this ensemble could be beaten (possibly by either of the base models), but for the sake of example we'll stick with it ;)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"r2_score (train):\n",
"lm_f = 0.94, gbm_f = 0.98, ensemble_f = 0.97, \n",
"\n",
"mean_absolute_error (train):\n",
"lm_f = 13582.86, gbm_f = 6171.74, ensemble_f = 9150.23, \n",
"\n",
"r2_score (test):\n",
"lm_f = 0.81, gbm_f = 0.89, ensemble_f = 0.86, \n",
"\n",
"mean_absolute_error (test):\n",
"lm_f = 16021.18, gbm_f = 14592.74, ensemble_f = 13910.63, \n",
"\n"
]
}
],
"source": [
"categorical_columns = x_train.select_dtypes('category').columns.tolist()\n",
"linear_model = sklearn.pipeline.make_pipeline(\n",
" category_encoders.one_hot.OneHotEncoder(\n",
" cols=categorical_columns, handle_unknown='drop', handle_missing='drop'),\n",
" sklearn.impute.SimpleImputer(),\n",
" sklearn.preprocessing.StandardScaler(),\n",
" sklearn.linear_model.LassoCV(cv=10)\n",
")\n",
"gbm_model = lgb.LGBMRegressor()\n",
"linear_model.fit(x_train, y_train)\n",
"gbm_model.fit(x_train, y_train)\n",
"\n",
"# define functions that map input to output using our models -- in Pandas!\n",
"def lm_f(x):\n",
" return linear_model.predict(x)\n",
"\n",
"def gbm_f(x):\n",
" return gbm_model.predict(x)\n",
"\n",
"def ensemble_f(x):\n",
" return (lm_f(x) + gbm_f(x))/2\n",
"\n",
"# scoring\n",
"for split_name, x, y in (('train', x_train, y_train), ('test', x_test, y_test)):\n",
" for loss_fn in (sklearn.metrics.r2_score, sklearn.metrics.mean_absolute_error):\n",
" print(f'{loss_fn.__name__} ({split_name}):')\n",
" for model_fn in (lm_f, gbm_f, ensemble_f):\n",
" print(f'{model_fn.__name__} = {loss_fn(y, model_fn(x)):.2f}', end=', ')\n",
" print()\n",
" print()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Feature Importance\n",
"We now have a nice model that works well ($r^2$ = 0.86, MAE < \\$14,000). Let's measure the feature importance!"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# here is a quick and dirty implementation of permutation-based \n",
"# random ablation feature importance\n",
"\n",
"def permutation_predictions(model_fn, x, seed=0):\n",
" \"\"\"Generates a DataFrame where each column is the predictions resulting\n",
" from permutation-based ablation of the corresponding feature.\"\"\"\n",
" shuffled_order = np.random.RandomState(seed).permutation(x.shape[0])\n",
" res = dict()\n",
" for column_name, column in x.iteritems():\n",
" permuted_x = x.copy()\n",
" permuted_x[column_name] = column.values[shuffled_order]\n",
" res[column_name] = model_fn(permuted_x)\n",
" return pd.DataFrame(res)\n",
"\n",
"\n",
"def ablation_importance(model_fn, x, y, n_repetitions=30, \n",
" loss_fn=sklearn.metrics.mean_absolute_error):\n",
" \"\"\"Reference implementation of ablation importance.\"\"\"\n",
" original_predictions = model_fn(x)\n",
" original_loss = loss_fn(y, original_predictions)\n",
" results = pd.DataFrame.from_dict({\n",
" seed: {\n",
" permuted_column_name: (loss_fn(\n",
" y, y_hat_permuted) - original_loss)\n",
" for permuted_column_name, y_hat_permuted\n",
" in permutation_predictions(model_fn, x, seed).iteritems()\n",
" }\n",
" for seed in tqdm.tqdm(range(n_repetitions))\n",
" }, orient='index')\n",
" results.index.name = 'seed'\n",
" return results"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def plot_importances(n_points, n_cycles, figname, n_features=10):\n",
" sampled_index = x_test.sample(n_points, random_state=1).index\n",
" res = ablation_importance(\n",
" model_fn=ensemble_f, \n",
" x=x_test.loc[sampled_index], \n",
" y=y_test.loc[sampled_index],\n",
" n_repetitions=n_cycles,\n",
" )\n",
" mean_imp = res.mean()\n",
" \n",
" # 95% CI based upon z-distribution approximation \n",
" # of t-distribution using MLE estimate of variance\n",
" imp_ci = 1.96 * res.std(ddof=0) / np.sqrt(n_cycles)\n",
"\n",
" imp = pd.DataFrame(\n",
" dict(\n",
" feature_importance=mean_imp, \n",
" ci_fixed=imp_ci\n",
" ), \n",
" ).sort_values(by='feature_importance', ascending=False)[:n_features]\n",
" x = np.arange(len(imp))\n",
" plt.barh(x[::-1], imp['feature_importance'], tick_label=imp.index, xerr=imp['ci_fixed'])\n",
" plt.title('Feature Importance')\n",
" plt.savefig(figname, dpi=300, bbox_inches='tight')\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 3/3 [01:17<00:00, 26.03s/it]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEICAYAAAA+16EyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de5xVVf3/8ddbREARvEAyYjpamKkoKuY1wyIt7/rVVOwrpEWWVtZXja/5TbI0ki54KyN/3lL8oqh4obykTXlDHAgFFbxiylcEbyh4Q/j8/thr5DDO7cw5Z85seD8fj3nMPmvvvdbn7GHmw1p7nb0UEZiZmeXNWtUOwMzMrD2cwMzMLJecwMzMLJecwMzMLJecwMzMLJecwMzMLJecwMzMLJecwGyNJ2mepHclLSn42rTEOodIeqlcMbaxzSsl/aIj22yOpNGSrql2HLZ6cwIzyxwcET0Lvv6vmsFIWrua7Zciz7FbvjiBmbVA0u6SHpT0pqRHJQ0p2PcNSU9KelvSc5K+ncrXA/4KbFrYo2vcQ2rcS0s9wR9LegxYKmntdN6NkhZJel7S99sYd62kSDG+KOkNSSdJ2lXSY+n9XFxw/AhJD0i6WNJiSXMkfalg/6aSbpX0uqRnJH2rYN9oSZMkXSPpLeAk4Ezg6PTeH23pehVeC0n/JWmhpJclfaNgfw9Jv5H0Qorvfkk9WvsZ2erN/1Mya4ak/sAU4D+BO4AvATdK2iYiFgELgYOA54B9gL9KeiQiZkj6KnBNRGxWUF9bmj0WOBB4FVgB3Abckso3A/4maW5E3NnGt7EbMCDFd2t6H0OBrsC/JN0QEf8oOHYS0Ac4ArhJ0pYR8Trwv8BsYFNgG+BuSc9GxL3p3EOBo4DjgW6pjk9HxNcLYmn2eqX9/YDeQH/gy8AkSZMj4g3g18B2wJ7AghTrijb8jGw15h6YWWZy+h/8m5Imp7KvA3+JiL9ExIqIuBuoBw4AiIgpEfFsZP4B3AV8vsQ4LoyIFyPiXWBXoG9EnBMRH0TEc8CfgGOKqO/nEfFeRNwFLAWui4iFETEfuA/YqeDYhcC4iFgWEROBucCBkj4J7AX8ONU1E7iMLFk1eCgiJqfr9G5TgbThei0Dzknt/wVYAnxG0lrACcAPImJ+RCyPiAcj4n1a+RnZ6s09MLPMYRHxt0ZlWwBHSTq4oKwr8HeA1Ms6G9ia7D+D6wKzSozjxUbtbyrpzYKyLmSJp61eKdh+t4nXPQtez49Vn+79AlmPa1Pg9Yh4u9G+wc3E3aQ2XK/XIuLDgtfvpPj6AN2BZ5uotsWfka3enMDMmvci8OeI+FbjHZK6ATeS9UJuiYhlqefWME7Y1DIPS8n+aDfo18Qxhee9CDwfEQPaE3w79JekgiS2Odmw4/8BG0lavyCJbQ7MLzi38ftd5XUbrldLXgXeAz4FPNpoX7M/I1v9eQjRrHnXAAdL2l9SF0nd02SDzYB1yO71LAI+TL2L/QrOfQXYWFLvgrKZwAGSNpLUDzi1lfanAW+niR09UgzbS9q1bO9wVZ8Avi+pq6SjgM+SDc+9CDwI/DJdgx2AE8muT3NeAWrT8B+0fr2aFRErgMuB36bJJF0k7ZGSYks/I1vNOYGZNSP94T6UbEbdIrL/7Z8OrJV6It8HrgfeAIaR9VYazp0DXAc8l+6rbQr8mawHMY/s/s/EVtpfTjbpYRDwPFlP5DKyiQ6V8DDZhI9XgXOBIyPitbTvWKCWrDd2M3B2E0OuhW5I31+TNKO169UGp5ENNz4CvA78iuzn0OzPqIi6LafkBS3NTNII4JsRsXe1YzFrK/8vxczMcskJzMzMcslDiGZmlkvugZmZWS75c2AV0KdPn6itra12GGZmuTF9+vRXI6JvMec4gVVAbW0t9fX11Q7DzCw3JL1Q7DkeQjQzs1xyAjMzs1xyAjMzs1xyAjMzs1xyAjMzs1xyAjMzs1xyAjMzs1xyAjMzs1zyB5krYNb8xdSOmlLtMMxsDTdvzIHVDqGi3AMzM7NccgIzM7NccgIzM7NccgIzM7NcqmgCk7SJpAmSnpM0XdJDkg5v47lLmig7SdLxRcawtqRFksYUc56ZmXVuFUtgkgRMBv4ZEVtFxC7AMcBmTRzbptmQEXFpRFxdZChfBp4CjkoxNRVrlyLrNDOzKqvkNPovAh9ExKUNBRHxAnARgKQRwBFAT6AL8IXWKpQ0GlgC3A5cHRGfS+W1wG0RMbCJ044FLgC+A+wBPJjOmQdMJEtw50t6BLgE6Au8A3wrIuZIOhg4C1gHeA04LiJeafNVMDNLFkwY1aHtDZk6tsPaqqur67C2GlQygW0HzGjlmJ2BHSLi9WIqTollHUlbRsTzwNFkyWgVkroDQ4FvAxuQJbMHCw55LSJ2TsfeA5wUEU9L2g34PVkSvh/YPSJC0jeBM4D/aqKtkcBIgC69ilpU1MzM2qHDPsgs6RJgb7Je2a6p+O5ik1eB68kS15j0/egmjjkI+HtEvCvpRuB/JJ0aEcvT/okptp7AnsANBaOM3dL3zYCJkmrIemHPNxVMRIwHxgN0qxkQ7XxPZrYa6zesY2/F1/mDzO32OFkPC4CIOBn4EtkQXYOlJdQ/EfiapK2z6uPpJo45FhiahgunAxuT9aoat78W8GZEDCr4+mzadxFwcRqe/DbQvYSYzcysTCqZwO4Fukv6TkHZuuWqPCKeBZYD/0PTw4e9gM8Dm0dEbUTUAieTJbXGdb0FPC/pqHSuJO2YdvcG5qft4eWK38zMSlOxBBYRARwGfEHS85KmAVcBP25jFetKeqng60dNHDMR+DrZcGJjhwP3RsT7BWW3AAdL6tbE8ccBJ0p6lKz3eGgqH002tDgdeLWNsZuZWYUpyzNWTt1qBkTN8HHVDsPM1nB5epivpOkRMbiYc/wkDjMzyyUnMDMzyyWvB1YBA/v3pj5HXXczszxyD8zMzHLJCczMzHLJCczMzHLJ98AqYNb8xdSOmlLtMMyskTxNK7fWuQdmZma55ARmZma55ARmZma55ARmZma5VJYEJmkzSbdIelrSs5IukLROOepupd0l6XutpNnNHLOdpHslzU2x/UxSu9+3pHmS+rT3fDMzK4+SE5iyFSBvAiZHxABga6AncG4Z6i5plqSkHsCtwJiI+AwwEPgc8INSYzMzs+oqxzT6LwLvRcQVABGxXNIPydbXOptsXbATI+JxAEl1wGnAk2SLRW4PdAVGR8QtkkYAR5AlwS6SDiRbBmXDdNxZEXFLG2MbBjwQEXel2N6RdApwH/A7SaOBJRHx6xTbbOCgiJgnaTLwSbIFLC9IKy6bWY4smDBqlddDpo5d5XVdXV0HRmPlVo4Eth3ZascfiYi3JP0b+DRp5WTgbEk1QE1E1Es6j2y9rhMkbQBMk/S3VMXOwA4R8XrqhR2e6uwDTJV0a7RtHZimYntWUo/UZktOSO33AB6RdGNEvNbcwZJGAiMBuvTq29xhZmZWJh3xQebrgbuAs8kS2aRUvh9wiKTT0uvuwOZp++6IeD1tCzhP0j7ACqA/sAmwoMJxf1/S4Wn7k8AAoNkElnpo4yFbD6zCsZlZG/QbNmaV13X+IPNqpRwJ7AngyMICSb3IktEzadjuNUk7AEcDJzUcBvxHRMxtdO5uwNKCouOAvsAuEbFM0jyyZNfW2PZpVP9WwGsR8aakD1n1PmD3dMwQYCiwR4q/rog2zcysA5RjFuI9wLqSjgeQ1AX4DXBlRLyTjpkInAH0jojHUtmdwPfSJBAk7dRM/b2BhSl57QtsUURs1wJ7Sxqa2ugBXEjWGwSYRzZciaSdgS0L2nwjJa9tgN2LaNPMzDpAyQks3Ys6HDhK0tPAU8B7wJkFh00CjiEbTmzwc7JJGY9Jejy9bsq1wGBJs4DjgTlFxPYucAjwE0lPAa+STeq4Nh1yI7BRav+UFDvAHcDakp4ExgBT29qmmZl1DLVtLsTqQdJhwG+BfSPihUq1061mQNQMH1ep6s2snfww385L0vSIGFzMOWvUkzgiYnJEbFXJ5GVmZh1jjUpgZma2+nACMzOzXPKClhUwsH9v6j3WbmZWUe6BmZlZLjmBmZlZLjmBmZlZLvkeWAXMmr+Y2lFTqh1GVfnzNmZWae6BmZlZLjmBmZlZLjmBmZlZLrXpHpikjcmeOg/QD1gOLEqvPxcRHzQ6fiPgaxFxaSv1rg28GhEbNFH+PjArFS0HTo6Ikh6qm544/4mIuKOJfT2By8gWwRTwBrA/8EGjWAAOjogXS4nFzMxK06YEllYiHgQgaTSwJCJ+3cIpG5Gt+9ViAmvF2xHR0OaBwLnAl0qoD7KlU7Yne9p8Yz8E/h0Rx6Q2twGWNY7FzMw6h5KHECWdIWl2+vpeKh4DfEbSTEljJPWSdK+kGZIek3RQkc30IusRIam/pPtT3bMl7SlpbUlvSvqtpMcl3SlpN0n/kPScpAPSWmA/BY5L5x7ZqI0aYH7Di4iYExHLMDOzTqmkafRp9eTjgF1TXdPS6sWjgE8X9KC6AodFxFuSPgE8ANzeSvXrS5pJthJyP2DfVP514LaI+FVaPLNHKu8N/DUifiTpNmA0WY9tR+CPETFY0jnA9hFxahPt/T/gDklHkw2XXhURzzSKBbJVphsnvzXaggmjPlY2ZOrYj5XV1dV1QDRmtqYo9XNgewM3poUjkTQZ+DxwV6PjBIyRtDewAvikpD7Amy3UXTiEuDdwNTAQeAT4o6TuwOSIeDTdM3s3Iu5O584CFkfEh2khzNrW3khETJe0FbAfMBSol/Q54DnaMIQoaSQwEqBLr76tNWdmZiXqqA8yH0/WQ9o5JZWXyHpWbRIR90vaVNJGEXGvpCHAgcDVks4HJpJNtmiwgmziRcN2W+/1vU22SvONkgR8FbikjeeOB8ZDtqBlW85ZXfQbNuZjZXX+ILOZVVip98DuAw6X1CPN4js0lb0NrF9wXG9gYUpeXwb6F9OIpO3IEtEbkrYAFqSEcQWwUxFVNY6rsI29JW2QtrsBnwW88KWZWSdVUg8sIqZJuo5sWA/gDxExC7LlodPw3RTgt8Bt6fU04Ok2VF943wng+IgISV8CfiRpGVlC+s8iQr4XOF3Sv4BzI2JSwb4BwB+yjhdrAbcBtwBdiqjfzMw6iCLWqNGuDtGtZkDUDB9X7TCqys9CNLNiSJoeEYOLOcdP4jAzs1xyAjMzs1xyAjMzs1zyemAVMLB/b+p9D8jMrKLcAzMzs1xyAjMzs1xyAjMzs1zyPbAKmDV/MbWjplQ7jA7nz36ZWUdyD8zMzHLJCczMzHLJCczMzHLJCczMzHKp1QQmKST9puD1aZJGt3LOIZI+vkzvqscMkdTkqsyS5qUFL9tF0mhJp7X3/I6u18zMiteWHtj7wBHFJJSIuDUiPr7KYQdIqzObmdlqri1/7D8kW2n4h8BPCndI6gtcCmyeik6NiAckjQAGR8Qpkj4FXAusR7a+1qkR0TMd31PSJGB7YDrw9Vi5vssZkr4KvAsMi4hnJNUClwN9gEXANyLi35KuBN4jW9zyAeAtYFtJdSm2cRFxYYr5R8AJqY3LImJcK+U/AYYDC4EXU5y5t2BCix3kdhkydWzZ66yrqyt7nWa2emjrPbBLgOMk9W5UfgHwu4jYFfgP4LImzr0AuCAiBgIvNdq3E3AqsC2wFbBXwb7F6ZyLgYbFtS4CroqIHciS4oUFx28G7BkRP0qvtwH2Bz4HnC2pq6RdgG8AuwG7A9+StFMr5ccAg4ADgF2bu0CSRkqql1S//J3FzR1mZmZl0qbhtoh4S9LVwPfJekQNhpL1dBpe95LUs9HpewCHpe0JwK8L9k2LiJcA0urLtcD9ad91Bd9/V1DXEWn7z8D5BXXdEBHLC15PiYj3gfclLQQ2AfYGbo6IpanNm4DPA2qmfK1U/k4qv7XJCwRExHiynirdagZ0+lVC+w0r/whvnT/IbGYdqJj7ReOAGcAVBWVrAbtHxHuFBxYktNa8X7C9vFE80cx2c5YWUbeZmeVcm6fRR8TrwPXAiQXFdwHfa3ghaVATp04lG16EbDiurY4u+P5Q2n6woI7jgPuKqI90/GGS1pW0HnB4Kmuu/J+pvIek9YGDi2zPzMwqpNheyW+AUwpefx+4RNJjqa5/Aic1OudU4Jo0GeIOoK03iDZM9b4PHJvKvgdcIel00iSOYoKPiBlpwse0VHRZRPwLoIXyicCjZJM4HimmPTMzqxytnPRXoQakdYF3IyIkHQMcGxGHVrTRKutWMyBqho9r/cDVjB/ma2btJWl6RAwu5pyOuC+0C3Cxshtjb7JyqrqZmVm7VTyBRcR9wI6VbsfMzNYsnplXAQP796bew2lmZhXlh/mamVkuOYGZmVkuOYGZmVku+R5YBcyav5jaUVOqHYantZvZas09MDMzyyUnMDMzyyUnMDMzyyUnMDMzy6V2JTBJyyXNlPSopBmS9iw1EEmDJB1Q8HqEpEWpnZlpPTIknSNpaBH1rivpWkmzJM2WdH/DmmUF76Phq7aJ8++Q9Kak20t9j2ZmVj7tnYX4bkQMApC0P/BL4AslxjIIGAz8paBsYkQUPv2eiPhpkfX+AHglre6MpM8Ay9K+j95HC8YC6wLfLrJdMzOroHJMo+8FvAEgqQaYmMrWBr4TEfdJWgL8ATgAeBk4k2w15c3Jllu5AzgH6CFpb7KE2KS07MntETFJ0jzgKrJ1uroCR0XEnEan1AAvNLyIiLnFvLmIuEfSkGLOqYYFE0Z9rGzI1LGrvK6rq+ugaMzMKq+998B6pCG3OcBlwM9T+TDgztSr2RGYmcrXA+6NiO2At4FfAF8mWzjynIj4APgpWY9rUERMTOcdXTC819zaX69GxM5kCfK0JvZfDvxY0kOSfiFpQBPvY6akm9tzIRpIGimpXlL98nfauuSZmZm1VzmGEPcArpa0PdmCj5dL6gpMjoiGBPYBWS8LYBbwfkQskzQLqG2hnY8NITbhpvR9OnBE450RMVPSVsB+wFDgEUl7RMSTtG0IsU0iYjwwHrL1wMpRZzH6DRvzsbI6f5DZzFZjJc9CjIiHgD5A34j4J7APMB+4UtLx6bBlsXLlzBVkqywTESsofRjz/fR9eXN1RcSSiLgpIr4LXEM2lGlmZjlWcgKTtA3QBXhN0hZkEyb+RDa0uHMRVb0NrF9qPE3Et5ekDdP2OsC2FNwTMzOzfGpv76eHpIbhQQHDI2J5muxwuqRlwBLg+OYqaMLfgVGp3mYncbTDp4A/pBWh1wKmADe29WRJ9wHbAD0lvQScGBF3ljE+MzNrB60c2bNy6VYzIGqGj6t2GH6Yr5nlhqTpETG4mHP8JA4zM8slJzAzM8slrwdWAQP796bew3dmZhXlHpiZmeWSE5iZmeWSE5iZmeWS74FVwKz5i6kdNaVD2vJUeTNbU7kHZmZmueQEZmZmueQEZmZmueQEZmZmuVSVBKbM/ZK+WlB2lKQ7WjqvHe38QtL8tGDlk5IukdTie5a0m6Tfpe1vShqXto9IT943M7NOoCoJLK0NdhLwW0ndJfUEzgNOLqVeSU3NqhybFq3cHtgF2KuV2B6OiB82sesIsqfSm5lZJ1C1afQRMVvSbcCPgfWAqyPiWUnDyRLZOsCDwCkRsULSeLL1xXqQrdR8DkBa4uQaYH+yJHhDM02uA3QD3kzn3Z/qnimpH3B/RHxa0tBUfljDiZI+T7YI5l6SRgOHRcS8Ml6OJi2YMKrVY4ZMHdvqMXV1dWWIxsysc6n258B+BswAPgAGS9oeOBzYMyI+TEnrGGACMCoiXk+9rL9LmhQRT6R6FkbETs20cbqkEcAWwG0RMavYICPiPkl/ASZFxOSmjpE0EhgJ0KVX32KbMDOzIlU1gUXEUkkTgSUR8X7q/ewK1GfrT9IDeDEdfqykE8li3pRsZeWGBDaxhWbGRsS4tBrzzZKOjIhJFXgv44HxkK0HVo46+w0b0+oxdf4gs5mtoardAwNYkb4gW9358oj4n8IDJA0AfgB8LiLelHQN0L3gkKWtNRIRH6RJIvsAk4APWXkPsHuzJ5qZWafU2abR/w34mqQ+AJI2lrQ50At4G3hLUg3Z/a6iKOvS7Qk8m4rmkU3qADiyDVW8DaxfbLtmZlYZnSqBpftTPwP+Jukx4C5gE7L7ZE8Ac4CrgQeKqPZ0STOB2cBy4I+pfCzwA0kzgA3bUM91wJlpSn5tEe2bmVkFKJvRbuXUrWZA1Awf1yFt+WG+ZrY6kDQ9IgYXc06n6oGZmZm1lROYmZnlUmeYhbjaGdi/N/Ue2jMzqyj3wMzMLJecwMzMLJecwMzMLJd8D6wCZs1fTO2oKRVtw9PnzWxN5x6YmZnlkhOYmZnlkhOYmZnlkhOYmZnlUrsTmKTNJN0i6WlJz0q6IK25VVGSlqTvtZJmN3PM1pL+kmKbIel6SZuUoe0Rki4utR4zMytduxJYWprkJmByRAwAtgZ6AueWGlBacbmU87sDU4A/RMSAiNgZ+D3gZZLNzFYj7e2BfRF4LyKuAIiI5cAPgRMkrStpqqTtGg6WVCdpsKT1JF0uaZqkf0k6NO0fIelWSfcC90jqKeme1Hua1XBcGw0DHoqI2xoKIqIuImZL6i7pilTnvyTtW9D+TZLuSL228wti/4akpyRNA/Zq5/UqqwUTRjFkyBCGDBlS7VDMzKqmvb2d7YDphQUR8ZakfwOfBiYCXwPOTgtQ1kREvaTzgHsj4gRJGwDTJP0tVbEzsENEvJ56YYenOvsAUyXdGm1b+2X7xrEVODkLNQZK2ga4S9LWad8gYCfgfWCupIvIVm3+GdnCl4uBvwP/aqpiSSOBkQBdermzZ2ZWaZX6IPP1ZItRnk2WyCal8v2AQySdll53BzZP23dHxOtpW8B5kvYBVgD9yRa2XFBiXHsDFwFExBxJL5ANfwLcExGLASQ9AWwB9AHqImJRKp9YcPwqImI8MB6y9cBKjLNF/YaNoc4fZDazNVx7E9gTwJGFBZJ6kSWjZyLiHUmvSdoBOBo4qeEw4D8iYm6jc3cDlhYUHUd2z2qXiFgmaR5ZsmuLx4EvFPl+IOt5NViOn1JiZtaptfce2D3AupKOB5DUBfgNcGVEvJOOmQicAfSOiMdS2Z3A99IkECTt1Ez9vYGFKXntS9YbaqsJwJ6SPuqiSNpH0vbAfWTJkTR0uDkwt8laMg8DX5C0saSuwFFFxGFmZhXUrgSW7kUdDhwl6WngKeA94MyCwyYBx5ANJzb4OdAVeEzS4+l1U64FBkuaBRwPzCkitneBg8gS5dNpOPC7wCKy2YhrpXonAiMi4v0W6noZGA08BDwAPNnWOMzMrLLUtnkRVoxuNQOiZvi4irbhh/ma2epE0vSIGFzMOX4Sh5mZ5ZITmJmZ5ZJn2lXAwP69qfcQn5lZRbkHZmZmueQEZmZmueQEZmZmueR7YBUwa/5iakdNqWgbnkZvZms698DMzCyXnMDMzCyXnMDMzCyXnMDMzCyXSkpgkpYUbB+QVi7eQtJJBU+qHyFp01bqGSHp4lJiaaHuvdMK0HMkzZX03RLrW9L6UWZmVmllmYUo6UvAhcD+EfECcGnB7hHAbOD/ytFWkXH1I1te5bCImJFWd75T0ssRcXNHx2NmZuVT8hBiWjX5T8BBEfFsKhst6TRJRwKDgWslzZTUQ9Kukh6U9GjqGa2fqtpU0h1pCZTzC+rfT9JDkmZIukFSz1Q+T9LPUvksSds0Ed7JZGuUzQCIiFfJ1ig7PdVxZYqxoa0l6XtPSfcU1H1oqdfJzMzKq9QeWDdgMjAkIj62ZldETJJ0CnBaRNRLWodsHa6jI+KRtIrzu+nwQcBOZCsjz5V0Udp3FjA0IpZK+jHwI+CcdM6rEbFzGhY8DfhmoxC2A65qVFYPbNvK+3oPODwi3kq9tqmSbo0qrz2zYMKoj7aHTB370XZdXV0VojEzq65SE9gy4EHgROAHbTj+M8DLEfEIQES8BZAWaL4nIhan10+QrcK8AVmyeSAdsw7Z4pINbkrfpwNHlPheCgk4L/UuVwD9gU2ABc2eII0ERgJ06dW3jKGYmVlTSk1gK4CvAfdIOjMiziuhrsKVkZeTxSbg7og4tpVzGo5v7AlgF+CWgrJdyHphAB+ShlElrUWWIAGOA/oCu0TEMknzgO4tBR8R44HxkC1o2dKx7dVv2JiPtuv8JA4zW8OVfA8sIt4BDgSOk3RiE4e8DTTc55oL1EjaFUDS+pJaSqJTgb0kfTodv56krYsI7xJghKRB6fyNgXOBn6f988gSGsAhQNe03RtYmJLXvmS9QTMz60TKMgsxIl6X9BXgn5IWNdp9JXCppHeBPYCjgYsk9SC7xzW0hXoXSRoBXCepWyo+C3iqjXG9LOnrwHhJvYFaYERE/CMd8ifgFkmPAncAS1P5tcBtkmaR9dY+dn/PzMyqS1Wel9Ch0mSP7wD7RMQblWqnW82AqBk+rlLVA36Yr5mtXiRNj4jBxZyzRj2JIyJ+HxEDK5m8zMysY6xRCczMzFYfTmBmZpZLXtCyAgb2702971GZmVWUe2BmZpZLTmBmZpZLTmBmZpZLvgdWAbPmL6Z21JSS6vDnvMzMWuYemJmZ5ZITmJmZ5ZITmJmZ5ZITmJmZ5VKHJjBJyyXNlPSopBmS9ixDnYMkHdDC/r0lTZM0R9Lc9EDfUtpbUsr5ZmZWHh09C/HdiGhYm2t/4JfAF0qscxAwGPhL4x2S+gETgMMiYoakPsCdkl6OiJtLbNfMzKqomtPoewFvAEiqASamsrWB70TEfam38wfgAOBl4EzgfGBz4FSyNbzOAXpI2hv4ZURMLGjjZODKiJgBEBGvSjqDbEHLmyVdCdweEZNSHEsioqeknmSrOG9ItsjlWRFRuKpz2S2YMGqV10Omjv3YMXV1dZUMwcwsVzo6gfWQNBPoDtQAX0zlw4A7I+JcSV2AdVP5esC9EXG6pJuBXwBfBrYFroqIWyX9FBgcEac00d52wFWNyurT+S15Dzg8It5Kvbapkm6NFhZPkzQSGAnQpVffVqo3M7NSVXMIcQ/gaknbA48Al0vqCkyOiJnp+A/IelkAs4D3I2JZWim5toJxCjhP0j7ACqA/sAmwoLkTImI8MB6yBS2LbabCaa4AAAo8SURBVLDfsDGrvK7zB5nNzFpUtVmIEfEQ0AfoGxH/BPYB5gNXSjo+HbasoNezAng/nbuCtiXfJ4BdGpXtQtYLA/iQdA0krQWsk8qPA/oCu6SE+wpZr9HMzDqJqiUwSdsAXYDXJG0BvBIRfwIuA3Yuoqq3gfWb2XcJMEJSQ69vY+BcsntgAPNYmeAOIbvfBdAbWJh6e/sCWxQRj5mZdYBq3QODbJhueEQslzQEOF3SMmAJcHxzFTTh78CoVO8qkzgi4mVJXwfGS+pNNuw4IiL+kQ75E3CLpEfJhiqXpvJrgdvSUGU9MKcd79XMzCpILcxLWO2kz4B9B9gnIt6oVDvdagZEzfBxJdXhh/ma2ZpE0vSIGFzMOWvUkzgi4vcRMbCSycvMzDrGGpXAzMxs9eH1wCpgYP/e1HsI0MysotwDMzOzXHICMzOzXHICMzOzXPI9sAqYNX8xtaOmFH2ep86bmbWde2BmZpZLTmBmZpZLTmBmZpZLTmBmZpZLHZbAJC2XNLPga1Qrx59ZhjYl6SxJT0t6StI/JO1QQn0jJF1calxmZla6jpyF+NFilm10JnBeMQ1I6hIRywuKTgb2BHaMiHck7QfcKmm7iFjadC1mZpYHVZ1Gn5Y4mQYcEhFzJV0H3At8ipVLrzweEcelZVG+T7bo5MPAd9NSLEuAPwJDyRLW/QVN/Bj4QkS8AxARd0m6j2zByvGSlkREzxTLkcBBETFC0sHAWamt14DjIuKVSlyDBRNWdkSHTB27yr66urpKNGlmtlroyHtgPRoNIR4dEYuBU8hWYT4G2DAi/hQRo0g9tpS8PgscDeyVenHLyZIQwHrAwxGxY0R8lLwk9QLWi4jnGsVRD2zbSqz3A7tHxE7A/wJntPbmJI2UVC+pfvk7i1u9GGZmVpqqDyFGxN2SjiJbPXnHZs79EtnKyY9IAugBLEz7lgM3ljnWzYCJkmrIemHPt3ZCRIwHxkO2HlhbG+o3bMxH23X+ILOZWZtVfRaipLWAzwLvABs2dxhwVeqRDYqIz0TE6LTvvUb3vQCIiLeApZK2arRrF7JeGEBhoulesH0RcHFEDAS+3WifmZl1AlVPYMAPgSeBYcAVkrqm8mUF2/cAR0r6BICkjSRt0Ya6xwIXSuqRzhsKbAdMSvtfkfTZlEQPLzivNzA/bQ9v5/syM7MK6sghxIZJGQ3uAK4Avgl8LiLelvRPsskTZ5MNxz0maUa6D3YWcFdKNsvIJmy80EqbFwEbpHq6kg0Hbh8R76X9o4DbgUVkvbKeqXw0cIOkN8gmlWxZwvs2M7MKUESbb9fkmqSewM3AIxFR8mfMWtKtZkDUDB9X9Hl+mK+ZrakkTY+IwcWcs8Y8jT4ilgBfrnYcZmZWHp3hHpiZmVnR1pgeWEca2L839R4ONDOrKPfAzMwsl5zAzMwsl5zAzMwsl5zAzMwsl5zAzMwsl5zAzMwsl5zAzMwsl5zAzMwsl5zAzMwsl9aYh/l2JElvA3OrHUcL+gCvVjuIVnT2GDt7fOAYy6GzxwerT4xbRETfYir1o6QqY26xT1XuSJLqO3N80Plj7OzxgWMsh84eH6zZMXoI0czMcskJzMzMcskJrDLGVzuAVnT2+KDzx9jZ4wPHWA6dPT5Yg2P0JA4zM8sl98DMzCyXnMDMzCyXnMDKSNJXJM2V9IykUR3Y7icl/V3SE5Iel/SDVL6RpLslPZ2+b5jKJenCFOdjknYuqGt4Ov5pScMrEGsXSf+SdHt6vaWkh1MsEyWtk8q7pdfPpP21BXX8dyqfK2n/Mse3gaRJkuZIelLSHp3pOkr6YfoZz5Z0naTu1b6Gki6XtFDS7IKysl0zSbtImpXOuVCSyhTj2PRzfkzSzZI2KNjX5PVp7ne8uZ9BKfEV7PsvSSGpT3rdaa5hKv9euo6PSzq/oLzy1zAi/FWGL6AL8CywFbAO8CiwbQe1XQPsnLbXB54CtgXOB0al8lHAr9L2AcBfAQG7Aw+n8o2A59L3DdP2hmWO9UfABOD29Pp64Ji0fSnwnbT9XeDStH0MMDFtb5uubTdgy3TNu5QxvquAb6btdYANOst1BPoDzwM9Cq7diGpfQ2AfYGdgdkFZ2a4ZMC0dq3TuV8sU437A2mn7VwUxNnl9aOF3vLmfQSnxpfJPAncCLwB9OuE13Bf4G9Atvf5ER17DivxBXRO/gD2AOwte/zfw31WK5Rbgy2RPA6lJZTVkH7AG+CNwbMHxc9P+Y4E/FpSvclwZ4toMuAf4InB7+mV6teCPyEfXMP3S7pG2107HqfF1LTyuDPH1JksQalTeKa4jWQJ7Mf2BWjtdw/07wzUEahv9YSvLNUv75hSUr3JcKTE22nc4cG3abvL60MzveEv/jkuND5gE7AjMY2UC6zTXkCzpDG3iuA65hh5CLJ+GPy4NXkplHSoNE+0EPAxsEhEvp10LgE3SdnOxVvo9jAPOAFak1xsDb0bEh02091Esaf/idHwlY9wSWARcoWyY8zJJ69FJrmNEzAd+DfwbeJnsmkync13DBuW6Zv3TdiVjBTiBrGfSnhhb+nfcbpIOBeZHxKONdnWma7g18Pk09PcPSbu2M8Z2XUMnsNWIpJ7AjcCpEfFW4b7I/ltTtc9MSDoIWBgR06sVQxusTTZE8oeI2AlYSjb89ZFqXsd0H+lQskS7KbAe8JVqxFKMav/ba42knwAfAtdWO5YGktYFzgR+Wu1YWrE22YjA7sDpwPXtub/WXk5g5TOfbLy6wWaprENI6kqWvK6NiJtS8SuSatL+GmBhK7FW8j3sBRwiaR7wv2TDiBcAG0hqeCZnYXsfxZL29wZeq3CMLwEvRcTD6fUksoTWWa7jUOD5iFgUEcuAm8iua2e6hg3Kdc3mp+2KxCppBHAQcFxKtO2J8TWa/xm016fI/qPyaPqd2QyYIalfO+Kr5DV8CbgpMtPIRlf6tCPG9l3D9oyD+qvJseG1yW6absnKm5PbdVDbAq4GxjUqH8uqN9LPT9sHsupN4GmpfCOye0Abpq/ngY0qEO8QVk7iuIFVb9x+N22fzKoTEK5P29ux6s3h5yjvJI77gM+k7dHpGnaK6wjsBjwOrJvavAr4Xme4hnz83kjZrhkfn4BwQJli/ArwBNC30XFNXh9a+B1v7mdQSnyN9s1j5T2wznQNTwLOSdtbkw0PqqOuYVn/MK3pX2Szg54im2Xzkw5sd2+yIZrHgJnp6wCyceV7gKfJZgo1/GMWcEmKcxYwuKCuE4Bn0tc3KhTvEFYmsK3SL9cz6R9ww2ym7un1M2n/VgXn/yTFPpd2zKZqJbZBQH26lpPTH4JOcx2BnwFzgNnAn9MfiKpeQ+A6sntyy8j+R35iOa8ZMDi932eBi2k0yaaEGJ8h+4Pb8DtzaWvXh2Z+x5v7GZQSX6P981iZwDrTNVwHuCbVPQP4YkdeQz9KyszMcsn3wMzMLJecwMzMLJecwMzMLJecwMzMLJecwMzMLJecwMzMLJecwMzMLJf+P/f/Ii1uSO8zAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plot_importances(100, 3, 'feature_importance.png')"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 30/30 [33:20<00:00, 103.45s/it]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAbEAAAEICAYAAADRFcoMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de5xVVf3/8ddblIsieIFkxMtooaaiiJh5ycYyLfP+1VTsK6RFlpbVV22+1jfJ0ki7UGoa9fOWYigqXihv2KTmBQFRvIBXTElEvKB4QYTP74+9Bg/jXM6Zc87MbHg/H495zD5r773W56xh5sNae5+9FBGYmZnl0RqdHYCZmVl7OYmZmVluOYmZmVluOYmZmVluOYmZmVluOYmZmVluOYmZmVluOYnZak/SXEnvSlpc8LVxmXXWSXqxUjEW2ealkn7ekW22RNJoSVd0dhy26nMSM8scGBG9C77+05nBSFqzM9svR55jt/xxEjNrhaRPS7pX0huSHpZUV7Dva5KekPSWpGclfTOVrwP8Hdi4cGTXdKTUdLSWRoQ/lPQI8LakNdN510p6RdJzkr5bZNy1kiLF+IKk1yWdIGkXSY+k93N+wfEjJf1L0vmSFkmaLenzBfs3lnSjpNckPS3pGwX7RkuaKOkKSW8CJwCnA0em9/5wa/1V2BeS/kfSAkkvSfpawf5ekn4t6fkU3z2SerX1M7JVn//HZNYCSQOBycB/A7cAnweulbRNRLwCLAAOAJ4F9gL+LunBiJgh6UvAFRGxSUF9xTR7NPBlYCGwHLgJuCGVbwLcIWlORNxa5NvYFRiU4rsxvY99gLWAhyRdExH/LDh2ItAPOAy4TtIWEfEa8FfgUWBjYBvgdknPRMSd6dyDgSOAY4EeqY5PRMRXC2Jpsb/S/gFAX2Ag8AVgoqRJEfE68CtgO2B3YH6KdXkRPyNbxXkkZpaZlP4n/4akSansq8DfIuJvEbE8Im4HpgH7A0TE5Ih4JjL/BG4DPlNmHL+PiBci4l1gF6B/RJwZEe9HxLPAn4CjSqjvZxHxXkTcBrwNXBURCyJiHnA3sFPBsQuAsRGxNCImAHOAL0vaFNgD+GGqaybwZ7KE1ei+iJiU+und5gIpor+WAmem9v8GLAa2lrQGcBxwckTMi4hlEXFvRCyhjZ+Rrfo8EjPLHBIRdzQp2xw4QtKBBWVrAf8ASKOtM4CtyP5DuDYwq8w4XmjS/saS3igo60aWfIr1csH2u8287l3wel6s/ETw58lGXhsDr0XEW032DWsh7mYV0V+vRsQHBa/fSfH1A3oCzzRTbas/I1v1OYmZtewF4C8R8Y2mOyT1AK4lG43cEBFL0wiucc6wueUh3ib7w91oQDPHFJ73AvBcRAxqT/DtMFCSChLZZmRTkP8BNpC0bkEi2wyYV3Bu0/e70usi+qs1C4H3gI8DDzfZ1+LPyFYPnk40a9kVwIGS9pPUTVLPdAPCJkB3sms/rwAfpFHGvgXnvgxsKKlvQdlMYH9JG0gaAHyvjfanAm+lmz16pRi2l7RLxd7hyj4GfFfSWpKOAD5JNlX3AnAv8IvUBzsAx5P1T0teBmrTVCC03V8tiojlwMXAb9INJt0k7ZYSY2s/I1sNOImZtSD98T6Y7E67V8j+138qsEYakXwXuBp4HRhONmppPHc2cBXwbLrOtjHwF7KRxFyy60ET2mh/GdmNEEOA58hGJH8mu/mhGh4guwlkIXAWcHhEvJr2HQ3Uko3KrgfOaGb6tdA16furkma01V9FOIVs6vFB4DXgl2Q/hxZ/RiXUbTkmL4ppZpJGAl+PiD07OxazUvh/K2ZmlltOYmZmllueTjQzs9zySMzMzHLLnxOrgn79+kVtbW1nh2FmlhvTp09fGBH9Sz3PSawKamtrmTZtWmeHYWaWG5Keb895nk40M7PcchIzM7PcchIzM7PcchIzM7PcchIzM7PcchIzM7PcchIzM7PcchIzM7Pc8oedq2DWvEXU1k/u7DDMzCpi7pgvd3YILfJIzMzMcstJzMzMcstJzMzMcstJzMzMcquqSUzSRpLGS3pW0nRJ90k6tMhzFzdTdoKkY0uMYU1Jr0gaU8p5ZmbW9VUtiUkSMAm4KyK2jIidgaOATZo5tqi7JCPiooi4vMRQvgA8CRyRYmou1m4l1mlmZl1ANW+x/xzwfkRc1FgQEc8D5wFIGgkcBvQGugGfbatCSaOBxcDNwOUR8alUXgvcFBGDmzntaOB3wLeA3YB70zlzgQlkSe4cSQ8CFwD9gXeAb0TEbEkHAj8GugOvAsdExMtF94KZWRc2f3x9m8fU3X9um8c0NDRUIJrSVTOJbQfMaOOYocAOEfFaKRWn5NJd0hYR8RxwJFlCWomknsA+wDeB9cgS2r0Fh7waEUPTsVOAEyLiKUm7An8gS8T3AJ+OiJD0deA04H+aaWsUMAqgW5+SFyc1M7N26LAPO0u6ANiTbHS2Syq+vdQEVuBqsuQ1Jn0/spljDgD+ERHvSroW+D9J34uIZWn/hBRbb2B34JqCGcce6fsmwARJNWSjseeaCyYixgHjAHrUDIp2viczsw41YHjbtws0rKYfdn6MbKQFQEScCHyebLqu0dtl1D8B+IqkrbLq46lmjjka2CdNHU4HNiQbXTVtfw3gjYgYUvD1ybTvPOD8NFX5TaBnGTGbmVkFVTOJ3Qn0lPStgrK1K1V5RDwDLAP+j+anEvsAnwE2i4jaiKgFTiRLbE3rehN4TtIR6VxJ2jHt7gvMS9sjKhW/mZmVr2pJLCICOAT4rKTnJE0FLgN+WGQVa0t6seDrB80cMwH4KtnUYlOHAndGxJKCshuAAyX1aOb4Y4DjJT1MNoo8OJWPJptmnA4sLDJ2MzPrAMpyjVVSj5pBUTNibGeHYWZWER3xAGBJ0yNiWKnn+YkdZmaWW05iZmaWW15PrAoGD+zLtC58S6qZ2arCIzEzM8stJzEzM8stJzEzM8stXxOrglnzFlFbP7mzwzCzCumIW8ytfTwSMzOz3HISMzOz3HISMzOz3HISMzOz3KpIEpO0iaQbJD0l6RlJv5PUvRJ1t9Hu4vS9VtKjLRyznaQ7Jc1Jsf1UUrvft6S5kvq193wzM6ucspOYslUkrwMmRcQgYCugN3BWBeou6+5JSb2AG4ExEbE1MBj4FHByubGZmVnnq8Qt9p8D3ouISwAiYpmk75Otz3UG2bpix0fEYwCSGoBTgCfIFpzcHlgLGB0RN0gaCRxGlgi7Sfoy2RIq66fjfhwRNxQZ23DgXxFxW4rtHUknAXcDv5U0GlgcEb9KsT0KHBARcyVNAjYlWwTzd2nlZjNbhcwfX1/UcXX3n1t0nQ0NDe2MxtqjEklsO7JVk1eIiDcl/Rv4BGkFZuAMSTVATURMk3Q22Xpfx0laD5gq6Y5UxVBgh4h4LY3GDk119gPul3RjFLeGTHOxPSOpV2qzNcel9nsBD0q6NiJebelgSaOAUQDd+vRv6TAzM6ugjviw89XAbcAZZMlsYirfFzhI0inpdU9gs7R9e0S8lrYFnC1pL2A5MBDYCJhf5bi/K+nQtL0pMAhoMYmlkdo4yNYTq3JsZlYBA4aPKeq4Bn/YucuqRBJ7HDi8sEBSH7KE9HSawntV0g7AkcAJjYcB/xURc5qcuyvwdkHRMUB/YOeIWCppLlnCKza2vZrUvyXwakS8IekDVr4u2DMdUwfsA+yW4m8ooU0zM+sglbg7cQqwtqRjASR1A34NXBoR76RjJgCnAX0j4pFUdivwnXRjCJJ2aqH+vsCClMD2BjYvIbYrgT0l7ZPa6AX8nmxUCDCXbOoSSUOBLQrafD0lsG2AT5fQppmZdZCyk1i6NnUocISkp4AngfeA0wsOmwgcRTa12OhnZDdqPCLpsfS6OVcCwyTNAo4FZpcQ27vAQcCPJD0JLCS70ePKdMi1wAap/ZNS7AC3AGtKegIYA9xfbJtmZtZxVNz9EasGSYcAvwH2jojnq9VOj5pBUTNibLWqN7MO5gcAV5+k6RExrNTzVqsndkTEpIjYspoJzMzMOs5qlcTMzGzV4iRmZma55UUxq2DwwL5M8xy6mVnVeSRmZma55SRmZma55SRmZma55WtiVTBr3iJq6yd3dhhdhj9jY2bV4pGYmZnllpOYmZnllpOYmZnlVptJTFJI+nXB61PSisitnXOQpFaXTJVUJ+nmFvbNTQtgtouk0QXrlFVMteo1M7P2KWYktgQ4rJSkEhE3RkRxq81VWFoJ2szMVgPFJLEPyFYs/n7THZL6S7pW0oPpa49UPlLS+Wn745LulzRL0s8lLS6oorekiZJmS7qycW2x5LR0zlRJn0h11Uq6U9IjkqZI2iyVXyrpIkkPAOek87eV1CDpWUnfLYj5B5IeTV/fK6L8R5KelHQPsHUR/WVmZh2k2FHLBWTrfp3TpPx3wG8j4p6UUG4FPtnMMb+LiKskndBk307AdsB/gH8BewD3pH2LImJwWmxzLHAAcB5wWURcJuk4sgUuD0nHbwLsHhHL0nTnNsDewLrAHEkXAjsAXwN2JVtZ+gFJ/yRL5i2VHwUMSX01A5heZJ+tluaP/+gsct395zZ7bENDQ5WjMbNVXVFJLCLelHQ58F3g3YJd+5CNeBpf95HUu8npu/FhohkP/Kpg39SIeBFA0kyglg+T2FUF339bUNdhafsvfDjqArgmIpYVvJ4cEUuAJZIWABsBewLXR8Tbqc3rgM+QJa7mytdI5e+k8hub7aBs3yhgFEC3Pv1bOszMzCqolOtHY8lGIpcUlK0BfDoi3is8cOVZwVYtKdhe1iSeaGG7JW+XUHfFRcQ4smlXetQMWn1WGm1iwPCPXgpt8IedzaxKir7FPiJeA64Gji8ovg34TuMLSUOaOfV+4L/S9lElxHZkwff70va9BXUcA9xdQn2k4w+RtLakdYBDU1lL5Xel8l6S1gUOLLE9MzOrolJHJ78GTip4/V3gAkmPpLruAppe9/oecIWkHwG3AIuKbGv9VO8S4OhU9h3gEkmnAq+QXccqWkTMkHQpMDUV/TkiHoLs5pAWyicADwMLgAdLac/MzKpLEdWd+ZK0NvBuRISko4CjI+LgqjbayXrUDIqaEWM7O4wuw89ONLO2SJoeEcNKPa8jPlO1M3B+un3+DeC4DmjTzMxWA1VPYhFxN7BjtdsxM7PVj5+daGZmueVHNFXB4IF9mebrQGZmVeeRmJmZ5ZaTmJmZ5ZaTmJmZ5ZaviVXBrHmLqK2f3NlhVIU/82VmXYlHYmZmlltOYmZmlltOYmZmlltOYmZmlltF3dghaUNgSno5gGx9rlfS609FxPtNjt8A+EpEXNRGvWsCCyNivWbKlwCzUtEy4MSIuL+YeFtpbyjwsYi4pZl9vYE/k600LeB1YD/g/SaxABwYES+UE4uZmZWv2JWdXwWGAEgaDSyOiF+1csoGZEuytJrE2vBWRDS2+WXgLODzZdQHMBTYnmxJmKa+D/w7Io5KbW4DLG0ai5mZdR1l32Iv6TTg2PTyjxFxHjAG2FrSTLKEcTYwCVgvtXl6RNxcQjN9yEZGSBoITAB6p7pGka0DthC4mGz09CLwE+AcYFOyNdD+kcp6SaoDfh4REwvaqAHmNL6IiNmpvdX+Ywjzx9ev2K67/9yV9jU0NHRwNGZmHyrrD7SkXclWWN4l1TVVUgNQD3yiYCS1FnBIRLwp6WPAv4C2kti6KQn2JJvC3DuVfxW4KSJ+Kakb0CuV9wX+HhE/kHQTMJps5LYjWXIdJulMYPuI+F4z7f0/4BZJR5JNnV4WEU83iQXg6Yg4vJm+GEWWUOnWp38bb83MzCqh3FHGnsC1EfEugKRJwGeA25ocJ2CMpD2B5cCmkvqRrS/WksLpxD2By4HBZKsr/1FST2BSRDycRkvvRsTt6dxZwKKI+EDSLKC2rTcSEdMlbQnsC+wDTJP0KeBZiphOjIhxwDjIFsVsq708GTB8zIrtBn/Y2cy6kI66O/FYspHS0JQMFpKNsIoSEfcAG0vaICLuBOqAl4DLJR2TDiu8uWQ52c0YjdvFXvt7KyKujYhvAX8FvlRsjGZm1vHKTWJ3A4dK6pXu7js4lb0FrFtwXF9gQRoZfQEYWEojkrYjS0avS9ocmJ9GPpcAO5VQVdO4CtvYU9J6absH8Eng+VLiNDOzjlXWdGJETJV0FdkUH8CFETELQNL0NJU3GfgNcFN6PRV4qojqC69DARwbESHp88APJC0lS0r/XULIdwKnSnoIOKvJjR2DgAslQZbcbwJuALqVUL+ZmXUgRaxSl2+6hB41g6JmxNjODqMq/ABgM6sGSdMjYlip5/mJHWZmlltOYmZmllur/Qd5q2HwwL5M87SbmVnVeSRmZma55SRmZma55SRmZma55WtiVTBr3iJq6yd3dhjN8i3yZrYq8UjMzMxyy0nMzMxyy0nMzMxyy0nMzMxyq1OSmDL3SPpSQdkRkm6pcDs/lzRP0kxJT0i6QFKr71nSrpJ+m7a/Lmls2j5M0jaVjM/MzMrTKUkssqcOnwD8RlLPtIzL2cCJ5dSbFsds6ty0htn2wM7AHm3E9kBEfL+ZXYcBTmJmZl1Ip91iHxGPSroJ+CGwDnB5RDwjaQRZMusO3AucFBHLJY0DhgK9gAkRcSaApBeBK4D9yBLhNS002R3oQVpNWtI9qe6ZkgYA90TEJyTtk8oPaTxR0meA/YE9JI0GDomIuRXsjqqZP75+pdd195+70uuGhoYOjMbMrLI6+3NiPwVmkK3KPEzS9sChwO5pAc1xwFHAeKA+Il5Lo61/SJoYEY+nehZEREuLY54qaSSwOXBT43pnpYiIuyX9DZgYEZOaO0bSKGAUQLc+/UttwszM2qFTk1hEvC1pArA4IpakUdAuwLS0OGUv4IV0+NGSjieLeWNgW6AxiU1opZlzI2KspO7A9ZIOb7IYZqXeyzhgHGTriVW6/vYaMHzMSq8b/GFnM1uFdPZIDGB5+gIQcHFE/F/hAZIGAScDn4qINyRdAfQsOOTtthqJiPfTjSN7AROBD/jwmmDPFk80M7Muq6vdYn8H8BVJ/QAkbShpM6AP8BbwpqQasutfJVE2tNsdeCYVzSW70QPg8CKqeAtYt9R2zcyserpUEkvXq34K3CHpEeA2YCOy62aPA7OBy4F/lVDtqZJmAo8Cy4A/pvJzgZMlzQDWL6Keq4DT0+36tSW0b2ZmVaLsbnerpB41g6JmxNjODqNZfgCwmXVFkqZHxLBSz+tSIzEzM7NSOImZmVludYW7E1c5gwf2ZZqn7czMqs4jMTMzyy0nMTMzyy0nMTMzyy1fE6uCWfMWUVs/uVPa9i30ZrY68UjMzMxyy0nMzMxyy0nMzMxyy0nMzMxyq11JTNKy9CDchyXNkLR7uYFIGiJp/4LXIyW9ktqZKenyVH5mWnes2HrXlnSlpFmSHpV0j6TeTd7HzJYe7CvpFklvSLq53PdoZmaV1d67E9+NiCEAkvYDfgF8tsxYhgDDgL8VlE2IiJMKD4qIn5RY78nAyxExGEDS1sDStG/F+2jFucDawDdLbNfMzKqsErfY9wFeB0hrfU1IZWsC34qIuyUtBi4E9gdeAk4HzgE2A74H3AKcCfSStCdZUmyWpEuBmyNioqS5wGXAgcBawBERMbvJKTXA840vImJOKW8uIqZIqivlnI42f3z9iu26+89daV9DQ0MHR2Nm1nHae02sV5p+mw38GfhZKh8O3JpGNzsCM1P5OsCdEbEd2eKSPwe+ABwKnBkR7wM/IRt5DYmICem8Iwum+r7WQiwLI2IoWZI8pZn9FwM/lHSfpJ+nVaKbvo+Zkq5vT0c0kjRK0jRJ05a9s6icqszMrEiVmE7cDbhc0vbAg8DFktYCJkVEYxJ7n2y0BTALWBIRSyXNAmpbaecj04nNuC59nw4c1nRnRMyUtCWwL7AP8KCk3SLiCYqbTixKRIwDxkG2nlgl6izWgOFjVmw3+MPOZrYaKfvuxIi4D+gH9I+Iu4C9gHnApZKOTYctjQ9X31wOLEnnLqf8Kc0l6fuyluqKiMURcV1EfBu4gmxa08zMcq7sJCZpG6Ab8KqkzcluovgT2TTj0BKqegtYt9x4molvD0nrp+3uwLYUXCMzM7P8au8oqJekxqlCASMiYlm6AeJUSUuBxcCxLVXQjH8A9aneFm/saIePAxdKElnSngxcW+zJku4GtgF6S3oROD4ibq1gfGZm1k76cJbPKqVHzaCoGTG2U9r2A4DNLI8kTY+IYaWe5yd2mJlZbjmJmZlZbnk9sSoYPLAv0zytZ2ZWdR6JmZlZbjmJmZlZbjmJmZlZbvmaWBXMmreI2vrJVW3Dt9KbmXkkZmZmOeYkZmZmueUkZmZmueUkZmZmudXuJCZpE0k3SHpK0jOSfpeeEl9VaZVoJNVKerSFY7aS9LcU2wxJV0vaqAJtj5R0frn1mJlZZbQriaUnwl9HtvDlIGAroDdwVrkBSSrrjklJPcmeVH9hRAxKqz7/AehfbmxmZta1tHck9jngvYi4BCAilgHfB46TtLak+yVt13iwpAZJwyStI+liSVMlPSTp4LR/pKQbJd0JTJHUW9KUNIqa1XhckYYD90XETY0FEdEQEY9K6inpklTnQ5L2Lmj/Okm3pNHbOQWxf03Sk5KmAnu0s78qZv74euaPr6eurm7Fl5nZ6qq9o57tgOmFBRHxpqR/A58AJgBfAc6QVAPURMQ0SWcDd0bEcZLWA6ZKuiNVMRTYISJeS6OxQ1Od/YD7Jd0Yxa0bs33T2AqcmIUag9NinrdJ2irtGwLsRLZS9BxJ5wEfAD8FdgYWka159lBzFUsaBYwC6NbHgz4zs45QrQ87Xw3cBpxBlswmpvJ9gYMknZJe9wQ2S9u3R8RraVvA2ZL2ApYDA4GNgPllxrUncB5ARMyW9DzZVCjAlIhYBCDpcWBzoB/QEBGvpPIJBcevJCLGAeMgW0+szDhbNGD4GAAa/GFnM7N2J7HHgcMLCyT1IUtIT0fEO5JelbQDcCRwQuNhwH9FxJwm5+4KvF1QdAzZNaydI2KppLlkCa8YjwGfLfH9QDYCa7QMP83EzKzLa+81sSnA2pKOBZDUDfg1cGlEvJOOmQCcBvSNiEdS2a3Ad9KNIUjaqYX6+wILUgLbm2xUVKzxwO6SVgxVJO0laXvgbrIESZpG3AyY02wtmQeAz0raUNJawBElxGFmZlXWriSWrk0dChwh6SngSeA94PSCwyYCR5FNLTb6GbAW8Iikx9Lr5lwJDJM0CzgWmF1CbO8CB5Aly6fS1OC3gVfI7lJcI9U7ARgZEUtaqeslYDRwH/Av4Ili4zAzs+pTcfdKWCl61AyKmhFjq9qGHwBsZqsSSdMjYlip5/mJHWZmlltOYmZmllu+A68KBg/syzRP95mZVZ1HYmZmlltOYmZmlltOYmZmllu+JlYFs+YtorZ+clXq9q31ZmYf8kjMzMxyy0nMzMxyy0nMzMxyy0nMzMxyq6wkJmlxwfb+aQXkzSWdUPCE+5GSNm6jnpGSzi8nllbq3jOtJD1b0hxJ3y6zvsVtH2VmZh2hIncnSvo88Htgv4h4HrioYPdI4FHgP5Voq8S4BpAtzXJIRMxIq0TfKumliLi+o+MxM7PKKns6Ma2+/CfggIh4JpWNlnSKpMOBYcCVkmZK6iVpF0n3Sno4jZDWTVVtLOmWtHzKOQX17yvpPkkzJF0jqXcqnyvpp6l8lqRtmgnvRLI1zmYARMRCsjXOTk11XJpibGxrcfreW9KUgroPLrefzMys8spNYj2ASWQjnY+s+RURE4FpwDERMYRsxeQJwMkRsSOwD/BuOnwI2SrQg4EjJW2aRk4/BvaJiKGprh8UNLEwlV8InNJMfNsB05uUTQO2beN9vQccmureG/h140KenWX++Hrmj6+nrq6uM8MwM+tSyp1OXArcCxwPnFzE8VsDL0XEgwAR8SZAyg9TImJRev042WrO65ElnH+lY7qTLVDZ6Lr0fTpwWJnvpZCAs9MoczkwENgImN/iCdIoYBRAtz79KxiKmZm1pNwkthz4CjBF0ukRcXYZdRWusLyMLDYBt0fE0W2c03h8U48DOwM3FJTtTDYaA/iANBqVtAZZkgQ4BugP7BwRSyXNBXq2FnxEjAPGQbYoZmvHtseA4WMAaPATO8zMVij7mlhEvAN8GThG0vHNHPIW0Hjdaw5QI2kXAEnrSmotkd4P7CHpE+n4dSRtVUJ4FwAjJQ1J528InAX8LO2fS5bUAA4C1krbfYEFKYHtTTYqNDOzLqYidydGxGuSvgjcJemVJrsvBS6S9C6wG9l1r/Mk9SK7HrZPK/W+ImkkcJWkHqn4x8CTRcb1kqSvAuMk9QVqgZER8c90yJ+AGyQ9DNwCvJ3KrwRukjSLbNT2ket9ZmbW+RRR8ZmvLit9RuxbwF4R8Xq12ulRMyhqRoytSt1+ALCZrYokTY+IYaWet1o9sSMi/hARg6uZwMzMrOOsVknMzMxWLU5iZmaWW14UswoGD+zLNF+7MjOrOo/EzMwst5zEzMwst5zEzMwst3xNrApmzVtEbf3ksuvxZ8LMzFrnkZiZmeWWk5iZmeWWk5iZmeWWk5iZmeVWhyYxScskzZT0sKQZknavQJ1DJO3fyv49JU2VNFvSnPQQ4HLaW1zO+WZmVjkdfXfiuxHRuLbXfsAvgM+WWecQYBjwt6Y7JA0AxgOHRMQMSf2AWyW9FBHXl9mumZl1ss6cTuwDvA4gqUbSXWmU9qikz6TyxZLOlfSYpDskfUpSg6RnJR0kqTtwJnBkOvfIJm2cCFwaETMAImIhcBpwaqr/UkmHNx7cOMqS1FvSlDRanCXp4Cr3xUrmj69n/vh66urqqKur68imzcxypaNHYr0kzQR6AjXA51L5cODWiDhLUjdg7VS+DnBnRJwq6Xrg58AXgG2ByyLiRkk/AYZFxEnNtLcdcFmTsmnp/Na8BxwaEW+m0dv9km6MVhZfkzQKGAXQrU//Nqo3M7NK6MzpxN2AyyVtDzwIXCxpLWBSRMxMx79PtuIywCxgSUQsTSsu11YxTgFnS9oLWA4MBDYC5rd0QkSMA8ZBtihmOY0PGD4GgAZ/2NnMrFWdNp0YEfcB/eH6npIAAAoYSURBVID+EXEXsBcwD7hU0rHpsKUFo5/lwJJ07nKKS8CPAzs3KduZbDQG8AGpDyStAXRP5ccA/YGdU9J9mWz0aGZmXUinJTFJ2wDdgFclbQ68HBF/Av4MDC2hqreAdVvYdwEwUlLj6G9D4CzgZ2n/XD5McgcBa6XtvsCCNOrbG9i8hHjMzKyDdNY1Mcim7EZExDJJdcCpkpYCi4FjW6qgGf8A6lO9v4iICY07IuIlSV8FxknqSzYFOTIi/pkO+RNwg6SHyaYt307lVwI3pWnLacDsdrxXMzOrMrVyr8IqJ31G7FvAXhHxerXa6VEzKGpGjC27Hj8A2MxWF5KmR8SwUs9brZ7YERF/iIjB1UxgZmbWcVarJGZmZqsWrydWBYMH9mWapwLNzKrOIzEzM8stJzEzM8stJzEzM8stXxOrglnzFlFbP7ld5/q2ejOz4nkkZmZmueUkZmZmueUkZmZmueUkZmZmudVhSUzSsrT6cuNXfRvHn16BNiXpx5KekvSkpH9K2qGM+kZKOr/cuMzMrDI68u7EFQtiFul04OxSGpDULSKWFRSdCOwO7BgR70jaF7hR0nYR8XbztZiZWV506nSipL6S5kjaOr2+StI3JI0hLdsi6cq076uSpqayP0rqlsoXS/p1Wk5ltyZN/BA4KSLeAYiI24C7yRa9RNLiglgOl3Rp2j5Q0gOSHpJ0h6SNqtoRyfzx9dTV1VFXV9cRzZmZ5V5HJrHGpNT4dWRELAJOIlvN+Shg/Yj4U0TUk0ZuEXGMpE8CRwJ7pNHcMlIiAtYBHoiIHSPinsbGJPUB1omIZ5vEMQ3Yto1Y7wE+HRE7AX8FTmvrzUkaJWmapGnL3lnUZmeYmVn5On06MSJul3QE2SrMO7Zw7ufJVmB+UBJAL2BB2rcMuLbCsW4CTJBUA3QHnmvrhIgYB4yDbD2x9jQ6YPgYGvxhZzOzonX63YmS1gA+CbwDrN/SYcBlaWQ2JCK2jojRad97Ta6DARARbwJvS9qyya6dyUZjAIXJpmfB9nnA+RExGPhmk31mZtZFdHoSA74PPAEMBy6RtFYqX1qwPQU4XNLHACRtIGnzIuo+F/i9pF7pvH2A7YCJaf/Lkj6ZEumhBef1Beal7RHtfF9mZlZlHTmd2EvSzILXtwCXAF8HPhURb0m6C/gxcAbZ1Nwjkmak62I/Bm5LCWcp2Z2Hz7fR5nnAeqmetcimBrePiPfS/nrgZuAVstFZ71Q+GrhG0uvAncAWZbxvMzOrEkW06/JN7kjqDVwPPBgRZX8GrTU9agZFzYix7TrXDwA2s9WRpOkRMazU81abp9hHxGLgC50dh5mZVU5XuCZmZmbWLqvNSKwjDR7Yl2meFjQzqzqPxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLdWmwcAdyRJbwFzOjuOVvQDFnZ2EG1wjJXR1WPs6vGBY6yUtmLcPCL6l1qpHztVHXPa8zTmjiJpWleODxxjpXT1GLt6fOAYK6VaMXo60czMcstJzMzMcstJrDrGdXYAbejq8YFjrJSuHmNXjw8cY6VUJUbf2GFmZrnlkZiZmeWWk5iZmeWWk1gFSfqipDmSnpZU38FtbyrpH5Iel/SYpJNT+QaSbpf0VPq+fiqXpN+nWB+RNLSgrhHp+KckjahwnN0kPSTp5vR6C0kPpDgmSOqeynuk10+n/bUFdfxvKp8jab8Kx7eepImSZkt6QtJuXbAPv59+xo9KukpSz87uR0kXS1og6dGCsor1m6SdJc1K5/xekioU47npZ/2IpOslrVewr9n+aen3vKWfQTnxFez7H0khqV963WX6MJV/J/XjY5LOKSivfh9GhL8q8AV0A54BtgS6Aw8D23Zg+zXA0LS9LvAksC1wDlCfyuuBX6bt/YG/AwI+DTyQyjcAnk3f10/b61cwzh8A44Gb0+urgaPS9kXAt9L2t4GL0vZRwIS0vW3q2x7AFqnPu1UwvsuAr6ft7sB6XakPgYHAc0Cvgv4b2dn9COwFDAUeLSirWL8BU9OxSud+qUIx7gusmbZ/WRBjs/1DK7/nLf0MyokvlW8K3Ao8D/Trgn24N3AH0CO9/lhH9mFV/qCujl/AbsCtBa//F/jfToznBuALZE8OqUllNWQfxAb4I3B0wfFz0v6jgT8WlK90XJkxbQJMAT4H3Jx+mRYW/BFZ0Yfpl3a3tL1mOk5N+7XwuArE15csQahJeVfqw4HAC+mP1JqpH/frCv0I1Db541aRfkv7ZheUr3RcOTE22XcocGXabrZ/aOH3vLV/y+XGB0wEdgTm8mES6zJ9SJZ49mnmuA7pQ08nVk7jH5dGL6ayDpemjHYCHgA2ioiX0q75wEZpu6V4q/k+xgKnAcvT6w2BNyLig2baWhFH2r8oHV/N+LYAXgEuUTbl+WdJ69CF+jAi5gG/Av4NvETWL9PpWv3YqFL9NjBtVzNWgOPIRijtibG1f8vtJulgYF5EPNxkV1fqw62Az6RpwH9K2qWdMbarD53EVjGSegPXAt+LiDcL90X235tO+UyFpAOABRExvTPaL9KaZFMlF0bETsDbZNNgK3RmHwKk60oHkyXcjYF1gC92VjzF6ux+a4ukHwEfAFd2diyNJK0NnA78pLNjacOaZDMDnwZOBa5uz/W29nISq5x5ZHPXjTZJZR1G0lpkCezKiLguFb8sqSbtrwEWpPKW4q3W+9gDOEjSXOCvZFOKvwPWk9T4DM/CtlbEkfb3BV6tYnyQ/c/vxYh4IL2eSJbUukofAuwDPBcRr0TEUuA6sr7tSv3YqFL9Ni9tVyVWSSOBA4BjUrJtT4yv0vLPoL0+TvaflYfT780mwAxJA9oRXzX78EXgushMJZtp6deOGNvXh+2ZE/VXs/PEa5JdRN2CDy9WbteB7Qu4HBjbpPxcVr64fk7a/jIrXxiemso3ILsutH76eg7YoMKx1vHhjR3XsPKF3G+n7RNZ+YaEq9P2dqx8sfhZKntjx93A1ml7dOq/LtOHwK7AY8Daqd3LgO90hX7ko9dKKtZvfPSmhP0rFOMXgceB/k2Oa7Z/aOX3vKWfQTnxNdk3lw+viXWlPjwBODNtb0U2VaiO6sOK/WHy14o7hp4ku/PmRx3c9p5k0zWPADPT1/5k88xTgKfI7iBq/Act4IIU6yxgWEFdxwFPp6+vVSHWOj5MYlumX66n0z/gxjuceqbXT6f9Wxac/6MU9xzacYdVG7ENAaalfpyU/hB0qT4EfgrMBh4F/pL+SHRqPwJXkV2jW0r2P/PjK9lvwLD0fp8BzqfJzTdlxPg02R/dxt+Zi9rqH1r4PW/pZ1BOfE32z+XDJNaV+rA7cEWqewbwuY7sQz92yszMcsvXxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLecxMzMLLf+PyiL8FxibZ0JAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# use 10x the iterations\n",
"plot_importances(100, 30, 'feature_importance_bigger_dataset.png')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Above we constructed confidence intervals around "
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"# NOTE: this implementation is not currently open sourced, sorry!\n",
"from explanation.global_explanations.random_ablation import random_ablation_feature_importance\n",
"\n",
"def pointwise_absolute_error(y_true: np.ndarray,\n",
" y_pred: np.ndarray) -> np.ndarray:\n",
" \"\"\"\n",
" :param y_true: Array of shape (N,), (N, 1)\n",
" :param y_pred: Array of shape (N,), (N, 1)\n",
"\n",
" :returns: An array of shape (N,) corresponding to the squared error of each909090p5r \n",
" row.\n",
"\n",
" \"\"\"\n",
" # demote (N, 1) shapes to (N,)\n",
" if y_true.ndim == 2:\n",
" y_true = y_true[:, 0]\n",
" if y_pred.ndim == 2:\n",
" y_pred = y_pred[:, 0]\n",
"\n",
" loss = np.abs(y_true - y_pred)\n",
" return loss\n",
"\n",
"\n",
"def plot_importances(n_points, n_cycles, figname, n_features=10):\n",
" sampled_index = x_test.sample(n_points, random_state=0).index\n",
" res = random_ablation_feature_importance(\n",
" model_fn=ensemble_f,\n",
" loss_fn=pointwise_absolute_error,\n",
" input_x=x_test.loc[sampled_index],\n",
" reference_x=x_train,\n",
" input_y=y_test.loc[sampled_index],\n",
" max_inferences_per_feature=n_points * n_cycles,\n",
" seed=0\n",
" )\n",
" imp = pd.DataFrame(\n",
" dict(\n",
" feature_importance=res.mean_loss_increase_importance, \n",
" ci_rand=res.random_sample_ci, \n",
" ci_fixed=res.fixed_sample_ci\n",
" ), \n",
" index=res.feature_names\n",
" ).sort_values(by='feature_importance', ascending=False)[:n_features]\n",
" x = np.arange(len(imp))\n",
" plt.barh(x[::-1], imp['feature_importance'], tick_label=imp.index, xerr=imp['ci_rand'])\n",
" plt.title('Feature Importance')\n",
" plt.savefig(figname, dpi=300, bbox_inches='tight')\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 81/81 [00:27<00:00, 2.85it/s]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAEICAYAAAAOW7ATAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZwdRb3+8c8DAgkG2YKyGSKIomwhDIiIGAVRUWS5IAheCKIR9+Ui8lOuxA254g4qRi8CshhlXxRBYBTUCBNkVXaCgATCTlhCCM/vj66R49xZzsmcmelJnvfrNa/pU91dVadnMt9UdZ3+yjYRERF1tcxIdyAiIqI/CVQREVFrCVQREVFrCVQREVFrCVQREVFrCVQREVFrCVQREVFrCVSx1JM0R9LTkuY3fK09yDqnSLqnXX1sss0TJH11ONvsi6Tpkk4e6X7EkiGBKqKyi+1xDV//HMnOSHrRSLY/GKO571FPCVQR/ZC0jaQ/SXpU0rWSpjTsO1DS3yU9IekOSR8q5S8GfgOs3ThC6zni6TnqKiO7z0m6DnhS0ovKeWdImifpTkmfaLLfEyW59PFuSY9IOljSVpKuK+/n2Ibjp0r6o6RjJT0m6SZJOzTsX1vSuZIelnSbpA827Jsu6XRJJ0t6HDgY+Dywd3nv1/Z3vRqvhaT/kvSApPskHdiwf6ykb0m6q/TvCkljB/oZxZIh//OJ6IOkdYALgP8ELgR2AM6QtJHtecADwLuAO4Dtgd9Iusr21ZLeAZxse92G+ppp9r3AO4EHgeeB84BzSvm6wO8k3Wz7t02+jdcBG5b+nVvex47AcsBfJf3K9u8bjj0dGA/sAZwp6RW2HwZ+AdwArA1sBFws6Xbbl5ZzdwX2AvYHVih1vNL2+xr60uf1KvvXBFYG1gHeCpwu6WzbjwDfBDYGtgXmlr4+38TPKJYAGVFFVM4u/yN/VNLZpex9wK9t/9r287YvBrqAnQFsX2D7dld+D1wEvHGQ/fi+7bttPw1sBaxh+8u2n7V9B/ATYJ8W6vuK7WdsXwQ8CZxm+wHb9wKXA1s0HPsA8F3bC23PBG4G3inp5cAbgM+Vuq4BfkoVlLr92fbZ5To93VtHmrheC4Evl/Z/DcwHXi1pGeD9wCdt32t7ke0/2V7AAD+jWDJkRBVR2c3273qUrQfsJWmXhrLlgMsAyqjpCOBVVP/pWxG4fpD9uLtH+2tLerShbFmqANOs+xu2n+7l9biG1/f6359SfRfVCGpt4GHbT/TY19FHv3vVxPV6yPZzDa+fKv0bD4wBbu+l2n5/RrFkSKCK6NvdwM9tf7DnDkkrAGdQjSrOsb2wjMS65/d6S0vwJNUf525r9nJM43l3A3fa3nBxOr8Y1pGkhmA1gWq68J/AapJWaghWE4B7G87t+X7/7XUT16s/DwLPABsA1/bY1+fPKJYcmfqL6NvJwC6S3iZpWUljyk3/dYHlqe7FzAOeK6OFnRrOvR9YXdLKDWXXADtLWk3SmsCnBmj/SuCJssBibOnDJpK2ats7/HcvBT4haTlJewGvoZpWuxv4E/D1cg02Aw6iuj59uR+YWKbtYODr1SfbzwPHA98uizqWlfT6Evz6+xnFEiKBKqIP5Q/0rlQr2OZR/e/9s8AyZWTxCeCXwCPAvlSjj+5zbwJOA+4o973WBn5ONSKYQ3V/ZuYA7S+iWnwwCbiTamTxU6oFB0PhL1QLLx4Evgbsafuhsu+9wESq0dVZwBG9TJU2+lX5/pCkqwe6Xk04hGqa8CrgYeB/qH4Off6MWqg7ak5JnBgRkqYCH7C93Uj3JaKn/K8jIiJqLYEqIiJqLVN/ERFRaxlRRUREreVzVENg/Pjxnjhx4kh3IyJi1Jg9e/aDttfobV8C1RCYOHEiXV1dI92NiIhRQ9Jdfe3L1F9ERNRaAlVERNRaAlVERNRaAlVERNRaAlVERNRaAlVERNRaAlVERNRaAlVERNRaPvA7Skw87IKR7kJEFHOOeudId2GpkhFVRETUWgJVRETUWgJVRETUWgJVRETU2pAGKkkvk3SqpDskzZb0Z0m7N3nu/F7KDpa0f4t9eJGkeZKOauW8iIiohyELVJIEnA38wfb6trcE9gHW7eXYplYf2j7O9kktduWtwC3AXqVPvfV12RbrjIiIYTKUy9PfAjxr+7juAtt3AccASJoK7AGMA5YF3jRQhZKmA/OB84GTbG9dyicC59netJfT3gt8D/gw8HrgT+WcOcBMqkD2DUlXAT8A1gCeAj5o+yZJuwCHA8sDDwH72b6/6asQMYrNPfWwke5CLU2ZdfRId6GWOjs7h6TeoQxUGwNXD3DMZGAz2w+3UnEJIMtLeoXtO4G9qYLOv5E0BtgR+BCwClXQ+lPDIQ/ZnlyOvQQ42Patkl4H/JAq2F4BbGPbkj4AHAr8Vy9tTQOmAUyYMKGVtxMREf0Ytg/8SvoBsB3VKGurUnxxq0GqwS+pAtRR5fvevRzzLuAy209LOgP4b0mfsr2o7J9Z+jYO2Bb4VcPs4Arl+7rATElrUY2q7uytM7ZnADMAOjo6vJjvKaJW1tw3t3Z705kP/A6roVxMcSPViAkA2x8FdqCaWuv25CDqnwm8R9Krqup9ay/HvBfYsUzzzQZWpxol9Wx/GeBR25Mavl5T9h0DHFumFT8EjBlEnyMiokVDGaguBcZI+nBD2Yrtqtz27cAi4L/pfdrvJcAbgQm2J9qeCHyUKnj1rOtx4E5Je5VzJWnzsntl4N6yfUC7+h8REc0ZskBl28BuwJsk3SnpSuBE4HNNVrGipHsavj7TyzEzgfdRTQP2tDtwqe0FDWXnALtIWqGX4/cDDpJ0LdVocNdSPp1qSnA28GCTfY+IiDZRFU+inTo6OtzV1dXWOvNQ2oj6yENp20/SbNsdve3LkykiIqLWEqgiIqLWko9qlMhUQ0QsrTKiioiIWkugioiIWkugioiIWss9qlEiy9OjW+5XxtImI6qIiKi1BKqIiKi1BKqIiKi1BKqIiKi1tgQqSetKOkfSrZJul/Q9Scu3o+4B2p1fvk+UdEMfx2ws6VJJN5e+fUnSYr9vSXMkjV/c8yMiojWDDlSqMg2eCZxte0PgVVTp5b/WhroHtSpR0ljgXOAo268GNgW2Bj452L5FRMTwaMfy9LcAz9j+GYDtRZI+TZXf6QiqvFQH2b4RQFIncAjwd6qkhJsAywHTbZ8jaSqwB1WwW1bSO6nSc6xajjvc9jlN9m1f4I+2Lyp9e0rSx4DLge9Img7Mt/3N0rcbgHfZniPpbODlVIkSv1cy+MYoMPfUw0a6C0NqyqyjR7oLQ6qzs3OkuxA1045AtTFV9tx/sf24pH8Ar6Rk4gWOKOnc17LdJelIqnxR75e0CnClpN+VKiYDm9l+uIyqdi91jgdmSTrXzeUn6a1vt0saW9rsz/tL+2OBqySdYfuhvg6WNA2YBjBhwoQmuhYREc0Yjg/8/hK4CDiCKmCdXsp3At4t6ZDyegzQ/Rf+YtsPl20BR0raHngeWAd4GTB3iPv9CUm7l+2XAxsCfQaqMuKaAVU+qiHuW/RjzX2PGukuDKnOfOA3ljLtCFR/A/ZsLChp4CcAt5XptockbQbsDRzcfRjwH7Zv7nHu64AnG4r2A9YAtrS9UNIcqqDWbN+271H/+sBDth+V9Bz/fp9uTDlmCrAj8PrS/84W2oyIiDZqx6q/S6jSxu8PIGlZ4FvACbafKsfMBA4FVrZ9XSn7LfDxshgDSVv0Uf/KwAMlSL0ZWK+Fvp0CbCdpx9LGWOD7VKM7gDlU04xImgy8oqHNR0qQ2gjYpoU2IyKijQYdqMq9ot2BvSTdCtwCPAN8vuGw04F9qKYBu32FanHEdZJuLK97cwrQIel6YH/gphb69jTwbuALkm4BHqRaXHFKOeQMYLXS/sdK3wEuBF4k6e/AUcCsZtuMiIj2UnNrEpYMknYDvg282fZdQ9VOR0eHu7q62lpnHkob3fJQ2lgSSZptu6O3fUvVkylsn217/aEMUhER0V5LVaCKiIjRJ4EqIiJqLYkTR4ncl4iIpVVGVBERUWsJVBERUWsJVBERUWu5RzVK1PVzVLl3FhFDLSOqiIiotQSqiIiotQSqiIiotREJVKpcIekdDWV7Sbqwze18VdK9kq6R9HdJP5DU73uW9DpJ3ynbH5D03bK9R3mSekREDKMRCVTliesHA9+WNEbSOOBI4KODqbdkA+7paNuTqFLebwm8YYC+/cX2p3vZtQeQQBURMcxGbOrP9g3AecDngC8CJ5U08QdIurKMgn7YPQKSNENSl6QbJX2xux5J90g6StJfqdKN9GV5YAXg0XLeFZImle01Jd1WtneUdHbjiZLeCOwMfKf0a2J7rkJERAxkpJenfwm4GniWKufUJlTBZlvbz0maQZXH6lTgMNsPl1HTZZJOt/23Us8DtvtKvPhZSVOpEi6eZ/v6Vjtp+3JJvwZOt332gCfU2NxTD2trfVNmHd3W+jo7O9taX0SMfiO6mML2k1TZf39uewFV+vetgC5J1wBvAjYoh79X0tVUge01wGsbqprZTzPdU38vA1aXtGeb3wYAkqaVEV/XvHnzhqKJiIil0kiPqACeL18AAo63/d+NB0jaEPgksLXtRyWdDIxpOOTJgRqx/WxZrLE9Vcbh53ghUI/p88Qm2Z4BzIAqceJg6xsqa+57VFvr68wHfiNiiNVtefrvgPdIGg8gaXVJE4CXAE8Aj0taC3hbqxVLErAtcHspmkO1uAKgmVHWE8BKrbYbERGDU6tAVe4ffQn4naTrgIuopuyuBv4G3AScBPyxhWo/W6YRbwAWAT8u5UcDnyzTias2Uc9pwOezmCIiYnipWike7dTR0eGurq621pln/UXEkkzSbNsdve2r1YgqIiKipwSqiIiotQSqiIiotTosT48m5F5QRCytMqKKiIhaS6CKiIhaS6CKiIhayz2qUaJOn6PK/bKIGE4ZUUVERK0lUEVERK0lUEVERK0lUEVERK0NGKgkWdK3Gl4fImn6AOe8W1K/qWQlTZF0fh/75nSn+lgckqZLOmRxzx/ueiMiom/NjKgWAHu0Ejhsn2u7vRn6mlRS1UdExBKimT/qz1Flrv008IXGHZLWAI4DJpSiT9n+o6SpQIftj0naADgFeDFwTjlmXDl+nKTTgU2A2cD7/ELekUMlvQN4GtjX9m0lD9TxwHhgHnCg7X9IOgF4BtiCKlfV48BrJXWWvn3X9vdLnz8DvL+08VPb3x2g/AvAAcADwN2ln7Uy99R+B69tN2XW0cPaXmdn57C2FxH10uw9qh8A+0lauUf594Dv2N4K+A/gp72c+z3ge7Y3Be7psW8L4FPAa4H1gTc07HusnHMs8N1Sdgxwou3NqILf9xuOXxfY1vZnyuuNqDIBbw0cIWk5SVsCBwKvA7YBPihpiwHK9wEmATsDW/V1gSRNk9QlqWvevHl9HRYRES1qaprM9uOSTgI+QTXC6bYj1cil+/VLJI3rcfrrgd3K9qnANxv2XWn7HoCShXcicEXZd1rD9+801LVH2f458I2Gun5le1HD6wtsLwAWSHqAKlPwdsBZtp8sbZ4JvBFQH+XLlPKnSvm5vV4gwPYMqpEnHR0dw5qNcs19h3eWtTMf+I2IYdTK/ZzvUqWE/1lD2TLANrafaTywIXANZEHD9qIe/XEf2315soW6IyJilGh6ebrth4FfAgc1FF8EfLz7haRJvZw6i2paEKpptGbt3fD9z2X7Tw117Adc3kJ9lON3k7SipBcDu5eyvsr/UMrHSloJ2KXF9iIiYpBaHWV8C/hYw+tPAD+QdF2p6w/AwT3O+RRwclmUcCHwWJNtrVrqXQC8t5R9HPiZpM9SFlO00nnbV5eFF1eWop/a/itAP+UzgWupFlNc1Up7ERExeHphkd0QNSCtCDxt25L2Ad5re9chbXSEdXR0uKurq6115qG0EbEkkzTbdkdv+4bjvs2WwLGqblw9ygtLwCMiIgY05IHK9uXA5kPdTkRELJmyEm6UyHRbRCyt8lDaiIiotQSqiIiotQSqiIiotdyjGiWGa3l67oVFRN1kRBUREbWWQBUREbWWQBUREbWWQBUREbXW1GIKSasDl5SXa1KlzejODri17Wd7HL8a8B7bxw1Q74uAB22v0kv5AuD6UrQI+KjtWc30t5/2JgMvtX1hL/vGUSV+3JgqP9UjVIkXn+3RF4BdbN89mL5ERERzmk2c+BBVllskTQfm2/5mP6esRvUU9X4D1QCesN3d5juBrwE7DKI+gMlUae//T6ACPg38w/Y+pc2NgIU9+xIREcNr0MvTJR0K7F9e/tj2McBRwKtL1t4LgSOBs4FVSpuft31+C828hGqEg6R1gJnAuFLXNKr0HA8Cx1ONgu4BvkiVAfjlVKlJLitlYyVNAb5q+/SGNtYCbu5+Yfum0t6oWMI/99TD2lLPlFlHt6UegM7OzrbVFRFLr0H9EZb0OqoEhluVuq6U1AkcBryyYUS0HLBbSWn/UuCPwECBaqUS6MZQTTe+uZS/DzjP9v9IWhYYW8pXBn5j+zOSzgOmU43ANqcKoB2SvgxsYvtTvbT3v8CFkvammuY80fZtPfoCcJvtPXu5FtOogiYTJkwY4K1FRESzBjta2A44w/bTAJLOBt5Ilfm3kYCjJG0HPA+8XNJ4qrQffWmc+tsOOAnYlCp54Y8ljQHOtn1tGfU8bfvicu71wGO2n5N0PTBxoDdie7ak9YGdgB2BLklbA3fQxNSf7RnADKjyUQ3UXrutue9RbamnMx/4jYiaGa5Vf/tTjXgmlz/4D1KNlJpi+wpgbUmr2b4UmALcB5wkab9yWOOCjuepFkB0bzd7L+4J22fY/jDwC+AdzfYxIiKGxmAD1eXA7pLGllVzu5ayJ4CVGo5bGXigjHDeCqzTSiOSNqYKOI9IWg+YW0YwPwO2aKGqnv1qbGM7SauU7RWA1wB3tdLPiIhov0FN/dm+UtJpVNNxAD+yfT1UaYXLtNsFwLeB88rrK4Fbm6i+8b4QwP4lnf0OwGckLaQKPP/ZQpcvBT4r6a/A13osptgQ+FGViJhlgPOAc4BlW6g/IiLaTPaw305Z4nV0dLirq6utdeahtBGxJJM023ZHb/vyZIqIiKi1BKqIiKi1UfFh1siUXEQsvTKiioiIWkugioiIWkugioiIWss9qlGincvTc78rIkaTjKgiIqLWEqgiIqLWEqgiIqLWEqgiIqLWFitQSVok6RpJ10q6WtK2g+2IpEmSdm54PVXSvNLONZJOKuVflrRjC/WuKOkUSddLukHSFeVJ743vo/trYi/nXyjpUUmtZCSOiIg2WdxVf083JDV8G/B14E2D7MskoAP4dUPZTNsfazzI9hdbrPeTwP22NwWQ9GpgYdn39EAJEYGjgRWBD7XYbkREtEE7lqe/BHgEQNJawMxS9iLgw7YvlzQf+BGwM1XCw88D3wAmAJ8CLgS+DIwt2Xy/3ldjkk4Azrd9uqQ5wInALsBywF62b+pxylo05JWyfXMrb872JZKmtHJOu0yZ8kKzc+94qH31zjq6bXX11NnZOWR1R8TSaXHvUY0tU2U3AT8FvlLK9wV+W0YpmwPd+aReDFxqe2OqHFJfBd4K7A582fazwBepRlCTbM8s5+3dMC13YB99edD2ZKpAeEgv+48HPifpz5K+KmnDXt7HNZLOWpwL0U3SNEldkrrmzZs3mKoiIqJBO6b+Xk+VEn4TqgSKx0taDjjbdnegepZq1ARwPbDA9sKSSHFiP+38n6m/XpxZvs8G9ui50/Y1ktYHdgJ2BK6S9Hrbf6e5qb+mlIzDM6DKR9WOOhtHJ+38wG9nPvAbEaPIoFf92f4zMB5Yw/YfgO2Be4ETJO1fDlvoFzI0Pg8sKOc+z+CnHxeU74v6qsv2fNtn2v4IcDLVFGRERIwCgw5UkjaiStf+kKT1qBYu/IRqSnByC1U9Aaw02P700r83SFq1bC8PvJaGe1YREVFvizuaGSupe1pPwAG2F5VFB5+VtBCYD+zfVwW9uAw4rNTb52KKxbAB8CNJogrMFwBnNHuypMuBjYBxku4BDrL92zb2LyIi+qEXZuSiXTo6OtzV1dXWOvNQ2ohYkkmabbujt315MkVERNRaAlVERNRa8lGNEpmui4ilVUZUERFRawlUERFRawlUERFRa7lHNUoszvL03NeKiCVBRlQREVFrCVQREVFrCVQREVFrCVQREVFrgwpUJXNv9/bOkm6RtJ6kg7tTfEiaKmntAeqZKunYwfSln7q3k3SlpJsk3SzpI4Osb/7AR0VERLu0ZdWfpB2A7wNvs30XcFzD7qnADcA/29FWi/1aEzgV2M321ZLGA7+VdJ/tQWX0jYiI4THoQCVpe+AnwM62by9l06nSfMwBOoBTJD0NvB7YBPgeVXr6BcAOpaq1JV1IlZbjLNuHlrp2Ar4ErADcDhxoe76kOcCJwC7AcsBetm/q0b2PAifYvhrA9oOSDgW+Apwl6QTgfNunl7bm2x4naRxwDrBqqftw2+cM9loN1txTD2vp+Cmzjm7p+MaMwhERdTHYe1QrAGdTjVh6BglKAOgC9isp3xcBM4FP2t6cKjX80+XwScDewKbA3pJeXkZAhwM72p5c6vpMQxMPlvIfAYf00r+NqVLUN+qiSp7Yn2eA3Uvdbwa+VfJZ9UnSNEldkrrmzZs3QPUREdGswY6oFgJ/Ag4CPtnE8a8G7rN9FYDtxwFKDLjE9mPl9d+A9YBVqILKH8sxywN/bqjvzPJ9NrDHIN9LIwFHltHi88A6wMuAuX2dYHsGMAOqfFRt7Mu/rLnvUS0d35kP/EbEEmCwI6rngfcAW0v6/CDrWtCwvYgqiAq42Pak8vVa2wf1ck738T39DdiyR9mWVKMqgOco10DSMlSBEGA/YA1gyzISvB8Ys1jvKiIiBmXQy9NtPwW8E9hP0kG9HPIEsFLZvhlYS9JWAJJWktTfqG4W8AZJryzHv1jSq1ro3g+AqZImlfNXB75GdY8Kqnto3YHs3VT3owBWBh6wvVDSm6lGdxERMQLasurP9sOS3g78QVLPGzQnAMc1LKbYGzhG0liq+1M79lPvPElTgdMkrVCKDwduabJf90l6HzBD0srARGCq7d+XQ34CnCPpWuBC4MlSfgpwnqTrqUZf/+f+W0REDA/ZQ3I7pZbKZ6g+DGxv+5Ghaqejo8NdXV0DH9iCPJQ2IpZkkmbb7uht31L1ZArbP7S96VAGqYiIaK+lKlBFRMTok3xUo0Sm8SJiaZURVURE1FoCVURE1FoCVURE1FruUY0SWZ4eEUurjKgiIqLWEqgiIqLWEqgiIqLWEqgiIqLWhiRQSVpT0i8k3S5ptqRft/jU877qnSLp/LL9bkmHle3dJL224bgvS+rzYbcttLe6pMskzZd07GDri4iI1rV91V/JhHsWcKLtfUrZ5lSJB5t66nkzbJ8LnFte7gacT5V/CttfbFMzzwD/DWxSviIiYpgNxYjqzcBC28d1F9i+1vblqhwt6QZJ10vaG/41UuqUdLqkmySd0p36XdLbS9nVNGTxlTRV0rGStqXKJXW0pGskbSDpBEl7luN2kPTX0t7x3elCJM2R9CVJV5d9G/V8I7aftH0FVcCKiIgRMBSfo9qEKjV8b/YAJgGbA+OBqyT9oezbAtgY+CfwR6qEiV1UOaPeAtwGzOxZoe0/SToXON/26fCv1PZIGkOVD2sH27dIOokqzcd3y+kP2p5c0n8cAnxgEO972M099bB+90+ZdXS/+zs7O9vYm4iIoTHciym2A06zvcj2/cDvga3Kvitt32P7eeAaqiSHGwF32r7VVeKsk1ts79Xl/O4pxxOB7Rv2n1m+zy7tLTZJ0yR1SeqaN69n7siIiFhcQzGiuhHYczHOW9CwvYjheWpGd5uDbs/2DGAGVIkTB9mvpqy571H97u/MkykiYgkwFCOqS4EVJE3rLpC0maQ3ApcDe0taVtIaVKObK/up6yZgoqQNyuv39nHcE8BKvZTfXM5/ZXn9n1SjuIiIGCXaHqjKFN3uwI5lefqNwNeBuVSrAa8DrqUKaIfanttPXc8A04ALymKKB/o49BfAZ8uiiQ16nH8g8CtJ1wPPA8f1UUevJM0Bvg1MlXRP4zL4iIgYeqriSrRTR0eHu7q62lpnHkobEUsySbNtd/S2L0+miIiIWkugioiIWkugioiIWkvixFEi95siYmmVEVVERNRaAlVERNRaAlVERNRa7lGNEs1+jir3siJiSZMRVURE1FoCVURE1FoCVURE1FoCVURE1NqwBSpJi0qq+O6vftPTSvp8G9qUpMMl3SrpFkm/l7TZIOqbKunYwfYrIiKaN5yr/p62PamF4z8PHNlKA5KWtb2ooeijwLbA5rafkrQTcK6kjW0/2UrdERExMkZ0ebqklakSJ77b9s2STqPKU7UBMFbSNcCNtveT9D7gE8DywF+Aj9heJGk+8GNgR6rAdEVDE58D3mT7KQDbF0m6HNgPmCFpvu1xpS97Au+yPVXSLsDhpa2HgP1s3z/El6Mpc0/tdyDKlFlHD1hHZ2dnm3oTETH0hvMe1dgeU397234M+BhwgqR9gFVt/8T2YZQRWAlSrwH2Bt5QRmWLqIINwIuBv9je3Pa/gpSklwAvtn1Hj350AQMlP7wC2Mb2FlRJGQ8d6M1JmiapS1LXvHnzBrwYERHRnBGf+rN9saS9gB8Am/dx7g7AlsBVkgDG8kK230XAGW3u67rATElrUY2q7hzoBNszgBlQJU5sc3/+Zc19j+p3f2c+8BsRS5gRX/UnaRngNcBTwKp9HQacWEZYk2y/2vb0su+ZHvelALD9OPCkpPV77NqSalQF0BhQxjRsHwMca3tT4EM99kVExDAa8UAFfBr4O7Av8DNJy5XyhQ3blwB7SnopgKTVJK3XRN1HA9+XNLactyOwMXB62X+/pNeUYLl7w3krA/eW7QMW831FREQbDOfUX/fiiG4XAj8DPgBsbfsJSX+gWsRwBNU02nWSri73qQ4HLipBZSHVwom7BmjzGGCVUs9yVNN4m9h+puw/DDgfmEc1yhpXyqcDv5L0CNXijlcM4n1HRMQgyB6y2ym1ImkccGZR8ecAAAgDSURBVBZwle1Bf0arPx0dHe7q6hr4wBbkobQRsSSTNNt2R2/7lpqnp9ueD7x1pPsRERGtqcM9qoiIiD4tNSOq0S5TehGxtMqIKiIiai2BKiIiai2BKiIiai33qEaJZpan5z5WRCyJMqKKiIhaS6CKiIhaS6CKiIhaS6CKiIhaW+xAJWldSedIulXS7ZK+J2n5dnauj3bnl+8TJd3QxzGvkvTr0rerJf1S0sva0PZUSccOtp6IiGjeYgUqVdkLzwTOtr0h8CqqJ49/bbAdkjSolYiSxgAXAD+yvaHtycAPgTUG27eIiBh+ixsU3kKVsPBnALYXSfo0cKekI6hSYxxk+0YASZ3AIVR5p44BNgGWA6bbPkfSVGAPqmC3rKR3AudQJVJcDjjc9jlN9m1f4M+2z+susN1Z+jEG+BHQATwHfMb2ZaX9dwMrAhsAZ9k+tJxzIPD/gEeBa4EFLV2pFk2ZMqXX8rl3PDTwubOOXqw2Ozs7F+u8iIjhsLiBamNgdmOB7ccl/QN4JTATeA9wREnnvpbtLklHApfafr+kVYArJf2uVDEZ2Mz2w2VUtXupczwwS9K5bi4nySY9+9bgo1VXvamkjajyW72q7JsEbEEViG6WdAxVMPsSVVbgx4DLgL/2VrGkacA0gAkTJjTRzYiIaMZQfeD3l8BFVAkQ38MLGXV3At4t6ZDyegzQ/Vf9YtsPl20BR0raHngeWAd4GTB3kP3ajmpEh+2bJN1FNW0JcIntxwAk/Q1YDxgPdNqeV8pnNhz/b2zPoEr2SEdHx2In+eprdNPMB34784HfiFgCLW6g+huwZ2OBpJdQBZ3bbD8l6SFJmwF7Awd3Hwb8h+2be5z7OuDJhqL9qO4pbWl7oaQ5VEGtGTcCb2rx/cC/T+ktIk/tiIiohcVd9XcJsKKk/QEkLQt8CzjB9lPlmJnAocDKtq8rZb8FPl4WYyBpiz7qXxl4oASpN1ONbpp1KrBtuc9FaWd7SZsAl1MFQcqU3wTg5l5rqfwFeJOk1Usq+71a6EdERLTBYgWqcq9od2AvSbcCtwDPAI0p3k8H9qGaBuz2FarFEddJurG87s0pQIek64H9gZta6NvTwLuoAuKtZRrvI8A8qtV/y5R6ZwJTbfe5OML2fcB04M/AH6kWg0RExDBSc+sTohUdHR3u6upqa515KG1ELMkkzbbd0du+PJkiIiJqLYEqIiJqLSvbRolM60XE0iojqoiIqLUEqoiIqLUEqoiIqLUEqoiIqLUEqoiIqLUEqoiIqLUEqoiIqLUEqoiIqLUEqoiIqLU8lHYISJoH3NXmascDD7a5ziVJrs/Aco36l+szsKG8RuvZXqO3HQlUo4Skrr6eLBy5Ps3INepfrs/ARuoaZeovIiJqLYEqIiJqLYFq9Jgx0h2ouVyfgeUa9S/XZ2Ajco1yjyoiImotI6qIiKi1BKqIiKi1BKqak/R2STdLuk3SYSPdn+EmaY6k6yVdI6mrlK0m6WJJt5bvq5ZySfp+uVbXSZrcUM8B5fhbJR0wUu9nsCQdL+kBSTc0lLXtekjaslzv28q5Gt53OHh9XKPpku4tv0fXSNq5Yd//K+/3Zklvayjv9d+epFdI+kspnylp+eF7d4Mn6eWSLpP0N0k3SvpkKa/v75HtfNX0C1gWuB1YH1geuBZ47Uj3a5ivwRxgfI+ybwCHle3DgP8p2zsDvwEEbAP8pZSvBtxRvq9atlcd6fe2mNdje2AycMNQXA/gynKsyrnvGOn33KZrNB04pJdjX1v+Xa0AvKL8e1u2v397wC+Bfcr2ccCHR/o9t3h91gIml+2VgFvKdajt71FGVPW2NXCb7TtsPwv8Ath1hPtUB7sCJ5btE4HdGspPcmUWsIqktYC3ARfbftj2I8DFwNuHu9PtYPsPwMM9ittyPcq+l9ie5eqvzUkNdY0afVyjvuwK/ML2Att3ArdR/bvr9d9eGRm8BTi9nN94vUcF2/fZvrpsPwH8HViHGv8eJVDV2zrA3Q2v7yllSxMDF0maLWlaKXuZ7fvK9lzgZWW7r+u1pF/Hdl2Pdcp2z/IlxcfK1NXx3dNatH6NVgcetf1cj/JRSdJEYAvgL9T49yiBKupuO9uTgXcAH5W0fePO8j+2fMaiyPXo04+ADYBJwH3At0a2OyNP0jjgDOBTth9v3Fe336MEqnq7F3h5w+t1S9lSw/a95fsDwFlUUzL3l+kFyvcHyuF9Xa8l/Tq263rcW7Z7lo96tu+3vcj288BPqH6PoPVr9BDV1NeLepSPKpKWowpSp9g+sxTX9vcogarergI2LKuMlgf2Ac4d4T4NG0kvlrRS9zawE3AD1TXoXmF0AHBO2T4X2L+sUtoGeKxMZfwW2EnSqmXKZ6dStqRoy/Uo+x6XtE25F7N/Q12jWvcf4GJ3qt8jqK7RPpJWkPQKYEOqhQC9/tsrI43LgD3L+Y3Xe1QoP9v/Bf5u+9sNu+r7ezTSK1DyNeAKnZ2pVuXcDnxhpPszzO99farVVtcCN3a/f6r7BJcAtwK/A1Yr5QJ+UK7V9UBHQ13vp7pRfhtw4Ei/t0Fck9Oopq4WUs39H9TO6wF0UP0Rvx04lvL0mtH01cc1+nm5BtdR/eFdq+H4L5T3ezMNq9P6+rdXfi+vLNfuV8AKI/2eW7w+21FN610HXFO+dq7z71EeoRQREbWWqb+IiKi1BKqIiKi1BKqIiKi1BKqIiKi1BKqIiKi1BKqIiKi1BKqIiKi1/w9EHqDvIuhjwAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plot_importances(100, 1, 'feature_importance_rand_ci.png')"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 81/81 [00:30<00:00, 2.76it/s]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAbcAAAEICAYAAADcC7pLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de5xd0/3/8ddbyIVIXJLKiMvQRhUhIu6Xhmq17rSK6FeCflNKVVuXVL8tVVSll2gpTf0UJRqCoFp3U9eIJEJcEtcoqYhriEtE8vn9sddwMj0zOWfmzJwzO+/n4zGP2Wftvdf6nJXMfGatvfdZigjMzMzyZIVqB2BmZlZpTm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mrSRptqQPJC0o+Fq7jXUOlfRypWIssc1LJZ3ZkW02R9Lpkq6odhzW+Tm5mbXNPhHRs+DrP9UMRtKK1Wy/LTpz7FZ7nNzM2oGk7SQ9IOltSY9KGlqw7whJT0l6V9Lzkr6TylcB/gmsXTgSbDqyajq6SyPIUyQ9BrwnacV03rWSXpP0gqTjS4y7XlKkGF+S9JakoyVtLemx9H7OLzh+hKT7JZ0vab6kmZK+VLB/bUk3SnpT0rOS/rdg3+mSJki6QtI7wNHAqcDB6b0/2lJ/FfaFpB9JmifpFUlHFOzvIek3kl5M8d0nqcey/o2s8/NfSmYVJqk/cDPwP8AtwJeAayVtHBGvAfOAvYHngV2Af0p6OCKmSfoacEVErFNQXynNHgrsBbwOLAFuAm5I5esAd0iaFRG3lvg2tgUGpPhuTO9jd2Al4BFJ10TEvwqOnQD0AQ4ErpO0QUS8CfwNeBxYG9gYuF3ScxFxVzp3P+Ag4HCgW6rjcxHxrYJYmu2vtL8f0BvoD3wZmCBpYkS8Bfwa2BTYAZibYl1Swr+RdXIeuZm1zcT0l//bkiamsm8B/4iIf0TEkoi4HZgC7AkQETdHxHOR+RdwG7BzG+P4fUS8FBEfAFsDfSPijIj4KCKeB/4MHFJGfb+IiA8j4jbgPeCqiJgXEXOAe4EtC46dB4yJiEURMR6YBewlaV1gR+CUVNd04GKyRNbowYiYmPrpg2KBlNBfi4AzUvv/ABYAn5e0AnAk8P2ImBMRiyPigYhYyDL+jazz88jNrG32j4g7mpStDxwkaZ+CspWAuwHS6Ow0YCOyPzBXBma0MY6XmrS/tqS3C8q6kCWlUr1asP1Bkdc9C17PiaU/gf1FspHa2sCbEfFuk31Dmom7qBL6642I+Ljg9fspvj5Ad+C5ItW2+G9knZ+Tm1nlvQT8NSL+t+kOSd2Aa8lGLzdExKI04muceyy2TMd7ZL/QG/UrckzheS8BL0TEgNYE3wr9Jakgwa1HNpX5H2ANSasWJLj1gDkF5zZ9v0u9LqG/WvI68CHwWeDRJvua/TeyfPC0pFnlXQHsI2kPSV0kdU83PqwDdCW7tvQa8HEalXyl4NxXgTUl9S4omw7sKWkNSf2AE5bR/mTg3XSTSY8Uw2aStq7YO1zaZ4DjJa0k6SDgC2RTfi8BDwC/TH2wOXAUWf8051WgPk0pwrL7q1kRsQS4BPhturGli6TtU8Js6d/IcsDJzazC0i/1/cju/HuNbJRwErBCGsEcD1wNvAUMIxvlNJ47E7gKeD5dx1sb+CvZyGM22fWm8ctofzHZDRiDgBfIRjAXk9100R4eIrv55HXgLOAbEfFG2ncoUE82irseOK3ING6ha9L3NyRNW1Z/leBEsinMh4E3gV+R/Ts0+29URt1Ww+TFSs2stSSNAL4dETtVOxazQv4rxczMcsfJzczMcsfTkmZmljseuZmZWe74Obca0adPn6ivr692GGZmncbUqVNfj4i+xfY5udWI+vp6pkyZUu0wzMw6DUkvNrfP05JmZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7foi7RsyYM5/6UTdXOwwzy4nZ5+xV7RCqyiM3MzPLHSc3MzPLHSc3MzPLHSc3MzPLnZpLbpLWkjRO0vOSpkp6UNIBJZ67oEjZ0ZIOLzOGFSW9Jumccs4zM7PaUFPJTZKAicA9EbFhRGwFHAKsU+TYku70jIiLIuLyMkP5MvA0cFCKqVisXcqs08zMOkitPQqwG/BRRFzUWBARLwJ/AJA0AjgQ6Al0Ab64rAolnQ4sAP4OXB4R26TyeuCmiBhY5LRDgfOAY4DtgQfSObOB8WTJ71xJDwMXAH2B94H/jYiZkvYB/g/oCrwBHBYRr5bcC2ZWs+aOG1XtEEoydNLoaodQkoaGhnapt9aS26bAtGUcMxjYPCLeLKfilHS6StogIl4ADiZLVEuR1B3YHfgOsBpZonug4JA3ImJwOvZO4OiIeEbStsAfyRL0fcB2ERGSvg2cDPyoSFsjgZEAXXoVXUzWzMxaodaS21IkXQDsRDaa2zoV315uYitwNVlSOyd9P7jIMXsDd0fEB5KuBX4q6YSIWJz2j0+x9QR2AK4pmLnslr6vA4yXVEc2enuhWDARMRYYC9CtbkC08j2ZWQfqN6xzXIpv8EPcNeUJspEZABFxLPAlsmm/Ru+1of7xwDclbZRVH88UOeZQYPc0BTkVWJNsNNa0/RWAtyNiUMHXF9K+PwDnpynP7wDd2xCzmZmVqdaS211Ad0nHFJStXKnKI+I5YDHwU4pPSfYCdgbWi4j6iKgHjiVLeE3regd4QdJB6VxJ2iLt7g3MSdvDKxW/mZmVpqaSW0QEsD/wRUkvSJoMXAacUmIVK0t6ueDrh0WOGQ98i2yKsqkDgLsiYmFB2Q3APpK6FTn+MOAoSY+SjTr3S+Wnk01XTgVeLzF2MzOrEGX5xKqtW92AqBs+ptphmFlOLA8fnCxpakQMKbavpkZuZmZmleDkZmZmuVPTjwIsTwb2782U5WAawcysI3jkZmZmuePkZmZmuePkZmZmueNrbjVixpz51I+6udphmFXF8nDbunUsj9zMzCx3nNzMzCx3nNzMzCx3nNzMzCx3qpbcJK0j6QZJz0h6TtJ5krp2QLsL0vd6SY83c8ymku6SNCvF9nNJre4rSbMl9Wnt+WZmVp6qJDdlq3teB0yMiAHARkBP4KwK1N2mO0Al9QBuBM6JiM8DA4FtgO+3NTYzM+sY1XoUYDfgw4j4C0BELJb0A7L10U4jW9ftqIh4AkBSA3Ai8BTZQqCbASsBp0fEDZJGAAeSJcgukvYiW6pm9XTc/0XEDSXGNgy4PyJuS7G9L+k44F7gd5JOBxZExK9TbI8De0fEbEkTgXXJFic9L620bVZz5o4bVe0QljJ00uhqh/CJhoaGaodgFVCt5LYp2SrXn4iIdyT9G/gcacVs4DRJdUBdREyRdDbZemtHSloNmCzpjlTFYGDziHgzjd4OSHX2ASZJujFKW9+nWGzPSeqR2mzJkan9HsDDkq6NiDeaO1jSSGAkQJdefZs7zMzMylSrD3FfDdwGnEaW5Cak8q8A+0o6Mb3uDqyXtm+PiDfTtoCzJe0CLAH6A2sBc9s57uMlHZC21wUGAM0mtzSyGwvZem7tHJvZJ/oNO6faISylwQ9xW4VVK7k9CXyjsEBSL7JE9WyaCnxD0ubAwcDRjYcBX4+IWU3O3RZ4r6DoMKAvsFVELJI0mywRlhrbLk3q3xB4IyLelvQxS1+r7J6OGQrsDmyf4m8oo00zM6ugat0teSewsqTDASR1AX4DXBoR76djxgMnA70j4rFUdivwvXRDCpK2bKb+3sC8lNh2BdYvI7YrgZ0k7Z7a6AH8nmwUCTCbbAoUSYOBDQrafCslto2B7cpo08zMKqgqyS1d+zoAOEjSM8DTwIfAqQWHTQAOIZuibPQLshtEHpP0RHpdzJXAEEkzgMOBmWXE9gGwL/ATSU8Dr5PdYHJlOuRaYI3U/nEpdoBbgBUlPQWcA0wqtU0zM6sslXaPxfJL0v7Ab4FdI+LF9mqnW92AqBs+pr2qN6tp/uBkaw1JUyNiSLF9/oSSZYiIiRGxYXsmNjMzqywnNzMzyx0nNzMzy51afc5tuTOwf2+m+LqDmVlFeORmZma54+RmZma54+RmZma542tuNWLGnPnUj7q52mFUlZ91MrNK8cjNzMxyx8nNzMxyx8nNzMxyp12Sm6SQ9JuC1yemFaxbOmdfSS0uDyxpqKS/N7NvdlqYtFUknV6wTlzFtFe9ZmbWvPYauS0EDiwn2UTEjRFRlRUU08rdZmaWE+2V3D4mW2H6B013SOor6VpJD6evHVP5CEnnp+3PSpokaYakMyUtKKiip6QJkmZKurJxbbfk5HTOZEmfS3XVS7pL0mOS7pS0Xiq/VNJFkh4Czk3nbyKpQdLzko4viPmHkh5PXyeUUP4TSU9Lug/4fBv70szMytSeI5YLyNZdO7dJ+XnA7yLivpRobgW+UOSY8yLiKklHN9m3JbAp8B/gfmBH4L60b35EDEyLoI4B9gb+AFwWEZdJOpJs4dH90/HrADtExOI0bboxsCuwKjBL0oXA5sARwLZkK4E/JOlfZH8YNFd+CDCIrH+nAVPL6LdOZ+64FmeTSzZ00uiK1NPQ0FCResys82q35BYR70i6HDge+KBg1+5kI6TG170k9Wxy+vZ8moDGAb8u2Dc5Il4GkDQdqOfT5HZVwfffFdR1YNr+K5+O0gCuiYjFBa9vjoiFwEJJ84C1gJ2A6yPivdTmdcDOZAmtWPkKqfz9VH5j0Q7K9o0ERgJ06dW3ucPMzKxM7X2taQzZyOUvBWUrANtFxIeFBy49u9iihQXbi1n6PUQz2815r4y6Ky4ixpJN39KtbkCnXTW237DKXCpt8EPcZlYh7fooQES8CVwNHFVQfBvwvcYXkgYVOXUS8PW0fUgZTR5c8P3BtP1AQR2HAfeWUR/p+P0lrSxpFeCAVNZc+T2pvIekVYF9ymzPzMzaqCPuEvwNcFzB6+OBCyQ9ltq/B2h6Xe0E4ApJPwFuAeaX2Nbqqd6FwKGp7HvAXySdBLxGdp2sZBExTdKlwORUdHFEPALZTSnNlI8HHgXmAQ+X056ZmbWdImpvNkzSysAHERGSDgEOjYj9qh1Xe+pWNyDqho+pdhhV5c+WNLNySJoaEUOK7avV57u2As5Pt/m/DRxZ5XjMzKwTqcnkFhH3AltUOw4zM+uc/NmSZmaWOzU5clseDezfmym+5mRmVhEeuZmZWe44uZmZWe44uZmZWe74mluNmDFnPvWjbq52GC3yc2hm1ll45GZmZrnj5GZmZrnj5GZmZrnj5GZmZrnTaZKbMvdJ+lpB2UGSbqlwO2dKmiNpuqSnJF0gqcV+krStpN+l7W9LGpO2D5S0cSXjMzOzZes0yS2y5QuOBn4rqXtavfts4Ni21Cup2B2joyNiELAZ2Yc477iM2B6KiB8U2XUg4ORmZtbBOtWjABHxuKSbgFOAVYDLI+I5ScPJklxXssVJj4uIJZLGAoOBHsD4iDgDQNLLwBXAHmQJ8ppmmuwKdCNbmQBJ96W6p0vqB9wXEZ+TtHsq37/xREk7A3sCO0o6Hdg/ImZXsDsqYu64USUfO3TS6LLqbmhoKDMaM7PK6FTJLfk5MA34CBgiaTOyVbB3iIiPU0I7BBgHjIqIN9Po7G5JEyLiyVTPvIjYspk2TpI0AlgfuCkiZpQbZETcK+kfwISImFjsGEkjgZEAXXr1LbcJMzNrRqdLbhHxXlrpekFELEyjpq2BKdnyb/QAXkqHHyrpKLL3uTawCdCY3Ma30MzoiBgjqStwvaRvRMSEdngvY4GxkC1WWun6S9Fv2DklH9vgh7jNrJPodMktWZK+AARcEhE/LTxA0gDg+8A2EfG2pCuA7gWHvLesRiLio3TDyi7ABOBjPr1O2b3ZE83MrKo6zQ0lLbgD+KakPgCS1pS0HtALeBd4R1Id2fW1sqSVwHcAnktFs8luMAH4RglVvAusWm67ZmbWNp0+uaXrYT8H7pD0GHAbsBbZdbkngZnA5cD9ZVR7kqTpwOPAYuBPqXw08H1J04DVS6jnKuDU9FhBfRntm5lZGyi7w96qrVvdgKgbPqbaYbTIH5xsZrVE0tSIGFJsX6cfuZmZmTXl5GZmZrnTWe+WzJ2B/XszxdN+ZmYV4ZGbmZnljpObmZnljpObmZnljq+51YgZc+ZTP+rmaofxCd/2b2admUduZmaWO05uZmaWO05uZmaWO05uZmaWO+12Q4mkNYE708t+ZB9A/Fp6vU1EfNTk+DWAb0bERcuod0Xg9YhYrUj5QqBxYdHFwLERMamN72Mw8JmIuKXIvp7AxcCmZEvvvEW2+sBHTWIB2CciXmpah5mZVV67JbeIeAMYBCDpdLLFRX/dwilrAEcDLSa3ZXg3Ihrb3As4C/hSG+oDGAxsBvxXcgN+APw7Ig5JbW4MLGoai5mZdayqPAog6WTg8PTyTxHxB+Ac4PNpqZlbgLOBicBqKc5TI+LvZTTTi2wkhaT+ZCtv90x1jQQmA68Dl5CNtl4GfgacC6wLHAfcncp6SBoKnNlkRe46YFbji4iYmdqrqUcs5o4bVfY5QyeNLvuchoaGss8xM2sPHf5LWNK2wGHA1qn9yZIagFHA5wpGXisB+0fEO5I+Q7Ye27KS26opOXYnmwrdNZV/C7gpIn4lqQvQI5X3Bv4ZET+UdBNwOtlIbwuypDtE0hnAZhFxQpH2/h9wi6SDyaZgL4uIZ5vEAvBsRPzX4qaSRpIlWrr06ruMt2ZmZqWqxghjJ+DaiPgAQNJEYGeyRUYLCThH0k7AEmDdtNr22y3UXTgtuRPZIqUDgYeBP0nqDkyMiEfT6OqDiLg9nTsDmB8RH0uaAdQv641ExFRJGwJfAXYHpkjaBnieEqYlI2IsMBay9dyW1V5r9Rt2TtnnNPghbjPrxGr5bsnDyUZWg1OSeJ1sRFaSiLgPWFvSGhFxFzAUeAW4XNJh6bDCm1qWkN0E0rhdUuKPiHcj4tqIOAb4G/C1UmM0M7P2UY3kdi9wgKQe6W7D/VLZu8CqBcf1BualkdSXgf7lNCJpU7Ik9Zak9YG5aaT0F2DLMqpqGldhGztJWi1tdwO+ALxYTpxmZlZ5HT4tGRGTJV1FNlUIcGFEzIBsyfA0JXgz8FvgpvR6MvBMCdUXXucCODwiQtKXgB9KWkSWrP6njJDvAk6S9AhwVpMbSgYAF0qC7A+Fm4AbgC5l1G9mZhWmiHa71GNl6FY3IOqGj6l2GJ/wByebWa2TNDUihhTbV8vX3MzMzFrFyc3MzHKnph42Xp4N7N+bKZ4KNDOrCI/czMwsd5zczMwsd5zczMwsd3zNrUbMmDOf+lE3d1h7vtXfzPLMIzczM8sdJzczM8sdJzczM8sdJzczM8udDktukhZLmi7pUUnTJO1QgToHSdqz4PUISa+ldqZLujyVnyFp9zLqXVnSlZJmSHpc0n1pBYPC99H4VV/k/FskvS2pnJXDzcysQjrybskPChYS3QP4JfDFNtY5CBgC/KOgbHxEHFd4UET8rMx6vw+8GhEDASR9HliU9n2wrEVIgdHAysB3ymzXzMwqoFqPAvQC3gKQVAeMT2UrAsdExL2SFgAXAnuSLTJ6KnAusB5wAnALcAbQI626/cvmGpN0KfD3iJggaTZwGbAPsBJwUETMbHJKHQXrskXErHLeXETcKWloOee01dxxo8o6fuik0WW30dDQUPY5ZmbV0JHX3HqkabyZwMXAL1L5MODWNBraAmhcj20V4K6I2JRsDbYzgS8DBwBnRMRHwM/IRmqDImJ8Ou/gginDI5qJ5fWIGEyWPE8ssv8S4BRJD0o6U9KAIu9juqTrW9MRjSSNlDRF0pTF789vS1VmZlagWtOS2wOXS9qMbNHSSyStBEyMiMbk9hHZ6AxgBrAwIhalxUvrW2jnv6Yli7gufZ8KHNh0Z0RMl7Qh8BVgd+BhSdtHxFOUNi1ZkrQy+FjI1nNrS139hp1T1vENfojbzHKsKndLRsSDQB+gb0TcA+wCzAEulXR4OmxRfLqS6hJgYTp3CW1PygvT98XN1RURCyLiuoj4LnAF2fSomZl1AlVJbpI2BroAb0han+zmjT+TTVcOLqOqd4FV2yG+HSWtnra7AptQcA3OzMxqW0dOS/aQ1DjlKGB4RCxON16cJGkRsAA4vLkKirgbGJXqbfaGklb4LHChJJH9AXAzcG2pJ0u6F9gY6CnpZeCoiLi1gvGZmVkL9OnMn1VTt7oBUTd8TIe15w9ONrPOTtLUiBhSbJ8/ocTMzHLHyc3MzHLH67nViIH9ezPFU4VmZhXhkZuZmeWOk5uZmeWOk5uZmeWOr7nViBlz5lM/6uZ2qdu3/ZvZ8sYjNzMzyx0nNzMzyx0nNzMzyx0nNzMzy50OTW6S1pF0g6RnJD0n6bz0qfvt3e6C9L1e0uPNHLORpH+k2KZJulrSWhVoe4Sk89taj5mZla7Dklv6hP3ryBYkHQBsBPQEzqpA3W2661NSd7JP/r8wIgakVbr/CPRta2xmZtbxOvJRgN2ADyPiLwBpuZsfAC9IOg24i2xpmCcAJDUAJwJPAX8ANgNWAk6PiBskjSBbRbsn0EXSXsANwOrpuP+LiBtKjG0Y8GBE3NRYEBENKY7uwIXAEOBj4IcRcXdqf19gZbIlcq6PiJPTOUcAPwbeBh7l08VRK2ruuFElHTd00uhlHtPQ0NDGaMzMakdHJrdNgamFBRHxjqR/A58DxgPfBE6TVAfURcQUSWcDd0XEkZJWAyZLuiNVMRjYPCLeTKO3A1KdfYBJkm6M0tb02axpbAWOzUKNgWmR1dskbZT2DQK2JEtesyT9gSwB/hzYCphPtubcI8UqljQSGAnQpZcHiWZmlVJLD3FfDdwGnEaW5Cak8q8A+0o6Mb3uDqyXtm+PiDfTtoCzJe0CLAH6A2sBc9sY105kI0ciYqakF8mmVAHujIj5AJKeBNYH+gANEfFaKh9fcPxSImIsMBay9dzKDazfsHNKOq7BD3Gb2XKmI5Pbk8A3Cgsk9SJLVM9GxPuS3pC0OXAwcHTjYcDXI2JWk3O3Bd4rKDqM7BrZVhGxSNJsskRYiieAL5b5fmDp6cbF1NYfC2Zmy62OvFvyTmBlSYcDSOoC/Aa4NCLeT8eMB04GekfEY6nsVuB76YYUJG3ZTP29gXkpse1KNooq1Thgh3TdjtTOLpI2A+4lS5yk6cj1gFlFa8k8BHxR0pqSVgIOKiMOMzOrgA5Lbuna1wHAQZKeAZ4GPgROLThsAnAI2RRlo1+Q3SDymKQn0utirgSGSJoBHA7MLCO2D4C9yZLoM2mK8bvAa2R3Ta6Q6h0PjIiIZm8QiYhXgNOBB4H7yW6IMTOzDqTS7rew9tatbkDUDR/TLnX7g5PNLI8kTY2IIcX2+RNKzMwsd5zczMwsd3x3X40Y2L83Uzx9aGZWER65mZlZ7ji5mZlZ7ji5mZlZ7viaW42YMWc+9aNuLusc3+JvZlacR25mZpY7Tm5mZpY7Tm5mZpY7Tm5mZpY7NZ/cJC2WNF3So5KmSdqhAnUOkrRnC/t3kjRZ0kxJsyR9t43tLWjL+WZmVp7OcLfkBxExCEDSHsAvad3aa4UGAUOAfzTdIakf2RI4+0fEtLSq962SXomI69vYrpmZdYCaH7k10Qt4C0BSnaR70qjucUk7p/IFkkZLekLSHZK2kdQg6XlJ+0rqCpwBHJzOPbhJG8eSrTE3DSAiXidbY+6kVP+lkj5ZdLVxVCapp6Q70+hyhqT92rkvzMysGZ1h5NZD0nSyVbXrgN1S+TDg1og4Ky18unIqXwW4KyJOknQ9cCbwZWAT4LKIuFHSz4AhEXFckfY2BS5rUjYlnd+SD4EDIuKdNNqbJOnGaMOaQnPHjWpx/9BJo1vc39DQ0Nqmzcw6tc6Q3AqnJbcHLk8rZD8MXJJWu54YEdPT8R8Bt6TtGcDCtDr3DKC+HeMUcLakXYAlQH9gLWBusydII4GRAF169W3H0MzMli+dIbl9IiIeTKOivhFxT0okewGXSvptRFwOLCoYLS0BFqZzl0gq5f0+CWwF3FBQthXZ6A3gY9J0rqQVgK6p/DCgL7BVSqazyUabLb2fscBYyBYrbbq/37BzWgy0wZ9QYmZWVKe65iZpY6AL8Iak9YFXI+LPwMXA4DKqehdYtZl9FwAjJDWOFtcEzgJ+kfbPJkt2APsCK6Xt3sC8lNh2BdYvIx4zM6ugzjBya7zmBtnU3/CIWCxpKHCSpEXAAuDwMuq8GxiV6v1lRIxv3BERr0j6FjBWUm+yqcwREfGvdMifgRskPUo2/fleKr8SuClNf04BZrbivZqZWQWoDfc7LBfSM27HALtExFvt1U63ugFRN3xMWef4g5PNbHkmaWpEDCm2r1NNS1ZDRPwxIga2Z2IzM7PKcnIzM7PccXIzM7Pc6Qw3lCwXBvbvzRRfQzMzqwiP3MzMLHec3MzMLHec3MzMLHd8za1GzJgzn/pRN5d1jp9zMzMrziM3MzPLHSc3MzPLHSc3MzPLHSc3MzPLnRaTmzL3SfpaQdlBkm5p6bxySTpT0hxJ0yU9Kemblay/hPa/LWlMwet1JP2z4PWJkt6X1NwyOaR+GrSsus3MrP21mNzSop9HA7+V1F1ST+Bs4Ni2NNrMoqGj04rbBwJ/ltSlLW200df4dDVvgEOBqcD+1QnHzMzKscxHASLicUk3AacAqwCXR8RzkoaTJbmuwAPAcWm167FkC4f2AMZHxBkAkl4GrgD2IEuQ1zTT3sy0Rltv4E1JA4DzgT5ka6d9OyKelnQFMB8YkvYdARwFbAvcHxFHpXa/lWIXcGNEnJrKvw2cDLwNPEa2JlyjrwI/TsdtlPrpdOBHwF9T+crAZcBmZKt3f7Lq9jLqLtvccaOKlg+dNLpoeUNDQ1uaMzPr9Ep9zu3nwDTgI2CIpM2AA4AdIuLjlNAOAcYBoyLizTQ6u1vShIh4MtUzLyK2bKkhSVsDj0fEm6loLFlCe07SjmSJ7itpX++I2FbS14GbgO3JFgmdlmJ8GziTLAHOB+6QtDcwHfgpWRJ+F7gHmJTaXxHYMCKeTm0cCvwNaAD+IqlPRLwOHAe8FRFfkLQl2QKlSFqnubqLvBKAJi8AAAvfSURBVNeRwEiALr36ttQtZmZWhpKSW0S8J2k8sCAiFkraHdgamCIJslHaS+nwQyUdlepeG9iEbGQDMJ7mnZR+2Q8A9gSQtBqwHXBtaqdpzDel7zOA/zQmUUlPkq2g3Q24KyUjJI0DdiEbZd0ZEW+k8quB9VJdO5CNRBsdAuyZVv+eCHwDuCjVc27qn0ckPZGO366FupcSEWPJkjfd6gY0u2psv2HnFC1v8EPcZmZFlfMJJUvSF2RTfJdExE8LD0hTiN8HtomIt9PUYfeCQ95rof7RETFG0oHA/0t1CXg9XYsrZmFBbAsLypfQ+k9f+eR6WxqRbUg2AoUsWT5NltzMzKxGtfZRgDuAb0rqAyBpTUnrAb3IpuLekVRHdn2tLBFxHdlI7Ftp9etXJB2Q2llB0hZlVPcQsGuKb0WyUdi/yKYJd5O0hqSuZKOxRrsCd6XtQ4H/i4j6iKgnG4lukKYe7wGGpbi2ADZN57RUt5mZdYBWjW4iYoakn5Ndw1oBWER2V+UUsinImcCLwP2tjOsMsutbl5AlpAslnU5288oVwKMlxvmypJ+SXS8TcFNE3AzZ4wdkiegtsmSKpH7AO2kaVsDBwJcK6os0NXkw2bW/yyQ9BTwBPFLQ5n/VbWZmHUfZ3f4GIGkE0Ccift3RbXerGxB1w8t7HM4fnGxmyzNJUyNiSLF9XhWgQERcWu0YzMys7fzxW2ZmljseudWIgf17M8XTjGZmFeGRm5mZ5Y6Tm5mZ5Y6Tm5mZ5Y6vudWIGXPmUz/q5pKP92MAZmbN88jNzMxyx8nNzMxyx8nNzMxyx8nNzMxyp2rJTdJaksZJel7SVEkPNn76fy2QtI2keyTNkvSIpIvT6ttmZlbjqpLc0ifuTwTuiYgNI2Irsk//X6eMOtrtTk9JawHXAKdExOfT6uG3AKtWOzYzM1u2av0S3g34KCI+WfQzIl4E/gAgqR74K7BK2n1cRDwgaSjwC7KlZDYGNkpL0KxLtijqeWl1a9Jq4KcAb5MtkbMwIo6T1JdssdHG1bFPiIimS/McC1wWEQ8WxDch1bsNcF5q7wPgiIiYlVYUOBDoCXSRdAjZyuO9yPr5mIi4t9QOmjtuVIv7h04a3ey+hoaGUpsxM8ulaiW3TYFpLeyfB3w5Ij5MK3JfBTQuazAY2CwiXkivj4yINyX1AB6WdC3Zitk/Tce+S7b4aOMacOcBv4uI+9ICq7cCX2jS/mbAZc3ENhPYOSI+lrQ7cDbw9YLYNk/x/Ai4NSLOktQF+K8pTUkjgZEAXXr1baE7zMysHDUxfSbpAmAnstHc1sBKwPmSBgGLgY0KDp9ckNgAji+4VrcuMADoB/wrIt5M9V9TUMfuwCbZzCgAvST1jIgFJYbbm2yR0gFApFgb3d7YJvAwcImklYCJETG9aUVplDkWsvXcCvf1G3ZOi0E0+CFuM7NmVeuGkifIRjkARMSxZCteNw5ffgC8CmxBNmLrWnDue40baZpyd2D7iNiCbDXs7stoewVgu4gYlL76F0lsTwBbNXP+L4C7I2IzYJ8m7X0SW0TcA+wCzAEulXT4MuIyM7MKqVZyuwvoLumYgrLCabvewCsRsQT4H6BLM/X0Bt6KiPclbQxsl8ofBr4oafV0c8fXC865Dfhe44s0OmzqfGC4pG0Ljjsw3WjSmyxhAYxo7g1KWh94NSL+DFxMQTI3M7P2VZXkFhEB7E+WgF6QNJnsGtcp6ZA/kiWXR8luHHmveE3cAqwo6SngHGBSqn8O2bWwycD9wGxgfjrneGCIpMckPQkcXSS+V8nu3vx1ehTgKWAPsut35wK/lPQILU/rDgUeTccdTHatz8zMOoCyPJM/jdfR0sjteuCSiLi+2nE1p1vdgKgbPqbk4/3ByWa2vJM0NSKGFNuX508oOV3SdOBx4AWy5+rMzGw5UBN3S7aHiDix2jGYmVl15Da5dTYD+/dmiqcazcwqIs/TkmZmtpxycjMzs9xxcjMzs9xxcjMzs9xxcjMzs9xxcjMzs9xxcjMzs9xxcjMzs9xxcjMzs9zJ7QcndzaS3gVmVTuOIvoAr1c7iCJqMa5ajAkcV7kcV3mqGdf6EdG32A5//FbtmNXcp1tXk6Qpjqs0tRgTOK5yOa7y1GpcnpY0M7PccXIzM7PccXKrHWOrHUAzHFfpajEmcFzlclzlqcm4fEOJmZnljkduZmaWO05uZmaWO05uVSbpq5JmSXpW0qgOaG9dSXdLelLSE5K+n8rXkHS7pGfS99VTuST9PsX3mKTBBXUNT8c/I2l4BWLrIukRSX9PrzeQ9FBqe7ykrqm8W3r9bNpfX1DHj1P5LEl7tDWmVOdqkiZIminpKUnb10h//SD9Gz4u6SpJ3avRZ5IukTRP0uMFZRXrH0lbSZqRzvm9JLUhrtHp3/ExSddLWm1Z/dDcz2hzfd2auAr2/UhSSOrTkf3VXEySvpf66wlJ53Z0X7VJRPirSl9AF+A5YEOgK/AosEk7t1kHDE7bqwJPA5sA5wKjUvko4Fdpe0/gn4CA7YCHUvkawPPp++ppe/U2xvZDYBzw9/T6auCQtH0RcEza/i5wUdo+BBiftjdJfdgN2CD1bZcK9NllwLfTdldgtWr3F9AfeAHoUdBXI6rRZ8AuwGDg8YKyivUPMDkdq3Tu19oQ11eAFdP2rwriKtoPtPAz2lxftyauVL4ucCvwItCnI/urmb7aFbgD6JZef6aj+6pNP7ft3YC/WvwPtT1wa8HrHwM/7uAYbgC+TPbpKHWprI7soXKAPwGHFhw/K+0/FPhTQflSx7UijnWAO4HdgL+nH8zXC34RfdJX6RfA9ml7xXScmvZf4XFtiKs3WRJRk/Jq91d/4KX0y23F1Gd7VKvPgPomvxgr0j9p38yC8qWOKzeuJvsOAK5M20X7gWZ+Rlv6/9nauIAJwBbAbD5Nbh3WX0X+Da8Gdi9yXIf2VWu/PC1ZXY2/oBq9nMo6RJqa2hJ4CFgrIl5Ju+YCa6Xt5mKsdOxjgJOBJen1msDbEfFxkfo/aTvtn5+Ob4/+3AB4DfiLsinTiyWtQpX7KyLmAL8G/g28QtYHU6mNPoPK9U//tF3p+ACOJBvZtCaulv5/lk3SfsCciHi0ya5q9tdGwM5pOvFfkrZuZUwV7atSObktpyT1BK4FToiIdwr3RfbnVYc9IyJpb2BeREztqDbLsCLZdM2FEbEl8B7ZNNsnOrq/ANI1rP3Iku/awCrAVzsyhlJVo3+WRdJPgI+BK2sglpWBU4GfVTuWJlYkmxnYDjgJuLrU6521wMmtuuaQzbM3WieVtStJK5Eltisj4rpU/KqkurS/Dpi3jBgrGfuOwL6SZgN/I5uaPA9YTVLj558W1v9J22l/b+CNCsfU6GXg5Yh4KL2eQJbsqtlfALsDL0TEaxGxCLiOrB9roc+gcv0zJ21XLD5JI4C9gcNS4m1NXG/QfF+X67Nkf6Q8mn4G1gGmSerXirgq2V8vA9dFZjLZrEqfVsRUyb4qXXvPe/qrxTnuFckuBG/ApxdgN23nNgVcDoxpUj6apW8AODdt78XSF7Qnp/I1yK5FrZ6+XgDWqEB8Q/n0hpJrWPoi9HfT9rEsfXPE1Wl7U5a+0P08lbmh5F7g82n79NRXVe0vYFvgCWDl1NZlwPeq1Wf89/WaivUP/32DxJ5tiOurwJNA3ybHFe0HWvgZba6vWxNXk32z+fSaW4f1V5G+Oho4I21vRDblqI7uq1b/jLR3A/5a5n+oPcnuWHwO+EkHtLcT2RTRY8D09LUn2bz4ncAzZHdINf6gCLggxTcDGFJQ15HAs+nriArFN5RPk9uG6Qf12fTD0XjXVvf0+tm0f8OC83+SYp1FiXfVlRDTIGBK6rOJ6ZdJ1fsL+DkwE3gc+Gv6ZdPhfQZcRXbdbxHZX/tHVbJ/gCHpPT4HnE+Tm3vKjOtZsl/Sjf/3L1pWP9DMz2hzfd2auJrsn82nya1D+quZvuoKXJHqmgbs1tF91ZYvf/yWmZnljq+5mZlZ7ji5mZlZ7ji5mZlZ7ji5mZlZ7ji5mZlZ7ji5mZlZ7ji5mZlZ7vx/Tzz+X31/9QsAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# use 5x as many points and 10x the iterations\n",
"plot_importances(500, 1, 'feature_importance_rand_ci_bigger_dataset.png')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.7"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment