Skip to content

Instantly share code, notes, and snippets.

@kiwamizamurai

kiwamizamurai/crossvalidation.ipynb

Created February 6, 2019 11:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kiwamizamurai/6d7ded92a2c13c50bf57b6c3027b1d9b to your computer and use it in GitHub Desktop.
Save kiwamizamurai/6d7ded92a2c13c50bf57b6c3027b1d9b to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
crossvalidation.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Understanding the usage of Cross-Validation\n",
"- https://kevinzakka.github.io/2016/07/13/k-nearest-neighbor/\n",
"- https://towardsdatascience.com/interactive-controls-for-jupyter-notebooks-f5c94829aee6\n",
"- https://jakevdp.github.io/PythonDataScienceHandbook/05.03-hyperparameters-and-model-validation.html\n",
"- https://towardsdatascience.com/train-test-split-and-cross-validation-in-python-80b61beca4b6"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>sepal length (cm)</th>\n",
" <th>sepal width (cm)</th>\n",
" <th>petal length (cm)</th>\n",
" <th>petal width (cm)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>5.1</td>\n",
" <td>3.5</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>4.9</td>\n",
" <td>3.0</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>4.7</td>\n",
" <td>3.2</td>\n",
" <td>1.3</td>\n",
" <td>0.2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4.6</td>\n",
" <td>3.1</td>\n",
" <td>1.5</td>\n",
" <td>0.2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5.0</td>\n",
" <td>3.6</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" sepal length (cm) sepal width (cm) petal length (cm) petal width (cm)\n",
"0 5.1 3.5 1.4 0.2\n",
"1 4.9 3.0 1.4 0.2\n",
"2 4.7 3.2 1.3 0.2\n",
"3 4.6 3.1 1.5 0.2\n",
"4 5.0 3.6 1.4 0.2"
]
},
"execution_count": 84,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.datasets import load_iris\n",
"import pandas\n",
"iris = load_iris()\n",
"X = iris.data\n",
"y = iris.target\n",
"pandas.DataFrame(X, columns=iris.feature_names).head()"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "86881cb5becb43e999f84aa75c691422",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"interactive(children=(Dropdown(description='column', options=('sepal length (cm)', 'sepal width (cm)', 'petal …"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import ipywidgets as widgets\n",
"from ipywidgets import interact, interact_manual\n",
"\n",
"df = pandas.DataFrame(X, columns=iris.feature_names)\n",
"\n",
"@interact\n",
"def show_articles_more_than(column=['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'], \n",
" x=(0, 4, 0.3)):\n",
" return df.loc[df[column] > x]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> No module named 'cufflinks'\n",
"\n",
"とエラーが出たから\n",
"\n",
"> pip install cufflinks\n",
"\n",
"とする"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "39293106d99c447eb4f5d03315519fbb",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"interactive(children=(Dropdown(description='x', options=('sepal length (cm)', 'sepal width (cm)', 'petal lengt…"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import cufflinks as cf\n",
"\n",
"@interact\n",
"def scatter_plot(x=list(df.select_dtypes('number').columns), \n",
" y=list(df.select_dtypes('number').columns)[1:],\n",
" theme=list(cf.themes.THEMES.keys()), \n",
" colorscale=list(cf.colors._scales_names.keys())):\n",
" \n",
" df.iplot(kind='scatter', x=x, y=y, mode='markers', \n",
" xTitle=x.title(), yTitle=y.title(), \n",
" text='title',\n",
" title=f'{y.title()} vs {x.title()}',\n",
" theme=theme, colorscale=colorscale)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"エラーがなおならない、このライブラリを使うのが少し楽しみだったのだが諦める\n",
"\n",
"気を取り直して**交差検証**を行う！"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# K-means　　(k=3)を使った例"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.neighbors import KNeighborsClassifier\n",
"model = KNeighborsClassifier(n_neighbors=3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. train_test_split でデータセットを分割する\n",
"2. trainで学習し, testで精度を確認する"
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/kiwamizamurai/anaconda3/lib/python3.6/site-packages/sklearn/cross_validation.py:41: DeprecationWarning:\n",
"\n",
"This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
"\n"
]
},
{
"data": {
"text/plain": [
"0.9777777777777777"
]
},
"execution_count": 88,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.cross_validation import train_test_split\n",
"from sklearn.metrics import accuracy_score\n",
"\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0, train_size=0.7)\n",
"\n",
"model.fit(X_train, y_train)\n",
"pred = model.predict(X_test)\n",
"accuracy_score(y_test, pred)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"上はcross-validationの一動作だった、なので次で交差検証を行う \n",
"下の例ではデータセットを５等分割して交差検証している"
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([0.96666667, 0.96666667, 0.93333333, 0.96666667, 1. ])"
]
},
"execution_count": 89,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.cross_validation import cross_val_score\n",
"cross_val_score(model, X, y, cv=5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# k=3がベストなのか？と疑問に思う\n",
"# そんな時、交差検証を行う\n",
"\n",
"下では次のようにしてベストなkを選んでいる\n",
"1. kを1から30まで回す\n",
"2. 各kに対してcv-10で交差検証を行う\n",
"3. プロットしてエラーが最も小さいところをみる\n"
]
},
{
"cell_type": "code",
"execution_count": 90,
"metadata": {},
"outputs": [],
"source": [
"neighbors = list(range(1, 30))\n",
"cv_scores = []\n",
"for k in neighbors:\n",
" knn = KNeighborsClassifier(n_neighbors=k)\n",
" scores = cross_val_score(knn, X_train, y_train, cv=10, scoring='accuracy')\n",
" cv_scores.append(scores.mean())"
]
},
{
"cell_type": "code",
"execution_count": 91,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The optimal number of neighbors is 6\n"
]
},
{
"data": {
"text/plain": [
"Text(0,0.5,'Misclassification Error')"
]
},
"execution_count": 91,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAnEAAAF3CAYAAADKPB24AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8nHd1L/7PmRntmhnJWqzdsi05kiUndmICpBCWQAgUEkoDJZcll6ZQCmlLaQuUvgpJetteaEv6g6ZQuBBSemnYujgQSGkp4QZSiLNKsmRLlryNdsmeRfvMnN8f8zyKImsZSfMsI33er5deHs088+gI7Ojo+/2ec0RVQURERETZxeN0AERERES0cUziiIiIiLIQkzgiIiKiLMQkjoiIiCgLMYkjIiIiykJM4oiIiIiyEJM4IiIioizEJI6IiIgoCzGJIyIiIspCTOKIiIiIspDP6QDsUF5ero2NjU6HQURERLSuJ598clxVK9a7bkckcY2NjTh+/LjTYRARERGtS0TOpnMdt1OJiIiIshCTOCIiIqIsxCSOiIiIKAsxiSMiIiLKQkziiIiIiLIQkzgiIiKiLMQkjoiIiCgLMYkjIiIiykJM4oiIiIiyEJM4IiIioizEJI6IiIgoC1maxInITSJyUkT6RORjK7yeJyLfMF7/uYg0LnntShF5XES6RKRDRPKN539s3PMZ46PSyu+BiIhoJzg/OY3I7ILTYdAGWJbEiYgXwH0AXg/gIIDbROTgssvuAHBRVZsA3AvgU8Z7fQD+EcD7VbUNwCsBLP2b9Q5VPWx8jFr1PRAREe0Ut33pv/EbXz0OVXU6FEqTlStx1wLoU9V+VZ0H8CCAW5ZdcwuAB4zH3wZwg4gIgBsBPKeqzwKAqk6oasLCWImIiHas2FwcFy7O4BdnJnHs2UGnw6E0WZnE1QI4v+TzC8ZzK16jqnEAYQBlAA4AUBF5RESeEpGPLHvf/cZW6p8YSR8RERFt0pnxKQBAfo4Hf/a9bsTm4g5HROmwMolbKblavka72jU+AC8D8A7jz18RkRuM19+hqocAvNz4eNeKX1zkfSJyXESOj42NbSZ+IiKiHaHfSOI++aY2jEbn8Lkf9TocEaXDyiTuAoD6JZ/XAVi+Rrt4jXEOLghg0nj+UVUdV9VpAA8DuBoAVDVk/BkF8HWktm0vo6pfVNWjqnq0oqIiY98UERHRdmOuxP3KkVrcek0dvvLYAE6PxRyOitZjZRL3BIBmEdkrIrkA3g7g2LJrjgG43Xh8K4AfaepE5SMArhSRQiO5ewWAEyLiE5FyABCRHABvBNBp4fdARES07Q2MT6EmmI/8HC8+elML8n1e3P3QCRY5uJxlSZxxxu1OpBKybgDfVNUuEblHRG42LvsygDIR6QPwYQAfM957EcBnkEoEnwHwlKp+D0AegEdE5Dnj+RCAL1n1PRAREe0E/eNT2FtRBACo8Ofhd1/TjJ+cGsMPT4w4HBmtxWflzVX1YaS2Qpc+94klj2cBvHWV9/4jUm1Glj43BeCazEdKRES0M6kqBsZiuPlwzeJzt1/XiG88cR5/+r0TuP5ABfJzvA5GSKvhxAYX+dbx83jfPxx3OgwiItpBJqfmEZmNY2958eJzOV4P7r65DecnZ/DFn/Q7GB2thUmcizz03BD+/cQIJmJzTodCREQ7xJmJVFHDvvKiFzx/XVM53nCoCvf9Vx8uXJx2IjRaB5M4l1BVdIbCAIDOwYjD0RAR0U7RP5ZK4vYuS+IA4I9/+SBEgD/7XrfdYVEamMS5xFB4FpNT8wCwmMwRERFZbWB8Cj6PoK604LLXaksK8MFXNuH7ncN4rHfcgehoLUziXMJM3LweQdcgkzgiIrLHwPgUGnYVwuddOSV47/X70LCrEHc91IWFRNLm6GgtTOJcojMUhkeAVxyoQAdX4oiIyCYD41MrbqWa8nO8+JM3HkTfaAwP/OyMfYHRupjEuUTnYATNlX4cbSzF+ckZhKcXnA6JiIi2uWRS103iAOA1rZV45RUV+Jv/6MVodNam6Gg9TOJcoiMURlttAO01QQBAJ7dUiYjIYsORWczFk4uNflcjIvjEGw9iLp7Ap75/0qboaD1M4lxgNDKLsegcDtUG0V5rJHHcUiUiIosNjK9embrcvopi3PGyffjOUxfw5NlJq0OjNDCJcwHzDFx7bRC7inJRW1LANiNERGS5/g0kcQDw269uwu5AHj55rAuJJOeqOo1JnAt0hiIQAQ5WBwAAbTUBrsQREZHlBsamUJDjxW5/flrXF+X58PE3tKIzFME3njhvcXS0HiZxLtA5GMa+8iIU5aVG2R6qDWJgfArRWRY3EBGRdQbGY2gsL4LHI2m/5+aranDt3l34y0d6cGl63sLoaD1M4lygMxRePAsHYPFxF7dUiYjIQmcmpi8bt7UeEcFdb2pDeGYBf/3vpyyKjNLBJM5h47E5DIVncWiFJI5bqkREZJWFRBLnJqfTPg+31MGaAN71kj34vz8/ixNccHAMkziHmYlaW83zSVyFPw+7A3lM4oiIyDLnJ6eRSCoaN5HEAcCHX3sFSgpz8cljnVBlkYMTmMQ5zNwybasNvOD5Q7VBVqgSEZFlNtJeZCXBwhz84euuwBNnLuLfnhnMZGiUJiZxDuu4EEZjWSEC+TkveL6tJojTYzFMz8cdioyIiLYzM4nb6Jm4pd52tB5X1gXx5w93IzbHn1d2YxLnsM7BMNqWnIczHaoNQhU8a0BERJboH59CSWEOSotyN30Pr0dw981tGI3O4XM/6s1gdJQOJnEOujQ9jwsXZ15Q1GBicQMREVnpTBozU9NxpKEUt15Th688NoDTY7EMREbpYhLnoM5QapWtvebyJG53IA/lxbnoCHEljoiIMi+dwffp+uhNLcj3eXH3QydY5GAjJnEOMofcty8ragBSfXjaa4PoGuRKHBERZdb0fBxD4VnsLctMElfhz8OHXnsAPzk1hh+eGMnIPWl9TOIc1BEKo660ACWFK59HaK8Jonc0htmFhM2RERHRdnZmfBoAsLciM0kcALz7pXvQXFmMe757gj+3bMIkzkFdofCK5+FM7bVBJJKK7iFuqRIRUeZstb3ISnK8Htx9cxsuXJzBl37Sn7H70uqYxDkkMruAMxPTLxi3tZy5zcp+cURElElnJlJJXGOGtlNN1zWV47r9Zfjuc0MZvS+tjEmcQ7qMgoW2msvPw5lqSwpQWpiDzgs8F0dERJnTPzaFqkA+ivJ8Gb/3NXtK0Tsaxcw8t1StxiTOIV2LRQ2rr8SZxQ2dLG4gIqIMGhiPZXQrdan22iCSCnQPcxfJakziHNIRCqM6mI/y4rw1r2urCeLUSBRzcf5GQ0REmTEwPrXpmanrMRcnutjn1HJM4hzSGQqvuQpnOlQbxEJCcWqYDRSJiGjrLk7N4+L0wpbGba2lJpiPXUW56GASZzkmcQ6IzcXRPz61YpPf5cziBv5jICKiTBiYyHxl6lIigraawGJDe7IOkzgHdA9FoAocqlu9qMHUsKsQ/nwfz8UREVFGnDHbi2SwR9xy7bWpo0DsF2ctJnEO6DCqTdNZiRMRtNcEebaAiIgyYmB8Cl6PoL600LKvcag2iHhScWokatnXICZxjugcDKPCn4fKQH5a1x+qC6J7OIqFRNLiyIiIaLvrH59CfWkBcn3WpQDmIgWPAlmLSZwDukKRNSc1LNdWE8B8PIneERY3EBHR1gyMWVeZaqrfVYBAvo/n4izGJM5mM/MJ9I5G0b5Gk9/lzISvk7/REBHRFqgqzkxMWVbUYDL7nHbxPLelmMTZrHs4gqSu3eR3ucayIhTlelncQEREWzIancP0fMKy9iJLHaoNomcoivk4jwJZhUmczczVtI0kcR6PoK0myLMFRES0Jf1jZnuRYsu/VlttEPOJJHpHWdxgFSZxNusMhbGrKBfVwfSKGkzttUF0D0UQZ3EDERFt0oAN7UVM5rEhHgWyDpM4m3WEImivDUJENvS+9toAZheS6Df+ARIREW3UwHgMeT4PqtPsjrAVjWVFKM5jcYOVmMTZaHYhgd6RjRU1mMziBrPHHBER0UYNjE+hsawIHs/GFhI2w+MRHKwJ8CiQhZjE2ejUSBTxpG6ovYhpX0UxCnJY3EBERJvXP259ZepSh3gUyFJM4mzUsYmiBpPX+I2GZwuIiGgz4okkzk9O23IeztReG8BcPInTYzwKZAUmcTbqDEUQLMhBXWnBpt7fXhNA12AEyaRmODIiItruQpdmsJBQ21fiAE5usAqTOBt1hsJorw1suKjB1FYbxPR8gsUNRES0YebPDjt6xJn2lhejMNfLXSSLMImzyXw8iZPD0bSG3q/G/I2GHbCJiGijBowtTatHbi3l9QgOVvMokFWYxNnk1EgU84nkps7DmZoqi5Hr8/AfAxERbdjA+BT8+T6UFeXa+nXba4M4MRRBgkeBMo5JnE3M1bOtJHE5Xg9aq1muTUREG3dmYgr7yos2faRns9pqApieT2BgPGbr190JLE3iROQmETkpIn0i8rEVXs8TkW8Yr/9cRBqXvHaliDwuIl0i0iEi+cbz1xif94nIZ8Xuv42b1BEKw5/nw55dhVu6T3tNAF0hFjcQEdHG9I/Z217EdKgutXjBpr+ZZ1kSJyJeAPcBeD2AgwBuE5GDyy67A8BFVW0CcC+ATxnv9QH4RwDvV9U2AK8EsGC85/MA3geg2fi4yarvIZM6QxEcrAlsucHiodogonNxnJuczlBkRES03c0uJDAYnrFlZupyTRXFyONRIEtYuRJ3LYA+Ve1X1XkADwK4Zdk1twB4wHj8bQA3GCtrNwJ4TlWfBQBVnVDVhIhUAwio6uOqqgD+AcCbLfweMiKeSKJ7KLKpJr/LmduxbPpLRETpOjsxDVV7ZqYu5+NRIMtYmcTVAji/5PMLxnMrXqOqcQBhAGUADgBQEXlERJ4SkY8suf7COvd0nb6xGObiWytqMB3Y7UeOV7gsTUREaTPPo+0tsz+JA1JNf0+wz2nGWZnErbRvuPz/vdWu8QF4GYB3GH/+iojckOY9UzcWeZ+IHBeR42NjY+lHbQEz4Wqv3fjM1OVyfR5cUeXnsjQREaXN7BHXWL61c9mbZR4FOsujQBllZRJ3AUD9ks/rAAyudo1xDi4IYNJ4/lFVHVfVaQAPA7jaeL5unXsCAFT1i6p6VFWPVlRUZODb2bzOUBiFud6MnUVorwmiczCM1I4yERHR2s6MT6HCnwd/fo4jX7+txixu4AJEJlmZxD0BoFlE9opILoC3Azi27JpjAG43Ht8K4EfGWbdHAFwpIoVGcvcKACdUdQhAVEReYpydezeAf7Pwe8iIzlAYB6sD8G6xqMHUXhvEpekFXLg4k5H7ERHR9jZg8+D75Q7s9iPXy+KGTLMsiTPOuN2JVELWDeCbqtolIveIyM3GZV8GUCYifQA+DOBjxnsvAvgMUongMwCeUtXvGe/5LQD/B0AfgNMAvm/V95AJiaSiazCSkfNwpnZObiAiog0YGJ+yddzWcotHgfhzK6N8Vt5cVR9Gait06XOfWPJ4FsBbV3nvPyLVZmT588cBtGc2UusMjMcws5DIaBLXUuWH1yPoCIVxU3t1xu5LRETbT3hmAeOxeUdX4oDUufCHO4ahqrY3HN6uOLHBYmZRQybai5jyc7xorixmhSoREa3rzLj9M1NX0l4bRHiGR4EyiUmcxTpCYeT5PNif4d48h2qD6AyxuIGIiNY2YCRxTm6nAqmiPIDFDZnEJM5inaEwWqsD8Hkz+z91e20QE1PzGI7MZvS+RES0vQyMT0EEaChzpr2I6YoqP3zGUSDKDCZxFkoaRQ2Z3Eo1LU5u4JYqERGtYWB8CnWlBcjzeR2NIz/Hi+bdfnQO8udWpjCJs9DZyWnE5uIZafK7XGu1Hx4Bf6MhIqI1pdqL2D8zdSWHagM8CpRBTOIsZCZYmaxMNRXm+rC/ohhdTOKIiGgVqup4e5Gl2muDmJyax1CYR4EygUmchbpCYeR6PWiu9Fty/0O1Qa7EERHRqsZic4jNxdHo8Hk4Eyc3ZBaTOAt1DobRUu1Hrs+a/5nbaoMYjc5hlMUNRES0gjPjqVmleyvcsZ16sDoAjzCJyxQmcRZRVXSGIou/dVjh0OLkBh4SJSKiyw2MxwA4317EVJDrRVNlMYsbMoRJnEUuXJxBeGbBkqIG08GaAITFDUREtIr+8Snkej2oKSlwOpRF7TwKlDFM4ixi/gW1or2IqTjPh73lRVyWJiKiFQ2MTWFPWSG8HveMuWqvCWKMR4EygkmcRTpDYfg8ggO7rSlqMLXXBJnEERHRilLtRdyxlWo6VJda3OBq3NYxibNIRyiMA7v9yM+xtrlie20Ag+FZTMTmLP06RESUXRJJxdmJadclca3VqaNAbFa/dUziLKCamtRg5Xk40+LkBh4SJSKiJQYvzWA+kXRdErd4FGiQK3FbxSTOAkPhWUxOzVt6Hs7EnjtERLQSc/C925I4IHVenD+3to5JnAXMff42G5K4YEEO9pQVoou/0RAR0RKLSVyF+5K49poghsKzGOdRoC1hEmeBrlAYHgFaq6zfTgVS/xh4QJSIiJYaGJ9CcZ4PFcV5TodymcWjQPzZtSVM4izQEQqjudKPglxrixpM7bVBnJ+cQXh6wZavR0RE7tc/PoXG8kKIuKe9iKnNODPOZvVbwyTOAp2DkcW/oHYwCyh4SJSIiExnxqewt9wd47aWC+TnoLGsEB0X+HNrK5jEZdhIZBZj0TlbihpM7SxuICKiJebiCVy46L72Iku11Qa5+LBFTOIyzEyk2m1M4kqLclFbUsBzcUREBAA4PzmNpLpnZupKDtUGceHiDC5NzzsdStZiEpdhnaEIRICD1fZtpwKpLVWeLSAiIgDoH3NvexHT87tI/Nm1WUziMqwjFMa+8iIU5fls/bqHaoMYGJ9CdJbFDUREO53ZXqTRxUlcWw3Pc28Vk7gM6xoM27qVajJ70nE1joiIBsanUFaUi2BBjtOhrKq0KBd1pTwKtBVM4jJoPDaHofCsrUUNJhY3EBGRyY2D71fSXhNEF39ubRqTuAwyEyhzFJadKvx5qArkM4kjIqKsSeIO1QVxZmIaER4F2hQmcRm0mMTZ2CNuqfbaIDq5nUpEtKPF5uIYjc65ctzWcua5uC4WN2wKk7gM6gxF0FhWiEC+M2cQ2msDOD0Ww9Rc3JGvT0REzjtjFDW4ub2IqX3xPDd3kTaDSVwGdYScKWowtdcEoQp0D/E3GiIit3v2/CUsJJIZv2+/OfjepdMaliovzkN1MJ/FDZvEJC5DLk7NI3RpxtEk7lAdixuIiLLBuYlp3HLfT/Gp7/dk/N7mStyessKM39sK7bVB/tzaJCZxGWK29mh3oKjBVOnPQ3lxHjp4toCIyNWeuXAJAPDVn51B32g0o/ceGJ9CbUkB8nO8Gb2vVdprgugfn0KMR4E2jElchnQsjttypqgBAEQEh2oDPFtARORynaEwcr0eFOZ6cdexE1DVjN27P0sqU03ttQEeBdqkNZM4EfGKyF/aFUw26xwMo660ACWFuY7G0V4bRO9oDLMLCUfjICKi1XWGwmip9uP3b7wCj/WN45Gu4YzcV1UxMBbLqiTO7K3acYELEBu1ZhKnqgkA14iI2BRP1uoMhR1p8rtce20QiaTyNxoiIpdSVXSGwmirCeIdL25AS5Uff/rdbszMb/2X78mpeURm41mVxFUG8lHhz+P4rU1IZzv1aQD/JiLvEpG3mB9WB5ZNwjMLODsx7WhRg8mMgYdEiYjc6fzkDCKzcbTXBuDzenDXzW0IXZrB5x89veV7mzNTs6FH3FKHaoPsFbcJ6SRxuwBMAHg1gDcZH2+0MqhsY55Bc0MSVxPMx66iXHTyHwMRkSuZK07m7s1L9pXh5qtq8IVHT+PcxPSW7r2YxJVlVxLXXhNA72g0I6uRO4lvvQtU9T12BJLNzN8ezM7TThIRtNUEuCxNRORSHaEwfB7Bgd3+xec+/oZW/Ef3CP70eyfwpXcf3fS9B8an4PMI6koLMhGqbdprg0gq0D0cwdUNpU6HkzXWXYkTkToR+RcRGRWRERH5jojU2RFctugcDKM6mI/y4jynQwGQ+sdwcjiKH54YcToUIiJapjMUxoHd/he0AKkK5uPOVzfhhydG8OipsU3fe2B8Cg1lhfB5s6v5BI8CbU46/y/fD+AYgBoAtQAeMp4jwysOVOCOl+11OoxF73rJHhzY7cd7/+E4PvzNZxCe4WBhIiI3MIsaVmpHdcfL9mJveRHuPtaF+fjmJjkMjE9lxbit5aoXjwIxiduIdJK4ClW9X1XjxsdXAVRYHFdWecvVdfiNl+9zOoxFNSUF+NcP/hJ+59VN+LdnBvG6e3+ypd/siIgoMwbDs7g4vbBiN4M8nxefeNNB9I9P4Ss/HdjwvZNJxUCW9YgziQjaa4NsVr9B6SRx4yLyTqNnnFdE3olUoQO5WK7Pgw/feAX+5QPXoTjfh9u/8gv80T93sCM2EZGDzJWmtlUK4V51RSVe01qJz/1nL0Yisxu693BkFnPxZFbMTF1Je00AvSNR9jndgHSSuF8H8DYAwwCGANxqPEdZ4Mq6Enz3t1+G37x+Hx584hxu+puf4PHTzMGJiJzQGQrDI0Br1eqFcH/yxoNYSCr+4uHuDd3brExtLM+OmanLHaoNIp5UnBrJ7Biy7WzdiQ0AflVVb1bVClWtVNU3q+pZm+KjDMjP8eKP3tCKb/3mS+HzCG770n/jrmNdLOUmIrJZZyiM5ko/CnJXn2u6p6wIv3n9PvzrM4P4xcBk2vfuN5K4fdm6EmdObuC5uLSlM7HhFptiIYsdbdyFh3/35fif1zXiqz87gzd89v/hybPp/weCiIg2T1XREYqgLY0Z2x94ZRNqgvn45LEuJJLpzVUdGJtCQY4XuwPu6JSwUXWlBQgW5LDP6Qaks536UxH5WxF5uYhcbX5YHhlZojDXh7tubsPX3/tizMeTeOsXHsdfPNzNMwhERBYbjc5hPDaX1ojGglwv/viXD6J7KIKv/zy9za+B8dTM1GydlJkqbgiwQnUD0knirgPQBuAeAH9tfPyVlUGR9a7bX45Hfu96/NqL6vH3P+nHmz73GIcPExFZyPxvbLrTfd5wqAov3VeGv/r3U5icml/3+oHxqawbt7Wc2ed0sy1Wdpr1zsR5AHxeVV+17OPV6dxcRG4SkZMi0iciH1vh9TwR+Ybx+s9FpNF4vlFEZkTkGePjC0ve82PjnuZrlRv6jmlRcZ4Pf/GWK/HV97wI0dk43vx3P8VnfniK/3iIiCzQORiGCHCwOr3pPiKCu29pQ2wujr985OSa1y4kkjh/cSbrxm0t114TxHwiyeKGNK13Ji4J4M7N3NgoirgPwOsBHARwm4gcXHbZHQAuqmoTgHsBfGrJa6dV9bDx8f5l73vHktdGNxMfPe+VV1TikQ9dj1uuqsFn/7MXb77vp+ge4pkEIqJM6gxFsK+8CEV56068XHRgtx+3v7QRDz5xbs3dkvOT00gkNSt7xC1lrlJ2cXRkWtLZTv2hiPyBiNSLyC7zI433XQugT1X7VXUewIO4vEjiFgAPGI+/DeAGydbN/CwXLMzBZ37tML74rmswGp3FzX/7GO77rz7EE1yVIyLKhM5QOK3zcMt96LXNKCvKxSePdSK5SpHD4uD7LN9O3bOrEP48H4sb0pRun7gPAvgJgCeNj+NpvK8WwPkln18wnlvxGlWNAwgDKDNe2ysiT4vIoyLy8mXvu9/YSv0TJn2ZdWNbFf79916BG9uq8JePnMTb/v5xFj0QEW3RWHQOw5HZtM/DLRXIz8FHbmrBU+cu4Z+fDq14zcBie5HsTuI8HsHBmgDbjKRp3SROVfeu8JHOjKmVkqvlv0Ksds0QgAZVPQLgwwC+LiLmIYJ3qOohAC83Pt614hcXeZ+IHBeR42NjHDm1EbuKcnHf/7ga99zShqfOXcJ/97M5MBHRVnQa24NtNRtP4gDg1qvrcLi+BP/7+z2IzF4+D7t/fAqlhTkoKczdUpxucKg2iO6hCHeC0rBqEiciH1ny+K3LXvvzNO59AUD9ks/rAAyudo2I+AAEAUyq6pyqTgCAqj4J4DSAA8bnIePPKICvI7VtexlV/aKqHlXVoxUVHPW6GW+5ug4iwDPnLzkdChFRVutaHLeVXlHDch6P4J5b2jAxNYfP/kfvZa+fydKZqStprw1iLp5E31jM6VBcb62VuLcvefxHy167KY17PwGgWUT2ikiucb9jy645BuB24/GtAH6kqioiFUZhBERkH4BmAP0i4hORcuP5HABvBNCZRiy0CcV5Phyo9OPpc0ziiIi2oiMURmNZIQL5OZu+x5V1Jfi1o/X46s/OoHdZ9ebA+BQat1ESB4Dn4tKwVhInqzxe6fPLGGfc7gTwCIBuAN9U1S4RuUdEbjYu+zKAMhHpQ2rb1GxDcj2A50TkWaQKHt6vqpMA8gA8IiLPAXgGQAjAl9aLhTbvcH0Jnr1wCarpdQwnIqLLdYYimzoPt9wfvu4KFOZ6cddDXYv/XZ6ej2MoPJv15+FMe8uLUJjrZdPfNKxV56yrPF7p85VvoPowgIeXPfeJJY9nAbx1hfd9B8B3Vnh+CsA16XxtyowjDSX4xvHzODMxvW2W6omI7HRxah6hSzN410v3bPleZcV5+P0br8Anj3XhB53DeP2hapwZnwYA7M3SmanLeT2Cg9Wc3JCOtVbirhKRiIhEAVxpPDY/P2RTfOSwww0lAICnz110OBIiouxkFjVspr3ISt7x4ga0VPnxv77XjZn5xPPtRbbRL9rttUGcGIqkPTd2p1o1iVNVr6oGVNWvqj7jsfn55jf1Kas0V/pRlOtlcQMR0SaZZ7vaajZX1LCcz+vBXTe3IXRpBp//cR8GxlMFAI3lhRm5vxu01wYxPZ9Y/N5oZen0iaMdzOsRXFlXwuIGIqJN6gyFUVdakNH2Hy/ZV4abr6rBF37Sj5/0jqM6mI/C3PQnQbjdIRY3pIVJHK3rSEMJuocibPpLRLQJnYObm9Swno+/oRU+j+AXA5NozPKZqcvtryhCfo6HTX/XsX3SdrLM4foSxJOKzlCwhqmQAAAgAElEQVQYRxvTmbhGREQAEJ5ZwNmJabztaP36F29QVTAfd766CZ/+wcmsH7e1nM/rQWt1AA89O4ih8IzT4bzA//7VK7fUKiaTmMTRuszihmfOX2ISR0S0AeYg90y0F1nJHS/bi85QGDe1VVlyfye97Wg9vvLYAHpH3HUuLpFwT7HFukmciLwFwKcAVCLVH04AqKpm5oQmuV6lPx+1JQU8F0dEtEFdxpmu9gwVNSyX5/Pi796xPTtv3XZtA267tsHpMFwtnZW4TwN4k6p2Wx0MudeRBhY3EBFtVEcojJpgPsqK85wOhbahdAobRpjA0eH6EoQuzWA0Mut0KEREWaNzMIw2i7ZSidJJ4o6LyDdE5DYReYv5YXlk5CpHGkoBAE+zXxwRUVpic3EMjE9ZUplKBKS3nRoAMA3gxiXPKYB/tiQicqW2mgByvIKnz13C67bhAVoiokw7MRiBKtBeyyPkZI11kzhVfY8dgZC75ed4cbA6gGfOc/wWEVE6zB5n7TVciSNrrLudKiJ1IvIvIjIqIiMi8h0RqbMjOHKXw/UleO5CmLPsiIjS0BUKo9Kfh8pAvtOh0DaVzpm4+wEcA1ADoBbAQ8ZztMMcaSjF9HwCp0aiTodCROR6nYNhy/rDEQHpJXEVqnq/qsaNj68CqLA4LnKhw/XPN/0lIqLVTc/H0TcaYxJHlkoniRsXkXeKiNf4eCeACasDI/fZU1aI0sIcPH2O5+KIiNbSPRRFUq1r8ksEpJfE/TqAtwEYBjAE4FbjOdphRASH60u4EkdEtI5Oo6jhUB1X4sg66VSnngNwsw2xUBY40lCKH58aQ3R2AX6XDAAmInKbzlAYZUW5qGJRA1lo1SRORD6iqp8Wkc8h1RfuBVT1dyyNjFzpcH0JVIHnLoTxS03lTodDRORKHaHUpAYRcToU2sbWWokzR20dtyMQyg5XGcUNT5+7yCSOiGgFswsJ9I7GcENrpdOh0Da3ahKnqg8ZD6dV9VtLXxORt1oaFblWsCAH+yuKeC6OiGgVJ4ejSCSVTX7JcukUNvxRms/RDnG4vhRPn7sEVTb9JSJabnFSA9uLkMXWOhP3egBvAFArIp9d8lIAQNzqwMi9jjSU4DtPXcCFizOo31XodDhERK7SNRhGsCAHdaUFTodC29xaK3GDSJ2HmwXw5JKPYwBeZ31o5FZm09+n2C+OiOgyHaEwDrGogWyw1pm4ZwE8KyJfV9UFG2Mil2up8iM/x4Nnzl/CLYdrnQ6HiMg15uNJnByO4tdfttfpUGgHWLdPHIBGEfkLAAcBLDa8UdV9lkVFrubzenBlbQmePsfiBiKipU6NRLGQUBzieTiyQTqFDfcD+DxS5+BeBeAfAHzNyqDI/Y40lODEYARz8YTToRARuYY5qYGVqWSHdJK4AlX9TwCiqmdV9S4Ar7Y2LHK7w/UlmE8kcWIw4nQoRESu0TkYhj/PhwYWfZEN0kniZkXEA6BXRO4UkV8BwA6GO9yRhlIAYL84IqIlOkIRtNUG4PGwqIGsl04S9yEAhQB+B8A1AN4J4HYrgyL3qwrmoyqQz3NxRESGhUQS3UMRbqWSbdYtbFDVJ4yHMQDvsTYcyiZHGkq4EkdEZOgbjWE+nsShOiZxZI91V+JE5IciUrLk81IRecTasCgbHK4vwbnJaUzE5pwOhYjIcWZRQxtX4sgm6Wynlqvq4nKLql4Ez8QReC6OiGipzlAYRble7CsvcjoU2iHSSeKSItJgfiIiewBwaCbhUG0QXo8wiSMiAtA5GMHBGhY1kH3Safb7xwAeE5FHjc+vB/A+60KibFGQ60VLlZ/FDUS04yWSihODEfzai+qdDoV2kHQKG34gIlcDeAkAAfB7qjpueWSUFQ7Xl+DYM4NIJpW/fRLRjtU/FsPMQoKTGshWq26nikiL8efVABoADAIIAWgwniPCkYZSROfiOD0WczoUIiLHdA4akxqYxJGN1lqJ+zBS26Z/vcJrCk5tIKRW4gDg6fOX0Lzb73A0RETO6LgQQX6OB/srWNRA9lkrifuh8ecdqtpvRzCUffaVFyGQ78PT5y7hbUd5FoSIdqbOwTBaqwPwedOpFyTKjLX+tv2R8ee37QiEspPHI7iqnk1/iWjnShpFDTwPR3ZbayVuQkT+C8BeETm2/EVVvdm6sCibHGkoxd/+qBdTc3EU5aVT8ExEtH2cmZhCbC7OcVtku7V+4v4ygKsBfA0rn4sjAgAcqS9BUoGOUBgv2VfmdDhERLbqHIwAYFED2W/VJE5V5wH8t4hcp6pjNsZEWWaxuOHcJSZxRLTjdIbCyPV60Ly72OlQaIdZNYkTkb9R1Q8B+IqIXDahgdupZCotykVjWSGeOX/R6VCIiGzXGQqjpdqPHBY1kM3W2k79mvHnX9kRCGW3w/Ul+NnpCagqRNj0l4h2BlVFZyiMN15V43QotAOt+muDqj5p/Pmo+QHgOQAXjcdEi440lGI0Ooeh8KzToRAR2eb85AwisyxqIGesu/YrIj8WkYCI7ALwLID7ReQz6dxcRG4SkZMi0iciH1vh9TwR+Ybx+s9FpNF4vlFEZkTkGePjC0vec42IdBjv+axw2ccVlp6LIyLaKTpCqUkNbC9CTkhnAz+oqhEAbwFwv6peA+A1671JRLwA7gPwegAHAdwmIgeXXXYHUit7TQDuBfCpJa+dVtXDxsf7lzz/eaQmSTQbHzel8T2QxVqrA8j1eXgujoh2lM7BMHK8ggNVLGog+6WTxPlEpBrA2wB8dwP3vhZAn6r2G5WuDwK4Zdk1twB4wHj8bQA3rLWyZsQRUNXHVVUB/AOAN28gJrJIrs+D9poAm/4S0Y7SGQrjwG4/8nxep0OhHSidJO4eAI8glZA9ISL7APSm8b5aAOeXfH7BeG7Fa1Q1DiAMwOxRsVdEnhaRR0Xk5Uuuv7DOPckhRxpK8dyFMBYSSadDISKynFnUwPNw5JR1kzhV/ZaqXqmqHzA+71fVX03j3iutqC1vVbLaNUMAGlT1CIAPA/i6iATSvGfqxiLvE5HjInJ8bIxt7uxwuL4Ec/EkTg5HnQ6FiMhyg+FZXJxeQHttwOlQaIdKp7Dh00ZhQ46I/KeIjIvIO9O49wUASyei1wEYXO0aEfEBCAKYVNU5VZ0AFqtkTwM4YFxft849Ybzvi6p6VFWPVlRUpBEubdWRBrO4gefiiGj767iQKmrgpAZySjrbqTcahQ1vRCqJOgDgD9N43xMAmkVkr4jkAng7gOUzWI8BuN14fCuAH6mqikiFURgBY/u2GUC/qg4BiIrIS4yzc+8G8G9pxEI2qC0pQHlxHp7muTgi2gG6BsPwegSt1VyJI2ekM608x/jzDQD+SVUn0+nqoapxEbkTqfN0XgBfUdUuEbkHwHFVPQbgywC+JiJ9ACaRSvQA4HoA94hIHEACwPtVddJ47bcAfBVAAYDvGx/kAiKCIw0leIZtRohoB+gMhdFcWYz8HBY1kDPSSeIeEpEeADMAPiAiFQDS6uiqqg8DeHjZc59Y8ngWwFtXeN93AHxnlXseB9Ceztcn+x2uL8EPT4zg0vQ8SgpznQ6HiMgSqoqOUASvOMDjOuScdAobPgbgpQCOquoCgClc3iqECMDz5+J2SquR7qEIhjmlgmjHGY3OYTw2h0MsaiAHpTuttxbAr4rIu5E6u3ajdSFRNruyrgQiOyeJe8/9T+DPH+52OgwishmLGsgN1t1OFZFPAnglUlMXHkZqAsNjSDXaJXqB4jwfrtjt3xHjty5OzWM4Moue4YjToRCRzToHwxABDtZwJY6ck85K3K0AbgAwrKrvAXAVgDxLo6Ksdri+BM+cv4TUUI3tq28sBgAYGJ9ig2OiHaYzFMb+imIU5qZztJzIGukkcTOqmgQQNxrujgLYZ21YlM2ONJQgPLOAgfEpp0OxVO9IKolbSCjOTkw7HA0R2akzFEE7V+HIYekkccdFpATAlwA8CeApAL+wNCrKaofrSwFs/3NxvaPPT6boG+WUCqKdYiw6h+HILM/DkePSqU79gKpeUtUvAHgtgNuNbVWiFTVVFqM4z7ftz8X1jcawv6IIwPOrckS0/XUOsqiB3GHVzXwRuXqt11T1KWtComzn9QiurAtu/5W4kRiu21+GuXgSvaNM4oh2iq5QKolr43YqOWytE5l/vcZrCuDVGY6FtpEjDSX4+0f7MbuQ2JbdzCOzCxiOzKJpdzEuTs8ziSPaQTpCYewtL4I/P2f9i4kstGoSp6qvsjMQ2l4O15cinlR0hsI42rjL6XAyrs9I2por/bg0vYCfnp5AIqnwetYfSUdE2a0zFMHVe0qdDoMorT5xHwTwf1X1kvF5KYDbVPXvrA6Ostfh+ucnN2zLJG7ETOJSK3Hz8STOT06jsbzI4ciI3GE8NofHeseh2F6thubjSYQuzeDdL93jdChEac1Ofa+q3md+oqoXReS9AJjE0aoq/HmoKy3YtsUNvaNR5Po8qN9ViIvT88ZzMSZxRIa/euQkHnzivNNhWOZFe7ffL6eUfdJJ4jwiImp0bhURLwBONqd1HWkoxVNnLzodhiV6R2PYX1EMr0fQVFlsPBfFaw/udjgyInfoGozg2sZd+PStVzodSsYV5HqxO5DvdBhEaSVxjwD4poh8AamChvcD+IGlUdG2cLi+BA89O4jRyCwqt9l/8HpHYrjGOBPjz89BdTB/cYuVaKeLJ5I4NRLFu1+6h6vTRBZKp9nvRwH8J4DfAvBB4/FHrAyKtgfzXNzT26zVyNRcHKFLM2g2VuCAVG88VqgSpZyZmMZcPImWKrbgILJSOs1+k6r6BVW9FcB7ATyuqgnrQ6Ns11YTQI5Xtt25uNPGzNTm3c8ncc2VfvSNxpBMbq9D3ESb0T0UAQC0VjOJI7LSukmciPxYRAIisgvAMwDuF5HPWB8aZbv8HC8OVgfwzPntdS7OnM7QVOlffK55dzFmFhIIXZpxKiwi1+gZjsDnEeyv5FYqkZXS2U4NqmoEwFsA3K+q1wB4jbVh0XZxpKEUz10II7GNVqh6R2PI8Qr2lBUuPmdurfZxS5UIPUNR7K8oRp5v+zX6JnKTdJI4n4hUA3gbgO9aHA9tM4frSzA9n8Cpke0zIL5vNIq95UXI8T7/z2dphSrRTtczHEVLtX/9C4loS9JJ4u5BqkK1T1WfEJF9AHqtDYu2iyMNRnHDNjoX1zsaQ3PlC39AlRTmosKft7jVSrRThacXELo0w/NwRDZIp7DhW6p6pap+wPi8X1V/1frQaDto2FWIXUW52+Zc3OxCAucmpxdX3pZqZoUqEXqGU0UNLVVciSOy2qp94kTkI6r6aRH5HHD53BRV/R1LI6NtQURwuL5k26zEnR6LQfWFlamm5spifOepEFQVIpyhSjtTz3DqSAFX4oist1az327jz+N2BELb1+H6EvzXyVFEZhcQyM9xOpwtMQsXDuy+fJWhabcfsbk4hiOzqA4W2B0akSv0DEdQWpiDSn+e06EQbXurJnGq+pDx5wP2hUPb0eH6EqgCHRfC+KWmcqfD2ZLekRi8HkFj2eWtE8wK1d6RGJM42rFODEXRUhXgajSRDdbaTj221htV9ebMh0PbUVtNaluleyiS/UncaBSNZYXI9V1+nHQxiRuN4foDFXaHRuS4RFJxajiK265tcDoUoh1hre3UlwI4D+CfAPwcAH+tok0pK85DpT8P3UPZ336jdzSGA5UrH9guK87DrqJc9LHNCO1Q5yanMbOQYHsRIpusVZ1aBeDjANoB/H8AXgtgXFUfVdVH7QiOto+W6sBi1Vq2mosncHZiesWiBlNTZTHbjNCO1WOO2+LMVCJbrJrEqWpCVX+gqrcDeAmAPgA/FpHfti062jZaq/3oHYkhnkg6HcqmnRmfRiKpK7YXMZltRlS3z4QKonR1D0fhkZWrt4ko89bsEycieSLyFgD/COCDAD4L4J/tCIy2l9aqAOYTSfSPTzkdyqaZ0xiWN/pdqrmyGOGZBYzF5uwKi8g1uoci2FdRjPwcjtsissNahQ0PILWV+n0Ad6tqp21R0bZjnpHpHoqs2J4jG/SOxOARYF/F6kO9ze+tbySGSn++XaERuULPcARX1ZU4HQbRjrHWSty7ABwA8LsAfiYiEeMjKiLZfbiJbLevvBg5XllsBJqN+kZjaNhVuOYqQ9Pu5ytUiXaS6OwCzk9y3BaRndbqE5fOXFWitOT6PNhfUbx48Dkb9Y5G0bTGVioAVBTnIViQs7j1SrRTnBpJ/Z3nuC0i+zBRI9scrA5k7UrcQiKJgfGpdQ9siwiaK4txihWqtMOcGOK4LSK7MYkj27RU+zEUnsWl6XmnQ9mwsxPTWEjoYkPftTTvLl4cz0W0U/QMRRDI96E6yLOgRHZhEke2aakyJzdk32pcXxqVqaamSj8mp+YxwQpV2kF6hqNoqea4LSI7MYkj25gVqtnY9Nds4Lu/cvXKVNPS8VtEO0EyqTg5HEUrz8MR2YpJHNmm0p+P8uJc9GThSlzvaAx1pQUozF1rUl1KMytUaYcJXZpBbC6OFp6HI7IVkziyVUtVAN3ZuBI3GkvrPBwAVAXyUZznQ99I9iWrRJtxwhy3xSSOyFZM4shWLVV+nByOIpHMnrFUiaTi9FgMzWk2KRaR1AxVrsTRDtEzFIUIcIDjtohsxSSObNVSHcBcPIkzE9kzfuv85DTm48k1Z6Yu18wkjnaQnuEIGsuK0jpuQESZwySObGU2As2mc3FmMpbudiqQOhc3Fp3LynYqRBvVMxxlk18iBzCJI1s17y6G1yPozqLJDeb0hY2txBkzVLkaR9vc9HwcZyameB6OyAFM4shWeT4v9lcUZVWbkb6RGKqD+fDn56T9nia2GaEd4uRwFKoct0XkBCZxZLuWqkBWNfztHY1taBUOAGpLClCQ413sL0e0XZmj9LgSR2Q/JnFku5ZqP0KXZhCZXXA6lHUlk4q+0VhakxqW8njMCtXsSVaJNqNnKILiPB9qSwqcDoVox2ESR7Yzf2M/Oez+BCd0aQYzC4l1B9+vpLmSM1Rp++sejuKKKj88Ho7bIrKbpUmciNwkIidFpE9EPrbC63ki8g3j9Z+LSOOy1xtEJCYif7DkuTMi0iEiz4jIcSvjJ2u0Ls5Qdf+5uN7FmakbT+KadhdjKDyLaBasOBJthqqieyiC1mqehyNygmVJnIh4AdwH4PUADgK4TUQOLrvsDgAXVbUJwL0APrXs9XsBfH+F279KVQ+r6tEMh0022B3IQ0lhTlacizPPtG30TBzAClXa/gbDs4jOxtFSxfNwRE6wciXuWgB9qtqvqvMAHgRwy7JrbgHwgPH42wBuEBEBABF5M4B+AF0WxkgOEBG0VPmzokK1dzSGCn8eSgpzN/zeZlao0jbXszhuiytxRE6wMomrBXB+yecXjOdWvEZV4wDCAMpEpAjARwHcvcJ9FcC/i8iTIvK+jEdNtmitDuDkcBRJl4/f2sjM1OXqdxUi1+fhShxtW2Zl6hVciSNyhJVJ3EqnXJf/xF7tmrsB3KuqK/30+yVVvRqpbdoPisj1K35xkfeJyHEROT42NraRuMkGrVUBTM8ncG5y2ulQVqWq6BuJbjqJ83oE+yuK0Tvi/m1jos04MRRBw65CFOdx3BaRE6xM4i4AqF/yeR2AwdWuEREfgCCASQAvBvBpETkD4EMAPi4idwKAqg4af44C+Bektm0vo6pfVNWjqnq0oqIiU98TZUiLsf3i5i3VofAspuYTaEpz8P1KOEOVtrOeoQib/BI5yMok7gkAzSKyV0RyAbwdwLFl1xwDcLvx+FYAP9KUl6tqo6o2AvgbAH+uqn8rIkUi4gcAY8v1RgCdFn4PZJHmSj88AlcXN2xmZupyzZXFuHBxBtPz8UyFReQKswsJDIxPoYVNfokcY1kSZ5xxuxPAIwC6AXxTVbtE5B4Rudm47MtInYHrA/BhAJe1IVlmN4DHRORZAL8A8D1V/YE13wFZqSDXi8Zyd4/fMrdBD2xlJc7oL3d6dCojMRG5Re9IDEkFWrkSR+QYSw8yqOrDAB5e9twnljyeBfDWde5x15LH/QCuymyU5JTW6gA6LoSdDmNVfaMxlBXlYlfRxitTTU1Gm5He0SgO1QUzFRqR48w+j1yJI3IOJzaQY1qr/Dg3OY3YnDu3GjczM3W5PWWFyPEKz8XRttM9HEFBjhd7dhU6HQrRjsUkjhxjNgh14/gtVUXvSHRT47aWyvF6sLe8aLFpMNF20TPEcVtETmMSR45xc4XqWHQOkdn4hgffr6S50o++UfclqkSbparoGea4LSKnMYkjx9SWFMCf70OPCytUM1GZamqqLMa5yWnMLiS2fC8iNxiNzuHi9ALHbRE5jEkcOUZE0FoVWDwg7SZmZWrTFrdTgVSFalKB/jFWqNL2cGJx3BaTOCInMYkjR7VU+9EzHIWqu8Zv9Y7GECzIQUVx3pbv1bykQpVoOzBXz69gexEiRzGJI0e1VAUQm4vjwsUZp0N5AXNmqsjWD203lhfC6xHOUKVto2c4gtqSAgQLcpwOhWhHYxJHjnq+uMFdq1R9o7EtV6aa8nxe7CkrZIUqbRs9Q1GO2yJyASZx5KgrdvshAledi5uIzWFyan6xUW8mpGaouitRJdqMuXgCp8dii7+AEZFzmMSRo4ryfNizq9BVbUYyWZlqaq7048zENObirFCl7NY3GkM8qSxqIHIBJnHkuJaqgKvajCwmcRnaTjXvlUgqzoxPZ+yeRE4w/62yvQiR85jEkeNaqv0YmJjCzLw7Vqn6RqIozvOhKpCfsXua47u4pUrZrmc4gjyfB41lHLdF5DQmceS41uoAVIFTI+5IcMyZqZmoTDXtryiGCFjcQFmvZziKA7v98Hn544PIafxXSI5rNbZl3FLcYLYXyaT8HC8adhWyzQhlve4hjtsicgsmceS4utICFOV6XdFm5NL0PMaicxk9D2dihSplu7HoHMZj8zwPR+QSTOLIcR6P4IoqvytW4voWK1Mzv9LQVOnHwPgUFhLJjN+byA5mFTnbixC5A5M4coWW6oArxm+ZlalNGd5OBVIrcQsJxdkJVqhSdjIrU1u5EkfkCkziyBVaqwMIzyxgKDzraBy9IzEU5HhRW1KQ8XubW7R93FKlLNU9FEFVIB+lRblOh0JEYBJHLtFaZY7fcnZLtXc0iqbKYng8matMNS22GWGFKmWp7uEot1KJXIRJHLnCASOJ63a46W+fBZWppsJcH+pKCxa3bImyyUIiib7RKIsaiFyESRy5QiA/B3WlBY5WqEZnU9u5TRZUpppSFapM4ij79I9NYSGhbC9C5CJM4sg1WqsDjlaoWlmZamre7cfpsRgSSWcLOIg2arEylStxRK7BJI5co7XKj/6xGGYXnBm/ZcXg++WaKosxH0/i/CQrVCm7nBiKINfrwb6KIqdDISIDkzhyjZbqAJIKx6Ya9I3GkOvzoH6XdTMhmxdnqHJLlbJLz1Cq6CeH47aIXIP/Gsk1WhaLG5zZUu0diWJ/RTG8FlSmmhYrVNlmhLJMz3CElalELsMkjlxjT1kRCnKcG79lxczU5fz5OagO5qOPbUYoi0xOzWMkMscmv0QuwySOXMPrERxwaPzW9HwcFy7OWJ7EAanVOG6nUjbpMf5NtlYziSNyEyZx5CqtRhJn9/it06NTAGDJ4Pvlmiv96BuNIckKVcoS3cbqOLdTidyFSRy5SkuVHxenFzAWnbP165pn1JosbC9iat5djJmFBEKXZiz/WkSZ0DMUQXlxHsqL85wOhYiWYBJHrtJibNd023wurnc0hhyvYE+ZdZWpJnPL1qkqXKKN6hmOsskvkQsxiSNXMQ9O230urnckhn3l9rRPYIUqZZN4IolTI9HF6nEicg8mceQqwcIc1ATzFw9S26VvNGrpuK2lSgpzUeHPQy8rVCkLnJmYwlw8yaIGIhdiEkeu01IdsLXNyOxCAucmp22pTDVxhipli+4ho6iB7UWIXIdJHLlOS1WqenM+nrTl650eiyGp1s5MXa65shh9ozHbq3CJNqpnOAKfR7C/kuO2iNyGSRy5Tmt1APGk2nbwf3HwvU3bqQDQtNuP2Fwcw5FZ274m0Wb0DKUmmeT5vE6HQkTLMIkj1zGr4HqG7TkX1zsSg9cjaCyzb6VhcYYqz8WRy3UPRViZSuRSTOLIdRrLipDr89h2Lq53NIrGskLk+uz757CYxPFcHLlYeHoBg+HZxdY/ROQuTOLIdXxeDw7sLratzUhqZqq9Kw1lxXnYVZSLPrYZIRczV8PZXoTInZjEkSu1VNlToToXT+DsxLSt5+FMTZXF3E4lVzP/DbK9CJE7MYkjV2qtDmAsOofxmLXjt86MTyOR1MUGvHYy24ywQpXcqnsogtLCHFT6OW6LyI2YxJErtRrbNz1D1q7GmVMT7N5OTX3NYoRn7J8TS5Su7uEoWqsDEBGnQyGiFTCJI1e6osqeCtXekRg8AuyrsL8HVvPu1PfI4gZyo0RScWo4yia/RC7GJI5cqaw4D5X+vMVu8VbpG42hYVch8nPs74H1fJsRFjeQ+5ybnMbMQgItbC9C5FpM4si1WqsDlleo9o5G0eTAVioAVPjzEMj3cSWOXMmcX9zKlTgi12ISR67VUp0av7WQsGb81kIiiYHxKUcqUwFARNC8288kjlypeygCj9g7yYSINoZJHLlWa1UA80aiZYWzE9NYSKitg++XM2eoErlN93AU+yqKHTlqQETpYRJHrmWexbFqS7XPwcpUU1NlMSan5jFhcSsVoo3qGY6wyS+Ry1maxInITSJyUkT6RORjK7yeJyLfMF7/uYg0Lnu9QURiIvIH6d6Tto995cXI8YplTX/NRrv7K+2vTDWxQpXcKDq7gPOTM2zyS+RyliVxIuIFcB+A1wM4COA2ETm47LI7AFxU1SYA9wL41LLX7wXw/Q3ek7aJXJ8HTZV+y1biekdjqCstQC5raDIAAA4cSURBVGGuz5L7p4MzVMmNTi5OauBKHJGbWbkSdy2APlXtV9V5AA8CuGXZNbcAeMB4/G0AN4jRVVJE3gygH0DXBu9J20hrld+yhr+pmanOHtquDuajKNeLPrYZIRfpNpI49ogjcjcrk7haAOeXfH7BeG7Fa1Q1DiAMoExEigB8FMDdm7gnbSMt1X4MR2ZxcWo+o/dNJBWnx2KL25lOERE0sUKVXKZnKIJAvg/VwXynQyGiNViZxK00p2X5kMjVrrkbwL2quvwnWzr3TF0o8j4ROS4ix8fGxtYNltzJXAnI9Lm485PTmI8nHZmZupw5Q5XILXqGo2jhuC0i17MyibsAoH7J53UABle7RkR8AIIAJgG8GMCnReQMgA8B+LiI3JnmPQEAqvpFVT2qqkcrKiq2/t2QI8yD1Zk+F2cmTU5vp5oxjEXncGk6s6uNRJuRTCpODkcX5xcTkXtZmcQ9AaBZRPaKSC6AtwM4tuyaYwBuNx7fCuBHmvJyVW1U1UYAfwPgz1X1b9O8J20jFf48lBfnZnyGqjn43hUrcUYzVfaLIze4cHEGsbk4K1OJsoBlSZxxxu1OAI8A6AbwTVXtEpF7RORm47IvI3UGrg/AhwGs2TJktXta9T2QO7RUBTK+ndo3EkN1MB/+/JyM3nczzD513FIlN+g2fmFqYRJH5HqW9lZQ1YcBPLzsuU8seTwL4K3r3OOu9e5J21tLlR9f+++zSCQVXk9mzuj0jsZcsQoHALUlBcjP8aBnKILZhYTT4dAO1xUKQwQ4wHFbRK7nXIMsojS1VgcwF0+N38pE4vXQs4M4ORzFu166JwPRbZ3HI2iu9OOBx8/igcfPOh0OEfaVFznaP5GI0sN/peR65vitnuHIlpK4yal5/Mm/duJ7HUO4qr4Ev/HyvZkKccv+15vb8bPTE06HQQQAeFFjqdMhEFEamMSR6zVVFsPrEfQMRfHGKzd3j0e6hvHH/9KB8MwC/vB1V+A3r98Hn9c9o4Ovqi/BVfUlTodBRERZhEkcuV6ez4v9FUWbqlANTy/g7oe68M9Ph3CwOoCv3fFiVt0REdG2wCSOskJLVQBPnr24off8+OQoPvqd5zAem8fv3NCMO1/VhFyfe1bfiIiItoJJHGWF1uoAjj07iPDMAoIFa7cFic4u4M++140HnziPA7uL8X/e/SIcqgvaFCkREZE9mMRRVjCLG04OR3Ht3l2rXvezvnH84befw1B4Br/5in34vdccQH6O164wiYiIbMMkjrJC6+IM1ciKSdz0fByf+n4PHnj8LPaWF+Fb778O1+xhhR0REW1fTOIoK+wO5KGkMAfdQ5dPbjh+ZhJ/8K1ncWZiGv/zukZ89KYWFORy9Y2IiLY3JnGUFUQErVUBdA89X6E6u5DAZ354Cl/6f/2oLSnAP733JXjp/jIHoyQiIrIPkzjKGi3Vfjz4i/NIJhUdoTB+/1vPom80hv/x4gZ8/A2tKM7jX2ciIto5+FOPskZrVeD/b+/eg6cq6ziOvz/8xCyw0CAHxURJx0spqZApCTle09BMVCSVckwrjS42mjVIzuh4icY/LE3NxElCvBD+4aQOA6hdAEEIkryCSZJgKoqlBXz74zxr287u78buHs7u5zXDsOfZc57zPd95ZvnynLP78K//bObyWcu5Z/EaBvV/H9O+MpLR+wzKOzQzM7OmcxFnhVH6huqMRS/xxYOHMPnz+3f5cyNmZmatykWcFcZ+gz/IxMOHcsTHBnLM/rvkHY6ZmVmuXMRZYfTt6MOUsQfkHYaZmdk2wWsQmZmZmRWQizgzMzOzAnIRZ2ZmZlZALuLMzMzMCshFnJmZmVkBuYgzMzMzKyAXcWZmZmYF5CLOzMzMrIBcxJmZmZkVkIs4MzMzswJyEWdmZmZWQC7izMzMzArIRZyZmZlZASki8o6h4SStB16s8fZA4NUmhtPOnOvmcJ6bx7luDue5eZzr5ugqz3tExKCuOmmLIq4zkp6IiEPzjqMdONfN4Tw3j3PdHM5z8zjXzVGvPPt2qpmZmVkBuYgzMzMzKyAXcXBL3gG0Eee6OZzn5nGum8N5bh7nujnqkue2fybOzMzMrIg8E2dmZmZWQG1bxEk6XtLTkp6TdFne8bQySaslLZe0VNITecfTSiTdLmmdpBVlbTtLekTSs+nvnfKMsRXUyPMUSX9L43qppM/lGWOrkLS7pLmSVkr6s6RJqd3juo46ybPHdZ1J2kHSQknLUq5/lNr3lLQgjem7JW3f477b8XaqpA7gGeAYYA2wCBgfEU/lGliLkrQaODQi/NtDdSbpSGAjcGdEfDy1XQe8FhHXpP+g7BQRl+YZZ9HVyPMUYGNE/DjP2FqNpMHA4IhYImlHYDFwCjARj+u66STPp+NxXVeSBPSLiI2S+gKPA5OA7wD3R8QMSTcDyyLipp703a4zcSOB5yLihYj4NzADODnnmMx6LCIeBV6raD4ZmJZeTyP7YLatUCPP1gARsTYilqTXbwErgd3wuK6rTvJsdRaZjWmzb/oTwFHAvam9V2O6XYu43YCXyrbX4MHbSAE8LGmxpK/mHUwb2CUi1kL2QQ18JOd4WtlFkv6Ubrf69l6dSRoKfBJYgMd1w1TkGTyu605Sh6SlwDrgEeB54I2I2JR26VUd0q5FnKq0td995eY5IiIOBk4AvpFuTZkV3U3AMGA4sBaYmm84rUVSf+A+4FsR8Wbe8bSqKnn2uG6AiNgcEcOBIWR3A/ertltP+23XIm4NsHvZ9hDg5ZxiaXkR8XL6ex0wi2wAW+O8kp53KT33si7neFpSRLySPpi3ALficV036bmh+4C7IuL+1OxxXWfV8uxx3VgR8QYwDzgMGCBpu/RWr+qQdi3iFgF7p2+GbA+cCTyQc0wtSVK/9NAskvoBxwIrOj/KttIDwLnp9bnA7BxjaVmlgiL5Ah7XdZEeAv8FsDIiflL2lsd1HdXKs8d1/UkaJGlAev1+4GiyZxDnAqel3Xo1ptvy26kA6WvTNwAdwO0RcVXOIbUkSXuRzb4BbAdMd67rR9KvgTHAQOAV4ArgN8BM4KPAX4FxEeGH8rdCjTyPIbvlFMBq4ILSM1vWe5JGAY8By4Etqflysue1PK7rpJM8j8fjuq4kHUj2xYUOssmzmRFxZfr3cQawM/Ak8KWIeLdHfbdrEWdmZmZWZO16O9XMzMys0FzEmZmZmRWQizgzMzOzAnIRZ2ZmZlZALuLMzMzMCshFnJk1hKSQNLVs+5K0aHw9+r5D0mld77nV5xknaaWkuRXtQ9P1XVzWdqOkiV30d6Gkc7rYZ6KkG2u8t7Fae72k61pRtn2+pCVeesls2+Qizswa5V3gVEkD8w6knKSOHux+HvD1iPhslffWAZPSD4Z3S0TcHBF39uD8dVP2y/Dd3f9s4GLg2Ih4vTFRmdnWcBFnZo2yCbgF+HblG5UzaaUZJkljJM2XNFPSM5KukTRB0kJJyyUNK+vmaEmPpf1OSsd3SLpe0qK0gPcFZf3OlTSd7MdNK+MZn/pfIena1DYZGAXcLOn6Kte3HpjD/1YRKO9vmKTfSlqcYtw3tU+RdEl6PSLF+IcUc/kv4++ajn9W0nUVfU9Ns2NzJA1KbcMl/TH1N6s0cyZpnqSrJc0nKzjHpWtcJunRKtdUOsfpwGVkBdyrtfYzs3y5iDOzRvopMEHSh3pwzEHAJOATwNnAPhExEriNbGaoZCgwGjiRrNDagWzmbENEjABGAOdL2jPtPxL4QUTsX34ySbsC1wJHkf1S/QhJp0TElcATwISI+F6NWK8Bvltldu8W4OKIOAS4BPhZlWN/CVwYEZ8GNle8Nxw4I+XgDEmltZ77AUsi4mBgPtnKEQB3ApdGxIFkReoVZX0NiIjRETEVmAwcFxEHAWNrXNMewI1kBdzfa+xjZtsAF3Fm1jAR8SZZgfHNHhy2KCLWpuVnngceTu3LyQq3kpkRsSUingVeAPYlW5v3HElLyZZp+jCwd9p/YUSsqnK+EcC8iFgfEZuAu4Aju3l9q4CFwFmlNkn9gcOBe1IcPwfK16MkraO4Y0T8PjVNr+h6TkRsiIh3gKfICivIlke6O73+FTAqFcgDImJ+ap9WEf/dZa9/B9wh6XyyJYCqWU+2rNXpNS/czLYJPXpGwsysF24AlpDNPJVsIv0nMi3EXf5cWfnagVvKtrfw/59ZlWsGBiCyGbCHyt+QNAZ4u0Z86vIKOnc1cC9Quj3ZB3gjIoZ3ckxX5yzPwWZqf1Z3Z93E9647Ii6U9Cmy2culkoZHxD8q9v8ncALwuKR1EXFXN85hZjnwTJyZNVRapHwm2a3OktXAIen1yUDfXnQ9TlKf9JzcXsDTwEPA1yT1BZC0j6R+XfSzABgtaWC6LTqe7FZlt0TEX8hmy05K228CqySNSzFI0kEVx7wOvCXpsNR0ZjdP1wcoPUt4FvB4RGwAXpf0mdR+dq34JQ2LiAURMRl4Fdi92n4RsR44Hrha0nHdjM3MmswzcWbWDFOBi8q2bwVmS1pI9uWAWrNknXmarFjZhezZsnck3UZ2y3VJmuFbD5zSWScRsVbS94G5ZDNkD0bE7B7GchXwZNn2BOAmST8kK1BnAMsqjjkPuFXS28A8YEM3zvM2cICkxWn/M1L7uWTPBX6A7Nbyl2scf72kvcmuc06VmN4TEaskjQUelHRqRCzoRnxm1kSK6M5svJmZ1ZOk/hFR+lbuZcDgiJiUc1hmViCeiTMzy8eJaQZwO+BFYGK+4ZhZ0XgmzszMzKyA/MUGMzMzswJyEWdmZmZWQC7izMzMzArIRZyZmZlZAbmIMzMzMysgF3FmZmZmBfRfAPgrzPY4TBMAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 720x432 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"%matplotlib inline\n",
"\n",
"MSE = [1 - x for x in cv_scores]\n",
"\n",
"optimal_k = neighbors[MSE.index(min(MSE))]\n",
"print(\"The optimal number of neighbors is %d\" % optimal_k)\n",
"\n",
"fig, ax = plt.subplots(figsize=(10, 6))\n",
"ax.plot(neighbors, MSE)\n",
"ax.set_xlabel('Number of Neighbors K')\n",
"ax.set_ylabel('Misclassification Error')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 前やったニューラルネットワークを使ってみよう\n",
"# 今回はkerasだよ"
]
},
{
"cell_type": "code",
"execution_count": 92,
"metadata": {},
"outputs": [],
"source": [
"from keras.models import Sequential\n",
"from keras.layers import Dense, Activation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 中間のactivationにはreluを、出力層のものにはsoftmaxを使うよ\n",
"# ちなみに最適化には最新のADAMを使うよ\n",
"- https://keras.io/ja/optimizers/"
]
},
{
"cell_type": "code",
"execution_count": 93,
"metadata": {},
"outputs": [],
"source": [
"model = Sequential()\n",
"model.add(Dense(6, input_dim=4))\n",
"model.add(Activation('relu'))\n",
"model.add(Dense(3, input_dim=12))\n",
"model.add(Activation('softmax'))\n",
"model.compile(optimizer='Adam', \n",
" loss='categorical_crossentropy',\n",
" metrics=['accuracy']\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 前回、前処理でやったエンコーダーを使うよ\n",
"これがないとエラーが起こるから注意。ソフトマックスではカテゴリカルなラベルが必要"
]
},
{
"cell_type": "code",
"execution_count": 94,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.preprocessing import LabelEncoder\n",
"from keras.utils import np_utils"
]
},
{
"cell_type": "code",
"execution_count": 95,
"metadata": {},
"outputs": [],
"source": [
"categol_y = np_utils.to_categorical(y)"
]
},
{
"cell_type": "code",
"execution_count": 96,
"metadata": {},
"outputs": [],
"source": [
"X_train, X_test, y_train, y_test = train_test_split(X, categol_y, random_state=0, train_size=0.7)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# とりあえず一回やってみる.その後に交差検証する"
]
},
{
"cell_type": "code",
"execution_count": 97,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/kiwamizamurai/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:1: UserWarning:\n",
"\n",
"The `nb_epoch` argument in `fit` has been renamed `epochs`.\n",
"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/100\n",
"105/105 [==============================] - 1s 5ms/step - loss: 2.6313 - acc: 0.3714\n",
"Epoch 2/100\n",
"105/105 [==============================] - 0s 312us/step - loss: 2.0706 - acc: 0.3714\n",
"Epoch 3/100\n",
"105/105 [==============================] - 0s 293us/step - loss: 1.7114 - acc: 0.3619\n",
"Epoch 4/100\n",
"105/105 [==============================] - 0s 258us/step - loss: 1.4884 - acc: 0.2000\n",
"Epoch 5/100\n",
"105/105 [==============================] - 0s 228us/step - loss: 1.3478 - acc: 0.1619\n",
"Epoch 6/100\n",
"105/105 [==============================] - 0s 228us/step - loss: 1.2316 - acc: 0.1619\n",
"Epoch 7/100\n",
"105/105 [==============================] - 0s 245us/step - loss: 1.1312 - acc: 0.1333\n",
"Epoch 8/100\n",
"105/105 [==============================] - 0s 257us/step - loss: 1.0486 - acc: 0.1619\n",
"Epoch 9/100\n",
"105/105 [==============================] - 0s 229us/step - loss: 0.9687 - acc: 0.2476\n",
"Epoch 10/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.9146 - acc: 0.4095\n",
"Epoch 11/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.8829 - acc: 0.4476\n",
"Epoch 12/100\n",
"105/105 [==============================] - 0s 226us/step - loss: 0.8590 - acc: 0.4667\n",
"Epoch 13/100\n",
"105/105 [==============================] - 0s 246us/step - loss: 0.8328 - acc: 0.4571\n",
"Epoch 14/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.8098 - acc: 0.4667\n",
"Epoch 15/100\n",
"105/105 [==============================] - 0s 235us/step - loss: 0.7952 - acc: 0.4762\n",
"Epoch 16/100\n",
"105/105 [==============================] - 0s 235us/step - loss: 0.7669 - acc: 0.4952\n",
"Epoch 17/100\n",
"105/105 [==============================] - 0s 247us/step - loss: 0.7486 - acc: 0.5238\n",
"Epoch 18/100\n",
"105/105 [==============================] - 0s 229us/step - loss: 0.7323 - acc: 0.5333\n",
"Epoch 19/100\n",
"105/105 [==============================] - 0s 232us/step - loss: 0.7151 - acc: 0.5048\n",
"Epoch 20/100\n",
"105/105 [==============================] - 0s 225us/step - loss: 0.6990 - acc: 0.5524\n",
"Epoch 21/100\n",
"105/105 [==============================] - 0s 243us/step - loss: 0.6869 - acc: 0.5333\n",
"Epoch 22/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.6699 - acc: 0.5429\n",
"Epoch 23/100\n",
"105/105 [==============================] - 0s 232us/step - loss: 0.6560 - acc: 0.5905\n",
"Epoch 24/100\n",
"105/105 [==============================] - 0s 229us/step - loss: 0.6444 - acc: 0.6095\n",
"Epoch 25/100\n",
"105/105 [==============================] - 0s 230us/step - loss: 0.6384 - acc: 0.5524\n",
"Epoch 26/100\n",
"105/105 [==============================] - 0s 221us/step - loss: 0.6206 - acc: 0.6286\n",
"Epoch 27/100\n",
"105/105 [==============================] - 0s 229us/step - loss: 0.6107 - acc: 0.6667\n",
"Epoch 28/100\n",
"105/105 [==============================] - 0s 226us/step - loss: 0.6002 - acc: 0.6286\n",
"Epoch 29/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.5915 - acc: 0.6190\n",
"Epoch 30/100\n",
"105/105 [==============================] - 0s 230us/step - loss: 0.5813 - acc: 0.6381\n",
"Epoch 31/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.5714 - acc: 0.6571\n",
"Epoch 32/100\n",
"105/105 [==============================] - 0s 231us/step - loss: 0.5660 - acc: 0.6286\n",
"Epoch 33/100\n",
"105/105 [==============================] - 0s 232us/step - loss: 0.5626 - acc: 0.6762\n",
"Epoch 34/100\n",
"105/105 [==============================] - 0s 238us/step - loss: 0.5501 - acc: 0.6571\n",
"Epoch 35/100\n",
"105/105 [==============================] - 0s 233us/step - loss: 0.5405 - acc: 0.6667\n",
"Epoch 36/100\n",
"105/105 [==============================] - 0s 233us/step - loss: 0.5335 - acc: 0.6952\n",
"Epoch 37/100\n",
"105/105 [==============================] - 0s 231us/step - loss: 0.5262 - acc: 0.6857\n",
"Epoch 38/100\n",
"105/105 [==============================] - 0s 237us/step - loss: 0.5243 - acc: 0.6857\n",
"Epoch 39/100\n",
"105/105 [==============================] - 0s 242us/step - loss: 0.5121 - acc: 0.6857\n",
"Epoch 40/100\n",
"105/105 [==============================] - 0s 231us/step - loss: 0.5128 - acc: 0.6762\n",
"Epoch 41/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.5009 - acc: 0.7238\n",
"Epoch 42/100\n",
"105/105 [==============================] - 0s 254us/step - loss: 0.4966 - acc: 0.7238\n",
"Epoch 43/100\n",
"105/105 [==============================] - 0s 273us/step - loss: 0.4872 - acc: 0.7238\n",
"Epoch 44/100\n",
"105/105 [==============================] - 0s 260us/step - loss: 0.4848 - acc: 0.7048\n",
"Epoch 45/100\n",
"105/105 [==============================] - 0s 253us/step - loss: 0.4780 - acc: 0.7524\n",
"Epoch 46/100\n",
"105/105 [==============================] - 0s 262us/step - loss: 0.4711 - acc: 0.7524\n",
"Epoch 47/100\n",
"105/105 [==============================] - 0s 239us/step - loss: 0.4715 - acc: 0.7429\n",
"Epoch 48/100\n",
"105/105 [==============================] - 0s 261us/step - loss: 0.4606 - acc: 0.7619\n",
"Epoch 49/100\n",
"105/105 [==============================] - 0s 244us/step - loss: 0.4564 - acc: 0.7619\n",
"Epoch 50/100\n",
"105/105 [==============================] - 0s 255us/step - loss: 0.4531 - acc: 0.7524\n",
"Epoch 51/100\n",
"105/105 [==============================] - 0s 268us/step - loss: 0.4487 - acc: 0.7524\n",
"Epoch 52/100\n",
"105/105 [==============================] - 0s 257us/step - loss: 0.4406 - acc: 0.7714\n",
"Epoch 53/100\n",
"105/105 [==============================] - 0s 248us/step - loss: 0.4386 - acc: 0.8000\n",
"Epoch 54/100\n",
"105/105 [==============================] - 0s 243us/step - loss: 0.4323 - acc: 0.8000\n",
"Epoch 55/100\n",
"105/105 [==============================] - 0s 239us/step - loss: 0.4298 - acc: 0.8190\n",
"Epoch 56/100\n",
"105/105 [==============================] - 0s 251us/step - loss: 0.4237 - acc: 0.8190\n",
"Epoch 57/100\n",
"105/105 [==============================] - 0s 252us/step - loss: 0.4256 - acc: 0.8000\n",
"Epoch 58/100\n",
"105/105 [==============================] - 0s 250us/step - loss: 0.4227 - acc: 0.7714\n",
"Epoch 59/100\n",
"105/105 [==============================] - 0s 270us/step - loss: 0.4136 - acc: 0.8000\n",
"Epoch 60/100\n",
"105/105 [==============================] - 0s 251us/step - loss: 0.4089 - acc: 0.8476\n",
"Epoch 61/100\n",
"105/105 [==============================] - 0s 247us/step - loss: 0.4015 - acc: 0.8476\n",
"Epoch 62/100\n",
"105/105 [==============================] - 0s 251us/step - loss: 0.4004 - acc: 0.8286\n",
"Epoch 63/100\n",
"105/105 [==============================] - 0s 254us/step - loss: 0.3967 - acc: 0.8667\n",
"Epoch 64/100\n",
"105/105 [==============================] - 0s 252us/step - loss: 0.3912 - acc: 0.8476\n",
"Epoch 65/100\n",
"105/105 [==============================] - 0s 275us/step - loss: 0.3853 - acc: 0.8667\n",
"Epoch 66/100\n",
"105/105 [==============================] - 0s 256us/step - loss: 0.3835 - acc: 0.8857\n",
"Epoch 67/100\n",
"105/105 [==============================] - 0s 256us/step - loss: 0.3799 - acc: 0.8857\n",
"Epoch 68/100\n",
"105/105 [==============================] - 0s 272us/step - loss: 0.3759 - acc: 0.8857\n",
"Epoch 69/100\n",
"105/105 [==============================] - 0s 246us/step - loss: 0.3760 - acc: 0.8667\n",
"Epoch 70/100\n",
"105/105 [==============================] - 0s 241us/step - loss: 0.3688 - acc: 0.9238\n",
"Epoch 71/100\n",
"105/105 [==============================] - 0s 252us/step - loss: 0.3638 - acc: 0.9143\n",
"Epoch 72/100\n",
"105/105 [==============================] - 0s 247us/step - loss: 0.3597 - acc: 0.9143\n",
"Epoch 73/100\n",
"105/105 [==============================] - 0s 261us/step - loss: 0.3589 - acc: 0.9048\n",
"Epoch 74/100\n",
"105/105 [==============================] - 0s 249us/step - loss: 0.3538 - acc: 0.9238\n",
"Epoch 75/100\n",
"105/105 [==============================] - 0s 252us/step - loss: 0.3497 - acc: 0.9238\n",
"Epoch 76/100\n",
"105/105 [==============================] - 0s 252us/step - loss: 0.3482 - acc: 0.9143\n",
"Epoch 77/100\n",
"105/105 [==============================] - 0s 245us/step - loss: 0.3436 - acc: 0.9333\n",
"Epoch 78/100\n",
"105/105 [==============================] - 0s 267us/step - loss: 0.3394 - acc: 0.9333\n",
"Epoch 79/100\n",
"105/105 [==============================] - 0s 258us/step - loss: 0.3370 - acc: 0.9238\n",
"Epoch 80/100\n",
"105/105 [==============================] - 0s 249us/step - loss: 0.3333 - acc: 0.9333\n",
"Epoch 81/100\n",
"105/105 [==============================] - 0s 251us/step - loss: 0.3330 - acc: 0.9333\n",
"Epoch 82/100\n",
"105/105 [==============================] - 0s 274us/step - loss: 0.3279 - acc: 0.9238\n",
"Epoch 83/100\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"105/105 [==============================] - 0s 243us/step - loss: 0.3233 - acc: 0.9143\n",
"Epoch 84/100\n",
"105/105 [==============================] - 0s 261us/step - loss: 0.3224 - acc: 0.9333\n",
"Epoch 85/100\n",
"105/105 [==============================] - 0s 289us/step - loss: 0.3206 - acc: 0.9143\n",
"Epoch 86/100\n",
"105/105 [==============================] - 0s 247us/step - loss: 0.3143 - acc: 0.9333\n",
"Epoch 87/100\n",
"105/105 [==============================] - 0s 229us/step - loss: 0.3108 - acc: 0.9333\n",
"Epoch 88/100\n",
"105/105 [==============================] - 0s 215us/step - loss: 0.3093 - acc: 0.9238\n",
"Epoch 89/100\n",
"105/105 [==============================] - 0s 220us/step - loss: 0.3068 - acc: 0.9333\n",
"Epoch 90/100\n",
"105/105 [==============================] - 0s 224us/step - loss: 0.3020 - acc: 0.9333\n",
"Epoch 91/100\n",
"105/105 [==============================] - 0s 237us/step - loss: 0.3049 - acc: 0.9143\n",
"Epoch 92/100\n",
"105/105 [==============================] - 0s 232us/step - loss: 0.2995 - acc: 0.9333\n",
"Epoch 93/100\n",
"105/105 [==============================] - 0s 239us/step - loss: 0.2972 - acc: 0.9238\n",
"Epoch 94/100\n",
"105/105 [==============================] - 0s 221us/step - loss: 0.2923 - acc: 0.9238\n",
"Epoch 95/100\n",
"105/105 [==============================] - 0s 225us/step - loss: 0.2909 - acc: 0.9429\n",
"Epoch 96/100\n",
"105/105 [==============================] - 0s 223us/step - loss: 0.2853 - acc: 0.9238\n",
"Epoch 97/100\n",
"105/105 [==============================] - 0s 228us/step - loss: 0.2838 - acc: 0.9333\n",
"Epoch 98/100\n",
"105/105 [==============================] - 0s 227us/step - loss: 0.2800 - acc: 0.9238\n",
"Epoch 99/100\n",
"105/105 [==============================] - 0s 224us/step - loss: 0.2774 - acc: 0.9333\n",
"Epoch 100/100\n",
"105/105 [==============================] - 0s 235us/step - loss: 0.2749 - acc: 0.9333\n"
]
},
{
"data": {
"text/plain": [
"<keras.callbacks.History at 0x1a37a03898>"
]
},
"execution_count": 97,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.fit(X_train, y_train, nb_epoch=100, batch_size=5)"
]
},
{
"cell_type": "code",
"execution_count": 98,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy = 0.93\n"
]
}
],
"source": [
"loss, accuracy = model.evaluate(X_test, y_test, verbose=0)\n",
"print(\"Accuracy = {:.2f}\".format(accuracy))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# この0.93のいう数値の妥当性を検証する！\n",
"ちなみにcross_val_score()を使うとエラーが出た。なのでkeras既存のもので行おう"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- https://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/\n",
"\n",
"関数を作らないとダメらしいので下ではmyとした"
]
},
{
"cell_type": "code",
"execution_count": 99,
"metadata": {},
"outputs": [],
"source": [
"def my():\n",
" model = Sequential()\n",
" model.add(Dense(6, input_dim=4))\n",
" model.add(Activation('relu'))\n",
" model.add(Dense(3, input_dim=12))\n",
" model.add(Activation('softmax'))\n",
" model.compile(optimizer='ADAM', \n",
" loss='categorical_crossentropy',\n",
" metrics=['accuracy']\n",
" )\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": 100,
"metadata": {},
"outputs": [],
"source": [
"from keras.wrappers.scikit_learn import KerasClassifier\n",
"from sklearn.model_selection import cross_val_score\n",
"from sklearn.model_selection import KFold"
]
},
{
"cell_type": "code",
"execution_count": 101,
"metadata": {},
"outputs": [],
"source": [
"estimator = KerasClassifier(build_fn=my, epochs=200, batch_size=5, verbose=0)"
]
},
{
"cell_type": "code",
"execution_count": 102,
"metadata": {},
"outputs": [],
"source": [
"import numpy\n",
"seed = 1\n",
"numpy.random.seed(seed)\n",
"\n",
"kfold = KFold(n_splits=10, shuffle=True, random_state=seed)"
]
},
{
"cell_type": "code",
"execution_count": 103,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Baseline: 96.67% (4.47%)\n"
]
}
],
"source": [
"results = cross_val_score(estimator, X, dummy_y, cv=kfold)\n",
"print(\"Baseline: %.2f%% (%.2f%%)\" % (results.mean()*100, results.std()*100))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# これで君はもうCross-Validatorだ！\n",
"\n",
"次回は回帰において交差検証をしてみるよ。楽しみにしておいて"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment