Skip to content

Instantly share code, notes, and snippets.

@ohtomi

ohtomi/learning-machine-learning-with-jupyter.ipynb

Last active November 23, 2015 20:03
Show Gist options
  • Save ohtomi/10f99cf133bf6244e912 to your computer and use it in GitHub Desktop.
Save ohtomi/10f99cf133bf6244e912 to your computer and use it in GitHub Desktop.
Download ZIP
Tech Talk #2 (2015/11/24)
Display the source blob
Display the rendered blob
Raw
learning-machine-learning-with-jupyter.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"internals": {
"slide_helper": "subslide_end",
"slide_type": "subslide"
},
"slide_helper": "slide_end",
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Jupyterで始める機械学習\n",
"<span></span> \n",
"<span></span> \n",
"<span></span> \n",
"<span></span> \n",
"<div style=\"text-align:right\">\n",
"2015/11/24 Kenichi Ohtomi\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 本日のゴール\n",
"\n",
"* 機械学習の手順を理解する\n",
"* 実際に機械学習をやってみたくなる\n",
"* 対話的環境の手軽さを感じてもらう\n",
"\n",
"<span></span> \n",
"<span></span> \n",
"\n",
"# 話さないこと\n",
"\n",
"* アルゴリズム, 手法の数学的証明\n",
"* 機械学習でどんなことができるか, 人工知能について\n",
"* Jupyter, Spark, Chainerのアーキテクチャー"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 機械学習の例題\n",
"\n",
"* データの予測\n",
"* データの分類\n",
"* データのクラスタリング"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# データの予測"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"N件の2つ組のデータ(x, y)がある.\n",
"データを眺めてみると, どうやらxが決まるとyが定まるようだ.\n",
"\n",
"このとき, 未知のxに対するyを予測したい."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from numpy.random import normal"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEepJREFUeJzt3X+MZedd3/H3p7uOtOFHTGRk4x9hoXFoXNHIQSwLaeWL\nwLsbIzndFhrS0pCAQhTJgKoBgg2qp+KPKKpWQsHCWFGSumobB0E2XWOn10vrCwaBWxP/ItkVXsBi\nHcIWcBxCsgg7/vaPe73ezs7cuTPn/th57vslXc358dx5nkdz5zNnnnOec1JVSJLa8g8W3QBJ0vQZ\n7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDeoU7kmuSfJgkk8n+cMkP7FBuQ8keSrJ40mu71KnJGlzuzu+\n/3ng31XVY0m+GviDJMer6sRLBZLcBLy2qq5N8h3AncD+jvVKksbodOReVX9RVY+Nlv8WOAFcuabY\nzcDdozIPA5cmubxLvZKk8aY25p5kL3A98PCaXVcBp89bfwa4elr1SpIuNJVwHw3J/Brwk6Mj+AuK\nrFn3ngeSNENdx9xJcgnw68B/qapPrFPks8A1561fPdq29vsY+JK0DVW19gC689UyAT4EfKaqfnGD\nYseAt4/K7weeq6ozGzRwbq/bb799rvXZN/tn/+zfLF4b6Xrk/ibgh4Ankjw62nYb8JpRWN9VVfcn\nuSnJKeBLwDs71ilJ2kSncK+q32GCo/+quqVLPZKkrVnaGaq9Xm/RTZiZlvsG9m+ns3/zkXFjNvOU\npC6WtkjSTpGEmvYJVUnSxclwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJek\nBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1qHO4J/lwkjNJntxgfy/JF5I8Onr9fNc6\nJUnjdXpA9shHgF8C/vOYMr9VVTdPoS5J0gQ6H7lX1UPA5zcpdsHz/SRJszOPMfcCvivJ40nuT3Ld\nHOqUpKU2jWGZzXwKuKaqvpzkzcAngNfNoV5JWlozD/eq+uJ5y59M8stJXl1Vz64tu7q6em651+vR\n6/Vm3TxJ2lEGgwGDwWDTcqmqzpUl2QvcW1Xfus6+y4H/W1WVZB/wq1W1d51yNY22SNIySUJVXXBe\ns/ORe5KPAjcAlyU5DdwOXAJQVXcB3w+8J8kLwJeBH+xapyRpvKkcuU+DR+6StHUbHbk7Q1WSGmS4\nS1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuC+Jfh8OHBi++v1Ft0bSrHn7gSXQ\n78Phw3D27HB9zx44ehQOHlxsuyR15+0HltiRIy8HOwyXjxxZXHskzZ7hLkkNMtyXwMrKcCjmJXv2\nDLdJapdj7kui3395KGZlxfF2qRUbjbkb7pK0g3lCVZKWiOEuSQ0y3CWpQZ3DPcmHk5xJ8uSYMh9I\n8lSSx5Nc37VOSdJ40zhy/whwaKOdSW4CXltV1wI/Btw5hTolSWN0Dveqegj4/JgiNwN3j8o+DFya\n5PKu9UqSNjaPMfergNPnrT8DXD2HeiVpae2eUz1rr8Fc94L21dXVc8u9Xo9erze7FknSDjQYDBgM\nBpuWm8okpiR7gXur6lvX2fcrwKCq7hmtnwRuqKoza8o5iUmStmiRk5iOAW8fNWI/8NzaYJckTVfn\nYZkkHwVuAC5Lchq4HbgEoKruqqr7k9yU5BTwJeCdXeuUJI3nvWUkaQfz3jKStEQMd0lqkOEuSQ0y\n3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlaQr6fThwYPjq9xfdGsNdc7DI\nD/3F9gunNvX7cPgwHD8+fB0+vPjPm3eF1Ey99KE/e3a4vmcPHD0KBw+2XbeWy4EDw1A/3403wgMP\nzL5u7wqphThy5OVwheHykSPt1y0tmuEuSR2trAz/M3zJnj3DbYtkuGumFvmhvxh/4dSmgweHQ343\n3jh8XQzDf465a+b6/ZeHQ1ZW5vuhX2Td0jxsNOZuuEvSDjazE6pJDiU5meSpJO9dZ38vyReSPDp6\n/XzXOiVJ4+3u8uYku4A7gO8FPgv8nyTHqurEmqK/VVU3d6lLkjS5rkfu+4BTVfV0VT0P3AO8ZZ1y\nF/zLIEmana7hfhVw+rz1Z0bbzlfAdyV5PMn9Sa7rWKckaROdhmUYBvdmPgVcU1VfTvJm4BPA69Yr\nuLq6em651+vR6/U6Nk+S2jIYDBgMBpuW63S1TJL9wGpVHRqt3wq8WFXvH/OePwW+raqeXbPdq2Uk\naYtmdbXMI8C1SfYmeQXwVuDYmoovT5LR8j6Gf1CevfBbSZKmpdOwTFW9kOQWoA/sAj5UVSeSvHu0\n/y7g+4H3JHkB+DLwgx3bLEnahJOYJGkH866QkrREDPc58sERkubFYZk58cERkmbBYZkF88ERkubJ\ncJekBhnuc+KDIyTNk2Puc+SDIyRNmw/rkKQGeUJVkpaI4S5JDTLcJalBhrskNchwl6QGGe6S1CDD\nXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQZ3DPcmhJCeTPJXkvRuU+cBo/+NJru9apyRpvE7hnmQXcAdw\nCLgOeFuS168pcxPw2qq6Fvgx4M4udUqSNtf1yH0fcKqqnq6q54F7gLesKXMzcDdAVT0MXJrk8o71\nSpLG6BruVwGnz1t/ZrRtszJXd6xXkjTG7o7vn/QG7GvvNbzu+1ZXV88t93o9er3etholSa0aDAYM\nBoNNy3V6WEeS/cBqVR0ard8KvFhV7z+vzK8Ag6q6Z7R+Erihqs6s+V4+rEOStmhWD+t4BLg2yd4k\nrwDeChxbU+YY8PZRI/YDz60NdknSdHUalqmqF5LcAvSBXcCHqupEkneP9t9VVfcnuSnJKeBLwDs7\nt1qSNJbPUJWkHcxnqErSEjHcJalBhrukZvT7cODA8NXvL7o1i+WYu6Qm9Ptw+DCcPTtc37MHjh6F\ngwcX265Zc8xdUtOOHHk52GG4fOTI4tqzaIa7JDXIcJfUhJWV4VDMS/bsGW5bVo65S2pGv//yUMzK\nSvvj7bDxmLvhLkk7mCdUR7xUStIyWKoj92W9VEpSuzxyx0ulJC2PpQp3SVoWSxXuXiolaVks1Zg7\nLOelUpLa5aWQktQgT6hK0hIx3CWpQYa7JDVo2w/ITvJq4GPANwJPA/+qqp5bp9zTwN8AXwGer6p9\n261TkjSZLkfuPwscr6rXAf9ztL6eAnpVdb3BLknz0SXcbwbuHi3fDfzzMWUvOJMrSZqdLuF+eVWd\nGS2fAS7foFwBv5nkkSTv6lCfJGlCY8fckxwHrlhn18+dv1JVlWSji9TfVFWfS/L1wPEkJ6vqofUK\nrq6unlvu9Xr0er1xzZOkpTMYDBgMBpuW2/YkpiQnGY6l/0WSbwAerKp/tMl7bgf+tqouuF2Xk5gk\naetmMYnpGPDDo+UfBj6xTqWvTPI1o+WvAg4AT3aoU5I0gS5H7q8GfhV4DeddCpnkSuCDVfV9Sb4Z\n+PjoLbuB/1pV79vg+3nkLklb5L1lJKlB3ltGkpaI4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIa\nZLhLUoMMd2lG+n04cGD46vcX3RotG2eoSjPQ78Phw3D27HB9zx44ehQOHlxsu9QeZ6hKc3TkyMvB\nDsPlIxfcC1WaHcNdkhpkuEszsLIyHIp5yZ49w23SvDjmLs1Iv//yUMzKiuPtmg1v+StJDfKEqiQt\nEcNdkhpkuEtSgwx3SWrQtsM9yQ8k+XSSryR545hyh5KcTPJUkvdutz5J0uS6HLk/CRwGfnujAkl2\nAXcAh4DrgLcleX2HOiVJE9i93TdW1UkYXoYzxj7gVFU9PSp7D/AW4MR265UkbW7WY+5XAafPW39m\ntE2SNENjj9yTHAeuWGfXbVV17wTff0uzklZXV88t93o9er3eVt4uacTZse0aDAYMBoNNy3WeoZrk\nQWClqj61zr79wGpVHRqt3wq8WFXvX6esM1SlKfB2w8tl1jNUNxp4fwS4NsneJK8A3gocm1Kdktbh\n7YYF3S6FPJzkNLAfuC/JJ0fbr0xyH0BVvQDcAvSBzwAfqypPpkrSjHnjMKkxix6Wcbx/vrwrpLRE\nFhWwi/7DsowMd0kzd+AAHD/+/2+78UZ44IHFtGcZeMtfSVoihrukqfHxghcPh2UkTZUnVOfLMXdJ\napBj7pK0RAx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWp\nQV2eofoDST6d5CtJ3jim3NNJnkjyaJL/vd36JEmT293hvU8Ch4G7NilXQK+qnu1QlyRpC7Yd7lV1\nEoa3m5zARIUkSdMxjzH3An4zySNJ3jWH+iRp6Y09ck9yHLhinV23VdW9E9bxpqr6XJKvB44nOVlV\nD221oZKkyY0N96q6sWsFVfW50de/THIU2AesG+6rq6vnlnu9Hr1er2v1ktSUwWDAYDDYtFznx+wl\neRD4qar6g3X2vRLYVVVfTPJVwAPAf6iqB9Yp62P2JGmLpv6YvSSHk5wG9gP3JfnkaPuVSe4bFbsC\neCjJY8DDwG+sF+ySpOnyAdmStIP5gGxJWiKGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5J\nDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBnV5QPZ/\nTHIiyeNJPp7kVRuUO5TkZJKnkrx3+02VJE2qy5H7A8A/rqo3AH8E3Lq2QJJdwB3AIeA64G1JXt+h\nzqkZDAaLbsLMtNw3sH87nf2bj22He1Udr6oXR6sPA1evU2wfcKqqnq6q54F7gLdst85pulh+ALPQ\nct/A/u109m8+pjXm/iPA/etsvwo4fd76M6NtkqQZ2j1uZ5LjwBXr7Lqtqu4dlfk54O+r6r+tU666\nN1GStFWp2n7+JnkH8C7ge6rq79bZvx9YrapDo/VbgRer6v3rlPUPgSRtQ1Vl7baxR+7jJDkE/DRw\nw3rBPvIIcG2SvcCfA28F3jZp4yRJ29NlzP2XgK8Gjid5NMkvAyS5Msl9AFX1AnAL0Ac+A3ysqk50\nbLMkaROdhmUkSRenpmeoTjKBKskHRvsfT3L9vNvYxWb9S/JvRv16IsnvJvkni2jndk06AS7Jtyd5\nIcm/mGf7uprw89kb/Wf8h0kGc25iJxN8Pl+V5N4kj436944FNHNbknw4yZkkT44ps9hsqaomX8Au\n4BSwF7gEeAx4/ZoyNwH3j5a/A/j9Rbd7yv37TuBVo+VDrfXvvHL/C/gN4F8uut1T/vldCnwauHq0\nftmi2z3l/t0GvO+lvgF/DexedNsn7N8/A64Hntxg/8KzpeUj90kmUN0M3A1QVQ8Dlya5fL7N3LZN\n+1dVv1dVXxitbjTR7GI16QS4Hwd+DfjLeTZuCibp378Gfr2qngGoqr+acxu7mKR/LwJfO1r+WuCv\na3ie7qJXVQ8Bnx9TZOHZ0nK4TzKBar0yOyUAtzpB7EdZf6LZxWrT/iW5imFg3DnatJNOIE3y87sW\neHWSB5M8kuTfzq113U3SvzuA65L8OfA48JNzats8LDxbtn0p5A4w6S/62kswd0pATNzOJN/NcBbx\nm2bXnKmbpH+/CPxsVVWScOHP8mI2Sf8uAd4IfA/wSuD3kvx+VT0105ZNxyT9OwR8qqq+O8k/ZHjl\n3Ruq6oszbtu8LDRbWg73zwLXnLd+DcO/nuPKXD3athNM0j9GJ1E/CByqqnH/Rl5sJunftwH3DHOd\ny4A3J3m+qo7Np4mdTNK/08BfVdVZ4GyS3wbeAOyEcJ+kf+8A3gdQVX+c5E+Bb2E4P2anW3i2tDws\nc24CVZJXMJxAtfaX/hjwdjg3m/a5qjoz32Zu26b9S/Ia4OPAD1XVqQW0sYtN+1dV31xV31RV38Rw\n3P09OyTYYbLP538H/mmSXUleyfDE3Gfm3M7tmqR/fwZ8L8BoPPpbgD+ZaytnZ+HZ0uyRe1W9kOSl\nCVS7gA9V1Ykk7x7tv6uq7k9yU5JTwJeAdy6wyVsySf+Afw98HXDn6Oj2+arat6g2b8WE/duxJvx8\nnkzyP4AnGJ58/GBV7Yhwn/Dn9wvAf0ryBMMhjJ+pqmcX1ugtSPJR4AbgsiSngdsZDqNdNNniJCZJ\nalDLwzKStLQMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGvT/AC+2Yfh+xoKJAAAAAElF\nTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb52c047cf8>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.ylim(-2, 2)\n",
"plt.xlim(-0.1, 1.1)\n",
"x = np.linspace(0, 1, 10)\n",
"y = list(map(lambda x: np.sin(2*np.pi*x) + normal(scale=0.3), x))\n",
"plt.scatter(x, y, marker=\"o\", color=\"blue\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### モデル\n",
"\n",
"以下の多項式を仮定する.\n",
"\n",
"\\begin{align*}\n",
" f(x) & = w_0 x^0 + w_1 x^1 + w_2 x^2 + \\cdots + w_M x^M \\\\\n",
" & = \\sum_{m=1}^M w_m x^m\n",
"\\end{align*}\n",
"\n",
"#### 評価基準\n",
"\n",
"以下の二乗誤差が最小であること.\n",
"\n",
"\\begin{align*}\n",
" E_D = \\frac{1}{2} \\sum_{n=1}^N (\\sum_{m=1}^M w_m x_n^m -t_n)^2\n",
"\\end{align*}"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 実装例"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import pandas as pd\n",
"from pandas import Series, DataFrame\n",
"from numpy.random import normal"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# データ準備\n",
"def linear_regression_prepare_dataset(num):\n",
" dataset = DataFrame(columns=['x', 'y'])\n",
" for i in range(num):\n",
" x = float(i)/float(num-1)\n",
" y = np.sin(2*np.pi*x) + normal(scale=0.3)\n",
" dataset = dataset.append(Series([x,y], index=['x','y']), ignore_index=True)\n",
"\n",
" return dataset"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# 計算定義(出力と正解の誤差)\n",
"def linear_regression_error(dataset, f):\n",
" err = 0.0\n",
" for index, line in dataset.iterrows():\n",
" x, y = line.x, line.y\n",
" err += 0.5 * (y - f(x))**2\n",
"\n",
" return np.sqrt(2 * err / len(dataset))"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# 学習\n",
"def linear_regression_resolve(dataset, m):\n",
" t = dataset.y\n",
" phi = DataFrame()\n",
" for i in range(0, m+1):\n",
" p = dataset.x**i\n",
" p.name=\"x**%d\" % i\n",
" phi = pd.concat([phi, p], axis=1)\n",
" tmp = np.linalg.inv(np.dot(phi.T, phi))\n",
" ws = np.dot(np.dot(tmp, phi.T), t)\n",
"\n",
" # モデル定義\n",
" def f(x):\n",
" y = 0\n",
" for i, w in enumerate(ws):\n",
" y += w * (x ** i)\n",
" return y\n",
"\n",
" return f"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"def linear_regression_run(n_degree):\n",
" train_set = linear_regression_prepare_dataset(10)\n",
" model = linear_regression_resolve(train_set, n_degree)\n",
" print(\"E(RMS): %.2f\" % linear_regression_error(train_set, model))\n",
"\n",
" plt.title(\"linear regression degree: %d\" % n_degree)\n",
" plt.ylim(-2, 2)\n",
" plt.xlim(-0.1, 1.1)\n",
" plt.plot(train_set.x, np.sin(2*np.pi*train_set.x), color='green', linestyle='--')\n",
" plt.scatter(train_set.x, train_set.y, marker='o', color='blue')\n",
" plt.plot(np.linspace(0,1,101), model(np.linspace(0,1,101)), color='red')\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false,
"scrolled": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"E(RMS): 0.55\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVdXZ9//PJb2oiFhBRRRFEQtKsU9EEBEpFixRLLGX\nmPvB38+SRMfkvmM0D7lNLIgYS6KCFRVFRywHEamigKEIQQRBkSJIZ2Cu54+10eMwDU7Zc85836/X\nfnH22Wvvfe0zw3XWrL3W2ubuiIhIftkp7gBERCT9lNxFRPKQkruISB5SchcRyUNK7iIieUjJXUQk\nDym51wBmNt/MTote32lmQ+KOKReZ2f5mttrMLAvnSpjZrzJ9HslfSu41w4+DGdz9T+5+dZzB5Cp3\nX+DuO3t2Boc4ST+3XGBme5jZUDNbZGYrzewjM+sYd1w1lZK7ZIWZVfi7ZpE0nq92uo6VT8ysVgYP\n3xiYALQHdgOeBt40s0YZPKeUQ8m9hjGzQjP7V/S6pZmVmFl/M/vKzJaa2Z1JZc3MbjezuWa2zMye\nN7Pdkra/aGbfRLW00WZ2eNK2p8xskJmNNLM1QEEZsSTM7L/NbCywFjjQzNqY2SgzW25ms8zs/KTy\nu5vZCDNbZWYTo33HJG0vMbMbzGwOMDt6r6eZfWZm35vZWDNrl1T+NjP72sx+iM61temqo5lNjs7z\nrZkNLPV57RSt72tmr0exzjGzq0p9zi+Y2dPR8T83s2Mr+Ll0jWJYaWYPAhYtW7dfaWYzzGyFmb1t\nZvsnbetmZrOjfR+Ofha/irZdHl33X81sGXC3mdU1s/8b/cy/jX5O9ZOOV+5nVhF3/9LdH3D3JR4M\nAeoCh1Rlf0kzd9eS5wvwJXBa9Ppu4F/R65ZACTAYqAccCWwADo223wJ8DOwL1AEeBZ5LOu7lQKNo\n2/8CnyZtewpYCRwfrdcrI64EMB84jFDR2BVYCFwWrR8NLAUOi8oPA54D6kf7LAA+TDpeCVAENImu\n5xhgCdCBkCj7R59FHeDQaP+9o333B1pFr8cBv4xeNwQ6lfq8dorWPwQeIiSwo4DvgF9E2wqB9UD3\n6Nx/AsaV8/NpBvwAnAPUAn4DFANXRtt7A3OimHcCfguMTdp3FdAn2vZrYFPSvpdHx7ox2l4/+lm9\nGn1OjYHXgT9F5cv9zKLtDwMPV/H37ujoM9g57v8DNXGJPQAtWfgh/zy5F7Jtct83qewEoF/0eubW\n/aL1faLEsVMZ52gSHWvnaP0p4KlK4voAKExav4CkZB29Nxi4K0p6m4DWSdv+CIxJWi8BCpLWBwF/\nKHW8WcApwEFREuuyNXEllRkdfU7NSr2/9fPaCdgP2Aw0Str+J+DJpM/5naRthwPryvkc+gMfl3pv\nYVKCfmvr62h9J8JfOvtH+44tte+CUsn9q6RtBqwh+iKL3jsemFfZZ7adv3O7ANOB2+L+/a+pi5pl\nBODbpNfrCLU5gAOA4dGf598DMwgJbS8zq2Vmf46abFYRvkAg1CQh3AxcWIVzJ5c5AOi09XzROS8G\n9oqOW7tU+a+rcLwBpY7XAtjH3f9DqCEXAkss3AjcJ9rvV4SmhJlR889ZZZxnX2CFu69Nem8B0Dxp\nfUnS63VAfSv73sO+ZVxL6ev4W9I1LI/eb074wi29b0XH2oPw18gnScd7i59+buV+ZmXEXSYzawCM\nIHxh3VfV/SS9lNylIguA7u6+W9LS0N2/ISTdXkAXd98VODDaZ3tviib3CFkAjC51vp3d/UZgGeGL\nZb+k8smvyzve/5Q6XmN3fx7A3Ye6+8mEhObAfdH7c939YnffI3rvpShhJVsMNDWzxknv7U/ZXziV\nWZx8LWZmpa5tAXBNqeto5O7jgG8IyTd53xb8XPJnsozQVHJ40rGauPsuSecq9zOrjJnVIzT5LHD3\na6t09ZIRSu5SkUeBP229eWehq1uvaFtjYCOwwkJviD+V2reqST653BvAIWZ2iZnViZYOZtbG3bcA\nrwCFZtbAzNoAl1Jxd8EhwHXRDVIzs0ZmdpaZNTazQ8zstCgZbSTca9gSXeclZrZHdIxV0TlKkg/s\n7gsJ9yPuNbN6ZnYkcCXwTBWvO9mbQFsz62uhl8+vgb2Ttj8K3GnRDWsz29V+utE8EmhnZr2jfW8s\nte/PuHtJ9Lk8sPUazay5mXWLipT7mVV2EWZWB3iJ8FfK5VW+eskIJfeap3T/6YqS498IN9veMbMf\nCDcat/Zb/ifwFbAI+DzaVvq4VemnndwHfw3QDbgwOu43wL2EG5YANxFuun5L6GY3lNAOX+a1uPsn\nwNWEm54rCDcl+0eb60XHXhqdpxlwR7TtDOBzM1tNuPl4obtvLOMcFxHa4RcTvnjucvf3K7j+Mj8P\nd18OnA/8mVCzPhj4KGn7q4S/IIZFTWDToxhx92XRvvdH+x4GTCZ8YZUXx23AXGB8dLxRRD1aKvjM\nQoN96FkzqKzrAE4AzgK6AistDPhabWYnllNeMsjcd3ychJntR/hPvifhh/+Yu/+9jHJ/B84k+kZ3\n9093+KQiETO7D9jT3a+IO5bqImrTXwhc7O6j445H4pNqzb0Y+C93bwt0Bm40s8OSC5hZD+Bgd28N\nXEO4Gy+y3czsUDM7Mmou6EhoBhked1xxi/q5N4mamLaOUxgfZ0wSv5SSu7t/6+6fRa/XELrO7Vuq\nWC/Cn9C4+wSgiZntlcp5pcbaGXiZ0JVvGPB/3f31eEOqFo4nNLMsJTSL9ElqRpIaKm1DtM2sJWEA\nxIRSm5qzbfe1Fvy8m5hIpdx9MtA67jiqG3e/B7gn7jikeknLDdXoTvpLwC1RDX6bIqXWc2pCJBGR\nXJNyzT3q/vQy8Ex0V7+0Rfy8z26L6L3Sx1HCFxHZAe6+TdfjlGru0YCJfwAz3P2Bcoq9TtT9zMw6\nAyvdvcwmmWwOzb377rtjHx6sa9P16fryb8n29ZUn1Zr7icAlwDQz29q98U7CSD3cfbC7jzSzHmY2\nlzAfhrqtiYhkWErJ3d0/ogq1f3e/KZXziIjI9qmxI1QLCgriDiFj8vnaQNeX63R92ZHSCNV0MjOv\nLrGIiOQKM8PTfUNVRESqJyV3EZE8pOQuIpKHlNxFRPKQkruISB5SchcRyUNK7iIieUjJXUQkDym5\ni4jkISV3EZE8pOQuIpKHlNxFRPKQkruISB5SchcRyUNK7iIieUjJXUQkDym5i4jkoZSTu5k9YWZL\nzGx6OdsLzGyVmX0aLb9L9ZwiIlKxlB6QHXkSeBD4ZwVlRrt7rzScS0REqiDlmru7jwG+r6TYNs/3\nExGRzMlGm7sDJ5jZVDMbaWaHZ+GcIiI1WjqaZSozBdjP3deZ2ZnAq8AhWTiviEiNlfHk7u6rk16/\nZWaPmFlTd19RumxhYeGPrwsKCigoKMh0eCIiOSWRSJBIJCotZ+6e8snMrCUwwt3blbFtL+A7d3cz\n6wi84O4tyyjn6YhFRKQmMTPcfZv7minX3M1sKHAq0MzMFgJ3A3UA3H0wcB5wvZltBtYBF6Z6ThER\nqVhaau7poJq7iMj2K6/mrhGqIiJ5SMm9higqgm7dwlJUFHc0IpJpapapAYqKoG9fWL8+rDdoAMOH\nwxlnxBuXiKROzTI12MCBPyV2CK8HDowvHhHJPCV3EZE8pOReAwwYEJpitmrQILwnIvlLbe41RFHR\nT00xAwaovV0kX5TX5q7kLiKSw3RDVUSkBlFyFxHJQ0ruIiJ5SMldRCQPKbmLiOQhJXcRkTyk5C4i\nkoeU3EVE8pCSu4hIHlJyFxHJQ0ruIiJ5KOXkbmZPmNkSM5teQZm/m9kcM5tqZsekek4REalYOmru\nTwLdy9toZj2Ag929NXANMCgN5xQRkQqknNzdfQzwfQVFegFPR2UnAE3MbK9UzysiIuXLRpt7c2Bh\n0vrXQIssnFdEpMaqnaXzlJ5ruMyJ2wsLC398XVBQQEFBQeYiqoE2bdnEqg2r2KPRHnGHIiI7KJFI\nkEgkKi2Xlod1mFlLYIS7tytj26NAwt2HReuzgFPdfUmpcnpYR4ZN/XYqpzx1Cs0aNqNT8050bN6R\nTs07ccw+x1C/dv24wxORHZDRJzFVktx7ADe5ew8z6ww84O6dyyiXveQ+dy4cdBDYNp9H3ivxEmYt\nm8XERROZ8PUEJiyawKHNDmXouUPjDk1EdkDGkruZDQVOBZoBS4C7gToA7j44KvMQoUfNWuAKd59S\nxnGyk9zdoXPnkNgLC8PDRPMwyW/asom6tepWqay7Y2V8BiPnjGTcwnF0atGJTs07qTlHpBrSM1ST\nlZTASy/BPfdA48Zw991w5pl5keRXbVjFb9//LQtWLeD1i15P6ViffvMpr8x8hQmLJjBp8SSa1G9C\np+aduLnjzZy4/4lpilhEUqHkXpaSEnj5ZfjDH6B+fbjrLujZMyeTvLsz7PNh3DrqVnq27sm9p99L\n0wZN03b8Ei/hi+VfMHHRRI7a6yiO2vuobcosW7eMpg2aspNp4LNItii5V6SkBIYPD0m+Vq2Q5Hv1\ngp1yI0nNWT6HG0bewHdrv+PRsx7l+P2OjyWO8144j3fnvUuH5h3ouG/HH5tz9mqsYQ0imaLkXhUl\nJTBiBPzxj7BpE/zud3DuuSHhV2NPfvok32/4nl93+jW1d8pW79ayLV27NNysXTSBiYsmhuXqiRzc\n9OBY4xLJV0ru28Md3norJPmVK+HOO+Gii6B2vIkzF7k7RUXw17+G370BA8I9bBFJDyX3HeEO770H\n//3fsHAh3HYbXHYZ1KsXd2Q5o6gI+vaF9evDeoMGoQVMCV4kPcpL7rnRqBwXMzj9dEgk4Omn4ZVX\n4OCD4W9/g3XrshrKlpItPDTxIf459Z9ZPW+qBg78KbFDeP2XgZuZtWxWfEGJ1ABK7lV10knw9tuh\n2vnhh3DggfCnP4VmmwybvHgynR7vxIszXuS4fY/L+PkybXXjzzjlyVP48KsP4w5FJG8puW+v444L\n3Sc/+ABmzQo1+TvvhO++S/upVm1Yxc0jb6bncz25uePNJC5LcPgeh6f9PJk0YEBoitmqQQP4w7XH\nMfTcoZz7wrmMnDMyvuBE8pja3FP15Zfwl7/AsGFw8cVw663QsmVaDt1raC/2brw3fz79z2nts55t\nRUWheQZ+fkN13MJx9Hm+Dw+e+SD92vaLL0CRHKYbqpn27bfwwAMwZAj06BFuvh5xREqH3Lh5I/Vq\n5/fN26nfTuXMZ89kcM/BnH3o2XGHI5JzlNyzZeVKePTRcNP1uONCkj/ppLijqtbmrpjLno32ZJd6\nu8QdikjOUXLPtg0bQg+b+++HvfcOSb5nzzJHvb47712O3vtomjVsFkOgIpLL1BUy2+rXh2uvhS++\ngFtuCVMbHHEEPPEEbNwIwLdrvuXily/m6hFXs2DVgpgDFpF8opp7triHHjb33Yd//jnj+nbg0j3H\n0O+Ea/j9qb+nYZ2GcUdYrWzasolaVotaO1XvqR9E4qaae9zM4LTT2DRyBJde1ZQ1n3zM7P/dzL1v\nbaLhN8vijq7aue+j+7hk+CUUbymOOxSRnKTknmV1a9Xlxqseo+vHS6g9dXp48+ijQzfKKds8w6TG\nuvWEW1m9cTXnvHAO64vXV76DiPyMmmWqg1WrQhfKv/0tDIoaMCB0p8yRKYczpXhLMf1f7c+SNUt4\n7cLX2LneznGHJFLtqLdMLiguhhdeCCN+1q2D//ovuPRSaFhz2+O3lGzh+jevZ9qSabz1y7fYrcFu\ncYckUq0ouecSdxg9OiT5CRNCr5sbboB99ok7sli4O49OfpTLjr5MN55FSsnYDVUz625ms8xsjpnd\nVsb2AjNbZWafRsvvUj1nKoqKoFu3sBQVZfZc7s6Nb97ItCXTtm9HMygoCA8O+egjWLECDj88TDf8\n6acZibU6MzOu73C9ErvIdkip5m5mtYDZwOnAImAScJG7z0wqUwD8H3fvVcmxMl5zz/bc4r9///e8\n9+V7vNf/PRrUaVD5DhVZsSK0yz/0EBx0UOg736tXtX9KlIhkVqZq7h2Bue4+392LgWFA77LOn+J5\n0qKsucW3TmiVbk98+gTPff4cr134WuqJHaBp0zDKdd48uP76MPL14IPhr3/NyrTDIpJbUk3uzYGF\nSetfR+8lc+AEM5tqZiPNLLfmrN0B7857lzveu4ORF49kj0Z7pPfgderABRfAuHFhJspPPoFWreDG\nG8MUxDWEu3P5q5czbuG4uEMRqZZSfShoVdpRpgD7ufs6MzsTeBU4pKyChYWFP74uKCigoKAgxfB+\nbsCA0ISd3CwzYEBaT8Hqjau57NXLePH8Fzm02aHpPXhpnTrBs8/C4sUwaFBopz/qKPj1r+HMM/O6\nK6WZcUHbC+g9rDdDzx1Kl1Zd4g5JJCsSiQSJRKLScqm2uXcGCt29e7R+B1Di7vdVsM+XwLHuvqLU\n+1npLVPe3OLp9N3a79iz0Z7pP3BlNmwIXSn//nf4/vtQm7/ySmjSJPuxZMno+aM5/8XzebzX4/Q6\ntMLbOiJ5KSNdIc2sNuGGahdgMTCRbW+o7gV85+5uZh2BF9y9ZRnHUlfIdHEPXSgffBBGjoR+/eCm\nm6Bdu7gjy4hJiyZx9tCz+esZf+XidhfHHY5IVmXkhqq7bwZuAoqAGcDz7j7TzK41s2ujYucB083s\nM+AB4MJUzilVYAadO4cmm5kzoUUL6N4dTj0Vnn8+DJbKIx2ad+Dd/u/y7rx3UQVBJNAgppqiuBhe\nfRUeeQRmz4arrw5LixZxRyYiKdCskBny6ORH+eDLD+IOo3J16sD554dph0eNguXL4cgj4ZxzwnpJ\nSdwRikgaqeaegtdnv871b17Px1d+zAFNDog7nO23enVouhk0KHQhuvZauPxy2H33uCMTkSpSzT3N\nJi+ezFWvX8WrF7yam4kdYOed4brr4LPP4Kmnwr8HHQT9+8PYseHGbA5bV7yOv4z9C5tLNscdikjW\nKbnvgK9WfkXvYb0ZcvYQOjTvEHc4qTODE06Af/0L/vOf0Ff+yitDs81DD+XsCFh3Z9ikUTS/5UJO\nP6M443MJiVQnapbZTiVeQochHbj0yEv5TeffxB1O5rhDIgGDB4fBAX36hBuwxx8fvgxyQFER9Dlv\nIxv69Iblh9Ag8feMziUkEgdN+ZtGc1fM5aDdDsK2M8llYwBVRixdCk8/DY89BnXrwjXXwCWXhPlu\nqrFu3cK9YuqvhKs6wdjb6NrsSt55J+7IRNJHyT1m2Z6RMiO2zjP/2GNhcFTPnnDVVaH/fDWszf+Y\n3AGazYIrTqHznHcYN/zoWOMSSScl95j9LNFEunYld2uRy5fDM8+EaYg3bgxt9JddBvvuG3dkPyr9\nhVpv/2m8/OhhnHVmnXgDE0kj9ZaR9Np99zCn/PTp4UbsvHnQti2cfXYYLFUNRsGecUb466hr17C8\n9tiRSuxSY6jmXokxX41h6bqlnHPYOSkdJy+aZSqzZg289BL84x/wxRfh+a9XXBGSvohkhGruO+CL\n5V9w3ovn0bhu45SPVboWmXeJHaBx4zAIaswY+PDDMCq2W7cwNfGjj+Zsl0qRXKSaezmWrl3K8f84\nnttPup2r2l8Vdzi5a/PmcLPhySfDDYYzzwxt8127xv6IwGXrljF3xVw6t+gcaxwiqVDNfTts2LyB\nPs/3oV/bfkrsqapdOyT0F14IA6ROOgnuugv22y88NnDGjNhCm71sNr2H9WbuirmxxSCSKaq5l+GK\n165gffF6njv3OXYyff9lxIwZoe/8M8/APvuE2vyFF8IeaX4sYSUGTRrEgxMfZPxV49ml3i5ZPbdI\nOqgr5HaYvmQ6rXdvTf3a9eMOJf9t2QLvvQf//Ce88Qacckq4EXv22VA/85+/u3PdG9fx7dpvGX7B\ncH2ZS85Rcpfqb/VqePnlUJufMgXOPTeMhD355Iw+D3bTlk2c9vRpdDmwC/f84p6MnUckE9TmLtXf\nzjuH3jbvvgvTpkHr1uFh3y1bhvb5adMyctq6terycr+XqVe7np7kJHlDNXep/qZPD/POP/cc7Lor\n/PKXcNFFcECOTrUseSmuuaPULFOOxasXk5if0IOVc0FJCXz0UUj0L78MbdqEJH/++bDnnnFHJzVY\nnIMUM9YsY2bdzWyWmc0xs9vKKfP3aPtUMzsm1XOmy5pNa+j5XE++WvlV3KFIVey0U7jhOngwLF4c\nmmrGjoVDDgn/i556SgOlJBYDB/6U2CG83lqLj0tKyd3MagEPAd2Bw4GLzOywUmV6AAe7e2vgGmBQ\nKudMl80lm7nwpQtpv097bj/p9rjDke1Vt27oUfPcc7BoUZi47LXXQlNNnz4wdGiYDiEFqzeuTlOw\nItmXas29IzDX3ee7ezEwDOhdqkwv4GkAd58ANDGzvVI8b0rcnd+8/Rs2bdnEoLMGbfe87FLNNGoE\nF1wQ/g5esCD8ffyvf0Hz5qHJ5sUXYe3a7T5sn+f78OSnT2YgYMk3AwaEppitGjQI78Up1eTeHFiY\ntP519F5lZVqkeN6UPDTxIUZ/NZoXz3+ROrU0S2Be2XXXMCBq5MgwU+UZZ8Djj4epiPv1CxObrVtX\npUM93ONhbnv3NsZ/PT7DQUuu2zp3VJeum6vN3FG1U9y/qndAS1eNy9yvsLDwx9cFBQUUFBTsUFCV\nOe3A0+jdpje71t81I8eXamL33cPDRK66CpYtC//jHnsMfvWr8D/vvPOgR48w4VkZ2jRrwxO9n+C8\nF85j4tUT2Xfn6jNXvVQ/6/Yfzk6XDuKdSzP7kIZEIkEikai0XEq9ZcysM1Do7t2j9TuAEne/L6nM\no0DC3YdF67OAU919SaljqSukZMeyZWHO+Zdego8/hi5dwoCps88ONf9S7h1zL6/OfpXRl4/WqGUp\n08RFE+n5XE/evuRt2u/TPqvnzlRvmclAazNraWZ1gQuA10uVeR3oHwXRGVhZOrGLZFWzZqE2//bb\nMH8+9O4Nzz8fJjPr0SPMR7906Y/Fbz/pdlrt1oq3574dX8xSbX35/Zf0GdaHJ3o/kfXEXpGU+7mb\n2ZnAA0At4B/ufq+ZXQvg7oOjMlt71KwFrnD3KWUcRzV3idcPP4S2+ldeCR2Xjzkm3Jzt25eS/Vpo\n3hnZxsoNKznhHydwQ4cbuKnjTbHEoEFMIttj/fowD/3w4TBixE9dLPv0gSOOqJYPBJfsW1+8nhdn\nvEj/o/rHFoOSu8iO2rw5jIwdPjz0pa9VKzTl9O4NJ54Y5qwXiYmSu0g6uMPUqSHJv/46fPVVaKfv\n1Ss8UnAXzQkv2aVZIUXSwQyOPppZN17ANYXHUjLlk/CM2McfhxYtQoJ/8EH48su4I5UaTsldZAcc\n2ORA/r303/xh3pNw442h582iRXDddWEu+k6doG3bMP/NmDGhaUdy3pRvprBmU2rTWmSLmmVEdtCS\nNUvoMKQDf+v+N/oe1vfnG7dsgcmTw9Ol3ngjTIvQrVtowunePeuPE5TUzVw6k4KnC3il3yucuP+J\ncYfzI7W5i2TA5MWT6fFsD97r/x7t9mpXfsFFi+Ctt+DNN+H998N0xT16hIeHH3tsuEkr1dZ3a7+j\n8+OduevUu7j86MvjDudnlNxFMuTZac9yz+h7+PyGz6lbq27lO2zaFHrfjBwZmnOWLAm1+u7dw797\nxTqvnpSyvng9v3j6F3Rt1ZU/nvbHuMPZhpK7SAbNXjabQ5sdumM7L1gQknxRUXhYeKtWYe6bM86A\nE04I0xtLLEq8hH4v9qNe7Xo80/eZajmDrJK7SC4oLobx40OiLyqCL74IDyjp2jUsbdpoAFUWFW8p\n5n/H/y+3dLqFerXrxR1OmZTcRXLR8uWhNl9UFEbMusPpp4dE36WLmnBEyV0k57mHmvyoUSHhJxKh\nb32XLmE59VQNoqqBlNxFsuj9L9+nTbM2mZ0DfvNm+OSTkOjfew8mTIB27eAXv4DTTgvt9Q0bZu78\nUi0ouYtk0f98+D+M+GIEicsT2ZsDfsMGGDcudLX84AP47DNo3z4k+4IC6Nz558+Ck20sXr2YBrUb\nsFuD3eIOpcqU3EWyyN254KULaFinIU/2fjKeXhZr1sDYsaH55oMP4PPPQ7IvKAhNOMcfr5p9kh82\n/sDJT57MjR1u5Jpjr4k7nCpTchfJsrWb1nLqU6dywn4n8ED3B+KfD3716vDkqUQCRo+GadPgyCND\nb5yTTw4zXDZpEm+MMdlcspmzh57NAbsewKCzBlXLLo/lUXIXicHKDSvp+VxPjtzrSB4565G4w/m5\ndetCM86YMWGZODH0sT/55J+SfYtYn2WfFe7OdW9cx4IfFjDiohHU3im3pnBWcheJydpNa/n8u8/p\n1KJT3KFUrLg4THr20Uch2Y8dC40ahSR/4onhBm27dnk3VcL9Y+/n2enPMuaKMexSL/d6Gym5i8j2\n2dr1cuzY0JwzdmyYI6djx9Bef/zx4SZt06ZxR7rD3J17Rt/DVe2vosUuuflXipK7iKRuxYowgvbj\nj0OTzqRJsM8+PyX6Tp1C7V5Pp8qatCd3M2sKPA8cAMwH+rn7yjLKzQd+ALYAxe7esZzjKblLjbK5\nZHPOte9uY8uW0Atn/PjQz378+DBXzjHHhETfsWNYDjhA0yZkSCaS+/3AMne/38xuA3Zz99vLKPcl\ncKy7r6jkeEruUmPMWDqDi16+iLd++VZGBjoVFcHAgeH1gAFhDrKsWbUqzGU/YUKo2U+YENrzjzsO\nOnQIy3HHhRq/pCwTyX0WcKq7LzGzvYGEu7cpo9yXwHHuvryS4ym5S43y54/+zJApQxh16Sha7dYq\nbcctKoK+fWH9+rDeoEF4tndWE3xpixaFRD9pUhhVO3ky1KsX5rI/9tjQ//7YY0PCz2ANf+Pmjfyw\n8Qf2aJQ/D0vJRHL/3t13i14bsGLreqly84BVhGaZwe4+pJzjKblLjfPIpEe496N7KbqkiMP3ODwt\nx+zWLUw/k6xrV3jnnbQcPj3cYf780Dvnk0/CMmVK6InTvn1o1jn66PBvq1awU+pjBNydS4dfym71\nd+PBHg+mfg3VRHnJvcIGPzMbBexdxqbfJq+4u5tZeZn5RHf/xsz2AEaZ2Sx3H1NWwcLCwh9fFxQU\nUFBQUFF6yYK4AAAMuUlEQVR4Ijnvhg43sEu9Xejyzy68efGbtN+nfdwhZYcZHHhgWM49N7znDl9/\nHZL8p5/CM8/ArbfC99/DUUf9fDniiO0eXXt34m7mrpjL+5e9n4ELyp5EIkEikai0XKrNMgXu/q2Z\n7QN8UFazTKl97gbWuPvAMrap5i411quzXqVZw2actP9JKR+rWjbLpGLFijBPztSpYfnss9BFc7/9\nwgjbI48MPXTatQtfFmXU8p/67Cn+MPoPjL9qPHs22jOGi8icTN1QXe7u95nZ7UCT0jdUzawhUMvd\nV5tZI+Ad4B533+YPRCV3kfSJ9YZqNhQXw+zZYQqF6dN/+nfFCjj8cGjbNtTu27blo8bfc+64W0hc\nPprD9jgs7sjTLlNdIV8A9iepK6SZ7QsMcfezzKwV8Eq0S23gWXe/t5zjKbmLSGpWrQpdM//97x//\nXfPZJOoXl1C7bTs47LCfLy1b5vyIWw1iEpGaa/lymDkzLDNmhGXWLPjuO2jdOjy+8NBDf77kyINP\nlNxFcszLM15m8erF3Nzp5rhDyV9r14bmndmzQ7Lf+nrOHGjcGA45JCT/1q3h4IPDvwcdFLZVE0ru\nIjlm/sr5nP7P07nymCu546Q7cmoa2pznDosXhyT/xRfh363LvHlhauSDDvr50qpVWPbcM6ujcZXc\nRXLQN6u/oeu/unJW67P48+l/VoKvRPGWYhatXkTLJi0zd5KSkjAoa948+M9/wjJv3k/LlCmhJ0+W\nKLmL5Kjl65bT/dnuHLfPcTx81sPxP/Sjmhq7YCzXvXkdJ+9/cvWbOz+Dykvu+i0RqeZ2b7g77/V/\nj8VrFjPv+3lxh1PtLF+3nKtfv5p+L/Xjdyf/jod7PBx3SNWCkrtIhhQVhakAunULr1OxS71deO3C\n1zi46cHpCS5PvDTjJdo+0pYGdRow44YZXHDEBWq6iqhZRiQD8m6UaDX1zn/eoWmDphy373FxhxIb\ntbmLZFFOTN6Vh/J+ZG4Z1OYukodemvESS9YsiTuMrKis8rf1r6VRo8LSt2/qzWG5TMldJAMGDAhN\nMVs1aBDeS7d/f/dvTnnqFBauWpj+g1cTC1ct5Jznz+GRSRX3gBk48KdmMAivB24zRWHNoeQukgFn\nnBHa2Lt2DUum2tvvLriba4+9lpOfPJk5y+ek/wQxKt5SzMCPB3LM4GM4aq+j+FX7X8UdUk5Rm7tI\nHhjyyRAKRxfy9i/fpt1e7eIOJ2UfL/yY6964jr0b783DPR6m9e6tK92npt7E1g1VkTw37PNh/PHD\nPzL1uqk5/+Dt/sP7c1brs+jXtt92dW3UDdWk96tLQlVyF0nd2k1raVS3UdxhSBYpuYuI5CF1hRSp\noWYvm83Tnz3NzKUzKfGSuMP50bridfz+/d+zYNWCuEPJS0ruInlu9abVvP2ft+k5tCdN72tK1391\n5bfv/ZZJiybFFtObX7xJ20faMvf7udSrVS+2OPKZmmVEapCla5cycdFEJi6ayBF7HsH5bc/fpoy7\nZ2x+lq9/+Jpb3r6FaUum8UiPR+h6UNeMnKcmUZu7iFTJdW9cx4RFE+jUvBOdmneiY/OOHLbHYSlP\nNbx642raPNyGq9tfze0n3U792vXTFHHNlokHZJ8PFAJtgA7uPqWcct2BB4BawOPufl855ZTcRaqB\nDZs3MPXbqUxYNCEsX09g6bqljLx4JCfuf2JKx165YSVN6jdJU6QCmUnubYASYDAwoKzkbma1gNnA\n6cAiYBJwkbvPLKOskrtINbVs3TIa1mlIwzoNt9n2zLRn2H/X/Tl2n2PVDTMG5SX3HR7p4O6zth64\nAh2Bue4+Pyo7DOgNbJPcRaT6atawWbnbpi2ZxoMTH2T6kum03r01nZp3onOLzlx5zJVZjFBKy/Qw\ntuZA8oxGXwOdMnxOEcmi+7veD8DGzRuZumQqE76ewKRFkzj/8PPZud7OMUdXc1WY3M1sFLB3GZvu\ndPcRVTj+drWzFBYW/vi6oKCAgoKC7dldRGJUr3Y9OjbvSMfmHeMOJa8lEgkSiUSl5VLuLWNmH1B+\nm3tnoNDdu0frdwAlZd1UVZu7iMj2y/QI1fIa3icDrc2spZnVBS4AXk/TOUVEpBw7nNzNrK+ZLQQ6\nA2+a2VvR+/ua2ZsA7r4ZuAkoAmYAz5fVU0ZERNJLg5hERHKYJg4TkawoKgoPCO/WrWY/wzRuqrmL\nSNrU1KchxUk1dxHJOD2kuvpQchcRyUNK7iKSNgMGhKaYrRo0CO9J9qnNXUTSqiY+pDpOms9dRCQP\n6YaqiEgNouQuIpKHlNxFRPKQkruISB5SchcRyUNK7iIieUjJXUQkDym5i4jkISV3EZE8pOQuIpKH\nlNxFRPJQKs9QPd/M/m1mW8ysfQXl5pvZNDP71Mwm7uj5RESk6mqnsO90oC8wuJJyDhS4+4oUziUi\nItthh5O7u8+CMCNZFVSpkIiIpEc22twdeNfMJpvZ1Vk4n4hIjVdhzd3MRgF7l7HpTncfUcVznOju\n35jZHsAoM5vl7mO2N1AREam6CpO7u3dN9QTu/k3071IzGw50BMpM7oWFhT++LigooKCgINXTi4jk\nlUQiQSKRqLRcyk9iMrMPgFvd/ZMytjUEarn7ajNrBLwD3OPu75RRVk9iEhHZTml/EpOZ9TWzhUBn\n4E0zeyt6f18zezMqtjcwxsw+AyYAb5SV2EVEJL30DFURkRymZ6iKiNQgSu4iInlIyV1EJA8puYuI\n5CEldxGRPKTkLiKSh5TcRUTykJK7iEgeUnIXEclDSu4iInlIyV1EJA8puYuI5CEldxGRPKTkLiKS\nh5TcRUTykJK7iEgeUnIXEclDSu4iInlIyV1EJA+l8oDsv5jZTDObamavmNmu5ZTrbmazzGyOmd22\n46GKiEhVpVJzfwdo6+5HAV8Ad5QuYGa1gIeA7sDhwEVmdlgK50ybRCIRdwgZk8/XBrq+XKfry44d\nTu7uPsrdS6LVCUCLMop1BOa6+3x3LwaGAb139JzpVF1+AJmQz9cGur5cp+vLjnS1uV8JjCzj/ebA\nwqT1r6P3REQkg2pXtNHMRgF7l7HpTncfEZX5LbDJ3Z8ro5ynHqKIiGwvc9/x/GtmlwNXA13cfUMZ\n2zsDhe7ePVq/Ayhx9/vKKKsvAhGRHeDuVvq9CmvuFTGz7sD/B5xaVmKPTAZam1lLYDFwAXBRVYMT\nEZEdk0qb+4NAY2CUmX1qZo8AmNm+ZvYmgLtvBm4CioAZwPPuPjPFmEVEpBIpNcuIiEj1lNcjVKsy\ngMrM/h5tn2pmx2Q7xlRUdn1m9svouqaZ2VgzOzKOOHdUVQfAmVkHM9tsZudkM75UVfH3syD6y/hz\nM0tkOcSUVOH3c1czG2Fmn0XXd3kMYe4QM3vCzJaY2fQKysSbW9w9LxegFjAXaAnUAT4DDitVpgcw\nMnrdCRgfd9xpvr7jgV2j193z7fqSyr0PvAGcG3fcaf75NQH+DbSI1pvFHXear+9O4N6t1wYsB2rH\nHXsVr+9k4BhgejnbY88t+Vxzr8oAql7A0wDuPgFoYmZ7ZTfMHVbp9bn7OHdfFa2WN9CsuqrqALib\ngZeApdkMLg2qcn0XAy+7+9cA7r4syzGmoirXVwLsEr3eBVju4T5dtefuY4DvKygSe27J5+RelQFU\nZZXJlQS4vQPEfkXZA82qq0qvz8yaExLGoOitXLqBVJWfX2ugqZl9YGaTzezSrEWXuqpc30PA4Wa2\nGJgK3JKl2LIh9tyyw10hc0BV/6OX7oKZKwmiynGa2S8Io4hPzFw4aVeV63sAuN3d3cyMbX+W1VlV\nrq8O0B7oAjQExpnZeHefk9HI0qMq19cdmOLuvzCzgwg9745y99UZji1bYs0t+ZzcFwH7Ja3vR/j2\nrKhMi+i9XFCV6yO6iToE6O7uFf0ZWd1U5fqOBYaFvE4z4EwzK3b317MTYkqqcn0LgWXuvh5Yb2Yf\nAkcBuZDcq3J9lwP3Arj7f8zsS+BQwviYXBd7bsnnZpkfB1CZWV3CAKrS/+lfB/rDj6NpV7r7kuyG\nucMqvT4z2x94BbjE3efGEGMqKr0+d2/l7ge6+4GEdvfrcySxQ9V+P18DTjKzWmbWkHBjbkaW49xR\nVbm+BcDpAFF79KHAvKxGmTmx55a8rbm7+2Yz2zqAqhbwD3efaWbXRtsHu/tIM+thZnOBtcAVMYa8\nXapyfcBdwG7AoKh2W+zuHeOKeXtU8fpyVhV/P2eZ2dvANMLNxyHunhPJvYo/vz8CT5nZNEITxv/v\n7itiC3o7mNlQ4FSgmZktBO4mNKNVm9yiQUwiInkon5tlRERqLCV3EZE8pOQuIpKHlNxFRPKQkruI\nSB5SchcRyUNK7iIieUjJXUQkD/0/LAwhqzXrPAMAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb5177d8f60>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"linear_regression_run(2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# データの分類"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"N件の3つ組のデータ(x, y, label)がある.\n",
"(labelは **XXXである** と **XXXではない** の2値のデータ).\n",
"どうやらうまく分割する直線を引けそうだ.\n",
"\n",
"このとき, 未知のxとyに対するlabelを求めたい."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from numpy.random import normal"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEnNJREFUeJzt3X2MXNV5x/Hfg3cXsIPkRElsMIkIUZIWy1JqXBopiRi1\nieWKKDGq3RTUCCWO46gKQlX+MGmQ18YxZVFMVlVpWdwkSqXUIbHkBKgguFFHRara4IBaC+wCMpb8\nxhKpOIXYYANP/7gz3tnx7Oy83HNfzv1+pBFzZ2bnXi3jZ3/z3HPONXcXACBOF+V9AACAcCjyABAx\nijwARIwiDwARo8gDQMQo8gAQsaGLvJl9z8ymzexAy2PvMrN9ZvacmT1uZouH3Q8AoH9pJPnvS1rT\n9tjtkva5+4cl/aKxDQDImKUxGcrMrpL0sLuvaGwfknS9u0+b2VJJdXf/naF3BADoS6ie/BJ3n27c\nn5a0JNB+AABdBD/x6slXBdZOAIAcjAR632kzW+ruL5nZ5ZJebn+BmVH4AWAA7m69vjZUkn9I0i2N\n+7dI+mmnF7k7t5Ru4+PjuR9DTDd+n/w+i3rrVxpDKHdL+ndJHzGzo2b2RUl3S/q0mT0n6Q8b2wCA\njA3drnH3m+Z46lPDvjcAYDjMeI1ErVbL+xCiwu8zXfw+85PKOPmBdmzmee0bAMrKzOQFOPEKACgA\nijwARIwiDwARo8gDQMQo8gAQMYo8AESMIg8AEaPIA0DEKPIAEDGKPABEjCIPABGjyANAxCjyABAx\nijwARIwiDwARo8gDQMQo8gAQMYo8AESMIg8AEaPIA0DEKPIAEDGKPABEjCIPABGjyANAxCjyABAx\nijyA8PbsSW7InLl7Pjs287z2DSBDZ85IV1yR3D95UrrkknyPp+TMTO5uvb6eJA8grKkp6dy55DY1\nlffRVA5JHkA4zRR/6lSyvXgxaX5IJHkAxTE1JZ0+PbN9+jRpPmMjeR8AgIi98Ya0YsXsx15/PZ9j\nqSjaNQBQIrRrAADnUeQBIGIUeQCIGEUeACJGkQeAiOVa5N96K8+9A0D8ci3yV18tbd8unTiR51EA\nQLxyLfJ790rHj0vLl0tr10qPPkq6B4A0FWIy1GuvSbt3Sw88IL38svTlL0sbNswsXAcASJRyMtQ7\n3iFt3Cg9+STpHgDSVIgk3wnpHgAuVMok3wnpHgCGV9gk3wnpHkDV9ZvkgxZ5Mzsi6f8kvSXpnLtf\n1/LcUKtQPvVUUuwffFC6/npp0yZp9WppwYKhDxsACqtoRf5FSde6+/92eC6VpYZJ9wCqpIg9+Z4P\nZhD07gFgbqGT/GFJv1HSrply910tzwW7aAjpHkCs+k3yoS//93F3P2lm75G0z8wOufsTzSe3bt16\n/oW1Wk21Wi2VnTbT/caNM7375cvp3QOlsGdP8t916/I9joKo1+uq1+sD/3xmo2vMbFzSa+6+s7Gd\n6eX/SPdAAGkX5DNnZv5RnjwpXXJJOu8bkcL05M1soZld1ri/SNJqSQdC7W8+9O6BlJ05M/OVOa2L\nc09NSefOJbepqXTes+KCJXkz+4CkvY3NEUk/dPe/bnk+9wt5k+6BIUxOSnfckdzfsUO67bbh3q+Z\n4k+dSrYXLybNd1CoIZRdd1yAIt+KcfdAH0IU5MlJafNm6ezZZHtsTLrnnuH/eESmMO2aslm5Urr/\nfunoUemGG6QtW1jvHpjT1JR0+vTM9unTw7dX3nhDWrFCuvba5LZiRXptoAojyXdBugfmMDEh/eQn\nsx9bvz5J4giKdk0A9O4RnW6jYhjCWGgU+cBI9yi9bsMUGcJYePTkA6N3j9LrNkyx03N79syk+9b7\nWch6fxEiyaeAdI/S6DYqptNzhw8nKUaafT+LlM+3io5I8jkg3aM0uo2K6fTcl740k+xb72cxUYmJ\nUakgyQdCukchdRsV0/7c229LBw/ODGM0k5r/ZkNPVGJi1Jw48VowjMxBabVPTmoVeqJSGhOjIh0l\nVLRVKCuPFTFRWs3JSZL00kvSK69I73yntHRp8ljIiUqt+27qZ3/NdXUk6TOfqfQ3AJJ8Dkj3yESk\nSbYnaa+rUyC0a0qG3j2CqPLIlMj7+YyuKRlG5iCIKo9MCbGuTolR5AuC9e6RmjNnpG3bpN/+Nrlt\n3Vq8hb5CTnJiobNZaNcUGL17DKToS/ZWuZWUAto1ESHdYyBFT7JVbiXlgCRfMqR7lFqnk6L33Zd8\n26jiKKABMLqmQhiZg76EHlLZy/u3t5JGR6WREenii2nd9IgiX0Gke8wrdB+81/dvXzphejr50I6O\nRjeePRR68hVE7x7zCt0H7/X9N2+W9u9Pbk88kSSUs2eLOwooAiT5SJHucV7oyUGDvn/RRwEVFEke\nkkj3aBF6ctCg71/0UUCRYIGyCmjOqv32t5N0v2WL9NWvku4rY9jFvkK9/+bNXPg7A7RrKoqROUA5\nMboGfaF3D0nVXrGyZCjyGBjpvqJYZqBUOPGKgbEiZkWxzEDUSPLoinQfucjXXo8RSR6pIt1HjrXX\no0eRR08Ydx+pso1VD7kOfaRo12BgjMxBV2mP2Gk/QfzII+m+f0kwuga5oHePWUKM2Gm9OPf4uHTX\nXem+f0nQk0cu6N1jlrRH7LRf0nDLlv7ev8JtHpI8giHdV1SIETvti5m1mu/9I5sHQJJHYZDuKyrE\niJ3WE8RXXilZS42b7/0rPg+AJI9Mke4roP3CIJK0fn16i5H18/4RzgPgxCtKgZE5yESEa9bTrkEp\nMO4emSjbPIAASPIoDNI9MD+SPEqLdA+kjySPQiPdA7OR5BEV0j0wHJI8Sod0jyojySN6pHugdyR5\nRIF0j6ogyaOSSPdAZyR5RIt0jxgVJsmb2RozO2Rmz5tZSotWAL0j3QOBkryZLZD0P5I+Jem4pCcl\n3eTuB1teQ5JH5kj3KLuiJPnrJL3g7kfc/ZykH0n6XKB9lcrEhLRq1ezbxETeR1UdpHtUTagiv0zS\n0ZbtY43HKu/ii6UDB6Rf/Sq5HThQ6lVPS4317iNT4as/dROqXfMnkta4+8bG9p9L+gN3v7XlNT4+\nPn7+Z2q1mmq1WurHEsqgS2ZHuLx1VFjvvqQiu/pTq3q9rnq9fn5727ZtfbVr5O6p3yR9TNJjLdvf\nkLS57TVeZt/5jvvYmLuU3MbG3Ccne//ZRYuSW68/g2y9+qr7Aw+4r1rl/v73u995p/vx43kfFeZU\noX9UjdrZcz0OleRHlJx4/SNJJyT9UpGdeO2WyOdL+RGHjiiR7guuYl+PC3Hi1d3flPQ1ST+X9Kyk\nB1sLfAwuvVQaH5cWLUpuW7fOfKbm67tfeqm0a1dyi/RzGBV69wUX4pqyEWEy1BDmSuQVCxaVRLov\nkNDXlC0YrvGasebJ/HXrZj8+OSndcUdyf8eOUl9SEl0w7h5Zo8gXBH336iHdIwsU+QKZK+UjbqR7\nhESRBwqEdI+0UeSBAiLdIy2FGEIJYDbWzEFeSPJATkj3GARJHigJ0j2yQJIHCoR0j/mQ5IESI90j\nbSR5oOBI92hFkgciQ7rHMEjyQAmR7quLJA9UAOkevSLJl0TFVlPFAEj31UCSj1TWFwCfmJBWrZp9\nm5gItz8Mj3SPTkjyJZH1hUgmJ5NvCWfPJtsXXSQtWya9970zr+GbRPGR7uNDko9Ut8sNhrBpk7Rw\n4cz22Jg0PZ3dNwmkg3QPknyB5H0B8Pb9T08n6W90NPkDc9ddXNIwBqT7cus3yY+EPBj0p9l3b7ZI\nxsakL3xh5vnmBcClMMW1ff+jo9LISPLfW29N/tu8pGHobxIIp5nuN26cWe9++XLWu48VST5H7cn5\n7belgwel119PtrNOy536/vfdl/yxWbeOSxrGjHRfHvTkS6R9xMwzz0hr1mTXd2/Xqe9/880zly9s\nfpPYtYsCHxt69/EiyeeoU3I+fFi6+upkO4+0TFpHE+m+mLj8X8lMTs70uXfskG67LbsLgM91oveD\nH8xm/ygPrlVbHBT5kskzObePhR8bk+65J/lD0w9m41YH6T5/9ORLJs8+d/tY+IULk8f6lfVsXOSH\n3n35kOQjMEyS7tQu6lfWs3FRLKT7bJHkK2iYJL1pUzL+fXS09xTfvq7NJz8pfeIT+Y0KQr5I98VG\nko/AsEm63xO9nXr53/pWMiNWIsWDdB8SJ14rqte2SxonSef6o/LII8k2o3LQipE56aJdU1G9tl3S\nOEk612Jp69ZR4HGhlSul+++Xjh6VbrhB2rIlmQuyfbt04kTeRxc/knxEemm7pHWSlElTGAbpfnC0\nazCvNEbUSNlN2kK86N33jyJfAf301Tu9du1aaefO5D4pHEVBuu8NPfkK6Kev3um1l13GQmMoHnr3\nYZDkS6ifvjoTlVBmpPsLkeQroJ9LAWZ92UAgTaT74ZHkS6qf0S2MhEFMqp7uOfFaIf2MbmEkDGJT\n1ZE5FPmCYRleILwqpXt68gXDMrxAePTu50aSD4zRLUA+Yk33JPmCYXQLkA/SfYIkn4EsR7dwDgCY\nWwzpniRfQINc4q/9whyrViWPzYdzAMDcqpjuSfIFNehFtjkHAPSnbOmeIZSRGKZYp7XKJFAlZRl3\nX4h2jZltNbNjZvZ047YmxH6KYtDWSjfDnLAd5LqtQNXFeq3aIEnezMYlveru93Z5TTRJftDWynyG\nOWHLDFdgeEVM94Vo1zSK/GvuvrPLa6Ip8iH74BRroBiK0rsvUpH/oqTfSNov6evufqrtNdEUeYk+\nOFAVeaf7zIq8me2TtLTDU9+U9B+Sft3Y3i7pcnff0PbzPj4+fn67VqupVqsNdCxFwEqPQPVkke7r\n9brq9fr57W3btuWf5GftwOwqSQ+7+4q2x6NK8hKtFaCqskz3RWnXXO7uJxv3/1LS77v7zW2via7I\nA0DodF+UIv+Pkj4qySW9KGmTu0+3vYYiDyBaodJ9IYp8TzumyAOoiDTTPUUeAAoqjXRfiBmvAIAL\n5TGrliQPADnqN92T5AGgREKne5I8ABRMt3TPiVcAiEj7yJyf/YwiDwDRaab7r3yFIg8A0eLEKwDg\nPIo8AESMIg8AEaPIA0DEKPIAEDGKPABEjCIPABGjyANAxCjyABAxijwARIwiDwARo8gDQMQo8gAQ\nMYo8AESMIg8AEaPIA0DEKPIAEDGKPABEjCIPABGjyANAxCjyABAxijwARIwiDwARo8gDQMQo8gAQ\nMYo8AESMIg8AEaPIA0DEKPIAEDGKPABEjCIPABGjyANAxCjyABAxijwARIwiDwARo8gDQMQo8gAQ\nMYo8AERs4CJvZuvN7Bkze8vMVrY99w0ze97MDpnZ6uEPEwAwiGGS/AFJN0r6t9YHzewaSZ+XdI2k\nNZL+zsz4xhBYvV7P+xCiwu8zXfw+8zNw8XX3Q+7+XIenPidpt7ufc/cjkl6QdN2g+0Fv+EeULn6f\n6eL3mZ8QCfsKScdato9JWhZgPwCAeYx0e9LM9kla2uGpv3L3h/vYj/d1VACAVJj7cPXXzP5V0tfd\n/anG9u2S5O53N7YfkzTu7v/Z9nMUfgAYgLtbr6/tmuT70LrDhyT9k5ndq6RN8yFJv2z/gX4OEgAw\nmGGGUN5oZkclfUzSP5vZo5Lk7s9K+rGkZyU9KukvfNivCwCAgQzdrgEAFFfm49eZRBWOmW01s2Nm\n9nTjtibvYyobM1vT+Pw9b2ab8z6esjOzI2b2343P4wVtW3RnZt8zs2kzO9Dy2LvMbJ+ZPWdmj5vZ\n4m7vkcckJSZRheOS7nX332vcHsv7gMrEzBZI+lsln79rJN1kZr+b71GVnkuqNT6PzJfp3/eVfB5b\n3S5pn7t/WNIvGttzyryIMokqOE5oD+46SS+4+xF3PyfpR0o+lxgOn8kBufsTkl5pe/izkn7QuP8D\nSWu7vUeRkjKTqNJxq5n9l5l9d76vcbjAMklHW7b5DA7PJf2Lme03s415H0wklrj7dOP+tKQl3V6c\n1hDKWZhEFU6X3+03Jf29pDsb29sl7ZS0IaNDiwGft/R93N1Pmtl7JO0zs0ONdIoUuLvPN+coSJF3\n908P8GPHJb2vZfvKxmNo0evv1sz+QVI/f1Bx4WfwfZr97RJ9cveTjf/+2sz2KmmJUeSHM21mS939\nJTO7XNLL3V6cd7umfRLVn5nZmJl9QHNMosLcGv/Dm25UcpIbvdsv6UNmdpWZjSkZCPBQzsdUWma2\n0Mwua9xfJGm1+Eym4SFJtzTu3yLpp91eHCTJd2NmN0r6G0nvVjKJ6ml3/2N3f9bMmpOo3hSTqAYx\nYWYfVdJ2eFHSppyPp1Tc/U0z+5qkn0taIOm77n4w58MqsyWS9pqZlNSaH7r74/keUrmY2W5J10t6\nd2Py6RZJd0v6sZltkHRE0p92fQ/qKADEK+92DQAgIIo8AESMIg8AEaPIA0DEKPIAEDGKPABEjCIP\nABGjyANAxP4foj/lqSS4iDQAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb511372128>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.ylim(-10, 10)\n",
"plt.xlim(-10, 10)\n",
"p_x = np.linspace(-7, -3, 20)\n",
"p_y = list(map(lambda x: x* normal(scale=0.3) - 3, p_x))\n",
"plt.scatter(p_x, p_y, marker=\"v\", color=\"blue\")\n",
"n_x = np.linspace(3, 7, 20)\n",
"n_y = list(map(lambda x: x* normal(scale=0.3) + 3, n_x))\n",
"plt.scatter(n_x, n_y, marker=\"^\", color=\"red\")\n",
"plt.plot(np.linspace(-10, 10, 2), np.linspace(-10, 10, 2) * -0.6)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### モデル\n",
"\n",
"XY平面を分割する関数を仮定する.\n",
"(f(x, y) = 0 を境界として、直線の上下で分割される).\n",
"\n",
"\\begin{align*}\n",
" f(x, y) = w_0 + w_1 x + w_2 y\n",
"\\end{align*}\n",
"\n",
"#### 評価基準\n",
"\n",
"以下の誤差関数が最小であること.\n",
"(tは, +1 あるいは -1 とする).\n",
"\n",
"\\begin{align*}\n",
" E & = \\sum_{n} | f(x_n, y_n) | \\\\\n",
" & = - \\sum_{n} t_n (w_0, w_1, w_2) \\left(\n",
" \\begin{array}{c}\n",
" 1 \\\\\n",
" x_n \\\\\n",
" y_n\n",
" \\end{array}\n",
" \\right)\n",
"\\end{align*}"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 実装例"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import pandas as pd\n",
"from pandas import Series, DataFrame\n",
"from numpy.random import normal, multivariate_normal"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# データ準備\n",
"def perceptron_prepare_dataset(N1, Mu1, N2, Mu2, variance):\n",
" cov1 = np.array([[variance, 0],[0, variance]])\n",
" cov2 = np.array([[variance, 0],[0, variance]])\n",
"\n",
" df1 = DataFrame(multivariate_normal(Mu1, cov1, N1),columns=['x', 'y'])\n",
" df1['type'] = 1\n",
" df2 = DataFrame(multivariate_normal(Mu2, cov2, N2),columns=['x', 'y'])\n",
" df2['type'] = -1 \n",
" df = pd.concat([df1,df2],ignore_index=True)\n",
" df = df.reindex(np.random.permutation(df.index)).reset_index(drop=True)\n",
"\n",
" return df"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# 計算定義(出力と正解の誤差)\n",
"def perceptron_error(train_set, w0, w1, w2, bias):\n",
" err = 0\n",
" for index, point in train_set.iterrows():\n",
" x, y, type = point.x, point.y, point.type\n",
" if type * (w0*bias + w1*x + w2*y) <= 0:\n",
" err += 1\n",
" err_rate = err * 100 / len(train_set)\n",
"\n",
" return err_rate"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# 学習\n",
"def perceptron_simulation(train_set):\n",
" w0 = w1 = w2 = 0.0\n",
" bias = 0.5 * (train_set.x.mean() + train_set.y.mean())\n",
"\n",
" for i in range(30):\n",
" for index, point in train_set.iterrows():\n",
" # モデル定義\n",
" x, y, type = point.x, point.y, point.type\n",
" if type * (w0*bias + w1*x + w2*y) <= 0:\n",
" w0 += type * 1 \n",
" w1 += type * x\n",
" w2 += type * y\n",
"\n",
" return w0, w1, w2, bias"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"def perceptron_run(n_A, center_A, n_B, center_B, variance):\n",
" train_set = perceptron_prepare_dataset(n_A, center_A, n_B, center_B, variance)\n",
" w0, w1, w2, bias = perceptron_simulation(train_set)\n",
" print(\"ERR %.2f%%\" % perceptron_error(train_set, w0, w1, w2, bias))\n",
"\n",
" plt.title(\"perceptron\")\n",
" ymin, ymax = train_set.y.min() - 5, train_set.y.max() + 10\n",
" xmin, xmax = train_set.x.min() - 5, train_set.x.max() + 10\n",
" plt.ylim([ymin-1, ymax+1])\n",
" plt.xlim([xmin-1, xmax+1])\n",
" train_set1 = train_set[train_set['type']==1]\n",
" train_set2 = train_set[train_set['type']==-1]\n",
" plt.scatter(train_set1.x, train_set1.y, marker='^', color=\"red\")\n",
" plt.scatter(train_set2.x, train_set2.y, marker='v', color=\"blue\")\n",
" plt.plot(\n",
" np.arange(xmin - 5, xmax + 5),\n",
" - np.arange(xmin - 5, xmax + 5) * w1 / w2 - bias * w0 / w2\n",
" )\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ERR 8.00%\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu81XPa//HX1VH0ayolhGjG4c5NhQ5G7nZmSjETM5i5\nS4SQe4zc4zCVcU81Zcgxg6EoOpDDHIgwxbRn4jYMahDG6dbUpDCYkULq+v3xWVur3T7vtdb39H4+\nHuvROuy1v9f+7t31/azrczJ3R0REkq9J1AGIiEhhKKGLiKSEErqISEoooYuIpIQSuohISiihi4ik\nhBK6iEhKKKGLNICZ3WFmk6OOQySfErqklpk1y+KxJbuU0CWWzOxtMxtnZivM7AMzm2VmLXOvfcvM\nlpvZh2b2pJkdVOl9PzazF4CPzayJmfUzs//Nff3fzGxk7mtbmtnVZrbSzNaa2c1mtkPutTIzW21m\n483sPTP7PzMbnnvtbGA48GMz+9jMHqjm2E3NbGjuZ/jQzJaY2QGVYr3QzP5iZh+Z2d0VP6NIQyih\nS5wNBwYBXwX2Ay41s57ATOAsoD0wHVhgZs3z3vefwBCgLbAn8DBwPdAB6AEsz33dFcDXgO65fzsD\nP837Pp2AnYHdgZHADDPb191nAHcCU939/7n7cdUc+6vAXcCY3LEfBh7Ma707cBJwNLAPcDBwWgPO\nkwighC7x5cCN7v53d/8QuAwYRkjk0939zx7MAT4D+ua97xe5931GuCgsdvd73H2zu3/g7n8xM8t9\nrwvc/SN3Xw9cTkjI+f7H3Te5+x+BhcD3c89b7lY55vxjfx94yN0fd/fNwNVAK+Dree/5hbuvzf2M\nDxIuOCINojqfxNmqvPt/I7SUuwAjzey8vNea516r6n17AG9V8b07AjsCz4XcDoQEnd/I+dDdN+Y9\nXgnslrtf3ap2+cfeLRd3eIO7m9kqwieBCmvz7m+s9HOI1IsSusTZXpXuryEkyMvc/ec1vC8/2a4C\nelfxNe8TEmg3d3+nmu/Tzsx2dPcNucddgBeqOEZ1x14D5Nf3jVAC+nsd3itSbyq5SFwZ8AMz62xm\n7YGfAHcDtwHnmFlvC3Yys2PNrHU13+dO4JtmdpKZNTOznc2su7tvAW4FpplZR4DcsQZVev8kM2tu\nZkcCxwL35Z5fB3St5We4FzjWzI7K1fgvBD4F/reGn1mkwZTQJa6c0KG4CHgTeB2Y4u7PEWrfNwIf\n5J4/lWpat+6+CjiGkEz/ASwjdD4CjAXeAP5kZv8EFhM6XyusBT4ktLTnAqPd/bXcazOBbrnRK7+p\n5tivASOAG4D3CBeEb7v7FzX8zGqlS4OZNriQODKz/wNGufvvIzp+GTDX3feM4vgiDaEWuohISiih\ni1RPH18lUVRyERFJCbXQRURSIsrFi/TRQESkAdy9yiGukbbQ3b2gtwkTJhT8eyb9pnOic6Jzkq5z\nUhOVXEREUkIJXUQkJVKV0MvKyqIOIXZ0Tranc7I9nZPtJfGcRDZs0cw8qmOLiCSVmeFx7BQVEZHC\nUUIXEUkJJXQRkZRQQhcRSQkldBGRlFBCFxFJCSV0EZGUUEIXEUkJJXQRkZRQQhcRSQkldBGRlFBC\nFxFJCSV0EZGUUEIXEUkJJXQRkZRQQhcRSQkldBGRlFBCFxFJCSV0EZGUUEIXEUkJJXQRkZRQQhcR\nSQkldBGRlFBCFxFJCSV0EZGUUEIXEUkJJXQRkZRQQhcRSQkldBGRlFBCFxFJCSV0EZGUiDShn3EG\nrF8fZQQiIukRaUI3g0MOgWefjTIKEZF0aFRCN7M9zWyJma0ws5fMbEzu+fZmttjMXjOzRWbWtqr3\nz5wJU6bAMcfA1KmwZUtjohERyTZz94a/2WxXYFd3X25mrYHngOOB04H33f1KMxsLtHP3cZXe6xXH\nXrkSRoyAFi1gzhzo3LnBIYmIpJqZ4e5W1WuNaqG7+1p3X567vx54BegMDAVm575sNiHJV6tLFygv\nhwEDQgnmt79tTFQiItnUqBb6Nt/IbG/gD8C/A39z93a55w34oOJx3td7Vcd+6ik4+WQYOBCuvRZ2\n2qkg4YmIpEJNLfRmBTpAa+DXwPnu/nHI4YG7u5lVedWYOHHil/fLysooKyvj8MNh+XI491w47DC4\n6y7o2bMQUYqIJE95eTnl5eV1+tpGt9DNrDnwEPCIu0/LPfcqUObua81sN2CJux9Q6X1VttDz3Xkn\n/Pd/w/jx4d8mGjUvIhlXtBp6rpwyE3i5IpnnLABG5u6PBO5vyPc/+WR4+mm47z4YMgTeeacx0YqI\npFtj27xHACOAAWa2LHcbDFwBDDSz14Cjco8bpGtXWLoU+vQJHaYPPdTIiEVEUqpgnaL1PnAdSi6V\nLV0Kp5wC3/oWXHUVtGpVpOBERGKqaCWXUjvyyNBh+v770KsXvPhi1BGJiMRHohI6QNu2MH8+XHwx\nHHUU3HADRPQhQ0QkVhJVcqnsjTdg+HDo2BFuvx122aVAwYkUwq9+Ff498cRo45BUqankkuiEDrBp\nE0yYAHfcAbNmweDBjY9NpNE2boTddw/333kHdtgh2ngkNVJTQ69K8+bw85+HCUhnnw0/+hF89lnU\nUUnmTZ8eWhubNoX7IiWQ+BZ6vg8+gLPOgjffDAm+W7eCfnuRuqlonX/0UXjctq1a6VIwqW6h52vf\nPpQtf/hD6N8fbr5ZHaYSgenTYcOGrY83bFArXUqiIGu5xIkZnHlmGOI4fDg8+mhYd71Dh6gjk8z4\n7DM46KBtn/v002hikUxJVcmlss8/h0svDeWXO+6Ab36zqIcTESm6VI9yqYvHHoPTTgst9ilTwkYa\nIiJJlJkaenW++c0ww/Svf4XDDw//ioikTSYSOoQa+v33h1Ew/fqFuro6TEUkTTJRcqns5Zdh2DDY\nbz+YMQPatav9PSIicZD5kktl3bqFddY7d4bu3eEPf4g6IhGRxstkCz3fI4/AqFFw+ukwcWKYeSoi\nEldqoddgyBBYtizc+vULs0xFRJIo8wkdoFMnWLgwbHnXty/Mnq0OUxFJnsyXXCp74YXQYXrwwWHp\ngLZto45IRGQrlVzq4eCD4dlnYeedoUcPeOKJqCMSEakbtdBr8OCDYdz6OeeEJQSapW7lGxFJmsxP\n/W+Md96BkSPhk09g3jzYZ5+oIxKRLFPJpRF22y2s2HjCCdCnT1joS0QkjtRCr4dly0KHaa9ecNNN\n0KZN1BFJZmm/0sxSyaWAPvkELrggrOB4551hmKNISWm/0kxTyaWAdtopbD5z9dVw3HFhOd7Nm6OO\nSjJF+5VKNdRCb4TVq+HUU+GLL0KH6V57RR2RpJ72K808tdCLZI89YPFiOPZYOOwwuPfeqCOS1NN+\npVIDjaxupKZNYexYOOqorXuY/uIX0Lp11JFJKqVxv1J18BaMSi4FtH49jBkDS5eG4Y29ekUdkUjM\nqYO33lRyKZHWrWHWLLjsslCGmToVtmyJOiqRGFMHb0GphV4kK1fCiBFhQ+o5c8JmGiKSRx28DaIW\negS6dIHychgwAA45BH7726gjEokZdfAWnDpFi6hp07Co1ze+EdZaf/RRuPbaMJZdpKTi2PGYxg7e\niKnkUiL/+hece25Ymveuu6Bnz6gjksxQx2OqqOQSA23awNy5ocU+aFBoqavDVEpCHY+ZoRZ6BN56\nK5Rg2rSBO+4IKzqKFIU6HlNHLfSY6do1jFXv0yd0mD70UNQRSWqp4zFT1CkakWbN4Gc/g4EDw/DG\nRx+Fq66CVq2ijkxSRR2PmaKSSwx89BGMHg0rVsD8+dv//xMRqaCSS8y1bQt33w0XXxzWhLnhBtC1\nTkTqq9EtdDObBRwLvOvuB+Weaw/cA3QB3ga+5+4fVXqfWuhVeP310GHasSPcfjvsskvUEYlInBS7\nhX47MLjSc+OAxe6+H/B47rHUwb77wpNPQvfu0KNHqK1LDP3qV1sn64jEREFq6Ga2N/BgXgv9VaC/\nu68zs12Bcnc/oNJ71EKvRXl52EDjhBPgiiugZcuoIxJAE3UkUlHU0Du5+7rc/XVApyIdJ9XKymD5\ncli1KgxxfPnlqCMSQBN1JLaK1UL/0N3b5b3+gbu3r/QenzBhwpePy8rKKCsra3QsaeQOM2fC+PEw\neXIYEWNVXp+l6DRRR0qsvLyc8vLyLx9PmjSp2hZ6MUsuZe6+1sx2A5ao5NJ4f/0rDBsW9i697Tbo\n0CHqiDJo2rSwRdXnn4fHLVrAlVfC+edHG5dkRhQllwXAyNz9kcD9RTpOpuy/Pzz1VOg47dEDHn88\n6ogyqGKizqGHhttBB2mijsRGIYYtzgf6Ax0I9fKfAg8A9wJ7oWGLRbF4MZx2WphlOnlyaChKAsRx\nGVtJlJpa6JopmmDvvQejRsGaNWFJ3v32izoiqZFGx0gBaKZoSnXsCA88AGecAUccETpOdY2MMY2O\nkSJTCz0lVqwIHab77w8zZkC7drW/R0qoqtExN90UamUqv0g9qIWeAQceCM88Ezaj7t4d/vCHqCNq\nhDTOwqxqGdtRo+Css9SpKgWjFnrCTJ0K99237XMnnRRG0lV45JGQK04/HSZOhObNSxpi46S1zlz5\nF7duXbi1aAGXXRb/YY/qzI0NtdBTpGVLePFFeO65cHvxxe1z3pAhsGxZuPXrB2++GU2sDZKEOnND\nPkGMHRs2lH322bC7yfr14Wf85JNw1Y1zK33jxvBJQp8mYk8JPWFGj4Ydd9z6eMcdw3OVdeoECxeG\nlRv79oU5cxLQYbpxI0yaFJJcXBNdIZJbMXcRKka5KgkXWQGU0BOnVSuYMAF22incJk6sviphBmPG\nhAlIU6fC8OFb++RiKQnbpRUiuRVrclIxWtJJuMjKl5TQE2j06FAXb9686tZ5ZQcfHD7pt28fZpg+\n8UTxY2yQuM/CLFRyyy+/VNzyO0Eaqhgt6SRcZOVL6hRNqIb2US1YAGefHS4E//M/YW9TqaM4r+NS\nrEXD6tILLyVVU6eo/jsnVEMHGwwdCr16wciR0L8/zJsH++xT2NhSK84bLlfXkm7sxWbsWCXvBFFC\nz5iKBpc7vPtuWC7gxBPD5tRSizgntzhfbKRkVHLJmMpVg2bNQm396KPhxhuhTZto4xORmmkcunyp\n8rDH1q3DTkitWkHPnvD009HFJiKNo4SeMVUNe9x551BuvfrqUGOfMgU2by5eDFOnwmGHbXubOrV4\nxxPJCiX0DKpu2ON3vhNmn/7+9zBgAPztb8U5fl1mu4pI/amGnlE1DXvcvDm01q+5JiwIeNJJhT22\ntuUUaThtcCEN8uc/h9ml//EfcP31od5eKNOmwaWXhvtJWJtKJC6U0KXB1q8Pywc88UTYFemwwwrz\nfdO6qOJ2tEqhFJgSujTavffCD38IF10Ubk0K0PuS+lyXmauWlJKGLUqjfe97oQTz4IMwcCD8/e+N\n/54nnpjiZA7JWKUwjZuJZJha6FKr/OU83GHtWvjXv2DuXDj++Ghji6269PxG/RFFnyASSS30lCv2\nuO78YYbPPw/vvx92RLrgAjjnnG2XEJGc2lYpvPNOOO20aDeNSMInCKkXJfQUKPa47qo21bjiCli+\nPKwie+ih4b7kqWkp4I0b4cwzw8mLKplqnfN0cvdIbuHQUggbNri3beseCiLh/saNhT3Gdde577RT\nuE2btu1r8+a5d+jgfs017ps3F/a4qTR16tZfVrF+YbW57jr3Fi22xtCixfa/WImlXO6sMq+qhV5k\npZjmXp9djBqqpk01Tj45rAFz331hP9N33inssVNl48bwy8oXxaYRcd9MRBpEy+cWWUU5JH9PhFNO\nKfxxRo8On6Ar7hdaq1Zw663hflUXi65dw97HP/sZHHJI+NpvfavwcSTe9Olb/xgg7BPYqVPpk2mc\nlwKWBtMolyIr5TT3qAdNVFi6FEaMgG9/G666KlwMJEc7AEkjaWJRxLI4zf2jj8InhRUrwuYZlfde\nEJGGUUKvp0I3orI63NcdZs+Giy+Gn/40zDS1Kv8MRaSutKdoPRW67l1b/TmtzMJQ6xUrYNy48Cll\n771Dx6qqDCKFpxZ6FepT91ZJtHbTpsGPfxyGXEPY9u7qqwtUeopLx0FlcY1LEk8zReupPsMAtVlD\n7UaPDuexwubN8MYbYeRco2zcGGZaRjnbsipxjUtSTwm9GjWNu678dZVnURZj2GCSVb5ATpkCa9ZA\nnz5hP9MGi+vU9bjGJamnkksN6vqpOYujWOqrcsdwy5YwcyaMHw+TJ4eL4JVX1qN8Fddtj+IaVzGo\nrBQJlVwaqK7Lu9a1NZ9lFR3Dt94acptZWM7kiSdgxoywn+mmTfUoX9W2+FVU4hpXoamsFEsa5VIA\nWR3FUl9VXRz33x+eeip8wrn55nBRrBhdVGP5qmLqer44JJa4xlVoFWWlivv6WBoLKrlIbCxeHEos\nGzaExP7znytPxFKWykoxpJKLJMLAgfDCC2FC0mefwVFHRR1RTFW3y1Cpdh/KSlkpgVRySbA0joHf\na6+wGfWiRSGhX3EFnHGGZph+qaJ2DWH1s4pWcXXPF0NWykoJpBZ6gqV1DPxJJ4U+ifJyuP76sJ/p\nhx9GHVVMVDckspRDJceOhWef3faW5FZEihSthm5mg4FpQFPgNnefWul11dAbKQulzE8/Dbni/vth\nzhzo379h3ycVn2aq+4W7p/8PQb5U8hq6mTUFbgQGA92AYWb2b8U4VpaVYmOLqO2wQ2il33ILDBsG\nP/nJ1sEV9ZGKTzPV1a5V05acYpVcegNvuPvb7r4JuBs4rkjHyrT8MfAff1z83ZGiMmQILFsWbv36\nwZtv1u/9qZjRW90uQ9p9SHKK1SnaGViV93g10KdIx8q0/DHwq1eXZnekqHTqBAsXwg03QN++cM01\n4eerS4dpxaeZihm9ifw0U9MuQ4mqHUmxFKWGbmYnAIPd/azc4xFAH3c/L+9rfELe3oplZWWUlZUV\nPJYsyUJNvcILL4QSTPfuYULSV75S+3uyui69JFt5eTnl5eVfPp40aVJpN7gws77ARHcfnHs8HtiS\n3zGqTtHiyNK6Mhs3wkUXwcMPw7x5cMQRtb9Hy49I0pV8xyIzawb8FfgGsAZ4Bhjm7q/kfY0SehEU\nqxUa51EiCxbA2WfDOeeEi1kzza6QFCv5jkXu/oWZ/RD4HWHY4sz8ZJ5mUSe+Yq0rU+hdnApp6NDQ\nATxyZBjW2K8fPP74tl8Tl4uPSDFpLZcCmzYtJI78xHfllckvfSShPr9lC1x3Xejw3LgxbKQB6fkd\niIDWcimp9eu3JhII99evjy6eQknCmPcmTeDCC8OyAflthUQOURRpACX0Amvdum7PJVGx132fOrUw\n4+gPPzys1Nis2daNquN28REpBpVcCqwhpYmo6+71UcxRIoUsV1X8Hj7/PFxQx4yBceOgadPCxixS\naiq5lFBFaaJly3CrS2miENPSC9W6rU1dd3FqiELO5qzoHJ49O5zTxx4LqzeuWlX7e0WSSgm9CEaP\nDgmlVau6JaRCJLI4rFXS2ItKoev0FRefPfYICX3IkDAzvhRLhotEQQm9CCrvn1mXr29sIovDWiWF\nuKgUq07ftGkouSxcGDamPvPMdHRWi+RTDT0mCjEhKOpZooUa2ljs2Zwffxxq6k8+CfPnh1a7SFKU\nfKZoXSihb6+xiSwOa5VEfVGpj3vugfPOC8sHXHRRGPYoEndK6ClRl9EwUa9VEoeLSn2sXAkjRoQR\nNXPmQOfOUUckUrOST/2X4qjL9PuoF50q1tIDxdKlCyxZApdfHkovt9wCxx8fdVQiDaMWeoIkYfp9\nkj31FJx8MgwaBNdeu20ns0hcaBx6SiRh+n0hlGpMfWWHHw7Ll8Mnn4TW+vLlxT+mSCEpoSdMsaff\nx0GUY+rbtIG5c8PepQMHhsW+tmwpzbFFGksllwSqb8dnkpYWgPiUlt56K5Rg2rQJM0533bW0xxep\nikouKVPf6fdxmEVaH3EpLXXtCn/8I/TpAz17wkMPlT4GkfpQCz0D4tLirY+4DX9cujQMbxw6NCwY\n1qpVtPFIdqmFnnFxafHWR32XTyi2I48MnaTvvgu9e8NLL0Udkcj21ELPiLi1eJPKPdTTL744XCTP\nPTesuS5SKpopKkD0s0jT5PXXYfhw6NQJZs2CXXaJOiLJCiV0kSLYtCm00mfPDkn96KOjjkiyQAld\nEi/OQy+XLIFTTw3xXH55GFUkUizqFJXEi/PQywEDQofpypVhiOMrr0QdkWSVWuiSCEkYeukOt90G\nl1wCU6bA2Werw1QKTy10SbwkDL00g7POCmPWb7kFvvtd+Mc/oo5KskQtdEmMJA29/OyzsB7MPffA\nHXfAN74RdUSSFuoUldRI2tDLRYvg9NPDLNPJk8Ma9iKNoYQudRbn0SRJ9d57MGoUrFkDd90F++0X\ndUSSZKqhS53FeTRJUnXsCA88AGecAUccEcasqy0jxaAWumwjCaNJkmzFChg2DPbfH2bMgHbtoo5I\nkkYtdKmzJIwmSbIDD4RnngkXzR49wvK8IoWiFrpsp7bRJKqzF8bDD4fa+qhR4SLavHnUEUkS1NRC\nb1bqYCT+Kpauhapb5xV19s8/D49btIBTTildfGlxzDFhhulpp4Xlee+8E7761aijkiRTC13qTXX2\nwtqyBW64IcwuveaacHHUDFOpjmroUlCqsxdWkyZw/vnw+OOhnDV8+NaLpUh9qIWeAHGsWSdp1maS\nbNwIF10U6uvz5oVhjiL5VENPuDjWrGurs0vDtGoFN90ECxbACSfAOefApZdCM/1PlTpQCz1GqmuJ\njxmjmnUWrVkDI0fChg2hw3TvvaOOSOJANfSEqG6WpmrW2bT77vC734VVG3v3hvnzo45I4k4t9Bip\nafSIatbZ9vzzobO0d2+48UZo0ybqiCQqaqEnRE0t8Yqa9a23Kpln0SGHhE9tO+wAPXvC009HHZHE\nUYNb6GZ2EjAROADo5e7P5702HjgD2AyMcfdFVbxfLfQqqCUutfnNb+C//iv0rYwbB02bRh2RlFJR\nls81swOALcB04MKKhG5m3YC7gF5AZ+AxYD9331Lp/Uro1Ujamt9SeqtXh5FOW7aE4Y177hl1RFIq\nRSm5uPur7v5aFS8dB8x3903u/jbwBtC7ocfJohNPVDKXmu2xBzz2GAwZAocdtrURINlWjBr67sDq\nvMerCS11ESmgpk1DyeWhh2D8+LDI1/r1UUclUapxuoKZLQZ2reKlS9z9wXocp8raysSJE7+8X1ZW\nRllZWT2+ZfrFcYaoxE+vXmEUzJgxofN0/nw49NCoo5JCKS8vp7y8vE5f2+hhi2a2hG1r6OMA3P2K\n3ONHgQnu/nSl96mGXotp00Lyzp8heuWVYd0Pkarccw+cd15YPuCii8I6MZIuRd1TNJfQL3L353KP\nKzpFe7O1U/RrlbO3EnrttKqhNMTKlWFT6pYtYfZs6KyCZ6oUpVPUzL5jZquAvsBCM3sEwN1fBu4F\nXgYeAX6gzN0wmiEqDdGlCyxZAv37h9LLAw9EHZGUimaKxpzGpUtjPPUUnHwyDBoE114LO+4YdUTS\nWJopmmCaISqNcfjhsGxZGP1y6KFhhyRJL7XQRTJi3jz40Y/gkktCx7o6TJOpqJ2iDaWELlJ6b70V\nSjBt2oQO012rGpQssaaSi4gA0LUr/PGP0KdPWORr4cKoI5JCUgtdJKOWLg3DG4cODfMbWrWKOiKp\nC7XQRWQ7Rx4ZOknffTess/7SS1FHJI2lhC6SYe3awd13wwUXwIABYfMMfXBOLpVcRASA118PuyJ1\n6gSzZsEuu0QdkVRFJRcRqdW++8KTT8JBB0GPHmE/U0kWtdBFZDtLlsCpp4bVPS+/PKwLI/GgFrqI\n1MuAAaHD9O23wxDHV16JOiKpCyV0EanSzjvDr38NP/hBGBEzfbo6TONOJRcRqdWrr8KwYWElx9tu\ngw4doo4ou1RyEZFGOeAA+NOf4GtfCx2mv/991BFJVdRCF5F6WbQITj89zDKdPDnspCWloxa6iBTM\noEGhw/Tll+HrX4fXXos6IqmghC4i9daxIyxYEFrqRxwRJiLpA3f0VHIRkUZZsSJ0mO6/P8yYEZYT\nkOJRyUVEiubAA+GZZ8JWiT16hOV5JRpqoYtIwTz8MIwaFW4TJkDz5lFHlD5qoYtISRxzTOgwfe65\nMBnpzTejjihblNBFpKA6dQo7IQ0bBn37wty56jAtFZVcRKRo/vKXsCRv9+5w883wla9EHVHyqeQi\nIpHo3h3+/Gdo2zZ0mD75ZNQRpVuqEnp5eXnUIcSOzsn2dE62V8xzsuOO8MtfwvXXwwknwMSJ8MUX\nRTtcwSTx70QJPeV0Tranc7K9UpyToUPh+edDK71//7A0b5wl8e8kVQldROJt993DTkjf/W7YmHr+\n/KgjShcldBEpqSZN4MIL4dFHYdIkGDkSPv446qjSIdJRLpEcWEQk4aob5RJZQhcRkcJSyUVEJCWU\n0EVEUiIVCd3MTjKzFWa22cwOqfTaeDN73cxeNbNBUcUYBTMbnPu5XzezsVHHEwUzm2Vm68zsxbzn\n2pvZYjN7zcwWmVnbKGMsJTPb08yW5P6/vGRmY3LPZ/mc7GBmT5vZ8tw5mZh7PnHnJBUJHXgR+A6w\nzcKdZtYN+D7QDRgM/NLM0vIz18jMmgI3En7ubsAwM/u3aKOKxO2Ec5BvHLDY3fcDHs89zopNwI/c\n/UCgL3Bu7u8is+fE3T8FBrh7D6AHMNjM+pDAc5KK5Obur7p7VRthHQfMd/dN7v428AbQu6TBRac3\n8Ia7v+3um4C7CecjU9x9KfBhpaeHArNz92cDx5c0qAi5+1p3X567vx54BehMhs8JgLtvyN1tATQH\nnASek1Qk9BrsDqzOe7ya8MebBZ2BVXmPs/Sz16aTu6/L3V8HdIoymKiY2d5AT+BpMn5OzKyJmS0n\n/OyL3P0ZEnhOmkUdQF2Z2WJg1ypeusTdH6zHt8rKOM2s/JyN4u6exTkRZtYa+DVwvrt/bLZ1WHMW\nz4m7bwF6mNlXgN+a2b9Xej0R5yQxCd3dBzbgbX8H9sx7vEfuuSyo/LPvybafVrJsnZnt6u5rzWw3\n4N2oAypf1XVIAAABEklEQVQlM2tOSOZz3f3+3NOZPicV3P2fZrYEOJoEnpM0llzyZ1AtAP7TzFqY\n2T7AvsAz0YRVcs8C+5rZ3mbWgtA5vCDimOJiATAyd38kcH8NX5sqFpriM4GX3X1a3ktZPicdKkaw\nmFkrYCChbyFx5yQVM0XN7DvAL4AOwD+BZe4+JPfaJcAZwBeEj5e/iyzQEjOzIcA0oCkw090vjzik\nkjOz+UB/wt/GOuCnwAPAvcBewNvA99z9o6hiLCUz60cYDfYCW8ty4wkNnayek4MInZ5NCY3ce9x9\nipm1J2HnJBUJXURE0llyERHJJCV0EZGUUEIXEUkJJXQRkZRQQhcRSQkldBGRlFBCFxFJCSV0EZGU\n+P/4TJ+XS1YkrgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb51126ff98>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"perceptron_run(20, [15, 10], 30, [0, 0], 15)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# データのクラスタリング"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"N件の2つ組のデータ(x, y)がある.\n",
"データを眺めてみると, どうやらいくつかのグループに分けられそうだ.\n",
"\n",
"このとき, グルーピングしてみよう."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from numpy.random import normal"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAExtJREFUeJzt3V+oI+d5x/Hfs3s4oMQFc9iwdt2wrrHTxksg6oWJycUe\nSkfa3jQ9F63rC2OKGwoluah1EW8N9WmyBvvi+KKE5qZOiE3tODcn2C2RjhJ6Ui8NCaauY+ws3sXZ\njV2cTZYWWsPCeuOnFyPtSjr6MxrNP736fmCIRmc0MysrP73zvO87MncXACBMh8o+AQBAfgh5AAgY\nIQ8AASPkASBghDwABIyQB4CALRzyZvZ1M7tkZq8PPLdhZl0ze8vM9szs5kWPAwCYXxYt+W9IOjny\n3COSuu7+CUnf760DAApmWUyGMrPbJb3k7p/qrZ+VdMLdL5nZLZL23f13Fz4QAGAuedXkj7r7pd7j\nS5KO5nQcAMAUuXe8enypwL0TAKAEaznt95KZ3eLuvzCzWyX9cnQDMyP4ASAFd7ek2+bVkn9R0oO9\nxw9K+s64jdydJaPlscceK/0cQlp4P3k/q7rMK4shlM9L+ndJv2Nm75jZn0t6QlJkZm9J+v3eOgCg\nYAuXa9z9/gl/+oNF9w0AWAwzXgOxublZ9ikEhfczW7yf5clknHyqA5t5WccGgGVlZvIKdLwCACqA\nkAeAgBHyABAwQh4AAkbIA0DACHkACBghDwABI+QBIGCEPAAEjJAHgIAR8gAQMEIeAAJGyANAwAh5\nAAgYIQ8AASPkASBghDwABIyQB4CAEfIAEDBCHgACRsgDQMAIeQAIGCEPAAEj5AEgYIQ8AASMkAeA\ngBHyABAwQh4AAkbIA0DACHkACBghDwABI+QBIGCEPAAEjJAHgIAR8gAQMEIeAAJGyANAwAh5AAgY\nIQ8AASPkASBghDwABIyQB4CAEfIAEDBCHgACRsgDQMDW8ty5mV2Q9L+Sfi3pA3e/J8/jAQCG5Rry\nklzSprv/d87HAQCMUUS5xgo4BgBgjLxD3iV9z8xeMbPP53wsAMCIvMs1n3X398zsY5K6ZnbW3V/u\n/3F7e/v6hpubm9rc3Mz5dABguezv72t/fz/1683dszubaQcye0zS++6+01v3oo4NoCSdjrSzEz9u\ntaRms9zzCYCZyd0Tl8FzC3kz+4ikw+7+f2b2UUl7kv7O3fd6fyfkgZB1OtLWlnTlSrxeq0m7uwT9\nguYN+Txr8kclvWxm/ynpR5L+uR/wAFbAzs6NgJfix/1WPQqTW03e3X8m6dN57R8AMBszXgHko9WK\nSzR9tVr8HApVWMfrgQNTkwfCR8dr5irT8TrzwIQ8AMytSh2vAICSEfIAEDBCfoV0Oh01Gg01Gg11\nOp2yTwdAAajJr4hOp6OtrS1d6Y1brtVq2t3dVZOOMGCpUJPHWDs7O9cDXpKuXLmiHSamAMEj5AEg\nYIT8imi1WqoNTEyp1WpqMTEFCB41+RXS6XSul2harRb1eGAJMRkKqYz7AuBLAageQh5zGzfy5tFH\nH9Xjjz9+YDSOJIJ/FFP3USBCHjONttB3dnbU7XaHtllbW9O1a9eGnrvpppt09epVXb16VRLDMCVx\nz3QUjiGUmKrfau92u+p2u9ra2tLly5cPbDca8JL0/vvvXw94aXgYZrATrTodqdGIl3H/rrT3TJ+1\nXyAjtORXTKPRONBqr9frOnv27NA4+qQ2Njb08MMPjy3tLGULf7D0cuKE9PjjB1vp0o1tLl+WXn11\neB/1unTkSPx4XPmG1j8WMG9LXu5eyhIfGkWLosglDS31et3b7bZHUeQbGxsH/p5miaKo7H/q/Npt\n91rNXYqXQ4duPO4v9frwNuvr8TJpvVaL9zsoig7ut8j3q92OjxdFB88NldfLzsRZS7lmxbRaLa2v\nrw8998Ybb0iS9vb29Nxzzw2Np19fXz+wfbBGSy8ffnhwm4sXh7e5elU6flyKong5fjx+ri/rn7xb\ntMzTv4roduNla4tyUejm+UbIchEt+dz1W+dRFHl7oMVWr9entrxHX9dut71er/uhQ4cSteJrtdrQ\n8ZbGuBb2YGu+Votb8tNa4Ula6aNXDONa++Okfd2sf+MyXnWtMM3Zkifkl9ykIG+3216r1cYG77iS\nTZLySv9Y9Xrd19fXxwb8xsbGcga8+/gQPX16uLQxK2iTBnGakkkWAZ1nyFMGKgQhv0JGg/zQoUND\n9fVJQT7tC2DSl8a4Y4+27pe2BT8oSVDN2iarsBvdTxYBncXVQJH7xQGE/AoZF+SS3Mz88OHDiUoy\n9Xrd6/W6R1Hkp0+fnhj+kyT9UsCcJl1VZBGkebS4KQMVhpBfIZNCPmmdfNyVQJoyDnIwKTSrWhIh\n5Aszb8gzumaJjd5ZcpKNjY2x49ZH7zH/4bjRJKiWZlPa24uXpOPqi5h41WrF4/37arX4OZSOkF9i\nzWZTu7u7qtfrOnRo8n/KY8eOaWdnJ9Fs1MH9cDvigg2G8YkT2YRmUUMmm814Qld/KCmTu6pjnmZ/\nloso12Sq3xFqZkPllrW1taGRMKOdrKM1+NOnT1NjL0OSkT1pUEYJjuYs13Bbg8B0Oh2dOnVKFy9e\n1LFjxyRJr45Mu4+iSHt7e9e3566SFdBoxC3tQVEUl2WquF+UZt7bGqzleTIoXrPZHArqRqMx1/bI\nSVm3I261pDNnhu+TQwlupVCTDxw/+1cBSerieXVcUitfeZRrVgAlmZIlLZlU+cdHqnxuK4YfDQGq\npsi6eB5hzK2RK4WQB6qmqJDM6zh03lYKvwwFVE1RdfG0v1KFoDG6BihCs7m85Q1G6Cw1WvJAKBih\ngzGoyQPLbLSjVWIUTODoeAVWBaNeVhIdr8CqyLOjtYg7V6IQdLwCGDZ6hXDmDFcIS4yWPLCs8upo\nZShmUAh5YFkx6gUJ0PEKYBgdupXG6BoAi+OGZJVFyANAwBhCCQC4jpAHkA5j6ZcC5RoA86NztjSV\nKdeY2UkzO2tm58zsS3kdB0AJGEu/NHIJeTM7LOmrkk5KulvS/Wb2yTyOBQCYLK+W/D2Szrv7BXf/\nQNK3JH0up2MBKFpes22RubxC/jZJ7wysv9t7DkAImG27NPK6QVmiHtXt7e3rjzc3N7W5uZnT6QDI\n3DL/2tUS2d/f1/7+furX5zK6xsw+I2nb3U/21k9J+tDdnxzYhtE1ADCnqoyueUXSXWZ2u5mtS7pP\n0os5HQsAMEEuIe/u1yR9QVJH0puSXnD3n+ZxLAAZYGJTsJgMBaw6JjYtlaqUawAsCyY2BY2QB4CA\nEfLAqmNiU9CoyQPgR0KWCD8aAiAInfMd7fww/uJp3dtS806+eCRCHkAAOuc72nphS1euxR3CtbWa\ndu/bJejF6BoAAdj54c71gJekK9euXG/VYz6EPAAEjJAHUDmte1uqrd0Y8VNbq6l1LyN+0qAmD6CS\nknS8rmLnLB2vAFbCqnbO0vEKYCXQOZsMIQ8AASPkASylpJ2znfMdNZ5tqPFsQ53zq3cbZWryS47Z\n6FhlszpeQ6zb0/G6QrgNODBd49mGum93h56L7oi098BeSWe0ODpeVwi3AQcwCyEPYKosatpZ1cXn\n3Q+TqijXLLUsyzXU9sOTxUShLGraWdXF0+4ntAlT1ORXTBbhTG0/PFkFaxY17azq4iHW19OgJr9i\nmk1pby9e0oZy1Wr7nY7UaMRLZ/VGvGWCiULoI+RXwDKFZv+qotuNl62t6p9z1qo0rjuLmnZWdXHq\n6+lQrglcklJMlco1jUYc7oOiKL5SWQVl168n7SuL2n4WdfHQ6utpUJPHkKShmaS2X0TnbNqQD6Xj\nOMu6c5pALOo1SG/ekF/L82SwPJrN6cE42to/cyZda39WGLda8b4HrypaM67Iszq30DTvbC40CubM\nz8/MbP2neU3e+NIZRk0+cK1WHJR9SUJznCw6Z5PU25vNOKCjKF52d+Pnp/UpTDq30b6IWetVUOb9\nWNJ01latg7f/pdN9u6vu211tvbBVer9G2WjJB64fmlUoZUz7ohg9v/45TmqlD77m8uWDx7p8efh1\nP/hB/L9Xr45fH7ffMt6r5p1N7d63O9f9WKrQeq6KSV86q/zeEPIrYFYpJok0ZZQkRsN4tNQy7ovh\n1Cnp7Nkbz6+vx0s/sPtXLoOv6/9t0vq4/ZZV9plVZskryFr3tnTm52eGOmtnjV5J8xoUi3INEhlX\nRpk3/MaVjqT5y0AXLx4M8OPHh8/tyJH5zm3cfsueL1C0/lVEdEek6I4o0dVBmtdMkkUJimGWY7h7\nKUt8aJSh3XaPonhpt9Nvk8Wxo8hdGl6iaHj7Wu3G32o193p9+mvGvW59PV4mrSfdbxW0z7W9drrm\n2pZrW147XfP2uQz/I5Ugy39T+1zbo2cij56Jlv59GaeXnYmzliGUKybPcfNphjEmPZ/B/UrJzm/c\n62atV2W+wCyhjSDhlgXJzTuEkpb8ipnVck66zahxLe6kVwBprhqKutIISZVbuNEz0fVWfH+Jnqng\nZVQFaM6WPB2vyMSkkTNJWsFpOoaz6Ewucr9lq/qIHDpw80PH64pJMm4+q7H1UjXHoodqWsdl1caz\nj8qyAxfDaMkHaFptPMm4+TRj68cNsTxxgpmoRal6Sz2JeWfoIhk6XgNT5s3GRr9cdnZW+2ZjWZrV\n0Tqr43LSDcskBdWBuwq4d82KW6Q2vqjRevYqjTHPUxat9HEzaSUtfesfs1GTR26yrO2vsiT19CST\ngJp3NrX3wJ72HthT885m5ev0yAYhH5gqBWsWs2SRTNU6Lqv0wyerjpp8gEK5tzpiWf4AyDLvFzF+\nNAQIUF4zXPPYb9LZq6HN2i0KIY/CcMWAcZKEPK399OYNeWrySIUf3MYkSTqB6fQtDiGPVLL4pSiE\nqWqdwKuOcfIAMjdr9ir3qikONXmkUubMWoSBjtd06HhFYeh4BYpXiZA3s21JfyHpV72nTrl7e2Qb\nQh4A5lSVe9e4pKfc/amc9g8ASCDP0TXJf54KAJCLPEP+i2b2mpk9bWY353gcAAXivjTLJXW5xsy6\nkm4Z86dHJX1N0pd761+RtCPpodENt7e3rz/e3NzU5uZm2tMBUIAQfpxk2ezv72t/fz/163MfXWNm\nt0t6yd0/NfI8Ha/Akkl6XxrkpxK3NTCzWwdWtyS9nsdxAADT5VWTf9LMfmJmr0k6IemvczoOgAIl\nuS8NqoXJUADmwkzVclViMlSiAxPyADC3StTkAQDVQMgDQMAIeQAIGCEPAAEj5AEgYIQ8AASMkAeA\ngBHyABAwQh4AAkbIA0DACHkACBghDwABI+QBIGCEPAAEjJAHgIAR8gAQMEIeAAJGyANAwAh5AAgY\nIQ8AASPkASBghDwABIyQB4CAEfIAEDBCHgACRsgDQMAIeQAIGCEPAAEj5AEgYIQ8AASMkAeAgBHy\nABAwQh4AAkbIA0DACHkACBghDwABI+QBIGCEPAAEjJAHgIAR8gAQMEIeAAJGyANAwAh5AAgYIQ8A\nASPkASBgqUPezP7EzN4ws1+b2e+N/O2UmZ0zs7Nm1lj8NAEAaSzSkn9d0pakfxt80szulnSfpLsl\nnZT0D2bGFUPO9vf3yz6FoPB+Zov3szypw9fdz7r7W2P+9DlJz7v7B+5+QdJ5SfekPQ6S4f9E2eL9\nzBbvZ3nyaGH/pqR3B9bflXRbDscBAMywNu2PZtaVdMuYP/2Nu780x3F8rrMCAGTC3BfLXzP7V0kt\nd/+P3vojkuTuT/TW25Iec/cfjbyO4AeAFNzdkm47tSU/h8EDvijpOTN7SnGZ5i5JPx59wTwnCQBI\nZ5EhlFtm9o6kz0j6FzP7riS5+5uSvi3pTUnflfRXvujlAgAglYXLNQCA6ip8/DqTqPJjZttm9q6Z\nvdpbTpZ9TsvGzE72Pn/nzOxLZZ/PsjOzC2b2k97n8UDZFtOZ2dfN7JKZvT7w3IaZdc3sLTPbM7Ob\np+2jjElKTKLKj0t6yt3rvaVd9gktEzM7LOmrij9/d0u638w+We5ZLT2XtNn7PDJfZn7fUPx5HPSI\npK67f0LS93vrExUeokyiyh0d2undI+m8u19w9w8kfUvx5xKL4TOZkru/LOl/Rp7+I0nf7D3+pqQ/\nnraPKrWUmUSVjS+a2Wtm9vSsyzgccJukdwbW+QwuziV9z8xeMbPPl30ygTjq7pd6jy9JOjpt46yG\nUA5hElV+pry3j0r6mqQv99a/ImlH0kMFnVoI+Lxl77Pu/p6ZfUxS18zO9lqnyIC7+6w5R7mEvLtH\nKV72X5I+PrD+W73nMCDpe2tm/yhpni9UHPwMflzDV5eYk7u/1/vfX5nZruKSGCG/mEtmdou7/8LM\nbpX0y2kbl12uGZ1E9Wdmtm5mv60Jk6gwWe8/eN+W4k5uJPeKpLvM7HYzW1c8EODFks9paZnZR8zs\nN3qPPyqpIT6TWXhR0oO9xw9K+s60jXNpyU9jZluS/l7SEcWTqF519z909zfNrD+J6pqYRJXGk2b2\nacVlh59J+suSz2epuPs1M/uCpI6kw5Kedveflnxay+yopF0zk+Ks+Sd33yv3lJaLmT0v6YSkI73J\np38r6QlJ3zazhyRdkPSnU/dBjgJAuMou1wAAckTIA0DACHkACBghDwABI+QBIGCEPAAEjJAHgIAR\n8gAQsP8HtRdzkVkJDRcAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb5111e8240>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.ylim(-10, 10)\n",
"plt.xlim(-10, 10)\n",
"a_x = np.linspace(-5, 0, 20)\n",
"a_y = list(map(lambda x: x* normal(scale=0.2) - 3, a_x))\n",
"plt.scatter(a_x, a_y, marker=\"o\", color=\"blue\")\n",
"b_x = np.linspace(0, 5, 20)\n",
"b_y = list(map(lambda x: x* normal(scale=0.4) + 3, b_x))\n",
"plt.scatter(b_x, b_y, marker=\"o\", color=\"red\")\n",
"c_x = np.linspace(2, 7, 20)\n",
"c_y = list(map(lambda x: x* normal(scale=0.2) - 3, c_x))\n",
"plt.scatter(c_x, c_y, marker=\"o\", color=\"green\")\n",
"d_x = np.linspace(-5, -3, 20)\n",
"d_y = list(map(lambda x: x* normal(scale=0.1) + 3, d_x))\n",
"plt.scatter(d_x, d_y, marker=\"o\", color=\"black\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 実装例"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.cm as cm\n",
"import numpy as np\n",
"import pandas as pd\n",
"from pandas import Series, DataFrame\n",
"from numpy.random import normal, randint"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# データ準備\n",
"def kmeans_prepare_dataset():\n",
" dataset = []\n",
" for i in range(100):\n",
" dataset.append(np.array([randint(100), randint(100)]))\n",
"\n",
" return dataset"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# 計算定義\n",
"def kmeans_cluster(point, center, k):\n",
" kix = 0\n",
" min_dist = 100*100*2\n",
" for i in range(k):\n",
" d = sum([x*x for x in point - center[i]])\n",
" if d < min_dist:\n",
" min_dist = d\n",
" kix = i\n",
"\n",
" return kix, min_dist\n",
"\n",
"def kmeans_center(sum_points, num_points, k):\n",
" center = []\n",
" for i in range(k):\n",
" center.append(sum_points[i] / num_points[i])\n",
"\n",
" return center"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# 学習\n",
"def kmeans_simulation(train_set, k):\n",
" cls = [0] * len(train_set)\n",
" center = []\n",
" for i in range(k):\n",
" center.append(np.array([randint(100), randint(100)]))\n",
" distortion = 0.0\n",
"\n",
" for iter_num in range(50):\n",
" sum_points = []\n",
" for i in range(k):\n",
" sum_points.append(np.array([0, 0]))\n",
" num_points = [0] * k\n",
" sum_dist = 0.0\n",
"\n",
" for pix, point in enumerate(train_set):\n",
" point = np.array(point)\n",
" kix, min_dist = kmeans_cluster(point, center, k)\n",
" cls[pix] = kix\n",
"\n",
" sum_points[kix] += point\n",
" num_points[kix] += 1\n",
" sum_dist += min_dist\n",
"\n",
" center = kmeans_center(sum_points, num_points, k)\n",
"\n",
" if iter_num > 0 and distortion - sum_dist < distortion * 0.001:\n",
" break\n",
" distortion = sum_dist\n",
"\n",
" return cls, center, distortion"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"def kmeans_run(k):\n",
" train_set = kmeans_prepare_dataset()\n",
" cls, center, distortion = kmeans_simulation(train_set, k)\n",
" print(\"distortion: %f\" % distortion)\n",
"\n",
" plt.ylim(-10, 110)\n",
" plt.xlim(-10, 110)\n",
" for pix, point in enumerate(train_set):\n",
" plt.scatter(point[0], point[1], marker=\"o\", color=cm.hot(cls[pix] / k))\n",
" for pix, point in enumerate(center):\n",
" plt.scatter(point[0], point[1], marker=\"D\", color=cm.hot(pix / k), s=100)\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"distortion: 104357.552703\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD7CAYAAAB37B+tAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGrRJREFUeJzt3W+QXNV55/HvE+TJtkViaVCtIJiAt8qsQYuygsRhnT+0\nWPUMVumP9WKxlUrKRZVfxetoycTJKLjicRUKApcwuFK8SBxYrWtR7HVWwLgczbSwGke7S3BARkSg\ngKsCBrskR5JJ2SlcwuVnX9zuUU+r50933773nnN/n6ouum93T597Z3h0+jnPOcfcHRERCdfP5N0A\nEREZjAK5iEjgFMhFRAKnQC4iEjgFchGRwCmQi4gEbkXWH2hmqncUEemDu1u347n0yN09s9unP/3p\nTD8v65vOL+ybzi/cW9bnthilVkREAqdALiISuOgDebVazbsJQ6XzC5vOL1xFOjdbKveS+geaedaf\nKSISOjPDizTYKSIi6VEgFxEJnAK5iEjgFMhFRAKnQC4iEjgFchGRwCmQi4gEToFcRCRwiwZyM3vY\nzE6b2Qttx0bNrG5mL5vZrJmtantut5m9YmYnzWxsmA0XEZHEUj3yR4DbOo5NAnV3vxZ4svkYM7se\n+DBwffM9D5mZevwiIkO2aKB1978FftBxeBuwv3l/P/Ch5v3twAF3f9vdXwW+Dbw/vaaKiEg3/fSY\n17r76eb908Da5v1fAN5oe90bwJUDtE36cHhmhq1jY2wdG+PwzEzezREZ3PEZ2DuW3I7rb7qbgXYI\ncndfYsefrs9NTU3N3a9Wq4VaRSxkh2dm2LljB2+99RYATx89yoGDB9k0Pp5zy0T6dHwGHtwB55O/\naV45CrsOwvr4/6YbjQaNRmNZr11y9UMzuwaYdvcbmo9PAlV3P2VmVwBH3P19ZjYJ4O57m687BHza\n3f+u4+dp9cMh2To2xpF6fd6xjbUa07OzObWoZI7PwNf2Jfc3TxQ/2ITQ3r1jcGL+3zTrajBZvr/p\nxVY/7KdH/gTwUeDe5n8fazv+qJndT5JSeS/wTB8/XyQ8ofUcQ2uvLGqp8sMDwP8F/r2ZvW5mdwB7\ngZqZvQzc2nyMu78IfBl4Efgb4HfV9c7WrokJKpXK3ONKpcKuiYkcW1QiX9t3IShCcr/V2y2iUNq7\neQJGLvxNM1JJjsk8i/bI3X3nAk9tWuD1fwr86aCNkv5sGh/nwMGDPLgv+R9y18SE8uMStvXjyTeF\noqeAcqYdgkTScHwGPrcNfnI+ebxiBO58orhBpzO1MlJRaqWbAo0jpJ0jF5HQqae7tIDGEdQjF0mD\nqiviU7DfqfbsFBGJmAK5SBpUXRGfgH6nSq2IpKVAA2OSkgL9ThdLrSiQi4gEQDlyEZGIKZCLiARO\ngVxEJHAK5CIigVMgFxEJnAK5iEjgtNaKiPSvQHXWZaY6chHpj1ZQzFQp68g/u2cPV112GVdddhmf\n3bMn7+aIxCeUzSlKIMrUymf37OEzn/rU3OPW/U/edVdeTRIRGZooUytXXXYZPzh3bt6x1aOjvH72\n7FA/V6RUlFrJVClTK9KbwzMzbB0bY+vYGIdnZvJujoSgtTnFulpyUxDPTZSpld/7/d+fl1ppHZPu\nDs/MsHPHDt56K+lZPX30KAcOHtR+n7K09eMK3gUQZWoFkjz55++/H0iCuPLjC9s6NsaR+vydUDbW\nakzPancbGZJ+yxZLXO6oZWxlUQrkkql+c+slz8krRy6L2jUxQaVyYSeUSqXCroli7oQiEei3bFHl\njgtSIBc2jY9z4OBBNtZqbKzVlB8XCYxSK1JIh2dmeHBf0tvaNTGhf1hiotRKX5Qjl6B0VtFUKhV9\nS4iNBjt7pkAuhdWt563BV5GLLRbIo6wjlzAsVL8uIr3pe7DTzO40s38wsxfM7FEz+1kzGzWzupm9\nbGazZrYqzcZKXB7ct28uiAO89dZbPLhv30VVNCMjI5w7c0azTkUW0FcgN7MrgU8AN7n7DcAlwEeA\nSaDu7tcCTzYfR0fT2fuz3OvWXkWzfsMGAJ4/dowj9To7d+zQNRfp5O4934Arge8Aq0nSM9NADTgJ\nrG2+5nLgZJf3esjqhw75mkrFV4KvBF9TqXj90KG8m1V43a7bfXffveS13FKrzT3fum2p1XI6C5H8\nNGNn15jcV4/c3b8L7GsG8+8Bb7p7vRnETzdfdhpY298/L8W1UDpAFtftun3jqadUvy6Sgr4GO81s\nNbANuAb4F+B/mdlvt7/G3d3MupanTE1Nzd2vVqtUq9V+miER2DQ+vmjw3jUxwdNHj84rRdSsUymD\nRqNBo9FY1mv7Kj80s/8CjLv7x5qPfwe4GbgV2Ojup8zsCuCIu7+v473ez2cWhWqc+zPIdetpclCJ\n64yFqH//qdeRm9n7gYeBXwF+DPx34BngauCsu99rZpPAKnef7Hhv0IEcNOuwX0O/biWf+Vd6kf/+\nhzIhyMymgA8DPwGeAz4G/BzwZeAXgVeB2939zY73BR/IpaD2jsGJ+ROJWFeDyYgmEkXc4xxY5L//\noUwIcvcpYKrj8DlgU78/U0QW0dnjfOVoVD1O6Z9mdko8Nk8kwa39q/Xmifm92OtugZeeuvD6kILg\nQsu4hnQOw7TQ778EFMglHq09JNtTDzC/F9v+1Vs92rh0+/2X5HerRbMkbt3ypu1CyqEWfTBP+fuh\n0qJZIjEoco9T+ftcKZBL3Drzpu1CzKEWddd65e9zpUC+CNWLR6CzFxvyYOcglPaImnLkC9AMTolG\nFrn1oufvI7BYjlybL3doLbV6x2/9lhbHkjhksft865vPulpyyyKIH59JBrP3jiX3S0yplTadvXAR\n6UGW+XsNrs6jHnmbzqVW22nVPQnW5okk1dFiPwM/OhN2LzaLbxkBUSBfwurRUa2VLWFrpT2u3pAE\ncf8pvHYs6dGGHMxljgJ5m869IiuVCo88+ijTs7MK4iFLI5caej52/ThcuiYJ4i0h92I7v2WEWEqa\nIuXI27T2ilTJYUTSyKUqH1s8RZ4clQOVH0ZGte8d0ljaNJblUVUiGDRN0S+Jzqqbp48eVW5fLlAv\nNlrKkUdkmBtDt+rrt46NcXgmoBxxGrnUmPKx68eTbxKTswrivSrwOIl65LKkoHv6afRC1ZOVgo+T\nKEcekWEtK7B1bIwj9fk54o21GtOzgeWIRfpVgHESTdEviVbVzcZaTbXvIiWiHrksSQuISekVoOJn\nsR65AnlBFa2MsGjtEelJGsv45rwUsAJ5YNQDFklRAXrTaVCOPDDDLCMMRbDljlI8JVhgS+WHUjhB\nlzuK5EA98gLqtnhXmZbQ1TcSSdXmCVgxcuHxipFwJ3QtQIG8gFRGKCK90GCnFI4GeyVVBZjMkwYN\ndkpQ9I1EpDd998jNbBXwBWAd4MAdwCvAl4CrgVeB2939zY73qUcuIotLs2a7BOWHgwTy/cBT7v6w\nma0AVgJ3AWfc/T4z+yNgtbtPdrxPgVxEFjaMwJvzZJ40pB7IzexdwDF3/3cdx08Ct7j7aTO7HGi4\n+/s6XqNALiILiySnnbZh5MjfA/yzmT1iZs+Z2V+Y2Upgrbufbr7mNLC2z59fCJqUIhKhAq8r3q9+\ne+S/DPw/4APu/k0zewD4IfBf3X112+vOuftox3uD6JGrckIkJ8PMaQecLx/GVm9vAG+4+zebj78C\n7AZOmdnl7n7KzK4Avt/tzVNTU3P3q9Uq1Wq1z2YMz0KTUhTIu9OiWinoJY8bQc53QcPcyGOh6foF\nvH6NRoNGo7Gs1w4y2PkN4GPu/rKZTQHvbD511t3vNbNJYFWog53aTGH59O0lBb30FAPuVebuUzfC\na8fmH7t6A9z9XD7t6cGw6sg/AfxPM3seWA/sAfYCNTN7Gbi1+ThIZZ8m34vFptRrnGGZelnYqQSL\nQElv+l40y92fB36ly1Ob+m9OcbQmpShd0D8tfiWFc+ma5R0LjGZ2LmLT+DjTs7NMz84q+CxioW8v\nWvyqB5snkhRJy0hl4YWdenmtzBfptdNaK5KKboOdGmfoUZ6DnS80f0831Ab7OSEIdKBYOwRJLjQI\nGohnH4eHdib3f/cA3LQ93/ZIVwrkkhuVJRZcK4i3V8AomBeSArksSQF3SIr8Nb4ziLcomBeSArks\nSimQISlyvfdCQbxFwbxwtB65LErVJUNS1HrvpYI4JM89tDN5rRReKQK5JqWINC0niLcsJ5hHuABV\niKJPrShtsDRdoyEpWmrlhTo8sH15QbzdSAX+2+MXlyYW7fwiV+rUitIGS9PWakPSWvxpXS25xRbk\nipo6KqG+p+hLXDaNjyt4D8P68eIE7xtqyQDmclMrcGHQswwThQIWfY9ci1+JtLlpexKY26epL2Sp\nypVIp7uHKPocOahGWuQiaZUfFrlOPjKqIy8R/aMViSwCpCYEBUWBvCRUfRKJLKtBNEU/GKWuWllM\nt/ryPGvOB/1sVehEIstqkPacuYJ4sEpbtdJt04M/vOsu7tuzJ5eNELQJg+Tmpu1JnTioOiVQpe2R\nd+u9fv7++3Pr0abRm941McHIyMjc45GREVXohCiPapAbagriASttjzxGP/7xj8HaUmhmyTEJyzB3\nkZc4uXumt+Qj81c/dMjXVCq+Enwl+JpKxe+7++6LjtUPHcqtPb189vRjj817f/vPmX7ssSG2fPjq\nhw75llrNt9Rqmf0+RIqmGTu7x9WFnhjWrSiB3L17gMgzaPT72QsF8RiC+aD/wInEYrFArvLDHKRZ\n6/3Vxx/njp075+XXu6lUKjxy4ABbtodVkaB9P0USKj8skMMzM9y+bRtH6nWO1Ovcvm1b32WOyw3i\nkAye3rFzJ199XOtLi8RGgTxjf7J7N+fPn597fP78ef5k9+6ef87X6/VlB/GWVjD/ekcPt8i0Vo7I\n0hTIM/b6a68t65gktMSuyNKUI8/Yr914I88fOzbv2C9t2MD/ee65nn9WL6kVCDdPLiLKkRfKZ+65\n56JJO5+5556+ftaW7dt55MCBeamHhcQWxEu7fV/WW6tpK7cgqEeeg7RXKFyqZx5jEC/l4mBZb62m\nrdwKZWirH5rZJcDfA2+4+1YzGwW+BFwNvArc7u5vdrwn80BehqVdFwrmsQVxKHFJ4t4xONExUL2u\nBpNDOu+sP69fJVkTfZiplV3Ai0ArMk8CdXe/Fniy+ThXrd5bq9xv544dUX4V75ZmiTGIi8zT+tZw\nop7cHtxRyhRQ34HczN4NbAa+ALT+ldgG7G/e3w98aKDWpaBMS7u2B/OYg3hpSxKXs5hWmjntELZy\n0wbQwGCLZn0O+CTw823H1rr76eb908DaAX6+9GHL9u18qTnp59ZanKvZtUoSY0+XXWSpxbQ6c9qv\nHB0sp63Fu4LRV47czLYAH3T3j5tZFZho5sh/4O6r2153zt1HO96baY68c2BsZGSE69atY3TNmvIE\nACmHUHLaaSrRgOxiOfJ+e+QfALaZ2Wbg3wA/b2ZfBE6b2eXufsrMrgC+3+3NU1NTc/er1SrVarXP\nZiytvfd29swZTp44MVfHrc0bRAIX8beGRqNBo9FY1msHLj80s1uAP2j2yO8Dzrr7vWY2Caxy98mO\n1+dWfljaagcphxL1TssoiwlBrci8F6iZ2cvArc3HuWtNHvnWs8/m3RSR4Wn1TtfVkpuCeGlEPyGo\nM0ferjQTSUQkeKWeot9ZfgiwenQ09QWYSjtlPGO6ziIXK+Wenf/xpptSzYt39vo1iDocus4i3UXf\nI89i8kiZJh3lSdc5Bb1OGNKiWUGIvkde2skjIp16nTCU9gQjGZroe+SQBPPp2VmmZ2eHEsRLO2U8\nY7rOA+p1OrumvwejFIF82LSLTTZ0nUW6i778UESaep0wpAlGhTK09cj7bIwCuUheel27uyRrfYdA\ngVwKqwybfoikQYFcCqm0W7aJ9KHUMzuluFQXLpIOBXIRkcApkEtuVBcukg7lyCVXGuwUWR4NdoqI\nBE6DnSIiEVMgFxEJnAK5iEjgFMhFRAIX/XrkIrnSWiWSAVWtiAyLVg+UFKlqRVKlDZCXSRszSEaU\nWpGeaANkkeJRj1x6ooWuerB5IkmntIxUkmMiKVMg76C0Qe++9eyzul7drB9PcuLraslN+XEZEg12\nttH62EvrvEbtdL1EhkeDncuktMHS2jdAXj06Ou+5IlyvTL5RHZ+BvWPJ7bi+hUj+NNgpPds0Ps6m\n8XG2jo1xpF7PuzlzMhmI7SwpfOWoUiaSO/XI22h97OVp9XrPnTnDyMjI3PG8r1cm36iGUVKoHv7F\ndE160leP3MyuAv4H8G8BB/7c3T9vZqPAl4CrgVeB2939zZTaOnSttIHWx15YZ693ZGSE9Rs2cNma\nNbpe/VAP/2K6Jj3rt0f+NnCnu68DbgY+bmbXAZNA3d2vBZ5sPpaIdPZ6z58/z2Vr1jA9O5t7EM/k\nG1XaJYWaNHQxXZOe9dUjd/dTwKnm/R+Z2UvAlcA24Jbmy/YDDQIK5prsErZMvlG1SgrTWj/lR2eW\nd0xkEQOXH5rZNcBTwH8AvuPuq5vHDTjXetz2+sKWH3YbvNtYqzE9O5tTi4pHJZop+9SN8Nqx+ceu\n3gB3P5dPe4pAa9R0tVj54UBVK2Z2KfDXwC53/2ESuxPu7mbWNWJPTU3N3a9Wq1Sr1UGaIRnSOELK\nLl2zvGNlkva3nkA1Gg0ajcayXtt3j9zM3gF8Ffgbd3+geewkUHX3U2Z2BXDE3d/X8b7C9sjV25TM\nqfcpy5T65svNtMl+4Ky739l2/L7msXvNbBJY5e6THe8tbCAH7eouOdCa5bIMwwjkvw58AzhOUn4I\nsBt4Bvgy8IssUH5Y9EDei6IE/TTaUZRzKQJdCymixQI57p7pLfnI8NUPHfI1lYqvBF8JvqZS8fqh\nQ0G2oyjnUgS6FlJUzdjZNa5qZmefirIuSxrtKMq5FIGuhYRIgVxEJHAK5H0qyrosabSjKOdSBLoW\nEiKtRz6AogyKabAzXboWUkSpV60M2JhoArmISFaGNrNTpJRU9y0Fox65SC80E1Nyoq3epBCi2Nh6\nkCVWtVmCDIlSK5KJ0i8RrM0SZIjUI5dMRDPRpt+NJbRZggyReuQivdASq1JAGuyUTJR+iWANksqA\nVEceidAnqoTe/oGpbFEGoEAegdL3aEVKTuWHEYhmsFAkKyUq99Rgp4jEp2TlnuqRB0Kr8on0oGTl\nngrkgWjtXr+xVmNjrab8uIjM0WCniMQnwnJPVa2ISPlEVu6pQC5zSl/LLRIoBXIBVIsuEjLVkQug\nWnSRWCmQi4gEToG8RDpr0UdGRjh35kzYGz2IZKXAM0WVI89B+4Djb95yC9946ikgm8HH1mefPXOG\nkydOcP78eUD5cpFFFaCcUYOdBdI54Nguy2C6dWyMI/X6vGMbazWmZ2eH/tkiwdk7Bifm///CuhpM\nZvf/iwY7C6RzwLGdBh+XFsW+n4PI6+t9gdMKCwqxzX1KfdEsM7sNeAC4BPiCu9+b9mfI4HZNTPD0\n0aPzShGLvnaL9v3MaSGoEBegSrvNmyeSn9GeWlnOFn8ZSbVHbmaXAH8G3AZcD+w0s+vS/IzQdQ44\ntssymIa4dkshyyez7PXltRBUiAtQpd3m1hZ/62rJrWD/kKXdI38/8G13fxXAzP4K2A68lPLnBKsV\nQPMa7OxsS9GDd6GF2FOV/q0fL+zvNu1AfiXwetvjN4BfTfkzgtcZQD951105tiYchUsHLdTrG9b/\n7Hl9vS94WqGrENs8gLQD+bLKUaampubuV6tVqtVqys2QGHV+myndWjGtr/dZLwSV1+cOIsQ2d2g0\nGjQajWW9NtXyQzO7GZhy99uaj3cDP20f8Cx7+aFEpAC1xVIemdWRm9kK4B+B/wx8D3gG2OnuL7W9\nRoFc4hHZUqlSXJlOCDKzD3Kh/PAv3f2ejucVyEVEeqSZnSIigdPMThGRiCmQi4gEToFcRCRwCuQi\nIoFTIBcRCZwCuYhI4BTIRUQCp0AuIhI4BXIRkcApkIuIBC76QL7cZSBDpfMLm84vXEU6NwXywOn8\nwqbzC1eRzi36QC4iEjsFchGRwOWyjG2mHygiEonCrEcuIiLpUmpFRCRwCuQiIoGLOpCb2W1mdtLM\nXjGzP8q7PYMys6vM7IiZnTCzfzCz32seHzWzupm9bGazZrYq77b2y8wuMbNjZjbdfBzTua0ys6+Y\n2Utm9qKZ/Wpk53dn8+/yBTN71Mx+NuTzM7OHzey0mb3QdmzB8zGz3c1Yc9LMxrJsa7SB3MwuAf4M\nuA24HthpZtfl26qBvQ3c6e7rgJuBjzfPaRKou/u1wJPNx6HaBbwItAZvYjq3B4Gvuft1wHrgJJGc\nn5ldCXwCuMndbyDZfP0jhH1+j5DEj3Zdz8fMrgc+TBJrbgMeMrPM4mu0gRx4P/Btd3/V3d8G/grY\nnnObBuLup9z9W837PwJeAq4EtgH7my/bD3wonxYOxszeDWwGvgC0RudjObd3Ab/h7g8DuPtP3P1f\niOT8mlYA7zSzFcA7ge8R8Pm5+98CP+g4vND5bAcOuPvb7v4q8G2SGJSJmAP5lcDrbY/faB6Lgpld\nA2wA/g5Y6+6nm0+dBtbm1KxBfQ74JPDTtmOxnNt7gH82s0fM7Dkz+wszW0kk5+fu3wX2Ad8hCeBv\nunudSM6vzULn8wskMaYl03gTcyCPtq7SzC4F/hrY5e4/bH/Ok3rS4M7dzLYA33f3Y1zojc8T6rk1\nrQBuBB5y9xuBf6UjzRDy+ZnZapLe6jUkQe1SM/vt9teEfH7dLON8MjvXmAP5d4Gr2h5fxfx/MYNk\nZu8gCeJfdPfHmodPm9nlzeevAL6fV/sG8AFgm5n9E3AAuNXMvkgc5wbJ394b7v7N5uOvkAT2U5Gc\n3ybgn9z9rLv/BPjfwH8invNrWejvsTPevLt5LBMxB/K/B95rZteY2QjJQMQTObdpIGZmwF8CL7r7\nA21PPQF8tHn/o8Bjne8tOnf/Y3e/yt3fQzJI9nV3/x0iODdIxjeA183s2uahTcAJYJoIzg94DbjZ\nzCrNv9NNJIPWsZxfy0J/j08AHzGzETN7D/Be4JnMWuXu0d6ADwL/SDLwsDvv9qRwPr9Okj/+FnCs\nebsNGAUOAy8Ds8CqvNs64HneAjzRvB/NuQG/BHwTeJ6kx/quyM5vimQA/gWSgcB3hHx+JN8Mvwec\nJxlvu2Ox8wH+uBlrTgLjWbZVU/RFRAIXc2pFRKQUFMhFRAKnQC4iEjgFchGRwCmQi4gEToFcRCRw\nCuQiIoFTIBcRCdz/B041o44YaVXRAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb5112c4710>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"kmeans_run(2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# ここまでのまとめ"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 機械学習のワークフロー\n",
"\n",
"* データをモデルに投入できる形式へ加工する\n",
"* モデルを定義する\n",
"* モデルを評価する基準を定義する\n",
"* モデルにデータを投入して学習する\n",
"* モデルの学習結果を考察する\n",
"\n",
"データの背後にある原理(関数関係)を, 再現可能な手順で見つけるのが, データサイエンスである."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 学習結果の精度を上げる\n",
"\n",
"* 定義したモデル(関数関係)を見直す\n",
"* 学習のアルゴリズムを変える\n",
"* ハイパーパラメーターの最適値を探す"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### 定義したモデル(関数関係)を見直す\n",
"\n",
"たとえば...\n",
"\n",
"* 特徴変数を増やす\n",
"* 誤差をプロットしてみる\n",
"\n",
"ドメイン知識を活用して, モデルを組み立てるのに必要なデータを集めることが大事."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### 学習のアルゴリズムを変える\n",
"\n",
"たとえば...\n",
"\n",
"* 多項式回帰のかわりに, リッジ回帰を使ってみる\n",
"* 確率的勾配降下法のかわりに, Adamを使ってみる"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### ハイパーパラメーターの最適値を探す\n",
"\n",
"たとえば...\n",
"\n",
"* 学習ループの繰り返し回数\n",
"* 多項式回帰における次数\n",
"* クラスタリングにおけるクラスター数"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 教師あり学習と教師なし学習\n",
"\n",
"**教師あり学習** は, 学習するときに正解データ(ラベル)と学習結果との差異(誤差)を算出して, モデルを更新する学習方法.\n",
"例題の「データの予測」, 「データの分類」が該当する.\n",
"\n",
"**教師なし学習** は, 学習するときに正解データ(ラベル)がなく, 学習結果をモデルを更新する学習方法.\n",
"例題の「データのクラスタリング」が該当する."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## さまざまなアルゴリズム, 手法\n",
"\n",
"たとえば...\n",
"\n",
"---\n",
"\n",
"* 線形回帰\n",
"* リッジ回帰\n",
"* 決定木\n",
"\n",
"---\n",
"\n",
"* ロジスティック回帰\n",
"* サポートベクターマシン"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 初学者の疑問, つまずきポイント "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## アルゴリズム, 手法の選びかた\n",
"\n",
"Q. どのようにアルゴリズムや手法を選べばよいか.\n",
"\n",
"A. 解きたい問題にあわせて選ぶ.\n",
"どのようなアルゴリズム, 手法があるのかを知りたいときは, [scikit-learnのチートシート](http://scikit-learn.org/stable/tutorial/machine_learning_map/)が参考になる.\n",
"\n",
"<img src=\"http://scikit-learn.org/stable/_static/ml_map.png\" width=\"30%\" />\n",
"\n",
"<span></span> \n",
"\n",
"<div style=\"font-size: small;\">\n",
"[scikit-learn から学ぶ機械学習の手法の概要](http://qiita.com/ynakayama/items/9c5867b6947aa41e9229)より\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 機械学習に必要なデータ件数\n",
"\n",
"Q. どのくらいのデータがあればよいのか.\n",
"\n",
"A. 大量にあるほうがよい.\n",
"実際にデータを集めてみて, 機械学習をやってみないと, 満足のいく精度が出るのか分からない.\n",
"データの収集期間や網羅性, 多くの種類のデータを集めるのがよい.\n",
"\n",
"<span></span> \n",
"\n",
"<div style=\"font-size: small;\">\n",
"データサイエンティスト養成読本 機械学習入門編 第1部 特集1 第4章より\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 機械学習ライブラリー\n",
"\n",
"* scikit-learn\n",
"* Chainer\n",
"* Spark"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
" # scikit-learn"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 多項式回帰によるデータの予測"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from numpy.random import normal\n",
"from sklearn.linear_model import LinearRegression\n",
"from sklearn.preprocessing import PolynomialFeatures\n",
"from sklearn.pipeline import make_pipeline"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"def sklearn_simulation(degree=3, trainsize=10, testsize=20):\n",
" # データ準備\n",
" x_train = np.linspace(0, 1, trainsize).reshape(trainsize, 1)\n",
" y_train = list(map(lambda x: np.sin(2*np.pi*x) + normal(scale=0.3), x_train))\n",
"\n",
" # モデル定義\n",
" model = make_pipeline(\n",
" PolynomialFeatures(degree),\n",
" LinearRegression()\n",
" )\n",
"\n",
" # 学習\n",
" model.fit(x_train, y_train)\n",
"\n",
" # モデルの評価\n",
" x_test = np.linspace(0, 1, testsize).reshape(testsize, 1)\n",
" y_test = model.predict(x_test)\n",
" \n",
" return x_train, y_train, model, x_test, y_test"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEPCAYAAACjjWTcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYVNX+P/D3DDAMjMNlRpCALDT4SuORi1idg+UlKDU7\nxzRJLW8Ilpa30oOgfsNDSmreSDEviGl1FOV4yd8jRiReUFMUKkcT8FZeEJ0RQUS5zPr94WG+IqBz\n3zOzP6/n6XnYM2v2+nwY+8yw9lprCxhjDIQQQuyakOsACCGEmB8Ve0II4QEq9oQQwgNU7AkhhAeo\n2BNCCA9QsSeEEB6gYk+MdvHiRQiFQhw+fLjV49aMGTMG0dHRBvVnzGu5tGHDBjg5OXEdBuEpKvbE\naB07dkR5eTleeOEFnV/z5ZdfYtu2bdrjuLg49OnTR6fXCgQCCAQCveMk5tHY2IikpCSEh4fDzc0N\nXl5e6NevH44dO8Z1aOQhVOyJ0YRCIby9veHo6Kjza6RSKdzd3Q3qjzEGS6wFrKurM3sf5qDRaKDR\naCzW371793D06FFMnz4dP//8M/Lz8+Hj44OoqCicP3/eYnGQx6NiTwAAhw4dQmRkJNzc3ODm5obQ\n0FD88MMP2ucrKiowduxY+Pj4wMXFBV26dEFmZiYA3YZtPv/8c8jlchw6dAhA86GY5ORkrF+/Hvv3\n74dQKIRQKMTGjRv1in/z5s0IDQ2Fi4sLAgIC8Mknn+Du3bva53Nzc9G7d2/I5XJ4eHigd+/eOH78\neLNzCIVCfPnllxgxYgQ8PDwwatQo7dDL4cOHER4eDolEgoiICBQWFj42HsYY5syZA29vb0ilUgwb\nNgy3bt1q0S43NxeRkZFwdXWFv78/YmNjoVarm50nKSkJXl5ecHNzw3vvvYfly5c3Gw5KTk5GYGAg\nsrKy0KVLFzg7O6O0tBR37tzBlClT4O/vD4lEgvDwcGzfvr1Z/9evX8eYMWPg7e0NNzc39OzZEwcP\nHtTrdy+RSPDTTz9hxIgRCA4OhkKhwPr16+Ho6Ig9e/bodS5iRozwXn19PfP09GSffPIJKysrY2Vl\nZWzHjh3s4MGDjDHG7t69y7p06cK6d+/O8vLy2MWLF1leXh7bsmULY4yxCxcuMIFAwAoKClocNzY2\nso8++oj5+/uzU6dOafscPXo0i46OZowxdufOHfbuu++yyMhIdv36dXb9+nVWW1vbZryjR49mUVFR\n2uPMzEzm6enJvvnmG3bhwgV24MAB1q1bNzZy5Ehtm+3bt7OtW7eykpISdvr0aRYXF8dkMhlTqVTa\nNgKBgMnlcrZy5Up2/vx5VlpayjIzM5lQKGS9evVihw4dYr///jvr378/CwgIYA0NDW3GuGzZMiaR\nSNjGjRtZaWkpW7hwIXN3d2dOTk7aNnl5eczV1ZWtWLGClZWVsePHj7M+ffqwXr16adssXryYtWvX\njn3zzTesrKyMLVmyhMlksmbn+fTTT5mrqyvr3bs3O3bsGCstLWXV1dWsd+/erE+fPqygoIBduHCB\nrVmzholEIpaXl6d9X4ODg9nbb7/NTpw4wc6dO8fmzZvHnJ2d2ZkzZ7Tnf+aZZ9iYMWPazLU11dXV\nTCKRsMzMTL1eR8yHij1harWaCQQClp+f3+rz69atY2KxmF25cqXV59sq9nl5eeztt99mzz//PPvz\nzz+bvebRgj1u3DjWu3dvneJ99LXPPPMMW716dbM2+/fvZwKBgFVWVrZ6jsbGRubp6cm+/fZb7WMC\ngYDFxcU1a5eZmckEAgErKirSPvbzzz8zgUDASkpK2ozRz8+PzZ49u9ljb7/9drMi3atXL5aYmNis\nzaVLl5hAIGC//PILY4wxX19f9r//+7/N2gwbNow5Ojpqjz/99FMmFAqb/Y737dvHxGIxu337drPX\njh07lg0aNEibm7+/f4sPrb59+7KpU6dqj1999VWWlJTUZq6tGTduHAsICGA1NTV6vY6YDw3jEHh6\neiIuLg6vv/46BgwYgAULFqCkpET7/IkTJ6BQKODr66vXeceOHYtTp07h0KFD8Pf31zuu+fPnQyqV\nav8rKCho0ebGjRv4448/MG3atGZtBwwYAIFAgLKyMgDAhQsXMHLkSAQGBsLd3R3u7u64ffs2/vjj\nj2bna+0is0AgQEhIiPb4qaeeAvBgCKQ1VVVVuHr1Kv72t781ezwyMrLZtYbjx49j6dKlzeJWKBQQ\nCAQoLS3F7du3ce3aNbz00kvNzvPoMQB06NCh2e/4+PHjqKurg5+fX7Pzf/vtt9rfyfHjx1FeXg4P\nD49mbQ4ePKhtAwA//vgj5s2b12qurZk5cyZ27dqFXbt2wdXVVefXEfPS/YoasWtr1qzBlClT8MMP\nPyA3Nxdz5szBihUrMH78eAAw6ILowIEDkZGRgT179mDEiBF6v37ChAkYNmyY9ri1D5umC5FpaWmt\nzubx8/PTxuLt7Y309HQ8/fTTcHJyQs+ePVtchJVIJC3OIRQKm83+afrZ2IugjDHMnDkTI0eObPFc\nhw4d0NDQ0Ky/x3k0bo1GA3d391avLYhEIm2b4OBg7Nixo0UbQ4o0YwxTpkzBli1bkJeXh65du+p9\nDmI+VOyJlkKhgEKhwLRp0zBhwgSsWbMG48ePR/fu3ZGZmYkrV65oi6cu3n33XbzyyisYPXo0Ghoa\nMGrUqDbbikQiNDY2NnvM09MTnp6ej+2jQ4cOePrpp/H7779j3LhxrbZRqVQ4c+YMlixZor0ofPny\nZVRUVOiciz7c3Nzg5+eHgoIC9O/fX/t4QUFBs8IdERGBU6dOoVOnTm2ey9fXF4cPH0a/fv20jx09\nevSJMfTo0QOVlZWora2FQqFos82mTZsglUrh5eWlS2ptamxsRFxcHH744Qfk5+cjODjYqPMR06Nh\nHIJz584hISEBBQUFuHTpEo4cOYKDBw9qi8SIESPwzDPP4O9//zvy8vJw4cIF5OXlISsr64nnfued\nd/Ddd9/h/fffR0ZGRpvtOnXqhN9//x2nT5/GzZs39Zr2OG/ePKSlpWH+/Pk4deoUzp49ix07duCD\nDz4A8OBDw8vLC2vWrEFpaSmOHDmC4cOHw8XFRec+9PXJJ59g+fLl+Oabb1BaWorFixcjLy+v2V9I\n//rXv7Bz50588sknKC4uxrlz55CTk4O4uDjcu3dPe55ly5bhu+++Q2lpKZYtW4bc3FwIhY//X7dv\n376IiorC4MGDsXPnTpw/fx4nTpzAl19+iXXr1gF48GEcEBCAN954A7m5ubh48SJ+/vlnpKamYufO\nndpzvfrqq0hKSmqzr8bGRgwbNgzff/89tmzZAk9PT5SXl6O8vBw1NTXG/BqJKXF6xYBYhWvXrrHB\ngwczf39/5uzszHx9fdn48eNZVVWVtk15eTkbNWoUa9++PROLxSw4OJh9/fXXjLEHF2SFQmGzC7QP\nHzPG2K5du5iLiwtbtWoVY4yxMWPGaGfjMPbgIvGAAQOYu7s7EwgE2nO35tHXMsbYjh072F//+lfm\n6urK3NzcWGhoKEtJSdE+v3//fhYSEsLEYjHr0qULy87OZs899xybO3euto1AIGh2wZaxBxcxH76o\nyhhjf/75JxMKhWz//v1txqjRaFhSUhJr3749k0gkbOjQoWzp0qUtznXw4EEWFRXFpFIpk0gkLDg4\nmE2bNk170VSj0bDExETWvn171q5dOzZ8+HA2f/58JpVKtedITk5mgYGBLWKora1lM2fOZAEBAUwk\nEjEfHx/Wv39/tm/fPm0blUrFJkyYwPz8/JhIJGJ+fn5s8ODBrLi4WNvm2WefZWPHjm0z16YL8kKh\nkAkEgmb/Pfz7JdwSMGb46pSbN29i5cqVuH37NgQCAV599VUMGDCgRbv169ejuLgYzs7OmDhxIgIC\nAoz6gCKEz2JjY/Hbb7+1WCdAyOMYNYzj6OiI0aNHY8mSJZg3bx727t2Ly5cvN2tz8uRJXL9+HWlp\naRg/frz2T0hro1QquQ7BbOw5N8C+87t27Rpmz56N06dP4+zZs/jiiy+wadMmxMfHcx2aydjz+wdY\nT35GFXsPDw88++yzAACxWAw/P78WqwQLCwvRq1cvAEBgYCBqampQWVlpTLdmYS1viDnYc26Afefn\n4OCA7du34+WXX0Z4eDi++eYbfPXVV9pZUvbAnt8/wHryM9lsnIqKCly8eBGBgYHNHler1ZDL5dpj\nuVwOtVoNDw8PU3VNiN3y9vbGp59+ipiYGK5DITbOJLNx7t27hyVLlmDMmDEQi8UtnjfisgAhhBAT\nMOoCLQA0NDRgwYIFCA0NxRtvvNHi+TVr1kChUCAyMhIAMHXqVCQnJ7f4Zq9UKpv9uUPfZAghxDAP\nT4tuWj9j1DAOYwxfffUV/Pz8Wi30wIOFI3v37kVkZCRKSkogkUhaHcJpCuhhV69eNSY8vUilUlRX\nV1usP0uy59wAys/WUX6m5evr2+qXZaOK/dmzZ3Hw4EF07NgR//znPwEAw4cPx82bNwEA0dHRCA8P\nR1FRESZNmgSxWIwJEyYY0yUhhBADGD2MY070zd407Dk3gPKzdZSfabW1YSFtl0AIITxAxZ4QQniA\nij0hhPAAFXtCCOEBKvaEEMIDVOwJIYQHqNgTQggPULEnhBAeoGJPCCE8QMWeEEJ4gIo9IYTwABV7\nQgjhASr2hBDCA1TsCSGEB6jYE0IID1CxJ4QQHqBiTwghPEDFnhBCeICKPSGE8IBRNxwHgPT0dBQV\nFcHNzQ2LFy9u8bxSqcTChQvRoUMHAMCLL76IIUOGGNstIYQQPRhd7Pv06YP+/ftjxYoVbbZ5/vnn\nkZCQYGxXhBBCDGT0ME5wcDAkEslj2zDGjO2GEEKIEYz+Zv8kAoEAJSUlmDFjBmQyGUaOHAl/f39z\nd0sIIeQhZi/2AQEBWLVqFZydnVFUVIRFixZh+fLl5u6WEELIQ8xe7F1cXLQ/h4WFYd26dbhz5w7a\ntWvXrJ1SqYRSqdQex8TEQCqVmjs8LZFIZNH+LMmecwMoP1tH+ZleVlaW9meFQgGFQmH+Yl9ZWQl3\nd3cIBAKUlZUBQItC/3BAD6uurjZ3eFpSqdSi/VmSPecGUH62jvIzfX8xMTEtHje62C9btgxnzpxB\nVVUVJkyYgKFDh6KxsREAEB0djaNHjyI3NxdCoRDOzs6YMmWKsV0SQgjRk4BZ8VSZq1evWqwve/52\nYc+5AZSfraP8TMvX17fVx2kFLSGE8AAVe0II4QEq9oQQwgNU7AkhhAeo2BNCCA9QsSeEEB6gYk8I\nITxAxZ4QQnjA7NslkLap1UKsXftge+j4+BrIZBqOIyKE2Csq9hxRq4UYMkSOkhInAEBOjhjZ2Soq\n+IQQs6BhHI6sXSvRFnoAKClx0n7LJ4QQU6NiTwghPMD7Yi9UqyFdsACilBQI1WqL9RsfX4OgoHrt\ncVBQPeLjayzWPyGEX3g9Zi9UqyEfMgROJSUAAPnOnVBlZ0Mjk5m9b5lMg+xsFV2gJYRYBK+LvWTt\nWm2hBwCnkhJI1q5FdUKCRfqXyTRISLDfrV0JIdaD98M4hBDCB7wu9jXx8agPCtIe1wcFoSY+nsOI\nCCHEPHg9jKORyaDKzoZk7VqInJ1xa9Qoi4zXE0KIpfG62AMPCn51QgKkUik0dnxrNEIIvxld7NPT\n01FUVAQ3NzcsXry41Tbr169HcXExnJ2dMXHiRAQEBBjbLSGEED0YPWbfp08fJCUltfn8yZMncf36\ndaSlpWH8+PFYt26dsV0SQgjRk9HFPjg4GBJJ28v8CwsL0atXLwBAYGAgampqUFlZaWy3hBBC9GD2\n2ThqtRpyuVx7LJfLobbgSlVCCCEWukDLGHtiG6VSCaVSqT2OiYmBVCo1Z1jNiEQivfoTqFRwSk8H\nANRPnAj20AeatWkrt7rGOlTdr0J71/YcRGU6+r53tobys21c5JeVlaX9WaFQQKFQmL/Yy2QyqFQq\n7bFKpYKslemNTQE9rNqCs2OkUqnO/T26zYLQgtssGKK13NRqIeavP49saX94Sz0R4ROGUK9QhHmH\noau8K8SOYo6i1Z8+750tovxsm6Xzk0qliImJafG42YdxIiIicODAAQBASUkJJBIJPDw8zN2tWbW1\nzYKtaNpL/99L/4a6lFtw2LIT3T1649ztc5hVMAsfH/iY6xAJISZm9Df7ZcuW4cyZM6iqqsKECRMw\ndOhQNDY2AgCio6MRHh6OoqIiTJo0CWKxGBMmTDA6aKK/usY6iBxEAB7ZS58JcakwBDdyO+HzhKEP\nHmpj2C3vjzycqDiBMK8whHuHQ+5ivUNXhJDmjC72U6dOfWKbcePGGduNVamJj4c4J0f77d6at1mo\nqqtCcl4yLt66iA2vb9DpNQKBoNXHO7h2AACsV67HlPwpcBO5Icw7DLGKWPTw6WGqkAkhZsD7FbSG\neHibBeBB8be28XrGGHae24mUn1PQv3N/LO21VPtcfHwNcnLE2m/3uu6l37V9V3Rt3xUAoGEanL99\nHkUVRXB1cm21vfqeGh7OHhAKeL0FEyFWQcB0mSrDkatXr1qsL3u6SHT+9nkkFSThZu1NfN7zc/R5\nrk+rF2jNvZd+/I/xOHTlEEK8QhDqFYpw73CEeYXBy9XLpP3Y03vXGsrPtlk6P19f31Yfp2L/X/b0\nD27L2S2ovF+JcV3HwVHoyGluqloVim4UoaiiCMU3ilF8oxi7/7EbAe6m2zLDnt671lB+ts1aij0N\n49ihd/7nHa5D0JK7yBHVMQpRHaMA6LbmghBielTsiUW1dfGXEGJedOWMA003OZcuWGDwTc4bNY3I\nVGZia8lWE0dneQ2aBpRVlnEdBiF2jYq9hTWtvpWmpUGalgb5kCF6F/xfbvyCgTsHYvf53QjxCjFT\npJajVCkx+PvBOHrtKNehEGK3qNhbmDGrb6vqqjC7YDZG7x2NsYqx2DZwG4I8g578QisX4hWClX1X\nIv7HeOT9kcd1OITYJSr2NmTyvsmo09Rh39v7EBMUY1fj3y/7vYzM1zLx8YGPsevcLq7DIcTu0AVa\nCzNm9e3qqNVwdnA2Z3iciugQge/6f4eROSMhdhTjtWde4zokQuwGFXsLM2b1rT0X+iYKuQLZA7PR\n3sW2t10mxNpQsedA003O23LgygF0lXeFTGxdWzBYiikXXBFCHqAxeytScbcCH/70If554J+4cucK\n1+EQQuwIFXsr0KhpxAblBrya/Sr82/lj39B9+Ev7v3AdllWpa6xDo6aR6zAIsVk0jMOxusY6DN49\nGCKhCNve2Ib/kf0P1yFZpZW/rERpZSmW914OJ6ET1+EQYnPomz3HRA4iJL+UjOyB2ToVelOsvrVF\nH3T7AHfq7iAuNw61DbVch0OIzaFibwUiOkToNGfeFKtvbZWLowsyXsuAxEmCUTmjcKfuDtchEWJT\nqNjbEFu/962xnIRO+LL3lwhwD8CwPcNQeb+S65AIsRk0Zk8szpgbpzgIHbCg5wJsPLORF+sOCDEV\no4t9cXExNmzYAI1Gg759+2LQoEHNnlcqlVi4cCE6dHhw/9IXX3wRQ4YMMbZbm8QYw6zDs/Bel/fw\nvPx5vV9vS/e+bYtaLcSQIXLtLRFzcsTIzlbpVfAFAgFGPz/aXCESYpeMKvYajQYZGRmYM2cOZDIZ\nEhMTERERAX9//2btnn/+eSQ8ZhERXyw6sQinbp4yeNGQLdz79knWrpVoCz0AlJQ4Ye1aCRIS7PdO\nRYRYA6OKfVlZGXx8fODt7Q0AiIyMRGFhYYtiT3cnAjaf3YwdZTvw/T++h4uji8HnedLqW0IIaY1R\nF2jVajXkcrn2WCaTQf3I7BCBQICSkhLMmDEDqampuHz5sjFd2qQDVw4g9XgqNvbbCLmL/MkvsGPx\n8TUICqrXHgcF1SM+vsbo8zLGMDV/KgqvFxp9LkLskdkv0AYEBGDVqlVwdnZGUVERFi1ahOXLl7do\np1QqoVQqtccxMTGQSqXmDk9LJBKZpb/qump8vP9jbHpzE8L8w0x+fl2IRCK41dXBKT0dAFA/cSKY\nnJsPHakU2Lv3PtLTH4zRT5xYD7lcYtQ5RSIR3Nzc8E7XdzAuZxzWv7EevTv2NkG01sFc/zatBeVn\nellZWdqfFQoFFAoFBMyIMZaSkhJs3boVs2bNAgBs374dAoGgxUXah3344YdYsGAB2rVr98TzX716\n1dDQ9GbOO8DfrL3J6S6ObnV1cH799WYXdlXZ2TY33t+Wh9+7I9eO4P0f38cXr3xhN1skm/PfpjWg\n/EzL19e31ceNGsbp3LkzysvLUVFRgYaGBhw+fBgRERHN2lRWVmrH7MvKHtxnVJdCb0+43q7XKT2d\nN/Pz//rUX7Gx30b88+A/sb1sO9fhEGI1jBrGcXBwQGxsLObNm6edeunv74/c3FwAQHR0NI4ePYrc\n3FwIhUI4OztjypQpJgmckLaEeoVi84DNWPPbGgzqPMiu7uhFiKGMGsYxN3sZxuEan4Zx7BHlZ9us\nZRiHVtCa2MbTG9HZozMifSO5DkWLyeU2Pz+fEFvTtFLc2VmEUaOEei0cNAcq9ib0w6UfsLxoOXb+\nfSfXobRA8/MJsZxHV4rv3CnXe6W4qfF+IzS1WogFC6RISRFBrTb81/HLjV8w/cB0ZLyWAX+p/5Nf\nQCyutqEWq35ZhQZNA9ehEDvX1kpxLvH6m72pPn0vV19G7A+xWPTyIoR6hZojVE4J1Wq7GAJijOHA\nlQM4eeMk0vum001QCK/w+pu9KT59NUyDuB/j8EG3D/D6s6+bOkTO2dMe+q5Ortjw+gbcrb+Lfx39\nF9fhEDtmrpXixuB1sTcFoUCIVX1XIa5rHNehmIW97aHv7OCM9L7pyL+cj81nN3MdDrFTMpkG2dkq\nTJ5cjRkz7nM+Xg/wfBgnPr4GOTli7bd7Qz99Dd3FknDD3dkdma9lYvD3g9G1fVd0lXflOiRih2Qy\nDRISqv879ZLbQg/wvNg3ffr+3/SoW5x/+lobe9hDvzXPeTyHzQM2I9AzkOtQCLEIWlT1X/a8sMPY\n3Kz9Aq09v3cA5WfraFGVjfr52s9Q3VNhQMAArkOxGJqjT4jtowu0ejhXeQ7j88ZD4sTtfFlim5rW\ndCxYIDVqTQchhqBv9jpS1aowau8oJEQkoJd/L67DsRnWPgT0MPU9NS7cvoDuHbqb/twmuPcuIcag\nrxc6uNdwD7G5sRjYaSBGdBnBdTg2w9bm6J+rPIfY3FhcuH3B5Oe2xhWVhF+o2OsgsSARvhJfJETQ\nuLU+bG2Ofg+fHvg4/GOM/WEsquvs94Ih4Scq9joY/5fxWNprKYQC+nXZu1HBo/CCzwuYnD8ZGma6\nIRZrXFFJ+IWqlw6CZcEQO4q5DsPm1MTHoz4oSHtsC3P0BQIBPvvbZ7h17xaWnFxisvM+vKJy8uRq\nGq8nFkcXaInZaGQym9xHX+Qgwtqotfj32X+DMWayO101ragkhAtU7IlZ2eocfS9XL0wOm8x1GISY\nDA3jPKK8ppxuVG0FhGo1pAsWQLpggVXP4CHEVhj9zb64uBgbNmzQ3nB80KBBLdqsX78excXFcHZ2\nxsSJExEQYJ0bh9XU12D03tEY2Gkg16HwWtOUzaaZPOKcHLu6Zy4hXDDqm71Go0FGRgaSkpKwZMkS\nFBQU4PLly83anDx5EtevX0daWhrGjx+PdevWGRWwuTRoGjAhbwL+0v4v+CjkI67D4TVrnbJ5p+4O\n1yEQYjCjin1ZWRl8fHzg7e0NR0dHREZGorCwsFmbwsJC9Or1YMVpYGAgampqUFlZaUy3JscYQ8K+\nBNRr6pHaM9VkF+SIfYnNjcWWs1u4DoMQgxhV7NVqNeRyufZYJpNB/cj46qNt5HJ5izZcy1RmouBy\nAVZHraZb1VkBa52yOT9yPuYdm4cT109wHQqxEdZ0v2OLzMbRZRdlpVIJpVKpPY6JiYFUKjVnWFqv\nBb6Gd0LfgY+Lj0X6szSRSGSx36VJSKW4v3cvNOnpAID6iRMheegLw6MslV+YNAzp/dLxwY8fYN+I\nfXiq3VNm7xOwwfdPT/aa3/el32Pdr+uwZ/gei+eXlZWl/VmhUEChUBhX7GUyGVQqlfZYpVJB9shF\nNF3aPBzQwyy1B7S/sz+kLva7p7ZN7hcuEgFTp/7f8WPifzQ/c26+1tOrJ0YFj8Kw7cOwbeA2iyy2\ns8n3Tw/2mF9RRREm507Gt/2/RV1dnUXzk0qliImJafG4UcM4nTt3Rnl5OSoqKtDQ0IDDhw8jIiKi\nWZuIiAgcOHAAAFBSUgKJRAIPDw9juiWkTZbYfO2jkI/Q0a0j8i/nm/S8xD78UfUHxuWOw+JXFuMv\n7f/CdThaRn2zd3BwQGxsLObNm6edeunv74/c3FwAQHR0NMLDw1FUVIRJkyZBLBZjwoQJJgmckNa0\nNZOnaWGXWi3U7jYZH19j0JYFAoEAK/qsoL2SSAu379/GqL2jMCl0EqKfieY6nGaMHrMPCwtDWFhY\ns8eio5snOW7cOGO7IcRoptxTngo9aY3IQYQPQz7E0KChXIfSAv2LJXblcTN5aE95Ym4uji5WWegB\n2huH2BljN1+zpTtrEaIPKvbE7rS1+Vp8fA1ycsTab/eP7ilvzDYNZZVlWPPbGnze83Ma4iFWif5V\nEt540p7yxmzT8LT0aZy9dRZLTy41edzEev128zfU1NvGTWio2BNeadpTPiGh2qQ3D3F2cMa6qHXY\nfHYz9lzYY7LzEutVeqsU7+W8h9Oq01yHohMq9oT8l7HbNHi5emFd9DokHErAGfUZc4RIrMTN2psY\ntXcUkl5IQg+fHlyHoxMasyfkv4y5uNt0YbcngLmvT0d8bjx+evsniBxEZoyYcKG2oRZjfhiDt557\nC+8EvcN1ODqjYk/IQwy5s9ajF3bH5wQhZMNyKvR2SMM0mJw/Gc9Kn8WM7jO4DkcvNIxDiJFau7Ab\nsvlHDiMi5tLIGhHmFYbFvRbb3Fbo9M2eEI7R3H7b4SR0wsSQiVyHYRAq9oQYqSY+HuKcHO23e30u\n7NItGIml0DAOIUZqurBbPXkyqidPbrVYH7pyCOU15S1ea623YCT2h77ZE2ICT7qwe6LiBD4v/Bzb\n3rDMHviMMBjKAAASFElEQVTENMpryiF2FMPD2fa3Zadv9oRYwOTQyfCT+GHmoZnN7txm7Nx+oVoN\n6YIFkC5YYPJ9+/muuq4a7+W8h93nd3MdikkImC73DOTI1atXLdaXPd4tp4k95wbYTn536+9iyO4h\niOgQgbl/navdQ+dJF2jbyu/R8f76oCCbHO+3xvevQdOAMXvHwK+dHz7v+blRM28snZ+vr2+rj9M3\ne0IsxNXJFZsHbMZvN3/DrIJZ2sebhoCqExL0KtTGjvfTXwWtY4whqSAJADAvcp7NTbFsC43ZE2JB\n7s7u+K7/d/j91u+cxkGzgNq26tdVOFlxEtvf3A5HoeElsukvNpGzM4SjRnH+u6Vv9oRYmKuTK8K9\nw40+jzHj/cb8VWDPfxEwxlDbUIuNr2+EVCQ1ONeH74XsvGiRWe6FrC/6Zk+IjTL2Ri2GMPYvAi4W\nkOnTp0AgwCfdP9G+ztBcn3QvZC4YXOzv3LmDpUuX4ubNm/Dy8sK0adMgkbS8xduHH34IFxcXCIVC\nODg4IDU11aiACbFXDZoGvYcNDNnLBzB8IZgxRayt4gmpVKeYDfmgsLeCbQyDh3F27NiBbt26Yfny\n5ejatSt27NjRZtvk5GQsXLiQCj0hbSi5VYJ+2/u1uvDKHHRZCGZqxg4dNQ2LSNPSdB4W4WrRmrFT\nas3B4GJfWFiIXr16AQB69+6N48ePt9nWimd3EmIVgjyDMKjzIAz+fjAuVV2ySJ+GzALiqoiZq2jf\nb7wPVa2q1eeMyfXhD9P7M2ZYxcVvg4dxbt++DQ+PB6vK3N3dcfv27VbbCQQCpKSkQCgUIioqClFR\nUYZ2SYhd+yj0I7QTtcOQ3UPwXf/vEOQZ9OQXWZgx1wnaGjpqOfhrOo8brmKMYfqB6XAXueOzyM9a\nvNbYayJNH6ZSqRQaK1hH8NhFVSkpKaisrGzx+PDhw7Fy5UpkZmZqHxs7dmyz4ya3bt2Cp6cnqqqq\nkJKSgtjYWAQHB7dop1QqoVQqtccxMTEWXYggEolQV1dnsf4syZ5zA+wvv82nN2POwTnYOmgrQjuE\n2lV+ApUKTunpAID6iRPB5HKd8hOoVHDp3x8Ovz+YstrYpQtq9+wBk8sN6hMAPiv4DD9d+gm7h+6G\nq5OrMWk9lqXfP6lUiqysLO2xQqGAQqEwfAXt1KlTkZycDA8PD9y6dQtz587FsmXLHvuarVu3QiwW\n480339SpD1pBaxr2nBtgn/nlXMyBTCzDCz4v2GV+D9M1P1PO5NlSsgXLTi7D9//4Hu1d2ht8Hl1Y\nywpag4dxIiIikJ+fj0GDBmH//v3o0aPlfRjv378PjUYDFxcX3Lt3D7/++ivefvttQ7skhDf6PduP\n6xCsjqEzjx518MpBzD82H9ve2Gb2Qm9NDC72gwYNwtKlS7Fv3z7t1EsAUKvVWL16NRITE1FZWYkv\nvvgCAKDRaNCzZ0+EhISYJnJCCDHALzd+waq+qxDoGch1KBZFG6H9lz3/qWzPuQGUn62j/EyLNkIj\nxMb9vwv/D+tPrec6DGKjqNgTYiO6te+GDGUG0orSaO0K0RsVe0JsxNPSp/GfN/+DHed2YP6x+VTw\ndVCvqcef1X9yHYZVoGJPiA3p4NoB2wZuQ8HVAiQWJELDNFyHZLWOlx9Hv//0Q/ov6VyHYhWo2BNi\nY2RiGba8sQXX71632NYKtkR9T40ZB2bgg7wPMDlsMuZHzuc6JKtAxZ4QGyQVSZH5WiYC3AO4DsWq\n7D6/G3239YXYUYx9Q/fhZc+3sHChGxYskEKt5ne5o/3sCSF2QyqS4uvXv0aIVwjUaiGGDJGjpMQJ\nAJCTI0Z2tgoyGT+Hvvj9UUcIsSu9/HshxOvBws21ayXaQg8AJSVOWLvWnNuuWTcq9oTYkd3nd+PG\n3Rtch2ERNBtJP1TsCbEjJbdKMHj3YFy5c4XrUMzmyp0riMuNw9env35su/j4GgQF1WuPg4LqER9f\nY+7wrBaN2RNiRz7u/jEkThIM/n4w/j3g3+jk3onrkEymXlOPjFMZWFG8ArGKWAz7n2GPbS+TaZCd\nrdIO3cTH1/B2vB6gYk+I3Xm/2/toJ2qHobuH4pv+3yBY1vL+Ebbm+PXjSDyUCC8XL+z6xy6dP8Rk\nMg0SEux33x19ULEnxA692+VdtHNqh4l5E5E7JFfvG5lbm02nN2FS6CT8vdPfIRAIuA7HJtn2vwBC\nSJv+0fkfiO4YbfOFHgDS+qRxHYLNowu0hNgxc95uj9gWKvaE8ExZZRmySrJQeqvUqvbWqW2oxcLC\nhXY9k4hLtv/3HSFELzX1Nci/nI9lJ5fh1v1bCPEKQahXKPo92w+hXqGcxPTjHz9idsFshHcIh0go\n4iQGe0fFnhCeCfEKQXrfBztBqmpVKLpRhOIbxfiz+s9Wiz1jzGwXRa/euYqUfSn4teJXLHx5IV7x\nf8Us/RAq9oTwmtxFjqiOUYjqGNVmm5mHZqLoRhHCvMIQ7h2OUK9QBHoGQigwbhT4Tt0dvLnzTYzp\nNgZLX14KsaPYqPORxzP4HrRHjhzB1q1bceXKFaSmpqJTp9bnvRYXF2PDhg3QaDTo27cvBg0apHMf\ndA9a07Dn3ADKz9zuNdzDafVpFFUUPfjvRhFUtSps6rcJPXx6GHXu2/dvw7+9P71/JtTWPWgN/mbf\nsWNHTJ8+HWvXrm2zjUajQUZGBubMmQOZTIbExERERETA39/f0G4JIRYmdhQj3Dsc4d7h2sfU99Rw\ncXRptX12aTb82vmhW/tuT5wN5O7sbtJYSdsMLvZ+fn5PbFNWVgYfHx94e3sDACIjI1FYWEjFnhAb\nJxPL2nzujPoMMpWZOKM+gwD3AO0HxZO2NyDmZdYxe7VaDblcrj2WyWQoKyszZ5eEEI7NfnE2AOB+\n432cVj0Y/im+UYyBAQPRTtSO4+j467HFPiUlBZWVlS0eHz58OCIiIkwaiFKphFKp1B7HxMRAKpWa\ntI/HEYlEFu3Pkuw5N4Dys1ZSSPGKxyt4pfPjZ9jYan664iK/rKws7c8KhQIKheLxxX7OnDlGdSiT\nyaBSqbTHKpUKMlnrf/41BfQwS17U4PoimDnZc24A5WfrKD/T9xcTE9PicbOuoO3cuTPKy8tRUVGB\nhoYGHD582OR/ERBCCHkyg8fsjx07hszMTFRVVSE1NRUBAQFISkqCWq3G6tWrkZiYCAcHB8TGxmLe\nvHnaqZd0cZYQflGrhbSnvBUweJ69JdA8e9Ow59wAys+aPXrT76Cg+hY3/bbl/HRhLfPsaSM0QojZ\n0E2/rQcVe0II4QEq9oQQs6GbflsP2giNEGI2dNNv60HFnhBiVnTTb+tAwziEEMIDVOwJIYQHqNgT\nQggPULEnhBAeoGJPCCE8QMWeEEJ4gIo9IYTwAM2zJ4QHaOdJQsWeEDv36M6TOTniFjtPEvtHwziE\n2DnaeZIAVOwJIYQXqNgTYudo50kC0Jg9IXaPdp4kABV7QniBdp4kBhf7I0eOYOvWrbhy5QpSU1PR\nqVOnVtt9+OGHcHFxgVAohIODA1JTUw0OlhBCiGEMLvYdO3bE9OnTsXbt2ie2TU5ORrt27QztihBC\niJEMLvZ+fn46t2WMGdoNIYQQEzD7mL1AIEBKSgqEQiGioqIQFRVl7i4JIYQ84rHFPiUlBZWVlS0e\nHz58OCIiInTqICUlBZ6enqiqqkJKSgr8/PwQHBzcop1SqYRSqdQex8TEQCqV6tSHKYhEIov2Z0n2\nnBtA+dk6ys/0srKytD8rFAooFAoImJFjLHPnzsXIkSPbvED7sK1bt0IsFuPNN9/U6dxXr141JjS9\nSKVSVFfb52wFe84NoPxsHeVnWr6+vq0+btZFVffv30dtbS0A4N69e/j111/RsWNHc3ZJCCGkFQaP\n2R87dgyZmZmoqqpCamoqAgICkJSUBLVajdWrVyMxMRGVlZX44osvAAAajQY9e/ZESEiIyYInhBCi\nG6OHccyJhnFMw55zAyg/W0f5mRYnwziEEEKsAxV7QgjhASr2hBDCA1TsCSGEB6jYE0IID1CxJ4QQ\nHqBiTwghPEDFnhBCeICKPSGE8AAVe0II4QEq9oQQwgNU7AkhhAeo2BNCCA9QsSeEEB6gYk8IITxA\nxZ4QQniAij0hhPAAFXtCCOEBg+9Bu2nTJpw8eRKOjo7o0KEDJk6cCFdX1xbtiouLsWHDBmg0GvTt\n2xeDBg0yKmBCCCH6M/ibfUhICBYvXoxFixbhqaeewvbt21u00Wg0yMjIQFJSEpYsWYKCggJcvnzZ\nqIAJIYToz+Bi361bNwiFD14eGBgIlUrVok1ZWRl8fHzg7e0NR0dHREZGorCw0PBoCSGEGMQkY/Y/\n/fQTwsPDWzyuVqshl8u1xzKZDGq12hRdEkII0cNjx+xTUlJQWVnZ4vHhw4cjIiICAPCf//wHjo6O\n6Nmzp3kiJIQQYrTHFvs5c+Y89sX5+fkoKipqs51MJms2vKNSqSCTyVptq1QqoVQqtccxMTHw9fV9\nbP+mJpVKLdqfJdlzbgDlZ+soP9PKysrS/qxQKKBQKABmoKKiIjZt2jR2+/btNts0NDSwjz76iF2/\nfp3V19ez6dOnsz///NPQLs1qy5YtXIdgNvacG2OUn62j/CzD4KmX69evR0NDAz777DMAQFBQEOLi\n4qBWq7F69WokJibCwcEBsbGxmDdvnnbqpb+/v/EfW4QQQvRicLFPS0tr9XGZTIbExETtcVhYGMLC\nwgzthhBCiAnQCtr/UigUXIdgNvacG0D52TrKzzIEjDHGdRCEEELMi77ZE0IID1CxJ4QQHjD4Aq0t\n0mVTtvXr16O4uBjOzs6YOHEiAgICOIjUME/K7+DBg9i1axcYY3BxcUFcXByeeeYZjqLVn66b6pWV\nlWH27NmYNm0aXnzxRQtHaThd8lMqlfj666/R2NgIqVSK5ORkywdqoCfld/fuXaSlpUGlUkGj0eDN\nN99E7969uQlWT+np6SgqKoKbmxsWL17cahvOawvHUz8tprGx8Ylz/k+cOMHmz5/PGGOspKSEJSUl\ncRGqQXTJ7+zZs6ympoYx9mCdhL3l19QuOTmZpaamsiNHjnAQqWF0ye/OnTts2rRp7ObNm4wx9tg1\nLtZGl/yys7PZt99+yxh7kNvYsWNZQ0MDF+Hq7fTp0+z8+fPs448/bvV5a6gtvBnG0WVTtsLCQvTq\n1QvAg83dampqWt0uwhrpkl9QUJB2G+rnnnuu1c3rrJWum+rt2bMHL730Etzc3DiI0nC65Hfo0CG8\n+OKL2v2mbClHXfITCoW4e/cuAKC2thZSqRQODg5chKu34OBgSCSSNp+3htrCm2Kvy6Zsj7aRy+U2\ns3GbvpvO/fTTTza1/kHX96+wsBCvvfYaAEAgEFg0RmPokt+1a9dw584dzJ07FzNnzsSBAwcsHabB\ndMmvX79+uHLlCt5//33MmDEDY8aMsXCU5mMNtYU3xV5XjAczUU+dOoV9+/bh3Xff5ToUk9qwYQNG\njBgBgUAAxpjdvZeNjY24cOECEhMTMWvWLGRnZ+PatWtch2UyxcXFCAgIwOrVq7Fw4UJkZGSgtraW\n67BMhut/j7y5QKvLpmz6bNxmbXSN/dKlS1i9ejVmzZqFdu3aWTJEo+iS3/nz57Fs2TIAQHV1NYqL\ni+Ho6KjdodWa6ZKfXC6HVCqFSCSCSCRCcHAwLl26hKeeesrS4epNl/zy8/O1F22bhnyuXr2Kzp07\nWzRWc7CG2sKbb/adO3dGeXk5Kioq0NDQgMOHD7coAhEREdo/jUtKSiCRSODh4cFFuHrTJb+bN2/i\niy++wKRJk+Dj48NRpIbRJb8VK1Zg5cqVWLlyJV566SXExcXZRKEHdMuvR48eOHv2LDQaDe7fv4/S\n0lKb2WtKl/zat2+P3377DQBQWVmJq1evokOHDlyEa3LWUFt4tYK2qKio2dSvt956C7m5uQCA6Oho\nAEBGRgaKi4shFosxYcIEdOrUicuQ9fKk/L766iscO3YM7du3BwA4ODggNTWVy5D1osv71yQ9PR3d\nu3e3qamXuuS3a9cu5OfnQyAQ4NVXX8WAAQO4DFkvT8rv1q1bSE9Px61bt8AYw1tvvWUz98lYtmwZ\nzpw5g6qqKnh4eGDo0KFobGwEYD21hVfFnhBC+Io3wziEEMJnVOwJIYQHqNgTQggPULEnhBAeoGJP\nCCE8QMWeEEJ4gIo9IYTwABV7Qgjhgf8PxgEl8hp36FgAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb52c1e7550>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def sklearn_run():\n",
" n_degree = 2\n",
" x_train, y_train, model, x_test, y_test = sklearn_simulation(n_degree)\n",
"\n",
" plt.style.use('ggplot')\n",
" plt.title(\"scikit-learn degree: %s\" % n_degree)\n",
" plt.ylim(-2, 2)\n",
" plt.xlim(-0.1, 1.1)\n",
" plt.plot(x_train, np.sin(2*np.pi*x_train), color='green', linestyle='--')\n",
" plt.scatter(x_train, y_train, marker='o', color='blue')\n",
" plt.scatter(x_test, y_test, marker='o', color='red')\n",
" plt.show()\n",
"\n",
"sklearn_run()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Chainer "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 多層パーセプトロンによる手書き数字認識"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## ニューラルネットワーク\n",
"\n",
"\n",
"\n",
"![ニューラルネットワーク](http://nnadl-ja.github.io/nnadl_site_ja/images/tikz1.png)\n",
"\n",
"<div style=\"font-size:small;\">\n",
"[ニューラルネットワークを用いた手書き文字認識](http://nnadl-ja.github.io/nnadl_site_ja/chap1.html)より\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## パーセプトロン\n",
"\n",
"![パーセプトロン](http://nnadl-ja.github.io/nnadl_site_ja/images/tikz0.png)\n",
"\n",
"<div style=\"font-size:small;\">\n",
"[ニューラルネットワークを用いた手書き文字認識](http://nnadl-ja.github.io/nnadl_site_ja/chap1.html)より\n",
"</div>\n",
"\n",
"<span></span> \n",
"\n",
"それぞれのユニットは, 前の層から入力を受け取り, 重み, バイアスとの積和を活性化関数に適用して, 後ろの層へ出力を送る.\n",
"\n",
"次式では, 入力が (0.5, -2.0, 3.0), 重みが (-1.0, 0.3, 0.4), バイアスが 0.3 とする.\n",
"\n",
"\\begin{align*}\n",
" f(X) & = f(0.5 * -1.0 + -2.0 * 0.3 + 3.0 * 0.4 + 0.3) \\\\\n",
" & = f(0.4)\n",
"\\end{align*}"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 活性化関数\n",
"\n",
"* ReLU(Rectified Linear Unit)\n",
"* 双曲線正接関数(tanh)\n",
"* シグモイド関数(sigmoid)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from sklearn.datasets import fetch_mldata\n",
"from chainer import cuda, Variable, FunctionSet, optimizers\n",
"import chainer.functions as F\n",
"import sys\n",
"\n",
"x_data = np.linspace(-10, 10, 100, dtype=np.float32)\n",
"x = Variable(x_data)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEPCAYAAABShj9RAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFUxJREFUeJzt3X2UbXV93/H373IZbRYH1CLKwG2xRlIEoT6AVBOZKqYs\nopi1mvtLSUSFtF0+YFkkJlGyyr1JfYwxwatJVixPYlDyLbG2GI1gmvGpJV4UJQFiLoRbuY6INRXn\nBvDO5e7+cfaQwzAzd2b2PrPP+c37tdasmf3029/fDPdzvmeffQ6pqiokSeNvU9cFSJLaYaBLUiEM\ndEkqhIEuSYUw0CWpEAa6JBXCQNeGkFJ6Tkrpyymlh1JKfzsC9UyllA6klCa7rkXlMNDVipTS1XVA\nHUgp7U8p3ZtS+vBqAyultD2ltGsIJf4m8H3gx4BThzD+kurfx2sWrP4S8HTg2+tZi8pmoKtNn6cf\nUluAnwOeC/zXNYwzjHe7/Sjw+aqqvllV1feGMP5yKiA9ZkVVzVVVdX/lO/vUIgNdbdpXh9S3q6r6\nAvBfgH+ZUjpsfoeU0stTSl9KKT2YUtqTUroypfSUBeMkllA/A/i5Bes+m1K6aon9j0spHQCeCfxG\nffyl8+tTSi9asP9dKaVtC873hpTSR1JKP6ifebx1wTGbU0rbUkp3p5Qerue1o962GzgEuKoe65F6\n/eMuuaSUTk8pfb7+3fxdSunalNJTB7ZvTyntSimdk1L665TS3pTSn6eUfnSp35c2FgNdbXo0iOug\n+hlgPzAfYi8FPgF8FHgO8NPAccDHG563Yumu/pvA0cAe4N30n0G8b5VjbQOmgVOAdwHvrOcy7wrg\njcClwAn053VXve0F9Od/UX3uoxc7aUrp6cCNdb2nAq8ETgKuX7Dr0cDrgXOBFwE94Mpl5qMNZHPX\nBagoUymlWfqNwj+q1/1WVVUP1T9fCry/qqrfnT8gpfQ6YHdK6eSqqm5ru6Cqqg4A36k7471VVd1f\nn/epyx/5GNdVVXVF/fPvpZQuBM4E/mfdHZ8H/ExVVfMPTPcAt9Tn/78pJYAH5s+9hDfRv8b/uqqq\n9tc1ngd8LaX041VVfbHe7wnAefOXjVJKvwl8LKU0UVXVvlXMSQWyQ1ebbqbfxZ4G/GfgfwH/aWD7\nqcDFKaXZ+S/gdvod8bPWu9hV+NqC5RngqPrn59Xfb2x4jhOBm+fDHKB+gHug3vbouRe8BvBt+s+M\njkIbnh262vRwVVXztwRuSyk9E/gA8B/qdYn+ZY+PLHLsd1Z4jse9wAhMrLZQ4MBATYMOXWTfxTrf\ntpuhxea1mIW1zF8esjmT/xGoVQuvPW8Hzk8pPb9evgU4qaqqv13k6+9XeI77gWPmF1JKTwCevYZa\nv1t/HxzrqMHlZQzO86v193+9zP776L8wupzbgdNTSo8+oKSUTgGOAP5qBTVJBrpatfDWvLuAG4B3\n1KsuBV6VUnpfSulfpJSemVI6K6V0eUrpiQOHTqSUTqn3mf86ud72WeD19R0hJwFXs3hXfbDaHqJ/\nL/ivpJROrh90rgF+uMKx0sAcr6V/bf3n6zmdmlL6jwP73wO8NKV0dErpyCXG/CBwOHB1SunElNKP\n038m8/mqqr60gpr6haV0TH0HzE+v9BiVw0BXW5a60+S9wMtTSi+pqmoaeClwMv171r8O/DbwA2Bu\nYJwtwK30u9/5r5vr7W+h37F+BvgT+nef7FxhfQtdAOylf63/o8AfsLI3+iyc6/n1sW8H7qB/185x\nA9t/CXg+sJvHXlp6dIz6BdOfBI6lP58bgNvo3ym01HkfNw79B7fj6T84aINJK3lfQ875SuCngPsj\n4jn1uqcAfwT8U/r/oeaI+P7wSpUkLWelHfpVwFkL1r0VuCkijgf+rF4+qJzz1IqrG0Mlz6/kuYHz\nG3fOb4WBHhFfAP7fgtXnAB+uf/4w/TdTrMRBixpzU10XMERTXRcwZFNdFzBkU10XMGRTXRcwZFMH\n26HJNfSnRcT89cDvAE9rMJYkqaFWXhSNiOXeei1JWgcrelEUIOd8HHDDwIuifw1MRcR9OeejgT+P\niH++yHFTDDxViIhtC/eRJB1czvnXBxanI2J6cHuTd4r+D+C1wHvq759YbKf6hIMn3TYzM9PgtKOt\n1+sxOzvbdRlDUfLcwPmNu9LnNzk5SURsX26fFQV6zvljwBnAkTnne+m/QeTdQOScf4H6tsVG1UqS\nGllRoEfEuUtsOrPFWiRJDfhOUUkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJ\nKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RC\nGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBVic9MBcs4XA78AVMBfAudHxA+bjitJWp1G\nHXrO+RjgzcDzI+I5wCHAv22jMEnS6jTu0OsxfiTn/AjwI8C3WhhTkrRKjTr0iPgW8D7gm8AM8P2I\n+GwbhUmSVqfpJZcnA+cAxwGTwGE5559voS5J0io1veRyJnBPRHwPIOf8ceBFwLXzO+Scp4Cp+eWI\noNfrNTzt6JqYmCh2fiXPDZzfuCt9fgA55+0Di9MRMT24PVVV1WTw04ArgVOBh4GrgS9HxO8uc1g1\nMzOz5nOOul6vx+zsbNdlDEXJcwPnN+5Kn9/k5CRAWm6fptfQvwxcD3wVuK1e/aEmY0qS1qbxXS4R\nsR3Y3rgSSVIjvlNUkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEM\ndEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCX\npEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQm5sOkHN+EnA5cCJQARdExM1Nx5UkrU4bHfr7\ngU9FxAnAycCdLYwpSVqlRh16zvkI4Cci4rUAEbEfeKCNwiRJq9P0ksszgO/mnK8CTgG+AlwUEQ82\nrkySRDU3R/XJ6+BNv3rQfZsG+mbgecCFEbEz53wZ8Fbg0vkdcs5TwNT8ckTQ6/UannZ0TUxMFDu/\nkucGzm/clTi//Xd/gwd//91sPvpYAHLO2wc2T0fE9OD+TQN9D7AnInbWy9fTD/RH1SccPOm22dnZ\nhqcdXb1ej1LnV/LcwPmNu5LmN9+VV1+4kfSz/45HTnsJABGxfbnjGr0oGhH3AffmnI+vV50J3N5k\nTEnayKrduzjw9oupZu5l07YdbHrhGaSUVnRs49sWgTcD1+acJ4C7gfNbGFOSNpSFXXk67SUrDvJ5\njQM9Ir4OnNp0HEnaqKrduzhw9Q546tPZtG0H6Ygnr2mcNjp0SdIatNGVDzLQJakDbXXlgwx0SVpH\nbXflgwx0SVonw+jKBxnokjRkw+zKBxnokjREw+7KBxnokjQE69WVDzLQJall69mVDzLQJaklXXTl\ngwx0SWpBV135IANdkhrouisfZKBL0hqNQlc+yECXpFUapa58kIEuSaswal35IANdklZgVLvyQQa6\nJB3EKHflgwx0SVrCOHTlgwx0SVrEuHTlgwx0SRowbl35IANdkmrV7l0cuOr9cNTk2HTlgwx0SRte\nNTdHdcPHqL5409h15YMMdEkbWnXPLg5cdRk87Zix7MoHGeiSNqRSuvJBBrqkDaekrnyQgS5pwyix\nKx9koEvaEKp7dnHg6vG9g2UlDHRJRSu9Kx9koEsqVqnXypfSSqDnnA8BbgH2RMQr2xhTktZqI3Xl\ng9rq0C8C7gB6LY0nSWuyEa6VL6VxoOecjwXOBt4B/GLjiiRpDaq5fRz4+DUbrisf1EaH/jvALwOH\ntzCWJK1adc8uZq/ZQXXk0RuuKx/UKNBzzq8A7o+IW3POU0vsMwU8ui0i6PXKvTIzMTFR7PxKnhs4\nv3FUze3j4es/zL7pT9O74KLiu/Kc8/aBxemImB7cnqqqajL4O4HzgP3AE+l36X8cEa9Z5rBqZmZm\nzeccdb1ej9nZ2a7LGIqS5wbOb9w85g6WV7+Bw4/9J0XNb6HJyUmAZR+tGnXoEXEJcAlAzvkM4C0H\nCXNJamSj3sGyEm3fh772dl+SDmKj3Ve+Wq0FekR8DvhcW+NJ0jy78pXxnaKSRppd+coZ6JJGkl35\n6hnokkbOuP+/PbtioEsaGdXcHNUnr6P6wo125WtgoEsaCf/QlW/sd3s2YaBL6pRdeXsMdEmdsStv\nl4Euad3ZlQ+HgS5pXdmVD4+BLmld2JUPn4EuaejsyteHgS5paOzK15eBLmko7MrXn4EuqVV25d0x\n0CW1xq68Wwa6pMbsykeDgS6pEbvy0WGgS1oTu/LRY6BLWjW78tFkoEtaMbvy0WagS1oRu/LRZ6BL\nWpZd+fgw0CUtya58vBjokh7Hrnw8GeiSHsOufHwZ6JIAu/ISGOiS7MoLYaBLG5hdeVkaBXrOeQtw\nDXAUUAEfiogdbRQmabjsysuzqeHxc8DFEXEicDrwppzzCc3LkjQs1dwcB/7bRziw4zdIZ29l0xsv\nMcwL0ahDj4j7gPvqn/fmnO8EJoE7W6hNUsvsysvW2jX0nPNxwHOBv2hrTEntqObmeOi6yznwZ5/0\nWnnBWgn0nPNhwPXARRGxd8G2KWBqfjki6PV6bZx2JE1MTBQ7v5LnBuXOb//d3+DB33831eQWDn/v\nlWx60lO6LmkoSv37Dco5bx9YnI6I6cHtqaqqpic4FPgk8OmIuGwFh1QzMzONzjnKer0es7OzXZcx\nFCXPDcqb38I7WA5/2U+xd+/egx84pkr7+y00OTkJsOzTqqZ3uSTgCuCOFYa5pHWw2LVyL7GUr+kl\nlxcDrwZuyznfWq97W0T8acNxJa2B95VvbE3vcvkizW99lNQC72CR7xSVxpxdueYZ6NIYsyvXIANd\nGkN25VqMgS6NGbtyLcVAl8aEXbkOxkCXxsA/dOWTduVakoEujTC7cq2GgS6NKLtyrZaBLo0Yu3Kt\nlYEujRC7cjVhoEsjwK5cbTDQpY7ZlastBrrUEbtytc1AlzpgV65hMNCldWRXrmEy0KV1YleuYTPQ\npSGzK9d6MdClIbIr13oy0KUhsCtXFwx0qWV25eqKgS61xK5cXTPQpRbYlWsUGOhSA3blGiUGurRG\nduUaNQa6tEp25RpVBrq0CnblGmUGurQCduUaBwa6dBB25RoXjQM953wWcBlwCHB5RLyncVXSCLAr\n17jZ1OTgnPMhwAeBs4BnA+fmnE9oozCpS9XuXRx4+8VUM/eyadsONr3wDMNcI69ph34acFdE7AbI\nOV8HvAq4s+G40rqr9s9R3baTaucXqW7/ql25xk7TQD8GuHdgeQ/wwoZjSuum2j8Hd3yN6pYv8YPb\ndlIdvYX0ghezKV9A6h3RdXnSqjQN9GotBz3yltc1PO3oemBTojqwpl/LyCtybj98CI59BukFL6Z3\n3uv5+0Of2HVF0po1DfRvAVsGlrfQ79IflXOeAqbmlyOCw9/1Bw1PO7oOnTiUuX1zXZcxFEXObeIJ\nbDqs1/9xYoJN+/Z1XNDwTExM0Ov1ui5jaEqfH0DOefvA4nRETA9uT1W19o4r57wZ+AbwMmAG+DJw\nbkQsdw29mpmZWfM5R12v12N2drbrMoai5LmB8xt3pc9vcnISYNkXdBrd5RIR+4ELgc8AdwB/dJAw\nlyQNSeP70CPi08CnW6hFktRAow5dkjQ6DHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgph\noEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6\nJBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRCb13pgzvm9wCuAfcDdwPkR8UBbhUmS\nVqdJh34jcGJEnAL8DfC2dkqSJK3Fmjv0iLhpYPEvgH/TvBxJ0lq1dQ39AuBTLY0lSVqDZTv0nPNN\nwNMX2XRJRNxQ7/NrwL6I+OgQ6pMkrVCqqmrNB+ecXwf8e+BlEfHwEvtMAVPzyxGxbc0nlKQNLOf8\n6wOL0xEx/Zgdqqpa09fWrVvP2rp16+1bt249cpXHbV/rOcfhq+T5lTw35zf+X86vanQN/QPAYcBN\nOedbc86/12AsSVJDTe5yeVabhUiSmuninaLTHZxzPU13XcAQTXddwJBNd13AkE13XcCQTXddwJBN\nH2yHRi+KSpJGh5/lIkmFMNAlqRBrflG0iZzzm4E3Ao8AfxIRv9pFHcOUc/4l4L3AkRHxd13X05ZS\nP5Qt53wWcBlwCHB5RLyn45Jak3PeAlwDHAVUwIciYke3VbUr53wIcAuwJyJe2XU9bco5Pwm4HDiR\n/t/vgoi4ebF9171Dzzn/K+Ac4OSIOAn4rfWuYdjqf0AvB/5P17UMQXEfylaHwQeBs4BnA+fmnE/o\ntqpWzQEXR8SJwOnAmwqbH8BFwB30A6807wc+FREnACcDdy61Yxcd+huAd0XEHEBEfLeDGobtt4Ff\nAf5714W0rdAPZTsNuCsidgPknK8DXsUy/3DGSUTcB9xX/7w353wnMEkh88s5HwucDbwD+MWOy2lV\nzvkI4Cci4rUAEbEfWPIZcReB/izgJTnndwIPA2+JiFs6qGMocs6vov+077acc9flDNsFwMe6LqIF\nxwD3DizvAV7YUS1DlXM+Dngu/QfjUvwO8MvA4V0XMgTPAL6bc74KOAX4CnBRRDy42M5DCfRlPtTr\n1+pzPjkiTs85nwoE8M+GUcewHGR+bwN+cmBdWpeiWrQBP5StxKfpj5NzPgy4nn4g7O26njbknF8B\n3B8Rt9afG1WazcDzgAsjYmfO+TLgrcClS+3cuoh4+VLbcs5vAD5e77cz53wg5/yPI+J7w6hlGJaa\nX875JPqPqF+vu/Njga/knE+LiPvXscRGlvv7waMfynY28LJ1KWj4vgVsGVjeQr9LL0bO+VDgj4E/\njIhPdF1Pi14EnJNzPht4InB4zvmaiHhNx3W1ZQ/9Z/w76+Xr6Qf6orq45PIJ4KXA53LOxwMT4xTm\ny4mIvwKeNr+cc74HeH5hd7mcRf/p7RlLfcLmGLoFeFZ9OWIG+Fng3E4ralHOOQFXAHdExGVd19Om\niLgEuAQg53wG/Uu4pYQ5EXFfzvnenPPxEfE3wJnA7Uvt30WgXwlcmXP+S/q3vhXzy19EiU/lPwBM\n0P9QNoD/HRFv7LakZiJif875QuAz9G9bvCIiinjBsPZi4NXAbTnnW+t1b4uIP+2wpmEp8d/cm4Fr\nc84T1LcKL7Wjb/2XpEL4TlFJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIf4/pDld\njduhzmgAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb4fcd96a90>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"y = F.relu(x)\n",
"plt.title(\"ReLu function.\")\n",
"plt.ylim(-2, 10)\n",
"plt.xlim(-6, 6)\n",
"plt.plot(x.data, y.data)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEPCAYAAABFpK+YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH3hJREFUeJzt3Xu8VHW9//HXd/ZFQAbxCm1AocLjNTUVOYq6U/TgJdSK\nT1k90jy/Ms30cX55UutX8ntkF+vUsbLjz0rNTqV+TVNMU8DaipgcUVEUyBsksFVURDcXYe89398f\ns7YOm31jz2XNrPV+Ph7zmFlrfWd9P18G3qz5zpo1LoSAiIikRybuAkREpLIU/CIiKaPgFxFJGQW/\niEjKKPhFRFJGwS8ikjIKfkkU51yzcy7nnGsaxHNnOOdecM51OOeuL0d921nPTOfcc3HXIcmj4JeK\nc87Ndc7dEHcdhZxzdcD1wM3AOOCiCvY9JfrPas9um34IHFGpOiQ96uMuQKRKNAE7An8OIbwcUw2u\ncCGEsAHYEFMtkmA64peKcs79GjgOOCs6ys05546Jtn3HObfEObfBOfeSc+4a59yIguee7Zxrd84d\n6Zx7PGq30Dl3WA9d7eecezBq84xzblofNZ0N/CNafDCq6diu/rq1Hdut5q6ppal99eec28M5d4Nz\n7hXn3Cbn3DLn3Oedc3sBD0bNlkf7+kv0nG2mepxzZ0V/Rpudcyudc9+O3q10bW9xzv3SOfdN59zL\nzrk3nHM3Oud27ONlkZRR8EulXQjMA24BRke3v0XbNgJfAPYFzgaagZ92e34G+C7wFeDDwBrAF4Zf\n5D+AK4APAQuAW5xzI3up6WZgUvR4elTTw9s5rl77c84NBR4ADgQ+DewDnE/+aH4lcFq0j8Ojvj/W\nUwfOuVOA64Abgf2BrwJfBi7v1vQTwEjgWOBTwKnAJds5HkmyEIJuulX0BswBrh9AuzOAdwqWzwZy\nwMEF6yZF6yZGy83R8ukFbfaI1p3QR1/jozZHduuvvVu7sVG7YwbaH/CvwCagqZe+p0Tt9+y2fibw\nXMHyPODmbm0uJP8fZn203AI80a3NfwEPx/2661Y9Nx3xS9Vwzn0smi5Z7ZxrA34LNDjnRhc0C8CT\nBctd8/Gjuu1u0btPCGEN0NlDm1Lqq79DgWdCCK1F9rEf700LdXkQGAJ8oGDdk93avEx5xy41RsEv\nVcE5dwTgyR+xng4cAnyJ/AeejQVNcyGEwkvKdj3u/nd5Sw/dbO/f91wP6xp6adtff66H7eUQeqgl\noH/rUkB/GSQOW9j2jLIpwOshhG+FEB4NITxP/rTKOK0B6pxzexSs+/Ag9rOQ/IfNY3rZ3hXU3T+n\n6O4Z8vP2hY4lP9XzwiDqkpRS8EsclgOHOufe75zbzTlXDywDdnfOnROt/xxwXrxlsgBoA77vnJsY\nnanzrUHs5ybyZw3Ncs4d75ybEN1btP0f5N9dnBKd/bNTL/v5HvBx59wlzrm9o+dfDvwohNARtXH0\n8+7COXdGdFbRdn/JTZJBwS9x+BHwOvm56FfJf6B6N/Ad8mfsPAUY8O+8N5XTpadfDhpIm4HY6nkh\nhDeBM4HJUa3f2I6aCvezifyR+dPkzyBaAvyM/Nw8IYRXgcuAS4FW4I8F+w0F+/kzcA5wFrAY+DHw\nc+D/dqulp/oK1+0ETETf40ktt/V06fYzs+uBU4A13vsDe9jeDNwJvBitus17f0VRnYqIyKCV4oj/\nBqDXL8dEHvDeHxLdBhT60X8YiZXk8SV5bKDx1TqNrwTB772fB7zZT7PBnNHQPIjn1JLmuAsoo+a4\nCyiz5rgLKLPmuAsos+a4Cyiz5v4aVGKOLwBHmtmTwGrgYu/9kgr0KyIiPajEh7uPA+O89weR/0Dr\njgr0KSIivSj6w10AMxsP3NXTh7s9tF0OHOq9X9ttfTMFb1G8992vPyIiIgNgZoVnerV471sKt5c9\n+M1sFPkzfoKZTQK89378AHYbWluL/YZ79cpms7S1tcVdRlkkeWyg8dW6pI+vqakJ+vlcteg5fjO7\nifw5yruZ2UryXyhpAPDeX0v+SoHnmVkH+W8YfqrYPkVEZPBKcsRfJjrir1FJHhtofLUu6eMbyBG/\nvrkrIpIyCn4RkZRR8IuIpIyCX0QkZRT8IiIpo+AXEUkZBb+ISMoo+EVEUkbBLyKSMgp+EZGUUfCL\niKSMgl9EJGUU/CIiKaPgFxFJGQW/iEjKKPhFRFJGwS8ikjIKfhGRlFHwi4ikjIJfRCRlFPwiIimj\n4BcRSRkFv4hIyij4RURSRsEvIpIyCn4RkZRR8IuIpIyCX0QkZeqL3YGZXQ+cAqzx3h/YS5ufAicB\nG4GzvfdPFNuviIgMTimO+G8ApvW20cxOBj7ovZ8IfBG4pgR9iojIIBUd/N77ecCbfTSZDtwYtV0A\njDSzUcX2KyIig1OJOf4xwMqC5VXA2Ar0KyIiPSh6jn+AXLflUKF+RSQmIQToaId3NsGWLbBlM7Rv\nzt9v2QztHdDZTmhvh86OfNuODsh1Qmcuv67rca4Tcrn3bqHrPuQfh5C/5QoehwB0Pebdxxvq68m1\nt0MIhK4oejeRupYLIqr7tm0HWoo/rOL30eV7/c+mVyL4VwPjCpbHRuu2YmbNQHPXsveebDZb7tpi\n09jYmNjxJXlskN7xhVwnYe3r5F57hdybbxDefotc2zrC2+sIb79FaHuLsHE9YeNGwqYNhI0bAHBD\nh0HjDrgdhuB22AEadsA1NkJjI66+EerroaEBV9+Qf1xXj8vUQV3+5hoaIZMpuNVBJoNzLr/sMpBx\n+Xvn3r25rsd0rQNw1Dc00NHZmR/Uu23y2wrv3ltfuK37MWyfq9n2mLcPve9ku5nZzILFFu99S+H2\nSgT/LOAC4GYzmwys896/2r1RVFhhcZe3tbVVoLx4ZLNZkjq+JI8Nkj++HTvbWb/sacKq5dC6kvDG\nGnj9VXhrLQwfAbvuASN3wWV3guE7we7vgwn74IZnYdhwGDoUN3RH3JBhuIaGHvsI3e4rqTGbZXOC\nXz8A7/3Mvra7UORbDDO7CTgW2A14FbgcaIg6vzZqczX5M382AJ/33j8+gF2H1tbWomqrZkkOjySP\nDZI1vrBlMzy3hLDsKcJLL8DK5bgQCGP2wo0dD2P2wu02CnbdHXbevdcgryVJev160tTUBP281Sg6\n+MtIwV+jkjw2qO3xhVwOVr5IWLKIsGQRLH8Wxk3A7XsQbvxEGDuB7J7jWb9+fdyllk0tv34DMZDg\nr9SHuyISo7D2NcL8+wnz5+bn0/c7hMzU6bD3Afk5+AKuhHPNUp0U/CIJFTo64KlHyc2bDcufxR1+\nNJnzL8Pt+YG4S5OYKfhFEiZ0dBAevJdwz60wqgk35UTcly7Nn1EjgoJfJDFCCLB4Iblbb4CddyVz\n0UzcuAlxlyVVSMEvkgBh1Qpyt14Pa18jM+McOPAwzdVLrxT8IjUstLcT/nAD4dF5uFM/iTtmGq5e\n/6ylb/obIlKjwro3yF3zfRi5C5lvX4PbcXjcJUmNUPCL1KDw/FJy116Jaz4Zd/IMTevIdlHwi9SY\n3AP3Eu78HZnPX4Q78LC4y5EapOAXqRGhvZ1w07WE55eSueRK3KimuEuSGqXgF6kBobOT3C9+ACGQ\n+foPcUOG9f8kkV7ox9ZFqlwIgfC7a2DLZjJfukShL0VT8ItUuTDr94SXXiRz3qX5a9aLFEnBL1LF\ncn+9h/A/D5K58Fs60peS0Ry/SJUKj80n3OPJfO37uBEj4y5HEkRH/CJVKPx9Mbnf/T8yX/kmbvfR\ncZcjCaPgF6ky4fVXyV37AzJfuFiXUJayUPCLVJGQy5H79U9xJ56O2/eguMuRhFLwi1SR8Nd7oKMd\nd+LpcZciCabgF6kS4ZXVhD/dRObsi3CZurjLkQRT8ItUgZDrJPfrn+BO/RRu9Ji4y5GEU/CLVIEw\n506ob8B95JS4S5EUUPCLxCysfolw7+1kzr4Ql9E/SSk//S0TiVHo6CB3w1W4Mz6L221U3OVISij4\nRWIU7rsdhmdxR/9L3KVIiij4RWIS3n6TMOdOMp89X7+gJRWl4BeJSbj7Vtw/f0RTPFJxCn6RGITX\nXiEseAB38oy4S5EUUvCLxCDMugl33Cm47E5xlyIpVPRlmc1sGnAVUAf8ynt/ZbftzcCdwIvRqtu8\n91cU269IrQqrVhCeeZzMd66NuxRJqaKC38zqgKuBqcBq4FEzm+W9X9qt6QPe++nF9CWSFLk7fos7\n+RO4ofphFYlHsVM9k4DnvfcrvPftwM3AaT200ykLIkB4fgmsXI479qS4S5EUK3aqZwywsmB5FXBE\ntzYBONLMniT/ruBi7/2SIvsVqTkhBHK3/wY3/dO4hsa4y5EUKzb4wwDaPA6M895vNLOTgDuAvbs3\nij4LaO5a9t6TzWaLLK96NTY2JnZ8SR4bDH587U8sYNOmDWRP/GhVX31Tr1/tM7OZBYst3vuWwu0u\nhIFkd687nwzM9N5Pi5YvA3LdP+Dt9pzlwKHe+7X97D60trYOurZql81maWtri7uMskjy2GBw4wu5\nHLlv/xuZ6WfiDplcpspKQ69fbWtqaoJ+pteLPeJfCEw0s/FAK/BJ4MzCBmY2CljjvQ9mNglwAwh9\nkWR54m/Q0AAHd58JFam8oj7c9d53ABcA9wFLgFu890vN7FwzOzdq9glgsZktIn/a56eK6VOkFuXm\n3Elm2sd0aQapCkVN9ZSZpnpqVJLHBts/vvDi38n94odkvnttVc/td9HrV9sGMtWjb+6KlFmYOwt3\nfHV/oCvpouAXKaOw9jXCkkW4KSfEXYrIuxT8ImUU/nJ3/gqc+pauVBEFv0iZhM3vEObPwR13atyl\niGxFwS9SJuHhv8DE/XG7j467FJGtKPhFyiDkcoS5s8hM7enSVSLxUvCLlMPix2DoMJi4X9yViGxD\nwS9SBrn7Z+GmflRf2JKqpOAXKbGwajm8vBJ32JS4SxHpkYJfpMTC3Fm45pNx9Q1xlyLSIwW/SAmF\nDW2Exx/BHTst7lJEeqXgFymh8EgL7sDDcMNHxF2KSK8U/CIlEkIgzJuNO+bEuEsR6ZOCX6RUlj8L\n7Vtg7wPirkSkTwp+kRIJD83BTTlBp3BK1VPwi5RAeGcj4bH5uH8+Lu5SRPql4BcpgfDoQ7D3gbiR\nu8Rdiki/FPwiJRDmzSZztK65L7VBwS9SpLBqBbz5Buz/4bhLERkQBb9IkcJDc3BHHY+r008rSm1Q\n8IsUIbRvISx4AHfU1LhLERkwBb9IEcLjf4M9368fW5GaouAXKUL+3H19U1dqi4JfZJDCmpdh1Qrc\nwUfEXYrIdlHwiwxSmD8XN7kZ16DLL0ttUfCLDELIdRIe/os+1JWapOAXGYylT8FOO+PGjo+7EpHt\npuAXGYQwfy7uqOPjLkNkUOqL3YGZTQOuAuqAX3nvr+yhzU+Bk4CNwNne+yeK7VckLrkN6wlPP0bm\n0+fGXYrIoBR1xG9mdcDVwDRgP+BMM9u3W5uTgQ967ycCXwSuKaZPkbi1/+0vsN/B+pUtqVnFTvVM\nAp733q/w3rcDNwOndWszHbgRwHu/ABhpZqOK7FckNlta7iWjD3WlhhUb/GOAlQXLq6J1/bUZW2S/\nIrEIL68k9/qrsN8hcZciMmjFzvGHAbbr/pNE2zzPzJqB5q5l7z3ZbHbQhVW7xsbGxI4vyWPbdNdD\n1DVPo3HkyLhLKZskv36Q/PEBmNnMgsUW731L4fZig381MK5geRz5I/q+2oyN1m0lKqywuMvb2tqK\nLK96ZbNZkjq+pI4tdHaSe+A+st/8USLH1yWpr1+XNIzPez+zrzbFBv9CYKKZjQdagU8CZ3ZrMwu4\nALjZzCYD67z3rxbZr0jlLVkEu+xG3djxkODgkOQrao7fe99BPtTvA5YAt3jvl5rZuWZ2btTmHuBF\nM3seuBY4v8iaRWIRHr4fd6TO3Zfa50IY6DR9xYXW1ta4ayibJL/dTOLYwoY2cpd9kcz3fsmI0e9L\n3PgKJfH1K5T08TU1NcG2n6tuRd/cFRmA8D/zcAd8GLfj8LhLESmagl9kADTNI0mi4BfpR1j9Eqx7\nA/Y7KO5SREpCwS/Sj/DQbNyRU3EZ/Zi6JIOCX6QPob2d8EgLboou0SDJoeAX6UNY9AiMm6AfU5dE\nUfCL9CHMm407Wj+mLsmi4BfpRXjtFVi5HHfw5LhLESkpBb9IL8JD+jF1SSYFv0gPQmcn4eG5uCma\n5pHkUfCL9OTpx2DXPXBj9oy7EpGSU/CL9CA3bzZuyglxlyFSFgp+kW7CujfguSW4w6bEXYpIWSj4\nRboJ8+/HHXYUbsjQuEsRKQsFv0iBkMsR5utDXUk2Bb9Iob8vhh2GwPgPxl2JSNko+EUKdH1T17k+\nf8dCpKYp+EUiYd1awjOP445ojrsUkbJS8ItEwl/vwU06Vr+yJYmn4BcBwpbNhHn34Y7/aNyliJSd\ngl8ECI+0wIS9caPHxF2KSNkp+CX1QgiEubPITJ0edykiFaHgF3nmCairg30+FHclIhWh4JfUy829\nEzd1uk7hlNRQ8EuqhdaX8j+2MumYuEsRqRgFv6RauP8u3LHTcA2NcZciUjEKfkmt0PY2YeFDuOaT\n4i5FpKIU/JJa4cF7cYdMxo3YOe5SRCqqfrBPNLNdgFuAvYAVgHnv1/XQbgXwNtAJtHvvJw22T5FS\nCR3thJZ7yFx0edyliFRcMUf8lwJzvPd7A/dHyz0JQLP3/hCFvlSL8OhDMHosbuyEuEsRqbhign86\ncGP0+Ebg9D7a6jw5qRqho4Nw101kTp4RdykisSgm+Ed571+NHr8KjOqlXQDmmtlCM/tCEf2JlER4\naA7sPhq370FxlyISiz7n+M1sDjC6h03fKFzw3gczC73s5ijv/ctmtjswx8yWee/n9dBXM9BcsE+y\n2Ww/5deuxsbGxI6vmscWNr/D2/fcyvCLr6B+kDVW8/hKQeOrfWY2s2CxxXvfUrjdhdBbXve742Xk\n5+5fMbP3AX/13u/Tz3MuB9Z77380gC5Ca2vroGqrBdlslra2trjLKItqHlvuz3+Af7xA5kuXDHof\n1Ty+UtD4altTUxP0M71ezFTPLOCs6PFZwB3dG5jZMDPLRo93BE4EFhfRp8ighQ3rCbPvwJ3+mbhL\nEYlVMcH/feAEM3sWOC5axsyazOzuqM1oYJ6ZLQIWAH/y3s8upmCRwQr33ZY/b3/02LhLEYnVoKd6\nKkBTPTWqGscW1r1BbuaFZL71E9wuuxW1r2ocXylpfLWt3FM9IjUj/OkW3FFTiw59kSRQ8EvihTWt\nhMfm4076eNyliFQFBb8kXrjz97ipp+GGj4i7FJGqoOCXRAvPLyH8/Wn9iLpIAQW/JFbY/A65G35C\n5jNfwg0ZGnc5IlVDwS+JFW67Eff+fXCHTI67FJGqouCXRApLnyQsWoD7lC4PJdKdgl8SJ2zaSO7G\nn5H53JdxOw6PuxyRqqPgl8QJt16P2+9g3AGHxl2KSFVS8EuihMWPEZYswtk5cZciUrUU/JIYYcN6\ncv/9czJnX4gbMizuckSqloJfEiHkcoTf/lf+Imz7fCjuckSqmoJfEiHcfiPhzddxHzur/8YiKafg\nl5qXm30H4amFZL7yTdwOO8RdjkjV6/OnF0WqXe6RFsL9s8hcciVux2T/nJ5IqeiIX2pWeOYJgr+O\nzIUzcbvsHnc5IjVDwS81KSx/jtx1PyZz/mW4MXvGXY5ITVHwS80JrS+R+/kVZD53Ae6D+8VdjkjN\n0Ry/1JSwaAG531yNs3/FHXxE3OWI1CQFv9SEkMsR/nQz4aG5ZC74P7j3/1PcJYnULAW/VL2wcQO5\n6/8TNrSR+caPcDvtHHdJIjVNc/xS1cLLq8h972LczruS+eoVCn2REtARv1Sl0L6FMPcuwuzbcR87\ni8zRJ8ZdkkhiKPilqoQQCAvnE277NYybQOaSH+BGj4m7LJFEUfBL1QjLnyXnr4PN7+SvsKmLrYmU\nhYJfYhVyOVj2JLkH7oMXl+FO+wzuyONwmbq4SxNJLAW/xCKsfZ3w8P2Eh+bAsB1xR/8L7vMX4YYM\njbs0kcRT8EvFhLWvE5YuIjz2MLywDHf4FDLnXYrb64NxlyaSKoMOfjObAcwE9gEO994/3ku7acBV\nQB3wK+/9lYPtU2pL2LQR/r6YsGQRYekiWN+G2/cg3GFTcOd+DbfDkLhLFEmlYo74FwNnANf21sDM\n6oCrganAauBRM5vlvV9aRL9SZUII8MYaWLWCsGo5G15ZTefy52DdG/CBfXD7Hkzmf10M4ybgMvrq\niEjcBh383vtlAGbWV7NJwPPe+xVR25uB0wAFf40InZ2w4W1oexva3iK89SasfQ1eX0NYuwZeXwNr\n18Cw4TB2Am7seBomHU3nSTNgVBOuXrOJItWm3P8qxwArC5ZXAbqy1gCEEODdWy5/n8u9dwsFjzs7\nIde59X1HB3S25+873rsPW7bAls3QHt1v2QKbN8E7G/NTM5s2wjubYNMGWP82bNyQD/XsTpAdgcuO\nhN32gLF7kTnocNh1FOy6+1YfyjZms2xua4vxT09E+tJn8JvZHGB0D5u+7r2/awD7D4OqKtJ5cZG/\nnxq2s/v+2m+1PWz78N3toVuTrgB/b+U6iAK94J7CsA/gXHTLgAMyGXB10b3L32cyUFf/3n1dtL2+\nAerr8/d19dCQv3eNO0BjIzTuAA3R/fAsDB1GZugwGDIMhgyFocNg+AgYntWplSIJ02fwe+9PKHL/\nq4FxBcvjyB/1b8PMmoHmgr4Z8f1fFtk9+YAs5T622p/b9uG7291Wy/k7926YNzQ00N7eng9pXL65\nc/nHUbC7UtQeg8bGRrLZ5P4MosZX25I+PgAzm1mw2OK9byncXqqpnt4SaiEw0czGA63AJ4Eze2oY\nFVZY3OUb6htLVF4FdX/T0Mu7hOyQoazfuKkiJVVaNpulLcFTPRpfbUvD+Lz3M/tqM+hTLMzsDDNb\nCUwG7jazP0frm8zsbgDvfQdwAXAfsAS4RWf0iIjEy4XtnQevnNDa2hp3DWWT5KOOJI8NNL5al/Tx\nNTU1Qe+zMICuxy8ikjoKfhGRlFHwi4ikjIJfRCRlFPwiIimj4BcRSRkFv4hIyij4RURSRsEvIpIy\nCn4RkZRR8IuIpIyCX0QkZRT8IiIpo+AXEUkZBb+ISMoo+EVEUkbBLyKSMgp+EZGUUfCLiKSMgl9E\nJGUU/CIiKaPgFxFJGQW/iEjKKPhFRFJGwS8ikjIKfhGRlFHwi4ikjIJfRCRl6gf7RDObAcwE9gEO\n994/3ku7FcDbQCfQ7r2fNNg+RUSkeIMOfmAxcAZwbT/tAtDsvV9bRF8iIlIigw5+7/0yADMbSHM3\n2H5ERKS0KjHHH4C5ZrbQzL5Qgf5ERKQPfR7xm9kcYHQPm77uvb9rgH0c5b1/2cx2B+aY2TLv/bzt\nLVREREqjz+D33p9QbAfe+5ej+9fM7I/AJGCb4DezZqC54Hk0NTUV231Vy2azcZdQNkkeG2h8tS7p\n4zOzmQWLLd77lq0ahBCKus2YMeOvM2bMOLSXbcNmzJiRjR7vOGPGjPkzZsw4cYD7nVlsbdV8S/L4\nkjw2ja/2bxpfGPwcv5mdYWYrgcnA3Wb252h9k5ndHTUbDcwzs0XAAuBP3vvZg+1TRESKV8xZPX8E\n/tjD+lbglOjxi8DBg65ORERKrpq/udsSdwFl1hJ3AWXUEncBZdYSdwFl1hJ3AWXWEncBZdbSXwMX\nQqhAHSIiUi2q+YhfRETKQMEvIpIyxVyrp+zM7CvA+eQv8Ha39/6SmEsqOTP7KvBDYLckXc/IzH4I\nnApsAV4APu+9fyveqopnZtOAq4A64Ffe+ytjLqlkzGwc8BtgD/LfuP+F9/6n8VZVWmZWBywEVnnv\nPxp3PaVkZiOBXwH7k3/9zvHeP9JT26o94jezjwDTgQ957w8A/iPmkkou+od2AvCPuGspg9nA/t77\ng4BngctirqdoUWhcDUwD9gPONLN9462qpNqBf/Pe70/+NO0vJ2x8ABcBS8gHY9L8BLjHe78v8CFg\naW8Nq/mI/zzge977dsh/8zfmesrhx8DXgDvjLqTUvPdzChYXAB+Pq5YSmgQ8771fAWBmNwOn0cc/\nsFrivX8FeCV6vN7MlgJNJGR8ZjYWOBn4DvC/Yy6npMxsJ+Bo7/1ZAN77DqDXd9jVHPwTgWPM7LvA\nO8DF3vuFMddUMmZ2Gvm3m08N8Aqntewc4Ka4iyiBMcDKguVVwBEx1VJWZjYeOIT8f9pJ8Z/AvwMj\n4i6kDCYAr5nZDcBBwGPARd77jT01jjX4+7gI3DfI17az936ymR0OeOD9layvWP2M7zLgxIJ1NXfp\n6oFcxM/MvgFs8d7/vqLFlUcSpwe2YWbDgT+QD471cddTCmZ2KrDGe/9EdF2wpKkHPgxc4L1/1Myu\nAi4FvtVb49j0dRE4MzsPuD1q96iZ5cxsV+/9GxUrsEi9jc/MDiD/P/ST0dH+WOAxM5vkvV9TwRKL\n0t9F/MzsbPJvrY+vSEHltxoYV7A8jvxRf2KYWQNwG/Bb7/0dcddTQkcC083sZGAIMMLMfuO9/1zM\ndZXKKvIzCI9Gy38gH/w9quapnjuA44AHzGxvoLGWQr8v3vungVFdy2a2HDg0YWf1TCP/tvpY7/07\ncddTIguBidE0SCvwSeDMWCsqITNzwHXAEu/9VXHXU0re+68DXwcws2PJTx0nJfTx3r9iZivNbG/v\n/bPAVOCZ3tpXc/BfD1xvZovJnxKYmBepB0mcQvgZ0Ej+NxgA/ua9Pz/ekorjve8wswuA+8ifznmd\n9z4RH3xGjgI+CzxlZk9E6y7z3t8bY03lksR/c18BfmdmjUSnUPfWUJdsEBFJmao9j19ERMpDwS8i\nkjIKfhGRlFHwi4ikjIJfRCRlFPwiIimj4BcRSRkFv4hIyvx/910M0IIdTCYAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb4fcd3f780>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"y = F.tanh(x)\n",
"plt.title(\"tanh function.\")\n",
"plt.ylim(-1.5, 1.5)\n",
"plt.xlim(-6, 6)\n",
"plt.plot(x.data, y.data)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEPCAYAAACukxSbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8FfW9//HX5ySEzYCIogZRXNCqiFYEvVo12mpxqdRW\nvtZqq1Vb6na7+dO2Xgu9rbd6a92Kda1a9Sp+q617a2k1V687igvghooCEUFACIRAkvP9/TGDHkMS\nQibJnOX9fDzmcc7MfM/M50vCO3O+M2eOhRAQEZHSkEm7ABER6TkKfRGREqLQFxEpIQp9EZESotAX\nESkhCn0RkRKi0JduZWZZM/tm2nUAmNnwuJ79N9BurpldsIE21WY208zWmtmjXVvpxjOzU8ysMe06\nJP8p9KW7bQXck3YRsfeJ6nluA+1CPLXnGmA6sD3wteSldYyZbRP/4TqoxaqpQFVP1SGFS6Ev3SqE\nsCiEsCbtOgBCCNm4nqYk2zEzA3YC/hlCWBBC+LhrKty4MnJnQggNIYTFKdQhBUahL4mY2RfM7Ekz\nWxFPL5nZ4TnrPzO8Y2bbm9k/zGx1PIwy0cxqzOyGnDZzzew/zewaM1tmZh+a2Vlm1tvMfm9mS81s\nvpmd1aKWrc1savyaejN7zMxG56xfb3jHzPY0s6fMrMHM3jQzt4H+VgPNQBlwa7y9b8fDPVkzq2rR\nvsnMvt1i/xPM7EEzW2Vmb5vZyS1es4mZXWFm78d1vWtmP4tXvx8/PhZv6534NesN75jZkWb2QryN\nD83sajPrl7P+FjObZmbfM7P3zGy5md1nZkPa+zeQwqbQl04zs3LgfuBp4PPxNAmob6O9AX8FKoED\ngfHAMcBerD+ccg7wBjAauAr4PfAX4G1gH2AKcJWZ7Zqz7XuBnYGjgLHAh8A0MxvcRj19gYeBpcAY\n4NvAuUB7ofcksHX8/Cyi4SLfTvvWhokuBm4B9iAalrnRzEbk9ONB4GjgbOBzwElxXwD2jh+/Fu97\nTBt9G0X0s6kBRgEnx9u8tkXTMcDBwBHAl+OaLm2nP1LoQgiaNHVqAgYBWeDgdtpkgW/Gzw+L53do\nsY1VwPU5y+YCf8mZN2A5cF+LZUuBM+P5L8bb/lxOmwqgFrgwnh8et9k/nj8dqAMG5rxm97jNzzfQ\n90/6Fc9Xx8uqWrRrBL7dYv8/zFmfAVYA323Rj73b2O828fqDWiw/BWjMmb8NeKZFm2OI3qUMi+dv\nARYCvXLanAfUpv27pan7Jh3pS6eFEJYBNwKPmNnDZna+me3czkt2Az4KIbzTYhtvtNw08HJOmwAs\nBl5psWwRnx6V7w4sCSG8ntNmLfBsvK6temaHEJbnvGYW0R+Y7vRSzv6yRP3YMl40GlgWQngx4T52\nAx5vsexxoj+Wu+Usez2EkDss9EFOLVKEFPqSSAjhe0RBNY1omGCmmX2vvZe0ssxaWdby8sPQxrIN\n/Q5bG/tsb9+dkW25PTMro/X61raY70g/OqMjfWvt37Sr/k0kDyn0JbEQwqwQwuUhhCOBPwJthf5s\nYAsz22HdAjMbRDQOn9QsYPC6Mf54272BfYGZ7bxmVzMbmPOa3YGBbbRvz6L4cWjOsr3Y+AB9ARiU\newK6hXV/MMo2sJ1ZQMvLOg8mCvVZOct0b/USo9CXTjOzHc3sEjM7wMy2M7N/IzpBO6u19iGEaUTD\nNreZ2T5mtifR2HMjnw2f1oKy3WUhhH8RXX9/h5ntb2YjgVuJxvWvaaMLdxCN6d9uZqPMbD/gJmB1\n271u01vAe8BkM9vFzL4AXE7HQrVlP54A7jKzY+KrnQ4ws9PiJh8BK4Evm9lW8R/N1vwW2NvMLjOz\nz5nZOKKT4beHEOa3tu9WCzMba2avm1mrJ4yl8Cj0JYlVRNerTyUal7+b6OqWs9t5zbHx654gurrk\nofi1DTltWgvKjiz7KvB6vM3niMb7DwshLG3tNSGE1cCRwOC4/W3AZXx61N5hIYRm4Ph4nzOIAvbn\nfDrsszH9OIroqqJr4/7cFte47hzAWYAD5hG9M2itb68Snbg9iOgcwq3AA8D3W7TfUD39gBFA31ba\nSQGy6HxY5znnbiL6JV3kvd+jlfUnEl0RYERHVWd4719p2U5Kk5lVAvOJrpa5Ou16RIpdVxzp3wyM\na2f9O8BB3vtRwK+A6ze0QedcdRfUlbdKuX9m9pX4Q0Pbm9m+wF1ElxG2d617Xinln1+hK+a+Qcf6\nlzj0vfdPAMvaWf+0937dJXDPEl1nvCHVSevKc9VpF9DNqttZ149ovHkm0XADwBdCYd1CoDrtArpZ\nddoFdKPqtAvoZtUbalDeA0XkOo1orFJKVAjhLqKjexFJQY+FvnPuEOBU4ICe2qeIiHxW4hO5AM65\n4cADrZ3IjdePIrpvyjjv/ZwObFLXDouIdE67l+F2+5G+c25bosA/qYOBD0BtbW33FZWyyspK6urq\n0i6j26h/ha2Y+1fMfQOoqtrwVyokDn3n3J1En/Tb3Dk3j+gui70AvPfXAb8guqnWNc45gEbv/dik\n+xURkY3XJcM73SDoSL9wqX+FrZj7V8x9g0+O9Nsd3tEnckVESohCX0SkhCj0RURKiEJfRKSEKPRF\nREqIQl9EpIQo9EVESohCX0SkhCj0RURKiEJfRKSEKPRFREqIQl9EpIQo9EVESohCX0SkhCj0RURK\niEJfRKSEKPRFREqIQl9EpIQo9EVESohCX0SkhCj0RURKiEJfRKSEKPRFREpIeZIXO+duAo4CFnnv\n92ijzVXAEUA9cIr3fkaSfYqISOclPdK/GRjX1krn3JHATt77EcD3gGsS7k9ERBJIFPre+yeAZe00\nOQb4U9z2WWBT59yWSfYpIiKd191j+kOBeTnz84FtunmfIiLShkRj+h1kLeZDywbOuWqget28957K\nysrurSpFFRUV6l8BU/8KVxp9CyFAcxNhzRpYu4bQuBYaG+PHaApNTdGy5kZoaoKmxmhZUzQfmpui\n5c3N0fPm5uh5thly5vl/v8I5Nzln9zXe+5rcero79BcAw3Lmt4mXfUZcVE3Ookl1dXXdWliaKisr\nUf8Kl/pXuDrat5BthtX1sGol1K+bVhFW10PD6mhdQ/y8YTWhYTWsXQNrGqJp7ZqcaS1kMlBRAb3W\nTb2gPH7s1QvKyqE8em5l5VBeHi+LH8vK4il+Xt4LKvpAWQYy8fJMNHDjvZ/cXt+6O/TvB84Gpjrn\n9gM+9t5/2M37FBFZT2iop3nlckLtPMLyZbB8KSz/GFauINQth5UroG4FrFwOq1dD377QbxPo1/+T\nR+vbH/r0hb79YOBmsFVf6N2XTO8+8MnUF3r3horecdD3xsrK0u7+JyyE9UZbOsw5dydwMLA58CEw\nCegF4L2/Lm4zhegKn1XAd7z3L3Zg06G2trbTdeW7Yj6SAvWv0BVi/0IIUWgv+oCweCEsXQxLFhOW\nLo6eL10M2WYym25OdsBAGDgIG7gZDNgUKgdilQNhkwFQOQAqB0Lf/lim8D7GVFVVBesPqX9GotDv\nRgr9Aqb+FbZ87l9oaoJFtYQF70Pt+/DhAsKiD2DRB1HUDanCttgKNtsCBm+BxY8M2gL69WfAgAF5\n27eu0JHQ74kTuSIiGy2srof35hDefQvmzyUsmBuF+6DNYei2WNW2sMc+ZIZsDVtWQf9KzNrNO0Gh\nLyJ5IGSzsOA9wpzZ8O6bhLlzYMkiGLY9NnwE7L4XmcPHw1bDsN690y63oCn0RaTHhWwWat8jvDGT\n8Mar8OYs2GQANmI32GlXMl8aD1XbYuWKqK6mf1ER6RGhoR5mzSC8/Bxh5ovQtx+2yx7Y6AOwEyZi\ngwanXWJJUOiLSLcJSz8ivPQM4eXn4Z3XYcfPYXuOJTP+RGzwkLTLK0kKfRHpUqF+FeHFpwjP1MD8\nudioMWQO+jKccT7Wp1/a5ZU8hb6IJBaam+HV6YRnagizZ8DnRpE59CjYYx+sV0Xa5UkOhb6IdFqo\nW0F44hFCzd9gs82x/Q8l860zsf7Fee+eYqDQF5GNFt5/m/Dog4QZz2Cf/zcyZ/8Htu0OaZclHaDQ\nF5EOCSHAay+TfXAqLFmEVR9J5tfXYZUD0i5NNoJCX0Q2KLz+Ctn774AVy7Gjj8fGHJhXNxGTjlPo\ni0ibwhszo7D/eAl29DewsQcp7AucQl9E1hMWLiB71w3wYW10ZL9vtcK+SCj0ReQToaGe1fffQfax\nh7AjjsPOugAr75V2WdKFFPoiQgiB8NzjhLtvITtqNJlJv8c23SztsqQbKPRFSlxYOJ/srVNgTQOZ\niefR//Nji/qe86VOoS9SokI2S6h5mPDAndjRJ2CHHIFlNG5f7BT6IiUoLF1M9paroGE1mfP/G9tq\naNolSQ9R6IuUkBBCdH+cP9+EffEr2Liv66qcEqPQFykRoaGecMvvCQvnk/nhZGzbHdMuSVKg0Bcp\nAeGD+WT/8F/YzruTueB3uvNlCVPoixS5MOMZsrddjR37LTIHHp52OZIyhb5IkQrZZsJ9dxCeeYzM\nORdi2++cdkmSBxKHvnNuHHAFUAbc6L2/pMX6gcDtwLB4f5d6729Jul8RaVtYVUf2hkuhqYnMBZdh\nAzZNuyTJE5kkL3bOlQFTgHHAbsAJzrldWzQ7C5jpvd8LqAZ+55zTOwyRbhKWLCZ7yU+xrbYh86P/\nVODLZyQKfWAsMMd7P9d73whMBca3aJMF1t1wewCwxHvflHC/ItKKsOA9specj33hMDLf+K4ux5T1\nJD3iHgrMy5mfD+zbos0U4AHnXC1QCbiE+xSRVoQ3Z5K99hLs+NPJ7Htw2uVInkoa+qEDbcYBL3rv\nD3HO7QhMc87t6b3/5OYezrlqoqEfALz3VFYW73dsVlRUqH8FLB/7t/a5x1l94+X0//cL6bXH6ETb\nysf+dZVi7ts6zrnJObM13vua3PVJQ38B0QnadYYRHe3nOgX4DYD3/m3n3LvALsD0dQ3ionILm1TM\nN3yqrKws6htaqX89K1vzMOEhT+bfJ9Gw3Y40JKwt3/rXlYq5bxD1z3s/ub02SUN/OjDCOTccqAWO\nB05o0eZ94EvAk865LYkC/52E+xURIDvtPsJjD5E572Jsi63SLkcKQKITufEJ2bOBR4DZwF3e+9ec\ncxOdcxPjZr8C9nfOvQL8EzjPe780yX5FBLL/vD8K/HMvUuBLh1kIHRmW73GhtrY27Rq6TSm8xVT/\nulf20QcJ0+6LAn/wkC7ddj70r7sUc98AqqqqAKy9Nkkv2RSRHpZ97GHCP+4l85Nfd3ngS/FT6IsU\nkGzN3wh/vycK/M23TLscKUD6ZKxIgcg+8Q/C3/5M5tz/0hi+dJqO9EUKQJjxDOG+O8j8+NcKfElE\noS+S58Kc2WRvnULm7AuwLavSLkcKnEJfJI+F2vfJ/uE3ZE77MTZ8RNrlSBFQ6IvkqbBsCdkrf4kd\n9x1s5N5plyNFQqEvkodC/UqyV07Gqo8gs/+haZcjRUShL5JnQmMj2T/8Btt5JDbu62mXI0VGoS+S\nR0IIhNuuhn79sW+cjlm7H64U2WgKfZE8EqbdR5j3bnTiNqMvQJGup9AXyRPh1RcI//hrdGlm7z5p\nlyNFSqEvkgfCB/PJ3nwFmYnn63460q0U+iIpC6tWkr36IuzYb2Ejdku7HClyCn2RFIXmZrI3/BYb\nuTeZAw9PuxwpAQp9kRSFe26BbBabcGrapUiJUOiLpCT7/BOEGc+QmXgeVqYrdaRnKPRFUhA+mE+4\n4zoy3/8p1r8y7XKkhCj0RXpYWNNA9tqLoxO32+2YdjlSYhT6Ij1o3SdubbudMJ24lRQo9EV6UHj8\nEcL8udiJZ+gWC5IKhb5IDwnvzSHce3s0jt+7d9rlSIlS6Iv0gLCqjuy1l5A56Qxsq6FplyMlLPEX\nozvnxgFXAGXAjd77S1ppUw1cDvQCPvLeVyfdr0ihCCGQvflKbM+x2OgD0i5HSlyiI33nXBkwBRgH\n7Aac4JzbtUWbTYGrga9470cCxyXZp0ihCY89BB8vxY47Je1SRBIP74wF5njv53rvG4GpwPgWbb4J\n3OO9nw/gvf8o4T5FCkaY9y7hgalkvncuVt4r7XJEEg/vDAXm5czPB/Zt0WYE0Ms59xhQCVzpvb8t\n4X5F8l5Y00D2+t9ix5+GDalKuxwRIHnohw606QXsDXwR6Ac87Zx7xnv/1roG8Zh/9bp57z2VlcX7\nKcWKigr1r4B1tH/1d1xL2Hk3+h92TA9U1XWK+edXzH1bxzk3OWe2xntfk7s+aegvAIblzA8jOtrP\nNY/o5O1qYLVz7nFgT+CT0I+Lyi1sUl1dXcLS8ldlZSXqX+HqSP+yzz9BmPUSmQsvK7h/i2L++RVz\n3yDqn/d+cnttkob+dGCEc244UAscD5zQos19wJT4pG9vouGfyxLuVyRvhcULCXdeT+YHk7A+/dIu\nR+QzEp3I9d43AWcDjwCzgbu896855yY65ybGbV4H/g68AjwL3OC9n52sbJH8FJqayN74O+yI47Dt\ndkq7HJH1WAgdGZbvcaG2tjbtGrpNKbzFLNX+Zf96O+H9OWTO+QWWKczPPhbzz6+Y+wZQVVUF0O79\nPQrzt1IkD4W3ZhOenEbmlB8UbOBL8dNvpkgXCPWryP7xMjLfOgsbOCjtckTapNAX6QLhzuuwkXtj\ne45NuxSRdin0RRLKPvu/hLlvYRNOS7sUkQ1S6IskEJYsItx1I5nTf6LbJUtBUOiLdFLINpO96XLs\nsPG6PFMKhkJfpJPCI38FwL58bMqViHScQl+kE8J7bxOm3Ufm1B9jmbK0yxHpMIW+yEYKaxqiT90e\nfzo2eIu0yxHZKAp9kY20+o7rsWHbk9n34LRLEdloCn2RjRBmvkDj9CexE89IuxSRTlHoi3RQqFtB\n9k+/p98Z52P9N0m7HJFOUeiLdEAIgeztV2NjDqTXyL3TLkek0xT6Ih0QnvoXfFiLHfuttEsRSUSh\nL7IBYfFCwt23kDn9x1ivirTLEUlEoS/SjtDcTPaPl2FHfB3bZvu0yxFJTKEv0o7w8J+hojf2pfFp\nlyLSJRT6Im0Ib79OeOwhMt/5ob4URYqGfpNFWhEa6qMvRTnxDGzQ4LTLEekyCn2RVoSpN2I7746N\n3j/tUkS6lEJfpIXw4lOEN2di3/hu2qWIdDmFvkiOsGwJ2duvib4UpU+/tMsR6XIKfZFYyGbJ3nwF\ndshR2A67pF2OSLcoT7oB59w44AqgDLjRe39JG+3GAE8Dznv/l6T7FelqYdq9sHYNduSEtEsR6TaJ\njvSdc2XAFGAcsBtwgnNu1zbaXQL8HbAk+xTpDuHdNwmP/JXMd8/FyvSlKFK8kg7vjAXmeO/neu8b\ngalAa59iOQe4G1iccH8iXS6srid7w6XR5ZmDh6Rdjki3Shr6Q4F5OfPz42WfcM4NJfpDcE28KCTc\np0iXCSEQbr8G23UvXZ4pJSHpmH5HAvwK4Kfe++CcM1oZ3nHOVQPV6+a991RWViYsLX9VVFSof3li\nTc3fWVP7HpUXXYP17tOh1xRS/zqjmPtXzH1bxzk3OWe2xntfk7s+aegvAIblzA8jOtrPNRqY6pwD\n2Bw4wjnX6L2/f12DuKjcwibV1dUlLC1/VVZWov6lLyycH12eee5FrFzbCGsbO/S6QulfZxVz/4q5\nbxD1z3s/ub02SUN/OjDCOTccqAWOB07IbeC932Hdc+fczcADuYEvkobQ2Ej2hkux8SdiQ7dLuxyR\nHpNoTN973wScDTwCzAbu8t6/5pyb6Jyb2BUFinSHcM8tMHgIdvC4tEsR6VEWQl6eVw21tbVp19Bt\nSuEtZj73L7zwJNk/30zmwsux/hs/vpvv/UuqmPtXzH0DqKqqgg1cFq9P5EpJCQsXROP43z+/U4Ev\nUugU+lIywpo1ZK+9GPvqSdjwEWmXI5IKhb6UhBAC4X+uwYZtjx305bTLEUmNQl9KQvi/aYT35mAn\nnYmZ7gQipUuhL0UvvP824S+3kvn+Tzv8ASyRYqXQl6IW6leSvfYS7JsTsa23SbsckdQp9KVohWxz\n9AGsUWPIjDkw7XJE8oJCX4pWuOdWaGrCjvtO2qWI5A2FvhSl7FOPEmY8TWbieVh54u8KEikaCn0p\nOuHt1wl330zmrP/ANhmQdjkieUWhL0UlLFtC9tqLyZx8DjZ027TLEck7Cn0pGmHtGrJXX4QdejS2\n59i0yxHJSwp9KQohBMKfpmBbVmHjvp52OSJ5S6EvRSHcezth8QfYyefoE7ci7VDoS8HLPvYwYfqT\nZM65EKvonXY5InlN17JJQQsvPkV42JM572KscmDa5YjkPR3pS8EKb86K7o1/zoXYFlulXY5IQVDo\nS0EKC96PLs08/cfYtjumXY5IwVDoS8EJSxeTvWoy5k7Fdvt82uWIFBSFvhSUsGIZ2csnYYceTWa/\nQ9IuR6TgKPSlYIQVy8he+h/YmAPJfPlraZcjUpAU+lIQwoqPyf7uQmz0AWSOOSHtckQKlkJf8l6o\nW072sguxz++HKfBFEkl8nb5zbhxwBVAG3Oi9v6TF+hOB8wAD6oAzvPevJN2vlIZQtyIK/FFjsPEn\n6tO2IgklOtJ3zpUBU4BxwG7ACc65XVs0ewc4yHs/CvgVcH2SfUrpCCtXkL38QmzkaOzYbynwRbpA\n0iP9scAc7/1cAOfcVGA88Nq6Bt77p3PaPwvoi0plg8KSRWSvmBQN6Rz7bQW+SBdJOqY/FJiXMz8/\nXtaW04CHE+5TilyY/y7Zi8/HDh5H5msnK/BFulDSI/3Q0YbOuUOAU4EDWllXDVSvm/feU1lZmbC0\n/FVRUaH+taFx1kvUX/lL+p1yDhX7H9rFlXUN/fwKVzH3bR3n3OSc2RrvfU3u+qShvwAYljM/jOho\nv2URo4AbgHHe+2Ut18dF5RY2qa6uLmFp+auyshL1b33hhSfJ/s+1ZL57Lmt23ZM1efpvpJ9f4Srm\nvkHUP+/95PbaJA396cAI59xwoBY4HvjMNXXOuW2BvwAnee/nJNyfFKnsow8S/nY3mR/+Ett2h7TL\nESlaicb0vfdNwNnAI8Bs4C7v/WvOuYnOuYlxs18Ag4BrnHMznHPPJapYikpobCR76xRCzd+i2yMr\n8EW6lYXQ4WH5nhRqa2vTrqHblMJbzI70Lyz9iOy1F8Ogzcl859+xPv16oLrk9PMrXMXcN4CqqiqI\nPhPVJn2JiqQivPEq2Rt+h33xK9i4r+kKHZEeotCXHhVCIPzzfsLf7yFz2o90a2SRHqbQlx4TVq4g\ne/sfYPFCMj/7Lbb5lmmXJFJyFPrSI8Irz5O97WpszIHYqT/SF5iLpEShL90qNNQT/E2E2S+ROf0n\n2C57pF2SSElT6Eu3CW/OJHvzldgue5CZdBXWtzCuzhEpZgp96XLZuuVkb7uW8PJzZE46A9tr37RL\nEpGYQl+6TMg2Ex5/hLoHpsI+XyDzyylY/03SLktEcij0pUuEObPJ3nEd9O1H5QWXUr/ZkLRLEpFW\nKPQlkbColnD/nYQ3Z2HHnYKNOZCyAQOgiD/1KFLIFPrSKWHRB4SHPOGV57FDjyZz0plYn75plyUi\nG6DQl40SFi+Mwv7lZ7FDjiJz0bVYP43bixQKhb50SJj7FuHRBwmvTseqjyTz6+t0klakACn0pU2h\nqZHwwlOExx6Cj5di1UeQOf50rH9xf/OQSDFT6Mt6wpJFhCf/RXj8EdhqKJnDj4U9x2JlZWmXJiIJ\nKfQFgLBqJeGFJwnP1sCC97F9DiDzw8nYNsPTLk1EupBCv4SFhtUwawbZ5/4XXnsZdt2LzJfGw8jR\nWK9eaZcnIt1AoV9iwpLFhFeeI7z8HLz9OuywC7bPF7CTz9FVOCIlQKFf5EJDPbz1GuGNVwmzZsDH\nH2Ej9yFz4OEw8XzdBE2kxCj0i0xYtRLefYPw5kzC669C7fuw3U7RnS6/ORF23AXL6ISsSKlS6Bew\nsGYNzHuHMPdNeHdO9LjiY9h2R2znkWS+fjJsv7O+sEREPqHQLwChqQkWfwAL3iPEEwveg2VLoGpb\nbPsRsNueZI6cAFsP1ZG8iLRJoZ8nwup6WLoYliwiLPoAFtXGjx/Aso9g0OYwdDg2dFtszEHYV7eF\nIVVYuX6EItJxiRPDOTcOuAIoA2703l/SSpurgCOAeuAU7/2MpPstFKGpEepWwPKlsHwZYflSGhrq\nyS5aSFi2JA76xdDcBIOHwGabY0O2hiFbkxk5GoZUweZDsHJdQikiySUKfedcGTAF+BKwAHjeOXe/\n9/61nDZHAjt570c45/YFrgH2S7LfnhaamqChHhpWR4+rV8PqVYT6lbBqFdSvjKZVKwmr6qBuOaxc\nEU1r18AmA2DgIBi4GTZwEGGLrWDo8CjUN9sCBm8B/TbBzNLuqogUuaRH+mOBOd77uQDOuanAeOC1\nnDbHAH8C8N4/65zb1Dm3pff+w/Y2HEKAkIXmbHQUnG2OnmeboakpWtbc/OljU2P0fN26xkZCcxM0\nro3WNTbGj2th7dqcxzWwdi1h7ZoooNc0fDqtbYCGhmifffpB337Qp2/82C+6rr1/f+i3STT8ss1w\nMpsMiEJ+kwFQOQD69l8vzPtWVtKk+82LSAqShv5QYF7O/Hyg5ReittZmG6Dd0M9+bzxYBsoykCmD\nsniyDJT3iufLP11e3gvKy+N15VBejpWVQ68K6NUrWt6rInqs6B2FckUF9OoNFRVkelVA7z6fThW9\no4Cv6AMVFToKF5GikDT0QwfbtUzMz7zOOVcNVK+b994z8M5HizZoKyoqqKws3jtVqn+FrZj7V8x9\nW8c5NzlntsZ7X5O7PmnoLwCG5cwPIzqSb6/NNvGyT8RF5RY2aeXKlQlLy1+VlZXUFfHwjvpX2Iq5\nf8XcN4j6572f3F6bpKE/HRjhnBsO1ALHAye0aHM/cDYw1Tm3H/DxhsbzRUSke2SSvNh730QU6I8A\ns4G7vPevOecmOucmxm0eBt5xzs0BrgPOTFiziIh0koXQ0WH5HhVqa2vTrqHblMJbTPWvcBVz/4q5\nbwBVVVWw/jnUz0h0pC8iIoVFoS8iUkIU+iIiJUShLyJSQhT6IiIlRKEvIlJCFPoiIiVEoS8iUkIU\n+iIiJURwbe0aAAAFIklEQVShLyJSQhT6IiIlRKEvIlJCFPoiIiVEoS8iUkIU+iIiJUShLyJSQhT6\nIiIlRKEvIlJCFPoiIiVEoS8iUkIU+iIiJUShLyJSQso7+0Ln3GbAXcB2wFzAee8/btFmGHArMAQI\nwPXe+6s6Xa2IiCSS5Ej/p8A07/3OwL/i+ZYagR9573cH9gPOcs7tmmCfIiKSQJLQPwb4U/z8T8BX\nWzbw3i/03r8UP18JvAZUJdiniIgkkCT0t/Tefxg//xDYsr3GzrnhwOeBZxPsU0REEmh3TN85Nw3Y\nqpVVF+TOeO+Dcy60s51NgLuBH8RH/CIikgILoc2sbpdz7nWg2nu/0Dm3NfCY9/5zrbTrBTwI/M17\nf0Ub26oGqtfNe+8ndaooEZES55z7Zc5sjfe+5jMNQgidmiZMmPDfEyZMOD9+/tMJEyZc3EobmzBh\nwq0TJky4fCO3PbmzdRXCpP4V9qT+Fe5UzH3raP86fckmcDHgnXOnEV+yGf+VqQJu8N4fBRwAnAS8\n4pybEb/uZ977vyfYr4iIdFKnQ997vxT4UivLa4Gj4uf/hz4AJiKSN/I1kGvSLqCb1aRdQDerSbuA\nblaTdgHdrCbtArpRTdoFdLOaDTXo9IlcEREpPPl6pC8iIt1AoS8iUkKSXL3T7Zxz5wBnAs3AQ977\n81Muqcs5534C/BbYPD45XhScc78FjgbWAm8D3/HeL0+3qmScc+OAK4Ay4Ebv/SUpl9RlSuXmiM65\nMmA6MN97/5W06+lKzrlNgRuB3Yl+hqd6759p2S5vj/Sdc4cQ3d9nlPd+JHBpyiV1ufg/2mHAe2nX\n0g3+Aezuvd8TeBP4Wcr1JBKHxRRgHLAbcEKR3TywVG6O+ANgNlEoFpsrgYe997sCo4judbaefD7S\nPwP4jfe+EcB7vzjlerrDZcB5wH1pF9LVvPfTcmafBb6eVi1dZCwwx3s/F8A5NxUYTxv/sQqN934h\nsDB+vtI5t+7miEXRPwDn3DbAkcBFwI9TLqdLOecGAgd6708G8N43Aa2+s87n0B8BHOSc+y+gATjX\nez895Zq6jHNuPNFbzFecc2mX091OBe5Mu4iEhgLzcubnA/umVEu3KuKbI14O/D9gQNqFdIPtgcXO\nuZuBPYEXiO51Vt+yYaqhv4EbupUDg7z3+znnxgAe2KEn60tqA/37GXB4zjLrkaK6UDv9+7n3/oG4\nzQXAWu/9HT1aXNcrxuGA9RTrzRGdc0cDi7z3M+J7fRWbcmBv4Gzv/fPOuSuIvuPkF601TI33/rC2\n1jnnzgD+Erd73jmXdc4N9t4v6bECE2qrf865kUR/mV+Oj/K3AV5wzo313i/qwRITae/nB+CcO4Xo\n7fQXe6Sg7rUAGJYzP4zoaL9oxDdHvAe43Xt/b9r1dLH9gWOcc0cCfYABzrlbvfffTrmurjKfaOTg\n+Xj+blr/Yqu8Ht65FzgU+F/n3M5ARSEFfnu89zPJ+f4B59y7wOgiu3pnHNFb6YO99w1p19MFpgMj\n4qGPWuB44IRUK+pCzjkD/gjMbutuuIXMe/9z4OcAzrmDiYaLiyXwie92PM85t7P3/k2iW+TMaq1t\nPof+TcBNzrlXiS77K5ofUCuKcejg90AFMC1+N/O09/7MdEvqPO99k3PubOARoks2/+i9L5qTnJTe\nzRGL8f/cOcD/OOcqiC+Tbq2RbsMgIlJC8vY6fRER6XoKfRGREqLQFxEpIQp9EZESotAXESkhCn0R\nkRKi0BcRKSEKfRGREvL/Aat2RdDSi7BtAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7fb4fcd29be0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"y = F.sigmoid(x)\n",
"plt.title(\"sigmoid function.\")\n",
"plt.ylim(-.2, 1.2)\n",
"plt.xlim(-6, 6)\n",
"plt.plot(x.data, y.data)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 損失関数(コスト関数)\n",
"\n",
"* 交差エントロピー誤差(cross entropy)\n",
"* ソフトマックス関数(soft max)"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[1 2 3 4]]\n",
"[[ 0.0320586 0.08714432 0.23688284 0.64391428]]\n"
]
}
],
"source": [
"print(Variable(np.array([[1, 2, 3, 4]])).data)\n",
"print(F.softmax(Variable(np.array([[1, 2, 3, 4]]).astype(np.float32))).data)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 実装例"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# パラメーター\n",
"batchsize = 100\n",
"trainsize = 6000\n",
"testsize = 1000\n",
"\n",
"n_epoch = 20\n",
"n_units = 1000"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# データ準備\n",
"mnist = fetch_mldata('MNIST original')\n",
"mnist['data'] = mnist['data'][:trainsize + testsize]\n",
"x_all = mnist['data'].astype(np.float32) / 255\n",
"y_all = mnist['target'].astype(np.int32)\n",
"x_train, x_test = np.split(x_all, [trainsize])\n",
"y_train, y_test = np.split(y_all, [trainsize])"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# モデル定義\n",
"model = FunctionSet(\n",
" l1=F.Linear(784, n_units),\n",
" l2=F.Linear(n_units, n_units),\n",
" l3=F.Linear(n_units, 10)\n",
")\n",
"\n",
"# 最適化手法\n",
"optimizer = optimizers.SGD()\n",
"optimizer.setup(model)"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# 計算定義(入力層から出力層への順伝播, 出力と正解の誤差)\n",
"def forward(x_data, y_data):\n",
" x = Variable(x_data)\n",
" t = Variable(y_data)\n",
" h1 = F.dropout(F.relu(model.l1(x)))\n",
" h2 = F.dropout(F.relu(model.l2(h1)))\n",
" y = model.l3(h2)\n",
" return F.softmax_cross_entropy(y, t), F.accuracy(y, t)"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"collapsed": false,
"scrolled": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 0\n",
"epoch 1\n",
"epoch 2\n",
"epoch 3\n",
"epoch 4\n",
"epoch 5\n",
"epoch 6\n",
"epoch 7\n",
"epoch 8\n",
"epoch 9\n",
"epoch 10\n",
"epoch 11\n",
"epoch 12\n",
"epoch 13\n",
"epoch 14\n",
"epoch 15\n",
"epoch 16\n",
"epoch 17\n",
"epoch 18\n",
"epoch 19\n"
]
}
],
"source": [
"# 学習\n",
"for epoch in range(n_epoch):\n",
" print('epoch %d' % epoch)\n",
" indexes = np.random.permutation(trainsize)\n",
" for i in range(0, trainsize, batchsize):\n",
" x_batch = x_train[indexes[i : i + batchsize]]\n",
" y_batch = y_train[indexes[i : i + batchsize]]\n",
"\n",
" optimizer.zero_grads()\n",
" loss, accuracy = forward(x_batch, y_batch)\n",
" loss.backward()\n",
" optimizer.update()"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"mean loss: 0.693561\n",
"mean accuracy: 0.800000\n"
]
}
],
"source": [
"# モデルの評価\n",
"sum_loss, sum_accuracy = 0, 0\n",
"for i in range(0, testsize, batchsize):\n",
" x_batch = x_test[i : i + batchsize]\n",
" y_batch = y_test[i : i + batchsize]\n",
" loss, accuracy = forward(x_batch, y_batch)\n",
" sum_loss += loss.data * batchsize\n",
" sum_accuracy += accuracy.data * batchsize\n",
"\n",
"mean_loss = sum_loss / testsize\n",
"mean_accuracy = sum_accuracy / testsize\n",
"\n",
"print(\"mean loss: %f\" % mean_loss)\n",
"print(\"mean accuracy: %f\" % mean_accuracy)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Spark "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## ロジスティック回帰によるスパム判定"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"from pyspark import SparkContext\n",
"from pyspark.mllib.regression import LabeledPoint\n",
"from pyspark.mllib.classification import LogisticRegressionWithSGD\n",
"from pyspark.mllib.feature import HashingTF"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"def spark_simulation():\n",
" # データ準備\n",
" spam = sc.textFile(\"files/spam.txt\")\n",
" ham = sc.textFile(\"files/ham.txt\")\n",
"\n",
" tf = HashingTF(numFeatures=100)\n",
" spam_features = spam.map(lambda email: tf.transform(email.split(\" \")))\n",
" ham_features = ham.map(lambda email: tf.transform(email.split(\" \")))\n",
"\n",
" positive_examples = spam_features.map(lambda features: LabeledPoint(1, features))\n",
" negative_examples = ham_features.map(lambda features: LabeledPoint(0, features))\n",
" training_data = positive_examples.union(negative_examples)\n",
"\n",
" training_data.cache()\n",
"\n",
" # モデル定義, 学習\n",
" model = LogisticRegressionWithSGD.train(training_data)\n",
"\n",
" # モデルの評価\n",
" pos_test_example = tf.transform(\"O M G GET cheap stuff by sending money to ...\".split(\" \"))\n",
" neg_test_example = tf.transform(\"Hi Dad, I started studying Spark the other ...\".split(\" \"))\n",
"\n",
" return model.predict(pos_test_example), model.predict(neg_test_example)"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Prediction for positive test example: 1\n",
"Prediction for negative test example: 0\n"
]
}
],
"source": [
"def spark_run():\n",
" pos_test_predict, neg_test_predict = spark_simulation()\n",
"\n",
" print(\"Prediction for positive test example: %g\" % pos_test_predict)\n",
" print(\"Prediction for negative test example: %g\" % neg_test_predict)\n",
"\n",
"spark_run()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 付録\n",
"\n",
"* Jupyterのインストール手順\n",
"* Chainerのインストール手順\n",
"* Sparkのインストール手順\n",
"* 参考資料"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Jupyterのインストール手順"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Python\n",
"\n",
"JupyterはPythonで書かれています.\n",
"Windows環境向けにPython処理系をインストールするのであれば, NumPyやSciPy, matplotlib, scikit-learnなどがバンドルされている[Anaconda](https://www.continuum.io/downloads#_windows)をインストールしましょう.\n",
"サイトからインストーラーをダウンロードして実行すると, インストール画面が開きます.\n",
"\n",
"<span></span> \n",
"\n",
"#### 注意\n",
"\n",
"\"Anaconda for Windows\"には, Windows環境向けのインストーラーが4つあります.\n",
"**Python 3.5** の **Windows 64bit** を選びましょう.\n",
"\n",
"<span></span> \n",
"\n",
"※本資料のサンプルコードはPython 3.4で動作を確かめていますが, 3.5系でも動くと思います."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Jupyter\n",
"\n",
"インストーラーが終了したら, コマンドプロンプトを立ち上げて, Pythonが使えるようになっているか確かめましょう. \n",
"\n",
"<pre>\n",
"C:¥Windows>python --version\n",
"Python 3.4.3 :: Anaconda 2.3.0 (64-bit)\n",
"</pre>\n",
"\n",
"といった出力がされれば大丈夫です. \n",
"つづいて, そのコマンドプロンプトで, 以下の2つのコマンドを実行すると, Jupyterをインストールできます.\n",
"\n",
"<pre>\n",
"C:¥Windows>conda update conda\n",
"C:¥Windows>conda update ipython ipython-notebook ipython-qtconsole\n",
"</pre>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### 補足\n",
"\n",
"上記コマンドの意味:\n",
"condaは, Anacondaのパッケージ管理コマンドです.\n",
"\"conda update\"でパッケージをアップデートできます. \n",
"「Jupyterのインストールなのにアップデート?」と思うかも知れません. \n",
"Jupyterは, 以前IPythonと呼ばれていました.\n",
"AnacondaにはIPythonがバンドルされており, これをアップデートするとJupyterに変わります."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### 起動\n",
"\n",
"コマンドプロンプトを立ち上げて, 以下のコマンドを実行すると, Jupter notebookが立ち上がります.\n",
"(デフォルト設定では, このタイミングで自動的にブラウザーが開き, Jupyter notebookのトップページが表示されます).\n",
"\n",
"<pre>\n",
"C:¥Windows>jupyter notebook\n",
"</pre>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Chainerのインストール手順"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"ChainerはPythonで書かれており, Pythonの標準的な手順でインストールできます.\n",
"\n",
"<pre>\n",
"C:¥Windows>pip install chainer\n",
"</pre>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Sparkのインストール手順"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Java\n",
"\n",
"SparkはJavaVM上で動きます.\n",
"バージョン6以降のJavaがインストールされていない場合は, [Oracle](http://www.oracle.com/technetwork/java/javase/downloads/index.html)のサイトから適当なバージョンを選びましょう.\n",
"(今ならばJDK8でよいでしょう).\n",
"サイトからインストーラーをダウンロードして実行すると, インストール画面が開きます."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"インストーラーが終了したら, ユーザーの環境変数 **JAVA_HOME** と **PATH** を設定しましょう.\n",
"\n",
"* JAVA_HOME: C:¥Program Files¥Java¥jdk1.8.0\n",
"* PATH: %JAVA_HOME%¥bin (既存の設定値に追加する)\n",
"\n",
"※C:¥Program Files¥Java¥jdk1.8.0 にインストールした場合\n",
"\n",
"つづいて, コマンドプロンプトを立ち上げて, Javaが使えるようになっているか確かめましょう. \n",
"\n",
"<pre>\n",
"C:¥Windows>java -version\n",
"java version \"1.8.0_65\"\n",
"Java(TM) SE Runtime Environment (build 1.8.0_65-b17)\n",
"Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)\n",
"</pre>\n",
"\n",
"といった出力がされれば大丈夫です."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### 補足\n",
"\n",
"仕事用の環境を変更したくない(できない)場合は, Sparkを実行するためのバッチファイルを作るのがよいでしょう.\n",
"以下の内容を書いたバッチファイルを保存して実行すると, コマンドプロンプトが立ち上がります.\n",
"このようにすると, そのコマンドプロンプトで実行したプログラムにのみ環境変数を与えることができます.\n",
"\n",
"<pre>\n",
"set JAVA_HOME=C:¥Program Files¥Java¥jdk1.8.0\n",
"set PATH=%JAVA_HOME%¥bin;%PATH%\n",
"start \"Spark Shell environment\"\n",
"</pre>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Hadoop\n",
"\n",
"SparkはHadoopが提供するHDFS APIとYARN APIに依存しています.\n",
"そのため, Hadoopを利用しない場合でもインストールする必要があります.\n",
"Hadoopがインストールされていない場合は, [Hadoop](https://hadoop.apache.org/releases.html)のサイトから適当なバージョンを選びましょう.\n",
"(今ならば2.7.1でよいでしょう).\n",
"サイトからbinaryのtarballをダウンロードして, tarbalを解凍しましょう.\n",
"tarballを解凍するツールがない場合は, コマンドプロンプトを立ち上げて, 以下のコマンドを実行します.\n",
"\n",
"<pre>\n",
"C:¥Users¥tommy¥Downloads>python -c 'import tarfile; tar = tarfile.open(\"hadoop-2.7.1.tar.gz\"); tar.extractall(); tar.close()'\n",
"</pre>\n",
"\n",
"※ファイル名はダウンロードしたものにあわせて適宜変えてください.\n",
"(ファイルを保存したディレクトリーで実行するか, 絶対パスを指定します)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"tarballを解凍したら, ユーザーの環境変数 **HADOOP_HOME** と **PATH** を設定しましょう.\n",
"\n",
"* HADOOP_HOME: C:¥hadoop-2.7.1\n",
"* PATH: %HADOOP_HOME%¥bin (既存の設定値に追加する)\n",
"\n",
"※C:¥hadoop-2.7.1 に解凍した場合"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"つづいて, winutils.exeとhadoop.dllを入手します.\n",
"(本来は自身の環境でビルドするのが好ましいですが, ビルド手順が煩雑なので, 手軽な野良ビルドを採用します).\n",
"[このページ](http://hadoop-on-windows.blogspot.jp/2015/05/hadoop-270windows.html)からwinutils.exeとhadoop.dllをダウンロードして, %HADOOP_HOME%¥bin に保存します.\n",
"\n",
"また, winutils.exeを動かすにはVisual C++ 2010 再頒布可能パッケージが必要です.\n",
"(Visual C++ 2010でビルドされたものなので).\n",
"[Microsoft](http://www.microsoft.com/ja-jp/download/details.aspx?id=14632)のサイトからインストーラーをダウンロードして, インストールします.\n",
"\n",
"<span></span> \n",
"\n",
"#### 補足\n",
"\n",
"winutils.exeは, Hadoop内部で使っているUNIXコマンドをWindows環境でエミュレーションするプログラムです."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Spark\n",
"\n",
"ここまで準備が済んだら, あと一息です.\n",
"\n",
"[Spark]()のサイトから適当なバージョンを選びましょう.\n",
"(今ならば1.5.2でよいでしょう).\n",
"パッケージタイプは ** Pre-build with user-provided Hadoop [can use with most Hadoop distributions]** を選んでください.\n",
"サイトからtarballをダウンロードして, tarballを解凍しましょう."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"tarballを解凍したら, ユーザーの環境変数 **SPARK_HOME** と **PATH** , **SPARK_DIST_CLASSPATH** を設定しましょう.\n",
"\n",
"* SPARK_HOME: C:¥spark-1.5.2\n",
"* PATH: %SPARK_HOME%¥bin (既存の設定値に追加する)\n",
"* SPARK_DIST_CLASSPATH: (hadoop classpath(下記を参照)の実行結果をコピー, ペーストする)\n",
"\n",
"※C:¥spark-1.5.2 に解凍した場合\n",
"\n",
"<pre>\n",
"C:¥Windows>hadoop classpath\n",
"C:¥hadoop-2.7.1¥etc¥hadoop;C:¥hadoop-2.7.1...(以下略)\n",
"</pre>\n",
"\n",
"※コマンドの出力結果はHadoopのインストール状況によって異なります."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### 起動\n",
"\n",
"コマンドプロンプトを立ち上げて, 以下のコマンドを実行すると, Sparkのシェルプログラムが立ち上がります.\n",
"\n",
"<pre style=\"font-family: Consolas, 'Courier New', Courier, Monaco, monospace;\">\n",
"C:¥Windows>pyspark\n",
"(...中略...)\n",
"Welcome to\n",
" ____ __\n",
" / __/__ ___ _____/ /__\n",
" _\\ \\/ _ \\/ _ `/ __/ '_/\n",
" /__ / .__/\\_,_/_/ /_/\\_\\ version 1.5.2\n",
" /_/\n",
"\n",
">>> sc\n",
"&lt;pyspark.context.SparkContext at 0x7ff874140ba8&gt;\n",
"</pre>\n",
"\n",
"※scは, Sparkのシェルプログラムが生成したSparkContextオブジェクトです.\n",
"SparkContextを通じてSparkの分散処理エンジンに処理を渡し, 実行してもらいます."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### Jupyterとの連携\n",
"\n",
"Sparkのシェルプログラムは, Jupyter(とJupyter notebook)に対応しています.\n",
"環境変数を設定することで, Jupyterと連携できます.\n",
"\n",
"* PYSPARK_PYTHON: C:¥anaconda3¥bin¥python\n",
"* PYSPARK_DRIVER_PYTHON: C:¥anaconda3¥bin¥jupyter\n",
"* PYSPARK_DRIVER_PYTHON_OPTS: notebook (Jupyter notebookと連携する場合のみ)\n",
"\n",
"※Anacondaを C:¥anaconda3 にインストールした場合\n",
"\n",
"ユーザーの環境変数に設定するか, コマンドプロンプトで設定したのち, Sparkのシェルプログラムを立ち上げると, Jupyter(あるいはJupyter notebook)が立ち上がります."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Scala\n",
"\n",
"**このセクションは, ScalaでSparkを利用したい人向けの追加手順です.**\n",
"\n",
"SparkにはScala用のシェルプログラムもあります.\n",
"また, JupyterでScalaプログラムを書いて, Sparkと連携することも可能です.\n",
"\n",
"[Scala](http://www.scala-lang.org/download/all.html)のサイトからインストーラーをダウンロードして実行すると, インストール画面が開きます.\n",
"\n",
"<span></span> \n",
"\n",
"#### 注意\n",
"\n",
"ScalaでSparkを利用する場合, **2.10系** を選ぶのが無難です.\n",
"binary版のSparkは, 2.10系をターゲットにビルドされているため, 2.11系でScalaプログラムをコンパイルしようとすると, コンパイルエラーが発生します.\n",
"(すでにインストールされているなどの理由で)2.11系を使う場合は, Sparkを再ビルドする必要があります."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"インストーラーが終了したら, コマンドプロンプトを立ち上げて, Scalaが使えるようになっているか確かめましょう. \n",
"\n",
"<pre>\n",
"C:¥Windows>scala -version\n",
"Scala code runner version 2.10.6 -- Copyright 2002-2013, LAMP/EPFL\n",
"</pre>\n",
"\n",
"といった出力がされれば大丈夫です."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"つづいて, sbtをインストールします.\n",
"[sbt](http://www.scala-sbt.org/download.html)のサイトからインストーラーをダウンロードして実行すると, インストール画面が開きます.\n",
"\n",
"<span></span> \n",
"\n",
"#### 補足\n",
"\n",
"sbtは, ビルドシステムです.\n",
"Scalaプログラムをコンパイルして, Sparkに処理を投入できるJARファイルを作るときに使います."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"インストーラーが終了したら, コマンドプロンプトを立ち上げて, sbtが使えるようになっているか確かめましょう. \n",
"\n",
"<pre>\n",
"C:¥Windows>set http.proxyHost=<プロクシーサーバー名>\n",
"C:¥Windows>set http.proxyPort=<プロクシーポート>\n",
"C:¥Windows>set https.proxyHost=<プロクシーサーバー名>\n",
"C:¥Windows>set https.proxyPort=<プロクシーポート>\n",
"C:¥Windows>sbt --version\n",
"sbt launcher version 0.13.8\n",
"</pre>\n",
"\n",
"といった出力がされれば大丈夫です.\n",
"\n",
"※sbtを使う都度, プロクシーを設定するのが手間であれば, ユーザーの環境変数を設定するか, Javaのインストール手順で説明したバッチファイルを作るのがよいでしょう."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### Sparkのシェルプログラム\n",
"\n",
"コマンドプロンプトを立ち上げて, 以下のコマンドを実行すると, Sparkのシェルプログラムが立ち上がります.\n",
"\n",
"<pre style=\"font-family: Consolas, 'Courier New', Courier, Monaco, monospace;\">\n",
"C:¥Windows>spark-shell\n",
"(...中略..)\n",
"Welcome to\n",
" ____ __\n",
" / __/__ ___ _____/ /__\n",
" _\\ \\/ _ \\/ _ `/ __/ '_/\n",
" /___/ .__/\\_,_/_/ /_/\\_\\ version 1.5.2\n",
" /_/\n",
"\n",
"Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_65)\n",
"Type in expressions to have them evaluated.\n",
"Type :help for more information.\n",
"(...中略..)\n",
"scala> \n",
"</pre>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"#### Jupyterとの連携\n",
"\n",
"Jupyterの対話的実行環境は, Python以外のプログラム言語に対応するための機構(カーネル)を持っています.\n",
"現時点で多くのプログラム言語のカーネルがあり, Scalaのカーネルも[公式ドキュメントのリスト](http://jupyter-notebook.readthedocs.org/en/latest/examples/Notebook/rstversions/What%20is%20the%20Jupyter%20Notebook.html#kernels)に載っています.\n",
"\n",
"しかし, リストに載っているカーネルは, Sparkと連携させづらいので, ここでは[spark-kernel](https://github.com/ibm-et/spark-kernel)を採用します."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"spark-kernelの[ソースコード](https://github.com/ibm-et/spark-kernel/archive/master.zip)をダウンロードしてZIPファイルを解凍するか, あるいはgit cloneします.\n",
"\n",
"つづいて, [usejavacp問題](https://github.com/ibm-et/spark-kernel/issues/132#issuecomment-148652255)に対応するため, scala-interpreter¥src¥main¥scala¥com¥ibm¥spark¥kernel¥interpreter¥scala¥scalainterpreter.scala の206行目に \"this.settings.usejavacp.value = true\" を付け加えます."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"ファイルを編集したら, sbtを使ってビルドして, カーネルをパッケージングします.\n",
"\n",
"<pre>\n",
"C:¥Users¥tommy¥Downloads>python -c 'import zipfile; zf = zipfile.ZipFile(\"spark-kernel-master.zip\", \"r\"); zf.extractall(); zf.close()'\n",
"(このタイミングで scalainterpreter.scala を編集する)\n",
"C:¥Users¥tommy¥Downloads>cd spark-kernel-master\n",
"C:¥Users¥tommy¥Downloads¥spark-kernel-master>sbt compile\n",
"C:¥Users¥tommy¥Downloads¥spark-kernel-master>sbt pack\n",
"</pre>\n",
"\n",
"パッケージングが終了したら, kernel¥target¥pack フォルダーを適当なところへコピーします.\n",
"(フォルダー名を pack から spark-kernel などへ変えておくのが好ましい)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"カーネルをJupyterに登録するためのJSONファイル(kernel.json)を作ります.\n",
"\n",
"<pre>\n",
"{\n",
" \"display_name\": \"Spark 1.5.2 (Scala 2.10.6)\",\n",
" \"language\": \"scala\",\n",
" \"argv\": [\n",
" \"C:¥spark-kernel¥bin¥sparkkernel\",\n",
" \"--profile\",\n",
" \"{connection_file}\"\n",
" ],\n",
" \"codemirror_mode\": \"scala\"\n",
"}\n",
"</pre>\n",
"\n",
"※C:¥spark-kernel にコピーした場合"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Jupyterの設定フォルダー(%USERHOME%¥.ipython¥kernels)に, spark-kernel用のフォルダー(フォルダー名は任意. カーネル本体と同じフォルダー名にする必要はない. 例: spark)を作り, JSONファイルを置きます.\n",
"\n",
"※この設定手順は, spark-kernelで説明されている[手順](https://github.com/ibm-et/spark-kernel/wiki/Getting-Started-with-the-Spark-Kernel)から少し改変されています.\n",
"(標準的なWindows環境にはmakeコマンドがインストールされていないため)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"カーネルの設定が済んだら, コマンドプロンプトを立ち上げて, Jupyterがカーネルを見つけられているか確かめましょう.\n",
"\n",
"<pre>\n",
"C:¥Windows>jupyter kernelspec list\n",
"Available kernels:\n",
" spark C:¥Users¥tommy¥.ipython¥kernels¥spark\n",
" python3 C:¥anaconda3¥lib¥python3.4¥site-packages¥ipykernel¥resources\n",
"</pre>\n",
"\n",
"といった出力がされれば大丈夫です.\n",
"\n",
"カーネルを見つけられていることを確かめたのち, Sparkのシェルプログラムを立ち上げると, Jupyter(あるいはJupyter notebook)が立ちあがり, Sparkを利用できるようになります."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 参考資料 "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 書籍\n",
"\n",
"<span></span> \n",
"\n",
"<div style=\"display: flex;\">\n",
"<img src=\"http://image.gihyo.co.jp/assets/images/cover/2015/9784774176987.jpg\" style=\"margin: 10px;\" />\n",
"<img src=\"http://image.gihyo.co.jp/assets/images/cover/2015/9784774176314.jpg\" style=\"margin: 10px;\" />\n",
"<img src=\"http://image.gihyo.co.jp/assets/images/cover/2013/9784774158969.jpg\" style=\"margin: 10px;\" />\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## 入手可能なテストデータ\n",
"\n",
"* [MNIST](http://yann.lecun.com/exdb/mnist/) (手書き数字画像データベース)\n",
"* [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml/)\n",
"* [情報学研究データリポジトリ](http://www.nii.ac.jp/dsc/idr/)\n",
"* scikit-learnで[トイデータ](http://scikit-learn.org/stable/datasets/index.html#toy-datasets)や[サンプルデータ](http://scikit-learn.org/stable/datasets/index.html#sample-images)を生成する\n",
"---\n",
"* [朱鷺の杜Wiki](http://ibisforest.org/index.php?DataSet) (さまざまなデータベースのリンク集)"
]
}
],
"metadata": {
"celltoolbar": "Slideshow",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment