-
-
Save takatakamanbou/336496e66a1eb756a0cb2b79c318871d to your computer and use it in GitHub Desktop.
PIP2020-13-note2.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "nbformat": 4, | |
| "nbformat_minor": 0, | |
| "metadata": { | |
| "colab": { | |
| "name": "PIP2020-13-note2.ipynb", | |
| "provenance": [], | |
| "collapsed_sections": [], | |
| "toc_visible": true, | |
| "include_colab_link": true | |
| }, | |
| "kernelspec": { | |
| "name": "python3", | |
| "display_name": "Python 3" | |
| } | |
| }, | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "view-in-github", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<a href=\"https://colab.research.google.com/gist/takatakamanbou/336496e66a1eb756a0cb2b79c318871d/pip2020-13-note2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "NI3mY1FGZWgV", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "# 2020年度パターン情報処理第13回講義資料その2\n", | |
| "\n", | |
| "この科目のウェブサイトへ https://www-tlab.math.ryukoku.ac.jp/wiki/?PIP/2020\n", | |
| "\n", | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "ffaGVK2o5VWe", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## はじめに" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "0IpujRN_Wuib", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### これは何？\n", | |
| "\n", | |
| "これは，Google Colaboratory（以下 Google Colab) という，Google が提供しているサービスを利用して作成した資料です．Notebook と呼びます．クラウド上に仮想的に構築された Linux マシン上で Python のプログラムを実行することができます．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "QRNjWashW4pr", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### Notebook の動かし方\n", | |
| "\n", | |
| "この Notebook では，上の方のセルで作った変数や関数を後のセルで使うことがあります．そのため，上の方のセルを実行せずに下の方を実行すると，エラーになることがあります．上から順に実行していきましょう．\n", | |
| "\n", | |
| "また，メニューの「ランタイム」から，「すべてのセルを実行」したりすることも可能です．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "1jfSXIY6VNxP", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## 準備\n", | |
| "\n", | |
| "以下のセルは，プログラムを実行するための準備を行うためのものです．このセルを実行してから，先へ進みましょう．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "IFYQk7ONZENz", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 科学技術計算のライブラリ NumPy のモジュールを np という名前で使えるようにする\n", | |
| "import numpy as np\n", | |
| "# グラフを描くためのライブラリ matplotlib の pyplot を plt という名前でインポート\n", | |
| "import matplotlib.pyplot as plt\n", | |
| "# matplitlib で3次元グラフを描くためのほげ\n", | |
| "from mpl_toolkits.mplot3d import Axes3D\n", | |
| "# matplotlib のグラフをより美しく便利に描くためのほげ\n", | |
| "import seaborn as sns\n", | |
| "sns.set()\n", | |
| "\n", | |
| "# コンピュータビジョン・画像処理のためのライブラリ OpenCV のモジュール cv2 をインポート\n", | |
| "#import cv2\n", | |
| "# Google Colab 上へ／からのファイルのアップロード／ダウンロードのためのほげ\n", | |
| "#from google.colab import files" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "JRBp_fRf6tEe", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## ★ 最小二乗法による平面のあてはめ\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "aB0gqs7AE79V", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "$\\mathbf{x} = (x_1, x_2, \\dots, x_D)$ と $y$ の関係が \n", | |
| "$$ y = f(x; w_0, w_1, \\dots , w_D) = w_0 + w_1x_1 + w_2x_2 + \\dots + w_Dx_D\n", | |
| "$$\n", | |
| "という式で表せると仮定して，この式の$(D+1)$個のパラメータ $w_0, w_1, \\dots , w_D$ として最適な値を最小二乗法で推定する．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "CwaPywwTFn5m", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "以下では，$D=2$ の例を考える．入力を $(x_1, x_2)$, 出力を $y$ として，\n", | |
| "$$\n", | |
| "y = w_0 + w_1x_1 + w_2x_2\n", | |
| "$$\n", | |
| "とする．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "S0mw5aISMBj_", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "ここでは，乱数を使って生成した学習データを使うことにする．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "PVmPGEpsGaDm", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 学習データの個数\n", | |
| "N = 50\n", | |
| "\n", | |
| "# 学習データの次元数\n", | |
| "D = 2\n", | |
| "\n", | |
| "# 乱数を使って学習データを生成\n", | |
| "Xtrain = np.random.rand(N, D)\n", | |
| "Ytrue = 1.0 + Xtrain @ [2.0, 3.0]\n", | |
| "Ytrain = Ytrue + 0.2*np.random.randn(N) # y = 2x_1 + 3x_2 + 1" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "isSxC-yIMPNc", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "まずは学習データを可視化してみよう．\n", | |
| "elevation を変えると上下に視点が移動する．\n", | |
| "azimuth を変えると左右に．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "GKUk73S67HqN", | |
| "colab_type": "code", | |
| "cellView": "both", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "#@title 学習データ {run: \"auto\"}\n", | |
| "fig = plt.figure(facecolor='white', figsize=(10, 7.5))\n", | |
| "ax = fig.add_subplot(111, projection='3d')\n", | |
| "ax.scatter(Xtrain[:, 0], Xtrain[:, 1], Ytrain)\n", | |
| "ax.set_xlim(0, 1)\n", | |
| "ax.set_ylim(0, 1)\n", | |
| "elevation = 30 #@param {type:\"slider\", min:0, max:60, step:10} \n", | |
| "azimuth = 20 #@param {type:\"slider\", min:0, max:90, step:10}\n", | |
| "ax.view_init(elevation, azimuth)\n", | |
| "plt.show()" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "PilpavQGN5Sz", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "最小二乗法で平面を当てはめる．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "FqcEMSi1M30y", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step1**\n", | |
| "\n", | |
| "$N$ 個の学習データ $\\{ (\\mathbf{x}_n, y_n) \\}_{n=1}^{N}$ に対して，次式で表される $(D+1)\\times N$ 行列 $X$ および $N$ 次元ベクトル $Y$ を求める．\n", | |
| "\\begin{align}\n", | |
| "X = \\begin{pmatrix}\n", | |
| "1 & 1 & \\dots & 1 \\\\\n", | |
| "x_{1, 1} & x_{2, 1} & \\dots & x_{N, 1} \\\\\n", | |
| "x_{1, 2} & x_{2, 2} & \\dots & x_{N, 2} \\\\\n", | |
| "\\vdots & \\vdots & \\vdots & \\vdots \\\\\n", | |
| "x_{1, D} & x_{2, D} & \\dots & x_{N, D} \n", | |
| "\\end{pmatrix}\n", | |
| "\\qquad\n", | |
| "Y = (y_1\\ y_2\\ \\dots\\ y_N)\n", | |
| "\\end{align}" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "10kCOm9POUs5", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# X を計算\n", | |
| "X = np.empty((D+1, N))\n", | |
| "X[0, :] = 1\n", | |
| "X[1:, :] = Xtrain.T\n", | |
| "print(X)\n", | |
| "print(X.shape)\n", | |
| "print()\n", | |
| "\n", | |
| "# Y はそのまま\n", | |
| "Y = Ytrain\n", | |
| "print(Y)\n", | |
| "print(Y.shape)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "5wabSPkCO3RR", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step2**\n", | |
| "\n", | |
| "最適なパラメータ $\\widetilde{\\mathbf{w}} = (w_0, w_1, \\dots , w_D)$ （$(D+1)$次元ベクトル）は次式で表される正規方程式の解である．\n", | |
| "\\begin{align}\n", | |
| "X X^{\\top} \\widetilde{\\mathbf{w}} = XY^{\\top}\n", | |
| "\\end{align}\n", | |
| "行列 $A = X X^{\\top}$ とベクトル $\\mathbf{b} = XY^{\\top}$ を計算する．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "V69ssRzcO5po", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# A を計算\n", | |
| "A = X @ X.T\n", | |
| "print(A)\n", | |
| "\n", | |
| "# b を計算\n", | |
| "b = X @ Y\n", | |
| "print(b)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "MZqZTc5zPDQh", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step3**\n", | |
| "\n", | |
| "連立方程式 $ A\\widetilde{\\mathbf{w}} = \\mathbf{b}$ を解いて $\\widetilde{\\mathbf{w}}$ を求める．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "oOJZ2RVvPE_O", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 連立方程式を解く\n", | |
| "wt = np.linalg.solve(A, b)\n", | |
| "print(wt)\n", | |
| "print(wt.shape)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "3qDMv1-3PL2f", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step4**\n", | |
| "\n", | |
| "得られたパラメータを用いて平面を当てはめてみる．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "UETU0OK8C89f", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "x = np.arange(0, 1, 0.1)\n", | |
| "y = np.arange(0, 1, 0.1)\n", | |
| "xx, yy = np.meshgrid(x, y)\n", | |
| "zz = wt[0] + wt[1] * xx + wt[2] * yy" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "W1UsCJl-Peuq", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "#@title 平面当てはめ {run: \"auto\"}\n", | |
| "fig = plt.figure(facecolor='white', figsize=(10, 7.5))\n", | |
| "ax = fig.add_subplot(111, projection='3d')\n", | |
| "ax.scatter(Xtrain[:, 0], Xtrain[:, 1], Ytrain)\n", | |
| "ax.plot_wireframe(xx, yy, zz, color='red')\n", | |
| "ax.set_xlim(0, 1)\n", | |
| "ax.set_ylim(0, 1)\n", | |
| "elevation = 30 #@param {type:\"slider\", min:0, max:60, step:10} \n", | |
| "azimuth = 10 #@param {type:\"slider\", min:0, max:90, step:10}\n", | |
| "ax.view_init(elevation, azimuth)\n", | |
| "plt.show()" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "0A9Wi-Ijd9Jq", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### ★★★ 次のことをやりましょう★★★\n", | |
| "\n", | |
| "上記の elevation と azimuth をいろいろ変えてみて，平面の当てはめができていることを確認する．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "Xx-2MzFHeAfD", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## ★ ゴリゴリ君ふたたび（多項式当てはめ）" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "XE0FwT1QeTFh", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### データの入手と表示" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "PFmCtyoteJE5", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# ゴリゴリ君データの入手\n", | |
| "!wget -nc https://www-tlab.math.ryukoku.ac.jp/~takataka/course/PIP/gorigori-mac.csv\n", | |
| "! ls -l" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "UxlkqQNbeWgd", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# データ先頭5行を表示．数値がカンマで区切られた CSV (Comma-Separated Values)という形式．\n", | |
| "! head -n 5 gorigori-mac.csv" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "vwh_T6W2eYHG", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# データを読み込む\n", | |
| "data = np.loadtxt('gorigori-mac.csv', delimiter = ',')\n", | |
| "print(data.shape) # 配列 data の大きさを表示\n", | |
| "print(data[:5, :]) # 最初の5行を表示" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "VvLmKu8ZeZtO", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 0列目をx軸，1列目をy軸としてグラフを描く\n", | |
| "fig, ax = plt.subplots(facecolor=\"white\", figsize=(8, 6))\n", | |
| "ax.scatter(data[:, 0], data[:, 1], color=\"red\")\n", | |
| "ax.set_xlim(0, 40) # x 軸の範囲指定\n", | |
| "plt.show()" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "Q5wKQDVL0cPr", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### 最小二乗法による多項式のあてはめ\n", | |
| "\n", | |
| "$x$ と $y$ の関係が $D$ 次多項式 $ y = f(x; w_0, w_1, \\dots , w_D) = w_0 + w_1x + w_2x^2 + \\dots +\n", | |
| "w_Dx^D$ で表せると仮定して，この多項式の$(D+1)$個のパラメータ $w_0, w_1, \\dots , w_D$ として最適な値を最小二乗法で推定する．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "yo6vnASYQ5o9", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# あてはめる多項式の次数\n", | |
| "D = 1" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "h1i0zcF81kvu", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step1**\n", | |
| "\n", | |
| "$N$ 個の学習データ $\\{ (x_n, y_n) \\}_{n=1}^{N}$ に対して，次式で表される $(D+1)\\times N$ 行列 $X$ および $N$ 次元ベクトル $Y$ を求める．\n", | |
| "\\begin{align}\n", | |
| "X = \\begin{pmatrix}\n", | |
| "1 & 1 & \\dots & 1 \\\\\n", | |
| "x_{1} & x_{2} & \\dots & x_{N} \\\\\n", | |
| "\\vdots & \\vdots & \\vdots & \\vdots \\\\\n", | |
| "x_{1}^D & x_{2}^D & \\dots & x_{N}^D \n", | |
| "\\end{pmatrix}\n", | |
| "\\qquad\n", | |
| "Y = (y_1\\ y_2\\ \\dots\\ y_N)\n", | |
| "\\end{align}" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "ZeNA11cC3RiC", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# X をつくる\n", | |
| "X = np.empty((D+1, N))\n", | |
| "X[0, :] = 1\n", | |
| "for d in range(1, D+1):\n", | |
| " X[d, :] = data[:, 0]**d\n", | |
| "print(X.shape)\n", | |
| "\n", | |
| "# Y をつくる\n", | |
| "Y = data[:, 1]\n", | |
| "print(Y.shape)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "_kkxsPon4cIo", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step2**\n", | |
| "\n", | |
| "最適なパラメータ $\\widetilde{\\mathbf{w}} = (w_0, w_1, \\dots , w_D)$ （$(D+1)$次元ベクトル）は次式で表される正規方程式の解である．\n", | |
| "\\begin{align}\n", | |
| "X X^{\\top} \\widetilde{\\mathbf{w}} = XY^{\\top}\n", | |
| "\\end{align}\n", | |
| "行列 $A = X X^{\\top}$ とベクトル $\\mathbf{b} = XY^{\\top}$ を計算する．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "v1GbXgcZ5ZwP", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# A を計算\n", | |
| "A = X @ X.T\n", | |
| "print(A)\n", | |
| "\n", | |
| "# b を計算\n", | |
| "b = X @ Y\n", | |
| "print(b)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "4RZvFrND5ojp", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step3**\n", | |
| "\n", | |
| "連立方程式 $ A\\widetilde{\\mathbf{w}} = \\mathbf{b}$ を解いて $\\widetilde{\\mathbf{w}}$ を求める．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "U0kxUZfp55G5", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 連立方程式を解く\n", | |
| "wt = np.linalg.solve(A, b)\n", | |
| "print(wt.shape)\n", | |
| "print(wt)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "sTUBKicM6kdc", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "**Step4**\n", | |
| "\n", | |
| "得られたパラメータを用いて多項式をあてはめ予測してみる．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "U1t_twDb6ySm", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 多項式\n", | |
| "f_est = np.poly1d(wt[::-1])\n", | |
| "\n", | |
| "# グラフを描く\n", | |
| "xr = np.linspace(0, 40, 100)\n", | |
| "fig, ax = plt.subplots(facecolor=\"white\", figsize=(8, 6))\n", | |
| "ax.scatter(data[:, 0], data[:, 1], color=\"red\")\n", | |
| "ax.plot(xr, f_est(xr), '-', label=f\"D = {D}\")\n", | |
| "ax.set_xlim(0, 40) # x 軸の範囲指定\n", | |
| "ax.set_ylim(-5, 125) # y 軸の範囲指定\n", | |
| "ax.legend()\n", | |
| "plt.show()\n", | |
| "\n", | |
| "# 多項式を表示\n", | |
| "print(f_est)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "M6mPN7i0HVUo", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 予測してみる\n", | |
| "xx = np.arange(5, 45, 5)\n", | |
| "yy = f_est(xx)\n", | |
| "for i in range(xx.shape[0]):\n", | |
| " print('気温 {0} 度のときの売上の予測値は {1:.1f} 個ほげ'.format(xx[i], yy[i]))" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "W4TNyCSgefd1", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### ★★★ 次のことをやりましょう★★★\n", | |
| "\n", | |
| "```\n", | |
| "# あてはめる多項式の次数\n", | |
| "D = 1\n", | |
| "```\n", | |
| "上記のように書いてあるセルの D の値をいろいろ変えてそのセルとそれ以降のセルを実行し，結果がどのように変わるか観察する\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "HXUMt-TsY-2B", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## ★ 手書き数字画像の識別\n", | |
| "\n", | |
| "MNIST と呼ばれる手書き数字画像のデータを使って識別の実験をしてみよう．参考: http://yann.lecun.com/exdb/mnist/" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "_EAfO4znc1vS", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### 準備" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "TMq9Hz4LYiOK", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 上記サイトから 4 つのファイルを入手し， gunzip\n", | |
| "! for fn in train-images-idx3-ubyte t10k-images-idx3-ubyte train-labels-idx1-ubyte t10k-labels-idx1-ubyte; do if [ ! -e ${fn} ]; then wget -nc http://yann.lecun.com/exdb/mnist/${fn}.gz ; gunzip ${fn}.gz; fi; done\n", | |
| "! ls -l" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "KJcd4JC0Ytql", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# MNIST のためのクラスの定義\n", | |
| "\n", | |
| "import struct\n", | |
| "import os\n", | |
| "import numpy as np\n", | |
| "\n", | |
| "class MNIST:\n", | |
| " \n", | |
| " def __init__(self):\n", | |
| "\n", | |
| " fnImageL = 'train-images-idx3-ubyte'\n", | |
| " fnImageT = 't10k-images-idx3-ubyte'\n", | |
| " fnLabelL = 'train-labels-idx1-ubyte'\n", | |
| " fnLabelT = 't10k-labels-idx1-ubyte'\n", | |
| " \n", | |
| " if not os.path.exists(fnImageL) or not os.path.exists(fnImageT) or not os.path.exists(fnLabelL) or not os.path.exists(fnLabelT):\n", | |
| " print('Please get the MNIST files first.')\n", | |
| " return\n", | |
| " \n", | |
| " self.fnImage = {'L': fnImageL, 'T': fnImageT}\n", | |
| " self.fnLabel = {'L': fnLabelL, 'T': fnLabelT}\n", | |
| " self.nrow = 28\n", | |
| " self.ncol = 28\n", | |
| " self.nclass = 10\n", | |
| " \n", | |
| " \n", | |
| " def getLabel( self, LT ):\n", | |
| " \n", | |
| " return _readLabel( self.fnLabel[LT] )\n", | |
| " \n", | |
| " \n", | |
| " def getImage( self, LT ):\n", | |
| " \n", | |
| " return _readImage( self.fnImage[LT] )\n", | |
| " \n", | |
| " \n", | |
| "##### reading the label file\n", | |
| "#\n", | |
| "def _readLabel( fnLabel ):\n", | |
| " \n", | |
| " f = open( fnLabel, 'rb' )\n", | |
| " \n", | |
| " ### header (two 4B integers, magic number(2049) & number of items)\n", | |
| " #\n", | |
| " header = f.read( 8 )\n", | |
| " mn, num = struct.unpack( '>2i', header ) # MSB first (bigendian)\n", | |
| " assert mn == 2049\n", | |
| " #print mn, num\n", | |
| " \n", | |
| " ### labels (unsigned byte)\n", | |
| " #\n", | |
| " label = np.array( struct.unpack( '>%dB' % num, f.read() ), dtype = int )\n", | |
| " \n", | |
| " f.close()\n", | |
| " \n", | |
| " return label\n", | |
| "\n", | |
| " \n", | |
| "##### reading the image file\n", | |
| "#\n", | |
| "def _readImage( fnImage ):\n", | |
| " \n", | |
| " f = open( fnImage, 'rb' )\n", | |
| " \n", | |
| " ### header (four 4B integers, magic number(2051), #images, #rows, and #cols\n", | |
| " #\n", | |
| " header = f.read( 16 )\n", | |
| " mn, num, nrow, ncol = struct.unpack( '>4i', header ) # MSB first (bigendian)\n", | |
| " assert mn == 2051\n", | |
| " #print mn, num, nrow, ncol\n", | |
| " \n", | |
| " ### pixels (unsigned byte)\n", | |
| " #\n", | |
| " npixel = ncol * nrow\n", | |
| " #pixel = np.empty( ( num, npixel ), dtype = int )\n", | |
| " #pixel = np.empty( ( num, npixel ), dtype = np.int32 )\n", | |
| " pixel = np.empty( ( num, npixel ) )\n", | |
| " for i in range( num ):\n", | |
| " buf = struct.unpack( '>%dB' % npixel, f.read( npixel ) )\n", | |
| " pixel[i, :] = np.asarray( buf )\n", | |
| " \n", | |
| " f.close()\n", | |
| " \n", | |
| " return pixel\n" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "9B7_wzfnYaYd", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# データを可視化するための関数\n", | |
| "\n", | |
| "def display(data, nx, ny, nrow=28, ncol=28, gap=4):\n", | |
| "\n", | |
| " assert data.shape[0] == nx*ny\n", | |
| " assert data.shape[1] == nrow*ncol\n", | |
| "\n", | |
| " # 並べた画像の幅と高さ\n", | |
| " width = nx * (ncol + gap) + gap\n", | |
| " height = ny * (nrow + gap) + gap\n", | |
| "\n", | |
| " # 画像の作成\n", | |
| " img = np.zeros((height, width), dtype = int) + 128\n", | |
| " for iy in range(ny):\n", | |
| " lty = iy*(nrow + gap) + gap\n", | |
| " for ix in range(nx):\n", | |
| " ltx = ix*(ncol + gap) + gap\n", | |
| " img[lty:lty+nrow, ltx:ltx+ncol] = data[iy*nx+ix].reshape((nrow, ncol))\n", | |
| "\n", | |
| " # 画像の出力\n", | |
| " plt.axis('off')\n", | |
| " plt.imshow(img, cmap = 'gray')\n", | |
| " #cv2.imwrite(ID + '_proto.png', img)\n", | |
| " plt.show()" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "3p3d_5hDagv1", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### 学習データの準備" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "ZPjOgsYvZAgX", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# MNIST の学習データ\n", | |
| "mn = MNIST()\n", | |
| "datL = mn.getImage('L')\n", | |
| "labL = mn.getLabel('L')\n", | |
| "NL = datL.shape[0]\n", | |
| "print('# 学習データのデータ数: ', NL)\n", | |
| "D = mn.nrow * mn.ncol" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "jvpXmhq2ZNr9", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 学習データの最初の50個を可視化してみる\n", | |
| "display(datL[:50, :], 10, 5)\n", | |
| "\n", | |
| "# これらの正解ラベル\n", | |
| "for i in range(5):\n", | |
| " print(labL[i*10:((i+1)*10)])" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "V-1SZAataqVH", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### 最短距離法による識別" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "k1hLTBiuZPii", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 各クラスの学習データの平均を求めてプロトタイプとする\n", | |
| "K = mn.nclass\n", | |
| "proto = np.empty((K, D))\n", | |
| "for k in range(K):\n", | |
| " proto[k, :] = np.mean(datL[labL == k, :], axis = 0)\n", | |
| "\n", | |
| "# 可視化してみる\n", | |
| "display(proto, 10, 1)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab_type": "code", | |
| "id": "YlvNu2o_a5zv", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# 学習データを識別してみる\n", | |
| "\n", | |
| "ncorrect = 0 # 正解した数をカウント\n", | |
| "for n in range(NL):\n", | |
| " d2 = np.sum((proto - datL[n, :])**2, axis = 1) # プロトタイプとの距離の2乗\n", | |
| " lab_est = np.argmin(d2) # 距離最小のプロトタイプの番号\n", | |
| " if labL[n] == lab_est:\n", | |
| " ncorrect += 1\n", | |
| " \n", | |
| "print('# 識別率: {0}/{1} = {2:.3f}'.format(ncorrect, NL, ncorrect/NL))" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "zOPC7AWDavXY", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# MNIST のテストデータ\n", | |
| "datT = mn.getImage('T')\n", | |
| "labT = mn.getLabel('T')\n", | |
| "NT = datT.shape[0]\n", | |
| "print('# テストデータのデータ数: ', NT)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "D-nttbpJbD4H", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "# テストデータを識別\n", | |
| "ncorrect = 0 # 正解した数をカウント\n", | |
| "for n in range(NT):\n", | |
| " d2 = np.sum((proto - datT[n, :])**2, axis = 1) # プロトタイプとの距離の2乗\n", | |
| " lab_est = np.argmin(d2) # 距離最小のプロトタイプの番号\n", | |
| " if labT[n] == lab_est:\n", | |
| " ncorrect += 1\n", | |
| " \n", | |
| "print('# 識別率: {0}/{1} = {2:.3f}'.format(ncorrect, NT, ncorrect/NT))" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "dnx2z8jLk-hJ", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### ★★★ 次のことをやりましょう★★★\n", | |
| "\n", | |
| "上記のセルたちを実行し，結果を観察する．識別率などをメモしておく" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "ZiOmlH6Fij9a", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### おまけ課題: 最近傍法による識別\n", | |
| "\n", | |
| "ここから先はおまけ課題です．やらなくても減点はありません．やったら棒茄子？結果は Teams チャットで送ってください．\n", | |
| "\n", | |
| "MNIST を最近傍法で識別する実験をやろう．\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "rsPsM_T4b9-j", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "MNIST のように学習データ数が多い場合，すべての学習データとの距離を計算する最近傍法は計算コストが大きくなります．ですが，少しの工夫で改善できます．そのアイデアを以下のセルに書いておきました．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "DBJ60Fdzj_5N", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "テストデータ $\\mathbf{x}$ と学習データ $\\mathbf{x}_{n}$ のユークリッド距離を $d$ とおくと\n", | |
| "$$\n", | |
| "\\begin{align}d^2 &= ||\\mathbf{x} - \\mathbf{x}_{n}||^2 \\\\\n", | |
| "&= ||\\mathbf{x}||^2 - 2\\mathbf{x}\\cdot\\mathbf{x}_{n} + ||\\mathbf{x}_{n}||^2\n", | |
| "\\end{align}\n", | |
| "$$\n", | |
| "である．第1項はどの学習データ相手でも同じ値なので無視できる．したがって，ユークリッド距離をまじめに計算するかわりに\n", | |
| "$$\n", | |
| "\\tilde{d}^2 = - 2\\mathbf{x}\\cdot\\mathbf{x}_{n} + ||\\mathbf{x}_{n}||^2\n", | |
| "$$\n", | |
| "を計算すればよい．ここで，第3項はテストデータによらないので，事前に計算しておける．したがって，テストデータごとの計算はほぼ $\\mathbf{x}$ と $\\mathbf{x}_{n}$ の内積のみで済む．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "vSjWvfbijXVE", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "というわけで，学習データそれぞれのノルムの2乗 $||\\mathbf{x}_{n}||^2$ をあらかじめ計算しておきます．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "I94NnZmIijoy", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "n2XL = np.sum(datL**2, axis = 1)\n", | |
| "print(n2XL.shape)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "w1joyr60lqmp", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "識別．以下のセルの先頭行のコメントを読んで実行しよう．" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "13yOJjc_jphd", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "hoge # 不用意に実行してしまわないよう，エラーを起こすようにしてある．おまけ課題やるときはこの行を削除しよう\n", | |
| "\n", | |
| "import datetime\n", | |
| "\n", | |
| "lab_est = np.empty(NT, dtype = int) # 識別結果（クラス番号）を入れるベクトル\n", | |
| "\n", | |
| "start = datetime.datetime.now() # 現在の時刻を取得\n", | |
| "\n", | |
| "for n in range(NT):\n", | |
| " d2 = -2 * (datL @ datT[n, :]) + n2XL # 距離をさぼって計算．結果は学習データ数次元ベクトル\n", | |
| " #d2 = np.sum((datL - datT[n, np.newaxis, :])**2, axis = 1) # まじめにユークリッド距離を計算するバージョン\n", | |
| " lab_est[n] = labL[np.argmin(d2)]\n", | |
| " if n % 100 == 0: # 100個識別ができるたびに出力\n", | |
| " delta = datetime.datetime.now() - start\n", | |
| " print('# {0}/{1} {2}'.format(n, NT, delta)) # delta は経過時間\n", | |
| "\n", | |
| "delta = datetime.datetime.now() - start\n", | |
| "ncorrect = np.sum(labT == lab_est) # 正解数\n", | |
| "print('# 識別率: {0}/{1} = {2:.3f}'.format(ncorrect, NT, ncorrect/NT))\n", | |
| "print('# 経過時間:', delta)" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "kQr5qub6tmBf", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "### ★★★ （おまけ課題）次のことをやりましょう ★★★\n", | |
| "- 上記を実行し，識別率と実行にかかった時間を報告しなさい\n", | |
| "- さらに，**暇なひとは**，上記の\n", | |
| "` d2 = -2 * (datL @ datT[n, :]) + n2XL`\n", | |
| "の行をコメントにしてその下の行のコメントをはずして同様のことをやってみなさい．" | |
| ] | |
| } | |
| ] | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment