Tensor&Matrix Operation Cookbook.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TASK #1 (Matrix-Matrix=>Tensor)\n",
    "\n",
    "즉, 모든 X 각각에 대해 c1, c2, .. cH를 뺀 Tensor 를 얻고자 함\n",
    "\n",
    "```\n",
    "X: (N, I)\n",
    "Center: (H, I)\n",
    "```\n",
    "=>`X - Center: (N, H, I)` \n",
    "\n",
    "Braodcasting & reshaping\n",
    "\n",
    "```\n",
    "X_reshaped  (3d array):  N x 1 x I\n",
    "Center      (2d array):      H x I\n",
    "Result      (3d array):  N x H x I\n",
    "```\n",
    "\n",
    "Example Case N = 5, H = 3, I = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 1],\n",
       "       [2, 2],\n",
       "       [3, 3],\n",
       "       [4, 4],\n",
       "       [5, 5]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.array(\n",
    "    [\n",
    "        [1, 1],\n",
    "        [2, 2],\n",
    "        [3, 3],\n",
    "        [4, 4],\n",
    "        [5, 5]\n",
    "    ]\n",
    ")\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 2],\n",
       "       [2, 2],\n",
       "       [2, 2]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Center = np.array(\n",
    "    [\n",
    "        [2, 2],\n",
    "        [2, 2],\n",
    "        [2, 2]\n",
    "    ]\n",
    ")\n",
    "Center"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5, 2) (3, 2)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape, Center.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5, 3, 2)\n",
      "[[[-1 -1]\n",
      "  [-1 -1]\n",
      "  [-1 -1]]\n",
      "\n",
      " [[ 0  0]\n",
      "  [ 0  0]\n",
      "  [ 0  0]]\n",
      "\n",
      " [[ 1  1]\n",
      "  [ 1  1]\n",
      "  [ 1  1]]\n",
      "\n",
      " [[ 2  2]\n",
      "  [ 2  2]\n",
      "  [ 2  2]]\n",
      "\n",
      " [[ 3  3]\n",
      "  [ 3  3]\n",
      "  [ 3  3]]]\n"
     ]
    }
   ],
   "source": [
    "# reshaping & broadcasting\n",
    "Result = X.reshape((5,1,2)) - Center\n",
    "print(Result.shape)\n",
    "print(Result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TASK #2 (Matrix tensordot Tensor => Tensor)\n",
    "\n",
    "Matrix(아래 예에선 Vector)와 Tensor의 곱으로 Matrix 얻기 ( 뉴럴넷에서 흔히 등장하는 패턴 )\n",
    "\n",
    "$$\n",
    "\\frac {\\partial L} {\\partial Z} \\frac {\\partial Z} {\\partial W} \\\\\n",
    "$$\n",
    "\n",
    "(1, H) tensordot (H, H, I)\n",
    "\n",
    "즉, 오른쪽 Tensor는 (H, I) 매트릭스가 (H, 1)만큼 있는 벡터인 셈\n",
    "\n",
    "```\n",
    "Delta: (1, H)  (매칭될 axis = 1)\n",
    "A: (H, H, I)  (매칭될 axis = 0) // H, H 일 필요는 없음\n",
    "```\n",
    "\n",
    "Example Case H = 3, I = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 2, 3]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Delta = np.array(\n",
    "    [\n",
    "        [1, 2, 3]\n",
    "    ]\n",
    ")\n",
    "Delta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[2, 4],\n",
       "        [3, 5],\n",
       "        [3, 5]],\n",
       "\n",
       "       [[1, 2],\n",
       "        [3, 4],\n",
       "        [3, 4]],\n",
       "\n",
       "       [[1, 0],\n",
       "        [0, 1],\n",
       "        [0, 1]]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array(\n",
    "    [\n",
    "        [\n",
    "            [2,4],\n",
    "            [3,5],\n",
    "            [3,5]\n",
    "        ],\n",
    "        [\n",
    "            [1,2],\n",
    "            [3,4],\n",
    "            [3,4]\n",
    "        ],\n",
    "        [\n",
    "            [1,0],\n",
    "            [0,1],\n",
    "            [0,1]\n",
    "        ]\n",
    "    ]\n",
    ")\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[ 7,  8],\n",
       "        [ 9, 16],\n",
       "        [ 9, 16]]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using tensor dot with axis Concept\n",
    "np.tensordot(Delta, A, (1, 0))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}