Skip to content

Instantly share code, notes, and snippets.

@xiangze

xiangze/NMF.ipynb Secret

Last active December 24, 2018 15:06
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save xiangze/63e34c313aed4dafd01be30b8518ec55 to your computer and use it in GitHub Desktop.
Save xiangze/63e34c313aed4dafd01be30b8518ec55 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
NMF.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tensorflow Probabilityで非負行列分解(NMF)"
]
},
{
"cell_type": "markdown",
"metadata": {
"toc": true
},
"source": [
"<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
"<div class=\"toc\" style=\"margin-top: 1em;\"><ul class=\"toc-item\"><li><span><a href=\"#Tensorflow-Probabilityで非負行列分解(NMF)\" data-toc-modified-id=\"Tensorflow-Probabilityで非負行列分解(NMF)-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Tensorflow Probabilityで非負行列分解(NMF)</a></span><ul class=\"toc-item\"><li><span><a href=\"#data生成\" data-toc-modified-id=\"data生成-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>data生成</a></span></li><li><span><a href=\"#統計model\" data-toc-modified-id=\"統計model-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>統計model</a></span><ul class=\"toc-item\"><li><span><a href=\"#推定されるべき変数\" data-toc-modified-id=\"推定されるべき変数-1.2.1\"><span class=\"toc-item-num\">1.2.1&nbsp;&nbsp;</span>推定されるべき変数</a></span></li></ul></li><li><span><a href=\"#推論\" data-toc-modified-id=\"推論-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span>推論</a></span></li><li><span><a href=\"#結果\" data-toc-modified-id=\"結果-1.4\"><span class=\"toc-item-num\">1.4&nbsp;&nbsp;</span>結果</a></span></li><li><span><a href=\"#発展\" data-toc-modified-id=\"発展-1.5\"><span class=\"toc-item-num\">1.5&nbsp;&nbsp;</span>発展</a></span></li><li><span><a href=\"#Link\" data-toc-modified-id=\"Link-1.6\"><span class=\"toc-item-num\">1.6&nbsp;&nbsp;</span>Link</a></span></li></ul></li></ul></div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"行列分解、テンソル分解は協調フィルタリングや最近ではニューラルネットの重み行列(テンソル)の圧縮で注目されている。\n",
"最近は動作速度や機能の充実さからpytorch, pyroが話題になっているがTensorflowも名前の由来であるテンソルの計算に使えるのではないかという安易な考えがある。\n",
"\n",
"Edward1の時にはうまく動いていない場合も報告されていたが、TFPになってからはどうなのだろうか。\n",
"- https://discourse.edwardlib.org/t/loss-increasing-in-inference-of-poisson-factorization/221/7\n",
"- https://discourse.edwardlib.org/t/nonnegative-matrix-factorization/341/2"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import seaborn as sns\n",
"\n",
"import tensorflow as tf\n",
"import tensorflow_probability as tfp\n",
"from tensorflow_probability import edward2 as ed\n",
"import warnings\n",
"tfd = tfp.distributions\n",
"\n",
"plt.style.use(\"ggplot\")\n",
"warnings.filterwarnings('ignore')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## data生成"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"n*m行列Oに対し\n",
"\n",
"$O=ALB^T$\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(Lは対角行列)という関係を仮定する。また要素非負であるという仮定から\n",
"はPoisson分布を用いて生成するので\n",
"\n",
"$O_{i,j} \\sim { Poisson } \\left( A_{ ij } ^ { T } L_{jk} B_{kl} , \\sigma ^ { 2 } \\right) $\n",
"\n",
"となり、A,BはGamma分布で生成するのが良さそう。\n",
"\n",
"$A_{i,k} \\sim Gamma(0,\\sigma_A^{2}I)$\n",
"\n",
"$B_{k,j} \\sim Gamma(0,\\sigma_B^{2}I)$\n",
"\n",
"$L_{k,k} \\sim \\mathcal{N}(0,\\sigma_l^{2}I)$\n",
"\n",
"[Poisson Matrix Factorization](http://www.hongliangjie.com/2015/08/17/poisson-matrix-factorization/)を参照"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"実験用のデータは理想的な状況としてモデルと同じ分布から作成する。"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [],
"source": [
"n=30\n",
"m=40\n",
"r=6\n",
"\n",
"A=np.random.gamma(2.,2,n*r).reshape(n,r)\n",
"B=np.random.gamma(2.,2,r*m).reshape(r,m)\n",
"L=np.random.normal(0,3,r*r).reshape(r,r)\n",
"\n",
"D=np.matmul(np.matmul(A,L),B)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 統計model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"モデルを関数として定義する。lossの計算に必要な対数尤度を計算するのにed.make_log_joint_fnを使う。logprobの和を直接計算しても良い(参考 http://statmodeling.hatenablog.com/entry/statistical-modeling-with-tensorflow ) "
]
},
{
"cell_type": "code",
"execution_count": 118,
"metadata": {},
"outputs": [],
"source": [
"def c2(a,b,value=0.):\n",
" return tf.constant(value,shape=[a,b])\n",
"\n",
"c2_2=lambda n,r:c2(n,r,2.)\n",
"c2_3=lambda n,r:c2(n,r,3.)\n",
"c2_0=lambda n,r:c2(n,r,0.)\n",
"\n",
"def model(n,m,r):\n",
" \n",
" X=ed.Gamma(c2_2(n,r),c2_2(n,r),name=\"X\")\n",
" Y=ed.Gamma(c2_2(r,m),c2_2(r,m),name=\"Y\")\n",
" L=ed.Normal(loc=c2_0(r,r),scale=c2_3(r,r),name=\"L\")\n",
"\n",
" K=tf.matmul(X,L)\n",
" KK=tf.matmul(K,Y)\n",
" O=ed.Poisson(KK, name=\"O\")\n",
" return O,(X,Y,L)"
]
},
{
"cell_type": "code",
"execution_count": 119,
"metadata": {},
"outputs": [],
"source": [
"tf.reset_default_graph()\n",
"target=lambda X,Y,L:ed.make_log_joint_fn(model)(n,m,r, O=D,X=X,Y=Y,L=L)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 推定されるべき変数"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"stanのparameterに相当する。tf.Variableで定義する。常に正の数である場合、一定値以上の場合などにはtf.nn.softplusを使って制約を入れることができる。"
]
},
{
"cell_type": "code",
"execution_count": 121,
"metadata": {},
"outputs": [],
"source": [
"qX= tf.Variable(np.ones([n,r]), dtype=tf.float32)\n",
"qY= tf.Variable(np.ones([r,m]), dtype=tf.float32)\n",
"qL= tf.Variable(np.ones([r,r]), dtype=tf.float32)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"普通の統計モデルとの違いはデータが行列１セットしかないところ(時系列でも同様のことが多い)、ノイズや成分の分布を過程するのにtfpを用いる。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 推論"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"MAP推定を行う。Eagerでない場合Sessionを作って実行することになる。"
]
},
{
"cell_type": "code",
"execution_count": 122,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"loss=target(qX,qY,qL)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"optimizerを定義する。"
]
},
{
"cell_type": "code",
"execution_count": 123,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.train.AdamOptimizer(learning_rate = 0.01)\n",
"train = optimizer.minimize(loss)\n",
"init = tf.global_variables_initializer()\n",
"\n",
"t = []\n",
"num_epochs = 20000\n",
"\n",
"with tf.Session() as sess :\n",
" sess.run(init)\n",
" for i in range(num_epochs):\n",
" sess.run(train)\n",
" if i % 100 == 0:\n",
" t.append(sess.run([loss]))\n",
" \n",
" X_inferred = sess.run(qX)\n",
" Y_inferred = sess.run(qY) \n",
" L_inferred = sess.run(qL)"
]
},
{
"cell_type": "code",
"execution_count": 125,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[nan]"
]
},
"execution_count": 125,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"t[-1]"
]
},
{
"cell_type": "code",
"execution_count": 124,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAD8CAYAAABDwhLXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAH2pJREFUeJzt3X9Q1HXix/HnwqIGi8ouIqPSKJh3\nanJLrp3HTApJN11cjdk149U1A92dXZYNNc5cRTZNTkTzTfHwx+gZcU0505WBNTfNNccgOkXUksJl\nP8YjrJFEN3aRWNH4sZ/vH447EVniyhs3Xo+//Px8v1/wx4vPD3dtlmVZiIiIjLCY0Z6AiIiMDSoc\nERExQoUjIiJGqHBERMQIFY6IiBihwhERESNUOCIiYoQKR0REjFDhiIiIESocERExwj7aE7jcHDt2\nbLSnMCzJycl0dHSM9jSMUuaxQZmjw7Rp0y54X13hiIiIESocERExQoUjIiJGqHBERMQIFY6IiBih\nwhERESNG7LXozz//nJ07d3LmzBmmTJnCAw88QHx8PABffPEFf//73zl9+jQ2m42nn36acePG0dra\nytatW+nt7SUrK4vCwkJsNhvBYJCysjK++uorpkyZwoMPPojD4cCyLCorKzl48CDjx49n9erVpKen\nA1BXV0dVVRUAK1asICcnZ6SiiojIBRixK5wdO3Zw5513smHDBq699lreeOMNAAYGBti8eTN//vOf\n2bhxI0888QR2+9ne27lzJ/fccw/l5eUcP36cpqYmAPbs2cOCBQsoLy9nwYIF7NmzB4CDBw9y/Phx\nysvLWbVqFc899xwAwWCQ3bt3U1JSQklJCbt37yYYDI5UVBERuQAjVjjHjh1j7ty5AGRmZvLee+8B\n0NzczJVXXsnMmTMBSExMJCYmhs7OTk6fPs2cOXOw2WwsWbIEr9cLgNfrZenSpQAsXbo0vL6xsZEl\nS5Zgs9mYM2cOp06dorOzk6amJjIzM3E4HDgcDjIzM8PlJSIio2PECictLS1cDA0NDfj9fgDa29ux\n2Ww89dRT/PWvf+X1118HIBAI4HK5wse7XC4CgQAAXV1dJCUlATB58mS6urrCxyQnJw855rvncjqd\n4XOJiMjoiOgZzvr16zl58uSQ9StXruTee++lsrKS1157DY/HE75tNjAwwKeffsrTTz/N+PHjefLJ\nJ0lPTw8/3/kxNpsNm80WybQHqampoaamBoDS0tJBBRYN7HZ71M05Uso8NijzT09EhbNu3bof3P7Y\nY48BZ2+vHThwADh7FTJ37lwmTpwIQFZWFkeOHOG6664LXwUB+P1+nE4nAJMmTaKzs5OkpCQ6OzvD\nxzqdzkGfO3TuGKfTyccffxxeHwgEmDdv3vfOMS8vj7y8vPBytH2OUTR+9lKklHlsUObocFl8ltq5\n216hUIiqqipuuOEGAH7xi19w9OhRvvnmGwYGBvjkk0+YMWMGSUlJXHHFFRw+fBjLsti/fz8ejwcA\nj8fDvn37ANi3bx+LFi0Kr9+/fz+WZXH48GHi4+NJSkrC7XbT3NxMMBgkGAzS3NyM2+0eqagiInIB\nRuy16HfeeYe33noLgGuvvZbc3FwAHA4H+fn5PPLII9hsNrKysrjmmmsA+NOf/sS2bdvo7e3F7XaT\nlZUFwPLlyykrK6O2tjb8WjScvTo6cOAADzzwAOPGjWP16tXhMW677TYeeeQRAH73u9/hcDhGKqqI\niFwAm2VZ1mhP4nKirye4/Cnz2KDM0eGyuKUmIiLybSocERExQoUjIiJGqHBERMQIFY6IiBihwhER\nESNUOCIiYoQKR0REjFDhiIiIESocERExQoUjIiJGqHBERMQIFY6IiBihwhERESNUOCIiYoQKR0RE\njFDhiIiIESocERExQoUjIiJGqHBERMQIFY6IiBihwhERESNUOCIiYoQKR0REjFDhiIiIESocEREx\nQoUjIiJGqHBERMQIFY6IiBihwhERESPsI3Xizz//nJ07d3LmzBmmTJnCAw88QHx8PP39/Wzfvp0j\nR44QCoVYsmQJt956KwBNTU1UVlYSCoVYtmwZy5cvB8Dn87Fp0ya6u7tJT09nzZo12O12+vr62LJl\nC62trSQmJlJUVERKSgoA1dXV1NbWEhMTQ2FhIW63e6SiiojIBRixK5wdO3Zw5513smHDBq699lre\neOMNABoaGujv72fDhg2UlpZSU1ODz+cjFApRUVHBo48+SllZGe+88w5tbW0AvPTSS+Tn57N582YS\nEhKora0FoLa2loSEBDZv3kx+fj67du0CoK2tjfr6ejZu3EhxcTEVFRWEQqGRiioiIhdgxArn2LFj\nzJ07F4DMzEzee++98LYzZ84wMDBAb28vdrud+Ph4WlpaSE1NZerUqdjtdrKzs/F6vViWxUcffcTi\nxYsByMnJwev1AtDY2EhOTg4Aixcv5tChQ1iWhdfrJTs7m7i4OFJSUkhNTaWlpWWkooqIyAUYscJJ\nS0sLF0NDQwN+vx84WwwTJkxg1apVrF69mptvvhmHw0EgEMDlcoWPd7lcBAIBuru7iY+PJzY2FgCn\n00kgEAAYdExsbCzx8fF0d3cPOde3jxERkdER0TOc9evXc/LkySHrV65cyb333ktlZSWvvfYaHo8H\nu/3sUC0tLcTExLBjxw5OnTrF448/zoIFCyKZRkRqamqoqakBoLS0lOTk5FGby8Ww2+1RN+dIKfPY\noMw/PREVzrp1635w+2OPPQacvb124MABAN5++23cbjd2u51Jkybxs5/9jM8++4zk5OTwVRCA3+/H\n6XSSmJhIT08PAwMDxMbGEggEcDqdwNkrF7/fj8vlYmBggJ6eHhITE8Prz/n2Md+Vl5dHXl5eeLmj\no+PifhijJDk5OermHCllHhuUOTpMmzbtgvcdsVtqXV1dAIRCIaqqqrjhhhuAsz/QQ4cOAWef5fzv\nf/9j+vTpZGRk0N7ejs/no7+/n/r6ejweDzabjfnz59PQ0ABAXV0dHo8HgIULF1JXVwecvW03f/58\nbDYbHo+H+vp6+vr68Pl8tLe3M3v27JGKKiIiF8BmWZY1Eid+8803eeuttwC49tprueOOO7DZbJw5\nc4Zt27bR1taGZVnk5uZyyy23AHDgwAFeeOEFQqEQubm5rFixAoATJ06wadMmgsEgs2bNYs2aNcTF\nxdHb28uWLVs4cuQIDoeDoqIipk6dCkBVVRV79+4lJiaGgoICsrKyLmjex44dG4GfxsiJxr+IIqXM\nY4MyR4fhXOGMWOFEKxXO5U+ZxwZljg6XxS01ERGRb1PhiIiIESocERExQoUjIiJGqHBERMQIFY6I\niBihwhERESNUOCIiYoQKR0REjFDhiIiIESocERExQoUjIiJGqHBERMQIFY6IiBihwhERESNUOCIi\nYoQKR0REjFDhiIiIESocERExQoUjIiJGqHBERMQIFY6IiBihwhERESNUOCIiYoQKR0REjFDhiIiI\nESocERExQoUjIiJGqHBERMQIFY6IiBhhj+Tgd999l1dffZUvv/ySkpISMjIywtuqq6upra0lJiaG\nwsJC3G43AE1NTVRWVhIKhVi2bBnLly8HwOfzsWnTJrq7u0lPT2fNmjXY7Xb6+vrYsmULra2tJCYm\nUlRUREpKykWNISIioyeiK5y0tDTWrl3L3LlzB61va2ujvr6ejRs3UlxcTEVFBaFQiFAoREVFBY8+\n+ihlZWW88847tLW1AfDSSy+Rn5/P5s2bSUhIoLa2FoDa2loSEhLYvHkz+fn57Nq166LHEBGR0RNR\n4cyYMYNp06YNWe/1esnOziYuLo6UlBRSU1NpaWmhpaWF1NRUpk6dit1uJzs7G6/Xi2VZfPTRRyxe\nvBiAnJwcvF4vAI2NjeTk5ACwePFiDh06hGVZwx5DRERG14g8wwkEArhcrvCy0+kkEAgMWe9yuQgE\nAnR3dxMfH09sbOyg/b97rtjYWOLj4+nu7h72GCIiMrp+9BnO+vXrOXny5JD1K1euZNGiRSMyKZNq\namqoqakBoLS0lOTk5FGe0fDY7faom3OklHlsUOafnh8tnHXr1g37pE6nE7/fH14OBAI4nU6AQev9\nfj9Op5PExER6enoYGBggNjZ20P7nzuVyuRgYGKCnp4fExMRhj3E+eXl55OXlhZc7OjqGnXc0JScn\nR92cI6XMY4MyR4fve6xyPiNyS83j8VBfX09fXx8+n4/29nZmz55NRkYG7e3t+Hw++vv7qa+vx+Px\nYLPZmD9/Pg0NDQDU1dXh8XgAWLhwIXV1dQA0NDQwf/58bDbbsMcQEZHRZbMsy7rYg99//32ef/55\nvv76axISEpg5cybFxcUAVFVVsXfvXmJiYigoKCArKwuAAwcO8MILLxAKhcjNzWXFihUAnDhxgk2b\nNhEMBpk1axZr1qwhLi6O3t5etmzZwpEjR3A4HBQVFTF16tSLGuNCHDt27GJ/HKMiGv8iipQyjw3K\nHB2Gc4UTUeH8FKlwLn/KPDYoc3QY9VtqIiIi36XCERERI1Q4IiJihApHRESMUOGIiIgRKhwRETFC\nhSMiIkaocERExAgVjoiIGKHCERERI1Q4IiJihApHRESMUOGIiIgRKhwRETFChSMiIkaocERExAgV\njoiIGKHCERERI1Q4IiJihApHRESMUOGIiIgRKhwRETFChSMiIkaocERExAgVjoiIGKHCERERI1Q4\nIiJihApHRESMUOGIiIgRKhwRETHCHsnB7777Lq+++ipffvklJSUlZGRkhLdVV1dTW1tLTEwMhYWF\nuN1uOjo62Lp1KydPnsRms5GXl8dNN90EQDAYpKysjK+++oopU6bw4IMP4nA4sCyLyspKDh48yPjx\n41m9ejXp6ekA1NXVUVVVBcCKFSvIyckBoLW1la1bt9Lb20tWVhaFhYXYbLZIooqISIQiusJJS0tj\n7dq1zJ07d9D6trY26uvr2bhxI8XFxVRUVBAKhYiNjeWuu+6irKyMp556irfeeou2tjYA9uzZw4IF\nCygvL2fBggXs2bMHgIMHD3L8+HHKy8tZtWoVzz33HHC2oHbv3k1JSQklJSXs3r2bYDAIwM6dO7nn\nnnsoLy/n+PHjNDU1RRJTREQugYgKZ8aMGUybNm3Ieq/XS3Z2NnFxcaSkpJCamkpLSwtJSUnhq5Mr\nrriC6dOnEwgEwscsXboUgKVLl+L1egFobGxkyZIl2Gw25syZw6lTp+js7KSpqYnMzEwcDgcOh4PM\nzEyampro7Ozk9OnTzJkzB5vNxpIlS8LnEhGR0TMiz3ACgQAulyu87HQ6w8Vyjs/n48iRI8yePRuA\nrq4ukpKSAJg8eTJdXV3hcyUnJ4ePc7lcBAKB847x3fXn9hcRkdH1o89w1q9fz8mTJ4esX7lyJYsW\nLbqoQc+cOcOGDRsoKCggPj5+yHabzWbsmUtNTQ01NTUAlJaWDiq3aGC326NuzpFS5rFBmX96frRw\n1q1bN+yTOp1O/H5/eDkQCOB0OgHo7+9nw4YNXHfddfzyl78M7zNp0iQ6OztJSkqis7OTiRMnhs/V\n0dER3s/v9+N0OnE6nXz88ceDxpg3b96Qsc/tfz55eXnk5eWFl789VjRITk6OujlHSpnHBmWODt/3\nWOV8RuSWmsfjob6+nr6+Pnw+H+3t7cyePRvLsti+fTvTp0/nt7/97ZBj9u3bB8C+ffvCV08ej4f9\n+/djWRaHDx8mPj6epKQk3G43zc3NBINBgsEgzc3NuN1ukpKSuOKKKzh8+DCWZbF//348Hs9IxBQR\nkWGwWZZlXezB77//Ps8//zxff/01CQkJzJw5k+LiYgCqqqrYu3cvMTExFBQUkJWVxaeffsrjjz/O\nlVdeGb5l9vvf/55rrrmG7u5uysrK6OjoGPJadEVFBc3NzYwbN47Vq1eHX7+ura2luroaOPtadG5u\nLgCfffYZ27Zto7e3F7fbzd13333Bt+iOHTt2sT+OURGNfxFFSpnHBmWODsO5womocH6KVDiXP2Ue\nG5Q5Ooz6LTUREZHvUuGIiIgRKhwRETFChSMiIkaocERExAgVjoiIGKHCERERI1Q4IiJihApHRESM\nUOGIiIgRKhwRETFChSMiIkaocERExAgVjoiIGKHCERERI1Q4IiJihApHRESMUOGIiIgRKhwRETFC\nhSMiIkaocERExAgVjoiIGKHCERERI1Q4IiJihApHRESMUOGIiIgRKhwRETFChSMiIkaocERExAgV\njoiIGGGP5OB3332XV199lS+//JKSkhIyMjLC26qrq6mtrSUmJobCwkLcbnd4WygU4uGHH8bpdPLw\nww8D4PP52LRpE93d3aSnp7NmzRrsdjt9fX1s2bKF1tZWEhMTKSoqIiUl5QfHaGpqorKyklAoxLJl\ny1i+fHkkMUVE5BKI6AonLS2NtWvXMnfu3EHr29raqK+vZ+PGjRQXF1NRUUEoFApvf/PNN5k+ffqg\nY1566SXy8/PZvHkzCQkJ1NbWAlBbW0tCQgKbN28mPz+fXbt2/eAYoVCIiooKHn30UcrKynjnnXdo\na2uLJKaIiFwCERXOjBkzmDZt2pD1Xq+X7Oxs4uLiSElJITU1lZaWFgD8fj8HDhxg2bJl4f0ty+Kj\njz5i8eLFAOTk5OD1egFobGwkJycHgMWLF3Po0CEsyzrvGC0tLaSmpjJ16lTsdjvZ2dnhc4mIyOgZ\nkWc4gUAAl8sVXnY6nQQCAQD+8Y9/8Ic//AGbzRbe3t3dTXx8PLGxsUP2//a5YmNjiY+Pp7u7+7xj\nfHe9y+UKn0tEREbPjz7DWb9+PSdPnhyyfuXKlSxatGhYg33wwQdMmjSJ9PR0Pvroo2EdO1Jqamqo\nqakBoLS0lOTk5FGe0fDY7faom3OklHlsUOafnh8tnHXr1g37pE6nE7/fH14OBAI4nU4aGxtpbGzk\n4MGD9Pb2cvr0acrLy1mzZg09PT0MDAwQGxsb3v/b53K5XAwMDNDT00NiYuJ5xwAGrff7/eH13ycv\nL4+8vLzwckdHx7Dzjqbk5OSom3OklHlsUObo8H2PVc5nRG6peTwe6uvr6evrw+fz0d7ezuzZs7nj\njjvYvn07W7dupaioiKuvvpoHHngAm83G/PnzaWhoAKCurg6PxwPAwoULqaurA6ChoYH58+djs9nO\nO0ZGRgbt7e34fD76+/upr68Pn0tEREZPRK9Fv//++zz//PN8/fXXlJaWMnPmTIqLi0lLS+NXv/oV\nDz30EDExMfzxj38kJuaHu+3OO+9k06ZNvPzyy8yaNYvrr78egOuvv54tW7awZs0aHA4HRUVFAD84\nxt13381TTz1FKBQiNzeXtLS0SGKKiMglYLMsyxrtSVxOjh07NtpTGJZovASPlDKPDcocHUb9lpqI\niMh3qXBERMQIFY6IiBihwhERESNUOCIiYoQKR0REjFDhiIiIESocERExQoUjIiJGqHBERMQIFY6I\niBihwhERESNUOCIiYoQKR0REjFDhiIiIESocERExQoUjIiJGqHBERMQIFY6IiBihwhERESNUOCIi\nYoQKR0REjFDhiIiIESocERExQoUjIiJGqHBERMQIFY6IiBihwhERESNUOCIiYoQKR0REjFDhiIiI\nEfZIDn733Xd59dVX+fLLLykpKSEjIyO8rbq6mtraWmJiYigsLMTtdgNw6tQptm/fztGjR7HZbNx7\n773MmTOHYDBIWVkZX331FVOmTOHBBx/E4XBgWRaVlZUcPHiQ8ePHs3r1atLT0wGoq6ujqqoKgBUr\nVpCTkwNAa2srW7dupbe3l6ysLAoLC7HZbJFEFRGRCEV0hZOWlsbatWuZO3fuoPVtbW3U19ezceNG\niouLqaioIBQKAVBZWYnb7WbTpk383//9H9OnTwdgz549LFiwgPLychYsWMCePXsAOHjwIMePH6e8\nvJxVq1bx3HPPARAMBtm9ezclJSWUlJSwe/dugsEgADt37uSee+6hvLyc48eP09TUFElMERG5BCIq\nnBkzZjBt2rQh671eL9nZ2cTFxZGSkkJqaiotLS309PTwySefcP311wNgt9tJSEgIH7N06VIAli5d\nitfrBaCxsZElS5Zgs9mYM2cOp06dorOzk6amJjIzM3E4HDgcDjIzM2lqaqKzs5PTp08zZ84cbDYb\nS5YsCZ9LRERGT0S31M4nEAhw1VVXhZedTieBQIBx48YxceJEtm3bxhdffEF6ejoFBQVMmDCBrq4u\nkpKSAJg8eTJdXV3hcyUnJ4fP5XK5CAQCBAIBXC7XkDG+u/7c/udTU1NDTU0NAKWlpYPGigZ2uz3q\n5hwpZR4blPmn50cLZ/369Zw8eXLI+pUrV7Jo0aJhDTYwMMCRI0e4++67ueqqq6isrGTPnj2sXLly\n0H42m83YM5e8vDzy8vLCyx0dHUbGvVSSk5Ojbs6RUuaxQZmjw/fd5TqfHy2cdevWDXsCTqcTv98f\nXg4EAjidTlwuFy6XK3z1s3jx4vCzmkmTJtHZ2UlSUhKdnZ1MnDgxfK5v/wL8fj9OpxOn08nHH388\naIx58+YNGfvc/iIiMrpG5LVoj8dDfX09fX19+Hw+2tvbmT17NpMnT8blcnHs2DEAPvzwQ2bMmBE+\nZt++fQDs27cvfPXk8XjYv38/lmVx+PBh4uPjSUpKwu1209zcTDAYJBgM0tzcjNvtJikpiSuuuILD\nhw9jWRb79+/H4/GMREwRERkGm2VZ1sUe/P777/P888/z9ddfk5CQwMyZMykuLgagqqqKvXv3EhMT\nQ0FBAVlZWQB8/vnnbN++nf7+flJSUli9ejUOh4Pu7m7Kysro6OgY8lp0RUUFzc3NjBs3jtWrV4df\nv66traW6uho4+1p0bm4uAJ999hnbtm2jt7cXt9vN3XfffcG36M6VYbSIxkvwSCnz2KDM0WE4t9Qi\nKpyfIhXO5U+ZxwZljg7DKRx90oCIiBihwhERESNUOCIiYoQKR0REjFDhiIiIEXpLTUREjNAVTpR7\n+OGHR3sKxinz2KDMPz0qHBERMUKFIyIiRqhwoty3P+l6rFDmsUGZf3r00oCIiBihKxwRETFiRL7x\nUy6tYDBIWVkZX3311aBP0v6uuro6qqqqgLOfnp2TkzNo+zPPPIPP52PDhg0mph2RSDJ/8803bNy4\nkRMnThATE8PChQu58847TUe4YE1NTVRWVhIKhVi2bBnLly8ftL2vr48tW7bQ2tpKYmIiRUVFpKSk\nAFBdXU1tbS0xMTEUFhbidrtHI8KwXGze//73v+zatYv+/n7sdjt33XUXV1999SilGJ5Ifsdw9osh\nH3zwQW6//XZuueUW09O/dCy57L344otWdXW1ZVmWVV1dbb344otD9unu7rbuu+8+q7u7e9C/z2lo\naLA2bdpkPfTQQ8bmHYlIMp85c8b68MMPLcuyrL6+PmvdunXWgQMHjM7/Qg0MDFj333+/dfz4cauv\nr89au3atdfTo0UH7/Pvf/7Z27NhhWZZlvf3229bGjRsty7Kso0ePWmvXrrV6e3utEydOWPfff781\nMDBgPMNwRJK3tbXV8vv9lmVZ1hdffGGtWrXK7OQvUiSZz3n22WetDRs2WK+//rqxeY8E3VKLAl6v\nl6VLlwKwdOlSvF7vkH2amprIzMzE4XDgcDjIzMykqakJgDNnzvCvf/2L2267zei8IxFJ5vHjx4f/\n8rXb7cyaNWvQt8BeTlpaWkhNTWXq1KnY7Xays7OHZG1sbAxfrS5evJhDhw5hWRZer5fs7Gzi4uJI\nSUkhNTWVlpaWUUhx4SLJO2vWrPC396alpdHb20tfX5/pCMMWSWY4+71jKSkp4S+rjGYqnCjQ1dVF\nUlISAJMnT6arq2vIPoFAAJfLFV52Op0EAgEAXn75ZW6++WbGjRtnZsKXQKSZzzl16hQffPABCxYs\nGNkJX6TvZnC5XEMyfHuf2NhY4uPj6e7uvqD8l5tI8n7be++9R3p6OnFxcSM/6QhFkvnMmTO8/vrr\n3H777UbnPFL0DOcysX79ek6ePDlk/cqVKwct22y2C/72Ujj7DasnTpygoKAAn88X8TwvpZHKfM7A\nwAB/+9vf+M1vfsPUqVMvep5yeTl69Ci7du0Kf7vwT9krr7xCfn4+EyZMGO2pXBIqnMvEunXrzrtt\n0qRJdHZ2kpSURGdnJxMnThyyj9Pp5OOPPw4vBwIB5s2bx+HDh2ltbeW+++5jYGCArq4unnjiCZ54\n4omRiDEsI5X5nB07dpCamkp+fv6lnfgl5HQ6B93u8/v94dtG393H5XIxMDBAT08PiYmJQ44NBAJD\njr3cRJL33P7PPvss9913H6mpqUbnfrEiydzS0sJ7773Hrl27OHXqFDabjXHjxnHjjTeajnFJ6JZa\nFPB4POzbtw+Affv2sWjRoiH7uN1umpubCQaDBINBmpubcbvd/PrXv2bHjh1s3bqVJ598kmnTpl0W\nZfNjIskMZ28j9vT0UFBQYHLaw5aRkUF7ezs+n4/+/n7q6+vxeDyD9lm4cCF1dXUANDQ0MH/+fGw2\nGx6Ph/r6evr6+vD5fLS3tzN79uxRSHHhIsl76tQpSktLueOOO/j5z38+CrO/OJFkfvLJJ9m6dStb\nt27lpptu4tZbb43asgH9x8+o0N3dTVlZGR0dHYNeEf7ss8/4z3/+w1/+8hcAamtrqa6uBs6+Ipyb\nmzvoPD6fj2eeeSYqXouOJLPf7+fee+9l+vTp2O1nL+JvvPFGli1bNmp5fsiBAwd44YUXCIVC5Obm\nsmLFCv75z3+SkZGBx+Oht7eXLVu2cOTIERwOB0VFReFbhFVVVezdu5eYmBgKCgrIysoa5TQ/7mLz\nvvbaa+zZs2fQlc1jjz3GpEmTRjHNhYnkd3zOK6+8woQJE6L6tWgVjoiIGKFbaiIiYoQKR0REjFDh\niIiIESocERExQoUjIiJGqHBERMQIFY6IiBihwhERESP+H9qo89wYUeBhAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11e167518>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(t)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 結果"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"モデルの原因なのかうまくlossが計算できていない。\n",
"うまくいけば平均２乗誤差(MSE)を用いて評価したい。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"X_inferred \n",
"Y_inferred \n",
"L_inferred "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 発展"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- MCMC(or 変分ベイズ?)による分布の推定\n",
"- 階層的なモデル、GPU,TPUの利用\n",
"- 大規模な行列の場合データの読み込みを逐次的に行いたい(kerasのfit_generateのようなもの)\n",
"- 疎行列(tf.SparseTensor)をどう扱うのか"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Link"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- http://smrmkt.hatenablog.jp/entry/2014/08/23/211555\n",
"- Stanでの実装　https://gist.github.com/danstowell/151a3b68ceb9c5f4a51f\n",
"- http://kento1109.hatenablog.com/entry/2018/09/29/151248\n",
"- http://mlwiki.org/index.php/Non-Negative_Matrix_Factorization\n",
"\n",
"- [Bayesian Nonnegative Matrix Factorization with Stochastic Variational Inference(pdf)](http://www.columbia.edu/~jwp2128/Papers/PaisleyBleiJordan2014.pdf)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.3"
},
"latex_envs": {
"LaTeX_envs_menu_present": true,
"autoclose": false,
"autocomplete": true,
"bibliofile": "biblio.bib",
"cite_by": "apalike",
"current_citInitial": 1,
"eqLabelWithNumbers": true,
"eqNumInitial": 1,
"hotkeys": {
"equation": "Ctrl-E",
"itemize": "Ctrl-I"
},
"labels_anchors": false,
"latex_user_defs": false,
"report_style_numbering": false,
"user_envs_cfg": false
},
"toc": {
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"toc_cell": true,
"toc_position": {},
"toc_section_display": "block",
"toc_window_display": true
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment