euroscipy-2017.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FluRS - A Library for Streaming Recommendation Algorithms\n",
    "\n",
    "This notebook provides code snippets and supplemental information for [Takuya Kitazawa](https://takuti.me/about/)'s talk at [EuroSciPy 2017](https://www.euroscipy.org/2017/).\n",
    "\n",
    "First and foremost, let's import the fundamental \"SciPy\" tools here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import numpy as np\n",
    "import numpy.linalg as ln"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Past: The era of user-item matrix\n",
    "\n",
    "At the beginning, consider the following tiny user-item matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 5 users * 6 items\n",
    "R = np.array([[5, 0, 1, 1, 0, 2],\n",
    "              [0, 2, 0, 4, 0, 4],\n",
    "              [4, 5, 0, 1, 1, 2],\n",
    "              [0, 0, 3, 5, 2, 0],\n",
    "              [2, 0, 1, 0, 4, 4]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collaborative Filtering (CF; k-Nearest-Neighbors)\n",
    "\n",
    "- Compute similarities between users and items\n",
    "- Recommend based on the users who have the same taste"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def similarity(x, y):\n",
    "    return np.inner(x, y) / (ln.norm(x, ord=2) * ln.norm(y, ord=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.35921060405354982"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity(R[0], R[1])  # user a <-> user b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.65495314632780777"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity(R[0], R[2])  # user a <-> user c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can infer that, for user **a**, user **c** has more similar taste thatn user **b**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Singular Value Decomposition (SVD)\n",
    "\n",
    "- Fill missing values in the original matrix\n",
    "- Recommend based on the estimated values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "U, s, V = ln.svd(R, full_matrices=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.19741238,  1.98064059,  0.19763307,  0.50430074,  1.04148574,\n",
       "         2.47123826],\n",
       "       [ 1.20450954,  1.18625722,  1.50361641,  3.5812116 ,  1.61569345,\n",
       "         2.37803076],\n",
       "       [ 4.36792826,  2.68465163,  0.20157659,  0.52659617,  1.36419993,\n",
       "         3.30665072],\n",
       "       [-0.94009727,  0.07701659,  2.08296828,  4.93223799,  1.52652414,\n",
       "         1.44132726],\n",
       "       [ 2.67985286,  1.80342544,  0.63125085,  1.52750202,  1.27145836,\n",
       "         2.54266834]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# represent user/item characteristics in a lower dimensional space\n",
    "k = 2\n",
    "\n",
    "np.dot(np.dot(U[:, :k], np.diag(s[:k])), V[:k, :])  # estimated user-item matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since missing values in the original matrix has been filled, the valus can be used to predict how a user likely to interests with an item."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matrix Factorization (MF)\n",
    "\n",
    "- Improve SVD's feasibility and performance\n",
    "- Meaningless \"zero\"s are completely ignored"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_user, n_item = R.shape\n",
    "\n",
    "# represent user/item characteristics in a lower dimensional space\n",
    "k = 2\n",
    "\n",
    "# gess random user/item factors\n",
    "P = np.random.rand(n_user, k)\n",
    "Q = np.random.rand(n_item, k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for user in range(n_user):\n",
    "    for item in range(n_item):\n",
    "        if R[user, item] == 0:  # ignore zeros\n",
    "            continue\n",
    "\n",
    "        p, q = P[user], Q[item]\n",
    "\n",
    "        err = R[user, item] - np.inner(p, q)  # estimation error\n",
    "\n",
    "        # adjust factors based on the estimation error\n",
    "        # yes, it's gradient descent!\n",
    "        # `0.1` is learning rate, and `0.01` is regularization parameter\n",
    "        next_p = p - 0.1 * (-2. * (err * q - 0.01 * p))\n",
    "        next_q = q - 0.1 * (-2. * (err * p - 0.01 * q))\n",
    "        \n",
    "        P[user], Q[user] = next_p, next_q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.97609174,  2.95034485,  1.48379043,  1.39108191,  2.9887629 ,\n",
       "         1.93777429],\n",
       "       [ 2.27929456,  3.43604698,  1.70685394,  1.7165006 ,  3.44401982,\n",
       "         2.23479256],\n",
       "       [ 2.49554099,  3.73340825,  1.87278218,  1.78224133,  3.7736529 ,\n",
       "         2.44708166],\n",
       "       [ 2.29928562,  3.48414632,  1.71931996,  1.79248131,  3.47241687,\n",
       "         2.2542272 ],\n",
       "       [ 2.76347051,  4.10105204,  2.07847715,  1.86102454,  4.18214788,\n",
       "         2.71011532]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(P, Q.T)  # estimated user-item matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Present: Streamed complex user-item data\n",
    "\n",
    "- Particularly focus on **implicit feedback recommendation**\n",
    "    - Create model based on \"actions\" such as buy, watch, listen and click, rather than \"rating value\"\n",
    "- **FluRS** provides an easy and structured way to implement such recommender"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Unified data representation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to provide unified interface to a wide variety of real-world data, FluRS requests you to define `user`, `item` and `event`, components which are fed into recommender system, as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from flurs.data.entity import User, Item, Event"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# define a user with index 0\n",
    "user = User(0)\n",
    "\n",
    "# define an item with index 0\n",
    "item = Item(0)\n",
    "\n",
    "# interaction between a user and item\n",
    "# e.g., a user bought an item\n",
    "event = Event(user, item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more complex input, `user` and `item` can be defined with its features in a vector form. For instance, user's demographics such as age and sex, and item properties like category and price are considered as the \"features.\"\n",
    "\n",
    "Moreover, you can set contextual information (e.g., time, location, distance) to `event`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# define a user with his/her feature set\n",
    "user = User(0, feature=np.array([0, 0, 1]))\n",
    "\n",
    "# define an item with its properties\n",
    "item = Item(0, feature=np.array([2, 1, 1]))\n",
    "\n",
    "# interaction between a user and item under a certain context\n",
    "event = Event(user, item, context=np.array([0, 4, 0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Same interface regardless of internal algorithms\n",
    "\n",
    "It is certainly the scikit-learn's way! We mostly hit `fit` and `predict` on scikit-learn.\n",
    "\n",
    "To give an example, let us launch `Popular` recommender which simply returns the most popular items over every users. This kind of naive (but it works!) recommendation logic is called \"non-personalized recommender systems.\"\n",
    "\n",
    "First, define additional set of users and items as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_a = User(0, feature=np.array([0, 0, 1]))\n",
    "user_b = User(1, feature=np.array([1, 0, 1]))\n",
    "user_c = User(2, feature=np.array([1, 3, 1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "item_a = Item(0, feature=np.array([2, 1, 1]))\n",
    "item_b = Item(1, feature=np.array([0, 2, 1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Second, create a new recommendation instance and initialize it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from flurs.baseline.popular import Popular"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "recommender = Popular()\n",
    "recommender.initialize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, in a mannar of **dependency injection**, register potential user and item to the recommender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "recommender.register(user_a)\n",
    "recommender.register(user_b)\n",
    "recommender.register(user_c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "recommender.register(item_a)\n",
    "recommender.register(item_b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we are ready to launch recommendation! Let's feed a couple of events to the recommender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "recommender.update(Event(user_a, item_b))\n",
    "recommender.update(Event(user_b, item_b))\n",
    "recommender.update(Event(user_a, item_a))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Obviously, `item_b` (i.e., item indexed by `1`) is the most popular, so the item should be recommended to `user_c` in the following case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([1, 0]), array([ 2.,  1.]))"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# specify target user and list of item candidates' indices\n",
    "candicates = np.array([0, 1])  # item a and b\n",
    "recommender.recommend(user_c, candicates)  # => (sorted item indices, scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the score indicates the frequency of corresponding item on the `Popular` recommender.\n",
    "\n",
    "Even if you choose a different recommender, you use it in the same mannar. Following code tries to launch incremental variant of matrix factorization on the previous data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from flurs.recommender.mf import MFRecommender"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# initialize a recommendation instance\n",
    "recommender = MFRecommender(k=40)\n",
    "recommender.initialize()\n",
    "\n",
    "# register users and items\n",
    "recommender.register(user_a)\n",
    "recommender.register(user_b)\n",
    "recommender.register(user_c)\n",
    "\n",
    "recommender.register(item_a)\n",
    "recommender.register(item_b)\n",
    "\n",
    "# feed some events\n",
    "recommender.update(Event(user_a, item_b))\n",
    "recommender.update(Event(user_b, item_b))\n",
    "recommender.update(Event(user_a, item_a))\n",
    "\n",
    "# make recommendation to `user_c`\n",
    "candidates = np.array([0, 1])\n",
    "recommended_items, scores = recommender.recommend(user_c, candidates)\n",
    "recommended_items"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Internally, FluRS separates recommender-specific functionality from core algorithm implementation. So, implementing a new recommendation algorithm on the package is not so complicated."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Incremental evaluation scheme"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FluRS enables you to evaluate recommendation engine in a streaming environment by following to \"**[test-then-learn](http://link.springer.com/chapter/10.1007/978-3-319-08786-3_41)**\" shecme. In short, the evaluation framework first trains recommendation model by using initial 30% of samples in a batch fashion. After that, for the remaining 70% of samples, recommender sequentially launches top-N recommendation and check if observed item is correctly included in a recommendation list.\n",
    "\n",
    "For instance, let us initialize a `UserKNNRecommender` (i.e., CF recommender):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from flurs.recommender.user_knn import UserKNNRecommender"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "recommender = MFRecommender(k=3)  # consider 3 nearest-neighbors\n",
    "recommender.initialize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to launch evaluation on a little bit more large-scale data, here we define 10000 users and items, and register them to the recommender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "users = [User(u) for u in range(10000)]\n",
    "for user in users:\n",
    "    recommender.register(user)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "items = [Item(i) for i in range(10000)]\n",
    "for item in items:\n",
    "    recommender.register(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, the recommender is injected to `Evaluator` as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from flurs.evaluator import Evaluator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "evaluator = Evaluator(recommender)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on the 10000 users and items, randomly create 1000 events:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "random_interactions = zip(np.random.choice(users, 1000), np.random.choice(items, 1000))\n",
    "events = [Event(u, i) for u, i in random_interactions]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the initial 30% of events, launch batch training on a recommender, and following 20% of samples are used to validate the accuracy of recommendation as a result batch training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "evaluator.fit(events[:300], events[300:500])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define a window with a certain size, and sequentially evaluate the remaining 70% of events:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from collections import deque\n",
    "window = deque(maxlen=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "accuracy = []\n",
    "for top_score, rank, recommend_time, update_time in evaluator.evaluate(events[300:]):\n",
    "    window.append(int(rank < 10))  # consider top-10 recommendation\n",
    "    accuracy.append(sum(window) / len(window))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plotting variation of the accuracy of recommendation is insightful, isn't it? :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1093cea20>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjQAAAF5CAYAAACIpbAsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3XmcXFWZ//HPNwuBwCSikQTZEWXVQBoUFAFFgsCMjopA\nK4qAOCzzA+MCMqOiOIq4EGGGCIiyDNAjgoPIYtgGIhAEEvYtCoTNSSSAYcvSnTy/P86tSaWo6q6q\nrr2+79erXum699xzTp10dT31nHPvVURgZmZm1s5GNLsDZmZmZsPlgMbMzMzangMaMzMza3sOaMzM\nzKztOaAxMzOztueAxszMzNqeAxozMzNrew5ozMzMrO05oDEzM7O254DGzMzM2l7LBDSSjpH0pKQl\nku6QtNMQ5feQNEfSUknzJB0ySNmDJK2U9JvhtmtmZmatpyUCGkkHAj8BTgJ2AO4DZkqaUKL8psBV\nwI3AZOB04FxJe5Uo+yNg1nDbNTMzs9akVrg5paQ7gD9GxHHZcwHPAGdExA+LlD8V2Cci3p23rQ8Y\nHxH75m0bQQpkfgHslu3/RLXtmpmZWWtqeoZG0migh5RtASBSlHUDsEuJw3bO9uebWaT8ScDCiDiv\nRu2amZlZCxrV7A4AE4CRwMKC7QuBLUscM6lE+XGSxkTEMkm7AoeSpqRq1a6ZmZm1oFYIaGpO0jrA\nhcAREfFSDet9C7A3MB9YWqt6zczMusCawKbAzIh4odaVt0JAswhYAUws2D4RWFDimAUlyr+cZWe2\nAjYBfpeti4Fsek3SclIG5tkq2t0buHjQV2NmZmaD+QxwSa0rbXpAExH9kuYAewJXwv8tzt0TOKPE\nYbOBfQq2Tc22AzwKvKtg//eAdYBjgWciYqCKducDXHTRRWy99dblvLyONm3aNKZPn97sbjSdx2EV\nj0XicVjFY5F4HOCRRx7h4IMPhuyztNaaHtBkTgPOzwKMO4FpwFjgfABJpwBvi4jctWbOAo7Jznb6\nJSkI2R/YFyAilgEP5zcg6W9pVzxSbrtFLAXYeuutmTJlyjBebmcYP368xwGPQz6PReJxWMVjkXgc\nVlOXJRstEdBExKXZtV9OJk353AvsHRHPZ0UmARvllZ8vaT9gOinj8ixweEQUnvk03HbNzMysDbRE\nQAMQETOAGSX2HVpk2yzSadfl1v+GOoZq18zMzNpD069DY2ZmZjZcDmisar29vc3uQkvwOKzisUg8\nDqt4LBKPQ/21xK0P2oWkKcCcOXPmeHGXmZlZBebOnUtPTw9AT0TMrXX9ztCYmZlZ23NAY2ZmZm3P\nAY2ZmZm1PQc0ZmZm1vYc0JiZmVnbc0BjZmZmbc8BjZmZmbU9BzRmZmbW9hzQmJmZWdtzQGNmZmZt\nzwGNmZmZtT0HNGZmZtb2HNCYmZlZ23NAY2ZmZm3PAY2ZmZm1PQc0ZmZm1vYc0JiZmVnbc0BjZmZm\nbc8BjZmZmbU9BzRmZmbW9hzQmJmZWdtzQGNmZmZtzwGNmZmZtT0HNGZmZtb2HNCYmZlZ22uZgEbS\nMZKelLRE0h2Sdhqi/B6S5khaKmmepEMK9n9c0l2SXpL0qqR7JB1cUOYkSSsLHg/X4/WZmZlZ/bRE\nQCPpQOAnwEnADsB9wExJE0qU3xS4CrgRmAycDpwraa+8Yi8A/wbsDLwLOA84r6AMwIPARGBS9ti1\nJi/KzMzMGmZUszuQmQacHREXAkg6EtgPOAz4YZHyRwFPRMTx2fPHJO2a1XM9QETMKjjmjCyLs2uu\nTGYgIp6v2SsxMzOzhmt6hkbSaKCHlG0BICICuAHYpcRhO2f7880cpDyS9gTeCdxSsOsdkp6T9Lik\niyRtVOFLMDMzsyZrhQzNBGAksLBg+0JgyxLHTCpRfpykMRGxDEDSOOA5YAwwABwdETflHXMH8Hng\nMWB94NvALEnbRcRr1b4gM7NWdddd8Ic/VHfs2LFw6KEwZkxt+2RWC60Q0NTTK6Q1NusAewLTJT2R\nm46KiJl5ZR+UdCfwFHAAac1NUdOmTWP8+PGrbevt7aW3t7fG3Tczq60TTkgBzVprVXbcypXw2muw\n9daw++716Zt1jr6+Pvr6+lbbtnjx4rq22QoBzSJgBWlhbr6JwIISxywoUf7lXHYG/m/q6ons6f2S\ntgFOBArX1+TKL5Y0D9hisA5Pnz6dKVOmDFbEzKwlLVsGn/0s/PKXlR33zDOw8cbpeLOhFPuSP3fu\nXHp6eurWZtPX0EREPzCHlEEBQJKy57eXOGx2fvnM1Gz7YEaQpp+KkrQOKZj53yHqMTNrS/39MKqK\nr7K5Y/r7a9sfs1pphQwNwGnA+ZLmAHeSzlYaC5wPIOkU4G0RkbvWzFnAMZJOBX5JCm72B/bNVSjp\n68DdwOOkIGY/4GDgyLwyPwJ+R5pm2gD4DtAPrJ4nMzPrEAMDMHp05cfljhkYqG1/zGqlJQKaiLg0\nu+bMyaSpo3uBvfNOp54EbJRXfr6k/YDpwLHAs8DhEZF/5tPawJnAhsAS4FHgMxFxWV6ZDYFLgLcA\nzwO3AjtHxAu1f5VmZs033AyNAxprVS0R0ABExAxgRol9hxbZNot0unep+r4JfHOINr2K18y6ysCA\np5ysMzV9DY2ZmTWOp5ysUzmgMTPrIl4UbJ3KAY2ZWRepdsppxIhVx5u1Igc0ZmZdpL+/uiknKR3n\ngMZalQMaM7MuUm2GBtJxnnKyVuWAxsysi1S7KBicobHW5oDGzKyLVLsoGNJxDmisVTmgMTPrIp5y\nsk7lgMbMrEtEVL8oGDzlZK3NAY2ZWZdYuTL96wyNdSIHNGZmXSKXXXGGxjqRAxozsy6Ry654UbB1\nIgc0ZmZdIheMeMrJOpEDGjOzLpELRjzlZJ3IAY2ZWZdwhsY6mQMaM7Mu4TU01skc0JiZdQmf5WSd\nzAGNmVmX8JSTdTIHNGZmXcKLgq2TOaAxM+sSztBYJ6vy1xokbQxsAowFngceiohlteqYmZnVlhcF\nWyer6Nda0qbAUcBBwIaA8nYvl/QH4Bzg8ohYWaM+mplZDdRiUfDSpbXrj1ktlT3lJOkM4D5gM+Ab\nwDbAeGANYBKwL3ArcDJwv6Sdat5bMzOrmqecrJNV8mv9GrB5RLxQZN9fgZuyx3ckfQTYCLhr+F00\nM7Na8KJg62RlBzQRcWIFZX9fXXfMzKxeapGhcUBjrWpYZzlJGiNpTK06Y2Zm9VOLRcGecrJWVXFA\nI2kvSddIegl4HXhd0kvZtg/XvotmZlYLvlKwdbKKAhpJhwDXAIuBacDfZ49pwN+AayR9tpqOSDpG\n0pOSlki6Y6hFxZL2kDRH0lJJ87K+5e//uKS7smDrVUn3SDp4uO2ambUrZ2isk1X6a/2vwJci4swi\n+86XdCvwLeA/K6lU0oHAT4AvAneSAqSZkt4ZEYuKlN8UuAqYAXwa+DBwrqS/RMT1WbEXgH8DHgWW\nA/8AnCdpYa5Mpe2ambUzZ2isk1U65bQxcMMg+28kXZ+mUtOAsyPiwoh4FDiSNJ11WInyRwFPRMTx\nEfFYFmBdltUDQETMiojfZvufjIgzgPuBXYfRrplZ2/KiYOtklQY0DwGHD7L/MODhSiqUNBroIQVD\nAEREkAKnXUoctjNvDKxmDlIeSXsC7wRuGUa7ZmZty1NO1skq/bX+CnBVdp2ZG4CF2faJwJ7A5sB+\nFdY5ARiZV1fOQmDLEsdMKlF+nKQxuVswSBoHPAeMAQaAoyPipmG0a2bWtoabofGUk7Wyin6tI+Jm\nSduRpnx2JgUWAAuAa4GzImJ+TXs4PK8Ak4F1SAHXdElPRMSs5nbLzLrF1VfD2WcPr44DD4TPfKa6\nYy+4AC6/PP08fz6MHAnSoIeUNGoUPP88fPSjlR/7/vfDCSdU165ZOSqO07OApZa/louAFaQsT76J\npECpmAUlyr+cf4PMbArpiezp/ZK2AU4EZlXZLgDTpk1j/Pjxq23r7e2lt7d3sMPMrAv96ldw223p\nA70ad98Ny5dXH9Ccfz786U8wZQpsuil85CPV1QPp2LvugpUV3qnv0Ufhvvsc0HSTvr4++vr6Vtu2\nePHiurY5nLttjycvQxMRVfU0IvolzSFlUK7M6lb2/IwSh80G9inYNjXbPpgRpOmnatsFYPr06UyZ\nMmWIpszM0hTN5Mlw5ZXVHd/bm7Iiw2l/zz1Tpma4enrgiisqP+4734Fzzhl++9Y+in3Jnzt3Lj09\nPXVrs5oL631B0sPAi6QFwI8AL0p6WNJgC4YHcxpwhKTPSdoKOAsYC5yftXmKpPy341nA5pJOlbSl\npKOB/bN6cv38uqQPS9pM0laSvgIczOqnlA/arpnZcPX3V79mBYa/EHe47deCFxNbI1T0ay7pa8C3\nSRmMmay+KHgqcLqkdSPix5XUGxGXSppAulP3ROBeYO+IyH0vmUS62WWu/HxJ+wHTgWOBZ4HDIyL/\nzKe1gTNJp5EvIV2P5jMRcVkF7ZqZDcvAQPXXfYHhL8Qdbvu14MXE1giVxu3/DBwaEZcWbH8EuFnS\nfcCPgIoCGoCImEG6UF6xfYcW2TaLdNp1qfq+CXxzOO2amQ3XwMDwMzTDDWhaIUPjgMbqrdIpp/WA\nBwbZ/wDpdGgzM8NTTuApJ2uMSgOau4CvS3rD20PSSNLZT3fVomNmZp3AU06ecrLGqGbKaSawQNIs\nVl9DsxvpnklTa9c9M7P25gzNqimniOqvgWM2lIoyNBFxP+n2Ad8kXbRu8+zxCvANYKuIeLDWnTQz\na1fO0Kxqf8WK5vbDOls1F9Z7BfhZ9jAzs0F4UfCq9luhL9a5Kr4OjZmZlc9TTqva98Jgq6dqLqx3\ntKQbJF2a3cE6f98ESU+UOtbMrNt4ymlV+14YbPVUUUAj6VjSdWYeBZYB10g6Ma/ISGCT2nXPzKy9\nOUPjDI01RqW/5v8EHBERlwBI+hlwhaS1IuJbNe+dmVmb8xqa1dfQmNVLpb/mmwG3555ExO2SPgTc\nIGk08NNads7MrN15yslTTtYYlQY0i0j3VJqf2xARD2ZBzU3A22rXNTOz9tfMKaeIdKp0q2RoPOVk\n9VTpouBbgU8UboyIh4E9gX1q0Skzs07RzAxN7jhnaKwbVBq3/4ASN4SMiIeyTM0nh90rM7MOUYsM\nTS7TMnJk5W3n6mgmZ2isESr6Nc+uFHz/IPsfBHylYDOzTC0WBefqqTSgyWVEWiWgcYbG6skX1jMz\nq6NaTDnl6qmm7fw6msVTTtYIDmjMzOqoFlNOuXqqaTu/jmbxlJM1ggMaM7M6cobGGRprDAc0ZmZ1\nsnJlWtBbqzU0lfIaGusmDmjMzOqkFlM+nnIyK0/VAY2kGZImFP5sZmZJLaZ8POVkVp7hZGgOBsYV\n+dnMzHCGJscZGmuE4QQ0KvGzmZnhDE2OMzTWCF5DY2ZWJ7VYlOtFwWblcUBjZlYnnnJavX1POVk9\nOaAxM6sTTzmt3r4zNFZPDmjMzOrEGZokdw8qZ2isnoYT0ESJn83MDK+hyZFSUOMMjdWTz3IyM6sT\nTzmtMnq0Axqrr6oDmoj4u4h4ovDnakk6RtKTkpZIukPSTkOU30PSHElLJc2TdEjB/i9ImiXpxexx\nfWGdkk6StLLg8fBwXoeZWY6nnFYZNcpTTlZfLbGGRtKBwE+Ak4AdgPuAmaWuPixpU+Aq4EZgMnA6\ncK6kvfKK7Q5cAuwB7Aw8A1wnaf2C6h4EJgKTsseutXhNZmbO0KziDI3VWwvE7QBMA86OiAsBJB0J\n7AccBvywSPmjgCci4vjs+WOSds3quR4gIj6bf4CkLwCfBPYELsrbNRARz9fwtZiZAV5Dk2/UKAc0\nVl9Nz9BIGg30kLItAEREADcAu5Q4bOdsf76Zg5QHWBsYDbxYsP0dkp6T9LikiyRtVEn/zcxK8ZTT\nKp5ysnprekADTABGAgsLti8kTQEVM6lE+XGSxpQ45lTgOVYPhO4APg/sDRwJbAbMkrR2uZ03MyvF\nU06reMrJ6q0F4vb6k/R14ABg94hYntseETPzij0o6U7gqazseY3tpVnnOOEE+NnPBi8zZgxcfz1s\nv/0b9x11FFx8cXltjR0Lt90Gb3/70GUPOAB+//vy6q2F3Af4mFJfs8owenQ67fmww+DIIys7dvny\ndLr0iBb46jpmDJx6Kpx+enXHf/zjcMEFte1Tp1q4EKZMgVdeKb7/Ix+BSy9tbJ8aoeqARtIo0oLb\ntwOXRMQrkt4GvBwRr1ZQ1SJgBWlhbr6JwIISxywoUf7liFhW0M+vAscDe0bEQ4N1JCIWS5oHbDFY\nuWnTpjF+/PjVtvX29tLb2zvYYWZd4557YPPN4XOfK75/yRL4xjfg8ceLBzRz58I226QAZDAvvgjf\n+x489VR5Ac2cObDTTrDffkOXrZUJE2D9wlMRKjBqFPz61+k1VmOzzVJA1Gxnn51+L6pxxRXpd8LK\n8+yz8Je/wJe/DBtssPq+q69uzFj29fXR19e32rbFixfXtc2qAhpJmwC/BzYGxpAW4r4CnJA9L/t7\nRET0S5pDWqx7ZVa/sudnlDhsNrBPwbap2fb8fh4PnAhMjYgh30qS1iEFMxcOVm769OlMmTJlqOrM\nutbAAGy7bfqDWswrr6SAptQUxMAAvOc9pY/PeeaZFNCUO5UxMADvf//Q9baaT36y2T0Yvg9+MD2q\n8dxzcM01te1PJ8u9Hw49FLbbbvV9L72UvkjUW7Ev+XPnzqWnp6dubVabiDwduBtYF1iSt/2/SYFI\npU4DjpD0OUlbAWcBY4HzASSdIik/2XgWsLmkUyVtKeloYP+sHrJjTgBOJp0p9bSkidlj7bwyP5K0\nm6RNJL0v638/sHpYaWYV6e8ffCHqUAtdhzq+3HqqrddaixcUV2awxeCdPJbVvrU/ALwvIpZr9Vzm\nfGCDokcMIiIuza45czJp6uheYO+806knARvllZ8vaT9gOnAs8CxweETkL/g9knRW02UFzX0nawdg\nQ9K1at4CPA/cCuwcES9U+hrMbJWBgcEXog610HWo48utp9p6rbV4QXFlBlsM3sljWW1AM4J0ZlKh\nDUlTTxWLiBnAjBL7Di2ybRbpdO9S9W1WRpte9GJWB0NlQoa6WaEzNJavk7MK9dCtGZpqp5yuA76U\n9zyy9SffATzTadblBgYGDxyGullhuZmUSi86N1S/rDX5onyVGeyCip08ltW+tb9CujXBw8CapGmb\nd5DOWHLWw6zLlROQDJb6LjfwyLVR7jdOTzm1p06eJqkHTzlVICKelTQZOJB0L6V1gF8AF0fEkkEP\nNrOOV87UzmCp70qnnMr9A+0pp/bUydMk9dCtU05Vv7UjYgC4OHuYmf2fWmRoysmkjByZpq/KCWgi\nYMUKZ2jaUSdnFeqhnAxNRGtcn6iWqlpDI2mFpP+R9OaC7RMlrahN18ysXTUqQzNUPfla6UaNVplO\nzirUw1AZGkjBfaepdlGwSBfQu1vStkX2mVkXK2cNzGCLEytZ61LuIkcHNO0r938c0eyetIehFgXn\nl+kk1QY0AXwS+B0wW9LHCvaZWRdr1KLgXD2VZGg85dR+cv9nK1c2tx/tYrCAZjg3O211w8nQrIiI\n44CvAr+S9A2cnTEzGj/lVM4f58HS8NbaKr3eULfr71+1vqxQJ4/lsN/aEXGOpD8BvwZ2G36XzKzd\nDSdDs3JlmlooN5NS7oJRZ2jaVydnFephsPdfJ49ltRmap0h3yAYgIv4H2Jm82xOYWfcazhqaSte6\neFFw5+vkdR/1MNj7r5PHstrr0LzhtgIR8WdJO5DuxWRmXWw4U065bbVeFOwpp/bVydMk9TDY+6+T\nx7Kmb+2IWErK3phZlyr3ei+lpooqzaR4UXDn6+Rpknro1imnsgMaSS8C74yIRZJeYpCzmSLizaX2\nmVlnKzcgcYbGytXJWYV6cIZmaNNYdSftafj0bDMrotxMiDM0Vq5OzirUgzM0Q4iIC/J+Pr8uvTGz\ntldJhqZWi4J9Yb3O1skLWeuhWxcFV3vrgymS3pX3/GOSrpD0fUlr1K57ZtZuyp3a8ZSTlauTp0nq\noVunnKo9bfts4J0AkjYHfgW8DnwK+GFtumZm7chTTlZrnTxNUg/dOuVUbUDzTuDe7OdPAbdExKeB\nz5NuiWBmXcoZGqu1Ts4q1IMzNJVR3rEfBq7Jfn4GmDDcTplZ+2rGGhpfWK+zdfK6j3rwGprK3A18\nQ9Jngd2Bq7PtmwELa9ExM2tPzZhy8q0POlsnT5PUg6ecKvMlYArwH8D3IuLP2fb9gdtr0TEza0+e\ncrJa6+Rpknro1imnam99cD/wriK7vkbePZ7MrPs0I0OzfHnt+mWtp5OzCvXgDE0NRMTSiOjAuM/M\nylWrNTS1ztB4DU376uR1H/XgNTRmZjVQqymnWi8K9pRT++rkaZJ66NYpJwc0ZlZTtZpyKjdD40XB\nna+Tp0nqwVNOZmY10IwMjRcFd7ZOzirUgzM0FciuDmxm9gatfKVgCUaOLK9eax2dnFWoh8EyNLnf\n/04cy2q/q/xZ0rPALcDNpCsF/3nwQ8ysG7TyomBnZ9pTJy9krYfBftel8t8z7abaKaeNgBOBJcDx\nwDxJz0q6WNIXqqlQ0jGSnpS0RNIdknYaovwekuZIWippnqRDCvZ/QdIsSS9mj+uL1Vlpu2Y2uFZe\nFOyApj118jRJPQz1u17ue6bdVBXQRMRzEXFxRHwxIrYEtgRuAA4g3biyIpIOBH4CnATsANwHzJRU\n9DYKkjYFrgJuBCYDpwPnStorr9juwCXAHsDOpNsyXCdp/WrbNbOhtfKiYC8Ibk+5qcJOzCrUw1C/\n6+W+Z9pNtWtoxkqaKun7km4H7icFFv8BfKKKKqcBZ0fEhRHxKHAk6e7dh5UofxTwREQcHxGPRcSZ\nwGVZPQBExGcj4qyIuD8i5gFfIL3ePYfRrpkNoVYZmnLXujhD0x06NatQD92aoan27f034CXgYuAH\nwB8i4qVqKpI0GugBvp/bFhEh6QZglxKH7UzKCOWbCUwfpKm1gdHAi8No18yGUIs1NCNGpEc5vIam\nO3Tquo96GOp3vVPHstq39zXArsBBwCRgkqSbs0xIpSYAI3njTS0XkqayiplUovw4SWMiYlmRY04F\nnmNVIFRNu2ZdIwIeeqi82wrk+3N2ekA5U07LlsHcuatvf/LJyqaGRo+G119/Yz2FnnnGU07tbPRo\nmD9/6P/nSm2xBYwbV9s6BzN/PqyxBkycCA8+CCvqcLOgV18desrp6adXjeUmm8Bb3lL7fjRatfdy\n+kcASe8mrVWZCnxX0gBwc0R8pnZdHD5JXyet79k9Iir882zWnWbOhH32qe7YsWOHzoasu24KaHp6\n3rhv4sTy21p3XVi0qHg9hbbdtvx6rbWsuy6ccUZ61NLHPgZXXFHbOktZvBg22yz9fO658IWqTqEp\nz5veVHrfuuvC2WenB8B73gN//GP9+tIow03APpDVsQawJrA3cCBQSUCziHRDy8I/YROBBSWOWVCi\n/MuF2RlJXyWdibVnRDw0zHYBmDZtGuPHj19tW29vL729vYMdZtZWXsomkW+9FdZaq7Jj11tv6DUw\nBx8M229fPPW9/vpv3FbK0UfDbrvBypVDl91oo/LrtdYyezY891xt6/zGN+Bvf6ttnYN55ZVVP7/0\nEqyzDtxyS+3bkQYP3m+6CZ59Nv18+ulwxx2170NfXx99fX2rbVu8eHHtG8pTVUAj6cuks4d2Bf6O\ndHbQLOAc4A+V1BUR/ZLmkBbrXpnVr+x5qVh8NlD43XFqtj2/n8eTTi+fGhH31KBdAKZPn86UKVOG\nfnFmbSy3aHDHHWHMmNrXP2IEvPvdw69n1KgUGFlnmzixssxdOdZbD/70p9rWOZj8hbj9/bDmmtCM\nj5L11ksPgA02qM8C4WJf8ufOnUtPOanUKlWboeklXVTvHNKC4OGGXacB52cBxp2ks4/GAucDSDoF\neFtE5K41cxZwjKRTgV+SgpD9gX1zFUo6AfhO1tenJeXeCq9GxGvltGvWzXx3aut0jV4cm9/WsmWt\n8d7qpAXC1a6hqenF5yLi0uzaLyeTpnzuBfaOiOezIpNIF/PLlZ8vaT/SWU3HAs8Ch0dE/plPR5LO\narqsoLnvZO2U065Z1/KtAqzTNfp6LPltLV3aGovUO+maNFXHh5LeBBwObJ1tehj4RbXZmoiYAcwo\nse/QIttmkU67LlXfZsNt16yb+bot1ukafT2W/LaWLGmN91cnXZOm2gvr7Qg8TpqieXP2mAY8LsmL\nS8w6gK+sa52umRmaJUta4/3lDE2a6rkSOCIiBgAkjQLOBX4K7Fab7plZs/hCdNbpmrmGZunS1nh/\ndf0aGmBH8oIZgIgYkPRD4O6a9MzMmspTTtbpPOXkKSeAl4GNi2zfCHilyHYzazOecrJO5ymnzppy\nqjag+RXwC0kHStooexxEmnLqG+JYM2sDztBYp3OGJvVhxYp0q5N2V+1wfhUI4MK8OvqBnwFfr0G/\nzKzJnKGxTucMzao+dML7vdrr0CwHjpN0IvD2bPPjEfF6zXpmZk3lRcHW6Zq9KLjgDjpNkXuPd21A\nk5MFMA/UqC9m1kI85WSdrtlTTq1wh+vce7y/v/J7trWasv9cSfpNuWUj4hPVdcfMWkUnfGMzG4yn\nnFafcmp3lSwKXpz3eJl0/6Qd8/b3ZNvqeztNM2sIZ2is0zU7Q9MK76/8DE27K3s4828/kN0U8lLg\nyIhYkW0bSbqFwMu17qSZNZ7X0FinGzUqnd2zcmW6+3u9FWZoWuH9lb+Gpt1V+194GPDjXDADkP18\nWrbPzNqcp5ys0zV6uqVVb04J3R3QjAK2KrJ9q2HUaWYtxFNO1ukaPd2S306rvL+6csqpwHmkC+u9\nHbgz2/Ze0jVozqtFx8ysuZyhsU7XzAxNfvvN1EkZmuFcWG8B8BVg/Wzb/wI/An5Sg36ZWZN5DY11\nukavHylspxXeX520hqbaC+utBH4I/FDSuGybFwObdZBWSYmb1Uszp5zy228mTznlcSBj1pkGBmCN\nNZrdC7P6acaU05gxsGzZ6u03UydNOVW1gFfSREn/KekvkgYkrch/1LqTZtZ4ztBYp2tGhib/aryt\n8P5yhgYAFSBgAAAgAElEQVTOBzYGvktaO9MB9+k0s3xeFGydrhkZmrXWgr/9bfX2m6mTMjTVBjS7\nAh+IiHtr2Rkzax1eFGydrhmLgseMeWP7zdRJi4KrvWbMM4Bq2REzay2ecrJO14wpp9GjYeTI1dtv\npk6acqo2oPkS8ANJm9auK2bWSjzlZJ2uGVNOo0evarcV3l+ecoJfAWOBxyW9DqwW20XEm4fbMTNr\nLmdorNM1I0MzatSqdlvh/dVJGZpqh/NLNe2FmbUcr6GxTteMNTStGtB0bYYmIi6odUfMrLV4ysk6\nnaecunTKSdK43EX0clcHLsUX2zNrf55ysk7nKafunXJ6SdL6EfFX4G8Uv/aMsu0ja9E5M2seZ2is\n0zlD06UZGuBDwIt5P/tiemYdzBka63TNytC00mnbI0aA1BkZmrJP246IWyJiIPv55ux50Uc1HZF0\njKQnJS2RdIeknYYov4ekOZKWSpon6ZCC/dtIuiyrc6WkY4vUcVK2L//xcDX9N+s0XhRsna5Zi4IL\n22+2UaM6I0NT7b2cZkk6WdKektYcbickHQj8BDgJ2AG4D5gpaUKJ8psCVwE3ApOB04FzJe2VV2ws\n8DhwAun2DKU8CEwEJmWPXYfxUsw6hqecrNM1a8pJWr39Zhs9ujMCmmrjw+uA3YAvA6Mk3Q3cDNwC\n3BYRr1dY3zTg7Ii4EEDSkcB+wGHAD4uUPwp4IiKOz54/JmnXrJ7rASLibuDurL5TB2l7ICKer7C/\nZh3PU07W6Zox5bRmXgqgVd5fo0Z12ZRTvoj4t4iYCrwJ+CApW7IjcDWr1tmURdJooIeUbcnVH8AN\nwC4lDts5259v5iDlB/MOSc9JelzSRZI2qqIOs47jDI11umZlaArbb7Zuz9DkbA68izTt827gFWBW\nhXVMIJ0VtbBg+0JgyxLHTCpRfpykMRGxrMy27wA+DzwGrA98G5glabuIeK3MOsxq6sYb4b77mt0L\nWL68db5BmtVD7vf76qth0aL6t/fkk/DWt0LE6u0326hRcMst9Q+wnn22vvVXNZySLgF2B8aQAphb\ngB8A92fZlbYQETPznj4o6U7gKeAA4LxSx02bNo3x48evtq23t5fe3t669NO6yxFHpDf+msNenTY8\n48fDlqW+Uph1gBEjYMoUuPba9GiEyZNh7bXhkktg880b0+ZQdtwRZs2CW2+tXZ39/X309/cVbF1c\nuwaKqDY+PAhYBJwL3ATcWsW6mZxFwArSwtx8E4EFJY5ZUKL8yxVkZ94gIhZLmgdsMVi56dOnM2XK\nlGqbMRvUsmXwr/8KJ53U7J6Ydb45c5rT7jnnNKfdYq66qh619maPVebOnUtPT089GgOqv9v2W4Av\nAGsApwCLJN0u6fuSplZSUUT0A3OAPXPbJCl7fnuJw2bnl89MzbZXTdI6pGBmsLOizOrKi3HNzCpX\n7aLglyLiyoj4ckT0kNbPzAO+BlSTuDsNOELS5yRtBZxFOu36fABJp0jKv3/UWcDmkk6VtKWko4H9\ns3rIjhktabKk7UmB1wbZ87fnlfmRpN0kbSLpfcB/k+4cXpgnM2sYL8Y1M6tctWto3kJaQ7NH9tiG\ndDuE35HW01QkIi7NrjlzMmnq6F5g77zTqScBG+WVny9pP2A6cCzwLHB4ROSf+fQ24B5WXdH4q9nj\nFtKVjgE2BC4hZZyeB24Fdo6IFyp9DWa14gvamZlVrto/m38lrX35A/Bz4OaIeGA4HYmIGcCMEvsO\nLbJtFul071L1PcUQGaiI8CpeazmecjIzq1y1fzbfHREP1bQnZgZ4ysnMrBrVrqFxMGNWBxGecjIz\nq0bZAY2k30vauYxyfyfpBEnHDK9rZt1nxYr0rwMaM7PKVPJn89fA5ZIWkxb/3g38BVgKrEtaGLwr\nsC/pFghfq21XzTpf7vLjnnIyM6tM2RmaiPgF6VYH3ycFL+eQFgXfRbqP0hHA08BOEXFgRDxd++5a\nt/vLX9JF516v9jKOLS53gzhnaMzMKlPRn83sKrwXZQ8kjQfWAl7ILpBnVld/+AN8//tw111w5ZXN\nvz1ArTlDY2ZWnWqvFAykWwVExAIHM9YouQ/8W26BAw5IN1DsJLnX5wyNmVllhhXQmDVabtHsr38N\nM2fCZz7TGbe9z/GUk5lZdRzQWFvJBTT77QeXXgpXXAGHHrpqe7vzlJOZWXUc0FhbyQUuI0bAxz4G\nF18MfX0pU9PfAROfztCYmVXHfzatrQwMpGBGSs8POCB9+B90ECxdCr/6FYwZ09w+DoczNGZm1XGG\nxtrKihVvzF584hNp6un3v4ePfrS9T+n2omAzs+pUcqXglyS9WM6jnh227rZiBYwc+cbt++4LV18N\nt94K++wDixc3vm+14CknM7PqVPJn80t164VZmUoFNAB77gnXXQd///fwgQ/AtdfCBhs0tn/D5Skn\nM7PqlB3QRMQF9eyIWTkGBkoHNADvf/+qLM0uu6SgZtttG9e/4XKGxsysOpVMOY0r91HPDlt3K7aG\nptC228Ls2fCmN8Guu8KsWY3pWy14DY2ZWXUqWRT8N+ClIR65MmZ1MdiUU74NNki3SdhhB9hrL7ig\nTfKLnnIyM6tOJd8DP1i3XpiVqdyABmD8+DTldMwx8PnPwwMPwKmnln98M3jKycysOpWsobmlnh0x\nK8dQa2gKjRkDP/85vOtd8OUvw0MPwX/9Vwp2WpEzNGZm1RnWdWgkjZW0laR35z9q1TmzQuWsoSkk\nwXHHpWzNHXfAe98L8+bVp3/D5QyNmVl1qgpoJL1V0lXAK8BDwD0FD7O6qGTKqdDUqXDnnSnA2XFH\nuOyy2vatFrwo2MysOtVmaH4KvAl4L7AE+AhwCPAn4KO16ZrZG1U65VToHe+AP/4xndb9qU+lzM3y\n5bXr33B5ysnMrDrVBjQfAr4cEXcDK4GnIuIi4HjgxFp1zqzQcDI0OePGpXU0Z54JZ52VLsI3f35N\nujdsnnIyM6tOtQHN2sBfs59fAt6a/fwAMGW4nTIrpZo1NMVIcPTRcNtt8Ne/ptO7L798+PUOlzM0\nZmbVqTageQzYMvv5PuCfJG0AHAn8by06ZlZMLTI0+XbcEebOhQ99CPbfHw49FF5+uXb1VyoX0LTy\nqeVmZq2o2oDmdGD97OfvAPsATwPHAv9Sg36ZFTXcNTTFrLtuWiB83nnp38mT0+0TmqG/P70+qTnt\nm5m1q6oCmoi4KCLOz36eA2wC7ARsFBG/ql33zFZX6wxNjpQuvnfffekqw7vvDv/yL7BsWe3bGszA\ngKebzMyqMazr0ORExOsRMTciFtWiPrNSarWGppTNN4dbboHvfhd+/GPYfnu4/fb6tVeov98Lgs3M\nqlHtdWgul/S1ItuPl/TrKus8RtKTkpZIukPSTkOU30PSHElLJc2TdEjB/m0kXZbVuVLSsbVo15qr\nXhmafCNHpuzMPfekKwrvuisceyy8+mp92wVnaMzMqlVthmY34Joi26/N9lVE0oHAT4CTgB1IC41n\nSppQovymwFXAjcBk0pqecyXtlVdsLPA4cAIlFipX2q41Xz3W0JSy7bbpLKjTToNf/CI9nzmzvm0O\nDDhDY2ZWjWoDmnWAgSLb+4FxVdQ3DTg7Ii6MiEdJZ0u9DhxWovxRwBMRcXxEPBYRZwKXZfUAEBF3\nR8QJEXEpUOrSaZW2a03WiAxNvpEj4UtfggcfhHe+Ez7yETjoIHjuufq05yknM7PqVBvQPAAcWGT7\nQcDDlVQkaTTQQ8q2ABARAdwA7FLisJ2z/flmDlK+Vu1ak9V7DU0pm20G110HF14IN98MW22V1tjk\nLoRXK55yMjOrTrUfDd8FfiPp7cBN2bY9gV7gUxXWNQEYCSws2L6QVde6KTSpRPlxksZERDnnplTT\nrjVZozM0+ST47Gfhox+Fk06CE06Ac85JwU2tPPKIMzRmZtWo6k9nRPxO0j+SrjmzP+l+TvcDH46I\nW2rYv5Y0bdo0xo8fv9q23t5eent7m9Sj7tHINTSljB8PP/1pugjfD38Ir7xSu7q33jrdisHMrJ31\n9fXR19e32rbFixfXtc2qvwtGxNXA1TXowyJgBTCxYPtEYEGJYxaUKP9ymdmZatsFYPr06UyZ4js8\nNMOKFTBmTLN7kUyeDBdf3OxemJm1nmJf8ufOnUtPT0/d2qz6OjSS3iTpC5K+L+nN2bYp2S0QyhYR\n/cAc0pRVrm5lz0tdAWR2fvnM1Gx7Pdu1JmvWGhozM2ttVX00SHo3afHsYmBT4FzgReATwMbA5yqs\n8jTgfElzgDtJZx+NBc7P2jsFeFtE5K41cxZwjKRTgV+SgpD9gX3z+jga2AYQsAawgaTJwKsR8Xg5\n7VrraeYaGjMza13Vftc9DTg/Io6XlL+C4Brgkkori4hLs2u/nEya8rkX2Dsins+KTAI2yis/X9J+\nwHTS/aOeBQ6PiPwzn94G3ANE9vyr2eMW4ENltmstphXW0JiZWeupNqDZCfinItufIwUfFYuIGcCM\nEvsOLbJtFum061L1PUUZU2qDtWutxxkaMzMrpto1NMsofgG9dwLObljdeA2NmZkVU21AcyXwrWyd\nCkBI2hg4Fbi8Jj0zK8JTTmZmVky1Ac1XSLc/+CuwFmldyp+BV4B/rU3XzN7IU05mZlZMtRfWWwzs\nJen9pJtDrgPMLViUa1ZznnIyM7NiKv5oyKaZfg8cGRG3AbfVvFdmJThDY2ZmxVQ85ZRdkO7ddeiL\n2ZC8hsbMzIqpdg3NRcDhteyIdba//hV6e+GBB4ZXjzM0ZmZWTLWrEUYBh0n6MOn2Aa/l74yILw+3\nY9ZZ7rkH/uu/4Le/hbPPTnetrobX0JiZWTHVfjRsB8zNfn5nwb7ArMCKFenfPfeEz30Obr893bG6\n0htNOkNjZmbFVHuW0wdr3RHrbLmA5uc/h9/9Dv7f/4O774a+Pthii/Lr8RoaMzMrpuq7bZtVYuXK\n9O/IkXDEEXDbbfDSS7D99nDuuRBl5vWcoTEzs2Ic0FhD5DI0uWCkpwfuvTctFD7iCPjHf4Tny7hp\nhtfQmJlZMQ5orCFyAc2IvN+4ddZJU1BXXJHW1Gy3HVx66eDZGmdozMysGAc01hD5U06FPvaxdDr3\n+98PBx4IH/0oPP108Xq8hsbMzIpxQGMNUSxDk2/SJPjNb9Jj7lzYdls444xVx+XX44DGzMwKOaCx\nhhgsQ5Pv4x+Hhx9O16k57jh473vTdFSO19CYmVkxDmisIQoXBQ9m/HiYMSOdCRWRpqI+/Wl45hln\naMzMrDgHNNYQQ005FfO+98Fdd8EvfgE33QRbbgn9/Q5ozMzsjRzQWEPkppwqCWhy5Q87DObNg2OP\nhTXWgA03rH3/zMysvTmgsYZYsSIFJ1J1x48bBz/4Abz2Guy7b237ZmZm7c/LK60hcgHNcHlBsJmZ\nFeMMjTXEypVe+2JmZvXjgMYaolYZGjMzs2L8EWMN4QyNmZnVkwMaawhfP8bMzOrJAY01hKeczMys\nnvwRYw3hKSczM6unlgloJB0j6UlJSyTdIWmnIcrvIWmOpKWS5kk6pEiZT0l6JKvzPkn7FOw/SdLK\ngsfDtX5t5iknMzOrr5YIaCQdCPwEOAnYAbgPmClpQonymwJXATcCk4HTgXMl7ZVX5n3AJcDPge2B\n3wJXSNqmoLoHgYnApOyxa61el63iKSczM6unVvmImQacHREXRsSjwJHA68BhJcofBTwREcdHxGMR\ncSZwWVZPzrHAtRFxWlbmW8Bc4J8L6hqIiOcj4q/Z48WavjIDPOVkZmb11fSARtJooIeUbQEgIgK4\nAdilxGE7Z/vzzSwov0sZZQDeIek5SY9LukjSRhW+BCuDMzRmZlZPrfARMwEYCSws2L6QNAVUzKQS\n5cdJGjNEmfw67wA+D+xNygptBsyStHYF/bcyOENjZmb11NV3xomImXlPH5R0J/AUcABwXnN61Zm8\nKNjMzOqpFQKaRcAK0sLcfBOBBSWOWVCi/MsRsWyIMqXqJCIWS5oHbDFYh6dNm8b48eNX29bb20tv\nb+9gh3U1TzmZmXWPvr4++vr6Vtu2ePHiurbZ9IAmIvolzQH2BK4EkKTs+RklDpsN7FOwbWq2Pb9M\nYR17FZRZjaR1SMHMhYP1efr06UyZMmWwIlbAU05mZt2j2Jf8uXPn0tPTU7c2W+U782nAEZI+J2kr\n4CxgLHA+gKRTJF2QV/4sYHNJp0raUtLRwP5ZPTmnAx+R9OWszLdJi4//I1dA0o8k7SZpk+w07/8G\n+oHVw0obNk85mZlZPTU9QwMQEZdm15w5mTQtdC+wd0Q8nxWZBGyUV36+pP2A6aTTs58FDo+IG/LK\nzJb0aeB72eNPwMciIv/CeRuSrlXzFuB54FZg54h4oT6vtHt5ysnMzOqpJQIagIiYAcwose/QIttm\nkTIug9V5OXD5IPu96KVBPOVkZmb15O/M1hDO0JiZWT35I8YawmtozMysnhzQWEN4ysnMzOrJAY01\nhKeczMysnvwRYw3hDI2ZmdWTAxprCK+hMTOzenJAYw3hKSczM6snf8RYQ3jKyczM6skBjTWEMzRm\nZlZP/oixhvAaGjMzqycHNNYQnnIyM7N6ckBjDeEpJzMzqyd/xFhDOENjZmb15IDGGsJraMzMrJ4c\n0FhDeMrJzMzqyR8x1hCecjIzs3pyQGMN4SknMzOrJwc01hCecjIzs3ryR4w1hKeczMysnhzQWEM4\nQ2NmZvXkjxhrCGdozMysnhzQWEN4UbCZmdWTAxprCE85mZlZPfkjxhrCU05mZlZPDmisITzlZGZm\n9eSAxhrCU05mZlZP/oixhvCUk5mZ1ZMDGmsIZ2jMzKyeWuYjRtIxkp6UtETSHZJ2GqL8HpLmSFoq\naZ6kQ4qU+ZSkR7I675O0z3DbtVX6+vrKLtvJGZpKxqHTeSwSj8MqHovE41B/LRHQSDoQ+AlwErAD\ncB8wU9KEEuU3Ba4CbgQmA6cD50raK6/M+4BLgJ8D2wO/Ba6QtE217drqKnmDdvKiYP+hWsVjkXgc\nVvFYJB6H+muJgAaYBpwdERdGxKPAkcDrwGElyh8FPBERx0fEYxFxJnBZVk/OscC1EXFaVuZbwFzg\nn4fRrlXJU05mZlZPTf+IkTQa6CFlWwCIiABuAHYpcdjO2f58MwvK7zJYmSrbtSp18pSTmZk136hm\ndwCYAIwEFhZsXwhsWeKYSSXKj5M0JiKWDVJm0jDaBeCRRwbb2z0WL4a5c8sru3y5AxozM6ufVgho\n2smaAAcf7IgmWUxPT5kRDZUFQO1k8eLFzO3EF1YFj0XicVjFY5F4HOCRVdmANetRfysENIuAFcDE\ngu0TgQUljllQovzLWXZmsDK5Oqtpd9P0z8EldnejnrJLnnJKenSinp7yx6HTeSwSj8MqHovE4/B/\nNgVur3WlTQ9oIqJf0hxgT+BKAEnKnp9R4rDZQOEp2FOz7fllCuvYK1emynZnAp8B5gNLh351ZmZm\nllmTFMzMrEflSutgm0vSAcD5pLOM7iSdfbQ/sFVEPC/pFOBtEXFIVn5T4AFgBvBLUhDyU2DfiLgh\nK7MLcDNwInA10At8HZgSEQ+X0259X7WZmZnVStMzNAARcWl27ZeTSVM+9wJ75wUVk4CN8srPl7Qf\nMJ10evazwOG5YCYrM1vSp4HvZY8/AR/LBTNltmtmZmZtoCUyNGZmZmbD0fTr0JiZmZkNlwMaMzMz\na3sOaCrQ6TeylPQBSVdKek7SSkkfLVLmZEl/kfS6pOslbVGwf4ykMyUtkvSKpMskrde4VzF8kk6U\ndKeklyUtlPTfkt5ZpFxHj4WkI7Obui7OHrdL+khBmY4eg2IkfT17f5xWsL3jx0LSSdlrz388XFCm\n48chR9LbJP1n9lpez94vUwrKdPR4ZJ+Jhb8TKyX9e16ZhoyBA5oyqTtuZLk2aWH00cAbFldJOoF0\nL6wvAu8BXiONwRp5xX4K7Ad8EtgNeBtweX27XXMfAP4deC/wYWA0cJ2ktXIFumQsngFOAKaQLjh0\nE/BbSVtD14zBarIvMV8kvf/zt3fTWDxIOoliUvbYNbejm8ZB0puA24BlwN7A1sBXgJfyynTDeOzI\nqt+FSaTLowRwKTR4DCLCjzIewB3A6XnPRTq76vhm961Or3cl8NGCbX8BpuU9HwcsAQ7Ie74M+Hhe\nmS2zut7T7Nc0jLGYkL2GXT0WvAAc2o1jAKwDPAZ8CPgf4LRu+30gfaGbO8j+rhiHrN8/AG4ZokzX\njEde/38KzGvGGDhDUwb5RpZI2owUfeePwcvAH1k1BjuSLgWQX+Yx4Gnae5zeRPrG8SJ051hIGiHp\nIGAscHs3jgFwJvC7iLgpf2MXjsU7lKalH5d0kaSNoCvH4R+AuyVdqjQ1PVfSF3I7u3A8cp+VnwF+\nkT1v6Bg4oCnPYDeynPTG4h1pEulDfbAxmAgsz35hS5VpK5JE+sZxa6y6hlHXjIWk7SS9QvoGNYP0\nLeoxumgMALJgbnvShToLddNY3AF8njTFciSwGTBL0tp01zgAbA4cRcraTQV+Bpwh6bPZ/m4bD4CP\nA+OBC7LnDR2DlriwnlkLmwFsA7y/2R1pkkeByaQ/UvsDF0rarbldaixJG5KC2g9HRH+z+9NMEZF/\nyfoHJd0JPAUcQPpd6SYjgDsj4pvZ8/skbUcK9P6zed1qqsOAayOi1P0Q68oZmvJUcyPLTrOAtG5o\nsDFYAKwhadwgZdqGpP8A9gX2iIj/zdvVNWMREQMR8URE3BMR/0paDHscXTQGpOnmtwJzJfVL6gd2\nB46TtJz0TbJbxmI1EbEYmAdsQXf9TgD8L/BIwbZHgI2zn7tqPCRtTDqJ4ud5mxs6Bg5oypB9K8vd\nyBJY7UaWNb9jaCuKiCdJv1z5YzCOdCZQbgzmAAMFZbYkvcHzbxza8rJg5mPAByPi6fx93TYWBUYA\nY7psDG4A3kWacpqcPe4GLgImR8QTdM9YrEbSOqRg5i9d9jsB6QynLQu2bUnKWHXj34nDSMH9NbkN\nDR+DZq+IbpcHKaX6OvA5YCvgbNIZH29tdt9q+BrXJv2x3p60wvxL2fONsv3HZ6/5H0h/4K8g3SNr\njbw6ZgBPAnuQvtneBvyh2a+twnGYQTr18gOkbwm5x5p5ZTp+LIDvZ2OwCbAdcEr2h+dD3TIGg4xN\n4VlOXTEWwI9Ip9VuArwPuJ70IfaWbhqH7HXsSFpbdiLwduDTwCvAQV34eyFgPvC9IvsaNgZNH4h2\nepCuzzKfdMrZbGDHZvepxq9vd1Igs6Lg8cu8Mt8mnYb3OukW8FsU1DGGdA2XRdmb+9fAes1+bRWO\nQ7ExWAF8rqBcR48FcC7wRPb7vgC4jiyY6ZYxGGRsbiIvoOmWsQD6SJerWEI6C+USYLNuG4e817Iv\ncH/2Wh8CDitSpuPHg3TtmRWFr63RY+CbU5qZmVnb8xoaMzMza3sOaMzMzKztOaAxMzOztueAxszM\nzNqeAxozMzNrew5ozMzMrO05oDEzM7O254DGzMzM2p4DGjMzM2t7DmjMrOYknSfpN8Os40lJxw6z\njt0lrSxyJ9+ak/RdSWeV0Z8V9e6PpL0l3VPPNsxajQMaM2tVOwLn1KCeut/fRdJE4Fjg3/K2/Y+k\n0wqK3gasHxEv17M/ETETWC7pM/Vsx6yVOKAxs5YUES9ExNJm96NMXwBui4hnBysUEQMR8dcG9ekC\n4LgGtWXWdA5ozNqYpP0l3S/pdUmLJF0naa1s347Z8+cl/U3SzZJ2KDh+paQvSvqdpNckPSxpZ0lv\nzzIMr0q6TdJmececJOme7Lins+N+Ndg0ipITJT2R9fUeSZ8c4rWtNuWU9fVwSb/J2pwn6R8KjtlX\n0mNZGzcCmxapd1dJs7IyT0k6XdLYbN+WWd0H5ZU/ICu71SDdPQj4Xd4x55HuXn9c1u8VkjYunAKT\ndIiklyTtJ+nRrO1LJa2V7XtS0otZH5VX/xqSfizp2ez/aLak3Qv69Dtgx/z/O7NO5oDGrE1JmgRc\nApwLbEX6AP0NkPvg+zvgfOB9wHuBecA1ktYuqOobWbnJwCNZnWcB3wN6svr+o+CYLYBPAfsBewM7\nAGcO0t1/AQ4GvghsA0wH/lPSB8p/xQB8C/gv4F3ANcDFkt4EIGlD4HLgt9lrORf4Qf7Bkt4OXAv8\nGtgOOBB4P/DvABHxGPBV4GeSNszq/BnwtYh4tFiHJK2bvaa78zYfB8wGfg5MBNYHnsn2FU6BjQX+\nH3AAaSw/CPw38BFgH9K4/ROwf94xZ5L+Tw/IxuLXwLXZ6yN7Lc8AC4FKx9isPUWEH3740YYPUhCx\nAtiozPIjgMXAvnnbVgLfznv+3mzbIXnbDgRey3t+ErAcmJS3bW+gH1gve34e8Jvs5zWAV4H3FvTn\n58BFg/T3SeDYQfo6Nts2NXv+feCBgjpOycZoXF6bPysosyswAKyRt+1K4BbgeuCaIcZ1ctbGBgXb\n/wc4rWDb7gX9OSR7vmlemZ8BrwBr5W27FpiR/bxxNtaTCuq+Hvi3gm1zgG82+3fVDz8a8Rg1ZMRj\nZq3qPuBG4EFJM4HrgMsi4m8AktYjZVl2B9YDRgJrkT4Q8z2Q9/PC7N8HC7atKWmdiHg12/Z0RCzI\nKzM7q39LoHCNyBak4OP6/GkTYDRQ6Zk4/9fXiHhd0suk1wYpS/XHgvKzC55PBt4l6eC8bbk+bQY8\nlv18OCmjtQLYdog+rZX9W+16n9cjYn7e84XA/IhYUrAt9zq3I431vILxXANYVFD3EtLYm3U8BzRm\nbSoiVgJTJe0CTCVNW3xP0nsi4ingQmDdbPvTwDLgDtIHX77+/GoH2VbtFPU62b/7An8p2Leswrr6\nC54HlfVrHeBs4HRWBTI5T+f9vD2wNimgWZ9VgV4xuSBiXeCFCvqSU+w1DfY61yFllKaQMlT5Xi14\n/mbg+Sr6ZNZ2HNCYtbmImA3MlvRd4Cng48BPSWtnjop0Ci+SNgImlFNlGWU2ljQpL0uzC+nD/7Ei\nZZ1MbzsAAAJ3SURBVB8mBS6bRMStZdRdrUeAfyjYtkvB87nANhHxZKlKsjUx55FOwV4fuETSDhFR\nKvh6nDRFtA3w57zty0mZlFq7J6t3YkTcVqqQpDHA26k8C2bWlrwo2KxNSXpPduZQTxasfJIUsDyc\nFfkT8FlJW0l6L3AR8Ho5VZexbRlwgaR3Zwt7Twd+FUVOSc6mqX4MTJf0/9u5f9UowigM4897DanF\nIMZWlIBoYatWi9iIsCIiKdRKm1iIojZKRCSNgpVg4QX4B0llZROQCCKksfEGLCw9Ft8Wa9xdVhSy\nkzy/cndgzszsLu9+58ycT7IvyaEkV5P0pzrY6TwBFpI8SHIgyTnajMqw+8CxJKtJDibZn6SXZHVo\nm6e0YHgPuE77nXw4bqdVVcAabRZn2FfgSJK9SeaG2kOjzu/UqmqTNrj9PMnpJPODz8JyklNDmx6l\ntcG2tt2kHclAI3XXd+A48Iq2MnIHuFZV7wbvX6S1QdZpzyR5zJ/zLaNWY6Z5bZN2R9Vr4C3wEbgy\nrtCqugncBZZpgesNrQU1dqVkxD4n1lXtrp4zQG9QzxJwY0sdn2gzRQvAe9qKzW3gG8AgYJ0E+lX1\ns6p+AH3gUpITE2p9Rrt1e9gKbdXqM+2875lwHH/rAq2luAJ8oV2LRX5vm50FXlR3nuUj/ZO0PxeS\nNJ0kt4BeVR3e7lpmSZIPwKOqejkDtczRgs7iYJ5K2vFcoZGk/2OJ2ZlLnAcuG2a0m8zKl0+SOq2q\nNoCN7a4DoKrWaa1Gadew5SRJkjrPlpMkSeo8A40kSeo8A40kSeo8A40kSeo8A40kSeo8A40kSeo8\nA40kSeo8A40kSeq8X7VM7CMmWRviAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108df46a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(accuracy)\n",
    "plt.xlabel('sample index (time)')\n",
    "plt.ylabel('recall (window size = 200)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature-based recommendation\n",
    "\n",
    "In the above examples, even if we create `User` and `Item` instances with `feature`, the classical recommenders cannot handle the auxiliary information. So, let us try alternative feature-based recommender. In particular, `FMRecommender` is a recommender based on one of the most famous recommendation techniques named [Factorization Machines](http://dl.acm.org/citation.cfm?id=2168771) (FMs).\n",
    "\n",
    "We need to pass the number of dimensions of feature vector as `p=8`. In this case, our feature vector will have 8 different elements (3 from user, 3 from item, and 2 from context)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from flurs.recommender.fm import FMRecommender"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# initialize a recommendation instance\n",
    "recommender = FMRecommender(p=8, k=2)\n",
    "recommender.initialize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "User/item registration and incremental updating are completely same as what we did before (but don't forget to put `context=...` to define an `Event` instance):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# register users and items\n",
    "recommender.register(user_a)\n",
    "recommender.register(user_b)\n",
    "recommender.register(user_c)\n",
    "\n",
    "recommender.register(item_a)\n",
    "recommender.register(item_b)\n",
    "\n",
    "# feed some events\n",
    "recommender.update(Event(user_a, item_b, context=np.array([1, 1])))\n",
    "recommender.update(Event(user_b, item_b, context=np.array([0, 2])))\n",
    "recommender.update(Event(user_a, item_a, context=np.array([1, 3])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, context-aware recommendation is launched as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# make recommendation to `user_c`\n",
    "candidates = np.array([0, 1])\n",
    "recommended_items, scores = recommender.recommend(user_c, candidates, context=np.array([0, 4]))  # pass current context\n",
    "recommended_items"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Algorithms\n",
    "\n",
    "FluRS currently supports the following recommendation algorithms:\n",
    "\n",
    "- Incremental Collaborative Filtering: `UserKNN`\n",
    "  - M. Pepagelis et al. **Incremental Collaborative Filtering for Highly-Scalable Recommendation Algorithms**. In *Foundations of Intelligent Systems*, pp. 553–561, Springer Berlin Heidelberg, 2005.\n",
    "- Incremental Matrix Factorization: `MF`\n",
    "  - J. Vinagre et al. **[Fast Incremental Matrix Factorization for Recommendation with Positive-only ](http://link.springer.com/chapter/10.1007/978-3-319-08786-3_41)**. In *Proc. of UMAP 2014*, pp. 459–470, July 2014.\n",
    "- Incremental Matrix Factorization with BPR optimization: `BPRMF`\n",
    "  - S. Rendle et al. **BPR: Bayesian Personalized Ranking from Implicit Feedback**. In *Proc. of UAI 2009*, pp. 452–461, June 2009.\n",
    "- Incremental Factorization Machines: `FM`\n",
    "  - T. Kitazawa. **[Incremental Factorization Machines for Persistently Cold-Starting Online Item Recommendation](https://arxiv.org/abs/1607.02858)**. arXiv:1607.02858 [cs.LG], July 2016.\n",
    "- Matrix Sketching: `OnlineSketch\n",
    "  - T. Kitazawa. **[Sketching Dynamic User-Item Interactions for Online Item Recommendation](http://dl.acm.org/citation.cfm?id=3022152)**. In *Proc. of CHIIR 2017*, March 2017."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### OSS for recommender systems\n",
    "\n",
    "- [Surprise](http://surpriselib.com/) (Python)\n",
    "- [fastFM](http://ibayer.github.io/fastFM/) (Python)\n",
    "- [Implicit](http://implicit.readthedocs.io/en/latest/) (Python)\n",
    "- [MyMediaLite](http://www.mymedialite.net/) (C#)\n",
    "- [LibRec](https://www.librec.net/) (Java)\n",
    "- [LensKit](http://lenskit.org/) (Java)\n",
    "\n",
    "If you are interested in recommendation on Apache Hadoop, Hive, Spark:\n",
    "\n",
    "- [Apache Mahout](http://mahout.apache.org/)\n",
    "- [Apache Hivemall](https://github.com/apache/incubator-hivemall)\n",
    "- [Spark MLlib](https://spark.apache.org/mllib/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}