Skip to content

Instantly share code, notes, and snippets.

@chezou
Last active July 29, 2016 08:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chezou/88568ce2bb620107cfdbdd20f0c966ae to your computer and use it in GitHub Desktop.
Save chezou/88568ce2bb620107cfdbdd20f0c966ae to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"<table>\n",
" <tr>\n",
" <th>Magic</th>\n",
" <th>Example</th>\n",
" <th>Explanation</th>\n",
" </tr>\n",
" <tr>\n",
" <td>info</td>\n",
" <td>%%info</td>\n",
" <td>Outputs session information for the current Livy endpoint.</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cleanup</td>\n",
" <td>%%cleanup -f</td>\n",
" <td>Deletes all sessions for the current Livy endpoint, including this notebook's session. The force flag is mandatory.</td>\n",
" </tr>\n",
" <tr>\n",
" <td>delete</td>\n",
" <td>%%delete -f -s 0</td>\n",
" <td>Deletes a session by number for the current Livy endpoint. Cannot delete this kernel's session.</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs</td>\n",
" <td>%%logs</td>\n",
" <td>Outputs the current session's Livy logs.</td>\n",
" </tr>\n",
" <tr>\n",
" <td>configure</td>\n",
" <td>%%configure -f<br/>{\"executorMemory\": \"1000M\", \"executorCores\": 4}</td>\n",
" <td>Configure the session creation parameters. The force flag is mandatory if a session has already been\n",
" created and the session will be dropped and recreated.<br/>Look at <a href=\"https://github.com/cloudera/livy#request-body\">\n",
" Livy's POST /sessions Request Body</a> for a list of valid parameters. Parameters must be passed in as a JSON string.</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sql</td>\n",
" <td>%%sql -o tables -q<br/>SHOW TABLES</td>\n",
" <td>Executes a SQL query against the sqlContext.\n",
" Parameters:\n",
" <ul>\n",
" <li>-o VAR_NAME: The result of the query will be available in the %%local Python context as a\n",
" <a href=\"http://pandas.pydata.org/\">Pandas</a> dataframe.</li>\n",
" <li>-q: The magic will return None instead of the dataframe (no visualization).</li>\n",
" <li>-m METHOD: Sample method, either <tt>take</tt> or <tt>sample</tt>.</li>\n",
" <li>-n MAXROWS: The maximum number of rows of a SQL query that will be pulled from Livy to Jupyter.\n",
" If this number is negative, then the number of rows will be unlimited.</li>\n",
" <li>-r FRACTION: Fraction used for sampling.</li>\n",
" </ul>\n",
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td>local</td>\n",
" <td>%%local<br/>a = 1</td>\n",
" <td>All the code in subsequent lines will be executed locally. Code must be valid Python code.</td>\n",
" </tr>\n",
"</table>\n"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%%help"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"Current session configs: <tt>{'kind': 'pyspark', 'driverMemory': '1000M', 'executorCores': 2}</tt><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"No active sessions."
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%%info"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"No logs yet."
]
}
],
"source": [
"%%logs"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Creating SparkContext as 'sc'\n"
]
},
{
"data": {
"text/html": [
"<table>\n",
"<tr><th>ID</th><th>YARN Application ID</th><th>Kind</th><th>State</th><th>Spark UI</th><th>Driver log</th><th>Current session?</th></tr><tr><td>4</td><td>None</td><td>pyspark</td><td>idle</td><td></td><td></td><td>✔</td></tr></table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Creating HiveContext as 'sqlContext'\n",
"SparkContext and HiveContext created. Executing user code ...\n",
"1000"
]
}
],
"source": [
"sc.parallelize(range(1000)).count()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/spark\n",
"/etc/hadoop/conf:/etc/hive/conf:/etc/hive/conf"
]
}
],
"source": [
"import os\n",
"print(os.environ.get('SPARK_HOME', None))\n",
"print(os.environ.get('HADOOP_CONF_DIR', None))"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"Current session configs: <tt>{'kind': 'pyspark', 'driverMemory': '1000M', 'executorCores': 2}</tt><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table>\n",
"<tr><th>ID</th><th>YARN Application ID</th><th>Kind</th><th>State</th><th>Spark UI</th><th>Driver log</th><th>Current session?</th></tr><tr><td>4</td><td>None</td><td>pyspark</td><td>idle</td><td></td><td></td><td>✔</td></tr></table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%%info"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1000"
]
}
],
"source": [
"sc.parallelize(range(1000)).count()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2000"
]
}
],
"source": [
"sc.parallelize(range(2000)).count()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"%%sql \n",
"show tables"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%%sql\n",
"select * from movies_pq_s3 limit 100"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"%%sql -o ratings\n",
"select movieid, rating from ratings_pq_s3"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.axes._subplots.AxesSubplot at 0x115f7d0b8>"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAecAAAFkCAYAAAAaKfMiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHMNJREFUeJzt3X9sW/W9//GX3cRN/SvlS8N0xVDDghGdkq0iKUTLbRSh\ndgvjxwY0U5M2DG40LUybEB6soU1n2IC0ICjTVLSibtLqIprokgLTpDGiCvIlvSJdtZUfVaNVKQNa\nvqVpu/r4pLGT2t8/uPFgaWO7xPEH5/n4Kz0+x7z9icXTv3LsSCaTSQEAAGM48z0AAAD4LOIMAIBh\niDMAAIYhzgAAGIY4AwBgGOIMAIBhitLtsHv3bvX29srhcCgWi+nQoUN67rnn9Nhjj8npdCoQCCgU\nCkmSenp61N3dreLiYrW3t6uhoSHX8wMAUHAc2fyd8y9+8QstWbJEe/bsUVtbm2pqahQKhbR8+XIt\nXbpUd999t3bv3q2xsTE1Nzert7dXxcXFuZwfAICCk/HL2m+//bYOHz6spqYmvfvuu6qpqZEk1dfX\na+/evXrrrbdUXV2toqIieb1elZeXa2hoKGeDAwBQqDKO87PPPquf/OQnU7Z7PB5Fo1HZti2fz5fa\n7na7ZVnWzEwJAMAcklGcLcvSe++9p2XLln1ykPNfh9m2Lb/fL6/Xq2g0OmX7dDhzKAAAU6X9QJgk\n7du3T7W1tal/L1myRPv27dOyZcvU39+v2tpaVVVVacuWLYrH44rFYhoeHlYgEJj2eh0Oh06c4Nl1\nLpWV+VjjWcA65x5rnHusce6VlfnS76QM43zkyBFdccUVqX+vW7dOGzdu1Pj4uCoqKtTY2CiHw6HW\n1la1tLQomUwqGAzK5XJd3PQAAMxhWX1aOxd4lJZbPBKeHaxz7rHGucca516mz5w5CQkAAIYhzgAA\nGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMA\nAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGKYo3wMAAPIr\nmUzKsiJyuRKKRKyMj/P5/HI4HDmcbO4izgAwx1lWRK++eVhlZf9HUTuW0TFnR22tvP4q+f2lOZ5u\nbiLOAAAtcHvk8fqV0Fi+R4F4zxkAAOMQZwAADEOcAQAwDHEGAMAwxBkAAMMQZwAADEOcAQAwDHEG\nAMAwxBkAAMMQZwAADEOcAQAwTEbn1n722We1Z88eTUxMaO3atbr22mvV0dEhp9OpQCCgUCgkSerp\n6VF3d7eKi4vV3t6uhoaGXM4OAEBBShvnwcFB/fWvf9WuXbs0Ojqq7du365VXXlEwGFRNTY1CoZD6\n+vq0dOlShcNh7d69W2NjY2publZdXZ2Ki4tn43YAAFAw0sb5jTfe0NVXX60f/ehHsm1bDzzwgF54\n4QXV1NRIkurr6zUwMCCn06nq6moVFRXJ6/WqvLxcQ0NDqqyszPmNAACgkKSN8+nTp3Xs2DFt27ZN\nH3zwge655x4lEonU5R6PR9FoVLZty+fzpba73W5ZVuZf2g0AAD6RNs4LFy5URUWFioqKdOWVV2r+\n/Pk6fvx46nLbtuX3++X1ehWNRqdsT6eszJd2H3w+rPHsYJ1zjzXODZcrIa/nlCTJ5y3J6Bin4lq0\nyKfSUn4nuZA2ztXV1QqHw7rrrrt0/PhxnT17VrW1tRocHNR1112n/v5+1dbWqqqqSlu2bFE8Hlcs\nFtPw8LACgUDaAU6c4Nl1LpWV+VjjWcA65x5rnDuRiKWoHZPHK1nRsYyOGbVjGhmxFI/zRz/ZyPQB\nZto4NzQ06C9/+YtWrVqlZDKphx56SJdffrk6Ozs1Pj6uiooKNTY2yuFwqLW1VS0tLUomkwoGg3K5\nXJ/7hgAAMNc4kslkMp8D8Eg4t3i2MTtY59xjjXMnEjmjN97+SF/6UlkWz5wt/WfVf8jvL83xdIUl\n02fOvB4BAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYh\nzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBh\niDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBg\nGOIMAIBhiDMAAIYhzgAAGIY4AwBgmKJMdrr99tvl9XolSV/+8pfV3t6ujo4OOZ1OBQIBhUIhSVJP\nT4+6u7tVXFys9vZ2NTQ05GxwAAAKVdo4x+NxSdKOHTtS2+655x4Fg0HV1NQoFAqpr69PS5cuVTgc\n1u7duzU2Nqbm5mbV1dWpuLg4d9MDAFCA0sb50KFDGh0dVVtbm86dO6f77rtPBw8eVE1NjSSpvr5e\nAwMDcjqdqq6uVlFRkbxer8rLyzU0NKTKysqc3wgAAApJ2jiXlJSora1NTU1Neu+99/SDH/xAyWQy\ndbnH41E0GpVt2/L5fKntbrdblmXlZmoAAApY2jiXl5dr8eLFqZ8XLlyogwcPpi63bVt+v19er1fR\naHTK9nTKynxp98HnwxrPDtY591jj3HC5EvJ6TkmSfN6SjI5xKq5Fi3wqLeV3kgtp49zb26uhoSGF\nQiEdP35c0WhUdXV1Ghwc1HXXXaf+/n7V1taqqqpKW7ZsUTweVywW0/DwsAKBQNoBTpzg2XUulZX5\nWONZwDrnHmucO5GIpagdk8crWdGxjI4ZtWMaGbEUj/NHP9nI9AFm2jivWrVK69ev15o1a+RwOLRp\n0yYtXLhQnZ2dGh8fV0VFhRobG+VwONTa2qqWlhYlk0kFg0G5XK7PfUMAAJhrHMlPv4GcBzwSzi2e\nbcwO1jn3WOPciUTO6I23P9KXvlSWxTNnS/9Z9R/y+0tzPF1hyfSZM69HAABgGOIMAIBhiDMAAIYh\nzgAAGIY4AwBgGOIMAIBhiDMAAIYhzgAAGIY4AwBgGOIMAIBhiDMAAIZJ+8UXAJAPyWRSlhVJ/dvl\nSigSSX9ubZ/PL4fDkcvRgJwjzgCMZFkRvfrmYS1weyRJXs8pRe3YtMecHbW18vqr+DIGfOERZwDG\nWuD2yO355Ft8PN4SJZTZNyYBX3S85wwAgGGIMwAAhiHOAAAYhjgDAGAY4gwAgGGIMwAAhiHOAAAY\nhjgDAGAY4gwAgGGIMwAAhiHOAAAYhjgDAGAY4gwAgGGIMwAAhiHOAAAYhjgDAGAY4gwAgGGIMwAA\nhiHOAAAYhjgDAGAY4gwAgGGIMwAAhskozidPnlRDQ4OOHDmi999/Xy0tLVq7dq0efvjh1D49PT26\n4447tHr1ar322mu5mhcAgIKXNs4TExMKhUIqKSmRJHV1dSkYDGrnzp1KJBLq6+vTyMiIwuGwuru7\ntX37dj355JMaHx/P+fAAABSitHHevHmzmpubddlllymZTOrgwYOqqamRJNXX12vv3r166623VF1d\nraKiInm9XpWXl2toaCjnwwMAUIiKpruwt7dXl156qerq6vSb3/xGkpRIJFKXezweRaNR2bYtn8+X\n2u52u2VZVkYDlJX50u+Ez4U1nh2s88xyuRLyek7J4y1JbfN96ufzcSquRYt8Ki3ld5GNybWW0q/x\nJNY6t9LG2eFwaGBgQENDQ1q3bp1Onz6duty2bfn9fnm9XkWj0SnbM3HiRGYRx8UpK/OxxrOAdZ55\nkYilqB1TQmOSPomGFR2b9phRO6aREUvxOJ91zcbkWnu8SrvGk1jri5Ppg/hpV3Xnzp0Kh8MKh8O6\n5ppr9Pjjj2v58uXat2+fJKm/v1/V1dWqqqrS/v37FY/HZVmWhoeHFQgEPv+tAABgDpr2mfP5rFu3\nThs3btT4+LgqKirU2Ngoh8Oh1tZWtbS0KJlMKhgMyuVy5WJeAAAKXsZx3rFjR+rncDg85fKmpiY1\nNTXNzFQAAMxhvFkAAIBhsn5ZGwAASMlkUpYVyeqYTD8QRpwBALgIlhXRq28e1gK3J6P9z47aqqj4\nckb7EmcAAC7SArdHbs/M/6037zkDAGAY4gwAgGGIMwAAhiHOAAAYhjgDAGAY4gwAgGGIMwAAhiHO\nAAAYhjgDAGAY4gwAgGGIMwAAhiHOAAAYhjgDAGAY4gwAgGGIMwAAhiHOAAAYhjgDAGAY4gwAgGGI\nMwAAhiHOAAAYhjgDAGAY4gwAgGGIMwAAhiHOAAAYhjgDAGAY4gwAgGGIMwAAhiHOAAAYhjgDAGAY\n4gwAgGGIMwAAhiHOAAAYpijdDolEQp2dnTpy5IicTqcefvhhuVwudXR0yOl0KhAIKBQKSZJ6enrU\n3d2t4uJitbe3q6GhIdfzAwBQcNLGec+ePXI4HHr++ec1ODiop556SslkUsFgUDU1NQqFQurr69PS\npUsVDoe1e/dujY2Nqbm5WXV1dSouLp6N2wEAQMFIG+cVK1bohhtukCQdO3ZMpaWl2rt3r2pqaiRJ\n9fX1GhgYkNPpVHV1tYqKiuT1elVeXq6hoSFVVlbm9hYAAFBg0sZZkpxOpx588EG9+uqr+tWvfqWB\ngYHUZR6PR9FoVLZty+fzpba73W5ZlpX2usvKfGn3wefDGs8O1nlmuVwJeT2n5PGWpLb5PvXz+TgV\n16JFPpWW8rvIxuRaS+nXeBJrff776HScimd83RnFWZK6urp0//33a9WqVYrFYqnttm3L7/fL6/Uq\nGo1O2Z7OiRPpA46LV1bmY41nAes88yIRS1E7poTGJH0SDSs6Nu0xo3ZMIyOW4nE+65qNybX2eJV2\njSex1lPvo+mM2rH0O/2vtKv64osvatu2bZKk+fPny+l0qrKyUoODg5Kk/v5+VVdXq6qqSvv371c8\nHpdlWRoeHlYgEMh4EAAA8Im0z5wbGxvV0dGhtWvXamJiQp2dnfrKV76izs5OjY+Pq6KiQo2NjXI4\nHGptbVVLS0vqA2Mul2s2bgMAAAUlbZxLSkr09NNPT9keDoenbGtqalJTU9PMTAYAwBw1d98sAADA\nUMQZAADDEGcAAAxDnAEAMAxxBgDAMMQZAADDEGcAAAyT8ek7AUjJZFKWFZmy3eVKKBK58Ok7fT6/\nHA5HLkcDUECIM5AFy4ro1TcPa4Hb85ntXs8pRS9w3tyzo7ZWXn+V/P7S2RgRQAEgzkCWFrg9cns+\n+008Hm9Jxie/B4B0eM8ZAADDEGcAAAxDnAEAMAxxBgDAMMQZAADDEGcAAAxDnAEAMAxxBgDAMMQZ\nAADDEGcAAAxDnAEAMAxxBgDAMMQZAADDEGcAAAxDnAEAMAxxBgDAMMQZAADDEGcAAAxDnAEAMAxx\nBgDAMMQZAADDEGcAAAxDnAEAMAxxBgDAMMQZAADDFE134cTEhNavX6+jR49qfHxc7e3tuuqqq9TR\n0SGn06lAIKBQKCRJ6unpUXd3t4qLi9Xe3q6GhobZmB8AgIIzbZxffvllXXLJJXr88ccViUT0ne98\nR9dcc42CwaBqamoUCoXU19enpUuXKhwOa/fu3RobG1Nzc7Pq6upUXFw8W7cDAGZNMpmUZUWyPs7n\n88vhcORgIhSaaeN84403qrGxUZJ07tw5zZs3TwcPHlRNTY0kqb6+XgMDA3I6naqurlZRUZG8Xq/K\ny8s1NDSkysrK3N8CAJhllhXRq28e1gK3J+Njzo7aWnn9VfL7S3M4GQrFtHFesGCBJCkajeree+/V\nfffdp82bN6cu93g8ikajsm1bPp8vtd3tdsuyrByNjPO50CN5lyuhSOTCvwseyQMXZ4HbI7fHl35H\n4CJMG2dJ+uijj/TjH/9Ya9eu1U033aQnnngidZlt2/L7/fJ6vYpGo1O2Z6KsjDv3TDhz5oxe+Z8P\n5P73R/LDpy54zOiorVsbvqrS0sx+V/jkwY7Xc0oeb8mUy3zn2SZJTsW1aJFPpaXc17NxvrW+0BpP\nmq21nu5+cCEm3w8mb4+Ufo0nzdbtSSaTikSyfwvB78/9E49s7wdOxTO+7mnjPDIyora2Nv385z9X\nbW2tJGnJkiXat2+fli1bpv7+ftXW1qqqqkpbtmxRPB5XLBbT8PCwAoFARgOcOMEz7JkQiVhKJIuU\nkOsz233eElnRsfMek0jGNDJiKR7nQ/uZikQsRe2YEvrsmk63zqM263wx/n2tp1vjSbO11he6H0zH\n5PvB5O3xeJV2jSfN3lqfMfYthGzvB6N2LOPrnjbO27ZtUyQS0TPPPKOtW7fK4XBow4YNeuSRRzQ+\nPq6Kigo1NjbK4XCotbVVLS0tSiaTCgaDcrlc0101AAAZmYtvIUwb5w0bNmjDhg1TtofD4Snbmpqa\n1NTUNHOTAQAwR5n3+goAAHMccQYAwDDEGQAAwxBnAAAMQ5wBADAMcQYAwDDEGQAAwxBnAAAMQ5wB\nADAMcQYAwDDEGQAAwxBnAAAMQ5wBADAMcQYAwDDEGQAAwxBnAAAMQ5wBADAMcQYAwDDEGQAAwxBn\nAAAMQ5wBADAMcQYAwDDEGQAAwxBnAAAMQ5wBADAMcQYAwDDEGQAAwxBnAAAMQ5wBADAMcQYAwDDE\nGQAAwxBnAAAMQ5wBADAMcQYAwDDEGQAAwxBnAAAMk1GcDxw4oNbWVknS+++/r5aWFq1du1YPP/xw\nap+enh7dcccdWr16tV577bWcDAsAwFyQNs7bt29XZ2enxsfHJUldXV0KBoPauXOnEomE+vr6NDIy\nonA4rO7ubm3fvl1PPvlkan8AAJCdtHFevHixtm7dmvr3u+++q5qaGklSfX299u7dq7feekvV1dUq\nKiqS1+tVeXm5hoaGcjc1AAAFLG2cV65cqXnz5qX+nUwmUz97PB5Fo1HZti2fz5fa7na7ZVnWDI8K\nAMDcUJTtAU7nv3pu27b8fr+8Xq+i0eiU7ZkoK/Ol3wlpuVwJeT2n5PGWTLnMd55tkuRUXIsW+VRa\nyu8gU6zz7DnfWl9ojSfN1lpPdz+4EJPvB5O3R0q/xpNY6+xncyqe8XVnHeevfvWr2rdvn5YtW6b+\n/n7V1taqqqpKW7ZsUTweVywW0/DwsAKBQEbXd+IEz7BnQiRiKWrHlNDYZ7b7vCWyomPnPWbUjmlk\nxFI8zof2M8U6z55/X+vp1njSbK31he4H0zH5fjB5ezxepV3jSax19rON2rGMrzvrOK9bt04bN27U\n+Pi4Kioq1NjYKIfDodbWVrW0tCiZTCoYDMrlcmV71QAAQBnG+fLLL9euXbskSeXl5QqHw1P2aWpq\nUlNT08xOBwDAHGTe6ysAAMxxxBkAAMMQZwAADEOcAQAwDHEGAMAwxBkAAMMQZwAADEOcAQAwDHEG\nAMAwxBkAAMMQZwAADEOcAQAwTNbfSgXATMlkUpYVyfo4n88vh8ORg4kAXCziDBQIy4ro1TcPa4Hb\nk/ExZ0dtrbz+Kvn9pTmcDEC2iDNQQBa4PXJ7fPkeA8DnxHvOAAAYhjgDAGAYXtZGTvEhJQDIHnFG\nTvEhJQDIHnFGzvEhJQDIDu85AwBgGOIMAIBhiDMAAIYhzgAAGIY4Z+m/Nu3J9whZ2frfB/I9Qtb+\n+/8ey/cIWfuirfOOPw3le4SsfdHWWPrirTNrbA7iDACAYfhTqiwc+3/HJUl/OXAo42Pmu+apakkg\nVyMBAAoQcc7CqX9akqRoMvO/2T1z+mSuxgEAFChe1gYAwDDEGQAAwxBnAAAMQ5wBADAMcQYAwDDE\nGQAAwxBnAAAMQ5wBADDMjJ6EJJlM6qGHHtLQ0JBcLpceffRRXXHFFTP5nwAAoODN6DPnvr4+xeNx\n7dq1Sz/96U/V1dU1k1cPAMCcMKNx3r9/v5YvXy5J+vrXv6533nlnJq8eAIA5YUZf1o5Go/L5/nXe\n6aKiIiUSCTmdhfHW9jyHQ5IUs0YyPqZIcUUiZ3I1UoplRXR21D7vZaO2dd7tZ0dtWVYkl2NNO1e6\n40xk6jpL2a/1qG3N2mwX43y350JrPMnEtZ6c+Yuy1unWeJJJa/3vM5s026dls68jmUwmL2ao89m0\naZOWLl2qxsZGSVJDQ4Nee+21mbp6AADmhBl9Snvttdfq9ddflyT97W9/09VXXz2TVw8AwJwwo8+c\nP/1pbUnq6urSlVdeOVNXDwDAnDCjcQYAAJ9fYXxSCwCAAkKcAQAwDHEGAMAwxBkAAMPkJc7JZFKh\nUEirV6/WnXfeqQ8++CAfY8wJBw4cUGtra77HKEgTExP62c9+pjVr1uh73/ue9uzZk++RClIikdD6\n9evV3NysNWvW6PDhw/keqWCdPHlSDQ0NOnLkSL5HKUi333677rzzTt15551av379tPvO6BnCMvXp\nc3AfOHBAXV1deuaZZ/IxSkHbvn27XnrpJXk8nnyPUpBefvllXXLJJXr88cd15swZffe739UNN9yQ\n77EKzp49e+RwOPT8889rcHBQTz31FP+/yIGJiQmFQiGVlJTke5SCFI/HJUk7duzIaP+8PHPmHNyz\nY/Hixdq6dWu+xyhYN954o+69915Jnzy7KyrKy2PdgrdixQr98pe/lCQdPXpUpaWleZ6oMG3evFnN\nzc267LLL8j1KQTp06JBGR0fV1tamu+66SwcOHJh2/7zE+ULn4MbMWrlypebNm5fvMQrWggUL5Ha7\nFY1Gde+99+q+++7L90gFy+l06sEHH9Sjjz6qW265Jd/jFJze3l5deumlqqurE6e+yI2SkhK1tbXp\nt7/9rR566CHdf//903YvL3H2er2y7X+dALyQvhwDc8tHH32k73//+7rtttv07W9/O9/jFLSuri69\n8sor6uzs1NjYWL7HKSi9vb0aGBhQa2urDh06pHXr1unkyZP5HquglJeX69Zbb039vHDhQp04ceKC\n++eliJyDe3bxSDg3RkZG1NbWpgceeEC33XZbvscpWC+++KK2bdsmSZo/f76cTicP5mfYzp07FQ6H\nFQ6Hdc0112jz5s269NJL8z1WQent7dWmTZskScePH5dt2yorK7vg/nl5k2zlypUaGBjQ6tWrJX3y\niBi54/jfr7rEzNq2bZsikYieeeYZbd26VQ6HQ9u3b5fL5cr3aAWlsbFRHR0dWrt2rSYmJrRhwwbW\nOIf4/0VurFq1SuvXr9eaNWvkcDj02GOPTfsgk3NrAwBgGF4bAgDAMMQZAADDEGcAAAxDnAEAMAxx\nBgDAMMQZAADDEGdgDvjwww+1YcMGSdI777yjjRs35nkiANPhTP3AHHD06NHUV7NWVlaqsrIyzxMB\nmA4nIQG+4AYHB/XEE08okUiotLRUTqdTlmXpxIkTuvnmmxUMBnXrrbfqww8/1G233aZvfetb+vWv\nf61wOKzW1lZ97Wtf0/79+3X69Gl1dnZq+fLlOn78uO6//35FIhEFAgHt27cvdcpdALnHy9pAAfjH\nP/6h3//+91q+fLluvvlmdXd36+WXX9Zzzz2nf/7zn+rs7FRlZWXq5exPn6JxYmJCu3btUkdHh55+\n+mlJ0qOPPqqbbrpJL730khobG/Xxxx/n5XYBcxUvawMF4Morr5TX69Xdd9+tN998U7/73e/097//\nXRMTEzp79uy0x05+t3ogENCZM2ckSQMDA6mT9K9YsUJ+vz+3NwDAZxBnoADMnz9fkrRp0yYdPXpU\nt9xyi1asWKG9e/em/VayyWMdDkdq33nz5n3mu2Z59wuYXbysDRSQvXv3qq2tTd/85jd17Ngxffzx\nxzp37pzmzZunc+fOZXw9dXV1+sMf/iBJev3112VZVq5GBnAexBkoID/84Q/1wAMPaPXq1frjH/+o\n66+/Xh9++KEqKioUiUS0bt26z+x/oa8HfPDBB/XnP/9Zt99+u/70pz/xsjYwy/i0NoApwuGwvvGN\nb6iiokIHDx7Uxo0b9cILL+R7LGDO4D1nAFMsXrxYwWBQTqdT8+fP1yOPPJLvkYA5hWfOAAAYhvec\nAQAwDHEGAMAwxBkAAMMQZwAADEOcAQAwzP8HXUoqBJEZl74AAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x115ab34e0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%%local\n",
"%matplotlib inline\n",
"import matplotlib\n",
"import seaborn as sns\n",
"import matplotlib.pyplot as plt\n",
"sns.distplot(ratings.rating, kde=False, rug=True)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "PySpark",
"language": "",
"name": "pysparkkernel"
},
"language_info": {
"codemirror_mode": {
"name": "python"
},
"mimetype": "text/x-python",
"name": "pyspark",
"pygments_lexer": "python2"
},
"widgets": {
"state": {
"034842d547674db694a02e123211a305": {
"views": []
},
"074b23332439401592c6c4054f588dad": {
"views": []
},
"0885b1a904d3424096810bf1116a3e9d": {
"views": []
},
"0bfcee70c34247b3959bb63e7fc77a71": {
"views": []
},
"0e3bfbb104cd4b1fbc02a30de6580df2": {
"views": []
},
"11df1ff0971f4e4386a524901d9b20ba": {
"views": []
},
"1396d1c4cda74680a41c882715387c73": {
"views": [
{
"cell_index": 10
}
]
},
"141e57b199094edc8105b0610af1428f": {
"views": []
},
"14c9eb6b1db040fc8dcf09f9ec665d19": {
"views": []
},
"184fa1e2bb4b4521859a74402927ae8a": {
"views": []
},
"192cf0c7231949e4b145c3432a2491d3": {
"views": [
{
"cell_index": 9
}
]
},
"19dbb4c7963a41cd91db658ca2083f51": {
"views": []
},
"19e5be7f7ba547f58bc75e76ed36a2b8": {
"views": []
},
"214fecaca5e0460dafed45cb31da4be7": {
"views": [
{
"cell_index": 8
}
]
},
"22a5797ffc2d469784014404d82a47f9": {
"views": []
},
"22ca476ddb4645e6b6cb75071e9ee2f9": {
"views": []
},
"23139acf7a6f453396c3a3b68197586b": {
"views": []
},
"24320b3bddbc4ea4a0259ce82c1e4759": {
"views": []
},
"259edcc74da2485c8ce0866e44efa9b0": {
"views": []
},
"26554f042c674c83b6a81701545dac0b": {
"views": []
},
"28226795a7e44d0297339b6f5259f122": {
"views": []
},
"2aa925686dea495eab2d83bf3c43e32d": {
"views": []
},
"2e46ff0e5268466b8843f57a09643232": {
"views": []
},
"2feba535f6f7436dbb26aba1c2f84425": {
"views": []
},
"30ed6638416f4846999f2b6be3ecbb2c": {
"views": []
},
"3166b6555f7d495d9bb251f5ed1c7c9d": {
"views": []
},
"3268208b0a394cd99899a893292164ff": {
"views": []
},
"3504347b556948eb99c30ccb620040ac": {
"views": []
},
"38fb69c2361f4a089fa17ae4e8df3f68": {
"views": []
},
"3914382ac3f64a94819dd565145e668e": {
"views": []
},
"3a1fe50c31464de4bbe0f16e09d917b7": {
"views": []
},
"3a55f05a284046ff99eedf4fadd3652a": {
"views": []
},
"3cd222b658e34662af3896acbf2e7d55": {
"views": []
},
"3d62b5d649cb42eb8f56b1ae5b2d6d5b": {
"views": []
},
"402ea933da724f158e7138c55335da28": {
"views": []
},
"42dd42bef2e34611a06f00bccdfd0ed0": {
"views": []
},
"450b0a056d514f7ca645ad72699dccb2": {
"views": []
},
"468cdf8133814989aea154636da80330": {
"views": []
},
"47fd3f9fc86040db8da50fa5b2b783dc": {
"views": []
},
"484215ba5bce487baa14d4c8035c3651": {
"views": []
},
"49d228af2e2242ef80c1718bcc39c1c5": {
"views": []
},
"4c359ee4c0b541efb9f3c976f459f216": {
"views": []
},
"4f4dfd3a9c8e41569b04d329a76bd510": {
"views": []
},
"52ccff6a06204618bd8596c79138b5f6": {
"views": []
},
"5388ecfe135f4aceb2df557361c31b6f": {
"views": []
},
"5530df53ff8848aa9dfc7c0056a84c33": {
"views": []
},
"56fc65076377407da7595f81e85c526c": {
"views": []
},
"571268157ce744dead6260c9e50fc96a": {
"views": []
},
"5a11c59b42184e5bb099a546c424d7ae": {
"views": []
},
"5b80bd7f072c458b81be90643483b8bc": {
"views": []
},
"5ba42de38188412f812314be02f68ec0": {
"views": []
},
"5f3c200ee6644011b2562b7c912923a2": {
"views": []
},
"609023f4ee2c48b99c2b88d94da8f29a": {
"views": []
},
"60cec193ee9f4635a0cfaf72d5dbe53b": {
"views": []
},
"61c9b91b35214d73988fc9fc001f6176": {
"views": []
},
"62cf073829db4cf2b42c920cca3a5a50": {
"views": []
},
"62f91ed2b81542a69e38c8938fe838b1": {
"views": []
},
"6727f64ab0d74e6497eb574135490147": {
"views": [
{
"cell_index": 9
}
]
},
"67a6a7eb5d944e0ca5ce5423bd07e914": {
"views": []
},
"6c9e901f48294b1d9b081815ff7cda45": {
"views": []
},
"6dc281639cf4400f8ec7a6e3e9a03c1c": {
"views": []
},
"6edcd93c92c74cd2aece127bba97519a": {
"views": [
{
"cell_index": 8
}
]
},
"6f8eb4cfc0b84ef0a3b4bd2fff618049": {
"views": []
},
"727031474bfa4433a58e27db7e49c8b1": {
"views": []
},
"7671acb7e00c43348333a4590c145e17": {
"views": []
},
"76b5b5dd82fb4ef5a979068241be9bfc": {
"views": []
},
"78e92a0c28724e738d2cbe3fe996b31f": {
"views": []
},
"796da4892f12465a9a6fe1189ef08efc": {
"views": []
},
"7a8752a4c4884edaaf9693fa61cdb773": {
"views": []
},
"7e7c938d937247aa9f94858dba6f6a78": {
"views": []
},
"83f95564c1194571ade5109f6253c577": {
"views": []
},
"85f655ea9f054df7b13d017996651476": {
"views": []
},
"86a79e9cc4e448fd911a87c584f172a4": {
"views": []
},
"87f721cdf41549188f7aeb6e6122852b": {
"views": []
},
"881e643973b04333a0ceb8893853ef2d": {
"views": []
},
"89c7275548b14eccae10923d4611cfcc": {
"views": []
},
"89efa1f11c5849bcb09e122666fe976d": {
"views": []
},
"8c69e56e066e4945a878efd2a62fdcfe": {
"views": []
},
"93246d82cb0b4c77b85cf55b9dc99608": {
"views": []
},
"963372574c6d44199726238990ccce39": {
"views": []
},
"964bf254f1d74526824ca6f37dff7855": {
"views": []
},
"971bca7ae0d04308875eeb634914c4f2": {
"views": []
},
"9933ae3e398d4671aff037213914efb0": {
"views": []
},
"9990ec24af4e459487873d9d44085f27": {
"views": []
},
"9b73ecc79eb94cd2bd26031596c402c4": {
"views": []
},
"9b988e5ff1dc40a69e5cdb5fdb8a6bfd": {
"views": []
},
"9d664e398b4a4d3ab7e22cb119d4f5b4": {
"views": []
},
"9ef67d0355ca4345adb257bd1235f9e6": {
"views": []
},
"a2a07c5c277f4e38bb0029720fb22bdc": {
"views": []
},
"a3452b2fdfd24400aa3ad22171d4c8ff": {
"views": []
},
"a44af55cd42344948cf57b61f54ed0dd": {
"views": []
},
"a78f84a286b441c3ab4aa68f7f48960b": {
"views": []
},
"aa0848583b2f469ea2d96af460377c1e": {
"views": []
},
"ab814dce4cf347af91f8a74c0a9cdfee": {
"views": []
},
"acf967842b5f4f219d2c58a2bd90e2c5": {
"views": []
},
"b0e0f3e0ce2b4624bc771cca3c525d20": {
"views": []
},
"b34816d2685e4584bba941d762e012e1": {
"views": []
},
"b43f74458700439a986e303f951a53e1": {
"views": []
},
"b49b957cfbee4d308d0029f08e003a12": {
"views": []
},
"b844b723a3b541c9a634c3236ef81c97": {
"views": []
},
"b883c2740ec342c3bd02903e88587eb4": {
"views": [
{
"cell_index": 10
}
]
},
"b9022cf0242c487393de8378076c6598": {
"views": []
},
"b996694eb6fd4ca28b0f0b21dd495775": {
"views": []
},
"bcc2d991d65e431e8c2323e1f1187373": {
"views": []
},
"beb96ca03c3344c38be632180df76248": {
"views": []
},
"c1127c2009c24ae19eb30d384513a8a8": {
"views": []
},
"c19c4b9d5501489eb23ceb6d02114ef7": {
"views": []
},
"c1b1e1bd49b34efbbcd87687bc2ed373": {
"views": []
},
"c452a6e4a41f44289ec5bcea5d4d0f0e": {
"views": []
},
"c4aaea9cc5ec4cfe8e574e742ad981ee": {
"views": []
},
"c81fb3d731724b1a8397f61987fbba86": {
"views": []
},
"c8631c9cbbc24de4b8b110340b99877b": {
"views": []
},
"cb5e49411cc74b169587080b3d5f2da6": {
"views": []
},
"d0d913a1831b4d8d911daf918719634f": {
"views": []
},
"d54b02c556734ca183a850e3aba38978": {
"views": []
},
"d962348cde034617aadb66959a787c0b": {
"views": []
},
"dfa1aca753944c9493ad62f3210da30e": {
"views": []
},
"e15eaf8613ef4969b100caf585ae903c": {
"views": []
},
"e801a7365ba549179ee6318aed84c62e": {
"views": []
},
"ed58a4dc63fd43aaa3890b97bd56f0e1": {
"views": []
},
"ef2bf76cc81948b8bf473a24c773a7d5": {
"views": []
},
"f3ca79a38b7f47dc99d0bd05045563ca": {
"views": []
},
"f71f17bf315e44ed86e45c969149ce73": {
"views": []
},
"f7a26e628e124fe398cf923aa28cc3f1": {
"views": []
},
"f7f64dc4a61a4feba8a949fdd95cf160": {
"views": []
},
"f87fbf5a6a364f9eabae4e2359ec1211": {
"views": []
},
"fa183906e57f4bb288e28b5943c1ff8b": {
"views": []
},
"fa471c60fed2497c843aac04ddcd91d5": {
"views": []
},
"fdd9aebb66434141b34c5e7774f0ed58": {
"views": []
},
"fe0ca149686a4923bd12abbce63b3ca6": {
"views": []
},
"fe649b813cf149b394a8eedc55c57b40": {
"views": []
},
"ffe58d245f764d7090211256ef851500": {
"views": []
}
},
"version": "1.1.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment