Skip to content

Instantly share code, notes, and snippets.

@fletchjeff

fletchjeff/D3 v4 with Jupyter.ipynb

Created February 9, 2018 13:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save fletchjeff/5bbb578c0969e055efdbfd2f149d227e to your computer and use it in GitHub Desktop.
Save fletchjeff/5bbb578c0969e055efdbfd2f149d227e to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
D3 v4 with Jupyter.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# D3 v4 with Jupyter"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook shows the process of displaying a d3 v4 graph widget in a Jupyter notebook using data that lives inside a Pandas DataFrame. I orginally followed the process detailed [here](https://github.com/stitchfix/d3-jupyter-tutorial) by stitchfix. Its very well done but unfortunately it doesn't seem to work work for d3 v4. Calling d3 v3 using `HTML('<script src=\"lib/d3/d3.min.js\"></script>')` is fine, but v4 is not loading with require.js.\n",
"\n",
"I wanted to get this working with d3 v4 and also using the `Javascript` display function rather than pushing in a script tag through the `HTML` display function. This it not the most exciting example ever, but it gets the process of moving data in a Pandas DataFrame into the DOM and accessible by D3. You can then edit your data queries and modify the output that D3 is displaying in one environment.\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"from IPython.display import Javascript,HTML\n",
"import json\n",
"import seaborn as sns\n",
"import pandas"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"# This will use the Iris dataset that seems to accompany most R examples. Its part of seaborn. Handy!\n",
"iris = sns.load_dataset('iris')"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>sepal_length</th>\n",
" <th>sepal_width</th>\n",
" <th>petal_length</th>\n",
" <th>petal_width</th>\n",
" <th>species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>48</th>\n",
" <td>5.3</td>\n",
" <td>3.7</td>\n",
" <td>1.5</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>109</th>\n",
" <td>7.2</td>\n",
" <td>3.6</td>\n",
" <td>6.1</td>\n",
" <td>2.5</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>5.4</td>\n",
" <td>3.9</td>\n",
" <td>1.7</td>\n",
" <td>0.4</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>20</th>\n",
" <td>5.4</td>\n",
" <td>3.4</td>\n",
" <td>1.7</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>4.7</td>\n",
" <td>3.2</td>\n",
" <td>1.6</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>123</th>\n",
" <td>6.3</td>\n",
" <td>2.7</td>\n",
" <td>4.9</td>\n",
" <td>1.8</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>98</th>\n",
" <td>5.1</td>\n",
" <td>2.5</td>\n",
" <td>3.0</td>\n",
" <td>1.1</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" <tr>\n",
" <th>78</th>\n",
" <td>6.0</td>\n",
" <td>2.9</td>\n",
" <td>4.5</td>\n",
" <td>1.5</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" <tr>\n",
" <th>139</th>\n",
" <td>6.9</td>\n",
" <td>3.1</td>\n",
" <td>5.4</td>\n",
" <td>2.1</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99</th>\n",
" <td>5.7</td>\n",
" <td>2.8</td>\n",
" <td>4.1</td>\n",
" <td>1.3</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" sepal_length sepal_width petal_length petal_width species\n",
"48 5.3 3.7 1.5 0.2 setosa\n",
"109 7.2 3.6 6.1 2.5 virginica\n",
"5 5.4 3.9 1.7 0.4 setosa\n",
"20 5.4 3.4 1.7 0.2 setosa\n",
"29 4.7 3.2 1.6 0.2 setosa\n",
"123 6.3 2.7 4.9 1.8 virginica\n",
"98 5.1 2.5 3.0 1.1 versicolor\n",
"78 6.0 2.9 4.5 1.5 versicolor\n",
"139 6.9 3.1 5.4 2.1 virginica\n",
"99 5.7 2.8 4.1 1.3 versicolor"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.sample(10)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['setosa', 'versicolor', 'virginica']"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# There are only 3 species types in the data, so colors of the bars are assigned with `d3.schemeCategory10`\n",
"iris['species'].drop_duplicates().tolist()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Editing javascript a Python notebook is not great. You don't get syntax highlighting, code completion or other handy stuff that a good editor will do. I use Sublime Text and had a file that I edited and re-loaded using the code below."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"#Javascript(filename='some_js_file.js')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Actual Javascript\n",
"This was trickier than it should be as the functions and variables aren't accessible to other `Javascript` instances unless you use the `window.` prefix. This is one function `udpater` that does all the work. Since the `svg` variable also not accessiable outside this function its necessary to check if it existing on each iteration and create it on the first one. \n",
"\n",
"The d3 stuff was quite straight forware. This uses the new d3 v4 enter-update-exit pattern described [here](https://bl.ocks.org/mbostock/3808218). I also grabbed some code from [here](https://bl.ocks.org/d3noob/bdf28027e0ce70bd132edc64f1dd7ea4). "
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"hide_input": false
},
"outputs": [
{
"data": {
"application/javascript": [
"\n",
"require.config({\n",
" paths: {\n",
" d3: \"https://d3js.org/d3.v4.min\"\n",
" }\n",
"});\n",
"\n",
"require([\"d3\"], function(d3) {\n",
"\n",
" window.updater = function(data) {\n",
"\n",
" var svg_margin = { top: 20, right: 20, bottom: 20, left: 40 };\n",
" var svg_width = 960 - svg_margin.left - svg_margin.right;\n",
" var svg_height = 500 - svg_margin.top - svg_margin.bottom;\n",
"\n",
" var y = d3.scaleLinear()\n",
" .domain([0, d3.max(data, function(d) { return d.petal_length; })])\n",
" .range([svg_height, 0]);\n",
"\n",
" var x = d3.scaleBand()\n",
" .domain(d3.range(data.length))\n",
" .range([0, svg_width])\n",
" .padding(0.1);\n",
"\n",
" var species_list = d3.map(data, function (d) { return d.species;}).keys();\n",
"\n",
" if (d3.select(\"#svg_container\").select(\"svg\").empty()) {\n",
"\n",
" window.svg = d3.select(\"#svg_container\").append(\"svg\")\n",
" .attr(\"width\", svg_width + svg_margin.left + svg_margin.right)\n",
" .attr(\"height\", svg_height + svg_margin.top + svg_margin.bottom)\n",
" .append(\"g\")\n",
" .attr(\"transform\",\n",
" \"translate(\" + svg_margin.left + \",\" + svg_margin.top + \")\");\n",
"\n",
" svg.append(\"g\")\n",
" .attr(\"transform\", \"translate(0,\" + svg_height + \")\")\n",
" .attr(\"class\", \"x axis\")\n",
" .call(d3.axisBottom(x));\n",
"\n",
" // add the y Axis\n",
" svg.append(\"g\")\n",
" .attr(\"class\", \"y axis\")\n",
" .call(d3.axisLeft(y));\n",
" } else {\n",
" svg.selectAll(\"g.y.axis\")\n",
" .call(d3.axisLeft(y));\n",
"\n",
" svg.selectAll(\"g.x.axis\")\n",
" .call(d3.axisBottom(x));\n",
" }\n",
"\n",
" // DATA JOIN\n",
" // Join new data with old elements, if any.\n",
"\n",
" var bars = svg.selectAll(\".bar\")\n",
" .data(data);\n",
"\n",
" // UPDATE\n",
" // Update old elements as needed.\n",
"\n",
" bars\n",
" .attr(\"style\",function(d) { return \"fill:\" + d3.schemeCategory10[species_list.indexOf(d.species)];})\n",
" .attr(\"x\", function(d, i) { return x(i); })\n",
" .attr(\"width\", x.bandwidth())\n",
" .transition()\n",
" .duration(100)\n",
" .attr(\"y\", function(d) { return y(d.petal_length); })\n",
" .attr(\"height\", function(d) { return svg_height - y(d.petal_length); });\n",
"\n",
" // ENTER + UPDATE\n",
" // After merging the entered elements with the update selection,\n",
" // apply operations to both.\n",
"\n",
" bars.enter().append(\"rect\")\n",
" .attr(\"class\", \"bar\")\n",
" .attr(\"style\",function(d) { return \"fill:\" + d3.schemeCategory10[species_list.indexOf(d.species)];})\n",
" .attr(\"x\", function(d, i) { return x(i); })\n",
" .attr(\"width\", x.bandwidth())\n",
" .attr(\"y\", function(d) { return y(d.petal_length); })\n",
" .attr(\"height\", function(d) { return svg_height - y(d.petal_length); })\n",
" .merge(bars);\n",
"\n",
" // EXIT\n",
" // Remove old elements as needed.\n",
"\n",
" bars.exit().remove();\n",
"\n",
" };\n",
"});\n"
],
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Javascript(\"\"\"\n",
"require.config({\n",
" paths: {\n",
" d3: \"https://d3js.org/d3.v4.min\"\n",
" }\n",
"});\n",
"\n",
"require([\"d3\"], function(d3) {\n",
"\n",
" window.updater = function(data) {\n",
"\n",
" var svg_margin = { top: 20, right: 20, bottom: 20, left: 40 };\n",
" var svg_width = 960 - svg_margin.left - svg_margin.right;\n",
" var svg_height = 500 - svg_margin.top - svg_margin.bottom;\n",
"\n",
" var y = d3.scaleLinear()\n",
" .domain([0, d3.max(data, function(d) { return d.petal_length; })])\n",
" .range([svg_height, 0]);\n",
"\n",
" var x = d3.scaleBand()\n",
" .domain(d3.range(data.length))\n",
" .range([0, svg_width])\n",
" .padding(0.1);\n",
"\n",
" var species_list = d3.map(data, function (d) { return d.species;}).keys();\n",
"\n",
" if (d3.select(\"#svg_container\").select(\"svg\").empty()) {\n",
"\n",
" window.svg = d3.select(\"#svg_container\").append(\"svg\")\n",
" .attr(\"width\", svg_width + svg_margin.left + svg_margin.right)\n",
" .attr(\"height\", svg_height + svg_margin.top + svg_margin.bottom)\n",
" .append(\"g\")\n",
" .attr(\"transform\",\n",
" \"translate(\" + svg_margin.left + \",\" + svg_margin.top + \")\");\n",
"\n",
" svg.append(\"g\")\n",
" .attr(\"transform\", \"translate(0,\" + svg_height + \")\")\n",
" .attr(\"class\", \"x axis\")\n",
" .call(d3.axisBottom(x));\n",
"\n",
" // add the y Axis\n",
" svg.append(\"g\")\n",
" .attr(\"class\", \"y axis\")\n",
" .call(d3.axisLeft(y));\n",
" } else {\n",
" svg.selectAll(\"g.y.axis\")\n",
" .call(d3.axisLeft(y));\n",
"\n",
" svg.selectAll(\"g.x.axis\")\n",
" .call(d3.axisBottom(x));\n",
" }\n",
"\n",
" // DATA JOIN\n",
" // Join new data with old elements, if any.\n",
"\n",
" var bars = svg.selectAll(\".bar\")\n",
" .data(data);\n",
"\n",
" // UPDATE\n",
" // Update old elements as needed.\n",
"\n",
" bars\n",
" .attr(\"style\",function(d) { return \"fill:\" + d3.schemeCategory10[species_list.indexOf(d.species)];})\n",
" .attr(\"x\", function(d, i) { return x(i); })\n",
" .attr(\"width\", x.bandwidth())\n",
" .transition()\n",
" .duration(100)\n",
" .attr(\"y\", function(d) { return y(d.petal_length); })\n",
" .attr(\"height\", function(d) { return svg_height - y(d.petal_length); });\n",
"\n",
" // ENTER + UPDATE\n",
" // After merging the entered elements with the update selection,\n",
" // apply operations to both.\n",
"\n",
" bars.enter().append(\"rect\")\n",
" .attr(\"class\", \"bar\")\n",
" .attr(\"style\",function(d) { return \"fill:\" + d3.schemeCategory10[species_list.indexOf(d.species)];})\n",
" .attr(\"x\", function(d, i) { return x(i); })\n",
" .attr(\"width\", x.bandwidth())\n",
" .attr(\"y\", function(d) { return y(d.petal_length); })\n",
" .attr(\"height\", function(d) { return svg_height - y(d.petal_length); })\n",
" .merge(bars);\n",
"\n",
" // EXIT\n",
" // Remove old elements as needed.\n",
"\n",
" bars.exit().remove();\n",
"\n",
" };\n",
"});\n",
"\"\"\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The HTML\n",
"This needs some HTML scafholding for d3 to select. You can add more complex styling, HTML inputs etc here and also access it via a file. "
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"<style>\n",
"\n",
".axis {\n",
" font: 12px sans-serif;\n",
"}\n",
"\n",
"</style>\n",
"<div id='svg_container' class=\"\"></div>\n"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"HTML(\"\"\"\n",
"<style>\n",
"\n",
".axis {\n",
" font: 12px sans-serif;\n",
"}\n",
"\n",
"</style>\n",
"<div id='svg_container' class=\"\"></div>\n",
"\"\"\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Binding the Data\n",
"Re-run the code below to update the graph. You can change the sample size to add or remove bars from the graph."
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"application/javascript": [
"\n",
"require([\"d3\"], function(d3) {\n",
" updater([{\"sepal_length\": 6.9, \"sepal_width\": 3.1, \"petal_length\": 5.4, \"petal_width\": 2.1, \"species\": \"virginica\"}, {\"sepal_length\": 6.2, \"sepal_width\": 2.2, \"petal_length\": 4.5, \"petal_width\": 1.5, \"species\": \"versicolor\"}, {\"sepal_length\": 5.7, \"sepal_width\": 2.5, \"petal_length\": 5.0, \"petal_width\": 2.0, \"species\": \"virginica\"}, {\"sepal_length\": 5.6, \"sepal_width\": 2.7, \"petal_length\": 4.2, \"petal_width\": 1.3, \"species\": \"versicolor\"}, {\"sepal_length\": 7.1, \"sepal_width\": 3.0, \"petal_length\": 5.9, \"petal_width\": 2.1, \"species\": \"virginica\"}, {\"sepal_length\": 5.0, \"sepal_width\": 2.3, \"petal_length\": 3.3, \"petal_width\": 1.0, \"species\": \"versicolor\"}, {\"sepal_length\": 5.7, \"sepal_width\": 2.6, \"petal_length\": 3.5, \"petal_width\": 1.0, \"species\": \"versicolor\"}, {\"sepal_length\": 5.0, \"sepal_width\": 3.5, \"petal_length\": 1.3, \"petal_width\": 0.3, \"species\": \"setosa\"}, {\"sepal_length\": 4.6, \"sepal_width\": 3.1, \"petal_length\": 1.5, \"petal_width\": 0.2, \"species\": \"setosa\"}, {\"sepal_length\": 7.2, \"sepal_width\": 3.6, \"petal_length\": 6.1, \"petal_width\": 2.5, \"species\": \"virginica\"}])\n",
"});\n"
],
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# This selects a random sample of 10 rows from the Iris DataFrom. Its push to the DOM in d3 specific format by \n",
"# converting the data frame to a dictionary, oriented by `records`. At this point is just a long, correctly formatted\n",
"# string being run inside the browser that creates an oject that d3 can work with. \n",
"\n",
"record_set = json.dumps(iris.sample(10).to_dict(orient='records'))\n",
"\n",
"Javascript(\"\"\"\n",
"require([\"d3\"], function(d3) {{\n",
" updater({})\n",
"}});\n",
"\"\"\".format(record_set))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Registering the d3 module to use on the console\n",
"Occasionally you are going to need to debug things on the console, and if your run the code snippet below, it adds d3 as a global function."
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"application/javascript": [
"\n",
"require([\"d3\"], function(d3) {\n",
" window.d3 = d3;\n",
"});\n"
],
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Javascript(\"\"\"\n",
"require([\"d3\"], function(d3) {\n",
" window.d3 = d3;\n",
"});\n",
"\"\"\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment