Skip to content

Instantly share code, notes, and snippets.

@ruxi
Last active April 6, 2016 22:02
Show Gist options
  • Save ruxi/f8af093450e9a57853baae7de2528faa to your computer and use it in GitHub Desktop.
Save ruxi/f8af093450e9a57853baae7de2528faa to your computer and use it in GitHub Desktop.
guide to D3 in jupyter notebooks
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "!date #last updated",
"execution_count": 67,
"outputs": [
{
"output_type": "stream",
"text": "Wed Apr 6 00:46:35 CDT 2016\r\n",
"name": "stdout"
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "**notes:** \n- in notebook d3 wrapper done\n- TODO: templating to serve in an index.html page\n- ISSUE: pulling in external data from data.json doesn't work due to security permission not disallowing requests from file://. Serving or substituting data seems to be the best way"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "# What is D3?\n\nD3 is a javascript framework for ['data-driven documents'](https://d3js.org/)"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "**author**: github.com/ruxi\n\n**created**: 2016-03-06 00:46:13 "
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## /lib - D3 library for offline use\n```\nd3.js d3.min.js LICENSE\n```\n\n- d3.min.js for production (compressed)\n- d3.js for development (human-readable)"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "**by bash**\n\n```bash\nwget https://github.com/mbostock/d3/releases/download/v3.5.16/d3.zip\nunzip d3.zip -d lib\n```\n\n** by python**\n```python\nimport requests, zipfile, io\ndef wget(url, unzip_path):\n response = requests.get(url)\n print(response)\n os.makedirs(unzip_path, exist_ok=True)\n z = zipfile.ZipFile(io.BytesIO(response.content))\n return z.extractall(unzip_path)\n\n# download d3 and put in /lib folder\nurl = 'https://github.com/mbostock/d3/releases/download/v3.5.16/d3.zip'\nwget(url, 'lib')\n```\n"
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "\"\"\"http://stackoverflow.com/questions/9419162/\"\"\"\nimport requests, zipfile, io, os.path\ndef wget(url, unzip_path):\n response = requests.get(url)\n print(response)\n os.makedirs(unzip_path, exist_ok=True)\n z = zipfile.ZipFile(io.BytesIO(response.content))\n print(os.listdir(unzip_path))\n return z.extractall(unzip_path)\n\n# download d3 and put in /lib folder\nurl = 'https://github.com/mbostock/d3/releases/download/v3.5.16/d3.zip'\nwget(url, os.path.join('lib','d3'))",
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": "<Response [200]>\n['LICENSE', 'd3.min.js', 'd3.js']\n",
"name": "stdout"
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## /data - external json flatfile"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "data.json as flatfiles"
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "import pandas as pd\nimport os.path\ndata = pd.DataFrame({'x':['a','b','c'], 'y':[4,3,5]})\nos.makedirs('data', exist_ok=True)\ndata.to_json(os.path.join('data','data.json'))",
"execution_count": 4,
"outputs": []
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "os.path.join(os.path.expanduser('~'),'git','d3','d3.js')",
"execution_count": 5,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": "'/home/thinkruxi/git/d3/d3.js'"
},
"metadata": {},
"execution_count": 5
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "# Via notebook-inline"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## d3wrapper.py\n\nUse to prototype d3.js charts in jupyter notebooks. \n\n```python\n%%writefile d3wrapper.py\n__author__ = 'github.com/ruxi'\n__date__ = 'Apr 6, 2016'\n__liscense__ = 'MIT'\n\"\"\"\nFor making d3.js visualizations in jupyter notebooks\n\nRecipe inspired by Brian Coffey's PyData NYC 2015 \n talk: https://youtu.be/pfP0WZmfqug\n code: https://github.com/stitchfix/d3-jupyter-tutorial\n\n\"\"\"\n\n\nimport uuid\nfrom IPython.core.display import HTML, Javascript, display_javascript, display_html\nfrom string import Template\nimport json, os.path\ndef d3wrapper(js =\"\", content=\"add description here\", verbose=False, lib='web', css=''):\n \"\"\"js: start with 'd3.select('#$hook')\n \n d3lib: use 'web' unless locally installed in any of the paths below\n \n - 'web':'https://d3js.org/d3.v3.min.js'\n - 'localrel': lib/d3/d3.js \n - 'localabs': ~/git/d3/d3.js\n \"\"\"\n d3lib_options = {'web':'https://d3js.org/d3.v3.min.js'\n , 'localrel':'lib/d3/d3.js'\n , 'localabs':os.path.join(os.path.expanduser('~'),'git','d3','d3.js')\n }\n if d3lib in list(d3lib_options.keys()):\n d3lib = [d3lib_options[d3lib]]\n print('d3lib', d3lib)\n # d3lib\n d3lib=[\"https://d3js.org/d3.v3.min.js\"]\n \n # html template\n html_template = Template(\"\"\"<style> $css </style><div id='$hook'> $content </div>\"\"\")\n \n # unique identifier for div\n # prevent conflicts when re-run in different notebook cell\n uid = \"id\"+str(uuid.uuid4())\n \n # replacement\n html_params = {'hook': uid, 'content':content, 'css':css}\n html_template = html_template.substitute(html_params)\n \n js_template = Template(js)\n js_params = {'hook': uid}\n js_template = js_template.substitute(js_params)\n # serve\n display_html(HTML(html_template))\n display_javascript(Javascript(data=js_template, lib=d3lib))\n \n if verbose==True:\n print('uid',uid)\n print('html_template',(html_template))\n print('js_template',(js_template))\n```"
},
{
"metadata": {
"run_control": {
"marked": true
},
"trusted": true,
"collapsed": true
},
"cell_type": "code",
"source": "#%%writefile d3wrapper.py\n__author__ = 'github.com/ruxi'\n__date__ = 'Apr 6, 2016'\n__liscense__ = 'MIT'\n\"\"\"\nFor making d3.js visualizations in jupyter notebooks\n\nRecipe inspired by Brian Coffey's PyData NYC 2015 \n talk: https://youtu.be/pfP0WZmfqug\n code: https://github.com/stitchfix/d3-jupyter-tutorial\n\n\"\"\"\n\n\nimport uuid\nfrom IPython.core.display import HTML, Javascript, display_javascript, display_html\nfrom string import Template\nimport json, os.path\ndef d3wrapper(js =\"\", content=\"add description here\", verbose=False, lib='web', css=''):\n \"\"\"js: start with 'd3.select('#$hook')\n \n d3lib: use 'web' unless locally installed in any of the paths below\n \n - 'web':'https://d3js.org/d3.v3.min.js'\n - 'localrel': lib/d3/d3.js \n - 'localabs': ~/git/d3/d3.js\n \"\"\"\n d3lib_options = {'web':'https://d3js.org/d3.v3.min.js'\n , 'localrel':'lib/d3/d3.js'\n , 'localabs':os.path.join(os.path.expanduser('~'),'git','d3','d3.js')\n }\n if d3lib in list(d3lib_options.keys()):\n d3lib = [d3lib_options[d3lib]]\n print('d3lib', d3lib)\n # d3lib\n d3lib=[\"https://d3js.org/d3.v3.min.js\"]\n \n # html template\n html_template = Template(\"\"\"<style> $css </style><div id='$hook'> $content </div>\"\"\")\n \n # unique identifier for div\n # prevent conflicts when re-run in different notebook cell\n uid = \"id\"+str(uuid.uuid4())\n \n # replacement\n html_params = {'hook': uid, 'content':content, 'css':css}\n html_template = html_template.substitute(html_params)\n \n js_template = Template(js)\n js_params = {'hook': uid}\n js_template = js_template.substitute(js_params)\n # serve\n display_html(HTML(html_template))\n display_javascript(Javascript(data=js_template, lib=d3lib))\n \n if verbose==True:\n print('uid',uid)\n print('html_template',(html_template))\n print('js_template',(js_template))\n",
"execution_count": null,
"outputs": []
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## draw shapes with svg"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "https://www.dashingd3js.com/basic-building-blocks"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "(Be sure to comment out after test, instance of the function is updated by the JS)"
},
{
"metadata": {
"run_control": {
"marked": true
},
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "# content = \"\"\"\n# <svg width=\"50\" height=\"50\">\n# 2 <rect x=\"0\" y=\"0\" width=\"50\" height=\"50\" fill=\"green\" />\n# 3</svg>\n# \"\"\"\n# def foo():\n# showd3(content=content)\n# return None\n# foo()\n# print('end')",
"execution_count": 7,
"outputs": []
},
{
"metadata": {},
"cell_type": "markdown",
"source": "### chrome javascript console"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "**CTRL+SHIFT+J** to pull up the console in chrome"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "### checking if d3 is working inline"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "(Be sure to comment out after test, because only the first instance of the graph is updated by the JS)"
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "# js = Template(\"\"\"\n# d3.select('#content').append('p').text('d3 lib works')\n# \"\"\")\n# js = js.substitute()\n# showd3(js = js, content='html')",
"execution_count": 8,
"outputs": []
},
{
"metadata": {},
"cell_type": "markdown",
"source": "### try to read data"
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "js = Template(\"\"\"\nd3.select('#content').append('p').text('d3 lib works')\n\nvar test_data = [1,2,3]\nd3.select('#content').append('p').text('test_data: '+test_data)\n\n// data from notebook\nvar root = $data\nd3.select('#content').append('p').text('root '+JSON.stringify(root))\n\n// plot\nvar canvas = d3.select('#content')\n .append('svg')\n .attr({\n 'width': $width,\n 'height': $height,\n })\n \nva\n\n\"\"\")\n# vars\nparams = {'data':str(data.to_json(orient='records'))\n , 'height':100\n , 'width':100\n }\njs = js.substitute(params)\nshowd3(js = js, content='html')",
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": "d3lib ['lib/d3/d3.js']\n",
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/html": "\n <!DOCTYPE html>\n <html>\n <head>\n <title> </title>\n <style> </style>\n <meta charset=\"UTF-8\">\n </head>\n \n <body>\n <div id='content'> html </div>\n \n </body>\n </html>\n "
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"application/javascript": "$.getScript(\"lib/d3/d3.js\", function () {\n\nd3.select('#content').append('p').text('d3 lib works')\n\nvar test_data = [1,2,3]\nd3.select('#content').append('p').text('test_data: '+test_data)\n\n// data from notebook\nvar root = [{\"x\":\"a\",\"y\":4},{\"x\":\"b\",\"y\":3},{\"x\":\"c\",\"y\":5}]\nd3.select('#content').append('p').text('root '+JSON.stringify(root))\n\n// plot\nvar canvas = d3.select('#content')\n .append('svg')\n .attr({\n 'width': 100,\n 'height': 100,\n })\n \nva\n\n});\n"
},
"metadata": {}
}
]
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "data.to_json(orient='records')",
"execution_count": 10,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": "'[{\"x\":\"a\",\"y\":4},{\"x\":\"b\",\"y\":3},{\"x\":\"c\",\"y\":5}]'"
},
"metadata": {},
"execution_count": 10
}
]
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "print(js)",
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"text": "\nd3.select('#content').append('p').text('d3 lib works')\n\nvar test_data = [1,2,3]\nd3.select('#content').append('p').text('test_data: '+test_data)\n\n// data from notebook\nvar root = [{\"x\":\"a\",\"y\":4},{\"x\":\"b\",\"y\":3},{\"x\":\"c\",\"y\":5}]\nd3.select('#content').append('p').text('root '+JSON.stringify(root))\n\n// plot\nvar canvas = d3.select('#content')\n .append('svg')\n .attr({\n 'width': 100,\n 'height': 100,\n })\n \nva\n\n\n",
"name": "stdout"
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "### quick graph"
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "js = Template(\"\"\"\nd3.select('#content').append('p').text('d3 lib works')\n\n// add graph\nvar width = $width\nvar height = $height\nd3.select('#content').append('svg')\n .attr({\n 'width': width,\n 'height': height,\n })\n \nalert(height)\n\"\"\")\n# vars\njs = js.substitute({'height':100, 'width':100})\nshowd3(js = js)",
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"text": "d3lib ['lib/d3/d3.js']\n",
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/html": "\n <!DOCTYPE html>\n <html>\n <head>\n <title> </title>\n <style> </style>\n <meta charset=\"UTF-8\">\n </head>\n \n <body>\n <div id='content'> body content </div>\n \n </body>\n </html>\n "
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"application/javascript": "$.getScript(\"lib/d3/d3.js\", function () {\n\nd3.select('#content').append('p').text('d3 lib works')\n\n// add graph\nvar width = 100\nvar height = 100\nd3.select('#content').append('svg')\n .attr({\n 'width': width,\n 'height': height,\n })\n \nalert(height)\n});\n"
},
"metadata": {}
}
]
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "showd3(js = js)",
"execution_count": null,
"outputs": []
},
{
"metadata": {},
"cell_type": "markdown",
"source": "# Via index.html"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "### jinja2 render script\n\n```python\n\"\"\"\n__author__ = 'lewis.r.liu@gmail.com'\nuses jinja2 templating engine to generate index.html\n\"\"\"\nimport jinja2\ndef render_indexhtml(render_template, template_dir = 'html_templates'):\n loader = jinja2.FileSystemLoader(template_dir)\n env = jinja2.Environment(loader=loader)\n \n if render_template not in env.list_templates():\n print('render_template selection not in list: {}'.format(env.list_templates()))\n else:\n return env.get_template(render_template).render()\n \n```\n\n### skeleton.html\n\n```bash\n%%writefile html_templates/skeleton.html\n<!DOCTYPE html>\n<meta charset=\"utf-8\">\n<html>\n <head>\n {% block libs %}{% endblock %}\n <style> {% block css %}{% endblock %} </style>\n \n <title> {% block title %}{% endblock %} </title>\n {% block header %}{% endblock %}\n\n </head>\n\n <body>\n {% block body %}{% endblock %}\n <div id='content'> {% block content %}{% endblock %} </div>\n <script> {% block javascript %}{% endblock %} </script>\n\n </body>\n</html>\n```\n\n### base.html\n\n```bash\n%%writefile html_templates/base.html\n{% extends \"skeleton.html\" %}\n <!-- head -->\n {% block libs %}\n <meta charset=\"UTF-8\"> \n <script type=\"text/javascript\" src=\"https://d3js.org/d3.v3.min.js\"></script>\n {% endblock %}\n\n {% block css %}\n <link rel=\"stylesheet\" href=\"style.css\" />\n {% endblock %}\n \n \n {% block header %}{% endblock %}\n {% block title %}{% endblock %}\n \n <!-- body -->\n \n {% block content %}{% endblock %}\n {% block body %}{% endblock %}\n {% block javascript %}{% endblock %}\n```"
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "%%writefile html_templates/base.html\n{% extends \"skeleton.html\" %}\n <!-- head -->\n {% block libs %}\n <meta charset=\"UTF-8\"> \n <script type=\"text/javascript\" src=\"lib/d3/d3.js\"></script>\n {% endblock %}\n\n {% block css %}\n <link rel=\"stylesheet\" href=\"style.css\" />\n {% endblock %}\n\n\n {% block header %}{% endblock %}\n {% block title %}{% endblock %}\n\n <!-- body -->\n\n {% block content %}{% endblock %}\n {% block body %}{% endblock %}\n {% block javascript %}{% endblock %}",
"execution_count": null,
"outputs": []
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "!ls html_templates",
"execution_count": null,
"outputs": []
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## Usage"
},
{
"metadata": {
"trusted": true,
"collapsed": true
},
"cell_type": "code",
"source": "import jinja2\ndef render_indexhtml(render_template, template_dir = 'html_templates', write_indexhtml = False):\n loader = jinja2.FileSystemLoader(template_dir)\n env = jinja2.Environment(loader=loader)\n \n if render_template not in env.list_templates():\n return 'render_template selection not in list: {}'.format(env.list_templates())\n else:\n rendered_html = env.get_template(render_template).render()\n if write_indexhtml == True:\n with open('index.html', 'w') as fp:\n fp.writelines(rendered_html)\n \n \n return rendered_html",
"execution_count": null,
"outputs": []
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "%%writefile html_templates/worker.html\n{% extends \"base.html\" %}\n\n{% block content %}\n <p> the defaultd3end is nign! check </p>\n{% endblock %}\n\n\n{% block javascript %}\n\n// test d3\nd3.select('body').append('p').text('d3 lib is working')\n// load external data\nd3.json(\"data.json\", function (error, data) {\n \n console.log(data)\n \n});\n\n{% endblock %}",
"execution_count": null,
"outputs": []
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "data = pd.DataFrame({'x':['a','b','c'], 'y':[6,4,5]})\ndata.to_json('data/data.json')\nwith open('data/data.json', 'r') as fp:\n print(fp.readlines())",
"execution_count": null,
"outputs": []
},
{
"metadata": {
"scrolled": true,
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "out = render_indexhtml('worker.html', write_indexhtml=True)\nfrom IPython.display import IFrame\nIFrame('index.html', 1000, 500)",
"execution_count": null,
"outputs": []
},
{
"metadata": {
"collapsed": true
},
"cell_type": "markdown",
"source": "http://blog.thedataincubator.com/2015/08/embedding-d3-in-an-ipython-notebook/"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "# Via notebook-javascript"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "http://blog.thedataincubator.com/2015/08/embedding-d3-in-an-ipython-notebook/"
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "# recipe from http://www.xavierdupre.fr/blog/2013-11-30_nojs.html\n\nhtml_src =\"\"\"\n<h2>Simple D3 plot</h2>\n<div id=\"chart\"></div>\n\"\"\"\n\ntest_d3_js = \"\"\"\nvar width = 600;\nvar height = 100;\nvar root = d3.select('#chart').append('svg')\n.attr({\n'width': width,\n'height': height,\n})\n.style('border', '1px solid black');\n\nvar evenNumbers = [0, 2, 4, 6, 8, 10];\nvar maxDataValue = d3.max(evenNumbers);\nvar barHeight = height / evenNumbers.length;\nvar barWidth = function(datum) {\nreturn datum * (width / maxDataValue);\n};\nvar barX = 0;\nvar barY = function(datum, index) {\nreturn index * barHeight;\n};\nroot.selectAll('rect.number')\n.data(evenNumbers).enter()\n.append('rect')\n.attr({\n'class': 'number',\n'x': barX,\n'y': barY,\n'width': barWidth,\n'height': barHeight,\n'fill': '#A6D854',\n'stroke': '#444',\n});    \n\"\"\"\n\njs_libs = ['lib/d3/d3.min.js']\n\nimport IPython\nfrom IPython.core.display import display_html, display_javascript, Javascript\ndisplay_html(IPython.core.display.HTML(data=html_src))\ndisplay_javascript( Javascript(data=test_d3_js, lib= js_libs))",
"execution_count": null,
"outputs": []
},
{
"metadata": {
"trusted": true,
"collapsed": false
},
"cell_type": "code",
"source": "# recipe from http://www.xavierdupre.fr/blog/2013-11-30_nojs.html\n\nhtml_src =\"\"\"\n<div id='desc'>desc</div>\n<div id=\"chart\"></div>\n\"\"\"\n\ntest_d3_js = \"\"\"\nd3.select('#chart').append('p').text('D3 is working')\nd3.select('#chart).append('svg')\n\"\"\"\n\njs_libs = ['lib/d3/d3.min.js']\n\nimport IPython\nfrom IPython.core.display import display_html, display_javascript, Javascript\ndisplay_html(IPython.core.display.HTML(data=html_src))\ndisplay_javascript( Javascript(data=test_d3_js, lib= js_libs))",
"execution_count": null,
"outputs": []
},
{
"metadata": {
"trusted": true,
"collapsed": true
},
"cell_type": "markdown",
"source": "### Testing"
},
{
"metadata": {
"trusted": true,
"collapsed": true
},
"cell_type": "code",
"source": "",
"execution_count": null,
"outputs": []
}
],
"metadata": {
"toc": {
"toc_cell": false,
"toc_section_display": "block",
"toc_number_sections": true,
"toc_threshold": 6,
"toc_window_display": true
},
"toc_position": {
"left": "1409.93px",
"width": "320px",
"height": "305px",
"right": "20px",
"top": "120px"
},
"language_info": {
"mimetype": "text/x-python",
"pygments_lexer": "ipython3",
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"version": "3.5.1",
"nbconvert_exporter": "python",
"name": "python"
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3",
"language": "python"
},
"gist": {
"id": "",
"data": {
"description": "guide to D3 in jupyter notebooks",
"public": false
}
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment