kirkhunter/README.md

## README.md

      
    Raw
  

              README.md
            
          
    These are the materials for my workshop at Strata San Jose 2015 as well as resources and next steps. Videos of the workshop can be found here.
We will be using the following two tools to works through these exercises:

bl.ocks.org: D3.js Gist renderer and online community
Building Blocks: In-browser interactive D3.js bl.ock editor

I would love your feedback on the materials either on the Q&A forum (Google Group) or in the Github issues.
And please do not hesitate to reach out to me directly via email at jondinu@gmail.com or over twitter @clearspandex

Throughout this workshop, you will learn how to make this animated and interactive line plot of temperature over time in Noe Valley.

Exercises


Part 1: Multiple line chart of Shanghai vs. Bangalore pollution levels created with dimple
Part 2: Line chart of Shanghai pollution levels over time created with D3.js
Part 3 (you are here 👇): Interactive animated line chart with color gradient of Noe Valley (SF neighborhood) daily temperature series over time.

The Data

The data is from the Data Canvas project, which is sponsored by Gray Area, swissnex San Francisco, and Lift.  It contains data from 14 sensors in 7 cities which collect and stream information about their environment (temperature, dust, pollution, humidity, light, etc.).
You an access a bulk download of all the data (100+ MB) here.  You can also download samples or access the stream through the API (details on the data page).

There are 4 different granularities of measurement. Files ending in:

*-5md.csv: measurements every 5 minutes for a day
*-1hd.csv: measurements every 1 hour for a day
*-6hw.csv: measurements every 6 hours for a week
noevalley.csv (or grapealope.csv): entire history of the sensor near Noe Valley at 10 second resolution

The files are comma separated with headers and 8 fields:


timestamp
city
temperature
light
airquality_raw
sound
humidity
dust


2015-02-16T17:00:00.000Z
San Francisco
20.8856185354523
2231.45801048026
28.8458008730416
1674.71050009727
48.4880466992298
882.367404134883


2015-02-16T18:00:00.000Z
San Francisco
21.8623045793052
2542.46720508251
26.5113633058142
1652.25960948903
43.9341875396295
912.0280753969


2015-02-16T19:00:00.000Z
San Francisco
23.5113166041101
3215.03460441893
24.8987852323788
1690.5842506536
40.5058249680354
939.447105875158


2015-02-16T20:00:00.000Z
San Francisco
25.6472096479114
4558.69142401972
26.0867059045864
1704.29832357106
38.4312464272035
999.743066983922


Archival event link: Strata + Hadoop World San Jose 2015
<style>
@import url(http://fonts.googleapis.com/css?family=Crimson+Text:700italic,400,700,400italic);

div.gist-readme {
  font: "Helvetica Neue",Helvetica,Arial,sans-serif;
  color: #666;
  -webkit-font-smoothing: antialiased;
  text-rendering: optimizeLegibility; }
 
 .gist-readme pre {
    width: 200%;
    margin-left: -50%;
    overflow: hidden; }

  .gist-readme p.small {
    color: #bbb;
    font-size: 14px;
    line-height: 1.5;
    display: block; }
 .gist-readme blockquote {
    padding-left: 15px;
    border-left: 4px solid #5694f1;
    color: #aaa; }
  .gist-readme span.code {
    font-family: Menlo, Monaco, Courier;
    background-color: #EEE;
    font-size: 14px; }
  .gist-readme pre {
    font-family: Menlo, Monaco, Courier;
    white-space: pre-wrap;
    padding: 20px;
    font-size: 14px; }
  .gist-readme code {
    padding: .25em .5em;
    font-size: 85%;
    color: #bf616a;
    background-color: #f9f9f9;
    border-radius: 3px;
    border: 1px solid #eee; }
  .gist-readme table {
    width: 100%;
    margin: 40px 0;
    border-collapse: collapse;
    font-size: 13px;
    line-height: 1.5em; }
   .gist-readme th,  .gist-readme td {
    text-align: left;
    padding-right: 20px;
    vertical-align: top; }
  .gist-readme table td, .gist-readme td {
    border-spacing: none;
    border-style: solid;
    padding: 10px 15px;
    border-width: 1px 0 0 0; }
  .gist-readme tr > td {
    border-top: 1px solid #eaeaea; }
  .gist-readme tr:nth-child(odd) > td {
    background: #fcfcfc; }
  .gist-readme thead th, .gist-readme th {
    text-align: left;
    padding: 10px 15px;
    height: 20px;
    font-size: 13px;
    font-weight: bold;
    color: #444;
    border-bottom: 1px solid #dadadc;
    cursor: default;
    white-space: nowrap; }

</style>
forked from Jay-Oh-eN's block: Strata Interactive Data Visualization: Exercise 3
forked from Jay-Oh-eN's block: Simple Line Chart with Color Gradient
forked from anonymous's block: Simple Line Chart with Color Gradient

  
## index.html
<!DOCTYPE html>
<html>
<head>
 <meta charset="utf-8">
  <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.12/d3.min.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.10.6/moment.min.js"></script>
      <style>
        body {
          font-family: futura;
          width: 960px;
        }

        h2.title {
          color: black;
          text-align: center;
        }

        .axis {
          font-family: arial;
          font-size: 0.8em;
        }

        .axis text {
          fill: black;
          stroke: none;
        }

        path {
          fill: none;
          stroke: black;
          stroke-width: 2px;
        }

        .tick {
          fill: none;
          stroke: black;
        }

        .line {
          fill: none;
          stroke: url(#temperature-gradient);
          stroke-width: 1px;
        }

        div.day_buttons {
          text-align: center;
        }

        div.day_buttons div {
          text-align: center;
          background-color: #438CCA;
          color: white;
          padding: 6px;
          margin: 5px;
          font-size: 0.8em;
          cursor: pointer;
          display: inline-block;
        }
    </style>

    <script type="text/javascript">
      // D3 utility to convert from timestamp string to Date object
      var format = d3.time.format("%Y-%m-%dT%H:%M:%S.%LZ");

      function draw(data) {
        "use strict";

        // set margins according to Mike Bostock's margin conventions
        // http://bl.ocks.org/mbostock/3019563
        var margin = {top: 25, right: 40, bottom: 25, left: 75};

        // set height and width of chart
        var width = 960 - margin.left - margin.right,
            height = 350 - margin.top - margin.bottom;

        var field = 'temperature';

        // Append the title for the graph
        d3.select("body")
            .append("h2")
            .text("Noe Valley Temperature")
            .attr('class', 'title');

        // append the SVG tag with height and width to accommodate for margins
        var svg = d3.select("body")
                    .append("svg")
                      .attr("width", width + margin.left + margin.right)
                      .attr("height", height + margin.top + margin.bottom)
                    .append('g')
                      .attr('class','chart')
                      .attr("transform", "translate(" + margin.left + "," + margin.top + ")");

        // Q1: group data by day
        var data_sub = data.slice(1000, 10000);

        // Q2: remove the first 10 days of data (month of January)


        // Q3: extract the days we are visualizing
        // (to be used as labels in buttons)

        // Q4: dynamically create buttons to toggle days


        // Q5: toggle button of first day

        // Find min/max of temperature column
        var temp_extent = d3.extent(data_sub, function(d) {
            return d[field];
        });

        // get min/max times of first day of data
        // (we will need a new extent for each day)
        var time_extent = d3.extent(data_sub, function(d){
            return d['timestamp'];
        });

        // Create x-axis scale mapping dates -> pixels
        var time_scale = d3.time.scale()
            .range([0, width])
            .domain(time_extent);

        // Create y-axis scale mapping temperature -> pixels
        var temp_scale = d3.scale.linear()
            .range([height, 0])
            .domain(temp_extent);

        // Create D3 axis object from time_scale for the x-axis
        var time_axis = d3.svg.axis()
            .scale(time_scale)
            .ticks(d3.time.hours, 2)
            .tickFormat(d3.time.format("%I:00 %p"));

        // Create D3 axis object from temp_scale for the y-axis
        var temp_axis = d3.svg.axis()
            .scale(temp_scale)
            .orient("left");

        // determine thresholds for gradient stops
        var range = temp_extent[1] - temp_extent[0];
        var interval = range / 3;
        var low_threshold = temp_extent[0] + interval;
        var high_threshold = temp_extent[0] + 2 * interval;

        // Append a Gradient to double encode temperature
        svg.append("linearGradient")
          .attr("id", "temperature-gradient")
          .attr("gradientUnits", "userSpaceOnUse")
          .attr("x1", 0).attr("y1", temp_scale(low_threshold))
          .attr("x2", 0).attr("y2", temp_scale(high_threshold))
        .selectAll("stop")
          .data([
            {offset: "0%", color: "steelblue"},
            {offset: "50%", color: "gray"},
            {offset: "100%", color: "red"}
          ])
        .enter().append("stop")
          .attr("offset", function(d) { return d.offset; })
          .attr("stop-color", function(d) { return d.color; });

        // Append SVG to page corresponding to the x-axis
        svg.append('g')
            .attr('class', 'x axis')
            .attr('transform', "translate(0," + height + ")")
            .call(time_axis);

        // Append SVG to page corresponding to the y-axis
        svg.append('g')
            .attr('class', 'y axis')
            .call(temp_axis);

        // add label to y-axis
        d3.select(".y.axis")
            .append("text")
            .text("Degrees (Celsius)")
            .attr("transform", "rotate(-90, -43, 0) translate(-260)")
            .style('font-size', '1.2em');

        // define the values to map for x and y position of the line
        var line = d3.svg.line()
                       .x(function(d) { return time_scale(d['timestamp']); })
                       .y(function(d) { return temp_scale(d[field]); });

        // append a SVG path that corresponds to the line chart
        var path = svg.append("path")
                       .datum(data_sub)
                       .attr("class", "line")
                       .attr("d", line);

        // Q6: create a timeout interval to animate over the days

        // Q7: define a function to unselect all the buttons


        // Q8: highlight button with a specified id


        // Q9: add hover to buttons


        // Q10: add hover to buttons

        // Q11: function to be run when on a clicked button

        // Q12: function which updates the data bound to our chart

      };
    </script>
  </head>
  <body>
    <script type="text/javascript">
      d3.csv("http://jay-oh-en.github.io/interactive-data-viz/data/datacanvas/noevalley.csv", function(d) {
        // convert from string to Date object
        var date = format.parse(d['timestamp']);

        // use moment.js to shift time back 8 hours (to PST)
        var mom = moment(date).subtract(8, 'hours');
        d['timestamp'] = mom.toDate();

        // coerce temperature from string to float
        d['temperature'] = +d['temperature'];
        return d;
      }, draw);
    </script>
  </body>
</html>
timestamp	city	temperature	light	airquality_raw	sound	humidity	dust
2015-02-16T17:00:00.000Z	San Francisco	20.8856185354523	2231.45801048026	28.8458008730416	1674.71050009727	48.4880466992298	882.367404134883
2015-02-16T18:00:00.000Z	San Francisco	21.8623045793052	2542.46720508251	26.5113633058142	1652.25960948903	43.9341875396295	912.0280753969
2015-02-16T19:00:00.000Z	San Francisco	23.5113166041101	3215.03460441893	24.8987852323788	1690.5842506536	40.5058249680354	939.447105875158
2015-02-16T20:00:00.000Z	San Francisco	25.6472096479114	4558.69142401972	26.0867059045864	1704.29832357106	38.4312464272035	999.743066983922
	<!DOCTYPE html>
	<html>
	<head>
	<meta charset="utf-8">
	<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.12/d3.min.js"></script>
	<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.10.6/moment.min.js"></script>
	<style>
	body {
	font-family: futura;
	width: 960px;
	}

	h2.title {
	color: black;
	text-align: center;
	}

	.axis {
	font-family: arial;
	font-size: 0.8em;
	}

	.axis text {
	fill: black;
	stroke: none;
	}

	path {
	fill: none;
	stroke: black;
	stroke-width: 2px;
	}

	.tick {
	fill: none;
	stroke: black;
	}

	.line {
	fill: none;
	stroke: url(#temperature-gradient);
	stroke-width: 1px;
	}

	div.day_buttons {
	text-align: center;
	}

	div.day_buttons div {
	text-align: center;
	background-color: #438CCA;
	color: white;
	padding: 6px;
	margin: 5px;
	font-size: 0.8em;
	cursor: pointer;
	display: inline-block;
	}
	</style>

	<script type="text/javascript">
	// D3 utility to convert from timestamp string to Date object
	var format = d3.time.format("%Y-%m-%dT%H:%M:%S.%LZ");

	function draw(data) {
	"use strict";

	// set margins according to Mike Bostock's margin conventions
	// http://bl.ocks.org/mbostock/3019563
	var margin = {top: 25, right: 40, bottom: 25, left: 75};

	// set height and width of chart
	var width = 960 - margin.left - margin.right,
	height = 350 - margin.top - margin.bottom;

	var field = 'temperature';

	// Append the title for the graph
	d3.select("body")
	.append("h2")
	.text("Noe Valley Temperature")
	.attr('class', 'title');

	// append the SVG tag with height and width to accommodate for margins
	var svg = d3.select("body")
	.append("svg")
	.attr("width", width + margin.left + margin.right)
	.attr("height", height + margin.top + margin.bottom)
	.append('g')
	.attr('class','chart')
	.attr("transform", "translate(" + margin.left + "," + margin.top + ")");

	// Q1: group data by day
	var data_sub = data.slice(1000, 10000);

	// Q2: remove the first 10 days of data (month of January)


	// Q3: extract the days we are visualizing
	// (to be used as labels in buttons)

	// Q4: dynamically create buttons to toggle days


	// Q5: toggle button of first day

	// Find min/max of temperature column
	var temp_extent = d3.extent(data_sub, function(d) {
	return d[field];
	});

	// get min/max times of first day of data
	// (we will need a new extent for each day)
	var time_extent = d3.extent(data_sub, function(d){
	return d['timestamp'];
	});

	// Create x-axis scale mapping dates -> pixels
	var time_scale = d3.time.scale()
	.range([0, width])
	.domain(time_extent);

	// Create y-axis scale mapping temperature -> pixels
	var temp_scale = d3.scale.linear()
	.range([height, 0])
	.domain(temp_extent);

	// Create D3 axis object from time_scale for the x-axis
	var time_axis = d3.svg.axis()
	.scale(time_scale)
	.ticks(d3.time.hours, 2)
	.tickFormat(d3.time.format("%I:00 %p"));

	// Create D3 axis object from temp_scale for the y-axis
	var temp_axis = d3.svg.axis()
	.scale(temp_scale)
	.orient("left");

	// determine thresholds for gradient stops
	var range = temp_extent[1] - temp_extent[0];
	var interval = range / 3;
	var low_threshold = temp_extent[0] + interval;
	var high_threshold = temp_extent[0] + 2 * interval;

	// Append a Gradient to double encode temperature
	svg.append("linearGradient")
	.attr("id", "temperature-gradient")
	.attr("gradientUnits", "userSpaceOnUse")
	.attr("x1", 0).attr("y1", temp_scale(low_threshold))
	.attr("x2", 0).attr("y2", temp_scale(high_threshold))
	.selectAll("stop")
	.data([
	{offset: "0%", color: "steelblue"},
	{offset: "50%", color: "gray"},
	{offset: "100%", color: "red"}
	])
	.enter().append("stop")
	.attr("offset", function(d) { return d.offset; })
	.attr("stop-color", function(d) { return d.color; });

	// Append SVG to page corresponding to the x-axis
	svg.append('g')
	.attr('class', 'x axis')
	.attr('transform', "translate(0," + height + ")")
	.call(time_axis);

	// Append SVG to page corresponding to the y-axis
	svg.append('g')
	.attr('class', 'y axis')
	.call(temp_axis);

	// add label to y-axis
	d3.select(".y.axis")
	.append("text")
	.text("Degrees (Celsius)")
	.attr("transform", "rotate(-90, -43, 0) translate(-260)")
	.style('font-size', '1.2em');

	// define the values to map for x and y position of the line
	var line = d3.svg.line()
	.x(function(d) { return time_scale(d['timestamp']); })
	.y(function(d) { return temp_scale(d[field]); });

	// append a SVG path that corresponds to the line chart
	var path = svg.append("path")
	.datum(data_sub)
	.attr("class", "line")
	.attr("d", line);

	// Q6: create a timeout interval to animate over the days

	// Q7: define a function to unselect all the buttons


	// Q8: highlight button with a specified id


	// Q9: add hover to buttons


	// Q10: add hover to buttons

	// Q11: function to be run when on a clicked button

	// Q12: function which updates the data bound to our chart

	};
	</script>
	</head>
	<body>
	<script type="text/javascript">
	d3.csv("http://jay-oh-en.github.io/interactive-data-viz/data/datacanvas/noevalley.csv", function(d) {
	// convert from string to Date object
	var date = format.parse(d['timestamp']);

	// use moment.js to shift time back 8 hours (to PST)
	var mom = moment(date).subtract(8, 'hours');
	d['timestamp'] = mom.toDate();

	// coerce temperature from string to float
	d['temperature'] = +d['temperature'];
	return d;
	}, draw);
	</script>
	</body>
	</html>