Strata Interactive Data Visualization: Exercise 3

README.md

These are the materials for my workshop at Strata San Jose 2015 as well as resources and next steps. Videos of the workshop can be found here.

We will be using the following two tools to works through these exercises:

• bl.ocks.org: D3.js Gist renderer and online community
• Building Blocks: In-browser interactive D3.js bl.ock editor

I would love your feedback on the materials either on the Q&A forum (Google Group) or in the Github issues.

And please do not hesitate to reach out to me directly via email at jondinu@gmail.com or over twitter @clearspandex

Throughout this workshop, you will learn how to make this animated and interactive line plot of temperature over time in Noe Valley.

Exercises

• Part 1: Multiple line chart of Shanghai vs. Bangalore pollution levels created with dimple
• Part 2: Line chart of Shanghai pollution levels over time created with D3.js
• Part 3 (you are here 👇): Interactive animated line chart with color gradient of Noe Valley (SF neighborhood) daily temperature series over time.

The Data

The data is from the Data Canvas project, which is sponsored by Gray Area, swissnex San Francisco, and Lift. It contains data from 14 sensors in 7 cities which collect and stream information about their environment (temperature, dust, pollution, humidity, light, etc.).

You an access a bulk download of all the data (100+ MB) here. You can also download samples or access the stream through the API (details on the data page).

There are 4 different granularities of measurement. Files ending in:

• *-5md.csv: measurements every 5 minutes for a day
• *-1hd.csv: measurements every 1 hour for a day
• *-6hw.csv: measurements every 6 hours for a week
• noevalley.csv (or grapealope.csv): entire history of the sensor near Noe Valley at 10 second resolution

The files are comma separated with headers and 8 fields:

timestamp	city	temperature	light	airquality_raw	sound	humidity	dust
2015-02-16T17:00:00.000Z	San Francisco	20.8856185354523	2231.45801048026	28.8458008730416	1674.71050009727	48.4880466992298	882.367404134883
2015-02-16T18:00:00.000Z	San Francisco	21.8623045793052	2542.46720508251	26.5113633058142	1652.25960948903	43.9341875396295	912.0280753969
2015-02-16T19:00:00.000Z	San Francisco	23.5113166041101	3215.03460441893	24.8987852323788	1690.5842506536	40.5058249680354	939.447105875158
2015-02-16T20:00:00.000Z	San Francisco	25.6472096479114	4558.69142401972	26.0867059045864	1704.29832357106	38.4312464272035	999.743066983922

---

Archival event link: Strata + Hadoop World San Jose 2015

<style> @import url(http://fonts.googleapis.com/css?family=Crimson+Text:700italic,400,700,400italic); div.gist-readme { font: "Helvetica Neue",Helvetica,Arial,sans-serif; color: #666; -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; } .gist-readme pre { width: 200%; margin-left: -50%; overflow: hidden; } .gist-readme p.small { color: #bbb; font-size: 14px; line-height: 1.5; display: block; } .gist-readme blockquote { padding-left: 15px; border-left: 4px solid #5694f1; color: #aaa; } .gist-readme span.code { font-family: Menlo, Monaco, Courier; background-color: #EEE; font-size: 14px; } .gist-readme pre { font-family: Menlo, Monaco, Courier; white-space: pre-wrap; padding: 20px; font-size: 14px; } .gist-readme code { padding: .25em .5em; font-size: 85%; color: #bf616a; background-color: #f9f9f9; border-radius: 3px; border: 1px solid #eee; } .gist-readme table { width: 100%; margin: 40px 0; border-collapse: collapse; font-size: 13px; line-height: 1.5em; } .gist-readme th, .gist-readme td { text-align: left; padding-right: 20px; vertical-align: top; } .gist-readme table td, .gist-readme td { border-spacing: none; border-style: solid; padding: 10px 15px; border-width: 1px 0 0 0; } .gist-readme tr > td { border-top: 1px solid #eaeaea; } .gist-readme tr:nth-child(odd) > td { background: #fcfcfc; } .gist-readme thead th, .gist-readme th { text-align: left; padding: 10px 15px; height: 20px; font-size: 13px; font-weight: bold; color: #444; border-bottom: 1px solid #dadadc; cursor: default; white-space: nowrap; } </style>

index.html

<!DOCTYPE html>
<html>
<head>
 <meta charset="utf-8">
  <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.12/d3.min.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.10.6/moment.min.js"></script>
      <style>
        body {
          font-family: futura;
          width: 960px;
        }

        h2.title {
          color: black;
          text-align: center;
        }

        .axis {
          font-family: arial;
          font-size: 0.8em;
        }

        .axis text {
          fill: black;
          stroke: none;
        }

        path {
          fill: none;
          stroke: black;
          stroke-width: 2px;
        }

        .tick {
          fill: none;
          stroke: black;
        }

        .line {
          fill: none;
          stroke: url(#temperature-gradient);
          stroke-width: 1px;
        }

        div.day_buttons {
          text-align: center;
        }

        div.day_buttons div {
          text-align: center;
          background-color: #438CCA;
          color: white;
          padding: 6px;
          margin: 5px;
          font-size: 0.8em;
          cursor: pointer;
          display: inline-block;
        }
    </style>

    <script type="text/javascript">
      // D3 utility to convert from timestamp string to Date object
      var format = d3.time.format("%Y-%m-%dT%H:%M:%S.%LZ");

      function draw(data) {
        "use strict";

        // set margins according to Mike Bostock's margin conventions
        // http://bl.ocks.org/mbostock/3019563
        var margin = {top: 25, right: 40, bottom: 25, left: 75};

        // set height and width of chart
        var width = 960 - margin.left - margin.right,
            height = 350 - margin.top - margin.bottom;

        var field = 'temperature';
        
        // Append the title for the graph
        d3.select("body")
            .append("h2")
            .text("Noe Valley Temperature")
            .attr('class', 'title');

        // append the SVG tag with height and width to accommodate for margins
        var svg = d3.select("body")
                    .append("svg")
                      .attr("width", width + margin.left + margin.right)
                      .attr("height", height + margin.top + margin.bottom)
                    .append('g')
                      .attr('class','chart')
                      .attr("transform", "translate(" + margin.left + "," + margin.top + ")");

        // object to keep track on animation state
        var animation = {
          "timeout": false,
          "animating": false
        };

        // group data by day
        var nested = d3.nest()
                       .key(function(d) {
                          var ts = d['timestamp'];
                          return (ts.getMonth() + 1).toString() + '/' + ts.getDate();
                        })
                       .entries(data);

        // remove the first 10 days of data (month of January)
        var day_data = nested.slice(10);

        // extract the days we are visualizing
        // (to be used as labels in buttons)
        var days = day_data.map(function(d, i) {
          return {
            key: d.key,
            selected: false,
            idx: i
          };
        });

        // dynamically create buttons to toggle days
        var buttons = d3.select("body")
          .append("div")
          .attr("class", "day_buttons")
          .selectAll("div")
          .data(days)
          .enter()
          .append("div")
          .text(function(d) {
             return d.key;
          })
          .attr('id', function(d) {
             // convert mm/dd to mm_dd for more readable #id
             return 'd' + d.key.split('/').join('_');
          });

        // toggle button of first day
        d3.select('.day_buttons')
              .select('div:first-child')
                .transition()
                .duration(500)
                .style("background", "#F7977A")
                .style('color', 'black');

        // Find min/max of temperature column
        var temp_extent = d3.extent(data, function(d) {
            return d[field];
        });

        console.log(nested);
        // get min/max times of first day of data
        // (we will need a new extent for each day)
        var time_extent = d3.extent(day_data[0].values, function(d){
            return d['timestamp'];
        });

        // Create x-axis scale mapping dates -> pixels
        var time_scale = d3.time.scale()
            .range([0, width])
            .domain(time_extent);

        // Create y-axis scale mapping temperature -> pixels
        var temp_scale = d3.scale.linear()
            .range([height, 0])
            .domain(temp_extent);

        // Create D3 axis object from time_scale for the x-axis
        var time_axis = d3.svg.axis()
            .scale(time_scale)
            .ticks(d3.time.hours, 2)
            .tickFormat(d3.time.format("%I:00 %p"));

        // Create D3 axis object from temp_scale for the y-axis
        var temp_axis = d3.svg.axis()
            .scale(temp_scale)
            .orient("left");

        // determine thresholds for gradient stops
        var range = temp_extent[1] - temp_extent[0];
        var interval = range / 3;
        var low_threshold = temp_extent[0] + interval;
        var high_threshold = temp_extent[0] + 2 * interval;

        // Append a Gradient to double encode temperature
        svg.append("linearGradient")
          .attr("id", "temperature-gradient")
          .attr("gradientUnits", "userSpaceOnUse")
          .attr("x1", 0).attr("y1", temp_scale(low_threshold))
          .attr("x2", 0).attr("y2", temp_scale(high_threshold))
        .selectAll("stop")
          .data([
            {offset: "0%", color: "steelblue"},
            {offset: "50%", color: "gray"},
            {offset: "100%", color: "red"}
          ])
        .enter().append("stop")
          .attr("offset", function(d) { return d.offset; })
          .attr("stop-color", function(d) { return d.color; });

        // Append SVG to page corresponding to the x-axis
        svg.append('g')
            .attr('class', 'x axis')
            .attr('transform', "translate(0," + height + ")")
            .call(time_axis);

        // Append SVG to page corresponding to the y-axis
        svg.append('g')
            .attr('class', 'y axis')
            .call(temp_axis);

        // add label to y-axis
        d3.select(".y.axis")
            .append("text")
            .text("Degrees (Celsius)")
            .attr("transform", "rotate(-90, -43, 0) translate(-260)")
            .style('font-size', '1.2em');

        // define the values to map for x and y position of the line
        var line = d3.svg.line()
                       .x(function(d) { return time_scale(d['timestamp']); })
                       .y(function(d) { return temp_scale(d[field]); });

        // append a SVG path that corresponds to the line chart
        var path = svg.append("path")
                       .datum(day_data[0].values)
                       .attr("class", "line")
                       .attr("d", line);

        // create a timeout interval to animate over the days
        var interval_timeout = function(day_idx) {

          // set timeout to run callback function every 2000 milliseconds
          var day_interval = setInterval(function(){
            // run the update function to update the bound data appropriately
            update(day_data[day_idx].key);

            // increment the day visualized by one
            day_idx++;

            if(day_idx >= day_data.length) {
               // clear callback if we have reached the end of days
               clearInterval(day_interval);
            }
          }, 2000);

          // change animation state on the global object to true
          animation.animating = true;

          // return a function we can use to stop the animation
          return function() {
            clearInterval(day_interval);
          };
        };

        // define a function to unselect all the buttons
        var button_unhighlight = function() {
          d3.select(".day_buttons")
              .selectAll("div")
              .datum(function(d) {
                // change the internal state of each button to unselected
                d.selected = false;
                return d;
              })
              .transition()
              .duration(500)
              .style("color", "white")
              .style("background", '#438CCA');
        };

        // highlight button with a specified id
        var button_highlight = function(id) {
          d3.select('#' + 'd' + id.split('/').join('_'))
          .datum(function(d) {
                // change internal state to selected
                d.selected = true;
                return d;
              })
              .transition()
              .duration(500)
              .style("background", "#F7977A")
              .style('color', 'black');
        };

        // add hover to buttons
        buttons.on('mouseover', function(d) {
          if (!d.selected) {
            d3.select(this)
              .transition()
              .duration(500)
              .style("background", "#F7977A")
              .style('color', 'black');
          }
        });

        // add hover to buttons
        buttons.on('mouseout', function(d) {
          if (!d.selected) {
           d3.select(this)
              .transition()
              .duration(500)
              .style("color", "white")
              .style("background", '#438CCA');
          } else {
            d3.select(this)
              .transition()
              .duration(500)
              .style("background", "#F7977A")
              .style('color', 'black');
          }
        })


        // function to be run when on a clicked button
        buttons.on("click", function(d) {
          if(d.selected === true) {
              // if button is selected an the animation is stopped
              if (animation.animating === false) {
                // change internal state of this button
                d.selected = false;

                button_unhighlight();

                // start the animation on the next button
                animation.timeout = interval_timeout(d.idx + 1);
              } else {
                // if the button is selected but animation
                // is running, stop the animation
                animation.timeout();
                animation.animating = false;
              }
          } else {
              // if button is not selected, selected it
              d.selected = true;

              // stop the animation
              animation.animating = false;
              animation.timeout();
          }

          // update the bound data for this button
          d3.select(this).datum(d);

          // run our update function for the line chart
          update(d.key);
        });

        // function which updates the data bound to our chart
        function update(key) {
            // filter our data for the specified day in 'key'
            var data_subset = day_data.filter(function(d) {
              return d.key === key;
            })[0].values;

            // highlight buttons in sync with animation
            button_unhighlight();
            button_highlight(key);

            var time_extent = d3.extent(data_subset, function(d) {
              return d['timestamp'];
            });

            // Create x-axis scale mapping dates -> pixels
            var time_scale = d3.time.scale()
                .range([0, width])
                .domain(time_extent);

            // define the values to map for x and y position of the line
            var line = d3.svg.line()
                           .x(function(d) { return time_scale(d['timestamp']); })
                           .y(function(d) { return temp_scale(d[field]); });

            // update data bound to path
            path.datum(data_subset);

            // transition the line to animate with the changed data
            path.transition().duration(1000).attr("d", line);
        };

        // start initial animation for first day
        animation.timeout = interval_timeout(1);
      };
    </script>
  </head>
  <body>
    <script type="text/javascript">
      d3.csv("http://jay-oh-en.github.io/interactive-data-viz/data/datacanvas/noevalley.csv", function(d) {
        // convert from string to Date object
        var date = format.parse(d['timestamp']);

        // use moment.js to shift time back 8 hours (to PST)
        var mom = moment(date).subtract(8, 'hours');
        d['timestamp'] = mom.toDate();

        // coerce temperature from string to float
        d['temperature'] = +d['temperature'];
        return d;
      }, draw);
    </script>
  </body>
</html>

thumbnail.png