Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Realtimeliness: 140 Journos + CartoDB

Realtimeliness: Mapping RealTime Data

July 29, 2015

Find this document here:

Outline

  1. RealTime Data
    • Types + Topics
    • Mapping Samples
  2. Mapping in CartoDB
    • Examples
    • Tour of the interface
    • APIs / JS Libs
    • Data Import + Sync Tables
    • Customizing UI + CartoCSS
  3. Building a Map App
    • Mapping in the Editor
    • Mapping in Code
      • Python
      • Node
      • R
  4. Building a Narrative
  5. Resources

RealTime Data

Journalism Projects

Types + Topics

Election Maps

Elections

Source: WSJ 2012 Election Map.

Check out the Toronto Municpal Election Maps or the 2011 Spanish Election Results.

Lots of journalists need to make realtime election trackers, preferably with mapped admin districts. More info on how to do that with CartoDB here.

Weather Maps

Realtime Precipitation

Source: Realtime Precipitation Map

Data: NOAA Weather data

Traffic Maps

Realtime Traffic Map

Source: Traffic Change Data for SF

Demo: Municipal Traffic in SF App

Blog: CartoDB + Firebase

Climate Tracking Maps

Realtime Climate Tracking

Source: Vizonomy + NOAA collaboration on Asterra's Climate Dashboard

Crisis Monitoring Maps

Realtime Earthquake Map

Source: Andy's Earthquake Workshop

Media Watch Maps

GDELT Geographic News Search Tool

Source: GDELT Geographic News Search Tool

Journalists have used the GDELT data to track wildlife crime, and the spread of the Islamic State in the Middle East among other things.

You can fork the GDELT hourly synced data set from the CartoDB DataLibrary and add it as a layer on your map or use the Geographic Search Tool linked above to search for tags of interest.

Data Library

Read more about how to filter and author queries via the GDELT API to build dynamic media maps to mashup with your data:

Other

Mapping in CartoDB

General Examples

Time Travel Map

Source: Time Travel Between Counties, Carto.JS

County Chart

Source: Georgia County Car Crash Counts, C3.JS

RealTime Mapping Templates in CartoDB

Source: Realtime Mapping templates here

Tour of the interface

int-data int-upload int-vis int-share

APIs / JS Libs

  • CartoJS - JS library for interacting with CartoDB
  • Maps API - generate public/private maps with data hosted on your CDB account
  • SQL API - run sql in your code that dynamically filters/affects/queries your mapped data stored in CartoDB
  • Import API - CRUD files in your CartoDB Account

Data Import + Sync Tables

Sync Tables

The Editor is especially setup to process realtime data updates via sync tables.

You can import data that lives online via a URL, and set it to pull and update your map at regular intervals.

Check the file types supported in sync tables; keep in mind that it also works with dropbox + google drive.

Notes:

  • to auto-geocode a sync table, verify that it contains the following:
    • country column, a latitude column and a separate longitude column
    • a column of IP addresses
  • to edit sync tables you need to be disconnected from the datasource, so during syncing, you can use SQL to manipulate the dataset on the fly
  • you can create sync tables in both the Editor and via the Import API

Customizing UI + CartoCSS

Basemaps

You have myriad customization options in the in-browser editor:

  • sql - run sql and postgis functions across your data
  • wizard - adjust the type, colors and fills in your map
  • infowindow - create hovers, tooltips with information from your datatables
  • css - customize the css and style of your map outside the wizard
  • legends - create keys for your map
  • filters - filter the data without sql

Likewise, many types of visualizations:

  • Simple -- most basic visualization
  • Cluster -- counts number of points within a certain binned region
  • Choropleth -- makes a histogram of your data and gives bins different colors depending on the color ramp chosen
  • Category -- color data based on unique category (works best for a handful of unique types)
  • Bubble -- size markers based on column values
  • Intensity -- colors by density
  • Density -- data aggregated by number of points within a hexagon
  • Torque -- temporal visualization of data
  • Heat -- more fluid map of concentration; emphasis on far over near-view

Torque and heatmaps might be most relevant for realtime data.

Building a Map App

Mapping in the Editor

The Wizard allows you to select your visualization "type" and customize color palettes, design details, and otherwise set the tone for your map.

Styles
  • Marker Fill: change the size, color, and opacity of all markers
  • Marker Stroke: change the width, color, and opacity of every marker's border
  • Composite Operation: change the color of markers when they overlap
  • Label Text: Text appearing by a marker (can be from columns)
Infowindows/hovers

Infowindow options

  • Select which column data appear in infowindow by toggling column on
  • Customize further by selecting HTML-view
Basemaps

Basemap options

Check out the base CartoDB Basemaps from Stamen.

Mapping in Code

Python

Use the CartoDB Python module to create and update tables; check out Andy's IPythonNB demos.

Node

Use the CartoDB Uploader package available in NPM for working in Node.js

R

Use Kyle Walker's R2CartoDB project to create and update tables in R.

Building a Narrative

Case Study: Realtime Earthquake Mapping

Move over to this tutorial on how to use CartoDB's sync tables to track realtime earthquakes vis-à-vis districts in Nepal.

Or, Stuart made this rad proxy for EONET and we can play with importing it into a CartoDB table and setting up a sync.

Storytelling: Odyssey + Torque

Outside of the CartoJS library, we have others to help you build dynamic narrative with your data.

Maps that tell Time - Torque

<iframe src="https://srogers.cartodb.com/viz/337d9194-6458-11e3-85b5-e5e70547d141/embed_map" width="100%" height="500px"></iframe>
  1. Demonstrations in Brazil
  2. Animal migration patterns
  3. Diwali Celebrated
  4. Ramadan Tweets w/OdysseyJS
  5. Alcatraz Escapees
  6. Lynching and the Press

Twitter Import

Maps that tell Stories - Odyssey JS

  1. Al Jazeera: Israeli-Palestinian Conflict by Tweets
  2. The Sounds of 11M
  3. Berlin Wall Historic Tour

Graphs + Charts

You can use CartoDB's SQL API to query your data and pull it into any charting library of your choosing.

You can easily wire up a chart of pollution data for example, set it as a sync table to update continuously, check the code here.

chart

Learn more about it here!

Here are some examples:

Type Title Link/Demo BlogPost
Chart.js Bar Graph Traffic Data Aurelia's Block
Highcharts Sensor Data Github / Demo MOW Post
Highcharts Weather Data Aurelia's Block Tutorial
Chart.js Line Graph Tornado Data Andrew's Block
Plot.ly Earthquake Data Plotly Tutorial CartoDB Blog

Resources

CartoDB

  1. Map Academy
  2. CartoDB Tutorials
  3. CartoDB Editor Documentation
  4. CartoDB APIs
  5. Community help on StackExchange
  6. CartoDB Map Gallery
  7. CartoDB Bootstrap Template by Chris Wong

Visualization

  1. Charting Tools Repository
  2. Workshops @ CartoDB
  3. Recommended tools for Visualizations
  4. Perception Concerns
  5. Gestalt Theory
  6. Color Brewer or Geocolor

Please give us feedback on the workshop here! Whoot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.