ajschumacher

import csv

r = csv.reader(open('train.csv','r'))
r.next()

edges = set()
#commutative_graph = dict()
for edge in r:
    edges.add((edge[0], edge[1]))
#    commutative_graph.setdefault(edge[0], set()).add(edge[1])

ajschumacher

#!/usr/bin/env python

import sys
import csv

def MeanAveragePrecision(valid_filename, attempt_filename, at=10):
    at = int(at)
    valid = dict()
    for line in csv.DictReader(open(valid_filename,'r')):
        valid.setdefault(line['source_node'],set()).update(line['destination_nodes'].split(" "))

ajschumacher

This shows New York City subway usage based on turnstile data. Hovering over a day shows the number of entries through subway turnstiles in that day.

Notice the clear effects of hurricanes Irene and Sandy.

More details and source code available. Visualization based (heavily) on Mike Bostock's excellent example.

ajschumacher

This d3.js visualization uses public data from NYC public schools. Original data available largely at http://schools.nyc.gov/Accountability/tools/report/default.htm This is a hacked-together prototype; it does not represent best practices.

ajschumacher

oh nothing, nothing...

ajschumacher

movie_reviews = dict()
for line in open('movies.txt.small'):
  pieces = line.split(':')
  if len(pieces) > 1:
    key = pieces[0]
    value = ':'.join(pieces[1:]).strip()
    if key == 'product/productId':
      id = value
    if key == 'review/text':
      movie_reviews.setdefault(id, []).append(value)

ajschumacher

set.seed(54)
n <- rnorm(3000,0.5,0.1)
par(mar=c(1,1,3,1))
hist(c(runif(10000),runif(1900,0.35,0.65),n[abs(n-0.5)<0.15]),
     xlim=c(0,1),ylim=c(0,600),breaks=51,ylab="",#main="",
     main="Statistical Programming DC",col="#C9242D",
     axes=F, cex.main=2)

ajschumacher

##DC Hack and Tell

#Round 3: Hack... to the Future!

And now, a wrap-up... in random order!

• Mike showed the excellent audioverb for all your language in situ needs - and it even has a youtube explanation too!
• Aaron talked about rjstat, his R package for reading and writing the JSON-stat data format.

ajschumacher

##DC Hack and Tell

#Round 4: The Christmas Invasion

Time to wrap up the Christmas Invasion and put a bow on it... Here are all the good things we saw, in non-random order!

• Aaron talked about lots of graphs made from NYC test scores.

ajschumacher

test <- data.frame(state=c("Florida", "Virginia", "Texas", "California",
                           "Georgia", "North Carolina", "New York",
                           "Missouri", "Illinois", "Maryland", "Pennsylvania",
                           "Tennessee", "Colorado", "Washington", "Arizona",
                           "Ohio", "Wisconsin", "District of Columbia",
                           "Michigan", "New Jersey", "Utah", "Louisiana",
                           "Minnesota", "Alabama", "Kansas", "Massachusetts",
                           "Indiana", "South Carolina", "Arkansas", "Oregon",
                           "Nevada", "Kentucky", "Nebraska", "(not set)",
                           "Connecticut", "New Mexico", "Oklahoma",