lifewinning

OK. So I wrote some article about predictive policing. And it had lots of links, and it turns out sometimes editors see your links and are like ¯_(ツ)_/¯ to keeping those links when they publish the article. This is probably in part my own fault because I should have included those links as footnotes, and because working in print vs working online is Different, etc etc--the point is I feel really rotten that lots of really good reporting wasn't given due credit. Putting this list together does not totally fix it but will hopefully help people interested in the topic.

Also: there is a part at the end of this gist that explains my mixed feelings about the piece in general. The citations are probably more important, but if you want to watch me tableflip a little scroll down.

---

• Matt Stroud's The minority report: Chicago's new police computer predicts crimes, but is it racist? This is pretty much the s

kylemcdonald

dannguyen

The NYPD just released their 2014 Stop and Frisk data, and in CSV format too! Here's some Bash commands to get the data and scope it out (see the previous guide here on how to graph it with R)

Gettin the data

Using bash:

Download the files

baseurl="http://www.nyc.gov/html/nypd/downloads/zip/analysis_and_planning/YYYY_sqf_csv.zip"

dannguyen

# some convenience functions here, nothing new
'''
# usage:
from easypipe import easy_pipeline
from easypipe import print_metrics
data_folder = "data-hold/20news"
p = easy_pipeline()
print_metrics(p, data_folder)
'''

nderkach

#!/usr/bin/env python
#
# Simple script showing how to read a mitmproxy dump file
#

### UPD: this feature is now avaiable in mitmproxy: https://github.com/mitmproxy/mitmproxy/pull/619

from libmproxy import flow
import json, sys

briantjacobs

Storytelling from Space: Tools/Resources

This list of resources is all about acquring and processing aerial imagery. It's generally broken up in three ways: how to go about this in Photoshop/GIMP, using command-line tools, or in GIS software, depending what's most comfortable to you. Often these tools can be used in conjunction with each other.

Acquiring Landsat & MODIS

Web Interface

• USGS Earth Explorer - Browser and data access (create a login)
  http://earthexplorer.usgs.gov/

• Landsat archive

patriciogonzalezvivo

This tutorial is partialy based on Yuriy’s Czoli article ‘Processing LiDAR to extract building heights’.

Install PostGIS

On Mac OSX

• Install Postgres, PostGIS and OSM-PostGIS tools

brew install postgis 

YKCzoli

Processing LiDAR to extract building heights

Walk through

Detailed walk through of building extraction using postgis

First lets pull a data layer from of openstreetmap. You can do this any which way you’d like, as there are a variety of methods for pulling openstreetmap data from their database. Check the [wiki] (http://wiki.openstreetmap.org/wiki/Downloading_data) for a comprehensive list. My favourite method thus far is pulling the data straight into QGIS using the open layers plugin. For those who may want to explore this method, check [this tutorial] (http://www.qgistutorials.com/en/docs/downloading_osm_data.html). For building extraction you only need building footprints, and include the building tags. Not all polygons are of type building in OSM, so we can download all the polygons, and then filter the layer for only polygons tagged as buildings.

LiDAR data was pulled from USGS via the Earth Explorer site. [Here] (http://earthobservatory.nasa.gov/blogs/ele

mojodna

"use strict";

/**
 * Note:
 *
 * setImmediate(callback, null, {});
 *
 * is shorthand for:
 * 
 * setImmediate(function() {

bsweger

Useful Pandas Snippets

A personal diary of DataFrame munging over the years.

Data Types and Conversion

Convert Series datatype to numeric (will error if column has non-numeric values)
(h/t @makmanalp)