Skip to content

Instantly share code, notes, and snippets.

View evz's full-sized avatar

Eric van Zanten evz

80 followers · 0 following
View GitHub Profile
@evz
evz / config.json
Last active August 29, 2015 14:09
NullPointerException
{
"traceFileName":"S11900.gpx.csv",
"outputFileName":"tracking-result.json",
"otpGraphLocation":"/tmp",
"initialParameters": {
"obsCov":[100.0,100.0],
"onRoadStateCov":[6.25E-4],
"offRoadStateCov":[6.25E-4,6.25E-4],
"offTransitionProbs":[1.0,+Infinity],
"onTransitionProbs":[+Infinity,1.0],
@evz
evz / sample_query.sql
Last active August 29, 2015 14:06
Two indexes + intersect for 3D magic
(
SELECT
posting_time,
latitude,
longitude
FROM plows
ORDER BY lat_lon <-> ST_SetSRID(ST_MakePoint(-87.637148,41.887707),4326)
) INTERSECT (
SELECT
posting_time,
@evz
evz / quickie.py
Created June 4, 2014 22:07
IL Elections Quickie Scraper
import requests
from requests.sessions import Session
from BeautifulSoup import BeautifulSoup
from urllib import urlencode
get_params = {
'AddressSearchType': 'Starts with',
'Archived': 'false',
'CitySearchType': 'Starts with',
'ContributionType': 'All Types',
@evz
evz / snap_crime_to_grid.sql
Last active August 29, 2015 14:01
Snap Crime locations to street grid
=> CREATE TABLE crime_to_streets AS
WITH ordered_nearest AS (
SELECT ST_GeometryN(streets.geom, 1) AS streets_geom,
streets.gid AS streets_gid,
crime.geom AS crime_geom,
crime.id AS crime_id,
st_distance(streets.geom, crime.geom) AS distance
FROM streets
JOIN dat_chicago_crimes_all AS crime
ON ST_DWithin(streets.geom, crime.geom, 200)
@evz
evz / crime_streets.geojson
Last active August 29, 2015 14:01
Snapped geojson stuff
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@evz
evz / csv2html.py
Last active December 20, 2015 18:48
Take a CSV and make it an HTML table
import csv
def table_it(fname):
outs = '<table id="abms" class="dataTable" aria-describedby="abms_info">'
with open(fname, 'rb') as f:
reader = csv.reader(f)
headers = reader.next()
outs += '<thead><tr>'
outs += ''.join(['<th>%s</th>' % h for h in headers])
outs += '</tr></thead><tbody>'
@evz
evz / smallbell.py
Last active December 20, 2015 17:28
Tarbell guts for dumping Google Spreadsheet out as JSON to an S3 bucket. Lovingly stolen from the NewsApps team at the Chicago Tribune
import os
from ordereddict import OrderedDict
from gdata.spreadsheet.service import SpreadsheetsService
from gdata.spreadsheet.service import CellQuery
import json
import codecs
import shutil
# Source for this is here https://github.com/newsapps/flask-tarbell/blob/master/tarbell/slughifi.py
from slughifi import slughifi
@evz
evz / README
Created August 3, 2013 20:41 — forked from sasha-id/README
MongoDB upstart scripts for Ubuntu.
Run following commands after installing upstart scripts:
ln -s /lib/init/upstart-job /etc/init.d/mongoconf
ln -s /lib/init/upstart-job /etc/init.d/mongodb
ln -s /lib/init/upstart-job /etc/init.d/mongos
To start services use:
@evz
evz / json_query.py
Last active December 17, 2015 22:59
A class based method of accessing and running basic queries against JSON files stored on a filesystem
import os
import json
from datetime import datetime
# first a super stupid exception to use
class QueryError(Exception):
def __init__(self, value):
self.value = value
def __str__(self):
return repr(self.value)
@evz
evz / 2007-10-24.json
Last active December 17, 2015 05:09
Chicago Crime vs. Weather: Under the hood.
{
"weather": {
"FAHR_MIN": 42.980000000000004,
"CELSIUS_MIN": 6.1,
"CELSIUS_MAX": 12.8,
"FAHR_MAX": 55.040000000000006
},
"meta": {
"total": {
"key": "total",