Skip to content

Instantly share code, notes, and snippets.

@sconnelley
sconnelley / mapreduce.py
Last active August 29, 2015 14:04
Python parser for maps.stamen.com log files
# Parses maps.stamen.com log files
# 99% of the code comes from http://pymotw.com/2/multiprocessing/mapreduce.html
# Another useful article can be found here, http://effbot.org/zone/wide-finder.htm
# Per file processing time could be improved with threads I imagine
# roughly 2k-3k files totaling ~ 150mb takes about 30min
"""
Usage:
1. Download log files from s3 into local dated directories with your weapon of choice.
@rafacv
rafacv / fakecassandra.py
Created April 23, 2010 19:17 — forked from mmalone/fakecassandra.py
In-memory Cassandra-ish thingy
# In-memory Cassandra-ish thingy... useful for unit tests. Maybe useful for other
# stuff too? No support for SuperColumns, but that should be easy enough to add.
import bisect
import copy
from cassandra.ttypes import NotFoundException, Column, ColumnPath, ColumnOrSuperColumn
class SSTable(object):
@andrewxhill
andrewxhill / density_grid.sql
Created March 19, 2012 14:50
Create a density grid from geometries. For use in CartoDB
CREATE TYPE CDB_DensityGridPair AS (
geom GEOMETRY,
val FLOAT
);
CREATE OR REPLACE FUNCTION CDB_DensityGrid(table_name TEXT, geom_name TEXT, dimension FLOAT)
RETURNS SETOF CDB_DensityGridPair AS $$
DECLARE
BEGIN
RETURN QUERY
EXECUTE 'SELECT ST_Envelope(GEOMETRYFROMTEXT(''LINESTRING(''||(st_xmax(geom)+(seed/2))||'' ''||(st_ymax(geom)+(seed/2))||'', ''||(st_xmin(geom)-(seed/2))||'' ''||(st_ymin(geom)-(seed/2))||'')'',3857)) as geom, val FROM (SELECT '||dimension||' as seed, count(*)::float as val,
@springmeyer
springmeyer / dot-density-points.diff
Created December 6, 2012 04:42
Messing around with using halton sequences for quasi-random point filling of polygons (applied to mapnik @ ff8f3f1d4597eed89e04aa1dc7cb4755f6d4a744)
diff --git a/src/agg/process_point_symbolizer.cpp b/src/agg/process_point_symbolizer.cpp
index 6854903..abf1972 100644
--- a/src/agg/process_point_symbolizer.cpp
+++ b/src/agg/process_point_symbolizer.cpp
@@ -40,8 +40,58 @@
// boost
#include <boost/make_shared.hpp>
+#include <boost/utility.hpp>
+
@bmander
bmander / dotmap.pde
Created December 27, 2012 04:26
Generate dotmap tiles
/* I, Brandon Martin-Anderson, release this into the public domain or whatever. */
BufferedReader reader;
double ll, bb, rr, tt;
float A = 1000.0;
GlobalMercator proj = new GlobalMercator();
@OllyHodgson
OllyHodgson / sparkline.svg
Created January 7, 2013 09:50
Stuart Langridge's Simple SVG sparklines — http://kryogenix.org/days/2012/12/30/simple-svg-sparklines — but using an object element instead of an embed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@shawnbot
shawnbot / leaflet-compact.js
Created March 13, 2013 18:04
A Leaflet TileLayer class for loading "compact" URL schemes.
L.CompactTileLayer = L.TileLayer.extend({
getTileUrl: function(tilePoint) {
this._adjustTilePoint(tilePoint);
var col = String("000000" + tilePoint.x).slice(String(tilePoint.x).length),
row = String("000000" + tilePoint.y).slice(String(tilePoint.y).length),
zoom = this._getZoomForUrl();
return this._url
.replace("{z}", zoom)
.replace("{x}", col.slice(0, 3) + "/" + col.slice(3, 6))
.replace("{y}", row.slice(0, 3) + "/" + row.slice(3, 6));
al@zorak.tobert.org ~ $ sudo virsh dumpxml newsblur
<domain type='lxc'>
<name>newsblur</name>
<uuid>cf64daa0-9fd0-088f-2bf2-436f23c43e88</uuid>
<memory unit='KiB'>1048576</memory>
<currentMemory unit='KiB'>1048576</currentMemory>
<vcpu placement='static'>2</vcpu>
<os>
<type arch='x86_64'>exe</type>
<init>/sbin/init</init>
@kehh
kehh / st_split.sql
Last active December 17, 2015 10:39
Wrapping polygons so they don't sit on the date line Requires st_split http://postgis.17.x6.nabble.com/ST-Split-in-1-5-tp4999369p4999388.html st_split for postgis 1.5 st_wrapx from: https://github.com/mci/mpatlas/blob/master/mpatlas/ST_WrapX.sql
CREATE OR REPLACE FUNCTION st_split(polygon geometry,split_line geometry) RETURNS SETOF geometry AS
$$
DECLARE
v_geometry geometry;
BEGIN
FOR v_geometry IN SELECT (ST_Dump((
ST_Polygonize(ST_Union(ST_Boundary(polygon),split_line))
@mojodna
mojodna / gist:5609010
Created May 19, 2013 21:05
Importing Natural Earth into PostGIS w/ ogr2ogr
P900913="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs"
EXTENT="-180 -85.05112878 180 85.05112878"
ogr2ogr --config PG_USE_COPY YES --config OGR_ENABLE_PARTIAL_REPROJECTION TRUE --config SHAPE_ENCODING WINDOWS-1252 -t_srs "$P900913" -lco ENCODING=UTF-8 -lco SRID=900913 -clipsrc $EXTENT -segmentize 1 -skipfailures -f PGDump /vsistdout/ ne_10m_populated_places.shp | pv | psql -d natural_earth -q