Skip to content

Instantly share code, notes, and snippets.


Chris Herwig hrwgc

  • Google
  • San Francisco
View GitHub Profile
hrwgc /
Last active Aug 29, 2015
mercantile find web mercator tiles for a given zoom level based on geojson feature collection
function to_tiles(){
jq -c .features[] < "$geojson" | mercantile tiles $zoom | sort | uniq
hrwgc /
Last active Apr 22, 2020
aws-cli get total size of all objects within s3 prefix. (mimic behavior of `s3cmd du` with aws-cli)
function s3du(){
bucket=`cut -d/ -f3 <<< $1`
prefix=`awk -F/ '{for (i=4; i<NF; i++) printf $i"/"; print $NF}' <<< $1`
aws s3api list-objects --bucket $bucket --prefix=$prefix --output json --query '[sum(Contents[].Size), length(Contents[])]' | jq '. |{ size:.[0],num_objects: .[1]}'
s3du $1;
hrwgc / xml2json.js
Created Sep 17, 2014
convert xml to json nodejs command line utility
View xml2json.js
#!/usr/bin/env node
var parser = require('xml2json');
var fs = require('fs');
function onFile(err, data) {
if (err) {
console.log('Uh oh: ' + err);
var json = parser.toJson(data);

perf guide

Profiling tools are critical for trying to understand performance bottlenecks in code.

Without real data about what a program is doing while running, detecting bottlenecks is at best a process of trial and error for both users and developers. While thoughtful testing of various program configurations along with measuring time elapsed for decrete runs can often be enough - why not learn faster and funner ways to rapidly collect real data about what a program is doing?

Actual data about program execution like which functions are being called while a program is active helps point to hot parts of the code where most time may be being spent. While users of applications may not easily understand the output of profiling tools, being equipped to generate profiling output can be extremely useful for sharing with developers, since the time to set up robust test cases for developers is can be greater than the time it takes to understand and optimize slow code paths. Therefore it can be invaluable to get

hrwgc /
Last active Dec 29, 2015
postgres / postgis randomly stops working on osx mavericks - fix

Following advice of Psql: could not connect...:

Running the following command resets PostgreSQL's transaction log, which resolves the problem.

 pg_resetxlog -f /usr/local/Cellar/postgresql/9.3.1/data

you should replace /usr/local/Cellar/postgresql/9.3.1/data with your own pg_datadir (eg., pg_ctl -D $DATADIR start)

hrwgc /
Created Nov 13, 2013
bash wget - check if file exists at url before downloading
# simple function to check http response code before downloading a remote file
# example usage:
# if `validate_url $url >/dev/null`; then dosomething; else echo "does not exist"; fi
function validate_url(){
if [[ `wget -S --spider $1 2>&1 | grep 'HTTP/1.1 200 OK'` ]]; then echo "true"; fi
Landsat 4-5 Wavelength Landsat 7 Wavelength Landsat 8 Wavelength (micrometers)
Band 1 - Coastal aerosol 0.43 - 0.45
Band 1 0.45-0.52 Band 1 0.45-0.52 Band 2 - Blue 0.45 - 0.51
Band 2 0.52-0.60 Band 2 0.52-0.60 Band 3 - Green 0.53 - 0.59
Band 3 0.63-0.69 Band 3 0.63-0.69 Band 4 - Red 0.64 - 0.67
Band 4 0.76-0.90 Band 4 0.77-0.90 Band 5 - Near Infrared (NIR) 0.85 - 0.88
Band 9 - Cirrus 1.36 - 1.38
Band 5 1.55-1.75 Band 5 1.55-1.75 Band 6 - SWIR 1 1.57 - 1.65
Band 7 2.08-2.35 Band 7 2.09-2.35 Band 7 - SWIR 2 2.11 - 2.29
hrwgc /
Created Oct 22, 2013
remove all non master branches from git repo
function clean_git_branches(){
git checkout master;
BRANCHES=`git branch | grep -Ev '^\s{0,}\*{0,}\s{0,}master$'`
git branch -D ${BRANCHES[@]}
#!/usr/bin/env python
import os
def CopyToClipboard(text):
cmd = "echo '%s' | tr -d \"\\\n\" | pbcopy" % text
hrwgc / nearest_neighbor.sql
Last active Dec 19, 2015
postgis select across polygon layers based on nearest adjacent polygon from other layer.
View nearest_neighbor.sql
create table
closest_point as
select distinct on (a.featureid) a.featureid as image_id,
(select b.eventid from fl_events as b order by st_distance(a.wkb_geometry,b.wkb_geometry) limit 1) as event_id,
a.wkb_geometry from images as a,
fl_events as b;
update images set eventid = closest_point.event_id from closest_point where images.featureid = closest_point.image_id;