Skip to content

Instantly share code, notes, and snippets.

@robinkraft
robinkraft / gist:03c28924c8203b6ce6dc
Last active August 29, 2015 14:07
Rainforest Connection looking for talent

SOFTWARE DEVELOPMENT

Rainforest Connection is a San Francisco-based nonprofit building open-source solutions to illegal logging and poaching. Our flagship project is the world’s first real-time forest monitoring system, built from recycled Android smartphones.

We have a number of pilot deployments that will begin shortly. We’re looking to build an engineering team to join us and bring our platform to the next level throughout the stack.

Let us know what you can do and your requirements. We ourselves have been working on this project for 1.5 years without being paid, so we’re not afraid to ask others to dive in. If we’re successful, money is unlikely to be a problem. Our basic philosophy is that technology and innovation should be used to solve real world problems, rather than creating “shiny objects” or harnessing the best minds in the world to say ... democratize cab service across town.

We appreciate fast learners with a sense of hum

# A mill with less than 5% UMD loss from 2005-2012 is low risk, 5-28% is medium risk, and over 28% is high risk.
# One within concessions and one within entire radius
# Use the area of the entire radius or concession as the denominator (effectively assume 100% forested)
import json
import requests # sudo pip install requests
APIURL = "http://wri-01.cartodb.com/api/v2/sql"
def parse_concession_json(j):
@robinkraft
robinkraft / gist:40f0d9dc13db2b5eb458
Created January 29, 2015 00:19
CLI commands to convert FORMA geotiff to text, with thresholding
gdal_translate -of XYZ ~/Downloads/forma-30104_2005-04-23-0000000000-0000000000.tif /tmp/forma.txt
grep -v "nan" /tmp/forma.txt | awk '$3 > 0.5' > /tmp/forma_hits.txt
@robinkraft
robinkraft / check-data.clj
Last active August 29, 2015 14:21
debug MODIS time series in FORMA processing
(use 'forma.hadoop.jobs.forma)
(in-ns 'forma.hadoop.jobs.forma)
(use 'forma.hadoop.pail)
(let [pail-path "s3n://pailbucket/all-master"
out-path "s3n://pailbucket/output/run-2015-05-14/date-counts"
s-res "500"
t-res "16"
pail-src (split-chunk-tap pail-path ["ndvi" (format "%s-%s" s-res t-res)])
sink (hfs-textline out-path :sinkmode :replace)]
@robinkraft
robinkraft / f250-cdm.clj
Last active August 29, 2015 14:21
convert FORMA 250m data into common data model suitable for GlobalForestWatch.org website
(use 'forma.hadoop.jobs.postprocess)
(in-ns 'forma.hadoop.jobs.postprocess)
(defn f250->cdm
[src t-res zoom]
(let [epoch (date/datetime->period t-res "2000-01-01")]
(<- [?x ?y ?z ?min-period]
(src ?line)
(clojure.string/split ?line #"\t" :> ?lat-str ?lon-str ?date-str)
(read-string ?lat-str :> ?lat)
import os
import sys
import requests
basefname = 'Hansen_GFC2013_lossyear_%d%s_%03d%s.tif'
baseurl = 'http://commondatastorage.googleapis.com/earthenginepartners-hansen/GFC2013/%s'
def download(url, output):
@robinkraft
robinkraft / start.clj
Created July 8, 2011 18:44
BytesWritable error storing data using elephant-tap
(ns eleph.start
(:use [cascalog api])
(:require [elephantdb.cascalog.core :as e])
(:import [elephantdb.persistence JavaBerkDB]))
(def data [
[1 "dan"]
[2 "sam"]
[3 "robin"]
[4 "david"]
@robinkraft
robinkraft / sam.clj
Created January 25, 2012 17:53
first macro!
(defmacro sam [expr pos neg zero]
`(cond (pos? ~expr) ~pos
(neg? ~expr) ~neg
:else ~zero))
@robinkraft
robinkraft / Stata_monthly_replace.do
Created February 10, 2012 01:36
Replacing monthly values in Stata
forvalues year = 2005/2011 {
forvalues month = 1/12 {
if `month' < 10 {
local mo_str = "0`month'"
}
else {
local mo_str = "`month'"
}
local yearmo = "`year'`mo_str'"
di "`yearmo'"
@robinkraft
robinkraft / precl_loader.py
Created February 10, 2012 17:40
Load PRECL data, print min, max, sample vals
import numpy as np
for i in range(2000, 2013):
rain = np.fromfile("precl_mon_v1.0.lnx.%i.gri0.5m" % i, dtype="f", count=-1)
print "year:", i
print "max:", rain.max()
print "min:", rain.min()
print "first 100:", rain[:100]