danhammer

(def beta (let [src (hfs-seqfile "/mnt/hgfs/Dropbox/betas")]
            (vec (flatten
                  (??<- [?beta]
                        (src _ ?eco ?beta)
                        (= ?eco 40146))))))

(defn logistic-prob-wrap
  [beta-vec feature-vec]
  (let [beta-mat (to-double-rowmat beta-vec)
        features-mat (to-double-rowmat (vec (flatten feature-vec)))]

danhammer

dan@hammer-statistic:~/Dropbox/github/danhammer/forma-clj$ lein clean
Downloading: lein-swank/lein-swank/1.4.4/lein-swank-1.4.4.pom from repository central at http://repo1.maven.org/maven2
Unable to locate resource in repository
[INFO] Unable to find resource 'lein-swank:lein-swank:pom:1.4.4' in repository central (http://repo1.maven.org/maven2)
Downloading: lein-swank/lein-swank/1.4.4/lein-swank-1.4.4.pom from repository clojars at http://clojars.org/repo/
Transferring 1K from clojars
Downloading: lein-swank/lein-swank/1.4.4/lein-swank-1.4.4.jar from repository central at http://repo1.maven.org/maven2
Unable to locate resource in repository
[INFO] Unable to find resource 'lein-swank:lein-swank:jar:1.4.4' in repository central (http://repo1.maven.org/maven2)
Downloading: lein-swank/lein-swank/1.4.4/lein-swank-1.4.4.jar from repository clojars at http://clojars.org/repo/

danhammer

(ns forma-local.core
  (:use forma.thrift
        cascalog.api
        [forma.hadoop.pail :only (?pail- split-chunk-tap)])
  (:require [incanter.core :as i]
            [cascalog.ops :as c]
            [forma.trends.analysis :as analyze]
            [forma.thrift :as thrift]
            [forma.classify.logistic :as log])
  (:import [backtype.hadoop.pail Pail]))

danhammer

fire    2011-01-01      1000    21      8       177     234     <DataValue fireVal:FireValue(temp330:0, conf50:0, bothPreds:0, count:1)>
fire    2011-01-01      1000    21      8       178     201     <DataValue fireVal:FireValue(temp330:0, conf50:0, bothPreds:0, count:1)>
fire    2011-01-01      1000    21      8       178     239     <DataValue fireVal:FireValue(temp330:0, conf50:1, bothPreds:0, count:1)>
fire    2011-01-01      1000    21      8       179     504     <DataValue fireVal:FireValue(temp330:0, conf50:0, bothPreds:0, count:1)>
fire    2011-01-01      1000    21      8       180     239     FireValue(temp330:0, conf50:2, bothPreds:0, count:2)
fire    2011-01-01      1000    21      8       181     8       <DataValue fireVal:FireValue(temp330:0, conf50:1, bothPreds:0, count:1)>
fire    2011-01-01      1000    21      8       181     144     <DataValue fireVal:FireValue(temp330:0, conf50:1, bothPreds:0, count:1)>
fire    2011-01-01      1000    21      8       181     202     <DataValue fireVal:Fi

danhammer

# coding=utf-8
import os, sys, json
import numpy as np
import matplotlib.pylab as plt
from scikits.timeseries.lib.interpolate import *
import scikits.statsmodels.sandbox.stats.diagnostic as diag
from scikits.statsmodels.tsa.filters import hpfilter
import scikits.statsmodels.api as sm

def make_reliable(target_ts, reli_ts=None):

danhammer

(ns forma.trends.entropy
  (:use [forma.matrix.utils]
        [clojure.math.numeric-tower :only (sqrt floor abs expt)]
        [clojure-csv.core])
  (:require [forma.utils :as utils]
            [incanter.core :as i]
            [incanter.stats :as s]
            [incanter.charts :as c]
            [forma.trends.filter :as filter]))

danhammer

(ns computing.infotheory
  (:use [clojure.math.numeric-tower :only (sqrt floor abs expt)])
  (:require [incanter.core :as i]))
 
(defn- ordinal-idx
  "Returns a sequence of indices that rank the values of the supplied
  time series in ascending order.  If there are equal values, the
  lexicographic ordering kicks in and the order at which the values
  appear is used to order the indices.
 

danhammer

(ns computing.dice
  (:require [incanter.core :as i]
            [incanter.optimize :as optimize]
            [incanter.charts :as c]
            [incanter.stats :as s]))

;; Given the observed average from rolling a six-sided die many times,
;; what are the probabilities associated with each side of the die?
;; For a fair die, all sides are equally likely, such that the
;; probability is 1/6 for each side.  If we roll the fair die

danhammer

boss-clj

Clojure implementation of the Balance Optimization Subset Selection method.

Installation

If you have Git, Clojure, and Lein already installed, you're all set:

$ ./boss help

danhammer

We continue to work with Thau and colleagues to port the FORMA algorithm to Earth Engine. It is clear that the exact FORMA algorithm cannot be implemented on EE without considerable effort from the Google team. There is currently no logistic classifier, for example, among other constraints. However, we are working with Thau et al. to make minor modifications to the algorithm to balance fidelity with ease/speed of implementation. Many of these modifications do not seem like concessions at all, but rather enhancements to the current methodology. With that said, the following is a list, to date, of the items that we have completed in the ongoing effort to implement the FORMA algorithm with the EE API.

1. Built the cluster infrastructure to map across partitions and specifically to map the training algorithm across each tropical ecoregion. The objective is to build a classifier image for each ecoregion, independent of any other ecoregion. We had briefly experimented with the map() method in Earth Eng