Skip to content

Instantly share code, notes, and snippets.

@matthewdowney
Created November 6, 2018 23:22
Show Gist options
  • Save matthewdowney/3db5a5f054a240762b92789e269866df to your computer and use it in GitHub Desktop.
Save matthewdowney/3db5a5f054a240762b92789e269866df to your computer and use it in GitHub Desktop.
Running summary stats clojure
(defn stats
"Calculate :mean :min :max :stdev and :n given a data point, and possibly a prior `stats` result.
I.e. to calculate statistics from scratch:
```
(def statistics (reduce stats nil data-set))
```
Or to maintain statistics for a constantly updating data series that you don't want to keep in memory:
```
(def statistics (atom (stats 0))
(defn add-data-point! [dp]
(swap! statistics stats dp))
```"
([data-point]
{:mean data-point
:min data-point
:max data-point
:stdev 0
:n 1})
([{:keys [mean stdev n] :as prev} data-point]
(if-not prev
(stats data-point)
(let [;; This is (sum (x^2 for x in data))
sum-squared (* n (+ (* stdev stdev) (* mean mean)))
sum (* n mean) ;; This is just (sum data)
;; Variables after the new data point
sum' (+ sum data-point)
sum-squared' (+ sum-squared (* data-point data-point))
n' (inc n)]
{:mean (/ sum' n')
:min (min (:min prev) data-point)
:max (max (:max prev) data-point)
:stdev (Math/sqrt (/ (- (* n' sum-squared') (* sum' sum'))
(* n' n')))
:n n'}))))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment