Created
November 6, 2018 23:22
-
-
Save matthewdowney/3db5a5f054a240762b92789e269866df to your computer and use it in GitHub Desktop.
Running summary stats clojure
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(defn stats | |
"Calculate :mean :min :max :stdev and :n given a data point, and possibly a prior `stats` result. | |
I.e. to calculate statistics from scratch: | |
``` | |
(def statistics (reduce stats nil data-set)) | |
``` | |
Or to maintain statistics for a constantly updating data series that you don't want to keep in memory: | |
``` | |
(def statistics (atom (stats 0)) | |
(defn add-data-point! [dp] | |
(swap! statistics stats dp)) | |
```" | |
([data-point] | |
{:mean data-point | |
:min data-point | |
:max data-point | |
:stdev 0 | |
:n 1}) | |
([{:keys [mean stdev n] :as prev} data-point] | |
(if-not prev | |
(stats data-point) | |
(let [;; This is (sum (x^2 for x in data)) | |
sum-squared (* n (+ (* stdev stdev) (* mean mean))) | |
sum (* n mean) ;; This is just (sum data) | |
;; Variables after the new data point | |
sum' (+ sum data-point) | |
sum-squared' (+ sum-squared (* data-point data-point)) | |
n' (inc n)] | |
{:mean (/ sum' n') | |
:min (min (:min prev) data-point) | |
:max (max (:max prev) data-point) | |
:stdev (Math/sqrt (/ (- (* n' sum-squared') (* sum' sum')) | |
(* n' n'))) | |
:n n'})))) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment