Create a gist now

Instantly share code, notes, and snippets.

Embed
Specter 0.13.0 benchmark
Benchmark code at: https://github.com/nathanmarz/specter/blob/master/scripts/benchmarks.clj
Run against Clojure 1.7.0 and Java 1.7.0 on Mac OSX 10.11.6
Benchmark: get value in nested map (2500000 iterations)
Avg(ms) vs best Code
53.528 1.00 (-> data (get :a) (get :b) (get :c))
54.708 1.02 (-> data :a :b :c)
103.03 1.92 (compiled-select-any p data)
109.92 2.05 (select-any [:a :b :c] data)
111.81 2.09 (select-any [(keypath :a) (keypath :b) (keypath :c)] data)
158.11 2.95 (specter-dynamic-nested-get data :a :b :c)
170.28 3.18 (get-in data [:a :b :c])
********************************
Benchmark: update value in nested map (500000 iterations)
Avg(ms) vs best Code
83.648 1.00 (manual-transform data inc)
86.441 1.03 (transform [:a :b :c] inc data)
529.97 6.34 (update-in data [:a :b :c] inc)
********************************
Benchmark: transform values of a small map (500000 iterations)
Avg(ms) vs best Code
45.241 1.00 (transform MAP-VALS inc data)
74.505 1.65 (reduce-kv (fn [m k v] (assoc m k (inc v))) {} data)
93.615 2.07 (reduce-kv (fn [m k v] (assoc m k (inc v))) (empty data) data)
102.49 2.27 (persistent! (reduce-kv (fn [m k v] (assoc! m k (inc v))) (transient {}) data))
170.22 3.76 (transform [ALL LAST] inc data)
424.46 9.38 (into {} (for [[k v] data] [k (inc v)]))
476.91 10.5 (zipmap (keys data) (map inc (vals data)))
491.83 10.9 (into {} (map (fn [e] [(key e) (inc (val e))]) data))
********************************
Benchmark: transform values of large map (600 iterations)
Avg(ms) vs best Code
89.321 1.00 (persistent! (reduce-kv (fn [m k v] (assoc! m k (inc v))) (transient clojure.lang.PersistentHashMap/EMPTY) data))
94.117 1.05 (persistent! (reduce-kv (fn [m k v] (assoc! m k (inc v))) (transient {}) data))
94.839 1.06 (transform MAP-VALS inc data)
115.98 1.30 (reduce-kv (fn [m k v] (assoc m k (inc v))) {} data)
120.07 1.34 (reduce-kv (fn [m k v] (assoc m k (inc v))) (empty data) data)
132.00 1.48 (transform [ALL LAST] inc data)
188.62 2.11 (into {} (for [[k v] data] [k (inc v)]))
196.81 2.20 (into {} (map (fn [e] [(key e) (inc (val e))]) data))
258.00 2.89 (zipmap (keys data) (map inc (vals data)))
********************************
Benchmark: map a function over a vector (1000000 iterations)
Avg(ms) vs best Code
165.45 1.00 (into [] (map inc) data)
169.01 1.02 (mapv inc data)
188.44 1.14 (transform ALL inc data)
442.41 2.67 (vec (map inc data))
********************************
Benchmark: filter a sequence (500000 iterations)
Avg(ms) vs best Code
110.79 1.00 (filterv even? data)
129.44 1.17 (select [ALL even?] data)
167.40 1.51 (into [] (filter even?) data)
204.44 1.85 (doall (filter even? data))
246.84 2.23 (select-any (filterer even?) data)
********************************
Benchmark: even :a values from sequence of maps (500000 iterations)
Avg(ms) vs best Code
83.723 1.00 (into [] xf data)
128.32 1.53 (select [ALL :a even?] data)
148.40 1.77 (into [] (comp (map :a) (filter even?)) data)
312.08 3.73 (->> data (mapv :a) (filter even?) doall)
********************************
Benchmark: update every value in a tree (represented with vectors) (50000 iterations)
Avg(ms) vs best Code
81.533 1.00 (tree-value-transform (fn [e] (if (even? e) (inc e) e)) data)
88.786 1.09 (transform [TreeValuesProt even?] inc data)
94.852 1.16 (transform [TreeValues even?] inc data)
324.52 3.98 (walk/postwalk (fn [e] (if (and (number? e) (even? e)) (inc e) e)) data)
339.70 4.17 (transform [(walker number?) even?] inc data)
********************************
Benchmark: Traverse into a set (5000 iterations)
Avg(ms) vs best Code
815.69 1.00 (into #{} (traverse ALL data))
824.53 1.01 (set data)
883.21 1.08 (persistent! (reduce conj! (transient #{}) (traverse ALL data)))
997.47 1.22 (set (select ALL data))
1258.4 1.54 (reduce conj #{} (traverse ALL data))
Benchmark code at: https://github.com/nathanmarz/specter/blob/master/scripts/benchmarks.clj
Run against Clojure 1.8.0 and Java 1.7.0 on Mac OSX 10.11.6
Benchmark: get value in nested map (2500000 iterations)
Avg(ms) vs best Code
56.953 1.00 (-> data :a :b :c)
62.537 1.10 (-> data (get :a) (get :b) (get :c))
106.82 1.88 (compiled-select-any p data)
113.30 1.99 (select-any [(keypath :a) (keypath :b) (keypath :c)] data)
118.95 2.09 (select-any [:a :b :c] data)
136.59 2.40 (select-one [:a :b :c] data)
139.85 2.46 (select-first [:a :b :c] data)
147.64 2.59 (select-one! [:a :b :c] data)
165.78 2.91 (specter-dynamic-nested-get data :a :b :c)
175.39 3.08 (get-in data [:a :b :c])
********************************
Benchmark: update value in nested map (500000 iterations)
Avg(ms) vs best Code
88.003 1.00 (transform [:a :b :c] inc data)
100.64 1.14 (manual-transform data inc)
509.91 5.79 (update-in data [:a :b :c] inc)
********************************
Benchmark: transform values of a small map (500000 iterations)
Avg(ms) vs best Code
52.867 1.00 (transform MAP-VALS inc data)
78.891 1.49 (map-vals-map-iterable data inc)
124.30 2.35 (map-vals-map-iterable-transient data inc)
153.92 2.91 (persistent! (reduce-kv (fn [m k v] (assoc! m k (inc v))) (transient {}) data))
154.18 2.92 (reduce-kv (fn [m k v] (assoc m k (inc v))) (empty data) data)
158.01 2.99 (reduce-kv (fn [m k v] (assoc m k (inc v))) {} data)
177.76 3.36 (transform [ALL LAST] inc data)
414.94 7.85 (zipmap (keys data) (map inc (vals data)))
471.38 8.92 (into {} (for [[k v] data] [k (inc v)]))
471.92 8.93 (into {} (map (fn [e] [(key e) (inc (val e))]) data))
********************************
Benchmark: transform values of large map (600 iterations)
Avg(ms) vs best Code
115.11 1.00 (persistent! (reduce-kv (fn [m k v] (assoc! m k (inc v))) (transient {}) data))
119.70 1.04 (transform MAP-VALS inc data)
120.55 1.05 (persistent! (reduce-kv (fn [m k v] (assoc! m k (inc v))) (transient clojure.lang.PersistentHashMap/EMPTY) data))
135.85 1.18 (reduce-kv (fn [m k v] (assoc m k (inc v))) (empty data) data)
138.02 1.20 (map-vals-map-iterable-transient data inc)
142.85 1.24 (reduce-kv (fn [m k v] (assoc m k (inc v))) {} data)
160.36 1.39 (transform [ALL LAST] inc data)
162.97 1.42 (map-vals-map-iterable data inc)
208.67 1.81 (into {} (for [[k v] data] [k (inc v)]))
211.33 1.84 (into {} (map (fn [e] [(key e) (inc (val e))]) data))
236.55 2.06 (zipmap (keys data) (map inc (vals data)))
********************************
Benchmark: map a function over a vector (1000000 iterations)
Avg(ms) vs best Code
164.71 1.00 (into [] (map inc) data)
166.34 1.01 (mapv inc data)
198.93 1.21 (transform ALL inc data)
438.70 2.66 (vec (map inc data))
********************************
Benchmark: filter a sequence (500000 iterations)
Avg(ms) vs best Code
94.955 1.00 (filterv even? data)
121.26 1.28 (select [ALL even?] data)
160.50 1.69 (doall (filter even? data))
163.26 1.72 (into [] (filter even?) data)
229.03 2.41 (select-any (filterer even?) data)
********************************
Benchmark: even :a values from sequence of maps (500000 iterations)
Avg(ms) vs best Code
80.767 1.00 (into [] xf data)
125.40 1.55 (select [ALL :a even?] data)
148.34 1.84 (into [] (comp (map :a) (filter even?)) data)
257.12 3.18 (->> data (mapv :a) (filter even?) doall)
********************************
Benchmark: update every value in a tree (represented with vectors) (50000 iterations)
Avg(ms) vs best Code
72.504 1.00 (tree-value-transform (fn [e] (if (even? e) (inc e) e)) data)
76.809 1.06 (transform [TreeValuesProt even?] inc data)
89.608 1.24 (transform [TreeValues even?] inc data)
247.96 3.42 (transform [(walker number?) even?] inc data)
289.12 3.99 (walk/postwalk (fn [e] (if (and (number? e) (even? e)) (inc e) e)) data)
********************************
Benchmark: Traverse into a set (5000 iterations)
Avg(ms) vs best Code
788.05 1.00 (set data)
846.60 1.07 (into #{} (traverse ALL data))
879.60 1.12 (persistent! (reduce conj! (transient #{}) (traverse ALL data)))
962.73 1.22 (set (select ALL data))
1376.1 1.75 (reduce conj #{} (traverse ALL data))
@greywolve

This comment has been minimized.

Show comment
Hide comment
@greywolve

greywolve Dec 22, 2016

This is useful, but is there any reason you didn't use something like Criterium to benchmark? I'm really weary of micro benchmark results. Thanks for sharing though. :)

This is useful, but is there any reason you didn't use something like Criterium to benchmark? I'm really weary of micro benchmark results. Thanks for sharing though. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment