[Clojure] Benchmarks for different implementations of `for-map`

for-map-benchmarks.clj

;; Benchmarks for different implementations of `for-map`

;; Can be run with `lein try proteus criterium`

;; See http://www.lexicallyscoped.com/2013/04/17/transients-in-clojure.html
;; for some relevant discussion.

(defn pairs-to-map-without-transient [pairs]
  (reduce (fn [m [k v]] (assoc m k v)) {} pairs))

(defmacro for-map-without-transient [seq-exprs key-expr val-expr]
  `(pairs-to-map-without-transient (for [~@seq-exprs] [~key-expr ~val-expr])))

(defn pairs-to-map-with-into [pairs]
  ;; `into` uses `transient` internally
  (into {} pairs))

(defmacro for-map-with-into [seq-exprs key-expr val-expr]
  `(pairs-to-map-with-into (for [~@seq-exprs] [~key-expr ~val-expr])))

;; Borrowed from https://github.com/Prismatic/plumbing via article above
(defmacro for-map-with-atom [seq-exprs key-expr val-expr]
  `(let [m-atom# (atom (transient {}))]
     (doseq ~seq-exprs
       (reset! m-atom# (assoc! @m-atom# ~key-expr ~val-expr)))
     (persistent! @m-atom#)))

(use 'proteus) ;; https://github.com/ztellman/proteus

(defmacro for-map-with-mutable-let [seq-exprs key-expr val-expr]
  `(let-mutable [m-var# (transient {})]
     (doseq ~seq-exprs
       (set! m-var# (assoc! m-var# ~key-expr ~val-expr)))
     (persistent! m-var#)))

(use 'criterium.core) ;; https://github.com/hugoduncan/criterium

(defmacro bench-for-map [suffix]
  (let [impl (symbol (str "for-map-" suffix))]
    `(do (print "\n# Benchmarking" '~impl "\n\n")
         (bench (~impl [i# (range 20)
                        j# (range 30)]
                  [i# j#] (even? (+ i# j#)))))))

(bench-for-map without-transient)
(bench-for-map with-into)
(bench-for-map with-atom)
(bench-for-map with-mutable-let)

#_ " ^^^
# Benchmarking for-map-without-transient 

Evaluation count : 78900 in 60 samples of 1315 calls.
             Execution time mean : 816,078487 µs
    Execution time std-deviation : 45,740840 µs
   Execution time lower quantile : 750,233162 µs ( 2,5%)
   Execution time upper quantile : 911,396991 µs (97,5%)
                   Overhead used : 14,405629 ns

Found 2 outliers in 60 samples (3,3333 %)
        low-severe       2 (3,3333 %)
 Variance from outliers : 41,7768 % Variance is moderately inflated by outliers

# Benchmarking for-map-with-into 

Evaluation count : 94080 in 60 samples of 1568 calls.
             Execution time mean : 674,850101 µs
    Execution time std-deviation : 56,766687 µs
   Execution time lower quantile : 610,809842 µs ( 2,5%)
   Execution time upper quantile : 815,548529 µs (97,5%)
                   Overhead used : 14,405629 ns

Found 5 outliers in 60 samples (8,3333 %)
        low-severe       4 (6,6667 %)
        low-mild         1 (1,6667 %)
 Variance from outliers : 61,8571 % Variance is severely inflated by outliers

# Benchmarking for-map-with-atom 

Evaluation count : 118920 in 60 samples of 1982 calls.
             Execution time mean : 506,626136 µs
    Execution time std-deviation : 25,218949 µs
   Execution time lower quantile : 464,942166 µs ( 2,5%)
   Execution time upper quantile : 569,008926 µs (97,5%)
                   Overhead used : 14,405629 ns

Found 3 outliers in 60 samples (5,0000 %)
        low-severe       3 (5,0000 %)
 Variance from outliers : 35,2537 % Variance is moderately inflated by outliers

# Benchmarking for-map-with-mutable-let 

Evaluation count : 136080 in 60 samples of 2268 calls.
             Execution time mean : 483,249824 µs
    Execution time std-deviation : 78,265565 µs
   Execution time lower quantile : 427,748592 µs ( 2,5%)
   Execution time upper quantile : 613,729926 µs (97,5%)
                   Overhead used : 14,405629 ns

Found 4 outliers in 60 samples (6,6667 %)
        low-severe       3 (5,0000 %)
        low-mild         1 (1,6667 %)
 Variance from outliers : 85,9209 % Variance is severely inflated by outliers
"