Skip to content

Instantly share code, notes, and snippets.

@IGJoshua
Last active February 11, 2022 17:45
Show Gist options
  • Save IGJoshua/98f3302d09f1fdcb97dcb1690d718508 to your computer and use it in GitHub Desktop.
Save IGJoshua/98f3302d09f1fdcb97dcb1690d718508 to your computer and use it in GitHub Desktop.
Benchmarking clojure.core/not-any? and alternatives
;;;; bench.clj
;;; Run on a 10th gen core i7 pro mobile processor, JDK 17, Clojure 1.10.3
(use 'criterium.core)
(defn the-pred
[c]
(Character/isDigit c))
(def digit-str "blah1blah")
(def non-digit-str "blah'blah")
(defn not-any?* [pred s]
(not-any? pred s))
(bench (not-any?* the-pred digit-str) :verbose)
;; Evaluation count : 3129240 in 60 samples of 52154 calls.
;; Execution time sample mean : 19.020819 µs
;; Execution time mean : 19.020387 µs
;; Execution time sample std-deviation : 193.272969 ns
;; Execution time std-deviation : 196.852449 ns
;; Execution time lower quantile : 18.784042 µs ( 2.5%)
;; Execution time upper quantile : 19.510881 µs (97.5%)
;; Overhead used : 6.536755 ns
;; Found 2 outliers in 60 samples (3.3333 %)
;; low-severe 1 (1.6667 %)
;; low-mild 1 (1.6667 %)
;; Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
(bench (not-any?* the-pred non-digit-str) :verbose)
;; Evaluation count : 2183400 in 60 samples of 36390 calls.
;; Execution time sample mean : 34.140458 µs
;; Execution time mean : 34.124835 µs
;; Execution time sample std-deviation : 2.410481 µs
;; Execution time std-deviation : 2.444503 µs
;; Execution time lower quantile : 27.569290 µs ( 2.5%)
;; Execution time upper quantile : 39.855339 µs (97.5%)
;; Overhead used : 6.536755 ns
;; Found 11 outliers in 60 samples (18.3333 %)
;; low-severe 5 (8.3333 %)
;; low-mild 1 (1.6667 %)
;; high-mild 5 (8.3333 %)
;; Variance from outliers : 53.4648 % Variance is severely inflated by outliers
(defn reduced?* [pred s]
(not
(reduce #(when (pred %2) (reduced %2))
nil
s)))
(bench (reduced?* the-pred digit-str) :verbose)
;; Evaluation count : 3909360 in 60 samples of 65156 calls.
;; Execution time sample mean : 18.935604 µs
;; Execution time mean : 18.927436 µs
;; Execution time sample std-deviation : 797.250614 ns
;; Execution time std-deviation : 804.951727 ns
;; Execution time lower quantile : 15.560927 µs ( 2.5%)
;; Execution time upper quantile : 19.747532 µs (97.5%)
;; Overhead used : 6.536755 ns
;; Found 7 outliers in 60 samples (11.6667 %)
;; low-severe 4 (6.6667 %)
;; low-mild 3 (5.0000 %)
;; Variance from outliers : 28.7041 % Variance is moderately inflated by outliers
(bench (reduced?* the-pred non-digit-str) :verbose)
;; Evaluation count : 2170440 in 60 samples of 36174 calls.
;; Execution time sample mean : 35.058948 µs
;; Execution time mean : 35.077158 µs
;; Execution time sample std-deviation : 2.194051 µs
;; Execution time std-deviation : 2.216457 µs
;; Execution time lower quantile : 33.642224 µs ( 2.5%)
;; Execution time upper quantile : 42.079771 µs (97.5%)
;; Overhead used : 6.536755 ns
;; Found 7 outliers in 60 samples (11.6667 %)
;; low-severe 1 (1.6667 %)
;; low-mild 3 (5.0000 %)
;; high-mild 3 (5.0000 %)
;; Variance from outliers : 46.8097 % Variance is moderately inflated by outliers
(defn with-loop* [pred s]
(not
(loop [s (seq s)]
(when s
(if (pred (first s))
true
(recur (next s)))))))
(bench (with-loop* the-pred digit-str) :verbose)
;; Evaluation count : 3899220 in 60 samples of 64987 calls.
;; Execution time sample mean : 18.765890 µs
;; Execution time mean : 18.764823 µs
;; Execution time sample std-deviation : 1.181885 µs
;; Execution time std-deviation : 1.188561 µs
;; Execution time lower quantile : 15.404703 µs ( 2.5%)
;; Execution time upper quantile : 19.634806 µs (97.5%)
;; Overhead used : 6.536755 ns
;; Found 9 outliers in 60 samples (15.0000 %)
;; low-severe 7 (11.6667 %)
;; low-mild 1 (1.6667 %)
;; high-mild 1 (1.6667 %)
;; Variance from outliers : 46.8170 % Variance is moderately inflated by outliers
(bench (with-loop* the-pred non-digit-str) :verbose)
;; Evaluation count : 2073060 in 60 samples of 34551 calls.
;; Execution time sample mean : 33.502466 µs
;; Execution time mean : 33.493995 µs
;; Execution time sample std-deviation : 2.073731 µs
;; Execution time std-deviation : 2.100721 µs
;; Execution time lower quantile : 27.695386 µs ( 2.5%)
;; Execution time upper quantile : 34.837555 µs (97.5%)
;; Overhead used : 6.536755 ns
;; Found 9 outliers in 60 samples (15.0000 %)
;; low-severe 8 (13.3333 %)
;; low-mild 1 (1.6667 %)
;; Variance from outliers : 46.7868 % Variance is moderately inflated by outliers
(defn char-at* [pred ^String s]
(not
(reduce #(let [c (.charAt s %2)]
(when (pred c)
(reduced c)))
nil
(range (count s)))))
(bench (char-at* the-pred digit-str) :verbose)
;; Evaluation count : 2918220 in 60 samples of 48637 calls.
;; Execution time sample mean : 25.044015 µs
;; Execution time mean : 25.046351 µs
;; Execution time sample std-deviation : 2.866108 µs
;; Execution time std-deviation : 2.881106 µs
;; Execution time lower quantile : 22.254331 µs ( 2.5%)
;; Execution time upper quantile : 31.833576 µs (97.5%)
;; Overhead used : 6.582262 ns
;; Found 1 outliers in 60 samples (1.6667 %)
;; low-severe 1 (1.6667 %)
;; Variance from outliers : 75.5099 % Variance is severely inflated by outliers
(bench (char-at* the-pred non-digit-str) :verbose)
;; Evaluation count : 2120460 in 60 samples of 35341 calls.
;; Execution time sample mean : 45.939784 µs
;; Execution time mean : 46.065053 µs
;; Execution time sample std-deviation : 7.572220 µs
;; Execution time std-deviation : 7.715464 µs
;; Execution time lower quantile : 39.235873 µs ( 2.5%)
;; Execution time upper quantile : 65.948416 µs (97.5%)
;; Overhead used : 6.582262 ns
;; Found 6 outliers in 60 samples (10.0000 %)
;; low-severe 4 (6.6667 %)
;; low-mild 2 (3.3333 %)
;; Variance from outliers : 87.6090 % Variance is severely inflated by outliers
(defn char-at** [pred ^String s]
(let [cnt (count s)]
(not
(loop [idx 0]
(if (>= idx cnt)
false
(if (pred (.charAt s idx))
true
(recur (inc idx))))))))
(bench (char-at** the-pred digit-str) :verbose)
;; Evaluation count : 2488380 in 60 samples of 41473 calls.
;; Execution time sample mean : 26.404125 µs
;; Execution time mean : 26.451781 µs
;; Execution time sample std-deviation : 4.215944 µs
;; Execution time std-deviation : 4.332298 µs
;; Execution time lower quantile : 21.803346 µs ( 2.5%)
;; Execution time upper quantile : 35.273507 µs (97.5%)
;; Overhead used : 6.582262 ns
;; Found 2 outliers in 60 samples (3.3333 %)
;; low-severe 2 (3.3333 %)
;; Variance from outliers : 85.9375 % Variance is severely inflated by outliers
(bench (char-at** the-pred non-digit-str) :verbose)
;; Evaluation count : 2137320 in 60 samples of 35622 calls.
;; Execution time sample mean : 42.053556 µs
;; Execution time mean : 42.073829 µs
;; Execution time sample std-deviation : 2.992874 µs
;; Execution time std-deviation : 3.029927 µs
;; Execution time lower quantile : 38.881682 µs ( 2.5%)
;; Execution time upper quantile : 50.348634 µs (97.5%)
;; Overhead used : 6.582262 ns
;; Found 8 outliers in 60 samples (13.3333 %)
;; low-severe 1 (1.6667 %)
;; low-mild 5 (8.3333 %)
;; high-mild 2 (3.3333 %)
;; Variance from outliers : 53.4810 % Variance is severely inflated by outliers
@IGJoshua
Copy link
Author

IGJoshua commented Feb 11, 2022

I also ran a couple benchmarks with variants that had no reflection in them, and observed performance that was kinda in-line with what I expected, that the versions with charAt were faster than the other versions, but not by a startling margin. My final results for a version of char-at** got 60 ns on the digit-str while not-any?* got 160 ns on the same.
But I'm short on time and it wasn't worth the effort of trying to re-run all the benchmarks and update the gist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment