Comparing multimethod performance with table lookup

multi-vs-table.clj

; As I was reading about multimethods I began to wonder if they
; would be a reasonable way to organize code that would normally
; do a table lookup based on some data in order to get at something
; (like a function in the Clojure case) to use to continue to process
; the data. There are times that I like to set up a table with functioanlity
; that can be keyed into in lieu of explicit conditional code. But that's
; beside the point. I wondered if there would be a significant difference
; in performance. If my memory is correct, Richard O'Keefe in The Craft of
; Prolog instilled in me that the only way to know these things is to 
; benchmark them. I'm glad I did. I was a little surprised.

; I made two indentical files with different extensions because it
; was the easiest way I knew of to execute one in Clojure and one in
; ClojureScript. I just wrote them in LightTable and let it do the work.

;; This is the Clojure one.


(ns mvt-bench)

(defn eo [v]
  (if (even? (count v))
    :even
    :odd))

(defmulti multi eo)
(defmethod multi :even [v] (apply + v))
(defmethod multi :odd [v] (apply - v))

(multi [1 2])
(multi [1 2 3])

(defn table [v]
  (let [table {:even + :odd -}]
    (apply ((eo v) table) v)))

(table [1 2])
(table [1 2 3])

(defn bench [f ntimes]
  (let [v1 [1 2]
        v2 [1 2 3]]
    (time
      (dotimes [_ ntimes]
       (f v1)
       (f v2)))))

(def runs 100000)
(bench multi runs)
(bench table runs)

;; A typical run produces this console output.
;;
;; I wasn't too surprised with this. At least I was hoping that both versions
;; would run in similar times. If anything, I expected table to be only slightly
;; faster. But they are pretty neck and neck.
;;
;; multi-vs-table.clj: "Elapsed time: 76.878 msecs"
;; multi-vs-table.clj: "Elapsed time: 76.98 msecs"

multi-vs-table.cljs

; As I was reading about multimethods I began to wonder if they
; would be a reasonable way to organize code that would normally
; do a table lookup based on some data in order to get at something
; (like a function in the Clojure case) to use to continue to process
; the data. There are times that I like to set up a table with functioanlity
; that can be keyed into in lieu of explicit conditional code. But that's
; beside the point. I wondered if there would be a significant difference
; in performance. If my memory is correct, Richard O'Keefe in The Craft of
; Prolog instilled in me that the only way to know these things is to 
; benchmark them. I'm glad I did. I was a little surprised.

; I made two indentical files with different extensions because it
; was the easiest way I knew of to execute one in Clojure and one in
; ClojureScript. I just wrote them in LightTable and let it do the work.

;; This is the ClojureScript one.


(ns mvt-bench)

(defn eo [v]
  (if (even? (count v))
    :even
    :odd))

(defmulti multi eo)
(defmethod multi :even [v] (apply + v))
(defmethod multi :odd [v] (apply - v))

(multi [1 2])
(multi [1 2 3])

(defn table [v]
  (let [table {:even + :odd -}]
    (apply ((eo v) table) v)))

(table [1 2])
(table [1 2 3])

(defn bench [f ntimes]
  (let [v1 [1 2]
        v2 [1 2 3]]
    (time
      (dotimes [_ ntimes]
       (f v1)
       (f v2)))))

(def runs 100000)
(bench multi runs)
(bench table runs)

;; A typical run produces this console output.
;;
;; I expected the same here as for Clojure. What I got was basically
;; multi is 3 times slower than table. I don't know why, just posting for now.
;; Maybe I'm doing something horribly wrong in how I'm comparing them.
;; I wasn't trying to be super precise but I don't see anything that I'm
;; doing incorrectly that would show this kind of disparity.
;;
;; So, don't go assuming relative performance characteristics carry over from
;; Clojure to ClojureScript. But maybe this is known and I've missed it.
;;
;; multi-vs-table.cljs[eval39][43]: "Elapsed time: 1350 msecs"
;; multi-vs-table.cljs[eval39][43]: "Elapsed time: 459 msecs"

The question of multimethod performance in ClojureScript was on my mind recently and I came across this gist. As far as I understand, there have been some measures taken to improve the performance of multimethods, for example: clojure/clojurescript@8af3ae8

Just re-ran this code on some newer CLJS versions:

;; CLJS 0.0-2843:
;; "Elapsed time: 623 msecs"
;; "Elapsed time: 309 msecs"


;; CLJS 0.0-3211:
;; "Elapsed time: 704 msecs"
;; "Elapsed time: 522 msecs"

Not sure if there are any insights to be gleamed from this very cursory benchmark. It'd be great to hear more about this from someone in the know. In cases where I'm interested on dispatching to a handler based on keywords, it might be nice to use multimethods to structure the code; but not if there is a significant performance overhead.