I'm using the ring buffers from this library: https://github.com/amalloy/ring-buffer to define a new structure composed of N ring-buffers where the "overflow" of the first buffer goes to fill in the second one and the overflow of the Xth buffer goes to fill the X+1 buffer. This in order to later allow to keep the min and max in each ring-buffer without having to fully recompute them everytime a new value arrives.
(deftype RBuffers [bufs]
IPersistentCollection
(cons [this x]
(RBuffers. (map (fn [buf a] (if-not (nil? a)
(.cons buf a)
buf))
bufs
(concat (rest (map #(when (full? %) (.peek %)) bufs)) [x]))))
(seq [_] (apply concat bufs)))
; Constructor
(defn rbuffers [x y]
(RBuffers. (repeat x (ring-buffer y)) nil))Using criterium to benchmark the performance, I first tried to get an idea of the performance of ring-buffer on its own (quick-bench (into (ring-buffer 1500) 100000)) and get roughly 50ms to put 100 000 elements into a ring-buffer of size 1500.
Then I tried the same thing with an RBuffers of 20 ring-buffers: (quick-bench (into (rbuffers 20 1500) (range 100000))). This gives me an average result of 21 seconds. This is a 400 times increase and makes it waaaay too slow for my app to perform correctly.
- Am I missing something obvious performance wise?
- Is it time to turn to Java to code this data structure? The new values are added only in one place and only read by the rest of the app so having a non persistent structure would be ok for me here.
Trying to figure out where the increase was coming from, I first replaced the (cons) method to just map and add the same value to all the buffers which removes all the logic of when to add what to which ring-buffer. I expected to see a roughly 20 times increase, but I end up with times around 6 seconds. Or a 120 times increase.