Debugging Clojure/Vagrant for 7 concurrency models in 7 weeks

gistfile1.txt

I am trying to replicate an example from this book: https://uk.bookshop.org/books/seven-concurrency-models-in-seven-weeks-when-threads-unravel/9781937785659

Running the below commands in a lein repl.

In the book, the second `parallel-sum` function executes 2.5x as fast as `sum` but on my Vagrant box (which has 4 cores, and demonstrates other parallelised code having a speed up effect) the two functions take pretty much the same time every time. My Vagrantfile is here https://github.com/annashipman/7weeks-concurrency/blob/main/Vagrantfile

This works in a non-Vagrant Clojure environment, i.e parallel-sum runs much faster.

So, what should I do differently in Vagrant to get it to work?


(ns sum.core
	(:require [clojure.core.reducers :as r]))
	
(defn sum [numbers]
 	(reduce + numbers))

(defn parallel-sum [numbers]
	(r/fold + numbers))

(def numbers (into [] (range 0 10000000)))

(time (sum numbers))

(time (sum numbers))

(time (parallel-sum numbers))

(time (parallel-sum numbers))

Running the same test cases on a hardware machine (here a MBP Core i7 2.7GHz 16Gb RAM) and OpenJDK 17 (not OpenJDK 1.8), the garbage collector is now G1 by default and the results are:

$ lein repl 
[...]
Clojure 1.10.3
OpenJDK 64-Bit Server VM 17+0
[...]
user=> (load-file "test-sum.clj")
"Elapsed time: 307.124373 msecs"
"Elapsed time: 359.415788 msecs"
49999995000000

$ lein repl 
[...]
Clojure 1.10.3
OpenJDK 64-Bit Server VM 17+0
[...]
user=> (load-file "test-parallel-sum.clj")
"Elapsed time: 175.177959 msecs"
"Elapsed time: 137.79619 msecs"
49999995000000

and with -Xms3g -Xmx3g:

$ LEIN_JVM_OPTS="-Xms3g -Xmx3g" lein repl
[...]
Clojure 1.10.3
OpenJDK 64-Bit Server VM 1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10
[...]
user=> (load-file "test-sum.clj")
"Elapsed time: 144.63988 msecs"
"Elapsed time: 117.451834 msecs"
49999995000000

$ LEIN_JVM_OPTS="-Xms3g -Xmx3g" lein repl
[...]
Clojure 1.10.3
OpenJDK 64-Bit Server VM 1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10
[...]
user=> (load-file "test-parallel-sum.clj")
"Elapsed time: 178.446205 msecs"
"Elapsed time: 74.989178 msecs"
49999995000000

⚠️ The above examples are only ran 1 time each, I don't pretend to present any concurrency benchmarking, but demonstrate that GC impact is tremendous and can prevent to compare parallelism gains.