Skip to content

Instantly share code, notes, and snippets.

@reborg
Last active August 29, 2017 10:38
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save reborg/5fed47af3fc8c414040efd83603738e2 to your computer and use it in GitHub Desktop.
Save reborg/5fed47af3fc8c414040efd83603738e2 to your computer and use it in GitHub Desktop.
Updating a java.util.HashMap in parallel with reducers
;; main idea is to create a vector out of the keys
;; and use already existent foldvec to reduce-combine
;; on the key set. Each parallel thread takes an
;; unique set of keys and update the same HashMap
;; instance. "identity" is used as transforming function "f".
;; Just replace with the transformation you need.
(require '[clojure.core.reducers :as r])
(import 'java.util.HashMap)
(extend-protocol r/CollFold
java.util.HashMap
(coll-fold
[m n combinef reducef]
(#'r/foldvec (into [] (keys m)) n combinef reducef)))
(defn large-map [i]
(into {}
(map vector (range i) (range i))))
(def res
(let [m (HashMap. (large-map 1000000))
f identity]
(r/fold (constantly m) (fn [m k] (.put m k (f (.get m k))) m) m)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment