Last active
August 29, 2017 10:38
-
-
Save reborg/5fed47af3fc8c414040efd83603738e2 to your computer and use it in GitHub Desktop.
Updating a java.util.HashMap in parallel with reducers
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
;; main idea is to create a vector out of the keys | |
;; and use already existent foldvec to reduce-combine | |
;; on the key set. Each parallel thread takes an | |
;; unique set of keys and update the same HashMap | |
;; instance. "identity" is used as transforming function "f". | |
;; Just replace with the transformation you need. | |
(require '[clojure.core.reducers :as r]) | |
(import 'java.util.HashMap) | |
(extend-protocol r/CollFold | |
java.util.HashMap | |
(coll-fold | |
[m n combinef reducef] | |
(#'r/foldvec (into [] (keys m)) n combinef reducef))) | |
(defn large-map [i] | |
(into {} | |
(map vector (range i) (range i)))) | |
(def res | |
(let [m (HashMap. (large-map 1000000)) | |
f identity] | |
(r/fold (constantly m) (fn [m k] (.put m k (f (.get m k))) m) m))) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment