Skip to content

Instantly share code, notes, and snippets.

@bfabry
Last active August 29, 2015 14:06
Show Gist options
  • Save bfabry/ad830b1888e4fc550f88 to your computer and use it in GitHub Desktop.
Save bfabry/ad830b1888e4fc550f88 to your computer and use it in GitHub Desktop.
(let [num-workers 4
widgets-per-worker (ceil (/ (count widgets) num-workers))
bucketed-widgets (partition-all widgets-per-worker widgets)
workers (doall (map (fn [widgets]
(thread
(doseq [widget widgets]
(long-running-widget-processor widget))
true))
bucketed-widgets))]
(doall (map <!! workers)))
@johnwalker
Copy link

It probably makes more sense to use future and deref instead of thread and <!!, since long-running-widget-processor isn't streaming. (I am not an expert though :) )

With futures, one way you could rewrite this is

(doall (pmap (fn [widgets]
         (doseq [widget widgets]
            (long-running-widget-processor widget))
         true)
         bucketed widgets))

Ultimately, which you should use depends mostly on the properties of long-running-widget-processor. If the task is IO-bound, then you should certainly use futures. Otherwise it is a little gray, and it's best to check by trying them out. Note that there are many options other than core.async and futures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment