Skip to content

Instantly share code, notes, and snippets.

@t0yv0
Created July 20, 2010 15:56
Show Gist options
  • Save t0yv0/483152 to your computer and use it in GitHub Desktop.
Save t0yv0/483152 to your computer and use it in GitHub Desktop.
let MapReduce (map: 'T1 -> 'T2)
(reduce: 'T2 -> 'T2 -> 'T2)
(zero: 'T2)
(inputs: seq<'T1>) =
let ( >>= ) x f = async.Bind(x, f)
Async.FromContinuations <| fun (ok, no, _) ->
let n = Seq.length inputs
let reducer =
MailboxProcessor.Start <| fun inbox ->
let rec loop n s =
if n = 0 then
async.Return (ok s)
else
inbox.Receive() >>= fun x ->
try loop (n - 1) (reduce s x)
with e -> async.Return (no e)
loop n zero
for x in inputs do
async { return try reducer.Post (map x) with e -> no e }
|> Async.Start
@t0yv0
Copy link
Author

t0yv0 commented Jul 20, 2010

The reductions/joins are not completely parallel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment