Skip to content

Instantly share code, notes, and snippets.

@folone
Last active December 18, 2015 10:09
Show Gist options
  • Save folone/5766269 to your computer and use it in GitHub Desktop.
Save folone/5766269 to your computer and use it in GitHub Desktop.
Parallelizing lazy Streams.
final val BATCH_SIZE = (Runtime.getRuntime.freeMemory / 1024).toInt // kb per object should be enough, right?
def doStuff(m: MyType): Stream[MyType] = ??? // Costly operation
val s: Stream[MyType] = ???
s.grouped(BATCH_SIZE).toStream.flatMap(_.par flatMap doStuff)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment