Skip to content

Instantly share code, notes, and snippets.

@folone

folone/gist:5766269

Last active December 18, 2015 10:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save folone/5766269 to your computer and use it in GitHub Desktop.
Save folone/5766269 to your computer and use it in GitHub Desktop.
Download ZIP
Parallelizing lazy Streams.
Raw
gistfile1.scala
final val BATCH_SIZE = (Runtime.getRuntime.freeMemory / 1024).toInt // kb per object should be enough, right?
def doStuff(m: MyType): Stream[MyType] = ??? // Costly operation
val s: Stream[MyType] = ???
s.grouped(BATCH_SIZE).toStream.flatMap(_.par flatMap doStuff)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment