ParStream.sortBy performance test

gistfile1.fs

#time
#r "bin/Release/Streams.Core.dll"

open Nessos.Streams.Core
let rnd = new System.Random()
let data = [|1..10000000|] |> Array.map (fun _ -> int64 <| rnd.Next(1000000))

#r "../../packages/FSharp.Collections.ParallelSeq.1.0/lib/net40/FSharp.Collections.ParallelSeq.dll"
open FSharp.Collections.ParallelSeq

// Real: 00:00:03.095, CPU: 00:00:14.679, GC gen0: 3, gen1: 2, gen2: 2
data
|> PSeq.map (fun x -> x + 1L)
|> PSeq.sortBy id
|> PSeq.toArray

// Real: 00:00:00.917, CPU: 00:00:04.321, GC gen0: 1, gen1: 0, gen2: 0
data
|> ParStream.ofArray
|> ParStream.map (fun x -> x + 1L)
|> ParStream.sortBy id

I'll try to elaborate a little bit more. In Streams the map function is implemented like
let inline map (f : 'T -> 'R) (stream : Stream<'T>) : Stream<'R> =
let (Stream streamf) = stream
Stream (fun iterf -> streamf (fun value -> iterf (f value)))
It easy to see that the essence of Streams is the "CPS composition of lazy operations".
It is more lightweight because you pay only one virtual call per element per operation.
Check this https://github.com/nessos/Streams/blob/master/src/Streams.Core/Streams.fs
for more operations. Maybe another way to understand it is that in streams we compose lazy operations for internal iteration and in LINQ/IEnumerable/IEnumerator is based on external iteration.

Yes "optimized" in cotl is about ScalaBlitz and LinqOptimizer.

Ah, I see. Thanks for the detailed explanations!

Btw, it might be helpful to add that explanation to the readme. First time I discovered your Streams API and read "inspired by Java 8 Streams" I didn't understand what makes Java 8 Streams special since it I thought that Java 8 Streams were just the Java version of Seq/IEnumerable+LINQ.

Great suggestion, thnx for the feedback!