Skip to content

Instantly share code, notes, and snippets.

@krishnanraman
Created July 27, 2014 05:05
Show Gist options
  • Save krishnanraman/e8289eac92d28856e089 to your computer and use it in GitHub Desktop.
Save krishnanraman/e8289eac92d28856e089 to your computer and use it in GitHub Desktop.
sets in typed vs fields
scala> TypedPipe.from((0 until 10).toList).groupAll.size.dump
14/07/26 21:58:42 INFO flow.Flow: [ScaldingShell] starting
14/07/26 21:58:42 INFO flow.Flow: [ScaldingShell] source: MemoryTap["NullScheme"]["0.6803924901219981"]
14/07/26 21:58:42 INFO flow.Flow: [ScaldingShell] sink: MemoryTap["NullScheme"]["0.3895948780547176"]
14/07/26 21:58:42 INFO flow.Flow: [ScaldingShell] parallel execution is enabled: true
14/07/26 21:58:42 INFO flow.Flow: [ScaldingShell] starting jobs: 1
14/07/26 21:58:42 INFO flow.Flow: [ScaldingShell] allocating threads: 1
14/07/26 21:58:42 INFO flow.FlowStep: [ScaldingShell] starting step: local
((),10)
scala> TypedPipe.from((0 until 10).toSet).groupAll.size.dump
14/07/26 21:58:48 INFO flow.Flow: [ScaldingShell] starting
14/07/26 21:58:48 INFO flow.Flow: [ScaldingShell] source: MemoryTap["NullScheme"]["0.2681900590184605"]
14/07/26 21:58:48 INFO flow.Flow: [ScaldingShell] sink: MemoryTap["NullScheme"]["0.6702092691249959"]
14/07/26 21:58:48 INFO flow.Flow: [ScaldingShell] parallel execution is enabled: true
14/07/26 21:58:48 INFO flow.Flow: [ScaldingShell] starting jobs: 1
14/07/26 21:58:48 INFO flow.Flow: [ScaldingShell] allocating threads: 1
14/07/26 21:58:48 INFO flow.FlowStep: [ScaldingShell] starting step: local
((),1)
scala> IterableSource((0 until 10).toSet, 'a).read.toTypedPipe[Int]('a).groupAll.size.dump
14/07/26 22:03:58 INFO flow.Flow: [ScaldingShell] starting
14/07/26 22:03:58 INFO flow.Flow: [ScaldingShell] source: MemoryTap["NullScheme"]["0.4200594268446092"]
14/07/26 22:03:58 INFO flow.Flow: [ScaldingShell] sink: MemoryTap["NullScheme"]["0.8554432259895711"]
14/07/26 22:03:58 INFO flow.Flow: [ScaldingShell] parallel execution is enabled: true
14/07/26 22:03:58 INFO flow.Flow: [ScaldingShell] starting jobs: 1
14/07/26 22:03:58 INFO flow.Flow: [ScaldingShell] allocating threads: 1
14/07/26 22:03:58 INFO flow.FlowStep: [ScaldingShell] starting step: local
((),10)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment