Skip to content

Instantly share code, notes, and snippets.

@movermeyer

movermeyer/stream_test.py

Last active February 16, 2016 21:56
Show Gist options
  • Save movermeyer/a2638b28aab1131c1e89 to your computer and use it in GitHub Desktop.
Save movermeyer/a2638b28aab1131c1e89 to your computer and use it in GitHub Desktop.
Download ZIP
stream processing error
Raw
stream_test.py
from stream import map, ForkedFeeder, ProcessPool, PCollector
#Collector works with ForkedFeeder
collector = PCollector()
for i in range(10):
ForkedFeeder(lambda: (i for i in xrange(10))) >> collector
print len([x for x in collector])
#Collector does not work with ProcessPool
collector = PCollector()
for i in range(10):
ForkedFeeder(lambda: (i for i in xrange(10))) >> ProcessPool(map(lambda x: x*x)) >> collector
print len([x for x in collector])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment