Skip to content

Instantly share code, notes, and snippets.

@sahib
Created December 17, 2012 13:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sahib/4318466 to your computer and use it in GitHub Desktop.
Save sahib/4318466 to your computer and use it in GitHub Desktop.
Pipeline Example with Python Coroutines
'''
Python coroutine Example for a Pipeline model.
--------- ------- -------
| Source | => send() => | Pipe | => send() => | Sink |
--------- ------- -------
close() shutdown the *whole* Pipeline.
'''
def sink():
'Coroutine: Recv the data and finalize it'
try:
while True:
item = (yield)
print('Sink:', item)
except GeneratorExit:
print('Exit.')
def pipe():
'Coroutine: Recv. the data and process it'
endpoint = sink()
endpoint.send(None)
try:
while True:
item = (yield)
# Do some processing
print('Process:', item)
endpoint.send(item * 10)
except GeneratorExit:
pass
# endpoint gets closed automagically?
def source():
'No Coroutine: Produce some data'
target = pipe()
target.send(None)
for item in range(10):
print('Send:', item)
target.send(item)
target.close()
if __name__ == '__main__':
s = source()
@cow-engineer
Copy link

LMAO I’ve been Googling trying to figure out how pipeline endpoints/sink close and your comment just about summed it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment