Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
import logging
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions, SetupOptions
_log = logging.getLogger(__name__)
def main(argv=None):
pipeline_options = PipelineOptions(argv)
setup_options = pipeline_options.view_as(SetupOptions) # type: SetupOptions
setup_options.save_main_session = True"Running pipeline")
with beam.Pipeline(runner="DirectRunner", options=pipeline_options) as p:
p | beam.Create(["hello", "world"]) | beam.Map(lambda x: print(x))
Call to will now deadlock, since the logging handler's
threading.RLock() has been passed through dill.
When you press Ctrl-C, the traceback should confirm that the process is
stuck at:
File "/usr/lib/python3.5/logging/", line 810, in acquire
""")"Pipeline done")
print("Launching nukes")
if __name__ == '__main__':
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment