Skip to content

Instantly share code, notes, and snippets.

@polleyg

polleyg/beam_sql_part1.java

Last active January 24, 2019 03:00
Show Gist options
  • Save polleyg/e5f8b6f39e81a9b762aceecebaf64ef1 to your computer and use it in GitHub Desktop.
Save polleyg/e5f8b6f39e81a9b762aceecebaf64ef1 to your computer and use it in GitHub Desktop.
Download ZIP
beam_sql_part1
Raw
beam_sql_part1.java
[..]
PipelineOptionsFactory.register(DataflowPipelineOptions.class);
DataflowPipelineOptions options = PipelineOptionsFactory
.fromArgs(args)
.withValidation()
.as(DataflowPipelineOptions.class);
Pipeline pipeline = Pipeline.create(options);
pipeline.apply("read_from_gcs", TextIO.read().from("gs://batch-pipeline-sql/input/*"))
[..]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment