Skip to content

Instantly share code, notes, and snippets.

@feczo

feczo/run_StreamingLogExtract.sh

Last active October 5, 2016 14:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save feczo/c6aee14b35ed19cbabb9c2a7a9f51248 to your computer and use it in GitHub Desktop.
Save feczo/c6aee14b35ed19cbabb9c2a7a9f51248 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
run_StreamingLogExtract.sh
# export PROJECT=my-project-x
# export STAGE=gs://my-bucket-x/ds
mvn compile exec:java \
-Dexec.mainClass=com.google.cloud.dataflow.examples.complete.StreamingLogExtract \
-Dexec.args="--project=$PROJECT \
--stagingLocation=$STAGE \
--pubsubTopic='projects/""$PROJECT""/topics/custom_logs' \
--bigQueryDataset='dataflow_demo' \
--bigQueryTable='custom_logs' \
--runner=DataflowPipelineRunner --workerMachineType='n1-standard-1' --streaming=true"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment