Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save rjerrems/356dfa407802596417b16fc25154c826 to your computer and use it in GitHub Desktop.
Save rjerrems/356dfa407802596417b16fc25154c826 to your computer and use it in GitHub Desktop.
--gcs-location: gs://<INSERT_PATH_TO_TEMPLATE>
--subnetwork: https://www.googleapis.com/compute/v1/projects/<INSERT_PROJECT_HERE>/regions/<INSERT_REGION>/subnetworks/<INSERT_VPC_NAME>
--parameters:
inputLocations: >-
{"location1":"Path to file in GCS or BQ table name"}
outputLocations: >-
{"location1":"Path to file in GCS or BQ table name","location2":"gs://<INSERT_GCS_PATH_HERE>/.profiler/profilerTypeCheckHistograms.json/file","location3":"gs:///<INSERT_GCS_PATH_HERE>/.profiler/profilerValidValueHistograms.json/file"}
@rjerrems
Copy link
Author

Using this flags file you can run a Dataflow job which has been generated by Dataprep using the following gcloud command:

gcloud beta dataflow jobs run glcoud-run --flags-file flags.yml --verbosity debug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment