Skip to content

Instantly share code, notes, and snippets.

@alexvanboxel

alexvanboxel/medium_agc1_bq2.py

Last active January 19, 2017 18:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alexvanboxel/c011f42f734d69560aa5adf6df7b6430 to your computer and use it in GitHub Desktop.
Save alexvanboxel/c011f42f734d69560aa5adf6df7b6430 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
medium_agc1_bq2.py
bq_to_gcp_avro = BigQueryToCloudStorageOperator(
task_id='bq_to_gcp_avro',
source_project_dataset_table='{{var.value.gcq_dataset}}.gsod_partition{{ ds_nodash }}',
destination_cloud_storage_uris=[
'gs://{{var.value.gcs_bucket}}/path/{{ ds_nodash }}/part-*.avro'
],
export_format='AVRO',
bigquery_conn_id='gcp_smoke',
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment