Skip to content

Instantly share code, notes, and snippets.

@tuanchris
Last active May 23, 2020 04:06
Show Gist options
  • Save tuanchris/0f47551f52f97017e44c060367cf9d58 to your computer and use it in GitHub Desktop.
Save tuanchris/0f47551f52f97017e44c060367cf9d58 to your computer and use it in GitHub Desktop.
check_us_cities_demo = BigQueryCheckOperator(
task_id = 'check_us_cities_demo',
use_legacy_sql=False,
sql = f'SELECT count(*) FROM `{project_id}.{staging_dataset}.us_cities_demo`'
)
check_airports = BigQueryCheckOperator(
task_id = 'check_airports',
use_legacy_sql=False,
sql = f'SELECT count(*) FROM `{project_id}.{staging_dataset}.airport_codes`'
)
check_weather = BigQueryCheckOperator(
task_id = 'check_weather',
use_legacy_sql=False,
sql = f'SELECT count(*) FROM `{project_id}.{staging_dataset}.temperature_by_city`'
)
check_immigration_data = BigQueryCheckOperator(
task_id = 'check_immigration_data',
use_legacy_sql=False,
sql = f'SELECT count(*) FROM `{project_id}.{staging_dataset}.immigration_data`'
)
loaded_data_to_staging = DummyOperator(
task_id = 'loaded_data_to_staging'
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment