Skip to content

Instantly share code, notes, and snippets.

@tuanchris
Created May 23, 2020 04:22
Show Gist options
  • Save tuanchris/437294fd9b58e01c3843982a33542d08 to your computer and use it in GitHub Desktop.
Save tuanchris/437294fd9b58e01c3843982a33542d08 to your computer and use it in GitHub Desktop.
# Define task dependencies
dag >> start_pipeline >> [load_us_cities_demo, load_airports, load_weather, load_immigration_data]
load_us_cities_demo >> check_us_cities_demo
load_airports >> check_airports
load_weather >> check_weather
load_immigration_data >> check_immigration_data
[check_us_cities_demo, check_airports, check_weather,check_immigration_data] >> loaded_data_to_staging
loaded_data_to_staging >> [load_country, load_port, load_state] >> create_immigration_data >> check_f_immigration_data
check_f_immigration_data >> [create_d_time, create_d_weather, create_d_airport, create_d_city_demo] >> finish_pipeline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment