Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save justinnaldzin/90b4d0f793e6ca88f69fd9b55eaef6fd to your computer and use it in GitHub Desktop.
Save justinnaldzin/90b4d0f793e6ca88f69fd9b55eaef6fd to your computer and use it in GitHub Desktop.
Google Cloud Composer - Manually trigger DAG runs using Airflow v1.10+
# Google Cloud Composer - Manually trigger DAG runs using Airflow v1.10+
ENVIRONMENT_NAME=my-composer
LOCATION=us-east1
# Trigger DAG - individual
DAG_ID=my_daily_dag
EXEC_DATE=2019-02-11
gcloud composer environments run ${ENVIRONMENT_NAME} --location ${LOCATION} trigger_dag -- -r manual__${EXEC_DATE} -e ${EXEC_DATE} ${DAG_ID}
# Trigger DAG - multiple
DAG_ID=my_daily_dag
DATES=(
2019-02-11
2019-02-10
2019-02-09
...
2019-01-01
)
for EXEC_DATE in ${DATES[*]}
do
echo "Creating DAG Run for: ${EXEC_DATE}"
gcloud composer environments run ${ENVIRONMENT_NAME} --location ${LOCATION} trigger_dag -- -r manual__${EXEC_DATE} -e ${EXEC_DATE} ${DAG_ID}
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment