Last active
October 23, 2018 11:34
-
-
Save dukehoops-zz/dae9c45c2035d50e41fee7c7d75a50dd to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def next_task_id(): | |
| return 'branch_upload' # branch_skip_upload or branch_upload | |
| decide_upload = BranchPythonOperator(python_callable=next_task_id, task_id='decide_upload', dag=main_dag) | |
| parent >> decide_upload | |
| branch_skip_upload = DummyOperator(task_id='branch_skip_upload', dag=main_dag) | |
| branch_upload = DummyOperator(task_id='branch_upload', dag=main_dag) | |
| decide_upload >> [branch_skip_upload, branch_upload] | |
| one = DummyOperator(task_id='one', dag=main_dag) | |
| two = DummyOperator(task_id='two', dag=main_dag) | |
| branch_upload_finish = DummyOperator(task_id='branch_upload_finish', dag=main_dag) | |
| branch_upload >> [one, two] >> branch_upload_finish | |
| join = DummyOperator(task_id='join', dag=main_dag, trigger_rule='one_success') | |
| finish = DummyOperator(task_id='finish', dag=main_dag) | |
| [branch_skip_upload, branch_upload_finish] >> join >> finish |
Author
the key is to set
join.trigger_ruleto something other than the defaultall_success, since at least one parent (upstream) task will be skipped
Thanks! It helped me a lot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
the key is to set
join.trigger_ruleto something other than the defaultall_success, since at least one parent (upstream) task will be skipped