Skip to content

Instantly share code, notes, and snippets.

@naturalett
Last active June 5, 2023 21:56
Show Gist options
  • Save naturalett/a5e733685df0a5e4948e81ab3158e1bd to your computer and use it in GitHub Desktop.
Save naturalett/a5e733685df0a5e4948e81ab3158e1bd to your computer and use it in GitHub Desktop.
Airflow running bash tasks
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.bash import BashOperator
NUM_TASKS = 500
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'email': ['admin@admin.com'],
'email_on_failure': True,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=5),
}
with DAG(
'bash_tasks',
default_args=default_args,
start_date=datetime(2021, 1, 1),
schedule_interval=None,
catchup=False
) as dag:
for i in range(0, NUM_TASKS):
BashOperator(
task_id=f'task_{i}',
bash_command='date'
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment