Skip to content

Instantly share code, notes, and snippets.

@naturalett

naturalett/airflow_cfg.csv

Created November 21, 2019 15:00
Show Gist options
  • Save naturalett/776301419aaad2913a683691f994f937 to your computer and use it in GitHub Desktop.
Save naturalett/776301419aaad2913a683691f994f937 to your computer and use it in GitHub Desktop.
Download ZIP
Airflow Configuration
Raw
airflow_cfg.csv
Parameter Explanation
core.parallelism Maximum number of tasks running across an entire Airflow installation
core.dag_concurrency The number of task instances allowed to run concurrently by the scheduler
core.non_pooled_task_slot_count Number of task slots allocated to tasks not running in a pool
core.max_active_runs_per_dag Maximum number of active DAG runs - per DAG
scheduler.max_threads How many threads the scheduler process should use to use to schedule DAGs
celery.worker_concurrency Number of task instances that a worker will take if using CeleryExecutor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment