Skip to content

Instantly share code, notes, and snippets.

@zkan
Forked from kaxil/airflow_json_variables.py
Created April 30, 2021 05:42
Show Gist options
  • Save zkan/9350472d6ddfbbb4db2eb44489998f81 to your computer and use it in GitHub Desktop.
Save zkan/9350472d6ddfbbb4db2eb44489998f81 to your computer and use it in GitHub Desktop.
Using Airflow Json Variables
from airflow.models import Variable
# Common (Not-so-nice way)
# 3 DB connections when the file is parsed
var1 = Variable.get("var1")
var2 = Variable.get("var2")
var3 = Variable.get("var3")
# Recommended Way
# Just 1 Database call
dag_config = Variable.get("dag1_config", deserialize_json=True)
dag_config["var1"]
dag_config["var2"]
dag_config["var3"]
# You can directly use it Templated arguments {{ var.json.my_var.path }}
bash_task = BashOperator(
task_id="bash_task",
bash_command='{{ var.json.dag1_config.var1 }} ',
dag=dag,
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment