Skip to content

Instantly share code, notes, and snippets.

View LucasMMota's full-sized avatar
🤖
what's up human

Lucas Fonseca LucasMMota

🤖
what's up human
View GitHub Profile
@LucasMMota
LucasMMota / run_dbt_cloud_ci_job.yaml
Created September 27, 2023 21:40
Github Action pipeline to run CI job on dbt Cloud when PR is approved
name: Run CI job on dbt Cloud
# After PR is approved, validate the modifications in dbt Cloud
on:
pull_request_review: # triggers on PR approval
types:
- submitted
pull_request: # triggers on pull requests, so we make sure this won't be skipped
types:
- opened
@LucasMMota
LucasMMota / my_dag_to_test_sla.py
Created August 24, 2021 23:15
Example for testing SLA in Airflow
from datetime import datetime, timedelta
from airflow.models import DAG
from airflow.operators.bash_operator import BashOperator
dag = DAG(
dag_id="my_dag_to_test_sla",
catchup=False,
start_date=datetime(2021, 8, 28, 0, 0, 0),
# If schedule_interval=None, SLA won't be evaluated for this DAG!
schedule_interval="*/5 * * * *",
@LucasMMota
LucasMMota / my_dag_to_test_sla_with_sla_miss_callback.py
Last active August 24, 2021 23:19
Example for testing SLA in Airflow with a custom function (sla_miss_callback)
from datetime import datetime, timedelta
from airflow.models import DAG
from airflow.operators.bash_operator import BashOperator
def my_alerting_function(dag, task_list, blocking_task_list, slas, blocking_tis):
"""Executes the user defined code (for example, post on Slack)"""
message = (
f":alert: *Airflow SLA Miss*\n\n"
@LucasMMota
LucasMMota / extract of baseoperator.py
Last active August 24, 2021 22:34
SLA argument docstring in airflow.models.baseoperator.BaseOperator
"""
:param sla: time by which the job is expected to succeed. Note that
this represents the ``timedelta`` after the period is closed. For
example if you set an SLA of 1 hour, the scheduler would send an email
soon after 1:00AM on the ``2016-01-02`` if the ``2016-01-01`` instance
has not succeeded yet.
The scheduler pays special attention for jobs with an SLA and
sends alert
emails for sla misses. SLA misses are also recorded in the database
for future reference. All tasks that share the same SLA time