dylanbstorey

use git2::{Cred, RemoteCallbacks, Repository};

use git2;
use git2_credentials::CredentialHandler;
use tempfile;
use std::path::Path;

/// works after running ╰─❯ ssh-keygen -t ed25519 -C "dylan.storey@gmail.com.com"       
/// and uploading to github/gitlab

dylanbstorey

 breeze start-airflow --verbose                                                                                                                                                                                                                                                                                                   ─╯
The output of the asset compilation is stored in: /home/dstorey/airflow/.build/www/asset_compile.out


Running command: <docker info>


Working directory /home/dstorey/airflow

dylanbstorey

Pattern is useful when you want to trigger many copies of the same DAG with different parameters.

Driver DAG

This dag is run on a schedule, fetches the appropriate configurations needed and then triggers the external DAG as a configuration. There is a final "wait" step that is optional that simply ensures there is a blocking step at the end of the submissions to ensure that concurrency is kept at "1" for the parameterized DAG. This pattern uses the dynamic task mapping paradigm to determine how many external dags should be triggered during execution. As a general note, the generate_list task could be replaced with any operator even ones that use external service calls to determine if a specific configuration should be run at this execution cycle.

dylanbstorey

import os
import fnmatch
import glob
import shutil

package = os.path.basename(os.path.dirname(__file__))

def task_docs():
    """
    Build our documentation

dylanbstorey

"""

Adapted from Camille Scott's dammit logger :
https://github.com/camillescott/dammit/blob/master/dammit/log.py

and 

Robpol86 
https://github.com/Robpol86/Flask-Large-Application-Example/blob/master/manage.py#L81

dylanbstorey

from dateutil.parser import parse
from skyscout.db_models import *
from sqlalchemy.exc import IntegrityError
import logging

logger = logging.getLogger(__name__)


def drop_all():
    db.drop_all()

dylanbstorey

import pickle
import os
from functools import partial

def function(text , a = 3):
    print('this is a function with text {0}  and a = {1}'.format(text , a))

class Test(object):
    """  """
    

dylanbstorey

Dummy File

dylanbstorey

dylanbstorey

def is_outlier(list, threshold=3.5):
    """
    Returns a boolean array with True if points are outliers and False
    otherwise.

    Parameters:
    -----------
        points : An numobservations by numdimensions array of observations
        thresh : The modified z-score to use as a threshold. Observations with
            a modified z-score (based on the median absolute deviation) greater