Skip to content

Instantly share code, notes, and snippets.

@chandulal
Created September 8, 2019 06:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chandulal/e74fa05a8e5c47016ced6133251b7ca8 to your computer and use it in GitHub Desktop.
Save chandulal/e74fa05a8e5c47016ced6133251b7ca8 to your computer and use it in GitHub Desktop.
from airflow.operators.presto_to_mysql import PrestoToMySqlTransfer
from datetime import datetime
from airflow import DAG
default_args = {
'email': ['hello@world.com']
}
dag = DAG('presto_to_mysql', description='Presto to Mysql Transfer', default_args=default_args,
schedule_interval='0 12 * * *',
start_date=datetime(2017, 3, 20), catchup=False)
PrestoToMySqlTransfer(
presto_conn_id='presto-conn',
mysql_conn_id='mysql-conn',
task_id='presto_to_mysql_transfer',
sql="""
SELECT name, count(*) as count
FROM blackhole.default.region
GROUP BY name
""",
mysql_table='mysql_region',
mysql_preoperator='TRUNCATE TABLE mysql_region;',
dag=dag)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment