Skip to content

Instantly share code, notes, and snippets.

View NicholasTurner23's full-sized avatar
💭
I may be slow to respond.

Nicholas Bob NicholasTurner23

💭
I may be slow to respond.
  • kampala, Uganda
View GitHub Profile
@NicholasTurner23
NicholasTurner23 / bulk-insert.py
Created September 7, 2020 07:51 — forked from ellisvalentiner/bulk-insert.py
Recipe for (fast) bulk insert from python Pandas DataFrame to Postgres database
#!/usr/bin/env/python
import psycopg2
import os
from io import StringIO
import pandas as pd
# Get a database connection
dsn = os.environ.get('DB_DSN') # Use ENV vars: keep it secret, keep it safe
conn = psycopg2.connect(dsn)
from datetime import timedelta
# The DAG object; we'll need this to instantiate a DAG
from airflow import DAG
# Operators; we need this to operate!
from airflow.operators.bash_operator import BashOperator
from airflow.utils.dates import days_ago