Skip to content

Instantly share code, notes, and snippets.

View ongzexuan's full-sized avatar

Ze Xuan Ong ongzexuan

8 followers · 4 following
View GitHub Profile
@ongzexuan
ongzexuan / airflow-scheduler.service
Last active November 12, 2020 05:46
Systemd script for Airflow scheduler
[Unit]
Description=Airflow scheduler daemon
After=network.target
Wants=postgresql.target
[Service]
Environment="PATH=/home/airflow/.local/bin"
User=airflow
Group=airflow
Type=simple
@ongzexuan
ongzexuan / airflow-webserver.service
Last active November 12, 2020 05:46
Systemd script for Airflow webserver
[Unit]
Description=Airflow webserver daemon
After=network.target
Wants=postgresql.target
[Service]
Environment="PATH=/home/airflow/.local/bin"
User=airflow
Group=airflow
Type=simple
@ongzexuan
ongzexuan / darkgist.css
Created August 17, 2020 05:55
Custom CSS for Github gists for the Casper 3 theme on Hugo
@media (prefers-color-scheme: dark) {
body .gist .highlight {
background: #272822;
}
body .gist .blob-num,
body .gist .blob-code-inner,
body .gist .pl-s2,
body .gist .pl-stj {
color: #f8f8f2;
}
@ongzexuan
ongzexuan / lambda_transform_example.py
Last active November 8, 2022 16:39
Example of an AWS Lambda function to fetch data from S3, transform and then load to MySQL
import boto3
import csv
import json
import os
import pymysql
import sys
from os.path import join, dirname
# Load environment settings if exists
@ongzexuan
ongzexuan / csv_write.py
Created June 17, 2019 16:10
Example of writing to CSV file with quotations and line terminator
# Write to staging file
with open(TEMP_FILE, 'w') as f:
writer = csv.DictWriter(f, fieldnames=HEADERS),
quoting=csv.QUOTE_ALL,
lineterminator='\n')
writer.writeheader()
for data in staging_rows:
writer.writerow(data)
print('Written to {}'.format(TEMP_FILE))
@ongzexuan
ongzexuan / load_data_local_infile.sql
Created June 17, 2019 15:59
Example of LOAD DATA LOCAL INFILE SQL command
LOAD DATA LOCAL INFILE '<FILENAME>'
REPLACE
INTO TABLE <TABLENAME>
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;