Skip to content

Instantly share code, notes, and snippets.

Ze Xuan Ong ongzexuan

View GitHub Profile
@ongzexuan
ongzexuan / lambda_transform_example.py
Last active Jun 17, 2019
Example of an AWS Lambda function to fetch data from S3, transform and then load to MySQL
View lambda_transform_example.py
import boto3
import csv
import json
import os
import pymysql
import sys
from os.path import join, dirname
# Load environment settings if exists
@ongzexuan
ongzexuan / csv_write.py
Created Jun 17, 2019
Example of writing to CSV file with quotations and line terminator
View csv_write.py
# Write to staging file
with open(TEMP_FILE, 'w') as f:
writer = csv.DictWriter(f, fieldnames=HEADERS),
quoting=csv.QUOTE_ALL,
lineterminator='\n')
writer.writeheader()
for data in staging_rows:
writer.writerow(data)
print('Written to {}'.format(TEMP_FILE))
@ongzexuan
ongzexuan / load_data_local_infile.sql
Created Jun 17, 2019
Example of LOAD DATA LOCAL INFILE SQL command
View load_data_local_infile.sql
LOAD DATA LOCAL INFILE '<FILENAME>'
REPLACE
INTO TABLE <TABLENAME>
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
You can’t perform that action at this time.