Skip to content

Instantly share code, notes, and snippets.

@nicolas-modsy
nicolas-modsy / print_add_partition_query.py
Created July 10, 2019 17:37
Generate Athena / Hive query to add partitions
# All partitions in time range are added in one query
from datetime import timedelta, date
def daterange(start_date, end_date):
for n in range(int ((end_date - start_date).days)):
yield start_date + timedelta(n)
start_date = date(2019, 1, 1)
end_date = date(2019, 7, 1)

Keybase proof

I hereby claim:

  • I am nicolas-modsy on github.
  • I am nicolasmodsy (https://keybase.io/nicolasmodsy) on keybase.
  • I have a public key whose fingerprint is AFDB C8CF 380D FDEE 6078 F847 1AAA 466F 496F C836

To claim this, I am signing this object:

@nicolas-modsy
nicolas-modsy / pull_images_from_spreadsheet.py
Created May 16, 2018 20:21
The marketing team wanted to download some images... EP-4246
import csv
import urllib
# urllib.urlretrieve("http://www.gunnerkrigg.com//comics/00000001.jpg", "00000001.jpg")
# export as CSV from Google Sheets
# the header will be commented out, remove the '#' manually
dl_dir = '/Users/nicolasrichard/Desktop/ImagePull1/'
modsy_prod_bucket_url = 'https://s3.us-west-2.amazonaws.com/modsy/'