Skip to content

Instantly share code, notes, and snippets.

@KarthikRajashekaran
KarthikRajashekaran / s3_multipart_upload.py
Created April 23, 2019 17:48 — forked from octopy/s3_multipart_upload.py
Example of Parallelized Multipart upload using boto
#!/usr/bin/env python
"""Split large file into multiple pieces for upload to S3.
S3 only supports 5Gb files for uploading directly, so for larger CloudBioLinux
box images we need to use boto's multipart file support.
This parallelizes the task over available cores using multiprocessing.
Usage:
s3_multipart_upload.py <file_to_transfer> <bucket_name> [<s3_key_name>]
@KarthikRajashekaran
KarthikRajashekaran / FileSplitter.py
Created April 23, 2019 18:48 — forked from msharp/FileSplitter.py
python script to split a (large) file into multiple (smaller) files with specified number of lines
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys
import os
class FileSplitter:
def __init__(self):
self.parse_args(sys.argv)
'''
Sorting the list of dates
'''
from datetime import datetime
dt = ['01 Apr 2017', '01 Apr 2018', '01 Aug 2017', '01 Aug 2018', '01 Dec 2017',
'01 Dec 2018', '01 Feb 2017', '01 Feb 2018', '01 Jan 2017', '01 Jan 2018']
print(sorted(dt, key=lambda x: datetime.strptime(x, '%d %b %Y')))