Skip to content

Instantly share code, notes, and snippets.

@dsaiztc
dsaiztc / boto3.py
Created April 27, 2017 09:01
Boto3 tricks.
def list_objects_s3(s3_client, s3_bucket, s3_prefix):
"""List objects in S3 overcoming the limit of 1000 per usual request."""
s3_paginator = s3_client.get_paginator('list_objects')
page_iterator = s3_paginator.paginate(Bucket=s3_bucket, Prefix=s3_prefix)
for page in page_iterator:
object_list = page.get('Contents')
for object in object_list:
yield object
@dsaiztc
dsaiztc / parse_dynamodb_dict.py
Created April 19, 2017 08:47
Parse DynamoDB element to python dict.
# -*- coding: utf-8 -*-
def parse_dynamodb_dict(dynamodb_dict):
def parse_value(value):
value_mapper = {
'S': unicode,
'N': float,
'B': str,
@dsaiztc
dsaiztc / logging.py
Last active July 2, 2017 13:13
Logging
import logging
logging.basicConfig()
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
import os
from datetime import datetime
log_file_path = os.path.join(os.path.dirname(os.path.realpath(__name__)), 'logs', '{}.log'.format(datetime.utcnow().strftime('%Y%m')))
@dsaiztc
dsaiztc / update_dict.py
Created April 11, 2017 16:25
Update dict
a.update(b)
c = dict(a, **b)
# Items from b will override the ones from a if conflict
@dsaiztc
dsaiztc / cache_function.py
Last active June 21, 2017 16:17
Pure function result cache decorator.
from functools import wraps
def cache(func):
saved = {}
@wraps(func)
def newfunc(*args):
if args in saved:
return saved[args]
result = func(*args)
saved[args] = result
@dsaiztc
dsaiztc / GettingStarted.md
Last active March 17, 2017 13:33
Getting started

Install node.js from Node.js download page. npm comes installed with node.js but it gets updates more often. To get such updates:

npm install npm -g
npm ls
npm outdated

Cross platform interface for virtualenv

Unless you use some Windows specific libraries; or an alternate Python implementation (like IronPython), there is nothing to worry about.

Many people (including myself) use Windows for development and deploy on Linux for production and use virtualenv for this purpose. It is designed to make your environment portable.

You don't push the entire virtualenv to Linux.

Once you have your virtual environment ready and your code is working, you should freeze the requirements for your application:

@dsaiztc
dsaiztc / file_directory.py
Created March 2, 2017 15:44
File directory
import os
os.path.dirname(os.path.abspath(__file__))
@dsaiztc
dsaiztc / loading.css
Created February 2, 2017 17:29
Spinner icon loading.
/* glyphicon spinning http://stackoverflow.com/a/26283602/3149679 */
.glyphicon.spinning {
animation: spin 1s infinite linear;
-webkit-animation: spin2 1s infinite linear;
}
@keyframes spin {
from { transform: scale(1) rotate(0deg); }
to { transform: scale(1) rotate(360deg); }
}
@-webkit-keyframes spin2 {
@dsaiztc
dsaiztc / lm.r
Last active January 22, 2017 13:40
m1 <- lm(I(log(price)) ~ I(carat^(1/3)), data = diamonds)
m2 <- update(m1, ~ . + carat)
m3 <- update(m2, ~ . + cut)
m4 <- update(m3, ~ . + color)
m5 <- update(m4, ~ . + clarity)
mtable(m1, m2, m3, m4, m5)
diamondsbig$logprice <- log(diamondsbig$price)
m1 <- lm(logprice ~ I(carat^(1/3)), diamondsbig[diamondsbig$price < 10000 & diamondsbig$cert == "GIA", ])