Skip to content

Instantly share code, notes, and snippets.

View swayson's full-sized avatar

swayson

11 followers · 27 following
View GitHub Profile
@swayson
swayson / simple_example_zipfile.py
Created November 29, 2016 11:49
Simple example on how to use Python Zipfile to make a compressed archive. Here additional care is taken to provide an arcname, such that the root is avoided in the archive.
files = [path/to/file, path/to/file2, path/to/file3]
with zipfile.ZipFile('/tmp/test.zip', 'w', zipfile.ZIP_DEFLATED) as out_file:
for rel_filename in files:
absname = os.path.abspath(rel_filename)
root = os.path.dirname(absname)
filename = os.path.relpath(rel_filename, root)
out_file.write(rel_filename, filename)
print(absname, root, filename)
@swayson
swayson / rolling_block.py
Last active November 14, 2016 05:30
Rolling block Numpy using strided tricks and padding.
import numpy as np
from numpy.lib.stride_tricks import as_strided
def rolling_block(A, block=(3, 3)):
shape = (A.shape[0] - block[0] + 1, A.shape[1] - block[1] + 1) + block
strides = (A.strides[0], A.strides[1]) + A.strides
return as_strided(A, shape=shape, strides=strides)
X = np.random.randint(0, 200, (10, 10))
@swayson
swayson / niceness.txt
Created October 31, 2016 05:51
linux-niceness CPU priority
WHERE XX defines the priority. -19 being highest priority. 19 being the lowest.
sudo renice -n [XX] [PID]
nice -n [XX] [COMMAND]
@swayson
swayson / get_executing_script_dir.py
Created October 16, 2016 06:11
Snippet to construct the absolute path of the directory, in which the currently executing script is located.
import os
os.path.dirname(os.path.realpath(__file__))
@swayson
swayson / python-cmd-biolerplate-model.py
Created October 8, 2016 11:40
Simple boilerplate code, with command line interface, for general data-related tasks.
# -*- coding: utf-8 -*-
import os
import sys
import glob
import numpy as np
try:
import ujson as json
except ImportError:
import json
@swayson
swayson / python-cmd-boilerplate.py
Last active October 8, 2016 11:20
Simple boilerplate code to setup a Python program.
# -*- coding: utf-8 -*-
import os
import sys
import glob
try:
import ujson as json
except ImportError:
import ujson as json
import argparse
@swayson
swayson / logger.py
Created March 14, 2016 05:51
Python logging decorator and simple config.
import logging
def log_event(func):
"""
"""
def wrapper(*args, **kwargs):
# Code before function call
# <ADD IT HERE>
@swayson
swayson / datascience-project-flow.txt
Created February 27, 2016 17:39
Data Science Project Flow
# Phase 1 - Preparation:
Acquire data
Reformat and clean data
# Phase 2 - Analysis:
Edit analysis scripts
Execute analysis scripts
Inspect outputs
Debug
@swayson
swayson / gist:0ea4386a150cf5686a2a
Created February 27, 2016 17:39
Dat a
# Phase 1 - Preparation:
Acquire data
Reformat and clean data
# Phase 2 - Analysis:
Edit analysis scripts
Execute analysis scripts
Inspect outputs
Debug
@swayson
swayson / jupyter_notebook_ec2.sh
Created February 19, 2016 16:52 — forked from rashmibanthia/jupyter_notebook_ec2.sh
#!/usr/bin/env bash
#Code adapted from https://gist.github.com/yangj1e/3641843c758201ebbc6c (Modified to Python3.5)
cd ~
#wget https://3230d63b5fc54e62148e-c95ac804525aac4b6dba79b00b39d1d3.ssl.cf1.rackcdn.com/Anaconda2-2.4.0-Linux-x86_64.sh
wget https://3230d63b5fc54e62148e-c95ac804525aac4b6dba79b00b39d1d3.ssl.cf1.rackcdn.com/Anaconda3-2.4.1-Linux-x86_64.sh
bash Anaconda3-2.4.1-Linux-x86_64.sh -b
echo 'PATH="/home/ubuntu/anaconda3/bin:$PATH"' >> .bashrc
. .bashrc