kjaanson

/*.egg-info
/.spyderproject
.spyderproject
.idea/*
/**/__pycache__
/**/.ipynb_checkpoints
/.doit.db.bak
/.doit.db.dat
/.doit.db.dir

kjaanson

kjaanson

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname

kjaanson

import glob
import os
from os.path import join
import nbformat as nbf


def generate_makehtmlscript_tasks(dir):
    """
    Builds html and scripts from deliver notebooks. Does not execute notebooks.

kjaanson

Run notebook with specified log level

jupyter nbconvert --Application.log_level=10

Running a notebook

jupyter nbconvert --to notebook --execute mynotebook.ipynb

kjaanson

#!/bin/sh
#
# Pre-commit hook for clearing output cells from commited analysis Jupyter notebooks.
#

echo "Running pre-commit hook to clear output from deliver/*.ipynb notebooks."
for notebook in git diff --cached --name-only -- 'deliver/*.ipynb'
do
	echo "Clearing output from $notebook"
	jupyter nbconvert --ClearOutputPreprocessor.enabled=True --ClearOutputPreprocessor.remove_metadata_fields=[] --to notebook --inplace $notebook

kjaanson

import os
import struct
import numpy as np

"""
Loosely inspired by http://abel.ee.ucla.edu/cvxopt/_downloads/mnist.py
which is GPL licensed.
"""

def read(dataset = "training", path = "."):

kjaanson

How-to setup a simple git push deployment

These are my notes basically. At first i created this gist just as a reminder for myself. But feel free to use this for your project as a starting point. If you have questions you can find me on twitter @thomasf https://twitter.com/thomasf This is how i used it on a Debian Wheezy testing (https://www.debian.org/releases/testing/)

Discuss, ask questions, etc. here https://news.ycombinator.com/item?id=7445545

On the server (example.com)

kjaanson

import time

def timeit(value_dict=None, print_values=True):
    """Decorator for timing functions with different parameters.

    Arguments:
        value_dict - dict where to save the timings, uses function args tupple as key
        print_values - boolean for printing timings to std out

    Usage:

kjaanson

"""
Monkeypatching Pandas Dataframe to include `to_bcp_sql` method
that uploads dataframe to mssql server using mssql-tools `bcp`.

Requires mssql-tools to be installed on system and in PATH.

Partly inspired by https://gist.github.com/ajsmith007/d1adb79e152f5f23503b

ToDo:
  * Add docs