martinapugliese

# Imports
import pandas as pd
import numpy as np
from scipy.stats import entropy

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

from matplotlib import pyplot as plt

martinapugliese

A collection of little libraries that help workflow

TQDM

Shows progress bar in a notebook's cell.

for i in tqdm(range(10), 'wasting time', unit='iterations wasted'):
    sleep(0.5)

martinapugliese

Pyplot reference stuff

Those things that I always forget how to do.

import pyplot as plt

Matplotlib styles

martinapugliese

A collection of useful command line hacks (Unix)

Memory usage

MACOS

vm_stat is the command, this makes output user friendly, thanks to this.

vm_stat | perl -ne '/page size of (\d+)/ and $size=$1; /Pages\s+([^:]+)[^\d]+(\d+)/ and printf("%-16s % 16.2f Mi\n", "$1:", $2 * $size / 1048576);'

martinapugliese

Pandas reference things

df is a DataFrame.

Grouping df on multiple functions and dropping hierarchical level

grouped_df = df.groupby(['colA', 'colB']) \
    .agg(
 {

martinapugliese

A reference on small Python stuffs

Various lil' things

• See Difference in for key in d.keys() and for key in d
• On underscores in Python
• Assigning more vars to same value
• Various ways, different in speed and in preserving the order, to uniquify a list: here and a followup with another suggestion (f7) here
• Various ways, with time benchmarks, to check if key is in dict, here

martinapugliese

A reference on git things

• When there's no master branch, will be created when you first create a file
• When .gitignore does not skip files

martinapugliese

# Copyright (C) 2016 Martina Pugliese


def run_methods():

    print '\n'

    print '* Count occurrences of substring in string'
    print 'Martina'.count('art')
    print 'Martina'.count('a')

martinapugliese

# Copyright (C) 2016 Martina Pugliese


def plot_freqdist_freq(fd,
                       max_num=None,
                       cumulative=False,
                       title='Frequency plot',
                       linewidth=2):
    """
    As of NLTK version 3.2.1, FreqDist.plot() plots the counts and has no kwarg for normalising to frequency. Work this around here.

martinapugliese

# Copyright (C) 2016 Martina Pugliese

# Imports
from datetime import datetime


# #################### ANSI Escape codes for terminal #########################

codes_dict = {