martinapugliese

# Imports
import pandas as pd
import numpy as np
from scipy.stats import entropy

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

from matplotlib import pyplot as plt

martinapugliese

Collection of sample Elasticsearch queries

Use the Python client elasticsearch.

Connect to cluster (the client)

from elasticsearch import Elasticsearch

es_client = Elasticsearch() # local

martinapugliese

# Copyright (C) 2016 Martina Pugliese

from boto3 import resource
from boto3.dynamodb.conditions import Key

# The boto3 dynamoDB resource
dynamodb_resource = resource('dynamodb')


def get_table_metadata(table_name):

martinapugliese

# Copyright (C) 2016 Martina Pugliese

# Imports
from datetime import datetime


# #################### ANSI Escape codes for terminal #########################

codes_dict = {

martinapugliese

# Copyright (C) 2016 Martina Pugliese


def run_methods():

    print '\n'

    print '* Count occurrences of substring in string'
    print 'Martina'.count('art')
    print 'Martina'.count('a')

martinapugliese

A collection of useful command line hacks (Unix)

Memory usage

MACOS

vm_stat is the command, this makes output user friendly, thanks to this.

vm_stat | perl -ne '/page size of (\d+)/ and $size=$1; /Pages\s+([^:]+)[^\d]+(\d+)/ and printf("%-16s % 16.2f Mi\n", "$1:", $2 * $size / 1048576);'

martinapugliese

A collection of little libraries that help workflow

TQDM

Shows progress bar in a notebook's cell.

for i in tqdm(range(10), 'wasting time', unit='iterations wasted'):
    sleep(0.5)

martinapugliese

Pyplot reference stuff

Those things that I always forget how to do.

import pyplot as plt

Matplotlib styles

martinapugliese

Pandas reference things

df is a DataFrame.

Grouping df on multiple functions and dropping hierarchical level

grouped_df = df.groupby(['colA', 'colB']) \
    .agg(
 {

martinapugliese

A reference on small Python stuffs

Various lil' things

• See Difference in for key in d.keys() and for key in d
• On underscores in Python
• Assigning more vars to same value
• Various ways, different in speed and in preserving the order, to uniquify a list: here and a followup with another suggestion (f7) here
• Various ways, with time benchmarks, to check if key is in dict, here