Skip to content

Instantly share code, notes, and snippets.

View ratchetwrench's full-sized avatar

David Wrench ratchetwrench

View GitHub Profile
@ratchetwrench
ratchetwrench / python_version.bash
Last active March 27, 2020 17:00
Know Which Version of Python You're Using
# To find out exactly which Python version you're using, you can use the --version flag
python --version
# Python 3 is usually available under the name python3
# python3 --version
# You can also figure out the version with the runtime configuration
# import sys
# print(sys.version_info)
# print(sys.version)
@ratchetwrench
ratchetwrench / _zipcode.py
Created October 25, 2017 23:01
Get geolocation information from a zipcode.
"""Get geolocation information from a zipcode.
https://pypi.python.org/pypi/zipcode
"""
import zipcode
def location_from_zipcode(zip=None):
return zipcode.isequal(zip).location_text
@ratchetwrench
ratchetwrench / parallel-web-scraping.py
Created August 15, 2017 23:42
speed up your python web scraper using multiprocessing
import requests
from bs4 import BeautifulSoup
from time import sleep
from multiprocessing import Pool
def get_listing(url):
headers = {
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'}
html = None
@ratchetwrench
ratchetwrench / images_request.py
Created August 15, 2017 23:23
download image using requests and shutil
"""You can either use the response.raw file object, or iterate over the response.
To use the response.raw file-like object will not, by default, decode compressed responses (with GZIP or deflate).
You can force it to decompress for you anyway by setting the decode_content attribute to True
(requests sets it to False to control decoding itself). You can then use shutil.copyfileobj()
to have Python stream the data to a file object:
"""
import requests
import shutil
@ratchetwrench
ratchetwrench / requests_api.py
Created August 15, 2017 20:40
Create a Python Request with API token authorization.
import requests
response = requests.get('https://website.com/id', headers={'Authorization': 'access_token myToken'})
@ratchetwrench
ratchetwrench / to_parquet.py
Created August 9, 2017 00:23
Convert CSV to Parquet file
import dask.dataframe as dd
import blaze
df = dd.read_csv('csv/yellow_tripdata_2015-*.csv') # '*' all files with the 'csv/yellow_tripdata_2015' prefix
df.to_parquet('yellow_tripdata.parquet')
@ratchetwrench
ratchetwrench / plot_cm.py
Created August 8, 2017 20:59
sklearn confusion matrix
import itertools
import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm, datasets
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix
# import some data to play with
iris = datasets.load_iris()
@ratchetwrench
ratchetwrench / print_cm.py
Created August 8, 2017 20:53 — forked from zachguo/print_cm.py
Pretty print for sklearn confusion matrix
from sklearn.metrics import confusion_matrix
def print_cm(cm, labels, hide_zeroes=False, hide_diagonal=False, hide_threshold=None):
"""pretty print for confusion matrixes"""
columnwidth = max([len(x) for x in labels]+[5]) # 5 is value length
empty_cell = " " * columnwidth
# Print header
print " " + empty_cell,
for label in labels:
print "%{0}s".format(columnwidth) % label,
@ratchetwrench
ratchetwrench / plot_tree.py
Last active August 8, 2017 01:54
Plot a CART Decision Tree from sklearn with pydotplus and graphviz
def plot_tree():
"""Make sure you have the correct imports.
Imports:
from sklearn import tree
import pydotplus
from IPython.display import Image
Note: Make sure you already have a 'tree'.
...
clf = tree.DecisionTreeClassifier()
@ratchetwrench
ratchetwrench / .bash_profile
Created July 21, 2017 01:29 — forked from natelandau/.bash_profile
Mac OSX Bash Profile
# ---------------------------------------------------------------------------
#
# Description: This file holds all my BASH configurations and aliases
#
# Sections:
# 1. Environment Configuration
# 2. Make Terminal Better (remapping defaults and adding functionality)
# 3. File and Folder Management
# 4. Searching
# 5. Process Management