Skip to content

Instantly share code, notes, and snippets.

View wkarney's full-sized avatar

Will Karnasiewicz wkarney

View GitHub Profile
@wkarney
wkarney / simple_pickling_snippet.py
Last active February 24, 2020 21:05
[Pickling in Python] #pickle #python #basics #deployment
# Python pickling example:
import pickle
with open('pickledfile.pickle', 'wb') as f:
pickle.dump(model, f, pickle.HIGHEST_PROTOCOL)
with open('pickledfile.pickle', 'rb') as f:
model = pickle.load(f)
@wkarney
wkarney / va_dept_api_intro.py
Last active February 24, 2020 21:05
[VA Facilities API Intro] Some examples for US VA Dept Facilities API #Python #API
# Import requests package
import requests
# API Key (request one at https://developer.va.gov/apply)
apikey = 'your-api-key-here'
# Typical request
url = 'https://dev-api.va.gov/services/va_facilities/v0/facilities/'
response = requests.get(
@wkarney
wkarney / infinite_scroll_scraping_selenium.py
Last active May 5, 2023 17:04
[Infinite Scroll Pages] Scraping infinite scroll pages with selenium #selenium #webscraping #python
from time import sleep
from selenium import webdriver
from bs4 import BeautifulSoup
# Headless/incognito Chrome driver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--incognito")
chrome_options.add_argument('headless')
driver = webdriver.Chrome(executable_path='CHROMEDRIVER_PATH',chrome_options=chrome_options)
@wkarney
wkarney / standardize_column_names.py
Last active February 24, 2020 21:04 — forked from georgerichardson/standardise_column_names.py
[Standardizing Column Names for pandas] Make DataFrame column names lowercase and replace whitespace (and punctuation) with '_' #python #pandas
import re
import string
def standardise_column_names(df, remove_punct=True):
""" Converts all DataFrame column names to lower case replacing
whitespace of any length with a single underscore. Can also strip
all punctuation from column names.
Parameters
----------
@wkarney
wkarney / py_setup.md
Last active February 26, 2020 23:29
[Python Environment] Python environment setup tips #python #shell #pyenv

Homebrew

Update homebrew

brew upgrade && brew update

Pyenv Tips

How to show all available install versions? pyenv install --list

@wkarney
wkarney / excel_pd_import.py
Created March 16, 2020 15:23
[Dealing with Excels with Multiple Tabs etc. in Pandas] #pandas #python
import pandas as pd
# Read in excel file and gathering sheet names
xls = pd.ExcelFile('file_path')
sheet_names = xls.sheet_names
sheet_names
for name in sheet_names:
<df_name> = pd.read_excel('file_path', name)
@wkarney
wkarney / git-clearHistory
Last active September 16, 2020 18:59 — forked from stephenhardy/git-clearHistory
Steps to clear out the history of a git/github repository
-- Remove the history from
rm -rf .git
-- recreate the repos from the current content only
git init
git add .
git commit -m "Initial commit"
-- push to the github remote repos ensuring you overwrite history
git remote add origin https://github.com/<YOUR ACCOUNT>/<YOUR REPO>.git