Skip to content

Instantly share code, notes, and snippets.

View akanik's full-sized avatar

Alexandra Kanik akanik

View GitHub Profile
@akanik
akanik / python-process-multiple-census.py
Created March 6, 2019 17:19
If you've downloaded a bunch of the same type of census file, for example multiple years of the same data or multiple geography types, and you want to process them all the same way
#If you've downloaded a bunch of the same type of census file, for example
#multiple years of the same data or multiple geography types, and you want to
#process them all the same way
#works with python3 and python2
import pandas as pd
import os
#Directories that contain census data downloaded from FactFinder
poverty_dir = './census-data/poverty/'
@akanik
akanik / python-scrape-delayed.py
Created March 6, 2019 17:16
Scrape a website that has a delayed page load
#Big thank you to Mike Stucka for all his help on this one <3
import pandas as pd
import requests, time, csv
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import UnexpectedAlertPresentException
#fac_data is created from segment.py
@akanik
akanik / terminal-download-youtube
Last active March 7, 2019 01:08
Download videos from Youtube
#Requires some config: https://github.com/rg3/youtube-dl/blob/master/README.md#readme
youtube-dl URL https://www.youtube.com/watch?v=7NcP-KZr7g4
@akanik
akanik / terminal-reload-environment-vars
Last active March 7, 2019 01:09
Reload environment variables
source ~/.bash_profile
@akanik
akanik / terminal-python-styleguide
Last active March 7, 2019 01:09
Detect lines over 200 characters long
#https://pycodestyle.readthedocs.io/en/latest/
pycodestyle --max-line-length=200 yourdoc.py
@akanik
akanik / terminal-geo-file-convert
Last active March 7, 2019 01:09
Convert a geographic file to another type
#https://www.gdal.org/ogr2ogr.html
ogr2ogr -f KML tl_2015_ky_county.kml tl_2015_ky_county.shp
@akanik
akanik / terminal-lowercase-filenames
Last active March 7, 2019 15:20
Terminal command to lowercase all the filenames in a folder
#terminal
#h/t @brentajones
for i in *; do mv "$i" "$(echo $i|tr A-Z a-z|tr -d " ")"; done
@akanik
akanik / terminal-geo-metadata
Last active March 7, 2019 01:20
get summary information like projection, schema, feature count and extents on a geographic file such as a .shp or .geojson
# more info: https://www.gdal.org/ogrinfo.html
ogrinfo -so -al geofile.shp
############ Example
ogrinfo -so -al tl_2015_ky_county.shp
output>>
INFO: Open of `tl_2015_ky_county.shp'
using driver `ESRI Shapefile' successful.
@akanik
akanik / terminal-start-local-server
Last active March 7, 2019 01:21
Start a local server at http://localhost:8000
#python2
python -m SimpleHTTPServer 8000
#python3
python -m http.server
@akanik
akanik / pandas-transform-apply
Created January 17, 2019 16:43
pandas .transform() vs .apply()
https://stackoverflow.com/questions/27517425/apply-vs-transform-on-a-group-object