Skip to content

Instantly share code, notes, and snippets.

View ryanpitts's full-sized avatar

Ryan Pitts ryanpitts

141 followers · 28 following
View GitHub Profile
@ryanpitts
ryanpitts / scraper.py
Created March 4, 2020 01:12
example scraper code for NICAR 2020
import requests
import bs4
import csv
URL = 'http://www.dllr.state.md.us/employment/warn.shtml'
warn_page = requests.get(URL)
soup = bs4.BeautifulSoup(warn_page.text, 'html.parser')
table = soup.find('table')
rows = table.find_all('tr')
@ryanpitts
ryanpitts / fixcsv.py
Last active August 29, 2018 21:07
quick csv grouping script in python re: https://twitter.com/rachelwalexande/status/1034236011830697986
import csv
from collections import OrderedDict
CLEANED_DICT = OrderedDict()
METADATA_FIELDS = ['districtID','district','schoolID','school']
VALUE_FIELDS = []
with open('schools.csv') as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
@ryanpitts
ryanpitts / bible_objects.py
Created February 29, 2016 23:20
Python list of every object mentioned in the Bible, in order. Source: http://www.cabinetmagazine.org/issues/16/kay.php
bible_objects = ['seed', 'gold', 'resin', 'onyx', 'fruit', 'fig leaves', 'fruit', 'vegetables', 'portion of meat', 'ark', 'olive leaf', 'burnt offerings', 'wine', 'bricks', 'silver', 'gold', 'tents', 'bread', 'wine', 'heifer carcass', 'goat carcass', 'ram carcass', 'brazier', 'torch', 'flour', 'curds', 'milk', 'veal', 'bread', 'one thousand silver shekels', 'food', 'skin of water', 'wood', 'knife', 'burnt offering', 'four hundred silver shekels', 'jar', 'gold nose ring', 'gold bracelets', 'straw', 'fodder', 'gold and silver jewellery', 'clothing', 'bread', 'lentil stew', 'quiver', 'bow', 'clothes', 'goatskins', 'bread', 'food', 'wine', 'stone', 'oil', 'mandrake plants', 'poplar branches', 'almond branches', 'watering troughs', 'tent', 'gods', 'saddle', 'stone', 'stones', 'food', 'one hundred pieces of silver', 'tent', 'altar', 'idols', 'earring', 'stone', 'oil', 'ornamented robe', 'food', 'spices', 'balm', 'myrrh', 'twenty shekels', 'clothes', 'goat carcass', 'sackcloth', 'clothes', 'veil', 'seal', 'cord', 's
@ryanpitts
ryanpitts / data.json
Last active August 29, 2015 14:26
data! from editdata.org!
[{"key":1,"value":{"City":"Kansas City","Team Name":"Royals"}},{"key":2,"value":{"City":"Detroit","Team Name":"Tigers"}},{"key":3,"value":{"City":"Minnesota","Team Name":"Twins"}}]
@ryanpitts
ryanpitts / gist:5d3767bcf67ad724b9f3
Last active August 29, 2015 14:22
giffffffffffs
<div class="image-gif-wrapper">
<img class="lazy-load" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" width="600" data-src="https://assets.opennews.org/img/loops/gif-placeholder.png" rel:animated_src="https://assets.opennews.org/img/loops/Phenakistoscope_12.gif">
</div>
<script src="https://cdn.jsdelivr.net/jquery/2.1.4/jquery.min.js"></script>
<script src="https://cdn.jsdelivr.net/blazy/latest/blazy.min.js"></script>
<script src="https://assets.opennews.org/js/libgif_wtf.js"></script>
<style>
.jsgif {
@ryanpitts
ryanpitts / postrock_song_titles.txt
Last active April 14, 2023 08:51
1,115 postrock song titles
War On Christmas
Glow
Cambian
Somnambulance
Long March
Seperate But Equal
Peterson
Thompson
Twelve Days Awake
Everyone Gets Shot
@ryanpitts
ryanpitts / aliases.txt
Created September 4, 2014 16:10
some aliases
git config --global alias.amend 'commit --amend --no-edit'
@ryanpitts
ryanpitts / fuzzyfields.py
Created August 14, 2014 23:05
FuzzyDateFields for Django
import datetime
import dateutil.parser
from django import forms
class FuzzyDateTimeField(forms.Field):
'''
Like a DateTimeField, but uses dateutil.parser to parse datetime.
http://www.djangosnippets.org/snippets/1422/
'''
@ryanpitts
ryanpitts / censusreporter_geo_example.json
Created May 20, 2014 19:43
Example API response from Census Reporter, showing geograhic metadata (including coordinates) for Spokane, WA. If you make a request to http://api.censusreporter.org/1.0/geo/tiger2012/16000US5367000?geom=true, you'll get results like this.
{
"geometry": {
"type": "MultiPolygon",
"coordinates": [
[
[
[
-117.604044,
47.615714
],
@ryanpitts
ryanpitts / censusreporter_data_example.json
Created May 20, 2014 19:05
Example API response from Census Reporter, showing "Sex By Age" data (Table B01001) for all counties in Washington state, from the ACS 2012 5-year release. If you make a request to http://api.censusreporter.org/1.0/data/show/latest?table_ids=B01001&geo_ids=050|04000US53, you'll get results like this.
{
"release": {
"id": "acs2012_5yr",
"name": "ACS 2012 5-year",
"years": "2008-2012"
},
"tables": {
"B01001": {
"title": "Sex by Age",
"universe": "Total Population",