Skip to content

Instantly share code, notes, and snippets.

View tylerpearson's full-sized avatar

Tyler Pearson tylerpearson

42 followers · 25 following
View GitHub Profile
@tylerpearson
tylerpearson / wv-2014-bill-simhashes.json
Created September 22, 2015 08:00
[{"number":"sb1","simhash":10465963670795915591437538580288343},{"number":"sb10","simhash":487027916618294144867790272464470},{"number":"sb100","simhash":15417219191613797111851361355020582},{"number":"sb101","simhash":174235579860163965932794724421648363634281},{"number":"sb102","simhash":174224946036199000632598510514291194019433},{"number":"sb103","simhash":16293922182844048823895779226340105833},{"number":"sb104","simhash":16294244205711778087649610992689196617},{"number":"sb105","simhash":15961615025286754683278567006434571881},{"number":"sb106","simhash":357241552772617246496639237862765729481},{"number":"sb107","simhash":16283538143724116768489081306850858569},{"number":"sb108","simhash":44933233886409270007578544672615798677065},{"number":"sb109","simhash":16293435400061772618678165817901260489},{"number":"sb11","simhash":12007205831240123030670276864467796},{"number":"sb110","simhash":11007677203025090129678931535412797005},{"number":"sb111","simhash":1739057128525615842437022722062446663401},{"numbe
@tylerpearson
tylerpearson / wv-2015-bill-simhashes.json
Created September 22, 2015 07:57
[{"number":"hb2001","simhash":10750382305925881435},{"number":"hb2002","simhash":6005931239459267833},{"number":"hb2003","simhash":6198456749687090249},{"number":"hb2004","simhash":2046119339972015101},{"number":"hb2005","simhash":12698328043044517563},{"number":"hb2006","simhash":14669717309915314933},{"number":"hb2007","simhash":4633573315411999419},{"number":"hb2008","simhash":9718120274308512840},{"number":"hb2009","simhash":10552301992618270341},{"number":"hb2010","simhash":7995558916276905492},{"number":"hb2011","simhash":16894390210490807912},{"number":"hb2012","simhash":16852415292677939129},{"number":"hb2013","simhash":5158062529398811993},{"number":"hb2014","simhash":12042236986170163360},{"number":"hb2015","simhash":5367546299161745337},{"number":"hb2016","simhash":14442452911993652433},{"number":"hb2017","simhash":11650636210421876007},{"number":"hb2018","simhash":9231621006084078688},{"number":"hb2019","simhash":7820279729124833737},{"number":"hb2020","simhash":5688545524043365879},{"number":"hb2
@tylerpearson
tylerpearson / yc_city_data_check.rb
Last active September 11, 2015 20:46
require 'csv'
yc_companies_names = []
def simple_name(name)
name.downcase
end
# mattermark export
CSV.foreach("mattermark.csv") { |row| yc_companies_names << simple_name(row[0]) }
@tylerpearson
tylerpearson / import_maxmind_csv.sql
Created August 11, 2015 23:18
SQL import MaxMind GeoIP CSV into MySQL
USE users_info;
CREATE TABLE IF NOT EXISTS `geoips` (
`id` INT(1) UNSIGNED ZEROFILL NOT NULL AUTO_INCREMENT,
`ipstart` VARCHAR(50) COLLATE UTF8_GENERAL_CI NOT NULL,
`ipend` VARCHAR(50) COLLATE UTF8_GENERAL_CI NOT NULL,
`locid_start` INT(1) UNSIGNED ZEROFILL NOT NULL,
`locid_end` INT(1) UNSIGNED ZEROFILL NOT NULL,
`country_code` VARCHAR(4) COLLATE UTF8_GENERAL_CI NOT NULL,
`country` VARCHAR(100) COLLATE UTF8_GENERAL_CI NOT NULL,
@tylerpearson
tylerpearson / results.json
Created July 27, 2015 22:14
Calculate the percentage of sponsored bills passed (as primary and cosponsor) for each delegate with a breakdown of bill subjects. These results are for WV House during the 2011 session, but the script should be able to be modified to be used with any JSON dump from the Sunlight Foundation Open States API.
{
"Morgan":{
"total":{
"count":117,
"passed":19,
"subjects":{
"Boards and Commissions":17,
"Licenses":9,
"Health (And Related Subheadings)":6,
"Governmental Agencies":6,
@tylerpearson
tylerpearson / .bashrc
Last active August 29, 2015 14:24
function parse_git_dirty {
[[ $(git status 2> /dev/null | tail -n1) != "nothing to commit (working directory clean)" ]] && echo "*"
}
function parse_git_branch {
git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e "s/* \(.*\)/[\1$(parse_git_dirty)]/"
}
export PS1='\[\033[34;40m\]\W \[\033[0;33m\]$(parse_git_branch)\[\e[0m\] $ '
@tylerpearson
tylerpearson / parse_dump_downloader.rb
Last active August 29, 2015 14:22
Ruby Capybara script to trigger a dump of Parse data. For example, this could be used nightly to get a backup.
require 'cgi'
require 'timeout'
require 'capybara'
class ParseDownloader
include Capybara::DSL
def initialize(email, password, app_name)
Capybara.default_driver = :selenium
@email = email
@tylerpearson
tylerpearson / unicorn.rb
Created June 1, 2015 23:14
Unicorn + Octopus on Heroku
worker_processes Integer(ENV["WEB_CONCURRENCY"] || 4)
timeout 15
preload_app true
before_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
Process.kill 'QUIT', Process.pid
end
@tylerpearson
tylerpearson / scrapenc.js
Created May 24, 2015 01:59
var fs = require('fs');
var request = require('request');
var cheerio = require('cheerio');
console.log("Starting scraping");
var allResults = [];
for (var chapter=1; chapter <= 168; chapter++) {
@tylerpearson
tylerpearson / pass_with_subjects.rb
Last active August 29, 2015 14:20
Calculate the percentage of sponsored bills passed (as primary and cosponsor) for each delegate with a breakdown of bill subjects. These results are for WV House during the 2011 session, but the script should be able to be modified to be used with any JSON dump from the Sunlight Foundation Open States API.
require 'httparty'
require 'json'
require 'pp'
require 'uri'
require 'hashie'
def load_json(filename)
Hashie::Mash.new(JSON.parse(IO.read(filename)))
end