Skip to content

Instantly share code, notes, and snippets.

View philshem's full-sized avatar
💭
🐙

@philshem philshem

💭
🐙
View GitHub Profile
@philshem
philshem / revgeo.py
Last active August 29, 2015 14:09
Google reverse geocoding for a list of latitude & longitude (CSV output)
import requests
urlbase = 'http://maps.googleapis.com/maps/api/geocode/json?latlng='
key = None
# list of latitude, longitude pairs
latlong = [(40.714224,-73.961452), (47.3667, 8.5500)]
for xy in latlong:
@philshem
philshem / Get_stackexchange_stats.py
Last active August 29, 2015 14:10
Collect traffic stats to compare Stack Exchange network of sites.
# -*- coding: utf-8 -*-
# collect traffic data from the stackexchange sites page
import requests
from bs4 import BeautifulSoup
from collections import defaultdict
def main():
url = 'http://stackexchange.com/sites?view=list#traffic'
@philshem
philshem / twitter-search-language-mapping.csv
Created January 12, 2015 19:21
Data file to map Twitter advanced search language code (lang:en) to language name
Language Name Language Code
Amharic am
Arabic ar
Bulgarian bg
Bengali bn
Tibetan bo
Cherokee chr
Danish da
German de
Maldivian dv
@philshem
philshem / swissa4.py
Created February 27, 2015 19:41
Comparing Switzerland to an A4 document
from geopy.distance import great_circle
# how similar is switzerland to an A4?
# http://isithackday.com/geoplanet-explorer/index.php?woeid=23424957
topright = (47.808380, 10.492030)
topleft = (47.808380, 5.955870)
bottomleft = (45.818020, 5.955870)
bottomright = (45.818020, 10.492030)
@philshem
philshem / twitter_notification_website_change.py
Last active August 29, 2015 14:19
Send a Twitter status or message when a webpage has a change or is updated
# -*- coding: utf-8 -*-
# 1. scrape a webpage
# 2. compare to previous version
# 3. send a tweet (or direct message) when page is updated
import requests
import os
from lxml import html
from datetime import datetime
@philshem
philshem / get_geo.py
Last active January 13, 2018 15:34
Source code for "Switzerland Tweets" dataviz
# -*- coding: utf-8 -*-
import glob
import json
import requests
delim = '\t'
def get_language(tweet,lang_dict):
@philshem
philshem / coke_donors.tsv
Last active September 29, 2015 11:39
Machine readable public list of Coca Cola donations http://www.coca-colacompany.com/transparency-search?noCache=true
Title Type Organization Amount Date
100 Healthy Living Project 2012 Partnership 100 Black Men of America, Inc. $150,000 2012
100 Healthy Living Project SOAR 2014 Partnership 100 Black Men of America, Inc. $200,000 2014
100 Swimming and Nutrition Education Program* Partnership One Hundred Black Men, Inc. of New York $25,000 2013
123Entalik (321Move)* Partnership Mercy Corps $150,000 2014
2010 Hope for a Cure Gala Partnership Juvenile Diabetes Research Foundation $10,000 2010
2010 Legendary Party Partnership Shepherd Center Foundation $10,000 2010
2010 Obesity Summit Partnership National Hispanic Caucus of State Legislators $20,000 2010
2010 State Competitions Partnership Special Olympics Georgia, Inc. $35,000 2010
2011 'A Family Affair' dinner & concert Partnership Emory University $25,000 2011
@philshem
philshem / swiss_bio_religion.csv
Last active January 9, 2016 21:00
Supporting text for Swiss Bio Religion dataviz
Kanton short Kanton Percent bio 2015 Römisch-katholisch Evangelisch-reformiert Konfessionslos
ZH Zurich 20.2 27.4494002999 32.146057653 24.2100824588
BE Berne 18.3 15.5754304486 55.4993347313 16.209321905
LU Lucerne 17.5 64.7856993061 11.0363370592 13.8023857444
UR Uri 16.6 81.6174974568 4.6998982706 8.0535774839
SZ Schwyz 17.6 63.7955490732 11.2731702235 14.3969931802
OW Obwalden 16.3 73.9953826078 7.2506440927 12.0989058788
NW Nidwalden 15.6 68.8285171426 10.7109434818 13.9352862774
GL Glarus 18.7 34.8267730818 35.4797750771 15.0311385211
ZG Zug 17.5 54.3763370849 14.2177988611 19.6095453653
@philshem
philshem / play_neiss.py
Last active December 26, 2017 20:14
python script to parse NEISS tsv files
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import pandas as pd
import numpy as np
import glob
import csv
# export options - default is just CSV
csv_tf = True #.csv
@philshem
philshem / upwork_skill_tests.csv
Last active October 4, 2018 03:30
Summary of Upwork Skill Tests (collected 2018-10-04 from https://www.upwork.com/ab/tests/)
Category Title Qualified Freelancers Tests Taken Success Ratio
English Language English Spelling Test (U.S. Version) 901778 1394513 0.647
Office Skills Office Skills Test 242375 416129 0.582
Computer Skills Windows XP Test 158157 294835 0.536
Upwork Upwork Readiness Test 131245 258871 0.507
Web Development HTML5 Test 105085 222286 0.473
English Language English Spelling Test (UK Version) 134264 207299 0.648
Office Skills Email Etiquette Certification 135632 194920 0.696
Web Development CSS Test 79855 156032 0.512
Web Development PHP Test 73244 149854 0.489