Skip to content

Instantly share code, notes, and snippets.

View nickjevershed's full-sized avatar

Nick Evershed nickjevershed

61 followers · 2 following
View GitHub Profile
@nickjevershed
nickjevershed / checkloop.py
Created April 1, 2014 05:45
checks for source/target loops and tells you what they are
dons = #source-target dataset here
for s in dons:
for ss in dons:
if s['sourceName'] == ss['targetName'] and s['targetName'] == ss['sourceName']:
print s['sourceName'], s['targetName']
@nickjevershed
nickjevershed / csv-read-write.py
Last active August 29, 2015 13:57
A template for reading a csv, then writing to a new csv with a new row
import csv
with open('input.csv','rU') as csvinput:
with open('output.csv', 'w') as csvoutput:
writer = csv.writer(csvoutput, lineterminator='\n')
reader = csv.reader(csvinput, lineterminator='\n')
#Get the headers from the old csv, add to new csv and add our new column header
headers = reader.next()
@nickjevershed
nickjevershed / Labor-network-graph.html
Created March 31, 2014 22:37
Network graph for Labor funds via associated entities
<!DOCTYPE html>
<meta charset="utf-8">
<title>How money flows to the Labor party | The Guardian</title>
<style>
body {
margin:0;
}
.donorContainer {
@nickjevershed
nickjevershed / donations.py
Created February 3, 2014 05:02
Scraper for Australian political party annual returns
import csv
import mechanize
import lxml.html
import scraperwiki
annDonorsurl = "http://periodicdisclosures.aec.gov.au/AnalysisDonor.aspx"
annReportingPeriods={
"1998-1999":"1",
"1999-2000":"2",
@nickjevershed
nickjevershed / getgenres.py
Created January 24, 2014 00:09
Classifies a csv of songs with last.fm tags, EchoNest terms and genres, gets the number of YouTube views, and a few other things
#!/usr/bin/env python
#coding=utf-8
import pylast
import csv
import urllib
import requests
import simplejson
import time
#Gets Last FM data - top 3 tags, duration of track, and lastFM listener count
@nickjevershed
nickjevershed / classify-politician.py
Last active December 31, 2015 21:59
Takes a csv containing names of Australian politicians in the first column, and assigns them a party
#!/usr/bin/env python
# coding=utf-8
import csv
coalition = ["Judith Troeth","Alex Somlyay","Alexander Somlyay","Nick Minchin","Nigel Scullion","Alan Ferguson","Russell Trood","Guy Barnett","Chris Back","Eric Abetz","Judith Adams"," Chris Back","Cory Bernardi","Simon Birmingham","Ron Boswell","Ronald Boswell","Sue Boyce","George Brandis","David Bushby","Michaelia Cash","Richard Colbeck","Helen Coonan","Mathias Cormann","Sean Edwards","Alan Eggleston","David Fawcett","Concetta Fierravanti-Wells","Mitch Fifield","Mary Jo Fisher","Bill Heffernan","Gary Humphries","David Johnston","Barnaby Joyce","Helen Kroger","Ian Macdonald","Bridget McKenzie","Brett Mason","Fiona Nash","Marise Payne","Stephen Parry","Michael Ronaldson","Anne Ruston","Scott Ryan","Arthur Sinodinos","Dean Smith","John Williams","Patrick Secker","Barry Haase","Peter Slipper","Paul Neville","Alby Schultz","John Forrest","Mal Washer","Tony Crook","Judi Moylan","Sophie Mirabella","Alex Somlyay","Alan Tudge","John Alexander","Philip Ruddock","Ross
@nickjevershed
nickjevershed / flight-scraper.py
Created December 19, 2013 04:13
Scraper for Australian politician's expenses
#!/usr/bin/env python
import scraperwiki
import urllib2
import lxml.etree
urls = ["http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/ABBOTT_Tony.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/ALBANESE_Anthony.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/ALEXANDER_John.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/ARBIB_Mark.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/BALDWIN_Bob.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/BIRD_Sharon.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/BISHOP_Bronwyn.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/BOWEN_Chris.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/BRADBURY_David.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T29/BURKE_Tony.pdf"
@nickjevershed
nickjevershed / hide-header.html
Created December 15, 2013 05:55 — forked from gabrieldance/hide-header.html
<style type="text/css">
#gia-header div,
#gia-header h1,
#gia-header p,
#gia-header ul,
#gia-header li {
margin: 0;
padding: 0;
border: 0;
font-size: 100%;
@nickjevershed
nickjevershed / rfs-fires-to-kml
Created October 22, 2013 02:43
Gets latest polygons of fires from RFS site, converts to KML for mapping
#NOTE: please do not DDoS the RFS site with a bunch of automated queries
import lxml.etree
import simplekml
import urllib2
import datetime
kml = simplekml.Kml()
date_string = datetime.datetime.now().strftime("%d-%m-%Y-%H-%M")
#open latest feed
@nickjevershed
nickjevershed / PDF scraper
Created October 14, 2013 22:08
Another scraper for older expense PDFs
#!/usr/bin/env python
import scraperwiki
import urllib2
import lxml.etree
urls = ["http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/ABBOTT_Tony.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/ALBANESE_Anthony.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/ALEXANDER_John.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/ARBIB_Mark.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/BALDWIN_Bob.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/BISHOP_Bronwyn.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/BOWEN_Chris.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/BRADBURY_David.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/CAMERON_Doug.pdf","http://www.finance.gov.au/publications/parliamentarians-reporting/docs/T27/CLARE_Jason.pd