Skip to content

Instantly share code, notes, and snippets.

View neilhawkins's full-sized avatar

neilhawkins

4 followers · 1 following
View GitHub Profile
@neilhawkins
neilhawkins / 2013q2.csv
Last active December 21, 2015 01:49
UK political donations info
name color entity
Conservative Party #0087DC party
Labour Party #DC241f party
Liberal Democrats #FDBB30 party
UKIP #70147A party
Unite #aabbcc donor
Joan Edwards* #aabbcc donor
GMB #aabbcc donor
UNISON #aabbcc donor
USDAW #aabbcc donor
@neilhawkins
neilhawkins / proms_scraper.py
Created January 15, 2014 17:02
Scraper collecting all pieces performed at proms from BBC archive
import scraperwiki, urllib2, re, time
import lxml.html
from pprint import pprint
decades = [x*10 for x in range(189,202)]
for decade in decades:
years = [decade+y for y in range(10)]
for year in years:
url = 'http://www.bbc.co.uk/proms/archive/search/'+str(decade)+'s/'+str(year)
@neilhawkins
neilhawkins / gist:9069114
Created February 18, 2014 11:20
Newspaper circulation in dc js
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8'>
<title>dc.js Experiment</title>
<script src='bower_components/d3/d3.js' type='text/javascript'></script>
<script src='bower_components/crossfilter/crossfilter.js' type='text/javascript'></script>
<script src='dc.js' type='text/javascript'></script>
@neilhawkins
neilhawkins / README.md
Last active August 29, 2015 13:56
My 10,000 pixel wide bar chart.

Why a bar chart for 800 rows of data will end in tears and repetitive strain injury.

Data is political donations above £7500 made to Labour, Conservative and Liberal Democrat parties in the United Kingdom between May 2010 and August 2013.

@neilhawkins
neilhawkins / sorting-array-of-objects.js
Last active August 29, 2015 14:02
Ways of sorting an array of objects
// Ways to sort an array of objects
var arr = [{"member_id":"40323","person_id":"24709","name":"Bridget Phillipson","party":"Labour","constituency":"Houghton and Sunderland South","office":[]},{"member_id":"40636","person_id":"11592","name":"Sharon Hodgson","party":"Labour","constituency":"Washington and Sunderland West","office":[]},{"member_id":"40592","person_id":"24710","name":"Julie Elliott","party":"Labour","constituency":"Sunderland Central","office":[]},{"member_id":"40068","person_id":"13814","name":"Naomi Long","party":"Alliance","constituency":"Belfast East","office":[]},{"member_id":"40428","person_id":"13852","name":"Ian Paisley Jnr","party":"DUP","constituency":"North Antrim"},{"member_id":"40653","person_id":"10858","name":"Pat Doherty","party":"Sinn Fein","constituency":"West Tyrone"},{"member_id":"40434","person_id":"11003","name":"Kevan Jones","party":"Labour","constituency":"North Durham","office":[]},{"member_id":"40606","person_id":"10626","name":"Steve Webb","party":"Liberal Democrat","c
@neilhawkins
neilhawkins / .gistup
Created July 23, 2014 22:18
gistup
@neilhawkins
neilhawkins / wayback-scraper.js
Created August 12, 2014 06:37
wayback-scraper.js
var casper = require("casper").create({
verbose: true,
loglevel: "debug"
});
var response = [], url, site, viewportWidth = 1280, viewportHeight = 1024;
var captureSize = {top: 0, left: 0, width: viewportWidth, height: viewportHeight};
// to get screenshot folder name from url
var re = /url=(\w+)\./;
@neilhawkins
neilhawkins / lazynoding.js
Created January 11, 2015 17:22
reads a node modules directory and outputs all packages and versions for when you forget to --save-dev...
var fs = require("fs");
function main() {
fs.readdir("./node_modules", function (err, dirs) {
if (err) {
console.log(err);
return;
}
dirs.forEach(function(dir){
if (dir.indexOf(".") !== 0) {
@neilhawkins
neilhawkins / LinkedOut.js
Last active August 29, 2015 14:23 — forked from berstend/LinkedOut.js
var INCR = 10; // Number of messages per page
var SENT_URL = '//www.linkedin.com/inbox/sent?startRow=';
var loMessageLinks = []; // Main array for all contacts
var fetchMessages = function(i, cb) {
console.log('Fetching message page #' + (i+1));
$.get(SENT_URL + (i*INCR), function(data){
var $dom = $(data);
@neilhawkins
neilhawkins / gist:120cf80b1a2f33ef85c2ce51580723a1
Last active December 23, 2020 10:28
Initial transforms needed to clean up UK Electoral Commission data exports
[
{
"op": "core/text-transform",
"engineConfig": {
"facets": [],
"mode": "row-based"
},
"columnName": "Value",
"expression": "grel:value.replace(/[,£]/,'').toNumber()",
"onError": "keep-original",