Skip to content

Instantly share code, notes, and snippets.

View stuartlynn's full-sized avatar

Stuart Lynn stuartlynn

119 followers · 8 following
  • Two Sigma
  • Brooklyn
View GitHub Profile
@stuartlynn
stuartlynn / gist:7dad3e6b01a12d2957cb
Created October 20, 2014 20:43
Planet hunter mini course analysis
{
"metadata": {
"name": "Minicourse Analysis"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
@stuartlynn
stuartlynn / gist:28e5bb8b294d68603a64
Created November 4, 2014 18:45
Cancer research processing
require 'pry'
require 'bson'
require 'json'
manifest = []
outfile = 'for_testing'
blank_images = []
["MRE11 TMAs.zip"].each do |zip_name|
@stuartlynn
stuartlynn / gist:f79b4f520af600a7ad73
Last active August 29, 2015 14:09
Data extract
import pprint
import json
import csv
file = open('all.txt')
j_file = open("all.json","w")
csv_file = open("all.csv","w")
writer = csv.writer(csv_file, delimiter="\t")
pp = pprint.PrettyPrinter(depth=6)
@stuartlynn
stuartlynn / gist:859846929d4c2f9c666e
Created November 9, 2014 23:58
Example amazon dataset
{
"metadata": {
"name": "Untitled0"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
@stuartlynn
stuartlynn / gist:c9926e7faebed6d2b4c3
Last active August 29, 2015 14:10
Purge unsubscribed emails
from mailbox import mbox
# from bs4 import BeautifulSoup
from pyquery import PyQuery as pq
import code
m = mbox("noreply.mbox")
reject_emails = []
warnings = []
@stuartlynn
stuartlynn / gist:f1ba2270032ef2722b9e
Last active August 29, 2015 14:10
Potential project manifest i dig bio hackathon
{
"project":{
"name" : "my bio project",
"description": "Dolore voluptate sint proident ad quis esse non exercitation deserunt elit est consectetur id. Tempor veniam sint voluptate voluptate eiusmod. Non deserunt ex eiusmod laboris commodo consectetur quis reprehenderit voluptate aliqua ea. Elit fugiat ad voluptate eu Lorem dolor eiusmod sunt laborum. Laboris reprehenderit excepteur sint amet tempor mollit occaecat aliquip sint minim. Minim enim dolor voluptate quis labore deserunt nulla est fugiat Lorem dolor. Aliquip Lorem anim mollit est proident fugiat dolore dolore.
",
"principle_locations": [ "England", "Scotland" ],
"principle_species":["birds", "bugs"],
},
@stuartlynn
stuartlynn / gist:fa7900292b8fd571ec1d
Created March 24, 2015 00:28
Locations of Wikipedia edits about countries
SELECT DISTINCT rev_user_text AS IP, COUNT(*) AS edits , page_title as country
FROM revision
INNER JOIN page ON rev_page = page_id
WHERE rev_user = 0
AND page_namespace = 0
AND page_title IN ("Member_states_of_the_United_Nations","United_Nations_General_Assembly_observers","Afghanistan","Albania","Algeria","Andorra","Angola","Antigua_and_Barbuda","Argentina","Armenia","Australia","Austria","Azerbaijan","The_Bahamas","Bahrain","Bangladesh","Barbados","Belarus","Belgium","Belize","Benin","Bhutan","Bolivia","Bosnia_and_Herzegovina","Botswana","Brazil","Brunei","Bulgaria","Burkina_Faso","Burma","Burundi","Cambodia","Cameroon","Canada","Cape_Verde","Central_African_Republic","Chad","Chile","China","Colombia","Comoros","Democratic_Republic_of_the_Congo","Republic_of_the_Congo","Costa_Rica","Croatia","Cuba","Cyprus","Czech_Republic","Denmark","Djibouti","Dominica","Dominican_Republic","East_Timor","Ecuador","Egypt","El_Salvador","Equatorial_Guinea","Eritrea","Estonia","Ethiopia","Fiji","Finland","France","Gabon","Th
@stuartlynn
stuartlynn / index.html
Created April 29, 2015 15:15
Gits for creating an upload to s3 site
<html>
<head>
<title>S3 POST Form</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<form action="https://zooniverse-data.s3.amazonaws.com/" method="post" enctype="multipart/form-data">
<input type="hidden" name="key" value="cruk_upload/${filename}">
<input type="hidden" name="AWSAccessKeyId" value={AWSAccessKeyId}>
@stuartlynn
stuartlynn / PH2 ingest
Last active August 29, 2015 14:22
Planet Hunters K2 data ingest
# PlanetHunterSubject.destroy_all
# redis = Ouroboros.redis[Rails.env]
# redis.keys('planet_hunter*').each{ |key| redis.del(key) }
json = JSON.parse(File.read('/home/ubuntu/manifest_C1.json')); nil
project = PlanetHunterSubject.project
workflow = PlanetHunterSubject.first.workflows.first
@stuartlynn
stuartlynn / gist:fc57e207358e222df77c
Created June 16, 2015 01:49
Sample iffT carto integration server
#This is actually running and dumping data here http://cdb.io/1BeHh9S
express = require 'express'
app = express()
redis = require("redis")
url = require('url');
CartoDB = require('cartodb');
bodyParser = require('body-parser')