Skip to content

Instantly share code, notes, and snippets.

View damonpetta's full-sized avatar

Damon damonpetta

View GitHub Profile
@max-mapper
max-mapper / index.js
Last active May 9, 2021 02:20
fast loading of a large dataset into leveldb
// data comes from here http://stat-computing.org/dataexpo/2009/the-data.html
// download 1994.csv.bz2 and unpack by running: cat 1994.csv.bz2 | bzip2 -d > 1994.csv
// 1994.csv should be ~5.2 million lines and 500MB
// importing all rows into leveldb took ~50 seconds on my machine
// there are two main techniques at work here:
// 1: never create JS objects, leave the data as binary the entire time (binary-split does this)
// 2: group lines into 16 MB batches, to take advantage of leveldbs batch API (byte-stream does this)
var level = require('level')
@gsomoza
gsomoza / gist:3910863
Created October 18, 2012 10:11
EC2 Bash Prompt
#!/bin/bash
#Set colour prompt using the name, platform or instance id and avzone
if [ ! -f "/opt/aws/AWS-INSTID" ]; then
curl -s --fail http://169.254.169.254/latest/meta-data/instance-id > /opt/aws/AWS-INSTID
fi
export INSTID=`cat /opt/aws/AWS-INSTID`
if [ ! -f "/opt/aws/AWS-AVZONE" ]; then
curl -s --fail http://169.254.169.254/latest/meta-data/placement/availability-zone > /opt/aws/AWS-AVZONE
@nikcub
nikcub / mails.sh
Created September 12, 2011 19:43
Send email with Mail.app from command line script
#!/bin/sh
/usr/bin/osascript > /dev/null <<ASCPT
set stdinText to "$(cat | sed -e 's/\\/\\\\/g' -e 's/\"/\\\"/g')"
set recName to "Nik Cubrilovic"
set recAddr to "nikcub@gmail.com"
set theSubject to "Email from standard input"
tell application "Mail"