Skip to content

Instantly share code, notes, and snippets.

View shawnhermans's full-sized avatar

Shawn Hermans shawnhermans

7 followers · 3 following
View GitHub Profile
@shawnhermans
shawnhermans / tlepreprocess.py
Created January 19, 2013 19:12
f = open('gps-ops.txt')
out = open('gps-ops.tsv','wb')
count = 0
output = ''
for row in f:
if count % 3 != 2:
output += row.strip() + '\t'
@shawnhermans
shawnhermans / PutExample.java
Last active December 14, 2015 22:18
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException; public
class PutExample {
public static void main(String[] args) throws IOException {
Configuration conf = HBaseConfiguration.create();
@shawnhermans
shawnhermans / entry.json
Last active December 25, 2015 13:38
Example of Avro schema for storing a collection of Atom entries.
{
"name": "Entry",
"namespace": "com.shawnhermans.avro.atom",
"doc": "A sample Atom entry",
"type": "record",
"fields": [
{"name": "title", "type": "string"},
{"name": "id", "type": "string"},
{"name": "updated", "type": ["long", "string"]},
{"name": "published", "type": ["long", "null", "string"]},
@shawnhermans
shawnhermans / entry-to-avro.py
Created October 15, 2013 07:33
import avro.schema
from avro.datafile import DataFileWriter
from avro.io import DatumWriter
schema = avro.schema.parse(open('entry.avsc').read())
writer = DataFileWriter(open("entries.avro", "w"), DatumWriter(), schema)
entry = {u'authors': [{u'email': u'f8dy@example.com',
u'name': u'Mark Pilgrim',
u'uri': u'http://example.org/'}],
@shawnhermans
shawnhermans / tupletomap.py
Created November 27, 2013 23:08
Simple example of converting tuples to a map
@outputSchema("output:map[]")
def tupleToMap(inputs):
field_names = ["field1", "field2", "field3"]
result = {}
if len(field_names) == len(inputs):
for i in range(len(inputs)):
result[field_names[i]] = inputs[i]
@shawnhermans
shawnhermans / top-machine-learning-algorithms.md
Created September 9, 2015 16:30
@shawnhermans
shawnhermans / encode.sh
Created September 21, 2015 17:33
Example of command line encoding with ffmpeg
ffmpeg -i $1 -codec:v libx264 -profile:v high -preset slower -crf 25 -threads 0 -codec:a libfdk_aac -b:a 128k -f mp4 $2
@shawnhermans
shawnhermans / create-iso.sh
Created September 23, 2015 15:58
Script I used to create a Yosemite ISO image
#!/bin/bash
# Mount the installer image
hdiutil attach /Applications/Install\ OS\ X\ Yosemite.app/Contents/SharedSupport/InstallESD.dmg -noverify -nobrowse -mountpoint /Volumes/install_app
# Convert the boot image to a sparse bundle
hdiutil convert /Volumes/install_app/BaseSystem.dmg -format UDSP -o /tmp/Yosemite
# Increase the sparse bundle capacity to accommodate the packages
hdiutil resize -size 8g /tmp/Yosemite.sparseimage
@shawnhermans
shawnhermans / foundations-of-cognitive-science.md
Created October 6, 2015 14:06

Chapter 1 - Herbert A. Simon and Craig A. Kaplan

From a sociological standpoint, disciplines are defined less by their intellectual structure and content than by the scientists who identify with them. But more accurately, over time the intellectual content of a discipline gradually defines its boundaries and membership, whereas its membership gradually redefines its content.

@shawnhermans
shawnhermans / markdown-example.md
Created December 16, 2015 19:19
Simple example of Markdown

Heading 1

Heading 2

  • Bullet 1
  • Bullet 2
  • Bullet 3

Heading 3