chrishamant

#!/usr/bin/env python
"""Split large file into multiple pieces for upload to S3.

S3 only supports 5Gb files for uploading directly, so for larger CloudBioLinux
box images we need to use boto's multipart file support.

This parallelizes the task over available cores using multiprocessing.

Usage:
  s3_multipart_upload.py <file_to_transfer> <bucket_name> [<s3_key_name>]

smougenot

  # stacktrace java as one message
  multiline {
    #type => "all" # no type means for all inputs
    pattern => "(^.+Exception: .+)|(^\s+at .+)|(^\s+... \d+ more)|(^\s*Caused by:.+)"
    what => "previous"
  }

guyrt

#!/bin/bash
# Install sqoop and mysql connector. Store in s3 and load 
# as bootstrap step.

bucket_location='<put your bucket here>'
sqoop_jar='sqoop-1.4.3.bin__hadoop-1.0.0'
sqoop_jar_gz=$sqoop_jar.tar.gz
mysql_dir='mysql-connector-java-5.1.24'
mysql_jar='mysql-connector-java-5.1.24-bin.jar'
mysql_dir_gz=$mysql_dir.tar.gz

ramn

import java.io._

@SerialVersionUID(15L)
class Animal(name: String, age: Int) extends Serializable {
  override def toString = s"Animal($name, $age)"
}

case class Person(name: String)

// or fork := true in sbt

fonnesbeck

fonnesbeck

fonnesbeck

fonnesbeck

jed

How to set up stress-free SSL on an OS X development machine

One of the best ways to reduce complexity (read: stress) in web development is to minimize the differences between your development and production environments. After being frustrated by attempts to unify the approach to SSL on my local machine and in production, I searched for a workflow that would make the protocol invisible to me between all environments.

Most workflows make the following compromises:

• Use HTTPS in production but HTTP locally. This is annoying because it makes the environments inconsistent, and the protocol choices leak up into the stack. For example, your web application needs to understand the underlying protocol when using the secure flag for cookies. If you don't get this right, your HTTP development server won't be able to read the cookies it writes, or worse, your HTTPS production server could pass sensitive cookies over an insecure connection.

• Use production SSL certificates locally. This is annoying

ConnorDoyle

package test

import scala.reflect.runtime.universe._

object ReflectionHelpers extends ReflectionHelpers

trait ReflectionHelpers {

  protected val classLoaderMirror = runtimeMirror(getClass.getClassLoader)