Skip to content

Instantly share code, notes, and snippets.

@leifwickland
leifwickland / gist:970895
Created May 13, 2011 17:02
Append to Map of Lists
def appendAt[K,V](m: scala.collection.mutable.Map[K,List[V]], k: K, v: V) = m(k) = (m.getOrElse(k, List[V]()) :+ v)
type Graph[T] = scala.collection.mutable.MultiMap[T,T]
def newGraph[T]: Graph[T] = new scala.collection.mutable.HashMap[T,scala.collection.mutable.Set[T]] with Graph[T]
@leifwickland
leifwickland / gist:1009250
Created June 5, 2011 18:26
SBT Project.scala to slurp in CHD3 Hadoop Core
import sbt._
class Project(info: ProjectInfo) extends DefaultProject(info) {
val clouderaRepo = "cloudera release" at "https://repository.cloudera.com/content/repositories/releases"
val cdhVer = "0.20.2-cdh3u0"
val hadoopCore = "org.apache.hadoop" % "hadoop-core" % cdhVer % "provided"
}
@leifwickland
leifwickland / gist:1010403
Created June 6, 2011 14:57
Example of using futures to schedule work for later
import scala.actors.Futures
(Futures.awaitAll(9999999,1.to(100000).map(x => Futures.future{Thread.currentThread.getName}): _*).foldLeft(Set[String]()) {(set, name) => set + name.get.toString}).foreach(println);
@leifwickland
leifwickland / build.sbt
Created June 7, 2011 18:38
build.sbt which will pull in hadoop and hbase from Cloudera while excluding Thrift.
organization := "Example"
name := "example"
scalaVersion := "2.9.0-1"
resolvers += Resolver.url("ClouderaRepo", url("https://repository.cloudera.com/content/repositories/releases"))
libraryDependencies ++= Seq(
"org.apache.hadoop" % "hadoop-core" % "0.20.2-cdh3u0",
@leifwickland
leifwickland / gist:1013460
Created June 7, 2011 23:39
MD5 in scala
object MD5 {
def hash(s: String) = {
val m = java.security.MessageDigest.getInstance("MD5")
val b = s.getBytes("UTF-8")
m.update(b, 0, b.length)
new java.math.BigInteger(1, m.digest()).toString(16)
}
}
class ThreadLocal[T](init: => T) extends java.lang.ThreadLocal[T] with Function0[T] {
override def initialValue:T = init
def apply = get
def :=(value: T) { set(value) }
}
@leifwickland
leifwickland / gist:1072762
Created July 8, 2011 20:44
Example of slamming a site with lots of HTTP HEAD requests in Scala
// Note to self: Use https://github.com/scalaj/scalaj-http next time instead.
import java.net._
import actors.Future
import actors.Futures._
object woobox extends App {
val codes = ('a'.to('z').map(_.toString) ++ 0.to(9).map(_.toString)).combinations(6)
val blockSize = 100
var i = 0
@leifwickland
leifwickland / gist:1090136
Created July 18, 2011 17:43
Recommended way of pimping Scala types (rather than using implicit conversions) according to Jorge's laws in http://dl.dropbox.com/u/1679797/Scalathon/CanBeConfusing.pdf
class ByteArrayWrapper(bytes: Array[Byte]) {
def fromUtf8: String = {
"Converted to String."
}
}
object ByteArrayWrapper {
implicit def makeByteArrayWrapper(bytes: Array[Byte]): ByteArrayWrapper = new ByteArrayWrapper(bytes)
}
object byteImplicit extends App {
@leifwickland
leifwickland / gist:1099861
Created July 22, 2011 17:01
Here's one solution I found for fighting with type erasure warnings.
object typeErasure {
def main(args: Array[String]) {
val evenOdd = 1.to(10)
.map{x => if (x%2==0) { Option[Int](x) } else { None } }
.partition{ case Some(_) => true case _ => false }
println(evenOdd)
}
}