Skip to content

Instantly share code, notes, and snippets.

View erikerlandson's full-sized avatar

Erik Erlandson erikerlandson

View GitHub Profile
@erikerlandson
erikerlandson / find_git_dirs_example.txt
Created May 5, 2014 23:27
A find command that identifies directory names housing a git repo
[eje@localhost git]$ for dir in `find /home/eje/git -maxdepth 3 -exec test -d \{\}/.git \; -print` ; do echo $dir; cd $dir ; git reset --hard HEAD; done
@erikerlandson
erikerlandson / audit2allow_example.txt
Created May 7, 2014 16:20
Example of using audit2allow to generate an selinux patch rule from failure messages in the audit log
# grep brprintconf_mfc /var/log/audit/audit.log | audit2allow -M mypol
# semodule -i mypol.pp
scala> val f1 = (x:Int, t:Int) => (x > t)
f1: (Int, Int) => Boolean = <function2>
scala> f1
res22: (Int, Int) => Boolean = <function2>
scala> val f2 = (x:Int, t:Int=7) => (x > t)
<console>:1: error: ')' expected but '=' found.
val f2 = (x:Int, t:Int=7) => (x > t)
@erikerlandson
erikerlandson / scala_config_demo.scala
Created May 16, 2014 03:09
A crude prototype of a configuration system in scala that supports easy meta-config of policies for individual configuration parameters
import scala.collection.mutable
import scala.reflect.runtime.universe.{TypeTag, TypeRef, typeOf}
// A function that also keeps its domain and range types as a payload
class FunctionTP[D: TypeTag, R:TypeTag](f: D=>R) extends Function[D,R] {
val domainType = typeOf[D]
val rangeType = typeOf[R]
val func = f
override def apply(x: D):R = func(x)
def compose[E:TypeTag](g: E=>D):FunctionTP[E,R] = new FunctionTP(func compose g)
@erikerlandson
erikerlandson / generic_conversion_example.scala
Created May 19, 2014 21:33
Demonstrate generic conversion, where explicit invocations work but implicit invocations don't happen
trait CastOptionToV[V] {
def cast[T:TypeTag](o:Option[T]):V
}
class CastOptionToLong extends CastOptionToV[Long] {
def cast[T:TypeTag](o:Option[T]):Long = {
val tLong = typeOf[Long]
typeOf[T] match {
case `tLong` => o.asInstanceOf[Option[Long]].get
case _ => throw new Exception
}
@erikerlandson
erikerlandson / boost_date_time.cpp
Created June 14, 2014 18:07
Demonstrate basic boost date/time usage of clock, duration and time comparison. Compare cost of local_time() and universal_time(). The universal_time() function is about 2x faster.
#include <iostream>
#include "boost/date_time/posix_time/posix_time_types.hpp"
using namespace boost::posix_time;
int main(int argc, char** argv) {
const int dsec = 5;
{
ptime t0 = microsec_clock::local_time();
@erikerlandson
erikerlandson / xval_ALS.scala
Created June 26, 2014 20:42
Demonstrate a function that abstracts cross validation for an MLLib model - in this case org.apache.spark.mllib.recommendation.MatrixFactorizationModel
import java.lang.Math
import org.apache.spark.rdd.RDD
import org.apache.spark.mllib.recommendation.Rating
import org.apache.spark.mllib.recommendation.ALS
import org.apache.spark.mllib.recommendation.MatrixFactorizationModel
import org.apache.spark.mllib.util.MLUtils.kFold
// Preload some Rating data for my own convenience
val txt = sc.textFile("/home/eje/git/ratorade/data/bgr.dat")
val ratings = txt.map(_.split('\t') match { case Array(user, item, rating, _, _) => Rating(user.toInt, item.toInt, rating.toDouble / 100.0)})
@erikerlandson
erikerlandson / gist:8518b1879f8748cb9120
Last active August 29, 2015 14:04
start up spark master and slaves in a sandbox
# start-slaves wants sshd running
# start up master
# web ui: http://localhost:8080/
$ ./sbin/start-master.sh
# start up 2 slaves
# web ui: http://localhost:8081/ (8082, 8083, ...)
$ env SPARK_WORKER_INSTANCES=2 ./sbin/start-slaves.sh
@erikerlandson
erikerlandson / power.scala
Last active August 29, 2015 14:04
scala power function
def pow(n: Int, k:Int): Int = k match {
case kk if kk < 0 => 0 // error
case 0 => 1
case 1 => n
case 2 => n*n
case kk if kk % 2 == 0 => pow(pow(n, k/2), 2)
case _ => pow(n, k/2)*pow(n, (k+1)/2)
}
@erikerlandson
erikerlandson / power.scala
Created August 6, 2014 18:58
tail recursive power function in scala
import scala.annotation.tailrec
def pow(n: Int, k: Int): Int = {
if (k <= 0) return 1
@tailrec def p(acc: Int, k:Int):Int = if (k == 1) acc else p(n*acc, k-1)
p(n, k)
}