Skip to content

Instantly share code, notes, and snippets.


BigsnarfDude bigsnarfdude

View GitHub Profile
View gist:2d81595460859966d0d3
// this section rewrites the each file byte by byte back to new zip file
while (e.hasMoreElements()) {
val entryIn = e.nextElement().asInstanceOf[ZipEntry]
if (!entryIn.getName.equalsIgnoreCase("abc.txt")) {
val is = zipFile.getInputStream(entryIn)
val buf = Array.ofDim[Byte](10)
var len: Int = 0
bigsnarfdude / gist:0c29a729d0dce6b0caae
Created May 13, 2014
Scala Algebird for IP Address count attack traffic
View gist:0c29a729d0dce6b0caae
case class OutsideServer(val ipAddress: String, val LoginsPerMinute: Int) extends Ordered[OutsideServer] {
def compare(that: OutsideServer): Int = {
val count = this.LoginsPerMinute - that.LoginsPerMinute
if (count == 0) this.ipAddress.compareTo(that.ipAddress) else count
case class InsideServer(val ipAddress: String, val LoginsPerMinute: Int) extends Ordered[InsideServer] {
def compare(that: InsideServer): Int = {
val count = this.LoginsPerMinute - that.LoginsPerMinute
View gist:bd3f7b59e49d4c555bb0
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
consumer_key = "HsYoJa0sdf980as8fbFh0w"
consumer_secret = "k698asdf9870sdf09a7s8df908a7sdf6A7m4"
access_token = "364asdf987asdfuiytsdaf765sdfasdfoiyLgS"
access_token_secret = "SFasdf9807asd09f87asf9087asdfbik5D"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
bigsnarfdude / gist:992070c464c021644494
Created Jun 12, 2014
Dems alotta jars in Apache Spark
View gist:992070c464c021644494
[info] Including: scala-library-2.10.4.jar
[info] Including: netty-all-4.0.17.Final.jar
[info] Including: jetty-server-8.1.14.v20131031.jar
[info] Including: javax.servlet-3.0.0.v201112011016.jar
[info] Including: jetty-continuation-8.1.14.v20131031.jar
[info] Including: jetty-http-8.1.14.v20131031.jar
[info] Including: jetty-io-8.1.14.v20131031.jar
[info] Including: jetty-util-8.1.14.v20131031.jar
[info] Including: jetty-plus-8.1.14.v20131031.jar
[info] Including: javax.transaction-1.1.1.v201105210645.jar
View gist:03c5b513fdcc4aed9233
'[info] Including: JavaEWAH-0.6.6.jar\n',
'[info] Including: ST4-4.0.4.jar\n',
'[info] Including: activation-1.1.jar\n',
'[info] Including: akka-actor_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: akka-remote_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: akka-slf4j_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: akka-zeromq_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: algebird-core_2.10-0.1.11.jar\n',
'[info] Including: ant-1.9.0.jar\n',
'[info] Including: ant-launcher-1.9.0.jar\n',
bigsnarfdude / SparkSQLexperiments.ipynb
Created Jul 1, 2014
Load Spark SQL from File, JSON file, or arrays
View SparkSQLexperiments.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View gist:45bd47e17b7daae60bed
FROM ubuntu:14.04
# need to run to make sure the apt-get installs work
RUN apt-get update
RUN apt-get install -y wget
bigsnarfdude / gist:9ae01a7df8ffa10b5016
Created Aug 1, 2014
weather.scala video recreation "scala-in-action" talk
View gist:9ae01a7df8ffa10b5016
import org.apache.http.client.methods.HttpGet
object Weather {
private val locations = List(
bigsnarfdude / gist:55b3b6c394f923bcbe66
Created Aug 3, 2014
problem with algebird 0.7.0 jar from maven repo
View gist:55b3b6c394f923bcbe66
antigensAir:bin antigen$ ./scala -cp /Users/antigen/Downloads/algebird-core_2.10-0.7.0.jar
Welcome to Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_60).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import com.twitter.algebird._
import com.twitter.algebird._
scala> import HyperLogLog._
import HyperLogLog._
View gist:c48dff7c79228b6180f3
import com.twitter.algebird._
import HyperLogLog._
import com.twitter.algebird.Monoid
import com.twitter.algebird.DecayedValue
import com.twitter.algebird.Operators._
val hll = new HyperLogLogMonoid(4)
You can’t perform that action at this time.