Skip to content

Instantly share code, notes, and snippets.

View glortho's full-sized avatar

Jed Verity glortho

View GitHub Profile
@glortho
glortho / gist:1371866
Created November 16, 2011 23:30
JS: Helper for dealing with url vars/params in a string
/*
Param Helper (Jed Verity 2011)
USAGE
Given query = "?foo=bar&oof=rab":
query.param('foo') // 'bar'
query.param('oof') // 'rab'
@glortho
glortho / random_integer_source.hpy
Created May 14, 2015 18:03
Random Integer Source
#!/bin/sh
"exec" "twxec" "-e" "rand_ints()" "$0" "$@"
import random
import gevent
def rand_ints():
while True:
value = random.randint({{number minval}}, {{number maxval}} + 1)
@glortho
glortho / accrual_rate.hpy
Created May 14, 2015 20:27
Calculate the average rate at which the user has accrued friends and followers
#!/bin/sh
"exec" "twxec" "-e" "average_accrual_rate" "$0" "$@"
from dateutil import parser
{{docstring "Calculate the average rate at which the user has accrued friends and followers"}}
_source = {{string Message_Type ["Twitter", "GNIP"]}}
def average_accrual_rate(msg):
if _source == "GNIP":
@glortho
glortho / anomaly_detection.hpy
Created May 14, 2015 20:29
Given the keys to the values of the average and standard deviation of a distribution, this transform identifies whether the value of a particular variable is an outlier with respect to that distribution. This enrichment returns a user-specified key with an * anomaly * field that contains _True_ or _False_
#!/bin/sh
"exec" "twxec" "-e" "anomaly_detection" "$0" "$@"
import json
{{ docstring "Given the keys to the values of the average and standard deviation of a distribution, this transform
identifies whether the value of a particular variable is an outlier with respect to that distribution. This enrichment returns
a user-specified key with an * anomaly * field that contains _True_ or _False_." }}
avg_key = {{string Key_name_of_distribution_average_value}}
std_key = {{string Key_name_of_distribution_standard_deviation_value}}
@glortho
glortho / binary_classifier.hpy
Created May 14, 2015 20:30
Predict if a tweet belongs to a category(based on its text) using a pre-built[Supoort Vector Machine](http://en.wikipedia.org/wiki/Support_vector_machine)
#!/bin/sh
"exec" "twxec" "-e" "predict_tweet" "$0" "$@"
{{docstring "Predict if a tweet belongs to a category(based on its text) using a pre-built[Supoort Vector Machine](http://en.wikipedia.org/wiki/Support_vector_machine)"}}
from trickle.nner.en.nner import tokens as tokenize
import numpy as np
import csv
import importlib
import pickle
@glortho
glortho / exp_moving_averages.hpy
Created May 14, 2015 20:31
Exp Moving Averages
#!/bin/sh
"exec" "twxec" "-e" "calc_emas" "$0" "$@"
import numpy as np
from itertools import count
from math import sqrt
import time
class ExpMovingAvg(object):
@glortho
glortho / filter_language.hpy
Created May 14, 2015 20:32
Filter your simplified tweets by the language they're written in. Choose your language from the dropdown box below.
#!/bin/sh
"exec" "twxec" "-e" "filter_by_language" "$0" "$@"
{{docstring "Filter your simplified tweets by the language they're written in. Choose your language from the dropdown box below."}}
language = {{string Language ["English", "French", "Arabic", "Japanese", "Spanish", "German", "Italian", "Indonesian", "Portuguese", "Korean", "Turkish", "Russian"]}}
language_map = {
"English": "en",
"French": "fr",
@glortho
glortho / filter_tweets.hpy
Created May 14, 2015 20:34
Filter out retweets - because if it's not authentic, it's not for you
#!/bin/sh
"exec" "twxec" "-e" "filter_out_retweets" "$0" "$@"
{{docstring "Filter out retweets - because if it's not authentic, it's not for you"}}
def filter_out_retweets(msg):
# Look for both source style as well as transform style
# Timbr.io style:
if "is_retweet" in msg and msg["is_retweet"]:
@glortho
glortho / gnip_powertrack.hpy
Last active August 29, 2015 14:21
Gnip Powertrack
#!/bin/sh
"exec" "twxec" "-e" "stream()" "$0" "$@"
from trickle.gnip import PowerTrackStream
user = {{string User_Name}}
account = {{string Account_Name}}
password = {{password Account_Password}}
config = {{string GNIP_Rules_Configuration}}
@glortho
glortho / gnip_powertrackreplay.hpy
Created May 14, 2015 20:36
Gnip Powertrackreplay
#!/bin/sh
"exec" "twxec" "-e" "replay_stream()" "$0" "$@"
from trickle.gnip import PowerTrackReplayStream
import datetime as dt
user = {{string User_Name}}
account = {{string Account_Name}}
password = {{password Account_Password}}
config = {{string GNIP_Configuration}}