Skip to content

Instantly share code, notes, and snippets.

View sjqtentacles's full-sized avatar

sjqtentacles

View GitHub Profile
@gregburek
gregburek / rateLimitDecorator.py
Created December 7, 2011 01:51
Rate limiting function calls with Python Decorators
import time
def RateLimited(maxPerSecond):
minInterval = 1.0 / float(maxPerSecond)
def decorate(func):
lastTimeCalled = [0.0]
def rateLimitedFunction(*args,**kargs):
elapsed = time.clock() - lastTimeCalled[0]
leftToWait = minInterval - elapsed
if leftToWait>0:
@nicklasos
nicklasos / plists.erl
Last active November 14, 2017 10:40
Erlang. Parallel map
-module(plists).
-export([pmap/2,pforeach/2,npforeach/2, parmap/2]).
%%% Map
pmap(F, L) ->
S = self(),
Pids = lists:map(fun(I) -> spawn(fun() -> pmap_f(S, F, I) end) end, L),
pmap_gather(Pids).
@samuelsmal
samuelsmal / pyspark_udf_filtering.py
Created October 11, 2016 14:10
PySpark DataFrame filtering using a UDF and Regex
from pyspark.sql.functions import udf
from pyspark.sql.types import BooleanType
def regex_filter(x):
regexs = ['.*ALLYOURBASEBELONGTOUS.*']
if x and x.strip():
for r in regexs:
if re.match(r, x, re.IGNORECASE):
return True
@doyle
doyle / gist:92b0b34cd4804d237ed9661e20e33e5d
Created October 17, 2016 19:51
Use curl to upload a file to a S3 pre-signed url
curl -X PUT -T test_s3.txt -L "https://your-bucket.s3.amazonaws.com/url-stuff"