Skip to content

Instantly share code, notes, and snippets.

View dcramer's full-sized avatar
💭
I may be slow to respond.

David Cramer dcramer

💭
I may be slow to respond.
View GitHub Profile

Goal

Implement a capped pool which will start dropping messages once it reaches <size>. This will have a much lower rate of failure (due to load) than a database or standard queue.

For example, if there is a burst of data coming in, we'd first stick it into the pool, and then fire off a job to the queue (the job would be argless and simple say "get data from pool", it could also eventually be replaced with a continuous processor). This guarantees that your queue isnt overflowing with large amounts of data, but rather potential no-ops (just like our buffer implementation), and will also ensure you don't

"""
sentry.pool.redis
~~~~~~~~~~~~~~~~~
:copyright: (c) 2010-2012 by the Sentry Team, see AUTHORS for more details.
:license: BSD, see LICENSE for more details.
"""
import random
from nydus.db import create_cluster
@dcramer
dcramer / signals.py
Last active December 10, 2015 16:49
Signals with decorator syntax in Django
from functools import wraps
from django.dispatch import Signal
class BetterSignal(Signal):
def connect(self, receiver=None, **kwargs):
"""
Support decorator syntax:
>>> @signal.connect(sender=type)
@dcramer
dcramer / heapqueue.py
Last active December 11, 2015 04:09
Refined Heap Queue for Python (including capped and max heap implementation)
import heapq
from threading import Lock
class HeapQueue(object):
def __init__(self, values=None, maxsize=None, reversed=False):
"""
Create a new heap queue.
- ``maxsize`` will create a capped queue.
@dcramer
dcramer / gist:4569278
Created January 18, 2013 22:36
XPath / JSON / Sentry filters concept
The gist is, you give a path expression (xpath-like), a condition, and a action.
----
Scrub all local vars in a stackframe where the key matches something like a password:
Path:
//sentry.interfaces.Stacktrace/frames[*]/vars/key
Condition:
@dcramer
dcramer / fix_requests.py
Last active December 11, 2015 21:28
Because you should maintain API compatibility when you tell everyone to use your shit.
from requests.models import Response
class fixedjson(object):
def __init__(self, func):
self.func = func
def __get__(self, inst, cls):
result = self.func(inst)
class proxy(type(result)):
@dcramer
dcramer / raven.js
Created February 12, 2013 01:18
sourcemap bug
/*! Raven.js a75cbab | github.com/getsentry/raven-js */
/*
* Includes TraceKit
* https://github.com/getsentry/TraceKit
*
* Copyright 2013 Matt Robenolt and other contributors
* Released under the BSD license
* https://github.com/getsentry/raven-js/blob/master/LICENSE
*
@dcramer
dcramer / gist:4773531
Last active December 12, 2015 12:39
Track last notified
===================
Alert model
-> binds to many users
- alert_date
Alert on Threshold
==================
def meanstdv(x):
n, mean, std = len(x), 0, 0
for a in x:
mean = mean + a
mean = mean / float(n)
for a in x:
std = std + (a - mean) ** 2
std = math.sqrt(std / float(n - 1))
return mean, std
@dcramer
dcramer / 1 dyno
Last active December 13, 2015 17:48
gevent hello world w/ apache bench from an AWS m1.large to Heroku
$ ab -c 100 -t 60 http://pure-depths-6229.herokuapp.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking pure-depths-6229.herokuapp.com (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests