Skip to content

Instantly share code, notes, and snippets.

Created March 11, 2012 22:04
Show Gist options
  • Save chooper/2018362 to your computer and use it in GitHub Desktop.
Save chooper/2018362 to your computer and use it in GitHub Desktop.
Decorator to quickly add statsd (graphite) instrumentation to Celery task functions.
"""Decorator to quickly add statsd (graphite) instrumentation to Celery
task functions.
With some slight modification, this could be used to instrument just
about any (non-celery) function and be made abstract enough to customize
metric names, etc.
Stats reported include number of times the task was accepted by a worker
(`started`), the number of successes, and the number of times the task
raised an exception. In addition, it also reports how long the task took
to complete. Usage:
>>> @task
>>> @instrument_task
>>> def mytask():
>>> # do stuff
>>> pass
Please note that the order of decorators is important to Celery. See
for more information.
Uses `simple_decorator` from
Limitation: Does not readily work on subclasses of celery.tasks.Task
because it always reports `task_name` as 'run'
# statsd instrumentation
from celery import current_app
import statsd
def instrument_task(func):
"""Wraps a celery task with statsd instrumentation code"""
def instrument_wrapper(*args, **kwargs):
stats_conn = statsd.connection.Connection(
host = current_app.conf['STATSD_HOST'],
port = current_app.conf['STATSD_PORT'],
sample_rate = 1)
task_name = func.__name__
counter = statsd.counter.Counter('celery.tasks.status',stats_conn)
timer = statsd.timer.Timer('celery.tasks.duration', stats_conn)
ret = func(*args, **kwargs)
return ret
del timer
del counter
del stats_conn
return instrument_wrapper
def simple_decorator(decorator):
"""Borrowed from:
Original docstring:
This decorator can be used to turn simple functions
into well-behaved decorators, so long as the decorators
are fairly simple. If a decorator expects a function and
returns a function (no descriptors), and if it doesn't
modify function attributes or docstring, then it is
eligible to use this. Simply apply @simple_decorator to
your decorator and it will automatically preserve the
docstring and function attributes of functions to which
it is applied."""
def new_decorator(f):
g = decorator(f)
g.__name__ = f.__name__
g.__module__ = f.__module__ # or celery throws a fit
g.__doc__ = f.__doc__
return g
# Now a few lines needed to make simple_decorator itself
# be a well-behaved decorator.
new_decorator.__name__ = decorator.__name__
new_decorator.__doc__ = decorator.__doc__
return new_decorator
Copy link

blueyed commented Aug 28, 2014

This seems to be interesting in this regard, too:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment