Skip to content

Instantly share code, notes, and snippets.

@schavery
Created October 26, 2017 21:47
Show Gist options
  • Save schavery/d225679b77f280552d9682617a2a48ae to your computer and use it in GitHub Desktop.
Save schavery/d225679b77f280552d9682617a2a48ae to your computer and use it in GitHub Desktop.
New Relic RQ setup
NEW_RELIC_LICENSE_KEY=6xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx5
NEW_RELIC_LOG=stdout
NEW_RELIC_APP_NAME=proj--staging
from newrelic.agent import (shutdown_agent, application, register_application, BackgroundTask)
# [import-hook:rq.worker]
# enabled = true
# execute = hook_rq:instrument_rq_worker
# we're using django, not flask
# from flask import current_app
def instrument_rq_worker(module):
_perform_job = module.Worker.perform_job
register_timeout = 10.0
shutdown_timeout = 10.0
def perform_job(self, job, queue):
# This gets called in the forked process to trigger
# execution of the actual job. We first need to force
# register the application we are going to report data
# against. Allow up to 5.0 seconds for this to complete.
# Normally should take less than 2.0 seconds. If
# registration hasn't occurred after 5.0 seconds, the
# job will be run anyway.
register_application(timeout=register_timeout)
# Execute the actual task.
result = _perform_job(self, job, queue)
# Now shutdown the agent, which will force a data
# harvest. We do this explicitly because RQ calls
# os._exit() to force exit the process and that bypasses
# atexit callbacks, which is how we normally harvest on
# shutdown. Again wait for up to 5.0 seconds to allow
# data to be posted to the data collector.
shutdown_agent(timeout=shutdown_timeout)
return result
module.Worker.perform_job = perform_job
# [import-hook:rq.job]
# enabled = true
# execute = hook_rq:instrument_rq_job
def instrument_rq_job(module):
_perform = module.Job.perform
def perform(self):
# This is the jobs own method to execute the task. Time
# this as a background task where name is the name of
# the queued task. This should also capture any record
# any unhandled exceptions that occurred when running
# the task.
with BackgroundTask(application(), self.func_name):
return _perform(self)
module.Job.perform = perform
[import-hook:rq.worker]
enabled = true
execute = newrelic-config.hook_rq:instrument_rq_worker
[import-hook:rq.job]
enabled = true
execute = newrelic-config.hook_rq:instrument_rq_job
web: newrelic-admin run-program gunicorn proj.wsgi
worker: NEW_RELIC_CONFIG_FILE=newrelic-config/newrelic.ini newrelic-admin run-program python manage.py rqworker default
release: python manage.py migrate
# some extra stuff removed for clarity
proj
├── .env
├── manage.py
├── newrelic-config
│   ├── __init__.py (empty file)
│   ├── hook_rq.py
│   └── newrelic.ini
├── Procfile
├── proj
│    ├── models.py
│    ├── settings.py
│    ├── views.py
│    ├── wsgi.py
│ └── ...
└── requirements.txt
appdirs==1.4.3
boto3==1.4.7
botocore==1.7.13
cached-property==1.3.1
certifi==2017.7.27.1
chardet==3.0.4
click==6.7
defusedxml==0.5.0
dj-database-url==0.4.1
Django==1.11.1
django-log-request-id==1.3.2
django-request-logging==0.5.1
django-rq==0.9.6
django-storages==1.6.5
djangorestframework==3.6.3
djangorestframework-jwt==1.11.0
docutils==0.14
docxtpl==0.4.1
Faker==0.7.18
gunicorn==19.6.0
idna==2.6
isodate==0.6.0
Jinja2==2.9.6
jmespath==0.9.3
lxml==3.8.0
MarkupSafe==1.0
newrelic==2.96.0.80
olefile==0.44
Pillow==4.3.0
psycopg2==2.6.2
PyJWT==1.5.2
python-dateutil==2.6.1
python-docx==0.8.6
python-dotenv==0.6.5
pytz==2017.2
redis==2.10.6
requests==2.18.4
requests-mock==1.3.0
requests-toolbelt==0.8.0
rq==0.8.2
s3transfer==0.1.11
six==1.10.0
urllib3==1.22
whitenoise==3.2
yapf==0.17.0
zeep==2.4.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment