$ pip install celery
$ pip install redis
$ sudo apt-get install redis-server
CELERY_BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Asia/Karachi'
Inside your app, create a file called celery.py, and add this:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'spider_hospital.settings.production')
app = Celery('spider_hospital')
"""
We're using [django.conf:settings] here which means we're using
our project's settings file for our celery config instead of a separate
settings file so that all of our configuration lives in one place.
We can also create a separate settings file for celery configs
and mention that here as [spider_hospital.celery_config]
"""
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()
This is based on the assumption that you already have one or more
tasks.py file(s) in your project.
$ sudo apt-get install supervisor
Now we need to add the supervisor config files at /etc/supervisor/conf.d:
$ sudo nano /etc/supervisor/conf.d/celery_worker.conf
Paste the following content to the file (edit the paths to match your server):
; ==========================
; celery worker supervisor
; ==========================
; the name of your supervisord program
[program:celery-default]
; Set full path to celery program if using virtualenv
command=/home/itadmin/spiderenv/bin/celery -A spider_hospital worker --loglevel=INFO
; The directory to your Django project
directory=/home/itadmin/spider_hospital
; If supervisord is run as the root user, switch users to this UNIX user account
; before doing any processing.
user=itadmin
; Supervisor will start as many instances of this program as named by numprocs
numprocs=1
; Put process stdout output in this file
stdout_logfile=/var/log/celery/celery_worker.log
; Put process stderr output in this file
stderr_logfile=/var/log/celery/celery_worker.log
; If true, this program will start automatically when supervisord is started
autostart=true
; May be one of false, unexpected, or true. If false, the process will never
; be autorestarted. If unexpected, the process will be restart when the program
; exits with an exit code that is not one of the exit codes associated with this
; process’ configuration (see exitcodes). If true, the process will be
; unconditionally restarted when it exits, without regard to its exit code.
autorestart=true
; The total number of seconds which the program needs to stay running after
; a startup to consider the start successful.
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true
; if your broker is supervised, set its priority higher
; so it starts first
priority=998
And then...
$ sudo nano /etc/supervisor/conf.d/celery_beat.conf
With the following content:
; ========================
; celery beat supervisor
; ========================
; the name of your supervisord program
[program:celery-beat]
; Set full path to celery program if using virtualenv
command=/home/itadmin/spiderenv/bin/celery -A spider_hospital beat -l INFO --scheduler django_celery_beat.schedulers:DatabaseScheduler
; The directory to your Django project
directory=/home/itadmin/spider_hospital
; If supervisord is run as the root user, switch users to this UNIX user account
; before doing any processing.
user=itadmin
; Supervisor will start as many instances of this program as named by numprocs
numprocs=1
; Put process stdout output in this file
stdout_logfile=/var/log/celery/celery_beat.log
; Put process stderr output in this file
stderr_logfile=/var/log/celery/celery_beat.log
; If true, this program will start automatically when supervisord is started
autostart=true
; May be one of false, unexpected, or true. If false, the process will never
; be autorestarted. If unexpected, the process will be restart when the program
; exits with an exit code that is not one of the exit codes associated with this
; process’ configuration (see exitcodes). If true, the process will be
; unconditionally restarted when it exits, without regard to its exit code.
autorestart=true
; The total number of seconds which the program needs to stay running after
; a startup to consider the start successful.
startsecs=10
stopwaitsecs = 60
; if your broker is supervised, set its priority higher
; so it starts first
priority=999
Now, to run supervisor run:
$ sudo supervisord
It will load the .conf files we just created.
To start/stop and status use the following commands:
$ sudo supervisorctl stop celery-default
$ sudo supervisorctl start celery-default
$ sudo supervisorctl status celery-default