Deploy Django-Q to an EC2 instance with Elastic Beanstalk

django-q-cluster.sh

container_commands:
  01_makemigrations:
    command: "python manage.py makemigrations"
    leader_only: true
  02_migrate:
    command: "python manage.py migrate"
    leader_only: true
  # Patch the current version, circular imports cause an ImportError (see https://github.com/Koed00/django-q/issues/331)
  03_patch_django_q:
    # Replace line 12 ('from django_q.cluster import worker, monitor') by 'from django_q.cluster import *'
    command: "sed -i '12s/.*/from django_q.cluster import */' tasks.py"
    # Make sure it's the right path
    cwd: "/opt/python/run/venv/lib/python3.6/site-packages/django_q"
  04_mkdir_for_log_and_pid:
    command: "mkdir -p /var/log/djangoq/ /var/run/djangoq/"
  05_djangoq_configure:
    command: "cp .ebextensions/django-q-cluster.sh /opt/elasticbeanstalk/hooks/appdeploy/post/ && chmod 744 /opt/elasticbeanstalk/hooks/appdeploy/post/django-q-cluster.sh"
    cwd: "/opt/python/ondeck/app"
  06_djangoq_run:
    command: "/opt/elasticbeanstalk/hooks/appdeploy/post/django-q-cluster.sh"

option_settings:
  aws:elasticbeanstalk:container:python:
    WSGIPath: "{{ YOUR APP NAME }}/wsgi.py"

django.config

#!/usr/bin/env bash

# This script was made for Celery, modified it for Django-Q.
# Put both files in your .ebextensions folder.

# Get django environment variables
djangoqenv=`cat /opt/python/current/env | tr '\n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g'`
djangoqenv=${djangoqenv%?}

# Create djangoq configuraiton script
djangoqconf="[program:django-q]
command=/opt/python/run/venv/bin/python manage.py qcluster
directory=/opt/python/current/app
user=nobody
numprocs=1
stdout_logfile=/var/log/djangoq/worker.log
stderr_logfile=/var/log/djangoq/worker.log
autostart=true
autorestart=true
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
stopwaitsecs = 600

; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true
stopasgroup=true

environment=$djangoqenv
"

# Create the djangoq supervisord conf script
echo "$djangoqconf" | tee /opt/python/etc/djangoq.conf

# Add configuration script to supervisord conf (if not there already)
if ! grep -Fxq "[include]" /opt/python/etc/supervisord.conf
  then
  echo "[include]" | tee -a /opt/python/etc/supervisord.conf
  echo "files: djangoq.conf" | tee -a /opt/python/etc/supervisord.conf
fi

# Reread the supervisord config
/usr/local/bin/supervisorctl -c /opt/python/etc/supervisord.conf reread

# Update supervisord in cache without restarting all services
/usr/local/bin/supervisorctl -c /opt/python/etc/supervisord.conf update

# Start/Restart djangoqd through supervisord
/usr/local/bin/supervisorctl -c /opt/python/etc/supervisord.conf restart django-q

I think the name of files are reversed.

yes the names are reversed.

Also, in django-q-cluster.sh (the correct one), I had change the "command" to activate the venv. So it was like:

command=/opt/python/run/venv/bin/python /opt/python/current/app/{projectname}/manage.py qcluster

it looks like this: Q_CLUSTER = { 'name': 'LMSQueue', 'workers': 4, 'timeout': 90, 'retry': 120, 'queue_limit': 50, 'bulk': 10, 'max_attempts' : 4, 'orm': 'default' }

…

On Wed, Mar 24, 2021 at 2:23 AM Steven Flory ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ @simo97 <https://github.com/simo97> @hunainkapadia <https://github.com/hunainkapadia> when you got this successfully working, what did your Q_Cluster dict look like in settings.py for the production deploy? I have it working locally, but can't connect to redis on the prod deployment — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://gist.github.com/0e11ce6d585b3290b15a9ad163b9aa06#gistcomment-3677306>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABF3DRAVG3NIMCECWNSYJSDTFEBDXANCNFSM4QVUQMGQ> .

-- Hunain Kapadia

@hunainkapadia thanks for the reply. My problem was the path in the command alteration. My manage.py file was located in the root of my project, so didn't need the {projectname} in the path