Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Celery example with tasks of varying length + -Ofair
# tested with celery[redis]==3.1.17
# to run with default configuration -- tasks will take 14 seconds to complete the 20 tasks in start_all() below
celery worker -A cluster_project.celery_app -Q tester -lINFO --concurrency=4
# to run with -Ofair -- tasks will take 10 seconds to complete
celery worker -A cluster_project.celery_app -Q tester -lINFO --concurrency=4 -Ofair
import time
from cluster_project import celery_app
# To run this contrived example:
# 1. start worker with -Ofair or not
# 2. call start_all() and watch celery worker output
def start_all():
slow_task.delay()
for i in range(19):
fast_task.delay(i)
@celery_app.task(queue='tester')
def slow_task():
print 'starting slow task'
time.sleep(10)
print 'done with slow task'
@celery_app.task(queue='tester')
def fast_task(i):
print 'starting fast task %d' % i
time.sleep(1)
print 'done with fast task %d' % i
@sohamnavadiya

This comment has been minimized.

Copy link

@sohamnavadiya sohamnavadiya commented Aug 7, 2018

What is @celery_app here. Can you share it also. it would be great for me.

@sr105

This comment has been minimized.

Copy link

@sr105 sr105 commented Jan 25, 2019

Might be nice to change the imports as follows so it's easier for anyone to run:

from celery import Celery

celery_app = Celery('tasks', broker='redis://localhost')

and run as celery worker -A tasks ...

@w00lf

This comment has been minimized.

Copy link

@w00lf w00lf commented Mar 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment