Last active
March 29, 2023 21:40
-
-
Save lalatgithub/909ed6d38ed50393d9807a5cc066ac67 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Python RQ | Celery | |
|---|---|---|
| Purely based on Redis | works with Redis Rabbitmq SQS and Zookeeper (experimental) | |
| RQ does not use an advanced broker to do the message routing | Celery does this thing very well | |
| Does not speak a portable protocol since it depends on pickle to serialize the jobs so its a Python only system | supports json yaml pickle msgpack which makes it able to produce task from any programming language | |
| Workers can also be started in burst mode to finish all currently available work and quit as soon as all given queues are emptied | Nothing similar in Celery | |
| only run on systems that implement fork(). Each worker will process a single job at a time. Within a worker there is no concurrent processing going on. If you want to perform jobs concurrently simply start more workers (using supervisor or systemd) | Celery works with fork gevent eventlet and solo and does auto scaling process within a single worker process | |
| RQ need supervisor to scale up processes(workers) | Celery has built-in option to auto scale processes |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment