gistfile1.txt

> The benefit to multiple intent-based queues is that the tasks are separated from each other in a logical sense.

This makes sense we plan to throttle/scale up each individual queue.

Priyank: Thanks

> Get better view of which part of the system is experiencing more load.

the worker can make the node go down & more queues will get clogged.

Priyank: Yep. 100% true. For the critical queues, we'll add workers in multiple queues.
And later we can try using max-memory-per-child to keep the RAM usage in check for workers that are listening
to queues that have RAM consuming messages. Ex, voice_reports_queues, voice_imports_queue

> Better decoupling of tasks and queues.
(Ex, if a queue has tasks that are logically the same, they are decoupled too. Long reports will not interfere with batch processing task).

by the same worker brings node down logic, it will interfere

Priyank: Yep, true.
The advantage will be that if we can take that queue and move its worker to a dedicated node. 
Ex, during closing time, we can move the reports_queue worker to a separate machine with no RAM limits.

> Even within a queue, tasks can have different priorities. 

this has to be programmed right?

Priyank: Yep. The @task will get an argument.

questions:
1. what about redundancy? I dont want my imp queue to have workers only on 1 node
Priyank: We have limited scope here. But for critical queues, we'll try to distribute among nodes.
Ideally each queue should have workers in atleast 2 nodes.

2. there would be disproportionate division of workers between CPU bound and IO bound, we'll have to allocate infra based on this
Priyank: Yep, based on need we'll evolve the workers infra too.

3. please let OU/Google translate/etc know if they need to whitelist new IPs
Priyank: Not doing anything to the data side.

4. idempotency doesn't seem to be covered ()
Priyank: This is application code right? How's the messaging infra supposed to play a part here.

5. SUPPOSE A SCENARIO - DB is under 90% usage. Now would I be able to control DB usage by killing workers of a queue?
Priyank: Yep. That's an added advantage.
And since we have a reliable messaging system, we can create design that doesn't need to write everything on the DB.