Disque jobs are uniquely identified by an ID like the following:
DI0f0c644fd3ccb51c2cedbd47fcb6f312646c993c05a0SQ
Job IDs always start with "DI" and end with "SQ" and are always composed of exactly 48 characters.
We can split an ID into multiple parts:
DI | 0f0c644f | d3ccb51c2cedbd47fcb6f312646c993c | 05a0 | SQ
- DI is the prefix
- 0f0c644f is the first 8 bytes of the node ID where the message was generated.
- d3ccb51c2cedbd47fcb6f312646c993c is the 128 bit ID pesudo random part in hex.
- 05a0 is the Job TTL in minutes. Because of it, message IDs can be expired safely even without having the job representation.
- SQ is the suffix.
IDs are returned by ADDJOB when a job is successfully created, are part of the GETJOB output, and are used in order to acknowledge that a job was correctly processed by a worker.
Part of the node ID is included in the message so that a worker processing messages for a given queue can easily guess what are the nodes where jobs are created, and move directly to these nodes to increase efficiency instead of listeing for messages in a node that will require to fetch messages from other nodes.
Only 64 bits of the original node ID is included in the message, however in a cluster with 1000 Disque nodes, the probability of two nodes to have identical 64 bit ID prefixes is given by the birthday paradox:
P(100,2^64) = .000000000000027
In case of collisions, the workers may just do a non-efficient choice.