Skip to content

Instantly share code, notes, and snippets.

@termie
Last active March 31, 2017 09:40
Show Gist options
  • Save termie/b9e1e9a751f25ae40fa6486f40cefe2b to your computer and use it in GitHub Desktop.
Save termie/b9e1e9a751f25ae40fa6486f40cefe2b to your computer and use it in GitHub Desktop.
is there a job queue that lets me encode these semantics?
Is there a job queue system (or a job queue system plus an approach to using it) that lets
me only process jobs with a maximum concurrency based on some property of the job?
The workload looks like
U1, J1
U1, J2
U2, J3
U1, J4
U1, J5
U3, J6
U3, J7
U3, J8
U2, J9
The runners are
R1, R2, R3, R4 - runs jobs from all users not U3
R5-R10 - runs jobs from U3 only
The user U1 and U2 can have two jobs running concurrently (so as not to denial of service each other), U3 can have 6.
We can set aside U3 for a little bit because they have some separate semantics and can be their
own topic/queue and are known ahead of time (U1 and U2 are "generic" jobs).
The goal is to have the runners pick up the following jobs (starting from the talk of the queue):
R1: U1, J1
R2: U1, J2,
R3: U2, J3,
R4: U2, J9
R5+: U3, J*
Leaving the following in the queue to get picked up once U1's jobs free up
U1, J4
U1, J5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment