Why merge backburner with quebert?

sketch.md

Assessment of Quebert & Backburner to figure out how to merge the two.

Quebert

1. Build out an async proxy in place of #async_send so that TTR, delays, and priority can be specified in a cleaner way. This looks something like User.new.async(delay: 3).send(:notify_user) as opposed to the current User.new.async_send(:notify_user, :quebert => {delay: 3, pri: 1000})
2. Quebert supports many back-ends: InProcess, Sync, and Beanstalk. Sync and InProcess were created to aid with testing assertions so that Quebert.backend.queue.should have(1).items is possible. It only needs Beanstalk, so the multiple-backends could be removed for the sake of simplicity. A stub could be provided for peeps that want to assert jobs are thrown on the queue in a test env.
3. There's a controller layer between the Job and the Backend that communicates using exceptions, which is suppose to deal with multiple backends. Eww. I want to just pass a controller into the job and have it deal.

Backburner

Some stuff I'd change:

1. I'm not convinced that the Job mix-in is the best approach. A job should be a class itself. Mixin's make sense as syntatical sugar to make it easier to queue an instance of a class into a job, and from pulling the job off the queue and shoving it back into that class for processing.
2. I don't think having different queues per app is a good idea in a beanstalkd world. Resque does this because there's no concept of priorities in Redis. Since Beanstalkd lets people specify priories of jobs, its a moot point to run different numbers of workers for different queue names. Put more emphasis on priority to deal with this.

Stuff that's good:

1. Runners are much more sophisticated than Quebert (forking runner, etc)
2. The name.
3. The hooks, though I'd tweak the implementation of this a bit more.

Features that would be awesome

1. We had a mis-numbered priority in production once that brought our system down. We'd like to have "named" priorities that are stack-ranked. This is a simple DSL that looks like: Backburner.config.priorities = [:high, :medium, :custom_pri, :low], which is mapped to the int values that beanstalkd understands. When tossing a job on the queue, a named priority could be specified like User.new.async(pri: :high).blah.
2. Web based GUI admin tool. We could pull this in from a Ruby Sinatra beanstalkd admin tool with little effort. Would want to re-brand it with backburner so that it feels more put together.
3. CLI for querying/filtering jobs for performing batch operations on jobs that are in the queue. This is important for when things go bad in production and certain jobs with certain payloads may need to be buried until a patch can be pushed to prod and the jobs are re-run. Web GUI might be able to use this CLI interface.

Links to the issues relating to your comments:

Quebert supports many back-ends: InProcess, Sync, and Beanstalk. Sync and InProcess were created to aid with testing assertions so that Quebert.backend.queue.should have(1).items is possible. It only needs Beanstalk, so the multiple-backends could be removed for the sake of simplicity. A stub could be provided for peeps that want to assert jobs are thrown on the queue in a test env.

nesquena/backburner#25

I'm not convinced that the Job mix-in is the best approach. A job should be a class itself. Mixin's make sense as syntatical sugar to make it easier to queue an instance of a class into a job, and from pulling the job off the queue and shoving it back into that class for processing.

nesquena/backburner#26

I don't think having different queues per app is a good idea in a beanstalkd world. Resque does this because there's no concept of priorities in Redis. Since Beanstalkd lets people specify priories of jobs, its a moot point to run different numbers of workers for different queue names. Put more emphasis on priority to deal with this.

nesquena/backburner#27

We had a mis-numbered priority in production once that brought our system down. We'd like to have "named" priorities that are stack-ranked. This is a simple DSL that looks like: Backburner.config.priorities = [:high, :medium, :custom_pri, :low], which is mapped to the int values that beanstalkd understands. When tossing a job on the queue, a named priority could be specified like User.new.async(pri: :high).blah.

nesquena/backburner#23

Web based GUI admin tool. We could pull this in from a Ruby Sinatra beanstalkd admin tool with little effort. Would want to re-brand it with backburner so that it feels more put together.

nesquena/backburner#11

CLI for querying/filtering jobs for performing batch operations on jobs that are in the queue. This is important for when things go bad in production and certain jobs with certain payloads may need to be buried until a patch can be pushed to prod and the jobs are re-run. Web GUI might be able to use this CLI interface.

nesquena/backburner#24