Skip to content

Instantly share code, notes, and snippets.

@adamcooper
Created November 8, 2010 17:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save adamcooper/667942 to your computer and use it in GitHub Desktop.
Save adamcooper/667942 to your computer and use it in GitHub Desktop.
Process multi-jobs per fork of resque
Resque.after_fork do |job|
jobs_performed = 0
kill_fork_at = Time.now.to_i + (ENV['MINUTES_PER_FORK'].to_i * 60)
worker = job.worker
first_job = job
worker.procline "Processing #{job.queue} since #{Time.now.to_i} until #{kill_fork_at}"
first_job.perform
# rely on parent error handling
worker.log "done: #{first_job.inspect}"
worker.done_working
jobs_performed += 1
while Time.now.to_i < kill_fork_at && !worker.shutdown?
if job = worker.reserve
worker.working_on(job)
job.perform
# rely on parent error handling
worker.log "done: #{job.inspect}"
worker.done_working
jobs_performed += 1
else
sleep(1)
end
end
first_job.instance_eval("def perform; end")
end
@mguterl
Copy link

mguterl commented Mar 3, 2011

We're now using resque-multi-job-forks

@tispratik
Copy link

Thanks mguterl !

@tispratik
Copy link

i used resque-multi-job-forks, but the workers quit when all the jobs are completed.
$ MINUTES_PER_FORK=1 COUNT=2 QUEUE=db rake resque:workers
(in /home/my_app)
(in /home/my_app)
The signal QUIT is in use by the JVM and will not work correctly on this platform
The signal USR1 is in use by the JVM and will not work correctly on this platform
The signal QUIT is in use by the JVM and will not work correctly on this platform
The signal USR1 is in use by the JVM and will not work correctly on this platform
155 were processed in this fork
245 were processed in this fork

$ _

Do you know why could that be?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment