Patch HireFire so worker auto-scaling works on the Beta Heroku Cedar stack

heroku_cedar_hirefire_patch.rb

# hopefully a temporary patch so HireFire will run on the Heroku Cedar stack
# NB: the worker type is hard-coded as "worker" below, this must correlate to the type in Procfile
# NB: ENV['APP_NAME'] must be defined (e.g. 'heroku config:add APP_NAME=myherokuappname')

# using ps & ps_scale instead of info & set_workers
class HireFire::Environment::Heroku
  
  private

  def workers(amount = nil)
    
    if amount.nil?
      # return client.info(ENV['APP_NAME'])[:workers].to_i
      return client.ps(ENV['APP_NAME']).select {|p| p['process'] =~ /worker.[0-9]+/}.length
    end

    # client.set_workers(ENV['APP_NAME'], amount)
    return client.ps_scale(ENV['APP_NAME'], {"type" => "worker", "qty" => amount})

  rescue RestClient::Exception
    HireFire::Logger.message("Worker query request failed with #{ $!.class.name } #{ $!.message }")
    nil
  end
end

# quick override to ensure that HireFire is activated on Heroku. The change is that it is looking for
# ::Rails.env.production? instead of ENV.include?('HEROKU_UPID')
module HireFire
  module Environment

    module ClassMethods
      
      def environment
        @environment ||= HireFire::Environment.const_get(
          if environment = HireFire.configuration.environment
            environment.to_s.camelize
          else
            ::Rails.env.production? ? 'Heroku' : 'Noop'
          end
        ).new
      end
      
    end
  end
end

Are you placing this in config/intializers?

Yeah. config/initializers/patches. Scaling down is working, but I don't think scaling up is, and I haven't looked into why.

I'm having no problems with this patch and workers scaling up and down. I'm using a standard DelayedJob setup on rails 3.1.

Thanks @navyrain - after a little digging, I think the reason mine have stopped scaling up is related to this issue, where Resque workers on Heroku are not 'pruned' https://github.com/defunkt/resque/issues/319

@thinkgareth - I see you've had the same issues, and suggested Resque.workers.each do |w| w.unregister_worker end to prune the workers. Have you worked this into a patch that gets HireFire back to being fully automated on Heroku with Resque? Thanks.

For the time being, I've added Resque.workers.each do |w| w.unregister_worker end to my app to clear all workers before creating new jobs. This is OK in this particular app as it only creates new jobs when there are no jobs in the queue. Thanks for the code snippet @thinkgareth

Thanks - this worked really well. Can I ask a seemingly dumb question - how do you tell when all workers have been fired? Hirefire does say that it is firing all of my workers, but when I run heroku scale, it still says worker+1 http://cl.ly/0H1g441f2T1o1Y0C350g

Hi @ptagell use heroku ps to see what's running. heroku scale is for changing the number of dynos, that is a usage example you are referring to.

Cheers! Will do

…

On 27/12/2011, at 12:31 AM, Joe ***@***.*** wrote: Hi @ptagell use `heroku ps` to see what's running. `heroku scale` is for changing the number of dynos, that is a usage example you are referring to. --- Reply to this email directly or view it on GitHub: https://gist.github.com/1082673

Is this still up to date? I can't seem to get it working...