heroku_delayed_job.rb

## /config/initializers/dynamic_job.rb

require 'heroku'

# base class for all jobs that you wish to automatically scale and go down in Heroku
class DynamicJob
    #set a cap on maximum number of users ever - just in case. 
    MAX_CONCURRENT_WORKERS = 100
    
    def initialize
      #assumption: the job class is new-ed up before enqueueing, which is always the case
      workers_needed = [Delayed::Job.count + 1, MAX_CONCURRENT_WORKERS].min
      if(ENV["RAILS_ENV"] == "production")
          username = ENV['HEROKU_USERNAME']
          password = ENV['HEROKU_PASSWORD']
          app     = ENV['HEROKU_APP']
          client = Heroku::Client.new(username, password)
          client.set_workers(app, workers_needed) 
      end
    end
    
    # call this from the derived class when you are done.
    def clean_up
      #assumption: this method is always called from within a derived job class
      workers_needed = [Delayed::Job.count - 1, 0].max
      if(ENV["RAILS_ENV"] == "production")
        username = ENV['HEROKU_USERNAME']
        password = ENV['HEROKU_PASSWORD']
        app     = ENV['HEROKU_APP']
        client = Heroku::Client.new(username, password)
        client.set_workers(app, workers_needed)
      end
    end
end

#example job class LoadImagesJob
class LoadImagesJob < DynamicJob
  def initialize topic_id
    @topic = Topic.find(topic_id)
    #remember to call super() with parentheses to call superclass initializer without    parameters
    super()
  end
  
  def perform
    @topic.load_images! 
    clean_up
  end
end

#example enqueue
job = LoadImageJob.new topic.id  # => calls some.do_something later
Delayed::Job.enqueue(job)

Um, it looks like you are not decrementing correctly. If I had 500 jobs, initially I'd have only 100 workers, but it'd increase it to 499 after the first one completed. Also, do you know if Heroku is smart about which workers are terminated when decreasing the count? Or, will it pick workers at random, which may be in the middle of processing?

Yes, the value must be configured to be larger than anything you expect to see in production. The config is provided only for the off chance that a bug causes workers to never be decremented and we want to limit the number of workers so we dont lose our savings:)
Afaik, heroku will try to release workers gracefully, and will then send a force abort after some time. I dont remember the details, but I think an unfinished job whose worker is killed gets picked up by a free worker anyway, so it doesnt matter if an active worker is dropped. Lot of this is based on my testing a few weeks ago, so it may change.if heroku changes.