Skip to content

Instantly share code, notes, and snippets.

@stevenharman
Last active March 6, 2020 17:35
Show Gist options
  • Star 20 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save stevenharman/8590937 to your computer and use it in GitHub Desktop.
Save stevenharman/8590937 to your computer and use it in GitHub Desktop.
The best solution I've found for reliably configuring Sidekiq's ActiveRecord connection pool size on Heroku.
# /config/initializers/sidekiq.rb
current_web_concurrency = Proc.new do
web_concurrency = ENV['WEB_CONCURRENCY']
web_concurrency ||= Puma.respond_to?
(:cli_config) && Puma.cli_config.options.fetch(:max_threads)
web_concurrency || 16
end
local_redis_url = Proc.new do
config_file = File.join(File.expand_path('../..', __FILE__), 'redis.conf')
port = File.read(config_file).chomp.split.last
"redis://localhost:#{port}"
end
ENV['REDISTOGO_URL'] ||= local_redis_url.call
Sidekiq.configure_server do |config|
config.redis = { url: ENV['REDISTOGO_URL'], namespace: 'brewdega' }
Rails.application.config.after_initialize do
ActiveRecord::Base.connection_pool.disconnect!
ActiveSupport.on_load(:active_record) do
config = Rails.application.config.database_configuration[Rails.env]
config['reaping_frequency'] = ENV['DATABASE_REAP_FREQ'] || 10 # seconds
config['pool'] = ENV['WORKER_DB_POOL_SIZE'] || Sidekiq.options[:concurrency]
ActiveRecord::Base.establish_connection(config)
Rails.logger.info("Connection Pool size for Sidekiq Server is now: #{ActiveRecord::Base.connection.pool.instance_variable_get('@size')}")
end
end
end
Sidekiq.configure_client do |config|
config.redis = { url: ENV['REDISTOGO_URL'], namespace: 'brewdega', :size => 1 }
Rails.application.config.after_initialize do
ActiveRecord::Base.connection_pool.disconnect!
ActiveSupport.on_load(:active_record) do
config = Rails.application.config.database_configuration[Rails.env]
config['reaping_frequency'] = ENV['DATABASE_REAP_FREQ'] || 10 # seconds
config['pool'] = ENV['WEB_DB_POOL_SIZE'] || current_web_concurrency.call
ActiveRecord::Base.establish_connection(config)
# DB connection not available during slug compliation on Heroku
Rails.logger.info("Connection Pool size for web server is now: #{config['pool']}")
end
end
end
@euge
Copy link

euge commented Aug 14, 2014

hey thanks for this! I ran into an issue with config['pool'] = ENV['WORKER_DB_POOL_SIZE'] || Sidekiq.options[:concurrency]. I found that you want to allocate at least 1 more than what Sidekiq's concurrency is set to. Seems like Sidekiq server takes 1 connection itself and then each worker will also. So I found that at least one more connection than the concurrency does the trick.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment