Skip to content

Instantly share code, notes, and snippets.

@virolea
Created April 25, 2023 19:29
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save virolea/4045781114c834b0b56eda5aa1591685 to your computer and use it in GitHub Desktop.
Save virolea/4045781114c834b0b56eda5aa1591685 to your computer and use it in GitHub Desktop.
Code for the "A simple and pragmatic approach to fixing a memory leak" talk at RailsConf 2023
# This middleware dynamically switch a job queue based on its class name.
module TroubleshootMemoryMiddleware
module Client
class CustomQueueName
QUARANTINE_QUEUE = "quarantine"
def call(worker_class, job, queue, redis_pool)
# Jobs enqueued without ActiveJob are not wrapped.
job_class_name = if job["wrapped"].present?
job["wrapped"].name
else
job["class"]
end
quarantined_jobs = parse_quarantined_jobs
if quarantined_jobs.include?(job_class_name)
job["queue"] = QUARANTINE_QUEUE
end
yield
end
private
def parse_quarantined_jobs
if ENV["QUARANTINED_JOBS"].present?
ENV["QUARANTINED_JOBS"].split(";")
else
[]
end
end
end
end
end
# Scrip to get the jobs enqueued in the `default` queue.
# put this in your Rails repo and
# run it: rails runner jobs_list.rb
# jobs_list.rb
filepaths = Dir.glob("app/jobs/**/*.rb")
ActiveSupport::Inflector.inflections(:en) do |inflect|
inflect.acronym "EHawk"
end
jobs_queued_as_defaults = []
filepaths.each do |filepath|
_, job_path = filepath.split("app/jobs/")
class_name = job_path
.gsub(".rb", "")
.split("/")
.map { |s| ActiveSupport::Inflector.camelize(s, true) }
.join("::")
job_class = ActiveSupport::Inflector.constantize(class_name)
# Some jobs do not inherit from `ApplicationJob` like Periodic jobs, and do not respond to `queue_name`.
# Their queue is default
if !job_class.respond_to?(:queue_name) || (job_class.respond_to?(:queue_name) && job_class.queue_name == "default")
jobs_queued_as_defaults << job_class
end
end
puts jobs_queued_as_defaults
# config/initializers/sidekiq.rb
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
chain.add TroubleshootMemoryMiddleware::Client::CustomQueueName
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment