Skip to content

Instantly share code, notes, and snippets.

@maxivak
Last active October 29, 2024 13:12
Show Gist options
  • Save maxivak/690e6c353f65a86a4af9 to your computer and use it in GitHub Desktop.
Save maxivak/690e6c353f65a86a4af9 to your computer and use it in GitHub Desktop.
Sending emails with ActionMailer and Sidekiq

Sending emails with ActionMailer and Sidekiq

Send email asynchroniously using Sidekiq.

ActionMailer

Create your mailer us usual:

# app/mailers/users_mailer.rb

class UsersMailer < ActionMailer::Base

def welcome_email(user_id)
    @user = User.find(user_id)

    mail(   :to      => @user.email,
            :subject => "Welcome"
    ) do |format|
      format.text
      format.html
    end
  end
end

Views for email:

app/views/users_mailer/welcome_email.html.erb - HTML version
app/views/users_mailer/welcome_email.text.erb - TEXT version

Send email:

user = User.find(1)

mail = UsersMailer.welcome_email(user.id)
#mail.deliver_now
mail.deliver_later

    

Sidekiq

Gemfile:

gem 'sidekiq'

Install Redis

Redis provides data storage for Sidekiq. It holds all the job data along with runtime and historical data

Configure Sidekiq

To make #deliver_later work we need to tell ActiveJob to use Sidekiq. As long as Active Job is setup to use Sidekiq we can use #deliver_later.

# config/initializers/active_job.rb
  
  config.active_job.queue_adapter = :sidekiq

Environment file:

# config/environments/development.rb

Rails.application.configure do
  ...
  
  config.active_job.queue_adapter = :sidekiq
  
  config.action_mailer.perform_deliveries = true
  config.action_mailer.raise_delivery_errors = true

  config.action_mailer.delivery_method = :smtp
  
  config.action_mailer.smtp_settings = { ... }
  

end

Read more about ActionJob and Sidekiq: https://github.com/mperham/sidekiq/wiki/Active-Job

Configure Sidekiq

config/sidekiq.yml:

---
:concurrency: 1
:queues:
  - default
  - mailers

Specify Redis namespace for different environments:

# config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { url: 'redis://localhost:6379/0', namespace: "app3_sidekiq_#{Rails.env}" }
end

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://localhost:6379/0', namespace: "app3_sidekiq_#{Rails.env}" }
end

Sidekiq and ActionMailer (ActionJob)

By default, jobs to deliver emails will be placed in queue named :mailers. To change queue name for ActionMailer use this config:

# config/environments/development.rb

  config.active_job.queue_adapter = :sidekiq

  config.active_job.queue_name_prefix = "mysite"
  config.active_job.queue_name_delimiter = "_"

This will use queues named :mysite_mailers, etc.

!!! important. You may need to include new queue names in sidekiq.yml file:

# config/sidekiq.yml

---
:concurrency: 1
:queues:
  - default
  - mailers

  - mysite_default
  - mysite_mailers

Sidekiq and Devise

Read this: https://github.com/mperham/sidekiq/wiki/Devise

Run Sidekiq

bundle exec sidekiq --environment development -C config/sidekiq.yml 

God + Sidekiq

Use god for monitoring and running sidekiq automatically: https://gist.github.com/maxivak/05847dc7f558d5ef282e

RSpec tests

RSpec tests for ActionMailer

In these tests we do not use Sidekiq.

test environment:

# config/environments/test.rb

Rails.application.configure do
 ...
 
  config.active_job.queue_adapter = :test

  config.action_mailer.perform_deliveries = true
  config.action_mailer.delivery_method = :test
  config.action_mailer.raise_delivery_errors = true

 
 

Test that deliver_later method was called:

user = User.first

    #
    message_delivery = instance_double(ActionMailer::MessageDelivery)
    expect(UsersMailer).to receive(:welcome_email).with(user.id).and_return(message_delivery)
    expect(message_delivery).to receive(:deliver_later)
    

#   
mail = UsersMailer.welcome_email(user.email)
mail.deliver_later
    
    

RSpec tests for Sidekiq

test environment:

# config/environments/test.rb

Rails.application.configure do
 ...
   
  config.active_job.queue_adapter = :sidekiq

  config.action_mailer.perform_deliveries = true
  config.action_mailer.delivery_method = :test
  config.action_mailer.raise_delivery_errors = true

 

spec/rails_helper.rb:

# sidekiq
require 'sidekiq/testing'
Sidekiq::Testing.fake!  # by default it is fake

User Sidekiq::Worker.jobs.size to see the number of jobs in the queue.

Test that email was enqueued:


RSpec.describe "Test sending email with sidekiq", :type => :request do

  it 'send email to sidekiq' do

    user = User.first
      
    expect{
      UsersMailer.welcome_email(user.id).deliver_later
    }.to change( Sidekiq::Worker.jobs, :size ).by(1)

  end

end

@rgaufman
Copy link

Is there anyway to make sidekiq retry sending the email 5 times before throwing an exception? - I currently get 2-3 exceptions per day that are:

Net::ReadTimeoutActionMailer::MailDeliveryJob@default
gems/net-protocol-0.1.3/lib/net/protocol.rb:219 rbuf_fill

They seem harmless and happen when Google servers are busy, a retry fixes it but they are clocking up Bugsnag.

@abhchand
Copy link

Great guide!

I had an issue where the jobs were always getting queued in the :default queue and not :mailers. I'm not sure why, but I had to explicitly set the mailer queue, even though ActionMailer defaults it internally to :mailers

# application.rb
config.action_mailer.deliver_later_queue_name = :mailers

Secondly, I also wanted an easier way to test which mailers were enqueued in RSpec tests. I use Sidekiq + RSpec and came up with this helper method. The description is probably a bit too wordy, but I wanted to be explicit about what it's doing.

This just returns a hash of mailers that are enqueued so you can easily access the mailer name and args.

usage:

expect do
  Devise::Mailer.confirmation_instructions(User.last, 'abcdefg', {}).deliver_later
end.to change { enqueued_mailers.count }.by(1)

email = enqueued_mailers.last
expect(email[:klass]).to eq(Devise::Mailer)
expect(email[:mailer_name]).to eq(:confirmation_instructions)
expect(email[:args][:record]).to eq(User.last)

Method source:

# When a mailer is enqueued with `#deliver_later`, it generates an
# ActionMailer job (which implements the ActiveJob interface).
#
# To be able to enqueue this job onto Sidekiq, it has to be further wrapper
# in a Sidekiq Job Wrapper that will enqueue it with keys that sidekiq
# expects, like `jid`, `retry`, etc...
#
# For example, here's the result of calling
#
#   > Devise::Mailer.confirmation_instructions(
#       User.find(108),
#       'pU8s2syM1pYN523Ap2ix',
#       {}
#     ).deliver_later
#
#   > Sidekiq::Worker.jobs
#   => [
#     {
#       "class"=>"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper",
#       "wrapped"=>"ActionMailer::MailDeliveryJob",
#       "queue"=>"mailers",
#       "args"=>
#         [{
#           "job_class"=>"ActionMailer::MailDeliveryJob",
#           "job_id"=>"17464385-ed14-4490-ab10-a0770870c169",
#           "provider_job_id"=>nil,
#           "queue_name"=>"mailers",
#           "priority"=>nil,
#           "arguments"=>[
#             "Devise::Mailer",
#             "confirmation_instructions",
#             "deliver_now",
#             {
#               "args"=>[
#                 {"_aj_globalid"=>"gid://familyties/User/108"},
#                 "pU8s2syM1pYN523Ap2ix",
#                 {"_aj_symbol_keys"=>[]}
#               ],
#               "_aj_ruby2_keywords"=>["args"]
#             }
#           ],
#           "executions"=>0,
#           "exception_executions"=>{},
#           "locale"=>"en",
#           "timezone"=>"UTC",
#           "enqueued_at"=>"2023-05-31T15:31:34Z"
#         }],
#       "retry"=>true,
#       "jid"=>"0c6ebfceee4cddc9ccd557b4",
#       "created_at"=>1685547094.2727168,
#       "enqueued_at"=>1685547094.2728298
#     }
#   ]
#
# This nested structure can be inconvenient for testing, so the below
# method deserializes this information and produces a simple hash that
# can be used for testing
#
#   expect do
#     post :create, params: params
#   end.to change { enqueued_mailers.count }.by(1)
#
#   email = enqueued_mailers.last
#   expect(email[:klass]).to eq(Devise::Mailer)
#   expect(email[:mailer_name]).to eq(:confirmation_instructions)
#   expect(email[:args][:record]).to eq(User.last)
def enqueued_mailers
  Sidekiq::Worker.jobs.map do |job|
    # `Sidekiq:Worker.jobs` returns all jobs. We only want to filter on those
    # in our action_mailer queue
    queue_name =
      Rails.application.config.action_mailer.deliver_later_queue_name
    next unless job['queue'] == queue_name.to_s

    mailer_klass, mailer_name, _method_name, args =
      ActiveJob::Arguments.deserialize(job['args'][0]['arguments'])

    mailer_klass = mailer_klass.constantize
    params =
      mailer_klass.instance_method(mailer_name).parameters.map { |p| p[1] }

    args = args[:args]

    enqueued_at = Time.zone.at(job['enqueued_at']) if job['enqueued_at']
    at = Time.zone.at(job['at']) if job['at']

    {
      klass: mailer_klass,
      mailer_name: mailer_name.to_sym,
      args: Hash[params.zip(args)],
      enqueued_at: enqueued_at,
      at: at
    }
  end.compact
end

@rgaufman
Copy link

rgaufman commented Jun 15, 2023

To get proper error handling and control over retries, I ended up doing this:

class MyMailerWorker
  include Sidekiq::Worker
  sidekiq_options queue: :mailer, retry: 2, backtrace: true

  def perform(location_id, event_key, name = nil, event_id = nil)
    # Setup code
    begin
      MyMailer.welcome(params).deliver_now
    rescue StandardError => e
      # Error handling code
    end
  end
end

Not sure if there is a better solution.

@eclectic-coding
Copy link

Thanks for this great reference.

Note that redis-namespace is no longer supported. You might want to update that section.

Sidekiq.configure_client do |config|
  config.redis = { db: 1 }
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment