Skip to content

Instantly share code, notes, and snippets.

@gauravano gauravano/batch_digest.md Secret
Last active Mar 26, 2018

Embed
What would you like to do?
PART 2 : Weekly batch digest

Daily "batch digest" email subscriptions using Rails ActiveJob

Goal:

Implementation of feature which automatically sends each user digest on the basis of user's subscriptions.

💎 Gem used : Resque ,Whenever

Implementation Steps:-

1. Creation of method in subscription_mailer.rb for sending email to the user. Example:

  def sending_digest(user)
     @notes = user.content_followed_in_period(Time.now - 1.week, Time.now) # for weekly digest 
     @user = user
     mail(to: user.email, subject: "Your Weekly digest").deliver
  end

In user.rb ,there is method content_followed_in_period(start time, end time) which takes two arguments start time and end time, this function gives us nodes corresponding to the subscribed tag which were updated or created during the specified time interval.Then, nodes would be displayed in the email.

2. Creating sending_digest.html.erb file which would contain the template of the email. Here's the sample email design:

gsoc_part2_1

gsoc_part2_2

3. Creating a job in app/jobs for sending digest to the users

What is ActiveJob and why we need it ?

We use rake tasks for migration, setting up database and for many more functions,so we can also use rake tasks for sending digests ,but we will not use it 😕 . As we want to send emails to large number of users and using rake task for that would slow down the other processes as priority would be given to rake task,but we don't want that.So, here comes the use of Active Job.

Active Job is a framework for declaring jobs and making them run on a variety of queuing backends. These jobs can be everything from regularly scheduled clean-up taskts,to mailing digest, subscription emails and many more. So, activejob will help us to send emails to multiple users asynchronously so there would be minimum affect on performance of other processes.

Our project plots2 currently uses rails 4.1.16 which doesn't support activejob. But, thanks to @Souravirus ,we are making progress in upgradation to rails 4.2.9 which would allow us to use activejobs.

3.1 Creation of Job Let's see an example job for sending digest:

class SendDigestJob < ApplicationJob
  queue_as :default   

  def perform(*args)
    User.all.each do | user |
      SubscriptionMailer.sending_digest(user)
    end
  end
end

In above example,a job class is there namely SendDigestJob,and default queue is used. Most of the adapters support multiple queues. With Active Job we can schedule the job to run on a specific queue,such as low_priority or high_priority queue.

3.2 Execution of Job on queueing backend For enqueuing and executing jobs in production, we need to set up a queuing backend, that is we need to decide for a 3rd-party queuing library that Rails should use. Rails itself only provides an in-process queuing system, which only keeps the jobs in RAM. If the process crashes or the machine is reset, then all outstanding jobs are lost with the default async backend. This may be fine for smaller apps or non-critical jobs, but most production apps will need to pick a persistent backend.

Active Job has built-in adapters for multiple queuing backends (Sidekiq, Resque, Delayed Job and others).I prefer to use Resque and Sidekiq from my experience. In plots2, I would use Resque gem which is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.

First I will install resque gem and then,I will define in config/application.rb that which adapter to use. Example:

  class Application < Rails::Application
    config.active_job.queue_adapter = :resque
  end
end

Resque also provides web overview to us where we can see the active jobs, failed jobs and also provide us option to retry job if it's failed. In gemfile, just below line needs to be added.

 gem 'resque', :require => "resque/server"

Webview looks like this:

resque_server

:require => "resque/server" enables the above view and can be accessed at route /resque/overview

4. Final Step: Calling the SendDigestJob weekly This can be achieved by whenever gem that provides a clear syntax for writing and deploying cron jobs.

First, install whenever gem by writing gem 'whenever', require: false and then running wheneverize . which would create schedule.rb where we would write all the cron jobs in friendly syntax. For calling our job, schedule.rb file would be:

set :output, "log/cron_log.log" # For logging 

every 1.week, at: '12pm' do 
 runner "SendDigestJob.perform_later" # call to job
end

And, this would finally call our SendDigestJob present in app/jobs and then from there job would be enqueued in the specified queue for execution.

Overview of Implementation instructions

Weekly digest Flow

Testing of Daily batch digest

  1. In the testing period of Weekly digest, I would be sending digest daily to all the moderators, selected contributors and admin of Public Labs and would attach a form with it where they can provide their suggestions.

  2. Test for the job would also be written which would ensure that job gets enqueued correctly, their behaviour and working of other entities like Redis and Resque gem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.