Skip to content

Instantly share code, notes, and snippets.

@brianpattison
Last active January 27, 2017 18:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save brianpattison/53e3b1476f511dd3e507e86a15181eed to your computer and use it in GitHub Desktop.
Save brianpattison/53e3b1476f511dd3e507e86a15181eed to your computer and use it in GitHub Desktop.
Simple Scheduler Proposal (README first development)

This was the initial proposal for Simple Scheduler in the form of a README. The goal was to come up with a project that didn't exist, but solved all of our problems right in the README. And then write the code.

You can now find the actual README and the project here:

https://github.com/simplymadeapps/simple_scheduler

--

Simple Scheduler

Simple Scheduler is a scheduling add-on that is designed to be used with Active Job and Heroku Scheduler. It gives you the ability to schedule tasks at any interval without adding a clock process. Heroku Scheduler only allows you to schedule tasks every 10 minutes, every hour, or every day.

Requirements

You must be using:

Getting Started

Create a configuration file config/simple_scheduler.yml:

# Runs once every 2 minutes
simple_task:
  class: "SimpleJob"
  every: "2.minutes"

# Runs once every day at 4:00 UTC
overnight_task:
  class: "OvernightJob"
  every: "1.day"
  at: "4:00"
  tz: "UTC"

# Runs once every hour at the half hour
half_hour_task:
  class: "HalfHourTask"
  every: "30.minutes"
  at: "*:30"
  tz: "America/New_York"

# Runs once every minute from 10:00 PM to 10:59 PM Central Time
frequent_task:
  class: "FrequentJob"
  every: "1.minute"
  at: "22:**"
  tz: "America/Chicago"

Add the rake task to Heroku Scheduler and set it to run every 10 minutes:

rake simple_scheduler -C config/simple_scheduler.yml

The file config/simple_scheduler.yml will be used by default, but it may be useful to point to another configuration file in non-production environments.

Writing Your Jobs

When writing your jobs, you need to account for any possible server downtime. The most common downtime would be caused by Heroku's required daily restart.

To ensure that your tasks always run, the jobs are queued in advance and it's possible the jobs may not be executed at the exact time that you configured them to run. If there is extended downtime, your jobs may back up and there is no guarantee of the order they will be executed when your worker process comes back online.

Because there is no guarantee that the job is run at the exact time given in the configuration, the time the job was expected to run will be passed to the job so you can handle situations where the time it was run doesn't match the time it was expected to run.

How It Works:

Once the rake task is added to Heroku Scheduler, the Simple Scheduler library will load the configuration file every 10 minutes, and ensure that each task has jobs scheduled in the future using Active Job.

Server Downtime Example:

Your clock process is restarted at 11:59:59, your task is scheduled for 12:00:00, and your dyno isn't available until 12:00:20. If you're using a gem like clockwork, there is no way for the clock process to know that the task was never run.

Simple Scheduler would have already enqueued the task hours before the task should actually run, so you still have to worry about the worker dyno restarting, but when the worker dyno becomes available, the enqueued task will be there and will be executed immediately.

Daily Digest Email Example:

Here's an example of a daily digest email that needs to go out at 8:00 AM for users in their local time zone. We need to run this every 15 minutes to handle all time zone offsets.

config/simple_scheduler.yml:

# Runs every hour starting at the top of the hour + every 15 minutes
daily_digest_task:
  class: "DailyDigestEmailJob"
  every: "15.minutes"
  at: "*:00"

app/jobs/daily_digest_email_job.rb:

class DailyDigestEmailJob < ApplicationJob
  queue_as :default

  # Called by Simple Scheduler and is given the scheduled time so decisions can be made
  # based on when the job was scheduled to be run rather than when it was actually run.
  # @param scheduled_time [DateTime] The time the job was scheduled to be run
  def perform(scheduled_time)
    # Don't do this! This will be way too slow!
    User.find_each do |user|
      if user.digest_time == scheduled_time
        DigestMailer.daily(user).deliver_later
      end
    end
  end
end

app/models/user.rb:

class User < ApplicationRecord
  # Returns the time the user's daily digest should be
  # delivered today based on the user's time zone.
  # @return [DateTime]
  def digest_time
    "8:00 AM".in_time_zone(self.time_zone)
  end
end
@brianpattison
Copy link
Author

Use a scheduler job to queue up from Heroku Scheduler. The job should only be queued if no scheduler jobs exist. If the scheduler job doesn't execute, that's fine because it will catch up the next time Heroku Scheduler runs. We'll have plenty of jobs queued in the future so we can afford to skip a run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment