Handling API Rate Limits with Delayed::Job

rate_limit_post.md

Has your app ever encountered a 429 (Too Many Requests) status code when making requests to a third-party API? Getting rate limited can be a nussance and if not handled properly can cause a bad experience for you and your users. While one solution is to catch the exception and ignore it, a better solution is to retry the request.

Let's take a look at how we can alleviate rate-limiting woes by utilizing a background job system. In this example we'll use delayed_job, since it provides the ability to retry failed jobs.

Let's pretend that we are going to be accessing the API of a popular website. Let's setup a background job that makes a request to that API.

class MyCustomJob < Struct.new(:param1, :param2)
  def perform
    PopularSiteApi.get('/posts')
  end
end

When this job gets executed a bunch of times in the row, we will potentially reach a limit, to how many requests we can make, that is provided by the popular website. When that happens an exception will be raised and our backround job will fail. That's okay -- delayed_job will retry any failed job (up to 25 times by default).

Rate limiting can vary from amount of requests per day to amount of requests per minute. For the sake of example, the's assume the latter. Now, delayed_job retries failed jobs in the following manner: failed job is scheduled again in 5 seconds + N ** 4, where N is the number of retries. This does not reflect what we know about the time frame of our rate-limit rule. So, let's do something different for jobs that fail does to rate-limit errors.

delayed_job provides a method called error which we can define to acess the exception.

def error(job, exception)
  if is_rate_limit?(exception)
    @rate_limited = true
  else
    @rate_limited = false
  end
end

def is_rate_limit?
  exception.is_a?(Faraday::Error::ClientError) && exception.response[:status] == 429
end

Now, we can retry this job at our known time interval if it got rate limited, by overriding a reschedule_at method for our job. resquedule_at is a method that delayed_job uses to calculate when to re-run the particular job. Here's our version.

def reschedule_at(attempts, time)
  if @rate_limited
    1.minute.from_now
  end
end

Once our custom job is cofigured thusly, we will retry it every minute, twenty five times in a row until it works. If the job is still encountering a 429 status code after our retries, it will fail completely. At this point, we'll send out a notification about the failure and consider upgrading our API rate plan.

Here's the full code example:

class MyCustomJob < Struct.new(:param1, :param2)
  def perform
    PopularSiteApi.get('/posts')
  end
  
  def error(job, exception)
    if is_rate_limit?(exception)
      @rate_limited = true
    else
      @rate_limited = false
    end
  end

  def is_rate_limit?
    exception.is_a?(Faraday::Error::ClientError) && exception.response[:status] == 429
  end
  
  def reschedule_at(attempts, time)
    if @rate_limited
      1.minute.from_now
    end
  end
  
  def failure(job)
    Airbrake.notify(error_message: "Job failure: #{job.last_error}")
  end
end