Skip to content

Instantly share code, notes, and snippets.

@chinshr

chinshr/Scaling Rails

Created January 6, 2015 22:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chinshr/65e9a89097945b88a554 to your computer and use it in GitHub Desktop.
Save chinshr/65e9a89097945b88a554 to your computer and use it in GitHub Desktop.
Download ZIP
Some low and higher hanging fruits to scale Rails apps
Raw
Scaling Rails
<!DOCTYPE html>
<html>
<head>
<title>Scaling Rails</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<style type="text/css">
@import url(https://fonts.googleapis.com/css?family=Yanone+Kaffeesatz);
@import url(https://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic);
@import url(https://fonts.googleapis.com/css?family=Ubuntu+Mono:400,700,400italic);
body { font-family: 'Droid Serif'; }
h1, h2, h3 {
font-family: 'Yanone Kaffeesatz';
font-weight: normal;
}
.remark-code, .remark-inline-code { font-family: 'Ubuntu Mono'; }
</style>
</head>
<body>
<textarea id="source">
class: center, middle
# Scaling Rails
## Or, some answers to: Does Rails scale?
---
class: center, middle
# Scaling a site is about architecture, databases, caching, event queues, disk IO, CDNs, etc.
---
class: center, middle
# Common Pitfalls
---
# Iterators vs. Scopes
Careful with: `select`, `find`, `any?`, `sort_by`, `map`, etc.
<br/>
### Instead of
``` Ruby
@product.taxons.find {|t| t.permalink.starts_with? 'category'}
# ... not as bad as ...
Taxon.all.find {|t| t.permalink.starts_with? 'category'}
```
### Consider this
``` Ruby
Spree::Taxon.class_eval do
scope :start_with_category, -> {
where("spree_taxons.permalink LIKE ?", "%category%")
}
end
@product.taxons.start_with_category
```
---
# Doing the right thing...
### This beauty
``` Ruby
# views/collections/_base3.html.erb
...
@variants = (@variants.reject(&:soldout?) + @variants.select(&:soldout?))
...
```
### Really means
``` Ruby
# views/collections/_base3.html.erb
...
@variants = @variants.order_by_soldout_last
...
```
---
# Doing the right thing is hard
### Nice on the outside
``` Ruby
# views/collections/_base3.html.erb
...
@variants = @variants.order_by_soldout_last
...
```
### Ughhhhhly on the inside
``` Ruby
Spree::Variant.class_eval do
...
scope :order_by_soldout_last, -> {
count_on_hand_sql = "(SELECT SUM(`spree_stock_items`.`count_on_hand`) FROM `spree_stock_items` WHERE `spree_stock_items`.`deleted_at` IS NULL AND `spree_stock_items`.`variant_id` = `spree_variants`.`id`)"
on_demand_sql = "(1 = ANY(SELECT `spree_stock_items`.`backorderable` FROM `spree_stock_items` WHERE `spree_stock_items`.`deleted_at` IS NULL AND `spree_stock_items`.`variant_id` = `spree_variants`.`id`))"
never_available_sql = "('Never Available' = `spree_variants`.`availability` AND `spree_variants`.`availability` IS NOT NULL)"
select("spree_variants.*")
.select("((#{count_on_hand_sql} <= 0 AND #{on_demand_sql} <> 1) OR #{never_available_sql}) AS soldout")
.preorder("(soldout = 0) DESC")
}
...
end
```
---
# Eagerloading
### When to use `preload`, `includes`, `references`, `eager_load` and `join`?
* `User.preload(:posts)`: Always loads association in a separate query (posts can't appear in `where`)
* `User.includes(:posts)`: Sometimes creates 2nd query, except when `where("posts.desc = 'Foo'")`
* `User.includes(:posts).references(:posts)`: Force to use a single query
* `User.eager_load(:posts)`: Single query, same as `includes+references` and `includes+where`
* `User.joins(:posts)`: No eager loading, creates `INNER JOIN`
---
# Memoization + preloaded data
``` Ruby
def details_images
@details_images ||= begin
if images.loaded?
(list = images.uniq.select {|i| i.image_type == 'Details'}).empty? ? images : list
else
images.image_type_present_or_all('Details').uniq
end
end
end
memoize :details_images
cache_method :details_images
```
<br/>
Notes:
* `loaded?` returns `true` if association is in memory
* Don't use `:memoize` keyword (deprecated), use `||= begin ... end` instead
* Inconsistencies with `cache_method`: in https://github.com/seamusabshere/cache_method
---
# When you are in doubt: Benchmark
Use Benchmark with warmup:
``` Ruby
# test/benchmark/array_flatten.rb
require 'benchmark'
require 'active_support/core_ext/array'
n = 500000
Benchmark.bmbm do |x|
x.report("[].flatten") { n.times { [["a"]].flatten } }
x.report("Array.wrap") { n.times { Array.wrap(["a"]) } }
end
```
Results in:
``` bash
Rehearsal ----------------------------------------------
[].flatten 0.500000 0.020000 0.520000 ( 0.508462)
Array.wrap 0.160000 0.000000 0.160000 ( 0.163742)
------------------------------------- total: 0.680000sec
user system total real
[].flatten 0.490000 0.000000 0.490000 ( 0.501750)
Array.wrap 0.170000 0.000000 0.170000 ( 0.161944)
```
---
# "and" vs && and "or" vs ||
### `and` and `or` are used for control flow like chaining expressions together, similar to `if` and `else`.
``` Ruby
redirect_to "/" and return
```
### && and || is for logical operators only:
``` Ruby
if one && two || three
# do something
end
```
<br/>
<br/>
<br/>
More: http://www.rubyinside.com/and-or-ruby-3631.html
---
class: center, middle
# Background Jobs
---
# Why Background Jobs?
Consider...
``` Ruby
def really_slow_task
sleep 1000 and "slept 1000 seconds"
end
really_slow_task and puts "ready."
=> slept 1000 seconds
=> ready.
```
<br/>
What if...
``` Ruby
Thread.new { really_slow_task } and "ready."
=> ready.
# ... if the process is not killed before ...
=> slept 1000 seconds
```
---
# Background Infrastructure
* delayed_job
* Resque
* Sidekiq
* Sucker Punch
* ...
* YABI (Yet Another Background Infrastructure)
<br/>
<br/>
### From a developer's perspective, all these work very similar (Job class, queues, invocation, parallel processing) and that's why the Rails core team decided to abstract their behavior using ActiveJob.
---
# Background Jobs with ActiveJob
ActiveJob is officially available in Rails ~>4.2. For Rails <4.2 we use http://github.com/ankane/activejob_backport
``` Ruby
class HeavyJob < ActiveJob::Base
queue_as :default # other queues can be configured: :low, :paperclip, etc.
def perform(*args)
sleep(1000) and puts "Job done."
end
end
HeavyJob.perform_later and "Ready."
=> Ready.
# ... 1000 seconds later ...
=> Job done.
```
Other ways to invoke jobs:
``` Ruby
# Enqueue a job to be performed tomorrow at noon.
HeavyJob.set(wait_until: Date.tomorrow.noon).perform_later(record)
# Enqueue a job to be performed 1 week from now.
MyJob.set(wait: 1.week).perform_later(record)
```
---
# Passing Parameters
###You can pass any (complex) object to a job, however, try to pass the bare minimum inside the argument list. Why?
* Objects are decomposed in memory (YAML)
* Persisted and sent (e.g. using Redis, MySQL, etc.)
* Recreated on job instance
* Records and their associations may get stale
* No transaction isolation
* Many more...
### Also consider
* Sidekiq is multi-threaded
---
# Instead of
``` Ruby
class Order::DoSomethingJob < ActiveJob::Base
queue_as :default
def perform(order)
order.line_items.each do |li|
...
end
end
end
Order::DoSomethingJob.perform_later(@order)
```
---
# Do this
``` Ruby
class Order::DoSomethingJob < ActiveJob::Base
queue_as :default
def perform(order_id)
order = Spree::Order.find(order_id)
order.line_items.each do |li|
...
end
end
end
Order::DoSomethingJob.perform_later(@order.id)
```
### But what if...
* Record has to be reloaded from DB...small overhead, comparing the risk of stale data.
* Order is deleted in the meantime...a `RecordNotFound` error is thrown and job is aborted/retried.
---
# How to begin implemeting a job
``` bash
rails generate job <job-name>
```
<br/>
<br/>
Example:
``` bash
rails generate job awesome
app/jobs/awesome_job.rb
specs/jobs/awesome_job_spec.rb
```
---
# Specs
How to test that jobs are invoked correctly?
``` Ruby
it "should kick off a job "
@order.touch
expect(Order::DoSomethingJob).to receive(:perform_later).once
# Or
expect(ActiveJob::Base.queue_adapter.enqueued_jobs).to eq 1
end
```
<br/>
<br/>
How to test the implementation of a job:
``` Ruby
# lives in specs/jobs/order/do_something_job_spec.rb
describe Order::DoSomethingJob do
it "should remove all line items" do
order = FactoryGirl.create(:order)
Order::DoSomethingJob.new.perform(order.id)
order.reload
order.line_items.count.should == 0
end
end
```
---
# Configuration
<br/>
``` Ruby
# E.g. in config/initializers/active_job.rb
ActiveJob::Base.queue_adapter = [:inline | :test | :sidekiq | :delayed_job]
```
* `:sidekiq` is our default adapter on production/staging.
* `:inline` executes the job immediately on `perform_later` inside the process without scheduling.
* `foreman start -f Procfile.dev` loads up environment in `development`.
<br/>
Or, switch inside a spec:
``` Ruby
before(:all) do
stored_adapter, ActiveJob::Base.queue_adapter = ActiveJob::Base.queue_adapter, :inline
end
after(:all) do
ActiveJob::Base.queue_adapter = stored_adapter
end
```
---
class: center, middle
# Fin!
---
</textarea>
<script src="https://gnab.github.io/remark/downloads/remark-latest.min.js">
</script>
<script>
var slideshow = remark.create();
</script>
</body>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment