async-rails3.patch

# checkout: http://github.com/igrigorik/async-rails/

From bb2a78858cffa7c6937642986e9aca1a4f862c0d Mon Sep 17 00:00:00 2001
From: Ilya Grigorik <ilya@igvita.com>
Date: Thu, 10 Jun 2010 00:46:48 -0400
Subject: [PATCH] async rails3

---
 Gemfile                               |    6 ++++++
 app/controllers/widgets_controller.rb |    6 ++++++
 app/models/widget.rb                  |    2 ++
 config.ru                             |    1 +
 config/application.rb                 |    1 +
 config/database.yml                   |    4 ++--
 config/environments/development.rb    |    3 +++
 config/routes.rb                      |    2 +-
 8 files changed, 22 insertions(+), 3 deletions(-)
 create mode 100644 app/controllers/widgets_controller.rb
 create mode 100644 app/models/widget.rb

diff --git a/Gemfile b/Gemfile
index ed9e0a7..aabda3d 100644
--- a/Gemfile
+++ b/Gemfile
@@ -7,6 +7,12 @@ gem 'rails', '3.0.0.beta4'
 
 gem 'sqlite3-ruby', :require => 'sqlite3'
 
+gem 'rack-fiber_pool', :require => 'rack/fiber_pool'
+gem 'activerecord', :require => 'active_record'
+gem 'mysqlplus'
+gem 'em-mysqlplus'
+gem 'em-synchrony'
+
 # Use unicorn as the web server
 # gem 'unicorn'
 
diff --git a/app/controllers/widgets_controller.rb b/app/controllers/widgets_controller.rb
new file mode 100644
index 0000000..62cf1d0
--- /dev/null
+++ b/app/controllers/widgets_controller.rb
@@ -0,0 +1,6 @@
+class WidgetsController < ApplicationController
+  def index
+    Widget.find_by_sql("select sleep(1)")
+    render :text => "Oh hai"
+  end
+end
\ No newline at end of file
diff --git a/app/models/widget.rb b/app/models/widget.rb
new file mode 100644
index 0000000..9f4a113
--- /dev/null
+++ b/app/models/widget.rb
@@ -0,0 +1,2 @@
+class Widget < ActiveRecord::Base
+end
\ No newline at end of file
diff --git a/config.ru b/config.ru
index 49e7126..9ab69f6 100644
--- a/config.ru
+++ b/config.ru
@@ -1,4 +1,5 @@
 # This file is used by Rack-based servers to start the application.
 
 require ::File.expand_path('../config/environment',  __FILE__)
+use Rack::FiberPool
 run Asynctest::Application
diff --git a/config/application.rb b/config/application.rb
index 2d6a859..099a968 100644
--- a/config/application.rb
+++ b/config/application.rb
@@ -1,6 +1,7 @@
 require File.expand_path('../boot', __FILE__)
 
 require 'rails/all'
+require '/git/em-mysqlplus/lib/em-activerecord'
 
 # If you have a Gemfile, require the gems listed there, including any gems
 # you've limited to :test, :development, or :production.
diff --git a/config/database.yml b/config/database.yml
index 025d62a..6deae08 100644
--- a/config/database.yml
+++ b/config/database.yml
@@ -1,8 +1,8 @@
 # SQLite version 3.x
 #   gem install sqlite3-ruby (not necessary on OS X Leopard)
 development:
-  adapter: sqlite3
-  database: db/development.sqlite3
+  adapter: em_mysqlplus
+  database: widgets
   pool: 5
   timeout: 5000
 
diff --git a/config/environments/development.rb b/config/environments/development.rb
index 97ff104..d9abd6a 100644
--- a/config/environments/development.rb
+++ b/config/environments/development.rb
@@ -1,3 +1,4 @@
+
 Asynctest::Application.configure do
   # Settings specified here will take precedence over those in config/environment.rb
 
@@ -16,4 +17,6 @@ Asynctest::Application.configure do
 
   # Don't care if the mailer can't send
   config.action_mailer.raise_delivery_errors = false
+
+  config.threadsafe!
 end
diff --git a/config/routes.rb b/config/routes.rb
index da860ab..9005646 100644
--- a/config/routes.rb
+++ b/config/routes.rb
@@ -54,5 +54,5 @@ Asynctest::Application.routes.draw do |map|
 
   # This is a legacy wild controller route that's not recommended for RESTful applications.
   # Note: This route will make all actions in every controller accessible via GET requests.
-  # match ':controller(/:action(/:id(.:format)))'
+  match ':controller(/:action(/:id(.:format)))'
 end
-- 
1.6.5.3

development.log

Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:21 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:21 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:21 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:21 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:21 -0400
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Widget Load (1000.6ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1002ms (Views: 0.5ms | ActiveRecord: 1000.6ms)
  Widget Load (1000.8ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1001ms (Views: 0.3ms | ActiveRecord: 1000.8ms)
  Widget Load (1002.6ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1003ms (Views: 0.3ms | ActiveRecord: 1002.6ms)
  Widget Load (1001.0ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1002ms (Views: 0.3ms | ActiveRecord: 1001.0ms)
  Widget Load (1001.5ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1002ms (Views: 0.3ms | ActiveRecord: 1001.5ms)


Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:22 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:22 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:22 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:22 -0400
Started GET "/widgets" for 127.0.0.1 at 2010-06-10 00:42:22 -0400
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Processing by WidgetsController#index as */*
  Widget Load (1000.6ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1002ms (Views: 0.5ms | ActiveRecord: 1000.6ms)
  Widget Load (1001.2ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1002ms (Views: 0.4ms | ActiveRecord: 1001.2ms)
  Widget Load (1001.7ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1002ms (Views: 0.3ms | ActiveRecord: 1001.7ms)
  Widget Load (1004.1ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1005ms (Views: 0.3ms | ActiveRecord: 1004.1ms)
  Widget Load (1003.2ms)  select sleep(1)
Rendered text template (0.0ms)
Completed 200 OK in 1004ms (Views: 0.4ms | ActiveRecord: 1003.2ms)

love it!

Epic. I hacked on a toy node app last night, which got me thinking about exactly this. That's a really elegant proof of concept—awesome work dude :)

Can you suggest how a model could do a HTTP request using em-http-request in this application? I've been mucking about with it for a day or so and can't seem to get it working, clearly I'm missing something...

Aaron, it should be pretty simple: require 'em-synchrony/em-http'

Then, in your controller:

http = EM::HttpRequest.new("url").get
puts http.response

That should do the trick!

Gah! So simple! Thanks for that Ilya. I was trying to run the exact same thing, but in an EM.synchrony block...

The code I ended up with is:

if(EventMachine.reactor_running?)
  github_response = EventMachine::HttpRequest.new(self.gist_embed_url_for_id(id)).get
else
  EventMachine.synchrony do
    github_response = EventMachine::HttpRequest.new(self.gist_embed_url_for_id(id)).get
    EventMachine.stop
  end
end

Is there a better way I should be handling the case of the application server not having started EventMachine? This way passes all my existing tests and works fine in Thin...

Thanks for all your work on em-synchrony! I was very pleased to discover it via your article that hit Hacker News the other day...
-A

Aaron, if you're running under thin, then you shouldn't need the reactor check. In fact, if you're running rails + fiber pool, you don't need the synchrony block either.. All of that is already taken care of for you: reactor is running, and your request is already wrapped into a fiber. So just EM::HttpRequest.new and you're good to go.

Hey Ilya,
The code without the reactor check was running fine on the test server, but failing all of my unit/functional tests with the error:

RuntimeError: eventmachine not initialized: evma_connect_to_server

Is there some setup I am missing in my tests that will allow it to run without the reactor check?
Thanks for all your help by the way! I'm fine with event drive programming, but combining all this stuff with Rails has been messing with my head!
-A

Aaron, there is no transparent work-around for that. I usually wrap my tests in EM.run { ...; EM.stop}. Take a look at the specs for em-http or em-synchrony.

Ah gotcha.
I may actually stick with the reactor check for the moment then, that way I don't have to modify all my tests. It certainly doesn't seem to be hurting the performance of the app! It went from 5 requests/second to 50+

Thanks for your help!
-A

Hmm, well you will have overhead of spinning up another fiber, which is redundant. Not sure how thin reacts to EM.stop within that context. Not 100%, but I think you would see a bit of a boost.

Ok, I'll have a look into it. Are you saying that calling EM.reactor_running? is spinning up another Fiber? Where is that extra spin-up coming from?

Calling EM.stop from within a simple application run by Thin seems to kill the server after completing that request. My knowledge of EM isn't great though unfortunately, so there could be other stuff going on that I'm not aware of.

Doh, nevermind me.. The coffee must have worn off. Yes that would work just fine, you're just checking if the reactor is already running. :-)

Haha, don't worry I know the feeling. I was up until 5am this morning doing the migration to Rails 3 and trying to get the async code to work, not exactly feeling bright and sparkly right now :-p

Thanks for all your help and input, I really appreciate it! Next time you're in Toronto drop me an email and I'll shout you a beer to say thanks!

A working demo + some explanations @ http://github.com/igrigorik/async-rails/