gistfile1.txt

# Vertical application slices

The Rails < 2.3 engines plugin defined the term "engine" as a "full vertical
application slice". When Rails 2.3 included engines it went a few steps back
and only implemented most of the core features, making an "engine" rather a
"pimped plugin". Today in Rails 3 we have engines that are way more powerful
and flexible than what we had before but (apart from what Piotr's plans are
[1]) there's still been one feature in the original engines plugin that I miss
today: the ability to mix first-class-citizen code slices from various engines
automatically without having one engine know much about other engines.

Let's say we have a bunch of engines that contribute small applications (say,
a blog and a ticket tracker) and only share a few things, maybe a User model.
So, we'd have the engines: user, blog, tickets. Obviously users have many blog
posts and many tickets. Now, when the blog engine is installed the User.should
have_many(:posts). When the tickets engine is installed the User.should
have_many(:tickets). But obviously this needs to be defined in the blog and
tickets engines, not in the user engine where the User class is defined.

So, the blog and tickets engines somehow need to reopen the User class and add
that association:

# somewhere in the blog engine 
User.has_many :posts

# somewhere in the tickets engine 
User.has_many :tickets

Traditionally there were two approaches (to my knowledge) to place these code
bits in the blog and tickets engines:

1. Initializers. It was easy to hack Rails plugins to have their own
initializers, so they'd all get loaded at startup. Placing all of these things
into tons of initializers smelled a lot and also required to use to_prepare
hooks all over the place.

2. App folder. One could do something like:

# [blog-engine]/app/models/user.rb 
load 'relative/path/to/user-engine/app/models/user.rb'
User.has_many :posts

But this obviously requires the blog-engine to preceed the user-engine in the
load path. With engines being Rails plugins one could mess with the plugin
load order to enforce this but obviously this isn't a nice and reliable
option, too.

I believe this should be a first-class framework capability. There should be a
way to tell the framework: "Here's a file that relates to the User class but
it's only a slice of it. You have to look elsewhere to find the file defining
the User class. Once you did please load this slice, too."

Maybe this could be accomplished introducing an after_constant_load hook in
ActiveSupport::Dependencies. Once Dependencies has found the User class in
[user-engine]/app/models/user.rb it would call the after_constant_load hook
where we could then look for files named something like models/user_slice.rb
and load them. Obviously this approach would require a naming convention or
special subfolder.

Another approach could be to have Dependencies be able to work recursively (I
have no idea how much of the current implementation could support that):

When const_missing for the User class is triggered Dependencies would first
find the file in the blog-engine:

# [blog-engine]/app/models/user.rb
User.has_many(:posts)

... which again uses the not yet defined User constant which again would
trigger Dependencies const_missing. Now Dependencies would skip looking at the
file [blog-engine]/app/models/user.rb and instead look for the next one in the
load path and so forth until it finds:

# [user-engine]/app/models/user.rb
class User
end

Dependencies would also need to continue looking for user.rb files once it has
found one ... so that engines that come later in the load_path will be able to
contribute their stuff, too:

# [xyz-engine]/app/models/user.rb
User.has_many(:xyzs)

So, basically this approach would change Dependencies:

- from looking for a single file defining a missing constant
- to loading all files according to the naming convention
  (constant_name.underscore) and configuration (app folders etc).


---

[1] http://piotrsarnacki.com/2010/07/06/rsoc-status-briging-engine-closer-to-application

I like the last approach the best, by changing Dependencies to load several files instead of just one.

Btw, people frequently need something like this for Devise, in order to change a controller behavior. Their only option today is: creating another controller, inheriting from Devise controller and change Devise routes to use the new controller.

That's what we do, too :)

Afaik it's also totally possible to do it in an initializer like: initializer { |config| config.to_prepare { Devise::SessionController.class_eval { ... } } } Ugh.

Or one can load the Devise controller file from within the custom controller file using load(Devise.root.join("app/controllers/devise/sessions_controller.rb")). Ughugh.

I think all of this requires way too much knowledge about the framework. The framework should be able to figure this out itself.

E.g. we've started working on a few engines only looking at cucumber features and hooking up things using initializers. Once we've started the app in dev mode way later we noticed that on the second request (of course) things were missing and we were looking at seemingly random exceptions. The bit we missed was that we needed to wrap a lot of things into to_prepare blocks. That might be something Rails really could make easier for users.

Maybe it's appropriate to give users a way to explicitely opt into this feature as a whole or maybe we'd only want to look at explicitely opted-in places or even constants.

I agree completely. The remaining questions then are:

1. What is the order this should happen? First load stuff from Engines and then from Application? (So I can easily override Engines settings?)

2. If an engine has two models and I want to use just one. How to easily turn the other model off?

Here's how I handle things in a 2.3 app: http://gist.github.com/466668

My only concern with automatically searching is that whether the overhead of searching all those paths will slow development mode down. In terms of order then definitely Engines first and then Application. In terms of switching models off you could have a kind of exclusive_scope method that disabled merging within that block:

ActiveSupport::Dependencies.exclusive_scope do
class User < ActiveRecord::Base
end
end

The problem then is how other parts of the Engine which provides the conflicting model would cope.

However, my biggest pain with Engines is schema migrations - we need something to fix that.

Nice gist Andrew! It seems that explicitly asking an Engine to load its file seems to be a good idea.

# app/models/user.rb
Engine.require "user"

class User < ActiveRecord::Base
end

Pros:

1. No side effects or caveats in performance since it should not change current behavior;

2. Full control of the loading order;

3. Easily choose what to include and what not to include;

Cons:

1. If you have two engines defining User, with the current approach it means just one of them will be loaded. That said, you will need to create a app/models/user.rb just to require them both:

# app/models/user.rb
EngineA.require "user"
EngineB.require "user"

1 - I think so, yes.

2 - Hmm, I've never seen this being a requirement. Sounds like this would make the system much more complicated?

2 - If you have two engines that are defining the same method (or the same scope), you may want to choose which one comes first. Otherwise your only option to ensure it always works is to redefine the method yourself. Confirm/deny?

But why do you think it would let everything much more complicated?

Jose,

The Engine.require("user") approach works when you just have the application contribute a "slice" to the User model. It doesn't work well when you want several engines contribute stuff. Because ... put the other way around ... it only works find when the contributing file is the first one loaded (and can have all the knowledge about all the required slices). Err, does that make sense to you? :)

I would expect that EngineA.require would basically do what's being done in this message: http://lists.rails-engines.org/htdig.cgi/engine-users-rails-engines.org/2007-September/000591.html

EngineA.require 'user':
require_dependency 'path/to/gem/or/plugin/engine_a/app/models/user'

That way you can control the order of loading and override the behavior.

@svenfuchs: Yes. This is the con. If you have two engines contributing to one piece, you will need to define this piece in your application and tell it to load the pieces from the engine. The question is: will this be a common scenario?

@pixeltrix: Yes, exactly. Is just a helper so you don't have to specify the whole path for the thing.

Doh. Just accidentally closed the browser instead of posting a long comment.

Jose: yes, I believe it absolutely is. Actually, it's the only reason for me to ask these questions :) Our scenario is:

1. Devise contributes class User (I know it doesn't, just for the sake of this example)
2. My Engines (e.g. blog-engine, tickets-engine) use User and need to add a few things to it.
3. Custom Rails 3 applications (e.g. which we are building for clients) both need to use User and Blog and both need to add a few things to it.

By installing Devise I basically opt into using whatever Devise provides by default. But because Devise is pretty modular there's no huge issue here. Even if Devise defines something that I do not want I could still overwrite it from the custom app.

I also like the Engine.require, I will think more on that topic later today :)

UPDATE: I haven't refreshed the browser before writing this.

@svenfuchs: by saying "Devise contributes class User" do you mean that in your example Devise would be the one to create it?

With that in mind Engine.require is not so cool anymore ;-)

Wouldn't the simplest solution be to use include to mix in the appropriate functionality like so:

# vendor/plugins/blog/app/models/blog/user.rb
require 'active_support/concern'
module BlogEngine
  module User
    extend ActiveSupport::Concern
    included do
      # stuff
    end

    def method_a
      # stuff
    end
  end
end

# vendor/plugins/devise/app/models/devise/user.rb
require 'active_support/concern'
module AuthEngine
  module User
    extend ActiveSupport::Concern
    included do
      # stuff
    end

    def method_b
      # stuff
    end
  end
end

# app/models/user.rb
class User < ActiveRecord::Base
  include BlogEngine::User
  include AuthEngine::User

  # Don't want this method
  remove_method :method_b
end

Andrew, this goes in the opposite direction to mountable apps. Where you can use an application mounted in another or stand alone.

This would also give problems with Devise. Sometimes people want to change the behavior of Devise::SessionsController and they cannot simply reopen app/controller/devise/sessions_controller.rb. They need to create a new controller and inherit from it. However, if we adopt your solution, we wouldn't have a controller in the first place. :(

Explicitly mixing in plugin behavior makes more sense to me than a highly-engineered autoextending mechanism. Big +1 to Andrew's last example.

A mountable app trying to work standalone or as a hybrid, using models from another app, would be well-off configuring that explicitly. For example,

module Blahg
  User = ::User
end

# or
Blahg.user_model = User

To subclass a controller or model provided by a plugin, create a base class and stub subclass in the plugin. An app subclass of the plugin's base class will appear first in $LOAD_PATH, displacing the stub.

For the record, we've now solve this issue like this: http://gist.github.com/518869

I.e. we look for files named foo_slice.rb in controller and model directories in both the application and our own engines. Then we require the class the "slice" is referring to and all the slices. Each slice can then class_eval the already loaded actual class.

Even though this works fine for now I'd rather liked to have seen a "lazy" approach where classes and their "slices" get loaded when necessary. Obviously our current approach preloads all these classes on all requests all the time - which slows down dev mode.

Since you probably almost always have to load the User model, this is no additional overhead compared to what AS::Dependencies would do. You could also use my RSoC fork, that has real dependency tracking. That way you only would have to load your patch once, do a require_dependency in there and everytime User is loaded, it will load your patch immediately afterwards.

Thanks Konstantin, that sounds interesting. Will look into that!