ActiveRepository Strawman

ActiveRepository.rb

# MOTIVATION:  As rails apps are growing, people are noticing the drawbacks
# of the ActiveRecord pattern.  Several apps I have seen, and several
# developers I have spoken to are looking towards other patterns for object
# persistence.  The major drawback with ActiveRecord is that the notion
# of the domain object is conflated with what it means to store/retrieve
# it in any given format (like sql, json, key/value, etc).
#
# This is an attempt to codify the Repository pattern in a way that would
# feel comfortable to beginner and seasoned Ruby developers alike.
#
# In the spirit of "make it look the way you want it to work, then make it
# work like that", here is a vision of the Repository pattern ala Ruby. I
# don't want to fault ActiveRecord for being an implementation of the
# ActiveRecord pattern; At RailsConf 2006, in his keynote, Martin Fowler
# said that ActiveRecord was "the most faithful implementation of the
# Activerecord pattern [he] had ever seen", and he was the one that
# documented that pattern in the first place.  So lets respect AR for what
# it is, and look at the repository pattern, done with Rails Flair.



# Person is a plain old Ruby object. It knows nothing about its persistence,
# but as a domain model, it knows something about its own attributes,
# relationships to other domain objects, as well as understands what it
# means to be 'valid'. Of course, it also has behavior.

class Person
  include ActiveModel::Relationships
  include ActiveModel::Validations
  
  has_many :hobbies
  belongs_to :fraternity
  
  attr_accessor :name
  attr_accessor :password
  attr_accessor :sex
  attr_accessor :born_on
  
  validates_presence_of :name, :password
  
  def favorite_drink
    if born_on < 7.years.ago
      "apple juice"
    elsif born_on < 18.years.ago
      "coke or pepsi"
    elsif < 21.years.ago
      "mountain dew"
    else
      "beer"
    end
  end
end

# conventions like _on and _at are still used.


# The person has no clue it is 'Persistent Capable'.  All of that is handled
# in the Repository.  Notice we could call the Repo anything we want.

class PersonStore < ActiveRepository::Base
  repository_for :person
  
  # other things that control the persistence can go here
  table :users
  encrypts_attribute :password
  map_attribute_to_column :sex, :gender
end

# I'm using inheritance here to enforce an opinion.  Of course this could
# be a mixin, but my current thoughts are that this Repository should only
# ever be a repository - after all, we are trying to get away from
# ActiveRecord's notion of "I'm the model and my own data store!".
# Inheritance would be an appropriate way to signal this *is a* repository.
# My fear is as a mixin, someone would think they are being clever by mixing
# the repository directly into the domain model, essentially recreating
# ActiveRecord and making this effort all for nothing.

# I would really like to have the ability to 'new' model objects as normal:

p = Person.new

# but it might be necessary to create them through the store, like:

p = PersonStore.create #, or
p = PersonStore.build

# saving is no longer done on the object, but through the repository

PersonStore.save(p)

# the save would be smart enough to call is_valid? if Validations were present.

# all the usual finder suspects are on the repository
p = PersonStore.find(5)
p = PersonStore.first
p = PersonStore.all
p = PersonStore.find_by_name("Chris")

# we could also create things like scopes, etc.


# Since Person is nothing special, you could easily swap in a different
# persistance Repository:

Class PersonStore < RedisRepository::Base
...
end

# or even:

Class PersonStore < RestfulResource::Repository
  repository_for :person
  base_uri "http://coolstuff.livingsocial.com"
end


# Swapping repositories might give you radically different methods (only an
# ActiveRepository would give you a find_by_sql method, for instance), but
# thats ok.  The "Persistant Capable" classes don't change with a change in
# the persistence mechanism. The "Persistent Aware" classes, like the
# controllers, would.  

# And it might even be possible to have multiple repositories in the same
# app...

Class PersonStore < ActiveRepository::Base
#...
end

# and

Class RemotePersonStore < RestfulResource::Repository
#...
end

# and then you could do stuff like:

p = RemotePersonStore.find(5)
PersonStore.save(p)

# and essentially use two repositories as an ETL engine.


# One nice thing we get from ActiveRecord would have to change slightly -
# the migrations.
# Actually, the migrations could work pretty much as they do now, but devs
# would have to make the corresponding attr_accessor declarations in their
# models.
#
# if an attr_accessor was declared that didn't have a corresponding column in
# the db, then it could warn on application init.  That warning for that field
# could be silenced in the repository with something like:

not_persisted :current_location

# and in reverse, if a migration added a column that couldn't map to an
# attr_accessor, it could warn unless the repo had a line like:

ignore_column :eye_color

# The generator could stick the attr_accessors in the class
# automatically if we wanted it to.  I wouldn't do anything 'magical' like
# have the persistence engine inject it with meta... that would make the
# attributes hard to discover, and could make multiple repositories in an
# app step on each other in nondeterministic ways.  By having attr_accessors,
# the model becomes the 'system of record' for what it means to be that
# kind of domain object.  I like that.

# (of course, nosql dbs may have their own requirements met with their own
# dsls).

# You could even have the store do something like:

synthetic_field :age, { Date.today - born_on }

# while you could do exactly the same thing by adding 'age' method to the
# model, having it in he Repository could be useful for an ETL task, for
# defining the transform step.  Imagine one database that has a born_on
# field, and another one that has an age field, and you are transforming
# data to go into the one with age...  in the store with the born_on,
# set the store to have a synthetic field :age.  In the other store, set
# it to ignore the born_on date. Then in the model you define attr_reader
# for :age.  Both stores see the domain model as exactly what it needs in
# order to make each database happy, and the domain model code stays clean.
#
# if you needed to map an attribute to a different column:

map_attribute_to_column :sex, :gender

# One potentially awesome tweak to the dsl is how this would
# handle polymorphic tables:

class PersonStore < ActiveRepository::Base
  repository_for :person
  repository_for :client
  repository_for :administrator
  polymorphic_model_column :type # would automatically store the class
                                 # type, just like AR polymorphism
end



# Given this pattern, I think relationship declarations go in the models,
# since there they can add the appropriate methods to return collections,
# etc, and since the repo knows what models it is supposed to manage, it can
# get access to the same knowledge to do whatever it needs to do.  If they
# were declared in the repo, it would be inappropriate for the model to
# introspect there, otherwise the model would become 'persistence aware'.
# They 'feel' a little attr_accessor-like things to me.

# Finally, while the model as code looks completely unaware of its storage,
# underneath the covers at runtime the repository could 'enhance' it with
# things like dirty flags for fields, is_dirty? & is_new? methods, etc.
# In fact, for years I used an object-oriented database in Java named
# Versant, and it had a 'bytecode enhancement' step that did exactly this
# during the compile - it modified the classes bytecode with dirty flags
# and other persistence helpers.

Someone just sent me an email mentioning this project:

https://github.com/braintree/curator

and it looks damn good as an inspiration. I don't like the way the model object has to include Curator::Model... including is a form of inheritance, and it is certainly aware of something about the persistence framework. I want the domain model to be completely free of anything like that. a Vanilla Ruby Object.

Very interesting concept, I really like it.

In the final part you say something about enhancing the model with things like dirty options. It couldn't be done with some decorator-like object?

I'd prefer to see a version with no inheritance at all, but I haven't worked out exactly what that'd look like.

Andrewhr - Thanks for the comments. First, let me say I have done more than my fair share of metaprogramming at the moment, so I'd need to step away for a while and clear my head before I could talk rationally about a roper implementation. But my intent in talking about that is that the line in the repository:

repository_for :person

gives the repo a hook to do any metaprogramming that necessary. Whether we need that or not is debatable... but I think from the repository's point of view, it would be nice to be able to annotate the stored object so that if you called PersonStore.save(person) on an unmodified person, it essentially translated to a no-op... or if you called save on a RemotePersonStore that communicated ala ActiveResource, it could just send the fields that have been modified via the [patch] verb. For that, we'd need a way to decorate each field with a dirty flag. The repository could do that by saying something like "model_class.extend(ActiveRepository::DirtyFlags)". From the developers point of view, they are dealing with a Vanilla Ruby Object unless they poke at the deep meta-juju dragon.

steveklabnik - there is no inheritance in the domain model. Are you concerned about the inheritance in the store? In that case, the code could look like (still technically inheritance):

class PersonStore
include ActiveRepository::Base
repository_for :person
end

but in my notes I mention that someone could essentially degenerate to ActiveRecord by doing:

class Person
include ActiveRepository::Base
repository_for :self
end

that makes me feel kinda dirty, but in the end, maybe thats a feature.

Yes, in the store. That's still inheritance. I mean using delegation instead. I've found it vastly superior in almost all cases, especially testability.

steveklabnik -
One way to do that would be to make the Repositories instances with runtime configuration, like this:

person_store = ActiveRepository::Base.new {
... configuration goes here
}

no inheritance, but I don't think it improves the testability that much... although you could store the block as a proc and have a completely different Repository you drop in for testing:

person_store = FactoryGirl::InMemoryRepository.new &person_store_config

and now person_store works as an in-memory tester backed by factories so you can still test things like finders.

Transactions are another big part of ActiveRecord. I've always found transactions to feel forced, since they should be independent of models:

In the example below, why not Case.transaction? or any Model.transaction? they all do the same thing.
`Person.transaction do

do something with person

do something with his case

end`

I guess this would be better...
`ActiveRepository.transaction do

do something with person

do something with his case

end`

My two cents: this will be much more awesome if ActiveRecord is composed from it. Further, if there is a linear difficulty curve conceptual scaling curve for building up an AR model and then scaling it down by decoupling the repository and any business-y logic, then you've double-plus won.

One thing we have tried to do with Curator is remove the requirement for attr_accessor. A domain model with all properties exposing setters removes any notion of encapsulation. It would be awesome if this could support that as well.

I don't think exposing properties necessarily breaks encapsulation - if you could access the properties directly it would, but attr_accessor technically synthesizes methods. (Methods that can be protected by authorization code, and could even lead to some other solution for the bulk update problem whipping us up like frenzied bees lately).

As a plain Ruby object, if you don't want a setter for it, then you could just use attr_reader. For instance, its always bothered me about the acts_as_state_machine gem that the 'state' of the object can be updated directly, bypassing all of the event triggers that happens when you call the transition methods. By using attr_reader for that field, this encapsulation could be preserved. If you really have a field that is just internal to the behavior, we could certainly add something like attr_internal or attr_private.

therealadam - deriving ActiveRecord from composition would be cool, yes. If that would really work, and if maintainers of AR are interested, then I humbly submit all of these ideas for their vetting. consider the whole thing MIT licensed. CarlHuda probably knows the internals of AR well enough by now to make this a fun afternoon project.

OK, it's clear what this method costs - a slight boost in complexity to controllers and possibly some SERIOUSLY shady metaprogramming to make associations work.

What does that cost BUY? Is it just the minty-fresh feeling one gets from OO purity? Not trying to be a jerk, just trying to figure out what exactly this is supposed to be doing for me.

It doesn't, for example, offer the "swap out persistence layers" option that's the mainstay of "ABSTRACT ALL THE THINGS" arguments; unless you're redefining PersonStore, references to it are littered throughout the controller code.

@ashirazi - a quick note: the transaction methods are only equivalent if all your models use the same database connection. Admittedly, that's a fairly extreme corner case but it does happen...

Okay, so I promised you thoughts on this a week ago; havn't had time to write anything down.

First, I do like the fact that this pulls persistence responsibility away from the individual record and into a repository. The field mapper stuff looks alright as well.

My biggest complaint is that this is still class based. I understand why it's class based, and it's probably an easier transition away from ActiveRecord thinking. But from my perspective, it looks like it's replacing one class-based solution with another.

I'm not an expert on DDD, but from what I've been reading, one of the ideas of a repository is that it's usable as an Enumerable. Just drop in anywhere an array is expected, and off to the races. In an instance based system, and with a little care in the clients, this means you could drop any enumerable into the client code, and it would work fine. That's trivial in an instance (object) oriented implementation. It's much more annoying in a class oriented implementation. I've mentioned before that I'm trying to avoid Capitols in my code these days. One of the obvious wins is testability -- you can test the clients by feeding them arrays, rather than worrying about a persisting repository. Bypass the whole mock/stub argument altogether. The other advantage is reduced coupling. Yes, I really need to actually write a blog post about this.

My ad-hoc implementations of a repository have thus far been a repository object takes an AR scope object in it's initializer -- or an array of other data. The advantage of that is that diffferent controllers can scope things down to just the sorts of record that it's actually interested in. I havn't done much with putting things back in the repository, however. I'm working with more, smaller objects. Yes, I'm also risking object-soup, but I havn't had that bite me just yet.

bump

@al2o3cr Both the Active Record pattern and the Repository pattern can support flexible application architecture, in which all dependencies point INWARD to the domain model, with no dependencies pointing OUT from it. The Single Responsibility Principle is fulfilled when a class has only ONE reason to change. For your domain model classes to honour this principle, they should require NO change in response to ANY change in the persistence mechanism(s). As soon as you can call Account.save on your Account business object, it's doing more than one thing (modelling the behaviour of an account, and managing account persistence).

If you buy what I've said so far, then to honour SRP with the active_record library, your domain model can't include classes that extend ActiveRecord::Base. Instead, your ActiveRecord::Base subclasses should be Data Access Object implementations, and you have to develop an adaptor strategy for composing DAOs into business objects and back. At which point, you realize that a) you're implementing the Repository pattern, and b) you're fighting the framework. For some excellent coverage of dependencies in application architecture, see: http://confreaks.net/videos/759-rubymidwest2011-keynote-architecture-the-lost-years

EDIT: I'd like to amend. Entities and Services in the data model can't extend ActiveRecord::Base; I'm not so sure about Value Objects.

Has anyone implemented this in some library/gem yet?

There's http://engineering.nulogy.com/posts/tag/edr that describes https://github.com/nulogy/edr.

But I'm not convinced that this is anything more than a stop-gap.

Why not Data Mapper? http://martinfowler.com/eaaCatalog/dataMapper.html

That is to say DataMapper 2 https://github.com/datamapper

Hey guys,

I have started experimenting with a similar idea (with a slightly different approach) here: https://github.com/fredwu/datamappify And I'd love to get some feedback from you all! :)

Basically, we want to separate the behaviour and persistence - without losing the convenience ActiveRecord (which we heavily rely on) provides to us.

Any thoughts?

Hi guys,

I have made some more progress on Datamappify. It no longer tries to do clever things with association, and it now maps data from any ActiveRecord classes!

Any feedback is welcomed! :)

Thanks for sharing this @bokmann. I really like the design that you've sketched out.

I stumbled across this today while researching repository implementations in Ruby land. I went looking for DataMapper 2 as suggested by @elight, but I later lids covered that it's been renamed to Ruby Object Mapper or ROM https://github.com/rom-rb/rom.

I also came across a gem named active_repostory, which looks close to this design, but not quite the same.

I was hoping that I wouldn't need to build my own repository framework to address problems on my current project. The design in this gist is going to be used for inspiration.