During the last year, I have worked on couple of Ruby/Rails based projects where the primary datastore was CouchDB. CouchDB, to me, is a dream database for web developer come true. The simplicity, the HTTP-based API, the abandonment of SQL semantics, the inspiring community, that all reminds me of when I came into Rails years ago.
However, working with Couch in Ruby and Rails is very, very painful, in my opinion. I'd like to briefly summarize some of my frustrations here. Maybe they are shared, maybe not -- if they are, I think we should launch some coordinate effort to make using Couch in Ruby a pleasure and intelectual satisfaction, not endless loops of research and hacks to „make it work“.
Please note, that my interest is solely to stir the debate. I may be severely mistaken in any point. But, I'd like using Couch in a Ruby application to be a joy, not a frustration, which is what I've met more times than I'd have liked.
First and obvious problem is the plethora of gems you can choose from to interact with Couch. There's the granddaddy CouchRest, above which sits CouchPotato and above which sits SimplyStored. There's RelaxDB and Makura with different take on things. There's CouchFoo, aiming for drop-in ActiveRecord replacement. And possibly others -- due to the simple HTTP-based Couch's API, it's relatively simple to create some wrapper around it.
Why the choice is bad? First of all, you have to research your options. You have to read documentation for those gems, noticing that they are almost the same, that they offer almost the same features. So, which of those options are valuable for you? Which less so? Which of those gems seem to be more maintained or used?
The situation reminds me of the state of internationalization/localization in Ruby on Rails some two years ago. You could choose from Globalize, GlobaLITE, Gibberish and loads and loads of other plugins/gems, providing you with relatively simple, and similar funtionality: replacing identifiers in your code with locale-specific text. In the end, it became completely unberable to research and compare the options, and the plugin/gem authors decided to get together and try to nail the commonalities and differences in their approaches, and a generic, higly modular solution for Rails i18n was born: http://github.com/svenfuchs/i18n. In the video from Euruko 2010, Sven Fuchs gives a very good overview of the effort: http://vimeo.com/12665914.
Do you think this is something worthwile to do for Couch-related gems?
As briefly noted above, some of the solutions have grotesque numbers of layers. Consider using SimplyStored, a very advanced and Rails-compatible library, in your application. Apart from the CouchDB database itself, you deal with:
- RestClient -- Handling HTTP
- CouchRest -- Handling creating databases, saving documents, etc
- CouchPotato -- Handling saving documents, validations, declaring and querying views, etc
- SimplyStored -- Giving ActiveModel-like API, saving documents, validations, associations, etc
That's rather painful. Whenever you need to adapt, patch, or otherwise change something in your app, you have to constantly shift between these layers, their codebase, their test suites, documentation and quirks.
I think current libraries do not do true justice to the flexibility of Couch.
One real world example. On my current contract, we need to save different models into different databases. (And we need to use and interact with the Couch authentication/authorization system, but that's another story.) That's impossible to do eg. in RelaxDB or CouchPotato/SimplyStored. In the end, we had to abandon attempts to hack that into SimplyStored and ended with replicating much of it's functionality in customized code.
But, hell, we need to use different databases for different model instances! (think user „inboxes“). Maybe that's kinda idiosyncratic, but consider that even the good old ActiveRecord has use_table
.
And the list could continue. What if you, for example, want to use Typhoeus instead of RestClient? It should be trivial to do that.
Whatever our opinion is on Rails and "ActiveRecord", it's the dominant platform for Ruby web applications. CouchPotato has full ActiveModel compatibility -- but, as an example, it magically disappears in SimplyStored (http://github.com/karmi/simply_stored/commit/a168851). CouchPotato does not offer Model#save
and such features by itself.
Compare this with the MongoID gem. That is just true drop-in solution. For simple cases, you swap MyModel < ActiveRecord::Base
with some include
s and you're done. I'd love Couch to be usable on exactly the same level.
I also hear endless talk about Couch „not being the same as relational databases“ and such, repeated over and over. This is truly painful because some of the people who repeat such things confuse „relational“ with has_many
associations. Whatever the issue is, ActiveRecord semantics (Model#update_attributes, validations, etc) is (one) valid way for modelling in any data store. Even Ohm for Redis has similar semantics.
Another case in point is pagination. Some of the gems, like RelaxDB, implement pagination, some of them leave that as an exercise for the reader. RelaxDB implements pagination in it's own way, incompatible with the preferred way of pagination in a Rails app -- will_paginate. This way, we lose all the helpers, all the syntax sugar everybody is used to when working with Rails.
And of course, will_paginate's style of pagination is wrong in Couch. But, why should it not be one way of doing it? Why should you think about and implement pagination on your own?
It is my opinion, that anybody should be able to use Couch in Rails or Sinatra or plain Ruby application as easily as using ActiveRecord, or, maybe more importantly, the highly faved MongoDB. Please share your opinion in the comments.
Thanks for all the feedback. There's been some misunderstanding, which I expected. Thanks @mikeal for the long answer.
And I'm not saying that there should be "ActiveRecord for Couch". I said true drop-in gem for using Couch in Rails should be 100% ActiveModel compatible, making stuff like
form_for
etc. work painlessly. (And I still see no real argument against that.) By the way, it seems that current couchrest_model brings that support out of the box. I also think, that in an ideal world a Rubygem for working with Couch would provide a smooth interface to Couch features, optionally even stuff like plugging into authentication/authorization or using the_changes
feed. And pagination. Or meaningful exceptions, like most prominently Makura does. Right now, the functionality is difused across a dozen of gems, making it hard to evaluate.Now, it's not about whether I'll "find solution in the end that works out well", as Jon writes. Of course I will. I have been working with Couch for almost a year, on real, production apps in Ruby (or a couchapp, used by a hosting company to manage invoicing/billing). That's not the point. Compared to how (fuckin') awesome Couch is, working with Couch in Ruby is unnecessarily hard, in my eyes. That's all.
Indeed. But this is not about "my relational way of doing things". As said, encountering Couch was probably the most joyful experience in the
"developer" part of my life, during last year, period. I thoroughly enjoyed learning it, researching it, using it. I cannot praise stuff like "no off switch" or trading CPU cycles for HDD space enough. (And yes, Redis is probably on the same level of awesomeness, even sharing the level of design simplicity.)
But, why it's so, that almost every gem out there implements "some" ActiveRecord-like API, even lightweight like Makura? Probably because there's some natural way of thinking about modelling, about imagining entities in Ruby classes? Could we work with that? Could we adapt and change such pattern? (Note that I'm not sold on most, probably all, implementations of
has_many
andbelongs_to
. This is the real point where ActiveRecord semantics may not make much sense.)This point is misleading, however. Most of the commentary is focused on the "abstraction" problem ("close to the metal", etc). Regarding this point, I only said it should be trivially easy to use CouchDB instead of *SQL database in a Rails app. And that in Ruby, a notoriously „smooth“ language, you'd expect elegant, simple and flexible solutions to your problems. I could raise more issues (like storing custom JavaScript in Ruby strings, seriously?, or providing support for storing them on disk, but in a format different from a couchapp?, why?) but that's for another time, maybe.
As it is, it seems that the core of Ruby devs working with Couch think the situation is "good enough" or "great". Really thanks for the various feedback I have received!
Anyway, just imagine what do you say to a guy/gal asking you „so what Rubygem should I use to work with Couch“? „Well, make your pick, they're almost the same“? „First, make sure you know Couch really well, so you know how to evaluate all that stuff“? „Just use a
libcurl
wrapper, it's about the metal ,,/“?The gist of my opinion is this, again: it's hard to choose from so many very similar options. Researching and evaluating all the options is just hard, and I don't see any immediate benefit in that.