public
Last active

Rails 3.2 Multiple Databases - Migrations. - Expose the common rake tasks like db:migrate but for different dbs. - Use different dbs depending on the models

  • Download Gist
README.md
Markdown

Commands examples

If the namespace is not used then the commands will perform on top of the default database. bundle exec rake db:create bundle exec rake db:migrate

By using the namespace we are going to use all the configuration for our alternate DB. bundle exec rake store:db:create bundle exec rake store:db:migrate

Notes

  • From here having a more complex structure with more than one alternative database should be easy.
data_store_base.rb
Ruby
1 2 3 4 5 6 7 8 9 10 11 12 13 14
# models/data_store_base.rb
 
# Public: Provide a base class for accessing the store database.
# All models inheriting from this class will use by default the data store DB.
class DataStoreBase < ActiveRecord::Base
 
# Tell Active Record (AR) to not look up for a table called like this class,
# since this class is only used to add custom config for AR
self.abstract_class = true
 
databases = YAML::load(IO.read('config/database_store.yml'))
establish_connection(databases[Rails.env])
 
end
database.yml
YAML
1 2 3 4 5 6 7 8 9 10 11 12 13
# config/database.yml
 
development: &development
adapter: sqlite3
database: db/administrator_development.sqlite3
pool: 5
timeout: 5000
 
test:
adapter: sqlite3
database: db/administrator_test.sqlite3
pool: 5
timeout: 5000
database_store.yml
YAML
1 2 3 4 5 6 7 8 9 10 11 12 13
# config/database_store.yml
 
development: &development
adapter: sqlite3
database: db_store/data_store_development.sqlite3
pool: 5
timeout: 5000
 
test:
adapter: sqlite3
database: db_store/data_store_test.sqlite3
pool: 5
timeout: 5000
databases.rake
Ruby
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
# lib/tasks/databases.rake
 
# Public: This Rake file tries to add what rails provides on the
# databases.rake but for building on top of custom databases.
# Basically we get the nice db:migrate but for using it on a different DB than
# the default, by calling it with the namespace defined here.
#
# In order to be able to use the default rails rake commands but on a different
# DB, we are first updating the Rails.application.config.paths and then
# calling the original rake task. Rails.application.config.paths is getting
# loaded and available as soon as we call rake since the rakefile in a rails
# project declares that. Look at Rakefile in the project root for more details.
 
 
# Public: Access to the same commands you do for DB operations,
# like db:drop, db:migrate but using the store namespace, this will
# execute on top of the store DB.
namespace :store do
 
desc "Configure the variables that rails need in order to look up for the db
configuration in a different folder"
task :set_custom_db_config_paths do
# This is the minimum required to tell rails to use a different location
# for all the files related to the database.
ENV['SCHEMA'] = 'db_store/schema.rb'
Rails.application.config.paths['db'] = ['db_store']
Rails.application.config.paths['db/migrate'] = ['db_store/migrate']
Rails.application.config.paths['db/seeds'] = ['db_store/seeds.rb']
Rails.application.config.paths['config/database'] = ['config/database_store.yml']
end
 
namespace :db do
task :drop => :set_custom_db_config_paths do
Rake::Task["db:drop"].invoke
end
 
task :create => :set_custom_db_config_paths do
Rake::Task["db:create"].invoke
end
 
task :migrate => :set_custom_db_config_paths do
Rake::Task["db:migrate"].invoke
end
 
task :rollback => :set_custom_db_config_paths do
Rake::Task["db:rollback"].invoke
end
 
task :seed => :set_custom_db_config_paths do
Rake::Task["db:seed"].invoke
end
 
task :version => :set_custom_db_config_paths do
Rake::Task["db:version"].invoke
end
end
end

Also handy:

namespace :test do
  task :prepare => :set_custom_db_config_paths do
    Rake::Task["db:test:prepare"].invoke
  end
end

hey for Rails 4

change Rails.application.config.paths['db/seeds'] for Rails.application.config.paths['db/seeds.rb']

and add

ActiveRecord::Migrator.migrations_paths = 'db_store/migrate'

for work store:db:seed

Use ENV['DB_STRUCTURE'] for the structure file location.

What migrations, do we need to change anything into migrations, I mean need to change AR::Base class or something else ??

This is the best post i have seen till now. I was trying to create multiple databases from different tutorials available on internet, but failed. But I implemented this and its working correctly. Thanks.

Life saver :) thx a lot

UPDATE:
I used your idea HERE hope you don't mind, virtually saved my life :)

Failed to get this technique to work in my case. I am using Rails 3.1.10. I've followed the same steps mentioned above. When I execute rake support:db:migrate the migrations are run on the default database itself. Please HELP!

Combining rake tasks seems to not be working for me (I'm doing this from a Rails engine):

bundle exec rake db:create supply_client_engine:db:create

The above would only create a database for the "primary" configuration. It is behaving as if once a configuration has been loaded, the configuration cannot not change. I've tried playing around with resetting the Rails environment / connection, but so far nothing is working for me.

Has anyone found a solution for this?

In response to my comment above, i had a couple of problems: tasks being marked as invoked and switching database connections. After creating the Rakefile below in my Rails Engine, i was able to successfully run this command from the including Rails app:

bundle exec rake supply_client_engine:db:drop db:drop supply_client_engine:db:create db:create db:schema:load supply_client_engine:db:schema:load supply_client_engine:db:migrate db:migrate

Having to do so many hacks to get this to work feels really bad and really wrong.

class Rake::Task
  def already_invoked?
    @already_invoked
  end
end

require 'erb'

namespace :supply_client_engine do
  # https://gist.github.com/rafaelchiti/5575309
  namespace :db do
    def engine_root
      File.join(File.dirname(__FILE__), '..', '..')
    end

    def set_config_path(key, value)
      orig_value = Rails.application.config.paths[key]

      Rails.application.config.paths[key] = [File.join(engine_root, value)]

      orig_value
    end

    def restore_config_path(key, value)
      Rails.application.config.paths[key] = value
    end

    def engine_database_config
      @database_config ||= (
        config_path = File.join(engine_root, 'config', database_config_file_name)

        env_key = (Rails.application.config.respond_to?(:supply_client_database_environment) && Rails.application.config.supply_client_database_environment) || Rails.env
        YAML.load(ERB.new(File.read(config_path)).result)[env_key].symbolize_keys
      )
    end

    def database_config_file_name
      "database.yml#{%w(production staging).include?(Rails.env) ? ".#{Rails.env}" : nil}"
    end

    def run_with_engine_config(&block)
      orig_schema_env = ENV['SCHEMA']
      ENV['SCHEMA'] = File.join(engine_root, 'db', 'schema.rb')

      orig_db = set_config_path('db', 'db')
      orig_db_migrate = set_config_path('db/migrate', 'db/migrate')
      orig_db_seeds = set_config_path('db/seeds', 'db/seeds.rb')
      orig_db_config = set_config_path('config/database', "config/#{database_config_file_name}")

      Rake::Task["db:load_config"].execute # load config after we made our changes

      orig_connection_config = ActiveRecord::Base.connection_config if ActiveRecord::Base.connected?
      ActiveRecord::Base.establish_connection(engine_database_config)

      yield

      restore_config_path('config/database', orig_db_config)
      restore_config_path('db/seeds', orig_db_seeds)
      restore_config_path('db/migrate', orig_db_migrate)
      restore_config_path('db', orig_db_migrate)

      ENV['SCHEMA'] = orig_schema_env

      Rake::Task["db:load_config"].execute # restore original config after done

      ActiveRecord::Base.establish_connection(orig_connection_config) if ActiveRecord::Base.connected?
    end

    def invoke_engine_task(name)
      already_invoked = Rake::Task[name].already_invoked?

      run_with_engine_config do
        Rake::Task[name].execute # use execute to run even if run before
      end

      Rake::Task[name].reenable unless already_invoked # reenable task if it wasn't run before our execute
    end

    %w(drop create migrate rollback seed version schema:load schema:dump test:prepare).each do |name|
      task name do
        invoke_engine_task("db:#{name}")
      end
    end
  end
end

I also added this into the Rakefile of my app that includes my engine:

# Make tasks that invoke both supply client and bomb db tasks
%w(drop create migrate rollback seed version schema:load schema:dump test:prepare).each do |name|
  task "app:db:#{name}" => ["db:#{name}", "supply_client_engine:db:#{name}"]
end

I created a Gem to encapsulate a lot of the stuff i needed to do above for creating a Rails Engine that has models connecting to a different database. You can get it here: https://github.com/bkr/goa.

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.