Skip to content

Instantly share code, notes, and snippets.

@eprothro
Created April 12, 2013 19:28
Show Gist options
  • Star 6 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save eprothro/5374472 to your computer and use it in GitHub Desktop.
Save eprothro/5374472 to your computer and use it in GitHub Desktop.
Dynamic Octopus configuration for master/slave horizontal DB scaling with a Rails application on the Heroku stack. See the wiki page for more info: https://github.com/tchandy/octopus/wiki/Replication-with-Rails-on-Heroku. Props to Heroku for the idea, gleaned from the dynamic database.yml they inject at build-time for rails apps.
<%
require 'cgi'
require 'uri'
def attribute(name, value, force_string = false)
if value
value_string =
if force_string
'"' + value + '"'
else
value
end
"#{name}: #{value_string}"
else
""
end
end
configs = case Rails.env
when 'development', 'test'
# use dev and test DB as feaux 'follower'
Array.new(2){YAML::load_file(File.open("config/database.yml"))[Rails.env]}
else
# staging, production, etc with Heroku config vars for follower DBs
master_url = ENV['DATABASE_URL']
slave_keys = ENV.keys.select{|k| k =~ /HEROKU_POSTGRESQL_.*_URL/}
slave_keys.delete_if{ |k| ENV[k] == master_url }
slave_keys.map do |env_key|
config = {}
begin
uri = URI.parse(ENV["#{env_key}"])
rescue URI::InvalidURIError
raise "Invalid DATABASE_URL"
end
raise "No RACK_ENV or RAILS_ENV found" unless ENV["RAILS_ENV"] || ENV["RACK_ENV"]
config['color'] = env_key.match(/HEROKU_POSTGRESQL_(.*)_URL/)[1].downcase
config['adapter'] = uri.scheme
config['adapter'] = "postgresql" if config['adapter'] == "postgres"
config['database'] = (uri.path || "").split("/")[1]
config['username'] = uri.user
config['password'] = uri.password
config['host'] = uri.host
config['port'] = uri.port
config['params'] = CGI.parse(uri.query || "")
config
end
end
whitelist = ENV['SLAVE_ENABLED_FOLLOWERS'].downcase.split(', ') rescue nil
blacklist = ENV['SLAVE_DISABLED_FOLLOWERS'].downcase.split(', ') rescue nil
configs.delete_if do |c|
( whitelist && !c['color'].in?(whitelist) ) || ( blacklist && c['color'].in?(blacklist) )
end
%>
octopus:
replicated: true
fully_replicated: false
environments:
<% if configs.present? %>
<%= "- #{ENV["RAILS_ENV"] || ENV["RACK_ENV"] || Rails.env}" %>
<%= ENV["RAILS_ENV"] || ENV["RACK_ENV"] || Rails.env %>:
followers:
<% configs.each_with_index do |c, i| %>
<%= c.has_key?('color') ? "#{c['color']}_follower" : "follower_#{i + 1}" %>:
<%= attribute "adapter", c['adapter'] %>
<%= attribute "database", c['database'] %>
<%= attribute "username", c['username'] %>
<%= attribute "password", c['password'], true %>
<%= attribute "host", c['host'] %>
<%= attribute "port", c['port'] %>
<% (c['params'] || {}).each do |key, value| %>
<%= key %>: <%= value.first %>
<% end %>
<% end %>
<% else %>
- none
<% end %>
@jamesfzhang
Copy link

Does this still work? I can't seem to get reads to go to slave unless I specify the query with Octopus.using(:follower).

@ericktai
Copy link

ericktai commented Nov 9, 2016

Also curious about this Heroku setup ^

@jamesfzhang, did you happen to get around it or you simply have to call Octopus.using(:follower) explicitly?

@refs
Copy link

refs commented Nov 15, 2016

^ +1

@tahngarth825
Copy link

This guy's solutions worked for me:
CoughDrop/coughdrop@7e849a3

@arjan0307
Copy link

@jamesfzhang Isn't that because fully_replicated is set to false?

@pmatsinopoulos
Copy link

Does this still work? I can't seem to get reads to go to slave unless I specify the query with Octopus.using(:follower).

It works like that because fully_replicated is set to false.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment