Skip to content

Instantly share code, notes, and snippets.

@PharkMillups
Created March 29, 2011 00:05
Show Gist options
  • Save PharkMillups/891586 to your computer and use it in GitHub Desktop.
Save PharkMillups/891586 to your computer and use it in GitHub Desktop.
11:14 <reiddraper> drev1: at a higher level, my goal is to launch a
Riak cluster on ec2 and have them join a cluster together without having to manually call "join"
11:15 <reiddraper> drev1: should I just launch one node first,
call it the master, and then write a script which calls join with
that node -- for the other machines?
11:16 <drev1> yes
11:16 <reiddraper> drev1: if I am launching a large cluster,
should I stagger the nodes joining?
11:16 <drev1> there is no automated clustering system in Riak;
joining/leaving actions should be performed by the admin or deployment scripts
11:17 <drev1> reiddraper: yes
11:17 <drev1> there is a ringready command
11:17 <drev1> that you can poll to determine when to join another node
11:17 <reiddraper> ok, for an empty dataset, approximately how long will it take?
11:18 <drev1> a couple of seconds usually
11:18 <drev1> probably less
11:19 <reiddraper> drev1: cool. So would it be reasonable to
expect I could launch a 100-node cluster on ec2 and have the nodes join
over the course of ~20 minutes (1/12 seconds)?
11:20 <drev1> for that large of a cluster the time it takes to gossip will increase
11:20 <drev1> I am not sure how long that would take
11:21 <reiddraper> drev1: I suppose I'll find out!
11:22 <drev1> excellent; let us know if you have any more questions
11:22 <reiddraper> drev1: one more actually,
wrt http://blog.basho.com/2011/02/04/creating-a-local-riak-cluster-with-vagrant-and-chef/
11:23 <reiddraper> How are the nodes discovering each other in this example?
11:24 <drev1> that is a good question; I have not actually setup tht up locally
11:24 <lusis> reiddraper, that particular article doesn't address startup coordination
11:25 <lusis> it's an awesome demo but it's manual
11:25 <reiddraper> lusis: Maybe I'm missing something, but it says,
"start up dev2 (run vagrant up inside dev2/) and watch it connect to the cluster automatically"
11:26 <lusis> right but I mean it's manual in the sense that it you start one node, change the bootstrap script, start another node, change the script, start the last node
11:26 <drev1> apparently the riak:autoconf recipe handles it; put
it in a cluster named “vagrant” which will be used by the “riak::autoconf” recipe to
automatically join our nodes together.
11:26 <drev1> uses chef server to discover the nodes by cluster name
11:28 <lusis> I can't find the autoconf recipe anywhere though
11:28 <reiddraper> drev1: ok, so chef server is filling in this: https://github.com/opscode/cookbooks/blob/master/riak/recipes/autoconf.rb#L40
11:28 <reiddraper> lusis: https://github.com/opscode/cookbooks/blob/master/riak/recipes/autoconf.rb#L40
11:28 <lusis> ahh using a bash resource
11:28 <lusis> oh that's just the startup
11:29 <lusis> I think there's an LWRP then
11:29 <lusis> yeah there it is
11:29 <lusis> https://github.com/opscode/cookbooks/blob/master/riak/providers/cluster.rb
11:30 <lusis> doing ringready tests
11:31 <reiddraper> lusis: Ok, awesome, thanks
11:31 <lusis> reiddraper, np
11:31 <lusis> I never actually got around to looking at the cookbook
11:31 <lusis> never had time
11:32 <lusis> I figured it was doing something like that
11:32 <reiddraper> lusis: another resource about it, http://wiki.opscode.com/display/chef/Opscode+LWRP+Resources#OpscodeLWRPResources-riak
11:33 <lusis> yeah so any node with that attribute is a cluster member
11:33 <lusis> I like the implementation
11:34 <lusis> you could also key off a chef role too
11:34 <lusis> but the attribute method gives you
flexibility for multiple clusters
11:36 <reiddraper> right
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment