How to config rabbitmq server cluster [3 nodes]

rabbitmq-cluster.md

This article explains how to install RabbitMQ on a Ubuntu 16.04 server instance.

Update the system

Use the following commands to update your Ubuntu 16.04 system to the latest stable status:

sudo apt-get update
sudo apt-get upgrade

Install

To install rabbit with specify version. You can go to this url https://packagecloud.io/rabbitmq/rabbitmq-server/ then chosea version, for example 3.5.1 on ubuntu 16.04

curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.deb.sh | sudo bash
sudo apt-get install rabbitmq-server=3.6.15-1

Start the Server

After that you can start and verify following command below

sudo systemctl start rabbitmq-server.service
sudo systemctl enable rabbitmq-server.service
sudo rabbitmqctl status

Enable and use the RabbitMQ management console

Enable the RabbitMQ management console so that you can monitor the RabbitMQ server processes from a web browser:

sudo rabbitmq-plugins enable rabbitmq_management

Then add config below to /etc/rabbitmq#/rabbitmq.config allow remote access

[
  {rabbit,[
    {handshake_timeout, 100000},                
    {loopback_users, [<<"guest">>]},            
    {loopback_users, []},                       
    {log_levels, [{channel, info}, {connection, info}, {mirroring, info}]},
    {cluster_nodes, {['rabbit@rabbit01', 'rabbit@rabbit02', 'rabbit@rabbit03'], disc}}                                  
   ]}               
].

cat /etc/rabbitmq/rabbitmq-env.conf
NODENAME=rabbit@rabbit02

Now, visit the following URL: http://[your-urbn8-server-IP]:15672/

Log in with the credentials you had specified earlier. You will be greeted with the RabbitMQ remote management console, where you can learn more about RabbitMQ.

Rabbit cluster

Take a look at https://gist.github.com/duythien/bc038436ef0af3f045e00feca8a03a3c

How to config rabbitmq server cluster [3 nodes]

Do after install SLGInstallers

Edit /etc/hosts

vi /etc/hosts
192.168.10.157  rabbit01
192.168.10.159  rabbit02
192.168.10.161  rabbit03

Restart rabbitmq-server [all nodes]

service rabbitmq-server restart

Enable rabbitmq-server service

chkconfig rabbitmq-server on

Verify rabbitmq-server is running [all nodes]

rabbitmqctl status

Result

Status of node 'rabbit@rabbitmq-1' ...
[{pid,2163},
 {running_applications,[{rabbit,"RabbitMQ","3.6.0"},
                        {ranch,"Socket acceptor pool for TCP protocols.",
                               "1.2.1"},
                        {rabbit_common,[],"3.6.0"},
                        {xmerl,"XML parser","1.3.8"},
                        {mnesia,"MNESIA  CXC 138 12","4.13.1"},
                        {os_mon,"CPO  CXC 138 46","2.4"},
                        {sasl,"SASL  CXC 138 11","2.6"},
                        {stdlib,"ERTS  CXC 138 10","2.6"},
                        {kernel,"ERTS  CXC 138 10","4.1"}]},
 {os,{unix,linux}},
 {erlang_version,"Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:8:8] [async-threads:64] [hipe] [kernel-poll:true]\n"},
 {memory,[{total,46149608},
          {connection_readers,0},
          {connection_writers,0},
          {connection_channels,0},
          {connection_other,0},
          {queue_procs,2808},
          {queue_slave_procs,0},
          {plugins,0},
          {other_proc,18729656},
          {mnesia,60536},
          {mgmt_db,0},
          {msg_index,47200},
          {other_ets,871984},
          {binary,24216},
          {code,17244116},
          {atom,662409},
          {other_system,8506683}]},
 {alarms,[]},
 {listeners,[{clustering,25672,"::"},{amqp,5672,"::"}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,6713846988},
 {disk_free_limit,50000000},
 {disk_free,583758884864},
 {file_descriptors,[{total_limit,924},
                    {total_used,2},
                    {sockets_limit,829},
                    {sockets_used,0}]},
 {processes,[{limit,1048576},{used,135}]},
 {run_queue,0},
 {uptime,244},
 {kernel,{net_ticktime,60}}]

Add User

$ sudo rabbitmqctl add_user myuser mypassword
$ sudo rabbitmqctl set_user_tags myuser administrator
$ sudo rabbitmqctl set_permissions -p / myuser ".*" ".*" ".*"

Create rabbitmq-server cluster

Step 1 view /var/lib/rabbitmq/.erlang.cookie [Node 1]

[Node 1]
cat /var/lib/rabbitmq/.erlang.cookie

Result = AFYDPNYXGNARCABLNENP

Step 2 set cookie same node 1 (Make sure the erlang cookies are the same all node.) [Node 2-3]

[Node 2-3]
service rabbitmq-server stop
echo -n "AFYDPNYXGNARCABLNENP" > /var/lib/rabbitmq/.erlang.cookie
service rabbitmq-server start

Step 3 join cluster to node 1 [Node 2-3]

[Node 2-3]
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@rabbitmq-1
rabbitmqctl start_app

Verify rabbitmq-server cluster [all nodes]

rabbitmqctl cluster_status

Result

Cluster status of node 'rabbit@rabbitmq-1' ...
[{nodes,[{disc,['rabbit@rabbitmq-1','rabbit@rabbitmq-2',
                'rabbit@rabbitmq-3']}]},
 {running_nodes,['rabbit@rabbitmq-3','rabbit@rabbitmq-2','rabbit@rabbitmq-1']},
 {cluster_name,<<"rabbit@rabbitmq-1">>},
 {partitions,[]}]

RabbitMQ cluster is not reconnecting after network failure

Login to slave rabbit

rabbitmqctl stop_app

rabbitmqctl reset

rabbitmqctl join_cluster rabbit@rabbit01

rabbitmqctl start_app

If do not work, then login to master do same step above

In such cases, you'll need to run the following set of commands on one of the nodes that formed part of the original cluster (so that it joins the other master node (say rabbitmq1) in the cluster as a slave): https://stackoverflow.com/questions/8654053/rabbitmq-cluster-is-not-reconnecting-after-network-failure