Created
May 20, 2018 13:10
-
-
Save skamithi/28b7219dde4e063b45b103586c2aa8d9 to your computer and use it in GitHub Desktop.
RabbitMQ recovery
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Last login: Sun May 20 02:09:59 2018 from 192.168.121.1 | |
| [vagrant@tower3 ~]$ sudo tail -f /var/log/rabbitmq/rabbitmq\@tower3 | |
| tail: cannot open ‘/var/log/rabbitmq/rabbitmq@tower3’ for reading: No such file or directory | |
| tail: no files remaining | |
| [vagrant@tower3 ~]$ sudo tail -f /var/log/rabbitmq/rabbitmq\@tower3.log | |
| Mirrored queue '2e1669a9-2e90-3577-a332-644a2c24352d.reply.celery.pidbox' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:05:07 === | |
| Mirrored queue '2e1669a9-2e90-3577-a332-644a2c24352d.reply.celery.pidbox' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:05:07 === | |
| Mirrored queue '2e1669a9-2e90-3577-a332-644a2c24352d.reply.celery.pidbox' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:05:07 === | |
| Mirrored queue '2e1669a9-2e90-3577-a332-644a2c24352d.reply.celery.pidbox' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| rabbit on node rabbitmq@tower1 down | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| Keep rabbitmq@tower1 listeners: the node is already back | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| node rabbitmq@tower1 down: connection_closed | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| node rabbitmq@tower1 up | |
| =ERROR REPORT==== 20-May-2018::13:08:26 === | |
| Partial partition detected: | |
| * We saw DOWN from rabbitmq@tower1 | |
| * We can still see rabbitmq@tower2 which can see rabbitmq@tower1 | |
| * pause_minority mode enabled | |
| We will therefore pause until the *entire* cluster recovers | |
| =WARNING REPORT==== 20-May-2018::13:08:26 === | |
| Cluster minority/secondary status detected - awaiting recovery | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| RabbitMQ is asked to stop... | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| Stopping RabbitMQ applications and their dependencies in the following order: | |
| rabbitmq_federation_management | |
| rabbitmq_management_visualiser | |
| rabbitmq_management | |
| rabbitmq_web_dispatch | |
| cowboy | |
| cowlib | |
| rabbitmq_management_agent | |
| rabbitmq_federation | |
| rabbit | |
| mnesia | |
| amqp_client | |
| rabbit_common | |
| os_mon | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| Stopping application 'rabbitmq_federation_management' | |
| =INFO REPORT==== 20-May-2018::13:08:26 === | |
| Stopping application 'rabbitmq_management_visualiser' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Stopping application 'rabbitmq_management' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Stopping application 'rabbitmq_web_dispatch' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Mirrored queue 'tower3' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Mirrored queue 'tower3' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Stopping application 'cowboy' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Stopping application 'cowlib' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Stopping application 'rabbitmq_management_agent' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Stopping application 'rabbitmq_federation' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Stopping application 'rabbit' | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| stopped TCP Listener on [::]:5672 | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.2011.0> (192.168.121.54:39408 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Mirrored queue 'tower3' in vhost 'tower': Synchronising: all slaves already synced | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.8714.0> (192.168.121.54:43024 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.2001.0> (192.168.121.54:39406 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.2185.0> (192.168.121.54:39422 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.2342.0> (192.168.121.54:39502 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.2042.0> (192.168.121.54:39414 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.2037.0> (192.168.121.54:39412 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.3382.0> (192.168.121.54:40068 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =ERROR REPORT==== 20-May-2018::13:08:27 === | |
| Error on AMQP connection <0.2021.0> (192.168.121.54:39410 -> 192.168.121.54:5672, vhost: 'tower', user: 'awx', state: running), channel 0: | |
| operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Mirrored queue 'celery@tower1.celery.pidbox' in vhost 'tower': Slave <rabbitmq@tower3.1.2177.0> saw deaths of mirrors <rabbitmq@tower1.1.1824.0> | |
| =INFO REPORT==== 20-May-2018::13:08:27 === | |
| Mirrored queue 'bcast.2fe402ed-554e-4938-acc6-ba87c91ee038' in vhost 'tower': Slave <rabbitmq@tower3.1.2074.0> saw deaths of mirrors <rabbitmq@tower1.1.1665.0> | |
| =ERROR REPORT==== 20-May-2018::13:08:42 === | |
| ** Generic server <0.2036.0> terminating | |
| ** Last message in was {'EXIT',<0.2035.0>,killed} | |
| ** When Server state == {state, | |
| {<0.2036.0>,rabbit_connection_helper_sup}, | |
| one_for_one, | |
| [{child,<0.2039.0>,channel_sup_sup, | |
| {rabbit_channel_sup_sup,start_link,[]}, | |
| intrinsic,infinity,supervisor, | |
| [rabbit_channel_sup_sup]}, | |
| {child,<0.2038.0>,collector, | |
| {rabbit_queue_collector,start_link, | |
| [<<"192.168.121.54:39412 -> 192.168.121.54:5672">>]}, | |
| intrinsic,30000,worker, | |
| [rabbit_queue_collector]}], | |
| undefined,10,10,[],rabbit_connection_helper_sup, | |
| []} | |
| ** Reason for termination == | |
| ** killed | |
| =WARNING REPORT==== 20-May-2018::13:08:42 === | |
| Mirrored queue 'bcast.0a0be060-24ce-4ea3-ad5d-6d49ff80aaf6' in vhost 'tower': Missing 'DOWN' message from <0.2111.0> in node rabbitmq@tower3 | |
| =WARNING REPORT==== 20-May-2018::13:08:42 === | |
| Mirrored queue 'celeryev.8d9dcb9e-fafa-472d-a785-fc55519246de' in vhost 'tower': Missing 'DOWN' message from <33305.2186.0> in node rabbitmq@tower2 | |
| =WARNING REPORT==== 20-May-2018::13:08:57 === | |
| Mirrored queue 'bcast.0a0be060-24ce-4ea3-ad5d-6d49ff80aaf6' in vhost 'tower': Missing 'DOWN' message from <33305.2212.0> in node rabbitmq@tower2 | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Stopping message store for directory '/var/lib/rabbitmq/mnesia/rabbitmq@tower3/msg_store_persistent' | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Message store for directory '/var/lib/rabbitmq/mnesia/rabbitmq@tower3/msg_store_persistent' is stopped | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Stopping message store for directory '/var/lib/rabbitmq/mnesia/rabbitmq@tower3/msg_store_transient' | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Message store for directory '/var/lib/rabbitmq/mnesia/rabbitmq@tower3/msg_store_transient' is stopped | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Stopping application 'mnesia' | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Stopping application 'amqp_client' | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Stopping application 'rabbit_common' | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Stopping application 'os_mon' | |
| =INFO REPORT==== 20-May-2018::13:08:57 === | |
| Successfully stopped RabbitMQ and its dependencies | |
| =ERROR REPORT==== 20-May-2018::13:08:58 === | |
| Mnesia(rabbitmq@tower3): ** ERROR ** mnesia_event got {inconsistent_database, starting_partitioned_network, rabbitmq@tower1} | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| Starting RabbitMQ 3.6.9 on Erlang 19.0.4 | |
| Copyright (C) 2007-2016 Pivotal Software, Inc. | |
| Licensed under the MPL. See http://www.rabbitmq.com/ | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| node : rabbitmq@tower3 | |
| home dir : /var/lib/rabbitmq | |
| config file(s) : /etc/rabbitmq/rabbitmq.config | |
| cookie hash : eYNeo1ItP5mobegiCXaTyg== | |
| log : /var/log/rabbitmq/rabbitmq@tower3.log | |
| sasl log : /var/log/rabbitmq/rabbitmq@tower3-sasl.log | |
| database dir : /var/lib/rabbitmq/mnesia/rabbitmq@tower3 | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| Memory limit set to 735MB of 1839MB total. | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| Disk free limit set to 50MB | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| Limiting to approx 924 file handles (829 sockets) | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| FHC read buffering: OFF | |
| FHC write buffering: ON | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| Waiting for Mnesia tables for 30000 ms, 9 retries left | |
| =INFO REPORT==== 20-May-2018::13:08:58 === | |
| Waiting for Mnesia tables for 30000 ms, 9 retries left | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Waiting for Mnesia tables for 30000 ms, 9 retries left | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Starting rabbit_node_monitor | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Management plugin: using rates mode 'basic' | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| rabbit on node rabbitmq@tower1 up | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| msg_store_transient: using rabbit_msg_store_ets_index to provide index | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| msg_store_persistent: using rabbit_msg_store_ets_index to provide index | |
| =WARNING REPORT==== 20-May-2018::13:08:59 === | |
| msg_store_persistent: recovery terms differ from present | |
| rebuilding indices from scratch | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'asgi:tower:websocket:websocket.receive' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15616.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'asgi:tower:websocket:websocket.connect' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15620.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'tower' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15624.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'celery@tower2.celery.pidbox' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15628.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'callback_tasks' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15632.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'celeryev.c48d3f78-8239-4643-b78c-ad1f94fbb221' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15636.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'celeryev.8d9dcb9e-fafa-472d-a785-fc55519246de' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15640.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'tower_scheduler' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15644.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'bcast.2fe402ed-554e-4938-acc6-ba87c91ee038' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15648.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'asgi:tower:websocket:websocket.disconnect' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15652.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'tower1' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15656.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'tower2' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15660.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Mirrored queue 'tower3' in vhost 'tower': Adding mirror on node rabbitmq@tower3: <0.15664.0> | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| started TCP Listener on [::]:5672 | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| rabbit on node rabbitmq@tower1 up | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| rabbit on node rabbitmq@tower2 up | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Management plugin started. Port: 15672 | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Statistics database started. | |
| =INFO REPORT==== 20-May-2018::13:08:59 === | |
| Server startup complete; 9 plugins started. | |
| * rabbitmq_federation_management | |
| * rabbitmq_management_visualiser | |
| * rabbitmq_management | |
| * rabbitmq_management_agent | |
| * rabbitmq_web_dispatch | |
| * cowboy | |
| * rabbitmq_federation | |
| * amqp_client | |
| * cowlib | |
| =INFO REPORT==== 20-May-2018::13:09:00 === | |
| Mirrored queue 'celeryev.8d9dcb9e-fafa-472d-a785-fc55519246de' in vhost 'tower': Slave <rabbitmq@tower3.1.15640.0> saw deaths of mirrors <rabbitmq@tower2.1.2186.0> | |
| =INFO REPORT==== 20-May-2018::13:09:02 === | |
| accepting AMQP connection <0.15915.0> (192.168.121.54:46764 -> 192.168.121.54:5672) | |
| =INFO REPORT==== 20-May-2018::13:09:02 === | |
| connection <0.15915.0> (192.168.121.54:46764 -> 192.168.121.54:5672): user 'awx' authenticated and granted access to vhost 'tower' | |
| =INFO REPORT==== 20-May-2018::13:09:02 === | |
| Mirrored queue 'celery@tower1.celery.pidbox' in vhost 'tower': Slave <rabbitmq@tower3.1.15928.0> saw deaths of mirrors <rabbitmq@tower2.1.2265.0> | |
| =INFO REPORT==== 20-May-2018::13:09:02 === | |
| Mirrored queue 'bcast.0a0be060-24ce-4ea3-ad5d-6d49ff80aaf6' in vhost 'tower': Slave <rabbitmq@tower3.1.15937.0> saw deaths of mirrors <rabbitmq@tower2.1.2212.0> | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| accepting AMQP connection <0.15950.0> (192.168.121.54:46766 -> 192.168.121.54:5672) | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| connection <0.15950.0> (192.168.121.54:46766 -> 192.168.121.54:5672): user 'awx' authenticated and granted access to vhost 'tower' | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| accepting AMQP connection <0.15955.0> (192.168.121.54:46768 -> 192.168.121.54:5672) | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| connection <0.15955.0> (192.168.121.54:46768 -> 192.168.121.54:5672): user 'awx' authenticated and granted access to vhost 'tower' | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Adding mirror on node rabbitmq@tower1: <33304.12204.0> | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Adding mirror on node rabbitmq@tower2: <33305.12191.0> | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:06 === | |
| Mirrored queue '9bcec8f7-f66a-33e2-aeec-e3806003a72d.reply.celery.pidbox' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Adding mirror on node rabbitmq@tower1: <33304.12245.0> | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Adding mirror on node rabbitmq@tower2: <33305.12205.0> | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celeryev.ae27aa66-1728-4e19-8144-6d782c8129dc' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Adding mirror on node rabbitmq@tower1: <33304.12268.0> | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Adding mirror on node rabbitmq@tower2: <33305.12222.0> | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'bcast.106fb63d-9e5e-4cd1-8caf-16a4af592ec3' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Adding mirror on node rabbitmq@tower1: <33304.12290.0> | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Adding mirror on node rabbitmq@tower2: <33305.12234.0> | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Synchronising: 0 messages to synchronise | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Synchronising: batch size: 4096 | |
| =INFO REPORT==== 20-May-2018::13:09:07 === | |
| Mirrored queue 'celery@tower3.celery.pidbox' in vhost 'tower': Synchronising: all slaves already synced | |
| =INFO REPORT==== 20-May-2018::13:09:08 === | |
| accepting AMQP connection <0.16066.0> (192.168.121.54:46770 -> 192.168.121.54:5672) | |
| =INFO REPORT==== 20-May-2018::13:09:08 === | |
| connection <0.16066.0> (192.168.121.54:46770 -> 192.168.121.54:5672): user 'awx' authenticated and granted access to vhost 'tower' | |
| =INFO REPORT==== 20-May-2018::13:09:09 === | |
| accepting AMQP connection <0.16076.0> (192.168.121.54:46774 -> 192.168.121.54:5672) | |
| =INFO REPORT==== 20-May-2018::13:09:09 === | |
| connection <0.16076.0> (192.168.121.54:46774 -> 192.168.121.54:5672): user 'awx' authenticated and granted access to vhost 'tower' |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment