Skip to content

Instantly share code, notes, and snippets.

/gist:1134288

Created August 9, 2011 15:01
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anonymous/1134288 to your computer and use it in GitHub Desktop.
Save anonymous/1134288 to your computer and use it in GitHub Desktop.
Download ZIP
ES two masters
Raw
gistfile1.txt
=======================================================================================================================================================
Config
=======================================================================================================================================================
cluster:
name: default-cluster-name
index:
number_of_shards: 2
number_of_replicas: 1
discovery:
type: ec2
zen:
minimum_master_nodes: 1
cloud:
aws:
access_key: XXXXXXXXXX
secret_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
=======================================================================================================================================================
Trace from node "Punisher", ip 10.59.59.235
=======================================================================================================================================================
2011-08-09 14:04:26,393 TRACE - [Punisher] building dynamic unicast discovery nodes...
2011-08-09 14:04:26,401 TRACE - [Punisher] adding i-36f7c240, address 10.49.61.119, transport_address inet[/10.49.61.119:9300]
2011-08-09 14:04:26,403 TRACE - [Punisher] adding i-e0d1e496, address 10.48.198.181, transport_address inet[/10.48.198.181:9300]
2011-08-09 14:04:26,405 TRACE - [Punisher] adding i-a6d8eed0, address 10.49.51.91, transport_address inet[/10.49.51.91:9300]
2011-08-09 14:04:26,407 TRACE - [Punisher] adding i-96d8eee0, address 10.59.59.235, transport_address inet[/10.59.59.235:9300]
2011-08-09 14:04:26,407 DEBUG - [Punisher] using dynamic discovery nodes [[#cloud-i-36f7c240-0][inet[/10.49.61.119:9300]], [#cloud-i-e0d1e496-0][inet[/10.48.198.181:9300]], [#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]], [#cloud-i-96d8eee0-0][inet[/10.59.59.235:9300]]]
2011-08-09 14:04:26,469 DEBUG - [Punisher] Connected to node [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]]
2011-08-09 14:04:26,470 TRACE - [Punisher] [1] connecting to [Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]
2011-08-09 14:04:26,518 TRACE - [Punisher] [1] received response from [Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]: [ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}]
2011-08-09 14:04:26,530 DEBUG - [Punisher] Connected to node [[#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]]]
2011-08-09 14:04:26,530 TRACE - [Punisher] [1] connecting to [#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]]
2011-08-09 14:04:26,533 TRACE - [Punisher] [1] received response from [#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]]: [ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]], master [null], cluster_name[default-cluster-name]}]
2011-08-09 14:04:29,595 TRACE - [Punisher] building dynamic unicast discovery nodes...
2011-08-09 14:04:29,598 TRACE - [Punisher] adding i-36f7c240, address 10.49.61.119, transport_address inet[/10.49.61.119:9300]
2011-08-09 14:04:29,600 TRACE - [Punisher] adding i-e0d1e496, address 10.48.198.181, transport_address inet[/10.48.198.181:9300]
2011-08-09 14:04:29,601 TRACE - [Punisher] adding i-a6d8eed0, address 10.49.51.91, transport_address inet[/10.49.51.91:9300]
2011-08-09 14:04:29,603 TRACE - [Punisher] adding i-96d8eee0, address 10.59.59.235, transport_address inet[/10.59.59.235:9300]
2011-08-09 14:04:29,603 DEBUG - [Punisher] using dynamic discovery nodes [[#cloud-i-36f7c240-0][inet[/10.49.61.119:9300]], [#cloud-i-e0d1e496-0][inet[/10.48.198.181:9300]], [#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]], [#cloud-i-96d8eee0-0][inet[/10.59.59.235:9300]]]
2011-08-09 14:04:29,609 TRACE - [Punisher] [1] connecting to [#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]]
2011-08-09 14:04:29,609 TRACE - [Punisher] [1] connecting to [Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]
2011-08-09 14:04:29,611 TRACE - [Punisher] [1] received response from [Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]: [ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}]
2011-08-09 14:04:29,612 TRACE - [Punisher] [1] received response from [#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]]: [ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]], master [null], cluster_name[default-cluster-name]}]
2011-08-09 14:04:44,652 DEBUG - [Punisher] Disconnected from [[#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]]]
2011-08-09 14:04:44,656 DEBUG - [Punisher] Disconnected from [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]]
2011-08-09 14:04:44,656 DEBUG - [Punisher] ping responses:
--> target [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]], master [null]
2011-08-09 14:04:44,659 DEBUG - [Punisher] processing [zen-disco-join (elected_as_master)]: execute
2011-08-09 14:04:44,659 TRACE - [Punisher] cluster state updated:
version [1], source [zen-disco-join (elected_as_master)]
nodes:
[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]], local, master
routing_table:
routing_nodes:
---- unassigned
2011-08-09 14:04:44,660 INFO - [Punisher] new_master [Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]], reason: zen-disco-join (elected_as_master)
2011-08-09 14:04:44,668 DEBUG - [Punisher] Connected to node [[Punisher][fMzyq9N2SZiq8xtXF2VMvQ][inet[/10.59.59.235:9300]]]
2011-08-09 14:04:44,670 TRACE - [Punisher] initial state set from discovery
2011-08-09 14:04:44,670 INFO - [Punisher] default-cluster-name/fMzyq9N2SZiq8xtXF2VMvQ
2011-08-09 14:04:44,677 DEBUG - [Punisher] no state elected
2011-08-09 14:04:44,680 INFO - [Punisher] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.59.59.235:9200]}
=======================================================================================================================================================
Trace from node "Agent", ip 10.49.51.91
=======================================================================================================================================================
2011-08-09 14:04:16,188 TRACE - [Agent] building dynamic unicast discovery nodes...
2011-08-09 14:04:16,192 TRACE - [Agent] adding i-36f7c240, address 10.49.61.119, transport_address inet[/10.49.61.119:9300]
2011-08-09 14:04:16,194 TRACE - [Agent] adding i-e0d1e496, address 10.48.198.181, transport_address inet[/10.48.198.181:9300]
2011-08-09 14:04:16,248 TRACE - [Agent] adding i-a6d8eed0, address 10.49.51.91, transport_address inet[/10.49.51.91:9300]
2011-08-09 14:04:16,253 TRACE - [Agent] adding i-96d8eee0, address 10.59.59.235, transport_address inet[/10.59.59.235:9300]
2011-08-09 14:04:16,253 DEBUG - [Agent] using dynamic discovery nodes [[#cloud-i-36f7c240-0][inet[/10.49.61.119:9300]], [#cloud-i-e0d1e496-0][inet[/10.48.198.181:9300]], [#cloud-i-a6d8eed0-0][inet[/10.49.51.91:9300]], [#cloud-i-96d8eee0-0][inet[/10.59.59.235:9300]]]
2011-08-09 14:04:16,254 TRACE - [Agent] [1] connecting to [Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]
2011-08-09 14:04:16,255 TRACE - [Agent] [1] received response from [Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]: [ping_response{target [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]], master [null], cluster_name[default-cluster-name]}, ping_response{target [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]], master [null], cluster_name[default-cluster-name]}]
2011-08-09 14:04:16,324 TRACE - [Agent] (Ignoring) Exception caught on netty layer [[id: 0x43900f17]]
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
2011-08-09 14:04:16,325 TRACE - [Agent] [1] failed to connect to [#cloud-i-96d8eee0-0][inet[/10.59.59.235:9300]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/10.59.59.235:9300]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:535)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:501)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:480)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:125)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:267)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
... 3 more
2011-08-09 14:04:31,329 DEBUG - [Agent] Disconnected from [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]]
2011-08-09 14:04:31,330 DEBUG - [Agent] ping responses: {none}
2011-08-09 14:04:31,611 DEBUG - [Agent] processing [zen-disco-join (elected_as_master)]: execute
2011-08-09 14:04:31,611 TRACE - [Agent] cluster state updated:
version [1], source [zen-disco-join (elected_as_master)]
nodes:
[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]], local, master
routing_table:
routing_nodes:
---- unassigned
2011-08-09 14:04:31,613 INFO - [Agent] new_master [Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]], reason: zen-disco-join (elected_as_master)
2011-08-09 14:04:31,713 DEBUG - [Agent] Connected to node [[Agent][jPtPjtcLT1qyfCK_9nNlkA][inet[/10.49.51.91:9300]]]
2011-08-09 14:04:31,715 DEBUG - [Agent] processing [zen-disco-join (elected_as_master)]: done applying updated cluster_state
2011-08-09 14:04:31,715 TRACE - [Agent] initial state set from discovery
2011-08-09 14:04:31,716 INFO - [Agent] default-cluster-name/jPtPjtcLT1qyfCK_9nNlkA
2011-08-09 14:04:31,716 DEBUG - [Agent] processing [reroute_rivers_node_changed]: execute
2011-08-09 14:04:31,716 DEBUG - [Agent] processing [reroute_rivers_node_changed]: no change in cluster_state
2011-08-09 14:04:31,724 DEBUG - [Agent] no state elected
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment