Skip to content

Instantly share code, notes, and snippets.

@hub-cap

hub-cap/elasticsearch0

Last active August 16, 2017 14:04
Show Gist options
  • Save hub-cap/b848213c41919259701405853d384f16 to your computer and use it in GitHub Desktop.
Save hub-cap/b848213c41919259701405853d384f16 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
elasticsearch0
elasticsearch0 | [2017-08-16T14:02:05,010][TRACE][o.e.d.z.ZenDiscovery ] [QtBjZWS] full ping responses:
elasticsearch0 | --> ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[9], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}
elasticsearch0 | [2017-08-16T14:02:05,010][DEBUG][o.e.d.z.ZenDiscovery ] [QtBjZWS] filtered ping responses: (ignore_non_masters [false])
elasticsearch0 | --> ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[9], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}
elasticsearch0 | --> ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[9], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}
elasticsearch0 | [2017-08-16T14:02:05,014][TRACE][o.e.d.z.ZenDiscovery ] [QtBjZWS] candidate Candidate{node={QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, clusterStateVersion=-1} won election
elasticsearch0 | [2017-08-16T14:02:05,015][DEBUG][o.e.d.z.ZenDiscovery ] [QtBjZWS] elected as master, waiting for incoming joins ([1] needed)
elasticsearch0 | [2017-08-16T14:02:05,016][TRACE][o.e.d.z.NodeJoinController] [QtBjZWS] not enough joins for election. Got [0], required [1]
elasticsearch0 | [2017-08-16T14:02:05,103][DEBUG][o.e.t.n.Netty4Transport ] [QtBjZWS] connected to node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:05,123][TRACE][o.e.d.z.NodeJoinController] [QtBjZWS] have enough joins for election. Got [1], required [1]
elasticsearch0 | [2017-08-16T14:02:05,138][INFO ][o.e.c.s.ClusterService ] [QtBjZWS] new_master {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, added {{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300},}, reason: zen-disco-elected-as-master ([1] nodes joined)[{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:05,158][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [1]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:05,159][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [1]
elasticsearch0 | [2017-08-16T14:02:05,159][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [RkOa5nYkTRqctppoQB_EqA], version [1]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:05,196][INFO ][o.e.h.n.Netty4HttpServerTransport] [QtBjZWS] publish_address {172.18.0.3:9200}, bound_addresses {172.18.0.3:9200}
elasticsearch0 | [2017-08-16T14:02:05,196][INFO ][o.e.n.Node ] [QtBjZWS] started
elasticsearch0 | [2017-08-16T14:02:05,385][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [2]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:05,386][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [2]
elasticsearch0 | [2017-08-16T14:02:05,386][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [ryodRcn5TJSr8J9GZ585-w], version [2]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:05,824][INFO ][o.e.l.LicenseService ] [QtBjZWS] license [bd809bd5-f17d-400e-8723-4d67dbc18999] mode [trial] - valid
elasticsearch0 | [2017-08-16T14:02:05,834][INFO ][o.e.g.GatewayService ] [QtBjZWS] recovered [2] indices into cluster_state
elasticsearch0 | [2017-08-16T14:02:05,849][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [3]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:05,850][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [3]
elasticsearch0 | [2017-08-16T14:02:05,850][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [GlkMf3GCTvybWaqMs5H8Ug], version [3]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:06,157][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [4]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:06,157][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [4]
elasticsearch0 | [2017-08-16T14:02:06,157][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [oKLVIIR8SgK0Op2pMwvauw], version [4]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:06,235][INFO ][o.e.c.r.a.AllocationService] [QtBjZWS] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.monitoring-es-6-2017.08.15][0]] ...]).
elasticsearch0 | [2017-08-16T14:02:06,237][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [5]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:06,238][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [5]
elasticsearch0 | [2017-08-16T14:02:06,238][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [0a-Qdz8lQ-22SwEBY0GKAQ], version [5]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:06,285][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [6]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:06,285][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [6]
elasticsearch0 | [2017-08-16T14:02:06,285][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [vSfARXDvT3yHb6WtnbniRQ], version [6]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:06,370][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [7]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:06,370][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [7]
elasticsearch0 | [2017-08-16T14:02:06,370][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [UOzHXMW8SKi3ZlznJR5TLw], version [7]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:06,418][INFO ][o.e.c.r.a.AllocationService] [QtBjZWS] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.monitoring-es-6-2017.08.15][0]] ...]).
elasticsearch0 | [2017-08-16T14:02:06,420][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [8]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:06,420][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [8]
elasticsearch0 | [2017-08-16T14:02:06,420][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [SkT0-awRRIqry2dcpKCWSA], version [8]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:14,853][WARN ][o.e.t.n.Netty4Transport ] [QtBjZWS] exception caught on transport layer [[id: 0x3d8753f1, L:/172.18.0.3:60570 - R:elasticsearch1/172.18.0.2:9300]], closing connection
elasticsearch0 | java.lang.IllegalStateException: Message not fully read (response) for requestId [67], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction$1@158bbfb3], error [false]; resetting
elasticsearch0 | at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1410) ~[elasticsearch-5.5.1.jar:5.5.1]
elasticsearch0 | at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-5.5.1.jar:5.5.1]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
], id[7], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[10], master [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[11], master [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}],cluster_state_version [8], cluster_name[docker-cluster]}]
elasticsearch0 | [2017-08-16T14:02:15,188][TRACE][o.e.d.z.MasterFaultDetection] [QtBjZWS] checking ping from {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} under a cluster state thread
elasticsearch0 | [2017-08-16T14:02:18,206][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] sending to {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}
elasticsearch0 | [2017-08-16T14:02:18,206][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] sending to {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}
elasticsearch0 | [2017-08-16T14:02:18,206][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] received response from {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}: [ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[4], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[7], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[7], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[10], master [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[12], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[13], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[14], master [null],cluster_state_version [8], cluster_name[docker-cluster]}]
elasticsearch0 | [2017-08-16T14:02:18,207][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] received response from {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}: [ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[4], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[7], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[7], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[10], master [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[12], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[13], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[14], master [null],cluster_state_version [8], cluster_name[docker-cluster]}]
elasticsearch0 | [2017-08-16T14:02:21,539][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] sending to {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}
elasticsearch0 | [2017-08-16T14:02:21,539][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] sending to {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}
elasticsearch0 | [2017-08-16T14:02:21,539][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] received response from {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}: [ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[7], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[10], master [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[12], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[13], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[15], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[16], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[17], master [null],cluster_state_version [8], cluster_name[docker-cluster]}]
elasticsearch0 | [2017-08-16T14:02:21,542][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [2] received response from {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}: [ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[7], master [null],cluster_state_version [-1], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[10], master [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[12], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[13], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[15], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[16], master [null],cluster_state_version [8], cluster_name[docker-cluster]}, ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[17], master [null],cluster_state_version [8], cluster_name[docker-cluster]}]
elasticsearch0 | [2017-08-16T14:02:24,873][TRACE][o.e.d.z.ZenDiscovery ] [QtBjZWS] full ping responses:
elasticsearch0 | --> ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[18], master [null],cluster_state_version [8], cluster_name[docker-cluster]}
elasticsearch0 | [2017-08-16T14:02:24,873][DEBUG][o.e.d.z.ZenDiscovery ] [QtBjZWS] filtered ping responses: (ignore_non_masters [false])
elasticsearch0 | --> ping_response{node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], id[18], master [null],cluster_state_version [8], cluster_name[docker-cluster]}
elasticsearch0 | --> ping_response{node [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], id[19], master [null],cluster_state_version [8], cluster_name[docker-cluster]}
elasticsearch0 | [2017-08-16T14:02:24,873][TRACE][o.e.d.z.ZenDiscovery ] [QtBjZWS] candidate Candidate{node={QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, clusterStateVersion=8} won election
elasticsearch0 | [2017-08-16T14:02:24,873][DEBUG][o.e.d.z.ZenDiscovery ] [QtBjZWS] elected as master, waiting for incoming joins ([1] needed)
elasticsearch0 | [2017-08-16T14:02:24,873][TRACE][o.e.d.z.NodeJoinController] [QtBjZWS] not enough joins for election. Got [0], required [1]
elasticsearch0 | [2017-08-16T14:02:25,205][TRACE][o.e.d.z.NodeJoinController] [QtBjZWS] have enough joins for election. Got [1], required [1]
elasticsearch0 | [2017-08-16T14:02:25,205][DEBUG][o.e.d.z.NodeJoinController] [QtBjZWS] received a join request for an existing node [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:25,206][INFO ][o.e.c.s.ClusterService ] [QtBjZWS] new_master {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, reason: zen-disco-elected-as-master ([1] nodes joined)[{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:25,208][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] master node {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300} acked cluster state version [9]. processing ... (current pending [1], needed [1])
elasticsearch0 | [2017-08-16T14:02:25,209][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] committing version [9]
elasticsearch0 | [2017-08-16T14:02:25,209][TRACE][o.e.d.z.PublishClusterStateAction] [QtBjZWS] sending commit for cluster state (uuid: [OGCOgs2qTeyKkauQTI_fIw], version [9]) to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:34,823][WARN ][o.e.t.n.Netty4Transport ] [QtBjZWS] exception caught on transport layer [[id: 0x7539c114, L:/172.18.0.3:60592 - R:elasticsearch1/172.18.0.2:9300]], closing connection
elasticsearch0 | java.lang.IllegalStateException: Message not fully read (response) for requestId [104], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction$1@5375e69b], error [false]; resetting
elasticsearch0 | at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1410) ~[elasticsearch-5.5.1.jar:5.5.1]
elasticsearch0 | at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-5.5.1.jar:5.5.1]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.11.Final.jar:4.1.11.Final]
elasticsearch0 | at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
elasticsearch0 | [2017-08-16T14:02:34,825][DEBUG][o.e.t.n.Netty4Transport ] [QtBjZWS] disconnecting from [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}], channel closed
elasticsearch0 | [2017-08-16T14:02:34,827][TRACE][o.e.d.z.NodesFaultDetection] [QtBjZWS] [node ] [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}] transport disconnected
elasticsearch0 | [2017-08-16T14:02:34,828][WARN ][o.e.d.z.ZenDiscovery ] [QtBjZWS] not enough master nodes (has [1], but needed [2]), current nodes: nodes:
elasticsearch0 | {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, local, master
elasticsearch0 | {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}
elasticsearch0 |
elasticsearch0 | [2017-08-16T14:02:34,828][TRACE][o.e.d.z.NodeJoinController] [QtBjZWS] starting an election context, will accumulate joins
elasticsearch0 | [2017-08-16T14:02:34,829][TRACE][o.e.d.z.ZenDiscovery ] [QtBjZWS] starting to ping
elasticsearch0 | [2017-08-16T14:02:34,829][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] resolved host [elasticsearch0] to [172.18.0.3:9300]
elasticsearch0 | [2017-08-16T14:02:34,829][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] resolved host [elasticsearch1] to [172.18.0.2:9300]
elasticsearch0 | [2017-08-16T14:02:34,832][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [3] opening connection to [{YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}]
elasticsearch0 | [2017-08-16T14:02:34,832][TRACE][o.e.d.z.UnicastZenPing ] [QtBjZWS] [3] sending to {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}
Raw
elasticsearch1
elasticsearch1 | [2017-08-16T14:02:05,163][INFO ][o.e.c.s.ClusterService ] [YfH0pll] detected_master {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, added {{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300},}, reason: zen-disco-receive(from master [master {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300} committed version [1]])
elasticsearch1 | [2017-08-16T14:02:05,176][INFO ][o.e.h.n.Netty4HttpServerTransport] [YfH0pll] publish_address {172.18.0.2:9200}, bound_addresses {172.18.0.2:9200}
elasticsearch1 | [2017-08-16T14:02:05,176][INFO ][o.e.n.Node ] [YfH0pll] started
elasticsearch1 | [2017-08-16T14:02:05,611][INFO ][o.e.l.LicenseService ] [YfH0pll] license [bd809bd5-f17d-400e-8723-4d67dbc18999] mode [trial] - valid
elasticsearch1 | [2017-08-16T14:02:15,195][INFO ][o.e.d.z.ZenDiscovery ] [YfH0pll] master_left [{QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}], reason [no longer master]
elasticsearch1 | org.elasticsearch.transport.RemoteTransportException: [QtBjZWS][172.18.0.3:9300][internal:discovery/zen/fd/master_ping]
elasticsearch1 | Caused by: org.elasticsearch.cluster.NotMasterException: local node is not master
elasticsearch1 | [2017-08-16T14:02:15,199][WARN ][o.e.d.z.ZenDiscovery ] [YfH0pll] master left (reason = no longer master), current nodes: nodes:
elasticsearch1 | {YfH0pll}{YfH0pllrTLOvl9kWRlR5HQ}{v9vf5XpmR3ix03zvftqLXA}{elasticsearch1}{172.18.0.2:9300}, local
elasticsearch1 | {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, master
elasticsearch1 |
elasticsearch1 | [2017-08-16T14:02:25,210][INFO ][o.e.c.s.ClusterService ] [YfH0pll] detected_master {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300}, reason: zen-disco-receive(from master [master {QtBjZWS}{QtBjZWS2QDiwYd30394TgQ}{_BxOtw3aR3Wxz4TntFhCgA}{elasticsearch0}{172.18.0.3:9300} committed version [9]])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment