Skip to content

Instantly share code, notes, and snippets.

@Rostry
Created July 26, 2018 13:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Rostry/63c246c4601bf320984daacf2155b87a to your computer and use it in GitHub Desktop.
Save Rostry/63c246c4601bf320984daacf2155b87a to your computer and use it in GitHub Desktop.
LiferayElasticSearch.log
[2018-07-25 16:42:00,071][INFO ][node ] [lisyx] version[2.4.0], pid[1339], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 16:42:00,072][INFO ][node ] [lisyx] initializing ...
[2018-07-25 16:42:00,686][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 16:42:00,709][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 16:42:00,709][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 16:42:02,720][INFO ][node ] [lisyx] initialized
[2018-07-25 16:42:02,720][INFO ][node ] [lisyx] starting ...
[2018-07-25 16:42:02,800][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 16:42:02,805][INFO ][discovery ] [lisyx] LiferayElasticSearch/Io3FwQftRFqEX5bkITM-kw
[2018-07-25 16:42:32,810][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 16:42:32,813][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 16:42:32,813][INFO ][node ] [lisyx] started
[2018-07-25 17:42:17,653][INFO ][node ] [lisyx] stopping ...
[2018-07-25 17:42:17,675][INFO ][node ] [lisyx] stopped
[2018-07-25 17:42:17,675][INFO ][node ] [lisyx] closing ...
[2018-07-25 17:42:17,693][INFO ][node ] [lisyx] closed
[2018-07-25 17:53:34,694][INFO ][node ] [lisyx] version[2.4.0], pid[1427], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 17:53:34,695][INFO ][node ] [lisyx] initializing ...
[2018-07-25 17:53:35,385][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 17:53:35,412][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 17:53:35,412][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 17:53:37,424][INFO ][node ] [lisyx] initialized
[2018-07-25 17:53:37,425][INFO ][node ] [lisyx] starting ...
[2018-07-25 17:53:37,538][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 17:53:37,543][INFO ][discovery ] [lisyx] LiferayElasticSearch/p1nUBoGLTDyExrvcuC4owQ
[2018-07-25 17:54:07,556][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 17:54:07,560][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 17:54:07,561][INFO ][node ] [lisyx] started
[2018-07-25 17:56:34,892][INFO ][node ] [lisyx] stopping ...
[2018-07-25 17:56:36,148][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{lisyx}{p1nUBoGLTDyExrvcuC4owQ}{172.38.1.37}{172.38.1.37:9300}{master=true}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2$1.doRun(UnicastZenPing.java:253)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 17:56:36,148][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{#zen_unicast_1#}{172.38.1.37}{172.38.1.37:9300}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2$1.doRun(UnicastZenPing.java:253)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 17:56:37,954][WARN ][discovery.zen.ping.unicast] [lisyx] [1] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,954][WARN ][discovery.zen.ping.unicast] [lisyx] [1] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,955][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,955][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,956][WARN ][discovery.zen.ping.unicast] [lisyx] [4] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,955][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,956][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,956][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,956][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:37,961][INFO ][node ] [lisyx] stopped
[2018-07-25 17:56:37,961][INFO ][node ] [lisyx] closing ...
[2018-07-25 17:56:37,969][INFO ][node ] [lisyx] closed
[2018-07-25 17:58:03,770][INFO ][node ] [lisyx] version[2.4.0], pid[1246], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 17:58:03,772][INFO ][node ] [lisyx] initializing ...
[2018-07-25 17:58:04,461][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 17:58:04,487][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 17:58:04,487][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 17:58:06,285][INFO ][node ] [lisyx] initialized
[2018-07-25 17:58:06,285][INFO ][node ] [lisyx] starting ...
[2018-07-25 17:58:06,351][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 17:58:06,356][INFO ][discovery ] [lisyx] LiferayElasticSearch/YrCN_k8-TSOp14UP0_MI2A
[2018-07-25 17:58:36,358][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 17:58:36,362][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 17:58:36,362][INFO ][node ] [lisyx] started
[2018-07-25 17:59:50,801][DEBUG][action.admin.cluster.state] [lisyx] no known master node, scheduling a retry
[2018-07-25 18:00:20,803][DEBUG][action.admin.cluster.state] [lisyx] timed out while retrying [cluster:monitor/state] after failure (timeout [30s])
[2018-07-25 18:00:20,816][WARN ][rest.suppressed ] path: /_cluster/state, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:02:49,485][DEBUG][action.admin.cluster.state] [lisyx] no known master node, scheduling a retry
[2018-07-25 18:03:19,485][DEBUG][action.admin.cluster.state] [lisyx] timed out while retrying [cluster:monitor/state] after failure (timeout [30s])
[2018-07-25 18:03:19,486][WARN ][rest.suppressed ] path: /_cluster/state, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:03:54,473][DEBUG][action.admin.cluster.state] [lisyx] no known master node, scheduling a retry
[2018-07-25 18:04:24,473][DEBUG][action.admin.cluster.state] [lisyx] timed out while retrying [cluster:monitor/state] after failure (timeout [30s])
[2018-07-25 18:04:24,474][WARN ][rest.suppressed ] path: /_cluster/state, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:38:42,310][DEBUG][action.admin.cluster.state] [lisyx] no known master node, scheduling a retry
[2018-07-25 18:39:00,889][INFO ][node ] [lisyx] stopping ...
[2018-07-25 18:39:00,922][WARN ][rest.suppressed ] path: /_cluster/state, params: {pretty=}
NodeClosedException[node closed {lisyx}{YrCN_k8-TSOp14UP0_MI2A}{172.38.1.37}{172.38.1.37:9300}{master=true}]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onClusterServiceClose(TransportMasterNodeAction.java:228)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onClose(ClusterStateObserver.java:222)
at org.elasticsearch.cluster.service.InternalClusterService.doStop(InternalClusterService.java:214)
at org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:88)
at org.elasticsearch.node.Node.stop(Node.java:326)
at org.elasticsearch.node.Node.close(Node.java:351)
at org.elasticsearch.bootstrap.Bootstrap$4.run(Bootstrap.java:173)
[2018-07-25 18:39:00,923][WARN ][netty.channel.DefaultChannelPipeline] An exception was thrown by an exception handler.
java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36)
at org.jboss.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34)
at org.jboss.netty.channel.DefaultChannelPipeline.execute(DefaultChannelPipeline.java:636)
at org.jboss.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496)
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46)
at org.jboss.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:781)
at org.jboss.netty.channel.Channels.write(Channels.java:725)
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71)
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59)
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784)
at org.elasticsearch.http.netty.pipelining.HttpPipeliningHandler.handleDownstream(HttpPipeliningHandler.java:87)
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
at org.elasticsearch.http.netty.NettyHttpChannel.sendResponse(NettyHttpChannel.java:147)
at org.elasticsearch.http.HttpServer$ResourceHandlingHttpChannel.sendResponse(HttpServer.java:352)
at org.elasticsearch.rest.action.support.RestActionListener.onFailure(RestActionListener.java:60)
at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:95)
at org.elasticsearch.action.support.ThreadedActionListener$2.doRun(ThreadedActionListener.java:104)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:01,061][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{#zen_unicast_1#}{172.38.1.37}{172.38.1.37:9300}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.doRun(UnicastZenPing.java:249)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 18:39:01,061][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{lisyx}{YrCN_k8-TSOp14UP0_MI2A}{172.38.1.37}{172.38.1.37:9300}{master=true}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.doRun(UnicastZenPing.java:249)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 18:39:02,561][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{#zen_unicast_1#}{172.38.1.37}{172.38.1.37:9300}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2$1.doRun(UnicastZenPing.java:253)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 18:39:02,561][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{lisyx}{YrCN_k8-TSOp14UP0_MI2A}{172.38.1.37}{172.38.1.37:9300}{master=true}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2$1.doRun(UnicastZenPing.java:253)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 18:39:10,509][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,510][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,510][WARN ][discovery.zen.ping.unicast] [lisyx] [4] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,510][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,509][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,510][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,510][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,520][WARN ][discovery.zen.ping.unicast] [lisyx] [261] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,524][WARN ][discovery.zen.ping.unicast] [lisyx] [508] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:39:10,527][INFO ][node ] [lisyx] stopped
[2018-07-25 18:39:10,528][INFO ][node ] [lisyx] closing ...
[2018-07-25 18:39:10,574][INFO ][node ] [lisyx] closed
[2018-07-25 18:40:19,303][INFO ][node ] [lisyx] version[2.4.0], pid[1235], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 18:40:19,304][INFO ][node ] [lisyx] initializing ...
[2018-07-25 18:40:20,028][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 18:40:20,057][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 18:40:20,057][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 18:40:21,875][INFO ][node ] [lisyx] initialized
[2018-07-25 18:40:21,875][INFO ][node ] [lisyx] starting ...
[2018-07-25 18:40:22,023][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 18:40:22,028][INFO ][discovery ] [lisyx] LiferayElasticSearch/L0d6ZIpoRO-FEA2l8qF64Q
[2018-07-25 18:40:52,041][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 18:40:52,045][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 18:40:52,045][INFO ][node ] [lisyx] started
[2018-07-25 18:41:26,046][DEBUG][action.admin.cluster.state] [lisyx] no known master node, scheduling a retry
[2018-07-25 18:41:47,886][DEBUG][action.admin.cluster.state] [lisyx] no known master node, scheduling a retry
[2018-07-25 18:41:56,048][DEBUG][action.admin.cluster.state] [lisyx] timed out while retrying [cluster:monitor/state] after failure (timeout [30s])
[2018-07-25 18:41:56,061][WARN ][rest.suppressed ] path: /_cluster/state, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:42:17,887][DEBUG][action.admin.cluster.state] [lisyx] timed out while retrying [cluster:monitor/state] after failure (timeout [30s])
[2018-07-25 18:42:17,888][WARN ][rest.suppressed ] path: /_cluster/state, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:53:12,225][INFO ][node ] [lisyx] version[2.4.0], pid[1352], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 18:53:12,226][INFO ][node ] [lisyx] initializing ...
[2018-07-25 18:53:12,854][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 18:53:12,879][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 18:53:12,879][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 18:53:14,872][INFO ][node ] [lisyx] initialized
[2018-07-25 18:53:14,873][INFO ][node ] [lisyx] starting ...
[2018-07-25 18:53:15,009][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 18:53:15,014][INFO ][discovery ] [lisyx] LiferayElasticSearch/0LBoRUNFQ3aTiqP5ALVvzA
[2018-07-25 18:53:45,027][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 18:53:45,031][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 18:53:45,031][INFO ][node ] [lisyx] started
[2018-07-25 18:54:38,000][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 18:55:08,002][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 18:55:08,015][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:01:12,035][INFO ][node ] [lisyx] version[2.4.0], pid[1545], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 19:01:12,035][INFO ][node ] [lisyx] initializing ...
[2018-07-25 19:01:12,642][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 19:01:12,666][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 19:01:12,666][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 19:01:14,397][INFO ][node ] [lisyx] initialized
[2018-07-25 19:01:14,397][INFO ][node ] [lisyx] starting ...
[2018-07-25 19:01:14,578][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 19:01:14,582][INFO ][discovery ] [lisyx] LiferayElasticSearch/DLSjSdOUS_mnGr5ZnnojOg
[2018-07-25 19:01:44,598][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 19:01:44,602][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 19:01:44,603][INFO ][node ] [lisyx] started
[2018-07-25 19:02:41,175][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 19:03:11,178][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 19:03:11,190][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:06:39,886][INFO ][node ] [lisyx] version[2.4.0], pid[1679], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 19:06:39,887][INFO ][node ] [lisyx] initializing ...
[2018-07-25 19:06:40,496][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 19:06:40,524][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 19:06:40,524][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 19:06:42,310][INFO ][node ] [lisyx] initialized
[2018-07-25 19:06:42,311][INFO ][node ] [lisyx] starting ...
[2018-07-25 19:06:42,415][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 19:06:42,421][INFO ][discovery ] [lisyx] LiferayElasticSearch/_0rgMrsNSA-RaPT0Zv0QFA
[2018-07-25 19:07:12,442][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 19:07:12,446][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 19:07:12,446][INFO ][node ] [lisyx] started
[2018-07-25 19:08:39,827][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 19:09:09,829][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 19:09:09,843][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:18:31,079][INFO ][node ] [lisyx] version[2.4.0], pid[1927], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 19:18:31,080][INFO ][node ] [lisyx] initializing ...
[2018-07-25 19:18:31,697][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 19:18:31,722][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 19:18:31,722][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 19:18:33,594][INFO ][node ] [lisyx] initialized
[2018-07-25 19:18:33,594][INFO ][node ] [lisyx] starting ...
[2018-07-25 19:18:33,681][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9301}, bound_addresses {172.38.1.37:9301}
[2018-07-25 19:18:33,686][INFO ][discovery ] [lisyx] LiferayElasticSearch/34FCUSgSR3-sP5D5pmLhMA
[2018-07-25 19:18:38,261][INFO ][cluster.service ] [lisyx] new_master {lisyx}{34FCUSgSR3-sP5D5pmLhMA}{172.38.1.37}{172.38.1.37:9301}{master=true}, added {{lisyx}{_0rgMrsNSA-RaPT0Zv0QFA}{172.38.1.37}{172.38.1.37:9300}{master=true},}, reason: zen-disco-join(elected_as_master, [1] joins received)
[2018-07-25 19:18:38,281][INFO ][cluster.service ] [lisyx] detected_master {lisyx}{34FCUSgSR3-sP5D5pmLhMA}{172.38.1.37}{172.38.1.37:9301}{master=true}, added {{lisyx}{34FCUSgSR3-sP5D5pmLhMA}{172.38.1.37}{172.38.1.37:9301}{master=true},}, reason: zen-disco-receive(from master [{lisyx}{34FCUSgSR3-sP5D5pmLhMA}{172.38.1.37}{172.38.1.37:9301}{master=true}])
[2018-07-25 19:18:38,354][ERROR][bootstrap ] [lisyx] Exception
BindHttpException[Failed to bind to [9200]]; nested: ChannelException[Failed to bind to: /172.38.1.37:9200]; nested: BindException[Address already in use];
at org.elasticsearch.http.netty.NettyHttpServerTransport.bindAddress(NettyHttpServerTransport.java:391)
at org.elasticsearch.http.netty.NettyHttpServerTransport.createBoundHttpAddress(NettyHttpServerTransport.java:289)
at org.elasticsearch.http.netty.NettyHttpServerTransport.doStart(NettyHttpServerTransport.java:275)
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
at org.elasticsearch.http.HttpServer.doStart(HttpServer.java:107)
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
at org.elasticsearch.node.Node.start(Node.java:289)
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:222)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:288)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /172.38.1.37:9200
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at org.elasticsearch.http.netty.NettyHttpServerTransport$1.onPortNumber(NettyHttpServerTransport.java:379)
at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:58)
at org.elasticsearch.http.netty.NettyHttpServerTransport.bindAddress(NettyHttpServerTransport.java:374)
... 9 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:18:38,383][INFO ][gateway ] [lisyx] recovered [0] indices into cluster_state
[2018-07-25 19:18:38,389][INFO ][node ] [lisyx] stopping ...
[2018-07-25 19:18:38,393][INFO ][discovery.zen ] [lisyx] master_left [{lisyx}{34FCUSgSR3-sP5D5pmLhMA}{172.38.1.37}{172.38.1.37:9301}{master=true}], reason [shut_down]
[2018-07-25 19:18:38,394][WARN ][discovery.zen ] [lisyx] master left (reason = shut_down), current nodes: {{lisyx}{_0rgMrsNSA-RaPT0Zv0QFA}{172.38.1.37}{172.38.1.37:9300}{master=true},}
[2018-07-25 19:18:38,394][INFO ][cluster.service ] [lisyx] removed {{lisyx}{34FCUSgSR3-sP5D5pmLhMA}{172.38.1.37}{172.38.1.37:9301}{master=true},}, reason: zen-disco-master_failed ({lisyx}{34FCUSgSR3-sP5D5pmLhMA}{172.38.1.37}{172.38.1.37:9301}{master=true})
[2018-07-25 19:19:00,094][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 19:19:03,773][WARN ][discovery.zen.ping.unicast] [lisyx] [1] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:19:03,773][WARN ][discovery.zen.ping.unicast] [lisyx] [1] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:19:03,782][INFO ][node ] [lisyx] stopped
[2018-07-25 19:19:03,782][INFO ][node ] [lisyx] closing ...
[2018-07-25 19:19:03,787][INFO ][node ] [lisyx] closed
[2018-07-25 19:19:22,694][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 19:19:30,095][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 19:19:30,096][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:19:52,695][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 19:19:52,695][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:29:58,330][INFO ][node ] [lisyx] version[2.4.0], pid[2135], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 19:29:58,330][INFO ][node ] [lisyx] initializing ...
[2018-07-25 19:29:58,944][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 19:29:58,967][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 19:29:58,967][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 19:30:00,850][INFO ][node ] [lisyx] initialized
[2018-07-25 19:30:00,850][INFO ][node ] [lisyx] starting ...
[2018-07-25 19:30:00,949][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 19:30:00,954][INFO ][discovery ] [lisyx] LiferayElasticSearch/XAjP6a9qQquO1yCWMTdZPw
[2018-07-25 19:30:30,969][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 19:30:30,973][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 19:30:30,974][INFO ][node ] [lisyx] started
[2018-07-25 19:30:42,938][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 19:31:12,940][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 19:31:12,953][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 19:41:58,584][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 19:42:28,585][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 19:42:28,586][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:26:21,010][INFO ][node ] [lisyx] version[2.4.0], pid[4828], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:26:21,011][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:26:21,617][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:26:21,634][ERROR][bootstrap ] Exception
java.lang.IllegalStateException: Failed to obtain node lock, is the following location writable?: [/home/centos/elasticsearch-2.4.0/data/LiferayElasticSearch]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:199)
at org.elasticsearch.node.Node.<init>(Node.java:165)
at org.elasticsearch.node.Node.<init>(Node.java:140)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:194)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2018-07-25 22:34:46,001][INFO ][node ] [lisyx] version[2.4.0], pid[4968], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:34:46,001][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:34:46,630][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:34:46,647][ERROR][bootstrap ] Exception
java.lang.IllegalStateException: Failed to obtain node lock, is the following location writable?: [/home/centos/elasticsearch-2.4.0/data/LiferayElasticSearch]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:199)
at org.elasticsearch.node.Node.<init>(Node.java:165)
at org.elasticsearch.node.Node.<init>(Node.java:140)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:194)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2018-07-25 22:35:47,856][INFO ][node ] [lisyx] version[2.4.0], pid[5011], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:35:47,856][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:35:48,453][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:35:48,477][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:35:48,477][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:35:50,328][INFO ][node ] [lisyx] initialized
[2018-07-25 22:35:50,328][INFO ][node ] [lisyx] starting ...
[2018-07-25 22:35:50,407][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 22:35:50,412][INFO ][discovery ] [lisyx] LiferayElasticSearch/gzQ3ql29SYCr957fTXpKSw
[2018-07-25 22:36:20,423][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 22:36:20,427][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 22:36:20,427][INFO ][node ] [lisyx] started
[2018-07-25 22:36:41,000][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 22:37:11,002][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 22:37:11,015][WARN ][rest.suppressed ] path: /_cat/health, params: {v=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:42:46,808][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 22:43:16,809][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 22:43:16,810][WARN ][rest.suppressed ] path: /_cat/health, params: {v=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:47:27,562][INFO ][node ] [lisyx] version[2.4.0], pid[5234], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:47:27,563][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:47:28,161][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:47:28,187][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:47:28,187][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:47:30,106][INFO ][node ] [lisyx] initialized
[2018-07-25 22:47:30,106][INFO ][node ] [lisyx] starting ...
[2018-07-25 22:47:30,204][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 22:47:30,209][INFO ][discovery ] [lisyx] LiferayElasticSearch/WwMsb7JnTU2s8FgudQoJwA
[2018-07-25 22:48:00,231][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 22:48:00,235][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 22:48:00,235][INFO ][node ] [lisyx] started
[2018-07-25 22:48:13,300][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 22:48:29,093][INFO ][node ] [lisyx] stopping ...
[2018-07-25 22:48:29,134][WARN ][rest.suppressed ] path: /_cat/health, params: {v=}
NodeClosedException[node closed {lisyx}{WwMsb7JnTU2s8FgudQoJwA}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1}]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onClusterServiceClose(TransportMasterNodeAction.java:228)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onClose(ClusterStateObserver.java:222)
at org.elasticsearch.cluster.service.InternalClusterService.doStop(InternalClusterService.java:214)
at org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:88)
at org.elasticsearch.node.Node.stop(Node.java:326)
at org.elasticsearch.node.Node.close(Node.java:351)
at org.elasticsearch.bootstrap.Bootstrap$4.run(Bootstrap.java:173)
[2018-07-25 22:48:29,166][WARN ][netty.channel.DefaultChannelPipeline] An exception was thrown by an exception handler.
java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36)
at org.jboss.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34)
at org.jboss.netty.channel.DefaultChannelPipeline.execute(DefaultChannelPipeline.java:636)
at org.jboss.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496)
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46)
at org.jboss.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:781)
at org.jboss.netty.channel.Channels.write(Channels.java:725)
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71)
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59)
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784)
at org.elasticsearch.http.netty.pipelining.HttpPipeliningHandler.handleDownstream(HttpPipeliningHandler.java:87)
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
at org.elasticsearch.http.netty.NettyHttpChannel.sendResponse(NettyHttpChannel.java:147)
at org.elasticsearch.http.HttpServer$ResourceHandlingHttpChannel.sendResponse(HttpServer.java:352)
at org.elasticsearch.rest.action.support.RestActionListener.onFailure(RestActionListener.java:60)
at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:95)
at org.elasticsearch.action.support.ThreadedActionListener$2.doRun(ThreadedActionListener.java:104)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,274][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{#zen_unicast_1#}{172.38.1.37}{172.38.1.37:9300}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.doRun(UnicastZenPing.java:249)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 22:48:30,274][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{lisyx}{WwMsb7JnTU2s8FgudQoJwA}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.doRun(UnicastZenPing.java:249)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 22:48:30,407][WARN ][discovery.zen.ping.unicast] [lisyx] [14] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:916)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,407][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,408][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,408][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,408][WARN ][discovery.zen.ping.unicast] [lisyx] [4] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,407][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,407][WARN ][discovery.zen.ping.unicast] [lisyx] [1] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,407][WARN ][discovery.zen.ping.unicast] [lisyx] [1] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,408][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,408][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,407][WARN ][discovery.zen.ping.unicast] [lisyx] [14] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:916)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:48:30,412][INFO ][node ] [lisyx] stopped
[2018-07-25 22:48:30,412][INFO ][node ] [lisyx] closing ...
[2018-07-25 22:48:30,418][INFO ][node ] [lisyx] closed
[2018-07-25 22:49:25,546][INFO ][node ] [lisyx] version[2.4.0], pid[1230], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:49:25,547][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:49:26,225][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:49:26,255][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:49:26,255][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:49:28,193][INFO ][node ] [lisyx] initialized
[2018-07-25 22:49:28,193][INFO ][node ] [lisyx] starting ...
[2018-07-25 22:49:28,300][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 22:49:28,305][INFO ][discovery ] [lisyx] LiferayElasticSearch/r4cECX-4TwyabiLvjCvBmw
[2018-07-25 22:49:58,325][WARN ][discovery ] [lisyx] waited for 30s and no initial state was set by the discovery
[2018-07-25 22:49:58,329][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 22:49:58,329][INFO ][node ] [lisyx] started
[2018-07-25 22:50:05,709][DEBUG][action.admin.cluster.health] [lisyx] no known master node, scheduling a retry
[2018-07-25 22:50:35,711][DEBUG][action.admin.cluster.health] [lisyx] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 22:50:35,724][WARN ][rest.suppressed ] path: /_cat/health, params: {v=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:15,459][INFO ][node ] [lisyx] version[2.4.0], pid[1344], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:53:15,459][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:53:16,062][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:53:16,080][ERROR][bootstrap ] Exception
java.lang.IllegalStateException: Failed to obtain node lock, is the following location writable?: [/home/centos/elasticsearch-2.4.0/data/LiferayElasticSearch]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:199)
at org.elasticsearch.node.Node.<init>(Node.java:165)
at org.elasticsearch.node.Node.<init>(Node.java:140)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:194)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2018-07-25 22:53:24,565][INFO ][node ] [lisyx] stopping ...
[2018-07-25 22:53:25,470][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{#zen_unicast_1#}{172.38.1.37}{172.38.1.37:9300}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2$1.doRun(UnicastZenPing.java:253)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 22:53:25,470][WARN ][discovery.zen.ping.unicast] [lisyx] failed to send ping to [{lisyx}{r4cECX-4TwyabiLvjCvBmw}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1}]
SendRequestTransportException[[lisyx][172.38.1.37:9300][internal:discovery/zen/unicast]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestToNode(UnicastZenPing.java:440)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:426)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2$1.doRun(UnicastZenPing.java:253)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 7 more
[2018-07-25 22:53:28,819][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,819][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,819][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,819][WARN ][discovery.zen.ping.unicast] [lisyx] [2] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,821][WARN ][discovery.zen.ping.unicast] [lisyx] [34] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,820][WARN ][discovery.zen.ping.unicast] [lisyx] [14] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,820][WARN ][discovery.zen.ping.unicast] [lisyx] [4] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,820][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,820][WARN ][discovery.zen.ping.unicast] [lisyx] [3] failed send ping to {#zen_unicast_2#}{172.38.3.167}{172.38.3.167:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:53:28,826][INFO ][node ] [lisyx] stopped
[2018-07-25 22:53:28,826][INFO ][node ] [lisyx] closing ...
[2018-07-25 22:53:28,835][INFO ][node ] [lisyx] closed
[2018-07-25 22:54:23,658][INFO ][node ] [lisyx] version[2.4.0], pid[1232], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:54:23,659][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:54:24,352][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:54:24,378][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:54:24,378][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:54:26,271][INFO ][node ] [lisyx] initialized
[2018-07-25 22:54:26,271][INFO ][node ] [lisyx] starting ...
[2018-07-25 22:54:26,393][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 22:54:26,398][INFO ][discovery ] [lisyx] LiferayElasticSearch/-B7L-lcTRAmWwujWeuq8Aw
[2018-07-25 22:54:30,963][INFO ][cluster.service ] [lisyx] new_master {lisyx}{-B7L-lcTRAmWwujWeuq8Aw}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 22:54:30,974][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 22:54:30,974][INFO ][node ] [lisyx] started
[2018-07-25 22:54:30,994][INFO ][gateway ] [lisyx] recovered [0] indices into cluster_state
[2018-07-25 22:55:05,901][INFO ][node ] [lisyx] version[2.4.0], pid[1296], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:55:05,902][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:55:06,522][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:55:06,545][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:55:06,545][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:55:08,162][INFO ][node ] [lisyx] initialized
[2018-07-25 22:55:08,162][INFO ][node ] [lisyx] starting ...
[2018-07-25 22:55:08,222][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 22:55:08,227][INFO ][discovery ] [lisyx] LiferayElasticSearch/itOAN-U_R1Sr_lLT0dAI9A
[2018-07-25 22:55:12,748][INFO ][cluster.service ] [lisyx] new_master {lisyx}{itOAN-U_R1Sr_lLT0dAI9A}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 22:55:12,765][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 22:55:12,765][INFO ][node ] [lisyx] started
[2018-07-25 22:55:12,778][INFO ][gateway ] [lisyx] recovered [0] indices into cluster_state
[2018-07-25 22:58:23,388][INFO ][node ] [lisyx] version[2.4.0], pid[1397], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:58:23,389][INFO ][node ] [lisyx] initializing ...
[2018-07-25 22:58:24,009][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:58:24,029][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:58:24,029][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:58:25,837][INFO ][node ] [lisyx] initialized
[2018-07-25 22:58:25,837][INFO ][node ] [lisyx] starting ...
[2018-07-25 22:58:25,917][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 22:58:25,922][INFO ][discovery ] [lisyx] LiferayElasticSearch/fFAEyTl6T2OgsxjCU-X77g
[2018-07-25 22:58:30,451][INFO ][cluster.service ] [lisyx] new_master {lisyx}{fFAEyTl6T2OgsxjCU-X77g}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 22:58:30,468][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 22:58:30,468][INFO ][node ] [lisyx] started
[2018-07-25 22:58:30,484][INFO ][gateway ] [lisyx] recovered [0] indices into cluster_state
[2018-07-25 23:04:15,653][INFO ][node ] [lisyx] version[2.4.0], pid[1476], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 23:04:15,653][INFO ][node ] [lisyx] initializing ...
[2018-07-25 23:04:16,282][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 23:04:16,307][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 23:04:16,307][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 23:04:18,363][INFO ][node ] [lisyx] initialized
[2018-07-25 23:04:18,363][INFO ][node ] [lisyx] starting ...
[2018-07-25 23:04:18,449][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 23:04:18,454][INFO ][discovery ] [lisyx] LiferayElasticSearch/AYaDxU44Ti2kQf4d_iUQLA
[2018-07-25 23:04:22,984][INFO ][cluster.service ] [lisyx] new_master {lisyx}{AYaDxU44Ti2kQf4d_iUQLA}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 23:04:22,999][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 23:04:22,999][INFO ][node ] [lisyx] started
[2018-07-25 23:04:23,017][INFO ][gateway ] [lisyx] recovered [0] indices into cluster_state
[2018-07-25 23:24:50,149][INFO ][node ] [lisyx] version[2.4.0], pid[1580], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 23:24:50,150][INFO ][node ] [lisyx] initializing ...
[2018-07-25 23:24:50,763][INFO ][plugins ] [lisyx] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 23:24:50,783][INFO ][env ] [lisyx] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 23:24:50,783][INFO ][env ] [lisyx] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 23:24:52,671][INFO ][node ] [lisyx] initialized
[2018-07-25 23:24:52,671][INFO ][node ] [lisyx] starting ...
[2018-07-25 23:24:52,787][INFO ][transport ] [lisyx] publish_address {172.38.1.37:9300}, bound_addresses {172.38.1.37:9300}
[2018-07-25 23:24:52,792][INFO ][discovery ] [lisyx] LiferayElasticSearch/r2HXWmLETdmBv4GLabpcQw
[2018-07-25 23:24:57,359][INFO ][cluster.service ] [lisyx] new_master {lisyx}{r2HXWmLETdmBv4GLabpcQw}{172.38.1.37}{172.38.1.37:9300}{max_local_storage_nodes=1, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 23:24:57,370][INFO ][http ] [lisyx] publish_address {172.38.1.37:9200}, bound_addresses {172.38.1.37:9200}
[2018-07-25 23:24:57,370][INFO ][node ] [lisyx] started
[2018-07-25 23:24:57,390][INFO ][gateway ] [lisyx] recovered [0] indices into cluster_state
[2018-07-25 17:55:17,983][INFO ][node ] [lisyx-2] version[2.4.0], pid[1423], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 17:55:17,983][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 17:55:18,653][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 17:55:18,677][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 17:55:18,677][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 17:55:20,368][INFO ][node ] [lisyx-2] initialized
[2018-07-25 17:55:20,369][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 17:55:20,546][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 17:55:20,551][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/dYjP-02aQUG_KjvIW4WbSQ
[2018-07-25 17:55:25,135][INFO ][cluster.service ] [lisyx-2] new_master {lisyx-2}{dYjP-02aQUG_KjvIW4WbSQ}{172.38.3.167}{172.38.3.167:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 17:55:25,149][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 17:55:25,149][INFO ][node ] [lisyx-2] started
[2018-07-25 17:55:25,185][INFO ][gateway ] [lisyx-2] recovered [0] indices into cluster_state
[2018-07-25 17:55:45,138][INFO ][node ] [lisyx-2] version[2.4.0], pid[1481], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 17:55:45,139][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 17:55:45,751][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 17:55:45,772][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 17:55:45,772][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 17:55:47,680][INFO ][node ] [lisyx-2] initialized
[2018-07-25 17:55:47,680][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 17:55:47,818][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9301}, bound_addresses {172.38.3.167:9301}
[2018-07-25 17:55:47,823][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/W_di9chxRHu6NqPN1e8ebw
[2018-07-25 17:55:52,402][INFO ][cluster.service ] [lisyx-2] added {{lisyx-2}{W_di9chxRHu6NqPN1e8ebw}{172.38.3.167}{172.38.3.167:9301},}, reason: zen-disco-join(join from node[{lisyx-2}{W_di9chxRHu6NqPN1e8ebw}{172.38.3.167}{172.38.3.167:9301}])
[2018-07-25 17:55:52,420][INFO ][cluster.service ] [lisyx-2] detected_master {lisyx-2}{dYjP-02aQUG_KjvIW4WbSQ}{172.38.3.167}{172.38.3.167:9300}, added {{lisyx-2}{dYjP-02aQUG_KjvIW4WbSQ}{172.38.3.167}{172.38.3.167:9300},}, reason: zen-disco-receive(from master [{lisyx-2}{dYjP-02aQUG_KjvIW4WbSQ}{172.38.3.167}{172.38.3.167:9300}])
[2018-07-25 17:55:52,451][ERROR][bootstrap ] [lisyx-2] Exception
BindHttpException[Failed to bind to [9200]]; nested: ChannelException[Failed to bind to: /172.38.3.167:9200]; nested: BindException[Address already in use];
at org.elasticsearch.http.netty.NettyHttpServerTransport.bindAddress(NettyHttpServerTransport.java:391)
at org.elasticsearch.http.netty.NettyHttpServerTransport.createBoundHttpAddress(NettyHttpServerTransport.java:289)
at org.elasticsearch.http.netty.NettyHttpServerTransport.doStart(NettyHttpServerTransport.java:275)
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
at org.elasticsearch.http.HttpServer.doStart(HttpServer.java:107)
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
at org.elasticsearch.node.Node.start(Node.java:289)
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:222)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:288)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /172.38.3.167:9200
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at org.elasticsearch.http.netty.NettyHttpServerTransport$1.onPortNumber(NettyHttpServerTransport.java:379)
at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:58)
at org.elasticsearch.http.netty.NettyHttpServerTransport.bindAddress(NettyHttpServerTransport.java:374)
... 9 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:55:52,454][INFO ][node ] [lisyx-2] stopping ...
[2018-07-25 17:55:52,460][INFO ][cluster.service ] [lisyx-2] removed {{lisyx-2}{W_di9chxRHu6NqPN1e8ebw}{172.38.3.167}{172.38.3.167:9301},}, reason: zen-disco-node-left({lisyx-2}{W_di9chxRHu6NqPN1e8ebw}{172.38.3.167}{172.38.3.167:9301}), reason(left)
[2018-07-25 17:56:18,019][WARN ][discovery.zen.ping.unicast] [lisyx-2] [1] failed send ping to {#zen_unicast_1#}{172.38.1.37}{172.38.1.37:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:18,019][WARN ][discovery.zen.ping.unicast] [lisyx-2] [1] failed send ping to {#zen_unicast_1#}{172.38.1.37}{172.38.1.37:9300}
java.lang.IllegalStateException: can't add nodes to a stopped transport
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:925)
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906)
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:395)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 17:56:18,025][INFO ][node ] [lisyx-2] stopped
[2018-07-25 17:56:18,025][INFO ][node ] [lisyx-2] closing ...
[2018-07-25 17:56:18,029][INFO ][node ] [lisyx-2] closed
[2018-07-25 17:56:23,086][INFO ][node ] [lisyx-2] stopping ...
[2018-07-25 17:56:25,143][DEBUG][action.admin.cluster.node.stats] [lisyx-2] failed to execute on node [dYjP-02aQUG_KjvIW4WbSQ]
SendRequestTransportException[[lisyx-2][172.38.3.167:9300][cluster:monitor/nodes/stats[n]]]; nested: TransportException[TransportService is closed stopped can't send request];
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:340)
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.start(TransportNodesAction.java:165)
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.access$100(TransportNodesAction.java:107)
at org.elasticsearch.action.support.nodes.TransportNodesAction.doExecute(TransportNodesAction.java:77)
at org.elasticsearch.action.support.nodes.TransportNodesAction.doExecute(TransportNodesAction.java:46)
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137)
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85)
at org.elasticsearch.cluster.InternalClusterInfoService.updateNodeStats(InternalClusterInfoService.java:289)
at org.elasticsearch.cluster.InternalClusterInfoService.refresh(InternalClusterInfoService.java:325)
at org.elasticsearch.cluster.InternalClusterInfoService.maybeRefresh(InternalClusterInfoService.java:310)
at org.elasticsearch.cluster.InternalClusterInfoService.access$600(InternalClusterInfoService.java:69)
at org.elasticsearch.cluster.InternalClusterInfoService$SubmitReschedulingClusterInfoUpdatedJob$1.run(InternalClusterInfoService.java:255)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: TransportException[TransportService is closed stopped can't send request]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:320)
... 14 more
[2018-07-25 17:56:50,860][INFO ][node ] [lisyx-2] stopped
[2018-07-25 17:56:50,861][INFO ][node ] [lisyx-2] closing ...
[2018-07-25 17:56:50,864][INFO ][node ] [lisyx-2] closed
[2018-07-25 17:58:05,544][INFO ][node ] [lisyx-2] version[2.4.0], pid[1248], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 17:58:05,544][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 17:58:06,214][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 17:58:06,248][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 17:58:06,248][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 17:58:08,185][INFO ][node ] [lisyx-2] initialized
[2018-07-25 17:58:08,185][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 17:58:08,285][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 17:58:08,290][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/XKvEdxBiTBKCGBRiPGeM_w
[2018-07-25 17:58:12,854][INFO ][cluster.service ] [lisyx-2] new_master {lisyx-2}{XKvEdxBiTBKCGBRiPGeM_w}{172.38.3.167}{172.38.3.167:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 17:58:12,863][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 17:58:12,863][INFO ][node ] [lisyx-2] started
[2018-07-25 17:58:12,882][INFO ][gateway ] [lisyx-2] recovered [0] indices into cluster_state
[2018-07-25 18:38:54,050][INFO ][node ] [lisyx-2] stopping ...
[2018-07-25 18:38:54,071][INFO ][node ] [lisyx-2] stopped
[2018-07-25 18:38:54,071][INFO ][node ] [lisyx-2] closing ...
[2018-07-25 18:38:54,075][INFO ][node ] [lisyx-2] closed
[2018-07-25 18:40:31,895][INFO ][node ] [lisyx-2] version[2.4.0], pid[1236], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 18:40:31,896][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 18:40:32,578][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 18:40:32,604][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 18:40:32,605][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 18:40:34,618][INFO ][node ] [lisyx-2] initialized
[2018-07-25 18:40:34,618][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 18:40:34,794][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 18:40:34,799][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/o7FYwD-DQAST8HGR58bnHA
[2018-07-25 18:41:04,810][WARN ][discovery ] [lisyx-2] waited for 30s and no initial state was set by the discovery
[2018-07-25 18:41:04,814][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 18:41:04,814][INFO ][node ] [lisyx-2] started
[2018-07-25 18:55:05,996][INFO ][node ] [lisyx-2] version[2.4.0], pid[1360], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 18:55:05,996][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 18:55:06,583][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 18:55:06,605][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 18:55:06,605][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 18:55:08,564][INFO ][node ] [lisyx-2] initialized
[2018-07-25 18:55:08,564][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 18:55:08,647][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 18:55:08,652][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/exE8BcPsSTCk4hePkD8LBA
[2018-07-25 18:55:38,663][WARN ][discovery ] [lisyx-2] waited for 30s and no initial state was set by the discovery
[2018-07-25 18:55:38,667][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 18:55:38,667][INFO ][node ] [lisyx-2] started
[2018-07-25 18:56:27,841][DEBUG][action.admin.cluster.health] [lisyx-2] no known master node, scheduling a retry
[2018-07-25 18:56:57,844][DEBUG][action.admin.cluster.health] [lisyx-2] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 18:56:57,854][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 18:58:09,565][INFO ][node ] [lisyx-2] version[2.4.0], pid[1476], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 18:58:09,565][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 18:58:10,189][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 18:58:10,210][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 18:58:10,211][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 18:58:12,204][INFO ][node ] [lisyx-2] initialized
[2018-07-25 18:58:12,205][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 18:58:12,271][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 18:58:12,276][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/ZEKA5vf9T-y1qhPa9mDntg
[2018-07-25 18:58:16,800][INFO ][cluster.service ] [lisyx-2] new_master {lisyx-2}{ZEKA5vf9T-y1qhPa9mDntg}{172.38.3.167}{172.38.3.167:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 18:58:16,808][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 18:58:16,808][INFO ][node ] [lisyx-2] started
[2018-07-25 18:58:16,831][INFO ][gateway ] [lisyx-2] recovered [0] indices into cluster_state
[2018-07-25 19:23:35,688][INFO ][node ] [lisyx-2] version[2.4.0], pid[1589], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 19:23:35,689][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 19:23:36,290][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 19:23:36,314][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 19:23:36,315][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 19:24:47,197][INFO ][node ] [lisyx-2] version[2.4.0], pid[1246], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 19:24:47,199][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 19:24:47,970][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 19:24:47,999][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 19:24:47,999][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 19:24:49,986][INFO ][node ] [lisyx-2] initialized
[2018-07-25 19:24:49,987][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 19:24:50,085][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 19:24:50,090][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/5eNqU_hJSDyXzU70Q60a5A
[2018-07-25 19:24:54,620][INFO ][cluster.service ] [lisyx-2] new_master {lisyx-2}{5eNqU_hJSDyXzU70Q60a5A}{172.38.3.167}{172.38.3.167:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 19:24:54,649][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 19:24:54,649][INFO ][node ] [lisyx-2] started
[2018-07-25 19:24:54,651][INFO ][gateway ] [lisyx-2] recovered [0] indices into cluster_state
[2018-07-25 19:27:59,533][INFO ][node ] [lisyx-2] version[2.4.0], pid[1312], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 19:27:59,534][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 19:28:00,130][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 19:28:00,152][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 19:28:00,152][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 19:28:02,045][INFO ][node ] [lisyx-2] initialized
[2018-07-25 19:28:02,045][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 19:28:02,154][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 19:28:02,159][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/YN82Z25jR-yDYqRr9roQPQ
[2018-07-25 19:28:32,178][WARN ][discovery ] [lisyx-2] waited for 30s and no initial state was set by the discovery
[2018-07-25 19:28:32,182][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 19:28:32,182][INFO ][node ] [lisyx-2] started
[2018-07-25 19:28:47,173][DEBUG][action.admin.cluster.health] [lisyx-2] no known master node, scheduling a retry
[2018-07-25 19:29:17,175][DEBUG][action.admin.cluster.health] [lisyx-2] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 19:29:17,187][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 22:27:27,671][INFO ][node ] [lisyx-2] version[2.4.0], pid[4101], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:27:27,672][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 22:27:28,319][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:27:28,341][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:27:28,341][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:27:30,201][INFO ][node ] [lisyx-2] initialized
[2018-07-25 22:27:30,201][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 22:27:30,293][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 22:27:30,298][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/h8MJrWT6RoeMGrzV_1JNRQ
[2018-07-25 22:27:34,842][INFO ][cluster.service ] [lisyx-2] new_master {lisyx-2}{h8MJrWT6RoeMGrzV_1JNRQ}{172.38.3.167}{172.38.3.167:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 22:27:34,853][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 22:27:34,853][INFO ][node ] [lisyx-2] started
[2018-07-25 22:27:34,870][INFO ][gateway ] [lisyx-2] recovered [0] indices into cluster_state
[2018-07-25 22:31:05,644][INFO ][node ] [lisyx-2] stopping ...
[2018-07-25 22:31:05,653][INFO ][node ] [lisyx-2] stopped
[2018-07-25 22:31:05,653][INFO ][node ] [lisyx-2] closing ...
[2018-07-25 22:31:05,656][INFO ][node ] [lisyx-2] closed
[2018-07-25 22:31:17,327][INFO ][node ] [lisyx-2] version[2.4.0], pid[4163], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 22:31:17,327][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 22:31:17,976][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 22:31:17,999][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 22:31:18,000][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 22:31:19,840][INFO ][node ] [lisyx-2] initialized
[2018-07-25 22:31:19,841][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 22:31:20,024][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 22:31:20,029][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/OqRGor0OQdOwdmyEfYi1TA
[2018-07-25 22:31:24,598][INFO ][cluster.service ] [lisyx-2] new_master {lisyx-2}{OqRGor0OQdOwdmyEfYi1TA}{172.38.3.167}{172.38.3.167:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 22:31:24,629][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 22:31:24,629][INFO ][node ] [lisyx-2] started
[2018-07-25 22:31:24,632][INFO ][gateway ] [lisyx-2] recovered [0] indices into cluster_state
[2018-07-25 23:04:22,172][ERROR][bootstrap ] Exception
java.lang.RuntimeException: don't run elasticsearch as root.
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:94)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:160)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2018-07-25 23:04:40,125][INFO ][node ] [lisyx-2] version[2.4.0], pid[4316], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 23:04:40,125][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 23:04:40,729][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 23:04:40,754][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 23:04:40,754][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 23:04:42,660][INFO ][node ] [lisyx-2] initialized
[2018-07-25 23:04:42,672][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 23:04:42,756][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 23:04:42,760][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/hByhfG_lT9W6DspnenNZvA
[2018-07-25 23:05:12,770][WARN ][discovery ] [lisyx-2] waited for 30s and no initial state was set by the discovery
[2018-07-25 23:05:12,775][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 23:05:12,775][INFO ][node ] [lisyx-2] started
[2018-07-25 23:06:36,685][DEBUG][action.admin.cluster.health] [lisyx-2] no known master node, scheduling a retry
[2018-07-25 23:07:06,687][DEBUG][action.admin.cluster.health] [lisyx-2] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 23:07:06,700][WARN ][rest.suppressed ] path: /_cluster/health, params: {pretty=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 23:07:53,587][DEBUG][action.admin.cluster.health] [lisyx-2] no known master node, scheduling a retry
[2018-07-25 23:08:23,588][DEBUG][action.admin.cluster.health] [lisyx-2] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-07-25 23:08:23,589][WARN ][rest.suppressed ] path: /_cat/health, params: {v=}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:234)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2018-07-25 23:32:04,570][INFO ][node ] [lisyx-2] version[2.4.0], pid[1278], build[ce9f0c7/2016-08-29T09:14:17Z]
[2018-07-25 23:32:04,571][INFO ][node ] [lisyx-2] initializing ...
[2018-07-25 23:32:05,261][INFO ][plugins ] [lisyx-2] modules [reindex, lang-expression, lang-groovy], plugins [analysis-kuromoji, analysis-icu, analysis-smartcn, analysis-stempel], sites []
[2018-07-25 23:32:05,286][INFO ][env ] [lisyx-2] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [8.7gb], net total_space [9.9gb], spins? [unknown], types [rootfs]
[2018-07-25 23:32:05,286][INFO ][env ] [lisyx-2] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-07-25 23:32:07,120][INFO ][node ] [lisyx-2] initialized
[2018-07-25 23:32:07,120][INFO ][node ] [lisyx-2] starting ...
[2018-07-25 23:32:07,246][INFO ][transport ] [lisyx-2] publish_address {172.38.3.167:9300}, bound_addresses {172.38.3.167:9300}
[2018-07-25 23:32:07,251][INFO ][discovery ] [lisyx-2] LiferayElasticSearch/PGIGGarCSV68Gr5haQmiHA
[2018-07-25 23:32:11,793][INFO ][cluster.service ] [lisyx-2] new_master {lisyx-2}{PGIGGarCSV68Gr5haQmiHA}{172.38.3.167}{172.38.3.167:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2018-07-25 23:32:11,807][INFO ][http ] [lisyx-2] publish_address {172.38.3.167:9200}, bound_addresses {172.38.3.167:9200}
[2018-07-25 23:32:11,807][INFO ][node ] [lisyx-2] started
[2018-07-25 23:32:11,825][INFO ][gateway ] [lisyx-2] recovered [0] indices into cluster_state
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment