Skip to content

Instantly share code, notes, and snippets.

@prashant407

prashant407/gist:1399733b7a3f5d26f25da0faeb0377eb Secret

Last active January 18, 2021 07:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save prashant407/1399733b7a3f5d26f25da0faeb0377eb to your computer and use it in GitHub Desktop.
Save prashant407/1399733b7a3f5d26f25da0faeb0377eb to your computer and use it in GitHub Desktop.
Download ZIP
hot_threads output for elasticsearch
Raw
gistfile1.txt
::: {abc-elasticsearch-client-768fd6bcc6-68zgd}{cfIWxQEjS_qdtX9e-Z4mxw}{GoUBp8MNS8CLACv7oYItPg}{aa.aa.aa.aa}{aa.aa.aa.aa:9300}{i}
Hot threads at 2021-01-15T11:01:09.333Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (24.3micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (24.1micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (13.6micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
::: {abc-elasticsearch-master-1}{ddz44X4lQ66bhf26qIcr2A}{ki0dOpCgS_2AmQdN369t7g}{192.168.220.175}{192.168.220.175:9300}{m}
Hot threads at 2021-01-15T11:01:09.332Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (28.2micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (27.2micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (25.7micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-1][generic][T#6]'
10/10 snapshots sharing following 35 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:82)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:461)
app//org.apache.lucene.util.IOUtils.fsync(IOUtils.java:471)
app//org.apache.lucene.store.FSDirectory.fsync(FSDirectory.java:331)
app//org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:286)
app//org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(LockValidatingDirectoryWrapper.java:68)
app//org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:458)
app//org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:764)
app//org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4790)
app//org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3277)
app//org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3131)
app//org.elasticsearch.gateway.PersistedClusterStateService$MetadataIndexWriter.prepareCommit(PersistedClusterStateService.java:526)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.commit(PersistedClusterStateService.java:737)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.writeIncrementalStateAndCommit(PersistedClusterStateService.java:616)
app//org.elasticsearch.gateway.GatewayMetaState$LucenePersistedState.setLastAcceptedState(GatewayMetaState.java:543)
app//org.elasticsearch.cluster.coordination.CoordinationState.handlePublishRequest(CoordinationState.java:357)
app//org.elasticsearch.cluster.coordination.Coordinator.handlePublishRequest(Coordinator.java:330)
app//org.elasticsearch.cluster.coordination.Coordinator$$Lambda$2034/0x00000001008cc840.apply(Unknown Source)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.acceptState(PublicationTransportHandler.java:452)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.handleIncomingPublishRequest(PublicationTransportHandler.java:432)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.lambda$new$0(PublicationTransportHandler.java:100)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$$Lambda$2037/0x00000001008cd440.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:267)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:136)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1987/0x00000001008b8c40.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
::: {abc-elasticsearch-master-2}{LMAijYr4RweZrfudhWGh3A}{U_6taRr-SSeyC0EZiq6xOg}{192.168.152.108}{192.168.152.108:9300}{m}
Hot threads at 2021-01-15T11:01:09.331Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (46.2micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (30.6micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (26.5micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-2][generic][T#3]'
10/10 snapshots sharing following 35 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:82)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:461)
app//org.apache.lucene.util.IOUtils.fsync(IOUtils.java:471)
app//org.apache.lucene.store.FSDirectory.fsync(FSDirectory.java:331)
app//org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:286)
app//org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(LockValidatingDirectoryWrapper.java:68)
app//org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:458)
app//org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:764)
app//org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4790)
app//org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3277)
app//org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3131)
app//org.elasticsearch.gateway.PersistedClusterStateService$MetadataIndexWriter.prepareCommit(PersistedClusterStateService.java:526)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.commit(PersistedClusterStateService.java:737)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.writeIncrementalStateAndCommit(PersistedClusterStateService.java:616)
app//org.elasticsearch.gateway.GatewayMetaState$LucenePersistedState.setLastAcceptedState(GatewayMetaState.java:543)
app//org.elasticsearch.cluster.coordination.CoordinationState.handlePublishRequest(CoordinationState.java:357)
app//org.elasticsearch.cluster.coordination.Coordinator.handlePublishRequest(Coordinator.java:330)
app//org.elasticsearch.cluster.coordination.Coordinator$$Lambda$2034/0x00000001008cc840.apply(Unknown Source)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.acceptState(PublicationTransportHandler.java:452)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.handleIncomingPublishRequest(PublicationTransportHandler.java:432)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.lambda$new$0(PublicationTransportHandler.java:100)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$$Lambda$2037/0x00000001008cd440.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:267)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:136)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1987/0x00000001008b8c40.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
::: {abc-elasticsearch-data-2}{lT9ANczjRUmEyPjOh0BVbA}{pyz5F_jLTWS6uovOf7nRzg}{192.168.152.123}{192.168.152.123:9300}{d}
Hot threads at 2021-01-15T11:01:09.346Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (40.8micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (25.1micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (16.6micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][generic][T#6]'
10/10 snapshots sharing following 33 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62)
java.base@11.0.9/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)
java.base@11.0.9/sun.nio.ch.IOUtil.write(IOUtil.java:79)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
java.base@11.0.9/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
java.base@11.0.9/java.nio.channels.Channels.writeFully(Channels.java:97)
java.base@11.0.9/java.nio.channels.Channels$1.write(Channels.java:172)
app//org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:416)
java.base@11.0.9/java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:74)
java.base@11.0.9/java.io.BufferedOutputStream.write(BufferedOutputStream.java:123)
app//org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:53)
app//org.elasticsearch.common.lucene.store.FilterIndexOutput.writeBytes(FilterIndexOutput.java:59)
app//org.elasticsearch.index.store.Store$LuceneVerifyingIndexOutput.writeBytes(Store.java:1208)
app//org.elasticsearch.indices.recovery.MultiFileWriter.innerWriteFileChunk(MultiFileWriter.java:126)
app//org.elasticsearch.indices.recovery.MultiFileWriter.access$000(MultiFileWriter.java:46)
app//org.elasticsearch.indices.recovery.MultiFileWriter$FileChunkWriter.writeChunk(MultiFileWriter.java:213)
app//org.elasticsearch.indices.recovery.MultiFileWriter.writeFileChunk(MultiFileWriter.java:74)
app//org.elasticsearch.indices.recovery.RecoveryTarget.writeFileChunk(RecoveryTarget.java:471)
app//org.elasticsearch.indices.recovery.PeerRecoveryTargetService$FileChunkTransportRequestHandler.messageReceived(PeerRecoveryTargetService.java:533)
app//org.elasticsearch.indices.recovery.PeerRecoveryTargetService$FileChunkTransportRequestHandler.messageReceived(PeerRecoveryTargetService.java:507)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:267)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:136)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1988/0x00000001008b9040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][management][T#5]'
10/10 snapshots sharing following 36 elements
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:481)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3021/0x0000000100c3c440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2998/0x0000000100c2b440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2987/0x0000000100c2e840.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2120/0x000000010094bc40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1988/0x00000001008b9040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][generic][T#2]'
10/10 snapshots sharing following 18 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:1009)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1324)
java.base@11.0.9/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)
app//org.elasticsearch.common.util.concurrent.ReleasableLock.acquire(ReleasableLock.java:55)
app//org.elasticsearch.index.translog.Translog.trimUnreferencedReaders(Translog.java:1657)
app//org.elasticsearch.index.engine.InternalEngine.trimUnreferencedTranslogFiles(InternalEngine.java:1885)
app//org.elasticsearch.index.shard.IndexShard.trimTranslog(IndexShard.java:1105)
app//org.elasticsearch.index.IndexService.maybeTrimTranslog(IndexService.java:857)
app//org.elasticsearch.index.IndexService.access$700(IndexService.java:110)
app//org.elasticsearch.index.IndexService$AsyncTrimTranslogTask.runInternal(IndexService.java:999)
app//org.elasticsearch.common.util.concurrent.AbstractAsyncTask.run(AbstractAsyncTask.java:144)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:636)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][AsyncLucenePersistedState#updateTask][T#1]'
10/10 snapshots sharing following 20 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:82)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:461)
app//org.apache.lucene.util.IOUtils.fsync(IOUtils.java:471)
app//org.apache.lucene.store.FSDirectory.syncMetaData(FSDirectory.java:309)
app//org.apache.lucene.store.LockValidatingDirectoryWrapper.syncMetaData(LockValidatingDirectoryWrapper.java:62)
app//org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:763)
app//org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4790)
app//org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3277)
app//org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3131)
app//org.elasticsearch.gateway.PersistedClusterStateService$MetadataIndexWriter.prepareCommit(PersistedClusterStateService.java:526)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.commit(PersistedClusterStateService.java:737)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.writeIncrementalStateAndCommit(PersistedClusterStateService.java:616)
app//org.elasticsearch.gateway.GatewayMetaState$LucenePersistedState.setLastAcceptedState(GatewayMetaState.java:543)
app//org.elasticsearch.gateway.GatewayMetaState$AsyncLucenePersistedState$1.doRun(GatewayMetaState.java:428)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][management][T#4]'
10/10 snapshots sharing following 36 elements
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:481)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3021/0x0000000100c3c440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2998/0x0000000100c2b440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2987/0x0000000100c2e840.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2120/0x000000010094bc40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1988/0x00000001008b9040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][management][T#1]'
10/10 snapshots sharing following 42 elements
java.base@11.0.9/sun.nio.fs.UnixNativeDispatcher.readdir(Native Method)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.readNextEntry(UnixDirectoryStream.java:165)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.hasNext(UnixDirectoryStream.java:198)
app//org.elasticsearch.gateway.MetadataStateFormat.findMaxGenerationId(MetadataStateFormat.java:354)
app//org.elasticsearch.gateway.MetadataStateFormat.write(MetadataStateFormat.java:222)
app//org.elasticsearch.gateway.MetadataStateFormat.writeAndCleanup(MetadataStateFormat.java:185)
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:491)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3021/0x0000000100c3c440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2998/0x0000000100c2b440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2987/0x0000000100c2e840.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2120/0x000000010094bc40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1988/0x00000001008b9040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][management][T#2]'
10/10 snapshots sharing following 36 elements
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:481)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3021/0x0000000100c3c440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2998/0x0000000100c2b440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2987/0x0000000100c2e840.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2120/0x000000010094bc40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1988/0x00000001008b9040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][refresh][T#1]'
10/10 snapshots sharing following 25 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:1009)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1324)
java.base@11.0.9/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)
app//org.elasticsearch.common.util.concurrent.ReleasableLock.acquire(ReleasableLock.java:55)
app//org.elasticsearch.index.translog.Translog.getLastWriteLocation(Translog.java:576)
app//org.elasticsearch.index.engine.InternalEngine.getTranslogLastWriteLocation(InternalEngine.java:600)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2908/0x0000000100be3840.get(Unknown Source)
app//org.elasticsearch.index.shard.RefreshListeners.beforeRefresh(RefreshListeners.java:216)
app//org.apache.lucene.search.ReferenceManager.notifyRefreshListenersBefore(ReferenceManager.java:279)
app//org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:175)
app//org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:225)
app//org.elasticsearch.index.engine.InternalEngine.refresh(InternalEngine.java:1625)
app//org.elasticsearch.index.engine.InternalEngine.maybeRefresh(InternalEngine.java:1604)
app//org.elasticsearch.index.shard.IndexShard.scheduledRefresh(IndexShard.java:3246)
app//org.elasticsearch.index.IndexService.maybeRefreshEngine(IndexService.java:839)
app//org.elasticsearch.index.IndexService.access$200(IndexService.java:110)
app//org.elasticsearch.index.IndexService$AsyncRefreshTask.runInternal(IndexService.java:971)
app//org.elasticsearch.common.util.concurrent.AbstractAsyncTask.run(AbstractAsyncTask.java:144)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:636)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][write][T#1]'
10/10 snapshots sharing following 53 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:82)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:461)
app//org.elasticsearch.index.translog.TranslogWriter.syncUpTo(TranslogWriter.java:376)
app//org.elasticsearch.index.translog.Translog.ensureSynced(Translog.java:785)
app//org.elasticsearch.index.translog.Translog.ensureSynced(Translog.java:806)
app//org.elasticsearch.index.engine.InternalEngine.ensureTranslogSynced(InternalEngine.java:548)
app//org.elasticsearch.index.shard.IndexShard$5.write(IndexShard.java:3069)
app//org.elasticsearch.common.util.concurrent.AsyncIOProcessor.processList(AsyncIOProcessor.java:108)
app//org.elasticsearch.common.util.concurrent.AsyncIOProcessor.drainAndProcessAndRelease(AsyncIOProcessor.java:96)
app//org.elasticsearch.common.util.concurrent.AsyncIOProcessor.put(AsyncIOProcessor.java:84)
app//org.elasticsearch.index.shard.IndexShard.sync(IndexShard.java:3092)
app//org.elasticsearch.action.support.replication.TransportWriteAction$AsyncAfterWriteAction.run(TransportWriteAction.java:320)
app//org.elasticsearch.action.support.replication.TransportWriteAction$WritePrimaryResult.runPostReplicationActions(TransportWriteAction.java:163)
app//org.elasticsearch.action.support.replication.ReplicationOperation.handlePrimaryResult(ReplicationOperation.java:136)
app//org.elasticsearch.action.support.replication.ReplicationOperation$$Lambda$3003/0x0000000100c29840.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:336)
app//org.elasticsearch.action.bulk.TransportShardBulkAction$2.finishRequest(TransportShardBulkAction.java:197)
app//org.elasticsearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:166)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.bulk.TransportShardBulkAction.performOnPrimary(TransportShardBulkAction.java:202)
app//org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:114)
app//org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:81)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2998/0x0000000100c2b440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2987/0x0000000100c2e840.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2120/0x000000010094bc40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1988/0x00000001008b9040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][flush][T#1]'
10/10 snapshots sharing following 14 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:82)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:461)
app//org.elasticsearch.core.internal.io.IOUtils.fsync(IOUtils.java:280)
app//org.elasticsearch.index.translog.Translog.copyCheckpointTo(Translog.java:281)
app//org.elasticsearch.index.translog.Translog.rollGeneration(Translog.java:1633)
app//org.elasticsearch.index.engine.InternalEngine.flush(InternalEngine.java:1798)
app//org.elasticsearch.index.shard.IndexShard.flush(IndexShard.java:1093)
app//org.elasticsearch.indices.flush.SyncedFlushService$2.doRun(SyncedFlushService.java:148)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-2][management][T#3]'
10/10 snapshots sharing following 36 elements
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:481)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3021/0x0000000100c3c440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2998/0x0000000100c2b440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2987/0x0000000100c2e840.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2120/0x000000010094bc40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1988/0x00000001008b9040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
::: {abc-elasticsearch-master-0}{BY5zVLuHQBCl6pnJkFXs4g}{N8dOQDqESzaoj3spkRvG-A}{192.168.16.185}{192.168.16.185:9300}{m}
Hot threads at 2021-01-15T11:01:09.333Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (26.5micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (24.9micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (20micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#291]'
10/10 snapshots sharing following 11 elements
app//org.elasticsearch.cluster.coordination.Coordinator$5.onResponse(Coordinator.java:1137)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:286)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:277)
com.floragunn.searchguard.transport.SearchGuardInterceptor$RestoringTransportResponseHandler.handleResponse(SearchGuardInterceptor.java:259)
app//org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1163)
app//org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:213)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#293]'
10/10 snapshots sharing following 11 elements
app//org.elasticsearch.cluster.coordination.Coordinator$5.onResponse(Coordinator.java:1137)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:286)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:277)
com.floragunn.searchguard.transport.SearchGuardInterceptor$RestoringTransportResponseHandler.handleResponse(SearchGuardInterceptor.java:259)
app//org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1163)
app//org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:213)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#296]'
10/10 snapshots sharing following 11 elements
app//org.elasticsearch.cluster.coordination.Coordinator$5.onResponse(Coordinator.java:1137)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:286)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:277)
com.floragunn.searchguard.transport.SearchGuardInterceptor$RestoringTransportResponseHandler.handleResponse(SearchGuardInterceptor.java:259)
app//org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1163)
app//org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:213)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#297]'
10/10 snapshots sharing following 35 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:82)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:461)
app//org.apache.lucene.util.IOUtils.fsync(IOUtils.java:471)
app//org.apache.lucene.store.FSDirectory.fsync(FSDirectory.java:331)
app//org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:286)
app//org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(LockValidatingDirectoryWrapper.java:68)
app//org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:458)
app//org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:764)
app//org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4790)
app//org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3277)
app//org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3131)
app//org.elasticsearch.gateway.PersistedClusterStateService$MetadataIndexWriter.prepareCommit(PersistedClusterStateService.java:526)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.commit(PersistedClusterStateService.java:737)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.writeIncrementalStateAndCommit(PersistedClusterStateService.java:616)
app//org.elasticsearch.gateway.GatewayMetaState$LucenePersistedState.setLastAcceptedState(GatewayMetaState.java:543)
app//org.elasticsearch.cluster.coordination.CoordinationState.handlePublishRequest(CoordinationState.java:357)
app//org.elasticsearch.cluster.coordination.Coordinator.handlePublishRequest(Coordinator.java:330)
app//org.elasticsearch.cluster.coordination.Coordinator$$Lambda$2034/0x00000001008cc840.apply(Unknown Source)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.acceptState(PublicationTransportHandler.java:449)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.handleIncomingPublishRequest(PublicationTransportHandler.java:432)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler.lambda$new$0(PublicationTransportHandler.java:100)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$$Lambda$2037/0x00000001008cd440.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1987/0x00000001008b8c40.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#298]'
10/10 snapshots sharing following 11 elements
app//org.elasticsearch.cluster.coordination.Coordinator$5.onResponse(Coordinator.java:1137)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:286)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:277)
com.floragunn.searchguard.transport.SearchGuardInterceptor$RestoringTransportResponseHandler.handleResponse(SearchGuardInterceptor.java:259)
app//org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1163)
app//org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:213)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#299]'
10/10 snapshots sharing following 11 elements
app//org.elasticsearch.cluster.coordination.Coordinator$5.onResponse(Coordinator.java:1137)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:286)
app//org.elasticsearch.cluster.coordination.PublicationTransportHandler$3.handleResponse(PublicationTransportHandler.java:277)
com.floragunn.searchguard.transport.SearchGuardInterceptor$RestoringTransportResponseHandler.handleResponse(SearchGuardInterceptor.java:259)
app//org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1163)
app//org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:213)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#300]'
10/10 snapshots sharing following 5 elements
app//org.elasticsearch.cluster.coordination.Coordinator$CoordinatorPublication$3.run(Coordinator.java:1344)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:636)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][generic][T#301]'
10/10 snapshots sharing following 5 elements
app//org.elasticsearch.cluster.coordination.Coordinator$CoordinatorPublication$2.run(Coordinator.java:1330)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:636)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-master-0][masterService#updateTask][T#1]'
10/10 snapshots sharing following 20 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
app//org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:252)
app//org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:87)
app//org.elasticsearch.common.util.concurrent.FutureUtils.get(FutureUtils.java:56)
app//org.elasticsearch.cluster.service.MasterService.publish(MasterService.java:272)
app//org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:250)
app//org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:73)
app//org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151)
app//org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)
app//org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:636)
app//org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252)
app//org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
::: {abc-elasticsearch-client-768fd6bcc6-zvzkn}{DWpWR2ISQPuZj9E3eWnQPw}{0rWhSPPDSuSxzMkM-cHI2g}{192.168.16.175}{192.168.16.175:9300}{i}
Hot threads at 2021-01-15T11:01:09.333Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (24.5micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (24.4micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (22.4micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
::: {abc-elasticsearch-client-768fd6bcc6-fzrkw}{gt1aKKH1RWeX2i07YqMefA}{pWyjPKsEQQOZee58GxJ86Q}{192.168.152.88}{192.168.152.88:9300}{i}
Hot threads at 2021-01-15T11:01:09.335Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (44.1micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (37.8micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (24.2micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
::: {abc-elasticsearch-data-0}{oztY-9UbQhKBFbp608_1jQ}{ZyC2HyV5S3S5_-sECv6gwg}{192.168.35.141}{192.168.35.141:9300}{d}
Hot threads at 2021-01-15T11:01:09.333Z, interval=500ms, busiestThreads=9999, ignoreIdleThreads=true:
0.0% (41.1micros out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][management][T#1]'
10/10 snapshots sharing following 36 elements
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:481)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3150/0x0000000100c3b440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2910/0x0000000100bb1c40.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2912/0x0000000100bb2440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2104/0x00000001008fac40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (28.4micros out of 500ms) cpu usage by thread 'signals/_main_Worker-1'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (25micros out of 500ms) cpu usage by thread 'signals/_main_Worker-2'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (12.3micros out of 500ms) cpu usage by thread 'signals/_main_Worker-3'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][AsyncLucenePersistedState#updateTask][T#1]'
10/10 snapshots sharing following 20 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:82)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:461)
app//org.apache.lucene.util.IOUtils.fsync(IOUtils.java:471)
app//org.apache.lucene.store.FSDirectory.syncMetaData(FSDirectory.java:309)
app//org.apache.lucene.store.LockValidatingDirectoryWrapper.syncMetaData(LockValidatingDirectoryWrapper.java:62)
app//org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:763)
app//org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4790)
app//org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3277)
app//org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3131)
app//org.elasticsearch.gateway.PersistedClusterStateService$MetadataIndexWriter.prepareCommit(PersistedClusterStateService.java:526)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.commit(PersistedClusterStateService.java:737)
app//org.elasticsearch.gateway.PersistedClusterStateService$Writer.writeIncrementalStateAndCommit(PersistedClusterStateService.java:616)
app//org.elasticsearch.gateway.GatewayMetaState$LucenePersistedState.setLastAcceptedState(GatewayMetaState.java:543)
app//org.elasticsearch.gateway.GatewayMetaState$AsyncLucenePersistedState$1.doRun(GatewayMetaState.java:428)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
unique snapshot
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/7.8.0]'
10/10 snapshots sharing following 8 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
java.base@11.0.9/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
app//org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:89)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'signals/_main_QuartzSchedulerThread'
10/10 snapshots sharing following 2 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][refresh][T#1]'
10/10 snapshots sharing following 25 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:1009)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1324)
java.base@11.0.9/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)
app//org.elasticsearch.common.util.concurrent.ReleasableLock.acquire(ReleasableLock.java:55)
app//org.elasticsearch.index.translog.Translog.getLastWriteLocation(Translog.java:576)
app//org.elasticsearch.index.engine.InternalEngine.getTranslogLastWriteLocation(InternalEngine.java:600)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2748/0x0000000100b68040.get(Unknown Source)
app//org.elasticsearch.index.shard.RefreshListeners.beforeRefresh(RefreshListeners.java:216)
app//org.apache.lucene.search.ReferenceManager.notifyRefreshListenersBefore(ReferenceManager.java:279)
app//org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:175)
app//org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:225)
app//org.elasticsearch.index.engine.InternalEngine.refresh(InternalEngine.java:1625)
app//org.elasticsearch.index.engine.InternalEngine.maybeRefresh(InternalEngine.java:1604)
app//org.elasticsearch.index.shard.IndexShard.scheduledRefresh(IndexShard.java:3246)
app//org.elasticsearch.index.IndexService.maybeRefreshEngine(IndexService.java:839)
app//org.elasticsearch.index.IndexService.access$200(IndexService.java:110)
app//org.elasticsearch.index.IndexService$AsyncRefreshTask.runInternal(IndexService.java:971)
app//org.elasticsearch.common.util.concurrent.AbstractAsyncTask.run(AbstractAsyncTask.java:144)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:636)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][write][T#1]'
10/10 snapshots sharing following 50 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62)
java.base@11.0.9/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)
java.base@11.0.9/sun.nio.ch.IOUtil.write(IOUtil.java:79)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
java.base@11.0.9/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
java.base@11.0.9/java.nio.channels.Channels.writeFully(Channels.java:97)
java.base@11.0.9/java.nio.channels.Channels$1.write(Channels.java:172)
java.base@11.0.9/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
java.base@11.0.9/java.io.BufferedOutputStream.write(BufferedOutputStream.java:127)
app//org.elasticsearch.common.bytes.AbstractBytesReference.writeTo(AbstractBytesReference.java:62)
app//org.elasticsearch.common.bytes.ReleasableBytesReference.writeTo(ReleasableBytesReference.java:111)
app//org.elasticsearch.index.translog.TranslogWriter.add(TranslogWriter.java:174)
app//org.elasticsearch.index.translog.Translog.add(Translog.java:544)
app//org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:957)
app//org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:815)
app//org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:787)
app//org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnReplica(IndexShard.java:751)
app//org.elasticsearch.action.bulk.TransportShardBulkAction.performOpOnReplica(TransportShardBulkAction.java:455)
app//org.elasticsearch.action.bulk.TransportShardBulkAction.performOnReplica(TransportShardBulkAction.java:437)
app//org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnReplica(TransportShardBulkAction.java:408)
app//org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnReplica(TransportShardBulkAction.java:81)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncReplicaAction.onResponse(TransportReplicationAction.java:513)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncReplicaAction.onResponse(TransportReplicationAction.java:487)
app//org.elasticsearch.index.shard.IndexShard.lambda$innerAcquireReplicaOperationPermit$30(IndexShard.java:2995)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$3180/0x0000000100c51440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.lambda$acquireReplicaOperationPermit$28(IndexShard.java:2934)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$3179/0x0000000100c50c40.accept(Unknown Source)
app//org.elasticsearch.index.shard.IndexShard.innerAcquireReplicaOperationPermit(IndexShard.java:3034)
app//org.elasticsearch.index.shard.IndexShard.acquireReplicaOperationPermit(IndexShard.java:2933)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquireReplicaOperationPermit(TransportReplicationAction.java:849)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncReplicaAction.doRun(TransportReplicationAction.java:585)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handleReplicaRequest(TransportReplicationAction.java:472)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2106/0x0000000100938440.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:267)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:136)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'Common-Cleaner'
10/10 snapshots sharing following 5 elements
java.base@11.0.9/java.lang.Object.wait(Native Method)
java.base@11.0.9/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.9/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
java.base@11.0.9/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][management][T#2]'
10/10 snapshots sharing following 42 elements
java.base@11.0.9/sun.nio.fs.UnixNativeDispatcher.readdir(Native Method)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.readNextEntry(UnixDirectoryStream.java:165)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.hasNext(UnixDirectoryStream.java:198)
app//org.elasticsearch.gateway.MetadataStateFormat.findMaxGenerationId(MetadataStateFormat.java:354)
app//org.elasticsearch.gateway.MetadataStateFormat.write(MetadataStateFormat.java:222)
app//org.elasticsearch.gateway.MetadataStateFormat.writeAndCleanup(MetadataStateFormat.java:185)
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:491)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3150/0x0000000100c3b440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2910/0x0000000100bb1c40.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2912/0x0000000100bb2440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2104/0x00000001008fac40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][management][T#3]'
10/10 snapshots sharing following 32 elements
java.base@11.0.9/sun.nio.fs.UnixNativeDispatcher.readdir(Native Method)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.readNextEntry(UnixDirectoryStream.java:165)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.hasNext(UnixDirectoryStream.java:198)
app//org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:215)
app//org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:233)
app//org.elasticsearch.index.store.ByteSizeCachingDirectory.estimateSizeInBytes(ByteSizeCachingDirectory.java:53)
app//org.elasticsearch.index.store.ByteSizeCachingDirectory.access$200(ByteSizeCachingDirectory.java:37)
app//org.elasticsearch.index.store.ByteSizeCachingDirectory$1.refresh(ByteSizeCachingDirectory.java:90)
app//org.elasticsearch.index.store.ByteSizeCachingDirectory$1.refresh(ByteSizeCachingDirectory.java:72)
app//org.elasticsearch.common.util.SingleObjectCache.getOrRefresh(SingleObjectCache.java:54)
app//org.elasticsearch.index.store.ByteSizeCachingDirectory.estimateSizeInBytes(ByteSizeCachingDirectory.java:121)
app//org.elasticsearch.index.store.Store$StoreDirectory.estimateSize(Store.java:697)
app//org.elasticsearch.index.store.Store.stats(Store.java:360)
app//org.elasticsearch.index.shard.IndexShard.storeStats(IndexShard.java:1032)
app//org.elasticsearch.action.admin.indices.stats.CommonStats.<init>(CommonStats.java:177)
app//org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:110)
app//org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:48)
app//org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.onShardOperation(TransportBroadcastByNodeAction.java:427)
app//org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:405)
app//org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:392)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:267)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:136)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][flush][T#1]'
10/10 snapshots sharing following 21 elements
java.base@11.0.9/sun.nio.fs.UnixNativeDispatcher.open0(Native Method)
java.base@11.0.9/sun.nio.fs.UnixNativeDispatcher.open(UnixNativeDispatcher.java:71)
java.base@11.0.9/sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:267)
java.base@11.0.9/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:143)
java.base@11.0.9/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:156)
java.base@11.0.9/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:217)
java.base@11.0.9/java.nio.file.Files.newByteChannel(Files.java:370)
java.base@11.0.9/java.nio.file.Files.createFile(Files.java:647)
java.base@11.0.9/java.nio.file.TempFileHelper.create(TempFileHelper.java:137)
java.base@11.0.9/java.nio.file.TempFileHelper.createTempFile(TempFileHelper.java:160)
java.base@11.0.9/java.nio.file.Files.createTempFile(Files.java:867)
app//org.elasticsearch.index.translog.Translog.copyCheckpointTo(Translog.java:274)
app//org.elasticsearch.index.translog.Translog.rollGeneration(Translog.java:1633)
app//org.elasticsearch.index.engine.InternalEngine.flush(InternalEngine.java:1798)
app//org.elasticsearch.index.shard.IndexShard.flush(IndexShard.java:1093)
app//org.elasticsearch.indices.flush.SyncedFlushService$2.doRun(SyncedFlushService.java:148)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][management][T#4]'
10/10 snapshots sharing following 42 elements
java.base@11.0.9/sun.nio.fs.UnixNativeDispatcher.readdir(Native Method)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.readNextEntry(UnixDirectoryStream.java:165)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.hasNext(UnixDirectoryStream.java:198)
app//org.elasticsearch.gateway.MetadataStateFormat.findMaxGenerationId(MetadataStateFormat.java:354)
app//org.elasticsearch.gateway.MetadataStateFormat.write(MetadataStateFormat.java:222)
app//org.elasticsearch.gateway.MetadataStateFormat.writeAndCleanup(MetadataStateFormat.java:185)
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:491)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3150/0x0000000100c3b440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2910/0x0000000100bb1c40.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2912/0x0000000100bb2440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2104/0x00000001008fac40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][management][T#5]'
10/10 snapshots sharing following 42 elements
java.base@11.0.9/sun.nio.fs.UnixNativeDispatcher.readdir(Native Method)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.readNextEntry(UnixDirectoryStream.java:165)
java.base@11.0.9/sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator.hasNext(UnixDirectoryStream.java:198)
app//org.elasticsearch.gateway.MetadataStateFormat.findMaxGenerationId(MetadataStateFormat.java:354)
app//org.elasticsearch.gateway.MetadataStateFormat.write(MetadataStateFormat.java:222)
app//org.elasticsearch.gateway.MetadataStateFormat.writeAndCleanup(MetadataStateFormat.java:185)
app//org.elasticsearch.index.seqno.ReplicationTracker.persistRetentionLeases(ReplicationTracker.java:491)
app//org.elasticsearch.index.shard.IndexShard.persistRetentionLeases(IndexShard.java:2250)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.lambda$shardOperationOnPrimary$0(RetentionLeaseBackgroundSyncAction.java:158)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction$$Lambda$3150/0x0000000100c3b440.get(Unknown Source)
app//org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:325)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:154)
app//org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction.shardOperationOnPrimary(RetentionLeaseBackgroundSyncAction.java:64)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:895)
app//org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:109)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.runWithPrimaryShardReference(TransportReplicationAction.java:374)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.lambda$doRun$0(TransportReplicationAction.java:297)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction$$Lambda$2910/0x0000000100bb1c40.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)
app//org.elasticsearch.index.shard.IndexShard.lambda$wrapPrimaryOperationPermitListener$24(IndexShard.java:2802)
app//org.elasticsearch.index.shard.IndexShard$$Lambda$2912/0x0000000100bb2440.accept(Unknown Source)
app//org.elasticsearch.action.ActionListener$3.onResponse(ActionListener.java:113)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:285)
app//org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
app//org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2776)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryOperationPermit(TransportReplicationAction.java:836)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:293)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
app//org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:256)
app//org.elasticsearch.action.support.replication.TransportReplicationAction$$Lambda$2104/0x00000001008fac40.messageReceived(Unknown Source)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:147)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:86)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.TransportService$8.doRun(TransportService.java:801)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][generic][T#8]'
10/10 snapshots sharing following 18 elements
java.base@11.0.9/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:1009)
java.base@11.0.9/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1324)
java.base@11.0.9/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)
app//org.elasticsearch.common.util.concurrent.ReleasableLock.acquire(ReleasableLock.java:55)
app//org.elasticsearch.index.translog.Translog.trimUnreferencedReaders(Translog.java:1657)
app//org.elasticsearch.index.engine.InternalEngine.trimUnreferencedTranslogFiles(InternalEngine.java:1885)
app//org.elasticsearch.index.shard.IndexShard.trimTranslog(IndexShard.java:1105)
app//org.elasticsearch.index.IndexService.maybeTrimTranslog(IndexService.java:857)
app//org.elasticsearch.index.IndexService.access$700(IndexService.java:110)
app//org.elasticsearch.index.IndexService$AsyncTrimTranslogTask.runInternal(IndexService.java:999)
app//org.elasticsearch.common.util.concurrent.AbstractAsyncTask.run(AbstractAsyncTask.java:144)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:636)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][generic][T#9]'
10/10 snapshots sharing following 33 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62)
java.base@11.0.9/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)
java.base@11.0.9/sun.nio.ch.IOUtil.write(IOUtil.java:79)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
java.base@11.0.9/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
java.base@11.0.9/java.nio.channels.Channels.writeFully(Channels.java:97)
java.base@11.0.9/java.nio.channels.Channels$1.write(Channels.java:172)
app//org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:416)
java.base@11.0.9/java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:74)
java.base@11.0.9/java.io.BufferedOutputStream.write(BufferedOutputStream.java:123)
app//org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:53)
app//org.elasticsearch.common.lucene.store.FilterIndexOutput.writeBytes(FilterIndexOutput.java:59)
app//org.elasticsearch.index.store.Store$LuceneVerifyingIndexOutput.writeBytes(Store.java:1208)
app//org.elasticsearch.indices.recovery.MultiFileWriter.innerWriteFileChunk(MultiFileWriter.java:126)
app//org.elasticsearch.indices.recovery.MultiFileWriter.access$000(MultiFileWriter.java:46)
app//org.elasticsearch.indices.recovery.MultiFileWriter$FileChunkWriter.writeChunk(MultiFileWriter.java:213)
app//org.elasticsearch.indices.recovery.MultiFileWriter.writeFileChunk(MultiFileWriter.java:74)
app//org.elasticsearch.indices.recovery.RecoveryTarget.writeFileChunk(RecoveryTarget.java:471)
app//org.elasticsearch.indices.recovery.PeerRecoveryTargetService$FileChunkTransportRequestHandler.messageReceived(PeerRecoveryTargetService.java:533)
app//org.elasticsearch.indices.recovery.PeerRecoveryTargetService$FileChunkTransportRequestHandler.messageReceived(PeerRecoveryTargetService.java:507)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:267)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:136)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[abc-elasticsearch-data-0][generic][T#5]'
10/10 snapshots sharing following 33 elements
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
java.base@11.0.9/sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62)
java.base@11.0.9/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)
java.base@11.0.9/sun.nio.ch.IOUtil.write(IOUtil.java:79)
java.base@11.0.9/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
java.base@11.0.9/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
java.base@11.0.9/java.nio.channels.Channels.writeFully(Channels.java:97)
java.base@11.0.9/java.nio.channels.Channels$1.write(Channels.java:172)
app//org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:416)
java.base@11.0.9/java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:74)
java.base@11.0.9/java.io.BufferedOutputStream.write(BufferedOutputStream.java:123)
app//org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:53)
app//org.elasticsearch.common.lucene.store.FilterIndexOutput.writeBytes(FilterIndexOutput.java:59)
app//org.elasticsearch.index.store.Store$LuceneVerifyingIndexOutput.writeBytes(Store.java:1208)
app//org.elasticsearch.indices.recovery.MultiFileWriter.innerWriteFileChunk(MultiFileWriter.java:126)
app//org.elasticsearch.indices.recovery.MultiFileWriter.access$000(MultiFileWriter.java:46)
app//org.elasticsearch.indices.recovery.MultiFileWriter$FileChunkWriter.writeChunk(MultiFileWriter.java:213)
app//org.elasticsearch.indices.recovery.MultiFileWriter.writeFileChunk(MultiFileWriter.java:74)
app//org.elasticsearch.indices.recovery.RecoveryTarget.writeFileChunk(RecoveryTarget.java:471)
app//org.elasticsearch.indices.recovery.PeerRecoveryTargetService$FileChunkTransportRequestHandler.messageReceived(PeerRecoveryTargetService.java:533)
app//org.elasticsearch.indices.recovery.PeerRecoveryTargetService$FileChunkTransportRequestHandler.messageReceived(PeerRecoveryTargetService.java:507)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:164)
com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:267)
com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:136)
com.floragunn.searchguard.SearchGuardPlugin$7.lambda$interceptHandler$0(SearchGuardPlugin.java:683)
com.floragunn.searchguard.SearchGuardPlugin$7$$Lambda$1972/0x0000000100896040.messageReceived(Unknown Source)
app//org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63)
app//org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:263)
app//org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695)
app//org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.9/java.lang.Thread.run(Thread.java:834)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment