Skip to content

Instantly share code, notes, and snippets.

@codyaray

codyaray/hot_threads.txt

Created January 11, 2016 19:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save codyaray/0fd390ae9f64f28d317e to your computer and use it in GitHub Desktop.
Save codyaray/0fd390ae9f64f28d317e to your computer and use it in GitHub Desktop.
Download ZIP
Lots of Lucene Merges
Raw
hot_threads.txt
::: [svlchi6pesm03.peak6.net][0zQiOjCySx2lc2CjKKAuxQ][svlchi6pesm03][inet[/10.2.4.180:9300]]{data=false, box_type=mgmt, master=true}
Hot threads at 2016-01-11T19:40:08.163Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
0.0% (198.7micros out of 500ms) cpu usage by thread 'elasticsearch[svlchi6pesm03.peak6.net][keep_alive]'
10/10 snapshots sharing following 3 elements
java.lang.Thread.sleep(Native Method)
org.elasticsearch.marvel.agent.exporter.ESExporter$ConnectionKeepAliveWorker.run(ESExporter.java:788)
java.lang.Thread.run(Thread.java:745)
::: [svlchi6pes02.peak6.net][7QVV3C62QeyeH9RySjm7Cg][svlchi6pes02][inet[/10.2.4.182:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi6pes01.peak6.net][u3lgZbZoRy-zgTz7jOq33g][svlchi6pes01][inet[/10.2.4.181:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.164Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi5pfes08.peak6.net][uaLcTHohTiyaRCHHMtxkUQ][svlchi5pfes08][inet[/10.100.0.179:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
7.8% (38.9ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes08.peak6.net][[recovery_stream]][T#411]'
3/10 snapshots sharing following 9 elements
java.nio.Bits.copyToArray(Bits.java:781)
java.nio.DirectByteBuffer.get(DirectByteBuffer.java:274)
org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferIndexInput.java:95)
org.apache.lucene.store.DataInput.readBytes(DataInput.java:84)
org.elasticsearch.indices.recovery.RecoverySourceHandler$3.doRun(RecoverySourceHandler.java:313)
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
7/10 snapshots sharing following 16 elements
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:278)
org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:117)
org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture.java:45)
org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture.java:34)
org.elasticsearch.indices.recovery.RecoverySourceHandler$3$1.run(RecoverySourceHandler.java:323)
org.elasticsearch.common.util.CancellableThreads.execute(CancellableThreads.java:83)
org.elasticsearch.indices.recovery.RecoverySourceHandler$3.doRun(RecoverySourceHandler.java:319)
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
::: [svlchi6pesm01.peak6.net][YHffHzSlTcKcSDeer8L0hQ][svlchi6pesm01.peak6.net][inet[/10.2.4.186:9300]]{data=false, box_type=mgmt, master=true}
Hot threads at 2016-01-11T19:40:08.164Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi5pfes03.peak6.net][uWadUHGYSaCPfo3MnG0VsA][svlchi5pfes03][inet[/10.100.0.174:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi5pfes07.peak6.net][-AKwely9TviAFgz4m1COsA][svlchi5pfes07][inet[/10.100.0.178:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.166Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
84.9% (424.5ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes07.peak6.net][[logstash-2016.01.11][5]: Lucene Merge Thread #5267]'
2/10 snapshots sharing following 11 elements
org.apache.lucene.index.MultiTermsEnum.pushTop(MultiTermsEnum.java:293)
org.apache.lucene.index.MultiTermsEnum.next(MultiTermsEnum.java:319)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:103)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
8/10 snapshots sharing following 8 elements
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
14.7% (73.7ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes07.peak6.net][bulk][T#26]'
4/10 snapshots sharing following 2 elements
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
::: [svlchi5pfes01.peak6.net][E-nLOTFwQFSB0X4HxggcnA][svlchi5pfes01][inet[/10.100.0.172:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.163Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
101.1% (505.6ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes01.peak6.net][[logstash-2016.01.11][2]: Lucene Merge Thread #2040]'
2/10 snapshots sharing following 22 elements
sun.nio.ch.NativeThread.current(Native Method)
sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:695)
sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:684)
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:140)
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:116)
org.apache.lucene.codecs.lucene41.ForUtil.readBlock(ForUtil.java:207)
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsAndPositionsEnum.refillPositions(Lucene41PostingsReader.java:763)
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsAndPositionsEnum.nextPosition(Lucene41PostingsReader.java:960)
org.apache.lucene.codecs.MappingMultiDocsAndPositionsEnum.nextPosition(MappingMultiDocsAndPositionsEnum.java:120)
org.apache.lucene.codecs.PostingsConsumer.merge(PostingsConsumer.java:118)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:164)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
2/10 snapshots sharing following 13 elements
org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadNextFloorBlock(SegmentTermsEnumFrame.java:126)
org.apache.lucene.codecs.blocktree.SegmentTermsEnum.next(SegmentTermsEnum.java:917)
org.apache.lucene.index.MultiTermsEnum.pushTop(MultiTermsEnum.java:293)
org.apache.lucene.index.MultiTermsEnum.next(MultiTermsEnum.java:319)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:156)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
2/10 snapshots sharing following 22 elements
sun.nio.ch.NativeThread.current(Native Method)
sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:695)
sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:684)
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
org.apache.lucene.store.DataInput.readVInt(DataInput.java:122)
org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:221)
org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:152)
org.apache.lucene.codecs.blocktree.SegmentTermsEnum.next(SegmentTermsEnum.java:959)
org.apache.lucene.index.MultiTermsEnum.pushTop(MultiTermsEnum.java:293)
org.apache.lucene.index.MultiTermsEnum.next(MultiTermsEnum.java:319)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:156)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
3/10 snapshots sharing following 9 elements
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:164)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
unique snapshot
java.io.FileOutputStream.writeBytes(Native Method)
java.io.FileOutputStream.write(FileOutputStream.java:345)
org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:390)
java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:73)
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:51)
org.apache.lucene.store.DataOutput.writeBytes(DataOutput.java:53)
org.apache.lucene.store.RAMOutputStream.writeTo(RAMOutputStream.java:76)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.writeBlock(BlockTreeTermsWriter.java:838)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.writeBlocks(BlockTreeTermsWriter.java:647)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.pushTerm(BlockTreeTermsWriter.java:943)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.finishTerm(BlockTreeTermsWriter.java:912)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:166)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
101.0% (505.1ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes01.peak6.net][[stats-2016.01.11][2]: Lucene Merge Thread #4409]'
2/10 snapshots sharing following 16 elements
java.util.zip.CRC32.updateBytes(Native Method)
java.util.zip.CRC32.update(CRC32.java:65)
org.apache.lucene.store.BufferedChecksum.flush(BufferedChecksum.java:80)
org.apache.lucene.store.BufferedChecksum.update(BufferedChecksum.java:47)
org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:42)
org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:110)
org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:508)
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:378)
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:332)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:100)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
8/10 snapshots sharing following 7 elements
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:332)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:100)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
100.9% (504.6ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes01.peak6.net][[stats-2016.01.11][4]: Lucene Merge Thread #3981]'
2/10 snapshots sharing following 16 elements
org.apache.lucene.util.fst.BytesStore.writeByte(BytesStore.java:89)
org.apache.lucene.util.fst.FST.<init>(FST.java:302)
org.apache.lucene.util.fst.Builder.<init>(Builder.java:155)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$PendingBlock.compileIndex(BlockTreeTermsWriter.java:476)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.writeBlocks(BlockTreeTermsWriter.java:656)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.pushTerm(BlockTreeTermsWriter.java:943)
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.finishTerm(BlockTreeTermsWriter.java:912)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:112)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
8/10 snapshots sharing following 8 elements
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
::: [svlchi6pes07.peak6.net][wSEY25OARx2t_PYU7ke3dw][svlchi6pes07][inet[/10.2.4.190:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.164Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi6pesm02.peak6.net][U9YcfSMhRc2g-8wOMcp_Nw][svlchi6pesm02.peak6.net][inet[/10.2.4.187:9300]]{data=false, box_type=mgmt, master=true}
Hot threads at 2016-01-11T19:40:08.164Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi5pfes05.peak6.net][DHORYEdOQ7y7fmmDzckmKg][svlchi5pfes05][inet[/10.100.0.176:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
101.0% (504.9ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes05.peak6.net][[stats-2016.01.11][5]: Lucene Merge Thread #7348]'
6/10 snapshots sharing following 10 elements
org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:508)
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:378)
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:332)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:100)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
2/10 snapshots sharing following 8 elements
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:332)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:100)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
2/10 snapshots sharing following 12 elements
org.apache.lucene.codecs.compressing.LZ4.compress(LZ4.java:243)
org.apache.lucene.codecs.compressing.CompressionMode$LZ4FastCompressor.compress(CompressionMode.java:161)
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:236)
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:166)
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:388)
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:332)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:100)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
94.7% (473.2ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes05.peak6.net][http_server_worker][T#22]{New I/O worker #72}'
10/10 snapshots sharing following 18 elements
sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51)
sun.nio.ch.IOUtil.write(IOUtil.java:148)
sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:524)
java.nio.channels.SocketChannel.write(SocketChannel.java:493)
org.elasticsearch.common.netty.channel.socket.nio.SocketSendBufferPool$GatheringSendBuffer.transferTo(SocketSendBufferPool.java:267)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.writeFromTaskLoop(AbstractNioWorker.java:151)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioChannel$WriteTask.run(AbstractNioChannel.java:315)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
30.2% (150.9ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes05.peak6.net][bulk][T#10]'
4/10 snapshots sharing following 9 elements
org.elasticsearch.index.shard.IndexShard.create(IndexShard.java:483)
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:423)
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:148)
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.performOnPrimary(TransportShardReplicationOperationAction.java:574)
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase$1.doRun(TransportShardReplicationOperationAction.java:440)
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
2/10 snapshots sharing following 10 elements
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:735)
java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:644)
java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1137)
org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:162)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
::: [svlchi6pes05.peak6.net][u3OLXtSLQ02OtgHoWhkvMg][svlchi6pes05][inet[svlchi6pes05.peak6.net/10.2.4.188:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.164Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi5pfes02.peak6.net][8DmJ80nsTduNSpgiE6Pw_A][svlchi5pfes02][inet[/10.100.0.173:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.163Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
27.2% (135.9ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes02.peak6.net][bulk][T#8]'
4/10 snapshots sharing following 2 elements
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
26.1% (130.3ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes02.peak6.net][bulk][T#35]'
5/10 snapshots sharing following 2 elements
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
::: [svlchi6pes06.peak6.net][D3DDGBe4Qs-7ZL4zBb-Hdg][svlchi6pes06][inet[/10.2.4.189:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi5pfes06.peak6.net][V0YnfMScTy2SHBzADhibMQ][svlchi5pfes06][inet[/10.100.0.177:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
80.9% (404.5ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes06.peak6.net][[stats-2016.01.11][3]: Lucene Merge Thread #7254]'
2/10 snapshots sharing following 22 elements
sun.nio.ch.NativeThread.current(Native Method)
sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:695)
sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:684)
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:140)
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:116)
org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:171)
org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadNextFloorBlock(SegmentTermsEnumFrame.java:126)
org.apache.lucene.codecs.blocktree.SegmentTermsEnum.next(SegmentTermsEnum.java:917)
org.apache.lucene.index.MultiTermsEnum.pushTop(MultiTermsEnum.java:293)
org.apache.lucene.index.MultiTermsEnum.next(MultiTermsEnum.java:319)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:103)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
3/10 snapshots sharing following 22 elements
sun.nio.ch.NativeThread.current(Native Method)
sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:695)
sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:684)
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:140)
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:116)
org.apache.lucene.codecs.lucene41.ForUtil.readBlock(ForUtil.java:207)
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.refillDocs(Lucene41PostingsReader.java:404)
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.nextDoc(Lucene41PostingsReader.java:446)
org.apache.lucene.codecs.MappingMultiDocsEnum.nextDoc(MappingMultiDocsEnum.java:104)
org.apache.lucene.codecs.PostingsConsumer.merge(PostingsConsumer.java:83)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:110)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
3/10 snapshots sharing following 12 elements
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.nextDoc(Lucene41PostingsReader.java:446)
org.apache.lucene.codecs.MappingMultiDocsEnum.nextDoc(MappingMultiDocsEnum.java:104)
org.apache.lucene.codecs.PostingsConsumer.merge(PostingsConsumer.java:83)
org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:110)
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
2/10 snapshots sharing following 8 elements
org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:399)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:112)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
14.7% (73.6ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes06.peak6.net][bulk][T#24]'
2/10 snapshots sharing following 10 elements
org.elasticsearch.index.engine.InternalEngine.create(InternalEngine.java:269)
org.elasticsearch.index.shard.IndexShard.create(IndexShard.java:483)
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:423)
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:148)
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.performOnPrimary(TransportShardReplicationOperationAction.java:574)
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase$1.doRun(TransportShardReplicationOperationAction.java:440)
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
3/10 snapshots sharing following 2 elements
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
::: [svlchi6pes08.peak6.net][PIdBM5vTRaaBk4TO9-diOA][svlchi6pes08][inet[/10.2.4.191:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi6pes03.peak6.net][6sYeHP3tT3GXBqomc2MzGg][svlchi6pes03][inet[/10.2.4.183:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [sslchi6pglacieres01.peak6.net][N0B5vl5_RB-HwYTDXEmwvQ][sslchi6pglacieres01][inet[/10.2.4.87:9300]]{box_type=glacier, master=false}
Hot threads at 2016-01-11T19:40:08.163Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi6pes04.peak6.net][-YJ6doXGRGS76qM0r9te3Q][svlchi6pes04][inet[/10.2.4.184:9300]]{box_type=slow, master=false}
Hot threads at 2016-01-11T19:40:08.163Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: [svlchi5pfes04.peak6.net][84ZrcLMSSdyKrZFl8JWbIw][svlchi5pfes04][inet[/10.100.0.175:9300]]{box_type=fast, master=false}
Hot threads at 2016-01-11T19:40:08.165Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
100.5% (502.4ms out of 500ms) cpu usage by thread 'elasticsearch[svlchi5pfes04.peak6.net][[logstash-2016.01.11][5]: Lucene Merge Thread #2494]'
2/10 snapshots sharing following 22 elements
java.io.FileOutputStream.writeBytes(Native Method)
java.io.FileOutputStream.write(FileOutputStream.java:345)
org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:390)
java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:73)
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:51)
org.apache.lucene.store.DataOutput.writeBytes(DataOutput.java:53)
org.apache.lucene.util.packed.DirectWriter.flush(DirectWriter.java:86)
org.apache.lucene.util.packed.DirectWriter.add(DirectWriter.java:78)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addNumericField(Lucene410DocValuesConsumer.java:216)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedField(Lucene410DocValuesConsumer.java:472)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedSetField(Lucene410DocValuesConsumer.java:500)
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addSortedSetField(PerFieldDocValuesFormat.java:127)
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedSetField(DocValuesConsumer.java:626)
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:213)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
5/10 snapshots sharing following 27 elements
sun.nio.ch.NativeThread.current(Native Method)
sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:695)
sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:684)
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:269)
org.apache.lucene.util.packed.DirectReader$DirectPackedReader4.get(DirectReader.java:112)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesProducer$1.get(Lucene410DocValuesProducer.java:359)
org.apache.lucene.util.LongValues.get(LongValues.java:45)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesProducer$6.getOrd(Lucene410DocValuesProducer.java:508)
org.apache.lucene.index.SingletonSortedSetDocValues.setDocument(SingletonSortedSetDocValues.java:54)
org.apache.lucene.codecs.DocValuesConsumer$8$1.setNext(DocValuesConsumer.java:710)
org.apache.lucene.codecs.DocValuesConsumer$8$1.hasNext(DocValuesConsumer.java:672)
org.apache.lucene.codecs.DocValuesConsumer$10$1.hasNext(DocValuesConsumer.java:855)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addNumericField(Lucene410DocValuesConsumer.java:214)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedField(Lucene410DocValuesConsumer.java:472)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedSetField(Lucene410DocValuesConsumer.java:500)
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addSortedSetField(PerFieldDocValuesFormat.java:127)
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedSetField(DocValuesConsumer.java:626)
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:213)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
2/10 snapshots sharing following 30 elements
sun.nio.ch.NativeThread.current(Native Method)
sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:695)
sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:684)
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:269)
org.apache.lucene.util.packed.DirectReader$DirectPackedReader4.get(DirectReader.java:112)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesProducer$1.get(Lucene410DocValuesProducer.java:359)
org.apache.lucene.util.LongValues.get(LongValues.java:45)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesProducer$6.getOrd(Lucene410DocValuesProducer.java:508)
org.apache.lucene.index.SingletonSortedSetDocValues.setDocument(SingletonSortedSetDocValues.java:54)
org.apache.lucene.codecs.DocValuesConsumer$9$1.setNext(DocValuesConsumer.java:789)
org.apache.lucene.codecs.DocValuesConsumer$9$1.hasNext(DocValuesConsumer.java:743)
org.apache.lucene.codecs.DocValuesConsumer$9$1.next(DocValuesConsumer.java:753)
org.apache.lucene.codecs.DocValuesConsumer$9$1.next(DocValuesConsumer.java:729)
org.apache.lucene.codecs.DocValuesConsumer$10$1.next(DocValuesConsumer.java:864)
org.apache.lucene.codecs.DocValuesConsumer$10$1.next(DocValuesConsumer.java:851)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addNumericField(Lucene410DocValuesConsumer.java:214)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedField(Lucene410DocValuesConsumer.java:472)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedSetField(Lucene410DocValuesConsumer.java:500)
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addSortedSetField(PerFieldDocValuesFormat.java:127)
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedSetField(DocValuesConsumer.java:626)
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:213)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
unique snapshot
org.apache.lucene.util.packed.DirectReader$DirectPackedReader4.get(DirectReader.java:112)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesProducer$1.get(Lucene410DocValuesProducer.java:359)
org.apache.lucene.util.LongValues.get(LongValues.java:45)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesProducer$6.getOrd(Lucene410DocValuesProducer.java:508)
org.apache.lucene.index.SingletonSortedSetDocValues.setDocument(SingletonSortedSetDocValues.java:54)
org.apache.lucene.codecs.DocValuesConsumer$9$1.setNext(DocValuesConsumer.java:789)
org.apache.lucene.codecs.DocValuesConsumer$9$1.hasNext(DocValuesConsumer.java:743)
org.apache.lucene.codecs.DocValuesConsumer$9$1.next(DocValuesConsumer.java:753)
org.apache.lucene.codecs.DocValuesConsumer$9$1.next(DocValuesConsumer.java:729)
org.apache.lucene.codecs.DocValuesConsumer$10$1.next(DocValuesConsumer.java:864)
org.apache.lucene.codecs.DocValuesConsumer$10$1.next(DocValuesConsumer.java:851)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addNumericField(Lucene410DocValuesConsumer.java:214)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedField(Lucene410DocValuesConsumer.java:472)
org.apache.lucene.codecs.lucene410.Lucene410DocValuesConsumer.addSortedSetField(Lucene410DocValuesConsumer.java:500)
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addSortedSetField(PerFieldDocValuesFormat.java:127)
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedSetField(DocValuesConsumer.java:626)
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:213)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4223)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3811)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:107)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
@linajiang
Copy link

hi ,can you tell me how to get this thred info from my own elasticsearch cluster? thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment