-
-
Save brettrann/ce52343692696a45d5b9f4df723bcd14 to your computer and use it in GitHub Desktop.
log clean thread dying after compacting 100GB+ consumer offset
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[2017-10-12 01:47:01,013] INFO Cleaner 0: Beginning cleaning of log __consumer_offsets-36. (kafka.log.LogCleaner) | |
Last message '[2017-10-12 01:47:01' repeated 1 times, suppressed by syslog-ng on kafka9 | |
[2017-10-12 01:47:01,013] INFO Cleaner 0: Building offset map for __consumer_offsets-36... (kafka.log.LogCleaner) | |
Last message '[2017-10-12 01:47:01' repeated 1 times, suppressed by syslog-ng on kafka9 | |
[2017-10-12 01:47:01,029] INFO Cleaner 0: Building offset map for log __consumer_offsets-36 for 11 segments in offset range [0, 2694523870). (kafka.log.LogCleaner) | |
Last message '[2017-10-12 01:47:01' repeated 1 times, suppressed by syslog-ng on kafka9 | |
[2017-10-12 01:47:08,396] INFO Cleaner 0: Offset map for log __consumer_offsets-36 complete. (kafka.log.LogCleaner) | |
Last message '[2017-10-12 01:47:08' repeated 1 times, suppressed by syslog-ng on kafka9 | |
[2017-10-12 01:47:08,397] INFO Cleaner 0: Cleaning log __consumer_offsets-36 (cleaning prior to Thu Oct 12 00:53:07 UTC 2017, discarding tombstones prior to Thu Jan 01 00:00:00 UTC 1970)... (kafka.log.LogCleaner) | |
Last message '[2017-10-12 01:47:08' repeated 1 times, suppressed by syslog-ng on kafka9 | |
[2017-10-12 01:47:08,397] INFO Cleaner 0: Cleaning segment 0 in log __consumer_offsets-36 (largest timestamp Wed Oct 11 05:31:59 UTC 2017) into 0, retaining deletes. (kafka.log.LogCleaner) | |
Last message '[2017-10-12 01:47:08' repeated 1 times, suppressed by syslog-ng on kafka9 | |
[2017-10-12 01:47:08,402] ERROR [kafka-log-cleaner-thread-0]: Error due to (kafka.log.LogCleaner) | |
java.lang.IllegalArgumentException: requirement failed: largest offset in message set can not be safely converted to relative offset. | |
at scala.Predef$.require(Predef.scala:224) | |
at kafka.log.LogSegment.append(LogSegment.scala:121) | |
at kafka.log.Cleaner.cleanInto(LogCleaner.scala:547) | |
at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:443) | |
at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:385) | |
at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:384) | |
at scala.collection.immutable.List.foreach(List.scala:392) | |
at kafka.log.Cleaner.doClean(LogCleaner.scala:384) | |
at kafka.log.Cleaner.clean(LogCleaner.scala:361) | |
at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:256) | |
at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:236) | |
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64) | |
[2017-10-12 01:47:08,402] ERROR [kafka-log-cleaner-thread-0]: Error due to (kafka.log.LogCleaner) | |
java.lang.IllegalArgumentException: requirement failed: largest offset in message set can not be safely converted to relative offset. | |
at scala.Predef$.require(Predef.scala:224) | |
at kafka.log.LogSegment.append(LogSegment.scala:121) | |
at kafka.log.Cleaner.cleanInto(LogCleaner.scala:547) | |
at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:443) | |
at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:385) | |
at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:384) | |
at scala.collection.immutable.List.foreach(List.scala:392) | |
at kafka.log.Cleaner.doClean(LogCleaner.scala:384) | |
at kafka.log.Cleaner.clean(LogCleaner.scala:361) | |
at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:256) | |
at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:236) | |
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64) | |
[2017-10-12 01:47:08,406] INFO [kafka-log-cleaner-thread-0]: Stopped (kafka.log.LogCleaner) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment