Skip to content

Instantly share code, notes, and snippets.

@mimani

mimani/host1

Created December 9, 2018 17:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mimani/ff27b7272482efc91c4d145d59ab59be to your computer and use it in GitHub Desktop.
Save mimani/ff27b7272482efc91c4d145d59ab59be to your computer and use it in GitHub Desktop.
Download ZIP
Logs-1 After fix Provided from PR: 2923
Raw
host1
worker-6701.log:2018-12-09T22:47:14.866+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103, 1682104, 1682105, 1682106], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682105, numFails=0, emitted=true}], latestEmittedOffset=1682106}}, emitted=[{topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682104, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682106, numFails=0, emitted=true}]}
worker-6701.log:2018-12-09T22:47:17.655+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103, 1682104, 1682105, 1682106, 1682107], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682105, numFails=0, emitted=true}], latestEmittedOffset=1682107}}, emitted=[{topic-partition=myTopic-6, offset=1682104, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682107, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682106, numFails=0, emitted=true}]}
worker-6701.log:2018-12-09T22:47:17.867+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103, 1682104, 1682105, 1682106, 1682107, 1682108], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682104, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682105, numFails=0, emitted=true}], latestEmittedOffset=1682108}}, emitted=[{topic-partition=myTopic-6, offset=1682107, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682106, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682108, numFails=0, emitted=true}]}
worker-6701.log:2018-12-09T22:47:18.384+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103, 1682104, 1682105, 1682106, 1682107, 1682108, 1682109], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682104, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682105, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682109, numFails=0, emitted=true}], latestEmittedOffset=1682109}}, emitted=[{topic-partition=myTopic-6, offset=1682107, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682106, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682108, numFails=0, emitted=true}]}
worker-6701.log:2018-12-09T22:47:18.943+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103, 1682104, 1682105, 1682106, 1682107, 1682108, 1682109, 1682110], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682104, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682105, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682109, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682110, numFails=0, emitted=true}], latestEmittedOffset=1682110}}, emitted=[{topic-partition=myTopic-6, offset=1682107, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682106, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682108, numFails=0, emitted=true}]}
worker-6701.log:java.lang.IllegalStateException: Record:ConsumerRecord(topic = myTopic, partition = 6, offset = 1682099, CreateTime = 1544367700469, checksum = 3496326571, serialized key size = 36, serialized value size = 4370, key = a88f405b-6aaa-428d-a752-c7ee6f7c3e9e, value = [B@7b4d0be7), Tuple:["myTopic",6,1682099,"a88f405b-6aaa-428d-a752-c7ee6f7c3e9e","jIzOC4....xNDYifX0="], Message:{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}Attempting to emit a message that has already been committed. This should never occur when using the at-least-once processing guarantee.
worker-6701.log:java.lang.IllegalStateException: Record:ConsumerRecord(topic = myTopic, partition = 6, offset = 1682099, CreateTime = 1544367700469, checksum = 3496326571, serialized key size = 36, serialized value size = 4370, key = a88f405b-6aaa-428d-a752-c7ee6f7c3e9e, value = [B@7b4d0be7), Tuple:["myTopic",6,1682099,"a88f405b-6aaa-428d-a752-c7ee6f7c3e9e","jIzOC4....xNDYifX0="], Message:{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}Attempting to emit a message that has already been committed. This should never occur when using the at-least-once processing guarantee.
worker-6701.log.10.gz:2018-12-09T22:47:02.645+05:30 KafkaSpout [INFO] Emitted tuple [[myTopic, 6, 1682099, a88f405b-6aaa-428d-a752-c7ee6f7c3e9e, [123, 34, 105, .... 52, 54, 34, 125, 125]]] for record [ConsumerRecord(topic = myTopic, partition = 6, offset = 1682099, CreateTime = 1544367700469, checksum = 3496326571, serialized key size = 36, serialized value size = 4370, key = a88f405b-6aaa-428d-a752-c7ee6f7c3e9e, value = [B@778adecc)] with msgId [{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}]
worker-6701.log.10.gz:2018-12-09T22:47:09.701+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}], latestEmittedOffset=1682102}}, emitted=[{topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}]}
worker-6701.log.10.gz:2018-12-09T22:47:09.977+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}], latestEmittedOffset=1682103}}, emitted=[{topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}]}
worker-6701.log.10.gz:2018-12-09T22:47:13.030+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103, 1682104], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}], latestEmittedOffset=1682104}}, emitted=[{topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682104, numFails=0, emitted=true}]}
worker-6701.log.10.gz:2018-12-09T22:47:14.233+05:30 KafkaSpout [INFO] No offsets to commit. KafkaSpout{offsetManagers ={myTopic-6=OffsetManager{topic-partition=myTopic-6, committedOffset=1682098, emittedOffsets=[1682098, 1682099, 1682100, 1682101, 1682102, 1682103, 1682104, 1682105], ackedMsgs=[{topic-partition=myTopic-6, offset=1682099, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682100, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682101, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682102, numFails=0, emitted=true}], latestEmittedOffset=1682105}}, emitted=[{topic-partition=myTopic-6, offset=1682103, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682105, numFails=0, emitted=true}, {topic-partition=myTopic-6, offset=1682104, numFails=0, emitted=true}]}
worker-6701.log.9.gz:2018-12-09T22:46:38.708+05:30 KafkaSpout [INFO] Emitted tuple [[myTopic, 2, 1682099, 80b2878f-2145-46c7-a5d5-edb718261288, [123, 34, 105, 100, ...., 54, 57, 34, 125, 125]]] for record [ConsumerRecord(topic = myTopic, partition = 2, offset = 1682099, CreateTime = 1544367702783, checksum = 643044652, serialized key size = 36, serialized value size = 4372, key = 80b2878f-2145-46c7-a5d5-edb718261288, value = [B@40ad0c9f)] with msgId [{topic-partition=myTopic-2, offset=1682099, numFails=0, emitted=true}]
Raw
host2
worker-6701.log.9.gz:2018-12-09T22:46:21.112+05:30 KafkaSpout [INFO] Emitted tuple [[myTopic, 3, 1682099, 4062fc94-be3a-420c-907f-5c9b939e1762, [123, 34, 105, 54, .... 34, 125, 125]]] for record [ConsumerRecord(topic = myTopic, partition = 3, offset = 1682099, CreateTime = 1544367703014, checksum = 3572239661, serialized key size = 36, serialized value size = 4372, key = 4062fc94-be3a-420c-907f-5c9b939e1762, value = [B@cd80358)] with msgId [{topic-partition=myTopic-3, offset=1682099, numFails=0, emitted=true}]
worker-6701.log.9.gz:2018-12-09T22:46:22.770+05:30 KafkaSpout [INFO] Offsets successfully committed to Kafka [{myTopic-3=OffsetAndMetadata{offset=1682099, metadata='{"topologyId":"myTopo_myTopic-151-1544367275","taskId":9,"threadName":"Thread-133-KafkaSpout-executor[9 9]"}'}}]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment