Skip to content

Instantly share code, notes, and snippets.

@akheron

akheron/kafka-python.log

Created March 10, 2016 12:48
Show Gist options
  • Save akheron/8e54c00ab2d19196fe2c to your computer and use it in GitHub Desktop.
Save akheron/8e54c00ab2d19196fe2c to your computer and use it in GitHub Desktop.
Download ZIP
kafka-python heartbeat issue
Raw
kafka-python.log
[2016-03-10 14:34:50,186] DEBUG base 14747 139872076707584 Received successful heartbeat
[2016-03-10 14:34:50,186] DEBUG fetcher 14747 139872076707584 Adding fetch request for partition TopicPartition(topic='TOPIC-NAME', partition=0) at offset 143454
[2016-03-10 14:34:50,186] DEBUG fetcher 14747 139872076707584 Sending FetchRequest to node 1
[2016-03-10 14:34:50,187] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 975: FetchRequest(replica_id=-1, max_wait_time=500, min_bytes=1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, max_bytes=1048576)])])
[2016-03-10 14:34:50,683] DEBUG consumer 14747 139872076707584 Sending offset-commit request with {TopicPartition(topic='TOPIC-NAME', partition=0): OffsetAndMetadata(offset=143454, metadata='')} to 1
[2016-03-10 14:34:50,684] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 976: OffsetCommitRequest_v2(consumer_group='GROUP-NAME', consumer_group_generation_id=1, consumer_id='bafljmae-ec2ae490-c1ed-4721-86f6-3b16f991fed6', retention_time=-1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, metadata='')])])
[2016-03-10 14:34:50,687] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 975: FetchResponse(topics=[(topics='TOPIC-NAME', partitions=[(partition=0, error_code=0, highwater_offset=143454, message_set=[])])])
[2016-03-10 14:34:50,689] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 976: OffsetCommitResponse(topics=[(topic='TOPIC-NAME', partitions=[(partition=0, error_code=0)])])
[2016-03-10 14:34:50,689] DEBUG consumer 14747 139872076707584 Committed offset OffsetAndMetadata(offset=143454, metadata='') for partition TopicPartition(topic='TOPIC-NAME', partition=0)
[2016-03-10 14:34:50,690] DEBUG consumer 14747 139872076707584 Successfully auto-committed offsets
[2016-03-10 14:34:50,690] DEBUG fetcher 14747 139872076707584 Adding fetch request for partition TopicPartition(topic='TOPIC-NAME', partition=0) at offset 143454
[2016-03-10 14:34:50,690] DEBUG fetcher 14747 139872076707584 Sending FetchRequest to node 1
[2016-03-10 14:34:50,690] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 977: FetchRequest(replica_id=-1, max_wait_time=500, min_bytes=1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, max_bytes=1048576)])])
[2016-03-10 14:34:51,230] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 977: FetchResponse(topics=[(topics='TOPIC-NAME', partitions=[(partition=0, error_code=0, highwater_offset=143454, message_set=[])])])
[2016-03-10 14:34:51,230] DEBUG fetcher 14747 139872076707584 Adding fetch request for partition TopicPartition(topic='TOPIC-NAME', partition=0) at offset 143454
[2016-03-10 14:34:51,231] DEBUG fetcher 14747 139872076707584 Sending FetchRequest to node 1
[2016-03-10 14:34:51,231] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 978: FetchRequest(replica_id=-1, max_wait_time=500, min_bytes=1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, max_bytes=1048576)])])
[2016-03-10 14:34:51,690] DEBUG consumer 14747 139872076707584 Sending offset-commit request with {TopicPartition(topic='TOPIC-NAME', partition=0): OffsetAndMetadata(offset=143454, metadata='')} to 1
[2016-03-10 14:34:51,690] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 979: OffsetCommitRequest_v2(consumer_group='GROUP-NAME', consumer_group_generation_id=1, consumer_id='bafljmae-ec2ae490-c1ed-4721-86f6-3b16f991fed6', retention_time=-1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, metadata='')])])
[2016-03-10 14:34:51,732] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 978: FetchResponse(topics=[(topics='TOPIC-NAME', partitions=[(partition=0, error_code=0, highwater_offset=143454, message_set=[])])])
[2016-03-10 14:34:51,734] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 979: OffsetCommitResponse(topics=[(topic='TOPIC-NAME', partitions=[(partition=0, error_code=0)])])
[2016-03-10 14:34:51,735] DEBUG consumer 14747 139872076707584 Committed offset OffsetAndMetadata(offset=143454, metadata='') for partition TopicPartition(topic='TOPIC-NAME', partition=0)
[2016-03-10 14:34:51,735] DEBUG consumer 14747 139872076707584 Successfully auto-committed offsets
[2016-03-10 14:34:51,735] DEBUG fetcher 14747 139872076707584 Adding fetch request for partition TopicPartition(topic='TOPIC-NAME', partition=0) at offset 143454
[2016-03-10 14:34:51,735] DEBUG fetcher 14747 139872076707584 Sending FetchRequest to node 1
[2016-03-10 14:34:51,736] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 980: FetchRequest(replica_id=-1, max_wait_time=500, min_bytes=1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, max_bytes=1048576)])])
[2016-03-10 14:34:52,236] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 980: FetchResponse(topics=[(topics='TOPIC-NAME', partitions=[(partition=0, error_code=0, highwater_offset=143454, message_set=[])])])
[2016-03-10 14:34:52,237] DEBUG fetcher 14747 139872076707584 Adding fetch request for partition TopicPartition(topic='TOPIC-NAME', partition=0) at offset 143454
[2016-03-10 14:34:52,237] DEBUG fetcher 14747 139872076707584 Sending FetchRequest to node 1
[2016-03-10 14:34:52,237] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 981: FetchRequest(replica_id=-1, max_wait_time=500, min_bytes=1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, max_bytes=1048576)])])
[2016-03-10 14:34:52,735] DEBUG consumer 14747 139872076707584 Sending offset-commit request with {TopicPartition(topic='TOPIC-NAME', partition=0): OffsetAndMetadata(offset=143454, metadata='')} to 1
[2016-03-10 14:34:52,736] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 982: OffsetCommitRequest_v2(consumer_group='GROUP-NAME', consumer_group_generation_id=1, consumer_id='bafljmae-ec2ae490-c1ed-4721-86f6-3b16f991fed6', retention_time=-1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, metadata='')])])
[2016-03-10 14:34:52,738] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 981: FetchResponse(topics=[(topics='TOPIC-NAME', partitions=[(partition=0, error_code=0, highwater_offset=143454, message_set=[])])])
[2016-03-10 14:34:52,740] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Response 982: OffsetCommitResponse(topics=[(topic='TOPIC-NAME', partitions=[(partition=0, error_code=0)])])
[2016-03-10 14:34:52,740] DEBUG consumer 14747 139872076707584 Committed offset OffsetAndMetadata(offset=143454, metadata='') for partition TopicPartition(topic='TOPIC-NAME', partition=0)
[2016-03-10 14:34:52,740] DEBUG consumer 14747 139872076707584 Successfully auto-committed offsets
[2016-03-10 14:34:52,741] DEBUG fetcher 14747 139872076707584 Adding fetch request for partition TopicPartition(topic='TOPIC-NAME', partition=0) at offset 143454
[2016-03-10 14:34:52,741] DEBUG fetcher 14747 139872076707584 Sending FetchRequest to node 1
[2016-03-10 14:34:52,741] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 983: FetchRequest(replica_id=-1, max_wait_time=500, min_bytes=1, topics=[(topic='TOPIC-NAME', partitions=[(partition=0, offset=143454, max_bytes=1048576)])])
[2016-03-10 14:34:52,774] DEBUG client_async 14747 139872076707584 Sending metadata request MetadataRequest(topics=['TOPIC-NAME']) to node 1
[2016-03-10 14:34:52,774] DEBUG conn 14747 139872076707584 <BrokerConnection host=SERVER-HOSTNAME port=9092> Request 984: MetadataRequest(topics=['TOPIC-NAME'])
[2016-03-10 14:34:52,926] DEBUG base 14747 139872076707584 Skipping heartbeat: no auto-assignment or waiting on rebalance
Raw
kafka.log
[2016-03-10 14:33:44,127] INFO [Group Metadata Manager on Broker 1]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2016-03-10 14:35:20,185] INFO [GroupCoordinator 1]: Preparing to restabilize group GROUP-NAME with old generation 1 (kafka.coordinator.GroupCoordinator)
[2016-03-10 14:35:20,186] INFO [GroupCoordinator 1]: Group GROUP-NAME generation 1 is dead and removed (kafka.coordinator.GroupCoordinator)
[2016-03-10 14:35:20,693] INFO [GroupCoordinator 1]: Preparing to restabilize group GROUP-NAME with old generation 0 (kafka.coordinator.GroupCoordinator)
[2016-03-10 14:35:20,693] INFO [GroupCoordinator 1]: Stabilized group GROUP-NAME generation 1 (kafka.coordinator.GroupCoordinator)
[2016-03-10 14:35:20,734] INFO [GroupCoordinator 1]: Assignment received from leader for group GROUP-NAME for generation 1 (kafka.coordinator.GroupCoordinator)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment