gistfile1.txt

ment: { id: {0}, group_id: {1}, replicas: {{node_id: {1}, shard: 1}, {node_id: {3}, shard: 3}, {node_id: {0}, shard: 1}} }, previous_assignment: {{ id: {0}, group_id: {1}, replicas: {{node_id: {0}, shard: 1}, {node_id: {2}, shard: 3}} }}} result: Current node is not a leader for partition
TRACE 2021-11-02 06:30:48,441 [shard 3] cluster - controller_backend.cc:519 - executing ntp: {test-namespace/t_4/0} opeartion: {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}}
TRACE 2021-11-02 06:30:48,441 [shard 3] cluster - controller_backend.cc:622 - processing partiton {test-namespace/t_4/0} update command with replicas {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}}
INFO  2021-11-02 06:30:48,441 [shard 3] cluster - controller_backend.cc:437 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} finished
TRACE 2021-11-02 06:30:48,441 [shard 3] cluster - controller_backend.cc:519 - executing ntp: {test-namespace/t_1/0} opeartion: {type: update, ntp: {test-namespace/t_1/0}, offset: {176}, new_assignment: { id: {0}, group_id: {1}, replicas: {{node_id: {1}, shard: 1}, {node_id: {3}, shard: 3}, {node_id: {0}, shard: 1}} }, previous_assignment: {{ id: {0}, group_id: {1}, replicas: {{node_id: {0}, shard: 1}, {node_id: {2}, shard: 3}} }}}
TRACE 2021-11-02 06:30:48,441 [shard 3] cluster - controller_backend.cc:622 - processing partiton {test-namespace/t_1/0} update command with replicas {{node_id: {1}, shard: 1}, {node_id: {3}, shard: 3}, {node_id: {0}, shard: 1}}
TRACE 2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:75 - checking if partition {test-namespace/t_1/0} configuration {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {1}, revision: {176}}, {id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}} is up to date with {{node_id: {1}, shard: 1}, {node_id: {3}, shard: 3}, {node_id: {0}, shard: 1}}
TRACE 2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:143 - requested replica set {{node_id: {1}, shard: 1}, {node_id: {3}, shard: 3}, {node_id: {0}, shard: 1}} differs from partition replica set: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}
TRACE 2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:519 - executing ntp: {test-namespace/t_4/0} opeartion: {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}}
TRACE 2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:622 - processing partiton {test-namespace/t_4/0} update command with replicas {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}}
TRACE 2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:75 - checking if partition {test-namespace/t_4/0} configuration {current: {voters: {{id: {0}, revision: {124}}}, learners: {{id: {3}, revision: {177}}, {id: {4}, revision: {177}}}}, old:{{voters: {{id: {0}, revision: {124}}, {id: {2}, revision: {167}}}, learners: {}}}, revision: {177}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}} is up to date with {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}}
TRACE 2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:143 - requested replica set {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} differs from partition replica set: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}
INFO  2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:433 - partition operation {type: update, ntp: {test-namespace/t_1/0}, offset: {176}, new_assignment: { id: {0}, group_id: {1}, replicas: {{node_id: {1}, shard: 1}, {node_id: {3}, shard: 3}, {node_id: {0}, shard: 1}} }, previous_assignment: {{ id: {0}, group_id: {1}, replicas: {{node_id: {0}, shard: 1}, {node_id: {2}, shard: 3}} }}} result: Current node is not a leader for partition
INFO  2021-11-02 06:30:48,442 [shard 3] cluster - controller_backend.cc:433 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} result: Current node is not a leader for partition
INFO  2021-11-02 06:30:48,441 [shard 0] cluster - controller_backend.cc:437 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} finished
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:519 - executing ntp: {test-namespace/t_4/0} opeartion: {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}}
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:622 - processing partiton {test-namespace/t_4/0} update command with replicas {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}}
INFO  2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:437 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} finished
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:519 - executing ntp: {test-namespace/t_2/0} opeartion: {type: update, ntp: {test-namespace/t_2/0}, offset: {166}, new_assignment: { id: {0}, group_id: {2}, replicas: {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }, previous_assignment: {{ id: {0}, group_id: {2}, replicas: {{node_id: {1}, shard: 3}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }}}
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:622 - processing partiton {test-namespace/t_2/0} update command with replicas {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}}
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:75 - checking if partition {test-namespace/t_2/0} configuration {current: {voters: {{id: {1}, revision: {134}}, {id: {3}, revision: {134}}, {id: {2}, revision: {134}}}, learners: {}}, old:{nullopt}, revision: {134}, brokers: {{id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}} is up to date with {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}}
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:88 - partition {test-namespace/t_2/0} configuration revision '{134}' is smaller than requested update revision '{166}'
INFO  2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:433 - partition operation {type: update, ntp: {test-namespace/t_2/0}, offset: {166}, new_assignment: { id: {0}, group_id: {2}, replicas: {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }, previous_assignment: {{ id: {0}, group_id: {2}, replicas: {{node_id: {1}, shard: 3}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }}} result: Current node is not a leader for partition
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:519 - executing ntp: {test-namespace/t_4/0} opeartion: {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}}
TRACE 2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:622 - processing partiton {test-namespace/t_4/0} update command with replicas {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}}
INFO  2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:437 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} finished
INFO  2021-11-02 06:30:48,442 [shard 0] cluster - controller_backend.cc:437 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} finished
TRACE 2021-11-02 06:30:48,442 [shard 0] storage - readers_cache.cc:301 - {redpanda/controller/0} - removing reader: [{0},{177}] lower_bound: {178}
TRACE 2021-11-02 06:30:48,442 [shard 0] storage - readers_cache.cc:301 - {redpanda/controller/0} - removing reader: [{0},{177}] lower_bound: {178}
TRACE 2021-11-02 06:30:48,442 [shard 0] storage - readers_cache.cc:301 - {redpanda/controller/0} - removing reader: [{0},{177}] lower_bound: {178}
TRACE 2021-11-02 06:30:48,442 [shard 0] storage - readers_cache.cc:301 - {redpanda/controller/0} - removing reader: [{0},{177}] lower_bound: {178}
TRACE 2021-11-02 06:30:48,442 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:242 - Append entries response: {node_id: {id: {2}, revision: {0}}, target_node_id{id: {0}, revision: {0}}, group: {0}, term:{1}, last_dirty_log_index:{177}, last_committed_log_index:{177}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,442 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:319 - Updated node {id: {2}, revision: {0}} last committed log index: {177}
TRACE 2021-11-02 06:30:48,442 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:427 - Updated node {id: {2}, revision: {0}} match {177} and next {178} indices
TRACE 2021-11-02 06:30:48,442 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:242 - Append entries response: {node_id: {id: {3}, revision: {0}}, target_node_id{id: {0}, revision: {0}}, group: {0}, term:{1}, last_dirty_log_index:{177}, last_committed_log_index:{177}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,442 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:319 - Updated node {id: {3}, revision: {0}} last committed log index: {177}
TRACE 2021-11-02 06:30:48,442 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:427 - Updated node {id: {3}, revision: {0}} match {177} and next {178} indices
TRACE 2021-11-02 06:30:48,443 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:242 - Append entries response: {node_id: {id: {4}, revision: {0}}, target_node_id{id: {0}, revision: {0}}, group: {0}, term:{1}, last_dirty_log_index:{177}, last_committed_log_index:{177}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,443 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:319 - Updated node {id: {4}, revision: {0}} last committed log index: {177}
TRACE 2021-11-02 06:30:48,443 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:427 - Updated node {id: {4}, revision: {0}} match {177} and next {178} indices
TRACE 2021-11-02 06:30:48,443 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:242 - Append entries response: {node_id: {id: {1}, revision: {0}}, target_node_id{id: {0}, revision: {0}}, group: {0}, term:{1}, last_dirty_log_index:{177}, last_committed_log_index:{177}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,443 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:319 - Updated node {id: {1}, revision: {0}} last committed log index: {177}
TRACE 2021-11-02 06:30:48,443 [shard 0] raft - [group_id:{0}, {redpanda/controller/0}] consensus.cc:427 - Updated node {id: {1}, revision: {0}} match {177} and next {178} indices
TRACE 2021-11-02 06:30:48,446 [shard 1] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=332, fragments=1}}
TRACE 2021-11-02 06:30:48,446 [shard 3] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=332, fragments=1}}
TRACE 2021-11-02 06:30:48,451 [shard 2] kvstore - kvstore.cc:203 - Apply op: delete: key={bytes:31} not found
TRACE 2021-11-02 06:30:48,451 [shard 1] kvstore - kvstore.cc:203 - Apply op: delete: key={bytes:31} not found
INFO  2021-11-02 06:30:48,452 [shard 2] cluster - partition_manager.cc:77 - Log created manage completed, ntp: {test-namespace/t_4/0}, rev: {177}, 0 segments, 0 bytes
TRACE 2021-11-02 06:30:48,452 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] configuration_manager.cc:41 - Initial configuration: {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}, idx: {0}
TRACE 2021-11-02 06:30:48,452 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}
INFO  2021-11-02 06:30:48,452 [shard 1] cluster - partition_manager.cc:77 - Log created manage completed, ntp: {test-namespace/t_4/0}, rev: {177}, 0 segments, 0 bytes
INFO  2021-11-02 06:30:48,452 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:984 - Starting
INFO  2021-11-02 06:30:48,452 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1016 - Starting with voted_for {id: {-2147483648}, revision: {-9223372036854775808}} term {0} initial_state true
TRACE 2021-11-02 06:30:48,452 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] configuration_manager.cc:41 - Initial configuration: {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}, idx: {0}
TRACE 2021-11-02 06:30:48,452 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}
INFO  2021-11-02 06:30:48,452 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:984 - Starting
INFO  2021-11-02 06:30:48,452 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1016 - Starting with voted_for {id: {-2147483648}, revision: {-9223372036854775808}} term {0} initial_state true
TRACE 2021-11-02 06:30:48,452 [shard 3] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=10370, fragments=6}}
TRACE 2021-11-02 06:30:48,452 [shard 0] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,452 [shard 0] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,452 [shard 0] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,452 [shard 0] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,456 [shard 1] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2311 - Follower commit index updated {0}
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:242 - Append entries response: {node_id: {id: {1}, revision: {176}}, target_node_id{id: {0}, revision: {100}}, group: {1}, term:{4}, last_dirty_log_index:{119}, last_committed_log_index:{0}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:427 - Updated node {id: {1}, revision: {176}} match {119} and next {120} indices
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [follower: {id: {1}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:495 - Recovery status - match idx: {119}, max offset: 164
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [follower: {id: {1}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:227 - Reading batches, starting from: {120}
TRACE 2021-11-02 06:30:48,456 [shard 1] storage - readers_cache.cc:87 - {test-namespace/t_1/0} - trying to get reader for: {start_offset:{120}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,456 [shard 1] storage - readers_cache.cc:120 - {test-namespace/t_1/0} - reader cache hit for: {start_offset:{120}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,456 [shard 1] storage - readers_cache.cc:301 - {test-namespace/t_1/0} - removing reader: [{0},{164}] lower_bound: {165}
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [follower: {id: {1}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:243 - Read batches in range [{120},{164}] for recovery
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:1500 - Append entries request: {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{119}, prev_log_term:{1}}
TRACE 2021-11-02 06:30:48,456 [shard 3] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,456 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2311 - Follower commit index updated {0}
TRACE 2021-11-02 06:30:48,456 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:242 - Append entries response: {node_id: {id: {3}, revision: {176}}, target_node_id{id: {0}, revision: {100}}, group: {1}, term:{4}, last_dirty_log_index:{119}, last_committed_log_index:{0}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,457 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:427 - Updated node {id: {3}, revision: {176}} match {119} and next {120} indices
TRACE 2021-11-02 06:30:48,457 [shard 1] raft - [follower: {id: {3}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:495 - Recovery status - match idx: {119}, max offset: 164
TRACE 2021-11-02 06:30:48,457 [shard 1] raft - [follower: {id: {3}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:227 - Reading batches, starting from: {120}
TRACE 2021-11-02 06:30:48,457 [shard 1] storage - readers_cache.cc:87 - {test-namespace/t_1/0} - trying to get reader for: {start_offset:{120}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,457 [shard 1] storage - readers_cache.cc:120 - {test-namespace/t_1/0} - reader cache hit for: {start_offset:{120}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,457 [shard 1] storage - readers_cache.cc:301 - {test-namespace/t_1/0} - removing reader: [{0},{164}] lower_bound: {165}
TRACE 2021-11-02 06:30:48,457 [shard 1] raft - [follower: {id: {3}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:243 - Read batches in range [{120},{164}] for recovery
TRACE 2021-11-02 06:30:48,457 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:1500 - Append entries request: {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{119}, prev_log_term:{1}}
INFO  2021-11-02 06:30:48,457 [shard 1] storage - segment.cc:628 - Creating new segment cluster_test.D8biUR.1/test-namespace/t_1/0_{176}/137-2-v1.log
INFO  2021-11-02 06:30:48,458 [shard 1] storage - segment.cc:628 - Creating new segment cluster_test.D8biUR.1/test-namespace/t_1/0_{176}/142-3-v1.log
INFO  2021-11-02 06:30:48,458 [shard 3] storage - segment.cc:628 - Creating new segment cluster_test.D8biUR.3/test-namespace/t_1/0_{176}/137-2-v1.log
INFO  2021-11-02 06:30:48,458 [shard 1] storage - segment.cc:628 - Creating new segment cluster_test.D8biUR.1/test-namespace/t_1/0_{176}/148-4-v1.log
INFO  2021-11-02 06:30:48,458 [shard 3] storage - segment.cc:628 - Creating new segment cluster_test.D8biUR.3/test-namespace/t_1/0_{176}/142-3-v1.log
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {1}, revision: {176}}, {id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {133} index {2}: {current: {voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {{id: {3}, revision: {27}}}}, old:{{voters: {{id: {1}, revision: {17}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {27}, brokers: {{id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {134} index {3}: {current: {voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}, old:{{voters: {{id: {1}, revision: {17}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {27}, brokers: {{id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
INFO  2021-11-02 06:30:48,459 [shard 3] storage - segment.cc:628 - Creating new segment cluster_test.D8biUR.3/test-namespace/t_1/0_{176}/148-4-v1.log
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {135} index {4}: {current: {voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}, old:{nullopt}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {136} index {5}: {current: {voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {137} index {6}: {current: {voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {138} index {7}: {current: {voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}, old:{nullopt}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {139} index {8}: {current: {voters: {{id: {2}, revision: {17}}}, learners: {{id: {4}, revision: {72}}}}, old:{{voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {72}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,459 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {140} index {9}: {current: {voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}, old:{{voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {72}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {141} index {10}: {current: {voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}, old:{nullopt}, revision: {72}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {142} index {11}: {current: {voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}, old:{nullopt}, revision: {72}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {143} index {12}: {current: {voters: {}, learners: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}}, old:{{voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}}, revision: {100}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {144} index {13}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {3}, revision: {100}}}}, old:{{voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}}, revision: {100}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {145} index {14}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}}, revision: {100}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {146} index {15}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {100}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {147} index {16}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {100}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {148} index {17}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {100}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,460 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {1}, revision: {176}}, {id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,461 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {149} index {18}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,461 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {150} index {19}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,461 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {133} index {2}: {current: {voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {{id: {3}, revision: {27}}}}, old:{{voters: {{id: {1}, revision: {17}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {27}, brokers: {{id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,461 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {151} index {20}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,461 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {152} index {21}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,461 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {153} index {22}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,462 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {154} index {23}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,461 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {134} index {3}: {current: {voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}, old:{{voters: {{id: {1}, revision: {17}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {27}, brokers: {{id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,462 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {155} index {24}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {2}, revision: {151}}, {id: {4}, revision: {151}}}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,470 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {135} index {4}: {current: {voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}, old:{nullopt}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,470 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {156} index {25}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {{id: {2}, revision: {151}}}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,470 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {157} index {26}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}, {id: {2}, revision: {151}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,470 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {136} index {5}: {current: {voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,470 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {158} index {27}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}, {id: {2}, revision: {151}}}, learners: {}}, old:{nullopt}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,462 [shard 2] kvstore - kvstore.cc:203 - Apply op: delete: key={bytes:10} not found
TRACE 2021-11-02 06:30:48,470 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {137} index {6}: {current: {voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {17}}, {id: {2}, revision: {17}}, {id: {3}, revision: {27}}}, learners: {}}}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {159} index {28}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}, {id: {2}, revision: {151}}}, learners: {}}}, revision: {156}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {138} index {7}: {current: {voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}, old:{nullopt}, revision: {27}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {160} index {29}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}, old:{nullopt}, revision: {156}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {139} index {8}: {current: {voters: {{id: {2}, revision: {17}}}, learners: {{id: {4}, revision: {72}}}}, old:{{voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {72}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {161} index {30}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {2}, revision: {159}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}}, revision: {159}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {140} index {9}: {current: {voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}, old:{{voters: {{id: {3}, revision: {27}}, {id: {0}, revision: {17}}, {id: {2}, revision: {17}}}, learners: {}}}, revision: {72}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {162} index {31}: {current: {voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}}, revision: {159}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {141} index {10}: {current: {voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}, old:{nullopt}, revision: {72}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {163} index {32}: {current: {voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}, old:{nullopt}, revision: {159}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {142} index {11}: {current: {voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}, old:{nullopt}, revision: {72}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,481 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {164} index {33}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {1}, revision: {176}}, {id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,491 [shard 2] kvstore - kvstore.cc:203 - Apply op: delete: key={bytes:10} not found
TRACE 2021-11-02 06:30:48,493 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1023 - Configuration manager started: {configurations: { offset: {-9223372036854775808}, idx: {0},cfg: {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}} }
 }
TRACE 2021-11-02 06:30:48,493 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {143} index {12}: {current: {voters: {}, learners: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}}, old:{{voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}}, revision: {100}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
DEBUG 2021-11-02 06:30:48,493 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1049 - Starting raft bootstrap from {-9223372036854775808}
TRACE 2021-11-02 06:30:48,493 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {144} index {13}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {3}, revision: {100}}}}, old:{{voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}}, revision: {100}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
INFO  2021-11-02 06:30:48,493 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1059 - Read bootstrap state: data_seen 0 config_seen 0 eol false commit {0} term {0} prev_idx {0} prev_term {0} config_tracker {-9223372036854775808} commit_base_tracker {-9223372036854775808} configurations {}
INFO  2021-11-02 06:30:48,493 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1060 - Current log offsets: {start_offset:{-9223372036854775808}, committed_offset:{-9223372036854775808}, committed_offset_term:{-9223372036854775808}, dirty_offset:{-9223372036854775808}, dirty_offset_term:{-9223372036854775808}, last_term_start_offset:{-9223372036854775808}}
INFO  2021-11-02 06:30:48,493 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1078 - Truncating configurations at {-9223372036854775808}
TRACE 2021-11-02 06:30:48,493 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] configuration_manager.cc:45 - Truncating configurations at {0}
TRACE 2021-11-02 06:30:48,493 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {145} index {14}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {2}, revision: {17}}, {id: {4}, revision: {72}}}, learners: {}}}, revision: {100}, brokers: {{id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,493 [shard 1] kvstore - kvstore.cc:203 - Apply op: delete: key={bytes:10} not found
TRACE 2021-11-02 06:30:48,493 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {146} index {15}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {100}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,499 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {147} index {16}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {100}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,499 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {148} index {17}: {current: {voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {100}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,499 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {149} index {18}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {3}, revision: {100}}}, learners: {}}}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,499 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {150} index {19}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,499 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {151} index {20}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,499 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {152} index {21}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,500 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {153} index {22}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,500 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {154} index {23}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {}}, old:{nullopt}, revision: {121}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,500 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {155} index {24}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {2}, revision: {151}}, {id: {4}, revision: {151}}}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,500 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {156} index {25}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {{id: {2}, revision: {151}}}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,500 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {157} index {26}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}, {id: {2}, revision: {151}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}}, learners: {}}}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,503 [shard 2] kvstore - kvstore.cc:192 - Apply op: insert: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,503 [shard 1] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=9288, fragments=5}}
TRACE 2021-11-02 06:30:48,503 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {158} index {27}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}, {id: {2}, revision: {151}}}, learners: {}}, old:{nullopt}, revision: {151}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,503 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {159} index {28}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}, {id: {2}, revision: {151}}}, learners: {}}}, revision: {156}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,504 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {160} index {29}: {current: {voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}, old:{nullopt}, revision: {156}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,504 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {161} index {30}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {2}, revision: {159}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}}, revision: {159}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,504 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {162} index {31}: {current: {voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {4}, revision: {151}}}, learners: {}}}, revision: {159}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,504 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {163} index {32}: {current: {voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}, old:{nullopt}, revision: {159}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,504 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {164} index {33}: {current: {voters: {{id: {0}, revision: {100}}}, learners: {{id: {1}, revision: {176}}, {id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,509 [shard 1] kvstore - kvstore.cc:203 - Apply op: delete: key={bytes:10} not found
TRACE 2021-11-02 06:30:48,513 [shard 2] storage-gc - disk_log_impl.cc:552 - [{test-namespace/t_4/0}] houskeeping with configuration from manager: {evicition_time:{timestamp: 1635229848513}, max_bytes:18446744073709551615, should_sanitize:false}
TRACE 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1023 - Configuration manager started: {configurations: { offset: {-9223372036854775808}, idx: {0},cfg: {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}} }
 }
TRACE 2021-11-02 06:30:48,518 [shard 2] storage-gc - disk_log_impl.cc:581 - [{test-namespace/t_4/0}] applying 'deletion' log cleanup policy with config: {evicition_time:{timestamp: 1635229848513}, max_bytes:18446744073709551615, should_sanitize:false}
DEBUG 2021-11-02 06:30:48,518 [shard 2] storage-gc - disk_log_impl.cc:300 - [{test-namespace/t_4/0}] time retention timestamp: {timestamp: 1635229848513}, first segment max timestamp: {timestamp: 0}
DEBUG 2021-11-02 06:30:48,518 [shard 2] storage-gc - disk_log_impl.cc:236 - [{test-namespace/t_4/0}] gc[time_based_retention] requested to remove segments up to {-9223372036854775808} offset
TRACE 2021-11-02 06:30:48,518 [shard 2] kvstore - kvstore.cc:192 - Apply op: insert: key={bytes:10} value={{bytes=38, fragments=1}}
DEBUG 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1049 - Starting raft bootstrap from {-9223372036854775808}
TRACE 2021-11-02 06:30:48,518 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}
INFO  2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1059 - Read bootstrap state: data_seen 0 config_seen 0 eol false commit {0} term {0} prev_idx {0} prev_term {0} config_tracker {-9223372036854775808} commit_base_tracker {-9223372036854775808} configurations {}
TRACE 2021-11-02 06:30:48,518 [shard 3] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
INFO  2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1060 - Current log offsets: {start_offset:{-9223372036854775808}, committed_offset:{-9223372036854775808}, committed_offset_term:{-9223372036854775808}, dirty_offset:{-9223372036854775808}, dirty_offset_term:{-9223372036854775808}, last_term_start_offset:{-9223372036854775808}}
INFO  2021-11-02 06:30:48,518 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1175 - started raft, log offsets: {start_offset:{-9223372036854775808}, committed_offset:{-9223372036854775808}, committed_offset_term:{-9223372036854775808}, dirty_offset:{-9223372036854775808}, dirty_offset_term:{-9223372036854775808}, last_term_start_offset:{-9223372036854775808}}, term: {0}, configuration: {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}
INFO  2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1078 - Truncating configurations at {-9223372036854775808}
TRACE 2021-11-02 06:30:48,518 [shard 2] cluster - controller_backend.cc:919 - adding {177} to shard table at {test-namespace/t_4/0}
TRACE 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] configuration_manager.cc:45 - Truncating configurations at {0}
TRACE 2021-11-02 06:30:48,518 [shard 1] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2311 - Follower commit index updated {148}
TRACE 2021-11-02 06:30:48,518 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] replicate_entries_stm.cc:246 - Skipping sending append request to {id: {3}, revision: {134}}
TRACE 2021-11-02 06:30:48,518 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] replicate_entries_stm.cc:246 - Skipping sending append request to {id: {2}, revision: {134}}
INFO  2021-11-02 06:30:48,518 [shard 2] cluster - controller_backend.cc:433 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} result: Waiting for partition to recover
TRACE 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:242 - Append entries response: {node_id: {id: {1}, revision: {176}}, target_node_id{id: {0}, revision: {100}}, group: {1}, term:{4}, last_dirty_log_index:{164}, last_committed_log_index:{164}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:427 - Updated node {id: {1}, revision: {176}} match {164} and next {165} indices
TRACE 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:384 - promoting node {id: {1}, revision: {176}} to voter
DEBUG 2021-11-02 06:30:48,518 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:1958 - Replicating group configuration {current: {voters: {{id: {0}, revision: {100}}, {id: {1}, revision: {176}}}, learners: {{id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,519 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] replicate_entries_stm.cc:177 - Self append entries - {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{164}, prev_log_term:{4}}
TRACE 2021-11-02 06:30:48,520 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2825 - should_reconnect_follower({id: {2}, revision: {167}}): 0/3 fails, last ok 200ms ago
TRACE 2021-11-02 06:30:48,533 [shard 1] raft - [follower: {id: {1}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:495 - Recovery status - match idx: {164}, max offset: 164
TRACE 2021-11-02 06:30:48,524 [shard 0] cluster - controller_backend.cc:519 - executing ntp: {test-namespace/t_2/0} opeartion: {type: update, ntp: {test-namespace/t_2/0}, offset: {166}, new_assignment: { id: {0}, group_id: {2}, replicas: {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }, previous_assignment: {{ id: {0}, group_id: {2}, replicas: {{node_id: {1}, shard: 3}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }}}
TRACE 2021-11-02 06:30:48,533 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2825 - should_reconnect_follower({id: {3}, revision: {177}}): 0/3 fails, last ok 748028620ms ago
TRACE 2021-11-02 06:30:48,533 [shard 1] raft - [follower: {id: {1}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:528 - Finished recovery
TRACE 2021-11-02 06:30:48,533 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2825 - should_reconnect_follower({id: {4}, revision: {177}}): 0/3 fails, last ok 748028620ms ago
TRACE 2021-11-02 06:30:48,533 [shard 0] cluster - controller_backend.cc:622 - processing partiton {test-namespace/t_2/0} update command with replicas {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}}
TRACE 2021-11-02 06:30:48,533 [shard 0] cluster - controller_backend.cc:75 - checking if partition {test-namespace/t_2/0} configuration {current: {voters: {{id: {1}, revision: {134}}, {id: {3}, revision: {134}}, {id: {2}, revision: {134}}}, learners: {}}, old:{nullopt}, revision: {134}, brokers: {{id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}} is up to date with {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}}
TRACE 2021-11-02 06:30:48,533 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {{id: {0}, revision: {100}}, {id: {1}, revision: {176}}}, learners: {{id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,528 [shard 3] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=9288, fragments=5}}
TRACE 2021-11-02 06:30:48,533 [shard 0] cluster - controller_backend.cc:88 - partition {test-namespace/t_2/0} configuration revision '{134}' is smaller than requested update revision '{166}'
TRACE 2021-11-02 06:30:48,533 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {165} index {34}: {current: {voters: {{id: {0}, revision: {100}}, {id: {1}, revision: {176}}}, learners: {{id: {3}, revision: {176}}}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
INFO  2021-11-02 06:30:48,533 [shard 0] cluster - controller_backend.cc:433 - partition operation {type: update, ntp: {test-namespace/t_2/0}, offset: {166}, new_assignment: { id: {0}, group_id: {2}, replicas: {{node_id: {4}, shard: 1}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }, previous_assignment: {{ id: {0}, group_id: {2}, replicas: {{node_id: {1}, shard: 3}, {node_id: {2}, shard: 2}, {node_id: {3}, shard: 0}} }}} result: Current node is not a leader for partition
TRACE 2021-11-02 06:30:48,533 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1500 - Append entries request: {raft_group:{4}, commit_index:{162}, term:{6}, prev_log_index:{162}, prev_log_term:{6}}
TRACE 2021-11-02 06:30:48,533 [shard 1] kvstore - kvstore.cc:192 - Apply op: insert: key={bytes:10} value={{bytes=8, fragments=1}}
DEBUG 2021-11-02 06:30:48,533 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1520 - Append entries request term:{6} is greater than current: {0}. Setting new term
TRACE 2021-11-02 06:30:48,533 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:2825 - should_reconnect_follower({id: {2}, revision: {134}}): 0/3 fails, last ok 748028634ms ago
TRACE 2021-11-02 06:30:48,533 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1500 - Append entries request: {raft_group:{4}, commit_index:{162}, term:{6}, prev_log_index:{162}, prev_log_term:{6}}
TRACE 2021-11-02 06:30:48,533 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:2825 - should_reconnect_follower({id: {3}, revision: {134}}): 0/3 fails, last ok 748028634ms ago
DEBUG 2021-11-02 06:30:48,533 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2330 - triggering leadership notification with term: {6}, new leader: {{id: {0}, revision: {124}}}
TRACE 2021-11-02 06:30:48,533 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:2825 - should_reconnect_follower({id: {4}, revision: {166}}): 0/3 fails, last ok 748028634ms ago
TRACE 2021-11-02 06:30:48,533 [shard 3] rpc - reconnect_transport.cc:70 - connecting to {host: 127.0.0.1, port: 11004}
DEBUG 2021-11-02 06:30:48,533 [shard 3] dns_resolver - Query name 127.0.0.1 (INET)
DEBUG 2021-11-02 06:30:48,534 [shard 3] dns_resolver - Query success: 127.0.0.1/127.0.0.1
TRACE 2021-11-02 06:30:48,534 [shard 3] dns_resolver - Poll sockets
TRACE 2021-11-02 06:30:48,534 [shard 3] dns_resolver - ares_fds: 0
TRACE 2021-11-02 06:30:48,534 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:242 - Append entries response: {node_id: {id: {4}, revision: {177}}, target_node_id{id: {0}, revision: {124}}, group: {4}, term:{6}, last_dirty_log_index:{-9223372036854775808}, last_committed_log_index:{-9223372036854775808}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,534 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:319 - Updated node {id: {4}, revision: {177}} last committed log index: {-9223372036854775808}
TRACE 2021-11-02 06:30:48,534 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:349 - Starting recovery process for {id: {4}, revision: {177}} - current reply: {node_id: {id: {4}, revision: {177}}, target_node_id{id: {0}, revision: {124}}, group: {4}, term:{6}, last_dirty_log_index:{-9223372036854775808}, last_committed_log_index:{-9223372036854775808}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,534 [shard 2] raft - [follower: {id: {4}, revision: {177}}] [group_id:{4}, {test-namespace/t_4/0}] - recovery_stm.cc:227 - Reading batches, starting from: {0}
TRACE 2021-11-02 06:30:48,534 [shard 2] storage - readers_cache.cc:87 - {test-namespace/t_4/0} - trying to get reader for: {start_offset:{0}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,534 [shard 2] storage - readers_cache.cc:116 - {test-namespace/t_4/0} - reader cache miss for: {start_offset:{0}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,534 [shard 2] storage - readers_cache.cc:74 - {test-namespace/t_4/0} - adding reader [{0},{162}]
TRACE 2021-11-02 06:30:48,534 [shard 3] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1500 - Append entries request: {raft_group:{4}, commit_index:{162}, term:{6}, prev_log_index:{162}, prev_log_term:{6}}
TRACE 2021-11-02 06:30:48,534 [shard 3] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2311 - Follower commit index updated {162}
DEBUG 2021-11-02 06:30:48,534 [shard 3] rpc - connected to {host: 127.0.0.1, port: 11004}
TRACE 2021-11-02 06:30:48,534 [shard 0] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:1500 - Append entries request: {raft_group:{2}, commit_index:{216}, term:{6}, prev_log_index:{219}, prev_log_term:{6}}
TRACE 2021-11-02 06:30:48,534 [shard 3] rpc - server.cc:147 - vectorized internal rpc protocol - Incoming connection from 127.0.0.1:53439 on ""
DEBUG 2021-11-02 06:30:48,534 [shard 0] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:2330 - triggering leadership notification with term: {6}, new leader: {{id: {1}, revision: {134}}}
TRACE 2021-11-02 06:30:48,534 [shard 2] raft - [follower: {id: {4}, revision: {177}}] [group_id:{4}, {test-namespace/t_4/0}] - recovery_stm.cc:243 - Read batches in range [{0},{112}] for recovery
TRACE 2021-11-02 06:30:48,534 [shard 1] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:1500 - Append entries request: {raft_group:{2}, commit_index:{216}, term:{6}, prev_log_index:{219}, prev_log_term:{6}}
TRACE 2021-11-02 06:30:48,543 [shard 3] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
DEBUG 2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:1520 - Append entries request term:{6} is greater than current: {0}. Setting new term
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:1500 - Append entries request: {raft_group:{2}, commit_index:{216}, term:{6}, prev_log_index:{219}, prev_log_term:{6}}
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2311 - Follower commit index updated {148}
DEBUG 2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:2330 - triggering leadership notification with term: {6}, new leader: {{id: {1}, revision: {134}}}
TRACE 2021-11-02 06:30:48,544 [shard 1] kvstore - kvstore.cc:192 - Apply op: insert: key={bytes:10} value={{bytes=38, fragments=1}}
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:242 - Append entries response: {node_id: {id: {3}, revision: {134}}, target_node_id{id: {1}, revision: {134}}, group: {2}, term:{6}, last_dirty_log_index:{217}, last_committed_log_index:{217}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,544 [shard 1] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=9690, fragments=5}}
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:319 - Updated node {id: {3}, revision: {134}} last committed log index: {217}
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}
TRACE 2021-11-02 06:30:48,544 [shard 2] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:1500 - Append entries request: {raft_group:{2}, commit_index:{216}, term:{6}, prev_log_index:{219}, prev_log_term:{6}}
DEBUG 2021-11-02 06:30:48,544 [shard 2] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:2330 - triggering leadership notification with term: {6}, new leader: {{id: {1}, revision: {134}}}
INFO  2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1175 - started raft, log offsets: {start_offset:{-9223372036854775808}, committed_offset:{-9223372036854775808}, committed_offset_term:{-9223372036854775808}, dirty_offset:{-9223372036854775808}, dirty_offset_term:{-9223372036854775808}, last_term_start_offset:{-9223372036854775808}}, term: {0}, configuration: {current: {voters: {}, learners: {}}, old:{nullopt}, revision: {177}, brokers: {}}
TRACE 2021-11-02 06:30:48,544 [shard 1] cluster - controller_backend.cc:919 - adding {177} to shard table at {test-namespace/t_4/0}
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:349 - Starting recovery process for {id: {3}, revision: {134}} - current reply: {node_id: {id: {3}, revision: {134}}, target_node_id{id: {1}, revision: {134}}, group: {2}, term:{6}, last_dirty_log_index:{217}, last_committed_log_index:{217}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [follower: {id: {3}, revision: {134}}] [group_id:{2}, {test-namespace/t_2/0}] - recovery_stm.cc:227 - Reading batches, starting from: {218}
TRACE 2021-11-02 06:30:48,544 [shard 3] storage - readers_cache.cc:87 - {test-namespace/t_2/0} - trying to get reader for: {start_offset:{218}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,544 [shard 3] storage - readers_cache.cc:116 - {test-namespace/t_2/0} - reader cache miss for: {start_offset:{218}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:242 - Append entries response: {node_id: {id: {3}, revision: {176}}, target_node_id{id: {0}, revision: {100}}, group: {1}, term:{4}, last_dirty_log_index:{164}, last_committed_log_index:{164}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,544 [shard 3] storage - readers_cache.cc:74 - {test-namespace/t_2/0} - adding reader [{218},{219}]
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:427 - Updated node {id: {3}, revision: {176}} match {164} and next {165} indices
TRACE 2021-11-02 06:30:48,544 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:1500 - Append entries request: {raft_group:{4}, commit_index:{112}, term:{6}, prev_log_index:{-9223372036854775808}, prev_log_term:{-9223372036854775808}}
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:384 - promoting node {id: {3}, revision: {176}} to voter
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [follower: {id: {3}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:495 - Recovery status - match idx: {164}, max offset: 165
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [follower: {id: {3}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:227 - Reading batches, starting from: {165}
TRACE 2021-11-02 06:30:48,544 [shard 3] storage - readers_cache.cc:301 - {test-namespace/t_2/0} - removing reader: [{218},{219}] lower_bound: {220}
TRACE 2021-11-02 06:30:48,544 [shard 1] storage - readers_cache.cc:87 - {test-namespace/t_1/0} - trying to get reader for: {start_offset:{165}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
INFO  2021-11-02 06:30:48,544 [shard 2] storage - segment.cc:628 - Creating new segment cluster_test.D8biUR.4/test-namespace/t_4/0_{177}/0-1-v1.log
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [follower: {id: {3}, revision: {134}}] [group_id:{2}, {test-namespace/t_2/0}] - recovery_stm.cc:243 - Read batches in range [{218},{219}] for recovery
TRACE 2021-11-02 06:30:48,544 [shard 1] storage - readers_cache.cc:116 - {test-namespace/t_1/0} - reader cache miss for: {start_offset:{165}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,544 [shard 1] storage - readers_cache.cc:74 - {test-namespace/t_1/0} - adding reader [{148},{165}]
TRACE 2021-11-02 06:30:48,544 [shard 1] storage - readers_cache.cc:301 - {test-namespace/t_1/0} - removing reader: [{148},{165}] lower_bound: {166}
TRACE 2021-11-02 06:30:48,544 [shard 1] raft - [follower: {id: {3}, revision: {176}}] [group_id:{1}, {test-namespace/t_1/0}] - recovery_stm.cc:243 - Read batches in range [{165},{165}] for recovery
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:242 - Append entries response: {node_id: {id: {4}, revision: {166}}, target_node_id{id: {1}, revision: {134}}, group: {2}, term:{6}, last_dirty_log_index:{-9223372036854775808}, last_committed_log_index:{-9223372036854775808}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,544 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:319 - Updated node {id: {4}, revision: {166}} last committed log index: {-9223372036854775808}
TRACE 2021-11-02 06:30:48,545 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:242 - Append entries response: {node_id: {id: {2}, revision: {167}}, target_node_id{id: {0}, revision: {124}}, group: {4}, term:{6}, last_dirty_log_index:{162}, last_committed_log_index:{162}, last_term_base_offset:{-9223372036854775808}, result: success}
TRACE 2021-11-02 06:30:48,558 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:349 - Starting recovery process for {id: {4}, revision: {166}} - current reply: {node_id: {id: {4}, revision: {166}}, target_node_id{id: {1}, revision: {134}}, group: {2}, term:{6}, last_dirty_log_index:{-9223372036854775808}, last_committed_log_index:{-9223372036854775808}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,558 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:319 - Updated node {id: {2}, revision: {167}} last committed log index: {162}
TRACE 2021-11-02 06:30:48,558 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:427 - Updated node {id: {2}, revision: {167}} match {162} and next {163} indices
TRACE 2021-11-02 06:30:48,554 [shard 1] kvstore - kvstore.cc:192 - Apply op: update: key={bytes:10} value={{bytes=8, fragments=1}}
TRACE 2021-11-02 06:30:48,558 [shard 3] raft - [follower: {id: {4}, revision: {166}}] [group_id:{2}, {test-namespace/t_2/0}] - recovery_stm.cc:227 - Reading batches, starting from: {0}
TRACE 2021-11-02 06:30:48,558 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] replicate_entries_stm.cc:87 - Sending append entries request {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{164}, prev_log_term:{4}} to {id: {2}, revision: {159}}
TRACE 2021-11-02 06:30:48,558 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] replicate_entries_stm.cc:246 - Skipping sending append request to {id: {3}, revision: {176}}
TRACE 2021-11-02 06:30:48,558 [shard 3] storage - readers_cache.cc:87 - {test-namespace/t_2/0} - trying to get reader for: {start_offset:{0}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,559 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] replicate_entries_stm.cc:87 - Sending append entries request {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{164}, prev_log_term:{4}} to {id: {1}, revision: {176}}
TRACE 2021-11-02 06:30:48,559 [shard 3] storage - readers_cache.cc:116 - {test-namespace/t_2/0} - reader cache miss for: {start_offset:{0}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,559 [shard 3] storage - readers_cache.cc:74 - {test-namespace/t_2/0} - adding reader [{0},{219}]
DEBUG 2021-11-02 06:30:48,559 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:1958 - Replicating group configuration {current: {voters: {{id: {0}, revision: {100}}, {id: {1}, revision: {176}}, {id: {3}, revision: {176}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,559 [shard 3] raft - [follower: {id: {4}, revision: {166}}] [group_id:{2}, {test-namespace/t_2/0}] - recovery_stm.cc:243 - Read batches in range [{0},{110}] for recovery
TRACE 2021-11-02 06:30:48,559 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] replicate_entries_stm.cc:177 - Self append entries - {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{165}, prev_log_term:{4}}
TRACE 2021-11-02 06:30:48,559 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {{id: {0}, revision: {20}}, {id: {1}, revision: {20}}, {id: {4}, revision: {20}}}, learners: {}}, old:{nullopt}, revision: {20}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,559 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:2320 - Updating follower stats with config {current: {voters: {{id: {0}, revision: {100}}, {id: {1}, revision: {176}}, {id: {3}, revision: {176}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,559 [shard 2] raft - [group_id:{4}, {test-namespace/t_4/0}] configuration_manager.cc:152 - Adding configuration at offset {0} index {1}: {current: {voters: {{id: {0}, revision: {20}}, {id: {1}, revision: {20}}, {id: {4}, revision: {20}}}, learners: {}}, old:{nullopt}, revision: {20}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {4}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9096}}}, rpc_address: {host: 127.0.0.1, port: 11004}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
TRACE 2021-11-02 06:30:48,559 [shard 3] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:1500 - Append entries request: {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{164}, prev_log_term:{4}}
TRACE 2021-11-02 06:30:48,559 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] configuration_manager.cc:152 - Adding configuration at offset {166} index {35}: {current: {voters: {{id: {0}, revision: {100}}, {id: {1}, revision: {176}}, {id: {3}, revision: {176}}}, learners: {}}, old:{{voters: {{id: {0}, revision: {100}}, {id: {2}, revision: {159}}}, learners: {}}}, revision: {176}, brokers: {{id: {0}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9092}}}, rpc_address: {host: 127.0.0.1, port: 11000}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {2}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9094}}}, rpc_address: {host: 127.0.0.1, port: 11002}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {1}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9093}}}, rpc_address: {host: 127.0.0.1, port: 11001}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}, {id: {3}, kafka_advertised_listeners: {{:{host: 127.0.0.1, port: 9095}}}, rpc_address: {host: 127.0.0.1, port: 11003}, rack: {i-am-rack}, properties: {cores 4, mem_available 0, disk_available 0}, membership_state: active}}}
INFO  2021-11-02 06:30:48,573 [shard 1] cluster - controller_backend.cc:433 - partition operation {type: update, ntp: {test-namespace/t_4/0}, offset: {177}, new_assignment: { id: {0}, group_id: {4}, replicas: {{node_id: {3}, shard: 1}, {node_id: {0}, shard: 2}, {node_id: {4}, shard: 2}} }, previous_assignment: {{ id: {0}, group_id: {4}, replicas: {{node_id: {2}, shard: 3}, {node_id: {0}, shard: 2}} }}} result: Waiting for partition to recover
TRACE 2021-11-02 06:30:48,573 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:242 - Append entries response: {node_id: {id: {2}, revision: {134}}, target_node_id{id: {1}, revision: {134}}, group: {2}, term:{6}, last_dirty_log_index:{217}, last_committed_log_index:{217}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,573 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:319 - Updated node {id: {2}, revision: {134}} last committed log index: {217}
TRACE 2021-11-02 06:30:48,573 [shard 3] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:349 - Starting recovery process for {id: {2}, revision: {134}} - current reply: {node_id: {id: {2}, revision: {134}}, target_node_id{id: {1}, revision: {134}}, group: {2}, term:{6}, last_dirty_log_index:{217}, last_committed_log_index:{217}, last_term_base_offset:{-9223372036854775808}, result: failure}
TRACE 2021-11-02 06:30:48,573 [shard 3] raft - [follower: {id: {2}, revision: {134}}] [group_id:{2}, {test-namespace/t_2/0}] - recovery_stm.cc:227 - Reading batches, starting from: {218}
TRACE 2021-11-02 06:30:48,573 [shard 3] storage - readers_cache.cc:87 - {test-namespace/t_2/0} - trying to get reader for: {start_offset:{218}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,573 [shard 3] storage - readers_cache.cc:116 - {test-namespace/t_2/0} - reader cache miss for: {start_offset:{218}, max_offset:{9223372036854775807}, min_bytes:1, max_bytes:32768, type_filter:nullopt, first_timestamp:nullopt}
TRACE 2021-11-02 06:30:48,573 [shard 3] storage - readers_cache.cc:74 - {test-namespace/t_2/0} - adding reader [{218},{219}]
ERROR 2021-11-02 06:30:48,573 [shard 2] assert - Assert failure: (../src/v/storage/segment_appender.cc:495) 'false' Could not dma_write: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > - {no_of_chunks:64, closed:0, fallocation_offset:33554432, committed_offset:32768, bytes_flush_pending:183}
TRACE 2021-11-02 06:30:48,559 [shard 0] raft - [group_id:{2}, {test-namespace/t_2/0}] consensus.cc:1500 - Append entries request: {raft_group:{2}, commit_index:{216}, term:{6}, prev_log_index:{217}, prev_log_term:{5}}
TRACE 2021-11-02 06:30:48,573 [shard 1] raft - [group_id:{1}, {test-namespace/t_1/0}] consensus.cc:1500 - Append entries request: {raft_group:{1}, commit_index:{164}, term:{4}, prev_log_index:{164}, prev_log_term:{4}}
TRACE 2021-11-02 06:30:48,573 [shard 3] storage - readers_cache.cc:301 - {test-namespace/t_2/0} - removing reader: [{218},{219}] lower_bound: {220}
ERROR 2021-11-02 06:30:48,573 [shard 2] assert - Backtrace below:
0x1c741de 0x1c746f0 0x1c74b38 0x19cba31 0x19cb966 0x19cb945 0x19cbbe6 0x1b3b345 0x1b3c7e7 0x1b5c0cb 0x1b1086a /lib64/libpthread.so.0+0x9298 /lib64/libc.so.6+0x100352
   --------
   seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void>::handle_exception<storage::segment_appender::dispatch_background_head_write()::$_21>(storage::segment_appender::dispatch_background_head_write()::$_21&&)::{lambda(auto:1&&)#1}, seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void>::handle_exception<storage::segment_appender::dispatch_background_head_write()::$_21>(storage::segment_appender::dispatch_background_head_write()::$_21&&)::{lambda(auto:1&&)#1}>(seastar::future<void>::handle_exception<storage::segment_appender::dispatch_background_head_write()::$_21>(storage::segment_appender::dispatch_background_head_write()::$_21&&)::{lambda(auto:1&&)#1}&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::future<void>::handle_exception<storage::segment_appender::dispatch_background_head_write()::$_21>(auto:1&&)::{lambda(auto:1&&)#1}&, seastar::future_state<seastar::internal::monostate>&&)#1}, void>
Test Exit code 132
Could not find vbuild in binary path /mnt/redpanda/build/bin/partition_moving_test_rpunit
Could not decode backtrace, seastar-addr2lines not found

      Start 39: leader_balancer_test_rpunit
33/58 Test #39: leader_balancer_test_rpunit .......................   Passed    0.10 sec
      Start 40: test_kafka_protocol_rpunit
34/58 Test #40: test_kafka_protocol_rpunit ........................   Passed    0.10 sec
      Start 41: test_kafka_client_unit_rpunit
35/58 Test #41: test_kafka_client_unit_rpunit .....................   Passed    0.09 sec
      Start 42: test_kafka_client_rpunit
36/58 Test #42: test_kafka_client_rpunit ..........................   Passed    0.09 sec
      Start 43: test_kafka_client_fixture_rpunit
37/58 Test #43: test_kafka_client_fixture_rpunit ..................   Passed    6.58 sec
      Start 44: test_kafka_server_rpunit
38/58 Test #44: test_kafka_server_rpunit ..........................   Passed    0.09 sec
      Start 45: test_kafka_server_fixture_rpunit
39/58 Test #45: test_kafka_server_fixture_rpunit ..................   Passed   12.55 sec
      Start 46: test_kafka_request_parser_rpunit
40/58 Test #46: test_kafka_request_parser_rpunit ..................   Passed    2.91 sec
      Start 47: arity_test_rpunit
41/58 Test #47: arity_test_rpunit .................................   Passed    0.10 sec
      Start 48: reflection_async_adl_test_rpunit
42/58 Test #48: reflection_async_adl_test_rpunit ..................   Passed    0.51 sec
      Start 49: pandaproxy_rest_fixture_rpunit
43/58 Test #49: pandaproxy_rest_fixture_rpunit ....................   Passed    4.13 sec
      Start 50: pandaproxy_schema_registry_unit_rpunit
44/58 Test #50: pandaproxy_schema_registry_unit_rpunit ............   Passed    0.09 sec
      Start 51: pandaproxy_schema_registry_single_thread_rpunit
45/58 Test #51: pandaproxy_schema_registry_single_thread_rpunit ...   Passed    0.37 sec
      Start 52: pandaproxy_schema_registry_fixture_rpunit
46/58 Test #52: pandaproxy_schema_registry_fixture_rpunit .........   Passed    6.47 sec
      Start 53: pandaproxy_schema_registry_requests_unit_rpunit
47/58 Test #53: pandaproxy_schema_registry_requests_unit_rpunit ...   Passed    0.51 sec
      Start 54: pandaproxy_json_types_rpunit
48/58 Test #54: pandaproxy_json_types_rpunit ......................   Passed    0.48 sec
      Start 55: pandaproxy_json_requests_rpunit
49/58 Test #55: pandaproxy_json_requests_rpunit ...................   Passed    0.52 sec
      Start 56: pandaproxy_parsing_unit_rpunit
50/58 Test #56: pandaproxy_parsing_unit_rpunit ....................   Passed    0.10 sec
      Start 57: test_http_client_rpunit
51/58 Test #57: test_http_client_rpunit ...........................   Passed    1.14 sec
      Start 58: s3_single_thread_rpunit
52/58 Test #58: s3_single_thread_rpunit ...........................   Passed    1.45 sec
      Start 59: test_archival_service_rpunit
53/58 Test #59: test_archival_service_rpunit ......................   Passed    2.83 sec
      Start 60: test_kafka_security_rpunit
54/58 Test #60: test_kafka_security_rpunit ........................   Passed    0.10 sec
      Start 61: serde_rpunit
55/58 Test #61: serde_rpunit ......................................   Passed    0.57 sec
      Start 63: test_cloud_storage_rpunit
56/58 Test #63: test_cloud_storage_rpunit .........................   Passed   10.39 sec
      Start 64: v8_script_rpunit
57/58 Test #64: v8_script_rpunit ..................................   Passed    0.97 sec
      Start 65: v8_executor_rpunit
58/58 Test #65: v8_executor_rpunit ................................   Passed   41.44 sec

95% tests passed, 3 tests failed out of 58

Label Time Summary:
archival          =   2.83 sec*proc (1 test)
bytes             =   0.66 sec*proc (1 test)
cloud_storage     =  10.39 sec*proc (1 test)
cluster           = 275.56 sec*proc (4 tests)
compression       =   0.67 sec*proc (1 test)
config            =   0.54 sec*proc (1 test)
coproc            =  33.84 sec*proc (1 test)
disable_on_ci     =  36.67 sec*proc (2 tests)
hashing           =   0.20 sec*proc (2 tests)
http              =   1.14 sec*proc (1 test)
json              =   0.50 sec*proc (1 test)
kafka             =  27.53 sec*proc (9 tests)
kafka_protocol    =   0.10 sec*proc (1 test)
model             =   0.43 sec*proc (2 tests)
pandaproxy        =  12.69 sec*proc (8 tests)
raft              =  54.42 sec*proc (3 tests)
random            =   0.10 sec*proc (1 test)
reflection        =   0.61 sec*proc (2 tests)
rpc               =   3.13 sec*proc (3 tests)
s3                =   1.45 sec*proc (1 test)
serde             =   0.57 sec*proc (1 test)
ssx               =   0.69 sec*proc (1 test)
storage           =  34.37 sec*proc (6 tests)
utils             =  14.17 sec*proc (3 tests)
v8_engine         =  42.40 sec*proc (2 tests)

Total Test time (real) = 525.48 sec

The following tests FAILED:
	 22 - coproc_fixture_rpunit (Failed)
	 24 - storage_single_thread_rpunit (Failed)
	 38 - partition_moving_test_rpunit (Failed)
Errors while running CTest
[root@3e751126da50 redpanda]# client_loop: send disconnect: Broken pipe
johndavis@Johns-MacBook-Air ~ %

ERROR 2021-11-02 06:30:48,573 [shard 2] assert - Assert failure: (../src/v/storage/segment_appender.cc:495) 'false' Could not dma_write: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > - {no_of_chunks:64, closed:0, fallocation_offset:33554432, committed_offset:32768, bytes_flush_pending:183}

looks like either flush/write is failing. are you low on disk space, perhaps?