Skip to content

Instantly share code, notes, and snippets.

@im-konge
Created April 13, 2023 20:56
Show Gist options
  • Save im-konge/82db06d04cf83f4db2f3aaf657fdfe41 to your computer and use it in GitHub Desktop.
Save im-konge/82db06d04cf83f4db2f3aaf657fdfe41 to your computer and use it in GitHub Desktop.
STRIMZI_BROKER_ID=2
Preparing truststore for replication listener
Adding /opt/kafka/cluster-ca-certs/ca.crt to truststore /tmp/kafka/cluster.truststore.p12 with alias ca
Certificate was added to keystore
Preparing truststore for replication listener is complete
Looking for the right CA
Found the right CA: /opt/kafka/cluster-ca-certs/ca.crt
Preparing keystore for replication and clienttls listener
Preparing keystore for replication and clienttls listener is complete
Preparing truststore for client authentication
Adding /opt/kafka/client-ca-certs/ca.crt to truststore /tmp/kafka/clients.truststore.p12 with alias ca
Certificate was added to keystore
Preparing truststore for client authentication is complete
Starting Kafka with configuration:
##############################
##############################
# This file is automatically generated by the Strimzi Cluster Operator
# Any changes to this file will be ignored and overwritten!
##############################
##############################
##########
# Broker ID
##########
broker.id=2
node.id=2
##########
# KRaft configuration
##########
process.roles=broker,controller
controller.listener.names=CONTROLPLANE-9090
controller.quorum.voters=0@my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090,1@my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090,2@my-cluster-1794df46-kafka-2.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090
##########
# Kafka message logs configuration
##########
log.dirs=/var/lib/kafka/data/kafka-log2
##########
# Control Plane listener
##########
listener.name.controlplane-9090.ssl.keystore.location=/tmp/kafka/cluster.keystore.p12
listener.name.controlplane-9090.ssl.keystore.password=[hidden]
listener.name.controlplane-9090.ssl.keystore.type=PKCS12
listener.name.controlplane-9090.ssl.truststore.location=/tmp/kafka/cluster.truststore.p12
listener.name.controlplane-9090.ssl.truststore.password=[hidden]
listener.name.controlplane-9090.ssl.truststore.type=PKCS12
listener.name.controlplane-9090.ssl.client.auth=required
##########
# Replication listener
##########
listener.name.replication-9091.ssl.keystore.location=/tmp/kafka/cluster.keystore.p12
listener.name.replication-9091.ssl.keystore.password=[hidden]
listener.name.replication-9091.ssl.keystore.type=PKCS12
listener.name.replication-9091.ssl.truststore.location=/tmp/kafka/cluster.truststore.p12
listener.name.replication-9091.ssl.truststore.password=[hidden]
listener.name.replication-9091.ssl.truststore.type=PKCS12
listener.name.replication-9091.ssl.client.auth=required
##########
# Listener configuration: PLAIN-9092
##########
##########
# Listener configuration: TLS-9093
##########
listener.name.tls-9093.ssl.keystore.location=/tmp/kafka/cluster.keystore.p12
listener.name.tls-9093.ssl.keystore.password=[hidden]
listener.name.tls-9093.ssl.keystore.type=PKCS12
##########
# Common listener configuration
##########
listeners=CONTROLPLANE-9090://0.0.0.0:9090,REPLICATION-9091://0.0.0.0:9091,PLAIN-9092://0.0.0.0:9092,TLS-9093://0.0.0.0:9093
advertised.listeners=REPLICATION-9091://my-cluster-1794df46-kafka-2.my-cluster-1794df46-kafka-brokers.namespace-0.svc:9091,PLAIN-9092://my-cluster-1794df46-kafka-2.my-cluster-1794df46-kafka-brokers.namespace-0.svc:9092,TLS-9093://my-cluster-1794df46-kafka-2.my-cluster-1794df46-kafka-brokers.namespace-0.svc:9093
listener.security.protocol.map=CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,PLAIN-9092:PLAINTEXT,TLS-9093:SSL
inter.broker.listener.name=REPLICATION-9091
sasl.enabled.mechanisms=
ssl.endpoint.identification.algorithm=HTTPS
##########
# User provided configuration
##########
default.replication.factor=3
inter.broker.protocol.version=3.3
log.message.format.version=3.3
min.insync.replicas=2
offsets.topic.replication.factor=3
transaction.state.log.min.isr=2
transaction.state.log.replication.factor=3
Kraft storage is already formatted
+ exec /usr/bin/tini -w -e 143 -- /opt/kafka/bin/kafka-server-start.sh /tmp/strimzi.properties
2023-04-13 20:50:17,312 INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) [main]
2023-04-13 20:50:17,562 INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util) [main]
2023-04-13 20:50:17,590 WARN inter.broker.protocol.version is deprecated in KRaft mode as of 3.3 and will only be read when first upgrading from a KRaft prior to 3.3. See kafka-storage.sh help for details on setting the metadata version for a new KRaft cluster. (kafka.server.KafkaConfig) [main]
2023-04-13 20:50:17,758 INFO [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data/kafka-log2] Recovering unflushed segment 0. 0/1 recovered for __cluster_metadata-0. (kafka.log.LogLoader) [main]
2023-04-13 20:50:17,759 INFO [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data/kafka-log2] Loading producer state till offset 0 with message format version 2 (kafka.log.UnifiedLog$) [main]
2023-04-13 20:50:17,759 INFO [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data/kafka-log2] Reloading from producer snapshot and rebuilding producer state from offset 0 (kafka.log.UnifiedLog$) [main]
2023-04-13 20:50:17,761 INFO [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data/kafka-log2] Producer state recovery took 2ms for snapshot load and 0ms for segment recovery from offset 0 (kafka.log.UnifiedLog$) [main]
2023-04-13 20:50:17,774 INFO [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data/kafka-log2] Loading producer state till offset 0 with message format version 2 (kafka.log.UnifiedLog$) [main]
2023-04-13 20:50:17,774 INFO [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data/kafka-log2] Reloading from producer snapshot and rebuilding producer state from offset 0 (kafka.log.UnifiedLog$) [main]
2023-04-13 20:50:17,774 INFO [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data/kafka-log2] Producer state recovery took 0ms for snapshot load and 0ms for segment recovery from offset 0 (kafka.log.UnifiedLog$) [main]
2023-04-13 20:50:17,812 INFO Initialized snapshots with IDs SortedSet() from /var/lib/kafka/data/kafka-log2/__cluster_metadata-0 (kafka.raft.KafkaMetadataLog$) [main]
2023-04-13 20:50:18,024 INFO [raft-expiration-reaper]: Starting (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper) [raft-expiration-reaper]
2023-04-13 20:50:18,185 INFO [RaftManager nodeId=2] Completed transition to CandidateState(localId=2, epoch=25, retries=1, electionTimeoutMs=1691) (org.apache.kafka.raft.QuorumState) [main]
2023-04-13 20:50:18,216 WARN inter.broker.protocol.version is deprecated in KRaft mode as of 3.3 and will only be read when first upgrading from a KRaft prior to 3.3. See kafka-storage.sh help for details on setting the metadata version for a new KRaft cluster. (kafka.server.KafkaConfig) [main]
2023-04-13 20:50:18,221 INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler) [main]
2023-04-13 20:50:18,225 INFO [kafka-raft-outbound-request-thread]: Starting (kafka.raft.RaftSendThread) [kafka-raft-outbound-request-thread]
2023-04-13 20:50:18,226 INFO [kafka-raft-io-thread]: Starting (kafka.raft.KafkaRaftManager$RaftIoThread) [kafka-raft-io-thread]
2023-04-13 20:50:18,226 INFO Starting controller (kafka.server.ControllerServer) [main]
2023-04-13 20:50:18,385 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:18,388 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:18,454 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:18,456 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:18,519 INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas) [main]
2023-04-13 20:50:18,524 INFO Awaiting socket connections on 0.0.0.0:9090. (kafka.network.DataPlaneAcceptor) [main]
2023-04-13 20:50:18,568 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:18,589 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:18,590 INFO [SocketServer listenerType=CONTROLLER, nodeId=2] Created data-plane acceptor and processors for endpoint : ListenerName(CONTROLPLANE-9090) (kafka.network.SocketServer) [main]
2023-04-13 20:50:18,613 INFO [Controller 2] Creating new QuorumController with clusterId iD5B9selQtujdUjasWtvLQ, authorizer Optional.empty. (org.apache.kafka.controller.QuorumController) [main]
2023-04-13 20:50:18,615 INFO [RaftManager nodeId=2] Registered the listener org.apache.kafka.controller.QuorumController$QuorumMetaLogListener@1854895689 (org.apache.kafka.raft.KafkaRaftClient) [kafka-raft-io-thread]
2023-04-13 20:50:18,619 INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Fetch]
2023-04-13 20:50:18,619 INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Produce]
2023-04-13 20:50:18,620 INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Request]
2023-04-13 20:50:18,622 INFO [ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-ControllerMutation]
2023-04-13 20:50:18,633 INFO [ExpirationReaper-2-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-AlterAcls]
2023-04-13 20:50:18,638 INFO [SocketServer listenerType=CONTROLLER, nodeId=2] Enabling request processing. (kafka.network.SocketServer) [main]
2023-04-13 20:50:18,643 INFO [BrokerServer id=2] Transition from SHUTDOWN to STARTING (kafka.server.BrokerServer) [main]
2023-04-13 20:50:18,644 INFO [BrokerServer id=2] Starting broker (kafka.server.BrokerServer) [main]
2023-04-13 20:50:18,647 WARN inter.broker.protocol.version is deprecated in KRaft mode as of 3.3 and will only be read when first upgrading from a KRaft prior to 3.3. See kafka-storage.sh help for details on setting the metadata version for a new KRaft cluster. (kafka.server.KafkaConfig) [main]
2023-04-13 20:50:18,655 INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Fetch]
2023-04-13 20:50:18,655 INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Produce]
2023-04-13 20:50:18,655 INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Request]
2023-04-13 20:50:18,655 INFO [ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-ControllerMutation]
2023-04-13 20:50:18,685 INFO [BrokerToControllerChannelManager broker=2 name=forwarding]: Starting (kafka.server.BrokerToControllerRequestThread) [BrokerToControllerChannelManager broker=2 name=forwarding]
2023-04-13 20:50:18,729 INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas) [main]
2023-04-13 20:50:18,730 INFO Awaiting socket connections on 0.0.0.0:9091. (kafka.network.DataPlaneAcceptor) [main]
2023-04-13 20:50:18,740 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:18,803 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:19,080 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:19,098 INFO [SocketServer listenerType=BROKER, nodeId=2] Created data-plane acceptor and processors for endpoint : ListenerName(REPLICATION-9091) (kafka.network.SocketServer) [main]
2023-04-13 20:50:19,099 INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas) [main]
2023-04-13 20:50:19,099 INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.DataPlaneAcceptor) [main]
2023-04-13 20:50:19,107 INFO [SocketServer listenerType=BROKER, nodeId=2] Created data-plane acceptor and processors for endpoint : ListenerName(PLAIN-9092) (kafka.network.SocketServer) [main]
2023-04-13 20:50:19,108 INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas) [main]
2023-04-13 20:50:19,108 INFO Awaiting socket connections on 0.0.0.0:9093. (kafka.network.DataPlaneAcceptor) [main]
2023-04-13 20:50:19,158 INFO [SocketServer listenerType=BROKER, nodeId=2] Created data-plane acceptor and processors for endpoint : ListenerName(TLS-9093) (kafka.network.SocketServer) [main]
2023-04-13 20:50:19,172 INFO [BrokerToControllerChannelManager broker=2 name=alterPartition]: Starting (kafka.server.BrokerToControllerRequestThread) [BrokerToControllerChannelManager broker=2 name=alterPartition]
2023-04-13 20:50:19,181 INFO [ExpirationReaper-2-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-Produce]
2023-04-13 20:50:19,182 INFO [ExpirationReaper-2-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-Fetch]
2023-04-13 20:50:19,182 INFO [ExpirationReaper-2-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-DeleteRecords]
2023-04-13 20:50:19,183 INFO [ExpirationReaper-2-ElectLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-ElectLeader]
2023-04-13 20:50:19,195 INFO [ExpirationReaper-2-Heartbeat]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-Heartbeat]
2023-04-13 20:50:19,195 INFO [ExpirationReaper-2-Rebalance]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-Rebalance]
2023-04-13 20:50:19,231 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:19,263 INFO [BrokerToControllerChannelManager broker=2 name=heartbeat]: Starting (kafka.server.BrokerToControllerRequestThread) [BrokerToControllerChannelManager broker=2 name=heartbeat]
2023-04-13 20:50:19,266 INFO [RaftManager nodeId=2] Registered the listener kafka.server.metadata.BrokerMetadataListener@591072738 (org.apache.kafka.raft.KafkaRaftClient) [kafka-raft-io-thread]
2023-04-13 20:50:19,266 INFO [BrokerLifecycleManager id=2] Incarnation PxUKS1KHToeGZzaPt3cmIw of broker 2 in cluster iD5B9selQtujdUjasWtvLQ is now STARTING. (kafka.server.BrokerLifecycleManager) [EventHandler]
2023-04-13 20:50:19,282 INFO [ExpirationReaper-2-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [ExpirationReaper-2-AlterAcls]
2023-04-13 20:50:19,575 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:19,678 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:19,855 INFO [RaftManager nodeId=2] Re-elect as candidate after election backoff has completed (org.apache.kafka.raft.KafkaRaftClient) [kafka-raft-io-thread]
2023-04-13 20:50:19,861 INFO [RaftManager nodeId=2] Completed transition to CandidateState(localId=2, epoch=26, retries=2, electionTimeoutMs=1918) (org.apache.kafka.raft.QuorumState) [kafka-raft-io-thread]
2023-04-13 20:50:19,863 INFO [Controller 2] In the new epoch 26, the leader is (none). (org.apache.kafka.controller.QuorumController) [QuorumControllerEventHandler]
2023-04-13 20:50:19,999 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:20,272 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:20,538 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:20,848 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:20,951 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:21,401 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:21,462 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:21,873 INFO [RaftManager nodeId=2] Re-elect as candidate after election backoff has completed (org.apache.kafka.raft.KafkaRaftClient) [kafka-raft-io-thread]
2023-04-13 20:50:21,880 INFO [RaftManager nodeId=2] Completed transition to CandidateState(localId=2, epoch=27, retries=3, electionTimeoutMs=1573) (org.apache.kafka.raft.QuorumState) [kafka-raft-io-thread]
2023-04-13 20:50:21,881 INFO [Controller 2] In the new epoch 27, the leader is (none). (org.apache.kafka.controller.QuorumController) [QuorumControllerEventHandler]
2023-04-13 20:50:21,974 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-0.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
2023-04-13 20:50:22,018 WARN [RaftManager nodeId=2] Error connecting to node my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local:9090 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient) [kafka-raft-outbound-request-thread]
java.net.UnknownHostException: my-cluster-1794df46-kafka-1.my-cluster-1794df46-kafka-brokers.namespace-0.svc.cluster.local
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1(InterBrokerSendThread.scala:103)
at kafka.common.InterBrokerSendThread.$anonfun$sendRequests$1$adapted(InterBrokerSendThread.scala:99)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
at kafka.common.InterBrokerSendThread.sendRequests(InterBrokerSendThread.scala:99)
at kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:73)
at kafka.common.InterBrokerSendThread.doWork(InterBrokerSendThread.scala:94)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment