Suppose topic with a partition P0.
- Producer PR1 publishes a message to P0 with required acks = 2 (two peers from the replicas set, aside from leader, needs to ack. it before the update is considered committed). Producer PR1 is waiting for ack from the leader. That message’s LSN is going to be 100.
- Producer PR2 publishes a message to P0 with required acks = 1(one peer from the replicas set, aside from the leader, needs to ack. it before the update is considered committed). Producer PR2 is waiting for ack from the leader. That message’s LSN is going to be 101
- One Peer asks to consume from x(< 100) and then immediately asks to consume from 102. Other peers in the replicas set are busy and haven’t gotten to consume any updates yet(but not too busy as to be expelled from the ISR).
What happens now? Is PR2 getting acknowledged (because one peer consumed > 101 and acks == 1 and PR's message LSN = 101) ? Or, will Kafka wait for all messages before it to be aknowledgd (e.g the message from PR1 with LSN 100) before acknowledging the message from PR2 with LSN 101?` That is to say, will it respond to PR2 with an acknowledgement before PR1 is provided an acknowledgement, considering that PR1's message LSN < PR2's message LSN?
Thanks Gwen. That makes sense.
I wasn't sure if PR2 would get an ack if there was another pending response to be ack for an LSN < PR2's LSN. I also thought that HWM would advance based on ack. semantics, not based on the maximum LSN attempted to be consumed by all nodes in the ISR.
That is to say, if a producer produces a message with LSN X and required peer acks = 1 (i.e 1 peer ack + the implicit ack from the leader), then as soon as just one replica consumed from LSN > X, the producer would get its ack and HWM would advance (because the producer who produced at X was acknowledged).