Skip to content

Instantly share code, notes, and snippets.

View DeathBorn's full-sized avatar
💭
⚔️

Vilius O DeathBorn

💭
⚔️
  • Vinted
  • Vilnius
View GitHub Profile
@DeathBorn
DeathBorn / Kafka performance tuning
Created March 26, 2020 08:07 — forked from idris75/Kafka performance tuning
kafka producer - consumer - broker tuning
1.Producer
1.request.required.acks=[0,1,all/-1] 0 no acknowledgement but ver fast, 1 acknowledged after leader commits, all acknowledged after replicated
2.use Async producer - use callback for the acknowledgement, using property producer.type=1
3.Batching data - send multiple messages together.
batch.num.messages
queue.buffer.max.ms
4.Compression for Large files - gzip, snappy supported
very large files can be stored in shared location and just the file path can be logged by the kafka producer.