Skip to content

Instantly share code, notes, and snippets.

@dmusicant-dk

dmusicant-dk/compaction_config.md

Created June 13, 2023 20:58
Show Gist options
  • Save dmusicant-dk/e1a32a7bb2a58ca047254986d8977369 to your computer and use it in GitHub Desktop.
Save dmusicant-dk/e1a32a7bb2a58ca047254986d8977369 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
compaction_config.md
Setting Description
segment.ms The period of time after which Kafka will force the log to roll even if the segment file isn’t full
retention.bytes | segment.bytes
  • retention.bytes sets the maximum size a partition (which consists of log segments) can grow to before it discards old log segments.
  • segment.bytes sets the segment file size for the log.
segment.jitter.ms The maximum random jitter to be subtracted from the scheduled segment roll time to avoid the "thundering herd" of segments rolling all at once
min.cleanable.dirty.ratio If the total number of duplicates is less than this value, e.g. 50%, then the messages will not be compacted
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment