Skip to content

Instantly share code, notes, and snippets.

@rhiroyuki
Created November 28, 2018 13:15
Show Gist options
  • Save rhiroyuki/bf28e7073705406308277162adc4058d to your computer and use it in GitHub Desktop.
Save rhiroyuki/bf28e7073705406308277162adc4058d to your computer and use it in GitHub Desktop.
Redis Stream

Memory usage and saving loading times

Because of the design used to model Redis streams, the memory usage is remarkably low. It depends on the number of fields, values, and their lengths, but for simple messages we are at a few millions of messages for every 100 MB of used memory. Moreover, the format is conceived to need very minimal serialization: the listpack blocks that are stored as radix tree nodes, have the same representation on disk and in memory, so they are trivially stored and read. For instance Redis can read 5 million entries from the RDB file in 0.3 seconds. This makes replication and persistence of streams very efficient.

It is planned to also allow deletion of items in the middle. This is only partially implemented, but the strategy is to mark entries as deleted in the entry flag, and when a given ratio between entries and deleted entires is reached, the block is rewritten to collect the garbage, and if needed it is glued to another adjacent block in order to avoid fragmentation.

Redis Stream commands are prefixed with X like (XADD, XRANGE and so on)

http://antirez.com/news/114

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment