Skip to content

Instantly share code, notes, and snippets.

@PatrickLef
Created November 9, 2012 09:36
Show Gist options
  • Save PatrickLef/4044822 to your computer and use it in GitHub Desktop.
Save PatrickLef/4044822 to your computer and use it in GitHub Desktop.
Redis comment store ideas
I'm trying to figure out how to fetch comments for objects and optimize memory usage and lookup speed. One video has multiple comments and comments will always be fetched using VideoID or VideoID + CommentID.
Our setup: One master in Amazon and a slave on each Web server. The web-servers will deliver this data, so the web servers will fetch this locally.
Some stats: Every video has in general 10-20 comments.
Idea 1: Store comments in a hash with VideoID as key, Comment ID as field and comment-data as value. Then fetching the whole hash and sorting it based on a Sorted set with VideoID as key and CommentIDs as values.
Pros: Only two lookups to fetch all comments (One for Sorted list and One for the hash).
Cons: VideoID will always be needed to fetch a comment (Okay in our application). We are not utilizing the memory savings of using a hash too keep down the memory size used by redis.
Redis storage:
cache:video:12345:comments = {
54345 => "SerializedComment",
573657 => "SerializedComment"
}
cache:video:12345:comments:video_id_sort = [54345, 573657]
Redis lookups:
ZRANGE cache:video:12345:comments:video_id_sort
HMGET cache:video:12345:comments 54345 573657
Idea 2: Store comments in a hash with CommentID's first part as key, CommentID's last to numbers as field and comment-data as value. Then fetching it (one by one) by a sorted set with VideoID as key and CommentIDs as values.
Pros: CommentID will be enough to fetch a comment. The hash can be optimized to 100 comments per hash.
Cons: When fetching a video with comments we'll have to do one lookup per comment since they'll be in different hashes.
Redis storage:
cache:comments:543 = {
...
44 => "SerializedComment",
45 => "SerializedComment",
...
}
cache:comments:5736 = {
...
57 => "SerializedComment"
...
}
cache:video:12345:comments:video_id_sort = [54345, 573657]
Redis lookups:
ZRANGE cache:video:12345:comments:video_id_sort
HMGET cache:comments:543 45
HMGET cache:comments:5736 57
What do you think is best to optimize for? My feeling is that fewer lookups will increase the speed for the end users but it'll have more memory usage on the servers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment