Notes for my redis talk at the Memphis Super User Group meeting (hosted by the Memphis Java User Group).

redis-notes.rst

Redis

These are my notes for a short presentation on Redis at the annual Super User Group meeting hosted by the Memphis JUG.

Intro

(What is it? see: http://redis.io/topics/faq)

Redis ("REmote DIctionary Server"), is an in-memory Key-Value Database. Also frequently called a "data structure server". Key Points:

• It's very Fast.
• It's Durable (it can be).
• In-Memory. Your data should fit in RAM
• Values are associated with unique Keys
• Values are Data Types: Strings, Lists, Sets, Sorted Sets, Hashes
• Every command is Atomic
• Keys can be segmented.

Clients

Many programming languages have libraries for Redis; A few of the recommended (e.g. actively developed) include:

• C
• C#
• Clojure
• Dart
• Erlang
• Go
• Haskell
• Java
• Lua
• Node.js
• Perl
• PHP
• Python
• Ruby
• Scala

See http://redis.io/clients for more info.

On Keys

• Can be anything (binary data)
• It's good to define a schema; e.g. object-type:id:field
• Can be queried (you don't query values, you query keys)

Data Types

See: http://redis.io/topics/data-types

Strings

Most basic data type, but it's very versitile; can be up to 512Mb; can be serialized data, e.g. JSON or JPEG image. Redis doesn't care what you store. Examples of string commands, follow:

Set some values:

SET users:brad "{email: brad@workforpie.com, password: SOMEHASH}"
SET users:brad:avatar "\xf8\xf8\xf8\xf0\xf0\xf0\xf8\xf8\xf8\xeb..."

Get some values:

GET users:brad
GET users:brad:avatar

Additional Utilities:

STRLEN users:brad                # returns 53
APPEND users:brad " moar data"   # Careful!

GET users:brad
# Returns: "{email: brad@workforpie.com, password: SOMEHASH} moar data"

Incrementing:

SET users:count 0
INCR users:count        # users:count -> 1
INCRBY users:count 5    # users:count -> 6

Lists

Store & manipulate a list (aka an Array). Lists maintain their order, and have fast index-based operations.

Adding to a list:

LPUSH users sally
LPUSH users jane
LPUSH users bill

Do an index-based lookup:

LINDEX users 0      # Returns "bill"
LINDEX users 2      # Returns "sally"

Access a range of items:

LRANGE users 0 3
# returns "bill", "jane", "sally"

Utilities:

LLEN users              # returns 3
LSET users 0 sallie     # changes "sally" to "sallie"

POP items from the list (removes the first item that was pushed):

LPOP users      # returns "bill"

Sets

Store unique values and provide set-based operations. Values have no order. Great for tagging content or tracking properties.

Adding a set:

SADD blog:10 redis nosql
SADD blog:11 couchdb nosql

Updating a Set:

SADD blog:11 json   # would now include "couchdb", "nosql", "json"

Remove an item from a set:

SREM blog:11 json   # removes the "json" value from the set

View the members of a set:

SMEMBERS blog:10       # returns "nosql", "redis"

Does an set contain a given value?:

SISMEMBER blog:10 python    # returns 0 or 1

Set-based operations:

SUNION blog:10 blog:11      # returns "redis", "couchdb", "nosql"
SINTER blog:10 blog:11      # returns "nosql"
SDIFF blog:10 blog:11       # returns "redis" (in blog:10 but not blog:11)

Sorted Sets

Just like sets, but each item also has a score that gives you sorting and ranking abilities. Great for a game leaderboard!

Add some users to the leaderboard:

ZADD leaders 30 sally
ZADD leaders 50 julie
ZADD leaders 10 bill

Find a person's rank (default sorting is low to high):

ZRANK leaders bill      # returns 0

Find a rank (sorted high to low):

ZREVRANK leaders bill   # returns 2

Count users with a score between 20 and 50 (inclusive):

ZCOUNT leaders 20 50    # returns 2

Find a user's score:

ZSCORE leaders bill     # returns 10

Hashes

Similar to strings, but hashes give you additional fields. This can result in a more granular access to a well-defined object. Think of it as a JSON object or a Python dictionary.

The signature to create a Hash object: HSET key field value [field value ...]

Given the JSON object for a user:

{
    id: 1234,
    username: johndoe,
    email: johndoe@example.com,
    fullname: 'John Doe'
}

Store in a Redis HASH with:

HSET users:1234 username johndoe
HSET users:1234 email johndoe@example.com
HSET users:1234 fullname "John Doe"

Get individual field values:

HGET users:1234 username    # returns "johndoe"

Get Multiple field values:

HMGET users:1234 username email     # returns "johndoe", "johndoe@example.com"

Get all keys/values for a hashed item:

HGETALL users:1234
# returns: "username", "johndoe", "email", "johndoe@example.com",
#          "fullname", "John Doe"

Additional Utilities:

HKEYS users:1234                # returns "username", "email", "fullname"
HLEN users:1234                 # returns 3
HEXISTS users:1234 password     # returns 0 (false)
HDEL users:1234 fullname        # removes the "fullname" field & value

Uses

Redis supplements other data storage systems; Use in addition to PostgreSQL.

Metrics

I use redis as the backend for django-redis-metrics. I store metrics as a string, incrementing it when necessary. Keys represent different granularities.

In python, the code looks something like this:

>>> metric('github-api')

And in Redis, keys are stored as:

m:github-api:2013-01-31 -> 100     ; daily usage
m:github-api:w:2013-05  -> 1000    ; weekly usage
m:github-api:m:2013-01  -> 10,000  ; monthly usage
m:github-api:y:2013     -> 100,000 ; yearly usage

I can then query that data and generate graphs (using Google Charts) of the data.

Cache/Ephemeral Data storage

Redis is commonly used as a cache or a data store for ephemeral data. You can set an Expiration for any redis key, so it's a good store for data that you want to automatically expire. Examples might include:

• User session keys
• "flash messages"
• Database query results

Redis command to expire a key:

expire messages:brad 30     # expires the "messages:brad" key in 30s

Simple Task Queue

Redis is also often used as a backend for a simple, asynchronous task queue (I use python-rq)

Pub/Sub

Redis also implements the Publish-Subscribe pattern.

Subscribe to a Channel called messages:

subscribe messages

Publish a message:

publish messages "Hello World!"

Durability

Multiple ways to persist.

• RDB: Write a Snapshot of your data to disk; Based on number of changed keys in a time period; e.g. if 1000 keys have changed in 60 seconds.
• AOF (append-only file): writes data to disk (always or every second)
• None. ONLY stores data in memory.
• Both! If you really care about your data, use AOF, with periodic snapshots.

Resources

• The official docs are at: redis.io
• Little Book of Redis
• Try Redis: try.redis-db.com