Last active

Embed URL

HTTPS clone URL

SSH clone URL

You can clone with HTTPS or SSH.

Download Gist

Notes for my redis talk at the Memphis Super User Group meeting (hosted by the Memphis Java User Group).

View redis-notes.rst

Redis

These are my notes for a short presentation on Redis at the annual Super User Group meeting hosted by the Memphis JUG.

Intro

(What is it? see: http://redis.io/topics/faq)

Redis ("REmote DIctionary Server"), is an in-memory Key-Value Database. Also frequently called a "data structure server". Key Points:

  • It's very Fast.
  • It's Durable (it can be).
  • In-Memory. Your data should fit in RAM
  • Values are associated with unique Keys
  • Values are Data Types: Strings, Lists, Sets, Sorted Sets, Hashes
  • Every command is Atomic
  • Keys can be segmented.

Clients

Many programming languages have libraries for Redis; A few of the recommended (e.g. actively developed) include:

  • C
  • C#
  • Clojure
  • Dart
  • Erlang
  • Go
  • Haskell
  • Java
  • Lua
  • Node.js
  • Perl
  • PHP
  • Python
  • Ruby
  • Scala

See http://redis.io/clients for more info.

On Keys

  • Can be anything (binary data)
  • It's good to define a schema; e.g. object-type:id:field
  • Can be queried (you don't query values, you query keys)

Data Types

See: http://redis.io/topics/data-types

Strings

Most basic data type, but it's very versitile; can be up to 512Mb; can be serialized data, e.g. JSON or JPEG image. Redis doesn't care what you store. Examples of string commands, follow:

Set some values:

SET users:brad "{email: brad@workforpie.com, password: SOMEHASH}"
SET users:brad:avatar "\xf8\xf8\xf8\xf0\xf0\xf0\xf8\xf8\xf8\xeb..."

Get some values:

GET users:brad
GET users:brad:avatar

Additional Utilities:

STRLEN users:brad                # returns 53
APPEND users:brad " moar data"   # Careful!

GET users:brad
# Returns: "{email: brad@workforpie.com, password: SOMEHASH} moar data"

Incrementing:

SET users:count 0
INCR users:count        # users:count -> 1
INCRBY users:count 5    # users:count -> 6
Lists

Store & manipulate a list (aka an Array). Lists maintain their order, and have fast index-based operations.

Adding to a list:

LPUSH users sally
LPUSH users jane
LPUSH users bill

Do an index-based lookup:

LINDEX users 0      # Returns "bill"
LINDEX users 2      # Returns "sally"

Access a range of items:

LRANGE users 0 3
# returns "bill", "jane", "sally"

Utilities:

LLEN users              # returns 3
LSET users 0 sallie     # changes "sally" to "sallie"

POP items from the list (removes the first item that was pushed):

LPOP users      # returns "bill"
Sets

Store unique values and provide set-based operations. Values have no order. Great for tagging content or tracking properties.

Adding a set:

SADD blog:10 redis nosql
SADD blog:11 couchdb nosql

Updating a Set:

SADD blog:11 json   # would now include "couchdb", "nosql", "json"

Remove an item from a set:

SREM blog:11 json   # removes the "json" value from the set

View the members of a set:

SMEMBERS blog:10       # returns "nosql", "redis"

Does an set contain a given value?:

SISMEMBER blog:10 python    # returns 0 or 1

Set-based operations:

SUNION blog:10 blog:11      # returns "redis", "couchdb", "nosql"
SINTER blog:10 blog:11      # returns "nosql"
SDIFF blog:10 blog:11       # returns "redis" (in blog:10 but not blog:11)
Sorted Sets

Just like sets, but each item also has a score that gives you sorting and ranking abilities. Great for a game leaderboard!

Add some users to the leaderboard:

ZADD leaders 30 sally
ZADD leaders 50 julie
ZADD leaders 10 bill

Find a person's rank (default sorting is low to high):

ZRANK leaders bill      # returns 0

Find a rank (sorted high to low):

ZREVRANK leaders bill   # returns 2

Count users with a score between 20 and 50 (inclusive):

ZCOUNT leaders 20 50    # returns 2

Find a user's score:

ZSCORE leaders bill     # returns 10
Hashes

Similar to strings, but hashes give you additional fields. This can result in a more granular access to a well-defined object. Think of it as a JSON object or a Python dictionary.

The signature to create a Hash object: HSET key field value [field value ...]

Given the JSON object for a user:

{
    id: 1234,
    username: johndoe,
    email: johndoe@example.com,
    fullname: 'John Doe'
}

Store in a Redis HASH with:

HSET users:1234 username johndoe
HSET users:1234 email johndoe@example.com
HSET users:1234 fullname "John Doe"

Get individual field values:

HGET users:1234 username    # returns "johndoe"

Get Multiple field values:

HMGET users:1234 username email     # returns "johndoe", "johndoe@example.com"

Get all keys/values for a hashed item:

HGETALL users:1234
# returns: "username", "johndoe", "email", "johndoe@example.com",
#          "fullname", "John Doe"

Additional Utilities:

HKEYS users:1234                # returns "username", "email", "fullname"
HLEN users:1234                 # returns 3
HEXISTS users:1234 password     # returns 0 (false)
HDEL users:1234 fullname        # removes the "fullname" field & value

Uses

Redis supplements other data storage systems; Use in addition to PostgreSQL.

Metrics

I use redis as the backend for django-redis-metrics. I store metrics as a string, incrementing it when necessary. Keys represent different granularities.

In python, the code looks something like this:

>>> metric('github-api')

And in Redis, keys are stored as:

m:github-api:2013-01-31 -> 100     ; daily usage
m:github-api:w:2013-05  -> 1000    ; weekly usage
m:github-api:m:2013-01  -> 10,000  ; monthly usage
m:github-api:y:2013     -> 100,000 ; yearly usage

I can then query that data and generate graphs (using Google Charts) of the data.

http://foib.us/im/github-api-calls-20130617-235634.png
Cache/Ephemeral Data storage

Redis is commonly used as a cache or a data store for ephemeral data. You can set an Expiration for any redis key, so it's a good store for data that you want to automatically expire. Examples might include:

  • User session keys
  • "flash messages"
  • Database query results

Redis command to expire a key:

expire messages:brad 30     # expires the "messages:brad" key in 30s
Simple Task Queue
Redis is also often used as a backend for a simple, asynchronous task queue (I use python-rq)
Pub/Sub

Redis also implements the Publish-Subscribe pattern.

Subscribe to a Channel called messages:

subscribe messages

Publish a message:

publish messages "Hello World!"

Durability

Multiple ways to persist.

  • RDB: Write a Snapshot of your data to disk; Based on number of changed keys in a time period; e.g. if 1000 keys have changed in 60 seconds.
  • AOF (append-only file): writes data to disk (always or every second)
  • None. ONLY stores data in memory.
  • Both! If you really care about your data, use AOF, with periodic snapshots.

Resources

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.