Redis is a commonly used as a mini 'database' for caching, shared queues, and similar. It is generally fast & reliable for these tasks. Redis does not support nested data types, and misusing redis for this type of thing can end up with the situation described below.
Imagine that we want to store the following data in Redis:
set users '[{"user":"tim", "pages": ["aaa", "bbb"]}]'
Now a better way to store the information in "pages" would be to break up the data into a list ,and keep it separate from the "user": "tim" detail. However we are programmers so we like to do things the wrong way sometimes. Lets keep it as a single value.
Imagine we have two tasks reading and writing to this value.
Task 1 reads from this users value, and decides to add a page - it reads the string from Redis, and parses the JSON into an object.
for (var i in users) {
users[i].pages.push("TASK1")
}
upd_users = JSON.stringify(users)
redis_client.set("users", upd_users)```
Our list of pages now reads "aaa", "bbb", "TASK1"
At the same time, Task 2 also decides to add a page. It reads the string
from Redis (with the original string), adds a page "TASK2", and writes
back to Redis. Unfortunately it reads at the same time as Task 1, and is
then unaware of the change that Task 1 subsequently makes, and so someone
has their change overridden.
Normally at this point we would decide to add locking, and prevent anyone
else from proceeding down the update path once we read the initial value.
Lets avoid that this time. Instead, we can apply the change as a diff to
the JSON object. Fortunately Redis has a useful programmable interface
to allow us to do this sort of thing, using the minimal Lua language.
Lua scripts in Redis do lock the DB, however they can be much faster
than the normal app which:
- sends a network request to the DB to lock an item
- performs calculations
- sends a network request to update the DB and complete the operation.
Lua scripts are evaluated with (this is slightly simplified): EVAL [script] arg1 arg2 ... argN
We can write a little script which can be run as follows:
EVAL [script] users ?user=tim .pages [append TASK1]
- take the JSON object from the users key
- Within that object, find an element with a key of user and a value of tim
- Descend into the pages object
- Append TASK1 to that object
Meanwhile, Task 2 also executes:
EVAL [script] users ?user=tim .pages [append TASK2]
We will end up with either: aaa, bbb, TASK1, TASK2 or aaa, bbb, TASK2, TASK1
We hardly locked at all. The lock is held for less than 1ms.