Skip to content

@aaronblohowiak /gist:3935383
Created

Embed URL

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Notes for My talk at RedisConf
Hi!
@aaronblohowiak
aaron@xitive.com
These notes will be / were published right after the talk.
10/22/2012
WHOAMI
Conributed to redis-doc: fdc12e2a214f044cb966652ccfcce5641361c41f :D
Founder of u2601.com, "Best cloud provider."
Sponsor of the.endoftimezones.com, "Most important political movement ever"
Instigator of EVALSHA.com, "An OSS repo of public domain Redis scripts"
64 public repos on github, most are useless ;D
Independent contractor.
EVAL, EVALSHA and EVALSHA.com
* Introduction to EVAL & EVALSHA
* Lua in Redis
* Examples of Lua Scripting
* Caveats
* EVALSHA.com
Introduction to EVAL & EVALSHA:
* Lua _in_ redis-server
* Uninteruptable kind of atomic
* Cached procedures vs Stored procedures
* Good client support already
* Interesting OSS projects
- http://evalsha.com
- https://github.com/andyet/redis-lua-example
- https://github.com/Shopify/wolverine
* Resources:
- http://redis.io/commands/eval !!! This is the best
- http://evalsha.com/documentation
- http://redis.io/commands#scripting
- http://tjholowaychuk.com/post/19321054250/redis-lua-scripting-is-badass
- http://blog.vishalshah.org/post/22175246910/redis-lua-for-processing-json-values
- http://dan.lecocq.us/wordpress/2012/03/11/redis-and-lua-for-robust-portable-libraries/
Lua in Redis:
* Easy to call redis:
- redis.call(CMD, [...ARGS...]) (raise)
- redis.pcall(CMD, [...ARGS...]) (error table)
- redis.log(loglevel,message)
* No Globals
- variable declaration: 'local myvar = redis.call("GET", KEYS[1])'
* JSON + MSGPACK
* Datatypes auto-converteted
- Lua Number is like JS Number
- Redis converts Number to Int! (use tostring(num))
- Lua Array-likes => Multi-Bulk
- Lua Hash-likes => :(
* The *script* is replicated, not its write commands
- BW - preserving, but complicates usage
* Deterministic OR Readonly
- math.random always the same sequence
- math.randomseed through ARGV
- TIME / SRANDOMKEY / SRANDMEMBER: Readonly!
* Develop using ruby client:
r.eval(File.read(filename), keys, args)
Thoonk from &Yet:
Convention-based event routing and dispatching system.
Gives you a pattern for publishing events and for managing subscriptions to them.
In your app, you define your event schema and the lua scripts that run in redis.. your object then "Lives in Redis."
If you use an object with these keys with these patterns, you get things for free.
RPC System, where you are remotely executing a locally defined script for each method on an object.
You should check it out!
- http://github.com/andyet/thoonk.js
- http://github.com/andyet/thoonk.py
- AndBang is implemented using it
- Questions? Ask @lancestout (py) @fritzy (js)
Why not Watch/Multi/Exec:
* Optimistic: while(fail?) retry
* Fewer network roundtrips
* Continues even if there are errors.
redis> multi
OK
redis> set hi 'hello'
QUEUED
redis> rpop hi
QUEUED
redis> set okay 'done'
QUEUED
redis> exec
1. OK
2. (error) ERR Operation against a key holding the wrong kind of value
3. OK
redis> get okay
"'done'"
redis> get hi
"'hello'"
redis>
Example: ZPOP WME vs EVAL:
* ZPOP is LPOP for ZSET
* (Remove lowest item and return it)
WME: (from redis.io)
WATCH zset
element = ZRANGE zset 0 0
MULTI
ZREM zset element
EXEC
EVAL: http://evalsha.com/scripts/00c37234520b7295684ccc128636b2413132684a
local elm = redis.call("ZRANGE", KEYS[1], 0, 0)[1]
if elm then
redis.call("ZREM", KEYS[1], elm)
end
return elm
Example: CAS
* Antirez thinks its dumb in redis https://groups.google.com/forum/?fromgroups=#!msg/redis-db/a4zK2k1Lefo/OU9ewOJro5MJ
* But it is a simple example
* http://evalsha.com/scripts/4298aecc807ec90f8bd4ec7f7006563cee4f7a1b
local current = redis.call('GET', KEYS[1])
if current == ARGV[1]
then
return redis.call('SET', KEYS[1], ARGV[2])
end
return false
Example: Publish to multiple channels
* publish a message to multiple channels
* avoid resending the message to redis
* http://evalsha.com/scripts/d639c8a2713f88d88d3ad6900e05a64a99aa3ead
local clients=0
for i,v in ipairs(ARGV) do
if i > 1 then
clients = clients + redis.call("PUBLISH", v, ARGV[1])
end
end
return clients
Example: Disconnected Subscriptions (mailboxes)
* Support connected & disconnected subscriptions
* Publish to channel & to mailboxes
* Single-Server only.
Lua script: http://evalsha.com/scripts/67e165f02e2830b2450acba0fae439a704edb515
local chan = KEYS[1]
local result = {}
result[1] = "clients"
result[2] = redis.call("PUBLISH", KEYS[1], ARGV[1])
result[3] = "boxes"
local boxes = redis.call("LRANGE", KEYS[1], 0, -1)
local boxCount = 0
for i,v in ipairs(boxes) do
redis.call("RPUSH", v, ARGV[1])
boxCount = boxCount + 1
end
result[4] = boxCount
return result
Results:
redis 127.0.0.1:6379> EVALSHA 7e992b1c3e547703c84e3901c4847b6ca92e84ce 1 News "My news!!!"
1) "clients"
2) (integer) 0
3) "boxes"
4) (integer) 2
Clients:
LTRIM listname 0 -1 to poll and/or BLPOP in a loop
Example: HMGETALL (like Crashalytics' Jeff wants):
- Get all keys for all hashes
Lua:
local result = {}
local hshValue = nil
for i,v in ipairs(KEYS) do
hshValue = redis.call("HGETALL", v)
result[i] = hshValue
end
return result
Example in Use:
redis 127.0.0.1:6379> HMSET hsh1 clicks 10 crashes 0
OK
redis 127.0.0.1:6379> HMSET hsh2 clicks 100 crashes 10
OK
redis 127.0.0.1:6379> HMSET hsh4 clicks 300 crashes 10
OK
1.9.2p320 :030 > r.eval(File.read('mhgetall.lua'), %w{hsh1 hsh2 hsh3 hsh4})
=> [["clicks", "10", "crashes", "0"], ["clicks", "100", "crashes", "10"], [], ["clicks", "300", "crashes", "10"]]
Caveats. AKA :(
* In Lua a[1]..a[N]
* Full script sent to slave/AOF (currently) scripting.c:867
* No Blocking operations!
* Single-server issues
* All scripts kept forever by default.
* No include/require: &Yet uses the c preprocessor to include multiple functions
* Testing this stuff is hard
* Error reporting =(
* Implicit sort:
- SMEMBERS
* What if script loops forever? ( BUSY )
- If readonly: SCRIPT KILL
- else: SHUTDOWN NOSAVE
* Logging is very expensive, churns fds
- redis.c:272
- fp = (server.logfile == NULL) ? stdout : fopen(server.logfile,"a");
- Don't log a lot in production!
* ASCIIZ-style arrays: First "nil" in Lua Array marks its end =(
- 1.9.2p320 :033 > r.eval("return {1,2,nil,3}", [])
=> [1, 2]
EVALSHA.com:
* Free/Open-Source repo of public domain scripts
* Based on redis.io code from CitrusByte
* Search using elasticsearch
* Needs your help!
- github.com/aaronblohowiak/evalsha.com
* What would you like to see added to this?
* Add other-licenses?
Questions?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.