redis-dev-day-2015.md

Antirez

• NoNoSQL - IDXSELECT for indices
  • based on sorted sets
  • but sorted sets have unique elements only
  • IDXSELECT myindex IFI FIELDS $3 WHERE $1 == 56 and $2 >= 10.00 AND $2 <= 30.00
• Timeseries database
  • not possible to built upon sorted sets
  • new data structure, API
• Multiple AOF file to avoid in-memory buffer on AOF rewrite
• RDB-prefixed AOF to save space
• Redis-CP
  • Redis with proxy in front for proper Raft/consensus

Redis as a Cloud native (Bill)

• Delayed Availability
  • Start up, read RDB, set everything up
  • but wait with serving clients to have the possibility to change config beforehand
• Config Replication / Sync
  • Sync configuration changes to slave instances
  • Only some configurations should be synced, opt-out for SAVE or similar
• Metadata for Instances
  • Setable via API, add administrative tags
  • Limit to a few key-value pairs (~100)
  • Persist it into the config file
• Sentinel
  • Sync sentinel options to other Sentinel nodes
  • Issue: 3 hosts with 1x Sentinel, 1x Redis
    • kill host with master → failover fails
  • Issue: Sentinel remove data lag
    • Sentinel Config state is reapplied after removing, then re-adding quickly
    • Some epoch/config changes still in flight to Sentinel node
  • Goodbye message on Sentinel removes
    • Lower seen Sentinel nodes
  • Slave remove
    • Forget this slave, don’t re-discover it
  • Let Sentinel distribute removals instead of user
    • make it blocking until it has ACKs from all other
• Config stored in the cloud
  • backing store (etcd, consul, …) or cloud storage (AWS, …)
  • Abstract used store, e.g. similar to https://github.com/docker/libkv
  • Sync back and forth between Redis and stored config
  • antirez: Redis-CP might cover a few things already
• Redis - Memory as a Service
  • Have a malloc-on-redis
  • Fixed-size data entries
  • fixed-size counters in contiguous memory
    • Possible: get substring (GETRANGE)
    • Next: Increment sub-part of value

Redis in Docker

• use --net=host: Use host network directly, no ip/port remapping
• Everything in Docker except Redis
  • Redis is foundation
  • Docker is immutable infrastructure

Microservices in Redis (Mario)

• IP-AS Lookup using Redis, Lua scripts and some logic in the scripts
• MONITOR: Include Lua script handle
• Aliases for Lua scripts

Itamar

• Quadtrees in Lua in Redis for 2-dimensional range queries
  • https://gist.github.com/itamarhaber/c1ffda42d86b314ea701
  • https://en.wikipedia.org/wiki/Quadtree
  • Geohashes are basically a flattened Quadtree, maybe usable for 2-dim queries
• We need more sophisticated APIs for queries
• Lua Debugging
  • http://www.trikoder.net/blog/foo-89
  • https://github.com/slembcke/debugger.lua
• Lua PRs
  • redis/redis#732

Redis Source Modularity

• libredis
  • RDB parsing
  • Protocol parsing
  • Command table
  • Generic Redis protocol server
• tension between performance and more abstraction/modularity
  • coupling avoids indirections, fast paths
  • organisation of dependencies, circular dependencies: hiredis → sds → redis → sds

Master ←→ Master Replication (Yftach, Redis Labs)

• No conflict resolution
• Introduce proxy master in between two Clusters
  • Gets writes from all masters, resolves conflicts and sends back fix commands

From a Client perspective

• Command composition
  • STORE dest SINTER keyA keyB