public
Created

RedisConf Notes Part II

  • Download Gist
gistfile1.txt
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59
Dr Josiah Carlson
- Redis user for 2 1/2 years. Extremely active on redis mailing list, #3 poster to the list (applause from PN)
- Author of Redis In Action (available electronically)
- What is search? scan text, use BM/BMG/KMP or regex or recursive-descent.. or suffix tries.. or you can search through the zip-format (BWT)
- Inverted index: words to documents, instead of crawling documents for words.
- SET per word with items for each docid, using redis' intersection for AND, union for OR
- "Simplest example that will be useful..[but it wont be that useful... buy the book!]"
- Demo of his editor.. its on sourceforge.
- Tokenization, stop words in some python code.. "with this stuff, we're going to implement search" already has failing tests written.. live coding, so not many notes... buy his book.
- "I have 5 minutes? I thought this was a 45-minute talk".. "Nope!"..."Shoot!"
- "If you like intersecting sets and unioning them...you're welcome" (Josiah created them in fork at old company...)
- "In the book, I cover fake-bayesian learning..[so it learns ads per word]"
- "You can build an ad-targeting engine in 3-4 weeks and be making money...if you do, I accpet donations!"
- User: "How do you do exact phrase search [if you use inverted index]?" A: "View content as non-stopword to non-stopword keys.. so you would store "lord" and 'rings' but also 'lord rings'" U: "Like trigrams?" A:"[Like fake trigrams, but more space-efficient]"
 
Aren Sandersen
- @Pinterest for 6 months, which is longer than 75% of the engineers. Pinterest is all scaling firedrill, use redis a lot.
- Overview of pinterest. Compressed-air powered Nerf gun pic + lols.. tour of screens
- Users can follow other users, or a subsection (board) of their posts, so graph is 'interesting'
- "Been called the fastest growing website..[it's all about scaling]" shows hockey-stick growth to jan, flat til may.
- 1 year ago: 4 Cassie, 15 membase, 10 memcache, 10 redis nodes, 2 engineers
- May: 59 redis, 51 memcache, 66 mysql, 6 engineers, sharded solr
- Nowish: over 200 redis, 50 engineers..
- "Would not have been able to keep up without the cloud...[sometimes, I spin up 50 servers for a project]"
- Justifies using all MySQL, Memcache and Redis.
- "Sharding is the #1 way we scale..[pre-shard so we can scale easily]...[earlier the better]"
- Started with many dbs per physical server. master-master, but only write to some. To scale, shift around where physically some of the dbs are located and update config.
- Pin ID is structured with shardId,type,localId (localid is id within table)
- They shard based on user join time, and maintain giant lookup structure for dbId to physical layout.
- Using redis for: pyres (resque for python), followers, feeds, caching in memcache and redis, anti-spam
- Truth in MySQL, redis as cache vs memcache because it allows paging. use memcache for id:small blob
- Starting to use redis persistently, so not sure how long mysql will be source of truth.
- Stagger bgsave.. but if a slave comes down and comes back up and all try to reconnect, the master's procs will all have to bgsave to let them sync. Use aof + hourly (staggared) bgsaves.
- use lua to avoid race conditions in updating list of followers.
- Twitter's twemproxy is adding redis support - run one per f/e host to pool connections.
- U: "Why not just redis instead of redis+memcache?" A:"We haven't explored that..[historical reasons]"
- U: "I noticed you went from having Mongo to not having Mongo..[why'd you leave it?]" A:"I wasn't there at the time..[they just weren't having good experiences..hard to say if that was the product or our usage..wanted something we could get support on if needed..]"
- U: "Any issues with lua scripting & expensive ops being slow and causing problems?" A: "Yes..there's a learning curve like everything.. but I think the team is very happy now with their decision.."
- U: "What happens when master fails?" A:"[It's all manual, there is a page..] code push to promote slave to master...working to streamline process..use zookeeper [to avoid code push]"
- Hourly backups of aof to S3
- U:"How do you force staggared BGSAVES?" A: "Cronjob"
 
Antirez - Sentinel Solution
- Sentinel is distributed monitoring so it is more reliable from Ops' point of view.
- Notification system
- Automatic failover, client config server
- Monitoring: if not distributed, availability from one point of view. We want to know if it is up from multiple POVs with an agreement protocol.
- Image of gossip protocol: N:N pings
- Demo using terminal. redis-sentinel IS redis-server (same sha), just detecting name. its a mode of execution
- Sentinel conf is easy because slaves and other sentinels are auto-detected
- You can run dozens of sentinels without issue if you have room for the processes. Set quorum as a fixnum in conf..
- In sentinal mode, redis responds to ping, 'info sentinal', and 'sentinal' command. NO set or other keyspace commands
- sdown vs odown: sdown is "down for me", owdown is once quorom is reached. "slaves don't require quorum because quorum is just used to promote masters"!
- You can subscribe to status using pub/sub channels, or you can have scripts that are executed when event happens. Demo of 'subscribe *' + bringing down slave yielding an 'sdown' event, and subbing to jus 'sdown' for selective message subscription.
- Automatic failover: requires client cooperation. odown detected, failover leader sentinal elected, slave selected, turned into master, other slaves repointed @ it. demo of it in-action. takes several seconds. uses timeouts to avoid race conditions.
- Walkthrough of sentinals following promotion-master and the slave promotion, use of sentinal get-master-addr-by-name
- Sentinal is 3k LOC. Uses redis codebase as framework; developed in few weeks instead of months. Planning to update sentinel with every point release of 2.6
- Ruby, Python, Node.js clients already support it.
- Redis 2.8 will support config through sentinal, so no need to say "slaveof" on startup.

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.