You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
LPUSH key value1, value2, ...
(The LPUSH command is used to insert a value at the head of the list. We can provide one or more values with this command. The syntax of this command is.)
LRANGE key start end or LRANGE cars 0 -1
LPOP key (removes the element at the head, i.e., the left of the list)
A message queue is a type of temporary storage for messages. The sender, also known as publisher, sends messages to the queue. The messages remain in the queue until a receiver also known as a subscriber, is ready to read the messages. The main use case for a message queue is when there are some larger jobs that need to be processed. The sender sends the jobs to a queue instead of directly sending it to the receiver. The receiver picks each job, one by one, and processes it.
Let’s say we are booking movie tickets on a website. There are a few things that should be done immediately, like selecting the seats, payment, etc. But a few things, like sending tickets via email, may take time and are not required to be done immediately. These tasks that time and are not required to be done immediately can be added to a queue and can be processed one by one.
SUBSCRIBE channel
PUBLISH channel message
UNSUBSCRIBE channel
PSUBSCRIBE string*
Redis Security
It is important that the Redis database is secured so that any unwanted user is unable to access the data. Security is also required to restrict an attacker from executing a command, like FLUSHALL, which can delete the entire data.
By default, authentication is not enabled in Redis. It can be enabled through the requirepass configuration. To check if authentication is enabled, a user can run the following command:
config get requirepass
The password field is blank. This means the authentication is not enabled.
config set requirepass tango
Now if a user tries to read or write data to Redis, they will get a NOAUTH authentication required error.
The user can authenticate using the command below:
AUTH password
How to restrict specific commands
Sometimes we may need to provide a user access to our Redis instance. However, we may also want to restrict them from executing any harmful commands. For example, we may need a particular user to read the data, but we don’t want them to be able to add or delete data from the database.
We can provide this type of limited access by renaming the commands that we want to restrict. By renaming the commands, only the user who is aware of the new name will be able to execute the command. Let’s say, for example, we need to restrict few users from executing the flushall command. The admin will rename this command to purgeall. By doing this the only the admin will be able to delete all the keys from the database.
Here are the steps to rename a command:
Add the following command to the redis.conf file.
rename-command flushall purgeall
Run the redis-server.exe, followed by the filename of your config file.
redis-server.exe redis.conf
Now the flushall command will not be available
Transactions in Redis
A transaction is a sequence of commands that are executed atomically and in order.
MULTI command
To start a new transaction, the user should execute the MULTI command. This command tells the Redis server that a transaction has been started.
EXEC command
When a transaction is started, the Redis client queues all commands sent by the user. When the user executes the EXEC command, the client sends the commands to the server. All commands are executed in the order that they were provided.
DISCARD command
If a user wants to end the transaction without executing the commands, the DISCARD command can be used. This will inform the Redis server that the transaction has ended.
When a transaction is running in Redis, no other client can run a query. Suppose a transaction has 70 commands. When EXEC is run, these 70 commands will run in the same order as they were queued. No other client will be able to run any command until these 70 commands are completed.
Redis transactions
The Redis transactions are atomic which means that either all transactions will execute or none will execute. Unlike in a relational database, you cannot rollback Redis transactions. We have five commands that need to be executed, but the second command has an error. Thus, the server will only execute the first, third, fourth, and fifth command.
In the example below, we are sending three commands in a transaction to the Redis server. The second command is incorrect, as we can’t pop an element from a value of String type. When we execute this transaction, the result for the first and third transactions are returned.
Please note that although the second command did not apply on the String value, it was syntactically correct. If we try to queue a syntactically incorrect command, Redis will not execute the transaction. This is shown in the example below.
Optimistic locking in Redis
When a Redis transaction is executing, the Redis server does not accept any queries from other clients. However, it is possible for data to be changed by another client while a user starts a transaction and before a transaction is executed.
Suppose we want to increment the value of a key number by 10. We will start a transaction, add our command to the queue, and then execute the transaction. It is possible that, while we are creating a transaction, another client also increases the value of number by 10. To avoid this, we can use the WATCH command. WATCH means that, before executing our transaction, the Redis server will check if the key we are watching key was modified by another client after we created our transaction. If it was, the Redis server will abort our transaction.
Redis is an in-memory database, which means the data is stored in memory(RAM). In the case of a server crash, all the data stored in the memory is lost. To prevent this, Redis has some mechanism in place to back up the data on the disk. In case of a failure, the data can be loaded from the disk to the memory when the server reboots.
Redis provides two persistence options:
RDB persistence(snapshotting)
Append-only file (AOF) persistence
RDB persistence(snapshotting)
In RDB(Redis Database) persistence, snapshots of the data are stored in a disk in a dump.rdb file. The interval for snapshotting is defined in the configuration file. The default configuration present in the redis.conf file is:
save 900 1
save 300 10
save 60 10000
In the example above, the behavior will be to save data as follows:
After 900 sec (15 min), if at least 1 key changed
After 300 sec (5 min), if at least 10 keys changed
After 60 sec, if at least 10000 keys changed
If the load is too high and a lot of data is being stored, Redis will take a backup every minute. If the load is medium, the backup will be taken after every five minutes and so on.
The durability of data depends upon how frequently the data is being dumped to the disk. If data is dumped after every five minutes, and the server crashes after minutes since the last dump then all the data stored within the last four minutes are lost. So, the duration of backup should be decided carefully.
A user can also save the data in an rdb file using the SAVE command. This command blocks the Redis server, so it should be avoided. Instead, the BGSAVE command, which creates a child process for snapshotting, should be used.
Configuration options in RDB
rdbcompression: If the value of this directive is true, Redis uses LZF compression. By default, this value is true, but, if you want to save some CPU cycles in the child process, then you can make it false.
rdbchecksum: If the value of this directive is yes, a CRC64 checksum is placed at the end of the file. This makes the format more resistant to corruption, but there is a performance hit to pay (around 10%) when saving and loading RDB files, so you can disable it for maximum performance. RDB files created with checksum disabled have a checksum of zero, which will tell the loading code to skip the check.
stop-writes-on-bgsave-error: If this option is true, Redis will stop accepting writes if the latest background save failed. If the background saving process starts working again, Redis will automatically allow writes again. The default value is true. It is advised not to change it to false because if there are some issues in saving the data on the disk that the user must know about it.
dbfilename: This option is used to specify the .rdb filename. The default value is dump.rdb.
Performance impact of snapshotting
Redis creates a child process to write data to the disk. If some data is changed while the child process is writing the data to the disk, the child process needs to make a copy of that data as well. This can be time-consuming, and if the data set is large, it may force Redis to stop serving clients for anywhere between a millisecond to a second. The AOF method, which we will discuss next, is far better in terms of performance.
Append-only file (AOF) persistence
When AOF is enabled, every write operation received by the server is logged into a file. When the Redis is restarted, all the commands in the AOF file are run again, and the entire dataset is created.
After a while, this file can get really big, since it contains the entire history of every key. To tackle this problem, Redis rewrites that file every once in a while to keep it as small as possible. So, instead of storing the entire history of a key, it starts with the latest state of that key.
Let’s say we enter a key count in Redis with the value 1. It is changed multiple times through the set command. The AOF file will look like this:
set count 1
set count 4
set count 12
set count 56
When Redis rewrites this file, it will simply write the following command in the file, as all other commands are not needed.
set count 56
We can configure how often this rewrite happens based on the current file size and the allowed increase percentage since the size of the latest rewrite.
Even though Redis sends the new command to be appended to the file, the OS usually keeps these commands in a buffer and flushes that buffer every X number of seconds. This means that if there is a power failure while the commands are in the buffer, you may lose some data. Redis forces a buffer flush every second so that you don’t lose anything. We can also configure Redis to force flush on every single command, but that makes Redis’s responses per command very slow.
Configuration options in AOF
appendonly: This directive is used to enable the AOF option in Redis. By default, the value is false. To enable AOF, change this valueto true and restart the server.
appendfsync: There is an fsync() call in the operating system, that tells the OS to flush data from the buffer to the disk. The fsync may happen every second or every ten seconds. It is upto the OS. Redis can control this time through the appendfsync directive. It has three options:
no: Let the OS decide when to flush.
always: Flush on every write. This makes Redis slow, but it is the safest option.
everysec- Flush every second.
auto-aof-rewrite-percentage: This directive specifies when an automatic rewrite of an AOF file should happen. If the value of this property is 20, Redis will automatically rewrite the log file by implicitly executing the BGREWRITEAOF command when the AOF size grows by 20 percent. The default value is 100.
Conclusion
Redis uses snapshotting as the default strategy. If you are fine with losing a few seconds of data, then this strategy should be used. If you need complete durability and can’t afford to lose any data, then AOF should be used. You can also use both strategies but Redis will use AOF to populate data, as it is the most accurate one.
When data is stored on a server and there is a server crash, there is a chance that the data will be lost. To avoid this, the technique of data replication is used. The data is stored on two or more servers instead of a single server. That way, if a server goes down, data is not lost since it is available on the other server. Also, the application does not crash because the other server can cater to the user requests. Another benefit of data replication is that it can reduce the load on the servers because the user requests can be load balanced to the servers instead of sending it to a single server.
Replication in Redis
Redis follows a master-slave approach for replication. One of the servers is a master, and the other servers are called slaves. The slaves are connected to the master. All the writes happen to the master and then the master, sends these changes to the slaves. The reads can then be handled by the slaves. Through this process, the load is distributed.
If a slave gets disconnected from the master, it automatically reconnects and tries to be the exact copy of the master. There are two approaches that a slave takes to get in sync with the master.
Partial Synchronization: The slave tries to get only the data that it missed while it was disconnected. The slave sends the master information about the point to which it has the data.
Full Synchronization: If partial synchronization is not possible, then full synchronization takes place. The master sends all the data to an rdb file. This rdb file is sent to the slave. The slave then saves it to its disk. After that is done, the slave loads the data from the rdb file to its memory. There is an option of disk-less replication as well. The child process can directly send the rdb to replicas over the wire, without using the disk as intermediate storage.
Synchronization of master and salves
The master is non-blocking and continues to handle queries when one or more replicas perform the initial synchronization or a partial re-synchronization.
The slave is also non-blocking. When a slave is synchronizing the data, it will also serve the clients from the old data if it is configured to do so.
When a slave gets disconnected from the master, it may or may not serve the clients. It depends on the slave-serve-stale-data configuration.
The slave sends a ping to the master after regular intervals. This tells the master if a slave is still connected or not. By default, a slave sends the ping every ten seconds.
We can configure how much bandwidth is used in syncing the slave by using the repl-disable-tcp-nodelay configuration.
When a slave disconnects from a master, the data being transmitted is saved in a buffer. When a slave reconnects, it gets the data from this buffer. If a slave is disconnected for too long, this buffer may not have all the data that the slave missed, so full sync is required.
If all the slaves are disconnected from the master, then the buffer is not required anymore. It is deleted after a certain period of time.
Each slave has a priority that is used to decide which slave will be promoted to master in case the master crashes. If there are three slaves with priorities 20, 30, and 50 then the slave with priority 20 will be promoted to master. The default priority of a server is 100.
Performance impact of replication in Redis
Since Redis stores data in-memory, it needs to store the data on the disk from time to time in order to protect the data in case of a failure. Storing the data on the disk is a costly operation so it is possible that, due to latency concerns, persistence is disabled on the master server. However, turning the persistence off is not advised unless it is absolutely necessary. If disabling persistence is necessary, the master should be configured to avoid restarting automatically after a reboot.
If persistence is disabled and the server restarts automatically, it may lead to data loss. If this happens, the slaves will try to sync with the master, and their data will also be lost.
How replication is done in Redis?
The replication process in Redis is asynchronous. The master sends the data to be replicated to the slave and does not check if it was actually saved in the slave server. The slave server asynchronously acknowledges the amount of data received periodically from the master. This gives the master an idea of what commands the slaves have processed.
We discussed earlier that if a slave is disconnected and reconnected, it syncs only the data that it missed. The question that arises here how the master knows what data the slave missed. The replication ID and offset are used to figure this out. Each master server has a replication ID that is also inherited by a slave. The master also has an offset that increments for every byte of replication stream that is produced to be sent to replicas. When a slave is reconnected, it sends its offset to the server.
Let’s suppose the master has a replication ID of 123gh3 and an offset as 12,000. The slave has the replication ID of 123gh3 and the offset of 10,000. When the slave connects with the master, it will send its offset of 10,000 to the master. The master will then know which bytes the slave is missing and it will resend those bytes to the slave.
There can be a situation where the offset provided by the slave is too old, and the master does not have that stream in the buffer. In that case, partial sync cannot be done and full sync is required.
How full sync is performed?
Full sync is done through the rdb file. The master creates an rdb file and sends it to the slave. The slave then saves it to the disk and loads the data into the memory. It is possible, however, that some writes are happening on the master after the master has created the rdb file. The master will store all these write operations on a buffer, the commands from the buffer will be sent to the slave after it is done loading the rdb file. Through this process, the slave will have the exact copy of the master.
Provide the slaveof property in the redis.conf file. This property takes two parameters, i.e., the host and the port of the master.
slaveof 127.0.0.1 6379
When this Redis instance is started, it will automatically connect to the master server.
We can use the REPLICAOF command to convert a Redis instance into a slave.
REPLICAOF hostname port will make the server a replica of another server listening at the specified hostname and port.
If a server is already a slave and REPLICAOF hostname port is run on this server with different hosts, it will become the slave of a new master. It will delete the data of the old master and replicate the data of the new master.
REPLICAOF NO ONE will convert a slave into a master. If a master is done, then this command can be used to convert its slave to the master. Later when the old master is up, it can be configured to become a slave.
By default, replicas are read-only. However, this behavior can be controlled using the replica-read-only property in the redis.conf file.
Authenticating a replica with the master
In the , we discussed that we can set a password for the master, and in order for a client to connect to the master, they must know the password. Similarly, a replica(slave) must also be aware of the master password to connect to it. There are two ways through which we can provide the master password to a slave.
If the slave is already running, the following command can be used to authorize it with the server.
config set masterauth <password>
We can also provide the master password in the redis.conf file of slave using the masterauth property, as shown below.
masterauth <password>
What is partitioning?
With replication, we make multiple copies of the data, and each server contains the complete data. But if the data is too large and can’t be saved on a single instance? In that case, we may need to split the data and store it in different instances. This is called partitioning the data.
Partitioning the data will not solve the problem of data safety. If a node goes down, the data stored in that node will be lost. So, replication is required in addition to partitioning. Although it will increase the amount of data that needs to be stored, the data will not be lost in the case of a failure.
Advantages of partitioning
Partitioning helps in horizontal scaling of the data. If the data is spread across different servers, then the traffic can be routed to different servers, and and each server will have less load.
Sometimes it may not be possible to store data on a single server because of its size. In this case, using partitioning is the only option left.
When we increase the memory, hard disk, and computing power of a server, it is called vertical scaling. But when we use the resources of multiple computers, it is called horizontal scaling. Horizontal scaling is more useful because when the data increases, we can add more servers, and when it decreases, we can reduce the servers
Disadvantages of partitioning
If a transaction involves multiple keys and the keys are available on different servers then the transaction is not supported.
Certain operations that involve multiple keys, such as the union of two sets are not possible if the keys are present in different instances.
It is very difficult to handle data when partitioning is used. Multiple rdb or aof files are generated, one for each server and they need to be combined together to get the complete data set.
Basic partitioning techniques
Range Partitioning
This technique can be used if the keys are in number format.
It may lead to an uneven distribution of keys.
Another issue is that if one or more instances are lost, then the data stored on that instance will also be lost. If new instances are added, then we will have to redefine how the range partitioning is calculated.
Hash partitioning
If the key is not a number, we can use a hash function to convert it into a number. We can divide this number by the number of nodes available, and the node can be decided based on the remainder.
The keys are evenly distributed.
The number of nodes must be a prime number to get the most effective partitioning.
If the number of nodes is even, there are more chances of a collision.
If the number of nodes is constantly changing, it will impact the performance as some data will be dumped upon the removal of a node. A technique called presharding is used to avoid this.
What is presharding?
It is very difficult to add or remove instances at runtime. A better approach is to start with a lot of instances instead of just one instance. The Redis instances are very lightweight and one server can store 64 instances. We can start with either 32 or 64 instances, but all the instances will be on the same server.
When the data increases, we can add one more server. Then the instances will be distributed among the two servers. We can keep adding the servers as the data size increases until we have one server per instance. If the data still increases then you may have to add more instances but 64 instances should be sufficient for most applications.
Since Redis stores data in-memory, it is possible that it may give an out of memory error if the memory gets full. We can configure the max memory that Redis can use through the maxmemory property in the redis.conf file. By default, this property is not available in the configuration file. The syntax to define this property in the configuration file is:
maxmemory <bytes>
We also need to define an eviction policy which will be used to evict keys in case the memory gets full.
Here are a few points about memory usage in Redis:
If maxmemory is not defined, Redis will throw an out-of-memory error once the heap is full.
If maxmemory is defined but the eviction policy is set to noeviction, the Redis server will stop taking any writes once the memory is full. It will still respond to get queries.
If some other eviction policy is defined, the keys will be evicted based on the policy.
NOTE: If slaves are connected to the master, then the memory is needed by the buffer(used for storing bytes to be sent to the slave) as well. The memory defined in the maxmemory property does not include that. So while defining maxmemory, please keep in mind the memory needed for buffers.
Eviction policies in Redis
noeviction: As the name suggests, no keys are evicted if the memory is full. If the client tries to insert more data, an error is returned.
allkeys-lru: Evicts the least recently used key. If there are two keys, key1, which was last used five minutes ago, and key2, which was last used six minutes ago, then key2 will be evicted.
allkeys-lfu: Evicts the least frequently used key. If there are two keys, key1, which was used 12 times since its creation, and key2, which was used 15 times since its creation, then key1 will be evicted.
allkeys-random: Evicts the key randomly.
volatile-lru: The term volatile is used for those keys for which an expiry time is set. This policy will evict the least recently used keys out of all the keys with an “expire” field set.
volatile-lfu: This policy will evict the least frequently used keys out of all the keys with an “expire” field set.
volatile-ttl: This policy will evict the least recently used keys with the shortest time to live out of all the keys with an “expire” field set.
volatile-random: Evicts the key randomly out of all the keys marked to be expired.
How are LRU keys determined?
The Least Recently Used and Time To Live algorithm are approximate algorithms for the purpose of saving memory. Redis randomly selects five keys from all the keys and evicts the LRU key out of these five keys. We can increase or decrease this sample size using the maxmemory-samples property in the redis.conf file. If we increase the sample size, then the LRU key found will be more accurate but will take more bandwidth. If it is decreased too much, then the key evicted is less effective but more efficient.
When the client requires data, they ask the Redis server to provide the data. Each back and forth from the server results in some bandwidth consumption, and it takes some time to get the result. It is possible to cache the result of the most frequently used keys on the client-side. This drastically improves the performance of the application, as it does not need to send requests to the database.
Challenges in client-side caching
The biggest challenge faced in caching the data on the client-side is how to invalidate the data. Suppose we have cached some data on the client-side, and it is changed on the server. The client will keep referring to the stale data present in its cache. There should be some mechanism to invalidate the data on the client’s cache if it is changed on the server. There are two mechanisms to deal for this situation:
A TTL (time to live) can be set for each key that is cached on the client-side. After a certain time period has elapsed, the key will be automatically removed from the cache.
Another method is to use the PUB/SUB model of Redis to send invalidate messages to the client. Whenever a key is changed. the server will send invalidate messages to the clients. The problem with this approach is that the server will need to send the message to all clients every time a key is changed, which can be very costly from the bandwidth point of view.
Client-Side caching in Redis
Redis provides support for client-side caching, called tracking.
Default mode
Broadcasting mode
Default mode
In the first approach, the server stores the information regarding which key is stored by which client. By doing this, if a key is changed, the server sends the message to only those clients who have cached that key. Although this saves a lot of bandwidth, it consumes some memory on the server side. The client needs to enable the tracking as it is not enabled by default.
Here are the steps for this approach:
The client sends a read request to the server.
The server checks if this client has enabled tracking. If yes, then the server maintains the record of the key that this client requested in a table called the Invalidation Table.
When the key is changed or expires, the server sends the invalidate message to all clients that have requested this key.
There is a limit to the number of records the table can maintain. If the table is full, the server deletes some mapping and sends an invalidation message to the clients.
In the second approach, the server does not need to keep track of the keys cached by the clients. The clients subscribe to key prefixes and will receive a notification message every time a key matching such prefix is touched. If the client does not specify any prefix, the invalidation message is received for each and every key. This approach saves a lot of memory on the server-side but can result in more bandwidth usage.
It is difficult to handle sharding if the nodes are constantly being added or removed. The Redis cluster is a way to do all of this automatically.
“According to the founder, it is:
“…basically a data sharding strategy, with the ability to reshard keys from one node to another while the cluster is running, together with a failover procedure that makes sure the system is able to survive certain kinds of failures.””
The two major advantages of Redis cluster are:
The ability to automatically split the dataset among multiple nodes.
The ability to continue operations when a subset of the nodes are experiencing failures or are unable to communicate with the rest of the cluster.
Redis cluster working
In Redis cluster, each node has two TCP connections open:
The normal port which is used for client communication, such as 6379. This connection is available to all clients as well as those cluster nodes that need to use client connection for key migration.
The cluster bus port, which is used by other cluster nodes for failure detection, configuration updates, and fail-over authorization. This channel uses less bandwidth by utilizing a binary protocol for data exchange. This port is always 10000 + the normal port. So if the normal port is 6379, then the cluster bus port will be 16379.
Hash slots
Redis cluster uses hash slots to shard the keys. A Redis cluster has 16,383 slots. These slots are distributed among the servers. So, if there are 3 servers in a cluster, then:
Server A contains hash slots from 0 to 5,500.
Server B contains hash slots from 5,501 to 11,000.
Server C contains hash slots from 11,001 to 16,383.
It is not necessary for the slots to be equally distributed among the servers.
Whenever a new server is added or deleted, these slots are redistributed. Moving hash slots from a node to another does not require you to stop operations. Thus, adding and removing nodes, or changing the percentage of hash slots held by nodes does not require any downtime.
When a key is inserted in Redis, the hash slot is found using the CRC-16 hash function to convert the key into an integer. Then modulo 16383 of that integer is calculated to get the hash slot for this key.
HASH_SLOT = CRC16(key) mod 16383
Fault tolerance in Redis cluster
The sharding of data won’t help in the case of a server failure. If a server goes down all the keys stored on that server will be lost. To handle this kind of situation, replication is required. Redis Cluster uses a master-slave model, where every hash slot has anywhere from one (the master itself) to N replicas (N-1 additional slave nodes). To be considered healthy, a cluster should have atleast three master nodes and one replica for each master. So, a cluster should have at-least 6 nodes. M1, M2, and M3 are the master nodes, and S1, S2, and S3 are the replicated nodes.
Redis cluster writes data to the slave node asynchronously. This means that when a key is inserted or updated, the master node sends this information to the slave node but does not wait for acknowledgement. If the master node crashes, it is possible that the slave does not contain all the keys that a client has entered. This problem can be solved by storing the data in a slave node synchronously, but doing this will slow down the server. Thus, Redis cluster makes a trade-off between performance and consistency.
Redis cluster configuration
cluster-enabled: This property should be yes if we need to start the Redis instance in cluster mode. If this property is no, then this instance cannot be used in a Redis cluster.
cluster-config-file: This directive sets the path for the configuration file that stores the changes happening to the cluster. This file is created by Redis and this directive should not be changed by the user. It stores things like the number of nodes in the cluster, their state etc. When a Redis cluster is started, this file is read.
cluster-node-timeout: This property defines the maximum amount of time in milliseconds, that a node can be unavailable without being considered failed.
cluster-slave-validity-factor: If a master is disconnected from the cluster, then a slave tries to fail-over a master. If the value of this property is 0, then a slave will always try to fail-over a master, regardless of the amount of time the link between the master and the slave remains disconnected.
cluster-migration-barrier: This directive defines the minimum number of slaves that a master will remain connected with before another slave migrates to a master that is no longer covered by any slave. Let’s say we have two masters, A and B. A has two slaves, called A1 and A2. B has one slave, called B1. If the master B goes down, then B1 is promoted to master. Now master B does not have any slaves. If cluster-migration-barrier is set to 2, then B1 will not be able to borrow a slave from A because 2 is the minimum requirement.
cluster-require-full-coverage: If a master server fails and it does not have any slaves, then the keys that were stored on this server are lost. If this property is set to yes, as it is by default, then the entire cluster will become unavailable. If the option is set to no, then the cluster will still serve the queries, but the keys that are lost will return an error.
There are two ways to create a Redis cluster, the easy way and the hard way. The easy way is to create a cluster using the create-cluster command. The hard way is to configure the Redis servers to create a cluster manually.
Creating Redis instances
The first step in creating a cluster is to create multiple Redis instances that will be added to the cluster. We will create six instances. Three will be masters, and three will be slaves.
We have created six folders and installed a Redis instance in each folder using the below commands. We have selected the folder names based on the ports that will be used for each server.
$ wget http://download.redis.io/redis-stable.tar.gz
$ tar xzf redis-stable.tar.gz
$ cd redis-stable
$ make
We will need to provide some configuration in the redis.conf file to start the server in cluster mode. Provide the following configuration in the redis.conf file of each instance.
port 7001
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 10000
appendonly yes
Once this configuration is done, open six terminals and start each instance using the following command:
./src/redis-server ./redis.conf
When all six instances are running, our next step is to create a cluster.
Creating a cluster
To create a cluster, run the following command in a new terminal:
The –cluster-replicas 1 option means that each master should have one slave. Since we have six instances, three masters and three slaves will be created.
In the below image, you can see that 3 masters are created and the slots are divided among the masters. Three slaves are also created.
Now that our cluster is created, the next step is to connect to the cluster and execute some commands.
Connecting to a cluster
To connect to a cluster, we will need to connect to one of the nodes. The command to connect to the instance running on port 7001 is shown below. This command is used to indicate that the client should be using the cluster mode.
./src/redis-cli -c -p 7001
The CLUSTER INFO command can be used to check the cluster status as shown below. It shows that the cluster state is ok, and all the slots are assigned.
When a new node is added to a cluster, it is considered a master. It does not have any hash slots assigned, so it cannot store any key. It is up to us to determine which hash slots we need to move to the newly added master. We can move the hash slot from one master to another and then migrate the keys stored on that hash slot.
Creating a new instance
In our cluster, we have six nodes running on ports 7001, 7002, 7003, 7004, 7005 and 7006. We will create a new instance that will be run on port 7007.
Introducing the new instance to the cluster
The CLUSTER MEET command is used to introduce a new instance to the cluster. Run the following command in a new terminal to inform the cluster that a new node is available.
redis-cli -c -p 7001 CLUSTER MEET 127.0.0.1 7007
We have introduced the new instance to the instance running on port 7001. This is enough as we don’t need to introduce the new instance to each node in the cluster.
Reshard the hash slots
We will move the hash slot, 9709, which is currently on the node running at port 7002. To move the hash slot, we will first need to find out the source and destination node id. This can be done using the CLUSTER NODES command.
The steps to migrate a hash slot from one node to another are listed below:
a) Import a hash slot
The receiving node will run a command to inform the cluster that it wishes to import a hash slot. The command for this is:
CLUSTER SETSLOT <hash-slot> IMPORTING <source-id>
Here, hash-slot is the slot that needs to be moved, and source-id is the node id of the node on which this hash slot is currently residing. In our case, we need to import a hash slot from the node running on port 7002. So, we will be running the following command from the node that is going to receive the hash slot.
Here, destination-id is the node id of the destination server.
Since we are moving the slot to a node running on port 7007, we will run the following command:
We have migrated the hash slot, 9709, to the new node. Now it’s time to move the keys that were stored on this slot to the new node. The following command will tell us how many keys are in a particular slot.
CLUSTER COUNTKEYSINSLOT 9707
To migrate the key, we should know the key name as well. This can be found using the following command:
CLUSTER GETKEYSINSLOT <slot> <amount>
Here, amount is the number of key names we want to get.
Once we know the key name, we can run the following command to migrate the key.
MIGRATE <host> <port> <key> <db> <timeout>
Informing all the nodes about hash slot migration
All the nodes in the cluster should be informed of the new location of the hash slot. This is required so that other nodes can look for the key at the correct location. The command for this operation is:
CLUSTER SETSLOT <hash-slot> NODE <owner-id>
Here, owner-id is the node id of the instance where the hash slot has been moved.
This command should only be run for master nodes.
$ redis-cli -c -p 7001 CLUSTER SETSLOT 9709 NODE
2206481813a1cbe55043f92b05726290737adb7b
$ redis-cli -c -p 7002 CLUSTER SETSLOT 9709 NODE
2206481813a1cbe55043f92b05726290737adb7b
$ redis-cli -c -p 7003 CLUSTER SETSLOT 9709 NODE
2206481813a1cbe55043f92b05726290737adb7b
Removing nodes from a cluster
To remove a node, first, we need to migrate all its hash slots to different nodes. A node can only be removed from a cluster when there are no hash slots assigned to it.
After all the slots are migrated from a node, then the following command should be run on all the master nodes.
CLUSTER FORGET <node-id>
As soon as CLUSTER FORGET is executed, the node is added to a ban list. This ban list exists in order to avoid adding the node back to the cluster when nodes exchange messages. The expiration time for the ban list is 60 seconds, so the above command should be executed in all of the master servers within 60 seconds.
Connection pooling means that connections are reused rather than created each time when the connection is requested. To facilitate connection reuse, a memory cache of database connections, called a connection pool, is maintained by a connection pooling module as a layer.
There are some concerns when using Redis with a multi-threaded environment. Redis connection instance is single-threaded. If we reuse a single Redis connection between multiple threads, it may not be enough to complete the required tasks in a limited time. If we create a separate Redis connection for a thread it will be overhead for the Redis server.
We need to have a pool of connections in multi-threaded environments to address these challenges. This allows us to talk to Redis from multiple threads while still getting the benefits of reused connections.
The idea is, take the connection from the pool and release it back to the pool once we have done. This pool should be configured once and can be reused many times.
