Redis Replication

Redis replication is the process that allows Redis instances to be exact copies of master instances. Replication by default is an asynchronous process. Redis replication is non-blocking on the master side as well as on the replica side. This means that the master will continue to handle queries when one or more replicas perform the initial synchronization or a partial resynchronization. And same on the replica side, during replication, it can handle queries using the old version of the dataset.

Read replicas improve read throughput and guard against data loss in cases of node failure. Replication can be used both for scalability, in order to have multiple replicas for read-only queries, or simply for improving data safety and high availability.

Apart from being read-only by default, one important distinction between master and replica is that Replicas don't expire/evict keys, they wait for masters to expire the keys and when the master expires or evicts a key, replica synthesizes a DEL command which is transmitted to all the replicas.

Replication uses three main mechanisms:

1. When a master and replica instances are well-connected, the master keeps the replica updated by sending a stream of commands to the replica to replicate the effects on the dataset happening on the master side due to: client writes, keys expired or evicted, any other action changing the master dataset.
2. When the link between the master and the replica breaks, for network issues or because a timeout is sensed in the master or the replica, the replica reconnects and attempts to proceed with a partial resynchronization: it means that it will try to just obtain the part of the stream of commands it missed during the disconnection.
3. When a partial resynchronization is not possible, the replica will ask for a full resynchronization. This will involve a more complex process in which the master needs to create a snapshot of all its data, send it to the replica, and then continue sending the stream of commands as the dataset changes.

Configuration

To configure basic Redis replication is trivial: just add the following line to the replica configuration file:

replicaof <redis master IP>

How replication happens

Master and replica synchronize each other with replicationID and offset. You can use info replication command to check master's and replica's offset and replicationID.

> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.31.2.117,port=6379,state=online,offset=5491126744,lag=1
master_replid:ba06e48871ad41ba08bd33fd77a9b4cdf4c5c705
master_replid2:55b76e9a9968c9e802ae5aaab4071c556dcea9e0
master_repl_offset:5491126797

> info replication
# Replication
role:slave
master_host:10.0.22.226
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:5491126465
repl_sync_enabled:1
slave_read_reploff:5491126465
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:ba06e48871ad41ba08bd33fd77a9b4cdf4c5c705
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:5491126465

When replicas connect to masters, they use the PSYNC command in order to send their old master replication ID and the offsets they processed so far. This way the master can send just the incremental part needed. However, if there is not enough backlog in the master buffers, or if the replica is referring to a history (replication ID) that is no longer known, then a full resynchronization happens: in this case, the replica will get a full copy of the dataset, from scratch.

How does full resynchronization happen?  

The master starts a background saving process to produce an RDB file. At the same time, it starts to buffer all new write commands received from the clients. When the background saving is complete, the master transfers the database file to the replica, which saves it on disk, and then loads it into memory. The master will then send all buffered commands to the replica. This is done as a stream of commands and is in the same format as the Redis protocol itself.

Since full resynchronization creates an RDB file on disk, this might create a performance bottleneck on both master and slow in case of slow IO. To fix this, Redis supports diskless replication in case of full-resynchronization from version 2.8.18. In this setup, the child process directly sends the RDB over the wire to replicas, without using the disk as intermediate storage.

Resources: