使用 redis-cli 搭建 Redis 集群

参考：Redis 集群教程

如何启动一个 Redis 实例

$ redis-server /path/to/redis.conf

redis.conf 文件中包含很多信息，如：端口号、持久化方式、持久化的文件等等。

启动多个 Redis 实例

使用写入了不同端口号的配置文件就可以启动多个 Redis 实例。
下面是一个最少选项的集群的配置文件:

port 7000
cluster-enabled yes
cluster-config-file nodes.7000.conf
cluster-node-timeout 5000
appendonly yes

文件中的 cluster-enabled 选项用于开实例的集群模式，而 cluster-conf-file 选项则设定了保存节点配置文件的路径，默认值为 nodes.conf。节点配置文件无须人为修改，它由 Redis 集群在启动时创建，并在有需要时自动进行更新。

要让集群正常运作至少需要三个主节点，不过在刚开始试用集群功能时，强烈建议使用六个节点：其中三个为主节点，而其余三个则是各个主节点的从节点。

首先，让我们进入一个新目录，并创建六个以端口号为名字的子目录，稍后我们在将每个目录中运行一个 Redis 实例：命令如下:

mkdir cluster-test
cd cluster-test
mkdir 7000 7001 7002 7003 7004 7005

在文件夹 7000 至 7005 中，各创建一个 redis.conf 文件，文件的内容可以使用上面的示例配置文件，但记得将配置中的 port 和 cluster-conf-file 中的端口号数字从 7000 改为与文件夹名字相同的号码。不同的集群节点要使用不同的 cluster-conf-file。
配置文件的路径是可以自定义的。创建完毕后分别启动一个实例。

创建 Redis 集群

使用 redis-trib.rb

网上看到的教程，包括参考的官方文档里的文章，大多是使用以下方式创建集群。

$ ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

这个命令在这里用于创建一个新的集群, 选项–replicas 1 表示我们希望为集群中的每个主节点创建一个从节点。
之后跟着的其他参数则是这个集群实例的地址列表，3 个 master 3 个 slave redis-trib 会打印出一份预想中的配置给你看，如果你觉得没问题的话，就可以输入 yes，redis-trib 就会将这份配置应用到集群当中，让各个节点开始互相通讯，最后可以得到如下信息：

[OK] All 16384 slots covered

这表示集群中的 16384 个槽都有至少一个主节点在处理，集群运作正常。

但是在 Redis 的 github 仓库中看到，该文件已经不建议使用。

redis-trib.rb 不建议使用

使用 redis-cli --cluster

我在手把手教你实现 Docker 部署 Redis 集群的评论中看到，现在 redis-cli --cluster 命令已经可以创建集群，分配槽，分配主从服务器了，于是使用以下命令了解到相关的命令。

$ redis-cli help
……
Cluster Manager Commands:
  Use --cluster help to list all available cluster manager commands.
……

$ redis-cli --cluster help
Cluster Manager Commands:
  create         host1:port1 ... hostN:portN
                 --cluster-replicas

可以看到，命令的组成形式和旧方式是一致的。

创建集群

$ redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7004 to 127.0.0.1:7000
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7003 to 127.0.0.1:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 588adadc1d226bd90e40a0ea0a0e7a959fcea8f0 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
M: ffad01907b25114fa2f64fc5235936914ddc91c6 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
M: d3fd5a876755fd5df7f9a15550c8b0b63a40c7c8 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
S: 98f9e3ac353f115640b6f6dc000b6b8e9bf6ebd0 127.0.0.1:7003
   replicates 588adadc1d226bd90e40a0ea0a0e7a959fcea8f0
S: 1df572376a8b54d78056a9f92a5baf9454694fd6 127.0.0.1:7004
   replicates ffad01907b25114fa2f64fc5235936914ddc91c6
S: 58e3ea1427cecafb50855bfb003714e3a0a9a852 127.0.0.1:7005
   replicates d3fd5a876755fd5df7f9a15550c8b0b63a40c7c8
Can I set the above configuration? (type 'yes' to accept):

可以看到执行命令后，redis 客户端做了以下工作：

在6个节点上分配 Hash 槽。
在节点间构建主从通过关系。
Trying to optimize slaves allocation for anti-affinity（不知道啥意思）

输入 yes 后，redis 客户端做了以下工作：

节点的配置更新
Assign a different config epoch to each node
发送 CLUSTER MEET 消息通知节点加入集群
执行 cluster check

>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 588adadc1d226bd90e40a0ea0a0e7a959fcea8f0 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 1df572376a8b54d78056a9f92a5baf9454694fd6 127.0.0.1:7004
   slots: (0 slots) slave
   replicates ffad01907b25114fa2f64fc5235936914ddc91c6
M: d3fd5a876755fd5df7f9a15550c8b0b63a40c7c8 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 58e3ea1427cecafb50855bfb003714e3a0a9a852 127.0.0.1:7005
   slots: (0 slots) slave
   replicates d3fd5a876755fd5df7f9a15550c8b0b63a40c7c8
S: 98f9e3ac353f115640b6f6dc000b6b8e9bf6ebd0 127.0.0.1:7003
   slots: (0 slots) slave
   replicates 588adadc1d226bd90e40a0ea0a0e7a959fcea8f0
M: ffad01907b25114fa2f64fc5235936914ddc91c6 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

集群搭建完毕。

查看效果

停止一个 Master 实例，对应的 Slave 实例中会有一个被提升为 Master

70135:S 24 Mar 2020 18:12:41.164 # Connection with master lost.
70135:S 24 Mar 2020 18:12:41.164 * Caching the disconnected master state.
70135:S 24 Mar 2020 18:12:41.889 * Connecting to MASTER 127.0.0.1:7001
70135:S 24 Mar 2020 18:12:41.889 * MASTER <-> REPLICA sync started
70135:S 24 Mar 2020 18:12:41.889 # Error condition on socket for SYNC: Connection refused
70135:S 24 Mar 2020 18:12:42.927 * Connecting to MASTER 127.0.0.1:7001
70135:S 24 Mar 2020 18:12:42.927 * MASTER <-> REPLICA sync started
70135:S 24 Mar 2020 18:12:42.928 # Error condition on socket for SYNC: Connection refused
70135:S 24 Mar 2020 18:12:43.966 * Connecting to MASTER 127.0.0.1:7001
70135:S 24 Mar 2020 18:12:43.967 * MASTER <-> REPLICA sync started
70135:S 24 Mar 2020 18:12:43.967 # Error condition on socket for SYNC: Connection refused
70135:S 24 Mar 2020 18:12:45.005 * Connecting to MASTER 127.0.0.1:7001
70135:S 24 Mar 2020 18:12:45.005 * MASTER <-> REPLICA sync started
70135:S 24 Mar 2020 18:12:45.005 # Error condition on socket for SYNC: Connection refused
70135:S 24 Mar 2020 18:12:46.035 * Connecting to MASTER 127.0.0.1:7001
70135:S 24 Mar 2020 18:12:46.035 * MASTER <-> REPLICA sync started
70135:S 24 Mar 2020 18:12:46.035 # Error condition on socket for SYNC: Connection refused
70135:S 24 Mar 2020 18:12:46.556 * FAIL message received from 588adadc1d226bd90e40a0ea0a0e7a959fcea8f0 about ffad01907b25114fa2f64fc5235936914ddc91c6
70135:S 24 Mar 2020 18:12:46.556 # Cluster state changed: fail
70135:S 24 Mar 2020 18:12:46.660 # Start of election delayed for 601 milliseconds (rank #0, offset 8190).
70135:S 24 Mar 2020 18:12:47.077 * Connecting to MASTER 127.0.0.1:7001
70135:S 24 Mar 2020 18:12:47.077 * MASTER <-> REPLICA sync started
70135:S 24 Mar 2020 18:12:47.078 # Error condition on socket for SYNC: Connection refused
70135:S 24 Mar 2020 18:12:47.283 # Starting a failover election for epoch 7.
70135:S 24 Mar 2020 18:12:47.286 # Failover election won: I'm the new master.
70135:S 24 Mar 2020 18:12:47.286 # configEpoch set to 7 after successful failover
70135:M 24 Mar 2020 18:12:47.286 # Setting secondary replication ID to 2e98e253879709e56d5ab44408ac57ea55d94aa6, valid up to offset: 8191. New replication ID is 3faa1f7d8d26d944219cffc2f5a5ab70f863d5cb
70135:M 24 Mar 2020 18:12:47.286 * Discarding previously cached master state.
70135:M 24 Mar 2020 18:12:47.287 # Cluster state changed: ok

重启该实例，会连接上顶替它成为 Master 的实例成为它的 Slave

72750:S 24 Mar 2020 18:15:42.949 # Cluster state changed: ok
72750:S 24 Mar 2020 18:15:43.985 * Connecting to MASTER 127.0.0.1:7004
72750:S 24 Mar 2020 18:15:43.985 * MASTER <-> REPLICA sync started
72750:S 24 Mar 2020 18:15:43.986 * Non blocking connect for SYNC fired the event.
72750:S 24 Mar 2020 18:15:43.986 * Master replied to PING, replication can continue...
72750:S 24 Mar 2020 18:15:43.986 * Trying a partial resynchronization (request 5c682ea81c1284fccb3a75bf4ddd734bf61b3d49:1).
72750:S 24 Mar 2020 18:15:43.987 * Full resync from master: 3faa1f7d8d26d944219cffc2f5a5ab70f863d5cb:8190
72750:S 24 Mar 2020 18:15:43.988 * Discarding previously cached master state.
72750:S 24 Mar 2020 18:15:44.031 * MASTER <-> REPLICA sync: receiving 192 bytes from master
72750:S 24 Mar 2020 18:15:44.031 * MASTER <-> REPLICA sync: Flushing old data
72750:S 24 Mar 2020 18:15:44.032 * MASTER <-> REPLICA sync: Loading DB in memory
72750:S 24 Mar 2020 18:15:44.032 * MASTER <-> REPLICA sync: Finished with success
72750:S 24 Mar 2020 18:15:44.032 * Background append only file rewriting started by pid 72752
72750:S 24 Mar 2020 18:15:44.056 * AOF rewrite child asks to stop sending diffs.
72752:C 24 Mar 2020 18:15:44.056 * Parent agreed to stop sending diffs. Finalizing AOF...
72752:C 24 Mar 2020 18:15:44.056 * Concatenating 0.00 MB of AOF diff received from parent.
72752:C 24 Mar 2020 18:15:44.056 * SYNC append only file rewrite performed
72750:S 24 Mar 2020 18:15:44.089 * Background AOF rewrite terminated with success
72750:S 24 Mar 2020 18:15:44.089 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
72750:S 24 Mar 2020 18:15:44.089 * Background AOF rewrite finished successfully

整个过程中，其它实例的输出如下：

70131:M 24 Mar 2020 18:12:46.556 * Marking node ffad01907b25114fa2f64fc5235936914ddc91c6 as failing (quorum reached).
70131:M 24 Mar 2020 18:12:46.556 # Cluster state changed: fail
70131:M 24 Mar 2020 18:12:47.285 # Failover auth granted to 1df572376a8b54d78056a9f92a5baf9454694fd6 for epoch 7
70131:M 24 Mar 2020 18:12:47.288 # Cluster state changed: ok
70131:M 24 Mar 2020 18:15:42.993 * Clear FAIL state for node ffad01907b25114fa2f64fc5235936914ddc91c6: master without slots is reachable again.

如果一对主从全挂了，此时写入：

(error) CLUSTERDOWN The cluster is down

因为 Redis Cluster 默认要求所有的槽位被覆盖，可以通过修改 cluster-require-full-coverage yes 配置来改变该行为。
CLUSTERDOWN The cluster is down in redis 这里的回答中提到：我们可以使用 N 个Master 和 N+1 个 Slave，正常情况下多余的一个实例作为随机一个 Master 的 Slave，一旦有实例宕机，可以迅速顶替，以保证每个主节点总是有至少一个从节点保持数据同步。

ps：搜索命令行的输出时，才看到深入理解Redis系列之集群环境搭建这篇文章，有时候搜索的关键词不合适容易走弯路啊……