Redis集群新增节点及移除节点

Redis集群新增节点及移除节点

前提：

此处以redis-5.0.9，redis-3.3.5.gem搭建的3主3从的redis集群，后续将新增主节点7007和从节点7008加入集群，最后再移除主节点7007和从节点7008

root     29813     1  0 09:12 ?        00:00:01 ./redis-server 127.0.0.1:7001 [cluster]
root     29821     1  0 09:13 ?        00:00:01 ./redis-server 127.0.0.1:7002 [cluster]
root     29827     1  0 09:13 ?        00:00:01 ./redis-server 127.0.0.1:7003 [cluster]
root     29832     1  0 09:13 ?        00:00:01 ./redis-server 127.0.0.1:7004 [cluster]
root     29837     1  0 09:13 ?        00:00:01 ./redis-server 127.0.0.1:7005 [cluster]
root     29850     1  0 09:13 ?        00:00:01 ./redis-server 127.0.0.1:7006 [cluster]
127.0.0.1:7002> CLUSTER nodes
cda5d93dbbf640ef75891e3cdb48d5f294e0e2e1 127.0.0.1:7006@17006 slave b3aaff67a3259bfa424449b5057ddd165e 7f49cc 0 1603157901000 6 connected
b3aaff67a3259bfa424449b5057ddd165e7f49cc 127.0.0.1:7002@17002 myself,master - 0 1603157902000 2 connec ted 5461-10922
bb1c3c68dfca2443f66b238bff0a789061ff0542 127.0.0.1:7001@17001 master - 0 1603157901000 1 connected 0-5 460
e2383dd1d6b9ed7972d18d015aa603e31fa6167b 127.0.0.1:7005@17005 slave bb1c3c68dfca2443f66b238bff0a789061 ff0542 0 1603157902935 5 connected
36e88ea5edc9f77efe0d2d6d2b9c112889824b75 127.0.0.1:7003@17003 master - 0 1603157901933 3 connected 109 23-16383
3488ea3da675e052a9bce0f50d55fa566d55f1eb 127.0.0.1:7004@17004 slave 36e88ea5edc9f77efe0d2d6d2b9c112889 824b75 0 1603157900931 4 connected
127.0.0.1:7002> CLUSTER info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:2
cluster_stats_messages_ping_sent:1174
cluster_stats_messages_pong_sent:1057
cluster_stats_messages_meet_sent:3
cluster_stats_messages_sent:2234
cluster_stats_messages_ping_received:1055
cluster_stats_messages_pong_received:1177
cluster_stats_messages_meet_received:2
cluster_stats_messages_received:2234

新增7007和7008节点，并启动

root     30461     1  0 09:32 ?        00:00:00 ./redis-server 127.0.0.1:7007 [cluster]
root     30467     1  0 09:32 ?        00:00:00 ./redis-server 127.0.0.1:7008 [cluster]

新增7007节点加入集群

命令注意：第一个ip:port 为新增的节点ip和端口，第二个ip:port为集群中的任意节点和端口
./redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7001

[root@localhost 7008]# ./redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7001
>>> Adding node 127.0.0.1:7007 to cluster 127.0.0.1:7001
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: bb1c3c68dfca2443f66b238bff0a789061ff0542 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: cda5d93dbbf640ef75891e3cdb48d5f294e0e2e1 127.0.0.1:7006
   slots: (0 slots) slave
   replicates b3aaff67a3259bfa424449b5057ddd165e7f49cc
S: e2383dd1d6b9ed7972d18d015aa603e31fa6167b 127.0.0.1:7005
   slots: (0 slots) slave
   replicates bb1c3c68dfca2443f66b238bff0a789061ff0542
S: 3488ea3da675e052a9bce0f50d55fa566d55f1eb 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 36e88ea5edc9f77efe0d2d6d2b9c112889824b75
M: b3aaff67a3259bfa424449b5057ddd165e7f49cc 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: 36e88ea5edc9f77efe0d2d6d2b9c112889824b75 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7007 to make it join the cluster.
[OK] New node added correctly.
[root@localhost 7008]#

查看当前集群信息

127.0.0.1:7002> CLUSTER nodes
d9c31d3c8cf1b3a22d85668706ac8f2335bac862 127.0.0.1:7007@17007 master - 0 1603163414000 7 connected
cda5d93dbbf640ef75891e3cdb48d5f294e0e2e1 127.0.0.1:7006@17006 slave b3aaff67a3259bfa424449b5057ddd165e7f49cc 0 1603163413000 6 connected
b3aaff67a3259bfa424449b5057ddd165e7f49cc 127.0.0.1:7002@17002 myself,master - 0 1603163411000 2 connected 5461-10922
bb1c3c68dfca2443f66b238bff0a789061ff0542 127.0.0.1:7001@17001 master - 0 1603163412000 1 connected 0-5460
e2383dd1d6b9ed7972d18d015aa603e31fa6167b 127.0.0.1:7005@17005 slave bb1c3c68dfca2443f66b238bff0a789061ff0542 0 1603163411787 5 connected
36e88ea5edc9f77efe0d2d6d2b9c112889824b75 127.0.0.1:7003@17003 master - 0 1603163413000 3 connected 10923-16383
3488ea3da675e052a9bce0f50d55fa566d55f1eb 127.0.0.1:7004@17004 slave 36e88ea5edc9f77efe0d2d6d2b9c112889824b75 0 1603163414792 4 connected
127.0.0.1:7002>

但是7007目前是没有哈希槽的。也不会存放数据。

主节点7007重新分配哈希槽

当前是三个主节点7001/7002/7003，新增7007后，变成4个主节点，总槽数16384除以4=4096槽，所以7007分配4096个哈希槽。

[root@localhost src]# redis-cli --cluster reshard 127.0.0.1:7001

再次查看集群信息，当前集群已经是4主3从

新增集群节点前，提前设置了10个key-value值，现在去7007节点查看确认一下，是否有数据分配

此处显示10个key都存在，查询k2，显示value值正常。

新增7008从节点加入集群

[root@localhost 7008]# ./redis-cli --cluster add-node 127.0.0.1:7008 127.0.0.1:7001
>>> Adding node 127.0.0.1:7008 to cluster 127.0.0.1:7001
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: bb1c3c68dfca2443f66b238bff0a789061ff0542 127.0.0.1:7001
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
S: cda5d93dbbf640ef75891e3cdb48d5f294e0e2e1 127.0.0.1:7006
   slots: (0 slots) slave
   replicates b3aaff67a3259bfa424449b5057ddd165e7f49cc
S: e2383dd1d6b9ed7972d18d015aa603e31fa6167b 127.0.0.1:7005
   slots: (0 slots) slave
   replicates bb1c3c68dfca2443f66b238bff0a789061ff0542
S: 3488ea3da675e052a9bce0f50d55fa566d55f1eb 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 36e88ea5edc9f77efe0d2d6d2b9c112889824b75
M: d9c31d3c8cf1b3a22d85668706ac8f2335bac862 127.0.0.1:7007
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
M: b3aaff67a3259bfa424449b5057ddd165e7f49cc 127.0.0.1:7002
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
M: 36e88ea5edc9f77efe0d2d6d2b9c112889824b75 127.0.0.1:7003
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7008 to make it join the cluster.
[OK] New node added correctly.
[root@localhost 7008]#

当前7008节点还是一个主节点，但没有哈希槽

将7008变成集群从节点

命令注意：
./redis-cli -p 7008 -c
###CLUSTER REPLICATE [主节点ID]
> CLUSTER REPLICATE d9c31d3c8cf1b3a22d85668706ac8f2335bac862  //此处填写主节点7007的节点ID

[root@localhost 7007]# ./redis-cli -p 7008 -c
127.0.0.1:7008> CLUSTER nodes
36e88ea5edc9f77efe0d2d6d2b9c112889824b75 127.0.0.1:7003@17003 master - 0 1603165509000 3 connected 12288-16383
e2383dd1d6b9ed7972d18d015aa603e31fa6167b 127.0.0.1:7005@17005 slave bb1c3c68dfca2443f66b238bff0a789061ff0542 0 1603165510355 1 connected
bb1c3c68dfca2443f66b238bff0a789061ff0542 127.0.0.1:7001@17001 master - 0 1603165507000 1 connected 1365-5460
d9c31d3c8cf1b3a22d85668706ac8f2335bac862 127.0.0.1:7007@17007 master - 0 1603165507349 7 connected 0-1364 5461-6826 10923-12287
3488ea3da675e052a9bce0f50d55fa566d55f1eb 127.0.0.1:7004@17004 slave 36e88ea5edc9f77efe0d2d6d2b9c112889824b75 0 1603165509000 3 connected
b3aaff67a3259bfa424449b5057ddd165e7f49cc 127.0.0.1:7002@17002 master - 0 1603165507000 2 connected 6827-10922
67d81a8ea6ff93b889a9f4926102c4269cb9b0b8 127.0.0.1:7008@17008 myself,master - 0 1603165508000 0 connected
cda5d93dbbf640ef75891e3cdb48d5f294e0e2e1 127.0.0.1:7006@17006 slave b3aaff67a3259bfa424449b5057ddd165e7f49cc 0 1603165509354 2 connected
127.0.0.1:7008>
127.0.0.1:7008> CLUSTER REPLICATE d9c31d3c8cf1b3a22d85668706ac8f2335bac862
OK
127.0.0.1:7008>

再次查看集群状态

7008节点已经变成7007的从节点了。
至此，新增主从节点完成。

移除集群节点部分

移除从节点7008，从节点去除很简单，因为并未分配哈希槽，所以直接移除即可。

去除命令：
 ./redis-cli --cluster del-node [节点ip]:[端口] [节点ID]
 ./redis-cli --cluster del-node 127.0.0.1:7008 67d81a8ea6ff93b889a9f4926102c4269cb9b0b8

查看集群状态

7008节点已移除，移除7008节点的同时，会停止7008节点的redis服务；

##########################################################
移除主节点7007，因为主节点上分配有哈希槽，所以要先将哈希槽重新分配给其他主节点，再进行移除操作。

[root@localhost 7007]# ./redis-cli --cluster reshard 127.0.0.1:7007
>>> Performing Cluster Check (using node 127.0.0.1:7007)
M: d9c31d3c8cf1b3a22d85668706ac8f2335bac862 127.0.0.1:7007
  slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
M: b3aaff67a3259bfa424449b5057ddd165e7f49cc 127.0.0.1:7002
  slots:[6827-10922] (4096 slots) master
  1 additional replica(s)
S: cda5d93dbbf640ef75891e3cdb48d5f294e0e2e1 127.0.0.1:7006
  slots: (0 slots) slave
  replicates b3aaff67a3259bfa424449b5057ddd165e7f49cc
S: 3488ea3da675e052a9bce0f50d55fa566d55f1eb 127.0.0.1:7004
  slots: (0 slots) slave
  replicates 36e88ea5edc9f77efe0d2d6d2b9c112889824b75
S: e2383dd1d6b9ed7972d18d015aa603e31fa6167b 127.0.0.1:7005
  slots: (0 slots) slave
  replicates bb1c3c68dfca2443f66b238bff0a789061ff0542
M: 36e88ea5edc9f77efe0d2d6d2b9c112889824b75 127.0.0.1:7003
  slots:[12288-16383] (4096 slots) master
  1 additional replica(s)
M: bb1c3c68dfca2443f66b238bff0a789061ff0542 127.0.0.1:7001
  slots:[1365-5460] (4096 slots) master
  1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)?

哈希槽迁移完后，再次查看集群信息

此时7007节点已经没有分配哈希槽，也没有key-value信息。

移除7007节点

最后查看集群信息

7001接收了7007的所有哈希槽，现在一共是8192个哈希槽。最后发现，移除主节点的时候，哈希槽并不能均匀分配到其他主节点，只能单节点接收。