Redis集群策略及集群实例（集群节点新增、删除、重新分配slot实战）
redis 集群配置示例
集群环境搭建，测试，添加节点, 重新分片, 删除节点
前沿:Redis Cluster（Redis集群）简介
redis是一个开源的key value存储系统，受到了广大互联网公司的青睐。redis3.0版本之前只支持单例模式，在3.0版本及以后才支持集群，我这里用的是redis3.0.0版本；
redis集群采用P2P模式，是完全去中心化的，不存在中心节点或者代理节点；
redis集群是没有统一的入口的，客户端（client）连接集群的时候连接集群中的任意节点（node）即可，集群内部的节点是相互通信的（PING-PONG机制），每个节点都是一个redis实例；
为了实现集群的高可用，即判断节点是否健康（能否正常使用），redis-cluster有这么一个投票容错机制：如果集群中超过半数的节点投票认为某个节点挂了，那么这个节点就挂了（fail）。这是判断节点是否挂了的方法；
那么如何判断集群是否挂了呢? -> 如果集群中任意一个节点挂了，而且该节点没有从节点（备份节点），那么这个集群就挂了。这是判断集群是否挂了的方法；
那么为什么任意一个节点挂了（没有从节点）这个集群就挂了呢？ -> 因为集群内置了16384个slot（哈希槽），并且把所有的物理节点映射到了这16384[0-16383]个slot上，或者说把这些slot均等的分配给了各个节点。当需要在Redis集群存放一个数据（key-value）时，redis会先对这个key进行crc16算法，然后得到一个结果。再把这个结果对16384进行求余，这个余数会对应[0-16383]其中一个槽，进而决定key-value存储到哪个节点中。所以一旦某个节点挂了，该节点对应的slot就无法使用，那么就会导致集群无法正常工作。
综上所述，每个Redis集群理论上最多可以有16384个节点。
以上原文链接：https://blog.csdn.net/qq_42815754/article/details/82912130
1. 环境
（1）VMware+Centos7 安装教程https://blog.csdn.net/babyxue/article/details/80970526
（2）虚拟机网络配置传送门：网络配置
（2）redis-4.0.10官网下载地址
2. 安装
（1）安装前需要先安装必要的包，yum -y install gcc gcc-c++ tcl，这里安装的都是一般遇到的错误所需要安装的库。
（2）将下载的redis拷贝到服务器解压tar -zxvf redis-4.0.10.tar.gz，
（3）进入redis-4.0.10目录，执行编译命令：make MALLOC=libc
[root@localhost local]#tar -zxvf redis-4.0.10.tar.gz
[root@localhost local]# cd redis-4.0.10/
[root@localhost local]# make MALLOC=libc
安装完可以直接启动测试一下：
默认配置文件redis.conf 端口6379 无密码验证
启动服务器，执行下面的命令：./src/redis-server redis.conf
启动客户端，执行下面的命令：./src/redis-cli
[root@localhost redis-4.0.10]# src/redis-server redis.conf
11241:C 13 Feb 19:39:25.541 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
11241:C 13 Feb 19:39:25.541 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=11241, just started
11241:C 13 Feb 19:39:25.541 # Configuration loaded
[root@localhost redis-4.0.10]# src/redis-cli
127.0.0.1:6379> set yy 123
OK
127.0.0.1:6379> get yy
"123"
127.0.0.1:6379>
以上显示证明我们的redis安装完成！
3. 集群
redis包含三种集群策略：
(1).主从复制（Master-Slave）
(2) .哨兵模式
(3).redis-cluster
三种策略具体差异优略自行查
本次集群测试只针对 redis-cluster。
Redis Cluster集群至少需要三个master节点，本文将以单机多实例的方式部署3个主节点及3个从节点，6个节点实例分别使用不同的端口及工作目录。
（一）首先，在redis安装目录/usr/local/下新建目录redis-cluster，并在该目录下再新建6个子目录，8001,8002,8003,8004,8005,8006，此时目录结构如下图所示
[root@localhost local]# mkdir redis-cluster
[root@localhost local]# cd redis-cluster/
[root@localhost redis-cluster]# mkdir 8001 8002 8003 8004 8005 8006
[root@localhost redis-cluster]# pwd
/usr/local/redis-cluster
[root@localhost redis-cluster]# ll
total 0
drwxr-xr-x. 2 root root 6 Feb 13 20:20 8001
drwxr-xr-x. 2 root root 6 Feb 13 20:20 8002
drwxr-xr-x. 2 root root 6 Feb 13 20:20 8003
drwxr-xr-x. 2 root root 6 Feb 13 20:20 8004
drwxr-xr-x. 2 root root 6 Feb 13 20:20 8005
drwxr-xr-x. 2 root root 6 Feb 13 20:20 8006
[root@localhost redis-cluster]#
（二）将redis安装目录/usr/local/redis-4.0.6下的redis.conf文件拷贝至8001文件夹
[root@localhost redis-4.0.10]# cp redis.conf ../redis-cluster/8001
[root@localhost redis-4.0.10]# cd ../redis-cluster/8001
[root@localhost 8001]# ll
total 60
-rw-r--r--. 1 root root 58756 Feb 13 20:32 redis.conf
并修改以下配置==修改的内容有
daemonize yes #开启后台运行
port 8001 #工作端口
bind 172.16.0.15 #绑定机器的内网IP,一定要设置呀老铁，不要用127.0.0.1
dir /usr/local/redis-cluster/8001/ #指定工作目录，rdb,aof持久化文件将会放在该目录下，不同实例一定要配置不同的工作目录
cluster-enabled yes #启用集群模式
cluster-config-file nodes-8001.conf #生成的集群配置文件名称，集群搭建成功后会自动生成，在工作目录下
cluster-node-timeout 5000 #节点宕机发现时间，可以理解为主节点宕机后从节点升级为主节点时间
appendonly yes #开启AOF模式
pidfile /var/run/redis_8001.pid #pid file所在目录
（三）把8001文件夹下的redis.conf文件拷贝到其他5个目录，并重新修改port 、dir、cluster-config-file pidfile三个属性:
快速修改命令如下
[root@localhost redis-cluster]# sed -i 's/8001/8002/g' 8002/redis.conf
[root@localhost redis-cluster]# sed -i 's/8001/8003/g' 8003/redis.conf
[root@localhost redis-cluster]# sed -i 's/8001/8004/g' 8004/redis.conf
[root@localhost redis-cluster]# sed -i 's/8001/8005/g' 8005/redis.conf
[root@localhost redis-cluster]# sed -i 's/8001/8006/g' 8006/redis.conf
由于创建集群需要用到redis-trib这个命令，它依赖Ruby（执行ruby脚本的环境）和RubyGems（redis和ruby接口依赖），因此我们要先安装一下
[root@localhost redis-cluster]# yum install ruby
[root@localhost redis-cluster]# yum install rubygems
[root@localhost redis-cluster]# gem install redis
遇到问题:(1)执行gem install redis --version 3.3.3报错
[root@localhost ~]# gem install redis
Fetching: redis-4.1.2.gem (100%)
ERROR: Error installing redis:
redis requires Ruby version >= 2.3.0.
查了查资料，CentOS7 库中的红宝石的版本支持到2.0.0，可ruby安装Redis（我用的redis版本是4.0.6）的需要最低是2.3.0，自己编译的ruby源码，再执行还是报错…最后谷歌结合几篇文章解决了：
采用rvm来更新ruby：
安装rvm：
[root@localhost /]#gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
[root@localhost /]#curl -L https://get.rvm.io | bash -s stable //下载两个文件（超级慢有超时）
[root@localhost /]# source ~/.bashrc
[root@localhost /]# source ~/.bash_profile
[root@localhost /]# source /usr/local/rvm/scripts/rvm //更新资源命令
[root@localhost /]# rvm -v //查看版本
rvm 1.29.9 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]
至此RVM安装完成
更新ruby
查看rvm库中已知的ruby版本
[root@localhost /]# rvm list known
# MRI Rubies
[ruby-]1.8.6[-p420]
[ruby-]1.8.7[-head] # security released on head
[ruby-]1.9.1[-p431]
[ruby-]1.9.2[-p330]
[ruby-]1.9.3[-p551]
[ruby-]2.0.0[-p648]
[ruby-]2.1[.10]
[ruby-]2.2[.10]
[ruby-]2.3[.8]
[ruby-]2.4[.6]
[ruby-]2.5[.5]
[ruby-]2.6[.3]
[ruby-]2.7[.0-preview1]
ruby-head
安装一个ruby版本
[root@localhost /]# rvm install 2.6.3 //安装2.6.3
如果安装失败（本人由于网络问题安装失败）
试试离线安装
此方式可以安装高版本
1：下载安装包，官网地址：下载Ruby；
2：文件放在$rvm_path/archives目录下
3：然后，rvm install x.x
rvm会自动用本地下载的文件进行安装
4：确认是否安装成功：ruby -v。
安装完成后执行以下命令
[root@localhost /]# rvm use 2.6.3 //使用2.6.3
[root@localhost /]# ruby --version //设置默认版本
[root@localhost /]# rvm remove 2.0.0 //卸载一个已知版本
以上执行完毕后再次执行：
[root@localhost ~]# gem install redis
Fetching redis-4.1.2.gem
Successfully installed redis-4.1.2
Parsing documentation for redis-4.1.2
Installing ri documentation for redis-4.1.2
Done installing documentation for redis after 1 seconds
1 gem installed
如果:执行gem install redis此命令还是无法获取资源。。。。经过网海搜寻–内网机器无rubygems源（也有可能跟网速有关，我的是网速慢）
如果你是内网集群而且公司没有rubygems源，你会发现你压根不能执行gem install redis，所以你需要离线安装：
下载地址 国外镜像 国内ruby gems镜像
至此redis集群环境Ruby安装完成
（四）Ruby安装完成之后，我们开始启动6个节点；
启动前释放缓存
[root@localhost redis-4.0.10]# echo 3 > /proc/sys/vm/drop_caches
[root@localhost redis-4.0.10]# src/redis-server ../redis-cluster/8001/redis.conf
[root@localhost redis-4.0.10]# src/redis-server ../redis-cluster/8002/redis.conf
[root@localhost redis-4.0.10]# src/redis-server ../redis-cluster/8004/redis.conf
[root@localhost redis-4.0.10]# src/redis-server ../redis-cluster/8004/redis.conf
[root@localhost redis-4.0.10]# src/redis-server ../redis-cluster/8004/redis.conf
[root@localhost redis-4.0.10]# src/redis-server ../redis-cluster/8006/redis.conf
或者使用命令
[root@localhost redis-4.0.10]# for dir in /usr/local/redis-cluster/{8001,8002，8003,8004，8005,8006} ; do /usr/local/redis-4.0.10/src/redis-server $dir/redis.conf ; done
也可使用启动脚本
[root@localhost redis-4.0.10]# vim startRedisAll.sh
#!/bin/sh
echo "-----------------------------------------------------------------------------" >> /usr/local/redis.log
/usr/local/redis-4.0.10/src/redis-server /usr/local/redis-cluster/8001/redis.conf
/usr/local/redis-4.0.10/src/redis-server /usr/local/redis-cluster/8002/redis.conf
/usr/local/redis-4.0.10/src/redis-server /usr/local/redis-cluster/8003/redis.conf
/usr/local/redis-4.0.10/src/redis-server /usr/local/redis-cluster/8004/redis.conf
/usr/local/redis-4.0.10/src/redis-server /usr/local/redis-cluster/8005/redis.conf
/usr/local/redis-4.0.10/src/redis-server /usr/local/redis-cluster/8006/redis.conf
echo -e "`date +%Y/%D/%H:%M` redis start end." >> /usr/local/redis.log
[root@localhost redis-4.0.10]# ./startRedisAll.sh
54356:C 14 Feb 13:57:25.984 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54356:C 14 Feb 13:57:25.984 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54356, just started
54356:C 14 Feb 13:57:25.984 # Configuration loaded
54358:C 14 Feb 13:57:25.996 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54358:C 14 Feb 13:57:25.997 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54358, just started
54358:C 14 Feb 13:57:25.997 # Configuration loaded
54363:C 14 Feb 13:57:26.007 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54363:C 14 Feb 13:57:26.008 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54363, just started
54363:C 14 Feb 13:57:26.008 # Configuration loaded
54368:C 14 Feb 13:57:26.027 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54368:C 14 Feb 13:57:26.027 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54368, just started
54368:C 14 Feb 13:57:26.027 # Configuration loaded
54373:C 14 Feb 13:57:26.045 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54373:C 14 Feb 13:57:26.045 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54373, just started
54373:C 14 Feb 13:57:26.045 # Configuration loaded
54378:C 14 Feb 13:57:26.063 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54378:C 14 Feb 13:57:26.063 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54378, just started
54378:C 14 Feb 13:57:26.063 # Configuration loaded
查看一下服务是否正常起来了
[root@localhost redis-4.0.10]# ps -ef |grep redis
root 54278 1 0 13:17 ? 00:00:00 src/redis-server 192.168.0.108:8001 [cluster]
root 54283 1 0 13:17 ? 00:00:00 src/redis-server 192.168.0.108:8002 [cluster]
root 54288 1 0 13:18 ? 00:00:00 src/redis-server 192.168.0.108:8003 [cluster]
root 54293 1 0 13:18 ? 00:00:00 src/redis-server 192.168.0.108:8004 [cluster]
root 54298 1 0 13:18 ? 00:00:00 src/redis-server 192.168.0.108:8005 [cluster]
root 54303 1 0 13:18 ? 00:00:00 src/redis-server 192.168.0.108:8006 [cluster]
root 54308 35349 0 13:18 pts/1 00:00:00 grep --color=auto redis
ok，6个节点都已经正常启动。
（五）开始创建集群
[root@localhost redis-4.0.10]# ./src/redis-trib.rb create --replicas 1 192.168.0.108:8001 192.168.0.108:8002 192.168.0.108:8003 192.168.0.108:8004 192.168.0.108:8005 192.168.0.108:8006
执行运行结果如下
[root@localhost redis-4.0.10]# ./src/redis-trib.rb create --replicas 1 192.168.0.108:8001 192.168.0.108:8002 192.168.0.108:8003 192.168.0.108:8004 192.168.0.108:8005 192.168.0.108:8006 // --replicas 1 表示为每个主节点分配一个从节点（3主3从）
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.0.108:8001
192.168.0.108:8002
192.168.0.108:8003
Adding replica 192.168.0.108:8005 to 192.168.0.108:8001
Adding replica 192.168.0.108:8006 to 192.168.0.108:8002
Adding replica 192.168.0.108:8004 to 192.168.0.108:8003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001
slots:0-5460 (5461 slots) master
M: 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002
slots:5461-10922 (5462 slots) master
M: 3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003
slots:10923-16383 (5461 slots) master
S: 8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004
replicates 3e79b85af9cdc988e126046f22928945285eb161
S: 1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005
replicates 5ffc6c0d316f3352443f1979239c852e6c8e4540
S: fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006
replicates 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 192.168.0.108:8001)
M: 5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006
slots: (0 slots) slave
replicates 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
S: 8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004
slots: (0 slots) slave
replicates 3e79b85af9cdc988e126046f22928945285eb161
S: 1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005
slots: (0 slots) slave
replicates 5ffc6c0d316f3352443f1979239c852e6c8e4540
M: 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
由控制台可以看到，集群已经创建完毕，那么我们到8001这个节点看一下集群节点状态
[root@localhost redis-4.0.10]# src/redis-cli -h 192.168.0.108 -p 8001
192.168.0.108:8001> cluster nodes
3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003@18003 master - 0 1581660685560 3 connected 10923-16383
fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006@18006 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581660686566 6 connected
8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004@18004 slave 3e79b85af9cdc988e126046f22928945285eb161 0 1581660687000 4 connected
1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005@18005 slave 5ffc6c0d316f3352443f1979239c852e6c8e4540 0 1581660686000 5 connected
1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002@18002 master - 0 1581660686000 2 connected 5461-10922
5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001@18001 myself,master - 0 1581660685000 1 connected 0-5460
192.168.0.108:8001>
connected 0-5460 表示主节点插槽（slots）范围
控制台信息显示: 当前集群中存在3个主节点和3个从节点，说明我们的集群已经搭建成功
master==8001-----------slave===8005
master==8002-----------slave===8006
master==8003-----------slave===8004
至此，Redis Cluster集群就搭建完成了！
（六）对集群受否可用进行测试
（1）测试集群后的redis是否可用
[root@localhost redis-4.0.10]# src/redis-cli -c -h 192.168.0.108 -p 8001 //正确打开客户端务必添加'-c'
192.168.0.108:8001> cluster info //查看集群信息
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:5451
cluster_stats_messages_pong_sent:5408
cluster_stats_messages_sent:10859
cluster_stats_messages_ping_received:5403
cluster_stats_messages_pong_received:5451
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:10859
192.168.0.108:8001> set name redis-cluster //写入值
-> Redirected to slot [5798] located at 192.168.0.108:8002
OK
192.168.0.108:8002> redis中总共设置了有16833个节点，在请求的时候会出现重定向，这里把命令重定向到8002端口。
验证集群是否成功，可以给每个节点进行set值，然后去观察运行结果，此处略过，可自行验证，下图是本人验证结果
192.168.0.108:8001> keys *
1) "3"
192.168.0.108:8002> keys *
1) "1"
2) "2"
3) "5"
4) "6"
192.168.0.108:8003> keys *
2) "4"
192.168.0.108:8004> keys *
1) "4"
192.168.0.108:8005> keys *
1) "3"
192.168.0.108:8006> keys *
1) "2"
2) "6"
3) "1"
4) "5"
主节点和从节点值一致。
出现如上提示证明集群后的redis可用
&&添加节点：
首先创建两个实例文件夹
[root@localhost redis-4.0.10]# mkdir -p /usr/local/redis-cluster/{8007,8008}
然后将8001实例中redis.conf复制到新建的实例文件夹中
[root@localhost redis-4.0.10]# for dir in /usr/local/redis-cluster/{8007,8008} ; do cp -v /usr/local/redis-cluster/8001/redis.conf $dir ; done
‘/usr/local/redis-cluster/8001/redis.conf’ -> ‘/usr/local/redis-cluster/8007/redis.conf’
‘/usr/local/redis-cluster/8001/redis.conf’ -> ‘/usr/local/redis-cluster/8008/redis.conf’
[root@localhost redis-4.0.10]#
再次修改8007，8008中的配置文件
[root@localhost redis-cluster]# sed -i 's/8001/8007/g' 8007/redis.conf
[root@localhost redis-cluster]# sed -i 's/8001/8008/g' 8008/redis.conf
启动8007，8008节点
[root@localhost redis-4.0.10]# for dir in /usr/local/redis-cluster/{8007,8008} ; do /usr/local/redis-4.0.10/src/redis-server $dir/redis.conf ; done
54646:C 14 Feb 15:59:43.860 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54646:C 14 Feb 15:59:43.860 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54646, just started
54646:C 14 Feb 15:59:43.860 # Configuration loaded
54648:C 14 Feb 15:59:43.872 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
54648:C 14 Feb 15:59:43.872 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=54648, just started
54648:C 14 Feb 15:59:43.872 # Configuration loaded
[root@localhost redis-4.0.10]#
查看
[root@localhost redis-4.0.10]# ps -ef|grep redis
root 54357 1 0 13:57 ? 00:00:17 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8001 [cluster]
root 54362 1 0 13:57 ? 00:00:17 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8002 [cluster]
root 54367 1 0 13:57 ? 00:00:18 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8003 [cluster]
root 54372 1 0 13:57 ? 00:00:18 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8004 [cluster]
root 54377 1 0 13:57 ? 00:00:17 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8005 [cluster]
root 54382 1 0 13:57 ? 00:00:17 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8006 [cluster]
root 54647 1 0 15:59 ? 00:00:00 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8007 [cluster]
root 54649 1 0 15:59 ? 00:00:00 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8008 [cluster]
root 54670 35349 0 16:02 pts/1 00:00:00 grep --color=auto redis
[root@localhost redis-4.0.10]#
8007 8008 都已启动，新增节点实例创建完成
加入集群 我们使用 redis-trib.rb add-node 命令
[root@localhost redis-4.0.10]# ./src/redis-trib.rb add-node 192.168.0.108:8007 192.168.0.108:8001
>>> Adding node 192.168.0.108:8007 to cluster 192.168.0.108:8001
>>> Performing Cluster Check (using node 192.168.0.108:8001)
M: 5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006
slots: (0 slots) slave
replicates 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
S: 8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004
slots: (0 slots) slave
replicates 3e79b85af9cdc988e126046f22928945285eb161
S: 1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005
slots: (0 slots) slave
replicates 5ffc6c0d316f3352443f1979239c852e6c8e4540
M: 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.108:8007 to make it join the cluster.
[OK] New node added correctly.
./src/redis-trib.rb add-node 192.168.0.108:8007 192.168.0.108:8001 删除线部分可以时集群中任何一个主节点
再看下集群节点, 发现8007这个实例已经做为主节点加到集群里边儿来了.
[root@localhost redis-4.0.10]# src/redis-cli -c -h 192.168.0.108 -p 8001
192.168.0.108:8001> cluster nodes
3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003@18003 master - 0 1581668050000 3 connected 10923-16383
fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006@18006 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581668051000 6 connected
8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004@18004 slave 3e79b85af9cdc988e126046f22928945285eb161 0 1581668050017 4 connected
1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005@18005 slave 5ffc6c0d316f3352443f1979239c852e6c8e4540 0 1581668051025 5 connected
1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002@18002 master - 0 1581668050521 2 connected 5461-10922
5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001@18001 myself,master - 0 1581668051000 1 connected 0-5460
39aeff3baa8ff12f49cfea887db724509be1d5b9 192.168.0.108:8007@18007 master - 0 1581668051528 0 connected
192.168.0.108:8001>
然后我们将8008作为8007的从节点加入集群add-node --slave --master-id master-id为8007主节点id “39aeff3baa8ff12f49cfea887db724509be1d5b9” 执行如下
[root@localhost redis-4.0.10]# ./src/redis-trib.rb add-node --slave --master-id 39aeff3baa8ff12f49cfea887db724509be1d5b9 192.168.0.108:8008 192.168.0.108:8001
>>> Adding node 192.168.0.108:8008 to cluster 192.168.0.108:8001
>>> Performing Cluster Check (using node 192.168.0.108:8001)
M: 5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006
slots: (0 slots) slave
replicates 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
S: 8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004
slots: (0 slots) slave
replicates 3e79b85af9cdc988e126046f22928945285eb161
S: 1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005
slots: (0 slots) slave
replicates 5ffc6c0d316f3352443f1979239c852e6c8e4540
M: 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 39aeff3baa8ff12f49cfea887db724509be1d5b9 192.168.0.108:8007
slots: (0 slots) master
0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.108:8008 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 192.168.0.108:8007.
[OK] New node added correctly.
[root@localhost redis-4.0.10]#
再次查看一下集群节点
[root@localhost redis-4.0.10]# src/redis-cli -c -h 192.168.0.108 -p 8001
192.168.0.108:8001> cluster nodes
3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003@18003 master - 0 1581668368023 3 connected 10923-16383
fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006@18006 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581668369534 6 connected
8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004@18004 slave 3e79b85af9cdc988e126046f22928945285eb161 0 1581668369231 4 connected
1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005@18005 slave 5ffc6c0d316f3352443f1979239c852e6c8e4540 0 1581668368000 5 connected
1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002@18002 master - 0 1581668369534 2 connected 5461-10922
76f82463d9088f62a99482818fb8690a139a60a9 192.168.0.108:8008@18008 slave 39aeff3baa8ff12f49cfea887db724509be1d5b9 0 1581668367218 0 connected
5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001@18001 myself,master - 0 1581668368000 1 connected 0-5460
39aeff3baa8ff12f49cfea887db724509be1d5b9 192.168.0.108:8007@18007 master - 0 1581668368225 0 connected
192.168.0.108:8001>
可以看到 8008 已经做为一个从节点加入到了集群.
从图中可以看到, 8007 这个节点实例并没有分配插槽（slots）加入到集群的新节点,我们还需要给它分配slots,才能真正用于集群存取数据,否则加进来没有任何意义.那边就需要重新分片
&&重新分片：
所谓重新分片,就是重新分配各个节点拥有的slots,这里我们的主要目的是从老的节点中迁移一部分slots放到新节点中去,以便让新节点真正成为集群中的一员.
同样, 还是利用redis-trib.rb reshard工具.
重新分片有交互式命令和非交互式命令进行迁移,交互式命令./src/redis-trib.rb reshard 192.168.0.108:8001按照提示操作即可.
非交互
[root@localhost redis-4.0.1]# ./src/redis-trib.rb reshard --from a2eee0ea546f2c3701b08981737c07938039857c --to 36d53c7f1896838249c0b4afdcf680bac2f4ec2e --slots 45 --yes 127.0.0.1:7001
如果执行报错：Wrong number of arguments for specified sub command
是因为redis-trib.rb脚本的兼容性问题，需要修改这个脚本migrate部分代码：
# Migrate all the keys from source to target using the MIGRATE command
while true
keys = source.r.cluster("getkeysinslot",slot,o[:pipeline])
break if keys.length == 0
begin
source.r.client.call(["migrate",target.info[:host],target.info[:port],"",0,@timeout,:keys,*keys])
rescue => e
if o[:fix] && e.to_s =~ /BUSYKEY/
xputs "*** Target key exists. Replacing it for FIX."
source.r.client.call(["migrate",target.info[:host],target.info[:port],"",0,@timeout,:replace,:keys,*keys])
将 source.r.client.中的client删除即可，即变成下面：
# Migrate all the keys from source to target using the MIGRATE command
while true
keys = source.r.cluster("getkeysinslot",slot,o[:pipeline])
break if keys.length == 0
begin
source.r.call(["migrate",target.info[:host],target.info[:port],"",0,@timeout,:keys,*keys])
rescue => e
if o[:fix] && e.to_s =~ /BUSYKEY/
xputs "*** Target key exists. Replacing it for FIX."
source.r.call(["migrate",target.info[:host],target.info[:port],"",0,@timeout,:replace,:keys,*keys])
不报错执行结果
[root@localhost redis-4.0.10]# ./src/redis-trib.rb reshard --from 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 --to 39aeff3baa8ff12f49cfea887db724509be1d5b9 --slots 456 --yes 192.168.0.108:8001
Moving slot 5911 from 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
Moving slot 5912 from 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
Moving slot 5913 from 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
Moving slot 5914 from 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
Moving slot 5915 from 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
Moving slot 5916 from 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7
......
Moving slot 5461 from 192.168.0.108:8002 to 192.168.0.108:8007:
Moving slot 5462 from 192.168.0.108:8002 to 192.168.0.108:8007:
Moving slot 5463 from 192.168.0.108:8002 to 192.168.0.108:8007:
Moving slot 5464 from 192.168.0.108:8002 to 192.168.0.108:8007:
Moving slot 5465 from 192.168.0.108:8002 to 192.168.0.108:8007:
Moving slot 5466 from 192.168.0.108:8002 to 192.168.0.108:8007:
Moving slot 5467 from 192.168.0.108:8002 to 192.168.0.108:8007:
Moving slot 5468 from 192.168.0.108:8002 to 192.168.0.108:8007:
......
查看一下8007的插槽
[root@localhost redis-4.0.10]# src/redis-cli -c -h 192.168.0.108 -p 8001
192.168.0.108:8001> cluster nodes
3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003@18003 master - 0 1581671683043 3 connected 10923-16383
fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006@18006 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581671683000 6 connected
8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004@18004 slave 3e79b85af9cdc988e126046f22928945285eb161 0 1581671683000 4 connected
1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005@18005 slave 5ffc6c0d316f3352443f1979239c852e6c8e4540 0 1581671684050 5 connected
1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002@18002 master - 0 1581671682000 2 connected 5917-10922
76f82463d9088f62a99482818fb8690a139a60a9 192.168.0.108:8008@18008 slave 39aeff3baa8ff12f49cfea887db724509be1d5b9 0 1581671682038 7 connected
5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001@18001 myself,master - 0 1581671683000 1 connected 0-5460
39aeff3baa8ff12f49cfea887db724509be1d5b9 192.168.0.108:8007@18007 master - 0 1581671683000 7 connected 5461-5916
192.168.0.108:8001>
39aeff3baa8ff12f49cfea887db724509be1d5b9 192.168.0.108:8007@18007 master - 0 1581671683000 7 connected 5461-5916 看控制台8007插槽范围 5461-5916
交互的
[root@localhost redis-4.0.10]# /src/redis-trib.rb reshard 192.168.0.108:8001
//PS: 这条命令是交互的,按照提示操作即可.
>>> Performing Cluster Check (using node 192.168.0.108:8001)
M: a2eee0ea546f2c3701b08981737c07938039857c 192.168.0.108:8001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 36d53c7f1896838249c0b4afdcf680bac2f4ec2e 192.168.0.108:8002
slots: (0 slots) master
1 additional replica(s)
S: 888b802702cb9304913a0b25d7ff3daf953a4507 192.168.0.108:8004
slots: (0 slots) slave
replicates fecbca3b75adc5cf6e79175630ff59c5764962ae
S: 2b577338d01f6be1502c493cd80af719e719ddbd 192.168.0.108:8005
slots: (0 slots) slave
replicates a2eee0ea546f2c3701b08981737c07938039857c
S: 66c0169faa7a00da01ddb681dcc56cae730e2ced 192.168.0.108:8006
slots: (0 slots) slave
replicates 36d53c7f1896838249c0b4afdcf680bac2f4ec2e
S: 590e206fe468991138e2fce92b6ccd10a9eddb2e 192.168.0.108:8008
slots: (0 slots) slave
replicates 8ab3d14eba181c06dc8826bea0db1becdead2533
M: fecbca3b75adc5cf6e79175630ff59c5764962ae 192.168.0.108:8003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: 8ab3d14eba181c06dc8826bea0db1becdead2533 192.168.0.108:8007
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)?4096 //输入一个数,这个4096表示迁移多少个slots数
What is the receiving node ID? 36d53c7f1896838249c0b4afdcf680bac2f4ec2e //输入目标节点ID,表示迁移到哪个目标节点
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:all //输入all表示从老的所有节点进行重分配,凑够4096个slots给到新节点.
//也可以输入源节点id, 可以输入多个源节点id,最后输入done.就开始从你输入的源节点id的节点进行迁移了.
Ready to move 4096 slots.
Source nodes:
M: a2eee0ea546f2c3701b08981737c07938039857c 192.168.0.108:8001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: fecbca3b75adc5cf6e79175630ff59c5764962ae 192.168.0.108:8003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: 8ab3d14eba181c06dc8826bea0db1becdead2533192.168.0.108:8002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
Destination node:
M: 36d53c7f1896838249c0b4afdcf680bac2f4ec2e 192.168.0.108:8007
slots: (0 slots) master
1 additional replica(s)
Resharding plan:
Moving slot 5461 from 8ab3d14eba181c06dc8826bea0db1becdead2533
Moving slot 5462 from 8ab3d14eba181c06dc8826bea0db1becdead2533
Moving slot 5463 from 8ab3d14eba181c06dc8826bea0db1becdead2533
Moving slot 5464 from 8ab3d14eba181c06dc8826bea0db1becdead2533
Moving slot 5465 from 8ab3d14eba181c06dc8826bea0db1becdead2533
Moving slot 5466 from 8ab3d14eba181c06dc8826bea0db1becdead2533
结果不在赘述
以上操作可完成分片
&删除节点：
删除节点：redis-trib.rb del-node
注意点：要删除集群中的某个节点(注:这里说的是集群中的主节点),首先必须确保这个节点没有拥有任何一个slots.我们现在来直接删除8007 这个节点看看结果
[root@localhost redis-4.0.10]# ./src/redis-trib.rb del-node 192.168.0.108:8001 39aeff3baa8ff12f49cfea887db724509be1d5b9
>>> Removing node 39aeff3baa8ff12f49cfea887db724509be1d5b9 from cluster 192.168.0.108:8001
[ERR] Node 192.168.0.108:8007 is not empty! Reshard data away and try again.
[root@localhost redis-4.0.10]#
执行报错：N*ode 192.168.0.108:8007 is not empty! Reshard data away and try again.*表示8007这个主节点上有slots，将节点上的slots全部迁移出去再试
采用交互式
[root@localhost redis-4.0.10]# ./src/redis-trib.rb reshard 192.168.0.108:8001
......
How many slots do you want to move (from 1 to 16384)?455 //输入一个大于或等于8007节点所拥有的slots数的数即可.
What is the receiving node ID? 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7//接收这些slots的目标节点,这里
......
Source node #1:39aeff3baa8ff12f49cfea887db724509be1d5b9 //因为我们要删除8007这个节点,所以源节点的id就是8007的节点ID
Source node #2:done //输入done,回车,就会开始从8007 这个节点迁移455个slot(没有这么多就迁移拥有的全部)到8002节点中去.
.......
Moving slot 5916 from 192.168.0.108:8007 to 192.168.0.108:8002:
迁移完成再看各个节点的状态:
[root@localhost redis-4.0.10]# src/redis-cli -c -h 192.168.0.108 -p 8001
192.168.0.108:8001> cluster nodes
3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003@18003 master - 0 1581674093352 3 connected 10923-16383
fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006@18006 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581674093000 8 connected
8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004@18004 slave 3e79b85af9cdc988e126046f22928945285eb161 0 1581674092345 4 connected
1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005@18005 slave 5ffc6c0d316f3352443f1979239c852e6c8e4540 0 1581674093000 5 connected
1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002@18002 master - 0 1581674093855 8 connected 5461-10922
76f82463d9088f62a99482818fb8690a139a60a9 192.168.0.108:8008@18008 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581674094000 8 connected
5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001@18001 myself,master - 0 1581674092000 1 connected 0-5460
39aeff3baa8ff12f49cfea887db724509be1d5b9 192.168.0.108:8007@18007 master - 0 1581674091842 7 connected
192.168.0.108:8001>
8007节点已经无任何一个slots。执行删除
[root@localhost redis-4.0.10]# ./src/redis-trib.rb del-node 192.168.0.108:8001 39aeff3baa8ff12f49cfea887db724509be1d5b9
>>> Removing node 39aeff3baa8ff12f49cfea887db724509be1d5b9 from cluster 192.168.0.108:8001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node. //同时停止了进程
[root@localhost redis-4.0.10]# ps -ef | grep redis //查看redis进程已无8007节点
root 54357 1 0 13:57 ? 00:00:34 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8001 [cluster]
root 54362 1 0 13:57 ? 00:00:34 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8002 [cluster]
root 54367 1 0 13:57 ? 00:00:34 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8003 [cluster]
root 54372 1 0 13:57 ? 00:00:34 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8004 [cluster]
root 54377 1 0 13:57 ? 00:00:33 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8005 [cluster]
root 54382 1 0 13:57 ? 00:00:33 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8006 [cluster]
root 54649 1 0 15:59 ? 00:00:15 /usr/local/redis-4.0.10/src/redis-server 192.168.0.108:8008 [cluster]
root 54890 54771 0 18:00 pts/0 00:00:00 grep --color=auto redis
[root@localhost redis-4.0.10]#
删除成功！
再次查看集群状态
[root@localhost redis-4.0.10]# src/redis-cli -c -h 192.168.0.108 -p 8001
192.168.0.108:8001> cluster nodes
3e79b85af9cdc988e126046f22928945285eb161 192.168.0.108:8003@18003 master - 0 1581674268505 3 connected 10923-16383
fbfaa42f64689db65dec42f5675dd37f66251371 192.168.0.108:8006@18006 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581674267000 8 connected
8706fb3d61e7ca1f433fc66534215aea8d91061e 192.168.0.108:8004@18004 slave 3e79b85af9cdc988e126046f22928945285eb161 0 1581674267497 4 connected
1924cebf2e62605d3b28a764add6b6da7e3bcbb1 192.168.0.108:8005@18005 slave 5ffc6c0d316f3352443f1979239c852e6c8e4540 0 1581674267000 5 connected
1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 192.168.0.108:8002@18002 master - 0 1581674267599 8 connected 5461-10922
76f82463d9088f62a99482818fb8690a139a60a9 192.168.0.108:8008@18008 slave 1cd0ce2458d65b65a5a634ec287f571dbbcbb4b7 0 1581674267000 8 connected
5ffc6c0d316f3352443f1979239c852e6c8e4540 192.168.0.108:8001@18001 myself,master - 0 1581674266000 1 connected 0-5460
192.168.0.108:8001>
我们看到8007这个节点已经被删除了，那么8007的从节点8008去哪了呢，查看控制到8008已经自动作为8002的从节点了，
以上就是利用官方提供的redis-trib.rb 工具来完成上述的各项工作,事实上这个工具也是利用cluster的内部命令进行的整合以方便我们使用和管理.如果想了解更多的细节,需要查看官方的文档！
后记：针对redis集群节点宕机 fail状态 redis的投票机制实测请看本人另一篇博文
传送门：redis集群节点宕机 fail状态 redis的投票机制实测
Tips :
- 如果想重新创建集群，需要登录到每个节点，执行flushdb，然后执行cluster reset，重启节点；
- 如果要批量杀掉Redis进程，可以使用pkill redis-server命令；
- 如果redis开启了密码认证，则需要在redis.conf中增加属性 : masterauth yourpassword ，并且需要修改/usr/local/share/gems/gems/redis-3.3.3/lib/redis目录下的client.rb文件，将password属性设置为redis.conf中的requirepass的值，不同的操作系统client.rb的位置可能不一样，可以使用 find / -name "client.rb"全盘查找一下；
- 插槽和Key的关系：key的有效部分使用CRC16算法计算出哈希值，再将哈希值对16384取余，得到插槽值。16384其实就是上面创建集群的时候看到的默认分配的插槽数量
参考文档：
https://my.oschina.net/lyyjason/blog/1842002
https://blog.csdn.net/bluesky588/article/details/102614977
https://www.cnblogs.com/xing901022/p/7327363.html
https://blog.csdn.net/reyleon/article/details/51454334
Redis命令手册：http://redisdoc.com/index.html