redis实战--主从--sentinel--Cluster
目录
一、主从模式
1.1主从测试
1.2通过配置快照快速存储数据
1.3通过AOF日志备份数据
二、sentinel模式
2.1特点
2.2搭建
2.3启动sentinel
2.4测试sentinel
2.5 sentinel 事件说明
三、Cluster模式
3.1Cluster集群特点
3.2Cluster搭建
3.4Cluster 伸缩
3.4.1 扩容--slot分配
3.4.2 缩容--slot分配
redis集群有三种模式
1.主从，2.sentinel（哨兵）,3.Cluster模式
环境：centos7 ,redis-6.0.10
一、主从模式
（一台电脑上测试）
特点：master挂了，redis从可读，影响master连接以及写操作。不会重选master。
1.1主从测试
[root@ol7-19 redis-6.0.10]# ps -aux|grep 6379
root 23328 0.5 0.1 164940 3004 pts/4 Sl+ 10:58 0:16 redis-server 127.0.0.1:6379
root 25068 0.0 0.0 25044 1696 pts/7 S+ 11:32 0:00 redis-cli -h 127.0.0.1 -p 6379
root 25744 0.0 0.0 112816 972 pts/0 S+ 11:45 0:00 grep --color=auto 6379
[root@ol7-19 redis-6.0.10]# kill -9 23328
[root@ol7-19 redis-6.0.10]# ps -aux|grep 6379
root 25068 0.0 0.0 25044 1696 pts/7 S+ 11:32 0:00 redis-cli -h 127.0.0.1 -p 6379
root 25761 0.0 0.0 112812 972 pts/0 S+ 11:46 0:00 grep --color=auto 6379
[root@ol7-19 redis-6.0.10]# redis-cli
Could not connect to Redis at 127.0.0.1:6379: Connection refused
not connected> exit
[root@ol7-19 redis-6.0.10]# redis-cli -p 7000
127.0.0.1:7000> get foo
"bar"
127.0.0.1:7000>
删除master的快照文件，dump.rdb,master挂掉之后重启不会恢复数据，从库会同步，导致所有数据清空，如果使用一个快照文件。
手动kill -9停止master之前，要先使用save命令，备份数据文件。
自动退出会在退出前自动保存数据（kill 不会自动保存）
27668:M 15 Feb 2021 12:19:21.878 * Saving the final RDB snapshot before exiting.
27668:M 15 Feb 2021 12:19:21.881 * DB saved on disk
27668:M 15 Feb 2021 12:19:21.882 # Redis is now ready to exit, bye bye...
1.2通过配置快照快速存储数据
save 900 1 // 900内,有1条写入,则产生快照
save 300 1000 // 如果300秒内有1000次写入,则产生快照
save 60 10000 // 如果60秒内有10000次写入,则产生快照
(这3个选项都屏蔽,则rdb禁用)
stop-writes-on-bgsave-error yes // 后台备份进程出错时,主进程停不停止写入?
rdbcompression yes // 导出的rdb文件是否压缩
Rdbchecksum yes // 导入rbd恢复时数据时,要不要检验rdb的完整性
dbfilename dump.rdb //导出来的rdb文件名
dir ./ //rdb的放置路径1.3通过AOF日志备份数据
appendonly no # 是否打开 aof日志功能
appendfsync always # 每1个命令,都立即同步到aof. 安全,速度慢
appendfsync everysec # 折衷方案,每秒写1次
appendfsync no # 写入工作交给操作系统,由操作系统判断缓冲区大小,统一写入到aof. 同步频率低,速度快,
no-appendfsync-on-rewrite yes: # 正在导出rdb快照的过程中,要不要停止同步aof
auto-aof-rewrite-percentage 100 #aof文件大小比起上次重写时的大小,增长率100%时,重写
auto-aof-rewrite-min-size 64mb #aof文件,至少超过64M时,重写
1.5 redis 生产环境启动方案（仅限只安装了一个版本的redis）
https://developer.aliyun.com/article/649990?spm=a2c6h.13813017.0.dArticle738638.316a6856d8vATl
以上配置待测试
复制redis.conf,7000.conf,7001.conf
修改7000.conf和7001.conf的端口分别不7000和7001
增加replicaof 127.0.0.1 6379
redis-server redis.conf
redis-server 7000.conf
redis-server 7001.conf
按顺序在一台服务器上分别启动不同端口的redis服务
[root@ol7-19 redis-6.0.10]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 127.0.0.1:6379 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 128 127.0.0.1:7000 *:*
LISTEN 0 128 127.0.0.1:7001 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 [::]:22 [::]:*
LISTEN 0 100 [::1]:25 [::]:*
[root@ol7-19 redis-6.0.10]#
12查看主从集群信息
集群命令：https://blog.csdn.net/zwjzqqb/article/details/79973909
client list：查看当前连接到redis的传话信息：id,来源ip,连接的db
master,会表明角色为主服务器，role:master
role命令返回当前实例在集群中的角色状态
127.0.0.1:6379> role
1) "master"
2) (integer) 3103
3) 1) 1) "127.0.0.1"
2) "7000"
3) "3103"
2) 1) "127.0.0.1"
2) "7001"
3) "3103"
127.0.0.1:6379>
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=7000,state=online,offset=2067,lag=0
slave1:ip=127.0.0.1,port=7001,state=online,offset=2067,lag=0
master_replid:ea56a307c4ba2b7516a6cc53206fabeac09c8502
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2067
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:2067
127.0.0.1:6379>
slave,会表明角色为从服务器，role:slave
127.0.0.1:7001> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:1927
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:ea56a307c4ba2b7516a6cc53206fabeac09c8502
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1927
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:1927
127.0.0.1:7001>
测试，主可以读写，从只能读，
[root@ol7-19 redis-6.0.10]# redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> del foo
(integer) 1
127.0.0.1:6379> set foo bar
OK
127.0.0.1:6379> get foo
"bar"
127.0.0.1:6379> exit
[root@ol7-19 redis-6.0.10]# redis-cli -h 127.0.0.1 -p 7000
127.0.0.1:7000> get foo
"bar"
127.0.0.1:7000> exit
[root@ol7-19 redis-6.0.10]# redis-cli -h 127.0.0.1 -p 7001
127.0.0.1:7001> get foo
"bar"
127.0.0.1:7001> del foo
(error) READONLY You can't write against a read only replica.
127.0.0.1:7001> set foo1 bar1
(error) READONLY You can't write against a read only replica.
127.0.0.1:7001>
结论，指定登陆不同的端口，在master上可以读写，在slave上写会提示：(error) READONLY You can't write against a read only replica.
不能在只读从服务器上写。
二、sentinel模式
主从模式集群不具备高可用，所以sentinel(哨兵)应运而生。
哨兵顾名思义就是监控redis集群的运行状况
2.1特点
1).建立在主从模式的基础上
2).master挂掉会在slave中选择一个为master,修改对应的配置文件，挂掉的master重启后变成slave.
3).sentinel也会挂掉，所以sentinel也要找成集群方式。
4).一个sentinel或sentinel集群可以管理多个主从Redis，多个sentinel也可以监控同一个redis
5).sentinel最好不要和Redis部署在同一台机器，不然Redis的服务器挂了以后，sentinel也挂了
2.2搭建
环境准备
| rediis服务器 | ip | 端口 |
| master | 172.18.28.245 | sentinel 26379 |
| slave | 172.18.28.244 |
sentinel 26379 |
| slave | 172.18.28.243 | sentinel 26379 |
master修改redis.conf
sed -i 's/^bind 127.0.0.1/bind 0.0.0.0/g' redis.conf
sed -i 's/protected-mode yes/protected-mode no/g' redis.conf
sed -i 's/appendonly no/appendonly yes/g' redis.conf
sed -i "/# requirepass foobared/a requirepass 123456" redis.conf
sed -i "/# masterauth
slave修改redis.conf
sed -i 's/^bind 127.0.0.1/bind 0.0.0.0/g' redis.conf
sed -i 's/protected-mode yes/protected-mode no/g' redis.conf
sed -i 's/appendonly no/appendonly yes/g' redis.conf
sed -i "/# replicaof
sed -i "/# requirepass foobared/a requirepass 123456" redis.conf
sed -i "/# masterauth
将它写成脚本config.sh,方便一个脚本多次安装:
#!/bin.bash
read -p "Setting the master?[y/n]:" input
echo $input
filename=redis.conf
filepath=/usr/local/redis-6.0.10
file=$filepath/$filename
masterIP=172.18.28.245
masterPort=6379
if [ -f "$file" ];then
echo "exist"
else
echo "not exist"
exit
fi
if [ $input = "n" ];then
IP=`ip addr|awk '{if($2~"172")print $2}'|awk -F '/' '{print $1}'`
sed -i "s/^bind 127.0.0.1/bind $IP/g" $file
sed -i 's/protected-mode yes/protected-mode no/g' $file
sed -i 's/appendonly no/appendonly yes/g' $file
sed -i "/# replicaof /a replicaof $masterIP $masterPort" $file
# requirepass foobared
sed -i "/# requirepass foobared/a requirepass 123456" $file
# masterauth
sed -i "/# masterauth /a masterauth 123456" $file
elif [ $input = "y" ];then
IP=`ip addr|awk '{if($2~"172")print $2}'|awk -F '/' '{print $1}'`
sed -i "s/^bind 127.0.0.1/bind $IP/g" $file
sed -i 's/protected-mode yes/protected-mode no/g' $file
sed -i 's/appendonly no/appendonly yes/g' $file
# requirepass foobared
sed -i "/# requirepass foobared/a requirepass 123456" $file
# masterauth
sed -i "/# masterauth /a masterauth 123456" $file
fi
当使用sentinel模式的时候，客户端就不要直接连接Redis，而是连接sentinel的ip和port，由sentinel来提供具体的可提供服务的Redis实现，这样当master节点挂掉以后，sentinel就会感知并将新的master节点提供给使用者。
查看sentinel.conf配置
[root@mycat redis-6.0.10]# grep -Ev "^#|^$" sentinel.conf
port 26379
daemonize no
pidfile /var/run/redis-sentinel.pid
logfile ""
dir /tmp
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
[root@mycat redis-6.0.10]#
修改为
sed -i "s/127.0.0.1/172.18.28.245/g" sentinel.conf
sed -i "/^sentinel monitor/a sentinel auth-pass mymaster 123456" sentinel.conf
在其他服务器使用同样的sentinel.conf配置文件
2.3启动sentinel
redis-sentinel sentinel.conf 在三台服务器分别启动sentinel
[root@mycat redis-6.0.10]# redis-sentinel sentinel.conf
28085:X 17 Feb 2021 17:52:56.503 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
28085:X 17 Feb 2021 17:52:56.503 # Redis version=6.0.10, bits=64, commit=00000000, modified=0, pid=28085, just started
28085:X 17 Feb 2021 17:52:56.503 # Configuration loaded
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 6.0.10 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 28085
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
28085:X 17 Feb 2021 17:52:56.505 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
28085:X 17 Feb 2021 17:52:56.511 # Sentinel ID is 2f928347f1644256ee2397a9a8c5bad3bb6d469c
28085:X 17 Feb 2021 17:52:56.511 # +monitor master mymaster 172.18.28.245 6379 quorum 2
28085:X 17 Feb 2021 17:52:56.512 * +slave slave 172.18.28.244:6379 172.18.28.244 6379 @ mymaster 172.18.28.245 6379
28085:X 17 Feb 2021 17:52:56.517 * +slave slave 172.18.28.243:6379 172.18.28.243 6379 @ mymaster 172.18.28.245 6379
2.4测试sentinel
登陆redis master服务器，查看主从集群信息
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.18.28.244,port=6379,state=online,offset=43562,lag=0
slave1:ip=172.18.28.243,port=6379,state=online,offset=43562,lag=0
master_replid:52f3edd73c636afd95d42dbe7b770d7474fb9269
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:43562
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:43562
127.0.0.1:6379>
关闭master的redis服务
如图提示redis的245这台master服务关闭，然后新的 master的选择了243这台，查看集群信息：
[root@mycat ~]# redis-cli -h 172.18.28.243
172.18.28.243:6379> auth 123456
OK
172.18.28.243:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.18.28.244,port=6379,state=online,offset=144931,lag=0
master_replid:ec86e81d6691795b6f5cd21cc54379fc2aa552db
master_replid2:52f3edd73c636afd95d42dbe7b770d7474fb9269
master_repl_offset:144931
second_repl_offset:84014
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:144931
172.18.28.243:6379>
注意查看role:master表示现在登陆的243这台为master,而connected_slaves:1表示只有一台从服务器连接，因为挂掉的master服务器也变为从服务器了，但是没有启动
启动原245的master服务器的redis服务
提示连接到master 172.18.28.243:6379,说明主从已变。
[root@mycat redis-6.0.10]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:172.18.28.243
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
这台服务器查看主从复制的集群信息，role:slave,表示是从服务器了。
master_host:172.18.28.243 说明了redis的主服务器
在243的新master服务器查看主从集群信息：
172.18.28.243:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.18.28.244,port=6379,state=online,offset=237502,lag=1
slave1:ip=172.18.28.245,port=6379,state=online,offset=237643,lag=1
master_replid:ec86e81d6691795b6f5cd21cc54379fc2aa552db
master_replid2:52f3edd73c636afd95d42dbe7b770d7474fb9269
master_repl_offset:237784
和没挂掉的主从一样的状态了，一主两从。
2.5 sentinel 事件说明
· +reset-master ：主服务器已被重置。
· +slave ：一个新的从服务器已经被 Sentinel 识别并关联。
· +failover-state-reconf-slaves ：故障转移状态切换到了 reconf-slaves 状态。
· +failover-detected ：另一个 Sentinel 开始了一次故障转移操作，或者一个从服务器转换成了主服务器。
· +slave-reconf-sent ：领头（leader）的 Sentinel 向实例发送了 [SLAVEOF](/commands/slaveof.html) 命令，为实例设置新的主服务器。
· +slave-reconf-inprog ：实例正在将自己设置为指定主服务器的从服务器，但相应的同步过程仍未完成。
· +slave-reconf-done ：从服务器已经成功完成对新主服务器的同步。
· -dup-sentinel ：对给定主服务器进行监视的一个或多个 Sentinel 已经因为重复出现而被移除 —— 当 Sentinel 实例重启的时候，就会出现这种情况。
· +sentinel ：一个监视给定主服务器的新 Sentinel 已经被识别并添加。
· +sdown ：给定的实例现在处于主观下线状态。
· -sdown ：给定的实例已经不再处于主观下线状态。
· +odown ：给定的实例现在处于客观下线状态。
· -odown ：给定的实例已经不再处于客观下线状态。
· +new-epoch ：当前的纪元（epoch）已经被更新。
· +try-failover ：一个新的故障迁移操作正在执行中，等待被大多数 Sentinel 选中（waiting to be elected by the majority）。
· +elected-leader ：赢得指定纪元的选举，可以进行故障迁移操作了。
· +failover-state-select-slave ：故障转移操作现在处于 select-slave 状态 —— Sentinel 正在寻找可以升级为主服务器的从服务器。
· no-good-slave ：Sentinel 操作未能找到适合进行升级的从服务器。Sentinel 会在一段时间之后再次尝试寻找合适的从服务器来进行升级，又或者直接放弃执行故障转移操作。
· selected-slave ：Sentinel 顺利找到适合进行升级的从服务器。
· failover-state-send-slaveof-noone ：Sentinel 正在将指定的从服务器升级为主服务器，等待升级功能完成。
· failover-end-for-timeout ：故障转移因为超时而中止，不过最终所有从服务器都会开始复制新的主服务器（slaves will eventually be configured to replicate with the new master anyway）。
· failover-end ：故障转移操作顺利完成。所有从服务器都开始复制新的主服务器了。
· +switch-master ：配置变更，主服务器的 IP 和地址已经改变。 这是绝大多数外部用户都关心的信息。
· +tilt ：进入 tilt 模式。
· -tilt ：退出 tilt 模式。
三、Cluster模式
主从集群--解决了一般的读写问题
sentinel集群+主从集群--解决了单个主从集群的master挂掉后的高可用问题
Cluster集群则上在上面的基础上增加了分片（分库）功能，解决单机redis的存储问题
3.1Cluster集群特点
多点redis节点互联，网络共享
所有节点都是一主一从（或一主多从），其中从不提供服务，仅作为备用
支持在线增加，删除节点
客户端可以连接任何一个主节点读写。
3.2Cluster搭建
cluster可以说是sentinel和主从模式的结合体，通过cluster可以实现主从和master重选功能，所以如果配置两个副本三个分片的话，就需要六个Redis实例
还是上面的三台机器，分别开启两个redis服务（端口），3台机器就是3个分片（分库），所以一共是3*2=6个redis服务。
172.18.28.245 端口7001,70002
172.18.28.244 端口7003,70004
172.18.28.243 端口7005,70006
同时防火墙端口要打开17001,17002,17003,17004,17005,17006[root@mycat redis-6.0.10]# cp redis.conf.bak redis_7001.conf
[root@mycat redis-6.0.10]# cp redis.conf.bak redis_7002.conf
[root@mycat redis-6.0.10]# pwd
/usr/local/redis-6.0.10
复制两个原始的配置文件
mkdir -p /data/redis/cluster/redis_7001
mkdir -p /data/redis/cluster/redis_7002
mkdir -p /data/redis/cluster/redis_7003
mkdir -p /data/redis/cluster/redis_7004
mkdir -p /data/redis/cluster/redis_7005
mkdir -p /data/redis/cluster/redis_7006
日志存放目录
mkdir -p /usr/local/redis-6.0.10/cluster/
redis_7001.conf修改
bind 127.0.0.1 改成bind 127.0.0.1 172.18.28.245 可以绑定多个ip地址，这里选择localhost地址以及内网地址
port 6379 改成port 7001
daemonize no改成daemonize yes #Redis will write a pid file in /var/run/redis.pid when daemonized,启动守护进程方式时会写一个pidfile文件
logfile "" 改为logfile "/usr/local/redis-6.0.10/cluster/redis_7001.log"
修改的脚本
#!/bin.bash
#$1==filename,$2==port
if [ -n $1 ];then
filename=$1
fi
filepath=/usr/local/redis-6.0.10
file=$filepath/$filename
Port=6379
if [ -f "$file" ];then
echo "exist"
else
echo "not exist"
exit
fi
#set the port,default 6379
if [ -n $2 ];then
Port=$2
fi
sed -i "s/port 6379/port $Port/g" $file
IP=`ip addr|awk '{if($2~"172")print $2}'|awk -F '/' '{print $1}'`
clusterDataPath="\/data\/redis\/cluster\/redis_$Port"
sed -i "s/^bind 127.0.0.1/bind $IP/g" $file
sed -i 's/protected-mode yes/protected-mode no/g' $file
sed -i "s/redis_6379/redis_$Port/g" $file
sed -i "s/logfile \"\"/logfile \/usr\/local\/redis-6.0.10\/cluster\/redis_${Port}.log/g" $file
sed -i "s/dir \.\//dir $clusterDataPath/" $file
sed -i 's/appendonly no/appendonly yes/g' $file
# requirepass foobared
sed -i "/# requirepass foobared/a requirepass 123456" $file
# masterauth
sed -i "/# masterauth /a masterauth 123456" $file
sed -i "/# cluster-enabled yes/a cluster-enabled yes" $file
sed -i "/# cluster-config-file nodes-6379.conf/a cluster-config-file nodes-${Port}.conf" $file
# cluster-config-file nodes-6379.conf
# cluster-node-timeout 15000
sed -i "/# cluster-node-timeout 15000/a cluster-node-timeout 15000" $file
sed -i "s/daemonize no/daemonize yes/g" $file
执行sh clusterConfig.sh redis_7001.conf 7001 根据不同的端口号修改生成上面所需要的参数，只要维护一个脚本 文件，不用一个一个改。
sh clusterConfig.sh redis_7001.conf 7001
sh clusterConfig.sh redis_7002.conf 7002
sh clusterConfig.sh redis_7003.conf 7003
sh clusterConfig.sh redis_7004.conf 7004
sh clusterConfig.sh redis_7005.conf 7005
sh clusterConfig.sh redis_7006.conf 7006
[root@mycat redis-6.0.10]# sh clusterConfig.sh redis_7001.conf 7001
exist
[root@mycat redis-6.0.10]#
redis-cli -a 123456 --cluster create 172.18.28.245:7001 172.18.28.245:7002 172.18.28.244:7003 172.18.28.244:7004 172.18.28.243:7005 172.18.28.243:7006 --cluster-replicas 1
[root@mycat redis-6.0.10]# redis-cli -a 123456 --cluster create 172.18.28.245:7001 172.18.28.245:7002 172.18.28.244:7003 172.18.28.244:7004 172.18.28.243:7005 172.18.28.243:7006 --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.18.28.244:7004 to 172.18.28.245:7001
Adding replica 172.18.28.243:7006 to 172.18.28.244:7003
Adding replica 172.18.28.245:7002 to 172.18.28.243:7005
M: fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001
slots:[0-5460] (5461 slots) master
S: 127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002
replicates e25e8945bc8dfa06b5fd7670ceae790d93ab58ed
M: 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003
slots:[5461-10922] (5462 slots) master
S: 2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004
replicates fa92015aa467aab2c2966237093bec6c10943172
M: e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005
slots:[10923-16383] (5461 slots) master
S: b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006
replicates 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.18.28.245:7001)
M: fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004
slots: (0 slots) slave
replicates fa92015aa467aab2c2966237093bec6c10943172
S: 127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002
slots: (0 slots) slave
replicates e25e8945bc8dfa06b5fd7670ceae790d93ab58ed
S: b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006
slots: (0 slots) slave
replicates 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b
M: 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
显示集群搭建成功。
3.3 Cluster集群测试
登陆集群：redis-cli -c -h 172.18.28.245 -p 7001 -a 123456
查看集群信息：cluster info
列出节点信息：cluster nodes
[root@mycat redis-6.0.10]# redis-cli -c -h 172.18.28.245 -p 7001 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.18.28.245:7001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:24441
cluster_stats_messages_pong_sent:24548
cluster_stats_messages_sent:48989
cluster_stats_messages_ping_received:24543
cluster_stats_messages_pong_received:24441
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:48989
172.18.28.245:7001>
172.18.28.245:7001> cluster nodes
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613677838498 5 connected 10923-16383
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 slave fa92015aa467aab2c2966237093bec6c10943172 0 1613677837000 1 connected
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613677840504 5 connected
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613677839000 3 connected
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 myself,master - 0 1613677839000 1 connected 0-5460
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613677839501 3 connected 5461-10922
172.18.28.245:7001>
写入数据验证分片功能
172.18.28.245:7001> set foo bar1
-> Redirected to slot [12182] located at 172.18.28.243:7005 说明数据到了7005,172.18.28.243：7005
OK
172.18.28.243:7005> set foo bar2
OK
172.18.28.243:7005> set foo bar3
OK
172.18.28.243:7005> set foo1 bar1
OK
172.18.28.243:7005> set foo2 bar2
-> Redirected to slot [1044] located at 172.18.28.245:7001 同上
OK
172.18.28.245:7001> set foo3 bar3
OK
172.18.28.245:7001> set foo4 bar4
-> Redirected to slot [9426] located at 172.18.28.244:7003 同上
OK
172.18.28.244:7003>
172.18.28.244:7003> get foo 在7003上执行get foo,foo是写在172.18.28.243:7005上的，所以自动转到172.18.28.243:7005获取这个值。
-> Redirected to slot [12182] located at 172.18.28.243:7005
"bar3"
172.18.28.243:7005>
可以看出redis cluster集群是去中心化的，每个节点都是平等的，连接哪个节点都可以获取和设置数据。
增加节点，增加的节点角色默认都是master
redis_7007.conf在245服务器，redis_7008.conf在244服务器
[root@mycat redis-6.0.10]# redis-cli -c -h 172.18.28.245 -p 7001
172.18.28.245:7001> auth 123456
OK
172.18.28.245:7001> cluster meet 172.18.28.245 7007
OK
172.18.28.245:7001> cluster meet 172.18.28.244 7008
OK
172.18.28.245:7001> cluster nodes
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 master - 0 1613679599000 7 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613679599000 5 connected 10923-16383
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 slave fa92015aa467aab2c2966237093bec6c10943172 0 1613679599000 1 connected
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613679601793 5 connected
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613679599000 3 connected
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 myself,master - 0 1613679595000 1 connected 0-5460
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 master - 0 1613679600000 0 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613679600791 3 connected 5461-10922
172.18.28.245:7001>
在集群中将7008改为7007的slave
172.18.28.245:7001> cluster nodes
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 master - 0 1613679981663 7 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613679981000 5 connected 10923-16383
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 slave fa92015aa467aab2c2966237093bec6c10943172 0 1613679979000 1 connected
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613679982665 5 connected
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613679981000 3 connected
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 myself,master - 0 1613679983000 1 connected 0-5460
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 master - 0 1613679983668 0 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613679984671 3 connected 5461-10922
172.18.28.245:7001> cluster replicate
[root@mycat redis-6.0.10]# redis-cli -c -h 172.18.28.244 -p 7008 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.18.28.244:7008> cluster replicate 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7
OK
172.18.28.244:7008> cluster nodes
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 master - 0 1613680071028 7 connected
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613680069000 5 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613680074037 5 connected 10923-16383
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 slave fa92015aa467aab2c2966237093bec6c10943172 0 1613680071000 1 connected
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 master - 0 1613680069000 1 connected 0-5460
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 myself,slave 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 0 1613680070000 7 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613680073034 3 connected 5461-10922
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613680072031 3 connected
172.18.28.244:7008>
如红色部分所示，7008的redis服务已成功7007redis服务的slave服务。
删除节点
172.18.28.244:7008> cluster nodes
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 master - 0 1613680219000 7 connected
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613680216000 5 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613680217335 5 connected 10923-16383
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 slave fa92015aa467aab2c2966237093bec6c10943172 0 1613680221000 1 connected
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 master - 0 1613680220341 1 connected 0-5460
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 myself,slave 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 0 1613680219000 7 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613680218000 3 connected 5461-10922
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613680221343 3 connected
172.18.28.244:7008> cluster forget a57ee406e9cb504babacdcd63ed8b2194029e316
(error) ERR I tried hard but I can't forget myself... 不能删除自己的登陆节点
172.18.28.244:7008> cluster forget 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7
(error) ERR Can't forget my master! 也不能删除登陆节点对应的master节点
成功删除节点：
172.18.28.244:7008> cluster forget fa92015aa467aab2c2966237093bec6c10943172
OK
172.18.28.244:7008> cluster forget 127e48427a23769f23c09df11b65d56db1e25938
OK
172.18.28.244:7008> cluster nodes
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 master - 0 1613680492892 7 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613680493000 5 connected 10923-16383
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 slave - 0 1613680494896 1 connected
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 myself,slave 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 0 1613680492000 7 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613680491000 3 connected 5461-10922
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613680494000 3 connected
172.18.28.244:7008>
过一会删除的节点自动恢复了
172.18.28.244:7008> cluster nodes
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613680558032 5 connected
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 master - 0 1613680556000 7 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613680556000 5 connected 10923-16383
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 master - 0 1613680557000 1 connected 0-5460
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 slave fa92015aa467aab2c2966237093bec6c10943172 0 1613680553019 1 connected
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 myself,slave 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 0 1613680555000 7 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613680554021 3 connected 5461-10922
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613680557031 3 connected
172.18.28.244:7008>
ps -e
模拟7001的master挂掉，查看它的slave 7004集群服务状态
172.18.28.244:7008> cluster nodes
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613707480000 5 connected
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 master - 0 1613707481000 7 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613707481464 5 connected 10923-16383
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 slave 2e500a178dffea762e15b2f92d5819041b5a2574 0 1613707482466 8 connected
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 master - 0 1613707483469 8 connected 0-5460
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 myself,slave 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 0 1613707482000 7 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613707480462 3 connected 5461-10922
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613707480000 3 connected
172.18.28.244:7008>
如上代码所示，7004现在是master,而7001的redis重启之后变成了slave.
master挂掉会由slave自动变成master,原master重启后变成slave.
但是如果使用7007和7008来测试，7007master挂掉，7008并不会自动切换，
这是因为7007节点上根本没数据。集群数据被分为三份，采用哈希槽 (hash slot)的方式来分配16384个slot的话，它们三个节点分别承担的slot 区间是
节点7004覆盖0－5460
节点7003覆盖5461－10922
节点7005覆盖10923－16383
在集群创建时有提示。
那么增加的节点还有什么用？
3.4Cluster 伸缩
3.4.1 扩容--slot分配
对主节点重新分配slot
新增加的主节点，是没有slots的，主节点如果没有slots的话，存取数据就都不会被选中
可以把分配的过程理解成打扑克牌，all表示大家重新洗牌；输入某个主节点的node id，然后在输入done的话，就好比从某个节点，抽牌:
redis-cli --cluster help
[root@mycat redis-6.0.10]# redis-cli --cluster reshard 172.18.28.245 7007 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 172.18.28.245:7007)
M: 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007
slots: (0 slots) master
1 additional replica(s)
S: fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001
slots: (0 slots) slave
replicates 2e500a178dffea762e15b2f92d5819041b5a2574
S: b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006
slots: (0 slots) slave
replicates 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b
M: 2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002
slots: (0 slots) slave
replicates e25e8945bc8dfa06b5fd7670ceae790d93ab58ed
M: 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008
slots: (0 slots) slave
replicates 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)?
给7007分配slot,总共16384个slot,加上7007一共4台master,平均slot=16384/4=4096
所以输入4096，然后要求输入node ID,直接输入all
分配完成后，再次执行分配 会出错。
可以看到7007,分别从原有的节点中分配了一小段slot：
[0-1364]后面接7004的slot
[5461-6826]后面接7003的slot
[10923-12287]后面接7005的slot
172.18.28.245:7007> cluster nodes
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 slave 2e500a178dffea762e15b2f92d5819041b5a2574 0 1613710623517 8 connected
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613710622000 3 connected
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 master - 0 1613710623000 8 connected 1365-5460
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613710625521 5 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613710623000 3 connected 6827-10922
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613710624518 5 connected 12288-16383
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 slave 28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 0 1613710625000 9 connected
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 myself,master - 0 1613710621000 9 connected 0-1364 5461-6826 10923-12287
172.18.28.245:7007>
至此,redis cluster 扩容成功。
再次测试7007挂掉的高可用是否正常切换
成功切换并且slot也正确，启动7007,要将7007,7008的slot划到7003
redis-cli --cluster reshard 172.18.28.244 7008 -a 123456
3.4.2 缩容--slot分配
计划将7007和7008下线
确认分配slot计划
执行分配slot计划完成，查看节点信息，里面有slot分布。然后可以将7007和7008下线了。
172.18.28.245:7001> cluster nodes
fa92015aa467aab2c2966237093bec6c10943172 172.18.28.245:7001@17001 myself,slave 2e500a178dffea762e15b2f92d5819041b5a2574 0 1613712605000 8 connected
a57ee406e9cb504babacdcd63ed8b2194029e316 172.18.28.244:7008@17008 master - 0 1613712608000 10 connected
5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 172.18.28.244:7003@17003 master - 0 1613712609489 11 connected 0-1364 5461-12287
b75ca6573efa61300316f501c2fda0e3dbd7aee0 172.18.28.243:7006@17006 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613712610000 11 connected
e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 172.18.28.243:7005@17005 master - 0 1613712610492 5 connected 12288-16383
2e500a178dffea762e15b2f92d5819041b5a2574 172.18.28.244:7004@17004 master - 0 1613712611000 8 connected 1365-5460
127e48427a23769f23c09df11b65d56db1e25938 172.18.28.245:7002@17002 slave e25e8945bc8dfa06b5fd7670ceae790d93ab58ed 0 1613712611495 5 connected
28e8dadc06b30b7a4dc3b3b536b388ba3f0389b7 172.18.28.245:7007@17007 slave 5e1834ac9b5c3798af4c850403ec0c3d8363ac9b 0 1613712608489 11 connected
172.18.28.245:7001>
未完待续。。。