linux在6台主机搭建3组一主两从的redis哨兵集群

序言

之前在测试环境搭建了3组一主两从的redis集群，最近又在生产环境成功搭建，为了便于以后自己查看，所以也做一下记录。

其实也可以当成搭建redis集群，搭建完成之后改一下配置就可以了。

我之前写了一篇搭建reids集群的博客：https://blog.csdn.net/fhf2424045058/article/details/105430060可供参考！

不过我还是要从头再来！！！

部署主机情况说明

我们的生产环境这次只给我分了6台主机，主机有限，但也不是不够，所以依然和测试一样搭建9个节点。

如果主机比较多，可以直接来它个12台，这feel真好！

说明：下表中的IP地址，在生产环境肯定不是如此的，为了保密，所以用127来代替。

主机列表和端口分布

主机IP	端口
127.0.0.1	7000、7008
127.0.0.2	7001
127.0.0.3	7002、7003
127.0.0.4	70004
127.0.0.5	7005、7006
127.0.0.6	7007

其中7000、7003和7006是master，其他的是slave，放上我写的我们生产的部署情况的一点点截图来说明一下关系

下面就开始从头搭建啦~

一、redis一主两从集群搭建

1、新建用户

在6台主机新建用户

groupadd rdsclsr
useradd -m -d /bal/rdsclsr -g rdsclsr -s /bin/bash -u 501 rdsclsr

2、解压

在6台主机解压redis安装包

tar -zxvf redis-5.0.8.tar.gz

3、编译redis

cd /bal/rdsclsr/redis/redis-5.0.8/
mkdir installs
make

4、安装redis

还是老习惯，安装到普通用户下的自定义的目录

make PREFIX=/bal/rdsclsr/redis/redis-5.0.8/installs install

5、创建节点目录

在不同主机创建9个redis节点配置文件目录

从这里开始：我就不详细再列出是哪台主机了，你一定可以根据我列出的命令来判断是在哪台主机。

cd  /bal/rdsclsr/redis/
mkdir -p rediscluster/s_1_1_7000/data
mkdir -p rediscluster/s_1_2_7001/data
mkdir -p rediscluster/s_1_3_7002/data
mkdir -p rediscluster/s_2_1_7003/data
mkdir -p rediscluster/s_2_2_7004/data
mkdir -p rediscluster/s_2_3_7005/data
mkdir -p rediscluster/s_3_1_7006/data
mkdir -p rediscluster/s_3_2_7007/data
mkdir -p rediscluster/s_3_3_7008/data

说明：可能有小伙伴看到这里就会疑问，为啥端口前面要加上s_1_1这样的，当然是为了方便运维，当你打开的主机过多的时候，只看端口会一时反映不过来，这是哪组的主还是备，当然也可以根据自己的喜好来创建。

6、修改配置文件（见附录一）

修改6台主机中redis的配置文件

vim /bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis-7000.conf
vim /bal/rdsclsr/redis/rediscluster/s_1_2_7001/redis-7001.conf
vim /bal/rdsclsr/redis/rediscluster/s_1_3_7002/redis-7002.conf
vim /bal/rdsclsr/redis/rediscluster/s_2_1_7003/redis-7003.conf
vim /bal/rdsclsr/redis/rediscluster/s_2_2_7004/redis-7004.conf
vim /bal/rdsclsr/redis/rediscluster/s_2_3_7005/redis-7005.conf
vim /bal/rdsclsr/redis/rediscluster/s_3_1_7006/redis-7006.conf
vim /bal/rdsclsr/redis/rediscluster/s_3_2_7007/redis-7007.conf
vim /bal/rdsclsr/redis/rediscluster/s_3_3_7008/redis-7008.conf

说明：配置文件Master和slave的配置文件内容和说明我会粘贴在本文的最后，可以直接跳到最后看（左侧有目录哦~）。

7、启动9个redis节点

cd /bal/rdsclsr/redis/redis-5.0.8/installs/bin
./redis-server  /bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis-7000.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_1_2_7001/redis-7001.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_1_3_7002/redis-7002.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_2_1_7003/redis-7003.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_2_2_7004/redis-7004.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_2_3_7005/redis-7005.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_3_1_7006/redis-7006.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_3_2_7007/redis-7007.conf
./redis-server  /bal/rdsclsr/redis/rediscluster/s_3_3_7008/redis-7008.conf

8、编写9个节点的启动脚本

（1）创建脚本目录

cd /bal/rdsclsr/redis/rediscluster
mkdir updownsh

（2）编辑脚本文件

vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_1_7000.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_2_7001.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_3_7002.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_1_7003.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_2_7004.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_3_7005.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_1_7006.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_2_7007.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_3_7008.sh

（3）9个节点启动脚本内容：

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis-7000.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_1_2_7001/redis-7001.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_1_3_7002/redis-7002.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_2_1_7003/redis-7003.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_2_2_7004/redis-7004.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_2_3_7005/redis-7005.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_3_1_7006/redis-7006.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_3_2_7007/redis-7007.conf

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_3_3_7008/redis-7008.conf

说明：从这里和后面的脚本中：9个脚本以#!/bin/bash开头为一个脚本，我也就不分开一行一行的写在这里了，在编辑的时候以#!/bin/bash为分割是一个脚本。

（4）给9个启动脚本赋予执行权限

chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_1_7000.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_2_7001.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_3_7002.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_1_7003.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_2_7004.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_3_7005.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_1_7006.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_2_7007.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_3_7008.sh

9、编写9个节点的停止脚本

（1）编辑脚本文件

vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_1_7000.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_2_7001.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_3_7002.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_1_7003.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_2_7004.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_3_7005.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_1_7006.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_2_7007.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_3_7008.sh

（2）9个节点停止脚本内容：

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.1  -p 7000  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.2  -p 7001  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.3  -p 7002  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.3  -p 7003  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.4  -p 7004  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.5  -p 7005  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.5  -p 7006  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.6  -p 7007  shutdown

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.1  -p 7008  shutdown

（3）给9个停止脚本赋予执行权限

chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_1_7000.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_2_7001.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_3_7002.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_1_7003.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_2_7004.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_3_7005.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_1_7006.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_2_7007.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_3_7008.sh

10、编辑登录客户端脚本

（1）脚本编辑命令

cd  /bal/rdsclsr/redis/rediscluster/updownsh
vim redis_cli_s_1_1_7000.sh
vim redis_cli_s_1_2_7001.sh
vim redis_cli_s_1_3_7002.sh
vim redis_cli_s_2_1_7003.sh
vim redis_cli_s_2_2_7004.sh
vim redis_cli_s_2_3_7005.sh
vim redis_cli_s_3_1_7006.sh
vim redis_cli_s_3_2_7007.sh
vim redis_cli_s_3_3_7008.sh

（2）登录客户端命令

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7000

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7001

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7002

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7003

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7004

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7005

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7006

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7007

#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7008

（3）赋予执行权限

chmod +x redis_cli_s_1_1_7000.sh
chmod +x redis_cli_s_1_2_7001.sh
chmod +x redis_cli_s_1_3_7002.sh
chmod +x redis_cli_s_2_1_7003.sh
chmod +x redis_cli_s_2_2_7004.sh
chmod +x redis_cli_s_2_3_7005.sh
chmod +x redis_cli_s_3_1_7006.sh
chmod +x redis_cli_s_3_2_7007.sh
chmod +x redis_cli_s_3_3_7008.sh

二、Redis哨兵部署

3个master端口和主机搭建和配置哨兵

主机IP	端口-角色
127.0.0.1	7000-master
127.0.0.3	7003-master
127.0.0.5	7006-master

1、创建sentinel目录

在3组哨兵的主机中创建sentinel的配置文件目录和日志目录

cd /bal/rdsclsr/redis
mkdir sentinel

2、修改哨兵的配置文件（见附录二）

cp /bal/rdsclsr/redis/redis-5.0.8/sentinel.conf /bal/rdsclsr/redis/sentinel
vim /bal/rdsclsr/redis/sentinel/sentinel.conf

说明：可以看左侧的目录直接找到附录二哦~

3、在3台master主机启动sentinel

（1）编辑启动脚本

cd   /bal/rdsclsr/redis/sentinel
vim upsentinel.sh

sentinel启动脚本内容

#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-sentinel /bal/rdsclsr/redis/sentinel/sentinel.conf

赋予执行权限并启动

chmod +x upsentinel.sh 
sh upsentinel.sh 

（2）编辑停止脚本

cd   /bal/rdsclsr/redis/sentinel
vim downsentinel.sh

3个sentinel的停止脚本内容

#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.1  -p 25379  shutdown
#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.3  -p 25379  shutdown
#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.5  -p 25379  shutdown

赋予执行权限

chmod +x downsentinel.sh 

三、查看主从信息

------------------------------------这里以7000这一组为例---------------------------------------

查看master所在主机的进程：

ps -ef|grep redis

显示内容：

rdsclsr  28017     1  0 14:40 ?        00:00:02 /bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-sentinel 127.0.0.1:25379 [sentinel]
rdsclsr  31013     1  0 15:28 ?        00:00:00 ./redis-server *:7000
rdsclsr  32555 26320  0 15:53 pts/1    00:00:00 grep --color=auto redis

（1）查看第一组redis哨兵的master

/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7000 info replication

显示内容：

# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.2,port=7001,state=online,offset=6412164,lag=0
slave1:ip=127.0.0.3,port=7002,state=online,offset=6412164,lag=0
master_replid:6008be2704b2e007b93d784613e41d205027f128
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:6412164
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:5363589
repl_backlog_histlen:1048576

（2）查看第一组redis哨兵slave1

/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7001 info replication 

显示内容：

# Replication
role:slave
master_host:127.0.0.1
master_port:7000
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:6418476
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:6008be2704b2e007b93d784613e41d205027f128
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:6418476
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:5369901
repl_backlog_histlen:1048576

（3）查看第一组redis哨兵slave2

/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7002 info replication 

显示内容：

# Replication
role:slave
master_host:127.0.0.1
master_port:7000
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:6418955
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:6008be2704b2e007b93d784613e41d205027f128
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:6418955
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:5370380
repl_backlog_histlen:1048576

（4）模拟master宕机，选举slave2为master

/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7002 info replication 

显示内容：

# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.2,port=7001,state=online,offset=277553,lag=0
master_replid:ccfd0f854252af7c7c3ae2da03a0a0ea143596ad
master_replid2:d6a8f4c9052a3169d6ea8f54c76a9b9169cd31fa
master_repl_offset:277553
second_repl_offset:271060
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:277553

到这里三组哨兵，每组一主两备就搭建完成了。

最后附上附录一和附录二

附录一

为了不让篇幅太长，我就把里面的注释去掉

protected-mode no

port 7000

tcp-backlog 511

timeout 300

tcp-keepalive 300

daemonize yes

supervised no

pidfile "/bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis_s_1_1_7000.pid"

loglevel notice

logfile "/bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis_s_1_1_7000.log"

databases 16

always-show-logo yes


save 900 1
save 300 10
save 60 10000

stop-writes-on-bgsave-error yes

rdbcompression yes

rdbchecksum yes

dbfilename "dump.s_1_1_7000.rdb"

dir "/bal/rdsclsr/redis/rediscluster/s_1_1_7000/data"

################################# REPLICATION #################################
# replicaof  
################################# REPLICATION #################################


replica-serve-stale-data yes

replica-read-only yes

repl-diskless-sync no

repl-diskless-sync-delay 5

repl-disable-tcp-nodelay no

replica-priority 100

lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no

appendonly yes

appendfilename "appendonly.aof"

appendfsync everysec

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

aof-load-truncated yes

aof-use-rdb-preamble yes

lua-time-limit 5000

slowlog-log-slower-than 10000

slowlog-max-len 128

latency-monitor-threshold 0

notify-keyspace-events ""

hash-max-ziplist-entries 512
hash-max-ziplist-value 64

list-max-ziplist-size -2

list-compress-depth 0

set-max-intset-entries 512

zset-max-ziplist-entries 128
zset-max-ziplist-value 64

hll-sparse-max-bytes 3000

stream-node-max-bytes 4096
stream-node-max-entries 100

activerehashing yes

client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60

hz 10

dynamic-hz yes

aof-rewrite-incremental-fsync yes

rdb-save-incremental-fsync yes

maxclients 4064

说明1 : redis.conf的配置项比较多，但如果要改成其他的节点的配置，需要修改的内容也并不多，
其他两组redis的master只需修改port、 pidfile、 logfile、 dbfilename、dir这5个值即可。

说明2：我的附录一是以7000为例的，而6个slave的redis.conf配置文件和master的redis.conf配置文件除了port、 pidfile、 logfile、 dbfilename、dir这5个值不同以外，
7000的2个slave的配置文件还需要添加 ”replicaof 127.0.0.1 7000” 配置，在这个附录一我特意留了几行注释:

################################# REPLICATION #################################
 replicaof  
################################# REPLICATION #################################

把里面的masterip和masterport换成master的ip和port即可，也就是slave需要添加对应的master的主机IP和端口。
所以，其他的5个slave的redis.conf的配置文件需要就只需要修改port、 pidfile、 logfile、 dbfilename、dir和replicaof 即可。

说明3：如果有时间和兴趣可以把注释都翻译一下，或者用翻译工具也可以，配置项的解释可以在网上找找，如果不想找可以看我之前参考的博客：文章链接

附录二

# sentinel configuration by fanhf in 202006152012

port 25379

daemonize yes

pidfile "/bal/rdsclsr/redis/sentinel/redis-sentinel.pid"

logfile "/bal/rdsclsr/redis/sentinel/redis-sentinel.log"

dir "/bal/rdsclsr/redis/sentinel"

protected-mode no

sentinel deny-scripts-reconfig no

sentinel monitor redis_22122_group1 127.0.0.1 7000 1 
sentinel down-after-milliseconds redis_22122_group1 10000 
sentinel failover-timeout redis_22122_group1 10000 
sentinel parallel-syncs redis_22122_group1 1 
sentinel config-epoch redis_22122_group1 0

sentinel monitor redis_22122_group2 127.0.0.3 7003 1 
sentinel down-after-milliseconds redis_22122_group2 10000 
sentinel failover-timeout redis_22122_group2 10000 
sentinel parallel-syncs redis_22122_group2 1 
sentinel config-epoch redis_22122_group2 0

sentinel monitor redis_22122_group3 127.0.0.5 7006 1 
sentinel down-after-milliseconds redis_22122_group3 10000 
sentinel failover-timeout redis_22122_group3 10000 
sentinel parallel-syncs redis_22122_group3 1
sentinel config-epoch redis_22122_group3 0

说明：在3台master所在主机同时还有sentinel，每个sentine的配置都是一样的，也就是说每个sentinel都会监控9个节点。

到这里就搭建完成了，在搭建的过程中有任何问题可以留言，相互交流，共同进步！

与君共勉！！！