redis主从方案

一、主从简单配置

系统环境: CentOS Linux release 7.6.1810 (Core)
redis版本: 4.1

主从配置明细

| IP | 端口 | 描述 | 配置文件 |
| 127.0.0.1 | 6379 | 主redis | redisMaster.conf |
| 127.0.0.1 | 6380 | 从redis-1 | – |
| 127.0.0.1 | 6381 | 从redis-2 | – |
| 127.0.0.1 | 6382 | 从redis-3 | – |

主redis配置文件redisMaster.conf

port 6379
requirepass qazwsxedcrfvtgbyhnujmik

从redis配置文件redisSlaveOne.conf

port 6381
slaveof 127.0.0.1 6379
masterauth qazwsxedcrfvtgbyhnujmik
slave-read-only yes  #开启只读
requirepass qazwsxedcrfvtgbyhnujmik

启动redis

 redis/redis-server ./redisMaster.conf
 redis/redis-server ./redisSlaveOne.conf
 redis/redis-server ./redisSlaveTwo.conf 
 redis/redis-server ./redisSlaveThree.conf  

查看各redis主从状态

命令: ./redis/redis-cli -h 127.0.0.1 -p 6379 -a qazwsxedcrfvtgbyhnujmik INFO replication
6379端口:

# Replication
role:master
connected_slaves:3
slave0:ip=127.0.0.1,port=6382,state=online,offset=588275,lag=1
slave1:ip=127.0.0.1,port=6381,state=online,offset=588275,lag=1
slave1:ip=127.0.0.1,port=6380,state=online,offset=588275,lag=1

6380/6381/6382端口:

# Replication
role:slave
master_host:127.0.0.1
master_port:6380
master_link_status:down
master_last_io_seconds_ago:-1
master_sync_in_progress:0

测试(从redis不支持写入操作)

[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6379 -a qazwsxedcrfvtgbyhnujmik 
127.0.0.1:6380> set tome eggs
OK
127.0.0.1:6380> exit
[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6382 -a qazwsxedcrfvtgbyhnujmik  
127.0.0.1:6382> get tome
"eggs"
127.0.0.1:6382> set tome rops
(error) READONLY You can't write against a read only slave.

二、主从高可用方案

一、Redis Sentinel（哨兵机构）方案

原理:

Sentinel(哨兵)是用于监控redis集群中Master状态的工具，是Redis 的高可用性解决方案，sentinel哨兵模式已经被集成在redis2.4之后的版本中。sentinel是redis高可用的解决方案，sentinel系统可以监视一个或者多个redis master服务，以及这些master服务的所有从服务；当某个master服务下线时，自动将该master下的某个从服务升级为master服务替代已下线的master服务继续处理请求。

sentinel可以让redis实现主从复制，当一个集群中的master失效之后，sentinel可以选举出一个新的master用于自动接替master的工作，集群中的其他redis服务器自动指向新的master同步数据。一般建议sentinel采取奇数台，防止某一台sentinel无法连接到master导致误切换。

优点：

Redis Sentinel 集群部署简单;
能够解决 Redis 主从模式下的高可用切换问题;
很方便实现 Redis 数据节点的线形扩展，轻松突破 Redis 自身单线程瓶颈，可极大满足 Redis 大容量或高性能的业务需求;
可以实现一套 Sentinel 监控一组 Redis 数据节点或多组数据节点。

缺点：

部署相对 Redis 主从模式要复杂一些，原理理解更繁琐;
资源浪费，Redis 数据节点中 slave 节点作为备份节点不提供服务;
Redis Sentinel 主要是针对 Redis 数据节点中的主节点的高可用切换，对 Redis 的数据节点做失败判定分为主观下线和客观下线两种，对于 Redis 的从节点有对节点做主观下线操作，并不执行故障转移。
不能解决读写分离问题，实现起来相对复杂。

验证

每一个配置文件对应一个redis进程:
启动四个sentinel进程，端口分别是:26379/26380/26381/26382

port 26379 #26379/26380/26381/26382
daemonize yes
sentinel monitor mymaster 127.0.0.1 6379 1
sentinel auth-pass mymaster qazwsxedcrfvtgbyhnujmik
sentinel down-after-milliseconds mymaster 5000
logfile /chroot/bak/redisGroup/logs/26379.log
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000

启动命令:

./redis/redis-sentinel ./sentinelMaster.conf 
./redis/redis-sentinel ./sentinelSlaveOne.conf 
./redis/redis-sentinel ./sentinelSlaveTwo.conf 
./redis/redis-sentinel ./sentinelSlaveThree.conf 

查看状态:

./redis/redis-cli -h 127.0.0.1 -p 26379 info Sentinel
./redis/redis-cli -h 127.0.0.1 -p 26380 info Sentinel
./redis/redis-cli -h 127.0.0.1 -p 26381 info Sentinel
./redis/redis-cli -h 127.0.0.1 -p 26382 info Sentinel

试验:

注: 主从切换后sentinel.conf及redis.conf配置文件会被系统修改
sentinel模板:

port 26379
daemonize yes
sentinel monitor mymaster 127.0.0.1 6379 1
sentinel auth-pass mymaster qazwsxedcrfvtgbyhnujmik
sentinel down-after-milliseconds mymaster 5000
logfile /chroot/bak/redisGroup/logs/26379.log
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000

正常启动master及Slave后状态:
查看master状态:

[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6379 -a qazwsxedcrfvtgbyhnujmik INFO replication  
# Replication
role:master
connected_slaves:3
slave0:ip=127.0.0.1,port=6380,state=online,offset=56,lag=0
slave1:ip=127.0.0.1,port=6381,state=online,offset=56,lag=0
slave2:ip=127.0.0.1,port=6382,state=online,offset=56,lag=0
master_replid:fe7261ac48b048c8bd7a208d0b76fe78530d2c85
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:56
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:56

查看Slave状态:

[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6382 -a qazwsxedcrfvtgbyhnujmik INFO replication 
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:4
master_sync_in_progress:0
slave_repl_offset:42
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:fe7261ac48b048c8bd7a208d0b76fe78530d2c85
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:42
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:42

启动sentinel，进程状态:

[root@oracle redisGroup]# ps -ef | grep redis
root     28897     1  0 14:04 ?        00:00:00 redis/redis-server *:6379
root     28963     1  0 14:04 ?        00:00:00 redis/redis-server *:6380
root     28980     1  0 14:04 ?        00:00:00 redis/redis-server *:6381
root     28996     1  0 14:04 ?        00:00:00 redis/redis-server *:6382
root     29600     1  0 14:08 ?        00:00:00 ./redis/redis-sentinel *:26379 [sentinel]
root     29650     1  0 14:08 ?        00:00:00 ./redis/redis-sentinel *:26380 [sentinel]
root     29662     1  0 14:08 ?        00:00:00 ./redis/redis-sentinel *:26381 [sentinel]
root     29678     1  0 14:08 ?        00:00:00 ./redis/redis-sentinel *:26382 [sentinel]

sentinel状态:

[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 26380 info Sentinel  
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=127.0.0.1:6379,slaves=3,sentinels=4

master异常挂掉后状态
master异常挂掉后查看master情况, 127.0.0.1:6381被选举为master身份.

[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6381 -a qazwsxedcrfvtgbyhnujmik INFO replication 
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6382,state=online,offset=43684,lag=0
slave1:ip=127.0.0.1,port=6380,state=online,offset=43684,lag=0

[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6380 -a qazwsxedcrfvtgbyhnujmik INFO replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6381
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0

master重启后查看状态:
重新启动原master 6379程序，需要手动修改一下原master主进程配置文件为从配置文件。

[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6381 -a qazwsxedcrfvtgbyhnujmik INFO replication
# Replication
role:master
connected_slaves:3
slave0:ip=127.0.0.1,port=6382,state=online,offset=323299,lag=1
slave1:ip=127.0.0.1,port=6380,state=online,offset=323432,lag=0
slave2:ip=127.0.0.1,port=6379,state=online,offset=323299,lag=1
master_replid:ff5862a7809c4ef5b6e181cfb92d0a5cc9b1dbf7
master_replid2:fe7261ac48b048c8bd7a208d0b76fe78530d2c85
[root@oracle redisGroup]# ./redis/redis-cli -h 127.0.0.1 -p 6379 -a qazwsxedcrfvtgbyhnujmik INFO replication  
# Replication
role:slave
master_host:127.0.0.1
master_port:6381
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0

二、keepalived方案

原理

Keepalived为负载平衡和高可用性提供了框架。负载平衡框架依赖于众所周知且广泛使用的Linux虚拟服务器（IPVS）内核模块，该模块提供第4层负载平衡。Keepalived实现了一组健康检查器，以根据其健康动态地自适应地维护和管理负载平衡的服务器池。虚拟冗余路由协议（VRRP）实现高可用性。VRRP是路由器故障转移的基础砖块。此外，keepalived实现了一组挂钩到VRRP有限状态机，提供低级别和高速协议交互。每个Keepalived框架可以独立使用或一起使用，以提供弹性基础设施。
在这种情况下，负载平衡器也可以被称为导向器或LVS路由器。
简而言之，Keepalived提供了两个主要功能：
健康检查LVS系统
实施VRRPv2堆栈以处理负载均衡器故障转移
Vip(虚拟路由器的IP地址): 一个虚拟路由器可以拥有多个ip地址,由用户进行设置；

验证

1、keepalived安装(Centos6.x安装测试存在问题)
安装依赖包

[root@localhost ~]# yum install -y curl gcc openssl-devel libnl3-devel net-snmp-devel
[root@localhost ~]# yum install -y keepalived

[root@localhost ~]# systemctl start keepalived   //启动keepalived
[root@localhost ~]# systemctl enable keepalived  //加入开机启动keepalived
[root@localhost ~]# systemctl restart keepalived  //重新启动keepalived
[root@localhost ~]# systemctl status keepalived   //查看keepalived状态

由于需要监听指定网络信息,需要三台虚拟主要进行测试，暂时没有测试验证成功。理论上用docker安装也可以实现,待验证
初步配置文件及配置结果如下:
1、查看keep运行日志tail -f /var/log/messages

2、keep配置文件 /etc/keepalived/keepalived.conf

! Configuration File for keepalived

#全局配置
global_defs {
   router_id oracle      #机器标识
   script_user root root #script执行用户名,如果这里不设置可能报无执行脚本权限
   #enable_script_security
}

vrrp_script chk_redis {
   script "/chroot/bak/keep/scripts/master/redis_check.sh 127.0.0.1 6389 qazwsxed"#监控脚本
   interval 2  #脚本执行间隔(秒)
   weight 2    #脚本结果导致的优先级变更：10表示优先级+10；-10则表示优先级-10   
}

#网卡需要注意,使用ifconfig查看一下当前活动网卡
#VRRPD子进程，VRRPD子进程就是来实现VRRP协议的
vrrp_instance VI_1 {
    state BACKUP       #设置为BACKUP
    nopreempt          #不抢占MASTER
    #interface eth0    #监控网卡
    interface enp1s0   #监控网卡
    virtual_router_id 51
    priority 100       #权重值 
    advert_int 1       #检查间隔，默认为1秒
    #smtp_alert         #表示切换时给global defs中定义的邮件地址发送邮件通知
    authentication {       #认证配置
        auth_type PASS     #认证方式，可以是PASS或AH两种认证方式
        auth_pass qazwsxed #认证密码
    }
    virtual_ipaddress {  #虚拟IP
       192.168.11.124
    }
    #设置额外的监控，里面的任意一个网卡出现问题，都会进入FAULT状态
    #track_interface {
       #enp1s0
    #}

    track_script {#引用健康检查脚本函数,执行上面定义的chk_redis 
       chk_redis
    }

    #当前节点成为master时,通知脚本执行任务
    notify_master "/chroot/bak/keep/scripts/master/redis_master.sh 127.0.0.1 6389 qazwsxed"

    #当前节点成为backup时
    notify_backup "/chroot/bak/keep/scripts/master/redis_backup.sh 127.0.0.1 6389 qazwsxed 6380"

    #当前节点出现故障，执行的任务
    notify_fault /chroot/bak/keep/scripts/master/redis_fault.sh

    #当前节点出现故障时,执行的任务
    notify_stop /chroot/bak/keep/scripts/master/redis_stop.sh
}

redis_check.sh脚本内容

#!/bin/bash  
ALIVE=`/chroot/bak/redisGroup/redisMaster/redis-cli -h $1 -p $2 -a $3 PING`
#ALIVE=`/chroot/bak/redisGroup/redisMaster/redis-cli -h 127.0.0.1 -p 6389 -a qazwsxed PING`
LOGFILE="/chroot/bak/keep/logs/keepalived-master-redis.log"
echo "———————————————–" >> $LOGFILE
echo "DATE:" $(date +"%Y-%m-%d %H:%M:%S") >> $LOGFILE
echo "PING: $ALIVE" >> $LOGFILE
echo "———————————————– " >> $LOGFILE
if [ "$ALIVE" == "PONG" ]; then
  echo $ALIVE
  exit 0
else
  echo $ALIVE 
  exit 1
fi

redis_master.sh脚本内容

#!/bin/bash
REDISCLI="/chroot/bak/redisGroup/redisMaster/redis-cli -h $1 -p $2 -a $3"
LOGFILE="/chroot/bak/keep/logs/keepalived-master-redis.log"
echo "———————————————–" >> $LOGFILE
echo "DATE:" $(date +"%Y-%m-%d %H:%M:%S") >> $LOGFILE
echo "[master]" >> $LOGFILE
#date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
#设置当前节点为主节点
#slaveof no one 取消同步
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
echo "———————————————– " >> $LOGFILE

redis_backup.sh脚本内容

#!/bin/bash  
REDISCLI="/chroot/bak/redisGroup/redisMaster/redis-cli  -h $1 -p $2 -a $3"
LOGFILE="/chroot/bak/keep/logs/keepalived-master-redis.log"
echo "———————————————–" >> $LOGFILE
echo "DATE:" $(date +"%Y-%m-%d %H:%M:%S") >> $LOGFILE
echo "[backup]" >> $LOGFILE
#date >> $LOGFILE
echo "Being slave...." >> $LOGFILE 2>&1
echo "Run SLAVEOF cmd ..." >> $LOGFILE
#设置主redis信息
$REDISCLI SLAVEOF $1 $4 >> $LOGFILE  2>&1  #IP为指定主redis的IP
echo "———————————————– " >> $LOGFILE

redis_fault.sh脚本内容

#!/bin/bash 
LOGFILE="/chroot/bak/keep/logs/keepalived-master-redis.log"
echo "———————————————–" >> $LOGFILE
echo "DATE:" $(date +"%Y-%m-%d %H:%M:%S") >> $LOGFILE
echo "STATUS: [fault]" >> $LOGFILE
echo "———————————————– " >> $LOGFILE

redis_stop.sh脚本内容

#!/bin/bash 
LOGFILE="/chroot/bak/keep/logs/keepalived-master-redis.log"
echo "———————————————–" >> $LOGFILE
echo "DATE:" $(date +"%Y-%m-%d %H:%M:%S") >> $LOGFILE
echo "STATUS: [stop]" >> $LOGFILE
echo "———————————————– " >> $LOGFILE

脚本监控日志:

[root@oracle logs]# tail -f keepalived-master-redis.log 
PING: 
———————————————– 
———————————————–
DATE: 2019-11-29 11:42:00
PING: 
———————————————– 
———————————————–
DATE: 2019-11-29 11:42:02
PING: 
———————————————– 
———————————————–
DATE: 2019-11-29 11:42:04
PING: 

keepalived运行日志:

Nov 29 11:41:39 oracle Keepalived_vrrp[4516]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on enp1s0 for 192.168.11.124
Nov 29 11:41:39 oracle Keepalived_vrrp[4516]: Sending gratuitous ARP on enp1s0 for 192.168.11.124
Nov 29 11:41:39 oracle Keepalived_vrrp[4516]: Sending gratuitous ARP on enp1s0 for 192.168.11.124
Nov 29 11:41:39 oracle Keepalived_vrrp[4516]: Sending gratuitous ARP on enp1s0 for 192.168.11.124
Nov 29 11:41:39 oracle Keepalived_vrrp[4516]: Sending gratuitous ARP on enp1s0 for 192.168.11.124
Nov 29 11:41:40 oracle Keepalived_vrrp[4516]: /chroot/bak/keep/scripts/master/redis_check.sh 127.0.0.1 6389 qazwsxed exited with status 1
Nov 29 11:41:42 oracle Keepalived_vrrp[4516]: /chroot/bak/keep/scripts/master/redis_check.sh 127.0.0.1 6389 qazwsxed exited with status 1

三、cacheCloud简单介绍

官网:https://cachecloud.github.io/
git地址: https://github.com/sohutv/cachecloud

CacheCloud提供一个Redis云管理平台：实现多种类型(Redis Standalone、Redis Sentinel、Redis Cluster)自动部署、解决Redis实例碎片化现象、提供完善统计、监控、运维功能、减少运维成本和误操作，提高机器的利用率，提供灵活的伸缩性，提供方便的接入客户端。

1、添加机器
2、添加/导入应用
注意: 这里的应用停用与启用是直接对实际的redis进行启动功关闭
全局统计中的应用下线也是对redis进行开启或关闭操作