Keepalived+Redis是一种比较成熟的Redis高可用解决方案，通过keepalived实现虚拟ip（vip）的切换以及Reids进程监控，并控制Redis的主从同步状态。以下是架构图：
环境介绍：
Saltstack-master: 192.168.39.185
Master：192.168.39.100
Slave：192.168.39.101
VIP：192.168.39.23
高可用状态情况：
1）当Master端与Slave端都运行正常时，Master负责服务，Slave作为备用。
2）当Master端运行不正常时，slave接管服务，同时关闭主从复制功能。
3）当Master端恢复正常时，则从slave同步数据，作为Slave的从，不切换成主。
4）当slave端挂掉，master端恢复成主。
以下通过saltstack完成keepalived+redis的高可用架构部署：
首先，修改master配置文件指定top file文件路径：
[root@centos7 ~]# vim /etc/salt/master file_roots: base: - /srv/salt
修改salt配置文件需要重启服务。
整个状态文件目录结构如下：
设置主从模式的Grains(Grains为模版文件设置变量，通常也可以用Pillar设置)：
设置master端和slave端的Grains，指定role、vip和redis的master_ip:
[root@centos7 ~]# salt "centos7-1" grains.setvals "{'role':'master','vip':'192.168.39.23','master_ip':'192.168.39.101'}"
[root@centos7 ~]# salt "centos7-2" grains.setvals "{'role':'slave','vip':'192.168.39.23','master_ip':'192.168.39.100'}"
可以查看到Grains已经设置成功：
设置入口文件top.sls：
以list匹配方式对两台主机应用对应模块：
[root@centos7 salt]# cat top.sls base: 'centos7-1,centos7-2': - match: list - redis - keepalived
一、部署Redis
安装redis并根据角色下发配置模版文件。
Redis安装入口文件：
[root@centos7 redis]# cat init.sls include: - .install_redis - .redis_running - .redis_conf
Redis安装：
[root@centos7 redis]# cat install_redis.sls
{% for s in ["jemalloc-3.6.0-1.el7.x86_64.rpm","redis-3.2.12-1.el7.x86_64.rpm",] %}
copy_{{ s }}:
file.managed:
- name: /usr/local/src/{{ s }}
- source: salt://redis/file/{{ s }}
- user: root
- group: root
- template: jinja
- mode: 755
{% endfor %}
install_redis:
cmd.run:
- name: cd /usr/local/src/ && yum install -y redis-3.2.12-1.el7.x86_64.rpm jemalloc-3.6.0-1.el7.x86_64.rpm
- require:
- file: copy_redis-3.2.12-1.el7.x86_64.rpm
这里通过require指定依赖关系，将rpm都下发到节点下之后，进行安装redis。
Redis配置洗发以及创建目录：
[root@centos7 redis]# cat redis_conf.sls redis_conf: file.managed: - name: /etc/redis.conf - source: salt://redis/templates/redis.j2 - user: root - group: root - template: jinja - mode: 644 - require: - cmd: install_redis redis_dir: cmd.run: - name: mkdir -pv /data/redis && chown redis.redis /data/redis - unless: test -d /data/redis
unless指定通过test命令判断目录是否存在，如果不存在则执行name。
Redis服务启动控制：
[root@centos7 redis]# cat redis_running.sls redis_running: service.running: - name: redis - enable: True - require: - cmd: install_redis - watch: - file: redis_conf
通过watch状态监控，如果就重启服务。
Redis配置文件模版：
[root@centos7 templates]# grep -v "#" redis.j2
bind 0.0.0.0
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis/redis.pid
loglevel notice
logfile /var/log/redis/redis.log
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /data/redis/
{% if grains['role'] == "slave" %}
slaveof {{grains['master_ip']}} 6379
{% endif %}
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
二、部署keepalived
keepalived入口文件：
[root@centos7 keepalived]# cat init.sls include: - .install_keepalived - .keepalived_running - .keepalived_conf
keepalived安装文件：
[root@centos7 keepalived]# cat install_keepalived.sls install_keepalived: pkg.installed: - name: keepalived
pkg指定通过yum源进行安装。
keepalived配置下发以及创建目录：
[root@centos7 keepalived]# cat keepalived_conf.sls
keepalived_conf:
file.managed:
- name: /etc/keepalived/keepalived.conf
- source: salt://keepalived/templates/keepalived.conf
- user: root
- group: root
- template: jinja
- mode: 644
- require:
- pkg: install_keepalived
- cmd: scripts_dir
scripts_dir:
cmd.run:
- name: mkdir -pv /etc/keepalived/scripts/
- unless: test -d /etc/keepalived/scripts/
{% for s in ["redis_backup.sh","redis_check.sh","redis_fault.sh","redis_master.sh","redis_stop.sh"] %}
keepalived_{{s}}:
file.managed:
- name: /etc/keepalived/scripts/{{s}}
- source: salt://keepalived/templates/scripts/{{s}}
- user: root
- group: root
- template: jinja
- mode: 755
- require:
- pkg: install_keepalived
{% endfor %}
keepalived服务启动文件：
[root@centos7 keepalived]# cat keepalived_running.sls keepalived_running: service.running: - name: keepalived - enable: True - require: - pkg: install_keepalived - watch: - file: keepalived_conf
keepalived模板文件和脚本文件：
keepalived.conf主配置文件根据role值动态下发配置文件：
[root@centos7 templates]# cat keepalived.conf
! Configuration File for keepalived
vrrp_script chk_redis {
script "/etc/keepalived/scripts/redis_check.sh"
interval 2
}
vrrp_instance VI_1 {
{% if grains['role'] == 'master' %}
state MASTER
{% else %}
state BACKUP
{% endif %}
interface eth0
virtual_router_id 51
{% if grains['role'] == "master" %}
priority 101
{% else %}
priority 100
{% endif %}
# advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass redis
}
track_script {
chk_redis
}
virtual_ipaddress {
{{ grains['vip'] }}
}
notify_master /etc/keepalived/scripts/redis_master.sh
notify_backup /etc/keepalived/scripts/redis_backup.sh
notify_fault /etc/keepalived/scripts/redis_fault.sh
notify_stop /etc/keepalived/scripts/redis_stop.sh
}
keepalived配置的几个脚本，当keeplaived在转换状态时会按照状态来呼叫：
当进入master状态时会呼叫notify_master；
当进入backup状态时会呼叫notify_backup；
当发现异常情况时进入fault状态呼叫notify_fault；
当keepalived程序终止时则呼叫notify_stop。
redis_backup.sh-Redis健康检测脚本：
[root@centos7 scripts]# cat redis_check.sh #!/bin/bash # ALIVE=`/usr/bin/redis-cli PING` if [ "$ALIVE" == "PONG" ];then echo $ALIVE exit 0 else echo $ALIVE exit 1 fi
redis_master.sh-切换master触发执行脚本：
[root@centos7 scripts]# cat redis_master.sh
#!/bin/bash
#
REDISCLI="/usr/bin/redis-cli"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master..." >> $LOGFILE 2>&1
echo "run slaveof no one cmd..." >> $LOGFILE
$RESISCLI SLAVEOF {{ grains['master_ip'] }} 6479>> $LOGFILE 2>&1
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
redis_backup.sh-切换成backup角色执行脚本：
[root@centos7 scripts]# cat redis_backup.sh
#!/bin/bash
#
REDISCLI="/usr/bin/redis-cli"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[backup]" >> $LOGFILE
date >> $LOGFILE
echo "being salve..." >> $LOGFILE 2>&1
#sleep 25
echo "run slaveof cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF {{ grains['master_ip'] }} 6379 $LOGFILE 2>&1
redis_fault.sh-进入fault状态执行脚本：
[root@centos7 scripts]# cat redis_fault.sh #!/bin/bash # $LOGFILE=/var/log/keepalived-redis-state.log echo "[fault]" >> $LOGFILE date >> $LOGFILE
redis_stop.sh-当keepalived服务终止时执行此脚本：
[root@centos7 scripts]# cat redis_stop.sh #!/bin/bash # LOGFILE=/var/log/keepalived-redis-state.log echo "[stop]" >> $LOGFILE date >> $LOGFILE
所有状态配置文件和脚本设置完成执行：
在执行时我们可以通过指定参数，模拟执行：
[root@centos7 salt]# salt 'centos7-[1-2]' state.highstate test=True
测试执行没有其他问题，执行安装：
[root@centos7 salt]# salt 'centos7-[1-2]' state.highstate
执行结构如下：
会看到centos7-1节点，执行完成之后的状态：
测试keepalived+redis高可用：
登陆centos7-1查看vip情况:
在saltstack服务端通过连接vip测试登陆情况：
关闭centos7-1下的redis服务：
[root@centos7-1:/usr/local/src] # service redis stop Redirecting to /bin/systemctl stop redis.service [root@centos7-1:/usr/local/src] # service redis status Redirecting to /bin/systemctl status redis.service ● redis.service - Redis persistent key-value database Loaded: loaded (/usr/lib/systemd/system/redis.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/redis.service.d └─limit.conf Active: inactive (dead) since Fri 2018-08-31 11:41:44 CST; 5s ago Process: 6414 ExecStop=/usr/libexec/redis-shutdown (code=exited, status=0/SUCCESS) Process: 6147 ExecStart=/usr/bin/redis-server /etc/redis.conf --supervised systemd (code=exited, status=0/SUCCESS) Main PID: 6147 (code=exited, status=0/SUCCESS) Aug 31 11:40:17 centos7-1 systemd[1]: Starting Redis persistent key-value database... Aug 31 11:40:17 centos7-1 systemd[1]: Started Redis persistent key-value database. Aug 31 11:41:44 centos7-1 systemd[1]: Stopping Redis persistent key-value database... Aug 31 11:41:44 centos7-1 systemd[1]: Stopped Redis persistent key-value database.
重新连接vip登陆redis查看设置name值：
登陆centos7-2下查看vip切换情况:
参考文献:《saltstack运维实战》