Redis + Keepalived主从集群的搭建及故障转移
Redis + Keepalived主从集群的搭建及故障转移
设计思路：
- 当 Master 与 Slave 均运作正常时, Master负责服务，Slave负责Standby；
- 当 Master 挂掉，Slave 正常时, Slave接管服务，同时关闭主从复制功能；
- 当 Master 恢复正常，则从Slave同步数据，同步数据之后关闭主从复制功能，恢复Master身份，于此同时Slave等待Master同步数据完成之后，恢复Slave身份。
- 然后依次循环。
注意事项:
需要在Master与Slave上都开启本地化策略，否则在互相自动切换的过程中，未开启本地化的一方会将另一方的数据清空，造成数据完全丢失。
环境准备
集群可以是单台机器的不同端口，也可以是多台机器，通常实际应用是，多台机器。假设目前已经安装好了2台机器，IP和端口分别是
主Redis服务器IP：192.168.1.148
主Redis服务器端口号：6379
从Redis服务器IP：192.168.1.158
从Redis服务器端口号：6379
设定一个虚拟IP
192.168.1.200
进入root用户
xiaoyao@xiaoyao-virtual-machine:~$ su
密码：
root@xiaoyao-virtual-machine:/home/xiaoyao#
主从两个机器上分别安装Keepalived
root@xiaoyao-virtual-machine:/home/xiaoyao# apt-get install keepalived
主从服务器分别修改hosts文件
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/hosts
在文件末尾添加
192.168.1.148 redis148
192.168.1.158 redis158
主服务器创建配置文件
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/keepalived.conf
配置内容：
! Configuration File for keepalived
global_defs {
router_id redis148
}
vrrp_script chk_redis
{
script "/etc/keepalived/scripts/redis_check.sh 127.0.0.1 6379"
interval 2
timeout 2
fall 3
}
vrrp_instance redis {
state MASTER # master set to SLAVE also
interface eth0
virtual_router_id 50
priority 150
nopreempt # no seize,must add
advert_int 1
authentication { #all node must same
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.200/24
}
track_script {
chk_redis
}
notify_master "/etc/keepalived/scripts/redis_master.sh 127.0.0.1 192.168.1.158 6379"
notify_backup "/etc/keepalived/scripts/redis_backup.sh 127.0.0.1 192.168.1.158 6379"
notify_fault /etc/keepalived/scripts/redis_fault.sh
notify_stop /etc/keepalived/scripts/redis_stop.sh
}
这里可能会出错，一般虚拟机配置成：interface eth0，但是有的机器可能是em1，这个要根据情况来配置，否则创建虚拟IP会失败。
举个例子：
查看IP后是下面的样子
root@ubuntu:/etc/init.d# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto em1
iface em1 inet static
address 192.168.0.33
netmask 255.255.255.0
gateway 192.168.0.1
那么就应该配置成
interface em1
从服务器创建配置文件
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/keepalived.conf
配置内容：
! Configuration File for keepalived
global_defs {
router_id redis158
}
vrrp_script chk_redis
{
script "/etc/keepalived/scripts/redis_check.sh 127.0.0.1 6379"
interval 2
timeout 2
fall 3
}
vrrp_instance redis {
state BACKUP
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication { #all node must same
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.200/24
}
track_script {
chk_redis
}
notify_master "/etc/keepalived/scripts/redis_master.sh 127.0.0.1 192.168.1.148 6379"
notify_backup "/etc/keepalived/scripts/redis_backup.sh 127.0.0.1 192.168.1.148 6379"
notify_fault /etc/keepalived/scripts/redis_fault.sh
notify_stop /etc/keepalived/scripts/redis_stop.sh
}
在主服务器和从服务器上分别创建监控Redis的脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# mkdir /etc/keepalived/scripts
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_check.sh
脚本内容：
#!/bin/bash
ALIVE=`/usr/local/bin/redis-cli -h $1 -p $2 PING`
LOGFILE="/var/log/keepalived-redis-check.log"
echo "[CHECK]" >> $LOGFILE
date >> $LOGFILE
if [ $ALIVE == "PONG" ]; then :
echo "Success: redis-cli -h $1 -p $2 PING $ALIVE" >> $LOGFILE 2>&1
exit 0
else
echo "Failed:redis-cli -h $1 -p $2 PING $ALIVE " >> $LOGFILE 2>&1
exit 1
fi
Keepalived在转换状态时会根据状态来呼叫：
- 当进入Master状态时会呼叫redis_master
- 当进入Backup状态时会呼叫redis_backup
- 当发现异常情况时进入Fault状态呼叫redis_fault
- 当Keepalived程序终止时则呼叫redis_stop
下面在主从服务器分别创建下列脚本
在主服务器上创建redis_master脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_master.sh
脚本内容
#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli -h $1 -p $3"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "Run MASTER cmd ..." >> $LOGFILE 2>&1
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE
sleep 10 #delay 10 s wait data async cancel sync
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
在主服务器上创建redis_backup脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_backup.sh
脚本内容
#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[backup]" >> $LOGFILE
date >> $LOGFILE
echo "Run SLAVEOF cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE 2>&1
# echo "Being slave...." >> $LOGFILE 2>&1
sleep 15 #delay 15 s wait data sync exchange role
在主服务器上创建redis_fault脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_fault.sh
脚本内容
#!/bin/bash
LOGFILE=/var/log/keepalived-redis-state.log
echo "[fault]" >> $LOGFILE
date >> $LOGFILE
在主服务器上创建redis_stop脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_stop.sh
脚本内容
#!/bin/bash
LOGFILE=/var/log/keepalived-redis-state.log
echo "[stop]" >> $LOGFILE
date >> $LOGFILE
在从服务器上创建redis_master脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_master.sh
脚本内容
#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli -h $1 -p $3"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "Run SLAVEOF cmd ... " >> $LOGFILE
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE 2>&1
#echo "SLAVEOF $2 cmd can't excute ... " >> $LOGFILE
sleep 10 ##delay 15 s wait data sync exchange role
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
在从服务器上创建redis_backup脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_backup.sh
脚本内容
#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[BACKUP]" >> $LOGFILE
date >> $LOGFILE
echo "Being slave...." >> $LOGFILE 2>&1
echo "Run SLAVEOF cmd ..." >> $LOGFILE 2>&1
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE
sleep 100 #delay 10 s wait data async cancel sync
exit(0)
在从服务器上创建redis_fault脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_fault.sh
脚本内容
#!/bin/bash
LOGFILE=/var/log/keepalived-redis-state.log
echo "[fault]" >> $LOGFILE
date >> $LOGFILE
在从服务器上创建redis_stop脚本
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_stop.sh
脚本内容
#!/bin/bash
LOGFILE=/var/log/keepalived-redis-state.log
echo "[stop]" >> $LOGFILE
date >> $LOGFILE
主服务器和从服务器分别给脚本添加可执行权限
root@xiaoyao-virtual-machine:/home/xiaoyao# chmod +x /etc/keepalived/scripts/*.sh
启动服务测试
启动主服务器上的Redis
root@xiaoyao-virtual-machine:/home/xiaoyao# service redis start启动从服务器上的Redis
root@xiaoyao-virtual-machine:/home/xiaoyao# service redis start启动主服务器上的Keepalived
root@xiaoyao-virtual-machine:/home/xiaoyao# /etc/init.d/keepalived start启动从服务器上的Keepalived
root@xiaoyao-virtual-machine:/home/xiaoyao# /etc/init.d/keepalived start通过虚拟ip连接Redis
root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.200 INFO # Replication role:master connected_slaves:1 slave0:ip=192.168.1.158,port=6379,state=online,offset=15,lag=1 master_repl_offset:15 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:14通过虚拟ip插入一条数据
root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.200 SET Hello Redis主从服务器均能读到此条数据
root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.200 GET Hello root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.148 GET Hello root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.158 GET Hello在主服务器上杀死redis进程
root@xiaoyao-virtual-machine:/home/xiaoyao# killall -9 redis-server
查看主服务器上的Keepalived日志
root@xiaoyao-virtual-machine:/home/xiaoyao# tailf /var/log/keepalived-redis-state.log查看从服务器上的Keepalived日志
root@xiaoyao-virtual-machine:/home/xiaoyao# tailf /var/log/keepalived-redis-state.log查看从服务器上的信息
root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.158 INFO # Replication role:master connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:0