Keepalived+redis高可用redis主从环境搭建

环境信息：

master ： 10.1.58.19
slave：10.1.58.29
VIP：10.1.58.150

安装软件包关联

#yum -y install libnl-devel
#yum -y install libnfnetlink-devel

1、确认lvs是否安装
如下命令检查kernel是否已经支持LVS的IPVS模块：

#modprobe -l |grep ipvs
kernel/net/netfilter/ipvs/ip_vs.ko
kernel/net/netfilter/ipvs/ip_vs_rr.ko
kernel/net/netfilter/ipvs/ip_vs_wrr.ko
kernel/net/netfilter/ipvs/ip_vs_lc.ko
kernel/net/netfilter/ipvs/ip_vs_wlc.ko
kernel/net/netfilter/ipvs/ip_vs_lblc.ko
kernel/net/netfilter/ipvs/ip_vs_lblcr.ko
kernel/net/netfilter/ipvs/ip_vs_dh.ko
kernel/net/netfilter/ipvs/ip_vs_sh.ko
kernel/net/netfilter/ipvs/ip_vs_sed.ko
kernel/net/netfilter/ipvs/ip_vs_nq.ko
kernel/net/netfilter/ipvs/ip_vs_ftp.ko
kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko

安装IPVS管理软件

#yum install ipvsadm -y

2、部署keepalived
主/备：

# wget http://www.keepalived.org/software/keepalived-1.3.6.tar.gz
# tar xzvf keepalived-1.3.6.tar.gz
# cd keepalived-1.3.6
# ./configure --prefix=/home/yzkf/app/keepalived 
# make
# make  install
# ln -s /home/yzkf/app/keepalived/sbin/keepalived /sbin/keepalived
# ll /sbin/keepalived
lrwxrwxrwx 1 root root 41 Oct 25 11:20 /sbin/keepalived -> /home/yzkf/app/keepalived/sbin/keepalived

制作随机启动，在源代码目录下(如果需要制作随机启动，可以用下面的方法)

#cp /home/yzkf/src_soft/keepalived-1.3.6/keepalived/etc/init.d/keepalived.rh.init /etc/init.d/keepalived
#chkconfig --add keepalived
#chkconfig  keepalived on

备注：因为启动脚本启动时会去读取默认的配置文件,即/etc/keepalived/keepalived.conf，所以需要修改/etc/init.d/keepalived的内容：如下

start() {
        echo -n "Starting Keepalived for LVS: "
        daemon keepalived  -f /home/yzkf/app/keepalived/etc/keepalived/keepalived.conf -D
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && touch /var/lock/subsys/keepalived
        return $RETVAL
}

3、部署redis主从
redis配置文件不能使用bind指令，具体安装步骤参考文档《Redis单实例安装部署》

4、配置keepalived
4.1、修改主节点keepalived配置：

#cat /home/yzkf/app/keepalived/etc/keepalived/keepalived.conf 
! Configuration File for keepalived
global_defs {
    notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server smtp.richinfo.cn    
   smtp_connect_timeout 30
   router_id redis-10.1.58.19        
}

vrrp_script chk_redis 
{ 
     script "/home/yzkf/app/script/redis/redis_check.sh 127.0.0.1 56379" 
     interval 2 
     timeout 2
     fall 3
     weight  -10
}

vrrp_script chk_redis_port 
{
     script      "/bin/echo 

redis监控脚本：

$ cat /home/yzkf/app/script/redis/redis_check.sh 
#!/bin/bash 

passwd=F33czco

ALIVE=`/home/yzkf/app/redis/bin/redis-cli  -h $1 -p $2 -a ${passwd} PING` 
LOGFILE="/home/yzkf/logs/redis/keepalived-redis-check.log" 

v_date=`date +"%Y%m%d %T"`

echo "${v_date} [CHECK]" >> $LOGFILE

if [ $ALIVE == "PONG" ]; then :
    echo "Success: redis-cli -h $1 -p $2 PING $ALIVE" >> $LOGFILE 2>&1
    exit 0 
else 
    echo "Failed:redis-cli -h $1 -p $2 PING $ALIVE " >> $LOGFILE 2>&1
    exit 1 
fi

redis监控脚本：redis_master.sh

$ cat  /home/yzkf/app/script/redis/redis_master.sh  
#!/bin/bash

passwd=F33czco

REDISCLI="/home/yzkf/app/redis/bin/redis-cli -h $1 -p $3 -a ${passwd}" 
LOGFILE="/home/yzkf/logs/redis/keepalived-redis-state.log" 

v_date=`date +"%Y%m%d %T"`

echo "${v_date} Begin [master]" >> $LOGFILE 2>&1 

###echo "Run MASTER cmd ..." >> $LOGFILE 2>&1
###$REDISCLI SLAVEOF $2 $3 >> $LOGFILE  
###sleep 10 #delay 10 s wait data async cancel sync
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

v_date=`date +"%Y%m%d %T"`

echo "${v_date} END [master]" >> $LOGFILE 2>&1

# send mail
echo "[WARNING] 10.1.58.19 SERVER GET VIP:10.1.58.150, Entering MASTER STATE"| mutt -s "Keepalived STATUS Warning-rcsoa_test env" [email protected] [email protected]

redis监控脚本：redis_backup.sh

$ cat  /home/yzkf/app/script/redis/redis_backup.sh
#!/bin/bash

passwd=F33czco

REDISCLI="/home/yzkf/app/redis/bin/redis-cli -h $1 -p $3 -a ${passwd}" 
LOGFILE="/home/yzkf/logs/redis/keepalived-redis-state.log" 

v_date=`date +"%Y%m%d %T"`

echo "${v_date} Begin [backup]" >> $LOGFILE 

echo "Run SLAVEOF cmd ..." >> $LOGFILE 
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE 2>&1 
#sleep 15 #delay 15 s wait data sync exchange role

v_date=`date +"%Y%m%d %T"`

echo "${v_date} End [backup]" >> $LOGFILE

# send mail
echo "[WARNING] 10.1.58.19 SERVER REMOVE VIP:10.1.58.150, Entering BACKUP STATE"| mutt -s "Keepalived STATUS Warning-rcsoa_test env" [email protected] [email protected]

redis_fault.sh 脚本：

$ cat /home/yzkf/app/script/redis/redis_fault.sh
#!/bin/bash

LOGFILE=/home/yzkf/logs/redis/keepalived-redis-state.log 
v_date=`date +"%Y%m%d %T"`

echo "${v_date} [fault]" >> $LOGFILE

#send mail
echo "[WARNING] Keepalived status is fail on server:10.1.58.19"| mutt -s "Keepalived STATUS" [email protected] [email protected]

redis_stop.sh 脚本：

$ cat /home/yzkf/app/script/redis/keepalived_stop.sh
#!/bin/bash

LOGFILE=/home/yzkf/logs/redis/keepalived-redis-state.log 
v_date=`date +"%Y%m%d %T"`

echo "${v_date} [stop]" >> $LOGFILE

# send mail
echo "[WARNING] Keepalived status is  stop on  SERVER:10.1.58.19"| mutt -s "Keepalived STATUS Warning" [email protected] [email protected]

4.2、修改备节点keepalived配置

$ cat /home/yzkf/app/keepalived/etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
    notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server pop.richinfo.cn    
   smtp_connect_timeout 30
   router_id redis-10.1.58.29        
}

vrrp_script chk_redis 
{ 
     script "/home/yzkf/app/script/redis/redis_check.sh 127.0.0.1 56379" 
     interval 2
     timeout 2
     fall 3
     weight  -10
}

vrrp_script chk_redis_port 
{
     script      "/bin/echo 

redis监控脚本（跟主一样）。

5.1、启动redis

5.2、启动keepalived

#service keepalived start

5.3、尝试通过VIP连接redis：

[root@localhost ~]# /home/middle/app/redis/bin/redis-cli -h 192.168.19.239 -p 6379 -a test1
192.168.19.239:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.19.242,port=6379,state=online,offset=14,lag=1
master_replid:ffa361951c84f0ce9b7cac18c52d527600a422c0
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:14
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:14

尝试写一些数据：

192.168.19.239:6379> set Hello Redis
OK
192.168.19.239:6379> get Hello
"Redis"

从Master读取数据：

[root@localhost ~]# /home/middle/app/redis/bin/redis-cli -h 192.168.19.240 -p 6379 -a test1 get Hello    
"Redis"

从slave读取数据：

[root@puppetm ~]# /home/middle/app/redis/bin/redis-cli -h 192.168.19.242 -p 6379 -a test1 get Hello 
"Redis"

5.4、模拟故障产生
使用kill -9 redis 模拟机器宕机

查看Master上的Keepalived日志

同时Slave上的日志显示：

可以发现，Slave已经接管服务，并且担任Master的角色了

同时VIP也漂移到slave上：