redis+keepalived 主从热备 实例

    应用环境：

    操作系统：centos 5.6 x64

   redis版本：redis-2.8.10.tar.gz 

keepalived版本：keepalived-1.2.12.tar.gz

redis  master : 192.168.101.50

redis     salve : 192.168.101.51

原理：如果返回时间超过keepalived vrrp_scripts的超时时间,keepalived会进入faild 状态,从而触发keepalived集群开始进行vip漂移

当 Master 挂掉，Slave 正常时, Slave接管服务，有写权限，同时关闭主从复制功能；

当 Master 恢复正常，则从Slave同步数据，同步数据之后关闭主从复制功能，恢复Master身份，同时Slave等待Master同步数据完成之后，恢复Slave身份。

部署安装：

1、在master  salve 上编译安装依赖包

      yum -y install gcc gcc+ gcc-c++ openssl openssl-devel pcre pcre-devel  

软件包download：

               

                redis-2.8.10.tar.gz               wget http://www.915c.com/softfile/redis-2.8.10.tar.gz

                keepalived-1.2.12.tar.gz      wget http://www.keepalived.org/software/keepalived-1.2.12.tar.gz 

                gperftools-2.1.tar.gz            wget http://down.51cto.com/data/1332597

                 libunwind-1.1.tar.gz      wget http://down.51cto.com/data/1332597

2、编译安装 (master salve  安装方法一致)

    

# tar xf  libunwind-1.1.tar.gz 

# cd libunwind-1.1

# CFLAGS=-fPIC ./configure 
# make CFLAGS=-fPIC 

# make CFLAGS=-fPIC install 

# tar xf gperftools-2.1.tar.gz 
# cd gperftools-2.1 

# /configure  --disable-cpu-profiler --disable-heap-profiler --disable-heap-checker --disable-debugalloc --enable-minimal 

#  make && make install 

# echo "/usr/local/lib" > /etc/ld.so.conf.d/usr_local_lib.conf 
# /sbin/ldconfig 

# cd /usr/local/lib 

# ln -sv libtcmalloc_minimal.so.4.1.2 libtcmalloc.so 

# tar xf redis-2.8.10.tar.gz 

# cd redis2.8.10 

# mkdir -p /opt/redis 

# make PREFIX=/opt/redis USE_TCMALLOC=yes FORCE_LIBC_MALLOC=yes install 

3、配置redis

创建配置文件夹以及文件 

mkdir -p /opt/redis/etc 

mkdir -p /opt/redis/run 
mkdir -p /opt/redis/data/6379 
mkdir -p /opt/redis/log 
cp /redis的解压路径 /redis.conf /opt/redis/etc/redis.conf 
cp /opt/redis/etc/redis.conf /opt/redis/etc/redis_6379.conf 
修改配置文件： 
vim /opt/redis/etc/redis_6379.conf 
daemonize yes 
pidfile /opt/redis/run/redis_6379.pid 
dir /opt/redis/data/6379 
logfile /opt/redis/log/redis_6379.log 
创建服务管理脚本 
vim /etc/init.d/redis 
#!/bin/sh 
PATH="/opt/redis/bin:$PATH" 
EXEC="/opt/redis/bin/redis-server" 
CLIEXEC="/opt/redis/bin/redis-cli" 
PIDFILE="/opt/redis/run/redis_6379.pid" 
CONF="/opt/redis/etc/redis_6379.conf" 
REDISPORT="6379" 

case "$1" in 
    start) 
        if [ -f $$PIDFILE ] 
        then 
                echo "$PIDFILE exists, process is already running or crashed." 
        else 
                echo "Starting Redis server..." 
                $EXEC $CONF 
        fi 
        ;; 
    stop) 
        if [ ! -f $PIDFILE ] 
        then 
                echo "$PIDFILE does not exist, process is not running." 
        else 
                PID=$(cat $PIDFILE) 
                echo "Stopping ..." 
                $CLIEXEC -p $REDISPORT shutdown 
                while [ -x /proc/${PID} ] 
                do 
                    echo "Waiting for Redis to shutdown ..." 
                    sleep 1 
                done 
                echo "Redis stopped." 
        fi 
        ;; 
    *)    
        echo "Usage: $0 {start|stop}" >&2 
        exit 1 
        ;; 
esac 

授权 chmod +x /etc/init.d/redis 

vim /etc/sysctl.conf 
在最后添加以下节点: 

vm.overcommit_memory = 1 

sysctl -p 

redis 就安装完毕了

4、启动测试

启动redis 服务

# /etc/init.d/redis start

#ps -aux |grep redis  查看进程

打开防火墙6379端口

5、测试redis的主从复制

[root@master ~]# /opt/redis/bin/redis-cli -p 6379 set test1 www.baidu.com       \\通过master  set设置一个key键为test1 写入values值  www.baidu.com  

返回一个ok

在master本地测试取test1 的key值 

[root@master ~]# /opt/redis/bin/redis-cli -p 6379 get test1

在salve端测试读取master写入的文件test1

[root@slave ~]# /opt/redis/bin/redis-cli -p 6379 get test1

6、安装keepalived 

# tar xf keepalived-1.2.12.tar.gz

# cd keepalived-1.2.12

# ./configure

# make && make install

# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/

# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

# chmod +x /etc/init.d/keepalived

# chkconfig -add keepalived

# mkdir /etc/keepalived

# ln -s /usr/local/sbin/keepalived /usr/sbin

7、首先在master上创建如下配置

# mkdir -p /etc/keepalived

# vim /etc/keepalived/keepalived.conf

vrrp_script chk_redis { 

                script "/etc/keepalived/scripts/redis_check.sh"   ###监控脚本 

                interval 2                                        ###监控时间 

} 

vrrp_instance VI_1 { 

        state MASTER                            ###设置为MASTER

        interface eth0                          ###监控网卡    

        virtual_router_id 51

        priority 101                            ###权重值

        authentication { 

                     auth_type PASS             ###加密 

                     auth_pass redis            ###密码 

        } 

        track_script { 

                chk_redis                       ###执行上面定义的chk_redis

        } 

        virtual_ipaddress { 

             192.168.101.52                       ###VIP 

        }

        notify_master /etc/keepalived/scripts/redis_master.sh

        notify_backup /etc/keepalived/scripts/redis_backup.sh

        notify_fault  /etc/keepalived/scripts/redis_fault.sh

        notify_stop   /etc/keepalived/scripts/redis_stop.sh 

}

 

在Redis Master上创建redis_master脚本 

# vim /etc/keepalived/scripts/redis_master.sh

 

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[master]" >> $LOGFILE

date >> $LOGFILE

echo "Being master...." >> $LOGFILE 2>&1

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.51(slave) 6379 >> $LOGFILE  2>&1

sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态

echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

在Redis Master上创建redis_backup脚本   

# vim /etc/keepalived/scripts/redis_backup.sh

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[backup]" >> $LOGFILE

date >> $LOGFILE

echo "Being slave...." >> $LOGFILE 2>&1

sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.51(slave)  6379 >> $LOGFILE  2>&1

 

在slave上创建如下文件：

# vim /etc/keepalived/keepalived.conf

vrrp_script chk_redis { 

                script "/etc/keepalived/scripts/redis_check.sh"   ###监控脚本 

                interval 2                                        ###监控时间 

} 

vrrp_instance VI_1 { 

        state BACKUP                                ###设置为BACKUP 

        interface eth0                              ###监控网卡

        virtual_router_id 51 

        priority 100                                ###比MASTRE权重值低 

        authentication { 

                     auth_type PASS 

                     auth_pass redis                ###密码与MASTRE相同

        } 

        track_script { 

                chk_redis                       ###执行上面定义的chk_redis

        } 

        virtual_ipaddress { 

             192.168.101.52                         ###VIP 

        } 

        notify_master /etc/keepalived/scripts/redis_master.sh

        notify_backup /etc/keepalived/scripts/redis_backup.sh

        notify_fault  /etc/keepalived/scripts/redis_fault.sh

        notify_stop   /etc/keepalived/scripts/redis_stop.sh 

}

在Redis salve上创建redis_master脚本 

vim /etc/keepalived/scripts/redis_master.sh

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[master]" >> $LOGFILE

date >> $LOGFILE

echo "Being master...." >> $LOGFILE 2>&1

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.50(master) 6379 >> $LOGFILE  2>&1

sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态

echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

在Redis Master上创建redis_backup脚本   

# vim /etc/keepalived/scripts/redis_backup.sh

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[backup]" >> $LOGFILE

date >> $LOGFILE

echo "Being slave...." >> $LOGFILE 2>&1

sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.50  6379 >> $LOGFILE  2>&1

 

以下脚本分别在master和slave上创建

#  mkdir /etc/keepalived/scripts

#  vim /etc/keepalived/scripts/redis_check.sh

#!/bin/bash

 

ALIVE=`/opt/redis/bin/redis-cli PING`

if [ "$ALIVE" == "PONG" ]; then

  echo $ALIVE

  exit 0

else

  echo $ALIVE

  exit 1

fi

 # vim /etc/keepalived/scripts/redis_fault.sh

#!/bin/bash

LOGFILE=/var/log/keepalived-redis-state.log

echo "[fault]" >> $LOGFILE

date >> $LOGFILE

 #  vim /etc/keepalived/scripts/redis_stop.sh

#!/bin/bash

LOGFILE=/var/log/keepalived-redis-state.log

echo "[stop]" >> $LOGFILE

date >> $LOGFILE

 

授执行权  chmod +x /etc/keepalived/scripts/*.sh

8、测试高可用性。

1、启动master和slave节点上keepalived

[root@master ~]# /etc/init.d/keepalived start

启动 keepalived：                                          [确定]

[root@slave ~]# /etc/init.d/keepalived start

Starting keepalived:                                       [  OK  ]

查看master  的网络地址 vip地址的绑定状态

查看slave

9、测试vip连接redis

[root@master ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 INFO |grep -A8 Replication

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 INFO |grep -A8 Replication

10、测试插入和取出key

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 set oneday www.baidu.com

OK

You have new mail in /var/spool/mail/root

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 get oneday 

"www.baidu.com"

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.50 get oneday 

"www.baidu.com"

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.51 get oneday 

"www.baidu.com"

11、测试vip漂移和redis高可用

关闭master节点的redis服务

[root@master ~]# /etc/init.d/redis stop

Stopping ...

Redis stopped.

查看vip 地址 slave变化状态

master

slave

12、测试故障恢复redis master恢复

启动master节点的redis 服务器，观察变化情况

slave节点

slave自动切换为slave节点服务

到此测试完成。

备注：

以上方案存在不足之处：1、当master节点故障后，虽然slave节点接管了服务，但是毕竟slave没有写入的权限，导致slave需要写入文件的时候报错

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 set twoday www.sina.net

(error) READONLY You can't write against a read only slave.

这个时候，如果是强制将slave节点手打或者脚本升级为master节点，则出现当master恢复时，原master与slave已经隔离，不能完成数据同步。

2、通过哨兵Redis Sentinel 监控master slave 监听可以实现当master出故障时，slave可以自动切换为master并接管工作，但当master端恢复后，会一直停留在slave状态，不会再抢占成为master节点。

3、抛出一个问题：假如master 是一台服务器性能比较高，而slave是一台性能较差的服务器，如何做到，当master节点故障时，slave自动切换为master并且有读写的权限，当原master节点恢复后，自动抢占成为master节点？