Redis之——搭建高可用及负载均衡的Redis

之前，给大家介绍了一些关于Redis的文章，大家可以参见博文中有关Redis的文章。今天，我们就一起来学习如何搭建高可用及负载均衡的Redis，好了，不多说了，我们直接进入正题吧。

一、测试环境

1、机器

   母机：centos6.5-64
   虚拟机:centos6.5-64 单核 1G 独立ip 3个
   虚拟机使用VMWare,centos为64位6.5.具体信息如下：

 192.168.0.231   Redis-master
 192.168.0.231   Redis-slave
 192.168.0.239   Virtual IP

2、软件

    Reidis
    Keepalived

二、安装前配置

1、虚拟机最小化安装

虚拟机最小化安装，更新系统及安装必须组件

 $ yum install -y openssl openssl-devel kernel kernel-devel gcc wget tcl libnl-devel
 $ yum upgrade
 $ reboot  //重启系统

2、安装Keepalived

安装keepalived 下载地址:http://www.keepalived.org/software/keepalived-1.2.13.tar.gz 安装

 $ wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz

 $ tar -zxvf keepalived-1.2.13.tar.gz

 $ cd keepalived-1.2.13

 $ ln -s /usr/src/kernels/`uname -r`-`uname -m`/ /usr/src/linux

 $ ./configure  --prefix=/usr/local/keepalived  --sysconf=/etc

 $ chkconfig --add keepalived

 //查看是否成功

 $ service keepalived status

3、安装redis

下载Redis,目前Redis的稳定版本是2.8.17，下载地址Redis-2.8.17

 $ wget http://download.redis.io/releases/redis-2.8.17.tar.gz

 $ tar -zxvf redis-2.8.17.tar.gz

 $ cd redis-2.8.17

 $ make && make test

 $ mkdir -p /usr/local/redis/{bin,var,etc}

 $ cp ./src/{redis-benchmark,redis-cli,redis-server,redis-check-dump,redis-check-aof} /usr/local/redis/bin/

 $ cp /usr/local/redis/bin/* /usr/local/bin/

三、配置

1、Master配置

 先配置好Redis，主从复制成功之后再行配置Keepalived。
 Master-Redis配置

 $ vim /usr/local/redis/etc/redis.conf

修改如下值

 daemonize yes
 pidfile /var/run/redis.pid
 port 6379
 tcp-backlog 511
 timeout 0
 tcp-keepalive 0
 loglevel notice
 logfile "/usr/local/redis/log/redis.log"
 databases 16
 save 900 1
 save 300 10
 save 60 10000
 stop-writes-on-bgsave-error yes
 rdbcompression yes
 rdbchecksum yes
 dbfilename dump.rdb
 dir /usr/local/redis/data
 slave-serve-stale-data yes
 slave-read-only no
 repl-disable-tcp-nodelay no
 slave-priority 100
 appendonly yes
 appendfilename "appendonly.aof"
 appendfsync everysec
 no-appendfsync-on-rewrite no
 auto-aof-rewrite-percentage 100
 auto-aof-rewrite-min-size 64mb
 lua-time-limit 5000
 slowlog-log-slower-than 10000
 slowlog-max-len 128
 notify-keyspace-events ""
 hash-max-ziplist-entries 512
 hash-max-ziplist-value 64
 list-max-ziplist-entries 512
 list-max-ziplist-value 64
 set-max-intset-entries 512
 zset-max-ziplist-entries 128
 zset-max-ziplist-value 64
 activerehashing yes
 client-output-buffer-limit normal 0 0 0
 client-output-buffer-limit slave 256mb 64mb 60
 client-output-buffer-limit pubsub 32mb 8mb 60
 hz 10
 aof-rewrite-incremental-fsync yes

Master-Keepalived配置

 global_defs {
     router_id redis
 }
 vrrp_script chk_redis {
     script "/etc/keepalived/scripts/redis_check.sh"   ###监控脚本
     interval 2                  ###监控时间
     timeout  2                                  ###超时时间
     fall     3                  ###重试次数
 }
 vrrp_instance redis {
   state MASTER                  ###设置为MASTER
   interface eth0              ###监控网卡
   virtual_router_id 52
   priority 101              ###权重值
   authentication {
          auth_type PASS          ###加密
          auth_pass redis        ###密码
   }
   track_script {
     chk_redis              ###执行上面定义的chk_redis
   }
   virtual_ipaddress {
        192.168.0.239            ###VIP
   }
   notify_master /etc/keepalived/scripts/redis_master.sh
   notify_backup /etc/keepalived/scripts/redis_backup.sh
   notify_fault  /etc/keepalived/scripts/redis_fault.sh
   notify_stop   /etc/keepalived/scripts/redsi_stop.sh
 }

也可在本地编辑好后使用 scp 上传到服务器。 Master-keepalived脚本redis_master.sh:

 #!/bin/bash
 REDISCLI="/usr/local/redis/bin/redis-cli"
 LOGFILE="/usr/local/redis/var/keepalived-redis-state.log"
 echo "[master]" >> $LOGFILE
 date >> $LOGFILE
 echo "Being master...." >> $LOGFILE 2>&1
 echo "Run SLAVEOF cmd ..." >> $LOGFILE
 $REDISCLI SLAVEOF 192.168.0.232 6379 >> $LOGFILE  2>&1
 sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态
 echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
 $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

Master-keepalived脚本redis_backup.sh:

 #!/bin/bash
 REDISCLI="/usr/local/redis/bin/redis-cli"
 LOGFILE="/usr/local/redis/log/keepalived-redis-state.log"
 echo "[backup]" >> $LOGFILE
 date >> $LOGFILE
 echo "Being slave...." >> $LOGFILE 2>&1
 sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色
 echo "Run SLAVEOF cmd ..." >> $LOGFILE
 $REDISCLI SLAVEOF 192.168.0.232 6379 >> $LOGFILE  2>&1

2、Backup配置

同样先配置好Redis，主从复制成功之后再行配置Keepalived。
Backup-Redis配置

 $ vim /usr/local/redis/etc/redis.conf

修改如下值

 daemonize yes
 pidfile /var/run/redis.pid
 port 6379
 tcp-backlog 511
 timeout 0
 tcp-keepalive 0
 loglevel notice
 logfile "/usr/local/redis/log/redis.log"
 databases 16
 save 900 1
 save 300 10
 save 60 10000
 stop-writes-on-bgsave-error yes
 rdbcompression yes
 rdbchecksum yes
 dbfilename dump.rdb
 dir /usr/local/redis/data
 slave-serve-stale-data yes
 slave-read-only no
 repl-disable-tcp-nodelay no
 slave-priority 100
 appendonly yes
 appendfilename "appendonly.aof"
 appendfsync everysec
 no-appendfsync-on-rewrite no
 auto-aof-rewrite-percentage 100
 auto-aof-rewrite-min-size 64mb
 lua-time-limit 5000
 slowlog-log-slower-than 10000
 slowlog-max-len 128
 notify-keyspace-events ""
 hash-max-ziplist-entries 512
 hash-max-ziplist-value 64
 list-max-ziplist-entries 512
 list-max-ziplist-value 64
 set-max-intset-entries 512
 zset-max-ziplist-entries 128
 zset-max-ziplist-value 64
 activerehashing yes
 client-output-buffer-limit normal 0 0 0
 client-output-buffer-limit slave 256mb 64mb 60
 client-output-buffer-limit pubsub 32mb 8mb 60
 hz 10
 aof-rewrite-incremental-fsync yes
 SLAVEOF 192.168.0.231 6379

Backup-Keepalived配置

 global_defs {
     router_id redis
 }
 vrrp_script chk_redis {
     script "/etc/keepalived/scripts/redis_check.sh"   ###监控脚本
     interval 2                  ###监控时间
     timeout  2                                  ###超时时间
     fall     3                  ###重试次数
 }
 vrrp_instance redis {
   state BACKUP                  ###设置为MASTER
   interface eth0              ###监控网卡
   virtual_router_id 51
   priority 100              ###权重值
   authentication {
          auth_type PASS          ###加密
          auth_pass redis        ###密码
   }
   track_script {
     chk_redis              ###执行上面定义的chk_redis
   }
   virtual_ipaddress {
        192.168.0.239            ###VIP
   }
   notify_master /etc/keepalived/scripts/redis_master.sh
   notify_backup /etc/keepalived/scripts/redis_backup.sh
   notify_fault  /etc/keepalived/scripts/redis_fault.sh
   notify_stop   /etc/keepalived/scripts/redsi_stop.sh
 }

也可在本地编辑好后使用 scp 上传到服务器。 Master-keepalived脚本redis_master.sh:

 #!/bin/bash
  REDISCLI="/usr/local/redis/bin/redis-cli"
  LOGFILE="/usr/local/redis/var/keepalived-redis-state.log"
  echo "[master]" >> $LOGFILE
  date >> $LOGFILE
  echo "Being master...." >> $LOGFILE 2>&1
  echo "Run SLAVEOF cmd ..." >> $LOGFILE
  $REDISCLI SLAVEOF 192.168.0.231 6379 >> $LOGFILE  2>&1
  sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态
  echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
  $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

Master-keepalived脚本redis_backup.sh:

 #!/bin/bash
 REDISCLI="/usr/local/redis/bin/redis-cli"
 LOGFILE="/usr/local/redis/log/keepalived-redis-state.log"
 echo "[backup]" >> $LOGFILE
 date >> $LOGFILE
 echo "Being slave...." >> $LOGFILE 2>&1
 sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色
 echo "Run SLAVEOF cmd ..." >> $LOGFILE
 $REDISCLI SLAVEOF 192.168.0.231 6379 >> $LOGFILE  2>&1

四、公共脚本配置

1、redis_chekc.sh

可以使用netcat来检测redis服务的状态.

安装nc，使用nc检测redis的服务而不是ping

 $ wget http://sourceforge.net/projects/netcat/files/netcat/0.7.1/netcat-0.7.1.tar.gz
 $ tar -zxvf netcat-0.7.1.tar.gz
 $ ./configure
 $ make && make install

建立脚本:

 $ vim /etc/keepalived/scripts/redis_check.sh

脚本内容:

 #!/bin/bash
 ALIVE=`/usr/local/redis/bin/redis-cli PING`

 if ["$ALIVE" == "PONG"] ;then
       echo $ALIVE
       exit 0
 else
       echo $ALIVE
       exit 1
 fi

2、redis_fault.sh

建立脚本:

 $ vim /etc/keepalived/scripts/redis_fault.sh

脚本内容:

 # !/bin/bash

 LOGFILE=/usr/local/redis/var/keepalived-redis-state.log
 echo "[faule]" >> $LOGFILE
 date >> $LOGFILE

3、redis_stop.sh

建立脚本:

 $ vim /etc/keepalived/scripts/redis_stop.sh

脚本内容:

 # !/bin/bash

 LOGFILE=/usr/local/redis/var/keepalived-redis-state.log
 echo "[stop]" >> $LOGFILE
 date >> $LOGFILE

五、测试

1、主从复制测试

先打开Master上的Redis，再打开Backup上的Redis，分别查看info

 $ redis-cli info

Master的rule显示master为正常，slave会显示为192.168.0.232
在Master上写入数据，在Backup读取

 $ redis-cli set a hello  //Master 写入数据
 $ redis-cli get a        //Backup 读取数据
 "hello"

主从复制正常。

2、主从切换测试

先后启动Master和Backup上的keepalived，在Master上查看vip

 $ ip a

从图中可以看出vip已经绑定在Master上，现在通过vip来进行数据测试

 $ redis-cli -h 192.168.0.239 set a test
 $ redis-cli -h 192.168.0.239 get a //测试虚拟ip数据读取
 //在主机和从机上分别进行数据读取
 $ redis-cli get a

从图中可以看出测试无问题。

现在关闭Master上的redis，来看VIP能否正常漂移，并进行数据测试。

 $ killall -9 redis-server
 $ redis-cli -h 192.168.0.232 info

现在给redis写入数据，测试看Master恢复服务后能否正常恢复数据

 $ redis-cli -h 192.168.0.239 set a nihao

恢复Master上的Redis，查看ip a

 $ service redis-server start
 $ ip a
 $ redis-cli get a

数据恢复，VIP切换回Master，Redis高可用环境搭建成功.

至此，已成功搭建高可用及负载均衡的Redis环境。