搭建高可用及负载均衡的REDIS
转载:http://blog.molibei.com/archives/87
项目决定数据存储使用Redis为主，Mysql只保存日志信息。需要对Redis的高可用及负载均衡做一个部署测试。测试使用虚拟机用。
测试环境
机器
母机：Ubuntu 4核 16G
虚拟机:centos6.5-64 单核 1G 独立ip 3个
虚拟机使用
virtual box,centos为64位6.5.具体信息如下192.168.0.231 Redis-master192.168.0.231 Redis-slave192.168.0.239 Virtual IP
软件
Reidis
Keepalived
安装前配置
虚拟机最小化安装，更新系统及安装必须组件
$ yum install -y openssl openssl-devel kernel kernel-devel gcc wget tcl libnl-devel$ yum upgrade $ reboot //重启系统
安装Keepalived
安装keepalived 下载地址:http://www.keepalived.org/software/keepalived-1.2.13.tar.gz 安装：
$ wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz$ tar -zxvf keepalived-1.2.13.tar.gz$ cd keepalived-1.2.13$ ln -s /usr/src/kernels/`uname -r`-`uname -m`/ /usr/src/linux$ ./configure --prefix=/usr/local/keepalived --sysconf=/etc$ chkconfig --add keepalived//查看是否成功$ service keepalived status
安装redis
下载Redis,目前Redis的稳定版本是2.8.17，下载地址Redis-2.8.17
$ wget http://download.redis.io/releases/redis-2.8.17.tar.gz$ tar -zxvf redis-2.8.17.tar.gz $ cd redis-2.8.17$ make && make test $ mkdir -p /usr/local/redis/{bin,var,etc} $ cp ./src/{redis-benchmark,redis-cli,redis-server,redis-check-dump,redis-check-aof} /usr/local/redis/bin/$ cp /usr/local/redis/bin/* /usr/local/bin/
配置
Master配置
先配置好Redis，主从复制成功之后再行配置Keepalived。
Master-Redis配置
$ vim /usr/local/redis/etc/redis.conf
修改如下值
daemonize yes pidfile /var/run/redis.pid port 6379tcp-backlog 511timeout 0tcp-keepalive 0loglevel notice logfile "/usr/local/redis/log/redis.log"databases 16save 900 1save 300 10save 60 10000stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb dir /usr/local/redis/dataslave-serve-stale-data yes slave-read-only no repl-disable-tcp-nodelay no slave-priority 100appendonly yes appendfilename "appendonly.aof"appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mb lua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128notify-keyspace-events ""hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64activerehashing yes client-output-buffer-limit normal 0 0 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsync yes
Master-Keepalived配置
global_defs { router_id redis } vrrp_script chk_redis { script "/etc/keepalived/scripts/redis_check.sh" ###监控脚本 interval 2 ###监控时间 timeout 2 ###超时时间 fall 3 ###重试次数} vrrp_instance redis { state MASTER ###设置为MASTER interface eth0 ###监控网卡 virtual_router_id 52 priority 101 ###权重值 authentication { auth_type PASS ###加密 auth_pass redis ###密码 } track_script { chk_redis ###执行上面定义的chk_redis } virtual_ipaddress { 192.168.0.239 ###VIP } notify_master /etc/keepalived/scripts/redis_master.sh notify_backup /etc/keepalived/scripts/redis_backup.sh notify_fault /etc/keepalived/scripts/redis_fault.sh notify_stop /etc/keepalived/scripts/redsi_stop.sh }也可在本地编辑好后使用
scp上传到服务器。 Master-keepalived脚本redis_master.sh:#!/bin/bash REDISCLI="/usr/local/redis/bin/redis-cli" LOGFILE="/usr/local/redis/var/keepalived-redis-state.log" echo "[master]" >> $LOGFILE date >> $LOGFILE echo "Being master...." >> $LOGFILE 2>&1 echo "Run SLAVEOF cmd ..." >> $LOGFILE $REDISCLI SLAVEOF 192.168.0.232 6379 >> $LOGFILE 2>&1 sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态 echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
Master-keepalived脚本redis_backup.sh:
#!/bin/bash REDISCLI="/usr/local/redis/bin/redis-cli" LOGFILE="/usr/local/redis/log/keepalived-redis-state.log" echo "[backup]" >> $LOGFILE date >> $LOGFILE echo "Being slave...." >> $LOGFILE 2>&1 sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色 echo "Run SLAVEOF cmd ..." >> $LOGFILE $REDISCLI SLAVEOF 192.168.0.232 6379 >> $LOGFILE 2>&1
Backup配置
同样先配置好Redis，主从复制成功之后再行配置Keepalived。
Backup-Redis配置
$ vim /usr/local/redis/etc/redis.conf
修改如下值
daemonize yes pidfile /var/run/redis.pid port 6379tcp-backlog 511timeout 0tcp-keepalive 0loglevel notice logfile "/usr/local/redis/log/redis.log"databases 16save 900 1save 300 10save 60 10000stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb dir /usr/local/redis/dataslave-serve-stale-data yes slave-read-only no repl-disable-tcp-nodelay no slave-priority 100appendonly yes appendfilename "appendonly.aof"appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mb lua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128notify-keyspace-events ""hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64activerehashing yes client-output-buffer-limit normal 0 0 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsync yes SLAVEOF 192.168.0.231 6379
Backup-Keepalived配置
global_defs { router_id redis } vrrp_script chk_redis { script "/etc/keepalived/scripts/redis_check.sh" ###监控脚本 interval 2 ###监控时间 timeout 2 ###超时时间 fall 3 ###重试次数} vrrp_instance redis { state BACKUP ###设置为MASTER interface eth0 ###监控网卡 virtual_router_id 51 priority 100 ###权重值 authentication { auth_type PASS ###加密 auth_pass redis ###密码 } track_script { chk_redis ###执行上面定义的chk_redis } virtual_ipaddress { 192.168.0.239 ###VIP } notify_master /etc/keepalived/scripts/redis_master.sh notify_backup /etc/keepalived/scripts/redis_backup.sh notify_fault /etc/keepalived/scripts/redis_fault.sh notify_stop /etc/keepalived/scripts/redsi_stop.sh }也可在本地编辑好后使用
scp上传到服务器。 Master-keepalived脚本redis_master.sh:#!/bin/bash REDISCLI="/usr/local/redis/bin/redis-cli" LOGFILE="/usr/local/redis/var/keepalived-redis-state.log" echo "[master]" >> $LOGFILE date >> $LOGFILE echo "Being master...." >> $LOGFILE 2>&1 echo "Run SLAVEOF cmd ..." >> $LOGFILE $REDISCLI SLAVEOF 192.168.0.231 6379 >> $LOGFILE 2>&1 sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态 echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
Master-keepalived脚本redis_backup.sh:
#!/bin/bash REDISCLI="/usr/local/redis/bin/redis-cli" LOGFILE="/usr/local/redis/log/keepalived-redis-state.log" echo "[backup]" >> $LOGFILE date >> $LOGFILE echo "Being slave...." >> $LOGFILE 2>&1 sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色 echo "Run SLAVEOF cmd ..." >> $LOGFILE $REDISCLI SLAVEOF 192.168.0.231 6379 >> $LOGFILE 2>&1
公共脚本配置
redis_chekc.sh
可以使用netcat来检测redis服务的状态.
安装nc，使用nc检测redis的服务而不是ping
$ wget http://sourceforge.net/projects/netcat/files/netcat/0.7.1/netcat-0.7.1.tar.gz$ tar -zxvf netcat-0.7.1.tar.gz$ ./configure$ make && make install建立脚本:
$ vim /etc/keepalived/scripts/redis_check.sh
脚本内容:
#!/bin/bashALIVE=`/usr/local/redis/bin/redis-cli PING`if ["$ALIVE" == "PONG"] ;then echo $ALIVE exit 0else echo $ALIVE exit 1fi
redis_fault.sh
建立脚本:
$ vim /etc/keepalived/scripts/redis_fault.sh
脚本内容:
# !/bin/bashLOGFILE=/usr/local/redis/var/keepalived-redis-state.log echo "[faule]" >> $LOGFILE date >> $LOGFILE
redis_stop.sh
建立脚本:
$ vim /etc/keepalived/scripts/redis_stop.sh
脚本内容:
# !/bin/bashLOGFILE=/usr/local/redis/var/keepalived-redis-state.log echo "[stop]" >> $LOGFILE date >> $LOGFILE
测试
主从复制测试
先打开Master上的Redis，再打开Backup上的Redis，分别查看info
$ redis-cli info
Master的rule显示master为正常，slave会显示为192.168.0.232
在Master上写入数据，在Backup读取
$ redis-cli set a hello //Master 写入数据$ redis-cli get a //Backup 读取数据"hello"
主从复制正常。
主从切换测试
先后启动Master和Backup上的keepalived，在Master上查看vip$ ip a
从图中可以看出vip已经绑定在Master上，现在通过vip来进行数据测试
$ redis-cli -h 192.168.0.239 set a test $ redis-cli -h 192.168.0.239 get a //测试虚拟ip数据读取//在主机和从机上分别进行数据读取$ redis-cli get a
从图中可以看出测试无问题。
现在关闭Master上的redis，来看VIP能否正常漂移，并进行数据测试。
$ killall -9 redis-server$ redis-cli -h 192.168.0.232 info
现在给redis写入数据，测试看Master恢复服务后能否正常恢复数据
$ redis-cli -h 192.168.0.239 set a nihao
恢复Master上的Redis，查看ip a
$ service redis-server start$ ip a$ redis-cli get a
数据恢复，VIP切换回Master，Redis高可用环境搭建成功.
下一篇进行redis调优的测试和读写分离测试。