nginx+keepalived构建双主负载均衡
引言
Nginx是一个高性能的代理服务器，单台Nginx容易出现单点故障，使用keepalived可以实现Nginx的故障转移，保证了网站的高可用性
一、 使用Nginx+keepalived的两种方案
1、主从模式
使用一个VIP，前端有2台服务器，一主一从，正常情况下是主服务器提供服务只有当主服务器不能正常提供服务之后，从服务器才提供服务，此时总会有一台服务器是空闲状态。
2、双主模式
使用两个VIP，前段有2台服务器，互为主从，两台服务器同时工作，不存在资源浪费情况。同时在前段的DNS服务器对网站做多条A记录，实现了Nginx的负载均衡，当一台服务器故障时候，资源会转移到另一台服务器，继续提供服务，在大型的网站中多数都使用此种架构。在此使用主主模式配置Nginx+keepalived的高可用性。
二、准备实验环境
1、服务器IP地址规划
VIP：172.16.10.8
VIP：172.16.10.9
Keepalived1:172.16.10.1
Keepalived2:172.16.10.2
2、服务器操作系统
Keepalived1 ：Centos 6.4 x86_64
Keepalived2 ：Centos 6.4 x86_64
3、网络拓扑图
4、修改主机名以及hosts文件keepalived1
####keepalived1 server############
sed -i 's@\(HOSTNAME=\).*@\1keepalived1@g'/etc/sysconfig/network
hostname keepalived1
[root@keepalived1 ~]# echo "172.16.10.1 keepalived1">> /etc/hosts
[root@keepalived1 ~]# echo "172.16.10.2 keepalived2">> /etc/hosts
[root@keepalived1 ~]# ssh-keygen -t rs
[root@keepalived1 ~]# ssh-copy-id -i .ssh/id_rsa.pub keepalived2
[root@keepalived1 ~]# scp /etc/hosts keepalived1:/etc/
####keepalived2 server############
sed -i 's@\(HOSTNAME=\).*@\1keepalived2@g'/etc/sysconfig/network
hostname keepalived2
[root@keepalived2 ~]# ssh-keygen -t rsa
[root@keepalived2 ~]# ssh-copy-id -i .ssh/id_rsa.pub keepalived1
三、编译安装Nginx
[root@keepalived1 ~]# yum install openssl-devel pcre-devel gcc -y
[root@keepalived1 ~]# tar xf nginx-1.4.2.tar.gz -C /usr/local/
[root@keepalived1 ~]# cd /usr/local/
[root@keepalived1 local]# groupadd -r nginx
[root@keepalived1 local]# useradd -r -g nginx nginx
[root@keepalived1 local]# cd nginx-1.4.2/
[root@keepalived1 nginx-1.4.2]# ./conf
conf/ configure
[root@keepalived1 nginx-1.4.2]# ./configure \
> --prefix=/usr \
> --sbin-path=/usr/sbin/nginx \
> --conf-path=/etc/nginx/nginx.conf \
> --error-log-path=/var/log/nginx/error.log \
> --http-log-path=/var/log/nginx/access.log \
> --pid-path=/var/run/nginx/nginx.pid \
> --lock-path=/var/lock/nginx.lock \
> --user=nginx \
> --group=nginx \
> --with-http_ssl_module \
> --with-http_flv_module \
> --with-http_stub_status_module \
> --with-http_gzip_static_module \
> --http-client-body-temp-path=/var/tmp/nginx/client/ \
> --http-proxy-temp-path=/var/tmp/nginx/proxy/ \
> --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ \
> --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \
> --http-scgi-temp-path=/var/tmp/nginx/scgi \
> --with-pcre
[root@keepalived1 nginx-1.4.2]# make && make install
[root@keepalived1 nginx-1.4.2]# vim /etc/init.d/nginx
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemon
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /etc/nginx/nginx.conf
# config: /etc/sysconfig/nginx
# pidfile: /var/run/nginx.pid
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
nginx="/usr/sbin/nginx"
prog=$(basename $nginx)
NGINX_CONF_FILE="/etc/nginx/nginx.conf"
[ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx
lockfile=/var/lock/subsys/nginx
make_dirs() {
# make required directories
user=`nginx -V 2>&1 | grep "configure arguments:" | sed 's/[^*]*--user=\([^ ]*\).*/\1/g' -`
options=`$nginx -V 2>&1 | grep 'configure arguments:'`
for opt in $options; do
if [ `echo $opt | grep '.*-temp-path'` ]; then
value=`echo $opt | cut -d "=" -f 2`
if [ ! -d "$value" ]; then
# echo "creating" $value
mkdir -p $value && chown -R $user $value
fi
fi
done
}
start() {
[ -x $nginx ] || exit 5
[ -f $NGINX_CONF_FILE ] || exit 6
make_dirs
echo -n $"Starting $prog: "
daemon $nginx -c $NGINX_CONF_FILE
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
killproc $prog -QUIT
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
configtest || return $?
stop
sleep 1
start
}
reload() {
configtest || return $?
echo -n $"Reloading $prog: "
killproc $nginx -HUP
RETVAL=$?
echo
}
force_reload() {
restart
}
configtest() {
$nginx -t -c $NGINX_CONF_FILE
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart|configtest)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac
[root@keepalived1 nginx-1.4.2]# chmod +x /etc/init.d/nginx
[root@keepalived1 nginx-1.4.2]# service nginx start
注意在此只上传了keepalived1的代码，keepalived2也需要同样的操作
1、修改默认网页以方便后期测试
###############keepalived1######################
[root@keepalived1 ~]# echo "<h1>keepalived1</h1>" > /usr/html/index.html
###############keepalived2######################
[root@keepalived2 ~]# echo "<h1>keepalived2</h1>" > /usr/html/index.html
四、 安装与配置keepalived
1、安装keepalived
###############keepalived1######################
[root@keepalived1 ~]# yum install keepalived -y
###############keepalived2######################
[root@keepalived2 ~]# yum install keepalived -y
2、修改配置文件
[root@keepalived1 keepalived]# grep -v "#" /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from Alexandre.Cassen@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nginx { #监控nginx脚本
script "killall -0 nginx" #监控nginx进程
interval 1 #监控间隔
weight -2 #优先级-2
}
vrrp_instance VI_1 {
state MASTER #主server
interface eth0
virtual_router_id 80
priority 100 #优先级
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.10.8 #定义vip
}
track_script {
chk_nginx #跟踪脚本
}
notify_master "/etc/keepalived/notify8.sh master" #定义邮件通知
notify_backup "/etc/keepalived/notify8.sh backup"
notify_fault "/etc/keepalived/notify8.sh fault"
}
vrrp_instance VI_2 {
state BACKUP #从server
interface eth0
virtual_router_id 81
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.10.9
}
track_script {
chk_nginx
}
notify_master "/etc/keepalived/notify9.sh master" #定义邮件通知
notify_backup "/etc/keepalived/notify9.sh backup"
notify_fault "/etc/keepalived/notify9.sh fault"
}
[root@keepalived1 keepalived]#
3、编辑邮件通知脚本（notify8.sh notify9.sh）
#####################notify8.sh##############
[root@keepalived1 keepalived]# cat notify8.sh
#!/bin/bash
# Author: xiaodong <[email protected]>
# description: An example of notify script
#
vip=172.16.10.8
contact='root@localhost'
notify() {
mailsubject="`hostname` to be $1: $vip floating"
mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
case "$1" in
master)
notify master
/etc/rc.d/init.d/nginx start
exit 0
;;
backup)
notify backup
/etc/rc.d/init.d/nginx stop
exit 0
;;
fault)
notify fault
exit 0
;;
*)
echo 'Usage: `basename $0` {master|backup|fault}'
exit 1
;;
esac
####################notfiy9.sh#################
[root@keepalived1 keepalived]# cat notify9.sh
#!/bin/bash
# Author: xiaodong <[email protected]>
# description: An example of notify script
#
vip=172.16.10.9
contact='root@localhost'
notify() {
mailsubject="`hostname` to be $1: $vip floating"
mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
case "$1" in
master)
notify master
exit 0
;;
backup)
notify backup
exit 0
;;
fault)
notify fault
exit 0
;;
*)
echo 'Usage: `basename $0` {master|backup|fault}'
exit 1
;;
esac
[root@keepalived1 keepalived]# chmod +x notify8.sh
[root@keepalived1 keepalived]# chmod +x notify9.sh
4、复制配置文件到keepalived2，并做修改.
[root@keepalived1 keepalived]# scp -p keepalived.conf notify8.sh notify9.sh keepalived2:/etc/keepalived/
[root@keepalived2 keepalived]# grep -v "#" /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
notification_email_from Alexandre.Cassen@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nginx {
script "killall -0 nginx "
interval 1
weight -2
}
vrrp_instance VI_1 {
state BACKUP #改为backup
interface eth0
virtual_router_id 80
priority 99 #改为99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.10.8
}
track_script {
chk_nginx
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state MASTER #改为MASTER
interface eth0
virtual_router_id 81
priority 100 #改为100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.10.9
}
track_script {
chk_nginx
}
notify_master "/etc/keepalived/notify9.sh master"
notify_backup "/etc/keepalived/notify9.sh backup"
notify_fault "/etc/keepalived/notify9.sh fault"
}
注释 ：此处使用本地的邮件服务器接受邮件，如果需要用其它邮件服务器请修改contact='root@localhost'
5、启动keepalived服务
###############keepalived1######################
[root@keepalived1 ~]# service keepalived start
###############keepalived2######################
[root@keepalived2 ~]# service keepalived start
6、查看两个节点的vip是否启动正常
五、测试nginx+keepalived的高可用性
1、使用游览器访问测试
2、模拟节点出现故障，nginx服务器是否能自动转移
[root@keepalived1 keepalived]# service keepalived stop
通过以上测试，节点出现故障的时候，服务可以自动转移到备用节点上
3、测试主节点服务down掉之后，备用节点服务是否能正常运行
[root@keepalived1 keepalived]# service keepalived start
[root@keepalived1 keepalived]# killall nginx
通过以上测试，实现了Nginx的高可用性，但是，运维人员是否能第一时间得知服务器出现故障，这时候就需要查看邮件了
4、查看邮件是否收到信息
[root@keepalived1 keepalived]# mail #查看邮件命令
Heirloom Mail version 12.4 7/29/08. Type ? for help.
"/var/spool/mail/root": 1 message 1 new
>N 1 root Wed Sep 25 20:15 18/728 "keepalived1 to be backup: 172.16.10.8 floating"
& 1
Message 1:
From [email protected] Wed Sep 25 20:15:46 2013
Return-Path: <[email protected]>
X-Original-To: root@localhost
Delivered-To: [email protected]
Date: Wed, 25 Sep 2013 20:15:46 +0800
To: [email protected]
Subject: keepalived1 to be backup: 172.16.10.8 floating
User-Agent: Heirloom mailx 12.4 7/29/08
Content-Type: text/plain; charset=us-ascii
From: [email protected] (root)
Status: R
2013-09-25 20:15:46: vrrp transition, keepalived1 changed to be backup
& quit #退出邮件
5、当nginx服务启动之后，主节点恢复
[root@keepalived1 keepalived]# service nginx start
[root@keepalived1 keepalived]# mail
Heirloom Mail version 12.4 7/29/08. Type ? for help.
"/var/spool/mail/root": 2 messages 1 unread
1 root Wed Sep 25 20:15 19/739 "keepalived1 to be backup: 172.16.10.8 floating"
>U 2 root Wed Sep 25 20:16 19/738 "keepalived1 to be master: 172.16.10.8 floating"
&
Message 2:
From [email protected] Wed Sep 25 20:16:22 2013
Return-Path: <[email protected]>
X-Original-To: root@localhost
Delivered-To: [email protected]
Date: Wed, 25 Sep 2013 20:16:22 +0800
To: [email protected]
Subject: keepalived1 to be master: 172.16.10.8 floating
User-Agent: Heirloom mailx 12.4 7/29/08
Content-Type: text/plain; charset=us-ascii
From: [email protected] (root)
Status: RO
2013-09-25 20:16:22: vrrp transition, keepalived1 changed to be master
Nginx+keepalived的高可用负载均衡配置完成。
相关文章:http://blog.csdn.net/luxiaoyu_sdc/article/details/7333024