负载均衡 Nginx+keepalive

1 规划和准备

两台相同配置的web
用途 IP
MASTER 192.168.1.100
BACKUP 192.1681.101

2 安装

两台接入服务器分别安装NginX和keepalived:
准备依赖包：

yum -y install gcc pcre-devel zlib-devel openssl-devel
yum -y install popt-devel

下载

wget http://nginx.org/download/nginx-1.2.4.tar.gz 
wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz

安装NginX
安装keepalive

tar zxvf keepalived-1.2.7.tar.gz
cd keepalived-1.2.7
./configure
make 
make install

cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/sbin/keepalived /usr/sbin/

加入启动服务

echo "/usr/local/nginx/sbin/nginx" >> /etc/rc.local
echo "/etc/init.d/keepalived start" >> /etc/rc.local
 

3 配置

3.1 配置NginX
两台接入服务器的NginX的配置完全一样,主要是配置/usr/local/nginx/conf/nginx.conf的http。其中多域名指向是通过虚拟主机（配置http下面的server）实现；同一域名的不同虚拟目录通过每个server下面的不同location实现；到后端的服务器在http下面配置upstream,然后在server或location中通过proxypass引用。要实现前面规划的接入方式，http的配置如下：

http {
     
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;

    upstream dev.hysec.com {
     
        server 50.1.1.21:80;
    }


    upstream opslinux.com {
     
      ip_hash;
      server 192.168.1.102:80
      server 192.168.1.103:80
      server 192.168.1.104:80
    }
    server {
     
        listen       80;
        server_name  opslinux.com;
        location / {
     
            proxy_pass http://opslinux.com;
        }
}
 

验证方法：
首先用IP访问前表中各个应用服务器的url
再用域名和路径访问前表中各个应用系统的域名/虚拟路径
3.2 配置keepalived
按照上面的安装方法，keepalived的配置文件在/etc/keepalived/keepalived.conf。主、从服务器的配置相关联但有所不同。如下：

Master:

! Configuration File for keepalived

global_defs {
     
  router_id NGINX_DEVEL
}

vrrp_instance VI_1 {
     
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 101
    advert_int 1
    authentication {
     
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
     
       192.168.1.100
    }
}

Backup:

! Configuration File for keepalived

global_defs {
     
  router_id NGINX_DEVEL
}

vrrp_instance VI_1 {
     
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
     
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
     
       192.168.1.100
    }
}

验证：
先后在主、从服务器上启动keepalived: /etc/init.d/keepalived start
在主服务器上查看是否已经绑定了虚拟IP： ip addr
停止主服务器上的keepalived: /etc/init.d/keepalived stop 然后在从服务器上查看是否已经绑定了虚拟IP：
启动主服务器上的keepalived，看看主服务器能否重新接管虚拟IP
3.3 让keepalived监控NginX的状态
经过前面的配置，如果主服务器的keepalived停止服务，从服务器会自动接管VIP对外服务；一旦主服务器的keepalived恢复，会重新接管VIP。 但这并不是我们需要的，我们需要的是当NginX停止服务的时候能够自动切换。
keepalived支持配置监控脚本，我们可以通过脚本监控NginX的状态，如果状态不正常则进行一系列的操作，最终仍不能恢复NginX则杀掉keepalived，使得从服务器能够接管服务。
如何监控NginX的状态
最简单的做法是监控NginX进程，更靠谱的做法是检查NginX端口，最靠谱的做法是检查多个url能否获取到页面。
如何尝试恢复服务
如果发现NginX不正常，重启之。等待3秒再次校验，仍然失败则不再尝试。
根据上述策略很容易写出监控脚本。这里使用nmap检查nginx端口来判断nginx的状态，记得要首先安装nmap。监控脚本如下:

#!/bin/bash
# check nginx server status
NGINX=/usr/local/nginx/sbin/nginx
PORT=8080

nmap localhost -p $PORT | grep "$PORT/tcp open"
#echo $?
if [ $? -ne 0 ];then
    $NGINX -s stop
    $NGINX
    sleep 3
    nmap localhost -p $PORT | grep "$PORT/tcp open"
    [ $? -ne 0 ] && /etc/init.d/keepalived stop
fi
 

不要忘了设置脚本的执行权限，否则不起作用。
假设上述脚本放在/opt/chk_nginx.sh，则keepalived.conf中增加如下配置：
主keepalived

vrrp_script chk_http_port {
     
    script "/opt/chk_nginx.sh"
    interval 1
    weight -2
}

track_script {
     
    chk_http_port
} 

例子：
! Configuration File for keepalived

global_defs {
     
   router_id NGINX_UPSTEAM
}

vrrp_script chk_http_port {
     
    script "/opt/chk_nginx.sh"
    interval 1
    weight -2
}

vrrp_instance VI_1 {
     
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
     
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
     
        192.168.1.100
    }

    track_script {
     
    chk_http_port
    }

}
 

更进一步，为了避免启动keepalived之前没有启动nginx , 可以在/etc/init.d/keepalived的start中首先启动nginx:

start() {
     
    /usr/local/nginx/sbin/nginx
    sleep 3
    echo -n $"Starting $prog: "
    daemon keepalived ${KEEPALIVED_OPTIONS}
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
}