apache+inotify-tools+keepalived+lvs-DR模式配置高可用负载均衡集群

环境:虚拟机VMware workstation 9

      操作系统:Redhat 5.6 i386

一、keepalived+LVS-DR模式配置高可用负载均衡

拓扑如下:wKioL1OpF3qQUXSnAAFmxuu-O0I274.jpg

二、服务器IP配置信息

四台服务器均有VMware虚拟机实现,两台HA主机、两台web服务器。网络方式都设置为NAT模式

1、master:192.168.80.145

2、slaver:192.168.80.137

3、web1:192.168.80.144

4、web2:192.168.80.134

5、VIP:192.168.80.200

三、安装配置keepalived

1、主辅服务器部署keepalived

yum install gcc gcc-c++ autoconf automake wget openssl-devel kernel-devel  -y
tar -zxf keepalived-1.1.20.tar.gz -C /usr/src
cd /usr/src/keepalived-1.1.20
./configure --prefix=/usr/local/keepalived --sysconf=/etc --with-kernel-dir=/usr/src/kernels/2.6.18-238.el5
make && make install
cp /usr/local/keepalived/sbin/keepalived /usr/sbin
cp /usr/local/keepalived/bin/genhash /usr/sbin/
/etc/init.d/keepalived start
chkconfig keepalived on

注:主要难点解决依赖关系


2、分别设置web服务器1和2

yum install httpd -y
echo "web1/web2 ok!!!">/var/www/html/index.html
echo "check web1 ok!!! /check web2 ok !!!">/var/www/html/check_web1.html
/etc/init.d/httpd start

3、主辅服务器部署ipvsadm

tar -zxf ipvsadm-1.24.tar.gz -C /usr/src
cd /usr/src/ipvsadm-1.24
make && make install

注:安装前检查内核是否支持ipvsadm  

 cat /boot/config-2.6.18-238.el5 |grep -i ipvs
# IPVS transport protocol load balancing support
# IPVS scheduler
# IPVS application helper

modprobe -l |grep ipvs
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_dh.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_ftp.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_lblc.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_lblcr.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_lc.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_nq.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_rr.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_sed.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_sh.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_wlc.ko
/lib/modules/2.6.18-238.el5/kernel/net/ipv4/ipvs/ip_vs_wrr.ko

4、配置主辅keepalived配置文件

/etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id httpd
}

vrrp_script check_http {
        script "/root/bash/check_httpd.sh"
        weight -5
        interval 1
    }

vrrp_instance VI_1 {
    state MASTER/BACKUP
    interface eth0
    virtual_router_id 240
    priority 100/98
    advert_int 1
#    track_script {
#        check_http
#    }
    authentication {
        auth_type PASS
        auth_pass 3333
    }
    virtual_ipaddress {
       192.168.80.200/24 dev eth0
    }
}
virtual_server 192.168.80.200 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    nat_mask 255.255.255.0
    persistence_timeout 0
    protocol TCP

    real_server 192.168.80.134 80 {
        weight 1
        HTTP_GET {
            url {
              path /check_web1.html
              digest 68f36ac34591233a3ca3b5def1bace34
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.80.144 80 {
        weight 1
        HTTP_GET {
            url {
              path /check_web1.html
              digest 68f36ac34591233a3ca3b5def1bace34
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

注:(1)、主辅设置优先级

    (2)、genhash值的计算、web1、web2的httpd服务开启时才能

    (3)、使用的是HTTP_GET检查web服务器的健康状态

    (4)、lvs模式设定

    (5)、persistence_timeout 设置为0 方便后面测试

5、检查部署情况

master检查ipvsadm 情况

ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.80.200:80 rr
  -> 192.168.80.144:80            Route   1      0          0         
  -> 192.168.80.134:80            Route   1      0          0

6、主服务器重启keepalived服务

tail -f /var/log/messages

May 17 00:37:48 master Keepalived: Terminating on signal
May 17 00:37:48 master Keepalived: Stopping Keepalived v1.1.20 (05/16,2014) 
May 17 00:37:48 master Keepalived_vrrp: Terminating VRRP child process on signal
May 17 00:37:49 master Keepalived_healthcheckers: Terminating Healthchecker child process on signal
May 17 00:37:49 master Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
May 17 00:37:49 master avahi-daemon[3543]: Withdrawing address record for 192.168.80.200 on eth0.
May 17 00:37:49 master Keepalived: Starting Keepalived v1.1.20 (05/16,2014) 
May 17 00:37:49 master Keepalived_healthcheckers: Netlink reflector reports IP 192.168.80.145 added
May 17 00:37:49 master Keepalived_healthcheckers: Registering Kernel netlink reflector
May 17 00:37:49 master Keepalived_healthcheckers: Registering Kernel netlink command channel
May 17 00:37:49 master Keepalived: Starting Healthcheck child process, pid=12821
May 17 00:37:49 master Keepalived: Starting VRRP child process, pid=12822
May 17 00:37:49 master Keepalived_vrrp: Netlink reflector reports IP 192.168.80.145 added
May 17 00:37:49 master Keepalived_vrrp: Registering Kernel netlink reflector
May 17 00:37:49 master Keepalived_vrrp: Registering Kernel netlink command channel
May 17 00:37:49 master Keepalived_vrrp: Registering gratutious ARP shared channel
May 17 00:37:49 master Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'. 
May 17 00:37:49 master Keepalived_vrrp: Configuration is using : 37747 Bytes
May 17 00:37:49 master Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
May 17 00:37:49 master Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
May 17 00:37:49 master Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'. 
May 17 00:37:49 master Keepalived_healthcheckers: Configuration is using : 14817 Bytes
May 17 00:37:49 master Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
May 17 00:37:49 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.134:80]
May 17 00:37:49 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.144:80]
May 17 00:37:50 master Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
May 17 00:37:51 master Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
May 17 00:37:51 master Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
May 17 00:37:51 master Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.80.200
May 17 00:37:51 master Keepalived_vrrp: Netlink reflector reports IP 192.168.80.200 added
May 17 00:37:51 master Keepalived_healthcheckers: Netlink reflector reports IP 192.168.80.200 added
May 17 00:37:51 master avahi-daemon[3543]: Registering new address record for 192.168.80.200 on eth0.
May 17 00:37:56 master Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.80.200
May 17 00:38:05 master Keepalived: Terminating on signal
May 17 00:38:05 master Keepalived: Stopping Keepalived v1.1.20 (05/16,2014) 
May 17 00:38:05 master Keepalived_vrrp: Terminating VRRP child process on signal
May 17 00:38:05 master Keepalived_healthcheckers: Terminating Healthchecker child process on signal
May 17 00:38:05 master Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
May 17 00:38:05 master avahi-daemon[3543]: Withdrawing address record for 192.168.80.200 on eth0.
May 17 00:38:52 master Keepalived: Starting Keepalived v1.1.20 (05/16,2014) 
May 17 00:38:52 master Keepalived_healthcheckers: Netlink reflector reports IP 192.168.80.145 added
May 17 00:38:52 master Keepalived_healthcheckers: Registering Kernel netlink reflector
May 17 00:38:52 master Keepalived_healthcheckers: Registering Kernel netlink command channel
May 17 00:38:52 master Keepalived: Starting Healthcheck child process, pid=12838
May 17 00:38:52 master Keepalived: Starting VRRP child process, pid=12839
May 17 00:38:52 master Keepalived_vrrp: Netlink reflector reports IP 192.168.80.145 added
May 17 00:38:52 master Keepalived_vrrp: Registering Kernel netlink reflector
May 17 00:38:52 master Keepalived_vrrp: Registering Kernel netlink command channel
May 17 00:38:52 master Keepalived_vrrp: Registering gratutious ARP shared channel
May 17 00:38:52 master Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'. 
May 17 00:38:52 master Keepalived_healthcheckers: Configuration is using : 14817 Bytes
May 17 00:38:52 master Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
May 17 00:38:52 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.134:80]
May 17 00:38:52 master Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'. 
May 17 00:38:52 master Keepalived_healthcheckers: Activating healtchecker for service [192.168.80.144:80]
May 17 00:38:52 master Keepalived_vrrp: Configuration is using : 37747 Bytes
May 17 00:38:52 master Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
May 17 00:38:52 master Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
May 17 00:38:53 master Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE

7、在web服务器上分别执行以下脚本

#!/bin/bash
#description:start realserver
#chkconfig
VIP1=192.168.80.200
case "$1" in
start)
echo " start LVS of REALServer"
/sbin/ifconfig lo:0 $VIP1 broadcast $VIP1 netmask 255.255.255.255 up
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
/sbin/ifconfig lo:0 down
echo "close LVS Directorserver"
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac

8、现在重启主辅服务器上的keepalived服务、web服务器上的httpd服务,关闭master的keepalived服务做测试。

keepalived+LVS-DR模式就部署就完成了 

9、在web1上部署inotify-tools工具、web2上部署rsync服务,用于当web1网页内容变化时,web2服务器的网页内容的一致性问题

web1上部署inotify-tools

tar -zxf inotify-tools-3.13.tar.gz -C /usr/src
cd /usr/src/inotify-tools-3.13
./configure --prefix=/usr/local/inotify_tools
make && make install 
cp /usr/local/inotify_tools/bin/inotifywait /usr/sbin/
cp /usr/local/inotify_tools/bin/inotifywatch /usr/sbin/


chmod 755 inotify-rsync.sh
inotify-rsync.sh
#!/bin/bash
host1=192.168.80.137
src=/var/www/html
dst1=test
user1=root
/usr/local/inotify_tools/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f%e' -e modify,delete,create,attrib  $src \
| while read files
        do
        /usr/bin/rsync -vzrtopg --delete --progress --password-file=/etc/rsync.passwd $src$user1@$host1::$dst1
                echo "${files} was rsynced" >>/tmp/rsync.log 2>&1
         done
         
         
inotify-rsync.sh  &


web2上部署rsync服务

chmod 600 /etc/rsync.passwd

/etc/rsync.passwd
root:strong

/etc/rsyncd.conf
strict modes = yes
port = 873
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
motd file = /etc/rsyncd.motd
use chroot =
timeout =

[test]
path = /var/www/
comment = rsync files
max connections = 5
uid = root
gid = root
ignore errors
read only = no
write only = no
list = no
hosts allow = *
hosts deny = 10.1.1.1
auth users = root
secrets file = /etc/rsync.passwd

启动rsync服务
chkconfig rsync on
/etc/init.d/xinted restart

10、至此apache+rsync+inotify-tools+keepalived+lvs-DR模式HA负载均衡部署就此结束


本文出自 “小杩的幸福生活” 博客,谢绝转载!

你可能感兴趣的:(虚拟机,操作系统,master,web服务器)