LVS+keepalived 实现负载均衡

一、资源规划

    在开始搭建之前，我们首先需要准备和规划好搭建所需的资源。因为性能等方面的考虑，我们使用当前比较流行的DR模式。

    搭建前我们需要了解，搭建环境的机器必须在同一网段内，此次环境搭建需要1个VIP和四台机器（我这里使用四台VM）

 

    具体清单如下：

服务器	IP地址	操作系统	网关	虚拟设备名	VIP	部署
Master Director	30.16.252.190	Red Hat Enterprise  6.7 (Santiago)	30.16.252.128	eth0:0	30.16.252.240	Lvs+keepalived
Backup Director	30.16.252.191	Red Hat Enterprise  6.7 (Santiago)	30.16.252.128	eth0:0	30.16.252.240	Lvs+keepalived
Real Server	30.16.252.188	Red Hat Enterprise  6.7 (Santiago)	30.16.252.128	lo:0	30.16.252.240	tomcat
Real Server	30.16.252.189	Red Hat Enterprise  6.7 (Santiago)	30.16.252.128	lo:0	30.16.252.240	tomcat

二、 配置real server

         在/usr/local/sbin下新建realserver.sh，并修改权限为可执行。

         在配置的过程中，需要注意VIP的掩码需要配置成32位

         下面的配置用来在real server上启用一个VIP，并让这个VIP忽略网络中的ARP请求。

        

         #realserver.sh

        =========================================================================

           #!/bin/sh

 

RS_VIP=30.16.252.240

 

sh /etc/rc.d/init.d/functions

 

case "$1" in

start)

       ifconfig lo:0 $RS_VIP netmask 255.255.255.255 broadcast $RS_VIP

       /sbin/route add -host $RS_VIP dev lo:0

       echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore

       echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce

       echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore

       echo "2">/proc/sys/net/ipv4/conf/all/arp_announce

       sysctl -p >/dev/null 2>&1

       echo "RealServer Start OK"

       ;;

stop)

       ifconfig lo:0 down

       route del $RS_VIP >/dev/null2>&1

       echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore

       echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce

       echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore

       echo "0">/proc/sys/net/ipv4/conf/all/arp_announce

       echo "RealServer Stoped"

       ;;

*)

       echo "Usage: $0 {start|stop}"

       exit 1

esac

exit 0

        ===========================================================================

 

         配置完后，执行：sh realserver.sh start

         需要配置开机自启动：echo "/usr/local/sbin/realserver.sh start" >> /etc/rc.local

         两台real server的配置是相同的，都要启动。

 

        搭建应用：

         在两台real server上部署tomcat，并新建应用test，在test下新建index.html

         #index.html

        [30.16.252.188:8080]   #根据两台机器真实值填写。

        

        

 

三、安装keepalived

 

    1. 我们选择keepalived的版本为1.2.8. 具体安装步骤如下：

 

         cd /usr/local

         tar -zxvf keepalived-1.2.8.tar.gz

         cd keepalived-1.2.8

         #在执行下面命令前请先做检查

         #如果没有这个目录：/usr/src/kernels/2.6.32-642.6.2.el6.x86_64， 可执行命令：yum install kernel-headers kernel-devel

         #如果没有/usr/src/linux这个目录，可以执行命令：ln -s /usr/src/kernels/2.6.32-642.6.2.el6.x86_64 /usr/src/linux

         #2.6.32-642.6.2.el6.x86_64为系统内核版本号，大家机器上的目录可能都不同，执行命令时，以机器上真实的值为准。

         ./configure  --sysconf=/etc  --with-kernel-dir=/usr/src/linux   

         make && make install

         ln -s /usr/local/sbin/keepalived /sbin/keepalived

         #中间任何一步有报错都是缺少依赖包造成，我们需要安装：

         yum install libnl* libpopt*

         yum install popt-devel popt-static

         yum install libnfnetlink libnfnetlink-devel

         然后编译和安装就会通过。

 

         2. 修改配置文件

         keepalived的配置文件位置：/etc/keepalived/keepalived.conf

         30.16.252.190 Master的配置如下：

        ============================================================================

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    virtual_router_id 163

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        30.16.252.240

    }

}

 

virtual_server 30.16.252.240 8080 {

    delay_loop 2

    lb_algo rr

    lb_kind DR

    #persistence_timeout50

    protocol TCP

 

    real_server 30.16.252.188 8080 {

      weight 1

      TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

      }

    }

 

    real_server 30.16.252.189 8080 {

      weight 1

      TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

      }

    }

}

        =============================================================================

           30.16.252.191Backup的配置如下：

        =============================================================================

        

vrrp_instance VI_1 {

    state BACKUP

    interface eth0

    virtual_router_id 163

    priority 99

    advert_int 1

    authentication {

    auth_type PASS

    auth_pass 1111

    }

         virtual_ipaddress {

               30.16.252.240

    }

}

 

virtual_server 30.16.252.240 8080 {

    delay_loop 2

    lb_algo rr

    lb_kind DR

    #persistence_timeout50

    protocol TCP

 

    real_server 30.16.252.188 8080 {

    weight 1

    TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

    }

    }

 

         real_server 30.16.252.189 8080 {

    weight 1

    TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

    }

    }

}

        =============================================================================

 

         3. 启动keepalived

             service keepalived start

             启动日志可查看：/var/log/message

            ===========================================================================================================

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Netlink reflector reports IP 30.16.252.190added

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Netlink reflector reports IPfe80::455:4ff:fe00:43ae added

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Registering Kernel netlink reflector

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Registering Kernel netlink command channel

Mar  2 10:31:44 SZC-L0051222Keepalived_vrrp[305580]: Opening file '/etc/keepalived/keepalived.conf'.

Mar  2 10:31:44 SZC-L0051222Keepalived_vrrp[305580]: Configuration is using : 60678 Bytes

Mar  2 10:31:44 SZC-L0051222Keepalived_vrrp[305580]: Using LinkWatch kernel netlink reflector...

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Opening file'/etc/keepalived/keepalived.conf'.

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Configuration is using : 18337 Bytes

Mar  2 10:31:44 SZC-L0051222Keepalived_vrrp[305580]: VRRP sockpool: [ifindex(2), proto(112), unicast(0),fd(10,11)]

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Using LinkWatch kernel netlink reflector...

Mar 2 10:31:44 SZC-L0051222 Keepalived_healthcheckers[305578]: Activatinghealthchecker for service [30.16.252.188]:8080    #检测要负载的real server的健康状况

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Activating healthchecker for service[30.16.252.189]:8080

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Activating healthchecker for service[30.16.252.188]:8088

Mar  2 10:31:44 SZC-L0051222Keepalived_healthcheckers[305578]: Activating healthchecker for service[30.16.252.189]:8088

Mar 2 10:31:45 SZC-L0051222 Keepalived_vrrp[305580]: VRRP_Instance(VI_1)Transition to MASTER STATE      #keepalived direcotor server 之间通信

Mar  2 10:31:46SZC-L0051222 Keepalived_vrrp[305580]: VRRP_Instance(VI_1) Entering MASTER STATE

Mar  2 10:31:46 SZC-L0051222Keepalived_vrrp[305580]: VRRP_Instance(VI_1) setting protocol VIPs.

Mar  2 10:31:46 SZC-L0051222Keepalived_vrrp[305580]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0for 30.16.252.240

Mar  2 10:31:46 SZC-L0051222Keepalived_healthcheckers[305578]: Netlink reflector reports IP 30.16.252.240added

Mar  2 10:31:51 SZC-L0051222Keepalived_vrrp[305580]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0for 30.16.252.240

 ==============================================================================================================================

 

 四、负载验证

          因在Director server上不能访问对外提供的VIP，所以又在同网段找了一台机器进行验证。

          因为防火墙等原因，不能直接使用browser来进行验证，所以就写了程序，因为使用了rr(轮询算法)，所以1000次访问，两台server各访问500次。

    注意：persistence_timeout配置可以进行会话保持，即同一台机器在设置的时间内，请求会定向到同一台real server.

          验证正常。