Keepalived双主ipvs集群实现后端nginx负载均衡

Keepalived双主ipvs集群实现后端nginx负载均衡

实验准备

初始设置
（1）时间同步
systemctl restart chronyd
（2）关闭SELinux和防火墙
（3）互相之间/etc/hosts文件添加对方主机名
（4）确认接口支持多播（组播），略，基本新的网卡都支持。

实验规划

主机：A B C D
A B Keepalived+LVS集群
C D 后端RS

实验步骤

RS主机配置

C D
1每个RS安装nginx服务并配置测试主页
yum -y install nginx
vim /usr/share/nginx/html/index/html

RS1

2 每个RS配置两个回环地址
vim setrs.sh

#!/bin/bash
vip1=172.16.42.199
vip2=172.16.42.200
mask=255.255.255.255
iface1="lo:0"
iface2="lo:1"

case $1 in
start)  
        echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
        echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

        ifconfig $iface1 $vip1 netmask $mask broadcast $vip1 up
        ifconfig $iface2 $vip2 netmask $mask broadcast $vip2 up
        route add -host $vip1 dev $iface1 
        route add -host $vip2 dev $iface2 
        ;;
stop)   
        ifconfig $iface1 down
        ifconfig $iface2 down

        echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
        echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
        ;;
*)      
        echo "Usage: $(basename $0) start|stop"
        exit 1  
        ;;
esac

bash -x setrs.sh start
ip a可看到每个RS主机上都配备了两个回环地址

Keepalived+lvs集群配置

A B
yum -y install ipvsadm
yum -y install keepalived
vim /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka1@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 60
   vrrp_mcast_group4 224.111.111.222     ##组播地址
}
####虚拟路由相关配置
vrrp_instance VI_1 {       ##虚拟路由1
    state BACKUP            ##此处B为MASTER
    interface eth1
    virtual_router_id 191  ##上下两个虚拟路由辨识号不能一样，但需主备一样
    priority 95                    
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 0702f7ab  ##上下两个虚拟路由验证不能一样，但需主备一样
    }
    virtual_ipaddress {
        172.16.42.199          ##上下两个虚拟路由虚拟路由ip不能一样，但需主备一样
    }
}
vrrp_instance VI_2 {        ##虚拟路由2
    state MASTER             ##此处B为BACKUP
    interface eth1
    virtual_router_id 192
    priority 100                   ##此处B为95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 85c9a27b
    }
    virtual_ipaddress {
        172.16.42.200
    }
}
####以下是ipvs规则的相关配置
virtual_server 172.16.42.199 80 {        ##虚拟路由1对应ipvs规则配置
    delay_loop 3
    lb_algo sh         ##算法为源地址哈希
    lb_kind DR        ##为dr模型
    protocol TCP
    real_server 172.16.100.216 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 1
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 172.16.250.170 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 1
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}
virtual_server 172.16.42.200 80 {     ##虚拟路由2对应ipvs规则配置
    delay_loop 3
    lb_algo sh
    lb_kind DR
    protocol TCP
    real_server 172.16.100.216 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 1
            nb_get_retry 3
            delay_before_retry 1
        }
    }
    real_server 172.16.250.170 80 {
        weight 1
        HTTP_GET {
            url {
                path /
                status_code 200
            }
            connect_timeout 1
            nb_get_retry 3
            delay_before_retry 1
        }
    }
}

systemctl stop keepalived
systemctl start keepalived
此时在A B两台主机上都可查看到ipvs规则

[root@centos nginx]# ipvsadm -Ln                   
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.16.42.199:80 sh
  -> 172.16.100.216:80            Route   1      0          0         
  -> 172.16.250.170:80            Route   1      0          0         
TCP  172.16.42.200:80 sh
  -> 172.16.100.216:80            Route   1      0          0         
  -> 172.16.250.170:80            Route   1      0          0        

ip a 可看到虚拟ip分别漂移在A B 主机端口上
for i in {1..10}; do curl http://172.16.42.200;done
for i in {1..10}; do curl http://172.16.42.199;done
返回的结果始终是第一次匹配到的RS主机页
当停掉A B其中一台主机上的Keepalived服务时，调度并不受影响且故障切换后，同一个客户端访问依然能关联至此前绑定的RS