lvs负载均衡和双机热备高可用配置（keepalived）

简介

keepalived的作用是检测服务器的状态，如果有一台web服务器宕机，或工作出现故障，Keepalived将检测到，并将有故障的服务器从系统中剔除，同时使用其他服务器代替该服务器的工作，当服务器工作正常后Keepalived自动将服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的服务器。

原理

Layer3,4,5工作在IP/TCP协议栈的IP层，TCP层，及应用层,原理分别如下：
Keepalived使用Layer3的方式工作式时，Keepalived会定期向服务器群中的服务器发送一个ICMP的数据包（既我们平时用的Ping程序）,如果发现某台服务的IP地址没有激活，Keepalived便报告这台服务器失效，并将它从服务器群中剔除，这种情况的典型例子是某台服务器被非法关机。Layer3的方式是以服务器的IP地址是否有效作为服务器工作正常与否的标准。

配置环境

server1：MASTER机
server4：BACKUP机
server2：Real server机1
server3：Real server机2

配置步骤

1、用server1做lvs负载均衡的DR模式

[root@server1 ~]# ip addr add 172.25.254.100/24 dev eth0
[root@server1 ~]# ipvsadm -A -t 172.25.254.100:80 -s rr
[root@server1 ~]# ipvsadm -a -t 172.25.254.100:80 -r 172.25.254.2:80 -g
[root@server1 ~]# ipvsadm -a -t 172.25.254.100:80 -r 172.25.254.3:80 -g
[root@server1 ~]# /etc/init.d/ipvsadm save 
ipvsadm: Saving IPVS table to /etc/sysconfig/ipvsadm:      [  OK  ]
[root@server1 ~]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.254.100:http rr
  -> server2:http                 Route   1      0          0         
  -> server3:http                 Route   1      0          0  

2、server2和server3做Read server配置

[root@server2 ~]# ip addr add 172.25.254.100/32 dev eth0
[root@server2 ~]# arptables -A IN -d 172.25.254.100 -j DROP
[root@server2 ~]# arptables -A OUT -s 172.25.254.100 -j mangle --mangle-ip-s 172.25.254.2
[root@server2 ~]# arptables -L
Chain IN (policy ACCEPT)
target     source-ip            destination-ip       source-hw          destination-hw     hlen   op         hrd        pro       
DROP       anywhere             172.25.254.100       anywhere           anywhere           any    any        any        any       

Chain OUT (policy ACCEPT)
target     source-ip            destination-ip       source-hw          destination-hw     hlen   op         hrd        pro       
mangle     172.25.254.100       anywhere             anywhere           anywhere           any    any        any        any       --mangle-ip-s server2 

Chain FORWARD (policy ACCEPT)
target     source-ip            destination-ip       source-hw          destination-hw     hlen   op         hrd        pro       
[root@server2 ~]# /etc/init.d/arptables_jf save 
Saving current rules to /etc/sysconfig/arptables:          [  OK  ]

此时，物理机访问vip实现lvs负载均衡轮询

3、keepalived配置

server1配置

1、下载keepalived软件包

[root@server1 ~]# ls
keepalived-1.4.3.tar.gz  
[root@server1 ~]# tar zxf keepalived-1.4.3.tar.gz 
[root@server1 ~]# ls
keepalived-1.4.3  keepalived-1.4.3.tar.gz  
[root@server1 ~]# yum install -y gcc openssl-devel   ##安装kp需要的依赖包

2、编译，检测并安装

[root@server1 keepalived-1.4.3]# ./configure --prefix=/usr/local/keepalived --with-init=SYSV   ##编译
[root@server1 keepalived-1.4.3]# make && make install  ##检测安装
[root@server1 keepalived-1.4.3]# cd /usr/local/keepalived/
[root@server1 keepalived]# ll    ##查看是否安装成功
total 16
drwxr-xr-x 2 root root 4096 Jun 21 23:09 bin
drwxr-xr-x 5 root root 4096 Jun 21 23:09 etc
drwxr-xr-x 2 root root 4096 Jun 21 23:09 sbin
drwxr-xr-x 5 root root 4096 Jun 21 23:09 share

3、创建软连接

[root@server1 keepalived]# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/     ##创建执行脚本软连接
[root@server1 keepalived]# ln -s /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/    ##创建全局配置文件软连接
[root@server1 keepalived]# ln -s /usr/local/keepalived/etc/keepalived/ /etc/                        ##创建配置文件连接
[root@server1 keepalived]# ln -s /usr/local/keepalived/sbin/keepalived /sbin/                          ##创建命令连接
[root@server1 keepalived]# chmod +x /usr/local/keepalived/etc/rc.d/init.d/keepalived                    ##给执行脚本给权限
[root@server4 ~]# which keepalived
/sbin/keepalived

4、修改配置文件

/etc/keepalived/keepalived.conf

  1 ! Configuration File for keepalived
  2 
  3 global_defs {
  4    notification_email {  
  5         root@localhost    #邮件设置
  6    }
  7    notification_email_from [email protected]
  8    smtp_server 127.0.0.1   ##回环接口
  9    smtp_connect_timeout 30
 10    router_id LVS_DEVEL
 11    vrrp_skip_check_adv_addr
 12 #   vrrp_strict     ##关闭  否则会在火墙加策略，阻止访问
 13    vrrp_garp_interval 0
 14    vrrp_gna_interval 0
 15 }
 16 
 17 vrrp_instance VI_1 {
 18     state MASTER      ##设置为MASTER
 19     interface eth0
 20     virtual_router_id 51   ##接口，BACKUP机需要和MASTER保持一致
 21     priority 100
 22     advert_int 1
 23     authentication {
 24         auth_type PASS
 25         auth_pass 1111
 26     }
 27     virtual_ipaddress {
 28         172.25.254.100    ##vip
 29     }
 30 }
 31 
 32 virtual_server 172.25.254.100 80 {   ##vip和接口
 33     delay_loop 1
 34     lb_algo rr
 35     lb_kind DR
 36 #    persistence_timeout 50    ##关闭持续连接
 37     protocol TCP
 38 
 39     real_server 172.25.254.2 80 {   Rip1和接口
 40         weight 1
 41         TCP_CHECK {
 42             connect_timeout 3
 43             retry 3
 44             delay_before_retry 3
 45         }
 46     }
 47     real_server 172.25.254.3 80 {   Rip2和接口
 48         weight 1
 49         TCP_CHECK {
 50             connect_timeout 3
 51             retry 3
 52             delay_before_retry 3
 53         }
 54     }
 55 }

server4

在server4上安装gcc ipvsadm openssl-devel等依赖包

[root@server4 ~]# yum install ipvsadm gcc openssl-devel httpd -y

1、将server1中的/usr/local/keepalived目录直接复制到server4中

[root@server1 keepalived]# scp -r /usr/local/keepalived/ server4:/usr/local/

2、在server4中创建软连接

[root@server4 ~]# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@server4 ~]# ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@server4 ~]# ln -s /usr/local/keepalived/etc/keepalived/ /etc/
[root@server4 ~]# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
[root@server4 ~]# chmod +x /usr/local/keepalived/etc/rc.d/init.d/keepalived 

3、将server1中的配置文件复制到server4中

[root@server1 keepalived]# scp /etc/keepalived/keepalived.conf server4:/etc/keepalived/
root@server4's password: 
keepalived.conf                               100% 1015     1.0KB/s   00:00   

4、修改server4配置文件

server4配置文件和server1差不多，只需要改两行即可

 17 vrrp_instance VI_1 {
 18     state BACKUP     ###
 19     interface eth0
 20     virtual_router_id 51
 21     priority 50    ###优先级要比server1低
 22     advert_int 1

server1和server4配置成功

1、开启server1和server4的keepalived

[root@server1 keepalived]# /etc/init.d/keepalived start 
Starting keepalived:                                       [  OK  ]
[root@server4 keepalived]# /etc/init.d/keepalived start 
Starting keepalived:                                       [  OK  ]

测试：

配置成功后，我们用物理机访问vip，访问结果为server2和server3轮询，且arp指向server1的物理地址，当server1的kp关闭后，物理机访问，轮询依旧正常，不过arp指向了server4的物理地址，所以说在主机出现故障时，备机能迅速代替工作，当server1开启kp后，物理机访问arp指向server1的物理地址，则表示当主机恢复正常后，主机会马上接管vip，从而实现双机热备高可用

[root@foundation77 Desktop]# arp -d 172.25.254.100
[root@foundation77 Desktop]# curl 172.25.254.100
server3
[root@foundation77 Desktop]# curl 172.25.254.100
server2
[root@foundation77 Desktop]# curl 172.25.254.100
server3
[root@foundation77 Desktop]# curl 172.25.254.100
server2
[root@foundation77 Desktop]# arp -an | grep 100
? (172.25.254.100) at 52:54:00:f9:4e:d3 [ether] on br0
[root@foundation77 Desktop]# arp -d 172.25.254.100
[root@foundation77 Desktop]# curl 172.25.254.100
server3
[root@foundation77 Desktop]# curl 172.25.254.100
server2
[root@foundation77 Desktop]# curl 172.25.254.100
server3
[root@foundation77 Desktop]# curl 172.25.254.100
server2
[root@foundation77 Desktop]# arp -an | grep 100
? (172.25.254.100) at 52:54:00:f2:23:ae [ether] on br0
[root@foundation77 Desktop]# curl 172.25.254.100
server3
[root@foundation77 Desktop]# curl 172.25.254.100
server2
[root@foundation77 Desktop]# curl 172.25.254.100
server3
[root@foundation77 Desktop]# curl 172.25.254.100
server2
[root@foundation77 Desktop]# arp -an | grep 100
? (172.25.254.100) at 52:54:00:f9:4e:d3 [ether] on br0