Keepalivd+Haproxy高可用负载均衡
一、Keepalivd+Haproxy介绍
1、简介
虽然keepalived+LVS是最好的搭配，但是其配置相对比较复杂，对于小型的集群可以采用Keepalivd+Haproxy搭配，而且自带健康检查，配置起来相对比较简单。通过Keepalived实现对Haproxy的高可用(通过脚本对Haproxy的状态进行检查)，Haproxy实现负载均衡。
2、keepalived原理图：
二、实验环境
1、安装环境
软件网盘链接： https://pan.baidu.com/s/15i6LvJ1H6A1JrNvvr9mSRw 密码: vp73
selinux iptables off
Operating System: Red Hat Enterprise Linux Server 7.0 (Maipo)
Keepalived官方下载链接：http://www.keepalived.org/download.html
Haproxy官方下载链接：https://www.haproxy.org/download/
2、功能说明
| 主机名 | IP | 搭建服务 | 功能说明 |
|---|---|---|---|
| server1(Master) | 10.10.10.1 | Keepalived+Haproxy | 通过keepavlied实现对Haproxy(负载均衡)高可用 |
| server2(Slave) | 10.10.10.2 | Keepalived+Haproxy | 通过keepavlied实现对Haproxy(负载均衡)高可用 |
三、Haproxy安装及配置
1、编译安装Haproxy
[root@server1 ~]# useradd -u 1001 yy
[root@server1 ~]# echo "1"|passwd --stdin yy
[root@server1 ~]# vim /etc/security/limits.conf
yy soft nofile 65535
yy hard nofile 65535
[root@server1 ~]# su - yy
[yy@server1 ~]$ wget https://www.haproxy.org/download/1.5/src/haproxy-1.5.3.tar.gz
[yy@server1 ~]$ tar xf haproxy-1.5.3.tar.gz
[yy@server1 ~]$ cd haproxy-1.5.3/
[yy@server1 haproxy-1.5.3]$ make TARGET=linux2628 ARCH=x86_64 PREFIX=/home/yy/haproxy
[yy@server1 haproxy-1.5.3]$ make install PREFIX=/home/yy/haproxy
[yy@server1 haproxy-1.5.3]$ cp /home/yy/haproxy-1.5.3/examples/haproxy.cfg /home/yy/haproxy
[yy@server1 haproxy-1.5.3]$ cp -r /home/yy/haproxy-1.5.3/examples/errorfiles/ /home/yy/haproxy
2、参数说明：
ARGET=linux26：内核版本，使用uname -r查看内核，如：2.6.18-371.el5，此时该参数就为linux26；内核大于2.6.28的用：TARGET=linux2628 ARCH=x86_64：系统位数
[yy@server1 haproxy-1.5.3]$ vim /home/yy/haproxy-1.5.3/README
3、配置haproxy.cfg ：
[yy@server1 haproxy-1.5.3]$ vim /home/yy/haproxy/haproxy.cfg
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 4096
#chroot /usr/share/haproxy ###注释掉
uid yy
gid yy
daemon
pidfile /home/yy/haproxy/haproxy.pid ###加入pid
#debug
#quiet
defaults
log global
mode http
option httplog
option dontlognull
retries 3
#redispatch
maxconn 2000
timeout connect 5000 ###下面3个地方都要修改，默认的不支持
timeout client 50000
timeout server 50000
listen appli1-rewrite 0.0.0.0:10001
cookie SERVERID rewrite
balance roundrobin
listen appli2-insert 0.0.0.0:10002
option httpchk
balance roundrobin
cookie SERVERID insert indirect nocache
server inst1 192.168.114.56:80 cookie server01 check inter 2000 fall 3
server inst2 192.168.114.56:81 cookie server02 check inter 2000 fall 3
capture cookie vgnvisitor= len 32
option httpclose # disable keep-alive
rspidel ^Set-cookie:\ IP= # do not let this cookie tell our internal IP address
listen appli3-relais 0.0.0.0:10003
dispatch 192.168.135.17:80
listen appli4-backup 0.0.0.0:10004
option httpchk /index.html
option persist
balance roundrobin
server inst1 192.168.114.56:80 check inter 2000 fall 3
server inst2 192.168.114.56:81 check inter 2000 fall 3 backup
listen ssl-relay 0.0.0.0:8443
option ssl-hello-chk
balance source
server inst1 192.168.110.56:443 check inter 2000 fall 3
server inst2 192.168.110.57:443 check inter 2000 fall 3
server back1 192.168.120.58:443 backup
listen appli5-backup 0.0.0.0:10005
option httpchk *
balance roundrobin
cookie SERVERID insert indirect nocache
server inst1 192.168.114.56:80 cookie server01 check inter 2000 fall 3
server inst2 192.168.114.56:81 cookie server02 check inter 2000 fall 3
server inst3 192.168.114.57:80 backup check inter 2000 fall 3
capture cookie ASPSESSION len 32
timeout server 20000 ###修改
option httpclose # disable keep-alive
option checkcache # block response if set-cookie & cacheable
rspidel ^Set-cookie:\ IP= # do not let this cookie tell our internal IP address
errorloc 502 http://192.168.114.58/error502.html
errorfile 503 /home/yy/haproxy/errorfiles/503.http ###修改为上面复制过去的路径
4、启动Haproxy
[yy@server1 haproxy-1.5.3]$ /home/yy/haproxy/sbin/haproxy -f /home/yy/haproxy/haproxy.cfg
5、查看是否启动成功
[yy@server1 ~]$ ps -aux|grep haproxy
yy 2070 0.0 0.2 12896 1368 ? Ss 23:33 0:00 /home/yy/haproxy/sbin/haproxy -f /home/yy/haproxy/haproxy.cfg
yy 2269 0.0 0.2 112640 984 pts/0 R+ 23:49 0:00 grep --color=auto haproxy
6、报错解决
（1）报错：
[yy@server1 haproxy-1.5.3]$ /home/yy/haproxy/sbin/haproxy -f /home/yy/haproxy/haproxy.cfg
[WARNING] 183/232300 (2455) : [/home/yy/haproxy/sbin/haproxy.main()] Cannot raise FD limit to 8217.
[WARNING] 183/232300 (2455) : [/home/yy/haproxy/sbin/haproxy.main()] FD limit (1024) too low for maxconn=4096/maxsock=8217. Please raise 'ulimit-n' to 8217 or more to avoid any trouble.
（2）解决方案：
[root@server1 ~]# vim /etc/security/limits.conf
yy soft nofile 65535
yy hard nofile 65535
重新登陆即可启动成功！！！
7、配置server2：
[root@server2 ~]# useradd -u 1001 yy
[root@server2 ~]# echo "1"|passwd --stdin yy
[yy@server1 ~]$ scp -r haproxy [email protected]: ###server1中发送到server2中
[root@server1 ~]# vim /etc/security/limits.conf
yy soft nofile 65535
yy hard nofile 65535
[yy@server2 ~]$ /home/yy/haproxy/sbin/haproxy -f /home/yy/haproxy/haproxy.cfg ###启动Haproxy
四、Keepalived安装
1、安装依赖包：
[root@server1 ~]# yum install -y openssl-devel
2、编译安装：
[root@server1 ~]# tar xf keepalived-2.0.4.tar.gz
[root@server1 ~]# cd keepalived-2.0.4/
[root@server1 keepalived-2.0.4]# ./configure --prefix=/home/yy/keepalived --with-init=SYSV
[root@server1 keepalived-2.0.4]# make && make install
3、可以看到所支持的：
4、查看是否安装成功
[root@server1 keepalived-2.0.4]# cd /usr/local/keepalived/
[root@server1 keepalived]# ll
total 0
drwxr-xr-x 2 root root 20 Jul 4 00:48 bin
drwxr-xr-x 5 root root 50 Jul 4 00:48 etc
drwxr-xr-x 2 root root 23 Jul 4 00:48 sbin
drwxr-xr-x 5 root root 37 Jul 4 00:48 share
5、进行链接：
[root@server1 keepalived]# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@server1 keepalived]# ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@server1 keepalived]# ln -s /usr/local/keepalived/etc/keepalived/ /etc/
[root@server1 keepalived]# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
[root@server1 keepalived]# chmod +x /usr/local/keepalived/etc/rc.d/init.d/keepalived
6、修改配置：
[root@server1 keepalived]# vim /etc/keepalived/keepalived.conf ###后面的注释掉既可
global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
#vrrp_strict ###严格执行VRRP协议规范，否则VIP不同
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state MASTER
interface eth0 ###网卡名，根据自己的网卡名进行设置
virtual_router_id 51 ###路由ID
priority 100 ###优先级
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.10.10.100 ###VIP（虚拟IP）
}
}
7、启动测试：
[root@server1 keepalived]# systemctl restart keepalived
[root@server1 keepalived]# ps aux |grep keepalived
root 14645 0.0 0.1 42376 676 ? Ss 00:56 0:00 keepalived -D
root 14647 0.0 0.2 42376 1136 ? S 00:56 0:00 keepalived -D
root 14659 0.0 0.2 112640 984 pts/0 R+ 00:57 0:00 grep --color=auto keepalived
这样keepalived就搭建好了，但是我们可以发现Haproxy和Keepalived没有任何的联系，因此我们许需要加入脚本来读取Haproxy的状态，从而达到高可用的作用！！！
五、加入脚本检测Haproxy状态
1、检查Haproxy状态脚本：
[root@server1 ~]# vim /etc/keepalived/chk_haproxy.sh
#!/bin/bash
Haproxy_Status=`ps -C haproxy --no-header |wc -l`
if [ $Haproxy_Status -eq 0 ];then
/home/yy/haproxy/sbin/haproxy -f /home/yy/haproxy/haproxy.cfg
sleep 3
if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
/etc/init.d/keepalived stop
fi
fi
[root@server1 ~]# chmod +x /etc/keepalived/chk_haproxy.sh
2、修改Keepalived.conf配置：
global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
#vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script check_haproxy { ###指定监控脚本
script "/etc/keepalived/chk_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.10.10.100
}
track_script { ###监控此模块
check_haproxy
}
}
[root@server1 ~]# chkconfig --add keepalived
[root@server1 ~]# chkconfig keepalived on ###设置为开机自动启动
3、安装server2
（1）发送脚本及配置文件
[root@server1 ~]# scp -r /usr/local/keepalived/ [email protected]:/usr/local/
[root@server1 ~]# scp /etc/keepalived/chk_haproxy.sh [email protected]:/etc/keepalived/
[root@server2 ~]# cd /usr/local/keepalived/
[root@server2 keepalived]# ll
total 0
drwxr-xr-x 2 root root 20 Jul 4 01:37 bin
drwxr-xr-x 5 root root 50 Jul 4 01:37 etc
drwxr-xr-x 2 root root 23 Jul 4 01:37 sbin
drwxr-xr-x 5 root root 37 Jul 4 01:37 share
[root@server2 keepalived]# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@server2 keepalived]# ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@server2 keepalived]# ln -s /usr/local/keepalived/etc/keepalived/ /etc/
[root@server2 keepalived]# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
[root@server2 keepalived]# chmod +x /usr/local/keepalived/etc/rc.d/init.d/keepalived
（2）Keepalived.conf中修改：
[root@server2 keepalived]# systemctl restart keepalived
[root@server2 keepalived]# chkconfig --add keepalived
[root@server2 keepalived]# chkconfig keepalived on
六、测试
1、在server1中我们把脚本的启动给禁了
2、关闭掉Haproxy
3、发现VIP飘移到server2
4、恢复server1后VIP恢复
七、VIP漂移设置
默认设置为：VIP会漂移回Master；另外一个设置：当Master挂掉后，VIP漂移到Slave上，当Master恢复后，VIP不漂移回来，在优先级高（即之前得Master）把
MASTER修改为BACKUP（虽然2个都为BACKUP，但是根据优先级选择出MASTER即server1），并加入nopreempt参数！！！
1、配置keepalived.conf（server1）
修改配置Master（server1）配置，slave（server2）配置文件不做修改！！！
[root@server1 ~]# /etc/init.d/keepalived restart
2、测试结果
当Master（server1）的haproxy挂掉后，VIP漂移到Slave（server2）上，在恢复Master（server1）的haproxy后，VIP并不会漂移回Master（server1）！！！