LVS+KeepAlived+Nginx+Tomcat高可用解决方案
1、环境：
1、操作系统：Centos7.6
2、服务器配置如下：
| 服务器 | 软件 |
|---|---|
| 192.168.0.13 | LVS+KeepAlived |
| 192.168.0.14 | LVS+KeepAlived |
| 192.168.0.15 | Nginx |
| 192.168.0.16 | Nginx |
| 192.168.0.17 | Tomcat |
| 192.168.0.18 | Tomcat |
| VIP 192.168.1.200 |
2、安装lvs+keepalived
本文安装采用源代码编译方式进行安装。
2.1 Lvs
从2.4版本开始，linux内核默认支持LVS。要使用LVS的能力，只需安装一个LVS的管理工具：ipvsadm。
yum -y install ipvsadm
2.2 keepalived
同时在192.168.0.13和192.168.0.14两台服务器上操作：
技巧：在Xshell的撰写窗格里同时对两台服务器进行操作。
2.2.1 下载
`进入到/usr/local/src目录下`
[root@henry004 ~]# cd /usr/local/src
`下载keepalived`
[root@henry004 src]# wget https://www.keepalived.org/software/keepalived-2.0.20.tar.gz
`解压缩`
[root@henry001 src]# tar -zxvf keepalived-2.0.20.tar.gz
2.2.2 安装
`在/usr/local目录下创建keepalived文件夹`
[root@henry001 keepalived-2.0.20]# mkdir /usr/local/keepalived
`将keepalived安装到/usr/local/keepalived下，conf配置文件指定到目录/etc下`
[root@henry001 keepalived-2.0.20]# ./configure --prefix=/usr/local/keepalived --sysconf=/etc
`编译安装`
[root@henry004 keepalived-2.0.20]# make && make install
编译过程中可能会出现如下常见问题：
1、缺少OpenSSL
`-------错误信息---------------`
hecking openssl/ssl.h usability... no
checking openssl/ssl.h presence... no
checking for openssl/ssl.h... no
configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!
`----- ---解决方案--------------------`
yum -y install openssl-devel
2、缺少libnl/libnl-3
`--------错误信息---------------`
*** WARNING - this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.
`-------解决方案--------------------`
yum -y install libnl libnl-devel
2.2.3 配置
`进入安装后的路径 cd /data/program/keepalived, 创建软连接`
[root@henry001 sbin]# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
`把 keepalived的启动文件复制到init.d下，加入开机启动项`
[root@henry001 keepalived-2.0.20]# cp /usr/local/src/keepalived-2.0.20/keepalived/etc/init.d/keepalived /etc/init.d
`添加keepalived到系统服务`
[root@henry001 sbin]# chkconfig –add keepalived
chkconfig version 1.7.4 - Copyright (C) 1997-2000 Red Hat, Inc.
This may be freely redistributed under the terms of the GNU Public License.
usage: chkconfig [--list] [--type <type>] [name]
chkconfig --add <name>
chkconfig --del <name>
chkconfig --override <name>
chkconfig [--level <levels>] [--type <type>] <name> <on|off|reset|resetpriorities>
`检测是否添加成功`
[root@henry001 sbin]# chkconfig keepalived on
Note: Forwarding request to 'systemctl enable keepalived.service'.
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
2.2.4 启动keepalived
`启动keepalived服务`
[root@henry001 sbin]# systemctl start keepalived.service
`查看keepalived状态`
[root@henry001 sbin]# systemctl status keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2020-03-07 22:13:54 CST; 3s ago
Process: 25684 ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 25685 (keepalived)
CGroup: /system.slice/keepalived.service
├─25685 /usr/local/keepalived/sbin/keepalived -D
├─25686 /usr/local/keepalived/sbin/keepalived -D
└─25687 /usr/local/keepalived/sbin/keepalived -D
-------------------------------------------------
操作keepalived的命令有如下：
`----启动-----`
systemctl start keepalived.service
`----重启-----`
systemctl restart keepalived.service
`----停止-----`
systemctl stop keepalived.service
`----查看状态-----`
systemctl status keepalived.service
2.3 防火墙
为方便测试，我直接关闭了防火墙，在实际应用中可以根据需要开启防火墙的端口，此外还要设置服务器的安全策略，我的是阿里云的服务器，就在阿里云服务器控制台设置了安全策略，开放了需要的端口。
关闭防火墙：
`关闭防火墙`
[root@henry001 sysconfig]# systemctl stop firewalld
`查看防火墙状态`
[root@henry001 sysconfig]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: `inactive (dead)`
Docs: man:firewalld(1)
Mar 07 22:39:08 henry001 systemd[1]: Starting firewalld - dynamic firewall daemon...
Mar 07 22:39:08 henry001 systemd[1]: Started firewalld - dynamic firewall daemon.
Mar 07 22:39:23 henry001 systemd[1]: Stopping firewalld - dynamic firewall daemon...
Mar 07 22:39:24 henry001 systemd[1]: Stopped firewalld - dynamic firewall daemon.
3、安装Nginx
同时在192.168.0.15和192.168.0.16两台服务器上操作，对Nginx+Tomcat的安装请参考文章：
实现Nginx+Tomcat负载均衡 https://blog.csdn.net/qq_33996921/article/details/104999852
4、配置keepalived
4.1master服务器
先来配置192.168.0.13的主机，指定其为master服务器；
`进入配置文件目录`
[root@henry001 ~]# cd /etc/keepalived
[root@henry001 keepalived]# ls
keepalived.conf samples
`编辑配置文件信息`
[root@henry001 keepalived]# vim keepalived.conf
编辑keepalived.conf文件
global_defs {
#notification_email {
# [email protected]
#}
# notification_email_from [email protected]
# smtp_server 192.168.80.1
#smtp_connection_timeout 30
router_id LVS_DEVEL # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_instance VI_1 {
state MASTER #指定Keepalived的角色，MASTER为主，BACKUP为备 记得大写
interface eth0 #网卡id 不同的电脑网卡id会有区别 可以使用:ip a查看
virtual_router_id 51 #虚拟路由编号，主备要一致
priority 100 #定义优先级，数字越大，优先级越高，主DR必须大于备用DR
advert_int 1 #检查间隔，默认为1s
authentication { #这里配置的密码最多为8位，主备要一致，否则无法正常通讯
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.200 #定义虚拟IP(VIP)为192.168.1.200，可多设，每行一个
}
}
# 定义对外提供服务的LVS的VIP以及port
virtual_server 192.168.0.200 80 {
delay_loop 6 # 设置健康检查时间，单位是秒
lb_algo rr # 设置负载调度的算法为wlc
lb_kind DR # 设置LVS实现负载的机制，有NAT、TUN、DR三个模式
#nat_mask 255.255.255.0
persistence_timeout 0
protocol TCP
real_server 192.168.0.15 80 { # 指定real server1的IP地址
weight 3 # 配置节点权值，数字越大权重越高
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.0.16 80 { # 指定real server2的IP地址
weight 3 # 配置节点权值，数字越大权重越高
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
4.1 backup服务器
配置192.168.0.14的主机，指定其为backup服务器；
`进入配置文件目录`
[root@henry004 ~]# cd /etc/keepalived
[root@henry004 keepalived]# ls
keepalived.conf samples
`编辑配置文件信息`
[root@henry004 keepalived]# vim keepalived.conf
编辑keepalived.conf 文件
global_defs {
#notification_email {
# [email protected]
#}
# notification_email_from [email protected]
# smtp_server 192.168.80.1
# smtp_connection_timeout 30
router_id LVS_DEVEL # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_instance VI_1 {
state BACKUP #指定Keepalived的角色，MASTER为主，BACKUP为备 记得大写
interface eth0 #网卡id 不同的电脑网卡id会有区别 可以使用:ip a查看
virtual_router_id 51 #虚拟路由编号，主备要一致
priority 50 #定义优先级，数字越大，优先级越高，主DR必须大于备用DR
advert_int 1 #检查间隔，默认为1s
authentication { #这里配置的密码最多为8位，主备要一致，否则无法正常通讯
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.200 #定义虚拟IP(VIP)为192.168.1.200，可多设，每行一个
}
}
# 定义对外提供服务的LVS的VIP以及port
virtual_server 192.168.0.200 80 {
delay_loop 6 # 设置健康检查时间，单位是秒
lb_algo rr # 设置负载调度的算法为wlc
lb_kind DR # 设置LVS实现负载的机制，有NAT、TUN、DR三个模式
#nat_mask 255.255.255.0
persistence_timeout 0
protocol TCP
real_server 192.168.0.16 80 { # 指定real server1的IP地址
weight 3 # 配置节点权值，数字越大权重越高
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.0.15 80 { # 指定real server2的IP地址
weight 3 # 配置节点权值，数字越大权重越高
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
5、查看虚拟IP
查看master服务器：
[root@henry001 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:16:3e:30:cc:a2 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.13/24 brd 192.168.0.255 scope global dynamic eth0
valid_lft 315332386sec preferred_lft 315332386sec
`inet 192.168.1.200/32 scope global eth0`
valid_lft forever preferred_lft forever
查看backup服务器
[root@henry004 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:16:3e:30:9f:0f brd ff:ff:ff:ff:ff:ff
inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic eth0
valid_lft 315348979sec preferred_lft 315348979sec
下面我们停止掉master服务器上的keepalived，虚拟ip将会漂移到backup服务器上
[root@henry004 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:16:3e:30:9f:0f brd ff:ff:ff:ff:ff:ff
inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic eth0
valid_lft 315348845sec preferred_lft 315348845sec
`inet 192.168.0.200/32 scope global eth0`
valid_lft forever preferred_lft forever
至此,一个LVS+KeepAlived+Nginx+Tomcat的高可用方案就搭建完成了。