Nginx 搭建LVS-DR模式
1 环境准备
1.1 服务器与IP规划
| LVS服务器 |
|---|
| VIP（虚拟IP）：192.168.51.103 |
| DIP（转发者IP/内网IP）：192.168.51.4 |
| Nginx服务器两台（RealServer） |
|---|
| RIP（真实IP/内网IP）：192.168.51.5 |
| RIP（真实IP/内网IP）：192.168.51.6 |
1.2 停止NetworkManager服务
三台服务器分别执行以下两行命令，因为服务器使用的是虚拟机，需要停止 NetworkManager，不然会出现问题
[root@localhost network-scripts]# systemctl stop NetworkManager
[root@localhost network-scripts]# systemctl disable NetworkManager
2 配置LVS节点与ipvsadm
2.1 创建子接口
LVS服务器
[root@localhost network-scripts]# cd /etc/sysconfig/network-scripts
[root@localhost network-scripts]# ls
ifcfg-ens33 ifcfg-lo ifdown-eth ifdown-isdn ifdown-routes ifdown-TeamPort ifup-aliases ifup-ippp ifup-plip ifup-ppp ifup-Team ifup-wireless network-functions-ipv6
ifdown ifdown-ippp ifdown-post ifdown-sit ifdown-tunnel ifup-bnep ifup-ipv6 ifup-plusb ifup-routes ifup-TeamPort init.ipv6-global
ifcfg-ens33.bak ifdown-bnep ifdown-ipv6 ifdown-ppp ifdown-Team ifup ifup-eth ifup-isdn ifup-post ifup-sit ifup-tunnel network-functions
[root@localhost network-scripts]# cp ifcfg-ens33 ifcfg-ens33:1
2.2 修改配置文件
[root@localhost network-scripts]# vi ifcfg-ens33:1
BOOTPROTO=static
DEVICE=ens33:1
ONBOOT=yes
IPADDR=192.168.51.103
NETMASK=255.255.255.0
2.3 刷新配置
[root@localhost network-scripts]# service network restart
2.4 安装ipvsadm
- 好像阿里云不支持虚拟IP,购买官网的负载均衡
- 好像腾讯云支持最大虚拟IP数量为10
ipvsadm -Ln
[root@localhost network-scripts]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
3 为两台RS配置虚拟IP
两台RS机器的配置相同，下面的操作需要在两台机器上都进行操作。
3.1 复制配置
[root@localhost network-scripts]# cd /etc/sysconfig/network-scripts
[root@localhost network-scripts]# ls
ifcfg-ens33 ifdown-eth ifdown-isdn ifdown-routes ifdown-TeamPort ifup-aliases ifup-ippp ifup-plip ifup-ppp ifup-Team ifup-wireless network-functions-ipv6
ifcfg-ens33.bak ifdown ifdown-ippp ifdown-post ifdown-sit ifdown-tunnel ifup-bnep ifup-ipv6 ifup-plusb ifup-routes ifup-TeamPort init.ipv6-global
ifcfg-lo ifdown-bnep ifdown-ipv6 ifdown-ppp ifdown-Team ifup ifup-eth ifup-isdn ifup-post ifup-sit ifup-tunnel network-functions
[root@localhost network-scripts]# cp ifcfg-lo ifcfg-lo:1
3.2 修改配置
[root@localhost network-scripts]# vi ifcfg-lo:1
DEVICE=lo:1
IPADDR=192.168.51.103
NETMASK=255.255.255.255
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
3.3 刷新配置
方式一
[root@localhost network-scripts]# ifup lo
方式二
[root@localhost network-scripts]# service network restart
4 为两台RS配置arp
4.1 arp介绍
ARP响应级别与通告行为 。
1.arp-ignore：ARP响应级别（处理请求）
- 0：只要本机配置了ip，就能响应请求
- 1：请求的目标地址到达对应的网络接口，才会响应请求
2.arp-announce：ARP通告行为（返回响应）
- 0：本机上任何网络接口都向外通告，所有的网卡都能接受到通告
- 1：尽可能避免本网卡与不匹配的目标进行通告
- 2：只在本网卡通告
4.2 修改配置文件
[root@localhost network-scripts]# vi /etc/sysctl.conf
# configration for lvs
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
4.3 刷新配置
[root@localhost network-scripts]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
4.4 增加网关
[root@localhost network-scripts]# route add -host 192.168.51.103 dev lo:1
设置开机添加
[root@localhost network-scripts]# echo "route add -host 192.168.51.103 dev lo:1" >> /etc/rc.local
5 使用ipvsadm配置集群规则
5.1 创建LVS节点
用户访问数据的集群调度者
[root@localhost network-scripts]# ipvsadm -A -t 192.168.51.103:80 -s rr -p 5
- -A：添加集群
- -t：tcp协议
- ip地址：设定集群的访问ip，也就是LVS的虚拟ip
- -s：设置负载均衡的算法，rr表示轮询
- -p：设置连接持久化的时间
5.2 创建2台RS真实服务器
[root@localhost network-scripts]# ipvsadm -a -t 192.168.51.103:80 -r 192.168.51.5:80 -g
[root@localhost network-scripts]# ipvsadm -a -t 192.168.51.103:80 -r 192.168.51.6:80 -g
5.3 保存到规则库
不保存，重启会失效
[root@localhost network-scripts]# ipvsadm -S
5.4 检查集群
查看集群列表
[root@localhost network-scripts]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.51.103:80 rr persistent 5
-> 192.168.51.5:80 Route 1 0 0
-> 192.168.51.6:80 Route 1 0 0
查看集群状态
[root@localhost network-scripts]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 192.168.51.103:80 0 0 0 0 0
-> 192.168.51.5:80 0 0 0 0 0
-> 192.168.51.6:80 0 0 0 0 0
5.5 其他命令
# 重启ipvsadm，重启后需要重新配置
service ipvsadm restart
# 查看持久化连接
ipvsadm -Ln --persistent-conn
# 查看连接请求过期时间以及请求源ip和目标ip
ipvsadm -Lnc
# 设置tcp tcpfin udp 的过期时间（一般保持默认）
ipvsadm --set 1 1 1
# 查看过期时间
ipvsadm -Ln --timeout
详细的帮助文档
ipvsadm -h
man ipvsadm
6 相关信息
- 博文不易，辛苦各位猿友点个关注和赞，感谢