前篇文章我们主要讲解了LVS-DR模型的架构方式以及如何实现,想了解的小伙伴点这个LVS-NAT模型实现负载均衡 ,今天我们来进行实践的LVS中三种模型中的DR模型的架构以及实现方式。(实验环境还以Web集群作为实验对象)
此处我们LVS-DR模型环境架构也用四台Linux服务器,都有以下配置。
基于DR机制实现。当用户请求到达director之后,此时Director知道client请求的为一个集群服务(Director有一块网卡,此块网卡配置了DIP【对内】,又配置了一个网卡别名VIP【对外公网访问】),那么director不会改变原有的IP头部,通过指定的调度算法,选择任意一台RealServer,通过DIP和RealServer的RIP在同一个物理网络内进行AIP广播请求,请求到此台RealServer的MAC之后,再将MAC地址封装为此台RealServer的MAC,然后发给此台RealServer,注意此时的源地址和目标地址没有改变,RealServer进行解包发现目标地址为VIP(RealServer有一块网卡:此块网卡配置了RIP【对内】,还在本地回环接口lookback口配置了一个隐藏网卡别名的VIP),自己有一个VIP,那么进行解封装,再将源地址为VIP目标地址为Client的CIP进行封装然后把数据发送给用户(此时不在经过Director,直接发给客户端),完成整个负载调度过程。
使用软件VMware WorkStation 11,Director的DIP和RealServer的RS1、RS2为桥接模式,让其在同一个物理局域网内。(只作为测试环境)
这里我们为了模拟的更切合真实环境,所以Client请求的和RealServer的RIP不在同一网段,那么就需要一个Route设备,这里可选,你也可以使用全部都在一个物理网络内
搭建LVS集群只需要在Director服务器上安装ipvsadmin工具,此处我们使用RedHat自带的rpm包进行安装
版本:ipvsadm-1.25-10.el6.x86_64.rpm
[root@route ~]# service ntpd restart # 重启ntp服务器
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@Director ~]# ntpdate 192.168.40.254 # 客户端同步三台都要同步这里我们就不多说了
现在LVS已经是Linux标准内核的一部分,在Linux2.4内核以前,使用LVS时必须要重新编译内核以支持LVS功能模块,但是从Linux2.4内核以后,已经完全内置了LVS的各个功能模块,无需给内核打任何补丁,可以直接使用LVS提供的各种功能。
[root@Director Packages]# uname -r # 此处我们环境内核版本为2.6.32-358.el6.x86_64,所以不需要再打补丁,如果你的内核低于2.4那么则需要提前打补丁
2.6.32-358.el6.x86_64
[root@Director ~]# modprobe -l | grep ipvs # 以下有之前所解释的十个内核所支持的算法(如果能有以下搜索到那么你的内核就支持ipvs)
kernel/net/netfilter/ipvs/ip_vs.ko
kernel/net/netfilter/ipvs/ip_vs_rr.ko
kernel/net/netfilter/ipvs/ip_vs_wrr.ko
kernel/net/netfilter/ipvs/ip_vs_lc.ko
kernel/net/netfilter/ipvs/ip_vs_wlc.ko
kernel/net/netfilter/ipvs/ip_vs_lblc.ko
kernel/net/netfilter/ipvs/ip_vs_lblcr.ko
kernel/net/netfilter/ipvs/ip_vs_dh.ko
kernel/net/netfilter/ipvs/ip_vs_sh.ko
kernel/net/netfilter/ipvs/ip_vs_sed.ko
kernel/net/netfilter/ipvs/ip_vs_nq.ko
kernel/net/netfilter/ipvs/ip_vs_ftp.ko
我们使用本地光盘来作为YUM源
[root@Director ~]# mount /dev/sr0 /media # 挂载本地光盘到本地目录
mount: block device /dev/sr0 is write-protected, mounting read-only
三台服务器我们都需要配置,因为后面两台RealServer我们还需要安装web服务来作为集群服务。
[root@Director ~]# vim /etc/yum.repos.d/rhel-source.repo
[localhost] # 库名称(可随意)
name=localhost # 名称描述 (自定义)
baseurl=file:///media # yum源目录,源地址
enabled=1 # 是否用该yum源,0为禁用,1为使用
gpgcheck=0 # 检查GPG-KEY,0为不检查,1为检查
到此我们安装前准备就做好了,来下面进行下一步
[root@route ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.40.254
NETMASK=255.255.255.0
[root@route ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=eth2
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.0.254
NETMASK=255.255.0.0
[root@route ~]# service network restart # 重启network服务
Shutting down interface eth1: [ OK ]
Shutting down interface eth2: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
Bringing up interface eth2: [ OK ]
[root@route ~]# ifconfig # 查看是否配置生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:22:E7:C7
inet addr:192.168.40.254 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe22:e7c7/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:609 errors:0 dropped:0 overruns:0 frame:0
TX packets:292 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:57640 (56.2 KiB) TX bytes:22985 (22.4 KiB)
eth2 Link encap:Ethernet HWaddr 00:0C:29:22:E7:D1
inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fe22:e7d1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5316 errors:0 dropped:0 overruns:0 frame:0
TX packets:4282 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:493923 (482.3 KiB) TX bytes:598669 (584.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:9965 errors:0 dropped:0 overruns:0 frame:0
TX packets:9965 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1049774 (1.0 MiB) TX bytes:1049774 (1.0 MiB)
[root@route ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1 # 将0更改为1即可
[root@route ~]# sysctl -p # 刷新重新生效
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
[root@Director Packages]# yum install -y ipvsadm-1.25-10.el6.x86_64.rpm
[root@Director ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 # 此处我们不指定GATEWAY(真实生产跟其环境架构而定)
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.40.11
NETMASK=255.255.255.0
[root@Director ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-eth1:0 # 拷贝DIP的配置文件
[root@Director ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1:0 # 此内网网卡可不指定GATEWAY,因为和后台RealServer在同一个局域网(但也要根据真实环境而定)
DEVICE=eth1:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.21.10
NETMASK=255.255.0.0
[root@Director ~]# service network restart
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
[root@Director ~]# ifconfig # 查看配置是否生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:E5:9A:47
inet addr:192.168.40.11 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fee5:9a47/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3289 errors:0 dropped:0 overruns:0 frame:0
TX packets:2421 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:305233 (298.0 KiB) TX bytes:324497 (316.8 KiB)
eth1:0 Link encap:Ethernet HWaddr 00:0C:29:E5:9A:47
inet addr:172.16.21.10 Bcast:172.16.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1284 (1.2 KiB) TX bytes:1284 (1.2 KiB)
[root@Director ~]# iptables -F
[root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 # 根据我们此时的环境一定要将网关指向RouteServer的192.168.40.254,因为ReaServer响应Client的时候不在一个网络不能直接路由
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.40.12
NETMASK=255.255.255.0
GATEWAY=192.168.40.254
NM_CONTROLLED=yes
由于我们Director和两台RealServer都配置VIP地址,那么客户在请求VIP时,都有配置VIP,都会响应,这样的话网络就乱了,不知道包应该发给谁了,也就达不到我们所说的负载均衡的作用了,下面我们需要只要Client请求VIP时,只能Director来响应,那么我们有以下三个方案来解决这么问题。
让前段路由将请求发往VIP时,只能是Dirctor上的VIP响应
第三种方法是我们最常用的,也是最实用的:
[root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore
[root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce
[root@RS1 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-lo:0
[root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo:0 # NETMASK要为三个255,自己在一个广播域里,此网络只有自己
DEVICE=lo:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.21.10
NETMASK=255.255.255.255
GATEWAY=172.16.0.254
[root@RS1 ~]# service network restart
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
[root@RS1 ~]# ifconfig # 查看配置是否生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:41:4A:CC
inet addr:192.168.40.12 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe41:4acc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2749 errors:0 dropped:0 overruns:0 frame:0
TX packets:1871 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:265901 (259.6 KiB) TX bytes:241716 (236.0 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2320 errors:0 dropped:0 overruns:0 frame:0
TX packets:2320 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:293374 (286.4 KiB) TX bytes:293374 (286.4 KiB)
lo:0 Link encap:Local Loopback
inet addr:172.16.21.10 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
[root@RS1 ~]# route add -host 172.16.21.10 dev lo:0
[root@RS1 ~]# route -n # 查看添加路由条目
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.21.10 0.0.0.0 255.255.255.255 UH 0 0 0 lo
192.168.40.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth1
0.0.0.0 192.168.40.1 0.0.0.0 UG 0 0 0 eth1
[root@RS1 ~]# yum install -y httpd
[root@RS1 ~]# service httpd start # 启动httpd服务
Starting httpd: httpd: apr_sockaddr_info_get() failed for RS1
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[ OK ]
[root@RS1 ~]# service httpd status # 查看httpd是否启动
httpd (pid 11249) is running...
[root@RS1 ~]# netstat -an | grep :80 # 查看web服务80端口是否监听
tcp 0 0 :::80 :::* LISTEN
[root@RS1 ~]# echo "RS1.xuxingzhuang.com" > /var/www/html/index.html # 给web服务提供网页界面
[root@RS1 ~]# curl http://localhost # 访问本地web是否可以正常访问
RS1.xuxingzhuang.com
[root@RS1 ~]# iptables -F
[root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.40.13
NETMASK=255.255.255.0
GATEWAY=192.168.40.254
[root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore
[root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce
[root@RS2 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-lo:0
[root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.21.10
NETMASK=255.255.255.255
GATEWAY=172.16.0.254
[root@RS2 ~]# service network restart
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
[root@RS2 ~]# ifconfig # 查看配置是否生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:9A:31:FB
inet addr:192.168.40.13 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9a:31fb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3401 errors:0 dropped:0 overruns:0 frame:0
TX packets:3027 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:291858 (285.0 KiB) TX bytes:1029009 (1004.8 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2242 errors:0 dropped:0 overruns:0 frame:0
TX packets:2242 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:273214 (266.8 KiB) TX bytes:273214 (266.8 KiB)
lo:0 Link encap:Local Loopback
inet addr:172.16.21.10 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
[root@RS2 ~]# route add -host 172.16.21.10 dev lo:0
[root@RS2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.21.10 0.0.0.0 255.255.255.255 UH 0 0 0 lo
192.168.40.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth1
0.0.0.0 192.168.40.254 0.0.0.0 UG 0 0 0 eth1
[root@RS2 ~]# yum install -y httpd
[root@RS2 ~]# service httpd start # 启动httpd服务
Starting httpd: httpd: apr_sockaddr_info_get() failed for RS1
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[ OK ]
[root@RS2 ~]# service httpd status # 查看httpd是否启动
httpd (pid 10833) is running...
[root@RS2 ~]# netstat -an | grep :80 # 查看web服务80端口是否监听
tcp 0 0 :::80 :::* LISTEN
[root@RS2 ~]# echo "RS2.xuxingzhuang.com" > /var/www/html/index.html # 给web服务提供网页界面
[root@RS2 ~]# curl http://localhost # 访问本地web是否可以正常访问
RS2.xuxingzhuang.com
[root@RS2 ~]# iptables -F
[root@Director ~]# ipvsadm -A -t 172.16.21.10:80 -s rr
[root@Director ~]# ipvsadm -a -t 172.16.21.10:80 -r 192.168.40.12 -g -w 2 # 此处的-w指定权重是没有意义的,因为我们使用的为rr调度算法(轮叫),不过你也可以指定,后边我们改变算法时就不用重新定义了,省去了时间
[root@Director ~]# ipvsadm -a -t 172.16.21.10:80 -r 192.168.40.13 -g -w 1
[root@Director ~]# ipvsadm -L -n # 查看集群服务
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.21.10:80 rr
-> 192.168.40.12:80 Route 2 0 0
-> 192.168.40.13:80 Route 1 0 0
到底我们就基于LVS-DR模型就搭建好了一个web服务集群,DR模型在我们生活中用的还是比较多的,最主要的还是理解,后面我们还会继续写一下HA高可用集群。