LVS-NAT &TUN实现负载均衡
工作原理：
基于NAT机制实现。当用户请求到达director之后，director将请求报文的目标地址（即VIP）改成选定的realserver 地址，同时将报文的目标端口也改成选定的realserver的相应端口，最后将报文请求发送到指定的realserver。在服务器端得到数据 后，realserver将数据返给director，而director将报文的源地址和源端口改成VIP和相应端口，然后把数据发送给用户，完成整个 负载调度过程。· 一台Director：
版本：Red Hat 6.5
双网卡：
eth0: VIP:192.168.43.1/24(真实生产环境下一定将网关指向运营商的公网IP)
eth2: DIP:172.25.53.1/24(此IP必须和后台的RealSever在同一个网段内)
· 两台RealServer：
· 版本：Red Hat 6.5单网卡：
RealServer1：RIP1:172.25.53.2/24(网关必须指向Director的DIP)
RealServer2: RIP2:172.25.53.3/24(网关必须执行Director的DIP)在Director服务器上安装ipvsadmin工具，此处我们使用RedHat自带的rpm包进行安装
yum install ipvsadm.x86_64
配置网卡，并设置IP
[root@server1 ~]# ip addr add 192.168.43.1/24 dev eth2
[root@server1 ~]# ip addr
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:07:35:d5 brd ff:ff:ff:ff:ff:ff
inet 172.25.53.1/24 brd 172.25.53.255 scope global eth0
inet6 fe80::5054:ff:fe07:35d5/64 scope link
valid_lft forever preferred_lft forever
3: eth2: mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 52:54:00:de:cc:dc brd ff:ff:ff:ff:ff:ff
inet 192.168.43.1/24 scope global eth2
[root@server1 ~]# ip link set up eth2
[root@server1 ~]# ping 192.168.43.250
PING 192.168.43.250 (192.168.43.250) 56(84) bytes of data.
64 bytes from 192.168.43.250: icmp_seq=1 ttl=64 time=0.718 ms
64 bytes from 192.168.43.250: icmp_seq=2 ttl=64 time=0.148 ms
^C
--- 192.168.43.250 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1870ms
rtt min/avg/max/mdev = 0.148/0.433/0.718/0.285 ms 打开本机的路由转发功能
查看本地路由功能是否打开(1 开启 0 关闭)
[root@server1 ~]# sysctl -a |grep ip_forward
net.ipv4.ip_forward = 0
[root@server1 ~]# sysctl -w net.ipv4.ip_forward=1
net.ipv4.ip_forward = 1
[root@server1 ~]# vim /etc/sysctl.conf 永久编辑配置RealServer服务器
配置Server2网卡，并设置IP
[root@server2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
BOOTPROTO="static"
ONBOOT="yes"
IPADDR=172.25.53.2
PREFIX=24
TYPE=Ethernet
GATEWAY=172.25.53.250
NM_CONTROLLED=yes
重启
[root@server2 ~]# service network restart
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: Determining if ip address 172.25.53.2 is already in use for device eth0...
[ OK ]
安装httpd服务
[root@server2 ~]# yum install httpd
Loaded plugins: product-id, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
rhel-source | 3.9 kB 00:00
Setting up Install Process
Package httpd-2.2.15-29.el6_4.x86_64 already installed and latest version
Nothing to do
[root@server2 ~]# /etc/init.d/httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.25.53.2 for ServerName
[ OK ]
[root@server2 ~]# netstat -an | grep :80
tcp 0 0 172.25.53.2:44804 172.25.53.250:80 TIME_WAIT
tcp 0 0 :::80 :::* LISTEN
访问本地web是否可以正常访问
[root@server2 ~]# curl 172.25.53.2
www.westos.org -server2
清空火墙策略
[root@server2 ~]# iptables -FServer3与上述相同，ip为172.25.53.3
测试Director是否可以正常访问Realserver的服务
[root@foundation63 ~]# curl 172.25.53.2
www.westos.org -server2
[root@foundation63 ~]# curl 172.25.53.3
www.westos.org -server3
测试正常将RealServer加入Web集群服务
[root@server1 ~]# ipvsadm -l
IP Virtual Server version 1.2.1 (size=4194304)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@server1 ~]# ipvsadm -A -t 192.168.43.1:80 -s rr
[root@server1 ~]# ipvsadm -a -t 192.168.43.1:80 -r 172.25.53.2:80 -m
[root@server1 ~]# ipvsadm -a -t 192.168.43.1:80 -r 172.25.53.3:80 -m最终测试
[root@foundation53 ~]# curl 192.168.43.1
www.westos.org - server3
[root@foundation53 ~]# curl 192.168.43.1
www.westos.org -server2
[root@foundation53 ~]# curl 192.168.43.1
www.westos.org - server3
[root@foundation53 ~]# curl 192.168.43.1
www.westos.org -server2成功
LVS-TUN
LVS、TUN简介
LVS 是Linux Virtual Server的简称，在实际环境中经常作为B/S结构的网络应用中的负载均衡器来使用，工作在7层网络模型中的，网络层，也就是通常说的IP层，由于数据的处理是在Linux内核态完成的，所以相对反向代理服务器来说，性能一般会高一些；
TUN 是IP Tunneling ，IP隧道的简称，它将调度器收到的IP数据包封装在一个新的IP数据包中，转交给应用服务器，然后实际服务器的返回数据会直接返回给用户。
优点
TUN模式可以解决DR模式下不能跨网段的问题，甚至可以跨公网进行。
[root@server1 ~]# ip addr
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:07:35:d5 brd ff:ff:ff:ff:ff:ff
inet 172.25.53.1/24 brd 172.25.53.255 scope global eth0
inet6 fe80::5054:ff:fe07:35d5/64 scope link
valid_lft forever preferred_lft forever
3: eth2: mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 52:54:00:de:cc:dc brd ff:ff:ff:ff:ff:ff
[root@server1 ~]# ip link set down eth2
[root@server1 ~]# ip addr
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:07:35:d5 brd ff:ff:ff:ff:ff:ff
inet 172.25.53.1/24 brd 172.25.53.255 scope global eth0
inet6 fe80::5054:ff:fe07:35d5/64 scope link
valid_lft forever preferred_lft forever
3: eth2: mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 52:54:00:de:cc:dc brd ff:ff:ff:ff:ff:ff 利用ipip建立tunnel（隧道）
[root@server1 ~]# modprobe ipip
[root@server1 ~]# ip addr
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:07:35:d5 brd ff:ff:ff:ff:ff:ff
inet 172.25.53.1/24 brd 172.25.53.255 scope global eth0
inet6 fe80::5054:ff:fe07:35d5/64 scope link
valid_lft forever preferred_lft forever
3: eth2: mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 52:54:00:de:cc:dc brd ff:ff:ff:ff:ff:ff
4: tunl0: mtu 1480 qdisc noop state DOWN
link/ipip 0.0.0.0 brd 0.0.0.0
[root@server1 ~]# ip link set up tunl0
[root@server1 ~]# ip addr add 172.25.53.100/24 dev tunl0
[root@server1 ~]# ip addr
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:07:35:d5 brd ff:ff:ff:ff:ff:ff
inet 172.25.53.1/24 brd 172.25.53.255 scope global eth0
inet6 fe80::5054:ff:fe07:35d5/64 scope link
valid_lft forever preferred_lft forever
3: eth2: mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 52:54:00:de:cc:dc brd ff:ff:ff:ff:ff:ff
4: tunl0: mtu 1480 qdisc noqueue state UNKNOWN
link/ipip 0.0.0.0 brd 0.0.0.0
inet 172.25.53.100/24 scope global tunl0
其他两台RealServer设置相同
==================
[root@server2 ~]# modprobe ipip
[root@server2 ~]# ip link set up tunl0
[root@server2 ~]# ip addr add 172.25.53.100/24 dev tunl0
[root@server3 ~]# modprobe ipip
[root@server3 ~]# ip link set up tunl0
[root@server3 ~]# ip addr add 172.25.53.100/24 dev tunl0
[root@server2 ~]# /etc/init.d/arptables_jf start
Flushing all current rules and user defined chains: [ OK ]
Clearing all current rules and user defined chains: [ OK ]
Applying arptables firewall rules: [ OK ]
[root@server2 ~]# sysctl -a|grep rp_filter
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.tunl0.rp_filter = 1
net.ipv4.conf.tunl0.arp_filter = 0
[root@server2 ~]# sysctl -w net.ipv4.conf.tunl0.rp_filter=0
net.ipv4.conf.tunl0.rp_filter = 0
[root@server3 ~]# /etc/init.d/arptables_jf start
Flushing all current rules and user defined chains: [ OK ]
Clearing all current rules and user defined chains: [ OK ]
Applying arptables firewall rules: [ OK ]
[root@server3 ~]# sysctl -w net.ipv4.conf.tunl0.rp_filter=0
net.ipv4.conf.tunl0.rp_filter = 0
将RealServer加入Web集群服务
[root@server1 ~]# ipvsadm -A -t 172.25.53.100:80 -s rr
[root@server1 ~]# ipvsadm -a -t 172.25.53.100:80 -r 172.25.53.2:80 -i
[root@server1 ~]# ipvsadm -a -t 172.25.53.100:80 -r 172.25.53.3:80 -i
[root@server1 ~]# ipvsadm -l
IP Virtual Server version 1.2.1 (size=4194304)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.53.100:http rr
-> server2:http Tunnel 1 0 0
-> server3:http Tunnel 1 0 0 客户访问
[root@foundation53 ~]# curl 172.25.53.100
www.westos.org - server3
[root@foundation53 ~]# curl 172.25.53.100
www.westos.org -server2
[root@foundation53 ~]# curl 172.25.53.100
www.westos.org - server3
[root@foundation53 ~]# curl 172.25.53.100
www.westos.org -server2
[root@foundation53 ~]# curl 172.25.53.100
www.westos.org - server3
[root@foundation53 ~]# curl 172.25.53.100
www.westos.org -server2