LVS-NAT && LVS-DR模型
Lvs三种模式
NAT
DR
TUN
LVS-NAT：目标地址转换
注意事项：
realserver的网关必须指向DIP，通常都是私有地址
directory位于client和realserver之间，并负责处理进出的所有通信
directory支持端口映射，向外提供服务的端口 和realserver的端口可以不相同
realserver可以使用任意操作系统
较大规模应用场景中，directory易成为系统瓶颈
LVS-DR：直接路由
各集群节点，必须要跟directory在同一物理网络中
RIP可以使用公网地址，实现远程管理（也可以使用私有地址）
directory仅负责处理入展请求，响应报文则有realserver 直接发往客户端
集群节点不能将网关指向DIP
directory不支持端口映射
LVS-TUN：隧道
集群节点可以跨越互联网internet
RIP必须是公网地址
directory仅处理入展请求，响应报文则由realserver直接发往客户端
realserver网关不能指向directory
只有支持隧道功能的OS才能用于realserver
不支持端映射
拓扑如下：
LVS-NAT
第一：确保两台web可以正常访问
第二：网络连接正常（Directory 两块网卡，NAT连接真机，vmnet2 连接两台WEB server）
第三：配置LVS Directory
#grep -i 'vs' /boot/config-2.6.32-431.el6.x86_64
说明已经支持ipvs
#yum list | grep ipvsadm
#yum install ipvsadm #安装ipvsadm工具
定义集群服务：
#ipvsadm -A -t 172.16.100.100:80 -s rr
添加集群主机：
ipvsadm-a -t 172.16.100.100:80 -r 10.0.0.2 �Cm
ipvsadm-a -t 172.16.100.100:80 -r 10.0.0.3 �Cm
打开Directory的路由转发功能，所有的主机均关闭防火墙。
个主机之间使用ntpdate进行实践同步
# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.100.100:80 rr
-> 10.0.0.2:80 Masq 1 0 5
-> 10.0.0.3:80 Masq 1 0 5
[root@localhost ~]#
可以看到访问统计信息
# ipvsadm -L --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 172.16.100.100:http 15 81 81 7392 8045
-> 10.0.0.2:http 7 35 35 3406 3832
-> 10.0.0.3:http 8 46 46 3986 4213
[root@localhost ~]#
Ipvs规则的保存：
ipvsadm -S > /etc/ipvs.conf
载入配置：
ipvsadm -R < /etc/ipvs.conf #下次启动的话，加入到开机启动即可
LVS-DR：直接路由
Directory配置VIP
# cat /etc/sysconfig/network-scripts/ifcfg-eth0:0
DEVICE=eth0:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=none
IPADDR=172.16.100.200
NETMASK=255.255.255.0
GATEWAY=172.16.100.2
IPV6INIT=no
USERCTL=no
[root@localhost ~]#
配置路由：
# route add -host 172.16.100.200 dev eth0:0
RS服务器上千万记得不能先配置VIP地址，必须等关掉arp广播才可以配置
配置RS1服务器，首先关闭ARP报文功能：
# sysctl -w net.ipv4.conf.eth0.arp_announce=2
# sysctl -w net.ipv4.conf.all.arp_announce=2
# sysctl -w net.ipv4.conf.eth0.arp_ignore=1
# sysctl -w net.ipv4.conf.all.arp_ignore=1
或者
echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore
这样的形式都可以的
配置VIP地址：
#ifconfig lo:0 172.16.100.200 broadcast 172.16.100.200 netmask 255.255.255.255 up
# route add -host 172.16.100.200 dev lo:0
配置RS2服务器：
# sysctl -w net.ipv4.conf.eth0.arp_announce=2
# sysctl -w net.ipv4.conf.all.arp_announce=2
# sysctl -w net.ipv4.conf.eth0.arp_ignore=1
# sysctl -w net.ipv4.conf.all.arp_ignore=1
#ifconfig lo:0 172.16.100.200 broadcast 172.16.100.200 netmask 255.255.255.255 up
# route add -host 172.16.100.200 dev lo:0
LVS-Directory配置：
# ipvsadm -A -t 172.16.100.200:80 -s wlc
# ipvsadm -a -t 172.16.100.200:80 -r172.16.100.101 -g -w 2
# ipvsadm -a -t 172.16.100.200:80 -r172.16.100.102 -g -w 1
# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.100.200:80 wlc
-> 172.16.100.101:80 Route 2 0 0
-> 172.16.100.102:80 Route 1 0 0
[root@localhost ~]#
通过浏览器访问：
http://172.16.100.200就可以访问到RS1 和RS2 两台服务器
这里提供两个脚本：
Directory的脚本：
#!/bin/bash
#
# LVS script for VS/DR
#
. /etc/rc.d/init.d/functions
#
VIP=172.16.100.200
RIP1=172.16.100.101
RIP2=172.16.100.102
PORT=80
#
case "$1" in
start)
/sbin/ifconfig eth0:1 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev eth0:1
# Since this is the Director we must beable to forward packets
echo 1 > /proc/sys/net/ipv4/ip_forward
# Clear all iptables rules.
/sbin/iptables -F
# Reset iptables counters.
/sbin/iptables -Z
# Clear all ipvsadm rules/services.
/sbin/ipvsadm -C
# Add an IP virtual service for VIP192.168.0.219 port 80
# In this recipe, we will use theround-robin scheduling method.
# In production, however, you should use aweighted, dynamic scheduling method.
/sbin/ipvsadm -A -t $VIP:80 -s wlc
# Now direct packets for this VIP to
# the real server IP (RIP) inside thecluster
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1 -g -w 1
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2 -g -w 2
/bin/touch /var/lock/subsys/ipvsadm &> /dev/null
;;
stop)
# Stop forwarding packets
echo 0 > /proc/sys/net/ipv4/ip_forward
# Reset ipvsadm
/sbin/ipvsadm -C
# Bring down the VIP interface
/sbin/ifconfig eth0:1 down
/sbin/route del $VIP
/bin/rm -f /var/lock/subsys/ipvsadm
echo "ipvs is stopped..."
;;
status)
if[ ! -e /var/lock/subsys/ipvsadm ]; then
echo "ipvsadm is stopped ..."
else
echo"ipvs is running ..."
ipvsadm -L -n
fi
;;
*)
echo "Usage: $0 {start|stop|status}"
;;
esac
RS脚本：
#!/bin/bash
#
# Script to start LVS DR real server.
# description: LVS DR real server
#
. /etc/rc.d/init.d/functions
VIP=172.16.100.200
host=`/bin/hostname`
case "$1" in
start)
# Start LVS-DR real server on this machine.
/sbin/ifconfig lo down
/sbin/ifconfig lo up
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
;;
stop)
# Stop LVS-DR real server loopback device(s).
/sbin/ifconfig lo:0 down
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 >/proc/sys/net/ipv4/conf/all/arp_announce
;;
status)
# Status of LVS-DR real server.
islothere=`/sbin/ifconfig lo:0 | grep $VIP`
isrothere=`netstat -rn | grep "lo:0" | grep $VIP`
if [ ! "$islothere" -o ! "isrothere" ];then
# Either the route or the lo:0 device
# not found.
echo "LVS-DR real server Stopped."
else
echo "LVS-DR real server Running."
fi
;;
*)
# Invalid entry.
echo "$0: Usage: $0 {start|status|stop}"
exit 1
;;
esac
使用以下脚本在Directory上进行主机的健康检查：
#!/bin/bash
#
VIP=172.16.100.200
CPORT=80
FAIL_BACK=127.0.0.1
RS=("172.16.100.101""172.16.100.102")
declare-a RSSTATUS
RW=("2""1")
RPORT=80
TYPE=g
CHKLOOP=3
LOG=/var/log/ipvsmonitor.log
addrs(){
ipvsadm -a -t $VIP:$CPORT -r $1:$RPORT -$TYPE-w $2
[ $? -eq 0 ] && return 0 || return 1
}
delrs(){
ipvsadm -d -t $VIP:$CPORT -r $1:$RPORT
[ $? -eq 0 ] && return 0 || return 1
}
checkrs(){
local I=1
while [ $I -le $CHKLOOP ]; do
if curl --connect-timeout 1 http://$1&> /dev/null; then
return 0
fi
let I++
done
return 1
}
initstatus(){
local I
local COUNT=0;
for I in ${RS[*]}; do
if ipvsadm -L -n | grep"$I:$RPORT" && > /dev/null ; then
RSSTATUS[$COUNT]=1
else
RSSTATUS[$COUNT]=0
fi
let COUNT++
done
}
initstatus
while:; do
let COUNT=0
for I in ${RS[*]}; do
if checkrs $I; then
if [ ${RSSTATUS[$COUNT]} -eq 0 ]; then
addrs $I ${RW[$COUNT]}
[ $? -eq 0 ] &&RSSTATUS[$COUNT]=1 && echo "`date +'%F %H:%M:%S'`, $I isback." >> $LOG
fi
else
if [ ${RSSTATUS[$COUNT]} -eq 1 ]; then
delrs $I
[ $? -eq 0 ] &&RSSTATUS[$COUNT]=0 && echo "`date +'%F %H:%M:%S'`, $I isgone." >> $LOG
fi
fi
let COUNT++
done
sleep 5
done
LVS的持久化连接：
在基于SSL，需要用到持久连接
PPC:将来自同于一个客户端对同一个集群的请求，始终定向值前端选定的RS（持久端口端连接）
PCC:将来自同一客户端对所有端口的请求，始终定向至此前选定的RS（持久客户端连接）
把所有端口统统定义为集群服务，所有请求指向RS
PNMPP：持久防火墙标记连接。
PCC
# ipvsadm �CC
# iptables �Ct mangle �CA PREROUTING �Cd 172.16.100.200 �Ci eth0 �Cp tcp �Cdport 80 �Cj MARK�Cset-mark 8
# iptables �Ct mangle �CA PREROUTING �Cd 172.16.100.200 �Ci eth0 �Cp tcp �Cdport 443 �Cj MARK�Cset-mark 8
# ipvsadm �CA �Cf 8 �Cs rr �Cp 600
# ipvsadm �Ca �Cf 8 �Cr 172.16.100.101 �Cg �Cw 2
# ipvsadm �Ca �Cf 8 �Cr 172.16.100.102 �Cg �Cw 2