lvs配置使用篇
1. lvs安装
根据内核选版本，我的内核是2.6.18，所以用1.24版本
[root@DS1 ~]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
[root@DS1 ~]# tar zxvf ipvsadm-1.24.tar.gz
[root@DS1 ~]# cd ipvsadm-1.24
[root@DS1 ipvsadm-1.24]# make 如果make报错，需要执行
[root@DS1 kernels]# ln -s /usr/src/kernels/2.6.18-371.11.1.el5-x86_64/ /usr/src/linux
如果/usr/src/kernels目录下是空的，则先执行
[root@DS1 kernels]# yum install kernel-devel
[root@DS1 ipvsadm-1.24]# make&&make install
[root@DS1 ipvsadm-1.24]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
至此 lvs安装完成
2. 配置lvs-DR(无高可用)
拓扑图
使用如下脚本配置
注意：dr模式不需要开启ip转发，REAL server上必须开启虚拟ip和抑制arp
+++++++++++++++++++++++++++++Direct server+++++++++++++++++++++++++++++++++++++++++++++++++++++++、
#!/bin/bash
# description: Start LVS of Director server
VIP=192.168.137.201
RIP1=192.168.137.10
RIP2=192.168.137.20
case "$1" in
start)
echo " start LVS of Director Server"
# set the Virtual IP Address and sysctl parameter
/sbin/ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 up
echo "1" >/proc/sys/net/ipv4/ip_forward #dr模式下此动作可以省略
#Clear IPVS table
/sbin/ipvsadm -C
#set LVS
/sbin/ipvsadm -A -t $VIP:80 -s rr -p 120
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
#Run LVS
/sbin/ipvsadm
;;
stop)
echo "close LVS Directorserver"
echo "0" >/proc/sys/net/ipv4/ip_forward
/sbin/ipvsadm -C
/sbin/ifconfig eth0:0 down
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
===========================REAL server=================================================
#!/bin/bash
#description; Start real server
VIP=192.168.137.201
case "$1" in
start)
echo " Start LVS of Real Server"
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0 #dr模式下此动作可以省略。
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
/sbin/ifconfig lo:0 down
echo " Stop LVS of Real Server"
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $0 {star|stop}"
exit 1
esac
测试效果：
因为我们加入了-p 120 两分钟的会话保持时间。如果从一个浏览器测试，请求会始终往一台机器上转发这个是正常的。
[root@DS1 init.d]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.137.201:http rr persistent 120
-> 192.168.137.20:http Route 1 2 6
-> 192.168.137.10:http Route 1 0 0
[root@DS1 init.d]#
此时请求都转给了192.168.137.20，这个时候我们把192.168.137.20上的apache停止，再刷新页面的时候会提示找不到页面。这就是lvs和nginx的区别，lvs不会自己检测real server的健康状态，而nginx有端口检测机制，一旦一台real server故障，nginx自己会检测并将其剔除。lvs实现此功能需要keepalived的tcpcheck或者用脚本实现。
3. lvs-NAT(无高可用)
注: nat模式的real server不需要虚拟ip和抑制arp
nat模式也可以不需虚拟ip，可以直接用dr的物理网卡ip
nat模式下real server不限制操作系统类型。
nat模式可以做端口映射
拓扑图
+++++++++++++++++++++++++++++Direct server+++++++++++++++++++++++++++++++++++++++++++++++++++++++
#!/bin/bash
# description: Start LVS of Director server
VIP=192.168.138.201
RIP1=192.168.137.10
RIP2=192.168.137.20
case "$1" in
start)
echo " start LVS of Director Server"
# set the Virtual IP Address and sysctl parameter
/sbin/ifconfig eth1:0 $VIP broadcast $VIP netmask 255.255.255.255 up
echo "1" >/proc/sys/net/ipv4/ip_forward #nat模式下该操作是必须的
#Clear IPVS table
/sbin/ipvsadm -C
#set LVS
/sbin/ipvsadm -A -t $VIP:80 -s rr
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -m
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -m
#Run LVS
/sbin/ipvsadm
;;
stop)
echo "close LVS Directorserver"
echo "0" >/proc/sys/net/ipv4/ip_forward
/sbin/ipvsadm -C
/sbin/ifconfig eth1:0 down
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
+++++++++++++++++++++++++++++++++++REAL SERVER+++++++++++++++++++++++++++++++++++++++++++++++
无需抑制arp，无需虚拟ip，只需要把网关指向dr的虚拟ip即可
route add defatul gw 192.168.137.100
测试效果：
访问192.168.138.201不停的刷新即可看到效果。
3. lvs-TUNNL(无高可用)
拓扑图：
++++++++++++++++++++++++++++++==dr配置++++++++++++++++++++++++++++++++++++++
#!/bin/bash
# description: Start LVS of Director server
modprobe ipip
VIP=192.168.137.201
#VIP=192.168.138.201
RIP1=192.168.137.10
RIP2=192.168.137.20
case "$1" in
start)
echo " start LVS of Director Server"
# set the Virtual IP Address and sysctl parameter
/sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
#Clear IPVS table
/sbin/ipvsadm -C
#set LVS
/sbin/ipvsadm -A -t $VIP:80 -s rr -p 30
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -i
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -i
#Run LVS
/sbin/ipvsadm
;;
stop)
echo "close LVS Directorserver"
/sbin/ipvsadm -C
/sbin/ifconfig tunl0 down
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
+++++++++++++++++++++++++++RS配置+++++++++++++++++++++++++++++++++++++++++++
#description; Start real server
modprobe ipip
VIP=192.168.137.201
case "$1" in
start)
echo " Start LVS of Real Server"
/sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
#/sbin/route add -host $VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
/sbin/ifconfig tunl0 down
echo " Stop LVS of Real Server"
echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $0 {star|stop}"
exit 1
esac
ipvsadm参数解释：
ipvsadm -L列出的状态含义解释：
[root@DS1 init.d]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.137.201:http rr persistent 30
-> 192.168.137.20:http Tunnel 1 0 0
-> 192.168.137.10:http Tunnel 1 0 0
ActiveConn是活动连接数,也就是tcp连接状态的ESTABLISHED;InActConn是指除了ESTABLISHED以外的,所有的其它状态的tcp连接.那既然这样,为什么从lvs里看的ActiveConn会比在真实机上通过netstats看到的ESTABLISHED高很多呢?问得好!这也是笔者一直迷惑而渐渐清晰的一个问题.原来lvs自身也有一个默认超时时间.可以用ipvsadm -L --timeout查看,默认是900 120 300,分别是TCP TCPFIN UDP的时间.也就是说一条tcp的连接经过lvs后,lvs会把这台记录保存15分钟,而不管这条连接是不是已经失效!所以如果你的服务器在15分钟以内有大量的并发请求连进来的时候,你就会看到这个数值直线上升.
其实很多时候,我们看lvs的这个连接数是想知道现在的每台机器的真实连接数吧?怎么样做到这一点呢?其实知道现在的ActiveConn是怎样产生的,做到这一点就简单了.举个例子:比如你的lvs是用来负载网站,用的模式是dr,后台的web server用的nginx.这时候一条请求过来,在程序没有问题的情况下,一条连接最多也就五秒就断开了.这时候你可以这样设置:ipvsadm --set 5 10 300.设置tcp连接只保持5秒中.如果现在ActiveConn很高你会发现这个数值会很快降下来,直到降到和你用nginx的status看当前连接数的时候差不多.你可以继续增加或者减小5这个数值,直到真实机的status连接数和lvs里的ActiveConn一致.
that's all.