LVS＋tomcat负载均衡部署

 

 

今天去公安部部署考试系统，一共有俩台机器，想跑8个tomcat出来，作负载。
原定于用tomcat自己实现负载，不过怕出万一就准备了lvs这种方式，不过没有用到，不过把准备的资料贴一下。

 

tomcat状态监控脚本
这个脚本实现的是tomcat端口状态的监控，web页面是否能访问的到，进程存活状态的监控，如果任何一个死掉，就重启tomcat并发送邮件报警。
在2太机器上都在后台执行

[root@bogon 22]# cat monitor.sh
#!/bin/bash
while [ 1 ]
do
#set server ipadd
SERVERA=192.168.0.38
STATUS=SERVERA
#set tomcat path
SVRA_TOMCAT_DIR=/opt/service
SVRB_TOMCAT_DIR=/opt/service
#set server port
TOMCAT1=8081
TOMCAT2=8082
TOMCAT3=8083
TOMCAT4=8084
#set mail
[email protected]
[email protected]
##########################################################################
NULL=/dev/null
LOG=/var/log/monitor-tomcat.log
SLEEP="sleep 10"
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
>.T1
>.T2
>.T3
>.T4
STARTUP=bin/startup.sh
SHUTDOWN=bin/shutdown.sh
AR1=`grep Restart $LOG |wc -l`
FILE="monitor tomcat status tomcat is alive"
if [ $STATUS = SERVERA ] ; then
T1WEB_ST=`links  http://$SERVERA:$TOMCAT1/monitor.jsp > .T1`
T1=`grep "$FILE" .T1 > $NULL ; echo $?`
T1STATUS=`nc -vv -z $SERVERA $TOMCAT1 > $NULL 2>&1 ; echo $?`
if [ $T1 -eq 0 ] ; then
echo $TIME  tomcat1 -------- is --------  alive >> $NULL
if [ $T1STATUS -eq 0 ]; then
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME  tomcat1 -------- is --------  alive >> $LOG
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat1 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat1 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat1 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat1/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat1/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat1/$STARTUP 2 > $NULL
$SLEEP
fi
fi
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat1 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat1 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat1 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat1/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat1/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat1/$STARTUP 2 > $NULL
$SLEEP
fi
fi
T2WEB_ST=`links  http://$SERVERA:$TOMCAT2/monitor.jsp > .T2`
T2=`grep "$FILE" .T2 > $NULL ; echo $?`
T2STATUS=`nc -vv -z $SERVERA $TOMCAT2 > $NULL 2>&1 ; echo $?`
if [ $T2 -eq 0 ] ; then
echo $TIME  tomcat2  is   alive >> $NULL
if [ $T2STATUS -eq 0 ]; then
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME  tomcat2 -------- is --------  alive >> $LOG
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat2 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat2 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat2 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat2/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat2/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat2/$STARTUP 2 > $NULL
$SLEEP
fi
fi
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat2 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat2 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat2 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat2/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat2/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat2/$STARTUP 2 > $NULL
$SLEEP
fi
fi

T3WEB_ST=`links  http://$SERVERA:$TOMCAT3/monitor.jsp > .T3`
T3=`grep "$FILE" .T3 > $NULL ; echo $?`
T3STATUS=`nc -vv -z $SERVERA $TOMCAT3 > $NULL 2>&1 ; echo $?`
if [ $T3 -eq 0 ] ; then
echo $TIME  tomcat3  is   alive >> $NULL
if [ $T3STATUS -eq 0 ]; then
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME  tomcat3 -------- is --------  alive >> $LOG
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat3 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat3 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat3 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat3/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat3/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat3/$STARTUP 2 > $NULL
$SLEEP
fi
fi
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat3 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat3 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat3 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat3/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat3/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat3/$STARTUP 2 > $NULL
$SLEEP
fi
fi

T4WEB_ST=`links  http://$SERVERA:$TOMCAT1/monitor.jsp > .T4`
T4=`grep "$FILE" .T4  > $NULL ; echo $?`
T4STATUS=`nc -vv -z $SERVERA $TOMCAT4 > $NULL 2>&1 ; echo $?`

if [ $T4 -eq 0 ] ; then
echo $TIME  tomcat4  is   alive >> $NULL
if [ $T4STATUS -eq 0 ]; then
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME  tomcat4 -------- is --------  alive >> $LOG
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat4 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat4 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat4 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat4/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat4/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat4/$STARTUP 2 > $NULL
$SLEEP
fi
fi
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat4 -------- is -------- shutdown" >> $LOG
echo $TIME "tomcat4 ------------------->>> Restart" >> $LOG
PID=`ps aux |grep tomcat4 > $NULL ;echo $?`
if [ $PID -eq 0 ] ; then
$SVRA_TOMCAT_DIR/tomcat4/$SHUTDOWN 2 > $NULL
$SVRA_TOMCAT_DIR/tomcat4/$STARTUP 2 > $NULL
$SLEEP
else
$SVRA_TOMCAT_DIR/tomcat4/$STARTUP 2 > $NULL
$SLEEP
fi
fi
fi
sleep 20
TOMCAT='(tomcat1|tomcat2|tomcat3|tomcat4)'
PIDSTATE=` ps aux |grep -v grep | egrep $TOMCAT  > /dev/null ;echo $?`
PORTSTATE1=`nc -vv -z $SERVERA $TOMCAT1 > $NULL 2>&1 ; echo $?`
PORTSTATE2=`nc -vv -z $SERVERA $TOMCAT2 > $NULL 2>&1 ; echo $?`
PORTSTATE3=`nc -vv -z $SERVERA $TOMCAT3 > $NULL 2>&1 ; echo $?`
PORTSTATE4=`nc -vv -z $SERVERA $TOMCAT4 > $NULL 2>&1 ; echo $?`
if [ $PIDSTATE -eq 0 ] && [ $PORTSTATE1 -eq 0 ] && [ $PORTSTATE2 -eq 0 ] &&  [ $PORTSTATE3 -eq 0 ] &&  [ $PORTSTATE4 -eq 0 ]  ; then
$SLEEP
else
$SLEEPMIN
fi
PIDSTATE=` ps aux |grep -v grep | egrep $TOMCAT  > /dev/null ;echo $?`
PORTSTATE1=`nc -vv -z $SERVERA $TOMCAT1 > $NULL 2>&1 ; echo $?`
PORTSTATE2=`nc -vv -z $SERVERA $TOMCAT2 > $NULL 2>&1 ; echo $?`
PORTSTATE3=`nc -vv -z $SERVERA $TOMCAT3 > $NULL 2>&1 ; echo $?`
PORTSTATE4=`nc -vv -z $SERVERA $TOMCAT4 > $NULL 2>&1 ; echo $?`
if [ $PIDSTATE -eq 0 ] && [ $PORTSTATE1 -eq 0 ] && [ $PORTSTATE2 -eq 0 ] &&  [ $PORTSTATE3 -eq 0 ] &&  [ $PORTSTATE4 -eq 0 ]  ; then
$SLEEP
else
$SLEEPMIN
fi
PIDSTATE=` ps aux |grep -v grep | egrep $TOMCAT  > /dev/null ;echo $?`
PORTSTATE1=`nc -vv -z $SERVERA $TOMCAT1 > $NULL 2>&1 ; echo $?`
PORTSTATE2=`nc -vv -z $SERVERA $TOMCAT2 > $NULL 2>&1 ; echo $?`
PORTSTATE3=`nc -vv -z $SERVERA $TOMCAT3 > $NULL 2>&1 ; echo $?`
PORTSTATE4=`nc -vv -z $SERVERA $TOMCAT4 > $NULL 2>&1 ; echo $?`
if [ $PIDSTATE -eq 0 ] && [ $PORTSTATE1 -eq 0 ] && [ $PORTSTATE2 -eq 0 ] &&  [ $PORTSTATE3 -eq 0 ] &&  [ $PORTSTATE4 -eq 0 ]  ; then
$SLEEP
else
$SLEEPMIN
fi
PIDSTATE=` ps aux |grep -v grep | egrep $TOMCAT > /dev/null ;echo $?`
PORTSTATE1=`nc -vv -z $SERVERA $TOMCAT1 > $NULL 2>&1 ; echo $?`
PORTSTATE2=`nc -vv -z $SERVERA $TOMCAT2 > $NULL 2>&1 ; echo $?`
PORTSTATE3=`nc -vv -z $SERVERA $TOMCAT3 > $NULL 2>&1 ; echo $?`
PORTSTATE4=`nc -vv -z $SERVERA $TOMCAT4 > $NULL 2>&1 ; echo $?`
if [ $PIDSTATE -eq 0 ] && [ $PORTSTATE1 -eq 0 ] && [ $PORTSTATE2 -eq 0 ] &&  [ $PORTSTATE3 -eq 0 ] &&  [ $PORTSTATE4 -eq 0 ]  ; then
$SLEEP
else
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME "tomcat  -------- is -------- shutdown" >> $LOG
$SLEEPMIN
fi

AR2=`grep Restart $LOG  |wc -l`
LINEB=`echo "$AR2-$AR1"|bc `
MAILSUBJECT=` grep Restart $LOG | tail -n $LINEB `

if [ $LINEB -gt 0 ] ; then
TIME=`date +"%Y-%m-%d  %H:%M:%S"`
echo $TIME send mail  to $MAILTO >> $LOG
sendmail -t <<EOF
from: $MAILFROM
to: $MAILTO
subject: warning
$MAILSUBJECT
EOF
fi
done

监控脚本就是这样的，需要将monitor.jsp文件放在应用目录中，确保可以访问的到
monitor.jsp文件内容

[root@bogon 22]# cat monitor.jsp
<html>
<head>
<tatle>monitor tomcat status</tatle>
<body>
<%out.println("tomcat is alive");%>
</body>
</head>
</html>

本来想用ipvsadm这个软件做的，把ipvsadm的命令也贴下。

[root@bogon 22]# cat ipvsadm_manager.sh
#!/bin/bash
ACCPORT=80
ACCIP=192.168.1.2
SVRAIP=192.168.0.38
SVRBIP=192.168.0.39
SVRA1PORT=8081
SVRA2PORT=8082
SVRA3PORT=8083
SVRA4PORT=8084
SVRB1PORT=8081
SVRB2PORT=8082
SVRB3PORT=8083
SVRB4PORT=8084
/sbin/ipvsadm -C
/sbin/ipvsadm -A -t $ACCIP:$ACCPORT -s rr
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRAIP:$SVRA1PORT -m
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRAIP:$SVRA2PORT -m
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRAIP:$SVRA3PORT -m
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRAIP:$SVRA4PORT -m
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRBIP:$SVRB1PORT -m
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRBIP:$SVRB2PORT -m
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRBIP:$SVRB3PORT -m
/sbin/ipvsadm -a -t $ACCIP:$ACCPORT -r $SVRBIP:$SVRB4PORT -m

需要iptables的支持
iptables脚本的内容，为了方便管理，将2台真实的机器22端口映射出来。

[root@bogon 22]# cat iptables_manager.sh
#!/bin/bash
NETWORK=192.168.0.0
LOCALHOST=192.168.1.2
NATPORTA=2222
NATPORTB=3333
SERVERA=192.168.0.38
SERVERB=192.168.0.39
iptables -F  
iptables -X
iptables -F -t nat
iptables -X -t nat
iptables -t nat -A POSTROUTING -s $NETWORK -j MASQUERADE
iptables -t nat -A PREROUTING -d $LOCALHOST -p tcp --dport $NATPORTA -j DNAT --to $SERVERA:22
iptables -t nat -A PREROUTING -d $LOCALHOST -p tcp --dport $NATPORTB -j DNAT --to $SERVERB:22
service iptables save
service iptables restart
[root@bogon 22]#

lvs使用nat方式
思路是，因为有俩台设备，做nat三台设备正好，所以再一台配置好的机器上装个虚拟己，呵呵，这也是虚拟化了。
在虚拟机搭建负载服务，安装ipvsadm软件，客户端访问的时候，访问的IP地址实际就是虚拟机的IP地址，而没有访问真实的服务器，而真实的页面是由2台真实的服务器提供的。
这样用ipvsadm对俩太机器的8个tomcat实现负载。