keepalived实现Tomcat服务双机热备

http://blog.csdn.net/u013256816/article/details/49127995

http://blog.csdn.net/kkdelta/article/details/39433137

项目中需要采用tomcat双机热备机制，以确保系统性能。本人通过查阅资料以及亲自实现完成，写下这篇博文，供个人读者参考。

1. 规划

1.1服务器环境规划

负载服务器master及WEB服务器1真实IP：10.10.195.53

负载服务器backup及WEB服务器2真实IP：10.10.195.190

负载服务器虚拟ip：10.10.195.212

1.2 软件环境规划

操作系统：Red Hat Enterprise Linux Server release 5.6 (Tikanga)

keepalived：keepalived-1.2.19

Java：jdk-1.7.0_79

Tomcat：apache-tomcat-7.0.64

2. 负载服务器配置

这里只叙述如何安装配置keepalived，至于java以及tomcat的安装及配置，这里不赘述。

2.1 安装keepalived

 [~]tar -zxvf keepalived-1.2.19.tar.gz
 [~] cd keepalived-1.2.19
 [keepalived-1.2.19] ./configure --prefix=/usr/local/keepalived --disable-fwmark
 #(如果直接输入./configure有可能报错：configure:error: No SO_MARK declaration in headers)
 [keepalived-1.2.19] make
 [keepalived-1.2.19] make install

2.2 配置keepalived服务

 [keepalived-1.2.19] cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
 [keepalived-1.2.19] cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
 [keepalived-1.2.19] cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
 [keepalived-1.2.19] mkdir /etc/keepalived
 [keepalived-1.2.19] cp /usr/local/keepalived/etc/keepalived/keepalived.conf/etc/keepalived/
 [keepalived-1.2.19] chkconfig --add keepalived
 [keepalived-1.2.19] chkconfig keepalived on

 重启\启动\关闭\查看状态keepalived

service keepalived restart

service keepalived start

service keepalived stop

service keepalived status

2.3 配置keepalived.conf文件

 [~]mv /etc/keepalived/keepalived.conf/etc/keepalived/keepalived.conf.bak
 [~]vi /etc/keepalived//keepalived.conf

主备机的keepalived.conf文件大致相同，只是注意红色标注的地方。

主机配置：

 global_defs {
    router_id NodeA
 }
 vrrp_script chk_http_port {
    script "/opt/tomcat.pid"
    interval 5
    weight 2
 }
 vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 52
    priority 150
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass 1111
     }

    track_script {
        chk_http_port
     }

    virtual_ipaddress {
        10.10.195.212
     }
 }

备机配置：

 global_defs {
    router_id NodeB
 }
 vrrp_script chk_http_port {
    script "/opt/tomcat.pid"
    interval 5
    weight 2
 }
 vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 52
    priority 100
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass 1111
     }

    track_script {
        chk_http_port
     }

    virtual_ipaddress {
        10.10.195.212
     }
 }

2.4配置/opt/tomcat.pid

 #!/bin/bash
 #description: check tomcat service anddecide whether stop the keepalived or not
 #edited by zzh: 2015-10-14

 CATALINA_HOME=/users/shr/apache-tomcat-7.0.64
 JAVA_HOME=/users/shr/util/JavaDir/jdk
 export CATALINA_HOME
 export JAVA_HOME

 ps ax --width=1000 | grep"org.apache.catalina.startup.Bootstrap start" | grep -v"grep" | awk '{printf $1 " "}' | wc | awk '{print $2}' >tomcat_process_count.txt
 read line < tomcat_process_count.txt

 start_tomcat=$CATALINA_HOME/bin/startup.sh

 if [ ${line} -lt 1 ]
 then
        echo -n "===Starting tomcat===:"
        ${start_tomcat}
        # :sudo service tomcat start
        echo "===tomcat start ok.==="
        sleep 3

        # check the tomcat status.
        ps ax --width=1000 | grep "org.apache.catalina.startup.Bootstrapstart" | grep -v "grep" | awk '{printf $1 " "}' | wc |awk '{print $2}' > tomcat_process_count.txt
        read line2 < tomcat_process_count.txt
        if [ ${line2} -lt 1 ]
        then
                 sudo service keepalived stop
        fi
 fi
 rm tomcat_process_count.txt
 #shell end.

3. 日志查看

3.1 正常启动

输入tail -f /var/log/messages查看启动keepalived日志：

（MASTER：10.10.195.53）

 Sep 29 15:49:16 shr Keepalived[5536]:Starting Keepalived v1.2.19 (09/21,2015)
 Sep 29 15:49:16 shrKeepalived_healthcheckers[5538]: Netlink reflector reports IP 10.10.195.53added
 Sep 29 15:49:16 shrKeepalived_healthcheckers[5538]: Netlink reflector reports IP 10.10.195.53added
 Sep 29 15:49:16 shrKeepalived_healthcheckers[5538]: Registering Kernel netlink reflector
 Sep 29 15:49:16 shrKeepalived_healthcheckers[5538]: Registering Kernel netlink command channel
 Sep 29 15:49:16 shrKeepalived_healthcheckers[5538]: Opening file'/etc/keepalived/keepalived.conf'.
 Sep 29 15:49:16 shrKeepalived_healthcheckers[5538]: Configuration is using : 6572 Bytes
 Sep 29 15:49:16 shr Keepalived[5537]:Starting Healthcheck child process, pid=5538
 Sep 29 15:49:16 shr Keepalived_healthcheckers[5538]:Using LinkWatch kernel netlink reflector...
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Netlink reflector reports IP 10.10.195.53 added
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Netlink reflector reports IP 10.10.195.53 added
 Sep 29 15:49:16 shr Keepalived[5537]:Starting VRRP child process, pid=5539
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Registering Kernel netlink reflector
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Registering Kernel netlink command channel
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Registering gratuitous ARP shared channel
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Opening file '/etc/keepalived/keepalived.conf'.
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Configuration is using : 36541 Bytes
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:Using LinkWatch kernel netlink reflector...
 Sep 29 15:49:16 shr Keepalived_vrrp[5539]:VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
 Sep 29 15:49:17 shr Keepalived_vrrp[5539]:VRRP_Instance(VI_1) Transition to MASTER STATE
 Sep 29 15:49:17 shr Keepalived_vrrp[5539]:VRRP_Instance(VI_1) Received lower prio advert, forcing new election
 Sep 29 15:49:18 shr Keepalived_vrrp[5539]:VRRP_Instance(VI_1) Entering MASTER STATE
 Sep 29 15:49:18 shr Keepalived_vrrp[5539]:VRRP_Instance(VI_1) setting protocol VIPs.
 Sep 29 15:49:18 shr Keepalived_vrrp[5539]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.10.195.212
 Sep 29 15:49:18 shr Keepalived_vrrp[5539]:Netlink reflector reports IP 10.10.195.212 added
 Sep 29 15:49:18 shrKeepalived_healthcheckers[5538]: Netlink reflector reports IP 10.10.195.212added
 Sep 29 15:49:23 shr Keepalived_vrrp[5539]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.10.195.212

(BACKUP:10.10.195.190)

 Sep 29 15:46:25 server1 Keepalived[18218]:Starting Keepalived v1.2.19 (09/25,2015)
 Sep 29 15:46:25 server1Keepalived_healthcheckers[18220]: Netlink reflector reports IP 10.10.195.190added
 Sep 29 15:46:25 server1Keepalived_healthcheckers[18220]: Netlink reflector reports IP 10.10.195.190added
 Sep 29 15:46:25 server1Keepalived_healthcheckers[18220]: Registering Kernel netlink reflector
 Sep 29 15:46:25 server1Keepalived_healthcheckers[18220]: Registering Kernel netlink command channel
 Sep 29 15:46:25 server1 Keepalived_healthcheckers[18220]:Opening file '/etc/keepalived/keepalived.conf'.
 Sep 29 15:46:25 server1Keepalived_healthcheckers[18220]: Configuration is using : 6682 Bytes
 Sep 29 15:46:25 server1 Keepalived[18219]:Starting Healthcheck child process, pid=18220
 Sep 29 15:46:25 server1 Keepalived[18219]:Starting VRRP child process, pid=18221
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: Netlink reflector reports IP 10.10.195.190 added
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: Netlink reflector reports IP 10.10.195.190 added
 Sep 29 15:46:25 server1Keepalived_healthcheckers[18220]: Using LinkWatch kernel netlink reflector...
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: Registering Kernel netlink reflector
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: Registering Kernel netlink command channel
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: Registering gratuitous ARP shared channel
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: Opening file '/etc/keepalived/keepalived.conf'.
 Sep 29 15:46:25 server1 Keepalived_vrrp[18221]:Configuration is using : 36651 Bytes
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: Using LinkWatch kernel netlink reflector...
 Sep 29 15:46:25 server1Keepalived_vrrp[18221]: VRRP_Instance(VI_1) Entering BACKUP STATE
 Sep 29 15:46:25 server1 Keepalived_vrrp[18221]:VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]

3.2 主备切换

l  当在主机（MASTER：10.10.195.53）中输入 sudoservice keepalived stop，此时就会进行主备切换，主机切换成备机。

主机（MASTER：10.10.195.53）输出如下

 Oct 14 13:25:09 shr Keepalived_vrrp[26683]:VRRP_Instance(VI_1) sending 0 priority
 Oct 14 13:25:09 shr Keepalived_vrrp[26683]:VRRP_Instance(VI_1) removing protocol VIPs.
 Oct 14 13:25:09 shrKeepalived_healthcheckers[26682]: Netlink reflector reports IP 10.10.195.212removed
 Oct 14 13:25:09 shr Keepalived[26681]:Stopping Keepalived v1.2.19 (09/21,2015)

备机(BACKUP:10.10.195.190)输出如下

 Oct 14 13:19:58 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) Transition to MASTER STATE
 Oct 14 13:19:59 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) Entering MASTER STATE
 Oct 14 13:19:59 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) setting protocol VIPs.
 Oct 14 13:19:59 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for10.10.195.212
 Oct 14 13:19:59 server1 Keepalived_healthcheckers[30889]:Netlink reflector reports IP 10.10.195.212 added
 Oct 14 13:19:59 server1Keepalived_vrrp[30890]: Netlink reflector reports IP 10.10.195.212 added
 Oct 14 13:20:04 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for10.10.195.212

l  当在主机（MASTER：10.10.195.53）中输入sudoservice keepalived start,此时就会切换成主机。备机(BACKUP:10.10.195.190)输出如下信息：

 Oct 14 13:25:11 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) Received higher prio advert
 Oct 14 13:25:11 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) Entering BACKUP STATE
 Oct 14 13:25:11 server1Keepalived_vrrp[30890]: VRRP_Instance(VI_1) removing protocol VIPs.
 Oct 14 13:25:11 server1Keepalived_healthcheckers[30889]: Netlink reflector reports IP 10.10.195.212removed
 Oct 14 13:25:11 server1Keepalived_vrrp[30890]: Netlink reflector reports IP 10.10.195.212 removed

4. 查看虚拟ip

可以通过ip add show命令查看添加的虚拟ip：

 [shr@shr bin]$ip add show
 1: lo:  mtu16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
 2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:6b:f2:a8 brd ff:ff:ff:ff:ff:ff
    inet 10.10.195.53/24 brd 10.10.195.255 scope global eth0
    inet 10.10.195.212/32scope global eth0

5. 常见错误

5. 1常见错误一：

 /var/log/messages has thousands of errorslike this:

 Jun 28 09:18:32 rust Keepalived_vrrp:receive an invalid ip number count
 associated with VRID!
 Jun 28 09:18:32 rust Keepalived_vrrp: bogusVRRP packet received on eth0 !!!
 Jun 28 09:18:32 rust Keepalived_vrrp:VRRP_Instance(VI_1) Dropping received
 VRRP packet...

 The backup director starts up, but doesn'tlisten on the virtual addresses
 at all. Its /var/log/messages has thousands of errors like this:

 Jun 28 06:25:05 stye Keepalived_vrrp:receive an invalid ip number count
 associated with VRID!
 Jun 28 06:25:05 stye Keepalived_vrrp: bogusVRRP packet received on eth0 !!!
 Jun 28 06:25:05 stye Keepalived_vrrp:VRRP_Instance(VI_1) ignoring received
 advertisment...

解决方法：

改变配置文件/etc/keepalived/keepalived.conf中virtual_router_id为另一个值即可。

（changed the vrid to another number and it worked fine）.

5.2 常见错误二：

主机执行到VRRP_Instance(VI_1) Entering BACKUP STATE之后没有执行VRRP_Instance(VI_1)setting protocol VIPs。

解决方法：

可能是配置文件keepalived.conf中{左边没有空格。

 

附

linux添加tomcat服务

在/etc/init.d中输入 sudo vim tomcat

脚本具体内容如下：

 #!/bin/bash
 #chkconfig: 2345 10 90
 #description: Starts and Stops the tomcatdaemon
 #edited by ZZH: 2015-10-14

 CATALINA_HOME=/users/shr/apache-tomcat-7.0.64
 JAVA_HOME=/users/shr/util/JavaDir/jdk
 export CATALINA_HOME
 export JAVA_HOME

 start_tomcat=$CATALINA_HOME/bin/startup.sh
 stop_tomcat=$CATALINA_HOME/bin/shutdown.sh

 if [ ! -f $CATALINA_HOME/bin/catalina.sh ]
 then
          echo"===Tomcat is not available.==="
          exit
 fi

 start()
 {
          echo-n "===Starting tomcat===:"
          ${start_tomcat}
          echo"===tomcat start ok.==="
 }

 stop()
 {
          echo-n "===Shutting down tomcat===:"
          ${stop_tomcat}
          echo"===tomcat stop ok.==="
 }

 status()
 {
          ps ax --width=1000 | grep"org.apache.catalina.startup.Bootstrap start" | grep -v"grep" | awk '{printf $1 " "}' | wc | awk '{print $2}'> tomcat_process_count.txt
          readline < tomcat_process_count.txt
          rmtomcat_process_count.txt
          if[ $line -gt 0 ]
                    then
                             echo-n "tomcat ( pid = "
                             ps ax --width=1000 | grep"org.apache.catalina.startup.Bootstrap start" | grep -v"grep" | awk '{printf $1 " "}' | awk '{print $1}'> tomcat_process_pid.txt
                             readpid < tomcat_process_pid.txt
                             rmtomcat_process_pid.txt
                             echo-n $pid
                             echo-n ") is running..."
                             echo
                    else
                             echo"tomcat is stopped"
          fi
 }

 case "$1" in
          start)
                    start
                    ;;
          stop)
                    stop
                    ;;
          restart)
                    stop
                    sleep1
                    start
                    ;;
          status)
                    status
                    ;;
          *)
                    echo"Usage:$0 {start|stop|restart}"
                    exit1
 esac

 exit 0
 # shell end.

在终端输入:

 sudo chmod 755 tomcat
 sudo chkconfig --add tomcat

tomcat 启动 service tomcatstart

tomcat 关闭 service tomcatstop

tomcat 重启 service tomcatrestart

tomcat状态查看 service tomcatstatus