【APP & Web架构】企业web高可用集群实战之apache整合tomcat实现tomcat负载均衡集群+session

一、环境：

系统最小化安装，系统初使化脚本：

http://linuxops.blog.51cto.com/2238445/841849

系统环境：centos 5.8 x86_64

应用软件：所以应用软件都放在/usr/local/src目录下

 httpd-2.2.22.tar.gz

         apache-tomcat-7.0.26.tar.gz

 jdk-7u4-linux-x64.gz

 tomcat-connectors-1.2.37-src.tar.gz

所需服务器：

         server1:192.168.8.222   -->安装httpd、apache-tomcat、jdk、tomcat-connectors

 server2:192.168.8.223   -->安装apache-tomcat、jdk

二、安装apache

cd /usr/local/src/
tar zxf httpd-2.2.22.tar.gz
cd httpd-2.2.22
./configure --prefix=/usr/local/apache2 --enable-modules=so --enable-so --enable-cache --enable-mem-cache --enable-rewrite --enable-ssl --with-mpm=prefork
make;make install
cd ..

三、安装tomcat、jdk

1.安装tomcat
tar zxf apache-tomcat-7.0.26.tar.gz
mv apache-tomcat-7.0.26 /usr/local/tomcat
2.安装jdk
tar zxf jdk-7u4-linux-x64.gz
mv jdk1.7.0_04 /usr/local/jdk
3.设置java、tomcat运行环境变量
vi /etc/profile
在文件最后加入如下内容：
#Set java & tomcat Environment
JAVA_HOME=/usr/local/jdk
export JAVA_HOME
JRE_HOME=/usr/local/jdk/jre
export JRE_HOME
CLASSPATH=/usr=/usr/local/tomcat/common/lib/:/usr/local/jdk/lib:/usr/local/jdk/jre/lib
export CLASSPATH
PATH=$PATH:/usr/local/tomcat/bin/:/usr/local/apache/bin:/usr/local/jdk/bin:/usr/local/jdk/jre/bin
export PATH
TOMCAT_HOME=/usr/local/tomcat
export TOMCAT_HOME
让环境变量立即生效：
source /etc/profile

四、安装jk,整合apache和tomcat

1.安装jk
tar zxf tomcat-connectors-1.2.37-src.tar.gz
cd tomcat-connectors-1.2.37-src/native
./configure --with-apxs=/usr/local/apache2/bin/apxs --with-java-home=/usr/local/jdk
make;make install
检测模块有没有生成
通过上面几步，会在/usr/local/apache2/modules/目录下产生mod_jk.so 这个模块文件
2.整合apache和tomcat
1.)修改apache配置文件/usr/local/apache2/conf/httpd.conf ,加入：Include conf/mod_jk.conf
echo "Include conf/mod_jk.conf" >> /usr/local/apache2/conf/httpd.conf
2.)创建mod_jk.conf配置文件
vi /usr/local/apache2/conf/mod_jk.conf
加入如下内容：
# Load mod_jk module
LoadModule jk_module modules/mod_jk.so
# Where to find workers.properties
JkWorkersFile conf/workers.properties
# Where to put jk logs
JkLogFile logs/mod_jk.log
JkShmFile logs/mod_jk.shm
# Set the jk log level [debug/error/info]
JkLogLevel info
# Select the log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
# JkOptions indicate to send SSL KEY SIZE,
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
# JkRequestLogFormat set the request format
JkRequestLogFormat "%w %V %T"
# Send JSPs for context /examples to worker named ajp13
JkMount /* controller
3.配置JK调度器
创建JK配置文件
cat /usr/local/apache2/conf/workers.properties
#
# workers.properties conf
#

# list the workers by name
worker.list=controller      #定义负载均衡名称
worker.controller.type=lb   #定义负载均衡类型，这里为lb
worker.controller.error_escalation_time=0   #切换错误状态时间
worker.controller.max_reply_timeouts=10

# -------- server 1  ---------
worker.jvm1.reference=worker.template     #jvm1引用worker.template模板
worker.jvm1.port=8009       #jvm1的ajp端口
worker.jvm1.host=192.168.8.222  #jvm1的的IP
worker.jvm1.lbfactor = 5

# -------- server 2  ---------
worker.jvm2.reference=worker.template
worker.jvm2.port=8009
worker.jvm2.host=192.168.8.223
worker.jvm2.lbfactor = 1
worker.jvm2.activation=A
worker.template.type=ajp13
worker.template.socket_connect_timeout=5000 #套接字连接超时为5000毫秒，默认为0
worker.template.socket_keepalive=true       #开启保持为真，默认为false
worker.template.ping_mode=A         #使用cping/cpong模式,有C,P,I,A四个模式,A表示选择所以模式
worker.template.ping_timeout=10000      #探测cping/cpong超时间默认10000毫秒
worker.template.connection_pool_minsize=0   #连接池关闭最小时间
worker.template.connection_pool_timeout=600 #连接池600秒
worker.template.reply_timeout=300000        #答复超时300000毫秒
worker.template.recovery_options=3      #重试次数
worker.controller.balance_workers=jvm1,jvm2 #负载均衡的客户名称
worker.controller.sticky_session=0
worker.controller.sticky_session_force=0
#worker.balancer.sticky_session=1

PS: 这里worker.jvm1.port=8009的端口设置为tomcat server.xml配置文件中ajp13的默认端口！也可以自己定义，两边必须相同！

worker.controller.sticky_session=0

worker.controller.sticky_session_force=0

这是实现session复制的关键！

把worker.balancer.sticky_session=1 设置为1貌似不成，索性用上面的方法OK！

五、tomcat集群和session复制配置

1.server1上配置
1.)在/usr/local/tomcat/conf/server.xml 中修改Enginx节点信息，大概在103行：
把<Engine name="Catalina" defaultHost="localhost">
修改成：
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
2.)加入Cluster节点信息，大概在110行左右处加入如下内容：
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
           expireSessionsOnShutdown="false"
           notifyListenersOnReplication="true"/>
  <Channel className="org.apache.catalina.tribes.group.GroupChannel">
    <Membership className="org.apache.catalina.tribes.membership.McastService"
                address="228.0.0.4"
                port="45564"
                frequency="500"
                dropTime="3000"/>
    <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
              address="192.168.8.222"
              port="4000"
              autoBind="100"
              selectorTimeout="5000"
              maxThreads="6"/>
    <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
      <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
    </Sender>
    <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
    <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
  </Channel>
  <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
         filter=""/>
  <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
  <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
            tempDir="/tmp/war-temp/"
            deployDir="/tmp/war-deploy/"
            watchDir="/tmp/war-listen/"
            watchEnabled="false"/>
  <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
  <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>

2.server2上配置

1.)在/usr/local/tomcat/conf/server.xml 中修改Enginx节点信息，大概在103行：
把<Engine name="Catalina" defaultHost="localhost">
修改成：
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm2">
2.)加入Cluster节点信息，大概在110行左右处加入如下内容：
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
           expireSessionsOnShutdown="false"
           notifyListenersOnReplication="true"/>
  <Channel className="org.apache.catalina.tribes.group.GroupChannel">
    <Membership className="org.apache.catalina.tribes.membership.McastService"
                address="228.0.0.4"
                port="45564"
                frequency="500"
                dropTime="3000"/>
    <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
              address="192.168.8.223"
              port="4000"
              autoBind="100"
              selectorTimeout="5000"
              maxThreads="6"/>
    <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
      <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
    </Sender>
    <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
    <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
  </Channel>
  <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
         filter=""/>
  <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
  <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
            tempDir="/tmp/war-temp/"
            deployDir="/tmp/war-deploy/"
            watchDir="/tmp/war-listen/"
            watchEnabled="false"/>
  <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
  <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
3.修改web站点文件存放主目录路径
1.)apache配置文件要修改的地方
把下面两处修改成：
DocumentRoot "/data/www/"    #大概在104行
<Directory "/data/www">      #大概在131行
2.)tomcat配置文件修改成：
   <Host name="localhost"  appBase="/data/www"
            unpackWARs="true" autoDeploy="true">
<Context path="/" docBase="" debug="0" reloadable="true" />

4.建立测试目录

mkdir -p /data/www/
cat /data/www/test.jsp
<%
 System.out.println("Welcome tomcat_cluster session repication");
%>
在/data/www目录下创建admin目录，再在admin目录下创建WEB-INF目录
mkdir -p /data/www/admin/WEB-INF
cd /data/www/admin/WEB-INF
vi web.xml
加入如下内容：
<web-app xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.co
m/xml/ns/j2ee/web-app_2_4.xsd" version="2.4">
<display-name>TomcatDemo</display-name>
<distributable/>
</web-app>

PS: <distributable/>这个必须加进去，不然也实现不了复制！

在admin目录下创建测试代码：

cat /data/www/admin/index.jsp
<%@ page contentType="text/html; charset=GBK" %>
<%@ page import="java.util.*" %>
<html><head><title>Cluster App Test</title></head>
<body>
Server Info:
<%
out.println(request.getLocalAddr() + " : " + request.getLocalPort()+"<br>");%>
<%
out.println("<br> ID " + session.getId()+"<br>");
// 如果有新的 Session 属性设置
String dataName = request.getParameter("dataName");
if (dataName != null && dataName.length() > 0) {
String dataValue = request.getParameter("dataValue");
session.setAttribute(dataName, dataValue);
}
out.print("<b>Session 列表</b>");
Enumeration e = session.getAttributeNames();
while (e.hasMoreElements()) {
String name = (String)e.nextElement();
String value = session.getAttribute(name).toString();
out.println( name + " = " + value+"<br>");
System.out.println( name + " = " + value);
}
%>
<form action="index.jsp" method="POST">
名称:<input type=text size=20 name="dataName">
<br>
值:<input type=text size=20 name="dataValue">
<br>
<input type=submit>
</form>
</body>
</html>

PS: 以上代码来源于网上！

浏览器测试结果：

http://192.168.8.222/admin/

再回到服务器上查看日志：

223服务器：

222服务器：

1.这里ID必须相同，且jvm1和jvm2来回切换！

2.ID不同，jvm1和jvm2来回切换只是实现了负载均衡！

后面的文章介绍: apache+tomcat集群用memcached保持session

               apache+tomcat集群用apache的 proxy-ajp和proxy-balance模块保持session

后面陆续介绍nginx+tomcat集群！欢迎大家来逛逛！！哈哈。。。。