linux+nginx+tomcat负载均衡，实现session同步

一、Nginx简介

Nginx ("engine x") 是一个高性能的HTTP和反向代理服务器，也是一个IMAP/POP3/SMTP 代理服务器 。Nginx 是由 Igor Sysoev 为俄罗斯访问量第二的Rambler.ru 站点开发的，它已经在该站点运行超过四年多了。Igor 将源代码以类BSD许可证的形式发布。自Nginx 发布四年来，Nginx 已经因为它的稳定性、丰富的功能集、 示例配置文件和低系统资源的消耗而闻名了。目前国内各大门户网站已经部署了Nginx，如新浪、网易、腾讯等；国内几个重要的视频分享网站也部署了Nginx，如六房间、酷6等。 新近发现Nginx 技术在国内日趋火热，越来越多的网站开始部署Nginx。

软件及环境：

虚拟机（IP：172.16.16.174）上安装redhat enterprise server linux5.5 OS

在虚拟机上安装nginx、Tomcat1和Tomcat2

二、JDK的安装

1、jdk-7-linux-i586.tar.gz解压

#tar zxvf jdk-7-linux-i586.tar.gz

#mkdir /usr/java

#mv jdk1.7.0 /usr/java

2、设置环境变量

#vi /etc/profile

export JAVA_HOME=/usr/local/jdk1.7.0

export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar

export PATH=$JAVA_HOME/bin:$PATH

保存退出

:wq!

3、使环境变量生效

#source /etc/profile

 

4、测试JDK

#java –version

三、TOMCAT的安装

1、tomcat安装目录

将TOMCAT安装在/usr/local/tomcat6

2、设置环境变量

#vi /etc/profile

export TOMCAT_HOME=/usr/local/tomcat6

export CATALINA_HOME=/usr/local/tomcat6

export CATALINA_BASE=/usr/local/tomcat6

保存退出

:wq!

3、使环境变量生效

#source /etc/profile

4、修改TOMCAT配置文件server.xml

修改TOMCAT配置文件/usr/local/tomcat6/conf/server.xml

将以下内容

<Host name="localhost" appBase="webapps"

.../>

修改为

<Host name="localhost" appBase="/usr/local/www/web"

.../>

5、启动TOMCAT

/usr/local/tomcat6/bin/startup.sh

6、关闭TOMCAT

/usr/local/tomcat6/bin/shutdown.sh

 

四、同一linux系统下安装多个TOMCAT

1、tomcat安装目录

将TOMCAT安装在/usr/local/tomcat6_2

2、设置环境变量

#vi /etc/profile

export TOMCAT_2_HOME=/usr/local/tomcat6_2

export CATALINA_2_HOME=/usr/local/tomcat6_2

export CATALINA_2_BASE=/usr/local/tomcat6_2

保存退出

:wq!

3、使环境变量生效

#source /etc/profile

4、修改TOMCAT配置文件server.xml

修改TOMCAT配置文件/usr/local/tomcat6_2/conf/server.xml

将以下内容

<Server port="8005" shutdown="SHUTDOWN">...

<Connector port="8080" protocol="HTTP/1.1" ...

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

<Host name="localhost" appBase="webapps"

.../>

修改为

<Server port="9005" shutdown="SHUTDOWN">...

<Connector port="9090" protocol="HTTP/1.1" ...

<Connector port="9009" protocol="AJP/1.3" redirectPort="8443" />

<Host name="localhost" appBase="/usr/local/www/web"

.../>

5、修改TOMCAT配置启动文件startup.sh

修改TOMCAT配置启动文件/usr/local/tomcat6_2/bin/startup.sh

在exec上添加以下内容

export JAVA_HOME=/usr/java/jdk1.7.0

export CLASSPATH=$JAVA_HOME/lib

export PATH=$JAVA_HOME/bin

export CATALINA_HOME=$CATALINA_2_HOME

export CATALINA_BASE=$CATALINA_2_BASE

6、修改TOMCAT配置启动文件shutdown.sh

修改TOMCAT配置启动文件/usr/local/tomcat6_2/bin/shutdown.sh

在exec上添加以下内容

export JAVA_HOME=/usr/java/jdk1.7.0

export CLASSPATH=$JAVA_HOME/lib

export PATH=$JAVA_HOME/bin

export CATALINA_HOME=$CATALINA_2_HOME

export CATALINA_BASE=$CATALINA_2_BASE

 

7、启动TOMCAT

/usr/local/tomcat6_2/bin/startup.sh

8、关闭TOMCAT

/usr/local/tomcat6_2/bin/shutdown.sh

五、Nginx 0.8.46的安装

1、安装Nginx所需的pcre库

tar zxvf pcre-8.10.tar.gz

cd pcre-8.10/

./configure

make && make install

cd ../

安装pcre成功后，继续安装nginx

2、安装Nginx

1、上传nginx-0.7.63.tar.gz 至/usr/local

2、执行如下命令解压nginx：

tar zxvf nginx-0.8.46.tar.gz

cd nginx-0.8.46/

1.#cd /usr/local

2.#tar zxvf nginx-0.7.63.tar.gz

3、编译安装nginx

1.#cd nginx-0.7.63

2.#./configure --user=wcity --group=wcity --prefix=/usr/local/nginx--with-http_stub_status_module --with-http_ssl_module

（user改为你安装的用户、group改为你安装的组）

#启动server 状态页和https模块

make && make install

cd ../

3、创建Nginx日志目录

mkdir -p /data/logs

chmod +w /data/logs

chown -R wcity:wcity /data/logs

4、创建并修改Nginx配置文件nginx.conf

①、在/usr/local//nginx/conf/目录中创建nginx.conf文件：

vi /usr/local/nginx/conf/nginx.conf

修改nginx.conf配置文件如下：

user wcity wcity;

worker_processes 4;

worker_rlimit_nofile 65535;

error_log /usr/local/nginx/logs/error.log crit;

pid /usr/local/nginx/nginx.pid;

 

events {

    use epoll;

    worker_connections 65535;

}

 

http {

    include       mime.types;

    default_type  application/octet-stream;

    server_names_hash_bucket_size 128;

    client_header_buffer_size 32k;

    large_client_header_buffers 4 32k;

    client_max_body_size 8m;

    sendfile on;

    tcp_nopush     on;

    keepalive_timeout 60;

    tcp_nodelay on;

    fastcgi_connect_timeout 300;

    fastcgi_send_timeout 300;

    fastcgi_read_timeout 300;

    fastcgi_buffer_size 64k;

    fastcgi_buffers 4 64k;

    fastcgi_busy_buffers_size 128k;

    fastcgi_temp_file_write_size 128k;

    fastcgi_intercept_errors on;

    gzip on;

    gzip_min_length  1k;

    gzip_buffers     4 16k;

    gzip_http_version 1.0;

    gzip_comp_level 2;

    gzip_types       text/plain application/x-javascript text/css application/xml;

    gzip_vary on;

 

upstream  http://10.46.51.159  {

         server   10.46.51.159:8080 weight=5;

         server   10.46.51.159:8081 weight=5;

  }

 

location ~ \.(jsp|jspx|do|htm)?$ {

          proxy_set_header  Host $host; 

          proxy_set_header  X-Real-IP  $remote_addr;

          proxy_pass http:// 10.46.51.159;  #转向tomcat处理      

         }

 

         location ~ (/dwr/)? {

          proxy_set_header  Host $host;

          proxy_set_header  X-Real-IP  $remote_addr;

          proxy_pass http:// 10.46.51.159;  #转向tomcat处理

         }

 

5、检查配置文件nginx.conf是否正确

修改/usr/local/nginx/conf/nginx.conf配置文件后，请执行以下命令检查配置文件是否正确：

1.#/usr/local/nginx/sbin/nginx -t

如果屏幕显示以下两行信息，说明配置文件正确：

the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok

the configuration file /usr/local/nginx/conf/nginx.conf was tested successfully

如果提示unknown host，则可在服务器上执行：ping www.baidu.com 如果也是同样提示unknown host则有两种可能：

a、服务器没有设置DNS服务器地址，查看/etc/resolv.conf下是否设置，若无则加上

b、防火墙拦截

6、启动nginx的命令

1.#/usr/local/nginx/sbin/nginx

这时，输入以下命令查看Nginx 主进程号：

1.ps -ef | grep "nginx: master process" | grep -v "grep" | awk -F ' ' '{print $2}'

7、停止nginx的命令

1.#/usr/local/nginx/sbin/nginx -s stop

8、在不停止Nginx服务的情况下平滑变更Nginx配置

a、修改/usr/local/nginx/conf/nginx.conf配置文件后，请执行以下命令检查配置文件是否正确：

1./usr/local/nginx/sbin/nginx -t

如果屏幕显示以下两行信息，说明配置文件正确：

1.the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok

2.the configuration file /usr/local/nginx/conf/nginx.conf was tested successfully

b、这时，输入以下命令查看Nginx 主进程号：

1.ps -ef | grep "nginx: master process" | grep -v "grep" | awk -F ' ' '{print $2}'

屏幕显示的即为Nginx 主进程号，例如：

6302

这时，执行以下命令即可使修改过的Nginx配置文件生效：

1.kill -HUP 6302

或者无需这么麻烦，找到Nginx 的Pid文件：

1.kill -HUP `cat /usr/local/nginx/nginx.pid`

9、nginx启动好后启动tomcat

此时输入http://主机ip地址即可看到“My web!”

10、配置开机自动启动Nginx

vi /etc/rc.local

在末尾增加以下内容：

/usr/local//nginx/sbin/nginx

---

六、Tomcat集群配置

1、分别打开tomcat1和tomcat2下conf中server.xml

<Server port="8005" shutdown="SHUTDOWN">  

<!-- Define a non-SSL HTTP/1.1 Connector on port 8080 -->

<Connector port="8080" maxHttpHeaderSize="8192"

   maxThreads="150" minSpareThreads="25" maxSpareThreads="75"

   enableLookups="false" redirectPort="8443" acceptCount="100"

   connectionTimeout="20000" disableUploadTimeout="true" />

<!-- Define an AJP 1.3 Connector on port 8009 -->

<Connector port="8009" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />

 

找到<Engine name="Catalina" defaultHost="localhost">
Tomcat1下的修改为<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">

Tomcat2下的修改为<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2">

2、修改tomcat1中的Cluster className

tomcat1找到  

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
修改为 

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" 
            channelSendOptions="6">

            <Manager className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>
            <!--
            <Manager className="org.apache.catalina.ha.session.BackupManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"
                mapSendOptions="6"/>
            -->
            <Channel className="org.apache.catalina.tribes.group.GroupChannel">
                <Membership className="org.apache.catalina.tribes.membership.McastService" 
                    address="228.0.0.5" 
                    bind="192.168.0.51" 
                    port="45564" 
                    frequency="500" 
                    dropTime="3000"/>
                <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" 
                    address="192.168.0.51" 
                    autoBind="100" 
                    port="4001" 
                    selectorTimeout="100" 
                    maxThreads="6"/>
                <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                    <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                </Sender>
                <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
            </Channel>
            <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;"/>
            
            <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
            <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>

 

 

3、修改tomcat2中的Cluster className

tomcat2找到
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
修改为 

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" 
            channelSendOptions="6">

            <Manager className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>
            <!--
            <Manager className="org.apache.catalina.ha.session.BackupManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"
                mapSendOptions="6"/>
            -->
            <Channel className="org.apache.catalina.tribes.group.GroupChannel">
                <Membership className="org.apache.catalina.tribes.membership.McastService" 
                    address="228.0.0.5" 
                    bind="192.168.0.50" 
                    port="45564" 
                    frequency="500" 
                    dropTime="3000"/>
                <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" 
                    address="192.168.0.50" 
                    autoBind="100" 
                    port="4000" 
                    selectorTimeout="100" 
                    maxThreads="6"/>
                <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                    <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                </Sender>
                <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
            </Channel>
            <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;"/>
            
            <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
            <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>

 

还需要在程序的web.xml里面</web-app>前面加入以下这句话
<distributable/> 

session同步到这里设置完毕。

服务器的启动顺序如下: 

tomcat1 --> |tomcat2 --> |nginx 

先启动tocmat1,等tomcat1启动完全的时候再启动tomcat2，等两个tocmat全启动之后，再启动nginx负载的时候可以用APACHE或NGINX，如果什么都不用，两台一接防问了话，可能会出错极session不同步的问题，网上也很多人介意不要做seeeion同步，这样会降低机器的性能，有一个方法可以，就是NGINX的ip_hash,这样至少可以保证客户端去防问同一台TOCMAT，除非那台TOCMAT挂了