nginx和httpd反向代理和负载均衡tomcat集群

实验准备：

• 虚拟机172.18.250.77 安装nginx和httpd 实现负载均衡和反向代理
• 虚拟机172.18.250.78 安装tomcat
• 虚拟机172.18.250.79 安装tomcat

nginx反向代理和负载均衡

安装nginx和tomcat

]# rpm -ivh nginx-1.8.0-1.el6.ngx.x86_64.rpm 
]# ls
apache-tomcat-7.0.55.tar.gz  jdk-7u79-linux-x64.rpm 
]# tar -xf apache-tomcat-7.0.55.tar.gz -C /usr/local/     //解压二进制编译包
]# cd /usr/local/
]# ln -s apache-tomcat-7.0.55/ tomcat                     //软链接
]# rpm -ivh jdk-7u79-linux-x64.rpm                        //安装官方的二进制jdk包
 
系统找java时是根据JAVA_HOME这个系统变量来寻找的,所以导出这个变量：
]# vim /etc/profile.d/java.sh
/usr/local/tomcat
JAVA_HOME=/usr/java/latest
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH
 
]# vim /etc/profile.d/tomcat.sh            //导出tomcat路径到环境变量中
CATALINA_HOME=/usr/local/tomcat
PATH=$CATALINA_HOME/bin:$PATH
export CATALINA_BASE PATH
 
]# . /etc/profile.d/java.sh 
]# . /etc/profile.d/tomcat.sh 
]# echo $JAVA_HOME
/usr/bin/
]# echo $CATALINA_HOME
/usr/local/tomcat

配置nginx和tomcat，在250.78和250.79上

]# vim /etc/nginx/conf.d/default.conf
 location / {
        root   /usr/share/nginx/html;
        proxy_pass   http://localhost:8080/ //所有请求nginx80端口的都反代到tomcat8080上 
        index  index.html index.htm;
    }
     
]# cd /usr/local/tomcat/webapps/          
]# mkdir myapp                                 //创建测试目录
]# cd myapp/
]# mkdir classes lib WEB-INF META-INF
]# vim index.jsp                               //提供测试页
<%@ page language="java" %>
<%@ page import="java.util.*" %>
  
    
     
    
    
        <% out.println("Hello World from 250.78");
        %>
         
  
   
]# nginx                                       //启动tomcat和nginx
]# catalina.sh start
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:        /usr/java/latest
Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started. 

然后直接可以用浏览器访问看是否可以访问。

配置nginx负载均衡

]# vim /etc/nginx/nginx.conf         //在250.77上配置
upstream webserver {
      server 172.18.250.78:80;
      server 172.18.250.79:80;
    }
]# vim /etc/nginx/conf.d/default.conf
     location / {
        root   /usr/share/nginx/html;
        proxy_pass http://webserver/;
        index  index.html index.htm;
    }    
]# nginx  
]# curl  http://172.18.250.77/myapp/index.jsp 
        
            
            Hello World from 250.78                                                                            
]# curl   http://172.18.250.77/myapp/index.jsp   //在服务器上curl请求，实现了负载均衡    
        
            
            Hello World from 250.79                                     

基于session sticky实现会话保持

]# vim /etc/nginx/nginx.conf 
upstream webserver {
      server 172.18.250.78:80;
      server 172.18.250.79:80;
      ip_hash; 
    }

源地址hash后，多次刷新，都是反代到相同的一台服务器上，保持了会话

httpd的反向代理

]# yum -y install httpd           //三台服务器全部装上httpd
]# nginx -s stop                  //停止nginx

配置httpd,实现反向代理：

]# httpd -M //httpd的反代有三种方式
proxy_ajp_module (shared) //ajp协议的反代模块
proxy_balancer_module (shared) //负载均衡模块
proxy_http_module (shared) //http协议的反代模块
还有一种模块是mod_jk，需要手动安装加载使用

1、配置http协议的反代模块
]# cd /etc/httpd/conf.d/  
]# vim http_proxy_tomcat.conf

  ServerName web.magedu.com
  ProxyVia On                   //用于在多级代理中控制代理请求的流向
  ProxyRequests Off             //关闭正向代理
  ProxyPreserveHost On          //如果启用此功能，代理会将用户请求报文中的Host:行发送给后端的服务器，而不再使用ProxyPass指定的服务器地址
                       //允许反代任何请求
    Require all granted
  
  ProxyPass /       http://localhost:8080/      //反代到指定服务器 
  ProxyPassReverse /      //用于让apache调整HTTP重定向响应报文中的Location、Content-Location及URI标签所对应的URL，在反向代理环境中必须使用此指令避免重定向报文绕过proxy服务器。
         
    Require all granted
  

根据mod_jk进行反代

]# ls
tomcat-connectors-1.2.40-src.tar.gz
]# tar -xf tomcat-connectors-1.2.40-src.tar.gz
]# cd tomcat-connectors-1.2.40-src/native
]# ./configure --with-apxs=/usr/bin/apxs       //编译httpd模块
]# make && make install
 
]# /etc/httpd/conf.d                           //在250.78上
]# vim mod_jk.conf
LoadModule jk_module modules/mod_jk.so              //加载mod_jk
JkWorkersFile /etc/httpd/conf.d/workers.properties  //jk的配置参数文件
JkLogFile logs/mod_jk.log                           //jk代理日志  
JkLogLevel debug                                    //jk的日志级别    
JkMount /* TomcatB                                  //反代到哪台主机上
JkMount /jk_status StatB                            //开启jk状态页
 
]# vim workers.properties
worker.list=TomcatB,StatB                           //worker的列表
worker.TomcatB.host=172.18.250.78                   //worker对应的主机
worker.TomcatB.port=8009                            //worker对应的端口  
worker.TomcatB.type=ajp13                           //worker对应的tomcat的协议
worker.StatB.type=status                            //状态页
 
250.79上只需要把TomcatB改成TomcatA就行，并改下IP地址
至此，一个基于mod_jk模块与后端名为TomcatA的worker通信的配置已经完成，重启httpd服务即可生效

httpd的负载均衡和session sticky（250.77上）

基于http协议的负载均衡至tomcat

]# cd /etc/httpd/conf.d/
]# vim http_balance_tomcat.conf

  BalancerMember http://172.18.250.78:8080 loadfactor=1   //loadfactor表示权重
  BalancerMember http://172.18.250.79:8080 loadfactor=2
  ProxySet lbmethod=byrequests             //定义调度算法

 

  ServerName tc.magedu.com
  ProxyRequests Off
  ProxyPreserveHost On
  ProxyVia On
  
    Require all granted 
  
  Proxypass /  balancer://webserver/
  ProxypassReverse / balancer://webserver/
  
    Require all granted
  

 
后端两台需要使用http_proxy_tomcat.conf的配置文件

设置session_sticky

]# vim http_balance_tomcat.conf
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

  BalancerMember http://172.18.250.78:8080 route=TomcatA
  BalancerMember http://172.18.250.79:8080 route=TomcatB
  ProxySet lbmethod=byrequests
  ProxySet stickysession=ROUTEID   //在将某请求调度至某worker后，源于此址的所有后续请求都将直接调度至此worker，实现将用户session与某worker绑定。

 

  ServerName tc.magedu.com
  ProxyRequests Off
  ProxyPreserveHost On
  ProxyVia On
  
    Require all granted
  
  ProxyPass /  balancer://webserver/
  ProxyPassReverse / balancer://webserver/
  
    Require all granted
  

基于ajp协议负载均衡至tomcat

]# cd /etc/httpd/conf.d/
]# vim ajp_balance_tomcat.conf

  BalancerMember ajp://172.18.250.78:8009 loadfactor=1   
  BalancerMember ajp://172.18.250.79:8009 loadfactor=2
  ProxySet lbmethod=byrequests             

 

  ServerName tc.magedu.com
  ProxyRequests Off
  ProxyPreserveHost On
  ProxyVia On
  
    Require all granted 
  
  Proxypass /  balancer://webserver/
  ProxypassReverse / balancer://webserver/
  
    Require all granted
  

 
后端两台需要使用ajp_proxy_tomcat.conf的配置文件

基于session_sticky的会话保持

]# vim ajp_balance_tomcat.conf 
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

  BalancerMember ajp://172.18.250.78:8009 route=TomcatA
  BalancerMember ajp://172.18.250.79:8009 route=TomcatB
  ProxySet lbmethod=byrequests
  ProxySet stickysession=ROUTEID  

 

  ServerName tc.magedu.com
  ProxyRequests Off
  ProxyPreserveHost On
  ProxyVia On
  
    Require all granted
  
  ProxyPass /  balancer://webserver/
  ProxyPassReverse / balancer://webserver/
  
    Require all granted
  
  

基于mod_jk来实现负载均衡

]# vim mod_jk.conf
LoadModule jk_module modules/mod_jk.so
 
JkWorkersFile /etc/httpd/conf.d/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* webserver                   //改成服务器池名称就行
JkMount /jk_status StatA
 
]# vim workers.properties
worker.list=webserver,StatA
worker.TomcatA.host=172.18.250.78
worker.TomcatA.port=8009
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.host=172.18.250.79
worker.TomcatB.port=8009
worker.TomcatB.type=ajp13
worker.TomcatB.lbfactor=2
worker.webserver.type=lb              //类型为负载均衡
worker.webserver.balance_workers=TomcatA,TomcatB
worker.StatA.type=status

基于session_sticky的会话保持

]# vim workers.properties
worker.list=webserver,StatA
worker.TomcatA.host=172.18.250.78
worker.TomcatA.port=8009
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.host=172.18.250.79
worker.TomcatB.port=8009
worker.TomcatB.type=ajp13
worker.TomcatB.lbfactor=2
worker.webserver.type=lb              
worker.webserver.balance_workers=TomcatA,TomcatB
worker.webserver.sticky_session=1     //开启会话粘性，默认是开启的
worker.StatA.type=status
 
mod_jk除了开启会话粘性外，还需在各个节点tomcat的配置文件server.xml上添加jvmRouet
   //250.78上
   //250.79上

这样就可以实现了。在做实验时每做一步都在浏览器中测试一下，因为测试步骤过于繁琐这里就不介绍了。