Centos6.2_x64位系统安装nginx+tomcat
一、网站需求简介:
网站架构简述:
我们公司是一家电子商务公司,现在线上运行的环境是在一台硬件F5做的负载均衡,用nginx结合tomcat做的代理,访问老网站的时候,静态数据交给165和168本地的nginx,访问动态数据的时候,交给对应的174和176来处理,通过nginx自带的upstream功能来实现的。
网站新需求:
由于访问量过大,为了更好的分摊网站访问的压力,需要再新搭建一台相同的Centos6.2_x64位的linux服务器,然后部署nginx+tomcat,访问网站动态数据的时候,让三台服务器分摊网站访问的压力,从而更好的实现负载均衡的效果。
系统环境介绍:
[root@Web-175 ~]# uname -a Linux Web-175 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux [root@Web-175 ~]# cat /etc/issue CentOS release 6.2 (Final) Kernel \r on an \m
所需软件版本:(由于线上跑的的是之前搭建的老环境,为了避免因版本不同,出现问题,安装的就与之前的版本一致了)
apache-tomcat-7.0.29.tar.gz jdk-6u25-linux-x64.bin nginx-0.8.33.tar.gz
二、nginx+tomcat环境搭建
1.安装gcc编译器及相关工具和依赖库:
[root@Web-175 ~]# yum -y install gcc gcc-c++ autoconf libjpeglibjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 libxml2-develzlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncursesncurses-devel curl curl-devel e2fsprogs e2fsprogs-devel krb5 krb5-devel libidnlibidn-devel openssl openssl-devel openldap openldap-devel nss_ldapopenldap-clients nfs-utils make patch openldap-servers
2.系统基本限制
vim /etc/security/limits.conf (在行尾添加) * soft noproc 10240 * hard noproc 10240 * soft nofile 65536 * hard nofile 65536 # End of file
说明: * 代表针对所有用户
noproc 是代表最大进程数
nofile 是代表最大文件打开数
(1)limits.conf描述
limits.conf文件实际是Linux PAM(插入式认证模块,Pluggable AuthenticationModules)中 pam_limits.so 的配置文件,而且只针对于单个会话
(2)limits.conf工作原理
limits.conf是 pam_limits.so的 配置文件,然后/etc/pam.d/下的应用程序调用pam_***.so模块。譬如说,当用户 访问服务器,服务程序将请求发送到PAM模块,PAM模块根据服务名称在/etc/pam.d目 录下选择一个对应的服务文件,然后根据服务文件的内容选择具体的PAM模块进行处理
3.修改一下linux操作系统的登陆配置文件
打开/etc/pam.d/login这个配置文件 在最后添加一行内容 session required /lib64/security/pam_limits.so # tail -1 /etc/pam.d/login session required /lib64/security/pam_limits.so
4.关闭防火墙和selinux.
# /etc/init.d/iptables stop [root@Web-175 ~]# setenforce 0
5.安装jdk.
[root@Web-175 opt]#chmod +x jdk-6u25-linux-x64.bin [root@Web-175 opt]# ./jdk-6u25-linux-x64.bin [root@Web-175 opt]# tar -zxvf apache-tomcat-7.0.29.tar.gz [root@Web-175 opt]# mv apache-tomcat-7.0.29 tomcat3
6.设置系统环境变量.
[root@Web-175 opt]# vim /etc/profile(在末尾添加三行代码,设置系统环境变量) export CLASSPATH=".:/opt/jdk1.6.0_25/jre/lib/rt.jar:/opt/jdk1.6.0_25/lib/dt.jar:/opt/jdk1.6.0_25/lib/tools.jar" export PATH=$PATH:/opt/jdk1.6.0_25/bin export JAVA_HOME="/opt/jdk1.6.0_25" [root@Web-175 opt]# source /etc/profile 让变量生效
7.修改tomcat的server.xml文件
(修改端口为9080,并且添加memcache共享session)
[root@Web-175 conf]# vim /opt/tomcat3/conf/server.xml <Connector port="9080"protocol="HTTP/1.1" connectionTimeout="20000" URIEncoding="utf-8" redirectPort="8443" maxThreads="500" /> <Context path=""docBase="/opt/newWebshop" useHttpOnly="false"> <ManagerclassName="de.javakaffee.web.msm.MemcachedBackupSessionManager" copyCollectionsForSerialization="false" sessionBackupTimeout="100" sessionBackupAsync="false" sticky="false" requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" memcachedNodes="n1:192.168.5.11:11211,n2:192.168.5.15:11211"/> </Context>
8.增加虚拟机可以使用的最大内存
[root@Web-175 conf]# vim ../bin/catalina.sh +99
在99行,添加下边这段代码
JAVA_OPTS="-server -Djava.awt.headless=true -Xms728m-Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=256m"
注意:tomcat中lib目录下的jar包,要跟174和176服务器上的一样,要是不一样,可以直接从174或176上拷贝一份。
[root@Web-175 tomcat3]# ls ../lib/ annotations-api.jar ecj-3.7.2.jar memcached-session-manager-1.6.3.jar tomcat-coyote.jar tomcat-jdbc.jar catalina-ant.jar el-api.jar memcached-session-manager-tc7-1.6.3.jar tomcat-dbcp.jar tomcat-util.jar catalina-ha.jar jasper-el.jar servlet-api.jar tomcat-i18n-es.jar catalina.jar jasper.jar spymemcached-2.8.4.jar tomcat-i18n-fr.jar catalina-tribes.jar jsp-api.jar tomcat-api.jar tomcat-i18n-ja.jar [root@Web-175 conf]# cd ../webapps/ [root@Web-175 webapps]# ls host-manager manager ROOT webapps下只要三个文件就可以,其余的可以删除
9.安装nginx
编译安装pcre库
PCRE(Perl Compatible RegularExpressions)是一个Perl库,包括 perl 兼容的正规表达式。Pcre的作用主要是使nginx支持HTTPrewrite模块。
[root@Web-175 ~]# cd /opt/ [root@Web-175 opt]# tar -zxvf pcre-7.6.tar.gz [root@Web-175 pcre-7.6]# ./configure [root@Web-175 pcre-7.6]# make && make install
编译安装jpeg
Jpeg是一个基本的图像压缩方式。
[root@Web-175 opt]# tar -zxvf jpegsrc.v7.tar.gz [root@Web-175 opt]# cd jpeg-7/ [root@Web-175 opt]# ./configure --enable-shared --enable-static [root@Web-175 opt]# make && make install
创建nginx用户和组
useradd -s /sbin/nologin -M nginx groupadd ngnix [root@Web-175 opt]# tar -zxvfnginx-upstream-jvm-route-0.2.tar.gz [root@Web-175 opt]# tar -zxvf nginx-0.8.33.tar.gz [root@Web-175 opt]# mv nginx-0.8.33 nginx5 [root@Web-175 opt]# cd nginx5
实现基于 Cookie 的 Session Sticky 的功能
patch -p0 < /opt/nginx_upstream_jvm_route/jvm_route.patch
编译安装nginx [root@Web-175 nginx5]# ./configure --prefix=/opt/nginx6--with-http_stub_status_module --with-pcre=/opt/pcre-7.6--add-module=../nginx_upstream_jvm_route/ --with-openssl=/usr/bin/openssl [root@Web-175 nginx5]# make && make install
为了方便,直接可以拷贝174或176上的nginx.conf的配置文件修改下代理转发的端口即可.
[root@Web-175 nginx5]# catnginx6/conf/nginx.conf user nobody; worker_processes 4; #error_log logs/error.log; #error_log logs/error.log notice; #error_log logs/error.log info; #pid logs/nginx.pid; worker_rlimit_nofile 51200; events { use epoll; worker_connections 51200; } http { include mime.types; default_type application/octet-stream; # limit_zone one $binary_remote_addr 10m; server_names_hash_bucket_size 256; client_header_buffer_size 256k; large_client_header_buffers 4 256k; client_max_body_size200m; client_body_buffer_size 256k; sendfile on; tcp_nopush on; tcp_nodelay on; send_timeout 3m; keepalive_timeout 120; server_tokens off; gzip on; gzip_min_length 1k; gzip_buffers 4 16k; gzip_http_version 1.0; gzip_comp_level 2; gzip_types text/plain application/x-javascripttext/css application/xml text/jsp; gzip_vary on; #proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $http_x_forwarded_for; proxy_set_header X-Forwarded-For$proxy_add_x_forwarded_for; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; # proxy_buffers 32 4k; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; proxy_temp_path/opt/proxy_temp_path; proxy_cache_path/opt/proxy_cache_path levels=1:2 keys_zone=cache_one:500m inactive=1dmax_size=30g; # access_log logs/access.log main; #sendfile on; #tcp_nopush on; #keepalive_timeout 0; #keepalive_timeout 65; #gzip on; upstream www.beyond.com{ server 127.0.0.1:9080; ip_hash; } server { listen 192.168.8.175:80; server_name www.beyomd.com localhost 192.168.8.175; # charset GBK; #charset UTF-8; access_log logs/access_beyond.log main; if ($host ~beyond.com){ rewrite ^(.*)http://www.taokey.com$1 permanent; } if ($host ~9h.com){ rewrite ^(.*)http://www.beyond.com$1 permanent; } location ^~/zhuanti/ { rewrite/zhuanti/(.*).html /html/zhuanti/$1.jsp; } location ^~/html/article/{ rewrite/html/article/(\d+)/(\d+)/(\d+)/(.*)\.html$ /wineCulture/detail-$1$2$3$4.shtml; } location / { proxy_pass http://www.beyond.com; proxy_set_header X-Real-IP $http_x_forwarded_for; index index.html index.shtmlindex.jsp login.jsp index.htm; #proxy_redirect off; } location ~^/static/ { root /opt/newWebshop/; access_log on; expires 12h; } location ~^/re- { proxy_pass http://www.beyond.com; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } location ~^/pi- { proxy_pass http://www.beyond.com; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } location ~^/wineCulture/?$ { rewrite^/wineCulture/?$ /wineCulture/content_cultureIndex.shtml; } location ~^/wineCulture/zuixinjiushili-(\d+)/?$ { rewrite/wineCulture/zuixinjiushili-(\d+)/? /wineCulture/content_showArticleList.shtml?more=3&page=$1 last; } location ~^/wineCulture/hongjiuyujiankang-(\d+)/?$ { rewrite/wineCulture/hongjiuyujiankang-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=5&seek=last; } location ~^/wineCulture/jiuyoutianxia-(\d+)/?$ { rewrite/wineCulture/jiuyoutianxia-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=6&seek=last; } location~^/wineCulture/hongjiuyishu-(\d+)/?$ { rewrite/wineCulture/hongjiuyishu-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=7&seek=last; } location~^/wineCulture/mingjiazhuanlan-(\d+)/?$ { rewrite/wineCulture/mingjiazhuanlan-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=8&seek=last; } location ~^/wineCulture/hongjiutouzi-(\d+)/?$ { rewrite/wineCulture/hongjiutouzi-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=11&seek=last; } location ~^/wineCulture/hongjiumeishi-(\d+)/?$ { rewrite/wineCulture/hongjiumeishi-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=13&seek=last; } location ~^/wineCulture/hongjiuzhuangyuan-(\d+)/?$ { rewrite/wineCulture/hongjiuzhuangyuan-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=15&seek=last; } location ~^/wineCulture/putaojiurumen-(\d+)/?$ { rewrite/wineCulture/putaojiurumen-(\d+)/? /wineCulture/content_showArticleList.shtml?page=$1&pageSize=9&more=19&seek=last; } location ~^/wineCulture/zuixinjiushili/?$ { rewrite/wineCulture/zuixinjiushili/? /wineCulture/content_showArticleList.shtml?more=3&pageSize=9 last; } location ~^/wineCulture/hongjiuyujiankang/?$ { rewrite/wineCulture/hongjiuyujiankang/? /wineCulture/content_showArticleList.shtml?more=5&pageSize=9 last; } location ~^/wineCulture/jiuyoutianxia/?$ { rewrite/wineCulture/jiuyoutianxia/? /wineCulture/content_showArticleList.shtml?more=6&pageSize=9 last; } location ~^/wineCulture/hongjiuyishu/?$ { rewrite/wineCulture/hongjiuyishu/? /wineCulture/content_showArticleList.shtml?more=7&pageSize=9 last; } location ~^/wineCulture/mingjiazhuanlan/?$ { rewrite/wineCulture/mingjiazhuanlan/?/wineCulture/content_showArticleList.shtml?more=8&pageSize=9 last; } location ~^/wineCulture/hongjiutouzi/?$ { rewrite/wineCulture/hongjiutouzi/? /wineCulture/content_showArticleList.shtml?more=11&pageSize=9 last; } location ~^/wineCulture/hongjiumeishi/?$ { rewrite/wineCulture/hongjiumeishi/?/wineCulture/content_showArticleList.shtml?more=13&pageSize=9 last; } location ~^/wineCulture/hongjiuzhuangyuan/?$ { rewrite/wineCulture/hongjiuzhuangyuan/?/wineCulture/content_showArticleList.shtml?more=15&pageSize=9 last; } location ~^/wineCulture/putaojiurumen/?$ { rewrite/wineCulture/putaojiurumen/?/wineCulture/content_showArticleList.shtml?more=19&pageSize=9 last; } location ^~/html/culture{ rewrite /html/culture/cultureIndex\.html /wineCulture permanent; } location ~^/p-list/* { proxy_next_upstream http_502 http_504 error timeout invalid_header; proxy_cache cache_one; proxy_cache_valid 200 304 12h; proxy_cache_key $host$uri$is_args$args; proxy_set_header Host $host; proxy_set_header X-Forwarded-For$remote_addr; proxy_pass http://www.beyond.com; expires 1h; } location ~^/wineCulture/detail-.*\.shtml { proxy_next_upstream http_502http_504 error timeout invalid_header; proxy_cache cache_one; proxy_cache_valid 200 304 12h; proxy_cache_key $host$uri$is_args$args; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_pass http://www.beyond.com; expires 12h; } location =/wineCulture/content_cultureIndex.shtml { proxy_next_upstream http_502 http_504 error timeout invalid_header; proxy_cache cache_one; proxy_cache_valid 200 304 12h; proxy_cache_key $host$uri$is_args$args; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_pass http://www.beyond.com; expires 4h; } location =/daogou/daogou_index.shtml { proxy_next_upstream http_502 http_504 error timeout invalid_header; proxy_cache cache_one; proxy_cache_valid 200 304 12h; proxy_cache_key $host$uri$is_args$args; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_pass http://www.beyond.com; expires 12h; } location ~ .*\.(jsp|do|shtml)?$ { proxy_pass http://www.beyond.com; proxy_set_header X-Real-IP $http_x_forwarded_for; } location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$ { root /opt/webshop/; access_log off; expires 30d; } location ~ .*\.(js|css|htm|html)?$ { # proxy_pass http://www.beyond.com; # proxy_set_header X-Real-IP $remote_addr; root /opt/webshop/; access_log off; expires 12h; } location /html { root /opt/webshop/; index index.html index.jsp index.shtml login.jspindex.htm; access_log off; expires 12h; } location ^~ /bg { return 404; } location ^~ /WEB-INF { return 404; } error_page 404 http://www.beyond.com/404.jsp; error_page 500 502 503 504 /50x.html; location =/50x.html { root html; } server_name_in_redirect off; } } 注意:网站域名我经过的修改。 10.挂载旧网站和新网站数据,网站数据是通过NFS实现挂载的
#mkdir /opt/webshop #mkdir /opt/newWebshop #mount 192.168.8.180:/root/webshop /opt/webshop/ -o nolock #mount 192.168.8.180:/opt/newWebshop /opt/newWebshop/ -o nolock
11.开启nginx和tomcat
[root@Web-175 ~]# /opt/tomcat3/bin/startup.sh Using CATALINA_BASE: /opt/tomcat3 Using CATALINA_HOME: /opt/tomcat3 Using CATALINA_TMPDIR: /opt/tomcat3/temp Using JRE_HOME: /opt/jdk1.6.0_25 Using CLASSPATH: /opt/tomcat3/bin/bootstrap.jar:/opt/tomcat3/bin/tomcat-juli.jar [root@Web-175 ~]# /opt/nginx6/sbin/nginx -t the configuration file /opt/nginx6/conf/nginx.conf syntax is ok configuration file /opt/nginx6/conf/nginx.conf test issuccessful [root@Web-175 ~]# /opt/nginx6/sbin/nginx
12.查看下是否已经开启成功
[root@Web-175 ~]# ps -ef | grep java root 3684 1 68 15:51 pts/0 00:01:04/opt/jdk1.6.0_25/bin/java-Djava.util.logging.config.file=/opt/tomcat3/conf/logging.properties -server-Djava.awt.headless=true -Xms728m -Xmx1024m -XX:PermSize=128m-XX:MaxPermSize=256m -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager-Djava.endorsed.dirs=/opt/tomcat3/endorsed -classpath/opt/tomcat3/bin/bootstrap.jar:/opt/tomcat3/bin/tomcat-juli.jar-Dcatalina.base=/opt/tomcat3 -Dcatalina.home=/opt/tomcat3-Djava.io.tmpdir=/opt/tomcat3/temp org.apache.catalina.startup.Bootstrap start root 3745 3427 015:52 pts/0 00:00:00 grep java [root@Web-175 ~]# ps -ef | grep nginx root 3711 1 015:51 ? 00:00:00 nginx: masterprocess /opt/nginx6/sbin/nginx nobody 3712 3711 015:51 ? 00:00:00 nginx: worker process nobody 3713 3711 015:51 ? 00:00:00 nginx: workerprocess nobody 3714 3711 015:51 ? 00:00:00 nginx: workerprocess nobody 3715 3711 015:51 ? 00:00:00 nginx: workerprocess nobody 3716 3711 015:51 ? 00:00:00 nginx: cachemanager process root 3747 3427 015:53 pts/0 00:00:00 grep nginx
查看下tomcat的日志,是否正常
[root@Web-175 ~]# tail -f /opt/tomcat3/logs/catalina.out 2013-10-16 15:51:48.636 -- LoopQuerierServiceJoinImpl: 0 个注册、评论事件,不需要处理。 2013-10-16 15:51:48.731 -- eventLog [130009] is handling byWeb-175#0 2013-10-16 15:51:48.918 -- eventLog [130008] is handling byWeb-175#0 2013-10-16 15:51:48.988 -- eventLog [130010] is handling byWeb-175#0 2013-10-16 15:51:49.056 -- 这 3 条记录由交线程Web-175#0 处理完 2013-10-16 15:51:51com.sun.xml.ws.transport.http.servlet.WSServletContextListenercontextInitialized 信息: WSSERVLET12: JAX-WS 上下文监听程序正在初始化 2013-10-16 15:51:52com.sun.xml.ws.transport.http.servlet.WSServletDelegate <init> 信息: WSSERVLET14: JAX-WS servlet 正在初始化 2013-10-16 15:51:57.449 -- 有 0 条 订单加分 order 记录,不需要处理。 2013-10-16 15:52:21 org.apache.catalina.util.SessionIdGeneratorcreateSecureRandom 信息: Creation of SecureRandom instance for session ID generation using[SHA1PRNG] took [29,220] milliseconds. 2013-10-16 15:52:21de.javakaffee.web.msm.MemcachedSessionService startInternal 信息: MemcachedSessionService starts initialization... (configured nodesdefinition n1:192.168.8.170:11211,n2:192.168.8.169:11211, failover nodes null) 2013-10-16 15:52:21.724 INFOnet.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.8.170:11211, #Rops=0, #Wops=0, #iq=0, topRop=null,topWop=null, toWrite=0, interested=0} to connect queue 2013-10-16 15:52:21.725 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.8.169:11211, #Rops=0,#Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connectqueue 2013-10-16 15:52:21de.javakaffee.web.msm.RequestTrackingHostValve <init> 信息: Setting ignorePattern to .*\.(ico|png|gif|jpg|css|js)$ 2013-10-16 15:52:21.735 INFOnet.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@1a6261c9 2013-10-16 15:52:21.736 INFOnet.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@33e1830b 2013-10-16 15:52:21de.javakaffee.web.msm.MemcachedSessionService setLockingMode 信息: Setting lockingMode to NONE 2013-10-16 15:52:21de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory 信息: Creating transcoder factoryde.javakaffee.web.msm.JavaSerializationTranscoderFactory 2013-10-16 15:52:21de.javakaffee.web.msm.MemcachedSessionService startInternal 信息: MemcachedSessionService finished initialization, sticky false,operation timeout 1000, with node ids [n1, n2] and failover node ids [] 2013-10-16 15:52:23 org.apache.catalina.startup.HostConfigdeployDirectory 信息: Deploying web application directory/opt/tomcat3/webapps/host-manager 2013-10-16 15:52:23 org.apache.tomcat.util.digester.DigesterendElement 警告: No rules found matching'Context/Context/Manager'. 2013-10-16 15:52:23 org.apache.tomcat.util.digester.DigesterendElement 警告: No rules found matching'Context/Context'. 2013-10-16 15:52:23 org.apache.catalina.startup.HostConfigdeployDirectory 信息: Deploying web application directory /opt/tomcat3/webapps/manager 2013-10-16 15:52:23 org.apache.tomcat.util.digester.DigesterendElement 警告: No rules found matching'Context/Context/Manager'. 2013-10-16 15:52:23 org.apache.tomcat.util.digester.DigesterendElement 警告: No rules found matching'Context/Context'. 2013-10-16 15:52:23 org.apache.coyote.AbstractProtocol start 信息: Starting ProtocolHandler ["http-bio-9080"] 2013-10-16 15:52:23 org.apache.coyote.AbstractProtocol start 信息: Starting ProtocolHandler ["ajp-bio-8009"] 2013-10-16 15:52:23 org.apache.catalina.startup.Catalina start 信息: Server startup in 58531 ms 从tomcat的日志来看,是正常的,说明tomcat正常开启,而且运行正常。
到此为止,nginx+tomcat的环境,已经搭建完毕。
测试:
测试正常,说明nginx结合tomcat,安装成功了。