shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-1/logs/catalina.out
Jan 26, 2013 12:03:25 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8088"]
Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8089"]
Jan 26, 2013 12:03:26 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1743 ms
Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.26
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n1)
2013-01-26 00:03:27.441 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-01-26 00:03:27.442 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.RequestTrackingHostValve
INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$
2013-01-26 00:03:27.459 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@5b0668
2013-01-26 00:03:27.460 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
INFO: Setting lockingMode to null
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n2] and failover node ids [n1]
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/host-manager
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/manager
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/docs
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/examples
Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/ROOT
Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8088"]
Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8089"]
Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 1487 ms
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-2/logs/catalina.out
Jan 26, 2013 12:04:34 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-9088"]
Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-9089"]
Jan 26, 2013 12:04:34 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 742 ms
Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.26
Jan 26, 2013 12:05:07 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom
INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [33,094] milliseconds.
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n2)
2013-01-26 00:05:07.873 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-01-26 00:05:07.876 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.RequestTrackingHostValve
INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$
2013-01-26 00:05:07.896 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@701a27
2013-01-26 00:05:07.897 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
INFO: Setting lockingMode to null
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n1] and failover node ids [n2]
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/host-manager
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/manager
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/docs
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/examples
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/ROOT
Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-9088"]
Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-9089"]
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 34163 ms
已经成功启动了。
Nginx安装、配置、启动
在Ubuntu系统下,如果没有安装Nginx,可以执行如下命令进行安装:
sudo apt-get install nginx
默认情况下,Nginx使用默认的配置文件/etc/nginx/nginx.conf,该配置文件直接包含了虚拟目录配置文件/etc/nginx/sites-available/default,修改该文件的内容为,如下所示:
# You may add here your
# server {
# ...
# }
# statements for each of your virtual hosts to this file
##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
#
# Generally, you will want to move this file somewhere, and start with a clean
# file but keep this around for reference. Or just disable in sites-enabled.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##
upstream dev.shirdrn.org {
server 192.168.1.104:8088 weight=1;
server 192.168.1.104:9088 weight=1;
}
server {
#listen 80; ## listen for ipv4; this line is default and implied
#listen [::]:80 default ipv6only=on; ## listen for ipv6
root /usr/share/nginx/www/session; # session是我们配置的虚拟目录,实际直接指向Tomcat下名为session的web应用
index index.html index.htm;
# Make site accessible from http://localhost/
####### server_name localhost;
server_name dev.shirdrn.org; # Nginx服务所在主机
charset utf-8;
location / {
proxy_pass http://dev.shirdrn.org; #直接代理tomcat集群
proxy_set_header X-Real-IP $remote_addr;
client_max_body_size 100m;
# First attempt to serve request as file, then
# as directory, then fall back to index.html
####### try_files $uri $uri/ /index.html;
# Uncomment to enable naxsi on this location
# include /etc/nginx/naxsi.rules
}
location ~ ^/(WEB-INF)/ { # 禁止访问Tomcat下web应用的WEB-INF目录下的资源
deny all;
}
location /doc/ {
alias /usr/share/doc/;
autoindex on;
allow 127.0.0.1;
deny all;
}
# Only for nginx-naxsi : process denied requests
#location /RequestDenied {
# For example, return an error code
#return 418;
#}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
#error_page 500 502 503 504 /50x.html;
#location = /50x.html {
# root /usr/share/nginx/www;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# fastcgi_split_path_info ^(.+\.php)(/.+)$;
# # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
#
# # With php5-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php5-fpm:
# fastcgi_pass unix:/var/run/php5-fpm.sock;
# fastcgi_index index.php;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# root html;
# index index.html index.htm;
#
# location / {
# try_files $uri $uri/ /index.html;
# }
#}
# HTTPS server
#
#server {
# listen 443;
# server_name localhost;
#
# root html;
# index index.html index.htm;
#
# ssl on;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
#
# ssl_session_timeout 5m;
#
# ssl_protocols SSLv3 TLSv1;
# ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
# ssl_prefer_server_ciphers on;
#
# location / {
# try_files $uri $uri/ /index.html;
# }
#}
上面配置中根目录为/usr/share/nginx/www,因为我们使用到了一个虚拟目录来对应Tomcat下部署的web应用的path,所以需要在/usr/share/nginx/www下面创建这个对应的虚拟目录:
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo mkdir /usr/share/nginx/www/session
最后,可以启动Nginx服务器,执行如下命令:
shirdrn@dev:~$ sudo /etc/init.d/nginx start
查询Nginx服务启动状态:
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo ps -ef | grep nginx
root 3199 1 0 Jan25 ? 00:00:00 nginx: master process /usr/sbin/nginx
www-data 3200 3199 0 Jan25 ? 00:00:00 nginx: worker process
www-data 3201 3199 0 Jan25 ? 00:00:00 nginx: worker process
www-data 3202 3199 0 Jan25 ? 00:00:00 nginx: worker process
www-data 3203 3199 0 Jan25 ? 00:00:00 nginx: worker process
shirdrn 3327 3095 0 00:01 pts/5 00:00:00 grep --color=auto nginx
这是就可以通过Nginx访问(代理)Web应用首页了。
集群验证
上面配置完成,服务都已经启动成功,可以直接请求链接 http://dev.shirdrn.org/session,就会直接访问部署在Tomcat服务器上的Java Web应用,看到登录页面。
使用前面准备的Web应用session中账号数据进行模拟登录,就可以查看Memcached缓存的数据:
- telnet 192.168.1.104 11211
telnet到memcached服务端口,查看缓存统计数据。
shirdrn@dev:~$ telnet 192.168.1.104 11211
Trying 192.168.1.104...
Connected to 192.168.1.104.
Escape character is '^]'.
ERROR
stats
STAT pid 3300
STAT uptime 2148
STAT time 1359131741
STAT version 1.4.13
STAT libevent 2.0.16-stable
STAT pointer_size 32
STAT rusage_user 0.052003
STAT rusage_system 0.084005
STAT curr_connections 7
STAT total_connections 9
STAT connection_structures 8
STAT reserved_fds 20
STAT cmd_get 23
STAT cmd_set 6
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 2
STAT get_misses 21
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 2675
STAT bytes_written 2835
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 262144
STAT hash_is_expanding 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT bytes 858
STAT curr_items 2
STAT total_items 6
STAT evictions 0
STAT reclaimed 0
END
- telnet 192.168.1.104 11311
shirdrn@dev:~$ telnet 192.168.1.104 11311
Trying 192.168.1.104...
Connected to 192.168.1.104.
Escape character is '^]'.
stats
STAT pid 3308
STAT uptime 2398
STAT time 1359132000
STAT version 1.4.13
STAT libevent 2.0.16-stable
STAT pointer_size 32
STAT rusage_user 0.004000
STAT rusage_system 0.128008
STAT curr_connections 7
STAT total_connections 9
STAT connection_structures 8
STAT reserved_fds 20
STAT cmd_get 1
STAT cmd_set 2
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 1
STAT delete_misses 1
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 644
STAT bytes_written 2109
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 262144
STAT hash_is_expanding 0
STAT expired_unfetched 1
STAT evicted_unfetched 0
STAT bytes 0
STAT curr_items 0
STAT total_items 2
STAT evictions 0
STAT reclaimed 1
END
我们可以停掉某个Tomcat服务,然后观察Session会话数据的复制过程,能够保证当前集群中的会话数据不丢失,另一个结点仍然能提供基于该存在Session的服务。
- https://code.google.com/p/memcached-session-manager
- https://code.google.com/p/memcached-session-manager/wiki/SetupAndConfiguration
- http://code.google.com/p/kryo/
- http://wiki.ubuntu.org.cn/Nginx
- http://www.iteye.com/topic/676347