每次tomcat在客户服务器上运行一个多月左右的时候,都会崩溃。然后tomcat自动停止,生成hs_err_pid.log文件。
因为在原来服务器上就出现了tomcat崩溃,也不知道因为什么原因崩溃,多长时间崩溃。而且也没注意有pid.log文件,只是进行重启操作。后来还跑过去换服务器,重新进行部署。但是还是会崩溃,这次注意到pid.log文件,然后进行分析,一直找不到解决的原因。
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 533160 bytes for Chunk::new
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (allocation.cpp:323), pid=5452, tid=4920
#
# JRE version: Java(TM) SE Runtime Environment (7.0_40-b43) (build 1.7.0_40-b43)
# Java VM: Java HotSpot(TM) Server VM (24.0-b56 mixed mode windows-x86 )
# Failed to write core dump.
根据这个日志前面部分分析,就是内存不足导致的。当时很奇怪,怎么会引起内存不足呢,因为按照以往的项目,根本不需要太大内存,何况服务器内存是32G的,怎么都不会引起内存不足。(当时没注意java 是32为的)
Java Threads: ( => current thread )
0x491d1c00 JavaThread "Timer-919" daemon [_thread_blocked, id=4840, stack(0x00110000,0x00190000)]
0x491d1800 JavaThread "Timer-915" daemon [_thread_blocked, id=5244, stack(0x7fee0000,0x7ff60000)]
0x491d1000 JavaThread "Timer-914" daemon [_thread_blocked, id=4928, stack(0x00050000,0x000d0000)]
0x491cd000 JavaThread "Timer-913" daemon [_thread_blocked, id=5200, stack(0x7fd40000,0x7fdc0000)]
0x491d0800 JavaThread "Timer-912" daemon [_thread_blocked, id=11684, stack(0x7fe60000,0x7fee0000)]
0x491ccc00 JavaThread "Timer-911" daemon [_thread_blocked, id=6312, stack(0x7fba0000,0x7fc20000)]
0x491d0400 JavaThread "Timer-910" daemon [_thread_blocked, id=8736, stack(0x7fcc0000,0x7fd40000)]
0x491cfc00 JavaThread "Timer-909" daemon [_thread_blocked, id=8556, stack(0x7fa00000,0x7fa80000)]
0x491cd800 JavaThread "Timer-908" daemon [_thread_blocked, id=12128, stack(0x7fb20000,0x7fba0000)]
0x491cec00 JavaThread "Timer-907" daemon [_thread_blocked, id=6568, stack(0x7f860000,0x7f8e0000)]
0x491cdc00 JavaThread "Timer-906" daemon [_thread_blocked, id=10204, stack(0x7f980000,0x7fa00000)]
0x491cf000 JavaThread "Timer-905" daemon [_thread_blocked, id=5100, stack(0x7f7a0000,0x7f820000)]
0x491cc400 JavaThread "Timer-904" daemon [_thread_blocked, id=1740, stack(0x7f6e0000,0x7f760000)]
还有就是发现,不知道什么原因,创建了九百多个Timer线程。但是定时都是用Spring的定时,没有用自己写的Timer那。
因为创建过多的Timer线程,于是在项目中查找,看什么地方创建了,于是找到了
@Scheduled(cron="0 0 0/1 * * ?") //每小时
public void init() {
Map map = queryEmail();
String str = queryWarning();
String user = map.get("monitorSenderMail");
if(null==user){
try {
Thread.sleep(60*60*1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
String password =map.get("monitorSenderPass");
String toaddress = map.get("monitorRecieverMail");
String emailstmp = map.get("emailstmp");
String emailport = map.get("emailport");
String theme = "测试";
Timer timer = new Timer(true);
timer.schedule(new WriteWarning(str),0,1000*60*60); //1个小时写向warning.txt文件写一次
timer.schedule(new SendEmail(user, password, toaddress, theme,emailstmp,emailport),0,1*60*60*1000); //1个小时向邮箱发送一次
}
这段代码,用的spring定时器,每小时调用一次。然后我。。。。。(我就不多说啥了,改吧)
然后发现客户的tomcat配置中
set JAVA_OPTS=-server -Xms128M -Xmx512M -Xss512k
然后恍然大悟,为啥一个多月就崩溃一次,每小时创建一个线程,一天24个,一个月下来就七百多,每个线程512K,结果不言而喻。。。。。
1. 修改代码,只用spring 定时,不创建Timer线程
2. 修改tomcat配置 因为是32为的java 目前直接设置-Xmx1024M