nginx 0.7.64 503错误解决方法

浏览器提示503错误：The page you are looking for is temporarily unavailable.Please try again later.

 

查看nginx的错误日志

#more error.log
2010/02/25 15:35:48 [alert] 28537#0: worker process 28541 exited on signal 11
2010/02/25 15:35:49 [alert] 28537#0: worker process 28540 exited on signal 11
2010/02/25 15:35:49 [alert] 28537#0: worker process 28538 exited on signal 11
2010/02/25 15:35:54 [alert] 28537#0: worker process 30384 exited on signal 11
2010/02/25 15:35:54 [alert] 28537#0: worker process 30385 exited on signal 11
2010/02/25 15:35:54 [alert] 28537#0: worker process 30386 exited on signal 11
2010/02/25 15:35:59 [alert] 28537#0: worker process 30389 exited on signal 11
2010/02/25 15:35:59 [alert] 28537#0: worker process 28539 exited on signal 11
2010/02/25 15:36:00 [alert] 28537#0: worker process 30390 exited on signal 11
2010/02/25 15:36:09 [alert] 28537#0: worker process 30398 exited on signal 11

 

2010/02/25 15:36:11 [error] 30399#0: *140 limiting connections by zone "one", client: 124.128.18.164, server: localhost, request: "GET /aim/380K/mv/201002/126665211551fcd03770.mp4/03563069d9a532c0183283d492cdcd9e/4b877865.mp4 HTTP/1.1", host: "60.*.236.22:*80", referrer: "http://*.com/theme/index/t2/"
2010/02/25 15:36:23 [error] 30399#0: *143 limiting connections by zone "one", client: 124.128.18.164, server: localhost, request: "GET /aim/380K/mv/201002/126673220943d6d9c8a4.mp4/be38df13158f338d7a2d754169a15a46/4b877865.mp4 HTTP/1.1", host: "60.*.236.22:*80", referrer: "http://*.com/theme/index/t2/"

 

升级到nginx0.7.65之前的状态

lai@ff:/var/log/nginx$ sudo more error.log|grep worke        
2010/02/26 07:38:08 [alert] 31300#0: worker process 31304 exited on signal 11
2010/02/26 08:40:07 [alert] 31300#0: worker process 31301 exited on signal 11
2010/02/26 09:08:03 [alert] 31300#0: worker process 2068 exited on signal 11
2010/02/26 11:08:55 [alert] 31300#0: worker process 31303 exited on signal 11
2010/02/26 11:38:34 [alert] 31300#0: worker process 2262 exited on signal 11
2010/02/26 13:08:32 [alert] 31300#0: worker process 2641 exited on signal 11
2010/02/26 13:40:57 [alert] 31300#0: worker process 31302 exited on signal 11
2010/02/26 14:07:55 [alert] 31300#0: worker process 4684 exited on signal 11
2010/02/26 15:08:02 [alert] 14374#0: worker process 14376 exited on signal 11
2010/02/26 16:37:55 [alert] 14374#0: worker process 14377 exited on signal 11
2010/02/26 17:37:59 [alert] 20435#0: worker process 20437 exited on signal 11
lai@ff:/var/log/nginx$

 

查看nginx的访问日志

#more iphone.access.log
124.128.18.164 - - [25/Feb/2010:15:36:11 +0800] "GET /aim/380K/mv/201002/126665211551fcd03770.m/03563069d9a532c0183283d492cdcd9e/4b877865.mp4 HTTP/1.1" 503 383 "http://*.com/theme/index/t2/" "Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac OS X; zh-cn) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7D11 Safari/528.16"

 

 

#dmesg

[8899569.894983] nginx[31582]: segfault at 1 ip 080727a8 sp bf8c1d00 error 4 in nginx[8048000+83000]
[8919934.402677] nginx[31604]: segfault at 1 ip 080727a8 sp bf8c1d00 error 4 in nginx[8048000+83000]
[8919935.259635] nginx[31225]: segfault at 1 ip 080727a8 sp bf8c1cc0 error 4 in nginx[8048000+83000]
[8919935.683643] nginx[31551]: segfault at 1 ip 080727a8 sp bf8c1d00 error 4 in nginx[8048000+83000]
[8920889.231712] nginx[994]: segfault at 1 ip 080727a8 sp bf8c1d00 error 4 in nginx[8048000+83000]
[8937845.432609] nginx[32105]: segfault at 1 ip 080727a8 sp bf8c1d00 error 4 in nginx[8048000+83000]

 

worker process 20437 exited on signal 11的错误原因：

11)SIGSEGV  建立CORE文件      段非法错误
 SIGSEGV 试图访问未分配给自己的内存, 或试图往没有写权限的内存地址写数据.

lai@kin:/var/log/nginx$ free
             total       used       free     shared    buffers     cached
Mem:       3107008    3090076      16932           0     515532    2190200
-/+ buffers/cache:     384344    2722664
Swap:      2931852        104    2931748

lai@kin:/var/log/nginx$ ps aux|grep pop3|wc
    129    1420    9803
lai@kin:/var/log/nginx$ ps aux|grep php|wc
     22     243    1796

 

 

现代操作系统中有两个关键性的概念，其一是作为程序运行实体的进程（也叫任务），其二是进程运行的空间广度，也即地址空间。在x86体系下，地址有三级，其一是逻辑地址，也就是你写汇编时用的那种指定段和偏移的地址；其二是线性地址，不同的进程都可以有一个重合的线性地址集合，即线性地址空间；其三才是真实的物理地址空间。

而你所说的C++的析构函数其实是实现了线性地址空间层次上的资源回收。现代的操作系统普遍支持虚存特性，即把进程运行在其中的地址空间与实际的物理内存空间隔离开来，从而实现多任务“同时”运行。而在主流的32位地址总线系统中，进程可以寻址的线性地址空间是4GB，也就是说，你写程序时可以认为系统有 4GB的内存——当然，大多数普通pc都没有这么奢侈的配置——而不必关心真实的物理内存大小，unix/linux的内核自然会把你的4GB的地址空间转换到也许只有512MB的真实物理内存上去运行。从这里就可以看出，当我在程序里malloc了10MB的一块内存时，其实只是将我的地址空间拉长了 10MB，也就是告诉内核，“我这个程序现在运行在110MB的空间上，请你分配新的目录和页表以便寻址物理地址”云云，而并不是说真的立刻就在内存中划出10MB空间供使用。事实上，此刻在内存中，你的程序所占的物理地址也许只有5MB不到。

那为什么书上又要说“分配了内存就一定要释放”呢？这是因为你最大的地址空间是有限，对于通常的系统，这个限制是4GB。如果你只malloc不 free，那么你的地址空间就被不断地拉长，最后超出4GB的限制，这时内核无法正确做地址转换，就会把向你的进程发送一个SIGSEGV信号，把你的进程强行kill掉，并释放起初为你的进程分配的所有资源，包括任务描述符，段表和页表等等。

而我们所说的操作系统的内存管理是负责逻辑、线性和物理地址转换，以及内存的监视，对缓冲和缓存进行管理的整个sub system，其重要的一个任务便是更加有效地使用内存，包括避免产生控制之外的内存，以及提高程序的响应速度。为此unix/linux的内核都会实现缓冲（cache）与缓存（buffer）两种机制（在2.6系列内核中似乎两部分已经统一了）。缓冲把程序的一部分内容放在内存内，因而你会发现当你关闭一个大的程序后，以后再启动同一程序的速度会变得很快，这其实是因为内核此时并没有从硬盘上读取可执行文件，而是直接执行了内存中的代码。缓存则主要用于硬件i/o操作，当你写一个大文件到硬盘时，你会发现硬盘灯并不是一直亮着，而是一下一下地闪烁，这就是因为数据先被缓存在内存中，到一定数量时才一次性写到硬盘中（也就是sync），这样大大提高了硬盘i/o的效率，也避免了硬盘无谓地损耗。

基于上述的原因，unix/linux内核会尽量使用多的物理内存做缓冲和缓存。但是当内存不够时，内核完全可以在可以忽略的时间内取得足够的内存分配给需要的程序。

上面这段话的关键点是不要搞混编程中的释放内存和内核中的管理内存两个概念。程序中操作的是线性地址空间，而只有内核才会和物理地址空间打交道。而不管你的程序写的多糟糕，只要进程结束，所有的内存立刻都会被内核回收做他用。并不是说如果我的程序malloc后没有free，那么这部分内存就再也没办法再使用了。

 

解决方法：重启nginx。减少pop3-login进程数。

 

其他关于此类错误的报告:

Debian Bug report logs - #515904

nginx worker exit with 11 signal if wildcard dns names are used with ssl  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=515904

nginx官方更新日志：














Changes with nginx


 0.7.65                                        01 Feb 2010

 *) Security: now nginx
 /Windows ignores trailing spaces in URI.















       Thanks to Dan Crowley, Core Security Technologies.































    *) Security: now nginx














/Windows ignores short files names.















       Thanks to Dan Crowley, Core Security Technologies.































    *) Change: now the "009" status code is written to an access log for 















       proxied HTTP/0.9 responses.































    *) Change: now the default buffer size of the 















       "large_client_header_buffers" directive is 8K.















       Thanks to Andrew Cholakian.































    *) Change: now default SSL ciphers are "HIGH:!ADH:!MD5".































    *) Change: now SSLv2 protocol is disabled by default.































    *) Change: now $host variable value is always low case.































    *) Feature: the conf/fastcgi.conf for simple FastCGI configurations.































    *) Feature: now URI part is not required a "proxy_pass" directive if 















       variables are used.































    *) Feature: the $ssl_session_id variable.































    *) Bugfix: if a proxied or FastCGI request was internally redirected to 















       another proxied or FastCGI location, then $upstream_response_time 















       variable may have abnormally large value; the bug had appeared in 















       0.7.63.































    *) Bugfix: if the "expires modified" set date in the past, then a 















       negative number was set in the "Cache-Control" response header 















       line.















       Thanks to Alex Kapranoff.































    *) Bugfix: nginx














 closed a connection if a cached response had an empty 















       body.















       Thanks to Piotr Sikora.































    *) Bugfix: nginx














 cached a 304 response if there was the "If-None-Match" 















       header line in a proxied request.















       Thanks to Tim Dettrick and David Kostal.































    *) Bugfix: nginx














 did not treat a comma as separator in the 















       "Cache-Control" backend response header line.































    *) Bugfix: cached HTTP/0.9 responses were handled incorrectly.































    *) Bugfix: nginx














 sent gzipped responses to clients those do not support 















       gzip, if "gzip_static on" and "gzip_vary off"; the bug had appeared 















       in 0.8.16.































    *) Bugfix: nginx














 always added "Content-Encoding: gzip" response header 















       line in 304 responses sent by ngx_http_gzip_static_module.































    *) Bugfix: the "!-x" operator did not work.















       Thanks to Maxim Dounin.































    *) Bugfix: a segmentation fault might occur in a worker














 process














, if 















       limit_rate was used in HTTPS server.















       Thanks to Maxim Dounin.































    *) Bugfix: a segmentation fault might occur in a worker














 process














 while 















       $limit_rate logging.















       Thanks to Maxim Dounin.































    *) Bugfix: nginx














 did not support dates in 2038 year on 32-bit platforms;































    *) Bugfix: nginx














/Windows tried to delete a temporary file twice if the 















       file should replace an already existent file.































    *) Bugfix: nginx














/Windows tried to rename a temporary file twice if the 















       file should replace an already existent file.































    *) Bugfix: nginx














/Windows might not create temporary file, a cache file, 















       or "proxy/fastcgi_store"d file if a worker














 had no enough access 















       rights for top level directories.































    *) Bugfix: in UTF-8 encoding support by "try_files" directive in 















       nginx














/Windows.































    *) Bugfix: UTF-8 encoding usage in the ngx_http_autoindex_module.















       Thanks to Maxim Dounin.































    *) Bugfix: the ngx_http_autoindex_module did not show the trailing 















       slash in links to a directory; the bug had appeared in 0.7.15.































    *) Bugfix: nginx














 did not close a log file set by the --error-log-path 















       configuration option; the bug had appeared in 0.7.53.































    *) Bugfix: "addition_types" directive was incorrectly named 















       "addtion_types".































    *) Bugfix: invalid request line in $request variable was written in 















       access_log only if error_log was set to "info" or "debug" level.















































Changes with nginx














 0.7.64














                                        16 Nov 2009































    *) Security: now SSL/TLS renegotiation is disabled.















       Thanks to Maxim Dounin.































    *) Bugfix: nginx














 sent gzipped responses to clients those do not support 















       gzip, if "gzip_static on" and "gzip_vary off"; the bug had appeared 















       in 0.7.63.































    *) Bugfix: if names .domain.tld, .sub.domain.tld, and .domain-some.tld 















       were defined, then the name .sub.domain.tld was matched by 















       .domain.tld.































    *) Bugfix: segmentation fault and infinite looping in resolver.































    *) Bugfix: in resolver.















       Thanks to Artem Bokhan.































    *) Bugfix: resolver cache poisoning.















       Thanks to Matthew Dempsky.































    *) Bugfix: memory leak in resolver.















       Thanks to Matthew Dempsky.