ORA-12537 TNS-12518 Process m000 died
最近客户邮件描述无法从客户端连接到数据库，其错误号为ORA-12537: TNS:connection closed，连接被关闭。直接通过tnsping没有任何问题。listener日志中出现TNS-12518: TNS:listener could not hand off client connection。即Listener无法分发客户端连接。下面是具体的错误信息、分析与解决的过程。
1、前端错误信息
C:\Program Files\VMware\VMware vSphere CLI>sqlplus robinson@sywgmr SQL*Plus: Release 10.2.0.1.0 - Production on Thu Mar 27 10:49:00 2014 Copyright (c) 1982, 2005, Oracle. All rights reserved. Enter password: ERROR: ORA-12537: TNS:connection closed
2、故障分析
oracle@SZDB:/users/oracle/OraHome10g/network/log> tail listener_sywgmr.log
TNS-12547: TNS:lost contact
TNS-12560: TNS:protocol adapter error
TNS-00517: Lost contact
Linux Error: 32: Broken pipe
27-MAR-2014 11:02:38 * (CONNECT_DATA=(SID=SYWGMR)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))) * (ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=63561)) * establish * SYWGMR * 12518
TNS-12518: TNS:listener could not hand off client connection
TNS-12547: TNS:lost contact
TNS-12560: TNS:protocol adapter error
TNS-00517: Lost contact
Linux Error: 32: Broken pipe
oracle@SZDB:~> oerr ora 12537
12537, 00000, "TNS:connection closed"
// *Cause: "End of file" condition has been reached; partner has disconnected.
// *Action: None needed; this is an information message.
oracle@SZDB:~> oerr ora 12518
12518, 00000, "TNS:listener could not hand off client connection"
// *Cause: The process of handing off a client connection to another process
// failed.
// *Action: Turn on listener tracing and re-execute the operation. Verify
// that the listener and database instance are properly configured for
// direct handoff. If problem persists, call Oracle Support.
// *Comment: The problem can be worked around by configuring dispatcher(s)
// to specifically handle the desired presentation(s), and connecting
// directly to the dispatcher, bypassing the listener.
#根据上面的2个error no，没有获得太多的有用信息
#ora-12518建议作一个trace.其comment说明可以通过配置disaptcher来解决，而当前我们的数据库使用的是dedicate方式
#关于如何配置监听器trace，可参考: http://blog.csdn.net/leshami/article/details/8254720
#此处不做trace，笔者尝试reload以及restart 监听，故障依旧
#下面查看alert log 日志
#下面的日志里表明无法创建m000进程，m000进程是SMON进程的奴隶进程
oracle@SZDB:/users/oracle> tail /u02/database/SYWGMR/bdump/alert_SYWGMR.log
Thu Mar 27 11:00:28 2014
ksvcreate: Process(m000) creation failed
Thu Mar 27 11:01:29 2014
Process m000 died, see its trace file
Thu Mar 27 11:01:29 2014
ksvcreate: Process(m000) creation failed
Thu Mar 27 11:02:30 2014
Process m000 died, see its trace file
Thu Mar 27 11:02:30 2014
ksvcreate: Process(m000) creation failed
#在06:19:02 2014时有一个Error，无法衍生job 奴隶进程
Thu Mar 27 06:19:02 2014
Process J000 died, see its trace file
Thu Mar 27 06:19:02 2014
kkjcre1p: unable to spawn jobq slave process
Thu Mar 27 06:19:02 2014
Errors in file /u02/database/SYWGMR/bdump/sywgmr_cjq0_7780.trc:
#从上面的alert log可以看出实例无法创建新的进程，推断有可能是由于超出了实例设定进程的最大值
oracle@SZDB:~> export ORACLE_SID=SYWGMR
oracle@SZDB:~> sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Thu Mar 27 10:56:27 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Release 10.2.0.3.0 - 64bit Production
-- Author : Leshami
-- Blog : http://blog.csdn.net/leshami
SQL> col RESOURCE_NAME for a20
SQL> col LIMIT_VALUE for a20
SQL> select resource_name,MAX_UTILIZATION,LIMIT_VALUE from v$resource_limit where resource_name in ('processes','sessions');
RESOURCE_NAME MAX_UTILIZATION LIMIT_VALUE
-------------------- --------------- --------------------
processes 1000 1000
sessions 1004 1105
#上面的查询结果果然如此
3、故障解决
#找到了引起故障的原因，就好处理了。下面是关于processes的描述
PROCESSES specifies the maximum number of operating system user processes that can simultaneously connect to Oracle.
Its value should allow for all background processes such as locks, job queue processes, and parallel execution processes.
The default values of the SESSIONS and TRANSACTIONS parameters are derived from this parameter.
Therefore, if you change the value of PROCESSES, you should evaluate whether to adjust the values of those derived parameters.
#修改实例的processes参数，如下，不支持memory的修改，因此需要加scope
sys> alter system set processes=1500;
alter system set processes=1500
*
ERROR at line 1:
ORA-02095: specified initialization parameter cannot be modified
sys> alter system set processes=1500 scope=spfile;
System altered.
#下面是修改并重启后的结果，如果你使用的是pfile参数，则直接修改pfile中processes的值再重启即可。
sys> show parameter processes
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
processes integer 1500
sys> show parameter session;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
sessions integer 1655
更多参考
有关Oracle RAC请参考
使用crs_setperm修改RAC资源的所有者及权限
使用crs_profile管理RAC资源配置文件
RAC 数据库的启动与关闭
再说 Oracle RAC services
Services in Oracle Database 10g
Migrate datbase from single instance to Oracle RAC
Oracle RAC 连接到指定实例
Oracle RAC 负载均衡测试(结合服务器端与客户端)
Oracle RAC 服务器端连接负载均衡(Load Balance)
Oracle RAC 客户端连接负载均衡(Load Balance)
ORACLE RAC 下非缺省端口监听配置(listener.ora tnsnames.ora)
ORACLE RAC 监听配置 (listener.ora tnsnames.ora)
配置 RAC 负载均衡与故障转移
CRS-1006 , CRS-0215 故障一例
基于Linux (RHEL 5.5) 安装Oracle 10g RAC
使用 runcluvfy 校验Oracle RAC安装环境
有关Oracle 网络配置相关基础以及概念性的问题请参考：
配置非默认端口的动态服务注册
配置sqlnet.ora限制IP访问Oracle
Oracle 监听器日志配置与管理
设置 Oracle 监听器密码(LISTENER)
配置ORACLE 客户端连接到数据库
有关基于用户管理的备份和备份恢复的概念请参考
Oracle 冷备份
Oracle 热备份
Oracle 备份恢复概念
Oracle 实例恢复
Oracle 基于用户管理恢复的处理
SYSTEM 表空间管理及备份恢复
SYSAUX表空间管理及恢复
Oracle 基于备份控制文件的恢复(unsing backup controlfile)
有关RMAN的备份恢复与管理请参考
RMAN 概述及其体系结构
RMAN 配置、监控与管理
RMAN 备份详解
RMAN 还原与恢复
RMAN catalog 的创建和使用
基于catalog 创建RMAN存储脚本
基于catalog 的RMAN 备份与恢复
RMAN 备份路径困惑
使用RMAN实现异机备份恢复(WIN平台)
使用RMAN迁移文件系统数据库到ASM
linux 下RMAN备份shell脚本
使用RMAN迁移数据库到异机
有关ORACLE体系结构请参考
Oracle 表空间与数据文件
Oracle 密码文件
Oracle 参数文件
Oracle 联机重做日志文件(ONLINE LOG FILE)
Oracle 控制文件(CONTROLFILE)
Oracle 归档日志
Oracle 回滚(ROLLBACK)和撤销(UNDO)
Oracle 数据库实例启动关闭过程
Oracle 10g SGA 的自动化管理
Oracle 实例和Oracle数据库(Oracle体系结构)