oracle 监听器(listener)启动慢的处理

1.问题描叙

启动Oracle 监听器非常慢，达2分钟之久

2.启动过程追踪

strace -f -t -o ~/listener.trc lsnrctl start

3.查看追踪log

vi ~/listener.trc

输出如下：
3564 17:47:12 recvmsg(14, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base=“\24\0\0\0\3\0\2\0\1\0\0\0\354\r\0\0\0\0\0\0”, iov_len=328}], msg_iovlen=1, msg_control=[{cmsg_len=20, cmsg_level=SOL_NETLINK, cmsg_type=0x3}, {cmsg_len=28, cmsg_level=SOL_SOCKET, cmsg_type=SCM_CREDENTIALS, cmsg_data={pid=0, uid=0, gid=0}}], msg_controllen=56, msg_flags=0}, MSG_TRUNC) = 20
3564 17:47:12 close(14)
3564 17:47:12 connect(13, {sa_family=AF_INET, sin_port=htons(199), sin_addr=inet_addr(“172.19.1.74”)}, 16) = -1 ETIMEDOUT (Connection timed out)
3564 17:49:19 times({tms_utime=2, tms_stime=5, tms_cutime=0, tms_cstime=0}) = 601764108^@3564 17:49:19 poll([{fd=8, events=POLLIN|POLLRDNORM}, {fd=11, events=POLLIN|POLLRDNORM}, {fd=12, events=POLLIN|POLLRDNORM}], 3, -1) = 1 ([{fd=8, revents=POLLIN|POLLRDNORM}])
3564 17:49:19 access(“/oracle/product/11.2.0.4/db_1/network/admin/snmp_ro.ora”, F_OK) = -1 ENOENT (No such file or directory)^@3564 17:49:19 close(13)
可以看到在17:47:12与17:49:19之间，长达2分钟等待，等待内容如下
3564 17:47:12 connect(13, {sa_family=AF_INET, sin_port=htons(199), sin_addr=inet_addr(“172.19.1.74”)}, 16) = -1 ETIMEDOUT (Connection timed out)

4.确认端口用途

可以看到199port是与snmp有关

[oracle@asics ~]$ cat /etc/services |grep 199/
smux            199/tcp                         # SNMP Unix Multiplexer
smux            199/udp

5.查看监听监听端口对应的pid

[oracle@asics ~]$ netstat -anp |grep :199
tcp        0      0 127.0.0.1:199               0.0.0.0:*                   LISTEN      -                   
tcp        4      0 127.0.0.1:13547             127.0.0.1:199               CLOSE_WAIT  10810/tnslsnr

确实是tnslsnr与199 port有连接记录，但是与199对应的pid是"-"， 网上有一段对此的解释如下：
A dash “-” is given when the kernel itself is listening, consequently there are no associated processes to show. I am not sure which kernel thread is using tcp/10080 though.

6.原因分析

Oracle listener启动过程中，尝试连接snmp port 199,此过程出现timeout, 由于这个端口是linux kernel负责启动监听，因此也无法透过尝试重启进程的方式解决问题，最终决定重启os
重启OS后，再次使用strace追踪listener启动过程

strace -f -t -o ~/listener.trc lsnrctl start

从strace log看，这次启动非常流畅：
11707 11:28:24 connect(13, {sa_family=AF_INET, sin_port=htons(199), sin_addr=inet_addr(“172.19.1.74”)}, 16) = -1 ECONNREFUSED (Connection refused)
11707 11:28:24 close(13) = 0
11707 11:28:24 access(“/oracle/product/11.2.0.4/db_1/network/admin/snmp_ro.ora”, F_OK) = -1 ENOENT (No such file or directory)

至于oracle listener启动过程为何会与snmp port连接，同样贴上来自网上的解释
SNMP refers to Simple Network Management Protocol. The SNMP status appearing in Listener status output indicates whether the Unix SNMP agent, the Peer master agent and encapsulator services are running. Oracle communicates with the Unix SNMP agents through the Oracle IntelligentAgent (dbsnmp). SNMP is a protocol used for gathering node status and task information.
简单来说就是要抓取节点信息，如os版本等