Oracle 网络问题错误日志典型格式

1.网络超时

alert日志中会出现如下提示：

SQLNET.LOG

Fatal NI connect error 12170.

VERSION INFORMATION:
TNS for 64-bit Windows: Version 10.2.0.4.0 - Production
Oracle Bequeath NT Protocol Adapter for 64-bit Windows: Version 10.2.0.4.0 - Production
Windows NT TCP/IP NT Protocol Adapter for 64-bit Windows: Version 10.2.0.4.0 - Production
Time: 10-DEC-2011 17:26:39
Tracing not turned on.
Tns error struct:
ns main err code: 12535
TNS-12535: TNS:operation timed out
ns secondary err code: 12560
nt main err code: 505
TNS-00505: Operation timed out
nt secondary err code: 60
nt OS err code: 0
Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=10.x.xx.yy)(PORT=52180))

LISTENER LOG

10-DEC-2011 12:32:16 * (CONNECT_DATA=(SID=redc)(CID=(PROGRAM=C:\Program Files\Quest Software\TOAD\TOAD.exe)(HOST=server1)(USER=srcat110786))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.x.xx.yy)(PORT=52180)) * establish * redc * 0

o 12170 cross referenced from SQLNET.LOG to LISTENER.LOG via IP ADDRESS and PORT number
o Listener accepted connection 12:32:16
o Connection timed out         17:26:39
o Lowest error code in the stack none oracle >> nt secondary err code: 60
(see errno.h file)

+ Device on network / TCPIP settings areas to look into. Might have to wireshark to prove.

2.数据库超时

alert日志中会出现如下提示：

SQLNET.LOG

Fatal NI connect error 12170.

VERSION INFORMATION:
TNS for 64-bit Windows: Version 10.2.0.4.0 - Production
Oracle Bequeath NT Protocol Adapter for 64-bit Windows: Version 10.2.0.4.0 - Production
Windows NT TCP/IP NT Protocol Adapter for 64-bit Windows: Version 10.2.0.4.0 - Production
Time: 11-DEC-2011 09:11:53
Tracing not turned on.
Tns error struct:
ns main err code: 12535
TNS-12535: TNS:operation timed out
ns secondary err code: 12606
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=10.x.yy.zzz)(PORT=63416))

LISTENER LOG

11-DEC-2011 09:10:52 * (CONNECT_DATA=(SID=redc)(CID=(PROGRAM=C:\orant\bin\ifrun60.exe)(HOST=CTX2ND)(USER=srca5026))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.x.yy.zzz)(PORT=63416)) * establish * redc * 0

o 12170 cross referenced from SQLNET.LOG to LISTENER.LOG via IP ADDRESS and PORT number
o Listener accepted connection 09:10:52
o Connection timed out         09:11:53
o Error codes stack all Oracle, no nt main and secondary errors reported for OS level
o 60 seconds timeout to the database, check INBOUND_CONNECT_TIMEOUT in database SQLNET.ORA file (Default is 60 seconds for some versions)

+ Check what the database is doing, level 16 client and server trace will help.

3.监听关闭或者错误

日志格式一般如下：

Listener Log

Error listening on: (ADDRESS=(PROTOCOL=ipc)(PARTIAL=yes)(QUEUESIZE=1))
21-NOV-2011 21:54:03 * establish * 1159
TNS-01159: Internal connection limit has been reached; listener has shut down
TNS-12540: TNS:internal limit restriction exceeded
No longer listening on:
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.x.xxx.yy)(PORT=1521)))
No longer listening on:
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.x.xxx.yy)(PORT=1521)))
No longer listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))

First 12518 seen

21-NOV-2011 21:51:47 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin
 Client)(HOST=__jdbc__)(USER=unicaprod))(SERVER=DEDICATED)(SERVICE_NAME=ORCL.uk.oracle.com)) *
(ADDRESS=(PROTOCOL=tcp)(HOST=10.x.yyy.zz)(PORT=47727)) * establish *
ORCL.uk.oracle.com * 12518

Listener Trace

Tracking back to around 21:51 and all we see in connect packets, no resend,redirect or refuse packets.

But massive time gap and very large number of waiting connections, 125 waiting connections.

[000123 21-NOV-2011 21:51:16:316] nsevwait: 125 registered connection(s)
[000123 21-NOV-2011 21:51:16:316] nsevwait: 0 pre-posted event(s)
[000123 21-NOV-2011 21:51:16:316] nsevwait: waiting for transport event (1 thru 330)...
[000123 21-NOV-2011 21:51:50:048] nsevwait: 1 newly-posted event(s)
[000123 21-NOV-2011 21:51:50:048] nsevfnt: cxd: 0x1bbf2c60 stage 0: NT events set:

Quick track back through the trace, grep on registered connection(s)and we see connections going from single figure

[000120 21-NOV-2011 19:38:09:491] nsevwait: 7 registered connection(s)
[000120 21-NOV-2011 19:38:10:503] nsevwait: 8 registered connection(s)
[000120 21-NOV-2011 19:38:10:516] nsevwait: 9 registered connection(s)

to double

[000123 21-NOV-2011 21:22:59:899] nsevwait: 47 registered connection(s)
[000123 21-NOV-2011 21:23:01:241] nsevwait: 47 registered connection(s)
[000123 21-NOV-2011 21:23:01:242] nsevwait: 47 registered connection(s)

Looking at a connection at time connection rate is high, and we see connect Packet from client coming in

[000123 21-NOV-2011 21:23:48:649] nscon: doing connect handshake...
[000123 21-NOV-2011 21:23:48:649] nscon: recving a packet

This is the 38 connection in Q right now

[000123 21-NOV-2011 21:23:48:662] nsevreg: begin registration process for 38

Now the listener continues to get other connections in.

[000123 21-NOV-2011 21:23:51:300] nscon: doing connect handshake...
[000123 21-NOV-2011 21:23:51:300] nscon: recving a packet

[000123 21-NOV-2011 21:23:51:301] nsprecv: normal exit
[000123 21-NOV-2011 21:23:51:301] nscon: got NSPTCN packet

We see the backlog q grow, upto 40 now

[000123 21-NOV-2011 21:23:51:313] nsevreg: begin registration process for 40

Back to that 38 connection in the Q

Resend packet to the client 21:28

[000123 21-NOV-2011 21:28:32:522] nscon: sending NSPTRS packet
[000123 21-NOV-2011 21:28:32:522] nspsend: entry
[000123 21-NOV-2011 21:28:32:522] nspsend: plen=8, type=11
[000123 21-NOV-2011 21:28:32:522] nttwr: entry
[000123 21-NOV-2011 21:28:32:523] nttwr: socket 67 had bytes written=8
[000123 21-NOV-2011 21:28:32:523] nttwr: exit
[000123 21-NOV-2011 21:28:32:523] nspsend: packet dump
[000123 21-NOV-2011 21:28:32:523] nspsend: 00 08 00 00 0B 00 00 00 
|........|
[000123 21-NOV-2011 21:28:32:523] nspsend: 8 bytes to transport

Connection removed from the listener q here:

[000123 21-NOV-2011 21:28:32:523] nsevunreg: entry
[000123 21-NOV-2011 21:28:32:523] nsevunreg: cid=38, sgt=0, rdm=0

Client connections coming into the TNS listener start to slow down.
This starts a chain reaction of a backlog of connections building up.
This backlog hits internal restriction of waiting connections and the
listener shuts down