一、错误现象与环境
前端应用程序运行时出现下面的错误提示：
事件添加失败:ORA-02409;超时：分布式事务处理等待锁定
ORA-02063:紧接着line(源于ITSPFDB.US.ORACLE.COM)
该应用程序后台对应的数据库为db01(oraclce 8)，使用了db link到数据库itspfdb(oracle 9)
下面的SQL信息来自Oracle 8的数据库
SQL> select * from v$version; BANNER ---------------------------------------------------------------- Oracle8i Enterprise Edition Release 8.1.7.4.1 - Production PL/SQL Release 8.1.7.4.0 - Production CORE 8.1.7.2.1 Production TNS for 32-bit Windows: Version 8.1.7.4.0 - Production NLSRTL Version 3.4.1.0.0 - Production SQL> select * from dba_db_links; OWNER DB_LINK USERNAME HOST CREATED --------------- ------------------------------ --------------- -------------------- ----------- PUBLIC ITSPFDB.US.ORACLE.COM HECH itspfdb 2008-7-3 18 SQL> select * from v$dblink; DB_LINK OWNER_ID LOGGED_ON HETEROGENEOUS PROTOCOL OPEN_CURSORS IN_TRANSACTION UPDATE_SENT COMMIT_POINT_STRENGTH --------------------- -------- --------- ------------- -------- ------------ -------------- ----------- --------------------- ITSPFDB.US.ORACLE.COM 0 YES YES UNKN 0 YES NO 1 创建了同义词 SQL> col TABLE_OWNER format a15 SQL> select * from dba_synonyms where db_link is not null; OWNER SYNONYM_NAME TABLE_OWNER TABLE_NAME DB_LINK --------------- ------------------------------ --------------- ------------------------------ ------------------------------ HECH T_EVENT_HECH511 ITSPF T_EVENT_HECH511 ITSPFDB.US.ORACLE.COM二、alert log file与dump信息
错误日志出现在Oracle 8中，如下：
Errors in file D:\oracle\admin\DB01\udump\ORA03992.TRC:
ORA-02049: 超时: 分布式事务处理等待锁定
ORA-02063: 紧接着line（源于ITSPFDB.US.ORACLE.COM）
Mon Jul 11 10:24:13 2011
Errors in file D:\oracle\admin\DB01\udump\ORA01084.TRC:
ORA-02049: 超时: 分布式事务处理等待锁定
ORA-02063: 紧接着line（源于ITSPFDB.US.ORACLE.COM）
dump文件信息：
Dump file D:\oracle\admin\DB01\udump\ORA03992.TRC
Mon Jul 11 10:23:10 2011
ORACLE V8.1.7.4.1 - Production vsnsta=0
vsnsql=f vsnxtr=3
Windows 2000 Version 5.2 , CPU type 586
Oracle8i Enterprise Edition Release 8.1.7.4.1 - Production
With the Partitioning option
JServer Release 8.1.7.4.1 - Production
Windows 2000 Version 5.2 , CPU type 586
Instance name: db01
Redo thread mounted by this instance: 1
Oracle process number: 21
Windows thread id: 3992, image: ORACLE.EXE
*** 2011-07-11 10:23:10.313
*** SESSION ID83.24650) 2011-07-11 10:23:10.266
FATAL ERROR IN TWO-TASK SERVER: error = 12571
*** 2011-07-11 10:23:10.313
ksedmp: internal or fatal error
ORA-02049: 超时: 分布式事务处理等待锁定
ORA-02063: 紧接着line（源于ITSPFDB.US.ORACLE.COM）
Current SQL statement for this session:
insert into T_EVENT_HECH511(id,event_id, start_unit_id, end_unit_id, event_level, state, event_type, start_date,
end_date, acc_id, event_source, summary, remark, event_reason, region_no, modify_date, acc_level)
values(73544,'110711100028',1354,1371,0,0,001,to_date('2011-7-11 10:22:01','yyyy-mm-dd hh24:mi:ss')
,null,null,2,'xxxx yyyy','','0','11',sysdate,null)
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
_ksedmp+a8 CALLrel _ksedst+0
8EE474
_opitsk+f4a CALLrel _ksedmp+0 2
_opiino+50c CALLrel _opitsk+0 0
_opiodr+507 CALLreg 00000000 3C 4
D8EFBFC
_opidrv+384 CALLrel _opiodr+0 3C 4
D8EFBFC
0
_sou2o+19 CALLrel _opidrv+0
_opimai+10c CALLrel _sou2o+0
_OracleThreadStart@4+4b8 CALLrel _opimai+0 2
D8EFE78
77E1A98D CALLreg 00000000
----- Argument/Register Address Dump -----
Argument/Register addr=d8ee474.
Dump of memory from 0xD8EE434 to 0xD8EE574
D8EE430 00000003 00000000 6552211C [.........!Re]
D8EE440 00000033 00000000 FFFFFFFD 00000048 [3...........H...]
D8EE450 00000000 00000000 00000000 0D001758 [............X...]
D8EE460 00000000 0000001D 0D8EE480 0041E6E4 [..............A.]
D8EE470 00000001 00E3F1BC 00E3F1BC 00E3F1BC [................]
D8EE480 00E3F1C4 00E3F1BC 00E3F1BC 00E3F1BC [................]
D8EE490 00E3F1BC 00E3F1C4 00000000 00000000 [................]
D8EE4A0 00000000 0D001758 0D8EE55C 0041F5F9 [....X...\.....A.]
D8EE4B0 0D8EE4C0 0000001C 0D001758 00000002 [........X.......]
D8EE4C0 2A2A2A0A 31303220 37302D31 2031312D [.*** 2011-07-11 ]
D8EE4D0 323A3031 30313A33 32332D25 2D252073 [10:23:10%-32s %-]
D8EE4E0 25207338 7332332D 00000020 0D8EE504 [8s %-32s .......]
D8EE4F0 60B15218 0D410030 45384420 38374546 [.R.`0.A. D8EFE78]
D8EE500 25207300 7332332D 342D2520 0D000A73 [.s %-32s %-4s...]
D8EE510 0000311B 655221B4 0D000033 00000000 [.1...!Re3.......]
D8EE520 00000000 00000000 00000000 00000000 [................]
D8EE530 00000000 0D8EFFEC 00000000 0D8EFE78 [............x...]
D8EE540 00000000 00000001 00000001 00000000 [................]
D8EE550 0D8EDCBC 00000003 0D8EE580 004844CD [.............DH.]
D8EE560 0D8EE498 00000000 0000311B 0D0043F0 [.........1...C..]
D8EE570 00000000 [....]
Argument/Register addr=d8efbfc.
Dump of memory from 0xD8EFBBC to 0xD8EFCFC
D8EFBB0 00000001 [....]
D8EFBC0 77F32AC3 77E2AAA3 0D414AFC 60840538 [.*.w...w.JA.8..`]
D8EFBD0 00000000 00000000 00000000 0D8EFBEC [................]
D8EFBE0 00000000 00000000 00000000 00000000 [................]
D8EFBF0 7FF93C00 0D8EFC14 77E2AB09 0D8EFD83 [.<.........w....]
D8EFC00 FFFFFFFF 00000000 00000001 00000000 [................]
D8EFC10 77F3331F 77F32397 0D8EFFB8 0040318D [.3.w.#.w.....1@.]
D8EFC20 00000002 0D8EFE78 00000000 00000000 [....x...........]
D8EFC30 00E79084 0D007188 77FB2340 77F4D0E7 [.....q..@#.w...w]
D8EFC40 00000000 7FFDF000 00000000 00000024 [............$...]
D8EFC50 00000001 00000000 44414853 00000000 [........SHAD....]
D8EFC60 FFFF0000 00000002 01208F98 00000000 [.......... .....]
D8EFC70 0000175C 00000001 00000000 00000000 [\...............]
D8EFC80 00000070 FFFFFFFF FFFFFFFF 77F4D045 [p...........E..w]
D8EFC90 77F4D10F 00246E68 71A81570 7FFDF000 [...whn$.p..q....]
D8EFCA0 77FB23B4 0D8EFC40 00000000 0D8EFD0C [.#.w@...........]
D8EFCB0 77F35A34 77F7C7B0 FFFFFFFF 77F4D0E7 [4Z.w...w.......w]
D8EFCC0 77F3363F 77F4CEE5 0D8EFD30 00000000 [?6.w...w0.......]
D8EFCD0 00E79084 00000000 00000000 00000000 [................]
D8EFCE0 00000000 00000000 00000000 00000000 [................]
D8EFCF0 00000000 00000000 7FFDF000 [............]
Argument/Register addr=d8efe78.
Dump of memory from 0xD8EFE38 to 0xD8EFF78
D8EFE30 F6C0AB75 E16EACF0 [u.....n.]
D8EFE40 82695620 00000000 F6C0AAF4 F6C0AB30 [ Vi.........0...]
D8EFE50 80585240 805691F3 FF8D730C 81A31A08 [@RX...V..s......]
D8EFE60 8132F238 C0015B3C 055CF000 00000000 [8.2.<[....\.....]
D8EFE70 00000005 000001B3 0D8EFD78 0D8EFD83 [........x.......]
D8EFE80 00000004 00000000 00000004 61726F2A [............*ora]
D8EFE90 77617073 65725F6E 5F796C70 31306264 [spawn_reply_db01]
D8EFEA0 3939335F 00002A32 F873F120 804E8905 [_3992*.. .s...N.]
D8EFEB0 00000246 C01FFE08 00000002 80500F2E [F.............P.]
D8EFEC0 00000002 80500EE0 00000000 7FF81000 [......P.........]
D8EFED0 804F0CB0 F6C0AB9C 00BD7867 7FF81000 [..O.....gx......]
D8EFEE0 02F81000 F6C0ABE8 804F7C80 00000000 [.........|O.....]
D8EFEF0 FFDFF548 00BD7867 804FBDFC 81A31C20 [H...gx....O. ...]
D8EFF00 81A31A08 81A31AD8 00000000 7FF81000 [................]
D8EFF10 820521D0 055D0000 00000000 81907800 [.!....]......x..]
D8EFF20 00000000 81A31AD8 FFFFFFFF 00000000 [................]
D8EFF30 804E7E36 F6C0ABFC 8147B680 F873F120 [6~N.......G. .s.]
D8EFF40 F873F900 804EF800 8147B720 8147B680 [..s...N. .G...G.]
D8EFF50 8147B810 00000001 8147B680 00000001 [..G.......G.....]
D8EFF60 00000000 00000000 F6C0AC30 804EFA20 [........0... .N.]
D8EFF70 8147B680 8147B7E0 [..G...G.]
----- End of Call Stack Trace -----
===================================================
Files currently opened by this process:
===================================================
PROCESS STATE
-------------
Process global information:
process: 23707e4, call: 0, xact: 268e070, curses: 0, usrses: 23b8c40
----------------------------------------
SO: 23707e4, type: 1, owner: 0, pt: 0, flag: INIT/-/-/0x00
(process) Oracle pid=21, calls cur/top: 0/3da179c, flag: (0) -
int error: 0, call error: 0, sess error: 0, txn error 0
(post info) last post received: 85 0 4
last post received-location: kslpsr
last process to post me: 236d23c 1 2
last post sent: 0 0 13
last post sent-location: ksasnd
last process posted by me: 236d23c 1 2
(latch info) wait_event=0 bits=0
Process Group: DEFAULT, pseudo proc: 2389f20
O/S info: user: SYSTEM, term: TESTCOMPUTER, ospid: 3992
OSD pid info: 3992
----------------------------------------
SO: 23b8c40, type: 3, owner: 23707e4, pt: 0, flag: INIT/-/-/0x00
(session) trans: 268e070, creator: 23707e4, flag: (8000041) USR/- -/-/-/-/-/-
DID: 0001-0015-00000025, short-term DID: 0000-0000-00000000
txn branch: 26afb2c
oct: 0, prv: 0, user: 43/HECH
O/S info: user: Administrator, term: CHINA-175944CD0, ospid: 2100:3876, machine: WORKGROUP\CHINA-175944CD0
program: VB6.EXE
last wait for 'SQL*Net break/reset to client' blocking sess=0x0 seq=827 wait_time=-2
driver id=54435000, break?=1, =0
----------------------------------------
SO: 27efcfc, type: 36, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT PIN: pin=27efcfc handle=0 lock=27e3dd4
user=23b8c40 session=23b8c40 count=0 mask=0000 savepoint=105 flags=[00]
----------------------------------------
SO: 27e3dd4, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27e3dd4 handle=2ee8d28 mode=N
call pin=0 session pin=27efcfc
user=23b8c40 session=23b8c40 count=1 flags=[00] savepoint=103
LIBRARY OBJECT HANDLE: handle=2ee8d28
namespace=CRSR flags=RON/KGHP/PN0/[10010000]
kkkk-dddd-llll=0000-0041-0041 lock=N pin=0 latch=2
lwt=2ee8d40[2ee8d40,2ee8d40] ltm=2ee8d48[2ee8d48,2ee8d48]
pwt=2ee8d58[2ee8d58,2ee8d58] ptm=2ee8db0[2ee8db0,2ee8db0]
ref=2ee8d30[4070294,4070294] lnd=2ee8dbc[2ee8dbc,2ee8dbc]
LIBRARY OBJECT: object=2be4a38
type=CRSR flags=EXS[0001] pflags= [00] status=VALD load=0
DEPENDENCIES: count=1 size=16
TRANSLATIONS: count=1 size=16
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 4147cd0 2dfbdf4 I/P/A 0 NONE
6 2be4acc 290e0f4 I/-/A 0 NONE
----------------------------------------
SO: 27f11b4, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27f11b4 handle=2e72ac0 mode=N
call pin=27e5a60 session pin=0
user=23b8c40 session=23b8c40 count=1 flags=[00] savepoint=102
LIBRARY OBJECT HANDLE: handle=2e72ac0
name=
insert into T_EVENT_HECH511(id,event_id, start_unit_id, end_unit_id, event_level, state, event_type, start_date,
end_date, acc_id, event_source, summary, remark, event_reason, region_no, modify_date, acc_level)
values(73544,'110711100028',1354,1371,0,0,001,to_date('2011-7-11 10:22:01','yyyy-mm-dd hh24:mi:ss'),
null,null,2,'xxxx yyyy','','0','11',sysdate,null)
hash=3ee5c0de timestamp=07-11-2011 10:22:08
namespace=CRSR flags=RON/KGHP/TIM/PN0/LRG/[10010001]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch=2
lwt=2e72ad8[2e72ad8,2e72ad8] ltm=2e72ae0[2e72ae0,2e72ae0]
pwt=2e72af0[2e72af0,2e72af0] ptm=2e72b48[2e72b48,2e72b48]
ref=2e72ac8[2e72ac8,2e72ac8] lnd=2e72b54[2e72b54,2e72b54]
LIBRARY OBJECT: object=407010c
type=CRSR flags=EXS[0001] pflags= [00] status=VALD load=0
CHILDREN: size=16
child# table reference handle
------ -------- --------- --------
0 40702c8 4070294 2ee8d28
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 3142c14 4070190 I/P/A 0 NONE
----------------------------------------
SO: 27e57d8, type: 36, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT PIN: pin=27e57d8 handle=0 lock=27f0db8
user=23b8c40 session=23b8c40 count=0 mask=0000 savepoint=52 flags=[00]
----------------------------------------
SO: 27f0db8, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27f0db8 handle=2d9063c mode=N
call pin=0 session pin=27e57d8
user=23b8c40 session=23b8c40 count=1 flags=[00] savepoint=52
LIBRARY OBJECT HANDLE: handle=2d9063c
namespace=CRSR flags=RON/KGHP/PN0/[10010000]
kkkk-dddd-llll=0000-0041-0041 lock=N pin=0 latch=2
lwt=2d90654[2d90654,2d90654] ltm=2d9065c[2d9065c,2d9065c]
pwt=2d9066c[2d9066c,2d9066c] ptm=2d906c4[2d906c4,2d906c4]
ref=2d90644[34cde88,34cde88] lnd=2d906d0[2d906d0,2d906d0]
LIBRARY OBJECT: object=30cdcd4
type=CRSR flags=EXS[0001] pflags= [00] status=VALD load=0
DEPENDENCIES: count=3 size=16
AUTHORIZATIONS: count=1 size=16 minimum entrysize=16
ACCESSES: count=1 size=16
TRANSLATIONS: count=1 size=16
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 3c48610 3be60e4 I/P/A 0 NONE
6 30cdd68 38f2bc8 I/-/A 0 NONE
----------------------------------------
SO: 27e4d4c, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27e4d4c handle=3c827f4 mode=N
call pin=27efbdc session pin=0
user=23b8c40 session=23b8c40 count=1 flags=[00] savepoint=51
LIBRARY OBJECT HANDLE: handle=3c827f4
name=select to_char(sysdate,'yyyy-mm-dd hh24:mi:ss') from dual
hash=58512020 timestamp=07-05-2011 07:30:48
namespace=CRSR flags=RON/KGHP/TIM/PN0/SML/[12010000]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch=2
lwt=3c8280c[3c8280c,3c8280c] ltm=3c82814[3c82814,3c82814]
pwt=3c82824[3c82824,3c82824] ptm=3c8287c[3c8287c,3c8287c]
ref=3c827fc[3c827fc,3c827fc] lnd=3c82888[3c82888,3c82888]
LIBRARY OBJECT: object=28cd708
type=CRSR flags=EXS[0001] pflags= [00] status=VALD load=0
CHILDREN: size=16
child# table reference handle
------ -------- --------- --------
0 28cd798 34cde88 2d9063c
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 432818c 3fa8550 I/P/A 0 NONE
----------------------------------------
SO: 27f29c0, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27f29c0 handle=2ff5230 mode=N
call pin=27f8f3c session pin=0
user=23b8c40 session=23b8c40 count=1 flags=[00] savepoint=26
LIBRARY OBJECT HANDLE: handle=2ff5230
name=HECH.SJ_T_HECH_EVENT_ID
hash=3353c42f timestamp=07-16-2008 16:10:12
namespace=TABL/PRCD/TYPE flags=KGHP/TIM/SML/[02000000]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch=1
lwt=2ff5248[2ff5248,2ff5248] ltm=2ff5250[2ff5250,2ff5250]
pwt=2ff5260[2ff5260,2ff5260] ptm=2ff52b8[2ff52b8,2ff52b8]
ref=2ff5238[2ff5238,2ff5238] lnd=2ff52c4[2e101c8,3ca2508]
LIBRARY OBJECT: object=3114000
type=SQNC flags=EXS/LOC[0005] pflags= [00] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 31650d8 3114084 I/-/A 0 NONE
----------------------------------------
SO: 3500484, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=3500484 handle=3a872e4 mode=N
call pin=27f24bc session pin=0
user=23b8c40 session=23b8c40 count=1 flags=[00] savepoint=16
LIBRARY OBJECT HANDLE: handle=3a872e4
name=HECH.SJ_T_HECH_EVENT_EVENTID
hash=16d2ed55 timestamp=07-11-2011 00:00:03
namespace=TABL/PRCD/TYPE flags=KGHP/TIM/SML/[02000000]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch=1
lwt=3a872fc[3a872fc,3a872fc] ltm=3a87304[3a87304,3a87304]
pwt=3a87314[3a87314,3a87314] ptm=3a8736c[3a8736c,3a8736c]
ref=3a872ec[3a872ec,3a872ec] lnd=3a87378[3c807b8,40a7a50]
LIBRARY OBJECT: object=339b2f8
type=SQNC flags=EXS/LOC[0005] pflags= [00] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 3402d74 339b37c I/-/A 0 NONE
----------------------------------------
SO: 26afb2c, type: 26, owner: 23b8c40, pt: 0, flag: INIT/-/-/0x00
(branch) trn = 0x268e070, flg = 0x02, state = 0x00 bno=1 ser=6
creator = 0x23b8c40 uid = 0x2b serial# = 24650
DB01.AIX.dfc38ba4.3.62.40474
----------------------------------------
SO: 268e070, type: 25, owner: 26afb2c, pt: 0, flag: INIT/-/-/0x00
(trans) bsn = 3, flg = 0x401e03, flg2 = 0x80, prx = 0x0, ros = 2147483647, spn = 105
efd = 10
parent xid: 0x0000.000.00000000
env: (scn: 0x0b31.6691d413 xid: 0x0003.03e.00009e1a uba: 0x00800407.9127.1b)
cev: (spc = 4792 usi = 3 ubk tsn: 1 rdba: 0x00800407 useg tsn: 1 rdba: 0x00800402
hwm uba: 0x00800407.9127.1b col uba: 0x00000000.0000.00
num bl: 1 bk list: 0x2673154)
(gtrid 499) DB01.AIX.dfc38ba4.3.62.40474
#branches = 1, type = 0x2
branches = 26afb2c, 0, 0, 0, 0, 0, 0, 0
transactions = 268e070, 0, 0, 0, 0, 0, 0, 0
branch#0 #refs = 1, #prep'd = 0, flags = 0x0, npi = N
(enqueue) TX-0003003E-00009E1A DID: 0001-0015-00000025
lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
res: 244a3e4, mode: X, prv: 244a3ec, sess: 23b8c40, proc: 23707e4
xga: 0x0, heap: UGA
----------------------------------------
SO: 265d584, type: 22, owner: 268e070, pt: 0, flag: INIT/-/-/0x00
DML LOCK: tab=24946 flg=01 chi=0
his[0]: mod=3 spn=92
(enqueue) TM-00006172-00000000 DID: 0001-0015-00000025
lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
res: 244a15c, mode: SX, prv: 265da50, sess: 23b8c40, proc: 23707e4
----------------------------------------
SO: 2673154, type: 24, owner: 268e070, pt: 0, flag: -/-/-/0x00
(List of Blocks) next index = 7
index savepoint itli buffer hint
--------------------------------------
0 71 3 0x 251cf50
1 74 2 0x 24b8ea8
2 76 2 0x 24d1eb0
3 78 2 0x 2500340
4 80 3 0x 257ea40
5 82 2 0x 2542690
6 95 3 0x 2562e20
----------------------------------------
SO: 265d494, type: 22, owner: 268e070, pt: 0, flag: INIT/-/-/0x00
DML LOCK: tab=25111 flg=01 chi=0
his[0]: mod=3 spn=68
(enqueue) TM-00006217-00000000 DID: 0001-0015-00000025
lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
res: 244a42c, mode: SX, prv: 265d618, sess: 23b8c40, proc: 23707e4
----------------------------------------
SO: 27eff84, type: 36, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT PIN: pin=27eff84 handle=422c49c mode=S lock=27f0f98
user=23b8c40 session=23b8c40 count=1 mask=0041 savepoint=2 flags=[00]
----------------------------------------
SO: 27f0f98, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27f0f98 handle=422c49c mode=N
call pin=0 session pin=27eff84
user=23b8c40 session=23b8c40 count=1 flags=PNS/[08] savepoint=2
LIBRARY OBJECT HANDLE: handle=422c49c
namespace=CRSR flags=RON/KGHP/PN0/[10010000]
kkkk-dddd-llll=0000-0041-0041 lock=N pin=S latch=2
lwt=422c4b4[422c4b4,422c4b4] ltm=422c4bc[422c4bc,422c4bc]
pwt=422c4cc[422c4cc,422c4cc] ptm=422c524[422c524,422c524]
ref=422c4a4[3971fc4,3971fc4] lnd=422c530[422c530,422c530]
LIBRARY OBJECT: object=38cb388
type=CRSR flags=EXS[0001] pflags= [00] status=VALD load=0
DEPENDENCIES: count=2 size=16
AUTHORIZATIONS: count=1 size=16 minimum entrysize=16
ACCESSES: count=1 size=16
TRANSLATIONS: count=2 size=16
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 2f96fe4 28895e8 I/P/A 0 NONE
6 38cb41c 4235720 I/P/A 1 NONE
----------------------------------------
SO: 27f131c, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27f131c handle=3c20354 mode=N
call pin=27ef75c session pin=0
user=23b8c40 session=23b8c40 count=1 flags=[00] savepoint=1
LIBRARY OBJECT HANDLE: handle=3c20354
name=
select region_name from t_region_info where region_id =
(select region_id from t_unit_info_branch where unit_id = '1354' and rownum=1)
hash=44f1d7ab timestamp=07-11-2011 09:07:18
namespace=CRSR flags=RON/KGHP/TIM/PN0/MED/[50010000]
kkkk-dddd-llll=0000-0001-0001 lock=N pin=0 latch=2
lwt=3c2036c[3c2036c,3c2036c] ltm=3c20374[3c20374,3c20374]
pwt=3c20384[3c20384,3c20384] ptm=3c203dc[3c203dc,3c203dc]
ref=3c2035c[3c2035c,3c2035c] lnd=3c203e8[3c203e8,3c203e8]
LIBRARY OBJECT: object=2a71a8c
type=CRSR flags=EXS[0001] pflags= [00] status=VALD load=0
CHILDREN: size=16
child# table reference handle
------ -------- --------- --------
0 3972190 3971fc4 422c49c
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 38dee34 2a71b10 I/P/A 0 NONE
----------------------------------------
SO: 350022c, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=350022c handle=42f5cf0 mode=N
call pin=27f8d8c session pin=0
user=23b8c40 session=23b8c40 count=1 flags=CBB/[02] savepoint=23
LIBRARY OBJECT HANDLE: handle=42f5cf0
name=SYS.DATABASE
hash=aea96fa0 timestamp=12-31-4712 23:59:59
namespace=EVNT flags=KGHP/TIM/SML/[02000000]
kkkk-dddd-llll=0000-0009-0009 lock=N pin=0 latch=1
lwt=42f5d08[42f5d08,42f5d08] ltm=42f5d10[42f5d10,42f5d10]
pwt=42f5d20[42f5d20,42f5d20] ptm=42f5d78[42f5d78,42f5d78]
ref=42f5cf8[42f5cf8,42f5cf8] lnd=42f5d84[42f5d84,42f5d84]
LIBRARY OBJECT: object=3254378
type=EVNT flags=EXS/LOC[0005] pflags= [00] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 310a4a0 325445c I/-/A 0 NONE
3 325440c 3d89b54 I/-/A 0 NONE
----------------------------------------
SO: 27f2df8, type: 35, owner: 23b8c40, flag: INIT/-/-/0x00
LIBRARY OBJECT LOCK: lock=27f2df8 handle=40abcf0 mode=N
call pin=27f14fc session pin=0
user=23b8c40 session=23b8c40 count=1 flags=CBB/[02] savepoint=23
LIBRARY OBJECT HANDLE: handle=40abcf0
name=HECH.HECH
hash=7677ed7 timestamp=12-31-4712 23:59:59
namespace=EVNT flags=KGHP/TIM/SML/[02000000]
kkkk-dddd-llll=0000-0009-0009 lock=N pin=0 latch=1
lwt=40abd08[40abd08,40abd08] ltm=40abd10[40abd10,40abd10]
pwt=40abd20[40abd20,40abd20] ptm=40abd78[40abd78,40abd78]
ref=40abcf8[40abcf8,40abcf8] lnd=40abd84[40abd84,40abd84]
LIBRARY OBJECT: object=2ab2cf4
type=EVNT flags=EXS/LOC[0005] pflags= [00] status=VALD load=0
DATA BLOCKS:
data# heap pointer status pins change
----- -------- -------- ------ ---- ------
0 41b89c8 2ab2dd8 I/-/A 0 NONE
3 2ab2d88 31ef814 I/-/A 0 NONE
----------------------------------------
SO: 326c1d8, type: 34, owner: 23b8c40, pt: 0, flag: INIT/-/-/0x00
user lock: lock=326c1d8 mode=S
user resource: user=27fc940 uid=43 mode=S
----------------------------------------
SO: 3d415a0, type: 34, owner: 23b8c40, pt: 0, flag: INIT/-/-/0x00
user lock: lock=3d415a0 mode=S
user resource: user=27fc940 uid=43 mode=S
----------------------------------------
SO: 2472308, type: 9, owner: 23707e4, pt: 0, flag: INIT/-/-/0x00
(broadcast handle) flag: (2) ACTIVE SUBSCRIBER, owner: 23707e4,
event: 43885, last message event: 43885, messages read: 0
channel: (2473f60) system events broadcast channel
scope: 101, event: 43889, last mesage event: 0,
publishers/subscribers: 0/28,
messages published: 0
----------------------------------------
SO: 3da179c, type: 2, owner: 23707e4, pt: 0, flag: INIT/-/-/0x00
(call) sess: cur 23b8c40, rec 0, usr 23b8c40; depth: 0
END OF PROCESS STATE
三、分析与解决
ORA-02049:
timeout:distributed transaction waiting for lock
cause:exceeded INIT.ORA distributed_lock_timeout seconds waiting for lock.
action:treat as a deadlock.
下面的ID 576929.1来自metalink
Oracle Trace Error:The 'FATAL ERROR IN TWO-TASK SERVER: error =12571' [ID 576929.1]
--------------------------------------------------------------------------------
Modified 22-APR-2010 Type HOWTO Status ARCHIVED
In this Document
Goal
Solution
--------------------------------------------------------------------------------
Applies to:
Oracle Agile Application Administration
Information in this document applies to any platform.
***Checked for relevance on 22-April-2010***
Oracle 8.1.7
Goal
Oracle Trace Error:The 'FATAL ERROR IN TWO-TASK SERVER: error =12571'
Solution
Restart the eHub services
The ora-12571 error usually occurs when the client abruptly terminates with a Ctrl-C or powers down.
OR
The ora-12571 error, usually, indicates an instability with Oracle with the network. Communication Gap between eHub
and Database Server.
从上面的情形可知，两种情况导致了ora-12571。一是意外终止或按下Ctrl-C，二是网络不稳定所致，而导致前面描述的ORA-02409
更多有关ORA-02409的文章
ID 164839.1
ID 19332.1
ID 789517.1
四、更多参考
有关性能优化请参考
Oracle硬解析与软解析
共享池的调整与优化(Sharedpool Tuning)
Buffercache 的调整与优化(一)
Oracle表缓存(cachingtable)的使用
有关ORACLE体系结构请参考
Oracle表空间与数据文件
Oracle密码文件
Oracle参数文件
Oracle联机重做日志文件(ONLINE LOG FILE)
Oracle控制文件(CONTROLFILE)
Oracle归档日志
Oracle回滚(ROLLBACK)和撤销(UNDO)
Oracle数据库实例启动关闭过程
Oracle10g SGA 的自动化管理
Oracle实例和Oracle数据库(Oracle体系结构)
有关闪回特性请参考
Oracle闪回特性(FLASHBACK DATABASE)
Oracle闪回特性(FLASHBACK DROP & RECYCLEBIN)
Oracle闪回特性(Flashback Query、FlashbackTable)
Oracle闪回特性(Flashback Version、Flashback Transaction)
有关基于用户管理的备份和备份恢复的概念请参考
Oracle冷备份
Oracle热备份
Oracle备份恢复概念
Oracle实例恢复
Oracle基于用户管理恢复的处理(详细描述了介质恢复及其处理)
SYSTEM表空间管理及备份恢复
SYSAUX表空间管理及恢复
有关RMAN的备份恢复与管理请参考
RMAN 概述及其体系结构
RMAN 配置、监控与管理
RMAN 备份详解
RMAN 还原与恢复
RMANcatalog 的创建和使用
基于catalog 创建RMAN存储脚本
基于catalog 的RMAN 备份与恢复
使用RMAN迁移文件系统数据库到ASM
RMAN 备份路径困惑(使用plus archivelog时)
有关ORACLE故障请参考
ORA-32004的错误处理
ORA-01658错误
CRS-0215错误处理
ORA-00119，ORA-00132 错误处理
又一例SPFILE设置错误导致数据库无法启动
对参数FAST_START_MTTR_TARGET= 0 的误解及设定
SPFILE错误导致数据库无法启动(ORA-01565)
有关ASM请参考
创建ASM实例及ASM数据库
ASM 磁盘、目录的管理
使用 ASMCMD 工具管理ASM目录及文件
有关SQL/PLSQL请参考
SQLPlus常用命令
替代变量与SQL*Plus环境设置
使用Uniread实现SQLplus翻页功能
SQL 基础-->SELECT 查询
SQL 基础--> NEW_VALUE的使用
SQL 基础--> 集合运算(UNION与UNION ALL)
SQL 基础--> 常用函数
SQL 基础--> 视图(CREATEVIEW)
SQL 基础--> 创建和管理表
SQL 基础--> 多表查询
SQL 基础--> 过滤和排序
SQL 基础--> 子查询
SQL 基础--> 分组与分组函数
SQL 基础--> 层次化查询(STARTBY ... CONNECT BY PRIOR)
SQL 基础--> ROLLUP与CUBE运算符实现数据汇总
PL/SQL--> 游标
PL/SQL--> 异常处理(Exception)
PL/SQL--> 语言基础
PL/SQL--> 流程控制
PL/SQL--> PL/SQL记录
PL/SQL--> 包的创建与管理
PL/SQL--> 隐式游标(SQL%FOUND)
PL/SQL--> 包重载、初始化
PL/SQL--> DBMS_DDL包的使用
PL/SQL--> DML 触发器
PL/SQL--> INSTEAD OF 触发器
PL/SQL--> 存储过程
PL/SQL--> 函数
PL/SQL--> 动态SQL
PL/SQL--> 动态SQL的常见错误
有关ORACLE其它特性
Oracle常用目录结构(10g)
使用OEM,SQL*Plus,iSQL*Plus 管理Oracle实例
日志记录模式(LOGGING、FORCE LOGGING 、NOLOGGING)
表段、索引段上的LOGGING与NOLOGGING
OralceOMF 功能详解
Oracle用户、对象权限、系统权限
Oracle角色、配置文件
Oracle分区表
Oracle外部表
使用外部表管理Oracle 告警日志(ALAERT_$SID.LOG)
簇表及簇表管理(Index clustered tables)
数据泵 EXPDP 导出工具的使用
数据泵 IMPDP 导入工具的使用
导入导出 Oracle 分区表数据
SQL*Loader使用方法
启用用户进程跟踪
配置非默认端口的动态服务注册
配置ORACLE 客户端连接到数据库
systemsys，sysoper sysdba 的区别
ORACLE_SID、DB_NAME、INSTANCE_NAME、DB_DOMIAN、GLOBAL_NAME
Oracle补丁全集 (Oracle 9i 10g 11g Path)
Oracle10.2.0.1 升级到10.2.0.4
Oracle彻底 kill session