bug之旅1.2：java.sql.SQLException: ORA-00060: 等待资源时检测到死锁

1、错误日志------------------------------------------------------------------

2020-XX-XX 15:32:01.808->
功能号：UPDATE StkInfo SET ExchIDEx=’’,stkName=‘19杭实01’,ShortPrefix=’’,…afterLowerLimit=0 WHERE exchId=‘0’ AND stkId=‘155400’
java.sql.SQLException: ORA-00060: 等待资源时检测到死锁
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:194)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1000)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at oracle.jdbc.driver.OracleStatement.executeUpdateInternal(OracleStatement.java:1814)
at oracle.jdbc.driver.OracleStatement.executeUpdate(OracleStatement.java:1779)
at oracle.jdbc.driver.OracleStatementWrapper.executeUpdate(OracleStatementWrapper.java:277)
at com.XXXXXXX
2020-XX-XX 15:32:02.033->java.sql.SQLException: ORA-00060: 等待资源时检测到死锁

2、oracle死锁日志------------------------------------------------------------------

2.1、获取db死锁日志，

/u01/oracle/diag/rdbms/pbdb/XXX/alert/log.xml
->

ORA-00060: Deadlock detected. More info in file /u01/oracle/diag/rdbms/pbdb/PBDB_VER/trace/PBDB_VER_ora_14465.trc.


->
/u01/oracle/diag/rdbms/pbdb/PBDB_VER/trace/PBDB_VER_ora_14465.trc

2.2、打开错误日志。------------------------------------------------------------------

->
1、
*** 2020-XX-XX 15:35:15.720
DEADLOCK DETECTED ( ORA-00060 )

[Transaction Deadlock]

The following deadlock is not an ORACLE error. It is a
deadlock due to user error in the design of an application
or from issuing incorrect ad-hoc SQL. The following
information may aid in determining the deadlock:

Deadlock graph:
---------Blocker(s)-------- ---------Waiter(s)---------
Resource Name process session holds waits process session holds waits
TX-000b0008-00030a61 322 408 X 321 243 X
TX-000a001e-0003e86a 321 243 X 322 408 X

session 408: DID 0001-0142-0000008E session 243: DID 0001-0141-00000074
session 243: DID 0001-0141-00000074 session 408: DID 0001-0142-0000008E

Rows waited on:
Session 408: obj - rowid = 000144B5 - AAAUS1AAQAAAIosAAI
(dictionary objn - 83125, file - 16, block - 35372, slot - 8)
Session 243: obj - rowid = 000144B5 - AAAUS1AAXAAAQK9AAV
(dictionary objn - 83125, file - 23, block - 66237, slot - 21)

----- Information for the OTHER waiting sessions -----
Session 243:
sid: 243 ser: 4235 audsid: 115281135 user: 72/COREDB_VER1
flags: (0x45) USR/- flags_idl: (0x1) BSY/-/-/-/-/-
flags2: (0x40009) -/-/INC
pid: 321 O/S info: user: oracle, term: UNKNOWN, ospid: 14463
image: oracle@xxxxx
client details:
O/S info: user: Administrator, term: unknown, ospid: 1234
machine: XXXXXprogram: JDBC Thin Client
application name: JDBC Thin Client, hash value=2546894660
current SQL:
UPDATE stkInfo SET closePrice = (case when :1 = 0 then closePrice else :2 end), SettlementPrice = (case when SettlementPrice = 0 then (case when :3 = 0 then closePrice else :4 end) else SettlementPrice end), IOPV = :5 , TotalMktKnockAmt = (case when :6 = 0 then TotalMktKnockAmt else :7 end), TotalMktKnockQty = (case when :8 = 0 then TotalMktKnockQty else :9 end) WHERE exchId = :10 AND stkId = :11

----- End of information for the OTHER waiting sessions -----

Information for THIS session:

----- Current SQL Statement for this session (sql_id=0turbt2d7jsbs) -----
****UPDATE StkInfo SET ExchIDEx=’’,stkName=‘19杭实01’,ShortPrefix=’’…MinBuyQtyTimes=10,afterHighLimit=0,afterLowerLimit=0 WHERE exchId=‘0’ AND stkId=‘155400’
===================================================****
UPDATE stkInfo SET closePrice = (case when :1 = 0 then closePrice else :2 end), SettlementPrice = (case when SettlementPrice = 0 then (case when :3 = 0 then closePrice else :4 end) else SettlementPrice end), IOPV = :5 , TotalMktKnockAmt = (case when :6 = 0 then TotalMktKnockAmt else :7 end), TotalMktKnockQty = (case when :8 = 0 then TotalMktKnockQty else :9 end) WHERE exchId = :10 AND stkId = :11

3、锁定到执行的sql------------------------------------------------------------------

根据第2步，找到发生死锁的两个sql：
SQL1:
UPDATE StkInfo SET ExchIDEx=’’,stkName=‘19杭实01’,ShortPrefix=’’,…afterLowerLimit=0 WHERE exchId=‘0’ AND stkId=‘155400’
SQL2:
UPDATE stkInfo SET closePrice = (case when :1 = 0 then closePrice else :2 end), SettlementPrice = (case when SettlementPrice = 0 then (case when :3 = 0 then closePrice else :4 end) else SettlementPrice end), IOPV = :5 , TotalMktKnockAmt = (case when :6 = 0 then TotalMktKnockAmt else :7 end), TotalMktKnockQty = (case when :8 = 0 then TotalMktKnockQty else :9 end) WHERE exchId = :10 AND stkId = :11

4、根据sql定位到应用程序的代码位置------------------------------------------------------------------

根据SQL1、SQL2定位到代码行，在两个线程中均位于如下类似结构体内。Thread1、Tread2.
Thread1（接受到通知时更新sktinfo）。
Thread2 (定时更新sktinfo表)。
Thread1
//db.beginTrasaction，开启事务
StkList排序->排序
for(Stk stk StkList1){
db.update
}
db.commit;//提交事务。
Thread2
//db.beginTrasaction，开启事务
for(Stk stk StkList2){
db.update
}
db.commit;//提交事务。

5、分析------------------------------------------------------------------

分析：以Stk1=600001、Stk2=600004为例。
在Thread1中，Stk1进行Update完之后。循环未结束。Stk1行处于被锁定状态，且继续循环，对Stk2进行更新（先申请锁）。->Session1
在Thread2中，Stk2进行Update完之后。循环未结束。Stk2行处于被锁定状态，且继续循环，对Stk1进行更新（先申请锁）。->Session2
由于Stk1被Session1锁定，Stk2被Session2锁定。导致 Session1与Session2相互等待锁，产生死锁。（注意区分死锁和锁等待超时的区别。）
解决方法：Thread2对StkList2按照 与Thread1中相同的排序规则进行排序。

6、总结：------------------------------------------------------------

->>后台日志日常维护分析
->>发现死锁log
->>通过运维人员获取到服务的oracle数据库日志
->>查询oracle死锁日志
->>定位到sql
->>定位到表及程序代码位置
->>分析代码运行逻辑，给出解决方案
->>反馈给相关开发人员进行修改

7、备注

此问题是在一次服务端日志检查时发现的。