locked <0xd01eb358> (a oracle.jdbc.driver.T4CConnection

发现写Oracle的线程挂住了，场景是从mysql读数据，然后写到Oracle。

 

1 定位线程

因为在同一台机器上运行了多个java进程，要找到对应的pid，就是连接mysql的的那个进程。

 

[admin@arch036075.sqa.cm4 logs]$ ss -p | grep "10.232.31.67"
ESTAB      0      0            10.232.36.75:41790         10.232.31.67:mysql    users:(("java",1978,184))
ESTAB      0      0            10.232.36.75:41789         10.232.31.67:mysql    users:(("java",1978,179))

 

红色部分就是挂住线程的pid

 

2 jstack

"Stage Task: mysql-to-dbsync" daemon prio=10 tid=0x00002aaac8699000 nid=0x7e33 runnable [0x0000000043cfb000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at oracle.net.ns.Packet.receive(Unknown Source)
        at oracle.net.ns.NSProtocol.sendReset(Unknown Source)
        at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1112)
        at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1075)
        at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:480)
        at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:216)
        at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:966)
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1170)
        at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3339)
        at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3423)
        - locked <0x00002aaab4575880> (a oracle.jdbc.driver.T4CPreparedStatement)
        - locked <0x00002aaab4cda408> (a oracle.jdbc.driver.T4CConnection)

发现写oracle的异常

- locked <0x00002aaab4575880> (a oracle.jdbc.driver.T4CPreparedStatement)

- locked <0x00002aaab4cda408> (a oracle.jdbc.driver.T4CConnection)

3 询问dba对应oracle数据库的情况，说写库有一张表被锁住了，解锁，重启精卫系统，恢复正常。

4 之前有人说过可能是oracle的bug，设置connection的timeout可解决，也用这种方法解决了第一次遇这个的问题的应用（这是第二次遇到），重启应用，发现是锁表导致的。