现场报网公司数据库连不上,先检查了下数据库processes=1500,session=2200.我觉得很大啊,这个数据库没有几个人用。
查看v$session中的session最多是哪个machine发起的,发现是省公司的数据库发起的session,找开发梳理了下业务,省公司同步dblink操作网公司表,且是通过weblogic的连接池。
哦,有点明白了,是dblink引起的,weblogic连接池是一直存在的,所以在网公司端session是不释放的,如果省公司把应用都停掉,那在网公司端的session都会释放。要验证想法,做个试验:
目标:数据库A上建dblink,修改数据库B上的表。
环境准备:
1.在数据库A上建dblink
create public database link TO_B
connect to TEST_DB identified by TEST_DB
using '(DESCRIPTION =(ADDRESS_LIST =(ADDRESS = (PROTOCOL = TCP)(HOST = 10.10.15.150)(PORT = 1521)))(CONNECT_DATA =(SID = orcl)))';
2.在数据库B上建表
create table TEST( ID NUMBER );
insert into test values(1);
开始测试:
1.在数据库B上select count(1) from v$session;
2.在数据库A上update test@TO_B set id=1;
commit;
3.在数据库B上select count(1) from v$session;可以看到涨了一个
对比测试:
1.在数据库B上select count(1) from v$session;
2.在数据库A上update test@TO_B set id=1;
commit;
alter session close database linkTO_B;
3.在数据库B上select count(1) from v$session;可以看到没有变化
总结: 出现这种问题,归根结底是操作dblink不规范,用了之后没有关闭。
有可能是bug:
|
Bug 18394488 : TOO MANY CONNECTIONS OVER SHARED DBLINK AFTER UPGRADE TO 11.2.0.4 |
|
|
|
类型 |
B - Defect |
已在产品版本中修复 |
|
严重性 |
2 - Severe Loss of Service |
产品版本 |
11.2.0.4 |
状态 |
36 - Duplicate Bug. To Filer |
平台 |
212 - IBM AIX on POWER Systems (64-bit) |
创建时间 |
2014-3-13 |
平台版本 |
7.1 |
更新时间 |
2015-6-5 |
基本 Bug |
18759589 |
数据库版本 |
11.2.0.4 |
影响平台 |
Generic |
产品源 |
Oracle |
与此 Bug 相关的知识, 补丁程序和 Bug |
产品线 |
Oracle Database Products |
系列 |
Oracle Database Suite |
区域 |
Oracle Database |
产品 |
5 - Oracle Database - Enterprise Edition |
Hdr: 18394488 11.2.0.4 RDBMS 11.2.0.4 PRG INTERFACE PRODID-5 PORTID-212 18759589
Abstract: TOO MANY CONNECTIONS OVER SHARED DBLINK AFTER UPGRADE TO 11.2.0.4
*** 03/13/14 12:14 am ***
PROBLEM:
--------
The production database system upgraded from 11.2.0.1 to 11.2.0.4 in the
previous weekend.
There are several shared db links on this database to other production
database that its version is 11.2.0.3 .
When trying to connect sessions on 11.2.0.4 to 11.2.0.3 via shared db links
it opens too many connection on connected database.
We tested this situation between 11.2.0.3 to 11.2.0.3 databases via shared db
link , opened session count is two (one of them is PSEUDO other is DEDICATED
)
but between 11.2.0.4 to 11.2.04 opened session count is at least 10. When
1000 session trying to connect 11.2.0.4 to 11.2.0.3 via shared db links about
10000 connection opening.
This system working correctly before database upgrade.
We didn't change any parameter, listener configuration or unix parameter.
DIAGNOSTIC ANALYSIS:
--------------------
Checked with the customer on:
1. Why are you using shared DBLINKS and SHARED servers ?
Because main application works with shared dblinks. (from many years ) ... We
didn't use shared servers , after upgrade too many connection is opening and
we have to use it because of handle too many connection with less resource .
2. Can you use standard DBLINKS and not use shared servers ?
We cannot use standard db links because of application design... If Oracle
solve this problem we can use dedicated. (before upgrade we had used
dedicated )
When checking with application development team on why are they using shared
db links ;
they said that main banking application works in webspehere connection pool
using XA transaction and it must use shared db link to handle connections !
It can't use standart db links . If oracle created shared db link feature why
are you offer to use standard db link ? Before upgrade everything works fine.
I am also repeating my words , we had used dedicated server , we have to use
shared server because of performance issue for now . If Oracle solve opening
many connection issue we will use dedicated server again . But shared db link
feautere will remain because of websphere XA transaction handling.
3. How many sessions to server B does each of your application connections to
Server A open ?
Its count didn't change , anyway after upgrade it is same that the previous
value before upgrade.
ct found metalink document why they use shared db link.
ORA-24777 reported when using a database link from within an XA coordinated
transaction (Doc ID 1506756.1)
Solution from document;
Two options
1. Configure the database to allow the use of shared servers and then let the
application use these as per the following documentation
http://docs.oracle.com/cd/E11882_01/server.112/e25494/manproc003.htm#ADMIN0050
2
or establish whether the Java application needs to start an XA transaction.
OR
2. Define the database link being utilised as a shared database link, i.e,
CREATE SHARED DATABASE LINK ..
They got this error many years ago ;
ORA-24777 reported by a JDBC/XA application when utilising a database link
between two Oracle databases. And db links was configured to shared db links.
I explained before why we are using shared server ( performance )
A complete test case epxlaining how a connection from 11.2.0.4. to 11.2.0.3
opens alot of connections: db_link.pdf
WORKAROUND:
-----------
none
RELATED BUGS:
-------------
Bug 18157790 - ORA-22 AFTER UPGRADE FROM 11.2.0.3 TO 11.2.0.4
Bug 1559258 - PSEUDO SESSION INCREASE WITH XA + SHARED DBLINK
REPRODUCIBILITY:
----------------
issue is reproducible on ct's env.
TEST CASE:
----------
provided
STACK TRACE:
------------
SUPPORTING INFORMATION:
-----------------------
24 HOUR CONTACT INFORMATION FOR P1 BUGS:
----------------------------------------
DIAL-IN INFORMATION:
--------------------
IMPACT DATE:
------------
附录为官方文档:
32 Managing a Distributed Database
Closing Database Links
If you access a database link in a session, then the link remains open until you close the session. A link is open in the sense that a process is active on each of the remote databases accessed through the link. This situation has the following consequences:
-
If 20 users open sessions and access the same public link in a local database, then 20 database link connections are open.
-
If 20 users open sessions and each user accesses a private link, then 20 database link connections are open.
-
If one user starts a session and accesses 20 different links, then 20 database link connections are open.
After you close a session, the links that were active in the session are automatically closed. You may have occasion to close the link manually. For example, close links when:
To close a link, issue the following statement, where linkname refers to the name of the link:
ALTER SESSION CLOSE DATABASE LINK linkname;
Note that this statement only closes the links that are active in your current session.