系统版本:aix6014
数据库版本:oracle11.2.0.2.0
故障情况,早晨公司清分系统执行异常缓慢,取了一份awr,发现由于一条频繁执行的sql原因,导致应用执行缓慢。查看该条sql的sqlplan情况
SQL> explain plan for
2 select /* +index(BMS_CHK_DTL, INDEX2_BMS_CHK_DTL) */
3 ACQ_DB_FEE, ACQ_CR_FEE, RSV1, RSV2, UMS_MCC, ORI_TRANS_DT
4 from YSSETTZGS.BMS_CHK_DTL
5 where RRN = :REF_NO
6 and RUNTIME_ID = :RUNTIME_ID
7 and MID = :MER_NO
8 and TID = :TERM_NO
9 and MSG_TYPE = :MSG_TYPE
10 and FILE_SEQ = '1000'
11 and PROC_CODE = :PROC_CODE
12 and SER_CONCODE = :SVR_COND_CODE
13 and TRANS_DT = :TRANS_DT;
Explained.
SQL> select * from table(DBMS_XPLAN.display); \
2
SQL> select * from table(DBMS_XPLAN.display);
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 142170735
-------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
-------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 169 | 0 (0)| 00:00:01 | | |
|* 1 | TABLE ACCESS BY GLOBAL INDEX ROWID| BMS_CHK_DTL | 1 | 169 | 0 (0)| 00:00:01 | ROWID | ROWID |
|* 2 | INDEX RANGE SCAN | INDEX1_BMS_CHK_DTL | 1 | | 0 (0)| 00:00:01 | | |
-------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("FILE_SEQ"=1000 AND "RRN"=:REF_NO AND "MID"=:MER_NO AND "TID"=:TERM_NO AND "MSG_TYPE"=:MSG_TYPE
AND "PROC_CODE"=:PROC_CODE AND "SER_CONCODE"=:SVR_COND_CODE AND "TRANS_DT"=:TRANS_DT)
2 - access("RUNTIME_ID"=TO_NUMBER(:RUNTIME_ID))
发现sql走得索引是INDEX1_BMS_CHK_DTL ,并非hint指定的索引INDEX2_BMS_CHK_DTL
这两条索引的字段差别:
SQL> col column_name for a30
SQL> select INDEX_NAME,TABLE_NAME,COLUMN_NAME,COLUMN_POSITION from dba_ind_columns where INDEX_NAME in (select b.INDEX_NAME from dba_indexes b
2 where b.owner=upper('&username') and b.TABLE_NAME=upper('&tablename')) order by INDEX_NAME,COLUMN_POSITION;
Enter value for username: yssettzgs
Enter value for tablename: BMS_CHK_DTL
old 2: where b.owner=upper('&username') and b.TABLE_NAME=upper('&tablename')) order by INDEX_NAME,COLUMN_POSITION
new 2: where b.owner=upper('yssettzgs') and b.TABLE_NAME=upper('BMS_CHK_DTL')) order by INDEX_NAME,COLUMN_POSITION
INDEX_NAME TABLE_NAME COLUMN_NAME COLUMN_POSITION
------------------------------ ------------------------------ ------------------------------ ---------------
BMS_CHK_DTL_IDX3 BMS_CHK_DTL INST_ID 1
INDEX1_BMS_CHK_DTL BMS_CHK_DTL RUNTIME_ID 1
INDEX1_BMS_CHK_DTL BMS_CHK_DTL PAN 2
INDEX2_BMS_CHK_DTL BMS_CHK_DTL RRN 1
INDEX2_BMS_CHK_DTL BMS_CHK_DTL TID 2
INDEX2_BMS_CHK_DTL BMS_CHK_DTL MID 3
INDEX_BMS_CHK_DTL BMS_CHK_DTL RUNTIME_ID 1
INDEX_BMS_CHK_DTL BMS_CHK_DTL SYSTRACE 1
INDEX_BMS_CHK_DTL BMS_CHK_DTL TID 2
INDEX_BMS_CHK_DTL BMS_CHK_DTL TRANS_DT 2
INDEX_BMS_CHK_DTL BMS_CHK_DTL MID 3
INDEX_BMS_CHK_DTL BMS_CHK_DTL ACQ_ID 3
INDEX_BMS_CHK_DTL BMS_CHK_DTL FWD_ID 4
INDEX_BMS_CHK_DTL BMS_CHK_DTL RRN 4
INDEX_BMS_CHK_DTL BMS_CHK_DTL CHK_FLG 5
查看那索引统计分析情况:
SQL> select INDEX_NAME,STATUS,TABLE_NAME,NUM_ROWS,LAST_ANALYZED,GLOBAL_STATS,VISIBILITY from dba_indexes where owner=upper('&owner') and table_name=upper('&tablename');
Enter value for owner: yssettzgs
Enter value for tablename: BMS_CHK_DTL
old 1: select INDEX_NAME,STATUS,TABLE_NAME,NUM_ROWS,LAST_ANALYZED,GLOBAL_STATS,VISIBILITY from dba_indexes where owner=upper('&owner') and table_name=upper('&tablename')
new 1: select INDEX_NAME,STATUS,TABLE_NAME,NUM_ROWS,LAST_ANALYZED,GLOBAL_STATS,VISIBILITY from dba_indexes where owner=upper('yssettzgs') and table_name=upper('BMS_CHK_DTL')
INDEX_NAME STATUS TABLE_NAME NUM_ROWS LAST_ANALYZED GLO VISIBILIT
------------------------------ -------- ------------------------------ ---------- ------------------- --- ---------
INDEX2_BMS_CHK_DTL VALID BMS_CHK_DTL 0 2013-08-29:22:03:03 YES VISIBLE
INDEX1_BMS_CHK_DTL VALID BMS_CHK_DTL 0 2013-08-29:22:03:03 YES VISIBLE
INDEX_BMS_CHK_DTL VALID BMS_CHK_DTL 0 2013-08-29:22:03:03 YES VISIBLE
BMS_CHK_DTL_IDX3 N/A BMS_CHK_DTL 0 2013-08-29:22:03:03 YES VISIBLE
查看该表的统计分析情况
SQL> select OWNER,TABLE_NAME,NUM_ROWS,BLOCKS,EMPTY_BLOCKS,NUM_FREELIST_BLOCKS,LAST_ANALYZED from dba_tables where owner=upper('&username') and table_name=upper('&tablename');
Enter value for username: yssettzgs
Enter value for tablename: BMS_CHK_DTL
old 1: select OWNER,TABLE_NAME,NUM_ROWS,BLOCKS,EMPTY_BLOCKS,NUM_FREELIST_BLOCKS,LAST_ANALYZED from dba_tables where owner=upper('&username') and table_name=upper('&tablename')
new 1: select OWNER,TABLE_NAME,NUM_ROWS,BLOCKS,EMPTY_BLOCKS,NUM_FREELIST_BLOCKS,LAST_ANALYZED from dba_tables where owner=upper('yssettzgs') and table_name=upper('BMS_CHK_DTL')
OWNER TABLE_NAME NUM_ROWS BLOCKS EMPTY_BLOCKS NUM_FREELIST_BLOCKS LAST_ANALYZED
------------------------------ ------------------------------ ---------- ---------- ------------ ------------------- -------------------
YSSETTZGS BMS_CHK_DTL 0 0 0 0 2013-08-29:22:03:01
查看该表当前数据总数
SQL> select count(*) from yssettzgs.BMS_CHK_DTL;
COUNT(*)
----------
232183
发现比较奇怪的是,20130829晚上22点左右,索引的统计分析自动更新, NUM_ROWS 都是0,说明那个时刻表的数据为0.
查看这两个索引第一个字段的筛选率
SQL> select count(*) from (select distinct RUNTIME_ID from YSSETTZGS.BMS_CHK_DTL);
COUNT(*)
----------
1
SQL> select count(*) from (select distinct RRN from YSSETTZGS.BMS_CHK_DTL);
COUNT(*)
----------
178663
可见INDEX2_BMS_CHK_DTL 的筛选率要远高于INDEX1_BMS_CHK_DTL ,但为什么不走INDEX2_BMS_CHK_DTL 呢?
继续查看该表的dml记录变化
TABLE_NAME PARTITION_NAME SUBPARTITION_NAME INSERTS UPDATES DELETES TRU DROP_SEGMENTS TIMESTAMP
------------------------------ ------------------------------ ------------------------------ ---------- ---------- ---------- --- ------------- -------------------
BMS_CHK_DTL BMS_CHK_DTL_48005840 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48020000 1376507 1762186 1144331 YES 0 2013-08-30:05:30:30
BMS_CHK_DTL BMS_CHK_DTL_48021000 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48021100 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48021210 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48021610 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48021910 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48022200 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48022220 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48022410 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48022610 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48022900 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48023010 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48023310 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48023320 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48023610 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48023900 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48023930 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48024210 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48024500 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48024520 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48024910 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48025200 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48025510 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48025800 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48025840 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48026100 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48026410 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48026510 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48026900 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48027010 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48027310 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48027700 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48027910 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48028210 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48028500 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48028700 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48028810 0 0 0 NO 0 2013-08-28:22:00:09
BMS_CHK_DTL BMS_CHK_DTL_48031000 0 0 0 NO 0 2013-08-28:22:00:09
可以发现,该表数据的变化,在20130830的凌晨五点左右,发生数据改变,但这时候当晚自动统计分析已经过了。因此统计分析数据相对当前情况是非常不准确。
可以通过立刻对该表和表上索引进行统计分析,可以纠正执行计划,恢复执行效率。
但为什么hint提示无效?
通过10053trace分析,看见结果:
****** finished trying bitmap/domain indexes ******
Best:: AccessPath: IndexRange
Index: INDEX1_BMS_CHK_DTL
Cost: 0.00 Degree: 1 Resp: 0.00 Card: 0.00 Bytes: 0
导致HINT 失效的原因有如下2点:
(1) 如果CBO 认为使用Hint 会导致错误的结果时,Hint将被忽略。
如索引中的记录因为空值而和表的记录不一致时,结果就是错误的,会忽略hint。
(2) 如果表中指定了别名,那么Hint中也必须使用别名,否则Hint也会忽略。
猜测可能是第一点,在执行sql时,rrn字段并未有值导致索引选择错误。
再研究分析后,再更新具体内容。。。先记录着。