谈到大表与大表之间的连接方式,总会下意识的选择走HASH JOIN+并行效率才高,其实具体情况必须具体问题,在有的时候,大表走嵌套循环的效率要优于走HASH JOIN,下面就用我之前优化过的一个案例来说明。
一个哥们找我优化一个SQL 跑的很慢,103万个逻辑读,103万个物理读,要跑19秒才能跑完。
下面是它的SQL 以及执行计划:
SELECT "A1"."SERV_ID"
,"A1"."ATTR_ID"
,"A1"."ATTR_VAL"
,"A1"."EFF_DATE"
,NVL("A1"."EXP_DATE", TO_DATE(NULL, NULL))
,"A2"."OP_SEQ"
,"A2"."OP_TYPE"
FROM COMM."USER_INFO_OPLOG_MASTER2" "A2"
,COMM."SERV_ATTR" "A1"
WHERE "A2"."MEMOP_DATE" IS NULL
AND "A2"."TABLE_NAME" = 'SERV_ATTR'
AND "A1"."SERV_ID" = "A2"."TABLE_COLUMN_1"
AND "A1"."AGREEMENT_ID" = "A2"."TABLE_COLUMN_2"
AND "A1"."ATTR_ID" = "A2"."TABLE_COLUMN_3"
ORDER BY "A2"."OP_SEQ"
563 rows selected.
Elapsed: 00:00:19.50
Execution Plan
----------------------------------------------------------
Plan hash value: 669020553
---------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
---------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 150K| 17M| | 61528 (3)| 00:12:19 | | | |
| 1 | PX COORDINATOR | | | | | | | | | |
| 2 | PX SEND QC (ORDER) | :TQ10002 | 150K| 17M| | 61528 (3)| 00:12:19 | Q1,02 | P->S | QC (ORDER) |
| 3 | SORT ORDER BY | | 150K| 17M| 20M| 61528 (3)| 00:12:19 | Q1,02 | PCWP | |
| 4 | PX RECEIVE | | 150K| 17M| | 61526 (3)| 00:12:19 | Q1,02 | PCWP | |
| 5 | PX SEND RANGE | :TQ10001 | 150K| 17M| | 61526 (3)| 00:12:19 | Q1,01 | P->P | RANGE |
|* 6 | HASH JOIN | | 150K| 17M| | 61526 (3)| 00:12:19 | Q1,01 | PCWP | |
| 7 | BUFFER SORT | | | | | | | Q1,01 | PCWC | |
| 8 | PX RECEIVE | | 147K| 11M| | 1069 (1)| 00:00:13 | Q1,01 | PCWP | |
| 9 | PX SEND BROADCAST | :TQ10000 | 147K| 11M| | 1069 (1)| 00:00:13 | | S->P | BROADCAST |
| 10 | TABLE ACCESS BY INDEX ROWID| USER_INFO_OPLOG_MASTER2 | 147K| 11M| | 1069 (1)| 00:00:13 | | | |
|* 11 | INDEX RANGE SCAN | IDX_INFO_OPLOG_M2COMB_11502 | 6113 | | | 33 (0)| 00:00:01 | | | |
| 12 | PX BLOCK ITERATOR | | 229M| 9427M| | 60101 (2)| 00:12:02 | Q1,01 | PCWC | |
|* 13 | TABLE ACCESS FULL | SERV_ATTR | 229M| 9427M| | 60101 (2)| 00:12:02 | Q1,01 | PCWP | |
---------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
6 - access("A1"."SERV_ID"="A2"."TABLE_COLUMN_1" AND "A1"."AGREEMENT_ID"="A2"."TABLE_COLUMN_2" AND "A1"."ATTR_ID"="A2"."TABLE_COLUMN_3")
11 - access("A2"."MEMOP_DATE" IS NULL AND "A2"."TABLE_NAME"='SERV_ATTR')
filter("A2"."TABLE_NAME"='SERV_ATTR')
13 - filter(SYS_OP_BLOOM_FILTER(:BF0000,"A1"."SERV_ID","A1"."AGREEMENT_ID","A1"."ATTR_ID"))
Note
-----
- dynamic sampling used for this statement (level=6)
Statistics
----------------------------------------------------------
30 recursive calls
0 db block gets
1039308 consistent gets
1003269 physical reads
716 redo size
31187 bytes sent via SQL*Net to client
927 bytes received via SQL*Net from client
39 SQL*Net roundtrips to/from client
11 sorts (memory)
0 sorts (disk)
563 rows processed
从执行计划来看 有大概3个问题:
<1> 原SQL并未用hint paraller 提示走并行,但是从实际的执行计划来看,还是走了并行,应该是这两张表上设置了默认的并行度。一般可以在数据仓库里面这样搞,但是问了一下这个哥们,他们的数据库系统是OLTP和OLAP混合系统,因此不建议在表上设置较高的并行度,这样会促使CBO更加倾向于选择走并行+全表扫描。
<2> ID=13 的谓词信息有了“(SYS_OP_BLOOM_FILTER(:BF0000,"A1"."SERV_ID","A1"."AGREEMENT_ID","A1"."ATTR_ID"))” ,从10g R2 以后 CBO 优化器算法加入了布隆过滤算法,一般在两表走HASH JOIN+并行的时候会在HASH JOIN的被驱动表中出现,但是从我之前优化过SQL的经验来看,这个新特性带了的BUG较多,尤其在被驱动表数据量特别大的时候使用该算法影响会更大,因此,建议数据系统解决 将该参数disable掉。
<3> 可以看到这个SQL最终返回的结果只有563行,由此推断,走Nested loop 嵌套循环应该会比有HASH 效率更高效。
下面我通过以下SQL 印证了我的判断:
A2 ------USER_INFO_OPLOG_MASTER2 这个表的总数据量为:67万条
A1-------SERV_ATTR 这个表的总数据量为:2亿多条
执行SQL:
select count(*) from COMM.USER_INFO_OPLOG_MASTER2 where MEMOP_DATE is null and TABLE_NAME='SERV_ATTR';
COUNT(*)
----------
53677
A2 这个表过滤以后只返回5万多条
再来看A1 表中索引的信息:
****************************************************************************************
INDEX INFO
****************************************************************************************
TABLE TABLE Index COLUMN Col
OWNER NAME Name Unique NAME Pos DESC
--------------- ----------------------------------- ------------------------------ --------- ------------------------- ---- ----
COMM SERV_ATTR IDX_SERV_ATTR_AGREEMENT_ID NONUNIQUE AGREEMENT_ID 1 ASC
IDX_SERV_ATTR_ATTR_ID NONUNIQUE ATTR_ID 1 ASC
IDX_SERV_ATTR_INTERNET NONUNIQUE ATTR_VAL 1 ASC
ATTR_ID 2 ASC
IDX_SERV_ATTR_SERV_ID NONUNIQUE SERV_ID 1 ASC
IDX_SERV_ATTR_VAL8 NONUNIQUE ATTR_VAL 1 ASC
PK_SERV_ATTR UNIQUE SERV_ID 1 ASC
AGREEMENT_ID 2 ASC
ATTR_ID 3 ASC
我让它加下以下的HINT 然后去set autotrace traceonly 去看执行计划和逻辑读:
/*+ use_nl(A1,A2) index(A1,PK_SERV_ATTR) leading(A2) */
Elapsed: 00:00:05.45
Execution Plan
----------------------------------------------------------
Plan hash value: 4263321111
----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 142K| 16M| | 286K (1)| 00:57:13 |
| 1 | SORT ORDER BY | | 142K| 16M| 19M| 286K (1)| 00:57:13 |
| 2 | NESTED LOOPS | | 142K| 16M| | 282K (1)| 00:56:26 |
| 3 | NESTED LOOPS | | 142K| 16M| | 282K (1)| 00:56:26 |
| 4 | TABLE ACCESS BY INDEX ROWID| USER_INFO_OPLOG_MASTER2 | 140K| 10M| | 1489 (1)| 00:00:18 |
|* 5 | INDEX RANGE SCAN | IDX_INFO_OPLOG_M2COMB_11502 | 9438 | | | 49 (0)| 00:00:01 |
|* 6 | INDEX UNIQUE SCAN | PK_SERV_ATTR | 1 | | | 2 (0)| 00:00:01 |
| 7 | TABLE ACCESS BY INDEX ROWID | SERV_ATTR | 1 | 43 | | 3 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
5 - access("A2"."TABLE_NAME"='SERV_ATTR' AND "A2"."MEMOP_DATE" IS NULL)
6 - access("A1"."SERV_ID"="A2"."TABLE_COLUMN_1" AND "A1"."AGREEMENT_ID"="A2"."TABLE_COLUMN_2" AND
"A1"."ATTR_ID"="A2"."TABLE_COLUMN_3")
Note
-----
- dynamic sampling used for this statement (level=6)
Statistics
----------------------------------------------------------
7 recursive calls
0 db block gets
60044 consistent gets
5440 physical reads
8824 redo size
19160 bytes sent via SQL*Net to client
762 bytes received via SQL*Net from client
24 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
345 rows processed