Optimizer Transformations: Subquery Unesting part 2

In Part one of our blog series on Optimizer transformations we discussed Subquery unesting. We explained how subqueries are evaluated, and the unnesting of EXISTS and ANY subqueries.

Here in part two of our series we will discuss the unnesting of NOT EXISTS and single-row subqueries, which appear in the WHERE clause. Once again I need to give credit to Rafi one of the senior Optimizer developers for the content of this blog!

Quick recap on what we mean by Subquery Unesting
Subquery unnesting is an optimization that converts a subquery into a join in the outer query and allows the optimizer to consider subquery table(s) during access path, join method, and join order selection. Unnesting either merges the subquery into the body of the outer query or turns it into an inline view.

NOT EXISTS Subqueries
Anti-join is always used for unnesting NOT EXISTS and ALL subqueries. We represent anti-join by the following non-standard syntax: T1.x A= T2.y, where T1 is the left table and T2 is the right table of the anti-join. The semantics of anti-join is the following: A row of T1 is rejected as soon as T1.x finds a match with any value of T2.y. A row of T1 is returned, only if T1.x does not match with any value of T2.y.

Consider query F, which has a NOT EXISTS subquery containing two tables.

F.
SELECT C.cust_last_name, C.country_id
FROM    customers C
WHERE NOT EXISTS (SELECT 1
                                  FROM sales S, products P
                                  WHERE P.prod_id = S.prod_id and
                                               P.prod_min_price > 90 and
                                               S.cust_id = C.cust_id);

The subquery in F can be unnested by using an anti-join; however, the inner join of the tables in the subquery, sales and products must take place before the anti-join is performed. An inline view is generated in order to enforce the join order. This unnesting produces query G; here the inline view becomes the right table of anti-join.

G.
SELECT C.cust_last_name, C.country_id
FROM    customers C,
             (SELECT S.cust_id AS s_cust_id
              FROM sales S, products P
              WHERE P.prod_id = S.prod_id and
                           P.prod_min_price > 90) VW
WHERE C.cust_id A= VW.s_cust_id;

The execution plan of G is shown as XG. Of the three join methods (i.e., nested-loop, hash and sort-merge), the hash method was selected by the optimizer to do the anti-join.

XG.



Single-Row Aggregated Subqueries
Consider query H that contains an aggregated, correlated, single-row subquery.

H.
SELECT C.cust_last_name, C.cust_income_level
FROM    customers C
WHERE C.cust_credit_limit < (SELECT SUM(S.amount_sold)
                                             FROM sales S
                                             WHERE S.cust_id = C.cust_id);

Doing aggregation for multiple values of equality correlation predicate is like doing aggregation and group-by on the local column, and then joining the view with the outer table on the group-by columns. The subquery in H is unnested by decorrelating it and converting it into a group-by view, which is inner joined with the outer table, customer; here both the correlating and connecting conditions have been turned into join predicates. The transformed query is shown as I.

I.
SELECT C.cust_last_name, C.cust_income_level
FROM    customers C,
             SELECT SUM(S.amount_sold) AS sum_amt, S.cust_id
             FROM sales S
             GROUP BY S.cust_id) VW
WHERE C.cust_credit_limit < VW.sum_amt and
             C.cust_id = VW.cust_id;

XI shows the execution plan of I.

XI.



Validity of Unnesting
Every subquery, before it can be unnested, goes through a set of validity checks. The optimizer decisions to unnest or not to unnest a subquery can be overridden by specifying an appropriate hint, but the validity requirements cannot be, since unnesting in such cases would not guarantee a semantically equivalent query.

In the following, we enumerate some important checks that currently invalidate subquery unnesting. Note that this list of checks is by no means exhaustive.

    Subqueries that are correlated to non-parent; for example, subquery SQ3 is contained by SQ2 (parent of SQ3) and SQ2 in turn is contained by SQ1 and SQ3 is correlated to tables defined in SQ1.
    A group-by subquery is correlated; in this case, unnesting implies doing join after group-by. Changing the given order of the two operations may not be always legal.
    Connecting or correlating conditions are not well-formed (e.g., they contains a mix of local and correlated columns on either side of the predicate) and the subquery requires inline view generation, as predicates of this kind do not allow separating out view columns and outer table columns.
    For disjunctive subqueries, the outer columns in the connecting or correlating conditions are not the same.

Using view-merging transformation, Oracle may merge the group-by or distinct inline view generated during unnesting, and therefore the execution plan may not show any view even when a view is expected.

Summary
In these posts we have tried to illustrate the basic ideas behind unnesting of different types of subquery by showing  simple example queries. Oracle can handle far more complex queries - query statements with multiple subqueries at one or more levels, multiple tables, correlated and connecting conditions containing inequality predicates and expressions, subqueries that contain set operators, subqueries with group-by and COUNT aggregate function, ALL subqueries containing nullable columns in its connecting condition, and subqueries in disjunction.

If unnesting of a subquery does not require generation of an inline view, then the subquery is always unnested, because this unnesting provides a more efficient execution plan by allowing more join methods and join orders. If the local column (e.g., S.cust_id in A) in the correlating predicate of the subquery has an index on it, then the subquery evaluation becomes akin to doing index-based nested-loop join; and thus, in some cases, not unnesting may be more efficient than unnesting that generates an inline view and enables only sort-merge and hash join of the view with outer tables. Therefore, in Oracle, subquery unnesting which generates inline views, is done based on cost under the cost-based query transformation framework.