String Aggregation
In some cases, it is necessary to aggregate data from number of rows into a single row, giving a list of data associated with a specific value. There are some methods to achieve this.
- LISTAGG Analystic Function in 11g Release 2
The LISTAGG analytic function was introduced in Oracle 11g Release 2, making it very easy to aggregate strings.
Example:
SELECT type, LISTAGG(oid, ',') FROM rcp_company group by type;
- WM_CONCAT function
If you are running a version of the database where WM_CONCAT function is available, then use this function to aggregate strings. Be note that this function is undocumented, and Tom doesn’t suggest to use it.
Example:
SELECT type, wm_concat(oid) AS oids FROM rcp_company group by type;
- Generic Function using Ref Cursor
Write a generic function to concatenate values passed using a ref cursor.
Example:
create or replace function join
(
pv_cursor sys_refcursor,
pv_del varchar2 := ','
) return varchar2
is
lv_value varchar2(4000);
lv_result varchar2(4000);
begin
loop
fetch pv_cursor into lv_value;
exit when pv_cursor%notfound;
if lv_result is not null then
lv_result := lv_result || pv_del;
end if;
lv_result := lv_result || lv_value;
end loop;
return lv_result;
end join;
The cursor function is used as below:
select type,
join(CURSOR (select t2.oid from rcp_company t2 where t2.type = t1.type)
from rcp_company t1
group by type;
- User-Defined Aggregate Function
Create a user-defined aggregate function, using the ODCIAggregate interface.
Exmaple:
CREATE OR REPLACE TYPE gtyp_strcat_object IS OBJECT
(
catstr VARCHAR2(32767),
STATIC FUNCTION odciaggregateinitialize(cs_ctx IN OUT gtyp_strcat_object)
RETURN NUMBER,
MEMBER FUNCTION odciaggregateiterate(SELF IN OUT gtyp_strcat_object,
VALUE IN VARCHAR2) RETURN NUMBER,
MEMBER FUNCTION odciaggregatemerge(SELF IN OUT gtyp_strcat_object,
ctx2 IN OUT gtyp_strcat_object)
RETURN NUMBER,
MEMBER FUNCTION odciaggregateterminate(SELF IN OUT gtyp_strcat_object,
returnvalue OUT VARCHAR2,
flags IN NUMBER)
RETURN NUMBER
)
;
/
CREATE OR REPLACE TYPE BODY gtyp_strcat_object IS
STATIC FUNCTION odciaggregateinitialize(cs_ctx IN OUT gtyp_strcat_object)
RETURN NUMBER IS
BEGIN
cs_ctx := gtyp_strcat_object(NULL);
RETURN odciconst.success;
END;
MEMBER FUNCTION odciaggregateiterate(SELF IN OUT gtyp_strcat_object,
VALUE IN VARCHAR2) RETURN NUMBER IS
BEGIN
SELF.catstr := SELF.catstr || ',' || VALUE;
RETURN odciconst.success;
END;
MEMBER FUNCTION odciaggregateterminate(SELF IN OUT gtyp_strcat_object,
returnvalue OUT VARCHAR2,
flags IN NUMBER) RETURN NUMBER IS
BEGIN
returnvalue := ltrim(rtrim(SELF.catstr, ','), ',');
RETURN odciconst.success;
END;
MEMBER FUNCTION odciaggregatemerge(SELF IN OUT gtyp_strcat_object,
ctx2 IN OUT gtyp_strcat_object)
RETURN NUMBER IS
BEGIN
SELF.catstr := SELF.catstr || ',' || ctx2.catstr;
RETURN odciconst.success;
END;
END;
/
CREATE OR REPLACE FUNCTION fun_agg_strcat(pv_str varchar2) RETURN varchar2
PARALLEL_ENABLE
AGGREGATE USING gtyp_strcat_object;
The aggregate function is implemented using a type and type body, and is used within a query as below:
select type,fun_agg_strcat(oid) from rcp_company group by type
- Hierarchical function
Use SYS_CONNECT_BY_PATH functions to achieve the same result without the use of PL/SQL or additional type definitions.
Example:
select type, ltrim(sys_connect_by_path(sp.oid, ','), ',')
from (select s.oid,
s.type,
row_number() over(partition by s.type order by s.oid) as num
from rcp_company s) sp
where connect_by_isleaf = 1
start with sp.num = 1
connect by prior sp.num + 1 = sp.num
and sp.type = prior sp.type;
- COLLECT function in Oracle 10g
Use the COLLECT function in Oracle 10g to get the same result. This method requires a table type and a function to convert the contents of the table type to a string.
Example:
CREATE OR REPLACE TYPE GTYP_STR_TABLE IS TABLE OF VARCHAR2 (32767);
/
CREATE OR REPLACE FUNCTION fun_tab_to_str(pv_tab IN gtyp_str_table,
pv_del IN VARCHAR2 DEFAULT ',')
RETURN VARCHAR2 IS
lv_result VARCHAR2(32767);
BEGIN
FOR i IN pv_tab.FIRST .. pv_tab.LAST
LOOP
IF i != pv_tab.FIRST
THEN
lv_result := lv_result || pv_del;
END IF;
lv_result := lv_result || pv_tab(i);
END LOOP;
RETURN lv_result;
END fun_tab_to_str;
/
The function is used as below:
select type, fun_tab_to_str(cast(collect(to_char(oid)) as gtyp_str_table))
from rcp_company c
group by type;
All above methods works fine if the combined string length is no more than 4000, however, there are occasions that the data from number of rows is so much that the combined length might more than 4000,to handle this problem, the return type of combined data can be lob type. There are some methods.
- Expanded generic Function using Ref Cursor
Example:
CREATE OR REPLACE FUNCTION join_clob(pv_cursor SYS_REFCURSOR,
pv_del VARCHAR2 := ',') RETURN CLOB IS
lv_value VARCHAR2(4000);
lv_result CLOB;
BEGIN
dbms_lob.createtemporary(lv_result, TRUE, dbms_lob.CALL);
LOOP
FETCH pv_cursor
INTO lv_value;
EXIT WHEN pv_cursor%NOTFOUND;
dbms_lob.writeappend(lv_result, length(pv_del), pv_del);
dbms_lob.writeappend(lv_result, length(lv_value), lv_value);
--lv_result := lv_result || lv_value;
END LOOP;
RETURN TRIM(pv_del FROM lv_result);
END join_clob;
The cursor function is used as below:
select type,
join_clob(CURSOR (select t2.oid from rcp_company t2 where t2.type = t1.type)
from rcp_company t1
group by type;
- Expended User-Defined Aggregate Function
CREATE OR REPLACE TYPE "GTYP_CLOBCAT2_OBJECT" IS OBJECT
(
catstr CLOB,
STATIC FUNCTION odciaggregateinitialize(cs_ctx IN OUT gtyp_clobcat2_object)
RETURN NUMBER,
MEMBER FUNCTION odciaggregateiterate(SELF IN OUT gtyp_clobcat2_object,
VALUE IN VARCHAR2) RETURN NUMBER,
MEMBER FUNCTION odciaggregatemerge(SELF IN OUT gtyp_clobcat2_object,
ctx2 IN OUT gtyp_clobcat2_object)
RETURN NUMBER,
MEMBER FUNCTION odciaggregateterminate(SELF IN OUT gtyp_clobcat2_object,
returnvalue OUT CLOB,
flags IN NUMBER)
RETURN NUMBER
)
/
CREATE OR REPLACE TYPE BODY gtyp_clobcat2_object IS
STATIC FUNCTION odciaggregateinitialize(cs_ctx IN OUT gtyp_clobcat2_object)
RETURN NUMBER IS
BEGIN
cs_ctx := gtyp_clobcat2_object(NULL);
dbms_lob.createtemporary(cs_ctx.catstr, TRUE, dbms_lob.CALL);
RETURN odciconst.success;
END;
MEMBER FUNCTION odciaggregateiterate(SELF IN OUT gtyp_clobcat2_object,
VALUE IN VARCHAR2) RETURN NUMBER IS
BEGIN
--dbms_lob.append(SELF.catstr, VALUE);
--dbms_lob.append(SELF.catstr, ',');
dbms_lob.writeappend(SELF.catstr, lengthb(VALUE), VALUE);
dbms_lob.writeappend(SELF.catstr, 1, ',');
RETURN odciconst.success;
END;
MEMBER FUNCTION odciaggregatemerge(SELF IN OUT gtyp_clobcat2_object,
ctx2 IN OUT gtyp_clobcat2_object)
RETURN NUMBER IS
BEGIN
dbms_lob.writeappend(SELF.catstr, 1, ',');
dbms_lob.append(SELF.catstr, ctx2.catstr);
RETURN odciconst.success;
END;
MEMBER FUNCTION odciaggregateterminate(SELF IN OUT gtyp_clobcat2_object,
returnvalue OUT CLOB,
flags IN NUMBER) RETURN NUMBER IS
BEGIN
returnvalue := TRIM(',' FROM SELF.catstr);
RETURN odciconst.success;
END;
END;
/
create or replace function fun_agg_clobcat2(pv_str varchar2) return clob
PARALLEL_ENABLE
AGGREGATE USING gtyp_clobcat2_object;
/
The aggregate function is implemented using a type and type body, and is used within a query as below:
select type, fun_agg_clobcat2(oid) from rcp_company group by type
- Expended COLLECT function in Oracle 10g
CREATE OR REPLACE TYPE GTYP_STR_TABLE IS TABLE OF VARCHAR2 (32767);
/
CREATE OR REPLACE FUNCTION fun_tab_to_clob(pv_tab IN gtyp_str_table,
pv_del IN VARCHAR2 DEFAULT ',')
RETURN CLOB IS
lv_result CLOB;
BEGIN
dbms_lob.createtemporary(lv_result, TRUE, dbms_lob.CALL);
FOR i IN pv_tab.FIRST .. pv_tab.LAST
LOOP
IF i != pv_tab.FIRST
THEN
dbms_lob.writeappend(lv_result, length(pv_del), pv_del);
--lv_result := lv_result || pv_del;
END IF;
dbms_lob.writeappend(lv_result, length(pv_tab(i)), pv_tab(i));
--lv_result := lv_result || pv_tab(i);
END LOOP;
RETURN lv_result;
END fun_tab_to_clob;
/
The function is used as below:
SELECT TYPE, fun_tab_to_clob(CAST(COLLECT(to_char(OID)) AS gtyp_str_table))
FROM rcp_company
GROUP BY TYPE;
As the number of rows might very large and return type is clob, the performance is badly impacted. Here is a simple test result for the three methods under the same circumstance in which the total records of rcp_company is 1190:
| Sql statement | SELECT TYPE, join_clob(CURSOR (SELECT t2.OID FROM rcp_company t2 WHERE t2.TYPE = t1.TYPE)) FROM rcp_company t1 GROUP BY TYPE; |
SELECT TYPE, fun_agg_clobcat2(OID) FROM rcp_company GROUP BY TYPE; |
SELECT TYPE, fun_tab_to_clob(CAST(COLLECT(to_char(OID)) AS gtyp_str_table)) FROM rcp_company GROUP BY TYPE; |
| Response time(s) | 0.215 | 1.497 | 0.185 |
See from the test result, Collect function method should be the better choose.