TimesTen的列压缩功能
本文出处为：http://ggsig.blogspot.jp/2012/02/columnar-compression-in-timesten.html
并参考了ＭOS Doc ID 1454651.1
Columnar Compression是11.2.2新增的功能，和Oracle一样，所谓的压缩其实就是重复数据删除，是通过类似于look-up table的技术实现的。
语法可参见SQL Reference. 原理如下图：
在字典表中存放了此压缩列不同的值，原始表中存放的是指针，实际数据在字典表中
先来看一个例子，了解一下原理:
create table emp_comp (
id number not null,
val_1 varchar2(40),
val_2 varchar2(40),
val_3 varchar2(40))
compress ((val_1,val_2) by dictionary maxvalues = 255,
val_3 by dictionary maxvalues = 255)
optimized for read;
Command> desc emp_comp;
Table ORACLE.EMP_COMP:
Columns:
ID NUMBER NOT NULL
VAL_1 VARCHAR2 (40) INLINE
VAL_2 VARCHAR2 (40) INLINE
VAL_3 VARCHAR2 (40) INLINE
COMPRESS ( ( VAL_1, VAL_2 ) BY DICTIONARY MAXVALUES=255,
VAL_3 BY DICTIONARY MAXVALUES=255 ) OPTIMIZED FOR READ
1 table found.
(primary key columns are indicated with *)
Command> desc emp_comp;
Table ORACLE.EMP_COMP:
Columns:
ID NUMBER NOT NULL
VAL_1 VARCHAR2 (40) INLINE
VAL_2 VARCHAR2 (40) INLINE
VAL_3 VARCHAR2 (40) INLINE
COMPRESS ( ( VAL_1, VAL_2 ) BY DICTIONARY MAXVALUES=255,
VAL_3 BY DICTIONARY MAXVALUES=255 ) OPTIMIZED FOR READ
1 table found.
(primary key columns are indicated with *)
在上例中, optimized for read启用了列压缩，compress定义了压缩的列/列组，在本例中，val_1和val_2为压缩组1，val_3为压缩组2
maxvalues定义了指针大小，255个不同值只需要一个字节存地址即可。
optimized for read定义了压缩的级别，目前仅支持一种级别，另外也说明了列压缩适用于读较多的查询分析，而不是DML较多的交易
Command> alltables;
ORACLE.CD$_1086064_2
ORACLE.CD$_1086064_4
ORACLE.EMP_COMP
可以看到，系统为EMP_COMP表建立了两个字典表，因为有两个压缩列组
数据字典表表名的格式为：
“CD$” + the table identifier (SYS.TABLES.TBLID) + compressed column number
Command> select tblname, tblid from sys.tables where tblname = 'EMP_COMP';
< EMP_COMP , 1086064 >
1 row found.知道了table ID为1086064后，可以查询出相关字典表的信息：
Command> select tblname from sys.tables where tblname like 'CD$_%1086064%';
< CD$_1086064_2 >
< CD$_1086064_4 >
2 rows found.
Command> desc CD$_1086064_2;
Table ORACLE.CD$_1086064_2:
Columns:
*VAL_1 VARCHAR2 (40) INLINE
*VAL_2 VARCHAR2 (40) INLINE
##CD_REFCNT TT_INTEGER NOT NULL
1 table found.
(primary key columns are indicated with *)
Command> desc CD$_1086064_4;
Table ORACLE.CD$_1086064_4:
Columns:
*VAL_3 VARCHAR2 (40) INLINE
##CD_REFCNT TT_INTEGER NOT NULL
1 table found.
(primary key columns are indicated with *)
以下是数据字典中关于表压缩的信息：
Command>
VARIABLE TABNAME VARCHAR2(50) := 'EMP_COMP';
select TABLE_NAME, COMPRESSION, COMPRESS_FOR from sys.all_tables
where TABLE_NAME = :TABNAME;
< EMP_COMP, ENABLED, QUERY HIGH >
1 row found.
以下查询表示有3列被压缩
select tblname, numcompress, valtblids from sys.tables where TBLNAME=:TABNAME;
< EMP_COMP , 3, 7892100088921000 >
1 row found.
以下是一个未启用压缩和启用压缩的表的空间使用对比
先建立一个没有压缩的表test2，并插入100万条数据
create table test2 ( id number not null, val_1 varchar2(40), val_2 varchar2(40), val_3 varchar2(40) );
begin for i in 1 .. 1000000 loop insert into test2 values (i,'1234567890123456789012345678901234567890', '1234567890123456789012345678901234567890', '1234567890123456789012345678901234567890');
end loop;
end;
/然后查看表的大小为171539960 bytes
Command> call ttComputeTabSizes('test2');
Command> tablesize test2;
Sizes of ORACLE.TEST2:
INLINE_ALLOC_BYTES: 171470416
NUM_USED_ROWS: 1000000
NUM_FREE_ROWS: 192
AVG_ROW_LEN: 171
OUT_OF_LINE_BYTES: 0
METADATA_BYTES: 69544
TOTAL_BYTES: 171539960
LAST_UPDATED: 2016-04-02 05:53:59.000000
1 table found.
再建一个只压缩一列的表test2_comp，插入100万条数据
create table test2_comp ( id number not null, val_1 varchar2(40), val_2 varchar2(40), val_3 varchar2(40)) compress (val_1 by dictionary maxvalues = 255) optimized for read;
begin for i in 1 .. 1000000 loop insert into test2_comp values (i,'1234567890123456789012345678901234567890', '1234567890123456789012345678901234567890', '1234567890123456789012345678901234567890');
end loop;
end;查看压缩表的空间占用
Command> call ttComputeTabSizes('test2_comp');
Command> tablesize test2_comp;
Sizes of ORACLE.TEST2_COMP:
INLINE_ALLOC_BYTES: 131462736
NUM_USED_ROWS: 1000000
NUM_FREE_ROWS: 192
AVG_ROW_LEN: 131
OUT_OF_LINE_BYTES: 0
METADATA_BYTES: 87612
TOTAL_BYTES: 131550348
LAST_UPDATED: 2016-04-02 05:59:20.000000
1 table found.
表的大小由171539960 减少到131550348字节，节省了38M
由于10万条数据是相同的，因此节省的空间为(1000000-1)*40=39999600字节
最后再做一个实验，注意maxvalues变为了1000000
先建表并仅插入一条记录
create table test2_comp ( id number not null, val_1 varchar2(40), val_2 varchar2(40), val_3 varchar2(40)) compress (val_1 by dictionary maxvalues = 1000000) optimized for read;
insert into test2_comp values (1,'1234567890123456789012345678901234567890', '1234567890123456789012345678901234567890', '1234567890123456789012345678901234567890');Command> call ttComputeTabSizes('test2_comp');
Command> tablesize test2_comp;
Sizes of ORACLE.TEST2_COMP:
INLINE_ALLOC_BYTES: 33648
NUM_USED_ROWS: 1
NUM_FREE_ROWS: 255
AVG_ROW_LEN: 205
OUT_OF_LINE_BYTES: 0
METADATA_BYTES: 18832
TOTAL_BYTES: 52480
LAST_UPDATED: 2016-04-02 08:43:44.000000
1 table found.
Command> select tblname, tblid from sys.tables where tblname = 'TEST2_COMP';
< TEST2_COMP , 1086096 >
1 row found.
Command> select tblname from sys.tables where tblname like 'CD$_%1086096%';
< CD$_1086096_2 >
1 row found.
Command> select count(*) from CD$_1086096_2;
< 1 >
1 row found.其中33648约为一个页面的大小，可以存放256行，已占用1行。字典表占用的空间包含在18832字节元数据中
接着再插入256条数据，我们可以看到，数据页面增加了一页，而元数据即字典表的空间不变
begin
for i in 2 .. 257 loop
insert into test2_comp
values (i,'1234567890123456789012345678901234567890',
'1234567890123456789012345678901234567890',
'1234567890123456789012345678901234567890');
end loop;
end;
/
Command> call ttComputeTabSizes('test2_comp');
Command> tablesize test2_comp;
Sizes of ORACLE.TEST2_COMP:
INLINE_ALLOC_BYTES: 67296
NUM_USED_ROWS: 257
NUM_FREE_ROWS: 255
AVG_ROW_LEN: 168
OUT_OF_LINE_BYTES: 0
METADATA_BYTES: 18832
TOTAL_BYTES: 86128
LAST_UPDATED: 2016-04-02 08:54:14.000000
1 table found.
Command> select count(*) from CD$_1086096_2; < 1 >
1 row found.
再插入255条数据，恰好把NUM_FREE_ROWS填满，我们可以看到数据页面和元数据空间都没有变化。注意此时插入了不同的值。结果中CD$_1086096_2行数为256，表示有256个不同的值
begin
for i in 257 .. 511 loop
insert into test2_comp
values (i,'1234567890123456789012345678901000000'||i,
'1234567890123456789012345678901234567890',
'1234567890123456789012345678901234567890');
end loop;
end;
/
Command> call ttComputeTabSizes('test2_comp');
Command> tablesize test2_comp;
Sizes of ORACLE.TEST2_COMP:
INLINE_ALLOC_BYTES: 67296
NUM_USED_ROWS: 512
NUM_FREE_ROWS: 0
AVG_ROW_LEN: 168
OUT_OF_LINE_BYTES: 0
METADATA_BYTES: 18832
TOTAL_BYTES: 86128
LAST_UPDATED: 2016-04-02 08:58:35.000000
1 table found.
Command> select count(*) from CD$_1086096_2; < 256 >
1 row found.
总之，重复的数据越多，压缩比越高
列压缩也存在一些限制，如
* LOB列不支持压缩
* cache table，复制表也不支持压缩