Postgres、hgdb物化视图实验
以下步骤依postgres 9.6版本为例：
postgres9.6@[local]:5432 postgres# select version();
version
---------------------------------------------------------------------------------------------------------
PostgreSQL 9.6.1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4), 64-bit
(1 row)
一、语法
CREATE MATERIALIZED VIEW table_name
[ (column_name [, ...] ) ]
[ WITH ( storage_parameter [= value] [, ... ] ) ]
[ TABLESPACE tablespace_name ]
AS query
[ WITH [ NO ] DATA ]
二、说明
storage_parameter是存储参数，诸如填充因子(fillfactor)等,tablespace可以指定表空间,比较关键的是后面的as query with [no] data,后面示例描述
三、示例
1.创建基础表
[postgres@primary ~]$ psql
psql (9.3.0)
Type "help" for help.
postgres=# create table highgo_T(id int,vname text);
CREATE TABLE
postgres=# insert into highgo_T select generate_series(1,20),'highgo_value_'||generate_series(1,20);
INSERT 0 20
postgres=# select * from highgo_T ;
id | vname
----+-------------------
1 | highgo_value_1
2 | highgo_value_2
3 | highgo_value_3
4 | highgo_value_4
5 | highgo_value_5
6 | highgo_value_6
7 | highgo_value_7
8 | highgo_value_8
9 | highgo_value_9
10 | highgo_value_10
11 | highgo_value_11
12 | highgo_value_12
13 | highgo_value_13
14 | highgo_value_14
15 | highgo_value_15
16 | highgo_value_16
17 | highgo_value_17
18 | highgo_value_18
19 | highgo_value_19
20 | highgo_value_20
(20 rows)
2.创建物化视图
postgres=# create materialized view mv_highgo_T as select * from highgo_T where id >10;
SELECT 10
postgres=# select * from mv_highgo_T;
id | vname
----+-------------------
11 | highgo_value_11
12 | highgo_value_12
13 | highgo_value_13
14 | highgo_value_14
15 | highgo_value_15
16 | highgo_value_16
17 | highgo_value_17
18 | highgo_value_18
19 | highgo_value_19
20 | highgo_value_20
(10 rows)
postgres=# \d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+----------------+-------------------+----------+-------+-------------
public | mv_highgo_T | materialized view | postgres | 16 kB |
public | highgo_T | table | postgres | 16 kB |
(2 rows)
--size有大小(默认空表是8kb,而这里是16kb)说明存储了数据，有相应的物理文件，并且有类似表的结构
postgres=# \d mv_highgo_T
Materialized view "public.mv_highgo_T"
Column | Type | Modifiers
--------+---------+-----------
id | integer |
vname | text |
--表和物化视图的文件地址
postgres=# select oid,pg_relation_filepath(oid),relpages from pg_class where relname = 'highgo_T'; -->无结果
postgres=# select oid,pg_relation_filepath(oid),relpages from pg_class where relname = 'highgo_t';
oid | pg_relation_filepath | relpages
-------+----------------------+----------
16396 | base/12896/16428 | 0
(1 row)
postgres=# select oid,pg_relation_filepath(oid),relpages from pg_class where relname = 'mv_highgo_t';
oid | pg_relation_filepath | relpages
-------+----------------------+----------
16459 | base/12896/16459 | 0
(1 row)
3.物化视图更新
postgres=# insert into highgo_T values(21,'bad boy');
INSERT 0 1
postgres=# insert into highgo_T values(22,'bad boy2');
INSERT 0 1
postgres=# select * from highgo_T where id>20;
id | vname
----+----------
21 | bad boy
22 | bad boy2
(2 rows)
postgres=# select * from mv_highgo_T where id>20;
id | vname
----+-------
(0 rows)
--物化视图的数据没有刷新过来
--手动刷新物化视图数据
postgres=# refresh materialized view mv_highgo_T;
REFRESH MATERIALIZED VIEW
postgres=# select * from mv_highgo_T where id>20;
id | vname
----+----------
21 | bad boy
22 | bad boy2
(2 rows)
--使用with no data刷新
postgres=# insert into highgo_T values(32,'bad boy3');
INSERT 0 1
postgres=# select * from mv_highgo_T where id>20;
id | vname
----+----------
21 | bad boy
22 | bad boy2
(2 rows)
postgres=# refresh materialized view mv_highgo_T with no data;
REFRESH MATERIALIZED VIEW
postgres=# \d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+----------------+-------------------+----------+------------+-------------
public | mv_highgo_T | materialized view | postgres | 8192 bytes |
public | highgo_T | table | postgres | 16 kB |
(2 rows)
postgres=# select * from mv_highgo_T;
ERROR: materialized view "mv_highgo_T" has not been populated
HINT: Use the REFRESH MATERIALIZED VIEW command.
说明：
使用了with no data刷新后会导致物化视图里面的数据清除干净，并使物化视图不可用，如果需要继续使用，需要使用REFRESH MATERIALIZED VIEW view_name来恢复。
四、增量刷新物化视图concurrently的使用说明（9.4之后的版本才支持增量刷新）
1、刷新语法：
--刷新语法
REFRESH MATERIALIZED VIEW [ CONCURRENTLY ] name
[ WITH [ NO ] DATA ]
2、源码
相关唯一索引的源码，在matview.c里面可以查看：
--先初始化唯一索引是false
foundUniqueIndex = false;
--如果找到唯一索引赋值为true
if (foundUniqueIndex)
appendStringInfoString(&querybuf, " AND ");
colname = quote_identifier(NameStr((tupdesc->attrs[attnum - 1])->attname));
appendStringInfo(&querybuf, "newdata.%s ", colname);
type = attnumTypeId(matviewRel, attnum);
op = lookup_type_cache(type, TYPECACHE_EQ_OPR)->eq_opr;
mv_GenerateOper(&querybuf, op);
appendStringInfo(&querybuf, " mv.%s", colname);
foundUniqueIndex = true;
--如果找不到唯一索引报error
if (!foundUniqueIndex)
ereport(ERROR,(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot refresh materialized view \"%s\" concurrently",matviewname),
errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
appendStringInfoString(&querybuf, " AND newdata OPERATOR(pg_catalog.*=) mv) "
"WHERE newdata IS NULL OR mv IS NULL " "ORDER BY tid");
3、增量刷新和非增量刷新时间对比
postgres9.6@[local]:5432 postgres# insert into highgo_T values(33,'bad boy3');
INSERT 0 1
Time: 4.961 ms
postgres9.6@[local]:5432 postgres# refresh materialized view mv_highgo_T;
REFRESH MATERIALIZED VIEW
Time: 27.304 ms --->非增量刷新耗时27.304 ms
postgres9.6@[local]:5432 postgres# insert into highgo_T values(34,'bad boy3');
INSERT 0 1
Time: 5.688 ms
postgres9.6@[local]:5432 postgres# refresh materialized view concurrently mv_highgo_T;
ERROR: cannot refresh materialized view "public.mv_highgo_t" concurrently
HINT: Create a unique index with no WHERE clause on one or more columns of the materialized view.
Time: 0.854 ms
postgres9.6@[local]:5432 postgres# create unique index idx_highgo_t_id on mv_highgo_T(id);
CREATE INDEX
Time: 18.879 ms
postgres9.6@[local]:5432 postgres# refresh materialized view concurrently mv_highgo_T;
REFRESH MATERIALIZED VIEW
Time: 43.970 ms --->增量刷新耗时43.970 ms
4、总结：
1.9.4之后版本的物化视图新增了concurrently参数，对物化视图的刷新不阻塞在该物化视图上的并发选择。如果没有这个选项，一次影响很多行的刷新将使用更少的资源并且更快结束，但是可能会阻塞其他尝试从物化视图中读取的连接。这个选项在只有少量行被影响的情况下可能会更快。
2.该参数的原理和优缺点与索引的concurrently类似，以时间来换取查询锁，刷新的速度会变得很慢
3.刷新参数with data是全量更新(replace)物化视图内容,且是默认参数；with no data会清除物化视图内容，释放物化视图所占的空间，并使物化视图不可用
4.当物化视图还未被填充时，concurrently选项不能被使用。
5.只有当物化视图上有至少一个UNIQUE索引（只用列名并且包括所有行）时，才允许concurrently选项。
6.即使带有concurrently选项，对于任意一个物化视图一次也只能运行一个REFRESH。
五、自动刷新雾化视图
1、使用\watch实现每隔几秒自动刷新一次
postgres9.6@[local]:5432 postgres# REFRESH MATERIALIZED VIEW CONCURRENTLY mv_highgo_T; \watch 3
REFRESH MATERIALIZED VIEW
Time: 26.326 ms
Thu 13 Dec 2018 12:34:40 AM CST (every 3s)
REFRESH MATERIALIZED VIEW
Time: 10.702 ms
Thu 13 Dec 2018 12:34:43 AM CST (every 3s) -->与上次间隔三秒
REFRESH MATERIALIZED VIEW
Time: 12.635 ms
Thu 13 Dec 2018 12:34:46 AM CST (every 3s) -->与上次间隔三秒
REFRESH MATERIALIZED VIEW
Time: 10.916 ms
Thu 13 Dec 2018 12:34:49 AM CST (every 3s) -->与上次间隔三秒
REFRESH MATERIALIZED VIEW
Time: 10.921 ms
Thu 13 Dec 2018 12:34:52 AM CST (every 3s) -->与上次间隔三秒
REFRESH MATERIALIZED VIEW
Time: 6.971 ms
Thu 13 Dec 2018 12:34:55 AM CST (every 3s) -->与上次间隔三秒
REFRESH MATERIALIZED VIEW
Time: 12.055 ms
# REFRESH MATERIALIZED VIEW CONCURRENTLY mv_highgo_T; \watch 3 & -->加上&后虽然可以执行，但ctrl+c结束后不能后台执行；示例如下：
REFRESH MATERIALIZED VIEW
Time: 14.529 ms
Thu 13 Dec 2018 12:35:23 AM CST (every 3s)
REFRESH MATERIALIZED VIEW
Time: 11.785 ms
Thu 13 Dec 2018 12:35:26 AM CST (every 3s) -->与上次间隔三秒
REFRESH MATERIALIZED VIEW
Time: 11.036 ms
postgres9.6@[local]:5432 postgres# insert into highgo_T values(36,'bad boy3');
INSERT 0 1
Time: 5.063 ms
postgres9.6@[local]:5432 postgres# select * from highgo_t;
id | vname
----+-----------------
...
36 | bad boy3
(27 rows)
Time: 0.436 ms
postgres9.6@[local]:5432 postgres# select * from mv_highgo_t;
id | vname
----+-----------------
...
35 | bad boy3
(16 rows)
postgres9.6@[local]:5432 postgres# REFRESH MATERIALIZED VIEW CONCURRENTLY mv_highgo_T;
REFRESH MATERIALIZED VIEW
Time: 14.629 ms
postgres9.6@[local]:5432 postgres# select * from mv_highgo_t;
id | vname
----+-----------------
...
35 | bad boy3
36 | bad boy3
(17 rows)
2、创建函数执行同步更新和触发器执行函数
create or replace function update_test_materialized_view() returns trigger as $$
declare
begin
REFRESH MATERIALIZED VIEW CONCURRENTLY public.mv_highgo_t;
return new;
end;
$$ language plpgsql;
CREATE TRIGGER insert_test_materialized_view AFTER INSERT ON public.highgo_t FOR EACH STATEMENT EXECUTE PROCEDURE update_test_materialized_view();
CREATE TRIGGER update_test_materialized_view AFTER UPDATE ON public.highgo_t FOR EACH STATEMENT EXECUTE PROCEDURE update_test_materialized_view();
CREATE TRIGGER delete_test_materialized_view AFTER DELETE ON public.highgo_t FOR EACH STATEMENT EXECUTE PROCEDURE update_test_materialized_view();
示例如下：
highgo=# create or replace function update_test_materialized_view() returns trigger as $$
highgo$# declare
highgo$# begin
highgo$# REFRESH MATERIALIZED VIEW CONCURRENTLY public.mv_highgo_t;
highgo$# return new;
highgo$# end;
highgo$# $$ language plpgsql;
CREATE FUNCTION
highgo=# CREATE TRIGGER update_test_materialized_view AFTER UPDATE ON public.highgo_t FOR EACH STATEMENT EXECUTE PROCEDURE update_test_materialized_view();
CREATE TRIGGER
highgo=# CREATE TRIGGER insert_test_materialized_view AFTER INSERT ON public.highgo_t FOR EACH STATEMENT EXECUTE PROCEDURE update_test_materialized_view();
CREATE TRIGGER
highgo=# CREATE TRIGGER delete_test_materialized_view AFTER DELETE ON public.highgo_t FOR EACH STATEMENT EXECUTE PROCEDURE update_test_materialized_view();
CREATE TRIGGER
highgo=# insert into highgo_t values (100,'a');
INSERT 0 1
highgo=# select * from mv_highgo_t where id=100;
id | vname
-----+-------
100 | a
(1 row)
highgo=# update highgo_t set vname='bb' where id=100;
UPDATE 1
highgo=# select * from mv_highgo_t where id=100;
id | vname
-----+-------
100 | bb
(1 row)
highgo=# delete from highgo_t where id=100;
DELETE 1
highgo=# select * from mv_highgo_t where id=100;
id | vname
----+-------
(0 rows)
highgo=# \df
...
oracle_catalog | vsize | integer | text | normal
public | pg_buffercache_pages | SETOF record | | normal
public | update_test_materialized_view | trigger | | trigger
(169 rows)
删除函数及触发器：
highgo=# drop function update_test_materialized_view() cascade;
注意: 串联删除3个其它对象
DETAIL: 递归删除 触发器 update_test_materialized_view 在 表 highgo_t
递归删除 触发器 insert_test_materialized_view 在 表 highgo_t
递归删除 触发器 delete_test_materialized_view 在 表 highgo_t
DROP FUNCTION
highgo=#
六.删除物化视图
postgres=# drop materialized view mv_highgo_T ;
DROP MATERIALIZED VIEW
postgres=#
--如果有其他约束在物化视图上，需要加cascade来级联删除
七、应用场景和优劣势
可以将复杂的SQL写成视图来调用，并可增大数据的安全性
另外物化视图与普通视图比因为直接扫描数据，通常扫描的数据更少，在有索引的支持下，效率更高，网络消耗也更少，特别是跨DB，跨服务器的查询
与普通视图相比的劣势是数据需要不定时地刷新才能获取到最实时的数据。
八、物化视图是在HGDB 3.0.0之后才有的功能。