Postgresql数据同步到Clickhouse

Postgresql的配置

在postgresql配置文件中将wal_level设置为logical，将max_replication_slots设置为2。

root@ugmwygs2obenrie:/etc/postgresql/12/main# ls
conf.d  environment  pg_ctl.conf  pg_hba.conf  pg_ident.conf  postgresql.conf  start.conf

root@ugmwygs2obenrie:/etc/postgresql/12/main# vim postgresql.conf 

root@ugmwygs2obenrie:/etc/postgresql/12/main# sudo systemctl restart postgresql  # 重启服务
root@ugmwygs2obenrie:/etc/postgresql/12/main# sudo systemctl status -l postgresql
● postgresql.service - PostgreSQL RDBMS
     Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
     Active: active (exited) since Mon 2023-06-19 22:51:16 CST; 14s ago
    Process: 30349 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
   Main PID: 30349 (code=exited, status=0/SUCCESS)

ClickHouse

ugmwygsWUYING.LOCAL :) CREATE DATABASE pg_demo_bin
                                                    ENGINE = MaterializedPostgreSQL('127.0.0.1:5432', 'demo_pg', 'ck', '123456');

CREATE DATABASE pg_demo_bin
ENGINE = MaterializedPostgreSQL('127.0.0.1:5432', 'demo_pg', 'ck', '123456')

Query id: 424f8216-f04f-4796-811e-5aa6977e8b35

Ok.

0 rows in set. Elapsed: 0.014 sec. 

坑

如果出现下面错误

ugmwygs2obenrie.CN-SHENZHEN-E-198-2.WUYING.LOCAL :) CREATE DATABASE pg_demo_bin
                                                    ENGINE = MaterializedPostgreSQL('127.0.0.1:5432', 'demo_pg', 'ck', '123456');

CREATE DATABASE pg_demo_bin
ENGINE = MaterializedPostgreSQL('127.0.0.1:5432', 'demo_pg', 'ck', '123456')

Query id: 424f8216-f04f-4796-811e-5aa6977e8b35

Ok.

0 rows in set. Elapsed: 0.014 sec. 

WUYING.LOCAL :) SET allow_experimental_database_materialized_postgresql=1   // 设置

SET allow_experimental_database_materialized_postgresql = 1

Query id: c3c6df01-7274-4827-8949-d02c4cd011c4

Ok.

参考链接
https://github.com/ClickHouse/ClickHouse/blob/master/docs/en/engines/database-engines/materialized-postgresql.md
https://clickhouse.com/docs/zh/engines/database-engines/materialized-postgresql
https://postgresqlco.nf/doc/zh/param/max_replication_slots/
pg配置归档
https://blog.csdn.net/whynotldch/article/details/78163480