postgres 简单流复制配置
postgres流复制类似于oracle的goldengate，在PostgreSQL在9.0之后引入了主备流复制机制，通过流复制，备库不断的从主库同步相应的数据，并在备库apply每个WAL record，这里的流复制每次传输单位是WAL日志的record。而PostgreSQL9.0之前提供的方法是主库写完一个WAL日志文件后，才把WAL日志文件传送到备库，这样的方式导致主备延迟特别大。
PG主备流复制的核心部分由walsender，walreceiver和startup三个进程组成。
walsender进程是用来发送WAL日志记录的
walreceiver进程是用来接收WAL日志记录的
startup进程是用来apply日志的
现在通过参考网上资料，把流复制的搭建过程重新整理如下：
配置环境
主机名 IP地址 角色
qxy 192.168.40.170 primary
qxy1 192.168.40.171 standby
主数据库配置
1、首先配置replica用户进行主从同步,并且具有登录和备份权限。
[postgres@qxy pgsql]$ psql
psql.bin (9.6.4)
Type "help" for help.
postgres=# CREATE ROLE replica login replication encrypted password 'replica';
CREATE ROLE
postgres=# 2、修改pg_hba.conf文件添加新增replica用户用于同步
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
#host all all 192.168.40.1/24 md5
host replication replica 192.168.40.1/24 md5 <====指定192.168.40网段的replica用户都可以连接
3、修改postgresql.conf 文件
wal_level = hot_standby # 备用服务器上增加了运行只读查询所需的信息
max_wal_senders = 5 # 这个设置了可以最多有几个流复制连接
wal_keep_segments = 128 # 设置流复制保留的最多的xlog数目
wal_sender_timeout = 60s # 设置流复制主机发送数据的超时时间4、启动postgres数据库
[postgres@qxy data]$ pg_ctl stop -D $PGDATA
waiting for server to shut down....LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
done
server stopped
[postgres@qxy data]$ pg_ctl start -D $PGDATA
server starting
[postgres@qxy data]$ LOG: database system was shut down at 2018-06-07 12:08:50 GMT
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started5、备数据库配置
备库执行
使用pg_basebackup建备库
[postgres@QXY1 data]$ pg_basebackup -F p --progress -D /spark/pgsql/data -h 192.168.40.170 -p 5432 -U replica --password
Password:
21856/21856 kB (100%), 1/1 tablespace
[postgres@QXY1 data]$
[postgres@QXY1 data]$ ls -ltr
total 116
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_stat_tmp
drwx------. 4 postgres dba 4096 Jun 7 20:23 pg_logical
-rw-------. 1 postgres dba 1636 Jun 7 20:23 pg_ident.conf
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_dynshmem
-rw-------. 1 postgres dba 206 Jun 7 20:23 backup_label
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_clog
drwx------. 3 postgres dba 4096 Jun 7 20:23 pg_xlog
drwx------. 4 postgres dba 4096 Jun 7 20:23 pg_multixact
-rw-------. 1 postgres dba 22592 Jun 7 20:23 postgresql.conf
-rw-------. 1 postgres dba 4 Jun 7 20:23 PG_VERSION
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_twophase
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_subtrans
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_snapshots
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_serial
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_notify
-rw-------. 1 postgres dba 4606 Jun 7 20:23 pg_hba.conf
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_commit_ts
drwx------. 5 postgres dba 4096 Jun 7 20:23 base
-rw-------. 1 postgres dba 88 Jun 7 20:23 postgresql.auto.conf
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_tblspc
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_stat
drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_replslot
drwx------. 2 postgres dba 4096 Jun 7 20:23 global
[postgres@QXY1 data]$ 6、修改备库recovery.conf，该文件在$PGHOME/share/postgres,把该文件copy到/spark/pgsql/data下面
[postgres@QXY1 postgresql]$ pwd
/spark/pgsql/share/postgresql
[postgres@QXY1 postgresql]$ ls -ltr recovery.conf.sample
-rw-r--r--. 1 postgres postgres 5587 Aug 9 2016 recovery.conf.sample
[postgres@QXY1 postgresql]$
[postgres@QXY1 postgresql]$mv recovery.conf.sample recovery.conf
修改如下内容：
standby_mode = on # 指定为从库
primary_conninfo = 'host=192.168.40.170 port=5432 user=replica password=replica' # 对应的主库信息
recovery_target_timeline = 'latest' # 这个说明这个流复制同步到最新的数据7、修改备库的postgresql.conf中几个参数
max_connections = 200
hot_standby = on
max_standby_streaming_delay = 30s
wal_receiver_status_interval = 10s
hot_standby_feedback = on 8、启动备库
[postgres@QXY1 pgsql]$ FATAL: database files are incompatible with server
DETAIL: The data directory was initialized by PostgreSQL version 9.6, which is not compatible with this version 9.4.9.
[postgres@QXY1 data]$ /spark/pgsql/pgsql/bin/pg_ctl start -D /spark/pgsql/data/
server starting
[postgres@QXY1 data]$ LOG: database system was shut down in recovery at 2018-06-07 14:29:47 GMT
LOG: entering standby mode
LOG: redo starts at 0/2000028
LOG: consistent recovery state reached at 0/3000000
LOG: database system is ready to accept read only connections
LOG: started streaming WAL from primary at 0/3000000 on timeline 1
9、主库查询replicate状态
[postgres@qxy spark]$ psql
psql.bin (9.6.4)
Type "help" for help.
postgres=# \x
Expanded display is on.
postgres=# select * from pg_stat_replication ;
-[ RECORD 1 ]----+------------------------------
pid | 5382
usesysid | 16385
usename | replica
application_name | walreceiver
client_addr | 192.168.40.171
client_hostname |
client_port | 58399
backend_start | 2018-06-07 14:33:04.238161+00
backend_xmin | 548
state | streaming
sent_location | 0/3001720
write_location | 0/3001720
flush_location | 0/3001720
replay_location | 0/3001720
sync_priority | 0
sync_state | async <======异步方式
postgres=#
10、修改主库postgresql.conf配置文件
synchronous_standby_names = 'standby' # standby servers that provide sync rep11、修改备库recovery.conf配置文件
primary_conninfo = 'application_name=standby host=192.168.40.170 port=5432 user=replica password=replica' # 对应的主库信息12、重启主库和备库
主库查看replicate状态
postgres=# select * from pg_stat_replication ;
-[ RECORD 1 ]----+------------------------------
pid | 5482
usesysid | 16385
usename | replica
application_name | standby
client_addr | 192.168.40.171
client_hostname |
client_port | 58404
backend_start | 2018-06-07 14:45:09.909992+00
backend_xmin | 548
state | streaming
sent_location | 0/3001988
write_location | 0/3001988
flush_location | 0/3001988
replay_location | 0/3001988
sync_priority | 1
sync_state | sync <=====更新为同步
postgres=# 13、测试同步情况
主库创建一个测试表T
postgres=#
postgres=# create table t(id int);
CREATE TABLE
postgres=# insert into t values (10);
INSERT 0 1
postgres=#
备库查询
[postgres@QXY1 ~]$ psql
psql.bin (9.6.4)
Type "help" for help.
postgres=#
postgres=#
postgres=# select * from t;
id
----
10
(1 row)
postgres=# 14、对应后台进程
主库
[postgres@qxy data]$ ps -ef |grep postgres
root 3820 2904 0 11:14 pts/0 00:00:00 su - postgres
postgres 3821 3820 0 11:14 pts/0 00:00:00 -bash
postgres 5473 1 0 14:45 pts/0 00:00:00 /spark/pgsql/pgsql/bin/postgres -D /spark/pgsql/data
postgres 5476 5473 0 14:45 ? 00:00:00 postgres: checkpointer process
postgres 5477 5473 0 14:45 ? 00:00:00 postgres: writer process
postgres 5478 5473 0 14:45 ? 00:00:00 postgres: wal writer process
postgres 5479 5473 0 14:45 ? 00:00:00 postgres: autovacuum launcher process
postgres 5480 5473 0 14:45 ? 00:00:00 postgres: stats collector process
postgres 5482 5473 0 14:45 ? 00:00:00 postgres: wal sender process replica 192.168.40.171(58404) streaming 0/3017528
postgres 5516 3821 17 14:48 pts/0 00:00:00 ps -ef
postgres 5517 3821 0 14:48 pts/0 00:00:00 grep postgres
[postgres@qxy data]$
备库
[postgres@QXY1 ~]$ ps -ef |grep postgres
root 2564 2019 0 20:49 pts/0 00:00:00 su - postgres
postgres 2565 2564 0 20:49 pts/0 00:00:00 -bash
root 2922 2901 0 22:29 pts/1 00:00:00 su - postgres
postgres 2923 2922 0 22:29 pts/1 00:00:00 -bash
postgres 3033 1 0 22:43 pts/0 00:00:00 /spark/pgsql/pgsql/bin/postgres -D /spark/pgsql/data
postgres 3034 3033 0 22:43 ? 00:00:00 postgres: startup process recovering 000000010000000000000003
postgres 3035 3033 0 22:43 ? 00:00:00 postgres: checkpointer process
postgres 3036 3033 0 22:43 ? 00:00:00 postgres: writer process
postgres 3037 3033 0 22:43 ? 00:00:00 postgres: stats collector process
postgres 3049 3033 0 22:44 ? 00:00:00 postgres: wal receiver process streaming 0/3017528
postgres 3061 2565 0 22:49 pts/0 00:00:00 ps -ef
postgres 3062 2565 0 22:49 pts/0 00:00:00 grep postgres
[postgres@QXY1 ~]$