postgresql 物理备份 pg_rman 之二 backup
os: centos 7.4
postgresql: 9.6.9
pg_rman: REL9_6_STABLE
pg_rman 备份有全量备份、增量备份、归档备份，没错，pg_rman 实现了增量备份。
备份的目录通过 init 时指定一个路径，一般来说，会首先备份在本地的某个路径下，然后再通过 scp 或者 rsync 传输到异机上。
pg_rman使用的是独占型备份，所以当pg_rman正在备份时，不能启动其它的备份。
可以参考 http://postgres.cn/docs/9.6/continuous-archiving.html#BACKUP-BASE-BACKUP 理解下。
$ pg_rman --help
pg_rman manage backup/recovery of PostgreSQL database.
Usage:
pg_rman OPTION init
pg_rman OPTION backup
pg_rman OPTION restore
pg_rman OPTION show [DATE]
pg_rman OPTION show detail [DATE]
pg_rman OPTION validate [DATE]
pg_rman OPTION delete DATE
pg_rman OPTION purge
pg_rman 全量备份
$ pg_rman backup --backup-mode=full --with-serverlog --progress
INFO: copying database files
Processed 2319 of 2319 files, skipped 0
INFO: copying archived WAL files
Processed 6 of 6 files, skipped 0
INFO: copying server log files
Processed 20 of 20 files, skipped 0
INFO: backup complete
INFO: Please execute 'pg_rman validate' to verify the files are correctly copied.
校验备份集
pg_rman 的备份必须都是经过验证过的，否则不能进行恢复和增量备份
$ pg_rman show
=====================================================================
StartTime EndTime Mode Size TLI Status
=====================================================================
2018-12-03 11:53:39 2018-12-03 11:54:00 FULL 461MB 1 DONE
未校验备份集 Status 显示为 DONE，开始校验
$ pg_rman validate
INFO: validate: "2018-12-03 11:53:39" backup, archive log files and server log files by CRC
INFO: backup "2018-12-03 11:53:39" is valid
再次查看备份集
$ pg_rman show
=====================================================================
StartTime EndTime Mode Size TLI Status
=====================================================================
2018-12-03 11:53:39 2018-12-03 11:54:00 FULL 461MB 1 OK
Status 已经显示 OK
查看os的物理文件
$ ls -l /mnt/pg_rman_backupset/
total 8
drwx------ 3 postgres postgres 20 Dec 3 11:53 20181203
drwx------ 4 postgres postgres 35 Dec 3 11:51 backup
-rw-r--r-- 1 postgres postgres 75 Dec 3 11:51 pg_rman.ini
-rw-r--r-- 1 postgres postgres 40 Dec 3 11:51 system_identifier
drwx------ 2 postgres postgres 6 Dec 3 11:51 timeline_history
多了个 20181203 文件夹
$ cd 20181203/115339/
$ ls -l
total 156
drwx------ 2 postgres postgres 214 Dec 3 11:53 arclog
-rw-r--r-- 1 postgres postgres 457 Dec 3 11:55 backup.ini
drwx------ 20 postgres postgres 4096 Dec 3 11:53 database
-rw-r--r-- 1 postgres postgres 442 Dec 3 11:53 file_arclog.txt
-rw-r--r-- 1 postgres postgres 132860 Dec 3 11:53 file_database.txt
-rw-r--r-- 1 postgres postgres 1363 Dec 3 11:54 file_srvlog.txt
-rwx------ 1 postgres postgres 901 Dec 3 11:53 mkdirs.sh
drwx------ 2 postgres postgres 4096 Dec 3 11:54 srvlog
$ cat backup.ini
# configuration
BACKUP_MODE=FULL
FULL_BACKUP_ON_ERROR=false
WITH_SERVERLOG=true
COMPRESS_DATA=false
# result
TIMELINEID=1
START_LSN=3/d0000060
STOP_LSN=3/d0000168
START_TIME='2018-12-03 11:53:39'
END_TIME='2018-12-03 11:54:00'
RECOVERY_XID=11025
RECOVERY_TIME='2018-12-03 11:53:59'
TOTAL_DATA_BYTES=405025779
READ_DATA_BYTES=405025564
READ_ARCLOG_BYTES=83886392
READ_SRVLOG_BYTES=2231440
WRITE_BYTES=461491903
BLOCK_SIZE=8192
XLOG_BLOCK_SIZE=8192
STATUS=OK
里面纪律了详细的信息(重点看# result 下面的备注),关键信息如下
TIMELINEID=1
START_LSN=3/d0000060
STOP_LSN=3/d0000168
START_TIME='2018-12-03 11:53:39'
END_TIME='2018-12-03 11:54:00'
RECOVERY_XID=11025
RECOVERY_TIME='2018-12-03 11:53:59'
查看 xlog/wal 的归档文件依然存在，并没有删除。
$ ls -l /mnt/walbackup/
total 81924
-rw------- 1 postgres postgres 16777216 Dec 3 11:53 00000001000000030000006A
-rw------- 1 postgres postgres 16777216 Dec 3 11:53 00000001000000030000006B
-rw------- 1 postgres postgres 16777216 Dec 3 11:52 0000000100000003000000CE
-rw------- 1 postgres postgres 16777216 Dec 3 11:53 0000000100000003000000CF
-rw------- 1 postgres postgres 16777216 Dec 3 11:53 0000000100000003000000D0
-rw------- 1 postgres postgres 312 Dec 3 11:53 0000000100000003000000D0.00000060.backup
pg_rman 增量备份
增量备份是基于文件系统的update time时间线
增量备份必须有个对应的全库备份
生成增量表
postgres=# create table tmp_t3(c0 varchar(100),c1 varchar(100));
CREATE TABLE
postgres=# insert into tmp_t3(c0,c1) select id::varchar, md5(id::text) from generate_series(1,10000) as id;
INSERT 0 10000
开始增量备份
$ pg_rman backup --backup-mode=incremental --progress --compress-data
INFO: copying database files
Processed 2321 of 2321 files, skipped 2267
INFO: copying archived WAL files
Processed 11 of 11 files, skipped 6
INFO: backup complete
INFO: Please execute 'pg_rman validate' to verify the files are correctly copied.
$ pg_rman show
=====================================================================
StartTime EndTime Mode Size TLI Status
=====================================================================
2018-12-03 12:16:05 2018-12-03 12:16:10 INCR 8083kB 1 DONE
2018-12-03 11:53:39 2018-12-03 11:54:00 FULL 461MB 1 OK
$ pg_rman validate
INFO: validate: "2018-12-03 12:16:05" backup and archive log files by CRC
INFO: backup "2018-12-03 12:16:05" is valid
$ pg_rman show
=====================================================================
StartTime EndTime Mode Size TLI Status
=====================================================================
2018-12-03 12:16:05 2018-12-03 12:16:10 INCR 8083kB 1 OK
2018-12-03 11:53:39 2018-12-03 11:54:00 FULL 461MB 1 OK
pg_rman 再次增量备份
postgres=# create table tmp_t4(c0 varchar(100),c1 varchar(100));
CREATE TABLE
postgres=# insert into tmp_t4(c0,c1) select id::varchar, md5(id::text) from generate_series(1,10000) as id;
INSERT 0 10000
$ pg_rman backup --backup-mode=incremental --progress --compress-data
$ pg_rman validate
$ pg_rman show
=====================================================================
StartTime EndTime Mode Size TLI Status
=====================================================================
2018-12-03 13:01:14 2018-12-03 13:01:20 INCR 11MB 1 OK
2018-12-03 12:16:05 2018-12-03 12:16:10 INCR 8083kB 1 OK
2018-12-03 11:53:39 2018-12-03 11:54:00 FULL 461MB 1 OK
pg_rman 再次全量备份
$ pg_rman backup --backup-mode=full --with-serverlog --progress
$ pg_rman validate
$ pg_rman show
=====================================================================
StartTime EndTime Mode Size TLI Status
=====================================================================
2018-12-03 13:04:57 2018-12-03 13:05:18 FULL 443MB 1 OK
2018-12-03 13:01:14 2018-12-03 13:01:20 INCR 11MB 1 OK
2018-12-03 12:16:05 2018-12-03 12:16:10 INCR 8083kB 1 OK
2018-12-03 11:53:39 2018-12-03 11:54:00 FULL 461MB 1 OK
pg_rman 再再次增量备份
postgres=# create table tmp_t5(c0 varchar(100),c1 varchar(100));
CREATE TABLE
postgres=# insert into tmp_t5(c0,c1) select id::varchar, md5(id::text) from generate_series(1,10000) as id;
INSERT 0 10000
$ pg_rman backup --backup-mode=incremental --progress --compress-data -v
$ pg_rman validate
$ pg_rman show detail
======================================================================================================================
StartTime EndTime Mode Data ArcLog SrvLog Total Compressed CurTLI ParentTLI Status
======================================================================================================================
2018-12-03 13:07:24 2018-12-03 13:07:29 INCR 3186kB 67MB ---- 8138kB true 1 0 OK
2018-12-03 13:04:57 2018-12-03 13:05:18 FULL 406MB 67MB 28kB 443MB false 1 0 OK
2018-12-03 13:01:14 2018-12-03 13:01:20 INCR 2932kB 100MB ---- 11MB true 1 0 OK
2018-12-03 12:16:05 2018-12-03 12:16:10 INCR 3203kB 67MB ---- 8083kB true 1 0 OK
2018-12-03 11:53:39 2018-12-03 11:54:00 FULL 405MB 83MB 2231kB 461MB false 1 0 OK
通过添加 -v 参数，可以较详细看到增量备份的日志输出。
通过多次的完整备份和增量备份，来模拟线上的备份操作。
pg_rman 归档备份
归档备份就是静态文件的拷贝，就这么简单。
$ pg_rman backup --backup-mode=archive --progress --compress-data
$ pg_rman validate
$ pg_rman show
参考：
https://github.com/ossc-db/pg_rman/tree/master
http://ossc-db.github.io/pg_rman/index.html
https://travis-ci.org/ossc-db/pg_rman