pg_rman理解以及模拟过程

pg_rman使用的是pg_start_backup(), copy, pg_stop_backup()的备份模式。
pg_rman跑的不是流复制协议，而是文件拷贝，所以pg_rman必须和数据库节点跑在一起。
如果在standby节点跑pg_rman，pg_rman则需要通过网络连接到主节点执行pg_start_backup和pg_stop_backup。
注意：
pg_rman只从postgresql.conf取log_directory和archive_command参数的值。
如果你的PostgreSQL的配置文件是include的或者配置在postgresql.auto.conf中。
这两个值将不准确。
所以建议你仅仅把参数配置在postgresql.conf中，而不要使用其他配置文件。



1、安装pg_rman
操作系统：redhat 6.5
数据库版本：postgresql9.6.1
下载地址：https://github.com/ossc-db/pg_rman/releases
yum install postgres* -y
[root@pg ~]# cd /tmp
[root@pg tmp]#  rpm -ivh pg_rman-1.3.3-1.pg94.rhel6.x86_64.rpm 
error: Failed dependencies:
        postgresql94-libs is needed by pg_rman-1.3.3-1.pg94.rhel6.x86_64
出现上面的报错后我们需要下载和安装依赖包：
下载地址：http://yum.postgresql.org/9.4/redhat/rhel-6.5-x86_64/

[root@pg tmp]# rpm -ivh postgresql94-libs-9.4.11-1PGDG.rhel6.x86_64.rpm 
warning: postgresql94-libs-9.4.11-1PGDG.rhel6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 442df0f8: NOKEY
Preparing...                ########################################### [100%]
   1:postgresql94-libs      ########################################### [100%]
[root@pg tmp]#  rpm -ivh pg_rman-1.3.3-1.pg94.rhel6.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:pg_rman                ########################################### [100%]

安装完成后，pg_rman会安装在默认的/usr/pgsql-9.4/目录下.
2、环境变量的设置：
PGDATA=/opt/pgsql-9.4.4/data
PATH=/opt/pgsql-9.4.4/bin:$PATH:/usr/pgsql-9.4/bin
export LD_LIBRARY_PATH=/opt/pgsql-9.4.4/lib
export BACKUP_PATH=/backups
export PGDATA PATH


3、查看pg_rman的情况和参数：
[postgres @ localhost] $ pg_rman -help
pg_rman管理PostgreSQL数据库的备份/恢复。

用法：
pg_rman OPTION init
pg_rman选项备份
pg_rman选项恢复
pg_rman选项show [DATE]
pg_rman OPTION show detail [DATE]
pg_rman OPTION validate [DATE]
pg_rman OPTION delete DATE
pg_rman选项purge

常用选项：
-D，--pgdata =数据库存储区域的PATH位置
-A，--arclog-path =归档WAL存储区域的PATH位置
-S，--srvlog-path =服务器日志存储区的PATH位置
-B，--backup-path =备份存储区的PATH位置
-c，--check显示会做什么
-v，--verbose显示什么详细消息
-P，--progress显示已处理文件的进度

备份选项（备份参数）
-b，--backup-mode = MODE full，incremental或archive
-s，--with-serverlog也备份服务器日志文件
-Z，--compress-data使用zlib压缩数据备份
-C，--smooth-checkpoint在备份之前做平滑的检查点
-F，--full-backup-on-error切换到完全备份模式
如果pg_rman找不到验证完全备份
当前时间轴
注意：此选项仅在--backup-mode = incremental或archive中使用。
--keep-data-generations = NUM​​保留NUM代的完整数据备份
--keep-data-days = NUM​​保留足够的数据备份以恢复到N天前
--keep-arclog-files = NUM​​保留NUM个归档WAL
--keep-arclog-days = DAY保持在DAY天内修改的归档WAL
--keep-srvlog-files = NUM​​保留NUM个伺服器记录
--keep-srvlog-days = DAY在DAY天内修改伺服器记录
--standby-host = HOSTNAME从备用数据库进行备份时的备用主机
--standby-port =从备用备份时的PORT备用端口

恢复选项
--recovery-target-time时间戳，恢复将继续进行
--recovery-target-xid事务ID，恢复将继续进行
--recovery-target-inclusive是否在恢复目标之后停止
--recovery-target-timeline恢复到特定的时间线
- 硬拷贝复制archivelog不是符号链接

目录选项：
-a，--show-all也显示已删除的备份

删除选项：
-f，--force强制删除早于给定DATE的备份

连接选项：
-d，--dbname =要连接的DBNAME数据库
-h，--host = HOSTNAME数据库服务器主机或套接字目录
-p，--port = PORT数据库服务器端口
-U，--username = USERNAME要连接的用户名
-w，--no-password从不提示输入密码
-W，--password强制口令提示

一般选项：
-q，--quiet不显示任何INFO或DEBUG消息
--debug显示DEBUG消息
--help显示此帮助，然后退出
--version输出版本信息，然后退出

阅读网站了解详情。 
向报告错误。

4、创建归档目录：
mkdir /archive_log
chown postgres:postgres /archive_log
[postgres@localhost data]$ vi postgresql.conf

listen_addresses = '*'

port = 5432

wal_level = archive ------->>若为HOT STANDBY环境则此处设置为hot_standby

archive_mode = on

archive_command = 'test ! -f /archive_log/%f && cp %p /archive_log/%f'
log_destination = csvlog 
log_directory = pg_log

5、创建初始化目录（备份存放在该目录下）
[root@localhost ~]# mkdir /backups
[root@localhost ~]# chown postgres:postgres /backups/
[postgres@pg data]$ pg_rman init -B /backups
INFO: ARCLOG_PATH is set to '/archive_log'
INFO: SRVLOG_PATH is set to '/opt/pgsql-9.4.4/data/pg_log'
[postgres@pg data]$ cd /backups/
[postgres@pg backups]$ ll
total 16
drwx------ 4 postgres postgres 4096 Mar  6 12:54 backup
-rw-rw-r-- 1 postgres postgres   71 Mar  6 12:54 pg_rman.ini
-rw-rw-r-- 1 postgres postgres   40 Mar  6 12:54 system_identifier
drwx------ 2 postgres postgres 4096 Mar  6 12:54 timeline_history
[postgres@pg backups]$ cat /backups/pg_rman.ini
ARCLOG_PATH='/archive_log'
SRVLOG_PATH='/opt/pgsql-9.4.4/data/pg_log'

6、pg_rman 支持三种备份方式，全库、增量和归档
完全备份
备份整个数据库集群。
增量备份
仅使用相同的时间轴仅备份上次验证备份后修改的文件或页面。
存档WAL备份
仅备份归档WAL文件。
建议在备份后尽快验证备份文件。未经验证的备份不能用于还原或增量备份。



6.1全库备份：



[postgres@localhost ~]$ cat /backups/pg_rman.ini
ARCLOG_PATH='/arclog'
SRVLOG_PATH='/opt/pgsql/data/pg_log'



COMPRESS_DATA = YES
KEEP_ARCLOG_FILES = 10
KEEP_ARCLOG_DAYS = 10
KEEP_DATA_GENERATIONS = 3
KEEP_DATA_DAYS = 120
KEEP_SRVLOG_FILES = 10
KEEP_SRVLOG_DAYS = 10
执行全库备份：
[postgres@localhost ~]$ pg_rman backup --backup-mode=full
INFO: database backup start
NOTICE: pg_stop_backup complete, all required WAL segments have been archived
查看备份集：
[postgres@localhost ~]$ pg_rman show
==========================================================
StartTime Mode Duration Size TLI Status
==========================================================
2016-12-08 10:49:51 FULL 0m 2655kB 1 DONE
校验备份集（每次备份完，必须要做一次校验，否则备份集不可用用来恢复，增量备份时也不会用它来做增量比较。）
[postgres@localhost ~]$ pg_rman validate
INFO: validate: 2016-12-08 10:49:51 backup and archive log files by CRC
再次查看备份：
[postgres@localhost ~]$ pg_rman show
==========================================================
StartTime Mode Duration Size TLI Status
==========================================================
2016-12-08 10:49:51 FULL 0m 2655kB 1 OK



6.2增量备份：
[postgres@localhost ~]$ pg_rman backup --backup-mode=incremental --with-serverlog
查看备份集以及验证同上全库备份
6.3归档备份：
[postgres@localhost ~]$ pg_rman backup --backup-mode=archive --with-serverlog
查看备份集以及验证同上全库备份
7、删除备份
pg_rman delete "2016-12-08 10:49:51"
查看某个备份集的详细信息：
[postgres@localhost ~]$ pg_rman show 2016-12-08 10:49:51
# configuration
BACKUP_MODE=FULL
FULL_BACKUP_ON_ERROR=false
WITH_SERVERLOG=false
COMPRESS_DATA=true
# result
TIMELINEID=1
START_LSN=0/0e000028
STOP_LSN=0/0e0000f8
START_TIME='2016-12-08 10:49:51'
END_TIME='2016-12-08 10:49:55'
RECOVERY_XID=1691
RECOVERY_TIME='2016-12-08 10:49:54'
TOTAL_DATA_BYTES=22892947
READ_DATA_BYTES=22892792
READ_ARCLOG_BYTES=33554741
WRITE_BYTES=2655690
BLOCK_SIZE=8192
XLOG_BLOCK_SIZE=8192
STATUS=OK
8、恢复数据;
在数据库建一张表并插入数据，
postgres=# create table b (b int);
CREATE TABLE
postgres=# insert into b values (1);
INSERT 0 1
postgres=# insert into b values (2);
INSERT 0 1
postgres=# insert into b values (3);
INSERT 0 1
postgres=# select * from b;
b
---
1
2
3
(3 rows)

执行备份
[postgres@localhost ~]$ pg_rman backup --backup-mode=incremental
INFO: database backup start
NOTICE: pg_stop_backup complete, all required WAL segments have been archived
[postgres@localhost ~]$ pg_rman show
==========================================================
StartTime Mode Duration Size TLI Status
==========================================================
2017-03-06 13:25:29 FULL 0m 2655kB 1 OK
模拟崩溃：
[postgres@localhost~]$ killall -9 postgres
[postgres@localhost ~]$ rm -rf /opt/pgsql-9.4.4/data/*
建恢复文件：
[postgres@localhost data]$ vi recovery.conf
# recovery.conf generated by pg_rman 1.3.3
restore_command = 'cp /archive_log/%f %p'
recovery_target_time = '2017-03-06 13:25:29'
recovery_target_timeline = '1'

开始恢复：
[postgres@pg data]$ pg_rman restore --recovery-target-time "2017-03-06 13:25:29"
WARNING: could not open pg_controldata file "/opt/pgsql-9.4.4/data/global/pg_control": No such file or directory
WARNING: could not open pg_controldata file "/opt/pgsql-9.4.4/data/global/pg_control": No such file or directory
INFO: the recovery target timeline ID is not given
INFO: use timeline ID of latest full backup as recovery target: 1
INFO: calculating timeline branches to be used to recovery target point
INFO: searching latest full backup which can be used as restore start point
INFO: found the full backup can be used as base in recovery: "2017-03-06 13:01:20"
INFO: copying online WAL files and server log files
INFO: clearing restore destination
INFO: validate: "2017-03-06 13:01:20" backup and archive log files by SIZE
INFO: backup "2017-03-06 13:01:20" is valid
INFO: restoring database files from the full mode backup "2017-03-06 13:01:20"
INFO: searching incremental backup to be restored
INFO: validate: "2017-03-06 13:03:38" backup, archive log files and server log files by SIZE
INFO: backup "2017-03-06 13:03:38" is valid
INFO: restoring database files from the incremental mode backup "2017-03-06 13:03:38"
INFO: searching backup which contained archived WAL files to be restored
INFO: backup "2017-03-06 13:03:38" is valid
INFO: restoring WAL files from backup "2017-03-06 13:03:38"
INFO: validate: "2017-03-06 13:04:03" archive log files and server log files by SIZE
INFO: backup "2017-03-06 13:04:03" is valid
INFO: restoring WAL files from backup "2017-03-06 13:04:03"
INFO: backup "2017-03-06 13:25:29" is valid
INFO: restoring WAL files from backup "2017-03-06 13:25:29"
INFO: restoring online WAL files and server log files
INFO: generating recovery.conf
INFO: restore complete
HINT: Recovery will start automatically when the PostgreSQL server is started.
启动数据库：
[postgres@pg data]$ pg_ctl start
server starting
[postgres@pg data]$ LOG:  redirecting log output to logging collector process
HINT:  Future log output will appear in directory "pg_log".
[postgres@pg data]$ psql
psql (9.4.4)
Type "help" for help.
postgres=# select * from b;
 b 
---
 1
 2
 3
(3 rows)

postgres=# \d
        List of relations
 Schema | Name | Type  |  Owner   
--------+------+-------+----------
 public | aaa  | table | postgres
 public | b    | table | postgres
(2 rows)
数据恢复完成。



补充：

当出现两个全备和归档。时间线TLI会显示2.
TLI=timeline。