pg备份还原工具--pg_rman
一、系统
IP HOSTNAME PG VERSION DIR BACKUPSETS PATH OS
192.168.100.147 pg 9.3.4 /opt/pgsql /backup CentOS6.4_x64
[root@barman ~]# cat /etc/issue CentOS release 6.5 (Final) Kernel \r on an \m [root@barman ~]# uname -a Linux barman 2.6.32-431.11.2.el6.x86_64 #1 SMP Tue Mar 25 19:59:55 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
二、安装
2.1 简介
pg_rman is an online backup and restore tool for PostgreSQL.
The goal of the pg_rman project is providing a method for online backup and PITR as easy as pg_dump. Also, it maintains a backup catalog per database cluster. Users can maintain old backups including archive logs with one command.
pg_rman is a utility program to backup and restore PostgreSQL database. It takes a physical online backup of whole database cluster, archive WALs, and server logs.
pg_rman supports getting backup from standby-site with PostgreSQL 9.0 later.
And pg_rman supports storage snapshot backup.
pg_rman的开发语言为C。
pg_rman的版本更新情况：
平台需求：
Linux (and some of UNIXes)
PostgreSQL 9.3, 9.2, 9.1, 9.0, 8.4 - 8.2
目前已通过测试的平台：
不支持的平台：
Windows
PostgreSQL 8.1 or prior versions.
2.2 安装依赖包
下载安装postgresql93-libs
http://yum.postgresql.org/9.3/redhat/rhel-6.4-x86_64/
[root@pg ~]# rpm -ivh postgresql93-libs-9.3.4-1PGDG.rhel6.x86_64.rpm warning: postgresql93-libs-9.3.4-1PGDG.rhel6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 442df0f8: NOKEY Preparing... ########################################### [100%] 1:postgresql93-libs ########################################### [100%]
2.3 安装pg_rman
下载地址：http://sourceforge.net/projects/pg-rman/files/
[root@pg ~]# rpm -ivh pg_rman-1.2.8-1.pg93.rhel6.x86_64.rpm Preparing... ########################################### [100%] 1:pg_rman ########################################### [100%]
将被安装在/usr/pgsql-9.3/bin/pg_rman
源码编译安装
安装类似pg扩展模块的安装
$ make USE_PGXS=1 $ make USE_PGXS=1 install
三、安装配置数据库
3.1 安装
安装过程略
3.2 创建用户
在pg上创建postgres用户并设置密码：
[root@pg ~]# useradd postgres [root@pg ~]# passwd postgres
配置postgres用户环境变量：
export PATH=/usr/pgsql-9.3/bin:/opt/pgsql/bin:$PATH:$HOME/bin export PGDATA=/opt/pgsql/data export PGUSER=postgres export PGPORT=5432 export LD_LIBRARY_PATH=/opt/pgsql/lib:$LD_LIBRARY_PATH export BACKUP_PATH=/backup
3.3 配置
创建归档目录：
[root@pg ~]# mkdir /arclog [root@pg ~]# chown postgres:postgres /arclog [postgres@pg data]$ vim postgresql.conf listen_addresses = '*' port = 5432 wal_level = archive 若为HOT STANDBY环境则此处设置为hot_standby archive_mode = on archive_command = 'test ! -f /arclog/%f && cp %p /arclog/%f'
四、pg_rman管理
[postgres@pg ~]$ pg_rman --help pg_rman manage backup/recovery of PostgreSQL database. Usage: pg_rman OPTION init pg_rman OPTION backup pg_rman OPTION restore pg_rman OPTION show [DATE] pg_rman OPTION show timeline [DATE] pg_rman OPTION validate [DATE] pg_rman OPTION delete DATE Common Options: -D, --pgdata=PATH location of the database storage area -A, --arclog-path=PATH location of archive WAL storage area -S, --srvlog-path=PATH location of server log storage area -B, --backup-path=PATH location of the backup storage area -c, --check show what would have been done Backup options: -b, --backup-mode=MODE full, incremental, or archive -s, --with-serverlog also backup server log files -Z, --compress-data compress data backup with zlib -C, --smooth-checkpoint do smooth checkpoint before backup --keep-data-generations=N keep GENERATION of full data backup --keep-data-days=DAY keep enough data backup to recover to DAY days age --keep-arclog-files=NUM keep NUM of archived WAL --keep-arclog-days=DAY keep archived WAL modified in DAY days --keep-srvlog-files=NUM keep NUM of serverlogs --keep-srvlog-days=DAY keep serverlog modified in DAY days --standby-host=HOSTNAME standby host when taking backup from standby --standby-port=PORT standby port when taking backup from standby Restore options: --recovery-target-time time stamp up to which recovery will proceed --recovery-target-xid transaction ID up to which recovery will proceed --recovery-target-inclusive whether we stop just after the recovery target --recovery-target-timeline recovering into a particular timeline --hard-copy copying archivelog not symbolic link Catalog options: -a, --show-all show deleted backup too Connection options: -d, --dbname=DBNAME database to connect -h, --host=HOSTNAME database server host or socket directory -p, --port=PORT database server port -U, --username=USERNAME user name to connect as -w, --no-password never prompt for password -W, --password force password prompt Generic options: -q, --quiet don't write any messages --debug debug mode --help show this help, then exit --version output version information, then exit Read the website for details. <http://sourceforge.net/projects/pg-rman/> Report bugs to <http://sourceforge.net/p/pg-rman/tickets/>.
4.1 初始化备份目录
[root@pg ~]# mkdir /backup [root@pg ~]# chown postgres:postgres /backup/ [postgres@pg ~]$ pg_rman init -B /backup INFO: ARCLOG_PATH is set to '/arclog' INFO: SRVLOG_PATH is set to '/opt/pgsql/data/pg_log' [postgres@pg ~]$ ll /backup/ total 12 drwx------ 4 postgres postgres 4096 Apr 11 07:17 backup -rw-rw-r-- 1 postgres postgres 60 Apr 11 07:17 pg_rman.ini drwx------ 2 postgres postgres 4096 Apr 11 07:17 timeline_history [postgres@pg ~]$ cat /backup/pg_rman.ini ARCLOG_PATH='/arclog' SRVLOG_PATH='/opt/pgsql/data/pg_log'
4.2 备份
Backup options: -b, --backup-mode=MODE full, incremental, or archive -s, --with-serverlog also backup server log files -Z, --compress-data compress data backup with zlib -C, --smooth-checkpoint do smooth checkpoint before backup --keep-data-generations=N keep GENERATION of full data backup --keep-data-days=DAY keep enough data backup to recover to DAY days age --keep-arclog-files=NUM keep NUM of archived WAL --keep-arclog-days=DAY keep archived WAL modified in DAY days --keep-srvlog-files=NUM keep NUM of serverlogs --keep-srvlog-days=DAY keep serverlog modified in DAY days --standby-host=HOSTNAME standby host when taking backup from standby --standby-port=PORT standby port when taking backup from standby
各项详细解释参见http://sourceforge.net/p/pg-rman/wiki/User%27s_Manual/中“Backup options”部分。
4.2.1 全库备份
[postgres@pg ~]$ cat /backup/pg_rman.ini ARCLOG_PATH='/arclog' SRVLOG_PATH='/opt/pgsql/data/pg_log' COMPRESS_DATA = YES KEEP_ARCLOG_FILES = 10 KEEP_ARCLOG_DAYS = 10 KEEP_DATA_GENERATIONS = 3 KEEP_DATA_DAYS = 120 KEEP_SRVLOG_FILES = 10 KEEP_SRVLOG_DAYS = 10
执行全库备份：
[postgres@pg ~]$ pg_rman backup --backup-mode=full INFO: database backup start NOTICE: pg_stop_backup complete, all required WAL segments have been archived
查看备份集：
[postgres@pg ~]$ pg_rman show ============================================================================ Start Time Total Data WAL Log Backup Status ============================================================================ 2014-04-11 08:14:31 0m 20MB ---- 83MB ---- 4426kB DONE
验证：
[postgres@pg ~]$ pg_rman validate INFO: validate: 2014-04-11 08:14:31 backup and archive log files by CRC
再次查看备份集：
[postgres@pg ~]$ pg_rman show ============================================================================ Start Time Total Data WAL Log Backup Status ============================================================================ 2014-04-11 08:14:31 0m 20MB ---- 83MB ---- 4426kB OK
各列含义解释：
Start : start time of backup
Time : total time of backup
Total : size of whole database cluster
Data : size of read data files
WAL : size of read WAL archive files
Log : size of read serverlog files
Backup: size of backup (= written size)
Status: status of backup. Possible values are:
OK : backup is done and validated.
DONE : backup is done, but not validated yet.
RUNNING : backup is running now.
DELETING : backup is being deleted now.
DELETED : backup has been deleted.
ERROR : backup is unavailable because some errors occur during backup.
CORRUPT : backup is unavailable because it is broken.
4.2.2 增量备份
执行备份：
[postgres@pg ~]$ pg_rman backup --backup-mode=incremental --with-serverlog INFO: database backup start NOTICE: pg_stop_backup complete, all required WAL segments have been archived
验证：
[postgres@pg ~]$ pg_rman validate INFO: validate: 2014-04-11 08:17:06 backup and archive log files by CRC
查看备份集信息：
[postgres@pg ~]$ pg_rman show ============================================================================ Start Time Total Data WAL Log Backup Status ============================================================================ 2014-04-11 08:17:06 0m ---- 16kB 33MB 0B 79kB OK 2014-04-11 08:14:31 0m 20MB ---- 83MB ---- 4426kB OK [postgres@pg ~]$ pg_rman show timeline ============================================================ Start Mode Current TLI Parent TLI Status ============================================================ 2014-04-11 08:17:06 INCR 1 0 OK 2014-04-11 08:14:31 FULL 1 0 OK
4.2.3 归档备份
[postgres@pg ~]$ pg_rman backup --backup-mode=archive --with-serverlog [postgres@pg ~]$ pg_rman validate INFO: validate: 2014-04-11 08:21:59 archive log files by CRC [postgres@pg ~]$ pg_rman show timeline ============================================================ Start Mode Current TLI Parent TLI Status ============================================================ 2014-04-11 08:21:59 ARCH 1 0 OK 2014-04-11 08:17:06 INCR 1 0 OK 2014-04-11 08:14:31 FULL 1 0 OK [postgres@pg ~]$ pg_rman show ============================================================================ Start Time Total Data WAL Log Backup Status ============================================================================ 2014-04-11 08:21:59 0m ---- ---- 16MB 0B 2000kB OK 2014-04-11 08:17:06 0m ---- 16kB 33MB 0B 79kB OK 2014-04-11 08:14:31 0m 20MB ---- 83MB ---- 4426kB OK
4.3 查看备份集信息
上面已经设计，除此之外，也可直接指定某个备份集来查看详细信息：
[postgres@pg ~]$ pg_rman show 2014-04-11 08:14:31 # configuration BACKUP_MODE=FULL WITH_SERVERLOG=false COMPRESS_DATA=true # result TIMELINEID=1 START_LSN=0/25000028 STOP_LSN=0/250000b8 START_TIME='2014-04-11 08:14:31' END_TIME='2014-04-11 08:14:35' RECOVERY_XID=1836 RECOVERY_TIME='2014-04-11 08:14:34' TOTAL_DATA_BYTES=20495299 READ_DATA_BYTES=20495299 READ_ARCLOG_BYTES=83887037 WRITE_BYTES=4426584 BLOCK_SIZE=8192 XLOG_BLOCK_SIZE=8192 STATUS=OK
注意RECOVERY_XID与RECOVERY_TIME，在恢复时分别对应参数--recovery-target-xid与--recovery-target-time。
4.4 查看全部备份集（包括已删除）
[postgres@pg ~]$ pg_rman show -a ============================================================================ Start Time Total Data WAL Log Backup Status ============================================================================ 2014-04-11 08:43:18 0m ---- 16kB 67MB 0B 2096kB OK 2014-04-11 08:43:09 0m 20MB ---- 33MB 0B 2402kB OK 2014-04-11 08:41:29 0m ---- 16kB 67MB 0B 102kB DELETED 2014-04-11 08:40:54 0m 20MB ---- 33MB 0B 2377kB DELETED 2014-04-11 08:21:59 0m ---- ---- 16MB 0B 2000kB DELETED 2014-04-11 08:17:06 0m ---- 16kB 33MB 0B 79kB DELETED 2014-04-11 08:14:31 0m 20MB ---- 83MB ---- 4426kB DELETED
4.5 删除备份集
（删除指定时间之前恢复时不再需要的备份，将保留指定时间点所在时间段及之后的全库备份及其它备份）
[postgres@pg ~]$ pg_rman show timeline ============================================================ Start Mode Current TLI Parent TLI Status ============================================================ 2014-04-11 08:43:18 INCR 1 0 OK 2014-04-11 08:43:09 FULL 1 0 OK 2014-04-11 08:41:29 INCR 1 0 OK 2014-04-11 08:40:54 FULL 1 0 OK [postgres@pg ~]$ pg_rman delete 2014-04-11 08:43:18 INFO: delete: 2014-04-11 08:41:29 INFO: delete: 2014-04-11 08:40:54 [postgres@pg ~]$ pg_rman show timeline ============================================================ Start Mode Current TLI Parent TLI Status ============================================================ 2014-04-11 08:43:18 INCR 1 0 OK 2014-04-11 08:43:09 FULL 1 0 OK
4.6 对STANDBY进行备份
与一般主库备份使用方法相同，不同之处在于以下参数的使用：
--standby-host=HOSTNAME standby host when taking backup from standby --standby-port=PORT standby port when taking backup from standby
4.7 恢复（PITR）
Restore options: --recovery-target-time time stamp up to which recovery will proceed --recovery-target-xid transaction ID up to which recovery will proceed --recovery-target-inclusive whether we stop just after the recovery target --recovery-target-timeline recovering into a particular timeline --hard-copy copying archivelog not symbolic link
各项详细解释参见http://sourceforge.net/p/pg-rman/wiki/User%27s_Manual/中“Restore options”部分。
注意--hard-copy参数：
The archive WAL are copied to archive WAL storage area. If not specified, pg_rman makes symlink to archive WAL where are in the backup catalog directory.
{不指定时不会真实拷贝归档，只做一个符号链接。但经测试该参数无效，结果均为实际拷贝}
例：
postgres=# \d List of relations Schema | Name | Type | Owner --------+------+-------+---------- public | t1 | table | postgres public | t2 | table | postgres public | t3 | table | postgres public | t4 | table | postgres public | t5 | table | postgres public | t6 | table | postgres (6 rows) [postgres@pg ~]$ pg_rman backup --backup-mode=full INFO: database backup start NOTICE: pg_stop_backup complete, all required WAL segments have been archived [postgres@pg ~]$ pg_rman show ============================================================================ Start Time Total Data WAL Log Backup Status ============================================================================ 2014-04-11 08:58:54 0m 20MB ---- 33MB ---- 2392kB DONE 2014-04-11 08:43:18 0m ---- 16kB 67MB 0B 2096kB OK 2014-04-11 08:43:09 0m 20MB ---- 33MB 0B 2402kB OK postgres=# drop table t1; DROP TABLE postgres=# drop table t2; DROP TABLE postgres=# drop table t3; DROP TABLE postgres=# select now(); now ------------------------------- 2014-04-11 09:01:11.396377+08 (1 row) postgres=# create table t7(id int); CREATE TABLE postgres=# select now(); now ------------------------------- 2014-04-11 09:01:41.613243+08 (1 row) [postgres@pg ~]$ pg_rman backup --backup-mode=incremental INFO: database backup start NOTICE: pg_stop_backup complete, all required WAL segments have been archived [postgres@pg ~]$ pg_rman show ============================================================================ Start Time Total Data WAL Log Backup Status ============================================================================ 2014-04-11 09:03:08 0m ---- 1671kB 67MB ---- 275kB DONE 2014-04-11 08:58:54 0m 20MB ---- 33MB ---- 2392kB DONE 2014-04-11 08:43:18 0m ---- 16kB 67MB 0B 2096kB OK 2014-04-11 08:43:09 0m 20MB ---- 33MB 0B 2402kB OK
模拟崩溃：
[postgres@pg ~]$ killall -9 postgres [postgres@pg ~]$ rm -rf /opt/pgsql/data/*
恢复：
[postgres@pg ~]$ pg_rman restore --recovery-target-time "2014-04-11 09:01:11" WARNING: can't open pg_controldata file "/opt/pgsql/data/global/pg_control": No such file or directory INFO: validate: 2014-04-11 08:58:54 backup and archive log files by SIZE INFO: restore complete. Recovery starts automatically when the PostgreSQL server is started. [postgres@pg ~]$ cat /opt/pgsql/data/recovery.conf # recovery.conf generated by pg_rman 1.2.8 restore_command = 'cp /arclog/%f %p' recovery_target_time = '2014-04-11 09:01:11' recovery_target_timeline = '1'
启动数据库并验证：
postgres=# \d List of relations Schema | Name | Type | Owner --------+------+-------+---------- public | t4 | table | postgres public | t5 | table | postgres public | t6 | table | postgres (3 rows)
五、总结
官方网站给出的特性：
Ease of use. Backup and restore can be done with just one command.
Online full backup, incremental backup, and archive backup. Incremental backup can reduce backup size extreamly.
Backup compression. Unused area in pages are removed and only actual data are compressed with gzip.
Automatic backup maintenance. Backup and archive WAL files older than specified days are deleted automatically.
Backup validation. Backup can be validated with CRC checks.
No transaction lost in restore. Configuration file generator for point-in-time recovery is supported.
六、参考文献
官方首页：https://code.google.com/p/pg-rman/
代码：http://sourceforge.net/projects/pg-rman/?source=navbar
使用说明文档：http://sourceforge.net/p/pg-rman/wiki/User%27s_Manual/
七、license
New BSD License