pg备份还原工具--pg_rman

2019独角兽企业重金招聘Python工程师标准>>>

一、系统

IP                      HOSTNAME   PG VERSION        DIR BACKUPSETS PATH         OS

192.168.100.147 pg                  9.3.4                     /opt/pgsql /backup                    CentOS6.4_x64

[root@barman ~]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@barman ~]# uname -a
Linux barman 2.6.32-431.11.2.el6.x86_64 #1 SMP Tue Mar 25 19:59:55 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

二、安装

2.1 简介

pg_rman is an online backup and restore tool for PostgreSQL.

The goal of the pg_rman project is providing a method for online backup and PITR as easy as pg_dump. Also, it maintains a backup catalog per database cluster. Users can maintain old backups including archive logs with one command.

pg_rman is a utility program to backup and restore PostgreSQL database. It takes a physical online backup of whole database cluster, archive WALs, and server logs.

pg_rman supports getting backup from standby-site with PostgreSQL 9.0 later.

And pg_rman supports storage snapshot backup.

pg_rman的开发语言为C。

pg_rman的版本更新情况：

平台需求：

Linux (and some of UNIXes)

PostgreSQL 9.3, 9.2, 9.1, 9.0, 8.4 - 8.2

目前已通过测试的平台：

不支持的平台：

Windows

PostgreSQL 8.1 or prior versions.

2.2 安装依赖包

下载安装postgresql93-libs

http://yum.postgresql.org/9.3/redhat/rhel-6.4-x86_64/

[root@pg ~]# rpm -ivh postgresql93-libs-9.3.4-1PGDG.rhel6.x86_64.rpm 
warning: postgresql93-libs-9.3.4-1PGDG.rhel6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 442df0f8: NOKEY
Preparing...                ########################################### [100%]
   1:postgresql93-libs      ########################################### [100%]

2.3 安装pg_rman

下载地址：http://sourceforge.net/projects/pg-rman/files/

[root@pg ~]# rpm -ivh pg_rman-1.2.8-1.pg93.rhel6.x86_64.rpm
Preparing...                ########################################### [100%]
   1:pg_rman                ########################################### [100%]

将被安装在/usr/pgsql-9.3/bin/pg_rman

源码编译安装

安装类似pg扩展模块的安装

$ make USE_PGXS=1
$ make USE_PGXS=1 install

三、安装配置数据库

3.1 安装

安装过程略

3.2 创建用户

在pg上创建postgres用户并设置密码：

[root@pg ~]# useradd postgres
[root@pg ~]# passwd postgres

配置postgres用户环境变量：

export PATH=/usr/pgsql-9.3/bin:/opt/pgsql/bin:$PATH:$HOME/bin
export PGDATA=/opt/pgsql/data
export PGUSER=postgres
export PGPORT=5432
export LD_LIBRARY_PATH=/opt/pgsql/lib:$LD_LIBRARY_PATH
export BACKUP_PATH=/backup

3.3 配置

创建归档目录：

[root@pg ~]# mkdir /arclog
[root@pg ~]# chown postgres:postgres /arclog

[postgres@pg data]$ vim postgresql.conf
listen_addresses = '*'
port = 5432
wal_level = archive        若为HOT STANDBY环境则此处设置为hot_standby
archive_mode = on
archive_command = 'test ! -f /arclog/%f && cp %p /arclog/%f'

四、pg_rman管理

[postgres@pg ~]$ pg_rman --help
pg_rman manage backup/recovery of PostgreSQL database.
Usage:
  pg_rman OPTION init
  pg_rman OPTION backup
  pg_rman OPTION restore
  pg_rman OPTION show [DATE]
  pg_rman OPTION show timeline [DATE]
  pg_rman OPTION validate [DATE]
  pg_rman OPTION delete DATE
Common Options:
  -D, --pgdata=PATH         location of the database storage area
  -A, --arclog-path=PATH    location of archive WAL storage area
  -S, --srvlog-path=PATH    location of server log storage area
  -B, --backup-path=PATH    location of the backup storage area
  -c, --check               show what would have been done
Backup options:
  -b, --backup-mode=MODE    full, incremental, or archive
  -s, --with-serverlog      also backup server log files
  -Z, --compress-data       compress data backup with zlib
  -C, --smooth-checkpoint   do smooth checkpoint before backup
  --keep-data-generations=N keep GENERATION of full data backup
  --keep-data-days=DAY      keep enough data backup to recover to DAY days age
  --keep-arclog-files=NUM   keep NUM of archived WAL
  --keep-arclog-days=DAY    keep archived WAL modified in DAY days
  --keep-srvlog-files=NUM   keep NUM of serverlogs
  --keep-srvlog-days=DAY    keep serverlog modified in DAY days
  --standby-host=HOSTNAME   standby host when taking backup from standby
  --standby-port=PORT       standby port when taking backup from standby
Restore options:
  --recovery-target-time    time stamp up to which recovery will proceed
  --recovery-target-xid     transaction ID up to which recovery will proceed
  --recovery-target-inclusive whether we stop just after the recovery target
  --recovery-target-timeline  recovering into a particular timeline
  --hard-copy                 copying archivelog not symbolic link
Catalog options:
  -a, --show-all            show deleted backup too
Connection options:
  -d, --dbname=DBNAME       database to connect
  -h, --host=HOSTNAME       database server host or socket directory
  -p, --port=PORT           database server port
  -U, --username=USERNAME   user name to connect as
  -w, --no-password         never prompt for password
  -W, --password            force password prompt
Generic options:
  -q, --quiet               don't write any messages
  --debug                   debug mode
  --help                    show this help, then exit
  --version                 output version information, then exit
Read the website for details. 
Report bugs to .

4.1 初始化备份目录

[root@pg ~]# mkdir /backup
[root@pg ~]# chown postgres:postgres /backup/
[postgres@pg ~]$ pg_rman init -B /backup
INFO: ARCLOG_PATH is set to '/arclog'
INFO: SRVLOG_PATH is set to '/opt/pgsql/data/pg_log'
[postgres@pg ~]$ ll /backup/
total 12
drwx------ 4 postgres postgres 4096 Apr 11 07:17 backup
-rw-rw-r-- 1 postgres postgres   60 Apr 11 07:17 pg_rman.ini
drwx------ 2 postgres postgres 4096 Apr 11 07:17 timeline_history
[postgres@pg ~]$ cat /backup/pg_rman.ini 
ARCLOG_PATH='/arclog'
SRVLOG_PATH='/opt/pgsql/data/pg_log'

4.2 备份

Backup options:
  -b, --backup-mode=MODE    full, incremental, or archive
  -s, --with-serverlog      also backup server log files
  -Z, --compress-data       compress data backup with zlib
  -C, --smooth-checkpoint   do smooth checkpoint before backup
  --keep-data-generations=N keep GENERATION of full data backup
  --keep-data-days=DAY      keep enough data backup to recover to DAY days age
  --keep-arclog-files=NUM   keep NUM of archived WAL
  --keep-arclog-days=DAY    keep archived WAL modified in DAY days
  --keep-srvlog-files=NUM   keep NUM of serverlogs
  --keep-srvlog-days=DAY    keep serverlog modified in DAY days
  --standby-host=HOSTNAME   standby host when taking backup from standby
  --standby-port=PORT       standby port when taking backup from standby

各项详细解释参见http://sourceforge.net/p/pg-rman/wiki/User%27s_Manual/中“Backup options”部分。

4.2.1 全库备份

[postgres@pg ~]$ cat /backup/pg_rman.ini 
ARCLOG_PATH='/arclog'
SRVLOG_PATH='/opt/pgsql/data/pg_log'

COMPRESS_DATA = YES
KEEP_ARCLOG_FILES = 10
KEEP_ARCLOG_DAYS = 10
KEEP_DATA_GENERATIONS = 3
KEEP_DATA_DAYS = 120
KEEP_SRVLOG_FILES = 10
KEEP_SRVLOG_DAYS = 10

执行全库备份：

[postgres@pg ~]$ pg_rman backup --backup-mode=full
INFO: database backup start
NOTICE:  pg_stop_backup complete, all required WAL segments have been archived

查看备份集：

[postgres@pg ~]$ pg_rman show
============================================================================
Start                Time   Total    Data     WAL     Log  Backup   Status  
============================================================================
2014-04-11 08:14:31    0m    20MB    ----    83MB    ----  4426kB   DONE

验证：

[postgres@pg ~]$ pg_rman validate
INFO: validate: 2014-04-11 08:14:31 backup and archive log files by CRC

再次查看备份集：

[postgres@pg ~]$ pg_rman show
============================================================================
Start                Time   Total    Data     WAL     Log  Backup   Status  
============================================================================
2014-04-11 08:14:31    0m    20MB    ----    83MB    ----  4426kB   OK

各列含义解释：

Start : start time of backup

Time : total time of backup

Total : size of whole database cluster

Data : size of read data files

WAL : size of read WAL archive files

Log : size of read serverlog files

Backup: size of backup (= written size)

Status: status of backup. Possible values are:

　　OK : backup is done and validated.

　　DONE : backup is done, but not validated yet.

　　RUNNING : backup is running now.

　　DELETING : backup is being deleted now.

　　DELETED : backup has been deleted.

　　ERROR : backup is unavailable because some errors occur during backup.

　　CORRUPT : backup is unavailable because it is broken.

4.2.2 增量备份

执行备份：

[postgres@pg ~]$ pg_rman backup --backup-mode=incremental --with-serverlog
INFO: database backup start
NOTICE:  pg_stop_backup complete, all required WAL segments have been archived

验证：

[postgres@pg ~]$ pg_rman validate
INFO: validate: 2014-04-11 08:17:06 backup and archive log files by CRC

查看备份集信息：

[postgres@pg ~]$ pg_rman show
============================================================================
Start                Time   Total    Data     WAL     Log  Backup   Status  
============================================================================
2014-04-11 08:17:06    0m    ----    16kB    33MB      0B    79kB   OK
2014-04-11 08:14:31    0m    20MB    ----    83MB    ----  4426kB   OK
[postgres@pg ~]$ pg_rman show timeline
============================================================
Start                Mode  Current TLI  Parent TLI  Status  
============================================================
2014-04-11 08:17:06  INCR            1           0  OK
2014-04-11 08:14:31  FULL            1           0  OK

4.2.3 归档备份

[postgres@pg ~]$ pg_rman backup --backup-mode=archive --with-serverlog
[postgres@pg ~]$ pg_rman validate
INFO: validate: 2014-04-11 08:21:59 archive log files by CRC
[postgres@pg ~]$ pg_rman show timeline
============================================================
Start                Mode  Current TLI  Parent TLI  Status  
============================================================
2014-04-11 08:21:59  ARCH            1           0  OK
2014-04-11 08:17:06  INCR            1           0  OK
2014-04-11 08:14:31  FULL            1           0  OK
[postgres@pg ~]$ pg_rman show 
============================================================================
Start                Time   Total    Data     WAL     Log  Backup   Status  
============================================================================
2014-04-11 08:21:59    0m    ----    ----    16MB      0B  2000kB   OK
2014-04-11 08:17:06    0m    ----    16kB    33MB      0B    79kB   OK
2014-04-11 08:14:31    0m    20MB    ----    83MB    ----  4426kB   OK

4.3 查看备份集信息

上面已经设计，除此之外，也可直接指定某个备份集来查看详细信息：

[postgres@pg ~]$ pg_rman show  2014-04-11 08:14:31
# configuration
BACKUP_MODE=FULL
WITH_SERVERLOG=false
COMPRESS_DATA=true
# result
TIMELINEID=1
START_LSN=0/25000028
STOP_LSN=0/250000b8
START_TIME='2014-04-11 08:14:31'
END_TIME='2014-04-11 08:14:35'
RECOVERY_XID=1836
RECOVERY_TIME='2014-04-11 08:14:34'
TOTAL_DATA_BYTES=20495299
READ_DATA_BYTES=20495299
READ_ARCLOG_BYTES=83887037
WRITE_BYTES=4426584
BLOCK_SIZE=8192
XLOG_BLOCK_SIZE=8192
STATUS=OK

注意RECOVERY_XID与RECOVERY_TIME，在恢复时分别对应参数--recovery-target-xid与--recovery-target-time。

4.4 查看全部备份集（包括已删除）

[postgres@pg ~]$ pg_rman show -a
============================================================================
Start                Time   Total    Data     WAL     Log  Backup   Status  
============================================================================
2014-04-11 08:43:18    0m    ----    16kB    67MB      0B  2096kB   OK
2014-04-11 08:43:09    0m    20MB    ----    33MB      0B  2402kB   OK
2014-04-11 08:41:29    0m    ----    16kB    67MB      0B   102kB   DELETED
2014-04-11 08:40:54    0m    20MB    ----    33MB      0B  2377kB   DELETED
2014-04-11 08:21:59    0m    ----    ----    16MB      0B  2000kB   DELETED
2014-04-11 08:17:06    0m    ----    16kB    33MB      0B    79kB   DELETED
2014-04-11 08:14:31    0m    20MB    ----    83MB    ----  4426kB   DELETED

4.5 删除备份集

（删除指定时间之前恢复时不再需要的备份，将保留指定时间点所在时间段及之后的全库备份及其它备份）

[postgres@pg ~]$ pg_rman show timeline
============================================================
Start                Mode  Current TLI  Parent TLI  Status  
============================================================
2014-04-11 08:43:18  INCR            1           0  OK
2014-04-11 08:43:09  FULL            1           0  OK
2014-04-11 08:41:29  INCR            1           0  OK
2014-04-11 08:40:54  FULL            1           0  OK
[postgres@pg ~]$ pg_rman delete 2014-04-11 08:43:18
INFO: delete: 2014-04-11 08:41:29
INFO: delete: 2014-04-11 08:40:54
[postgres@pg ~]$ pg_rman show timeline
============================================================
Start                Mode  Current TLI  Parent TLI  Status  
============================================================
2014-04-11 08:43:18  INCR            1           0  OK
2014-04-11 08:43:09  FULL            1           0  OK

4.6 对STANDBY进行备份

与一般主库备份使用方法相同，不同之处在于以下参数的使用：

  --standby-host=HOSTNAME   standby host when taking backup from standby
  --standby-port=PORT       standby port when taking backup from standby

4.7 恢复（PITR）

Restore options:
  --recovery-target-time    time stamp up to which recovery will proceed
  --recovery-target-xid     transaction ID up to which recovery will proceed
  --recovery-target-inclusive whether we stop just after the recovery target
  --recovery-target-timeline  recovering into a particular timeline
  --hard-copy                 copying archivelog not symbolic link

各项详细解释参见http://sourceforge.net/p/pg-rman/wiki/User%27s_Manual/中“Restore options”部分。

注意--hard-copy参数：

The archive WAL are copied to archive WAL storage area. If not specified, pg_rman makes symlink to archive WAL where are in the backup catalog directory.

{不指定时不会真实拷贝归档，只做一个符号链接。但经测试该参数无效，结果均为实际拷贝}

例：

postgres=# \d
        List of relations
 Schema | Name | Type  |  Owner   
--------+------+-------+----------
 public | t1   | table | postgres
 public | t2   | table | postgres
 public | t3   | table | postgres
 public | t4   | table | postgres
 public | t5   | table | postgres
 public | t6   | table | postgres
(6 rows)

[postgres@pg ~]$ pg_rman backup --backup-mode=full
INFO: database backup start
NOTICE:  pg_stop_backup complete, all required WAL segments have been archived
[postgres@pg ~]$ pg_rman show
============================================================================
Start                Time   Total    Data     WAL     Log  Backup   Status  
============================================================================
2014-04-11 08:58:54    0m    20MB    ----    33MB    ----  2392kB   DONE
2014-04-11 08:43:18    0m    ----    16kB    67MB      0B  2096kB   OK
2014-04-11 08:43:09    0m    20MB    ----    33MB      0B  2402kB   OK


postgres=# drop table t1;
DROP TABLE
postgres=# drop table t2;
DROP TABLE
postgres=# drop table t3;
DROP TABLE
postgres=# select now();
              now              
-------------------------------
 2014-04-11 09:01:11.396377+08
(1 row)
postgres=# create table t7(id int);
CREATE TABLE
postgres=# select now();
              now              
-------------------------------
 2014-04-11 09:01:41.613243+08
(1 row)

[postgres@pg ~]$ pg_rman backup --backup-mode=incremental
INFO: database backup start
NOTICE:  pg_stop_backup complete, all required WAL segments have been archived
[postgres@pg ~]$ pg_rman show
============================================================================
Start                Time   Total    Data     WAL     Log  Backup   Status  
============================================================================
2014-04-11 09:03:08    0m    ----  1671kB    67MB    ----   275kB   DONE
2014-04-11 08:58:54    0m    20MB    ----    33MB    ----  2392kB   DONE
2014-04-11 08:43:18    0m    ----    16kB    67MB      0B  2096kB   OK
2014-04-11 08:43:09    0m    20MB    ----    33MB      0B  2402kB   OK

模拟崩溃：

[postgres@pg ~]$ killall -9 postgres
[postgres@pg ~]$ rm -rf /opt/pgsql/data/*

恢复：

[postgres@pg ~]$ pg_rman restore --recovery-target-time "2014-04-11 09:01:11"
WARNING: can't open pg_controldata file "/opt/pgsql/data/global/pg_control": No such file or directory
INFO: validate: 2014-04-11 08:58:54 backup and archive log files by SIZE
INFO: restore complete. Recovery starts automatically when the PostgreSQL server is started.

[postgres@pg ~]$ cat /opt/pgsql/data/recovery.conf 
# recovery.conf generated by pg_rman 1.2.8
restore_command = 'cp /arclog/%f %p'
recovery_target_time = '2014-04-11 09:01:11'
recovery_target_timeline = '1'

启动数据库并验证：

postgres=# \d
        List of relations
 Schema | Name | Type  |  Owner   
--------+------+-------+----------
 public | t4   | table | postgres
 public | t5   | table | postgres
 public | t6   | table | postgres
(3 rows)

五、总结

官方网站给出的特性：

Ease of use. Backup and restore can be done with just one command.

Online full backup, incremental backup, and archive backup. Incremental backup can reduce backup size extreamly.

Backup compression. Unused area in pages are removed and only actual data are compressed with gzip.

Automatic backup maintenance. Backup and archive WAL files older than specified days are deleted automatically.

Backup validation. Backup can be validated with CRC checks.

No transaction lost in restore. Configuration file generator for point-in-time recovery is supported.

六、参考文献

官方首页：https://code.google.com/p/pg-rman/

代码：http://sourceforge.net/projects/pg-rman/?source=navbar

使用说明文档：http://sourceforge.net/p/pg-rman/wiki/User%27s_Manual/

七、license

New BSD License

转载于:https://my.oschina.net/lianshunke/blog/223707