PostgreSQL 数据备份和恢复

PostgreSQL 数据备份和恢复

pg_dump 备份

• 备份数据库 pgupgradedb 为 databak, p 为端口

postgres@clw-db2:/pgdata/10.3> pg_dump pgupgradedb -h 10.10.56.17 -p 5438 -U postgres > databak
postgres@clw-db2:/pgdata/10.3> ll -lh
total 480M
drwx------ 19 postgres postgres 4.0K Jun  7 16:44 data
-rw-r--r--  1 postgres postgres 480M Jun  7 18:07 databak
postgres@clw-db2:/pgdata/10.3>

• 查看备份的数据库文件如下：

postgres@clw-db2:/pgdata/10.3> cat databak
--
-- PostgreSQL database dump
--

-- Dumped from database version 10.3
-- Dumped by pg_dump version 10.3

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET client_min_messages = warning;
SET row_security = off;

--
-- Name: plpgsql; Type: EXTENSION; Schema: -; Owner:
--

CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;


--
-- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner:
--

COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';


SET default_tablespace = '';

SET default_with_oids = false;

--
-- Name: pgupgrade; Type: TABLE; Schema: public; Owner: postgres
--

CREATE TABLE public.pgupgrade (
    id integer NOT NULL,
    age bigint,
    inserttime timestamp without time zone DEFAULT now()
);


ALTER TABLE public.pgupgrade OWNER TO postgres;

--
-- Name: pgupgrade_id_seq; Type: SEQUENCE; Schema: public; Owner: postgres
--

CREATE SEQUENCE public.pgupgrade_id_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1;


ALTER TABLE public.pgupgrade_id_seq OWNER TO postgres;

--
-- Name: pgupgrade_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: postgres
--

ALTER SEQUENCE public.pgupgrade_id_seq OWNED BY public.pgupgrade.id;


--
-- Name: pgupgrade id; Type: DEFAULT; Schema: public; Owner: postgres
--

ALTER TABLE ONLY public.pgupgrade ALTER COLUMN id SET DEFAULT nextval('public.pgupgrade_id_seq'::regclass);


--
-- Data for Name: pgupgrade; Type: TABLE DATA; Schema: public; Owner: postgres
--

COPY public.pgupgrade (id, age, inserttime) FROM stdin;
5       91647434        2018-06-07 17:47:38.821321
13      46223030        2018-06-07 17:47:38.82556
21      114249409       2018-06-07 17:47:38.829053
28      315543381       2018-06-07 17:47:38.831503
35      383642413       2018-06-07 17:47:38.833049
45      109718816       2018-06-07 17:47:38.8367
64      77561788        2018-06-07 17:47:38.843235
79      164887727       2018-06-07 17:47:38.84546
93      203783237       2018-06-07 17:47:38.846801
109     38643627        2018-06-07 17:47:38.854703
127     323372478       2018-06-07 17:47:38.856251
150     213251316       2018-06-07 17:47:38.858198
166     119008838       2018-06-07 17:47:38.859787
...

• 注 意

pg_dump 默认会使用当前登录操作系统的用户为登录数据库的用户，所以备份时请指定 -U 指定用户

备份的数据库可以在任意架构服务器上运行（32位，64位）

备份的数据库也可以在比当前版本新的数据库上运行

pg_dump 备份类似一个快照，不会对数据库进行锁

允许任意连接数据库的用户进行备份，更多参数请查看api

允许任意连接数据库的用户进行备份，更多参数请查看api

确保在备份时，d对象或者用户所属关系全部创建好了，否则恢复时出现权限错误

pg-dump 恢复备份数据

• 创建需要恢复的数据库目录

postgres=# create database pgupgradedb;
CREATE DATABASE
postgres=# \l
                                   List of databases
    Name     |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-------------+----------+----------+-------------+-------------+-----------------------
 pgupgradedb | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 pocdb       | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres    | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
 template1   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
(5 rows)
postgres=#

• 恢复,通过 psql 命令进行恢复

psql dbname < databak

• 注 意： 执行改命令，必须先创建好需要恢复的数据库名称，上述命令不会自动创建不存在的 dbname

• 开始恢复

postgres@clw-db2:/pgdata/10/poc/data> psql pgupgradedb -h 10.10.56.17 -p 5432 -U postgres  < /pgdata/10.3/databak
Password for user postgres:
SET
SET
SET
SET
SET
 set_config
------------
(1 row)
SET
SET
SET
CREATE EXTENSION
COMMENT
SET
SET
CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
ALTER SEQUENCE
ALTER TABLE
COPY 11252994
  setval
----------
 11252998
(1 row)

看到上述表示数据恢复完成

• 验证是否恢复正确

postgres@clw-db2:/pgdata/10/poc/data> psql -p 5432
psql (10.3)
Type "help" for help.
pgupgradedb=# \l
                                   List of databases
    Name     |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-------------+----------+----------+-------------+-------------+-----------------------
 pgupgradedb | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 pocdb       | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres    | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
 template1   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
(5 rows)

pgupgradedb=# \d+
                             List of relations
 Schema |       Name       |   Type   |  Owner   |    Size    | Description
--------+------------------+----------+----------+------------+-------------
 public | pgupgrade        | table    | postgres | 560 MB     |
 public | pgupgrade_id_seq | sequence | postgres | 8192 bytes |
(2 rows)

pgupgradedb=# select max(id) from pgupgrade;
   max
----------
 11252998
(1 row)

发现结果与备份前一直，备份恢复成功！

pg_basebackup 基础备份

• 查询原数据库库数据

pgupgradedb=# \d+
                             List of relations
 Schema |       Name       |   Type   |  Owner   |    Size    | Description
--------+------------------+----------+----------+------------+-------------
 public | pgupgrade        | table    | postgres | 141 MB     |
 public | pgupgrade_id_seq | sequence | postgres | 8192 bytes |
(2 rows)

pgupgradedb=# \d+ pgupgrade
                                                            Table "public.pgupgrade"
   Column   |            Type             | Collation | Nullable |                Default                | Storage | Stats target | Description
------------+-----------------------------+-----------+----------+---------------------------------------+---------+--------------+-------------
 id         | integer                     |           | not null | nextval('pgupgrade_id_seq'::regclass) | plain   |              |
 age        | bigint                      |           |          |                                       | plain   |              |
 inserttime | timestamp without time zone |           |          | now()                                 | plain   |              |

pgupgradedb=# select count(1) from pgupgrade;
  count
---------
 2834653
(1 row)
pgupgradedb=#

• 备份此时的数据库状态

• 登陆数据库，创建复制备份的角色，

postgres=# \du
                                   List of roles
 Role name |                         Attributes                         | Member of
-----------+------------------------------------------------------------+-----------
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}

postgres=# CREATE USER repl WITH PASSWORD '123456' REPLICATION;
CREATE ROLE
postgres=# \c pgupgradedb
You are now connected to database "pgupgradedb" as user "postgres".
pgupgradedb=# \du
                                   List of roles
 Role name |                         Attributes                         | Member of
-----------+------------------------------------------------------------+-----------
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
 repl      | Replication 

• 修改 pg_hba.conf 文件，配置需要使用的角色、IP地址等信息,此处为Ip 10.10.56.0 repl用户

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
host    all             all             0.0.0.0/0               md5
# IPv6 local connections:
host    all             all             ::1/128                 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local   replication     postgres                                trust
host    replication     postgres        127.0.0.1/32            trust
host    replication     repl            10.10.56.0/0            trust
#host    replication     postgres        ::1/128                 trust
~

• 修改配置文件后，需要重启服务

postgres@clw-db2:/pgdata/10.3/data> pg_ctl -D /pgdata/10.3/data restart
waiting for server to shut down.... done
server stopped
waiting for server to start....2018-06-08 16:05:07.625 CST [24614] LOG:  listening on IPv4 address "0.0.0.0", port 5438
2018-06-08 16:05:07.625 CST [24614] LOG:  listening on IPv6 address "::", port 5438
2018-06-08 16:05:07.636 CST [24614] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5438"
2018-06-08 16:05:07.672 CST [24614] LOG:  ending log output to stderr
2018-06-08 16:05:07.672 CST [24614] HINT:  Future log output will go to log destination "csvlog".
2018-06-08 16:05:07.681 CST [24643] LOG:  database system was shut down at 2018-06-08 16:05:07 CST
2018-06-08 16:05:07.696 CST [24614] LOG:  database system is ready to accept connections
 done
server started
postgres@clw-db2:/pgdata/10.3/data>

• 使用 pg_basebackup 进行基础备份

postgres@clw-db2:/pgdata/10.3/data> /opt/pgsql-10/bin/pg_basebackup -h 10.10.56.17 -p 5438 -U repl -Fp -Xs -Pv -D /pgdata/10.3/backup/pgupgradedb_backup_20180608_data/
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/7B000060 on timeline 1
pg_basebackup: starting background WAL receiver
179855/179855 kB (100%), 1/1 tablespace
pg_basebackup: write-ahead log end point: 0/7B000130
pg_basebackup: waiting for background process to finish streaming ...
pg_basebackup: base backup completed
postgres@clw-db2:/pgdata/10.3/data>

• 参数说明

- h 数据库地址 -p 数据库端口 -U 备份使用的用户 -Fp 指定备份输出的文件格式，此处为原文件
-Xs 采用流复制的 wal 日志方法 -Pv 输出备份过程的信息 -D 备份数据库的存放目录

上述备份时，备份的存放目录，必须为空，该文件路径需要创建好，不会自动创建

• 查看备份后的数据目录

postgres@clw-db2:/pgdata/10.3/backup> ls
pgupgradedb_backup_20180608_data
postgres@clw-db2:/pgdata/10.3/backup> cd pgupgradedb_backup_20180608_data/
postgres@clw-db2:/pgdata/10.3/backup/pgupgradedb_backup_20180608_data> ls
backup_label  pg_commit_ts  pg_ident.conf  pg_notify    pg_snapshots  pg_subtrans  PG_VERSION  postgresql.auto.conf
base          pg_dynshmem   pg_logical     pg_replslot  pg_stat       pg_tblspc    pg_wal      postgresql.conf
global        pg_hba.conf   pg_multixact   pg_serial    pg_stat_tmp   pg_twophase  pg_xact
postgres@clw-db2:/pgdata/10.3/backup/pgupgradedb_backup_20180608_data>

此时基础备份就完成了