MongoDB热备方案
文章目录
-
- 一、PBM
-
- 1.1 PBM工具介绍
- 1.2 PBM工具使用
-
- 1.2.1 安装工具
- 1.2.2 创建用户
- 1.2.2 配置PBM_MONGODB_URI
- 1.2.3 配置文件存储
- 1.2.4 启动pbm-agent代理进程
- 1.2.5 相关操作
- 1.2.6 时间点恢复
- 1.2.7 PBM状态
- 1.3 PBM相关集合
- 二、Percona For Mongodb
一、PBM
1.1 PBM工具介绍
pbm全称percona backup mongodb，是percona公司提供的mongodb的备份恢复工具，支持percona mongodb,community mongodb，支持版本>v3.6，pbm底层是调用mongodump/mongorestore进行转储备份，目前对分片集群的支持力度较弱，不支持增量备份。
1.2 PBM工具使用
1.2.1 安装工具
yum localinstall percona-backup-mongodb-1.4.1-1.el7.x86_64.rpm -y
[root@zijie ~]# ll /usr/bin/pbm*
-rwxr-xr-x 1 root root 20M 1月 28 03:16 /usr/bin/pbm
-rwxr-xr-x 1 root root 22M 1月 28 03:16 /usr/bin/pbm-agent
-rwxr-xr-x 1 root root 21M 1月 28 03:16 /usr/bin/pbm-speed-test
1.2.2 创建用户
db.getSiblingDB("admin").createRole({ "role": "pbmAnyAction",
"privileges": [
{ "resource": { "anyResource": true },
"actions": [ "anyAction" ]
}
],
"roles": []
});
db.getSiblingDB("admin").createUser({user: "pbmuser",
"pwd": "secretpwd",
"roles" : [
{ "db" : "admin", "role" : "readWrite", "collection": "" },
{ "db" : "admin", "role" : "backup" },
{ "db" : "admin", "role" : "clusterMonitor" },
{ "db" : "admin", "role" : "restore" },
{ "db" : "admin", "role" : "pbmAnyAction" }
]
});
1.2.2 配置PBM_MONGODB_URI
[root@zijie ~]# cat /etc/sysconfig/pbm-agent
PBM_MONGODB_URI="mongodb://pbmuser:[email protected]:27017/?authSource=admin&replicaSet=xuehui"
#配置环境变量
export PBM_MONGODB_URI="mongodb://pbmuser:[email protected]:27017/?authSource=admin&replicaSet=xuehui"
1.2.3 配置文件存储
pbm备份的存储类型支持s3存储，远程文件系统服务器，仅单实例或单节点副本集支持本地存储
[root@zijie ~]# cat /etc/pbm_config.yaml
storage:
type: filesystem
filesystem:
path: /data/mongodbbackup
[root@zijie ~]# mkdir -pv /data/mongodbbackup
[root@zijie ~]# chown -R pbm.pbm /data/mongodbbackup/
[root@zijie ~]# pbm config --mongodb-uri="mongodb://root:[email protected]:27017/test?authSource=admin&replicaSet=xuehui" --file /etc/pbm_config.yaml
1.2.4 启动pbm-agent代理进程
[root@zijie ~]# systemctl start pbm-agent
[root@zijie ~]# systemctl status pbm-agent
● pbm-agent.service - pbm-agent
Loaded: loaded (/usr/lib/systemd/system/pbm-agent.service; enabled; vendor preset: disabled)
Active: active (running) since 三 2021-03-10 10:42:16 CST; 3s ago
Main PID: 11307 (pbm-agent)
CGroup: /system.slice/pbm-agent.service
└─11307 /usr/bin/pbm-agent
3月 10 10:42:16 zijie pbm-agent[11307]: 2021-03-10T10:42:16.000+0800 I pbm-agent:
3月 10 10:42:16 zijie pbm-agent[11307]: Version: 1.4.1
3月 10 10:42:16 zijie pbm-agent[11307]: Platform: linux/amd64
3月 10 10:42:16 zijie pbm-agent[11307]: GitCommit: e9d4b1b047209a82076a8a17b4b561cf2405ecd8
3月 10 10:42:16 zijie pbm-agent[11307]: GitBranch: release-1.4.1
3月 10 10:42:16 zijie pbm-agent[11307]: BuildTime: 2021-01-27_19:16_UTC
3月 10 10:42:16 zijie pbm-agent[11307]: GoVersion: go1.14.2
3月 10 10:42:16 zijie pbm-agent[11307]: 2021-03-10T10:42:16.000+0800 I node: xuehui/zijie:27017
3月 10 10:42:16 zijie pbm-agent[11307]: 2021-03-10T10:42:16.000+0800 I starting PITR routine
3月 10 10:42:16 zijie pbm-agent[11307]: 2021-03-10T10:42:16.000+0800 I listening for the commands
1.2.5 相关操作
#备份
[root@zijie data]# pbm backup
Starting backup '2021-03-10T03:27:02Z'....................
Backup '2021-03-10T03:27:02Z' to remote store '/data/mongodbbackup' has started
[root@zijie mongodbbackup]# pbm backup --compression=gzip
Starting backup '2021-03-10T03:29:21Z'....................
Backup '2021-03-10T03:29:21Z' to remote store '/data/mongodbbackup' has started
[root@zijie mongodbbackup]# ll
总用量 60K
-rw-rw-r-- 1 pbm pbm 1.4K 3月 10 11:27 2021-03-10T03:27:02Z.pbm.json
-rw-rw-r-- 1 pbm pbm 19K 3月 10 11:27 2021-03-10T03:27:02Z_xuehui.dump.s2
-rw-rw-r-- 1 pbm pbm 881 3月 10 11:27 2021-03-10T03:27:02Z_xuehui.oplog.s2
-rw-rw-r-- 1 pbm pbm 26K 3月 10 11:29 2021-03-10T03:29:21Z_xuehui.dump.gz
-rw-rw-r-- 1 pbm pbm 823 3月 10 11:29 2021-03-10T03:29:21Z_xuehui.oplog.gz
#备份列表
[root@zijie mongodbbackup]# pbm list
Backup snapshots:
2021-03-10T03:27:02Z [complete: 2021-03-10T03:27:23]
2021-03-10T03:29:21Z [complete: 2021-03-10T03:29:42]
#数据恢复
[root@zijie mongodbbackup]# pbm restore 2021-03-10T03:27:02Z
...Restore of the snapshot from '2021-03-10T03:27:02Z' has started
#取消备份
[root@zijie mongodbbackup]# pbm cancel-backup
Backup cancellation has started
#删除备份
[root@zijie mongodbbackup]# pbm delete-backup 2021-03-10T03:27:02Z
Are you sure you want delete backup(s)? [y/N] y
Waiting for delete to be done ...[done]
Backup snapshots:
2021-03-10T03:29:21Z [complete: 2021-03-10T03:29:42]
#查看备份恢复日志
[root@zijie mongodbbackup]# pbm logs --event=backup
2021-03-10T03:27:19Z I [xuehui/zijie:27017] [backup/2021-03-10T03:27:02Z] backup started
2021-03-10T03:27:23Z I [xuehui/zijie:27017] [backup/2021-03-10T03:27:02Z] mongodump finished, waiting for the oplog
2021-03-10T03:27:28Z I [xuehui/zijie:27017] [backup/2021-03-10T03:27:02Z] backup finished
2021-03-10T03:29:39Z I [xuehui/zijie:27017] [backup/2021-03-10T03:29:21Z] backup started
2021-03-10T03:29:42Z I [xuehui/zijie:27017] [backup/2021-03-10T03:29:21Z] mongodump finished, waiting for the oplog
2021-03-10T03:29:47Z I [xuehui/zijie:27017] [backup/2021-03-10T03:29:21Z] backup finished
[root@zijie mongodbbackup]# pbm logs --event=restore
2021-03-10T03:55:01Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] restore started
2021-03-10T03:55:07Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] mongorestore finished
2021-03-10T03:55:09Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] starting oplog replay
2021-03-10T03:55:09Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] oplog replay finished on {1615346842 1}
2021-03-10T03:55:09Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] restoring users and roles
2021-03-10T03:55:10Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] restore finished successfully
2021-03-10T03:27:28Z I [xuehui/zijie:27017] [backup/2021-03-10T03:27:02Z] backup finished
2021-03-10T03:29:22Z I [xuehui/zijie:27017] got command backup [name: 2021-03-10T03:29:21Z, compression: gzip]
2021-03-10T03:29:22Z I [xuehui/zijie:27017] got epoch {1615346848 1}
2021-03-10T03:29:39Z I [xuehui/zijie:27017] [backup/2021-03-10T03:29:21Z] backup started
2021-03-10T03:29:42Z I [xuehui/zijie:27017] [backup/2021-03-10T03:29:21Z] mongodump finished, waiting for the oplog
2021-03-10T03:29:47Z I [xuehui/zijie:27017] [backup/2021-03-10T03:29:21Z] backup finished
2021-03-10T03:55:01Z I [xuehui/zijie:27017] got command restore [name: 2021-03-10T03:55:00.602972345Z, backup name: 2021-03-10T03:27:02Z]
2021-03-10T03:55:01Z I [xuehui/zijie:27017] got epoch {1615346987 2}
2021-03-10T03:55:01Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] restore started
2021-03-10T03:55:07Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] mongorestore finished
2021-03-10T03:55:09Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] starting oplog replay
2021-03-10T03:55:09Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] oplog replay finished on {1615346842 1}
2021-03-10T03:55:09Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] restoring users and roles
2021-03-10T03:55:10Z I [xuehui/zijie:27017] [restore/2021-03-10T03:27:02Z] restore finished successfully
2021-03-10T03:56:57Z I [xuehui/zijie:27017] got command cancelBackup
2021-03-10T03:56:57Z I [xuehui/zijie:27017] got epoch {1615348510 2}
2021-03-10T03:58:14Z I [xuehui/zijie:27017] got command delete
2021-03-10T03:58:14Z I [xuehui/zijie:27017] got epoch {1615348510 2}
2021-03-10T03:58:14Z I [xuehui/zijie:27017] [delete/2021-03-10T03:27:02Z] deleting backup
2021-03-10T03:58:14Z I [xuehui/zijie:27017] [delete/2021-03-10T03:27:02Z] done
1.2.6 时间点恢复
针对分片集群的恢复，需要提前关闭balancer，最直接的方式是关闭所有mongos节点
[root@zijie pbmPitr]# pbm list
Backup snapshots:
2021-03-10T03:29:21Z [complete: 2021-03-10T03:29:42]
2021-03-10T05:31:13Z [complete: 2021-03-10T05:31:33]
PITR :
2021-03-10T05:31:33 - 2021-03-10T06:09:02
pbm restore --time="2021-03-10T06:09:01"
1.2.7 PBM状态
[root@zijie pbmPitr]# pbm status
Cluster:
========
xuehui:
- xuehui/zijie:27017: pbm-agent v1.4.1 OK
PITR incremental backup:
========================
Status [ON]
Currently running:
==================
(none)
Backups:
========
FS /data/mongodbbackup
Snapshots:
2021-03-10T05:31:13Z 35.38KB [complete: 2021-03-10T05:31:33]
2021-03-10T03:29:21Z 25.01KB [complete: 2021-03-10T03:29:42]
PITR chunks:
2021-03-10T05:31:33 - 2021-03-10T05:51:47 18.28KB
1.3 PBM相关集合
admin.pbmBackups-每个备份的日志/状态
admin.pbmAgents-包含有关pbm-agents状态和运行状况的信息
admin.pbmConfig-包含Percona Backup for MongoDB的配置信息
admin.pbmCmd-用于定义和触发操作
admin.pbmLock - pbm代理同步锁定结构
admin.pbmLockOp-用于协调不互斥的操作，例如make备份和delete备份。
admin.pbmLog-pbm-agents在MongoDB环境中存储所有日志信息。从1.4.0版开始的Percona Backup for MongoDB中可用
admin.pbmOpLog-存储操作ID
admin.pbmPITRChunks-存储时间点恢复操作日志片
admin.pbmPITRState-包含时间点恢复增量备份的当前状态
admin.pbmRestores-包含所有副本集的还原历史记录和还原状态
admin.pbmStatus-为Pergoa备份存储MongoDB状态记录
二、Percona For Mongodb
Percona Server for MongoDB为默认WiredTiger存储引擎提供了一个集成的开源热备份系统。它在运行中的服务器上创建物理数据备份，而不会造成显著的性能和操作降级。
chown mongod:mongod
xuehui:SECONDARY> use admin;
switched to db admin
xuehui:SECONDARY> db.runCommand({createBackup: 1, archive: "/data/mongodbbackup/archive.tar" })
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1615466288, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1615466288, 1)
}
xuehui:SECONDARY> db.runCommand({createBackup: 1, backupDir: "/data/mongodbbackup/"})
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1615466303, 2),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1615466303, 2)
}
要恢复备份，需要停止mongod服务，清理数据目录，然后从中复制备份目录文件到数据目录。