Mongodb PITR恢复

PITR（point-in-time recovery）是在有备份的情况下,可以恢复到任何一个时间点。

之前文章写过,mongodb的备份工具mongodump,mongorestore,mongoexport,mongoimport

以上工具就不再详细介绍

这里介绍一下mongodump的一个选项--oplog

--oplog                                               use oplog for taking a point-in-time snapshot

文档是这样介绍的，但是并不是任何时间点的snapshot哦，这个选项的意思是dump开始到dump结束这个时间段操作的oplog，oplog这个东西只有在replica sets里面有，因为主从节点之间就是通过这个东西同步的，所以建议大家即使是一个节点的mongodb也要配置一下--replSet选项，因为standalone模式下，local下面是没有oplog.rs这个表的。

PITR恢复其实主要是用到mongorestore的--oplogReplay --oplogLimit两个选项，一个是回放操作，另外一个oplogLimit可以选取任何一个点，这个点如何选择呢，接下来就要注意了，做个实例，希望大家理解。

这里replica sets的搭建就不要介绍了，直接主节点操作步骤如下：

创建两个测试表，一个test，一个tb1:

db.test.save({name:"apple"});

for(var i = 1; i <=100; i++) {db.tb1.insert({name:"hank"+i});}

然后做一下全备

mongodump -h 127.0.0.1 --port 5888  -o /home/mongo/

进入数据库做一些操作

db.test.drop();

db.tb1.remove({});

db.tb1.insert({name:"xxxxxxxxxxxxxxxxxxxx"});

现在，我们要恢复到操作之前，怎么做呢，第一个就是用我们刚才全备直接restore回去，另外如果想恢复到任何一个点的话，那么就需要oplog

导出oplog

mongodump -h 127.0.0.1 --port 5888 -d local -c oplog.rs -o /home/mongo/

mv oplog.rs.bson /home/mongo/local/oplog.bson  --重命名导出的oplog

bsondump oplog.bson 查看oplog内容，这类我截取了部分内容，这里标红加粗的就是--oplogLimit 要跟的参数内容

这个参数就是db.test.drop();操作之前的点

{"ts":{"$timestamp":{"t":1488181183,"i":1}},"t":{"$numberLong":"4"},"h":{"$numberLong":"-6545284460632360892"},"v":2,"op":"c","ns":"test.$cmd","o":{"drop":"test"}}

{"ts":{"$timestamp":{"t":1488181203,"i":1}},"t":{"$numberLong":"4"},"h":{"$numberLong":"-1071465797953742786"},"v":2,"op":"d","ns":"test.tb1","o":{"_id":{"$oid":"58b3d7395089b35311fa9b1e"}}}

{"ts":{"$timestamp":{"t":1488181203,"i":2}},"t":{"$numberLong":"4"},"h":{"$numberLong":"7922879569243271327"},"v":2,"op":"d","ns":"test.tb1","o":{"_id":{"$oid":"58b3d7395089b35311fa9b1f"}}}

{"ts":{"$timestamp":{"t":1488181203,"i":3}},"t":{"$numberLong":"4"},"h":{"$numberLong":"5203619554553799738"},"v":2,"op":"d","ns":"test.tb1","o":{"_id":{"$oid":"58b3d7395089b35311fa9b20"}}}

{"ts":{"$timestamp":{"t":1488181203,"i":4}},"t":{"$numberLong":"4"},"h":{"$numberLong":"-4057525037056875752"},"v":2,"op":"d","ns":"test.tb1","o":{"_id":{"$oid":"58b3d7395089b35311fa9b21"}}}

{"ts":{"$timestamp":{"t":1488181203,"i":5}},"t":{"$numberLong":"4"},"h":{"$numberLong":"428652878437400693"},"v":2,"op":"d","ns":"test.tb1","o":{"_id":{"$oid":"58b3d7395089b35311fa9b22"}}}

比如我们要恢复到db.tb1.insert({name:"xxxxxxxxxxxxxxxxxxxx"});之前

那么执行以下命令

mongorestore -h 127.0.0.1 --port 5888 --drop --oplogReplay --oplogLimit "1488181183:1" /home/mongo/local

2017-02-27T17:07:11.670+0800    building a list of dbs and collections to restore from /home/mongo/local dir

2017-02-27T17:07:11.671+0800    replaying oplog

2017-02-27T17:07:11.681+0800    done

 db.tb1.find() 

{ "_id" : ObjectId("58b3d7e25bdf4a8fd701f07d"), "name" : "xxxxxxxxxxxxxxxxxxxx" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b1e"), "name" : "hank1" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b1f"), "name" : "hank2" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b20"), "name" : "hank3" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b21"), "name" : "hank4" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b22"), "name" : "hank5" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b23"), "name" : "hank6" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b24"), "name" : "hank7" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b25"), "name" : "hank8" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b26"), "name" : "hank9" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b27"), "name" : "hank10" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b28"), "name" : "hank11" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b29"), "name" : "hank12" }

这里可以看到恢复到了之前的状态，但是这里"name" : "xxxxxxxxxxxxxxxxxxxx"的记录还是在的

也就是说，这里只是回放了oplog操作，因为这条记录本身就有的，我们只是回放了之前插入的操作，所以这条记录还保留在表中

再比如，以下操作是

mongorestore -h 127.0.0.1 --port 5888 --oplogReplay --oplogLimit "1488181203:20" /home/mongo/local/

--oplogLimit=[:ordinal]

这里是会恢复到1488181203这个操作的第20次

 db.tb1.find()

{ "_id" : ObjectId("58b3d7e25bdf4a8fd701f07d"), "name" : "xxxxxxxxxxxxxxxxxxxx" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b32"), "name" : "hank21" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b33"), "name" : "hank22" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b34"), "name" : "hank23" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b35"), "name" : "hank24" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b36"), "name" : "hank25" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b37"), "name" : "hank26" }

{ "_id" : ObjectId("58b3d7395089b35311fa9b38"), "name" : "hank27" }

可以看到删除了前面20行记录

{"ts":{"$timestamp":{"t":1488187878,"i":1}},"t":{"$numberLong":"4"},"h":{"$numberLong":"2891167049592106048"},"v":2,"op":"c","ns":"test.$cmd","o"

     :{"drop":"oplog"}}

1664 {"ts":{"$timestamp":{"t":1488187909,"i":1}},"t":{"$numberLong":"4"},"h":{"$numberLong":"4897030452135397992"},"v":2,"op":"c","ns":"test.$cmd","o"

     :{"create":"hank"}}

1665 {"ts":{"$timestamp":{"t":1488187909,"i":2}},"t":{"$numberLong":"4"},"h":{"$numberLong":"-366641111972103000"},"v":2,"op":"i","ns":"test.hank","o"

     :{"_id":{"$oid":"58b3f205ff3732609f8a099d"},"name":"apple"}}

1666 {"ts":{"$timestamp":{"t":1488187943,"i":1}},"t":{"$numberLong":"4"},"h":{"$numberLong":"448267919733245353"},"v":2,"op":"c","ns":"test.$cmd","o":

     {"drop":"tb1"}}

drop恢复演示：

db.tb1.drop();

db.hank.drop();

删除后通过以下命令恢复

mongorestore -h 127.0.0.1 --port 5888 --oplogReplay --oplogLimit "1488187943:1" /home/mongo/local/           

2017-02-27T17:41:02.631+0800    building a list of dbs and collections to restore from /home/mongo/local dir

2017-02-27T17:41:02.632+0800    replaying oplog

2017-02-27T17:41:03.107+0800    done

show tables; --可以看到2个表都已经恢复

hank

tb1

db.hank.find();

{ "_id" : ObjectId("58b3f205ff3732609f8a099d"), "name" : "apple" }

db.tb1.count();

82

记录也正常

另外如果有想通过备份+oplog做恢复的同学，一定注意，两次备份的时间间隔不要超过oplog的覆盖范围，

所以要权衡oplog的大小，要有足够的时间dump出oplog，发生故障，停止数据库的操作，以免超出oplog覆盖范围。

另外注意点，恢复点之后，新增的表，恢复之后，这个表不会被删除，还存在并且数据正常

比如：

这里我们的hank表在之前是没有的，那么我恢复到很早的一个点

show tables;

hank

tb1

db.hank.find();

{ "_id" : ObjectId("58b3f205ff3732609f8a099d"), "name" : "apple" }

 mongorestore -h 127.0.0.1 --port 5888 --drop --oplogReplay --oplogLimit "1488181183:1" /home/mongo/local

这里恢复的这个时间点其实并没有hank表，但是恢复后还是存在，但是tb1恢复到了那个时间点。

show tables;

hank

tb1

db.tb1.count();

101