mongodb备份恢复
备份mmongodb的方式多种多样,下面介绍几种备份方式
一.复制数据文件
复制数据库目录的所有文件,在复制的时候防止数据文件的改变,可以使用
fsyncLock()命令来锁定数据库,禁止任何写入.
> db.fsyncLock()
{
"info" : "now locked against writes, use db.fsyncUnlock() to unlock",
"seeAlso" : "http://dochub.mongodb.org/core/fsynccommand",
"ok" : 1
}
此时任何修改都会加入到队列等待.复制完成之后使用fsyncUnlock()命令来解除锁定.
> db.fsyncUnlock()
{ "info" : "unlock completed", "ok" : 1 }
二.使用mongodump/mongorestore备份恢复
mongodump/mongorestore是用来备份和恢复bson数据.mongodump不会备份local数据库.如果使用了访问控制,那么备份的用户必须拥有backup角色.
mongodump通过连接mongod或者mongos来备份数据,mongodump可以用来备份整个数据,数据库,集合或者还可以通过查询备份集合的一部分.
如果你运行mongodump而不添加任何参数,那么此命令将会连接本地的27017端口,然后创建一个数据库的备份在当前的dump目录下面.
从2.2版本以后的mongodump和以前版本的mongod不兼容,不要使用新版本的mongodump来备份老版本的mognodb.
mongodump的帮助如下:
[root@mongodb3 ~]# mongodump --help
Usage:
mongodump <options>
Export the content of a running server into .bson files.
Specify a database with -d and a collection with -c to only dump that database or collection.
See http://docs.mongodb.org/manual/reference/program/mongodump/ for more information.
general options:
--help print usage
--version print the tool version and exit
verbosity options:
-v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N)
--quiet hide all log output
connection options:
-h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets)
--port=<port> server port (can also use --host hostname:port)
authentication options:
-u, --username=<username> username for authentication
-p, --password=<password> password for authentication
--authenticationDatabase=<database-name> database that holds the user's credentials
--authenticationMechanism=<mechanism> authentication mechanism to use
namespace options:
-d, --db=<database-name> database to use
-c, --collection=<collection-name> collection to use
query options:
-q, --query= query filter, as a JSON string, e.g., '{x:{$gt:1}}'
--queryFile= path to a file containing a query filter (JSON)
--readPreference=<string>|<json> specify either a preference name or a preference json object
--forceTableScan force a table scan
output options:
-o, --out=<directory-path> output directory, or '-' for stdout (defaults to 'dump')
--gzip compress archive our collection output with Gzip
--repair try to recover documents from damaged data files (not supported by all storage engines)
--oplog use oplog for taking a point-in-time snapshot
--archive=<file-path> dump as an archive to the specified path. If flag is specified without a value, archive is written to stdout
--dumpDbUsersAndRoles dump user and role definitions for the specified database
--excludeCollection=<collection-name> collection to exclude from the dump (may be specified multiple times to exclude additional collections)
--excludeCollectionsWithPrefix=<collection-prefix> exclude all collections from the dump that have the given prefix (may be specified multiple times to exclude additional prefixes)
-j, --numParallelCollections= number of collections to dump in parallel (4 by default)
1.备份全库
最简单的一个备份全库语句如下:
[root@mongodb3 ~]# mongodump --host 127.0.0.1 --port 37017
2016-06-27T21:29:49.121+0800 writing admin.system.users to
2016-06-27T21:29:49.121+0800 done dumping admin.system.users (3 documents)
2016-06-27T21:29:49.121+0800 writing admin.system.roles to
2016-06-27T21:29:49.121+0800 done dumping admin.system.roles (2 documents)
2016-06-27T21:29:49.122+0800 writing admin.system.version to
2016-06-27T21:29:49.122+0800 done dumping admin.system.version (1 document)
2016-06-27T21:29:49.122+0800 writing suq.t2 to
2016-06-27T21:29:49.122+0800 writing suq.t1 to
2016-06-27T21:29:49.123+0800 done dumping suq.t2 (1 document)
2016-06-27T21:29:49.123+0800 done dumping suq.t1 (0 documents)
那么会在当前目录下创建一个dump目录,dump下面是备份的数据库名称:
[root@mongodb3 ~]# ls dump
admin suq
使用-o可以指定目录:
[root@mongodb3 ~]# mongodump --port 37017 -o backup/
如果数据使用的副本集,那么在备份的时候可以添加--oplog选项,在备份结束之后会将oplog的所有操作记录下来,得到某个时间点的快照.
2.备份数据库
使用-d选项备份某个数据库:
[root@mongodb3 ~]# mongodump --port 37017 -d suq -o backup
2016-06-27T21:35:00.406+0800 writing suq.t2 to
2016-06-27T21:35:00.406+0800 writing suq.t1 to
2016-06-27T21:35:00.406+0800 done dumping suq.t2 (1 document)
2016-06-27T21:35:00.407+0800 done dumping suq.t1 (0 documents)
[root@mongodb3 ~]# ls
anaconda-ks.cfg backup Desktop Documents Downloads install.log install.log.syslog mm mongod.conf Music Pictures Public soft Templates Videos
[root@mongodb3 ~]# ls backup
suq
3.备份指定集合
使用-c选项指定集合的名字
[root@mongodb3 ~]# mongodump --port 37017 -c t1 -d suq -o backup
2016-06-27T22:10:43.047+0800 writing suq.t1 to
2016-06-27T22:10:43.047+0800 done dumping suq.t1 (0 documents)
[root@mongodb3 ~]# ls backup
suq
[root@mongodb3 ~]# ls backup/suq/
t1.bson t1.metadata.json
4.使用mongorestore恢复
mongorestore用来恢复一个使用mongodump的备份.默认的mongorestore寻常当前目录下的dump目录.
下面是mongorestore的参数说明:
[root@mongodb3 ~]# mongorestore --help
Usage:
mongorestore <options> <directory or file to restore>
Restore backups generated with mongodump to a running server.
Specify a database with -d to restore a single database from the target directory,
or use -d and -c to restore a single collection from a single .bson file.
See http://docs.mongodb.org/manual/reference/program/mongorestore/ for more information.
general options:
--help print usage
--version print the tool version and exit
verbosity options:
-v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N)
--quiet hide all log output
connection options:
-h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets)
--port=<port> server port (can also use --host hostname:port)
authentication options:
-u, --username=<username> username for authentication
-p, --password=<password> password for authentication
--authenticationDatabase=<database-name> database that holds the user's credentials
--authenticationMechanism=<mechanism> authentication mechanism to use
namespace options:
-d, --db=<database-name> database to use
-c, --collection=<collection-name> collection to use
input options:
--objcheck validate all objects before inserting
--oplogReplay replay oplog for point-in-time restore
--oplogLimit=<seconds>[:ordinal] only include oplog entries before the provided Timestamp
--archive=<filename> restore dump from the specified archive file. If flag is specified without a value, archive is read from stdin
--restoreDbUsersAndRoles restore user and role definitions for the given database
--dir=<directory-name> input directory, use '-' for stdin
--gzip decompress gzipped input
restore options:
--drop drop each collection before import
--writeConcern=<write-concern> write concern options e.g. --writeConcern majority, --writeConcern '{w: 3, wtimeout: 500, fsync: true, j: true}' (defaults to 'majority')
--noIndexRestore don't restore indexes
--noOptionsRestore don't restore collection options
--keepIndexVersion don't update index version
--maintainInsertionOrder preserve order of documents during restoration
-j, --numParallelCollections= number of collections to restore in parallel (4 by default)
--numInsertionWorkersPerCollection= number of insert operations to run concurrently per collection (1 by default)
--stopOnError stop restoring if an error is encountered on insert (off by default)
--bypassDocumentValidation bypass document validation
默认的在restore的时候不会删除和清空原集合.可以使用--drop选项.
最简单的一个restore语法如下:
[root@mongodb3 ~]# mongorestore --port 37017 backup/
2016-06-27T22:18:27.571+0800 building a list of dbs and collections to restore from backup dir
2016-06-27T22:18:27.572+0800 reading metadata for suq.t1 from backup/suq/t1.metadata.json
2016-06-27T22:18:27.572+0800 restoring suq.t1 from backup/suq/t1.bson
2016-06-27T22:18:27.572+0800 restoring indexes for collection suq.t1 from metadata
2016-06-27T22:18:27.572+0800 finished restoring suq.t1 (1 document)
2016-06-27T22:18:27.572+0800 done
同样如果mongodump使用了--oplog,那么在恢复的hi和必须使用--oplogRelay来应用oplog的内容,得到一个时间点快照.
三.使用mongoexport/mongoimport备份恢复
1.mongoexport
mongoexport/mongoimport可以用来将数据导出成json和csv格式,注意mongoexport并不能完全用来备份数据库,因为json只是bson的子集,有一些bson type对于json是不支持的.
如果你只是想拷贝数据库或者集合到其它的实例,可以使用
copydb,
clone或者
clonecollection命令.详见官方文档的reference说明.
下面介绍mongoexport实例.在3.0以后移除了--csv选项,而使用--type=csv来代替.
[root@mongodb3 log]# mongoexport --help
Usage:
mongoexport <options>
Export data from MongoDB in CSV or JSON format.
See http://docs.mongodb.org/manual/reference/program/mongoexport/ for more information.
general options:
--help print usage
--version print the tool version and exit
verbosity options:
-v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N)
--quiet hide all log output
connection options:
-h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets)
--port=<port> server port (can also use --host hostname:port)
authentication options:
-u, --username=<username> username for authentication
-p, --password=<password> password for authentication
--authenticationDatabase=<database-name> database that holds the user's credentials
--authenticationMechanism=<mechanism> authentication mechanism to use
namespace options:
-d, --db=<database-name> database to use
-c, --collection=<collection-name> collection to use
output options:
-f, --fields=<field>[,<field>]* comma separated list of field names (required for exporting CSV) e.g. -f "name,age"
--fieldFile=<filename> file with field names - 1 per line
--type=<type> the output format, either json or csv (defaults to 'json')
-o, --out=<filename> output file; if not specified, stdout is used
--jsonArray output to a JSON array rather than one object per line
--pretty output JSON formatted to be human-readable
querying options:
-q, --query=<json> query filter, as a JSON string, e.g., '{x:{$gt:1}}'
--queryFile=<filename> path to a file containing a query filter (JSON)
-k, --slaveOk allow secondary reads if available (default true)
--readPreference=<string>|<json> specify either a preference name or a preference json object
--forceTableScan force a table scan (do not use $snapshot)
--skip=<count> number of documents to skip
--limit=<count> limit the number of documents to export
--sort=<json> sort order, as a JSON string, e.g. '{x:1}'
其中主要的参数:
--host:主机地址
--port:端口号
-u:用户名
-p:密码
-d:数据库名
-c:集合名
-f:列名,当--type为csv的时候必须指定
-o:文件名
-q:查询条件
下面是导出csv格式实例:
[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 --type=csv -f name,age -o 1.csv
2016-06-28T20:01:23.951+0800 connected to: localhost:37017
2016-06-28T20:01:23.951+0800 exported 2 records
[root@mongodb3 test]# cat field.txt name age
[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 --type=csv --fieldFile field.txt -o 1.csv
2016-06-28T20:08:55.504+0800 connected to: localhost:37017
2016-06-28T20:08:55.505+0800 exported 2 records
默认导出为json格式,下面是导出json格式:
[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 -o suq.json
2016-06-28T20:12:11.756+0800 connected to: localhost:37017
2016-06-28T20:12:11.756+0800 exported 2 records
[root@mongodb3 test]# cat suq.json
{"_id":{"$oid":"577266483ef351522beaefe4"},"name":"tony","age":10.0}
{"_id":{"$oid":"577266603ef351522beaefe5"},"name":"tony","age":10.0,"sex":1.0}
导出部分数据,使用--query接查询条件:
[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 --query '{"sex":1}' -o suq.json
2016-06-28T20:14:00.656+0800 connected to: localhost:37017
2016-06-28T20:14:00.657+0800 exported 1 record
2.mongoimport
mongoimport帮助说明:
[root@mongodb3 test]# mongoimport --help
Usage:
mongoimport <options> <file>
Import CSV, TSV or JSON data into MongoDB. If no file is provided, mongoimport reads from stdin.
See http://docs.mongodb.org/manual/reference/program/mongoimport/ for more information.
general options:
--help print usage
--version print the tool version and exit
verbosity options:
-v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N)
--quiet hide all log output
connection options:
-h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets)
--port=<port> server port (can also use --host hostname:port)
authentication options:
-u, --username=<username> username for authentication
-p, --password=<password> password for authentication
--authenticationDatabase=<database-name> database that holds the user's credentials
--authenticationMechanism=<mechanism> authentication mechanism to use
namespace options:
-d, --db=<database-name> database to use
-c, --collection=<collection-name> collection to use
input options:
-f, --fields=<field>[,<field>]* comma separated list of field names, e.g. -f name,age
--fieldFile=<filename> file with field names - 1 per line
--file=<filename> file to import from; if not specified, stdin is used
--headerline use first line in input source as the field list (CSV and TSV only)
--jsonArray treat input source as a JSON array
--type=<type> input format to import: json, csv, or tsv (defaults to 'json')
ingest options:
--drop drop collection before inserting documents
--ignoreBlanks ignore fields with empty values in CSV and TSV
--maintainInsertionOrder insert documents in the order of their appearance in the input source
-j, --numInsertionWorkers=<number> number of insert operations to run concurrently (defaults to 1)
--stopOnError stop importing at first insert/upsert error
--upsert insert or update objects that already exist
--upsertFields=<field>[,<field>]* comma-separated fields for the query part of the upsert
--writeConcern=<write-concern-specifier> write concern options e.g. --writeConcern majority, --writeConcern '{w: 3, wtimeout: 500, fsync: true, j: true}' (defaults to 'majority')
--bypassDocumentValidation bypass document validation
json格式的import:
[root@mongodb3 test]# mongoimport --port 37017 -d suq -c suq --file suq.json
2016-06-28T20:40:25.148+0800 connected to: localhost:37017
2016-06-28T20:40:25.176+0800 imported 1 document
csv格式的import:
其中注意使用--headerline指定列的名字.
[root@mongodb3 test]# mongoimport --port 37017 --type=csv -d suq -c suq2 --headerline --file= 1.csv
2016-06-28T20:45:28.430+0800 connected to: localhost:37017
2016-06-28T20:45:28.458+0800 imported 2 documents