副本集模式:相当于是自带故障转移功能的主从复制
分片集群:(将数据子集分散在集群中,每个分片维护着一个数据集合的子集。与副本集相比,使用集群架构可以使应用程序具有更强大的数据处理能力。)
分片服务器(Shard Server):将数据子集分散在集群中,每个分片维护着一个数据集合的子集。与副本集相比,使用集群架构可以使应用程序具有更强大的数据处理能力。在实际生产中,一个 Shard Server 可由几台机器组成一个副本集来承担,防止因主节点单点故障导致整个系统崩溃
配置服务器(Config Server)这是独立的一个 mongod 进程,保存集群和分片的元数据,在集群启动最开始时建立,保存各个分片包含数据的信息。
路由服务器(Route Server):这是独立的一个 mongos 进程,Route Server 在集群中可作为路由使用,客户端由此接入,让整个集群看起来像是一个单一的数据库,提供客户端应用程序和分片集群之间的接口。
Route Server 本身不保存数据,启动时从 Config Server 加载集群信息到缓存中,
并将客户端的请求路由给每个 Shard Server,在各 Shard Server 返回结果后进行聚合并返回客户端.
mongodb下载地址:
https://www.mongodb.com/download-center/community/releases
RedHat / CentOS 7.0 x64
Archive: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-4.4.9.tgz
Server Package: https://repo.mongodb.org/yum/redhat/7/mongodb-org/4.4/x86_64/RPMS/mongodb-org-server-4.4.9-1.el7.x86_64.rpm
Mongos Package: https://repo.mongodb.org/yum/redhat/7/mongodb-org/4.4/x86_64/RPMS/mongodb-org-mongos-4.4.9-1.el7.x86_64.rpm
Mongo Shell Package: https://repo.mongodb.org/yum/redhat/7/mongodb-org/4.4/x86_64/RPMS/mongodb-org-shell-4.4.9-1.el7.x86_64.rpm
mongodb副本集安装流程
目前最小节点是3个
1、在集群每个节点上安装一份MongoDB;
2、配置副本集;
3、配置副本集的用户、密码;
4、配置副本集的KeyFile安全鉴权;
5、配置开机自启动;
服务器配置
16核 64G内存 40G+2000G存储 RedHat 7.6
192.168.100.138 主节点
192.168.100.139 从节点
192.168.100.140 仲裁节点
groupadd mongod
useradd -g mongod mongod
yum install -y libcurl openssl glibc
mkdir /opt
tar -zxvf mongodb-linux-x86_64-rhel70-4.4.9.tgz
mv mongodb-linux-x86_64-rhel70-4.4.9 /opt/mongodb
mkdir /opt/mongodb/{data,conf,logs,pid} -p
mkdir /var/run/mongodb
/opt/mongodb/
├── bin #可执行文件
│ ├── install_compass
│ ├── mongo
│ ├── mongod
│ └── mongos
├── conf #存放配置文件和Keyfile
├── data #存放数据库文件
│ ├── collection #数据
│ ├── index #索引
│ └── journal #日志,类似mysql redo log
├── LICENSE-Community.txt
├── logs #存放系统日志
├── MPL-2
├── pid
├── README
└── THIRD-PARTY-NOTICES
#条件允许的话,可以对服务器的磁盘进行优化,数据、索引、日志放在不同的磁盘(也可以加大内存,满足数据读取,减低磁盘交换)
mount /dev/sdb1 /opt #450G
mount /dev/sdc1 /opt/mongodb/data/index #100G
mount /dev/sdd1 /opt/mongodb/data/journal #50G
# vi /etc/bashrc #或者vi /etc/profile
#MongoDB config
MONGO_HOME=/opt/mongodb
PATH=$MONGO_HOME/bin:$PATH
export MONGO_HOME PATH
# source /etc/bashrc
# vi /opt/mongodb/conf/mongod.conf
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /opt/mongodb/logs/mongod.log #日志文件
# Where and how to store data.
storage:
dbPath: /opt/mongodb/data #数据库存储位置
journal:
enabled: true
commitIntervalMs: 500 #降低journal刷盘时间,500ms(最大的配置)
engine: "wiredTiger"
wiredTiger:
engineConfig:
cacheSizeGB: 50 ##(64G内存配置50G)(96G内存配置75G整数)。如果一台机器启动一个实例这个可以注释选择默认,如果一台机器启动多个实例,需要设置内存大小,避免互相抢占内存
journalCompressor: "snappy"
directoryForIndexes: true
collectionConfig:
blockCompressor: "snappy"
indexConfig:
prefixCompression: true
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
# network interfaces
net:
port: 27017
maxIncomingConnections: 20000
bindIp: 0.0.0.0 # Listen to local interface only, comment to listen on all interfaces.
security:
authorization: 'enabled'
clusterAuthMode: "keyFile"
keyFile: /opt/mongodb/conf/rs.key
operationProfiling:
slowOpThresholdMs: 200
mode: slowOp
replication:
oplogSizeMB: 102400 #复制操作日志的最大大小,以M为单位,默认情况下,mongod进程基于最大可用空间创建oplog,对于64位系统,oplog通常占可用磁盘空间的5%;
replSetName: test #副本集的名称
enableMajorityReadConcern: false
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
5、分别启动每个节点:A、B、C节点
#创建认证文件
#使用openssl命令随机生成741个字节的随机数字,然后将数字写入rs.key文件中,编码方式为base64,rand表示随机
# cd /opt/mongodb/conf/
# openssl rand -base64 741 > rs.key
ohMhfKa8oX6IDO2NS+YXsoDh0pcz/AJi013D6Ao6L1cO2LFaKtBwDx5vR9VKJvEQ
wMBC2syOFDFuze3U1JZ+DwsIpahvZCG1e74KG4Mgy/l+pJKdeoW2Qk2aRKkf3K00
LxCFmwhbdRZ5ARQNoq3fLhQg0eH7fvajzEXnwfpsIbIeOv6h+uH16nmKklTb426V
rlThA779UFMKmUq17xvY3mZbiPjjEJgC8oMQUOu8c4xUGL8oaC39zM9QjiNG/EoH
TCjra6UN+O9SR2Q09R+u7GZi0yrnLhVpgDjHMOmfWStHkEgHCdsur6r06fQ3YQuq
UE/EHkPjn99hcgRJ+7bfJbR14h76g1lGKPFJ17W8N+UZA+63emhduxfG77PLExuU
g2NURjXV85W4ISSpbak53tjXISNUwT+KKlGoH1UPLtGKWKzgAGMPzsIAva/+fkJi
BKz8gKdWOp3ivaSHUGOZMRUoluAHeQxM77C+DJG9gTPzQ9JcCQnDZFPgIfiYLpl5
dPZu5jWuU0HVRu03fNkeapkkuqIFdiE9kMzyVXpFiJJXUt1YvP/DDbfo9HTeaFw1
AzeKmXasatO6AWvMXc7GBA5KQlqfgrirbViZfw3vb52vTjcQXwmKNPikDfzvKQEd
OdNWHMJPBbIKkMD3BHefMA/CimOEXj9Ryq9cMRnOOSLuUkkkKJlrsxCzaekRUs97
SnjcD5YuWx8YOF1F0ERgS3DBAtPHlcuybmRMTac6Gb3qeMkJEs13IZbA4pmmwTs2
Jj8yuJIKKqxr+Zebjg5DxBte1PF3r7e2YwC3fx7gPP+0gh2a2yOUcL1HK5I3FA69
ryKs6HQ7SHbVnfUpY/1O9MiahBckpi8NkWclYndbS5PWEi+IK7g7sHR8cYuZGrR9
YXlJQH7VOfVaat3+kLJQT0b40YtylFfCW0Nmu83TEyb7+9jucprpLqraNLbb9RvO
eFQJIkSBwvC4nf5voa9TPgRpX+5J
# chmod 600 rs.key
# scp rs.key [email protected]:/opt/mongodb/conf/ #将此文件rs.key传到另外2台
# chown -R mongod.mongod /opt
# chown -R mongod.mongod /var/run/mongodb
# sudo -u mongod /opt/mongodb/bin/mongod -f /opt/mongodb/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 1725
child process started successfully, parent exiting
# ps -ef | grep mongo
mongod 1725 1 19 19:37 ? 00:00:02 /opt/mongodb/bin/mongod -f /opt/mongodb/conf/mongod.conf
root 1771 1641 0 19:37 pts/1 00:00:00 grep --color=auto mongo
#关闭mongo服务
# sudo -u mongod /opt/mongodb/bin/mongod --shutdown --dbpath /opt/mongodb/data
设置MongoDB开机自启动
# vi /etc/rc.d/rc.local #加入下面的自启动脚本
sudo -u mongod /opt/mongodb/bin/mongod -f /opt/mongodb/conf/mongod.conf
1、使用mongo 客户端 连接主节点或备节点进行配置
#配置防火墙,允许27017端口可以相互访问
# mongo --port 27017
#连接成功后,开始配置副本集:
> use admin
switched to db admin
# test为配置文件中副本集名称
> config = {_id: 'test', members: [{_id: 0, host: '192.168.100.138:27017',priority:2},{_id: 1, host: '192.168.100.139:27017',priority:1},{_id: 2, host:'192.168.100.140:27017',arbiterOnly:true}]}
{
"_id" : "replSet",
"members" : [
{
"_id" : 0,
"host" : "192.168.100.138:27017",
"priority" : 2
},
{
"_id" : 1,
"host" : "192.168.100.139:27017",
"priority" : 1
},
{
"_id" : 2,
"host" : "192.168.100.140:27017",
"arbiterOnly" : true
}
]
}
> rs.initiate(config)
{ "ok" : 1 }
> rs.status() #查看副本集状态,找到private节点的IP
test:SECONDARY> rs.status()
{
"set" : "test",
"date" : ISODate("2021-09-28T12:10:56.025Z"),
"myState" : 1,
"term" : NumberLong(2),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"votingMembersCount" : 3,
"writableVotingMembersCount" : 2,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1632831055, 1),
"t" : NumberLong(2)
},
"lastCommittedWallTime" : ISODate("2021-09-28T12:10:55.675Z"),
"appliedOpTime" : {
"ts" : Timestamp(1632831055, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1632831055, 1),
"t" : NumberLong(2)
},
"lastAppliedWallTime" : ISODate("2021-09-28T12:10:55.675Z"),
"lastDurableWallTime" : ISODate("2021-09-28T12:10:55.675Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "priorityTakeover",
"lastElectionDate" : ISODate("2021-09-28T12:10:05.552Z"),
"electionTerm" : NumberLong(2),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(1632831015, 1),
"t" : NumberLong(1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1632831015, 1),
"t" : NumberLong(1)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 2,
"electionTimeoutMillis" : NumberLong(10000),
"priorPrimaryMemberId" : 1,
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2021-09-28T12:10:05.645Z")
},
"electionParticipantMetrics" : {
"votedForCandidate" : true,
"electionTerm" : NumberLong(1),
"lastVoteDate" : ISODate("2021-09-28T12:09:55.023Z"),
"electionCandidateMemberId" : 1,
"voteReason" : "",
"lastAppliedOpTimeAtElection" : {
"ts" : Timestamp(1632830733, 1),
"t" : NumberLong(-1)
},
"maxAppliedOpTimeInSet" : {
"ts" : Timestamp(1632830733, 1),
"t" : NumberLong(-1)
},
"priorityAtElection" : 2
},
"members" : [
{
"_id" : 0,
"name" : "192.168.100.138:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY", #主节点
"uptime" : 2049,
"optime" : {
"ts" : Timestamp(1632831055, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2021-09-28T12:10:55Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1632831015, 2),
"electionDate" : ISODate("2021-09-28T12:10:15Z"),
"configVersion" : 1,
"configTerm" : 2,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 1,
"name" : "192.168.100.139:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY", #从节点
"uptime" : 67,
"optime" : {
"ts" : Timestamp(1632831055, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1632831055, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2021-09-28T12:10:55Z"),
"optimeDurableDate" : ISODate("2021-09-28T12:10:55Z"),
"lastHeartbeat" : ISODate("2021-09-28T12:10:55.723Z"),
"lastHeartbeatRecv" : ISODate("2021-09-28T12:10:54.816Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "192.168.100.138:27017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 1,
"configTerm" : 2
},
{
"_id" : 2,
"name" : "192.168.100.140:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER", #仲裁节点
"uptime" : 67,
"lastHeartbeat" : ISODate("2021-09-28T12:10:55.723Z"),
"lastHeartbeatRecv" : ISODate("2021-09-28T12:10:55.819Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : 1,
"configTerm" : 2
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1632831055, 1),
"signature" : {
"hash" : BinData(0,"e44mHwkuw62xDz57ke9tFqtvRGA="),
"keyId" : NumberLong("7012955805024518149")
}
},
"operationTime" : Timestamp(1632831055, 1)
}
在主节点上,用客户端连接,创建用户权限(主节点,可以用 rs.status() 查看)
# mongo --port 27017
#配置密码
# 登录主节点,进入admin数据库,分别创建用户名密码root/!@cmcc1234以及family/Cmcc!2021,
# family用户为应用程序所使用的用户(即填入各个应用配置文件中的账户信息)
# root用户为数据库管理员用户
test:PRIMARY> use admin
test:PRIMARY> db.createUser(
{
user: "root",
pwd: "Mima*2021",
roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
}
)
test:PRIMARY> db.auth("root", "Mima*2021" )
test:PRIMARY> db.createUser(
{
user: "family",
pwd: "Cmcc@2021",
roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
}
)
# 更改用户名密码
db.changeUserPassword("root", "Mima*2021")
db.changeUserPassword("family", "Cmcc@2021")
# 创建用户名密码后登录方式:
#(1)直接连接实例验证
mongo --port 27017 -u root -p 'Mima*2021'
mongo --port 27017 -u "root" --authenticationDatabase "admin" -p
# (2) 连接实例之后,使用数据库的时候验证
mongo --port 27017
test:PRIMARY> use admin
test:PRIMARY> db.auth("root", "Mima*2021" )
#创建普通数据库名mytest、用户mytest/mytest
replSet:PRIMARY> use mytest #创建mytest数据库
replSet:PRIMARY> db.createUser({user:"mytest",pwd:"mytest",roles:[{role:"dbOwner",db:"mytest"}]})
# mongo 172.17.0.3:27018/mytest -u mytest -p mytest #验证
role角色
数据库用户角色:read、readWrite;
数据库管理角色:dbAdmin、dbOwner、userAdmin;
集群管理角色:clusterAdmin、clusterManager、clusterMonitor、hostManager;
备份恢复角色:backup、restore;
所有数据库角色:readAnyDatabase、readWriteAnyDatabase、userAdminAnyDatabase、dbAdminAnyDatabase
超级用户角色:root
内部角色:__system
# 角色说明
read:允许用户读取指定数据库
readWrite:允许用户读写指定数据库
dbAdmin:允许用户在指定数据库中执行管理函数,如索引创建、删除,查看统计或访问system.profile
userAdmin:允许用户向system.users集合写入,可以找指定数据库里创建、删除和管理用户
clusterAdmin:只在admin数据库中可用,赋予用户所有分片和复制集相关函数的管理权限。
readAnyDatabase:只在admin数据库中可用,赋予用户所有数据库的读权限
readWriteAnyDatabase:只在admin数据库中可用,赋予用户所有数据库的读写权限
userAdminAnyDatabase:只在admin数据库中可用,赋予用户所有数据库的userAdmin权限
dbAdminAnyDatabase:只在admin数据库中可用,赋予用户所有数据库的dbAdmin权限。
root:只在admin数据库中可用。超级账号,超级权限
dbOwner: readWrite + dbAdmin + dbAdmin
# mongodb支持哪些命令
> help
db.help() help on db methods
db.mycoll.help() help on collection methods
sh.help() sharding helpers
rs.help() replica set helpers
help admin administrative help
help connect connecting to a db help
help keys key shortcuts
help misc misc things to know
help mr mapreduce
show dbs show database names
show collections show collections in current database
show users show users in current database
show profile show most recent system.profile entries with time >= 1ms
show logs show the accessible logger names
show log [name] prints out the last segment of log in memory, 'global' is default
use <db_name> set current database
db.mycoll.find() list objects in collection mycoll
db.mycoll.find( { a : 1 } ) list objects in mycoll where a == 1
it result of the last line evaluated; use to further iterate
DBQuery.shellBatchSize = x set default number of items to display on shell
exit quit the mongo shell
# 当前数据库支持哪些方法
> db.help();
DB methods:
db.adminCommand(nameOrDocument) - switches to 'admin' db, and runs command [just calls db.runCommand(...)]
db.aggregate([pipeline], {options}) - performs a collectionless aggregation on this database; returns a cursor
db.auth(username, password) # 访问认证
db.cloneDatabase(fromhost) - will only function with MongoDB 4.0 and below
db.commandHelp(name) returns the help for the command
db.copyDatabase(fromdb, todb, fromhost) - will only function with MongoDB 4.0 and below #复制数据库
db.createCollection(name, {size: ..., capped: ..., max: ...}) #创建表
db.createUser(userDocument)
db.createView(name, viewOn, [{$operator: {...}}, ...], {viewOptions})
db.currentOp() displays currently executing operations in the db
db.dropDatabase(writeConcern) #删除当前数据库
db.dropUser(username)
db.eval() - deprecated
db.fsyncLock() flush data to disk and lock server for backups
db.fsyncUnlock() unlocks server following a db.fsyncLock()
db.getCollection(cname) same as db['cname'] or db.cname
db.getCollectionInfos([filter]) - returns a list that contains the names and options of the db's collections
db.getCollectionNames() # 获取当前数据库的表名
db.getLastError() - just returns the err msg string
db.getLastErrorObj() - return full status object
db.getLogComponents()
db.getMongo() get the server connection object
db.getMongo().setSecondaryOk() allow queries on a replication secondary server
db.getName()
db.getProfilingLevel() - deprecated
db.getProfilingStatus() - returns if profiling is on and slow threshold
db.getReplicationInfo()
db.getSiblingDB(name) get the db at the same server as this one
db.getWriteConcern() - returns the write concern used for any operations on this db, inherited from server object if set
db.hostInfo() get details about the server's host
db.isMaster() check replica primary status
db.hello() check replica primary status
db.killOp(opid) kills the current operation in the db
db.listCommands() lists all the db commands
db.loadServerScripts() loads all the scripts in db.system.js
db.logout()
db.printCollectionStats() #打印各表的状态信息
db.printReplicationInfo() #打印主数据库的复制状态信息
db.printShardingStatus()
db.printSecondaryReplicationInfo() #打印从数据库的复制状态信息
db.resetError()
db.runCommand(cmdObj) run a database command. if cmdObj is a string, turns it into {cmdObj: 1}
db.serverStatus()
db.setLogLevel(level,<component>)
db.setProfilingLevel(level,slowms) 0=off 1=slow 2=all
db.setVerboseShell(flag) display extra information in shell output
db.setWriteConcern(<write concern doc>) - sets the write concern for writes to the db
db.shutdownServer()
db.stats()
db.unsetWriteConcern(<write concern doc>) - unsets the write concern for writes to the db
db.version() current version of the server
db.watch() - opens a change stream cursor for a database to report on all changes to its non-system collections.
# 当前数据库下的表或者表collection支持哪些方法
> db.user.help(); #user为表名
DBCollection help
db.user.find().help() - show DBCursor help
db.user.bulkWrite( operations, <optional params> ) - bulk execute write operations, optional parameters are: w, wtimeout, j
db.user.count( query = {}, <optional params> ) - count the number of documents that matches the query, optional parameters are: limit, skip, hint, maxTimeMS #统计表的行数
db.user.countDocuments( query = {}, <optional params> ) - count the number of documents that matches the query, optional parameters are: limit, skip, hint, maxTimeMS
db.user.estimatedDocumentCount( <optional params> ) - estimate the document count using collection metadata, optional parameters are: maxTimeMS
db.user.convertToCapped(maxBytes) - calls {convertToCapped:'user', size:maxBytes}} command
db.user.createIndex(keypattern[,options])
db.user.createIndexes([keypatterns], <options>)
db.user.dataSize() #统计表数据的大小
db.user.deleteOne( filter, <optional params> ) - delete first matching document, optional parameters are: w, wtimeout, j
db.user.deleteMany( filter, <optional params> ) - delete all matching documents, optional parameters are: w, wtimeout, j
db.user.distinct( key, query, <optional params> ) - e.g. db.user.distinct( 'x' ), optional parameters are: maxTimeMS #按照给定的条件除重
db.user.drop() drop the collection #删除表
db.user.dropIndex(index) - e.g. db.user.dropIndex( "indexName" ) or db.user.dropIndex( { "indexKey" : 1 } ) # 删除指定索引
db.user.hideIndex(index) - e.g. db.user.hideIndex( "indexName" ) or db.user.hideIndex( { "indexKey" : 1 } )
db.user.unhideIndex(index) - e.g. db.user.unhideIndex( "indexName" ) or db.user.unhideIndex( { "indexKey" : 1 } )
db.user.dropIndexes() #删除所有索引
db.user.ensureIndex(keypattern[,options]) - DEPRECATED, use createIndex() instead
db.user.explain().help() - show explain help
db.user.reIndex()
db.user.find([query],[fields]) - query is an optional query filter. fields is optional set of fields to return.
e.g. db.user.find( {x:77} , {name:1, x:1} )
db.user.find(...).count()
db.user.find(...).limit(n)
db.user.find(...).skip(n)
db.user.find(...).sort(...)
db.user.findOne([query], [fields], [options], [readConcern])
db.user.findOneAndDelete( filter, <optional params> ) - delete first matching document, optional parameters are: projection, sort, maxTimeMS
db.user.findOneAndReplace( filter, replacement, <optional params> ) - replace first matching document, optional parameters are: projection, sort, maxTimeMS, upsert, returnNewDocument
db.user.findOneAndUpdate( filter, <update object or pipeline>, <optional params> ) - update first matching document, optional parameters are: projection, sort, maxTimeMS, upsert, returnNewDocument
db.user.getDB() get DB object associated with collection
db.user.getPlanCache() get query plan cache associated with collection
db.user.getIndexes()
db.user.insert(obj)
db.user.insertOne( obj, <optional params> ) - insert a document, optional parameters are: w, wtimeout, j
db.user.insertMany( [objects], <optional params> ) - insert multiple documents, optional parameters are: w, wtimeout, j
db.user.mapReduce( mapFunction , reduceFunction , <optional params> )
db.user.aggregate( [pipeline], <optional params> ) - performs an aggregation on a collection; returns a cursor
db.user.remove(query)
db.user.replaceOne( filter, replacement, <optional params> ) - replace the first matching document, optional parameters are: upsert, w, wtimeout, j
db.user.renameCollection( newName , <dropTarget> ) renames the collection.
db.user.runCommand( name , <options> ) runs a db command with the given name where the first param is the collection name
db.user.save(obj)
db.user.stats({scale: N, indexDetails: true/false, indexDetailsKey: <index key>, indexDetailsName: <index name>})
db.user.storageSize() - includes free space allocated to this collection
db.user.totalIndexSize() - size in bytes of all the indexes
db.user.totalSize() - storage allocated for all data and indexes
db.user.update( query, <update object or pipeline>[, upsert_bool, multi_bool] ) - instead of two flags, you can pass an object with fields: upsert, multi, hint
db.user.updateOne( filter, <update object or pipeline>, <optional params> ) - update the first matching document, optional parameters are: upsert, w, wtimeout, j, hint
db.user.updateMany( filter, <update object or pipeline>, <optional params> ) - update all matching documents, optional parameters are: upsert, w, wtimeout, j, hint
db.user.validate( <full> ) - SLOW
db.user.getShardVersion() - only for use with sharding
db.user.getShardDistribution() - prints statistics about data distribution in the cluster
db.user.getSplitKeysForChunks( <maxChunkSize> ) - calculates split points over all chunks and returns splitter function
db.user.getWriteConcern() - returns the write concern used for any operations on this collection, inherited from server/db if set
db.user.setWriteConcern( <write concern doc> ) - sets the write concern for writes to the collection
db.user.unsetWriteConcern( <write concern doc> ) - unsets the write concern for writes to the collection
db.user.latencyStats() - display operation latency histograms for this collection
参考文档:https://www.cnblogs.com/dbabd/p/10811523.html
1. 超级用户相关:
#增加或修改用户密码
db.addUser('admin','pwd')
#查看用户列表
db.system.users.find()
#用户认证
db.auth('admin','pwd')
#删除用户
db.removeUser('mongodb')
#查看所有用户
show users
#查看所有数据库
show dbs
#查看所有的collection
show collections
#查看各collection的状态
db.printCollectionStats()
#查看主从复制状态
db.printReplicationInfo()
#修复数据库
db.repairDatabase()
#设置记录profiling,0=off 1=slow 2=all
db.setProfilingLevel(1)
#查看profiling
show profile
#拷贝数据库
db.copyDatabase('mail_addr','mail_addr_tmp')
#删除collection
db.mail_addr.drop()
#删除当前的数据库
db.dropDatabase()
2. 客户端连接
/usr/local/mongodb/bin/mongo user_addr -u user -p 'pwd'
3. 增删改
#存储嵌套的对象
db.foo.save({'name':'ysz','address':{'city':'beijing','post':100096},'phone':[138,139]})
#存储数组对象
db.user_addr.save({'Uid':'[email protected]','Al':['[email protected]','[email protected]']})
#根据query条件修改,如果不存在则插入,允许修改多条记录
db.foo.update({'yy':5},{'$set':{'xx':2}},upsert=true,multi=true)
#删除yy=5的记录
db.foo.remove({'yy':5})
#删除所有的记录
db.foo.remove()
4. 索引
增加索引:1(ascending),-1(descending)
db.things.ensureIndex({firstname: 1, lastname: 1}, {unique: true});
#索引子对象
db.user_addr.ensureIndex({'Al.Em': 1})
#查看索引信息
db.deliver_status.getIndexes()
db.deliver_status.getIndexKeys()
#根据索引名删除索引
db.user_addr.dropIndex('Al.Em_1')
5. 查询
查找所有
db.foo.find()
#查找一条记录
db.foo.findOne()
#根据条件检索10条记录
db.foo.find({'msg':'Hello 1'}).limit(10)
#sort排序
db.deliver_status.find({'From':'[email protected]'}).sort({'Dt',-1})
db.deliver_status.find().sort({'Ct':-1}).limit(1)
#count操作
db.user_addr.count()
#distinct操作
db.foo.distinct('msg')
#>操作
db.foo.find({"timestamp": {"$gte" : 2}})
#子对象的查找
db.foo.find({'address.city':'beijing'})
6. 管理
查看collection数据的大小
db.deliver_status.dataSize()
#查看colleciont状态
db.deliver_status.stats()
#查询所有索引的大小
db.deliver_status.totalIndexSize()
目前,MongoDB使用的是内存映射存储引擎,它会把磁盘IO操作转换成内存操作,如果是读操作,内存中的数据起到缓存的作用,如果是写操作,内存还可以把随机的写操作转换成顺序的写操作,总之可以大幅度提升性能。MongoDB并不干涉内存管理工作,而是把这些工作留给操作系统的虚拟缓存管理器去处理,这样的好处是简化了MongoDB的工作,但坏处是你没有方法很方便的控制MongoDB占多大内存,事实上MongoDB会占用所有能用的内存,所以最好不要把别的服务和MongoDB放一起。
简而言之,就是mongod把这事交给操作系统了,缺了就跟OS要内存,多了也不还,爱咋咋地。
如果需要强行收回内存也不是没有办法:
(1)重启mongod,或者调用 db.runCommand({closeAllDatabases:1})来清除内存
(2)使用Linux命令清除缓存中的数据:echo 3 > /proc/sys/vm/drop_caches
在应用运行过程中重启mongod,是很不现实的操作,可以尝试使用下面的方法:
mongod提供了额外的可选参数来控制WiredTiger存储引擎所占用的cache size。需要注意的是,cache size设置较低,同时mongodb复杂查询很频繁的话,会有延迟发生。cacheSizeGB 指的就是Cache size,包括数据和索引。Mongod本身使用内存如连接池堆栈以及sorting buffer等都是额外的,不会被统计到这个数字里面。如果索引在内存,查询冷数据取决于你的IO能力。如果IO latency很低,系统也没有是高负载,那响应时间应该是毫秒级的区别。但是如果查询很频繁,又涉及到很多范围、批量查询,IOPS又跟不上,那就有可能到几百几千毫秒都有可能。
说明:4.4以上,mongoimport mongoexport就是报错,这些工具分离出去了,MongoDB不自带,需要另外安装
安装文档:https://docs.mongodb.com/database-tools/installation/installation-linux/
命令使用文档:https://docs.mongodb.com/database-tools/mongoimport/
下载地址:https://fastdl.mongodb.org/tools/db/mongodb-database-tools-rhel70-x86_64-100.5.0.tgz
MongoDB Tools 工具说明
===================================
- **bsondump** - _display BSON files in a human-readable format_
- **mongoimport** - _Convert data from JSON, TSV or CSV and insert them into a collection_
- **mongoexport** - _Write an existing collection to CSV or JSON format_
- **mongodump/mongorestore** - _Dump MongoDB backups to disk in .BSON format, or restore them to a live database_
- **mongostat** - _Monitor live MongoDB servers, replica sets, or sharded clusters_
- **mongofiles** - _Read, write, delete, or update files in [GridFS](http://docs.mongodb.org/manual/core/gridfs/)_
- **mongotop** - _Monitor read/write activity on a mongo server_
# 安装mongodb Tools
tar -zxf mongodb-database-tools-rhel70-x86_64-100.5.0.tgz
cd mongodb-database-tools-rhel70-x86_64-100.5.0/bin/
cp ./* /opt/mongodb/bin/
# 给管理账户授权角色
在监控MongoDB的运行状况上,常用的命令行命令是 mongostat 和 mongotop。但是,当开启了授权后,使用mongostat就不那么顺利了,需要给管理账户授权角色才可以
db.createRole(
{role: "mongostatRole",
privileges: [{ resource: { cluster: true }, actions: [ "serverStatus", "top" ] }
],roles: []}
)
db.grantRolesToUser("root",
[{role: "mongostatRole",db: "admin"}
])
#删除用户角色
db.dropRole("mongostatRole")
# 给root用户授权后可以正常使用mongostat和mongotop啦
mongostat --username='root' --password='Mima*2021' --authenticationDatabase=admin
mongotop --username='root' --password='Mima*2021' --authenticationDatabase=admin
#导入文件
mongoimport --username='root' --password='Mima*2021' --authenticationDatabase=admin -d cmhigateway -c SubDeviceStb --type=json --file=/opt/mongodb/SubDeviceStb.js