MongoDB3.4版本分片集群+副本集的搭建教程
前言：
最近单位在做系统迁移，我负责基础服务的代建，写一篇文章来记录一下MongoDB分片集群+副本集的搭建过程。
正文：
一、基础概念：
MongoDB分片集群+副本集主要包括：Mongos、Config Server、Shard、Replica set四个部分。Mongos是数据库的入口，相当于路由的功能。Config Server是配置服务器，存储所有数据库的元数据。Shard是分片，将数据库存储的数据拆分到不同的机器上，减少机器的负载。replica set是副本集，它就是shard的备份。
二、服务器规划：
| 服务器3 | 服务器8 | 服务器16 |
|---|---|---|
| mongos | mongos | mongos |
| config server | config server | config server |
| shard server1 副节点 | shard server1 主节点 | shard server1 仲裁 |
| shard server2 仲裁 | shard server2 副节点 | shard server2 主节点 |
| shard server3 主节点 | shard server3 仲裁 | shard server3 副节点 |
三、搭建过程：
1.上传mongoDB安装包(mongodb-linux-x86_64-rhel62-3.4.6.tgz)：
cd /usr/local
rz -bye 2.解压mongoDB安装包：
tar -zxvf ./mongodb-linux-x86_64-rhel62-3.4.6.tgz
mv ./mongodb-linux-x86_64-rhel62-3.4.6 ./mongodb3.4.63.创建conf文件夹：
mkdir -p /usr/local/mongodb3.4.6/conf4.创建data目录：
mkdir -p /data/mongodb/data/{conf,keyfile,mongos,shard1,shard2,shard3}5.上传security文件：
cd /data/mongodb/data/keyfile
rz -bye
chmod 400 /data/mongodb/data/keyfile/*mongodb集群增加登录密码验证参考：https://blog.csdn.net/welcome66/article/details/84916061
6.config server配置服务器：
6.1 配置文件：
#此处为配置文件可配置的内容
#Mongod config file
#MongoDB configuration files use the YAML format.
#The following example configuration file contains several mongod settings.
#
########Example Start########
#systemLog:
# destination: file
# path: "/var/log/mongodb/mongodb.log"
# logAppend: true
#storage:
# journal:
# enabled: true
#processManagement:
# fork: true
#net:
# bindIp: 127.0.0.1
# port: 27017
#setParameter:
# enableLocalhostAuthBypass: false
#
########Example End########
#
########Core Options
systemLog:
# verbosity: 0 #Default: 0; 1 to 5 increases the verbosity level to include Debug messages.
# quiet:
# traceAllException:
# syslogFacility: user
path: "/data/mongodb/data/conf/mongodb.log"
logAppend: true
# logRotate: #rename or reopen
destination: file
# timeStampFormat: iso8601-local
# component:
# accessControl:
# verbosity: 0
# command:
# verbosity: 0
# # COMMENT additional component verbosity settings omitted for brevity
# storage:
# verbosity: 0
# journal:
# verbosity:
# write:
# verbosity: 0
#
#
########ProcessManagement Options
processManagement:
fork: true
pidFilePath: "/data/mongodb/data/conf/mongod.pid"
#
#
#########Net Options
net:
port: 27020
bindIp: 0.0.0.0 #Default All interfaces.
# maxIncomingConnections: 65536
# wireObjectCheck: true
# ipv6: false
# unixDomainSocket:
# enabled: true
# pathPrefix: "/tmp"
# filePermissions: 0700
# http:
# enabled: false
# JSONPEnabled: false
# RESTInterfaceEnabled: false
# ssl:
# sslOnNormalPorts: # deprecated since 2.6
# mode:
# PEMKeyFile:
# PEMKeyPassword:
# clusterFile:
# clusterPassword:
# CAFile:
# CRLFile:
# allowConnectionsWithoutCertificates:
# allowInvalidCertificates:
# allowInvalidHostnames: false
# FIPSMode:
#
#
########security Options
security:
keyFile: /data/mongodb/data/keyfile/security
clusterAuthMode: keyFile
authorization: enabled
#javascriptEnabled: true
########security.sasl Options
# sasl:
# hostName:
# serviceName:
# saslauthdSocketPath:
#
#
#########setParameter Option
setParameter:
enableLocalhostAuthBypass: true
# :
# :
#
#
#########storage Options
storage:
dbPath: "/data/mongodb/data/conf"
# indexBuildRetry: true
# repairPath: "/data/db/_tmp"
# journal:
# enabled: true
# directoryPerDB: false
# syncPeriodSecs: 60
engine: "wiredTiger" #Valid options include mmapv1 and wiredTiger.
#########storage.mmapv1 Options
# mmapv1:
# preallocDataFiles: true
# nsSize: 16
# quota:
# enforced: false
# maxFilesPerDB: 8
# smallFiles: false
# journal:
# debugFlags:
# commitIntervalMs: 100 # 100 or 30
#########storage.wiredTiger Options
# wiredTiger:
# engineConfig:
# cacheSizeGB: #Default: the maximum of half of physical RAM or 1 gigabyte
# statisticsLogDelaySecs: 0
# journalCompressor: "snappy"
# directoryForIndexes: false
# collectionConfig:
# blockCompressor: "snappy"
# indexConfig:
# prefixCompression: true
#
#
##########operationProfiling Options
#operationProfiling:
# slowOpThresholdMs: 100
# mode
#########replication Options
replication:
# oplogSizeMB:
replSetName: cfgsvr
# secondaryIndexPrefetch: all
#
#
##########sharding Options
sharding:
clusterRole: configsvr #configsvr or shardsvr
# archiveMovedChunks: True
#
#
#########auditLog Options
#auditLog:
# destination: #syslog/console/file
# format: #JSON/BSON
# path:
# filter:
#
#
#########snmp Options
#snmp:
# subagent:
# master:
#
#
########mongos-only Options
#replication:
# localPingThresholdMs: 15
#
#sharding:
# autoSplit: true
# configDB: 10.50.16.34:27017,10.50.16.35:27017,10.50.16.36:27017,10.50.16.18:27017
# chunkSize: 64
#
#
########Windows Service Options
#processManagement:
# windowsService:
# serviceName:
# displayName:
# description:
# serviceUser:
# servicePassword:
6.2 启动三台服务器的 config server：
/usr/local/mongodb3.4.6/bin/mongod -f /usr/local/mongodb3.4.6/conf/config.conf
ps -ef |grep mongo
netstat -ntpl | grep 270206.3 登录任意一台配置服务器，初始化配置副本集：
/usr/local/mongodb3.4.6/bin/mongo --port 27020
config = {
_id: "cfgsvr",
members: [{
_id: 0,
host: "10.10.10.8:27020"
}, {
_id: 1,
host: "10.10.10.3:27020"
}, {
_id: 2,
host: "10.10.10.16:27020"
}]
}
rs.initiate(config)Tips：任意一台config server的配置文件中 setParameter.enableLocalhostAuthBypass属性要为true，该属性的意思是本地登录可以跳过登录验证。
7.配置分片副本集(三台机器):
7.1 Shard1配置文件：
#此处为配置文件可配置的内容
#Mongod config file
#MongoDB configuration files use the YAML format.
#The following example configuration file contains several mongod settings.
#
########Example Start########
#systemLog:
# destination: file
# path: "/var/log/mongodb/mongodb.log"
# logAppend: true
#storage:
# journal:
# enabled: true
#processManagement:
# fork: true
#net:
# bindIp: 127.0.0.1
# port: 27017
#setParameter:
# enableLocalhostAuthBypass: false
#
########Example End########
#
########Core Options
systemLog:
# verbosity: 0 #Default: 0; 1 to 5 increases the verbosity level to include Debug messages.
# quiet:
# traceAllException:
# syslogFacility: user
path: "/data/mongodb/data/shard1/mongodb.log"
logAppend: true
# logRotate: #rename or reopen
destination: file
# timeStampFormat: iso8601-local
# component:
# accessControl:
# verbosity: 0
# command:
# verbosity: 0
# # COMMENT additional component verbosity settings omitted for brevity
# storage:
# verbosity: 0
# journal:
# verbosity:
# write:
# verbosity: 0
#
#
########ProcessManagement Options
processManagement:
fork: true
pidFilePath: "/data/mongodb/data/shard1/mongod.pid"
#
#
#########Net Options
net:
port: 27011
bindIp: 0.0.0.0 #Default All interfaces.
# maxIncomingConnections: 65536
# wireObjectCheck: true
# ipv6: false
# unixDomainSocket:
# enabled: true
# pathPrefix: "/tmp"
# filePermissions: 0700
# http:
# enabled: false
# JSONPEnabled: false
# RESTInterfaceEnabled: false
# ssl:
# sslOnNormalPorts: # deprecated since 2.6
# mode:
# PEMKeyFile:
# PEMKeyPassword:
# clusterFile:
# clusterPassword:
# CAFile:
# CRLFile:
# allowConnectionsWithoutCertificates:
# allowInvalidCertificates:
# allowInvalidHostnames: false
# FIPSMode:
#
#
########security Options
security:
keyFile: /data/mongodb/data/keyfile/security
clusterAuthMode: keyFile
authorization: enabled
#javascriptEnabled: true
########security.sasl Options
# sasl:
# hostName:
# serviceName:
# saslauthdSocketPath:
#
#
#########setParameter Option
setParameter:
enableLocalhostAuthBypass: true
# :
# :
#
#
#########storage Options
storage:
dbPath: "/data/mongodb/data/shard1"
# indexBuildRetry: true
# repairPath: "/data/db/_tmp"
# journal:
# enabled: true
# directoryPerDB: false
# syncPeriodSecs: 60
engine: "wiredTiger" #Valid options include mmapv1 and wiredTiger.
#########storage.mmapv1 Options
# mmapv1:
# preallocDataFiles: true
# nsSize: 16
# quota:
# enforced: false
# maxFilesPerDB: 8
# smallFiles: false
# journal:
# debugFlags:
# commitIntervalMs: 100 # 100 or 30
#########storage.wiredTiger Options
wiredTiger:
engineConfig:
cacheSizeGB: 12 #Default: the maximum of half of physical RAM or 1 gigabyte
# statisticsLogDelaySecs: 0
# journalCompressor: "snappy"
# directoryForIndexes: false
# collectionConfig:
# blockCompressor: "snappy"
# indexConfig:
# prefixCompression: true
#
#
##########operationProfiling Options
#operationProfiling:
# slowOpThresholdMs: 100
# mode: "off"
#
#
##########replication Options
replication:
# oplogSizeMB:
replSetName: shard1
# secondaryIndexPrefetch: all
#
#
##########sharding Options
sharding:
clusterRole: shardsvr #configsvr or shardsvr
# archiveMovedChunks: True
#
#
#########auditLog Options
#auditLog:
# destination: #syslog/console/file
# format: #JSON/BSON
# path:
# filter:
#
#
#########snmp Options
#snmp:
# subagent:
# master:
#
#
########mongos-only Options
#replication:
# localPingThresholdMs: 15
#
#sharding:
# autoSplit: true
# configDB:
# chunkSize: 64
#
#
########Windows Service Options
#processManagement:
# windowsService:
# serviceName:
# displayName:
# description:
# serviceUser:
# servicePassword: 7.2 设置第一个分片副本集：
/usr/local/mongodb3.4.6/bin/mongod -f /usr/local/mongodb3.4.6/conf/shard1.conf
ps -ef |grep mongo
netstat -ntpl | grep 27011
/usr/local/mongodb3.4.6/bin/mongo --port 27011
config = {
_id: "shard1",
members: [{
_id: 0,
host: "10.10.10.8:27011",
priority: 1
}, {
_id: 1,
host: "10.10.10.3:27011"
}, {
_id: 2,
host: "10.10.10.16:27011",
arbiterOnly: true
}]
}
rs.initiate(config)7.3 设置第二个分片副本集：
/usr/local/mongodb3.4.6/bin/mongod -f /usr/local/mongodb3.4.6/conf/shard2.conf
ps -ef |grep mongo
netstat -ntpl | grep 27012
/usr/local/mongodb3.4.6/bin/mongo --port 27012
config = {
_id: "shard2",
members: [{
_id: 0,
host: "10.10.10.8:27012"
}, {
_id: 1,
host: "10.10.10.3:27012",
arbiterOnly: true
}, {
_id: 2,
host: "10.10.10.16:27012",
priority: 1
}]
}
rs.initiate(config)7.4 设置第三个分片副本集：
/usr/local/mongodb3.4.6/bin/mongod -f /usr/local/mongodb3.4.6/conf/shard3.conf
ps -ef |grep mongo
netstat -ntpl | grep 27013
/usr/local/mongodb3.4.6/bin/mongo --port 27013
config = {
_id: "shard3",
members: [{
_id: 0,
host: "10.10.10.16:27013"
}, {
_id: 1,
host: "10.10.10.3:27013",
priority: 1
}, {
_id: 2,
host: "10.10.10.8:27013",
arbiterOnly: true
}]
}
rs.initiate(config)Tips：不要去设置成仲裁节点的服务器写执行 。
8.配置路由服务器 mongos：
8.1 mongos配置文件：
#Mongod config file
#MongoDB configuration files use the YAML format.
#The following example configuration file contains several mongod settings.
#
########Example Start########
#systemLog:
# destination: file
# path: "/var/log/mongodb/mongodb.log"
# logAppend: true
#storage:
# journal:
# enabled: true
#processManagement:
# fork: true
#net:
# bindIp: 127.0.0.1
# port: 27017
#setParameter:
# enableLocalhostAuthBypass: false
#
########Example End########
#
########Core Options
systemLog:
# verbosity: 0 #Default: 0; 1 to 5 increases the verbosity level to include Debug messages.
# quiet:
# traceAllException:
# syslogFacility: user
path: "/data/mongodb/data/mongos/mongodb.log"
logAppend: true
# logRotate: #rename or reopen
destination: file
# timeStampFormat: iso8601-local
# component:
# accessControl:
# verbosity: 0
# command:
# verbosity: 0
# # COMMENT additional component verbosity settings omitted for brevity
# storage:
# verbosity: 0
# journal:
# verbosity:
# write:
# verbosity: 0
#
#
########ProcessManagement Options
processManagement:
fork: true
pidFilePath: "/data/mongodb/data/mongos/mongod.pid"
#
#
#########Net Options
net:
port: 27021
bindIp: 0.0.0.0 #Default All interfaces.
# maxIncomingConnections: 65536
# wireObjectCheck: true
# ipv6: false
# unixDomainSocket:
# enabled: true
# pathPrefix: "/tmp"
# filePermissions: 0700
# http:
# enabled: false
# JSONPEnabled: false
# RESTInterfaceEnabled: false
# ssl:
# sslOnNormalPorts: # deprecated since 2.6
# mode:
# PEMKeyFile:
# PEMKeyPassword:
# clusterFile:
# clusterPassword:
#CAFile:
#CRLFile:
# allowConnectionsWithoutCertificates:
# allowInvalidCertificates:
# allowInvalidHostnames: false
# FIPSMode:
#
#
########security Options
security:
keyFile: /data/mongodb/data/keyfile/security
#clusterAuthMode: keyFile
#authorization: enabled
#javascriptEnabled: true
########security.sasl Options
## sasl:
# hostName:
# serviceName:
# saslauthdSocketPath:
#
#
#########setParameter Option
setParameter:
enableLocalhostAuthBypass: true
# :
# :
#
#
#########storage Options
#storage:
# dbPath: "/data/mongo/data/mongos/"
# indexBuildRetry: true
# repairPath: "/data/db/_tmp"
# journal:
# enabled: true
# directoryPerDB: false
# syncPeriodSecs: 60
# engine: "wiredTiger" #Valid options include mmapv1 and wiredTiger.
#########storage.mmapv1 Options
# mmapv1:
# preallocDataFiles: true
# nsSize: 16
# quota:
# enforced: false
# maxFilesPerDB: 8
# smallFiles: false
# journal:
# debugFlags:
# commitIntervalMs: 100 # 100 or 30
#########storage.wiredTiger Options
# wiredTiger:
# engineConfig:
# cacheSizeGB: #Default: the maximum of half of physical RAM or 1 gigabyte
# statisticsLogDelaySecs: 0
# journalCompressor: "snappy"
# directoryForIndexes: false
# collectionConfig:
# blockCompressor: "snappy"
# indexConfig:
# prefixCompression: true
#
#
##########operationProfiling Options
#operationProfiling:
# slowOpThresholdMs: 100
# mode: "off"
#
#
##########replication Options
#replication:
# oplogSizeMB:
# replSetName: cfgsvr
# secondaryIndexPrefetch: all
#
#
##########sharding Options
#sharding:
# clusterRole: configsvr #configsvr or shardsvr
# archiveMovedChunks: True
#
#
#########auditLog Options
#auditLog:
# destination: #syslog/console/file
# format: #JSON/BSON
# path:
# filter:
#
#
#########snmp Options
#snmp:
# subagent:
# master:
#
#
########mongos-only Options
#replication:
# localPingThresholdMs: 15
#
sharding:
# autoSplit: true
configDB: cfgsvr/10.10.10.8:27020,10.10.10.16:27020,10.10.10.3:27020
# chunkSize: 1
#
#
########Windows Service Options
#processManagement:
# windowsService:
# serviceName:
# displayName:
# description:
# serviceUser:
# servicePassword:
8.2 启动mongos：
/usr/local/mongodb3.4.6/bin/mongos -f /usr/local/mongodb3.4.6/conf/mongos.conf9.启用分片：
# 登陆任意一台mongos
/usr/local/mongodb3.4.6/bin/mongo --port 27021
use admin
#串联路由服务器与分配副本集
sh.addShard("shard1/10.10.10.16:27011,10.10.10.3:27011,10.10.10.8:27011")
sh.addShard("shard2/10.10.10.16:27012,10.10.10.3:27012,10.10.10.8:27012")
sh.addShard("shard3/10.10.10.16:27013,10.10.10.3:27013,10.10.10.8:27013")
#查看集群状态（创建用户登录才可以看）
sh.status()10.创建用户:
use admin
db.createUser({
user:'admin',pwd:'admin',
roles:[
{role:'clusterAdmin',db:'admin'},
{role:'userAdminAnyDatabase',db:'admin'},
{role:'dbAdminAnyDatabase',db:'admin'},
{role:'readWriteAnyDatabase',db:'admin'}
]})
db.auth('admin', 'admin')
use test
db.createUser({
user:'test',pwd:'test',
roles:[
{role:'dbOwner',db:'test'}
]})
Tips：mongodb 授权数据库用户流程（权限认证）： https://blog.csdn.net/qq_36051316/article/details/92798568
11.指定分片生效：
use admin
db.auth('admin', 'admin')
db.runCommand( { enablesharding :"test"});
# 指定数据库里需要分片的集合和片键
db.runCommand( { shardcollection : "test.table1",key : {id: 1} } )Tips：必须在admin库设置
12.测试分片：
use test
db.auth('test', 'test')
for (var i = 1; i <= 100000; i++){ db.table1.save({id:i,"test1":"testval1"})};
db.table1.stats();