MongoDB分片群集由组件
shard key(分片键)
Sharding的优势
分片集群搭建前注意事项
组件名 | 地址和端口 |
---|---|
config servers | 192.168.137.137:7017 192.168.137.137:7018 192.168.137.137:7019 |
mongos | 192.168.137.137:2000 |
shard1 | 192.168.137.137:17017 192.168.137.137:17018 192.168.137.137:17019 |
shard2 | 192.168.137.137:27017 192.168.137.137:27018 192.168.137.137:27019 |
shard3 | 192.168.137.137:37017 192.168.137.137:37018 192.168.137.137:37019 |
[zzq@localhost software]$ tree mongo-conf/
mongo-conf/
├── config
│ ├── config1.conf
│ ├── config2.conf
│ └── config3.conf
├── mongodb-keyfile
├── mongos
│ └── mongos1.conf
├── shard1
│ ├── shard1.conf
│ ├── shard2.conf
│ └── shard3.conf
├── shard2
│ ├── shard1.conf
│ ├── shard2.conf
│ └── shard3.conf
└── shard3
├── shard1.conf
├── shard2.conf
└── shard3.conf
[zzq@localhost software]$ tree mongo-database/
mongo-database/
├── config
│ ├── config1
│ ├── config2
│ └── config3
├── mongos
│ └── mongos1
├── shard1
│ ├── shard1
│ ├── shard2
│ └── shard3
├── shard2
│ ├── shard1
│ ├── shard2
│ └── shard3
└── shard3
├── shard1
├── shard2
└── shard3
firewall-cmd --permanent --zone=public --add-port=xxx/tcp
firewall-cmd --reload
tar -zxvf mongodb-linux-x86_64-rhel70-4.0.13.tgz
mv mongodb-linux-x86_64-rhel70-4.0.13 mongodb
vim /etc/profile
#在/etc/profile后面追加,配置环境变量
export MONGO_HOME=/home/zzq/software/mongodb
export PATH=$PATH:$MONGO_HOME/bin
#使环境变量生效
source /etc/profile
openssl rand -base64 741 > ~/software/mongo-conf/mongodb-keyfile
chmod 600 mongodb-keyfile
否则可能启动报permissions on /home/zzq/software/mongo-conf/mongodb-keyfile are too open
TDaOROls3Af/glbKNz6cli78KVvpUYffuMaurOOcSEb4LThra+aTnOz+igsjeLIR
zibdla9dZmWcmjyw9ssm4EUOrsbcgWPBGCkBdInCqj8RDenlspEU5Lh7+KVJl7rb
qiDx/BP1N0BJvhHiRrUhZDreCnoEvFJUOVqRxncKki5PM8GmswHxg+QMu+yrr0Lr
pJJcKZmAG0wzSdCdfwWDrNAbY/0eNWIz8WMh47xdHvUPrj/TMq8pRlV92h4tjUTn
ywh0E7DWJH7MqXLVrgwujwwiMJBSh/1QdarfW+SuygFrN4bc9YibM1AVYNFGKsur
tjXhlwTeQvY9MRhjdrRUwtwRUxOvXzKbk1ZlcOeL4KD/+6+CI0jBo8CsrGjaPZwn
SNaCpYHQ58Hw567upSC2uut49Hgi3W82/np5+4PiMRLm3cGuEs3y6sQbYUR+O8RR
RowAzo8evJJqlYVbgtDUSQm2Fk4SwcyFN160RKlaimru/n3cEPE1aK/cg12sHYvi
6bBhM4CnZGX+kGNVXrqNd4/PQNG6f69+wZgRXowwa0XNWlvY8IwCOe+cTuVYAwyq
cq2IKn46yZwfm3CiVOjKKd3SJdF9GdMOfWz4ghj/bBjTFeYlFraWuL8SHPi0LQbw
/9dX26H0NUykuU2qM8+8lqi6ixt6xF4MT1B1Wa5m/M6vYpIk2hlyWCRDN6ycnoG8
/asjlMCWp/qEHmNwti5jpNccxe4dJBQVxnmMM16H+t4E3CIAjMQpQCZSO/kpXD6a
EoQ+UulN6Y6/i30k10s1rGkPCotvJa9g4RrLDsC0zxTJMhyzY+/3w8nL8KlTTxOb
rnAPvGVYjQ+4oQ3iTQXjxmL0ziUvovjL1ryBm+6LEEPTSNkc+3mw4LWiqUNkq3nR
j9ascnJd55nborTQp52aDlvb8hklkpBHoDkcArCspfTZWBKeUwJkYZkQP/PYbpA0
Lhn9kpVmUZtXcrMdRseMdpOi/4mh
vim mongo-conf/config/config1.conf
sharding:
clusterRole: configsvr
replication:
replSetName: confset
net:
port: 7017
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/config/config1/mongo.log
storage:
dbPath: /home/zzq/software/mongo-database/config/config1
journal:
enabled: true
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/config/config1/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
vim mongo-conf/config/config2.conf
sharding:
clusterRole: configsvr
replication:
replSetName: confset
net:
port: 7018
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/config/config2/mongo.log
storage:
dbPath: /home/zzq/software/mongo-database/config/config2
journal:
enabled: true
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/config/config2/mongod.pid # location of pidfile
itimeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
vim mongo-conf/config/config3.conf
sharding:
clusterRole: configsvr
replication:
replSetName: confset
net:
port: 7019
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/config/config3/mongo.log
storage:
dbPath: /home/zzq/software/mongo-database/config/config3
journal:
enabled: true
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/config/config3/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
mongod --config xxxx.conf
或者mongod -f xxxx.conf
mongod --config /home/zzq/software/mongo-conf/config/config1.conf
mongod --config /home/zzq/software/mongo-conf/config/config2.conf
mongod --config /home/zzq/software/mongo-conf/config/config3.conf
mongo --port 7017
rs.initiate(
{
_id: "confset",
configsvr: true,
members: [
{ _id : 0, host : "192.168.137.137:7017" },
{ _id : 1, host : "192.168.137.137:7018" },
{ _id : 2, host : "192.168.137.137:7019" }
]
}
)
vim mongo-conf/shard1/shard1.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard1
net:
port: 17017
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard1/shard1/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard1/shard1
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard1/shard1/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
此时clusterRole:要改成shardsvr,这样才能和config节点,mongos节点一起组成分片集群。replSetName可以随便填一个,但是对于分片一和它的几个副本都应该是相同的名字。分片一和分片二两个复制集的replSetName一般为不一样的,和config节点的集群也应该不一样。这里要注意,repSetName要和Master节点填一样的。
shard1->shard2 vim mongo-conf/shard1/shard2.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard1
net:
port: 17018
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard1/shard2/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard1/shard2
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard1/shard2/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
vim mongo-conf/shard1/shard3.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard1
net:
port: 17019
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard1/shard3/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard1/shard3
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard1/shard3/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
mongod
启动mongod --config /home/zzq/software/mongo-conf/shard1/shard1.conf
mongod --config /home/zzq/software/mongo-conf/shard1/shard2.conf
mongod --config /home/zzq/software/mongo-conf/shard1/shard3.conf
mongo --port 17017
rs.initiate(
{
_id : "shard1",
members: [
{ _id : 0, host : "192.168.137.137:17017" },
{ _id : 1, host : "192.168.137.137:17018" },
{ _id : 2, host : "192.168.137.137:17019" }
]
}
)
或者使用命令(二选一)
rs.initiate(
{
_id : "shard1",
members: [
{ _id : 0, host : "192.168.137.137:17019" ,priority : 2 },
{ _id : 1, host : "192.168.137.137:17018" ,priority : 1 },
{ _id : 2, host : "192.168.137.137:17017" ,arbiterOnly :true }
]
}
)
vim mongo-conf/shard2/shard1.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard2
net:
port: 27017
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard2/shard1/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard2/shard1
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard2/shard1/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
vim mongo-conf/shard2/shard2.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard2
net:
port: 27018
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard2/shard2/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard2/shard2
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard2/shard2/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
vim mongo-conf/shard2/shard3.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard2
net:
port: 27019
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard2/shard3/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard2/shard3
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard2/shard3/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
mongod --config /home/zzq/software/mongo-conf/shard2/shard1.conf
mongod --config /home/zzq/software/mongo-conf/shard2/shard2.conf
mongod --config /home/zzq/software/mongo-conf/shard2/shard3.conf
mongo --port 27017
rs.initiate(
{
_id : "shard2",
members: [
{ _id : 0, host : "192.168.137.137:27017" },
{ _id : 1, host : "192.168.137.137:27018" },
{ _id : 2, host : "192.168.137.137:27019" }
]
}
)
vim mongo-conf/shard3/shard1.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard3
net:
port: 37017
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard3/shard1/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard3/shard1
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard2/shard3/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
vim mongo-conf/shard3/shard2.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard3
net:
port: 37018
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard3/shard2/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard3/shard2
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard3/shard2/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
vim mongo-conf/shard3/shard3.conf
sharding:
clusterRole: shardsvr
replication:
replSetName: shard3
net:
port: 37019
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/shard3/shard3/mongod.log
storage:
dbPath: /home/zzq/software/mongo-database/shard3/shard3
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/shard3/shard3/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
authorization: enabled
mongod --config /home/zzq/software/mongo-conf/shard3/shard1.conf
mongod --config /home/zzq/software/mongo-conf/shard3/shard2.conf
mongod --config /home/zzq/software/mongo-conf/shard3/shard3.conf
mongo --port 37017
rs.initiate(
{
_id : "shard3",
members: [
{ _id : 0, host : "192.168.137.137:37017" },
{ _id : 1, host : "192.168.137.137:37018" },
{ _id : 2, host : "192.168.137.137:37019" }
]
}
)
PRIMARY
算是成功了vim mongo-conf/mongos/mongos1.conf
,我这里只配置了一个,可以配置多个。sharding:
configDB: confset/192.168.137.137:7017,192.168.137.137:7018,192.168.137.137:7019
net:
port: 2000
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: /home/zzq/software/mongo-database/mongos/mongos1/mongod.log
processManagement:
fork: true # fork and run in background
pidFilePath: /home/zzq/software/mongo-database/mongos/mongos1/mongod.pid # location of pidfile
security:
keyFile: /home/zzq/software/mongo-conf/mongodb-keyfile
注意configDB: confset这是config servers replSetName的值要一致。
启动mongos1
mongos -f /home/zzq/software/mongo-conf/mongos/mongos1.conf
mongo --port 2000
use admin
db.createUser(
{
user: "root",
pwd: "root",
roles: [ { role: "root", db: "admin" } ]
}
)
exit
退出来,重新登录进去,就能操作了。mongo --port 2000 -u root -p root --authenticationDatabase "admin"
show dbs
sh.addShard( "shard1/192.168.137.137:17017")
sh.addShard( "shard2/192.168.137.137:27017")
sh.addShard( "shard3/192.168.137.137:37017")
use admin
db.runCommand({listshards:1})
use admin
db.runCommand({enablesharding:"testdb"})
sh.shardCollection("testdb.t",{id:"hashed"})
use testdb
for(i=1;i<=1000;i++){db.t.insert({id:i,name:"Leo"})}
db.runCommand({shardcollection:"testdb.users" ,key:{id:1,name:1}})
或者
sh.shardCollection("testdb.users",{id:1,name:1})
#如果不是第一次创建的testdb.users表
db.users.ensureIndex({id:1})
use config
db.settings.save( { _id:"chunksize", value: 1 } )
db.settings.find()
use testdb
for(i=1;i<=100000;i++){db.users.insert({id:i,name:"Leo"+i})}
程序调用的话就是这样的格式spring.data.mongodb.uri=mongodb://root:[email protected]:2000/xxxx?authSource=admin&authMechanism=SCRAM-SHA-1
,这是java spring boot的方式,如果是多个mongos spring.data.mongodb.uri=mongodb://root:[email protected]:2000,192.168.137.137:2001/xxxx?authSource=admin&authMechanism=SCRAM-SHA-1
;逗号分隔即可。
异常
使用db.runCommand({listshards:1}) 先use admin
修改权限chmod 600 mongodb-keyfile
使用db.runCommand({enablesharding:“testdb”})先use admin
参考资料
最后如果文章有错误的地方,希望您能批评指出,感谢您的观看。