mongodb4.2.8单机分片集群搭建

官网地址：https://www.mongodb.com/

中文地址：https://www.mongodb.org.cn/

下载地址：https://www.mongodb.com/try/download/community

 

 

1，安装背景

由于原开发服务器mongodb数据库为单机安装，为了提高访问效率和数据安全性，需要配置成单机分片集群。

1.1 实验环境

1.1.1 分片集群架构官图

1.1.2 实验主机

主机	IP
Ubuntu 18.04.3 LTS	192.168.17.160

1.1.3 端口规划

服务器	端口
route1	27017
conf1	29010
conf2	29011
conf3	29012
shard1_primary	27010
shard1_secondary	27011
shard1_arbiter	27012
shard2_primary	28010
shard2_secondary	28011
shard2_arbiter	28012

1.2 安装包下载

cd /opt/mongodb
wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-4.2.8.tgz

#对文件夹做软连接，方便后期升级
tar xf mongodb-linux-x86_64-ubuntu1804-4.2.8.tgz
ln -sv mongodb-linux-x86_64-ubuntu1804-4.2.8 mongodb

1.3配置环境变量

# vim ~/.bashrc
在 /etc/profile 配置文件中添加 MongoDB 环境变量，内容如下：
export MONGODB_HOME=/opt/mongodb
export PATH=$MONGODB_HOME/bin:$PATH
# source ~/.bashrc

2,部署分片

• 配置分片数据Ⅰ

---

2.1端口规划

shard1:rs-1:27010
shard1:rs-1:27011
shard1:rs-1:27012

2.2 分片副本集路径规划创建

2.2.1 Pid文件路径

mkdir -p /var/run/mongodb27010
mkdir -p /var/run/mongodb27011
mkdir -p /var/run/mongodb27012
touch /var/run/mongodb2701{0,1,2}/mongod.pid

2.2.2 日志存储路径

mkdir -p /data/log/mongod27010
mkdir -p /data/log/mongod27011
mkdir -p /data/log/mongod27012
touch /data/log/mongod2701{0,1,2}/mongod.log

2.2.3 分片数据存储路径

mkdir -p /data/db/db27010
mkdir -p /data/db/db27011
mkdir -p /data/db/db27012

2.2.4 keyfile文件路径

mkdir -p /data/keyfile/autokey27010
mkdir -p /data/keyfile/autokey27011
mkdir -p /data/keyfile/autokey27012
touch /data/keyfile/autokey2701{0,1,2}/autokey

生成keyfile文件（所有的keyfile保持一致）：

openssl rand -base64 756 > /data/keyfile/autokey27010/autokey

cp /data/keyfile/autokey27010/autokey /data/keyfile/autokey27011/
cp /data/keyfile/autokey27010/autokey /data/keyfile/autokey27012/


chmod 400 /data/keyfile/autokey27010/autokey
chmod 400 /data/keyfile/autokey27011/autokey
chmod 400 /data/keyfile/autokey27012/autokey

2.4 分片副本集配置数据

配置文件路径：

mkdir -p /opt/mongos/conf2701{0,1,2}

2.4.1 rs-1-27010配置

• rs-1-27010yaml配置文件：

vi /opt/mongos/conf27010/shard27010.yaml
sharding:
    clusterRole: shardsvr
replication:
    replSetName: shard1
net:
    port: 27010
    bindIp: 0.0.0.0
systemLog:
    destination: file
    logAppend: true
    path: /data/log/mongod27010/mongod.log
storage:
    dbPath: /data/db/db27010
    journal:
        enabled: true
processManagement:
    fork: true
    pidFilePath: /var/run/mongodb27010/mongod.pid
    timeZoneInfo: /usr/share/zoneinfo
#security:
    #keyFile: /data/keyfile/autokey27010/autokey

2.4.2 rs-1-27011配置

• rs-1-27011yaml配置文件：

vi /opt/mongos/conf27011/shard27011.yaml

sharding:
    clusterRole: shardsvr
replication:
    replSetName: shard1
net:
    port: 27011
    bindIp: 0.0.0.0
systemLog:
    destination: file
    logAppend: true
    path: /data/log/mongod27011/mongod.log
storage:
    dbPath: /data/db/db27011
    journal:
        enabled: true
processManagement:
    fork: true
    pidFilePath: /var/run/mongodb27011/mongod.pid
    timeZoneInfo: /usr/share/zoneinfo
#security:
    #keyFile: /data/keyfile/autokey27011/autokey

2.4.3 rs-1-27012配置

• rs-1-27012yaml配置文件：

vi /opt/mongos/conf27012/shard27012.yaml

sharding:
    clusterRole: shardsvr
replication:
    replSetName: shard1
net:
    port: 27012
    bindIp: 0.0.0.0
systemLog:
    destination: file
    logAppend: true
    path: /data/log/mongod27012/mongod.log
storage:
    dbPath: /data/db/db27012
    journal:
        enabled: true
processManagement:
    fork: true
    pidFilePath: /var/run/mongodb27012/mongod.pid
    timeZoneInfo: /usr/share/zoneinfo
#security:
    #keyFile: /data/keyfile/autokey27012/autokey

• 配置分片数据Ⅱ

---

2.5端口规划

shard2:rs-2:28010
shard2:rs-2:28011
shard2:rs-2:28012

2.6 分片副本集路径规划创建

2.6.1 Pid文件路径

mkdir -p /var/run/mongodb28010
mkdir -p /var/run/mongodb28011
mkdir -p /var/run/mongodb28012
touch /var/run/mongodb2801{0,1,2}/mongod.pid

2.6.2 日志存储路径

mkdir -p /data/log/mongod28010
mkdir -p /data/log/mongod28011
mkdir -p /data/log/mongod28012
touch /data/log/mongod2801{0,1,2}/mongod.log

2.6.3 分片数据存储路径

mkdir -p /data/db/db28010
mkdir -p /data/db/db28011
mkdir -p /data/db/db28012

2.6.4 keyfile文件路径

mkdir -p /data/keyfile/autokey28010
mkdir -p /data/keyfile/autokey28011
mkdir -p /data/keyfile/autokey28012
touch /data/keyfile/autokey2801{0,1,2}/autokey

生成keyfile文件：

openssl rand -base64 756 > /data/keyfile/autokey28010/autokey

cp /data/keyfile/autokey28010/autokey /data/keyfile/autokey28011/autokey
cp /data/keyfile/autokey28010/autokey /data/keyfile/autokey28012/autokey

chmod 400 /data/keyfile/autokey28010/autokey
chmod 400 /data/keyfile/autokey28011/autokey
chmod 400 /data/keyfile/autokey28012/autokey

2.7分片副本集配置数据

配置文件路径：

mkdir -p /opt/mongos/conf2801{0,1,2}

2.7.1 rs-2-28010配置

• rs-2-28010yaml配置文件：

vi /opt/mongos/conf28010/shard28010.yaml

sharding:
    clusterRole: shardsvr
replication:
    replSetName: shard2
net:
    port: 28010
    bindIp: 0.0.0.0
systemLog:
    destination: file
    logAppend: true
    path: /data/log/mongod28010/mongod.log
storage:
    dbPath: /data/db/db28010
    journal:
        enabled: true
processManagement:
    fork: true
    pidFilePath: /var/run/mongodb28010/mongod.pid
    timeZoneInfo: /usr/share/zoneinfo
#security:
   #keyFile: /data/keyfile/autokey28010/autokey

2.7.2 rs-2-28011配置

• rs-2-28011yaml配置文件：

vi /opt/mongos/conf28011/shard28011.yaml

sharding:
    clusterRole: shardsvr
replication:
    replSetName: shard2
net:
    port: 28011
    bindIp: 0.0.0.0
systemLog:
    destination: file
    logAppend: true
    path: /data/log/mongod28011/mongod.log
storage:
    dbPath: /data/db/db28011
    journal:
        enabled: true
processManagement:
    fork: true
    pidFilePath: /var/run/mongodb28011/mongod.pid
    timeZoneInfo: /usr/share/zoneinfo
#security:
   #keyFile: /data/keyfile/autokey28011/autokey

2.7.3 rs-2-28012配置

• rs-2-28012yaml配置文件：

vi /opt/mongos/conf28012/shard28012.yaml

sharding:
    clusterRole: shardsvr
replication:
    replSetName: shard2
net:
    port: 28012
    bindIp: 0.0.0.0
systemLog:
    destination: file
    logAppend: true
    path: /data/log/mongod28012/mongod.log
storage:
    dbPath: /data/db/db28012
    journal:
        enabled: true
processManagement:
    fork: true
    pidFilePath: /var/run/mongodb28012/mongod.pid
    timeZoneInfo: /usr/share/zoneinfo
#security:
   #keyFile: /data/keyfile/autokey28012/autokey

---

3，config配置服务器配置

3.1 端口及文件路径规划

3.1.1 Pid文件路径

mkdir -p /var/run/config/mongodb29010
mkdir -p /var/run/config/mongodb29011
mkdir -p /var/run/config/mongodb29012
touch /var/run/config/mongodb2901{0,1,2}/mongod.pid

3.1.2 日志存储路径

mkdir -p /data/config/log/mongod29010
mkdir -p /data/config/log/mongod29011
mkdir -p /data/config/log/mongod29012
mkdir -p /data/config/log/mongod2901{0,1,2}/mongod.log

3.1.3 分片数据存储路径

mkdir -p /data/config/db/db29010
mkdir -p /data/config/db/db29011
mkdir -p /data/config/db/db29012

3.1.4 keyfile文件路径

mkdir -p /data/config/keyfile/autokey29010
mkdir -p /data/config/keyfile/autokey29011
mkdir -p /data/config/keyfile/autokey29012
touch /data/config/keyfile/autokey2901{0,1,2}/autokey

生成keyfile文件：

openssl rand -base64 756 > /data/config/keyfile/autokey29010/autokey

cp /data/config/keyfile/autokey29010/autokey /data/config/keyfile/autokey29011/autokey
cp /data/config/keyfile/autokey29010/autokey /data/config/keyfile/autokey29012/autokey


chmod 400 /data/config/keyfile/autokey29010/autokey
chmod 400 /data/config/keyfile/autokey29011/autokey
chmod 400 /data/config/keyfile/autokey29012/autokey

3.2 config配置文件

配置文件路径:

mkdir -p /opt/configsvr/conf2901{0,1,2}

3.2.1 config29010配置

29010-yaml配置文件： vi /opt/configsvr/conf29010/config29010.yaml

sharding:
  clusterRole: configsvr  
replication:
  replSetName: config  	 
net:
  port: 29010              
  bindIp: 0.0.0.0        
systemLog:
  destination: file        
  logAppend: true		   
  path: /data/config/log/mongod29010/mongod.log    
storage:
  dbPath: /data/config/db/db29010              
  journal:
    enabled: true           
processManagement:
  fork: true               
  pidFilePath: /var/run/config/mongodb29010/mongod.pid
  timeZoneInfo: /usr/share/zoneinfo    
#security:
  #keyFile: /data/config/keyfile/autokey29010/autokey    

3.2.2 config29011配置

29011-yaml配置文件： vi /opt/configsvr/conf29011/config29011.yaml

sharding:
  clusterRole: configsvr  
replication:
  replSetName: config  	 
net:
  port: 29011              
  bindIp: 0.0.0.0        
systemLog:
  destination: file        
  logAppend: true		   
  path: /data/config/log/mongod29011/mongod.log    
storage:
  dbPath: /data/config/db/db29011              
  journal:
    enabled: true           
processManagement:
  fork: true               
  pidFilePath: /var/run/config/mongodb29011/mongod.pid
  timeZoneInfo: /usr/share/zoneinfo    
#security:
  #keyFile: /data/config/keyfile/autokey29011/autokey    

3.2.3 config29012配置

29012-yaml配置文件： vi /opt/configsvr/conf29012/config29012.yaml

sharding:
  clusterRole: configsvr  
replication:
  replSetName: config  	 
net:
  port: 29012              
  bindIp: 0.0.0.0        
systemLog:
  destination: file        
  logAppend: true		   
  path: /data/config/log/mongod29012/mongod.log    
storage:
  dbPath: /data/config/db/db29012              
  journal:
    enabled: true           
processManagement:
  fork: true               
  pidFilePath: /var/run/config/mongodb29012/mongod.pid
  timeZoneInfo: /usr/share/zoneinfo    
#security:
  #keyFile: /data/config/keyfile/autokey29012/autokey    

---

4,mongos路由配置

4.1端口及文件路径配置

4.1.1 日志存储路径

mkdir -p /data/mongos/log/
touch route.log

4.1.1 keyfile

mkdir -p /data/mongos/keyfile/
touch autokey

写入autokey(可以直接拷贝上面生成文件，全局key保持一致):

openssl rand -base64 756 > /data/mongos/keyfile/autokey

chmod 400 /data/mongos/keyfile/autokey

4.1.2 pid文件路径

mkdir -p /data/mongos/pidfile
touch /data/mongos/pidfile/mongos.pid

4.2配置文件

mongos yaml配置文件：

vi /opt/mongoroute/mongoroute.yaml

sharding:
    configDB: config/192.168.17.160:29010,192.168.17.160:29011,192.168.17.160:29012
net:
    port: 27017
    bindIp: 0.0.0.0

systemLog:
    destination: file
    logAppend: true
    path: /data/mongos/log/route.log

processManagement:
    fork: true
    pidFilePath: /data/mongos/pidfile/mongos.pid
    timeZoneInfo: /usr/share/zoneinfo

security:
  keyFile: /data/mongos/keyfile/autokey

---

5,配置服务器、分片配置启动

5.1 启动所有分片副本集（rs-1,rs-2）和配置服务器(config)和mongos路由

5.1.1先启动配置服务器（config），进行初始化配置：

• 先将yaml配置文件中security部分注释掉，然后启动：

mongod -f /opt/configsvr/conf29010/config29010.yaml

mongod -f /opt/configsvr/conf29011/config29011.yaml

mongod -f /opt/configsvr/conf29012/config29012.yaml

• 登入，进行初始化：

注意 _id:副本集名称要和config.yaml中的replSetName配置名称相同; 只需初始化一次即可

mongo --port 29010

rs.initiate(
			  {
				_id: "config",
				configsvr: true,
				members: [
				  { _id : 0, host : "192.168.17.160:29010" },
				  { _id : 1, host : "192.168.17.160:29011" },
				  { _id : 2, host : "192.168.17.160:29012" }
				]
			  }
			)

• 查看集群状态:

rs.status();

• 为conf集群添加root用户:

1.这里必须要连接到primary节点下创建用户,在secondary节点上创建用户会报错

2,连接上primary节点后,一定要切换到admin数据库下创建root用户:

use admin;
db.createUser(
			  {
				user: "root",
				pwd: "root",
				roles: [ { role: "root", db: "admin" } ]
			  }
			);

• kill掉mongod服务进程,把config.yaml配置文件中的security认证注释打开,重启mongod服务;

 kill `ps -ef|grep mongo|awk {'print $2'}`
 
mongod -f /opt/configsvr/conf29010/config29010.yaml

mongod -f /opt/configsvr/conf29011/config29011.yaml

mongod -f /opt/configsvr/conf29012/config29012.yaml

mongo --port 29010      #连接到mongod服务器
use admin               #切换到admin数据库
db.auth('root','root')  #登录 返回1说明登录成功

登陆后：
show dbs;
rs.status()

输出如下：

config:PRIMARY> show dbs;
admin   0.000GB
config  0.000GB
local   0.000GB

5.1.2 启动分片副本集Ⅰ

• 使用yaml文件启动，先注释掉sercurity认证：

mongod -f /opt/mongos/conf27010/shard27010.yaml
mongod -f /opt/mongos/conf27011/shard27011.yaml
mongod -f /opt/mongos/conf27012/shard27012.yaml

• 初始化(一主一从一仲裁）：

登陆：
mongo --port 27010

注意 _id:副本集名称要和shard.yaml中的replSetName配置相同;

rs.initiate(
				  {
					_id : "shard1",
					members: [
					  { _id : 0, host : "192.168.17.160:27010" ,priority : 2 },
					  { _id : 1, host : "192.168.17.160:27011" ,priority : 1 },
					  { _id : 2, host : "192.168.17.160:27012" ,arbiterOnly :true }
					]
				  }
				)

• 查看状态：

rs.status();

• 为share1添加root用户(必须在PRIMARY状态下，可以按回车键可以看到PRIMARY）：

use admin;
db.createUser(
			  {
				user: "root",
				pwd: "root",
				roles: [ { role: "root", db: "admin" } ]
			  }
			);

• kill掉mongod服务进程,把shard1.yaml配置文件中的security认证注释打开,重启mongod服务：

kill `ps -ef |grep 'mongod -f /opt/mongos/conf270'|awk {'print $2'}`

mongod -f /opt/mongos/conf27010/shard27010.yaml
mongod -f /opt/mongos/conf27011/shard27011.yaml
mongod -f /opt/mongos/conf27012/shard27012.yaml

连接后登陆：
mongo --port 27010     #连接到mongod服务器（primary）
use admin               #切换到admin数据库
db.auth('root','root')  #登录 返回1说明登录成功

• 验证：

登录后验证:
show dbs; 
rs.status()

rs-1分片集搭建完成

5.1.3 启动分片副本集Ⅱ

• 使用yaml文件启动，先注释掉sercurity认证：

mongod -f /opt/mongos/conf28010/shard28010.yaml
mongod -f /opt/mongos/conf28011/shard28011.yaml
mongod -f /opt/mongos/conf28012/shard28012.yaml

• 初始化(一主一从一仲裁）：

登陆：
mongo --port 28010

注意 _id:副本集名称要和shard.yaml中的replSetName配置相同;
rs.initiate(
				  {
					_id : "shard2",
					members: [
					  { _id : 0, host : "192.168.17.160:28010" ,priority : 2 },
					  { _id : 1, host : "192.168.17.160:28011" ,priority : 1 },
					  { _id : 2, host : "192.168.17.160:28012" ,arbiterOnly :true }
					]
				  }
				)

• 查看状态：

rs.status();

• 为share2添加root用户(必须在PRIMARY状态下）：

use admin;
db.createUser(
			  {
				user: "root",
				pwd: "root",
				roles: [ { role: "root", db: "admin" } ]
			  }
			);

• kill掉mongod服务进程,把shard2.yaml配置文件中的security认证注释打开,重启mongod服务：

kill `ps -ef |grep 'mongod -f /opt/mongos/conf280'|awk '{print $2}'`

mongod -f /opt/mongos/conf28010/shard28010.yaml
mongod -f /opt/mongos/conf28011/shard28011.yaml
mongod -f /opt/mongos/conf28012/shard28012.yaml

连接后登陆：
mongo --port 28010     #连接到mongod服务器（primary）
use admin               #切换到admin数据库
db.auth('root','root')  #登录 返回1说明登录成功

• 验证：

登录后验证:
show dbs; 
rs.status()

rs-2分片集搭建完成

5.1.4 启动route路由集群，并整合conf集群和shard集群

• 启动route:

启动route：
mongos -f /opt/mongoroute/mongoroute.yaml

注意:这里启动的时候和config集群shard集群不同,这里的security认证要打开,不然会报错

• 连接到mongo路由： 这里的登录密码就是config集群配置的用户名和密码;端口为定义的27017，可以不指定，使用root账户登陆：

mongo;
use admin;
db.auth('root','root');
查看登陆是否成功

5.1.5 添加shard分片集群

把shard1、shard2分片集群都添加进去,每个分片添加一个节点即可,主 从都可以;

use admin;
sh.addShard( "shard1/192.168.17.160:27010");
sh.addShard( "shard2/192.168.17.160:28010");

输出如下：

root@host160:/data/db/mongod# mongo
MongoDB shell version v4.2.8
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("4a04867e-3c0f-4d92-8cfe-92c9e947833d") }
MongoDB server version: 4.2.8
mongos> use admin
switched to db admin
mongos> db.auth('root','root')
1
mongos> use admin
switched to db admin
mongos> sh.addShard( "shard1/192.168.17.160:27010");
{
	"shardAdded" : "shard1",
	"ok" : 1,
	"operationTime" : Timestamp(1592877175, 4),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1592877175, 4),
		"signature" : {
			"hash" : BinData(0,"Fth3AMDH5O+bGhqZypLH47IN9g0="),
			"keyId" : NumberLong("6841344940694306847")
		}
	}
}
mongos> sh.addShard( "shard2/192.168.17.160:28010");
{
	"shardAdded" : "shard2",
	"ok" : 1,
	"operationTime" : Timestamp(1592877191, 4),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1592877191, 4),
		"signature" : {
			"hash" : BinData(0,"94Eay4P619ftB8wz6JBWoPJj3F8="),
			"keyId" : NumberLong("6841344940694306847")
		}
	}
}

查看集群状态：

mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
  	"_id" : 1,
  	"minCompatibleVersion" : 5,
  	"currentVersion" : 6,
  	"clusterId" : ObjectId("5ef156fa69d9c7953fecb0b0")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/192.168.17.160:27010,192.168.17.160:27011",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/192.168.17.160:28010,192.168.17.160:28011",  "state" : 1 }
  active mongoses:
        "4.2.8" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours: 
                101 : Success
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard1	923
                                shard2	101
                        too many chunks to print, use verbose if you want to force print

至此一个带有权限控制的可水平扩容的分片mongodb4.2.8版本集群搭建完成;

6，将原mongo单机配置文件导出，导入该集群

6.1 mongodb4.2版本单机数据备份

导出全部数据：
mongodump --host=127.0.0.1 --port=37017 -o /home/mongodback/

导入全部数据：
mongorestore --dir /data/mongoback/mongodback/ -u root -p root --authenticationDatabase admin

6.1.1开启均衡器和分片功能

• 开启均衡器：

sh.startBalancer()，显示

mongos> sh.startBalancer()
{
	"ok" : 1,
	"operationTime" : Timestamp(1592882192, 3),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1592882192, 3),
		"signature" : {
			"hash" : BinData(0,"Irm0/RdXjYrJFzPNicDZKZiZy3w="),
			"keyId" : NumberLong("6841344940694306847")
		}
	}
}

查看结果：
mongos> sh.status()

• 开启文档分片功能(对导入的文档)

mongos> use admin
switched to db admin

mongos> show dbs

mongos> db.runCommand({enablesharding:"test"})

mongos> db.runCommand( { shardCollection : "test.fs.chunks" , key : { files_id : 1 , n : 1 } } )