docker使用 docker-compose 部署MongoDB4.0 部署replica set(副本集)集群
环境准备
- 安装docker，安装docker-compose
- 生产环境最好是将不同的节点部署在不同的服务器上，但限于实际情况可能只能在单台服务器上部署，以下是将副本集部署在同一台服务器上。
生成keyFile
- MongoDB使用keyfile认证，副本集中的每个mongod实例使用keyfile内容作为认证其他成员的共享密码。mongod实例只有拥有正确的keyfile才可以加入副本集。
- keyFile的内容必须是6到1024个字符的长度，且副本集所有成员的keyFile内容必须相同。
- 有一点要注意是的：在UNIX系统中，keyFile必须没有组权限或完全权限（也就是权限要设置成X00的形式）。Windows系统中，keyFile权限没有被检查。
- 可以使用任意方法生成keyFile。例如，如下操作使用
openssl生成复杂的随机的1024个字符串。然后使用chmod修改文件权限，只给文件拥有者提供读权限。
这是MongoDB官方推荐keyFile的生成方式：
# 400权限是要保证安全性，否则mongod启动会报错
openssl rand -base64 756 > mongodb.key
chmod 400 mongodb.key
- 每一个副本集成员都要使用相同的keyFile文件
配置文件
docker-compose.yml 文件
version: '3.1'
services:
mongodb1:
image: mongo
restart: always
container_name: mongo1
volumes:
- ./data/db/mongo1:/data/db
- ./mongodb.key:/data/mongodb.key
ports:
- 27017:27017
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
networks:
- mongoNet
command: mongod --replSet mongos --keyFile /data/mongodb.key
entrypoint:
- bash
- -c
- |
chmod 400 /data/mongodb.key
chown 999:999 /data/mongodb.key
exec docker-entrypoint.sh $$@
mongodb2:
image: mongo
restart: always
container_name: mongo2
volumes:
- ./data/db/mongo2:/data/db
- ./mongodb.key:/data/mongodb.key
ports:
- 27018:27017
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
networks:
- mongoNet
command: mongod --replSet mongos --keyFile /data/mongodb.key
entrypoint:
- bash
- -c
- |
chmod 400 /data/mongodb.key
chown 999:999 /data/mongodb.key
exec docker-entrypoint.sh $$@
mongodb3:
image: mongo
restart: always
container_name: mongo3
volumes:
- ./data/db/mongo3:/data/db
- ./mongodb.key:/data/mongodb.key
ports:
- 27019:27017
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
networks:
- mongoNet
command: mongod --replSet mongos --keyFile /data/mongodb.key
entrypoint:
- bash
- -c
- |
chmod 400 /data/mongodb.key
chown 999:999 /data/mongodb.key
exec docker-entrypoint.sh $$@
networks:
mongoNet:
driver: bridge
.env 文件，主要作用是初始化数据库账密
MONGO_INITDB_ROOT_USERNAME=root
MONGO_INITDB_ROOT_PASSWORD=admin
mongodb.key 文件，用于副本集之间授权验证
6iB3VOV1eUQKodAhMyqdQQxdVvvaL5dBr7k/TIYf0csRGpLX0NyvL3f3M3G6kuI0
yRXiLlPJBzZ0GqmLNeo8jZ0rso/938qK5rU7pv9GmyEMGbAs75edl3OpfQCxjKUX
iq/+qgYN4R+Slw9klfMOaiFepAdeKHpCY66AqoM9YsjEW04evF9K0SlRzez3rfdA
HOqoovkOqWjhi6wDNan2JjYCb1jgjx6wDnwrq/GOoFIoqUIUMF9xdFXavlRO+jVc
sAow0U2+svvSbSiokCR023CqL618N5jYiq8/v4ztkHjFQ797TTJR6O69VHQ9hyMp
jUzjyA2+NXdXUbnMMoVfVRq104G4ux75nwsjnIAW2lpUKNPGbEQtYpWiCn0Ey2J9
3Zo2MF/SKgY4rj5FHdztP0d30SvFWsoCThwgEs3z1SwR2KJEDTu+UOQy3icEAPoJ
F0ya0C4J398a2gqVnDh+HxT/lhI3OVp1T846LlhRPnLpLbUvUR3kpGvXerGVryyZ
mzxG0cxEHM/Ede26iqWGXRQA7BgYhDx/1SFYWhqrvt3gNI9WR0jDe2B4m4r0TrPK
qyjbgXrb4E3P8HigDbgvm54DBnbkkbCd3+sONuNOpwpg2RPiVK5fVkJPxoNR+dzi
CCsixN6CvY5yRBquow10eCCLHRgYcJ8axyB1e772uHQFlyo4tJTm3kXsPaHg1KVA
rP/v1H4FW8RjQDVUHs1n5V3RAZwVEC6cprKqo6t2ygwiMMgRHVuYiMJRaN+yd1ap
QFnfQnw+iGgaKcl6Cgr3sFYEeO3FOTwLBPs24lk/pimIJn7rzyQ8T+/APMJBsMog
2oLZ/rTzbFbfOxccrfZQpc2/ozvnjvLSUPnePEDE3bfHREmOAJwPBMI7BQQgNZS9
y1NHWGR3v0qkmcnu3O0nmutxdS/TraOLpQYImLPQ++8axQKVjcRkoKCTPcWdjwDw
+mR0XL/H/UR9obDcTZLb1MCKD+tBw2UC07WP6LzmcNREk40/
文件详解
chown 999:999 /data/mongodb.key999用户是容器中的mongod用户，通过chown修改文件用户权限mongod --replSet mongos --keyFile /data/mongodb.key启动命令，--replSet mongos以副本集形式启动并将副本集名字命名为mongos，--keyFile /data/mongodb.key设置keyFile，用于副本集通信，文件通过volumes映射到容器内networks创建容器在同一局域网下，容器之间通信
配置副本集
- 切换到
docker-compose.yml所在目录，执行命令docker-compose up -d启动数据库，-d表示后台启动并运行所有的容器。 - 通过命令
docker exec -it mongo1 /bin/bash进入容器进行配置 - 进入容器之后，通过
mongo命令行进入数据库，因为在docker-compose配置文件中设置了数据库账密，不输入账密会无法进行设置，查看副本集配置时会提示无权限。
biao@centos:~$ docker exec -it mongo1 /bin/bash
root@00aee3c527c3:/# mongo
MongoDB shell version v4.0.10
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("6f8c9ba7-ef16-4caf-8582-a3d1f19c341e") }
MongoDB server version: 4.0.10
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
> rs.status()
{
"ok" : 0,
"errmsg" : "command replSetGetStatus requires authentication",
"code" : 13,
"codeName" : "Unauthorized"
}
- 输入正确的数据库账密之后，会显示以下信息：
root@00aee3c527c3:/# mongo -u root -p admin
MongoDB shell version v4.0.10
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("02d0d226-f23e-41c2-af9f-c34610d93797") }
MongoDB server version: 4.0.10
Server has startup warnings:
2019-07-14T05:58:43.055+0000 I STORAGE [initandlisten]
2019-07-14T05:58:43.055+0000 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-07-14T05:58:43.055+0000 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
- 此时输入
rs.status()会提示没有配置副本集
> rs.status()
{
"ok" : 0,
"errmsg" : "no replset config has been received",
"code" : 94,
"codeName" : "NotYetInitialized"
}
>
- 复制以下内容初始化副本集配置，其中ip是服务器的公网IP，如果设置了容器IP，数据库之间能进行通信，但是通过连接无法正确连接副本集，
_id的值是docker-compose配置文件启动命令设置的--replSet mongos副本集名称，这是最基础的配置，rs.initiate()触发选举，并选举出一个成员作为primary。
rs.initiate({
_id: "mongos",
members: [
{ _id : 0, host : "192.168.1.111:27017" },
{ _id : 1, host : "192.168.1.111:27018" },
{ _id : 2, host : "192.168.1.111:27019" }
]
});
- 执行初始化配置命令，显示
{ "ok" : 1 }就代表已经成功了，等待一会儿就会选举出PRIMARY节点，可以用rs.status()查看当前副本集状态。
> rs.initiate({
... _id: "mongos",
... members: [
... { _id : 0, host : "192.168.1.111:27017" },
... { _id : 1, host : "192.168.1.111:27018" },
... { _id : 2, host : "192.168.1.111:27019" }
... ]
... });
{ "ok" : 1 }
mongos:SECONDARY>
mongos:PRIMARY>
mongos:PRIMARY> rs.status()
{
"set" : "mongos",
"date" : ISODate("2019-07-14T06:01:48.574Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
}
},
"lastStableCheckpointTimestamp" : Timestamp(1563084091, 1),
"members" : [
{
"_id" : 0,
"name" : "192.168.1.111:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 185,
"optime" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2019-07-14T06:01:32Z"),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "could not find member to sync from",
"electionTime" : Timestamp(1563084090, 1),
"electionDate" : ISODate("2019-07-14T06:01:30Z"),
"configVersion" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 1,
"name" : "192.168.1.111:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 28,
"optime" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2019-07-14T06:01:32Z"),
"optimeDurableDate" : ISODate("2019-07-14T06:01:32Z"),
"lastHeartbeat" : ISODate("2019-07-14T06:01:48.252Z"),
"lastHeartbeatRecv" : ISODate("2019-07-14T06:01:48.519Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "192.168.1.111:27017",
"syncSourceHost" : "192.168.1.111:27017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 1
},
{
"_id" : 2,
"name" : "192.168.1.111:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 28,
"optime" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1563084092, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2019-07-14T06:01:32Z"),
"optimeDurableDate" : ISODate("2019-07-14T06:01:32Z"),
"lastHeartbeat" : ISODate("2019-07-14T06:01:48.252Z"),
"lastHeartbeatRecv" : ISODate("2019-07-14T06:01:48.512Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "192.168.1.111:27017",
"syncSourceHost" : "192.168.1.111:27017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 1
}
],
"ok" : 1,
"operationTime" : Timestamp(1563084092, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1563084092, 1),
"signature" : {
"hash" : BinData(0,"5XoSC0ilUaKemlTAx+Cx/xkoKc8="),
"keyId" : NumberLong("6713395051742887937")
}
}
}
mongos:PRIMARY>
故障转移
- 在副本集模式下，如果Primary节点宕机不可用，集群会选举出新的Primary节点
biao@centos:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
00aee3c527c3 mongo "bash -c 'chmod 400 …" 2 hours ago Up 2 hours 0.0.0.0:27017->27017/tcp mongo1
a0adb341e83c mongo "bash -c 'chmod 400 …" 2 hours ago Up 2 hours 0.0.0.0:27019->27017/tcp mongo3
76f1792ae9ed mongo "bash -c 'chmod 400 …" 2 hours ago Up 2 hours 0.0.0.0:27018->27017/tcp mongo2
biao@centos:~$ docker stop mongo11
mongo11
biao@centos:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a0adb341e83c mongo "bash -c 'chmod 400 …" 2 hours ago Up 2 hours 0.0.0.0:27019->27017/tcp mongo3
76f1792ae9ed mongo "bash -c 'chmod 400 …" 2 hours ago Up 2 hours 0.0.0.0:27018->27017/tcp mongo2
- 此时原来的主节点已经停止了，进容器查看当前副本集状态
biao@centos:~$ docker exec -it mongo2 /bin/bash
root@76f1792ae9ed:/# mongo -u root -p admin
mongos:PRIMARY>
mongos:PRIMARY> rs.status()
{
"set" : "mongos",
# ...省略
"members" : [
{
"_id" : 0,
"name" : "192.168.1.111:27017",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2019-07-14T08:01:48.362Z"),
"lastHeartbeatRecv" : ISODate("2019-07-14T08:00:58.421Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Error connecting to 192.168.1.111:27017 :: caused by :: Connection refused",
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : -1
},
{
"_id" : 1,
"name" : "192.168.1.111:27018",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 7386,
"optime" : {
"ts" : Timestamp(1563091300, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-07-14T08:01:40Z"),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1563091258, 1),
"electionDate" : ISODate("2019-07-14T08:00:58Z"),
"configVersion" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "192.168.1.111:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 7227,
"optime" : {
"ts" : Timestamp(1563091300, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1563091300, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-07-14T08:01:40Z"),
"optimeDurableDate" : ISODate("2019-07-14T08:01:40Z"),
"lastHeartbeat" : ISODate("2019-07-14T08:01:48.353Z"),
"lastHeartbeatRecv" : ISODate("2019-07-14T08:01:49.067Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "192.168.1.111:27018",
"syncSourceHost" : "192.168.1.111:27018",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 1
}
],
"ok" : 1,
"operationTime" : Timestamp(1563091300, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1563091300, 1),
"signature" : {
"hash" : BinData(0,"X0eYxSxcFjZZpMEEkunHgtnhJVg="),
"keyId" : NumberLong("6713395051742887937")
}
}
}
- 可以看到
mongo1的状态是not reachable/healthy，现在将已经停掉的容器重新启动，可以看到状态改变，重新加回副本集列表
增加节点
- 在新的服务器上部署并启动容器，部署的时候需要将
mongodb.key和.env文件一起复制过去，端口根据实际情况修改映射
docker-compsoe.yaml文件
version: '3.1'
services:
mongodb1:
image: mongo
restart: always
container_name: mongo4
volumes:
- ./data/db/mongo4:/data/db
- ./mongodb.key:/data/mongodb.key
ports:
- 27020:27017
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
command: mongod --replSet mongos --keyFile /data/mongodb.key
entrypoint:
- bash
- -c
- |
chmod 400 /data/mongodb.key
chown 999:999 /data/mongodb.key
exec docker-entrypoint.sh $$@
- 在
Primary节点数据库中，进入shell添加新创建的容器，服务器IP和端口
mongos:PRIMARY> rs.add("192.168.1.111:27020")
{
"ok" : 1,
"operationTime" : Timestamp(1563095185, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1563095185, 1),
"signature" : {
"hash" : BinData(0,"jMFzkKdSymJWZIBXPq+C094jk8w="),
"keyId" : NumberLong("6713395051742887937")
}
}
}
mongos:PRIMARY> rs.status()
{
"set" : "mongos",
# ...省略
"members" : [
{
"_id" : 0,
"name" : "192.168.1.111:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 4150,
"optime" : {
"ts" : Timestamp(1563095496, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1563095496, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-07-14T09:11:36Z"),
"optimeDurableDate" : ISODate("2019-07-14T09:11:36Z"),
"lastHeartbeat" : ISODate("2019-07-14T09:11:41.876Z"),
"lastHeartbeatRecv" : ISODate("2019-07-14T09:11:41.881Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "192.168.1.111:27018",
"syncSourceHost" : "192.168.1.111:27018",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 2
},
{
"_id" : 1,
"name" : "192.168.1.111:27018",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 11580,
"optime" : {
"ts" : Timestamp(1563095496, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-07-14T09:11:36Z"),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1563091258, 1),
"electionDate" : ISODate("2019-07-14T08:00:58Z"),
"configVersion" : 2,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "192.168.1.111:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 11421,
"optime" : {
"ts" : Timestamp(1563095496, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1563095496, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-07-14T09:11:36Z"),
"optimeDurableDate" : ISODate("2019-07-14T09:11:36Z"),
"lastHeartbeat" : ISODate("2019-07-14T09:11:41.890Z"),
"lastHeartbeatRecv" : ISODate("2019-07-14T09:11:41.967Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "192.168.1.111:27018",
"syncSourceHost" : "192.168.1.111:27018",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 2
},
{
"_id" : 3,
"name" : "192.168.1.111:27020",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 317,
"optime" : {
"ts" : Timestamp(1563095496, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1563095496, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-07-14T09:11:36Z"),
"optimeDurableDate" : ISODate("2019-07-14T09:11:36Z"),
"lastHeartbeat" : ISODate("2019-07-14T09:11:41.886Z"),
"lastHeartbeatRecv" : ISODate("2019-07-14T09:11:41.505Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "192.168.1.111:27018",
"syncSourceHost" : "192.168.1.111:27018",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 2
}
],
"ok" : 1,
"operationTime" : Timestamp(1563095496, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1563095496, 1),
"signature" : {
"hash" : BinData(0,"00FWYUxtXfPcvL/wKox4JZ7y7tI="),
"keyId" : NumberLong("6713395051742887937")
}
}
}
mongos:PRIMARY>
- 此时看到新的节点已经加进去了，数据在进行同步，此法可以用来迁移数据
节点切换
切换Primary节点为其他节点，参考文章mongodb4.0 replica set(副本集) 切换主节点 配置主从