MongoDB复制集架构
mongodb的复制至少需要两个节点。其中一个是主节点，负责处理客户端请求，其余的都是从节点，负责复制主节点上的数据。
mongodb各个节点常见的搭配方式为：一主一从、一主多从。
主节点记录在其上的所有操作oplog，从节点定期轮询主节点获取这些操作，然后对自己的数据副本执行这些操作，从而保证从节点的数据与主节点一致。
MongoDB复制结构图如下所示：
副本集特征：
N 个节点的集群
任何节点可作为主节点
所有写入操作都在主节点上
自动故障转移
自动恢复
Replica Set角色
Replica Set 的成员是一堆有着同样的数据内容 mongod 的实例集合，包含以下三类角色：
主节点（Primary）
是 Replica Set 中唯一的接收写请求的节点，并将写入指令记录到 oplog 上。副本节点通过复制 oplog 的写入指令同步主节点的数据。Secondary。一个 Replica Set 有且只有Primary 节点，当Primar挂掉后，其他 Secondary 或者 Arbiter 节点会重新选举出来一个主节点。应用程序的默认读取请求也是发到 Primary节点处理的。副本节点（Secondary）
通过复制主节点 oplog 中的指令与主节点保持同样的数据集，当主节点挂掉的时候，参与主节点选举。仲裁者（Arbiter）
不存储实际应用数据，只投票参与选取主节点，但不会被选举成为主节点。
复制集架构配置：
架构：
Arbite：127.0.0.1:27017
primary：127.0.0.1:27018
secondary： 127.0.0.1:27019
配置文件内容如下，只要修改logpath、dbpath、pidfilepath和端口即可
arbiter.conf
# syslog配置 logpath=/var/log/mongo/logs/mongod1.log logappend=true timeStampFormat=iso8601-utc # storage 存储配置 dbpath=/data/mongo/repldb1 directoryperdb=true # processManagement 进程管理配置 fork=true # 进程 PID 文件保存目录 pidfilepath=/var/run/mongod1.pid # net 网络配置 bind_ip=127.0.0.1 port=27017 # security 配置 #auth=true # 复制集 replSet=test-set
primary.conf
# syslog配置 logpath=/var/log/mongo/logs/mongod2.log logappend=true timeStampFormat=iso8601-utc # storage 存储配置 dbpath=/data/mongo/repldb2 directoryperdb=true # processManagement 进程管理配置 fork=true # 进程 PID 文件保存目录 pidfilepath=/var/run/mongod2.pid # net 网络配置 bind_ip=127.0.0.1 port=27018 # security 配置 #auth=true # 复制集 replSet=test-set
secondary.conf
# syslog配置 logpath=/var/log/mongo/logs/mongod3.log logappend=true timeStampFormat=iso8601-utc # storage 存储配置 dbpath=/data/mongo/repldb3 directoryperdb=true # processManagement 进程管理配置 fork=true # 进程 PID 文件保存目录 pidfilepath=/var/run/mongod3.pid # net 网络配置 bind_ip=127.0.0.1 port=27019 # security 配置 #auth=true # 复制集 replSet=test-set
分别启动这三个实例：
mongod --config arbiter.conf mongod --config primary.conf mongod --config secondary.conf
初始化：
使用mongodb客户端登陆两个常规节点（primary和secondary）中的任何一个，执行如下命令
mongod 127.0.0.1:27018
登陆后执行初始化命令
> rs.initiate(
... {"_id":"test-set",
... "members":[
... {"_id":1,"host":"127.0.0.1:27018"},
... {"_id":2,"host":"127.0.0.1:27019"}
... ]
... });
{ "ok" : 1 }
test-set:SECONDARY> rs.addArb("127.0.0.1:27017")
{ "ok" : 1 }
查看集群状态：
test-set:PRIMARY> rs.status()
{
"set" : "test-set",
"date" : ISODate("2018-04-19T07:22:22.813Z"),
"myState" : 1,
"term" : NumberLong(1),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524122535, 1),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1524122535, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1524122535, 1),
"t" : NumberLong(1)
}
},
"members" : [
{
"_id" : 1,
"name" : "127.0.0.1:27018",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 1303,
"optime" : {
"ts" : Timestamp(1524122535, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-19T07:22:15Z"),
"electionTime" : Timestamp(1524121903, 1),
"electionDate" : ISODate("2018-04-19T07:11:43Z"),
"configVersion" : 2,
"self" : true
},
{
"_id" : 2,
"name" : "127.0.0.1:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 649,
"optime" : {
"ts" : Timestamp(1524122535, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1524122535, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2018-04-19T07:22:15Z"),
"optimeDurableDate" : ISODate("2018-04-19T07:22:15Z"),
"lastHeartbeat" : ISODate("2018-04-19T07:22:21.472Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T07:22:21.472Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "127.0.0.1:27018",
"configVersion" : 2
},
{
"_id" : 3,
"name" : "127.0.0.1:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 69,
"lastHeartbeat" : ISODate("2018-04-19T07:22:21.471Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T07:22:18.484Z"),
"pingMs" : NumberLong(0),
"configVersion" : 2
}
],
"ok" : 1
}
登录127.0.0.1:27019 角色显示为Secondary
mongo 127.0.0.1:27019
2018-04-19T07:00:44.075Z I CONTROL [initandlisten] test-set:SECONDARY> test-set:SECONDARY> test-set:SECONDARY>
在primary上创建数据，测试secondary上是否会同步
test-set:PRIMARY> show dbs
admin 0.000GB
local 0.000GB
test-set:PRIMARY> use testdb
switched to db testdb
test-set:PRIMARY> show collections
test-set:PRIMARY> db.test.insert({"name":"user1"});
WriteResult({ "nInserted" : 1 })
登录secondary，查看
test-set:SECONDARY> rs.slaveOk()
test-set:SECONDARY> show dbs
admin 0.000GB
local 0.000GB
testdb 0.000GB
test-set:SECONDARY> use testdb
switched to db testdb
test-set:SECONDARY> show collections
test
test-set:SECONDARY> db.test.find()
{ "_id" : ObjectId("5ad844efa46a748c5a90fe41"), "name" : "user1" }
secondary上插入数据报错，说明secondary上没有写的权限
test-set:SECONDARY> db.test.insert({"name":"user2"});
WriteResult({ "writeError" : { "code" : 10107, "errmsg" : "not master" } })
验证复制集架构容灾功能
kill掉primary的进程，登录127.0.0.1:27019
test-set:PRIMARY> show dbs
admin 0.000GB
local 0.000GB
testdb 0.000GB
test-set:PRIMARY> rs.status()
{
"set" : "test-set",
"date" : ISODate("2018-04-19T07:44:28.912Z"),
"myState" : 1,
"term" : NumberLong(2),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524123775, 1),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1524123862, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1524123862, 1),
"t" : NumberLong(2)
}
},
"members" : [
{
"_id" : 1,
"name" : "127.0.0.1:27018",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2018-04-19T07:44:27.023Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T07:42:59.830Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Connection refused",
"configVersion" : -1
},
{
"_id" : 2,
"name" : "127.0.0.1:27019",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2625,
"optime" : {
"ts" : Timestamp(1524123862, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-19T07:44:22Z"),
"infoMessage" : "could not find member to sync from",
"electionTime" : Timestamp(1524123790, 1),
"electionDate" : ISODate("2018-04-19T07:43:10Z"),
"configVersion" : 2,
"self" : true
},
{
"_id" : 3,
"name" : "127.0.0.1:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 1395,
"lastHeartbeat" : ISODate("2018-04-19T07:44:27.009Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T07:44:28.662Z"),
"pingMs" : NumberLong(0),
"configVersion" : 2
}
],
"ok" : 1
}
可以看到127.0.0.1:27018 有一个 "stateStr" : "(not reachable/healthy)"的报错，而127.0.0.1:27019已经变成了primary角色
启动原来的primary.conf配置的实例，角色由原来的primary变成了secondary
test-set:SECONDARY> show dbs
2018-04-19T15:50:04.113+0800 E QUERY [thread1] Error: listDatabases failed:{
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:62:1
shellHelper.show@src/mongo/shell/utils.js:782:19
shellHelper@src/mongo/shell/utils.js:672:15
@(shellhelp2):1:1
test-set:SECONDARY>
其他命令： rs.help()
test-set:PRIMARY> rs.help()
rs.status() { replSetGetStatus : 1 } checks repl set status
rs.initiate() { replSetInitiate : null } initiates set with default settings
rs.initiate(cfg) { replSetInitiate : cfg } initiates set with configuration cfg
rs.conf() get the current configuration object from local.system.replset
rs.reconfig(cfg) updates the configuration of a running replica set with cfg (disconnects)
rs.add(hostportstr) add a new member to the set with default attributes (disconnects)
rs.add(membercfgobj) add a new member to the set with extra attributes (disconnects)
rs.addArb(hostportstr) add a new member which is arbiterOnly:true (disconnects)
rs.stepDown([stepdownSecs, catchUpSecs]) step down as primary (disconnects)
rs.syncFrom(hostportstr) make a secondary sync from the given member
rs.freeze(secs) make a node ineligible to become primary for the time specified
rs.remove(hostportstr) remove a host from the replica set (disconnects)
rs.slaveOk() allow queries on secondary nodes
rs.printReplicationInfo() check oplog size and time range
rs.printSlaveReplicationInfo() check replica set members and replication lag
db.isMaster() check who is primary
rs.slaveOk()：secondary默认不允许读写，执行此命令后，从节点有可读权限
rs.conf()：查看配置情况
test-set:SECONDARY> rs.conf()
{
"_id" : "test-set",
"version" : 2,
"protocolVersion" : NumberLong(1),
"members" : [
{
"_id" : 1,
"host" : "127.0.0.1:27018",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "127.0.0.1:27019",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 3,
"host" : "127.0.0.1:27017",
"arbiterOnly" : true,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : 60000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5ad841247ffb05dbb6b4a13b")
}
}
rs.add()：新增一个节点到副本集中,注意只能在primary上操作，secondary角色执行该命令会报错，例如：
test-set:SECONDARY> rs.add("127.0.0.1:27020")
{
"ok" : 0,
"errmsg" : "replSetReconfig should only be run on PRIMARY, but my state is SECONDARY; use the \"force\" argument to override",
"code" : 10107,
"codeName" : "NotMaster"
}
测试：
登录primary，新建一个third.conf的配置文件，并启动实例，端口为127.0.0.1:27020
mongod --config third.conf # netstat -tunlp | grep 27020 tcp 0 0 127.0.0.1:27020 0.0.0.0:* LISTEN 13528/mongod
添加新的secondary角色
test-set:PRIMARY> rs.add("127.0.0.1:27020")
{ "ok" : 1 }
test-set:PRIMARY> rs.status()
{
"set" : "test-set",
"date" : ISODate("2018-04-19T08:16:36.910Z"),
"myState" : 1,
"term" : NumberLong(2),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
},
"appliedOpTime" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
}
},
"members" : [
{
"_id" : 1,
"name" : "127.0.0.1:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1661,
"optime" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-19T08:16:32Z"),
"optimeDurableDate" : ISODate("2018-04-19T08:16:32Z"),
"lastHeartbeat" : ISODate("2018-04-19T08:16:36.730Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T08:16:35.732Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "127.0.0.1:27019",
"configVersion" : 3
},
{
"_id" : 2,
"name" : "127.0.0.1:27019",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 4553,
"optime" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-19T08:16:32Z"),
"electionTime" : Timestamp(1524123790, 1),
"electionDate" : ISODate("2018-04-19T07:43:10Z"),
"configVersion" : 3,
"self" : true
},
{
"_id" : 3,
"name" : "127.0.0.1:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 3323,
"lastHeartbeat" : ISODate("2018-04-19T08:16:36.730Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T08:16:36.713Z"),
"pingMs" : NumberLong(0),
"configVersion" : 3
},
{
"_id" : 4,
"name" : "127.0.0.1:27020",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 110,
"optime" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1524125792, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-19T08:16:32Z"),
"optimeDurableDate" : ISODate("2018-04-19T08:16:32Z"),
"lastHeartbeat" : ISODate("2018-04-19T08:16:36.729Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T08:16:35.914Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "127.0.0.1:27019",
"configVersion" : 3
}
],
"ok" : 1
}
rs.remove(hostportstr)：删除一个secondary节点
test-set:PRIMARY> rs.remove("127.0.0.1:27020")
{ "ok" : 1 }
test-set:PRIMARY> rs.status()
{
"set" : "test-set",
"date" : ISODate("2018-04-19T08:21:46.241Z"),
"myState" : 1,
"term" : NumberLong(2),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1524126097, 1),
"t" : NumberLong(2)
},
"appliedOpTime" : {
"ts" : Timestamp(1524126097, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1524126097, 1),
"t" : NumberLong(2)
}
},
"members" : [
{
"_id" : 1,
"name" : "127.0.0.1:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1971,
"optime" : {
"ts" : Timestamp(1524126097, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1524126097, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-19T08:21:37Z"),
"optimeDurableDate" : ISODate("2018-04-19T08:21:37Z"),
"lastHeartbeat" : ISODate("2018-04-19T08:21:45.652Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T08:21:42.657Z"),
"pingMs" : NumberLong(0),
"configVersion" : 4
},
{
"_id" : 2,
"name" : "127.0.0.1:27019",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 4863,
"optime" : {
"ts" : Timestamp(1524126097, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2018-04-19T08:21:37Z"),
"electionTime" : Timestamp(1524123790, 1),
"electionDate" : ISODate("2018-04-19T07:43:10Z"),
"configVersion" : 4,
"self" : true
},
{
"_id" : 3,
"name" : "127.0.0.1:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 3632,
"lastHeartbeat" : ISODate("2018-04-19T08:21:45.656Z"),
"lastHeartbeatRecv" : ISODate("2018-04-19T08:21:42.656Z"),
"pingMs" : NumberLong(0),
"configVersion" : 4
}
],
"ok" : 1
}
编辑修改集群配置(只能在primary节点上操作)
登录到MongoDB的primary实例
执行cfg=rs.conf()命令，将集群配置保存在cfg变量中
test-set:PRIMARY> cfg=rs.conf()
{
"_id" : "test-set",
"version" : 4,
"protocolVersion" : NumberLong(1),
"members" : [
{
"_id" : 1,
"host" : "127.0.0.1:27018",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "127.0.0.1:27019",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 3,
"host" : "127.0.0.1:27017",
"arbiterOnly" : true,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : 60000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5ad841247ffb05dbb6b4a13b")
}
}
查看集群第一个成员的配置
test-set:PRIMARY> cfg.members[0]
{
"_id" : 1,
"host" : "127.0.0.1:27018",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
修改第一个成员的优先级配置
test-set:PRIMARY> cfg.members[0].priority=2
2
##修改后，再次查看配置，优先级已经被修改
test-set:PRIMARY> cfg.members[0]
{
"_id" : 1,
"host" : "127.0.0.1:27018",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 2,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
保存配置使生效
test-set:PRIMARY> rs.reconfig(cfg)
{ "ok" : 1 }