MongoShake实时同步（kafka）

一、MongoDB安装与配置参考链接（1，2，3步）

MongoShake实时同步，MongoDB1到MongoDB2(单向)：https://www.jianshu.com/p/011ea0f834e4

二、下载安装kafka

kafka下载地址：http://kafka.apache.org/downloads

image.png

kafka启动需要zookeeper，无需额外下载，下载包自带有

1.zookeeper配置与启动

vim config/zookeeper.properties

dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
# Disable the adminserver by default to avoid port conflicts.
# Set the port to something non-conflicting if choosing to enable this
admin.enableServer=false

启动命令：./bin/zookeeper-server-start.sh config/zookeeper.properties

2.kafka服务配置与启动

vim config/server.properties

#将Broker的Listener信息发布到Zookeeper中
#192.168.1.1 当前kafka的IP (用户发现kafka 服务)
advertised.listeners=PLAINTEXT://192.168.1.1:9092
zookeeper.connect=localhost:2181

启动命令：./bin/kafka-server-start.sh config/server.properties

三、MongoShake安装配置

1.安装（参考上面一）
2.配置

#kafka通道只支持增量
sync_mode = incr
tunnel = kafka
mongo_urls = mongodb://127.0.0.1:27017
#topic2代表kafka的topic
tunnel.address = topic2@localhost:9092
#json模式
tunnel.message = json
#初始值这个要配置，整数倍
incr_sync.tunnel.write_thread = 8

配置完成
启动：./collector.linux -conf=collector.conf
监控状态查看另起窗口./mongoshake-stat --port=9100

MongoShake实时同步，MongoDB1到MongoDB2(单向)：https://www.jianshu.com/p/011ea0f834e4