KAFKA 连接器发布（Microsoft SQL Server)

Zookeeper 端口为：12181
KAFKA 在 zookeeper 上的命名空间：kafka_cdh
KAFKA 端口：19092
KAFKA 版本： http://archive.cloudera.com/kafka/parcels/3.0.0/
链接器： debezium-debezium-connector-sqlserver-0.9.5.zip

Microsoft SQL Server 开启 CDC

连接器配置

mssql-connect-distributed.properties

bootstrap.servers=xx.xx.xx.xx:19092,xx.xx.xx.xx:19092,xx.xx.xx.xx:19092
group.id=connect-cluster-mssql
offset.storage.topic=connect-offsets
config.storage.topic=connect-configs
status.storage.topic=connect-status

# rest 连接器发布端口
rest.port=18083
# 插件
plugin.path=/path/to/plugs/debezium-connector-sqlserver-0.9.5,

连接器启动

nohup   $KAFKA_HOME/bin/connect-distributed.sh $KAFKA_HOME/config/mssql-connect-distributed.properties > ~/mssql-connector.log 2>&1 &

准备连接器发布脚本

curl 'http://localhost:18083/connectors' -X POST -i -H "Content-Type:application/json" -d '{ "name": "win",
"config": {
    "connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",
    "database.hostname": " 10.xx.xx.xx",
    "database.port": "1433",
    "database.user": "xx",
    "database.password": "xx",
    "database.dbname": "run",
    "database.server.name": "win5",
    "table.whitelist": "xx.match,xx.orderrec",        "database.history.kafka.bootstrap.servers":"10.xx.xx.xx:19092,10.xx.xx.xx:19092,10.xx.xx.xx:19092",
    "decimal.handling.mode": "string",
    "database.history.kafka.topic": "dbhistory.win5"
    }
}'

手动创建主题

执行第二个步骤，正式发布连接器
指定分片数和副本数

1. win5.xx.match

$KAFKA_HOME/bin/kafka-topics.sh --zookeeper zoo1:12181/kafka_cdh --create --topic win5.xx.match --partitions 5 --replication-factor 3

1. win5.xx.orderrec

$KAFKA_HOME/bin/kafka-topics.sh --zookeeper zoo1:12181/kafka_cdh --create --topic win5.xx.orderrec --partitions 5 --replication-factor 3

发布并校验链接器

1. 删除连接器

 curl -X DELETE  'http://localhost:18083/connectors/win'

1. 查看所有连接器

  curl -X GET  'http://localhost:18083/connectors' 

1. 查看指定连接器状态

 curl -X GET 'http://localhost:18083/connectors/win/status'