Docker多机集群部署之kafka集群

一、环境说明:

宿主机:Win10

虚拟机工具:VMware Workstation 15

系统及版本:Ubuntu16.04

Docker版本:18.09.3

kafka注册在zookeeper集群中,zookeeper集群可参看:

https://www.jianshu.com/p/269145168597

涉及到的虚拟主机:

192.168.1.104 node1

192.168.1.105 node2

192.168.1.106 node3


二、创建本地目录

mkdir -p /docker/kafka/logs


三、创建容器

# 第一个节点(192.168.1.104)

docker run -d --net host --restart unless-stopped -p 9092:9092 \

-e CLUSTER_NAME=KAFKA  \

-e KAFKA_ADVERTISED_HOST_NAME=192.168.1.104 \

-e KAFKA_ADVERTISED_PORT=9092 \

-e KAFKA_ZOOKEEPER_CONNECT=192.168.1.104:2181,192.168.1.105:2181,192.168.1.106:2181 \

-e KAFKA_HOST_NAME=192.168.1.104 \

-e KAFKA_BROKER_ID=1 \

-e KAFKA_LISTENERS=PLAINTEXT://192.168.1.104:9092 \

-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.1.104:9092 \

-e KAFKA_DELETE_TOPIC_ENABLE=true \

-v /docker/kafka/logs:/kafka \

--name kafka1 wurstmeister/kafka


# 第二个节点(192.168.1.105)

docker run -d --net host --restart unless-stopped -p 9092:9092 \

-e CLUSTER_NAME=KAFKA  \

-e KAFKA_ADVERTISED_HOST_NAME=192.168.1.105 \

-e KAFKA_ADVERTISED_PORT=9092 \

-e KAFKA_ZOOKEEPER_CONNECT=192.168.1.104:2181,192.168.1.105:2181,192.168.1.106:2181 \

-e KAFKA_HOST_NAME=192.168.1.105 \

-e KAFKA_BROKER_ID=2 \

-e KAFKA_LISTENERS=PLAINTEXT://192.168.1.105:9092 \

-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.1.105:9092 \

-e KAFKA_DELETE_TOPIC_ENABLE=true \

-v /docker/kafka/logs:/kafka \

--name kafka2 wurstmeister/kafka

# 第三个节点(192.168.1.106)

docker run -d --net host --restart unless-stopped -p 9092:9092 \

-e CLUSTER_NAME=KAFKA  \

-e KAFKA_ADVERTISED_HOST_NAME=192.168.1.106 \

-e KAFKA_ADVERTISED_PORT=9092 \

-e KAFKA_ZOOKEEPER_CONNECT=192.168.1.104:2181,192.168.1.105:2181,192.168.1.106:2181 \

-e KAFKA_HOST_NAME=192.168.1.106 \

-e KAFKA_BROKER_ID=3 \

-e KAFKA_LISTENERS=PLAINTEXT://192.168.1.106:9092 \

-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.1.106:9092 \

-e KAFKA_DELETE_TOPIC_ENABLE=true \

-v /docker/kafka/logs:/kafka \

--name kafka3 wurstmeister/kafka

四、测试集群

# 进入第一个节点容器(192.168.1.104, 进入哪个节点下面的创建指令就用那个节点的ip)

docker exec -it kafka容器编号 bash


# 创建topic

kafka-topics.sh --create --zookeeper 192.168.1.104:2181 --replication-factor 1 --partitions 1 --topic test1

# 容器1发送消息

kafka-console-producer.sh --broker-list 192.168.1.104:9092 --topic test1

# 进入节点二容器查看消息

kafka-console-consumer.sh --bootstrap-server 192.168.1.105:9092 --topic test1 --from-beginning


五、查看集群工具

kafka tool http://www.kafkatool.com/download.html


六、删除topic

# 指定kafka参数

delete_topic_enable=true

# 重启kafka后通过指令来删除

bin/kafka-topics.sh --delete --topic test1 --zookeeper 192.168.1.104:2181

# 如果为删除成功,只是标记为了marked for deletion则可以尝试第二种方式

# 进入zookeeper容器执行zkCli.sh进入节点目录通过

# 查看节点

ls /brokers/topics

# 通过命令删除节点

rmr /brokers/topics/test1

# 同时查看标记为deletion的节点

ls /admin/delete_topics

# 通过命令删除

rmr /admin/delete_topics/test1


参考博客:

https://blog.csdn.net/qq_23146763/article/details/53364891

https://blog.csdn.net/russle/article/details/82881297

你可能感兴趣的:(Docker多机集群部署之kafka集群)