Kafka 入门（三）
Kafka 入门（三）
| IP | hostname |
|---|---|
| 192.168.79.177 | master01 |
| 192.168.79.180 | worker01 |
| 192.168.79.181 | worker02 |
2种集群模式
- Zookeeper
- Kraft
zookeeper集群配置
config/zookeeper.properties
三台服务器 master01, worker01, worker02 增加如下内容
note: master01, worker01, worker02 是服务器别名非 master01在这里一定是master节点
# ...
dataDir=/home/tester/app/kafka/zkdata
# ...
admin.enableServer=false
# admin.serverPort=8080
server.1=192.168.79.177:2999:3999
server.2=192.168.79.180:2999:3999
server.3=192.168.79.181:2999:3999
设置myid
master01
$ mkdir -p /home/tester/app/kafka/zkdata/
$ echo 1 > /home/tester/app/kafka/zkdata/myid
worker01
$ mkdir -p /home/tester/app/kafka/zkdata/
$ echo 2 > /home/tester/app/kafka/zkdata/myid
worker02
$ mkdir -p /home/tester/app/kafka/zkdata/
$ echo 3 > /home/tester/app/kafka/zkdata/myid
启动zookeeper
master01
$ bin/zookeeper-server-start.sh config/zookeeper.properties
worker01
$ bin/zookeeper-server-start.sh config/zookeeper.properties
worker02
$ bin/zookeeper-server-start.sh config/zookeeper.properties
验证 是否成功
在master01上创建节点
$ bin/zookeeper-shell.sh 192.168.79.177:2181
ls /
[zookeeper]
create /new
ls /
[new, zookeeper]
在192.168.79.180上登入192.168.79.181查看节点
$ bin/zookeeper-shell.sh 192.168.79.181:2181
ls /
[new, zookeeper]
更多具体设置及命令 Zookeeper集群
kafka server 配置
master01 config/server.properties
broker.id=0
# ...
log.dirs=/home/tester/kafka/kafka-logs
# ...
num.partitions=10
# ...
zookeeper.connect=192.168.79.177:2181,192.168.79.180:2181,192.168.79.181:2181
# ...
worker01 config/server.properties
broker.id=1
# ...
log.dirs=/home/tester/kafka/kafka-logs
# ...
num.partitions=10
# ...
zookeeper.connect=192.168.79.177:2181,192.168.79.180:2181,192.168.79.181:2181
# ...
worker02 config/server.properties
broker.id=2
# ...
log.dirs=/home/tester/kafka/kafka-logs
# ...
num.partitions=10
# ...
zookeeper.connect=192.168.79.177:2181,192.168.79.180:2181,192.168.79.181:2181
# ...
Kafka kraft 集群
kraft/server.properties
master01 更新配置
process.roles=broker,controller
node.id=1
# ...
controller.quorum.voters=1@192.168.79.177:9093,[email protected]:9093,[email protected]:9093
# ...
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
# ...
advertised.listeners=PLAINTEXT://:9092
# ...
log.dirs=/home/tester/kafka/kraft-combined-logs
# ...
num.partitions=10
worker01 更新配置
process.roles=broker,controller
node.id=2
# ...
controller.quorum.voters=1@192.168.79.177:9093,[email protected]:9093,[email protected]:9093
# ...
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
# ...
advertised.listeners=PLAINTEXT://:9092
# ...
log.dirs=/home/tester/kafka/kraft-combined-logs
# ...
num.partitions=10
worker02 更新配置
process.roles=broker,controller
node.id=3
# ...
controller.quorum.voters=1@192.168.79.177:9093,[email protected]:9093,[email protected]:9093
# ...
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
# ...
advertised.listeners=PLAINTEXT://:9092
# ...
log.dirs=/home/tester/kafka/kraft-combined-logs
# ...
num.partitions=10
master01 启动
$ KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
$ echo $KAFKA_CLUSTER_ID
ViRIykyGQUqQfBDvMUoFFQ
$ echo 'ViRIykyGQUqQfBDvMUoFFQ' > /home/tester/app/kafka/current_kafka_cid
$ rm -rf /home/tester/kafka/kafka-logs/* /home/tester/kafka/zkdata/version-2/* /home/tester/kafka/kraft-combined-logs/*
$ bin/kafka-storage.sh format -t ViRIykyGQUqQfBDvMUoFFQ -c config/kraft/server.properties
worker01 启动
$ rm -rf /home/tester/kafka/kafka-logs/* /home/tester/kafka/zkdata/version-2/* /home/tester/kafka/kraft-combined-logs/*
$ bin/kafka-storage.sh format -t ViRIykyGQUqQfBDvMUoFFQ -c config/kraft/server.properties
worker02 启动
$ rm -rf /home/tester/kafka/kafka-logs/* /home/tester/kafka/zkdata/version-2/* /home/tester/kafka/kraft-combined-logs/*
$ bin/kafka-storage.sh format -t ViRIykyGQUqQfBDvMUoFFQ -c config/kraft/server.properties
验证
参考 Kafka 入门（二）
修改 consumer 服务的application.properties文件
spring.kafka.bootstrap-servers=192.168.79.177:9092,192.168.79.180:9092,192.168.79.181:9092
修改 producer 服务的application.properties文件
spring.kafka.bootstrap-servers=192.168.79.177:9092,192.168.79.180:9092,192.168.79.181:9092
运行 producer, consumer 服务验证