The coordinator is not available

kafka consumer group 无法消费数据
打开debug日志查看具体原因(此处是logback配置)

...

看到关键提示

2020-05-08 10:11:15.596 DEBUG   --- [Thread-3] org.apache.kafka.clients.consumer.internals.AbstractCoordinator        [onSuccess] [741] : [Consumer clientId=consumer--4, groupId=group_async] Received FindCoordinator response ClientResponse(receivedTimeMs=1588903875595, latencyMs=4, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=3, clientId=consumer-group_async-4, correlationId=11), responseBody=FindCoordinatorResponseData(throttleTimeMs=0, errorCode=15, errorMessage='The coordinator is not available.', nodeId=-1, host='', port=-1))

根据这个提示得知,是消费组协调leader不存在导致
查看kafka __consumer_offset topic

kafka_consume_offset.png

发现确实有Leader:none的

可能原因是 __consumer_offset topic的默认分区是50，但是备份只有1份; kafka集群部署了3台，也就是3个brokers
消费者组连接kafka,并会请求某一台来查找Coordinator(协调者)，如果连接的机器上没有备份就不会有Leader,就会出现
找不到Coordinator(协调者)

解决方式
1、停止kafka, 修改kafka配置 config/server.properties添加

修改成broker的数量

offsets.topic.replication.factor=3

2、删除zookeeper配置信息

./bin/zkCli.sh -server 127.0.0.1:2181

删除/config/topics/__consumer_offsets

delete /config/topics/__consumer_offsets

删除/brokers/topics/__consumer_offsets

deleteall /brokers/topics/__consumer_offsets

3、重启kafka

参考
https://stackoverflow.com/questions/51952398/kafka-connect-distributed-mode-the-group-coordinator-is-not-available
https://www.cnblogs.com/jun1019/p/6634545.html
https://blog.csdn.net/yf_li123/article/details/101552437
https://www.orchome.com/1296
https://github.com/edenhill/librdkafka/issues/2086