SpringBoot工程连远程Kafka报错UnknownHostException

2019-12-30 23:12:13.049  WARN 8604 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-1, groupId=test-hello-group] Error connecting to node test_host:9094 (id: 2 rack: null)

java.net.UnknownHostException: test_host
    at java.net.InetAddress.getAllByName0(Unknown Source) ~[na:1.8.0_231]
    at java.net.InetAddress.getAllByName(Unknown Source) ~[na:1.8.0_231]
    at java.net.InetAddress.getAllByName(Unknown Source) ~[na:1.8.0_231]
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:104) ~[kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403) ~[kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363) ~[kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151) ~[kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:943) [kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:288) [kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.trySend(ConsumerNetworkClient.java:474) [kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:252) [kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233) [kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224) [kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.position(KafkaConsumer.java:1707) [kafka-clients-2.3.1.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.position(KafkaConsumer.java:1665) [kafka-clients-2.3.1.jar:na]
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.seekPartitions(KafkaMessageListenerContainer.java:859) [spring-kafka-2.3.4.RELEASE.jar:2.3.4.RELEASE]
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.access$3700(KafkaMessageListenerContainer.java:460) [spring-kafka-2.3.4.RELEASE.jar:2.3.4.RELEASE]
    at 

可能的错误

• kafka版本不对，现在这种方式只能支持0.10.x.x
• kafka配置没有对外开放host、port

• advertised.host.name、advertised.port

• broker-list配置的不对

stackoverflow相关问题的解答

When you run > bin/kafka-console-consumer.sh command kafka loads a ConsoleConsumer, which will attempt to create a consumer with an auto generated consumer id. The way Kafka generates the consumer id is to concatenate the name of the local host to it. So, in the problem was the fact that java could not resolve the ip address for local host on the Open Stack VM I am working with.

So the answer was that the Open Stack VM was resolving the local host name to kafka, which is the name of the VM. I had everything setup in the Kafka and Zookeeper instances as kafka1.

So, when java was calling getLocalHost, it was trying to find the IP Address for kafka, which I did not have in my /etc/hosts file.

I simply added an entry for kafka in my /etc/hosts file and everything started working wonderfully!!!

I would have thought it would resolve to localhost, but it did not, it resolved to the name of the vm, kafka.

 

sudo vim /etc/hosts

编辑，例如

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 ip-10-180-128-217