SparkStream+kafka报错Fetching topic metadata with correlation id 0 for topics from broker

19/10/22 17:35:44 WARN ClientUtils$: Fetching topic metadata with correlation id 0 
for topics [Set(zhaogw)] from broker [id:0,host:dblab-VirtualBox,port:9092] failed
java.nio.channels.ClosedChannelException
	at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
	at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)

先说解决方法,因为我的主机名称为dblab-VirtualBox,而在我本机配置虚拟机的映射为bigdata,因为而kafka默认拿着主机名寻找映射信息,然而hosts文件中没有,故报错。
如果还不懂,看图:

修改前

这是我配置的linux主机名
在这里插入图片描述
这是我在windows的hosts文件所做的映射
SparkStream+kafka报错Fetching topic metadata with correlation id 0 for topics from broker_第1张图片
代码

   val kafkaDsream: ReceiverInputDStream[(String, String)] = KafkaUtils.createStream(
      streamContext,
      "bigdata:2181",
      "zhaogw",
      Map("zhaogw" -> 3)
    )

启动项目后就不断的刷那个异常

修改后

wondows的hosts文件
SparkStream+kafka报错Fetching topic metadata with correlation id 0 for topics from broker_第2张图片
代码也改一下
SparkStream+kafka报错Fetching topic metadata with correlation id 0 for topics from broker_第3张图片
项目就能正常启动消费了

SparkStream+kafka报错Fetching topic metadata with correlation id 0 for topics from broker_第4张图片

参考https://www.cnblogs.com/dongxiucai/p/10288275.html

你可能感兴趣的:(Fetching,topic,metadata,SparkStream,kafka,spark)