StructedStreaming Kafka 排错（AbstractCoordinator: Marking the coordinator *dead for grou

描述：这次要接收一个新业务的数据，这个数据已经用Flume采集到了kafka里，
考虑后决定采用SparkStructedStreaming去接收数据，可是在接收过程中一直报一个错

19/04/26 15:32:11 DEBUG AbstractCoordinator: Received group coordinator response ClientResponse(receivedTimeMs=1556263931753, disconnected=false, request=ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@4350f1a2, request=RequestSend(header={api_key=10,api_version=0,correlation_id=0,client_id=consumer-1}, body={group_id=spark-kafka-source-4b13731a-90b1-4a3d-8482-1dd466da5fa2--513118232-driver-0}), createdTimeMs=1556263931618, sendTimeMs=1556263931725), responseBody={error_code=0,coordinator={node_id=0,host=hbase-single,port=9092}})
19/04/26 15:32:11 INFO AbstractCoordinator: Discovered coordinator hbase-single:9092 (id: 2147483647 rack: null) for group spark-kafka-source-4b13731a-90b1-4a3d-8482-1dd466da5fa2--513118232-driver-0.

首先看了看日志，并没有太多详细的信息，然后检查配置文件并没有错。
接下来就是百度、Google、StackOverFlow, 在这些网站也找到类类似的，都指向了通信方面的问题，但本地环境与Kafka环境是通的呀。

被逼无赖只好用处杀手锏，调整日志级别为DEBUG

spark.sparkContext.setLogLevel("DEBUG")

然后重启程序，出现了我想要的，日志如下：

19/04/26 15:32:20 DEBUG NetworkClient: Give up sending metadata request since no node is available
19/04/26 15:32:20 DEBUG NetworkClient: Initialize connection to node 0 for sending metadata request
19/04/26 15:32:20 DEBUG NetworkClient: Initiating connection to node 0 at hbase-single:9092.
19/04/26 15:32:20 DEBUG NetworkClient: Error connecting to node 0 at hbase-single:9092:
java.io.IOException: Can't resolve address: hbase-single:9092
	at org.apache.kafka.common.network.Selector.connect(Selector.java:171)
	at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:498)
	at org.apache.kafka.clients.NetworkClient.access$400(NetworkClient.java:48)
	at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:645)
	at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:552)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:258)
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitMetadataUpdate(ConsumerNetworkClient.java:134)
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:183)
	at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:974)
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader$$anonfun$fetchLatestOffsets$1$$anonfun$apply$9.apply(KafkaOffsetReader.scala:174)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader$$anonfun$fetchLatestOffsets$1$$anonfun$apply$9.apply(KafkaOffsetReader.scala:172)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader$$anonfun$org$apache$spark$sql$kafka010$KafkaOffsetReader$$withRetriesWithoutInterrupt$1.apply$mcV$sp(KafkaOffsetReader.scala:263)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader$$anonfun$org$apache$spark$sql$kafka010$KafkaOffsetReader$$withRetriesWithoutInterrupt$1.apply(KafkaOffsetReader.scala:262)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader$$anonfun$org$apache$spark$sql$kafka010$KafkaOffsetReader$$withRetriesWithoutInterrupt$1.apply(KafkaOffsetReader.scala:262)
	at org.apache.spark.util.UninterruptibleThread.runUninterruptibly(UninterruptibleThread.scala:85)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader.org$apache$spark$sql$kafka010$KafkaOffsetReader$$withRetriesWithoutInterrupt(KafkaOffsetReader.scala:261)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader$$anonfun$fetchLatestOffsets$1.apply(KafkaOffsetReader.scala:172)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader$$anonfun$fetchLatestOffsets$1.apply(KafkaOffsetReader.scala:172)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader.runUninterruptibly(KafkaOffsetReader.scala:230)
	at org.apache.spark.sql.kafka010.KafkaOffsetReader.fetchLatestOffsets(KafkaOffsetReader.scala:171)
	at org.apache.spark.sql.kafka010.KafkaSource$$anonfun$initialPartitionOffsets$1.apply(KafkaSource.scala:132)
	at org.apache.spark.sql.kafka010.KafkaSource$$anonfun$initialPartitionOffsets$1.apply(KafkaSource.scala:129)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.sql.kafka010.KafkaSource.initialPartitionOffsets$lzycompute(KafkaSource.scala:129)
	at org.apache.spark.sql.kafka010.KafkaSource.initialPartitionOffsets(KafkaSource.scala:97)
	at org.apache.spark.sql.kafka010.KafkaSource.getOffset(KafkaSource.scala:163)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$9$$anonfun$apply$6.apply(StreamExecution.scala:518)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$9$$anonfun$apply$6.apply(StreamExecution.scala:518)
	at org.apache.spark.sql.execution.streaming.ProgressReporter$class.reportTimeTaken(ProgressReporter.scala:279)
	at org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:58)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$9.apply(StreamExecution.scala:517)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$9.apply(StreamExecution.scala:515)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
	at scala.collection.AbstractTraversable.map(Traversable.scala:105)
	at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$constructNextBatch(StreamExecution.scala:515)
	at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$populateStartOffsets(StreamExecution.scala:489)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches$1$$anonfun$apply$mcZ$sp$1.apply$mcV$sp(StreamExecution.scala:297)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches$1$$anonfun$apply$mcZ$sp$1.apply(StreamExecution.scala:294)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches$1$$anonfun$apply$mcZ$sp$1.apply(StreamExecution.scala:294)
	at org.apache.spark.sql.execution.streaming.ProgressReporter$class.reportTimeTaken(ProgressReporter.scala:279)
	at org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:58)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anonfun$org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches$1.apply$mcZ$sp(StreamExecution.scala:294)
	at org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:56)
	at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches(StreamExecution.scala:290)
	at org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:206)
Caused by: java.nio.channels.UnresolvedAddressException
	at sun.nio.ch.Net.checkAddress(Net.java:101)
	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
	at org.apache.kafka.common.network.Selector.connect(Selector.java:168)
	... 52 more
19/04/26 15:32:20 DEBUG NetworkClient: Give up sending metadata request since no node is available
19/04/26 15:32:20 DEBUG NetworkClient: Initialize connection to node 0 for sending metadata request

日志中很详细，在初始化连接的时候失败，失败的原因也很清楚，是Can’t resolve address: hbase-single:9092，然后一回想是我还没配置新环境的hosts映射。（我记得前几天配了的，细细回想，好像当时是写在了桌面，但因为一些事情打断了，欲哭无泪。）

总结

虽然被自己无意中坑了，但还是收获满满，那就是一定要重视日志，学会用日志