kafka和sparkstreaming整合打包运行错误处理

问题一：

WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.NoClassDefFoundError: kafka/serializer/StringDecoder
at DirectKafkaWordCountDemo$.main(DirectKafkaWordCountDemo.scala:19)
at DirectKafkaWordCountDemo.main(DirectKafkaWordCountDemo.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:736)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)

此问题是缺少kafka_2.11-0.8.2.1.jar包所导致,将这个jar包上传spark/jars下就可以了

问题二：

-org.apache.spark.streaming.kafka.KafkaUtils

如果是这个异常，是缺少spark-streaming-kafka_2.11-1.6.2.jar包所导致，将这个包上传到spark/jars下就可以了

问题三：

WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.NoClassDefFoundError: com/yammer/metrics/Metrics
at kafka.metrics.KafkaMetricsGroup$class.newTimer(KafkaMetricsGroup.scala:85)
at kafka.consumer.FetchRequestAndResponseMetrics.newTimer(FetchRequestAndResponseStats.scala:26)
at kafka.consumer.FetchRequestAndResponseMetrics.(FetchRequestAndResponseStats.scala:35)
at kafka.consumer.FetchRequestAndResponseStats.(FetchRequestAndResponseStats.scala:46)

此问题是缺少metrics-core-2.2.0.jar包所致，将这个包上传到spark/jars下就可以了。注：在spark/jars下，metrics-core-3.1.2.jar和metrics-core-2.2.0.jar都要存在，metrics-core-3.1.2.jar是spark core所需要的