Spark Streaming实时流处理项目实战笔记——使用IDEA+Maven构建开发环境

构建项目

Spark Streaming实时流处理项目实战笔记——使用IDEA+Maven构建开发环境_第1张图片

Pom文件

 
        2.11
        2.4.3
        2.6.0
    

    
        
            org.apache.spark
            spark-core_${scala.version}
            ${spark.version}
        
        
            org.apache.spark
            spark-streaming_${scala.version}
            ${spark.version}
        
        
            org.apache.spark
            spark-sql_${scala.version}
            ${spark.version}
        
        
            org.apache.spark
            spark-hive_${scala.version}
            ${spark.version}
        
        
            org.apache.spark
            spark-mllib_${scala.version}
            ${spark.version}
        
        
            org.apache.hbase
            hbase-client
            0.98.12-hadoop2
        
        
            org.apache.hbase
            hbase-server
            0.98.12-hadoop2
        
        
            org.apache.hadoop
            hadoop-mapreduce-client-core
            ${hadoop.version}
        

        
            org.apache.hadoop
            hadoop-client
            ${hadoop.version}
        

        
            org.apache.hadoop
            hadoop-common
            ${hadoop.version}
        
        
            org.apache.hbase
            hbase-protocol
            1.2.6
        
        
            org.apache.hbase
            hbase-shaded-client
            1.2.6
        
        
            org.apache.hadoop
            hadoop-hdfs
            ${hadoop.version}
        
        
            org.apache.spark
            spark-streaming-kafka-0-10_2.11
            ${spark.version}
        
        
            org.specs
            specs
            1.2.5
            test
        
    

IDEA导入SDK

你可能感兴趣的:(Hadoop)