IDEA使用Springboot整合SparkStreaming的环境搭建及pom.xml

1.如图选择maven项目的webapp,然后点击下一步，填写项目名

2.引入sparkstreaming需要的依赖

注意：这里需要注意版本要相对应，不然编写代码会出现很多版本冲突、类找不到等问题

   
        2.11.8
        0.9.0.0
        2.2.0
        2.6.0-cdh5.7.0
        1.2.0-cdh5.7.0
    

    
    
        
            cloudera
            https://repository.cloudera.com/artifactory/cloudera-repos
        
    

    
        
            org.springframework.boot
            spring-boot-starter-web
            
                
                    org.springframework.boot
                    spring-boot-starter-logging
                
                
                    com.google.guava
                    guava
                
            
        

        
            org.scala-lang
            scala-library
            ${scala.version}
        
        
        

        
        
            org.apache.hadoop
            hadoop-client
            ${hadoop.version}
        

        
        
            org.apache.hbase
            hbase-client
            ${hbase.version}
        

        
            org.apache.hbase
            hbase-server
            ${hbase.version}
        

        
        
            org.apache.spark
            spark-streaming_2.11
            ${spark.version}
        


        
        
            org.apache.spark
            spark-streaming-flume_2.11
            ${spark.version}
        

        
            org.apache.spark
            spark-streaming-flume-sink_2.11
            ${spark.version}
        
  		
  		  
        
            org.apache.spark
            spark-streaming-kafka-0-8_2.11
            ${spark.version}
        

        
            org.apache.commons
            commons-lang3
            3.5
        

        
        
            org.apache.spark
            spark-sql_2.11
            ${spark.version}
        


  

        
            net.jpountz.lz4
            lz4
            1.3.0
        

        
            mysql
            mysql-connector-java
            5.1.38
        

        
            org.apache.flume.flume-ng-clients
            flume-ng-log4jappender
           1.9.0
        

    
    
        
            
                org.scala-tools
                maven-scala-plugin
                
                    
                        
                            compile
                            testCompile
                        
                    
                
                
                    ${scala.version}
                    
                        -target:jvm-1.5
                    
                
            
            
                org.apache.maven.plugins
                maven-eclipse-plugin
                
                    true
                    
                        ch.epfl.lamp.sdt.core.scalabuilder
                    
                    
                        ch.epfl.lamp.sdt.core.scalanature
                    
                    
                        org.eclipse.jdt.launching.JRE_CONTAINER
                        ch.epfl.lamp.sdt.launching.SCALA_CONTAINER
                    
                
            
        
    
    
        
            
                org.scala-tools
                maven-scala-plugin
                
                    ${scala.version}
                
            
        
    

三：引入依赖后，然后等着仓库下

注：下载这段时间比较长，下载完毕后，需要查看依赖是否完全下载完成，会有一些包因为网速原因下载不下来。这个重新刷新下载就可以

四：然后就可以开始愉快的编写Sparkstreaming的业务代码了