Linux下Eclipse配置scala开发Spark-WordCount项目

Eclipse和Scala安装见：https://blog.csdn.net/qq_25948717/article/details/80404158‘

                                       https://blog.csdn.net/qq_25948717/article/details/80758713

去官网下载插件：http://scala-ide.org/download/prev-stable.html

解压后，把其features和plugins下的所有jar包都copy到eclipse下对应的features和plugins目录下

进入到eclipse安装目录：../eclipse/eclipse启动eclipse

Eclipse打开之后，从file-->new-->others-->Scala Wizards-->Scala Project新建一个project，我命名为SparkWordCount

右击SparkWordCount，-->properties-->Java Build Path-->Libraries-->Add External Jars，选择spark安装目录下lib/的assembly jar包

创建class：

package spark
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
class wordcount {
    def main(args: Array[String]) {
    val infile = "/public/users/yexin/SparkWorks/data/test.txt" // Should be some file on your system
    val conf = new SparkConf().setAppName("word count")
    val sc = new SparkContext(conf)
    val indata = sc.textFile(infile, 2).cache()
    val words = indata.flatMap(line => line.split(" ")).map(word => (word,1)).reduceByKey((a,b) => (a+b))
    words.saveAsTextFile("/public/users/yexin/SparkWorks/data/output")
    println("All words are counted!")
  }
}