Spark word count 实例

利用中午休息时间配置了个环境，写个Word count 本地模式测试下 环境 ， 环境Okay

Scala 2.10.4
Java  1.7.08
Spark 1.6.0

package com.ib.e3

import org.apache.spark.{SparkConf, SparkContext}

/**
  * Created by xxxooo on 8/29/2016.
  */
object WordCount {

  def main(args: Array[String]) {

    val conf = new SparkConf().setAppName("Myapp").setMaster("local")

    val sc = new SparkContext(conf)

    val linesRDD = sc.textFile("C:\\workplacebus\\business\\xxxooo\\xxxooo\\workspace\\spark\\lib\\wordcount.txt")

    val wordRDD = linesRDD.flatMap(line => line.split(" ")).map(word => (word,1)).reduceByKey((a,b) => a+b)

    wordRDD.foreach(word => println(word))
  }

}

每一个坚持大汗淋漓的人都不会太简单

转载于:https://www.cnblogs.com/TendToBigData/p/10501385.html